RAMageddon and loss of ownership

It is being widely reported that there is a global shortage of RAM due to AI hardware requirements. Data centers require an immense amount of HBM (high-bandwidth memory), which leads to RAM supply being funneled to them. There is a corresponding rise in prices for all kinds of consumer computing projects. Apple increased its prices to compensate for the increased price of components. Microsoft, Sony, Nintendo, Valve, and Samsung have all raised prices considerably on their hardware this year.

I’ve been starting to look at running LLM models locally, since I believe that is the only way LLMs will be viable in the long run. The offerings of OpenAI and Anthropic and the like have such high capital investment and ongoing power requirements that there is no way they can be viable functioning businesses without significant price hikes. If there is a future for broad LLM use I think it is in much lighter weight models running locally.

And yet, you need hardware to run the models. You need GPUs and RAM. These are in short supply. A lot of hardware is therefore designed to be used strictly networked, for compute to be remote. Chromebooks follow this pattern. If components continue to be in short supply, we can expect many more devices that only exist to provide the user access to remote networked services, with little-to-no local capabilities.

Remote compute is monetized through subscription or metered usage. If you cannot do computing on hardware you own, you are forced to use what is effectively a dumb terminal to do computing on hardware that you cannot control. Generally you must pay for the privilege, and you are also subject to potentially unwanted oversight like the current proliferation of age-verification schemes and the corresponding loss of privacy.

This is…convenient, for the providers of these computing services. I don’t think this is an overt conspiracy or anything like that. But, it is unfortunate that the concentration of available hardware in the hands of LLM providers - providers of remote computing - also forces people to use that remote computing. This further concentrates wealth and capital in the hands of these companies.

It is probably worthwhile for people to continue to make use of their existing hardware wherever they can. Keep that old 2013 Windows laptop - it can run Linux now. You’re unlikely to be able to run something like an LLM on it, but at least you own the hardware and the software on it.