I’m getting more and more questions about RAM, storage, and AI. I thought I’d take a moment and put some thoughts down on the subject. I’m by no means an expert in the business dealings of the parties involved, but I do work in this field and am doing my best to navigate this environment. So buckle up.
The Three Types of Memory at the Heart of the Shortage
Let’s start with types of memory and their uses. For this post, I’m going to focus on three players at the heart of the shortage and give a brief description of each.
- High Bandwidth Memory (HBM)
- Standard DRAM (DDR5)
- NAND Flash/SSD Storage
High Bandwidth Memory (HBM)
HBM is the memory fueling the AI craze. It’s super fast and geared toward memory-intensive tasks. This is the memory that every AI data center is eating up like a chocoholic at the Godiva factory with an Amex Black card. The demand is extreme, with no end in sight. AI data centers, backed by venture capital betting on AI someday being profitable, want more and more of these HBM chips.
Standard DRAM
Standard DRAM (DDR5) is what we find in every laptop, desktop, and even servers that aren’t being used for AI. It’s also the memory found in every smartphone, printer, TV, game console, and just about every other device that connects to the internet — including the devices powering the internet itself.
NAND Flash
This is the memory that computers and phones use for storage when a device is powered off. Every USB memory stick uses NAND flash memory.
I’d dare say that virtually all the electronics we use today utilize some form of memory — everything from cell phones to calculators. All your network gear utilizes memory. Even modern coffee makers, like my trusty Breville Grind and Brew, now have memory just to boot up their system. Almost every automobile manufactured after the 1970s has onboard computers that rely on memory chips to function.
Who’s Actually Making This Stuff
Now that we have a sense of the scope, let’s talk about supply. There are three main manufacturers of memory chips: Samsung Electronics, SK Hynix, and Micron Technology. These three control most of the world’s supply of memory chips. There are other players in the flash/NAND SSD space, but for DDR and HBM specifically, it’s these three.
When supply is concentrated into just three competitors, there’s real potential for issues. As with any product, it comes down to supply and demand. A quick Google search (so take it for what it’s worth) tells me that spinning up a modern semiconductor fabrication plant capable of producing modern DDR5 memory requires an investment of $15–30 billion and a timeline of 3–5 years. The fear these three — or any new players entering the market — have is that by the time new capacity comes online, the prices these chips are demanding now will have dropped, causing losses. Whether that fear is accurate is difficult to say.
The Shift from DDR to HBM
So we have manufacturers running at capacity, but what kind of chips are they actually producing? HBM for AI data centers, DRAM for standard servers and everyday computers, or NAND for storage? Right now, the big three are in the process of shifting production capacity from DDR to HBM.
What this means for consumers is less RAM available for desktops, laptops, and the like. It also means manufacturers of desktops, laptops, gaming systems, etc. are paying more due to the limited production capacity available. That’s driving prices up across the board — which in turn forces those manufacturers to raise prices on their own products, or risk losing money on them.
Two Letters: AI
So what’s driving the shift from DDR to HBM production? Two letters: AI. All over the news we’re hearing about AI data centers being constructed across the US and the world. Large tech firms like Microsoft, Google, Meta, and Apple, along with AI powerhouses like OpenAI and Anthropic, are spending hundreds of billions on capacity to run their models.
Being a kid born in the ’70s, I think about how the US and Russia spent billions on the nuclear arms race. These companies are in a race to become the de facto standard for AI — the go-to place for answers. Why? So they can monetize that service. Whether it’s ads, data farming, or pay-as-you-go transactions, they’re not in this just to be in it.
Right now, the AI build out is causing these new players to lose money year after year. That’s not sustainable, and at some point the venture capital will run out. Investors will demand a reckoning. Until then: spend, spend, spend to try to win the race.
Back to the Memory Companies
Now let’s get back to the memory guys — the companies supplying the drugs to this industry, so to speak. DDR vs. HBM. As I said, manufacturers are shifting traditional capacity from DDR to HBM. There’s a problem with that: DDR is selling at an all-time high price, and you can produce three times as many chips on a wafer with DDR compared to HBM. So what they’re effectively doing is cutting output right when DDR profits are at an all-time high.
That’s one of the reasons all three companies are being sued for collusion and price fixing — and this isn’t the first time they’ve faced this kind of lawsuit. I won’t get into the details here, but if you want more on it, leave a comment and I’ll dig deeper in a future post.
Where This Leaves Us
So the reality of today’s world is that the price per gigabyte of RAM for a computer has doubled over the last year. Companies across the board are raising prices to compensate. Microsoft has even announced it’s optimizing Windows 11 to run on 8GB of RAM. My response to that: why didn’t Microsoft do that two or three years ago? Because they got lazy — computer hardware was always fast enough to overcome the software bloat being produced. The efficient coder has become a thing of the past.
Well, maybe it’s about to be reborn. Maybe we’ll see a resurgence of secure, efficient coding from major players while we wait for the bubble to burst.
Do I think memory prices will actually come back down? Short answer: no. I do think prices will stabilize.
TXMerak's thoughts and musings 