The Painful Truth: The RAM Crisis Is Only Just the Beginning(madshrimps.be) |
The Painful Truth: The RAM Crisis Is Only Just the Beginning(madshrimps.be) |
Apple got caught out because it had a shorter contract that ended in the beginning of Q3, and they've tried to source their RAM from Chinese CXMT who declined because all capacity was already locked in contracts. They've gone with a lesser known company Kioxia.
Overall the consumer segment is completely neglected, companies don't care about end users in the current market conditions.
"CXMT currently has two 12-inch DRAM fabrication plants — or fabs - in Hefei and one in Beijing, with a combined capacity of about 300,000 wafers per month.
With the new Shanghai facility and other new capacity, CXMT will double its DRAM wafer output to approximately 600,000 wafers per month, all three sources added."
https://www.reuters.com/world/china/chinas-cxmt-wins-3-billi...
Formerly known as Toshiba's memory division. They spun off the business as Kioxia in 2017.
Mirroring trends throughout the entire economy (not just RAM and other inputs for AI).
Everyone is chasing the top 10% or higher of the K-shaped economy for all goods and services, servicing everyone else isn't seen as being worth the investment.
This will just keep getting worse and worse everywhere for everything as long as we allow income inequality to keep exploding, which seems to be the plan.
Because the end users keep rushing to use the megacorps' latest AI models, weaving them into their work and life. So the megacorps keep locking in contracts to build more compute.
If you use LLMs, you're responsible for this, there's no way to pass the buck. "I only use it as a companion for learning about history" - it's still your fault. "I only use it to help guide my solitions, not for vibecoding" - it's still your fault.
The US Government also wouldn't let Apple use Chinese RAM anyways, even for product that was just going to be used in China. China does have capacity issues though, and focusing on local brands first is probably the right call. Hopefully they can ramp even if they can't get the fancy lithography machines from the Netherlands.
Kioxia is Toshiba, The most known company from the list, and it doesnt make any ram
I wonder if they'll, at some point, have enough RAM? Or is this is the new normal? Will models keep scaling with the amount of ram chips openai and anthropic own?
I’d guess no. Past a certain point the model has all the capabilities it can possibly usefully offer and honestly we may already be past that. The next gen model just doesn’t seem like as clear a step up as it once was.
Some of their treasury (Non-Current Marketable Securities) could be bonds that are invested with targeted contracts for trade-secret benefits.
Say TSMC needs capital, and say Apple has spare capital: they are both likely to agree to an investment where Apple gets contractual benefits that other customers do not.
I would expect Apple to be very aggressively investing into suppliers to get results that strongly benefits Apple and perhaps that disadvantages competitors.
It's almost like these companies WANT a dystopia with a centralized winner-takes-all power structure. Anything in the name of profits, who gives a fuck about humanity and distribution of rights or freedoms.
Interesting effect is that since DRAM production tooling has been switched from DDR/GDDR to HBM, we may finally see the proliferation of HBM in consumer GPUs/accelerators after the bust cycle starts.
I would be completely unsurprised should we discover in a few years that peak RAM prices were at least partially due to price gouging and not merely due to shortages.
There's no indication that this will change, and every indication this will continue to grow. This is not some temporary thing. Demand is already exponentially higher than what is possible to produce, and there's no current reason to believe it has any ceiling.
- memory compression algorithms
- alternative LLM architectures that don't rely on memory or GPUs
- compatibility hardware (like DDR3 to DDR4 boards)
- distributed computing improvements, both at local GPU and networking levels (SLI for AI)
- GPU hacks to add more memory or support older architectures
I'm personally looking forward to the new LLM architectures that don't require as much compute, e.g. DLLMs, which can be good enough for CPU usage but lack the accuracy of frontier models currently.
When this happens the bottom will fall out of the GPU and memory markets, putting a glut of cheap hardware out there.
Doom mongering like this never seems to include these as viable future alternatives, which is standard market adjustments, I wonder who the doom narrative helps? :)
You are projecting things will happen when there is no guarantee.
The DRAM issue has halted the industry.
AI companies need to back down. That is the only objective solution.
And the article doesn't even appear in their homepage.
And the author is somebody from a company that "provides you with quality RAM"
Flagged. How did this even get to HN ?
Otherwise, there will be no end to this. There are no hard limits on the speed of a parallel bruteforce. It's an infinite complexity problem class. The more parallel bruteforce power you have, the more likely you are to be able to solve a problem. So there is no world where demand ends. So if something isn't done about this, we'll have million dollar GPUs and RAM sticks because they've priced everyone out of the market and are the only ones able to afford them. Say goodbye to owning your own hardware at that point.
https://en.wikipedia.org/wiki/Cornering_the_market
Right now, the RAM market is effectively cornered. Anti-trust enforcement is what I'd look to, but the GOP do not believe in ensuring competitive markets / enforcement. (The current admin is pretty clearly 110% pro corporation.) Vote in November, but in all likelihood, there won't be government intervention earlier than 2028, and even that is optimistic. One hopes the AI bubble pops, but I think this market can remain irrational for far longer than I can keep old hardware alive.
They'd try it if we weren't living in an age of persistent supply chain problems across every industry. But sadly we have lived in that world since 2020.
Good luck trying to capture the low-end of any market when you are competing with those serving the high-end for whatever your inputs are.
I guess hardware is not that expensive to them in the grand scheme of things, at least not at this stage. OTOH, their propietary models might be thoughly optimized and we can't know, because they're still bound by supply contracts to buy the same amount of hardware nevertheless.
The assumption about the singularity simply is a delusion with LLMs.
However, the models do provide a means to improve the harness universe, so that residual will continue to improve perception. Parameter cpunt will stagnate and training wont be justifiable from every angle.
but i feel like this will not last that long and that cxmt will be layering dram for hbm stacks with abandon, like everyone else, uninterested in selling ram to anyone else.
meanwhile now we are seeing vertically stacked ram in regular dimms too, for 512GB sticks. once again increasing the multipliers of how many ram chips go into servers. https://www.techpowerup.com/352730/micron-develops-512-gb-dd...
Why have pants down exposure to the market prices? Specially when they have 300B in the bank and a fab costs what 20-30B?
They are lucky they did not get squeezed out of tsmc too, otherwise they would have to re release the iPhone X in 2027.
What makes it risky? Do you think that Apple will pivot into something that doesn't require RAM? Maybe like an IBM, they pivot to services?
https://skeptics.stackexchange.com/questions/2863/did-bill-g...
The harness though will improve while parameter count stagnates. The Qwen3.8 models are strong enough when given proper context.
Huh? I'm not sure what the word "training" does in that sentence. But "never" my arse. Frontier models can make nontrivial software already.
For example, the other day I asked fable to reverse engineer the satisfactory blueprint file format. Then write a program to read the logistic flow graph in a blueprint. Then make an auditing tool that can analyse the graph to find problems.
Well, it totally knocked it out of the park:
https://seph.au/blueprints/#bp=0%3Aalumina.sbp
This is a relatively small program, but it's not trivial. I'd consider a trivial program to be something I could code up in 20 minutes. It would have taken me a couple weeks to make this blueprint auditing tool, including reverse engineering the file format, writing the analysis code, making the website, scraping all the in-game data on available recipes and icons and so on.
I've got a lot of mixed feelings about LLMs. But it seems very silly to lie about what they're capable of.
I'm well aware it can build apps. But if you arn't tracking what's going on, they're basically creating deterministic gates and tools to get it to do anything.
There's no lie here, it's simply about what you think is _LLM_ and what is the rest of the software that's making it go. I use opencode consistently to build non-trivial apps with it, but it's not doing it with zero guidance, and it's routinely wrong about it's assumptions, and the rest. The thing keeping it on track is opencode, not the LLM's training.
So, the same as skilled humans then?