I really can't imagine OpenAI or Anthropic turning off inference for a model that my workplace is happy to spend >$200*person/month on. Google still has piles of cash and no reason to turn off Gemini.

The thing is, if inference is truly heavily subsidized (I don't think it is, because places like OpenRouter charge less than the big players for proportionally smaller models) then we'd probably happily pay >$500 a month for the current frontier models if everyone gave up on training new models because of some oddball scaling limit.

There's wildly different reports about whether the cost of just inference (not the training) is expensive or not...

Sam Altman has said “We’re profitable on inference. If we didn’t pay for training, we’d be a very profitable company.”

But a lot of folks are convinced that inference prices are currently being propped up by burning through investor capital?

I think if we look at open source model hosting then it's pretty convincing - Look at say https://openrouter.ai/z-ai/glm-4.7 . There's about 10 different random API providers that are competing on price and they'll serve GLM 4.7 tokens at around $1.50 - $2.50 per output Mtokens. (which by the way is a tenth of the cost of Opus 4.5)

I seriously doubt that all these random services that no one has ever heard of are also being propped up by investor capital. It seems more likely that $1.50 - $2.50 is the "near cost" price.

If that's the actual cost, and considering that the open source models like GLM are still pretty useful when used correctly, then it's pretty clear that AI is here to stay.

How much would it cost you to deploy a model that you and maybe a few coworkers could effectively use? $400k probably to buy all the hardware required to host a top-tier model that could do a few hundred tokens per second for 10 concurrent users? That's $40k per person. Ammortize the hardware over 5 years, thats $8k per person per year (roughly), with no training costs (that's just you buying hardware and running it yourself). So that means, you need ~$800 per user monthly just to cover hardware to run the model (this is with no staffing costs, internet, taxes, electricity, hosting, housing, etc).

So just food for thought, but $200 claude code is probably still losing money even just on inference.

Since they are in the software realm, they are probably shooting for a 90% profit margin. Using the above example, it would be ($800 + R&D + opex) x 10. My guess is assuming no more training (which probably can never be profitable at current rates), they need $20k per month per user, which is why that number was floated by OpenAI previously.

1. AI disappears, goes up in price, etc. All the money you've spent goes up in smoke, or you have to spend a lot more money to keep the engine running.

2. AI does not disappear, becomes cheaper and eats your businesses primary revenue generation for lunch.

Number 1 could happen tomorrow. Number 1 could happen after number 2. Number 1 may never happen.

Also expect that even if the AI market crashes that AI has already massively changed the economy, and that at least some investment will go into making AI more efficient and at any point number 2 could spring out of nowhere yet again.

I know this is probably an annoying question, but… has the author actually tried self-hosting an AI with one's own hardware? I have; ollama (and various frontends thereof) makes it straightforward, and it's absolutely not cost-prohibitive — I've ran my share of LLMs even on laptops without dedicated GPUs at all, and while the experience wasn't great compared to the commercial options, it wasn't outright unusable, either. Locally-hosted LLMs are already finding their way into various applications; that's only going to get more viable over time, not less (unless the computing hardware industry takes a catastrophic nosedive, in which case AI affordability is arguably the least of our worries).

I'm sure the author understands this and is just being hyperbolic in the article's title, but the AI bubble bursting ≠ AI disappearing, for the same reason the dotcom bubble bursting ≠ the World Wide Web disappearing. The bubble will burst when AI shifts from being novel to being mundane, just as with any other technology-related bubble — and that entails a degree of affordability and ubiquity that's mutually exclusive with any notion of AI “disappearing”. Hopefully it'll mean companies being less motivated to shove AI “features” down everyone's throats, but the virtually-intelligent cat is already out of Pandora's box: the technology's here to stay, and I think it's presumptuous to think the race to the bottom w.r.t. cost is anywhere near the finish line.

Ads, obviously

For the valuable kick start usecase it pays off. It cant do all the magic bootsrraps but for baseline technical questions its perfect. Will put in a rag search eventuallly.

Im not optimistic any use case will come to substantiate todays valuations. But the intertwined fascist businesses is going to stunt a lot of people trying to chain their product to 3rd parties.

Never used a LLM or anything explicitly.

Got annoyed when I had to deal with AI chatbots as front-line customer service - although that only happened once or twice in the last couple of months.

So basically, keep doing what I'm doing.

I like AI for specifically targeted applications: - e.g. 100,000+ AI "eyeballs" vs. a few 100 for diagonstic imaging, working out whether there's something to worry about or not. I hate the idea of generalised AI, LLM's etc.

Lowering the bar to enable 'creative output' from non-creative individuals just fucks up the world, because natural talent is replaced by unnatural talent, especially in (late) capitalism, where money is worth more than human experience to those few control-freak managers.

I'm old. I even earnt enough to buy a house with lawn over 4 years ago during my (pre-AI) career as a Software Developer. Get off my damn lawn.