No, local models will not win(seangoedecke.com) |
No, local models will not win(seangoedecke.com) |
Okay, but I already have multiple computers capable of running local models with acceptable performance, so for me the cost is $0. I suspect that's true for most people of sufficient technical inclination to be interested in and capable of running models on their local machines.
Also, no, the monthly electricity price of even my power-hungriest machines ain't anywhere close to that figure. Hell, at the high end that's more than my power bill for my whole household.
The mean price of electricity for me is about 40c per kWh: about as high as anywhere in the US.
There are about 730 hours in a month, and a consumer machine can sustain maybe 500W, so we can spend 0.4 * 0.5 * 730 = 146 dollars on compute power per box unless we start buying special stuff.
Win what? Money, fame? That's not the point of small models, freedom is the point. What's this occultist obsession with "There shall be only one" monopolies? Why only one? That's so irrational and childish.
Every so often someone hypes local models as "Don't pay for Claude check out this local modal" and the local modal isn't at all equivalent.
That doesn't mean that local models are going anytwhere (as the article motes they're useful) but if you want the best we're stuck with expensive remote models right now.
Idk, I already don’t bother with Opus and stick with sonnet med. I really care more about speed. I use qwen3.6 27b for personal projects and I think it works pretty great.
So like the article mentions, if scaling stalls and small models get better it’s not impossible to imagine a convergence and hardware costs drop.
Having said that, self hosting will be a niche thing like it is today for other services.