I think it would be a good business model.
I mean if you look back to how expensive computers were when they first arrived, accounting for inflation, it's basically the same price point.
I think it would be awesome if I could pay 5/6k on really nice hardware that my devices could query directly via a app.
The problem is local is still not the easiest thing to set up, and tackling the networking aspect is not straight forward.
This sort of architecture is democratized by design and I am rooting for the person that figures this out for the masses.
I want claude power, in a box, at my house, for my entire family completely compartmentalized from my operating system.
This is still a six-figure commitment, possibly high five.
The cost of RAM and size of models. For Kimi K2.6 you need 2TB RAM. That’s $40k with DDR5. If you want it to run at the speeds you’re accustomed to with Claude, you need HBM memory, which costs more.
Practically speaking, you need to sink $250k+ into a 8x B200 node. So yeah, 6 figures to run properly. High 5 figures if you’re okay with really slow responses.