SambaNova chip for LLMs nearly as fast as Groq(fast.snova.ai) |
SambaNova chip for LLMs nearly as fast as Groq(fast.snova.ai) |
I'm hoping the rise of Asics leads to hardware agnostic representations of ML models, or at least seamless conversion from Cuda to other target hardware.
With LLMs standardizing around known instructions, I'm bullish about this family of hardware. Now, they may just get acquired given the infinite money Nvidia has. But, I see their IP being valuable by itself.
I've used Groq, and it's been one of my few 'oh wow' moments of the recent llm cycle.
I see nothing from either Groq or SambaNova that says that they will distribute dedicated inference chips in any other form than full data centers. If I can't slot it into my machine, is it real? How exactly are these companies envisioning the future of inference? As a walled corporate garden where we pay them with our thoughts code low complexity tasks and small change so they can cement their hold on our material lives and make themselves indispensable as they imperceptibly shape our outcomes? At least Tenstorrent is selling something I could slot, although I don't think they are at the point where it makes sense to do so
Groq are trying to launch their own cloud and running it at a loss to try to acquire users are raise venture capital so they can tape out a new chip (their current chip just is not very competitive, they get speed for having a good interconnect, but it is so cost ineffective). SambaNova sell systems to big companies and service providers, and don’t have a paid cloud API, so aren’t as visible as Groq. But SambaNova make real revenue whilst Groq have literally zero revenue, which is why they struggle to raise money.
Very soon people will realize that Groq are frauds and SambaNova is the legit challenger to Nvidia.
Plus, Groq claims to have converted over to only being a cloud provider now, and will keep their hardware to themself. Given that I can't even sign up for a pay-as-you-go API key with Groq right now, I think there's a lot of room for competitors.