Also kimi 2.6 at 1000tps (as of may), though when we reached out they had a >12 month waitlist and minimum 7-8 figure annual token spend.

[0] https://www.cerebras.ai/blog/cerebras-kimi-k2-Enterprise

Please do not try to use gpt-oss-120b over Cerebras. It is broken, screws up tool calls most of the time, forgets to end thinking blocks and has all sorts of other issues. The speed is amazing but it is absolutely not worth it, especially at that quite incredible cost. Think: $5–10/minute levels of cost with a single agent, because Cerebras also offers no cache pricing for input tokens at all.

Or better: Qwen 2.8 27b

The point is the speed.

From what I’ve heard, the issue is more that it’s harder to efficiently share the hardware across diffusion requests, so it’s more expensive to serve.

It sounds like there might be opportunities for local models (not open weight, but actually locally run) to use diffusion for faster responses on weaker hardware that doesn’t need to be shared.

But yea, it’s still a red-ish flag that big labs haven’t invested much in it. I could see Google/Apple getting value of this sort of local model, but maybe there’s enough research behind traditional models that it’s not worth the distraction at this point in time.

You can't think that a small startup versus Anthropic's training setup is anywhere near the same scale to make apples to apples comparisons.

Not sure how the Chinese labs pull it off though using autoregressive models. The secret sauce is probably going to be in the training data.

The main reason Google hasn't switched over to DiffusionGemma is because serving at larger batch sizes loses the speed gains you get from diffusion, and most of the primary use case is serving many users at once off a single device with a large batch size.

If you were to move to on-device low latency... like say in a robot or something, then the story might be different...

DiffusionGemma was released alongside the other gemma-4 models just a few months ago, so clearly google hasn't abandoned the idea.

K2-Horizon-7B has a diffusion and non-diffusion variant, and they claim the same level of intelligence from both models.

Personally, I think it's more that text diffusion is not the ideal driver of an agentic work loop than that text diffusion is a total dead end. I am still hoping to see how it does on authoring and editing with further scaling and optimization. I think the push for AGI has put a bit too much focus on the idea of one general model doing everything.

It's stupid and it's fast.

It means something, because it an iterative workflow. If you're willing to burn tokens, it's possible for weaker models to implement tasks by incrementally improving drafts.

Ah, the old "good, fast, or cheap; pick two" proves true once again.

Some of us want fast food

Not if your use case needs speed. For one of my products I can't use an LLM that has a p99 of >700ms for TTFT.