We can see this already with GPT how luna can do 90% of what sol is used for. The only reason why china still bothers 'distilling' models is accurate training data generation, something that oai and anthropic had to spend years collecting while trying to dodge legal challenges.
The more intelligent models get, the more people will offramp to cheaper solutions that get the job done. There's no real benefit to using a sota model when the accuracy is already 99% and I think that is the biggest danger to US labs.
Anything else? Sure use a cheaper model.
And sometimes it's not about the model, it's just about refining the search space. E.g. I've had Opus write tests and GPT 5.5 write the implementation passing all the tests. Then ask about that specific implementation and find some real corner cases. Add those to tests, etc.
But the other fun trick that's been working better and better on the GPT-5.6 series is that even the lower-end models can find the things they didn't think of first when inspecting the already-written output.
I think there's still a bit of hard-to-quantify "creativity" to the bigger models - especially when trying to untangle (a) is this edge case that the model built a complicated way to avoid real/worth worrying about and also (b) even if it is real, is there not a better way to mitigate it? But it might be confirmation bias, in a way that definitely didn't use to be true about GPT-5.3 for planning and Composer 2 for implementation, say.
As open weight models and cheap fine tuning services become the norm the whole economic framework of these mega models the labs are in an arms race building just completely crumbles. As does the economic picture that justified the massive infrastructure building that’s now broadly funded by a complex network of debt.
This is what makes open weight models so threatening to them. The political and “it’s China” angle is mostly just a cover for the real reasons why they’re freaked out.
The fact that models are now a pure commodity is bad enough for the big labs. If small open weight models become the norm the big labs are toast.
They want to moat where they don’t need to compete with other companies because they have something that other companies can’t have.
In my opinion this is a short-sighted and greedy worldview. Haven’t seen it work personally. It is a different version of the month-to-month salary guy thinking he will be a billionaire and having that thrash “mindset”.
The reality is that practically none of those companies will amount to anything and they would be better off weighing the usefulness aspect of their output more. Instead they are imagining they will be Google.
Anthropic and openai ofc are the pinnacle of this greed culture and they correspond to FTX from the crypto trash hype so I don’t think they fit into the scale of sensibility.
Coming from this perspective, it is pretty easy to see what they are.
If we were to take these at face value, why is it that the frontier labs' models are making legitimate new discoveries (e.g. Erdős and Jacobian conjectures) and these models are not?
To me, a better signal of capability would be similarly performing novel work at the same or better level, which they presently are not. I say this as someone who very much looks forward to open models being more capable, but to deny the gap is misguided hopeful hype.
The point of Fine-tuning small models is for specific downstream tasks, which SOTA models can do, but at higher costs. It is purely an economic play, not an attempt at pushing boundaries of SOTA.
If the appropriate qualifiers were added to say "cheap model X does better at test Y than expensive model Z when we fine tune X to take Y test of existing knowledge" then it would be a more accurate statement, but naturally less impressive.
It doesn't mean frontier models aren't good at harder tasks.
A fine-tuned small model is like a laser pointer. It only illuminates a tiny specific spot. But it can illuminate it as brightly as the huge floodlight, for a tiny fraction of cost.
From the article it’s not clear how the scorer grades every episode - was it a frontier model that assigned the grade? How does that continue to work as the model that is being fine-tuned becomes better at the task than the frontier model?
Share of the maximum achievable score our GRPO-trained 9B open-source model reached on catalog review, vs 76.9% for the best frontier configuration: a 13.5% relative improvement over the frontier, and 36% over its own untrained base (64.2%). The five frontier models, even with optimized prompts, plateaued within a tenth of a point of each other; the trained specialist cleared that ceiling."
_______
This seems very unlikely to be true. I have a lot of skepticism that frontier models like GPT 5.5 that are likely 2T+ parameters in size only got about 12% more accurate than an untrained 9b parameter LLM.
What if you have to do the task a billion times? Which model will you choose?