(Transcript: https://gist.github.com/simonw/c2a0d8ecd3056a2681319eae8fc3f...)
What do we think we are doing with this life?
Time for a reminder that OpenRouter leaderboards only show tokens sent through OpenRouter, which most Anthropic API users don’t use.
Your business data is probably worthless, even considered harmful for the pretrain corpus.
Your interactions and decision making process are most valuable parts of the whole business.
please tell me you are not in charge of the data of any business I'm a client of
What you need to know is who is the provider for the LLM, and whether their endpoints are zero data retention enabled and opted out of training. OpenRouter gives you an easy way to control this.
Its of course highly dependant on the use case and the environment, but simply saying that the only important part is to know where the data goes is too simple.
https://github.com/lechmazur/buyout_game 10th out 36.
https://github.com/lechmazur/pact/ 14th out 25.
https://github.com/lechmazur/nyt-connections/ 60th out 81.
https://github.com/lechmazur/debate 16th out of 29.
Which means if a surprise model tops the leaderboard one week we can never be sure if it was because a single whale user pushing billions of tokens a day switched to it, or if it represents a genuine community trend towards that model.
Down with reality!!
You might have the default settings on your account, which limit Deepseek as a provider. If you disable that feature you see them on openrouter as well (and they serve it at the same cost as their own API).
You're trying to think logically, which has no place in an AI discussion. :) People just jump to whatever the latest model is. Plenty of people also prefer price to "quality" (which is very subjective). It's new, it's cheap, so people use it. It's likely people will stop using it when something else is cheaper and/or newer.
(But maybe that's just my interpretation based on something else going wrong in the animation)
Bit like asking for CSS and then getting a HTML file back with the CSS embedded, that was not what I was asking for!
However, I just double checked, and OpenRouter's pricing page for Flash v4 with DeepSeek provider shows a cache hit rate of $0.0028, which is the same as on DeepSeek's official API pricing page ($0.0028), so they do seem to be the same price, (assuming DeepSeek is able to pin your specific OpenRouter requests to the same DeepSeek server). OpenRouter adds 5% to that cost, but still it might be cheaper than the other providers.
Also just found out OpenRouter has a new feature "Response Caching" where they can cache identical requests and return them immediately with no billing. The entire request must be identical, though, not just a prefix, and you have to enable this feature. I don't know who would need to send multiple identical requests, but it's better than nothing?
It's the same way we trust OpenAI to not train on our data if we've opted out although there is no control on whether they can retain the data indefinitely.
And even if they dont train on the data. Who guarantees us, they dont let another AI model analyse all the data, exfiltrating all kinds of intelligence and using it? I only can imagine what OpenAI and Anthropic know….
Ask yourself
1. How would you know the provider has violated the contract?
2. How could you prove it?
3. Why would OpenRouter take your side in this (unlike your example with OpenAI, you're not a signing party)?
4. How would OpenRouter enforce the contract after all three above are somehow resolved in your favor?
IANAL, but IMO it's all a legal theater.
EDIT: formatting
I mean sure there are investors and a little more open-ness, but with the example of Mythos we don't even know if public will get access to the "good" stuff because it's too dangerous.
If your only opinion on trusting these companies more than one based in China is, they are Chinese then good luck, all the best.
Sure "China bad, US good" is naive, but certainly not more naive than suggesting that companies and individuals have similar rights and protections as each other.
> and they need US govt approval before model releases
This is just not true and it would be a gigantic legal battle to make it true against the model companies' wishes, which is indicative of your entire misunderstanding here.
So it may not be strictly true for the moment, but it is certainly something that the current US govt can mandate at any time.
This isn't the case (yet).
The AI oligarchs have no loyalty and when it comes to making money and they will drop the king at their first opportunity and the king in return will do the same.
1) Far from them actually trying to do it
2) Very, very far from them actually doing it successfully
The US government absolutely cannot "just tell" private entities what products they're allowed to create and sell, and the fact that LLMs are arguably a form of expression will make these particular products extremely hard to regulate – especially as a broad "government checkpoint" on incremental product updates.
In China, it really is as simple as the government deciding that it doesn't like your products and ta-da, you can no longer sell them.
It's beyond naive to act like these are similar in any meaningful sense.
And just like that, I totally agree with you
There are several components of the Fair Use test, "transformation" is just one of them. The most important dimension is the effect on the market, i.e. the effect on incentives.
You probably shouldn't base your legal analysis on pithy internet comments regardless of how succinct or agreeable they are to you.