An Inside Look at the Token Reseller Market(vectoral.com) |
An Inside Look at the Token Reseller Market(vectoral.com) |
Here are the two open source proxies listed in the article: https://github.com/songquanpeng/one-api and https://github.com/QuantumNous/new-api
I know of a friend's company in India who purchased inference, at 4% of the actual price and states that it gave him an unbeatable competitive edge in their large running video influence pipelines. Any new competitors could not offer their pricing at all.
Primarily that operated because registering a new company getting free AWS credits was a very tiny cost
No wonder social media is so shit nowadays. All that brainwashing and propaganda from third world countries, now at 4% the price!
I don’t know if that’s true of India or not, but I like to give comment authors the benefit of the doubt that being specific about the geography was helpful context here
How would one even word a bulletproof subscription contract for agentic tokens, anyway? You can't forbid automation because sub-agents are automation. You could forbid "using tokens for the benefit of more than the human who signed up" but then what do families (especially with kids) need to do? What if your friend asks you a question and you turn to a chat model? Forbidding "reselling" tokens outside of a household sounds like the closest terms but that's leaky for anyone who travels a lot, etc.
Fixed cost per token simply works.
reverse the pricing structure; give modest discount once you go over certain amount of tokens, then you are incentivized NOT to start multiple accounts.
require first few transactions to be pre-paid to get around at least some of the card problems.
Of course, that would fuck over subsidized plans, but I don't see any option to keep them if you want to avoid the flood
Is this some kind of attempt to make the other side look better by making the worst argument you can?
"$0.13 of usage per $1 spent"
So I spend a dollar and I get 13 cents worth of usage?
I guess it means the otherway around but I'm not seeing how that phrasing works. Are they paying a premium to access US models?
It turns out to be a pretty complex program to solve at scale. Token fraud is a lucrative market and the adversaries are surprisingly sophisticated. It's a cat-and-mouse game, accelerated with AI.
(If you'd like to work on this, we are hiring :))
Client-side detection can always be sidestepped, and you need to intermediate the actual inference to get enough signals to make an accurate prediction. There are hundreds of listings for cursor tokens/credits right now.
We use canary values to detect the resellers, and I believe that's the only approach that will actually work at scale.
Do these numbers make sense? $0.13 usage per $1 spent?
I disabled automatic downgrading/rerouting because it sometimes takes me a second to tell when the answer came from a different model than I wanted. You could easily sell Opus as Fable for a good while.
There are quite a few other mitigations that could be done by providers that aren't mentioned in the article.
1. Tokens are model-specific: e.g. tokens used by Anthropic cannot be used in models of other companies.
2. Tokens are generated by GPU cards. They measure the power of GPU cards.
3. Tokens cannot be separated from the models. You sort of "connect" the software part (models) into the hardware part (GPU cards) to use the tokens generated from the hardware.
GPUs "generate tokens" in the same sense that human feet "generate steps"
You can't compare token counts across different providers to get an absolute measure of "total work done" for the same reason that you can't compare step counts across different people to get an absolute measure of "total distance traveled"
You should do a bit of reading on what a token is. The short answer is that it’s a series of 2-4 bytes of information turned into an integer.
Your comparisons are akin to asking “are amazon gift cards the same as a bunch of pesos?”
At $1 of usage for $0.13, the reseller is making a tidy profit on top of whatever subscriptions they're reselling.
I assume most of it is just people who want cheaper access to these models and don't mind subsidizing access via somebody who is simultaneously distilling the model.
This is about controlling who gets to use the tokens for what, not about payment fraud.
This is about people circumventing the model company's attempts to protect their intellectual "property" (which, if you insist on that incoherent usage of the word "property", they themselves stole from the rest of us).
It's equivalent to buying a DVD in the US which is region-locked to Asia. Grey market, not black market. If you use a stolen credit card to buy that DVD, well tat's a totally separate matter.