Every major AI shop has a ton of in-house classifiers already, big, small, generalist, specialized. Some are used in inference pipelines (e.g. safeguards), some are used in data preparation, training, various one-off and intermediate tasks etc. Offering them on a public API doesn't always make business sense. I don't see much substance to this buzz, looks like people that are new to all this are discovering that classifiers exist, they are more efficient at classification, and many tasks commonly done with generative models are classification in disguise. Which is not bad at all, a fresh look at their use is great to have.

This article is extraordinarily hard to read. It’s tummelvisioned on OpenAI and things like tool calling which are only relevant to the extent that llms have been tuned to make relative choices, but this applies to all LLMs. Also, some really outdated references. LLM written, perhaps?

Also, moat discussion is the lowest form of discussion. I don’t care if jev has a moat. Did it get the interface right? What other past ideas have we overlooked that if given some love, could kick the door down like jev did?

Really silly stuff.. people wanting to talk about moats when there’s no castle. Moat talk merely projects the illusion of being engaged but, much more often than not, it’s hollow engagement.

I find it unlikely. OpenAI is all in training models with reasoning with RL, and Jev-like models are the total opposite. They are made to not reason at all to be fast. If you want to add reasoning on top, you might as well use a conventional LLM because you lose the price and speed benefits when you output auto-regressive tokens. I don't think OpenAI will even bother with this.

> My main assumption is that Jev is using something quite close to a conventional large language model. As evidence of this, Latent Space reports that many of the early clones are indeed LLM-based.

Not proof that this is the case with Jev though. It might use non causal text encoder for the state, which could make sense given that it's very good for its price.

There is one benefit that Jev has, that it is not OpenAI and thus it's probably less likely to steal your own work.

I'd be surprised if they weren't aqui-hired by one of the big labs as a unicorn.

The ability to use classifiers under the hood for the larger models has the potential to dramatically improve cost and throughput, allowing them to increase margin on a very similar service.

For context on what "Jev" is: https://news.ycombinator.com/item?id=49717558

Rather than focusing on OpenAI in particular, let's just say that there are many smart people at other AI labs and if it seems like it will be popular, this technique will probably be copied. What would prevent them from adding another API?

Hopefully there will be some decent benchmarks and gateways for switching between providers easily.

Isn't this more and more likely on all shapes of model evolution? The providers will all copy each other.

And in this case I hope its true. I've been classifying a lot of email and while OpenAI `text-embedding-3-small` has been very helpful for fast and cheap embeddings, initial tests with Jev are very promising and much more ergonomic.

I put more thoughts here: https://housecat.com/blog/classifying-email

If OpenAI releases something similar to what Jev does, then that would be like admitting defeat. Their whole spin is AGI and world ending danger. Why would somebody with an AGI at home make something like Jev which is intended to be a part of some SW the AGI is going to replace anyway.

I'm not exactly following through with the claim, can someone explain how the built-in classification would not necessitate more tokens used, or be much different from turning on reasoning? Not that I don't see the difference, I just doing see how OpenAI would do it well.

comments are pretty weird here, there's no real moat to what jev is doing, it is certain that frontier labs are going to release their own jev and there are even open source alternatives (although nowhere near as accurate as jev).

so maybe typesafe's real plan is to front run and releasing their own new models for some time until they can get acquired which seems to be the only rational objective

I think the article is part right and part wrong.

The right part: autoregressive LLMs are indeed generating “probabilities” (scare quotes very much intentional). During pre-training and any SFT steps, those probabilities are nudged toward the probabilities, over the training distribution, of the next token conditioned on the previous tokens. (This is an explicit property of most training recipes: KL divergence is a “proper scoring function”.)

So if you prompt with “Paris is a city in ”, the next token probabilities estimate the probabilities over the input distribution that the next token in the sentence is the first token of France or of something else.

But there are huge caveats:

1. That is not at all the same thing as the probability that Paris is France under any distribution that you care about (the population of the various Parises, for example).

2. None of this necessarily usefully applies to RL or, as the article discusses, tool calling. The output probability of a tool call is not some Platonic idea of a probability that the input is worthy of a tool call. It’s a the result of a training process that tried to teach the model to be useful and to achieve its goals.

3. I suspect that reasoning makes this all much worse. Suppose that you prompt with “a help desk user with IP=a.b.c.d says they’re ‘in Paris’. What country are they in?” The model has been trained to generate a reasoning trace, which may well start with “let me think of where Paris could be. It could be in France or in Texas etc. The user was speaking English…” See the problem? The model is reasoning well, but it reasoned “France” before “Texas”, so the logprob for France was probably higher than “Texas”. At the end of the reasoning trade there will be an answer, but the logprobs for that answer are, at best, some representation of the probabilities of the answer conditioned on the sampled reasoning trace. And that is not the probability distribution that a Jev user wants.

Not directly related, but still jev related:

Would it be intesting/useful to use jev to generate a block of text like LLMs do ?

Like asking it to pick the n + 1 word given the starting text (using it's choice primitive), but also asking n + 2,n+3 and so on at the same time.

Would it give coherent or useful results ? Or does the fact that it computes it "all at once" means it cannot make one of it's answer influence the other ones ?

Jev doesn't code. It can be used with LLMs to simplify coding and token consumption, but still and LLM is needed. Will they complement each other? How can Jev replace LLMs? Are they even competing?

Can't they eat everybody's lunch simply by typing "Astra, please copy this product?"

They certainly have the token budget for it.

I’m shaking my head in disbelief.

Reading logits is the cornerstone of ML. It’s almost like many of the people reporting on and fawning over AI have no technical background and never knew about ML classifiers or calibration.

Even the article itself has the title as a question: "Will OpenAI eat Jev's lunch?". A more useful title would be "OpenAI is Positioned to Compete with Jev".

love JEV, but you are right - how much time before a Frontier Lab release the same?

Jev certainly feels vulnerable but whether it’s oai or someone else that goes after them seems unclear.

Wouldn’t be surprised if every single AI house spins up a copy

But like they usually also have an embeddings endpoint

I think the original idea originates from this author : https://laya.convaiinnovations.com/

maybe open source even eats Jevs lunch first

see here: https://news.ycombinator.com/item?id=49800574

Personally, I don't really care what OpenAI does here. What's going to be far more exciting is when DeepSeek, Qwen, or GLM start integrating classifiers into their open models.

I'm looking forward to next week when we never have to hear about Jev again.

I'm confused. Why OpenAI and not Anthropic? I don't see anything here that is specific to OpenAI.

gosh, for wanting TypeSafe to survive, this fellow just handed OpenAI detailed instructions and ideas for defeating them...

A lot of business automation doesn't need AGI, doesn't want to pay for AGI if they don't have to, and would be better off using a classifier than something increasingly unreliable with a mind of it's own.

There are many automation pipelines that use LLMs because there was no choice, but the multi-way classification that Jev provides is exactly what they need, and is going to be way faster and cheaper, as well as having the benefit of calibrated probabilities and structured output that can be relied on.

I would assume that they would fold this type of classification into their responses API next to existing ancillary tooling that they already ship. I think OpenAI positions themselves as being wholly focused on AGI - but I think their business model leans towards lock-in via superior tooling (Codex, ChatGPT, etc.). This feels like an easy win for them without muddying the larger vision.

Why release an image model, or a video model, or custom agents if AGI will just make them all obsolete?

Why build codex if AGI will replace SWEs?

Why build excel integrations if AGI will replace spreadsheets?

I didn’t realise they were competing. If anything Jev seems complimentary to LLMs.

The people that buy into "AGI is here" and the people that understand what Jev is and how useful it would be to hook it up to an LLM are two separate circles, so there's no "defeat" being admitted.

ASI isn't here yet. It could benefit people in the interim and help make the ASI.

It's just another tool. Luna exists for a reason: it's the right tool for the job. If they release AGI and it costs $1 and 5 seconds to decide "is the customer asking for a refund", then that's a terrible use case for AGI if another tool can do it with 95% accuracy for $0.002 and 50ms.