How accurately calibrated is Jev?(maximumeffort.substack.com) |
How accurately calibrated is Jev?(maximumeffort.substack.com) |
> I decided to check this on questions where the answer is well understood. For example:
> A classical particle of mass m is embedded in a system at thermodynamic equilibrium with temperature T. What is its velocity v?
If I feed that into a model, the answer I want is: “the combination of the model and the provided state has nothing useful to add to your prior”.
If I want to know the Maxwell-Boltzmann distribution, I can look it up or I can derive it or I can ask a fancy LLM to do it for me (at the cost of some reasoning tokens and some time - unless I’m using an ultraspeed inference system, I’m not getting this answer in 50ms). [0]
Similarly, if I want to know that 73% of incoming customer support requests are spam/fraud, I should measure that - it’s a property of my system, it takes some manual classification and a database query, and it will be a different percentage than your customer support system would see. I neither expect nor want my classifier to know this (unless I’m using a conventional classifier manually trained on my data, and the whole point of Jev is to avoid this).
What I want out of a system like Jev is to tell me how the probabilities change as a result of the per-sample data I provide. Which, is the case of this Boltzmann distribution question, is nothing: I provided no data and the classifier can infer nothing.
[0] A really good answer would observe that the answer depends on the dimension of the system (probably 3, but 2D systems are a thing) and also on whether the particles are hot enough for relativistic effects to matter (probably not). And maybe a good answer would check whether the material is a gas - the answer for a solid is not the same, but I suppose that’s not classical. Oh, and one shouldn’t forget drift: if you have a classical particle in a moving fluid or a classical charged particle in an electric field, you will again get a different answer.
Yes, I’m being pedantic. But if you want good answers you should be pedantic, and the Jev-like model is not where the pedantry should go.
The well-defined problems aren't well-defined in this sense.
If you asked a set of humans to generate a random distribution of heads or tails from coin flips, it wouldn't be similar to a real world distribution of coin flips either because we also have our own biases. [1]
[1] "Heads or tails?"--a reachability bias in binary choice" - https://pubmed.ncbi.nlm.nih.gov/24773285/
Claims like this needs to be deeply analyized. Models do have some emergent capabilities[1], and I think there's a lot of evidence to show that semantics is actually learned (Word2Vec), and some math seems like it also might be learned (e.g. modular arithmetic). But it's hard to exactly say where there's some internal mechanism generating a true answer and where we're just getting lucky with some distribution so the answer just seems right.
IMO the lesson here is: even in trivial cases, Jev's outputs are just ~reasonableness scores which do not correspond to actual probabilities. They should not be treated as actual probabilities without careful calibration and plenty of meta-uncertainty about how well that calibration extrapolates.
The problem is, most of the value proposition of Jev is that it gives you the probabilities without doing that, which it doesn't.
[0] https://kantahayashiai.github.io/posts/jev-does-not-play-dic...
I suppose trying to interface with the model like this is like asking an LLM how many times the letter E appears in a word - just not the correct way to ask that given its model
Of course, it's impossible to know for sure what was LLM processed or not, but this post got classified that way.