(1) they are claiming to produce apparently bijective closed-form symbolic representations/approximations of, among other things, LLMs. Is evaluating these closed-form representations more computationally efficient? The implications of that are potentially huge. It would be essentially analytic distillation. Fable on a chip and not a data center would be important — and disruptive - in many ways.
(2) Unsupervised, and even supervised, symbolic approaches to problem solving break down due to combinatorial explosion, among other things. This could potentially allow us to treat LLM training and inference as a search algorithm for novel symbolic approaches to solving new classes of complex problems hitherto unreachable through other approaches. If that works, I suspect it’s a feedback loop, too - the learnings from one representation push advances in the other. This would also increase the economic value of large training runs, since the model itself is now valuable, not just its inference.
(3) Per the above, can this push LLM design to greater capabilities?
The relationship between this and Anthropic’s J-space observation is also interesting. This is much, much deeper and more directly actionable, though.
EDIT: I ran my questions through Sonnet — yes, I appreciate the irony — and it was none too sanguine about questions (1) and (2), but thought (3) was reasonable. In any case, this is quite the paper. On reflection, I do think that the apparent reliance on very simple symbolic representations and tasks is underwhelming. But the approach is impressive. And obviously this is still early days, and the value of building a bridge between the very fuzzy LLM models and the rigorous, mechanically provable models would be enormous.
> "Second, there is no guarantee that a given neural network can be approximated by DISCOVER"
They train what appears as embeddings for outer product of roles and fillers. The role for language model can be a position in text, the filler can be an embedding of a word at that position. Then that matrix of a sum of these outer products is linearly mapped into NN encodings and then decoded by NN decoder.
The embeddings learned by this process are not necessarily smaller than original ones. Given that they participate in an outer product computation gives me impression that the resulting sum is much bigger than actual NN encoding, that is why it needs to be linearly mapped into NN encoding.
So, this paper will not necessarily lead to any computation savings.
But I am at page 6. ;)
That in itself is of course not super useful for tasks that we don't already know to do symbolically, but it's nice to see that they can manipulate the keys and values of their representation somewhat independently. So at least some of what LLMs do internally seems to involve a bit more structure than just linear combinations of some overcomplete dictionary.
The interesting question is whether this can be used to interpret the operation of a single layer by lifting it to operate on key-value stores instead, replacing matrix-vector products with reading from some keys and then writing to others. That could potentially move the balance of power more towards CPU inference instead of GPUs.
Well that's good to hear. Would have been rather exasperating to find out we could have somehow gotten Fable or Sol out of sufficiently advanced sparse-coding ages ago.
No it is way worse since computationally they are same things with symbolic having overhead of attending to the symbols.
I'd imagine that our brains have become similarly biased towards generating such a generalised symbolic structure over hundreds of millions of years of evolution?
I'm thinking how certain regions of the brain reliably represent things like motor skills, speech, abstract thought.
When symbolic algorithms are that large, they're equally hard to reason with most of the time. The upshot would be a lot more storage required in exchange for more generalized computing, lessening the need for so much GPU in a lot of cases.
I don't see why a model couldn't be represented that way. After all, if you just polled the output of a model, you could evolve genetic algorithms to predict it with fairly high accuracy in a limited domain. Take that out to the Nth degree and you're basically just unspooling the model into a giant set of equations.
The hope would be that this unlocks some substantially more efficient or parsimonious math that would fit better on a chip. And that’s clearly my words, not the authors’, per the comment above.
My forthcoming paper at EMNLP offers an alternative that instead grounds the notion of representation in a very simple notion of the effect it has on model learning/behavior when you adversarially perturb it. For example, if I tell a model that in the context "I saw a duck quacking" it should replace 'duck' with 'glam', how much does it desire to replace 'duck' with 'glam' in "I need to duck out of the meeting" vs. "At the park a duck protected her ducklings." This method turns out to work quite well, and as we use only a single example, avoids the need for supervision.
The linked paper argues that their method, DISCOVER, is not supervised in the same way as DAS, since it does not directly optimize for causal effect. I have only skimmed this, but I am not so sure it might not suffer from a similar issue. They're still supervising to align representations with their underlying hypothesis, even if they don't directly supervise for causal outcomes.
Refs
- Hewitt and Liang 2019. Designing and interpreting probes with control tasks
- Kumon and Yanaka, 2026. Fine-grained analysis of shared syntactic mechanisms
- Meloux et al., 2025. Everything everywhere all at once
- Rozner and Shain 2026. Perturbation: A simple and efficient adversarial tracer for representation learning in LMs. https://arxiv.org/abs/2603.23821
- Sutter et al. 2025. The nonlinear representation dilemma
That's pretty cool. I hope I've got that kinda-right.
Instead of distilling the geometry of a model into a huge knowledge graph, we start from the largest known open source graphs and build it up towards something that resembles this geometry.
Come and join us. Discuss on github.com/latentpedia. We have the basic tech covered. Need more compute, storage and enough business to cover the cost of serving.
Huh? Do you mean “isn’t especially conscious of the ingredients”?
Though, that interpretation seems confusing, because it seems to be meant to be contrasting with “macro slicing dicing stacking them” which doesn’t sound more attentive to what the ingredients are than “micro level spice and heat”.
So, I’m having trouble understanding. (This might be a me problem.)
They say this holds in... Some examples they found?
I don't enough about this area
Just going from 2D to 3D creates massive new positional potential (e.g. surface of the earth, vs. the atmosphere above earth...).
Now imagine 1,000 dimensions.
Training identifies these dimensions in the training data and links it with each word/token. Then given a stream of such tokens, each with its own set of dimensions (which can be huge), and LLM predicts the dimensions that the next token is most likely to have...
This shows a deep misunderstanding of the paper's claims, which in no way challenge the established view that these bots are next-token predictors.
Regardless, if all you want is a next-token selector, save your money and roll a die.
No, this shows an appreciation of the symbolic richness behind that token 'prediction' which the paper leads on.
> Regardless, if all you want is a next-token selector, save your money and roll a die.
Tell me, where is the emergent symbology guiding that dice?
I mean, of course they are, that's literally what the inference loop does. You can look at the source of your favorite model runner and you'll see exactly that.
What I find misleading about this term is that it focuses attention on the "next token" part and glosses over the "prediction" part as some sort of unspecified "statistical algorithm" - even though this is where most of the work happens and where the interesting questions are.
e.g. quickarxiv.org/abs/2608.29530
Thus putting things similar to neurons in a network and making them able to learn could create behaviors similar to the brain. The fact that that biology used chemistry + electrical signal and computers use ReLU-like activation could be merely choosing the most efficient way to enable training.
That is, yes, ANNs are not brains. There are countless differences. But are there differences at the computational level? ANNs are meant to model brain computation, not brain biology.
(There is still a lot to debate there, I'm not saying "ANNs are perfect computational models for the brain")
their algorithm is even named "DISCOVER" so they set out to discover the connective tissue of why the universe has invariants like math, and lo it was discovered.
i guess good job for having credentials & publishing the math so people 2years behind the curve can learn from your tenure?
yes. large matrices can gradient descend to understand arbitrary symbolic logic.
ENGLISH IS INSUFFICIENT but it is at least a few decades of math proofs & progress :) welcome to the future Slackernews
If this is true and easily computable, this might have big impact in AI safety, as it seems to be really lacking today.
Do you have a simple explanation of why this level of proof is not sufficient?
I mean, no disrespect, but that's the core of your argument, yes?
To my reading the nature of supervision creating the structure seems to be the core of the argument.
The trivial example that comes to mind is the character customization sliders in many video games.
Agreed, I don't think anyone has a good grasp on, say, 64K D, but we can do a bit better than just 3.
That is why the scam works, because investors are humans...
A good example is https://richardaragon.substack.com/p/a-universal-prime-funct... describing a supposed "universal prime function" which is simply a finite approximation using a sum of 50 sines (each applied to a linear term plus a sine-log offset). The 53 parameters are fitted to the first 10^3 or so prime numbers. This is followed by the *absolutely ridiculous* claim that if the function approximates the first 10^3 primes well, it must also fit the remaining prime numbers (of which there are infinitely more than 10^3000000000) equally well.
Then they suggest "A formal proof connecting this function to the RH would involve the following steps" using this great discovery: "1. Correspondence with the Explicit Formula: Demonstrate that the oscillatory correction term in our function corresponds to the sum over zeta zeros in the explicit formula for ψ(x) or π(x). 2. Error Bound: Prove that the error in the prime counting function derived from our function is bounded by O(√x log x). 3. Contradiction: Show that if any non-trivial zero were to lie off the critical line ℜ(s) = 1/2, the error would exceed the bound, leading to a contradiction."
This isn't even midwit math.
It's the kind of naive ideas I had as a high schooler, who was good at high school math and who knew how to code functions and plots in Mathematica, but who had no understanding of higher math. This kind of naive approach to RH signals that one doesn't even understand the problem.
that's all. easy to call things "vague slop", hard to be right first. Jensen's def'n of intelligence "can you see around corners and predict the future?" kinda all that matters at this point.
just an fyi. congrats on your nitpicking of what math counts as math lol
here's my own paper: https://computerfuture.substack.com/p/demoting-laplaces-demo... (pdf - https://drive.google.com/file/d/1s4VER7SNjUFuCBodX_0myXRpNCW... ) and here's my MIT degree on the blockchain somewhere: https://trattner.github.io/img/18c-diploma.png
feedback welcome from nits... you made zero comments on ghostbasin.com which makes some extraordinary claims too.
In the way digital is the symbolic representation of physical, ai programs may have internal symbolic representation. This paper shows that nn vectors can be closely approx with symbolic structures.
The paper claims no symbolic richness beyond that evident from the undisputed next-token prediction.
> Tell me, where is the emergent symbology guiding that dice?
There's none. That's my point.
formalisms come only after there is understanding of the phenomena and a specific need to communicate concepts across members of community - in a concise and commonly comprehensive way. my understanding is that education often fails to train understanding and jumps directly to formalisms. and this is right also for STEM schools, where it is presumed everyone has some innate inclination to the matter. but this presumption, in my experience as university teacher, is not entirely correct, and students are not universally prepared to express lack of understanding, and particularly when it is expected from them to rather indicate understanding.
it seems very weird, the concept, to reward the correct answer, while suppressing and discouraging the expression of misunderstanding or confusion. why is weird? because learning comes trough trial and error, and the error part is half of it. so we need to encourage people to express their errors or lack of understanding, so that we can then fill in the gaps and eventually stand on a common ground. sadly, many forums, HN included, does not cherish misunderstanding. our posts here mostly get upvoted whenever they cross a sort of threshold about the amount of truth in our writing, and not because of daring expressions of confusion.
On the other hand I deal with higher dimensions better than most, possibly because of my aphantasia.
65 years is a long time to be beating your head against an obvious dead end.
The article intro presents the claims as self-evident, when they’re not at all.
But, someone rise pointed out this may have been a dig at those attitudes, which makes more sense.
He's the axis of this particular group of researchers, being the most senior at the place where they all met, Johns Hopkins.
So this is less a straw man and more a quick reminder to his peers: "Right, so, remember this particular thread we've spent the last 40 years hashing out, here we've got another contribution to that particular conversation."
This is exactly why Fodor argues that psychology should be explained on its on level with symbols rather than appealing to neurology. But if you're interested in modeling symbols, there's much better options than ANNs (see nearly any programming language ever).
>ANNs are meant to model brain computation
But we don't really know how that works! So if you know if you're not modelling the low-level behaviour right, you can't assume that there's a correspondence of the higher level computation when you don't really know what that higher level computations are.
But we do have a hypothesis: that it is done by a large number of simple units with very high connectivity and in deep layers. This is what neural networks model.
Personally I was skeptical of this model of the brain, but they have achieved remarkable success in practice, as well as Nobel prizes. The neural networks people may have been onto something all along (I say that grudgingly).
I've seen various posters talk about "stochastic parrots" or about how LLMs were just "using very simple word statistics" to get their results, which sounded to me very much as if they thought of LLMs simply as glorified markov chains. I would consider that a big misunderstanding.
"In machine learning, the term stochastic parrot is a metaphor that frames large language models as systems that statistically mimic text without real understanding." https://en.wikipedia.org/wiki/Stochastic_parrot
"Large Language Models as Markov Chains" https://arxiv.org/abs/2410.02724
Seeing LLMs for what they really are will also make it clear they are fundamentally unfit for a lot of tasks they are currently marketed for...
They're borderline useless and certainly potentially inadvertently malicious for writing, customer service, speech to text, writing large amounts of complex code, therapy, medical diagnostics... the list goes on
Sounds like you're part of the problem?
It's really a serious problem just eroding the fabric of society in real-time. Being complacent in it or believing in the promise is just wholly foolish and bad for everyone.
It's been several years now of LLMs only appeasing those with low expectations and inexperience. Unless the only goal was generating boilerplate or really sloppy proofs of concept, LLMs are a waste time for everyone else. This argument is so over already. We're all just hoping for a soft landing when the hangover really kicks in.
But I don’t think it is possible for us. At least, for strict senses of visualization. I suspect that the architecture of our brains doesn’t support it. This is rather speculative because I don’t know basically any neuroscience, but I wonder if the whole “the brain is largely a very wrinkled surface (though with several layers to this surface), and the retinas are also surfaces” may have something to do with this. A retina in a eye-4-ball would have a 3D boundary (I.e. hypersurface), and I imagine that the amount of information that it would take in if it had an at all similar angular resolution, would be too great for our visual cortex to be able to represent all of it.
Of course, our vision only really has the high resolution it seems to everywhere in the narrow region we’re directly looking at, but still.
I mean, there are people with aphasia and hyperplasia - how do you prove actual 3D visualisation.
There are games that people play that need 4D visualisation to be able to do well and they perform at that.
I'm sure someone will point out something like the 4-color theorem as a counterargument. Where is that kind of theorem proving in this generation of AI? We seem to have hit a dead end rather quickly.
"Amish do not view technology as evil, and individuals may petition for acceptance of a particular technology in the local community" - https://en.wikipedia.org/wiki/Amish_way_of_life#Use_of_moder...
See also https://thereader.mitpress.mit.edu/how-the-amish-let-technol... about some Amish use of cell phones with adapters to be more payphone/landline-like and https://news.ycombinator.com/item?id=21349989 was a discussion about the use of pneumatic machinery by the Amish.
Maybe you forgot that important tidbit?
People are using them because they're being shoved down their throats and they're complacent.
Software quality, maintainability, exploitability, morale, competency are all at all-time lows and just worsening.
It's really bad to defend this.
Then they removed the approve everything manually mode and I thought, maybe I'm the one out of touch, I should try vibe coding. I gave it a big spec of things to implement and I was pretty vague on the details, but I was kind of hoping that the AI would get it right anyway.
I noticed that when you let the AI just do its thing, you spend a lot of time staring at the screen waiting for it to finish. No wonder all these people are obsessed with parallel agents.
Anyway, after a while it produced the end result and it was pretty bad and I don't even mean the code. This was with gpt 5.6 Terra. Definitively not the type of "did a week's worth of work in one hour" type of deal.
Meanwhile if you guide it with your own technical skills the end result is a tireless code generator that operates at 80% of the quality of your own code but is only roughly 1.5x to 2x faster than doing it yourself. The biggest multipliers come from making the AI do things humans don't want to do, like writing a bunch of thorough tests but those don't translate into increased productivity, the opposite in fact. They translate into higher software quality but if you vibe the tests and don't review them, even that disappears.
Got annoyed enough at this to build a FOSS tool for it: https://github.com/Roy3838/Observer.
A small local LLM watches your screen and notifies/calls you when the agent finishes or stalls on a prompt :D
i'm a worse programmer now because of it
