AI Isn't Outthinking Mathematicians. It's Out-Remembering Them(davidepiffer.com) |
AI Isn't Outthinking Mathematicians. It's Out-Remembering Them(davidepiffer.com) |
Precisely perfect for replacing lawyers, if nothing else..
Why would you want to replace your lawyer with a set of tensors that does not actually think and makes mistakes like this? Lawyers tend to get hired in high stakes situations. Why wouldn't you instead say that this would be a great tool for lawyers to use judiciously in researching precedents, etc?
I don't understand what people are doing with models that makes them assign agency or intelligence to them. When I manage to forget the financial fuckery of the AI buildout and its implications, when I manage to forget scaremongering by loathsome CEOs, I still have the same fascination and excitement at the idea of LLMs as I did when I was playing with the GPT API prior to the release of ChatGPT.
LLMs are, to me, truly amazing tech. It's so fascinating to me that they now DO have emergent properties that look at face value like reasoning and intelligence. But every day that I work with them, I am repeatedly clobbered over the head with the fact that they do NOT reason and are NOT intelligent.
Why can't we be fascinated by emergent properties of intelligence without immediately jumping 10 steps into the future and, like a limit in calculus, assume that "this is it-- we're on the cusp of AGI"? To me, the fact that LLMs can combine existing ideas that people hadn't thought of combining in solving a novel problem is extremely cool. But my first thought is-- this is an amazing new tool for mathematicians and researchers. Instead, most everyone seems to jump the gun to the "humans are obsolete next year" conclusion.
I think the real scandal is that we are almost 3-4 years into this (I think the release of GPT 3.5 is a good marker of when this public frenzy started) and all we’ve seen is OpenAI and the other major AI frontier companies constantly retracting their preposterous claims every time. We appear to have reach a local maxima in that it has some value in places that tend to be a little easier to scope and limit (computer programming, mathematical proofs). So, given the actual useful economic value this has provided, does this justify the investments? I think we are approaching 1 trillion in CapEx for AI [0]. For context, I believe the annual GDP of Norway is $600 billion.
[0] https://www.fool.com/research/ai-companies-spending-on-data-...
Whereas I see that and say - if we properly funded the sciences we could have had a bunch of grad students tackling that problem and found this application 20-30 years ago. Sure it's 'nice' that LLMs can fill in for people in brute force work like that but people are perfectly capable of doing that work and if we focused on properly staffing our research institutions we would achieve a lot more a lot faster. Instead this is obviously going to be used to replace staff and further reduce headcounts.
Yeah, as expected, an article about AI that's at the very least been polished using AI. For fucks sake we need an LLM flag to filter out slop.
Computers are simply better at math now, like in chess or go!
I don't think I've had a truly original idea in my life. Combine A + B, when it's rare for people to know A and B at the same time. So from that perspective, what LLMs are doing is basically the same thing. Sometimes I am faster than the LLM because my context might be better organized, but it typically needs just a hint from me to steer itself correctly. It claims something is a memory leak, but smelling a rat, I suggest it to double check the garbage collection statistics too, at which point it's clear it's no leak, but a tuning error, at which point the LLM is better at tuning than me, because it has more energy than I do.
Maybe there's true brilliance out there, when something doesn't come out of combining data and building hypothesis until you get really lucky. My experience is not comprehensive. But I look around me, and it sure seems I've not been lucky enough to see it. Even the shiniest people I've worked with, which most of the audience here would recognize, have never shown me that they can go past this.
The exam was huge, at least over 10 pages, and even when we technically ran out of time, the professor was kind enough to move the remaining exam takers to the neighboring lecture hall to continue taking it. I recall I spent a total of 2 hours on that exam.
Now mind you it was mostly short answer or multiple choice questions. The multiple choice questions were pretty sharp too, lots of traps and false but sounds right answers mixed in. But if it had been purely essay questions, I would have been screwed.
However with such a huge corpus of information in front of me, I ended up basically learning all the material on the spot. I just kept doing multiple passes through it, each time I noticed one of my answers contradicted one of the others, I would make adjustments to harmonize, which indirectly refined my understanding.
In the end I got B+ in the exam (which was curved to an A), and walked out understanding the material better than I did walking in.
Reflecting in the experience years later, I've wondered if a hypothetical LLM which was ignorant of microbiology could do the same thing if fed that exam. In some respects the traps they placed in the multiple choice questions actually were what helped me refine my understanding the most. Made me appreciate information theory more.
I know people that got to post grad math without understanding a thing but they could remember a lot easily, while many of those that understood but had a harder time remembering every last variation of everything got penalized.
There’s no such thing as a truly original idea. It is all just combining A+B!
I’ve met different people throughout my career whose intelligence came in 1 specific area. For instance, my friend is extremely good at trivia, he clearly has a lot of storage and can access it easily. I think I’ve only met one person who was excellent in all three areas of intelligence.
Obviously this is a simplification, but it’s how I like to illustrate my ideas on intelligence at parties and first dates.
Understanding especially in the context of unknowns is what intelligence is.
If a time traveler went back to 1600 and started spouting off about differential equations everyone would think them quite mad.
Even in a debate, if somebody just has the ability to remember tons of facts and figures, the other person will seem unintelligent by comparison, even if the other person is correct
Most humans cannot incrementally contribute since they don’t have many traits required to do so - extreme discipline, imagination etc.
We literally live off and benefit from the investments of the few, in relative terms.
> Many people's model of accomplished mathematicians is that they are astoundingly bright, with very high IQs, and the ability to deal with very complex ideas in their mind. A common perception is that their smartness gives them the ability to deal with very complex ideas. Basically, they have a higher horsepower engine.
> It's true that top mathematicians are usually very bright. But here's a different explanation of what's going on. It's that, per Simon, many top mathematicians have, through hard work, internalized many more complex mathematical chunks than ordinary humans. And what this means is that mathematical situations which seem very complex to the rest of us seem very simple to them. So it's not that they have a higher horsepower mind, in the sense of being able to deal with more complexity. Rather, their prior learning has given them better chunking abilities, and so situations most people would see as complex they see as simple, and they find it much easier to reason about.
I once tried out his Anki approach during a math lecture. Whenever I reiterated a card, say about some lemma, I noticed something interesting about it. This was delightful and many lemmas became much more streamlined over time. It's not a "solution" to mathematics, but I found it delightful while it lasted (before akrasia or lack of time kicked in and I stopped doing it).
But AI agents have no such limitations and can publish and re-use negative traces easily. There have been some recent projects (https://www.theoremdb.org) aimed at exploiting this fact. https://news.ycombinator.com/item?id=49227505
In general though, LLMs do not have the same limitations and incentives as human mathematicians, and the next year's tsunami of change will make this abundantly. clear.
The simpler explanation is that a working memory is a requirement for intelligence, and a larger working memory will make you more intelligent. Hence the AI can in fact be more intelligent than the mathematician.
I use coding agents. I think they're pretty good overall. They save me a lot of tedious coding. For example I probably wouldn't spend the time to implement native splash screens for all the build targets of a Flutter app, but I'll have the coding agent do it.
Nevertheless, for all the time that we've had coding agents, it's still trivially easy to find the jagged edges of their training. For example, Gemini evidently doesn't know if the Xcode part of a Flutter tool chain is misconfigured. That's not exactly a Millennium Prize problem. But it is shaped wrong for a training set for a coding agent.
I think where you could say it is out-remembering us is when it can contemplate the vast universe of patterns, gleaned from essentially all human disciplines, encoded in its weights, that may let it draw connections that a human could not, unless they just happen to be familiar with multiple disciplines.
Which is why I think the analogy with Von Neumann / Einstein is also a bit off. From TFA it seems Von Neumann was more akin to what AI does, than Einstein. I don’t get the impression that it was Einstein’s memory but his ability to look at things from a radically different perspective. So far I don’t know that we can categorically say that LLMs can or cannot do that.
The age of humans comprehending things is coming to an end: our brains just won't have the capacity to make meaningful contributions to science, math, or technology.
This has major implications that haven't been fully realized yet. On the math side, there are long machine generated proofs. On the code side, there are high volumes of code with similar code not being folded into functions.
Also I suspect that, apart from that, the results on smaller, cleaner codebases are better. LLMs degrade when following more than N instructions (where N depends on the model) even if the context window is not full yet; I suspect they also degrade when code has too many unnecessary concepts and details
Has anyone tried feeding all of human knowledge to an LLM prior to Einstein's work and tried to have it reinvent physics?
Trying to convince us that mathematics and software engineering are "solved" is getting very tiring.
The pushback would probably be too much for the soon-to-be IPO-ed companies.
This also nudges into how to use it best: By knowing where the "piles" of if training data are (i.e. when it comes to a CLI in rust, I just briefly describe the use cases, and I have a very high confidence the code will work exactly as intended by me since there will be a multitude of examples in the training data), one can predict where the LLM is likely to go wrong an prompt/guard accordingly. This skill grows with domain expertise, and is one of the many reasons LLMs can be (and probably should be) used to outsource busy work, but never understanding and learning. ("never" is a not meant literaly of course - I for one am glad that I do not have to wrap my head around CSS and other frontend topics and go straight to the topics that interest me most)
"Out-Remembering" captures that perfectly, I feel. Also goes nice along with "asking it leading questions" as we know how to do in real live; if you want a person (LLM) to confess (produce output tokens) something, sometimes you do that by leading the interogation (chat, context) to where you think the truth lies.
It is the scary thing actually. Cause once AI makes arguments that require a working memory of hundred items, then we as humans will have no way of understanding the arguments…
We can decompose and write things but only up to a point. when Ai can have a working memory that spans hundreds of books, we are necessarily going to have to trust the system.
That doesn't follow. We could still understand it just by studying it and committing it all to long-term memory, it just takes longer. And there's a hard cap on the working memory of LLMs, due to the quadratic scaling cost of the full attention layers that have proved unescapable for all SOTA LLMs.
But was it?
It’s not just about out remembering, it’s about breadth.
Mathematicians are all about depth. It’s pretty much impossible to become an expert in more than one narrow field of mathematics.
AI is happily applying techniques and abstractions across these silos.
That's at least true for current journals, since they're supposed to be read by actual humans. I suppose one could imagine a sort of "AI" pure data journal that just "publishes" (in actuality aggregates) any sort of partial result. This body of knowledge would be entirely useless to humans, but could serve as a sort of "computation cache" for these stochastic systems.
The incentives are not.
The incentives are skewed towards "a magician never reveals her secrets". The results are presented as if a rabbit got pulled out of a hat, with a maximum ta-da! effect, and little backstory of how the hell did we get there.
Don't get me wrong, these things are discussed, often over beers (you better drink it you want to make a career in the field).
But not published.
The younger mathematicians are trying to change that with the blogging culture. But the professional incentives aren't there. (In corp-speak: can't put blogging on perf). They burn out.
That's why math blogs usually come from either the top dogs in the field, like Terrence Tao, who don't need to care about perf, or people outside academia.
That's one thing that I hope the disruptive/destructive effects of LLMs will force mathematicians to face.
As one of my fellow mathematicians sarcastically wrote¹, we've reached a point where we should become a cult because we're acting like one anyway.
The other possibility is, of course, that the shake-up will take us precisely into that direction.
My point here is that the real problem here is not mathematical; it's a social one: incentives and politics, organizational structures, policies, allocation of jobs and funding.
All of this directly impacts how we do mathematics, who we do it with and teach it to, how we teach and communicate, and, of course, what math we even do and look at.
Given that, I'm neither too worried about humans vs. AI standoff, nor hyped about the Glorious New Future full of AI-assisted discoveries.
AI or not, the organizational issues in the field are still there, as are the incentive structures (including the infamous publish-or-perish).
We are doomed, yes, but by our own hands and committees. And it's up to us, not the AI, to get us out of there.
The little shove from the AI might be just the thing we need.
____
¹ https://www.mcsweeneys.net/articles/an-open-letter-to-the-ma...
https://en.wikipedia.org/wiki/J._Robert_Oppenheimer#:~:text=...
sitzfleisch: the ability to endure or carry on with an activity
Something Oppenheimer did not have, apparently.
People go whole lives without being able to make it pan out.
Out-ralphing them, you might say!
AGI ≈ artificial stupidity × infinite persistence
That is also approximately what people have always done to succeed.
I don't know how or why this would be trained on behavior, but no, it isn't true anymore that models don't say things like, "Ugh," or "this is going to take hours and maybe we should stop here."
Theres going to be this field day of low-hanging fruit that ML can round up, but after that I suspect it will be in fits and starts as a “connection maker” rather than some proof producer.
It's not out-thinking, it's just out-remembering
It's not out-thinking, it's just out-working
It's not out-thinking, it's just able to consider more things simultaneously
It's not creative, it's just randomly generating things and then selecting viable ones
We have known for a very long time that computers and machines are much faster than humans, more accurate, are scalable in certain ways that humans aren't, and they don't tire. I think most people who are not in the "AI cult" would agree that LLMs and modern generative AI are really just an extension of those faster/more accurate/more scalable and never tiring traits. But there does seem to be (and I'm sure folks much smarter than I have quantified this or described it better than I can) a fundamental difference in how humans think, especially as it applies to what true "understanding" really entails, and for the ability to think up truly novel and unique things that are not just a rejiggering/recombination of training data. I believe those skills really are at the heart of human cognition, and as impressive as LLMs are in replicating what this looks like, there are plenty of "LLM failure modes" where it's clear that LLMs lack a true understanding of concepts or the ability to generate useful, completely novel ideas.
Could is carrying a lot of weight here.
Because, what's really happening is we're saying "Oh these things are what defines intelligence" then implementing them and /discovering/ "oh wait, there's more to this than we knew".
We've known, for decades, for example that an IQ test is not a measure of Intelligence, even though people still refer to it as though it is. A computer passing an IQ test, therefore, would have been thought of as possessing intelligence way back when, but would not now.
Oh, on the point of "creativity" - is a RNG "creative"? It creates a value unbounded by human intervention (in theory, yes Pseudo RNGs have limitations) - therefore it must be creative... right?
Your timelines are a bit unambitious. There's nobody expecting to make significant progress with a week of work.
You underestimate my ADHD.
Source: I am mathematician.
(1+x*y)^3*z+y^2*(1+x*y)*(4+3*x*y);y+3*x*(1+x*y)^2*z+3*x*y^2*(4+3*x*y);2*x-3*x^2*y-x^3*z|0,0,-1/4|1,-3/2,13/2
If LLMs are monkeys, somehow trained LLM weights allow them to model and prune massive numbers of universes in parallel.
As to your second point, Terry Tao already has an answer [1]: the proof isn’t the contribution, shared understanding is. This issue was already raised back when the four-colour theorem was proved. Machine proving and machine proof checking are useful tools but they don’t mean anything without the interpretative work and the communication necessary to build shared understanding.
Mathematicians have never been known to communicate their ideas very clearly.
Regardless, even that target llms will likely win - an llm will likely be more efficient at teaching me string theory than a professor in a room with 463 other students.
The llm is the shared understanding.
Why is it so hard to imagine we can build tools to think thoughts we can't comprehend?
If there's commercial value, I think it's inevitable. We don't fund mathematicians because it's cute when they understand a problem, but because their work tends to have applications with commercial value. The value can be captured without understanding the details.
Ask yourself: is that really the world you want to live in? It's a world where people, all people, are sidelined.
I think the happy ending of that path is something like Idiocracy. And the more likely ending is something like "automated capitalist economy without the people, because the people couldn't compete."
Proofs also enable AIs to direct search and generate knowledge. Verifiability is immensely useful for keeping AI grounded.
One might imagine AI generating enormous numbers of hypotheses and then trying to prove or disprove them, and then mine that data for new abstractions and heuristics.
That's something AI companies would really want you to believe.
Why would I care what they want me to believe?
Intuitively it would make sense that you can put math ability on a chart with a value for “general public” “smart high schooler” “smart undergrad” “smart PhD/ professional”. And you could place frontier AI somewhere on that chart over time from GPT 2 to now and see the trend.
Then you’d have to consider that either you believe there is a fundamental limit that is below peak human mathematician level or there’s not.
I agree.
> The age of humans comprehending things is coming to an end: our brains just won't have the capacity to make meaningful contributions to science, math, or technology.
I don't know if I see this being true for quite a while, if ever.
There's an infinite space of possible statements and proofs. The only thing that makes certain proofs significant is that human mathematicians consider them significant; if AI came up with a proof of some statement that no humans could understand then no humans would bother investing further resources in building upon it, for the same reason we don't waste computational resources iterating over the infinite space of true statements in first-order logic.
That sentiment makes me cringe. If you understand how LLMs work, you'd know it'll never be possible without a fundamental change in how these work.
We're also supposed to be reaching that point, somehow, without the LLMs ever being intelligent (in the dictionary definition sense, not the "high reasoning model" marketing sense).
Based on observations, the ones who are fooled by the supposed emergent properties, are just that, fools. Any sufficiently unintelligent agent will perceive transformer based LLM text predictors as possessing high intelligence.
LLMs in agentic harnesses are Turing complete.
To my best knowledge, we don't know of any greater computational model that the brain is a part of, that LLMs are not.
This is not to say that a human couldn't understand a streamlined version or that the AI would not be better if it made more streamlined statements to begin with.
(I am not saying that everything mathematical that an AI produces is in any sense trivial.)
Humans can’t compete with AIs on vastness of material they are familiar with, or the depth of effort they are willing and able to throw at a problem.
But scale isn’t the only aspect of difficult scientific endeavours. There’s also theory. And advancements sometimes come through hard graft of knotting together many things. And sometimes they come through the revelation of a deeper truth, or a new framework, a fundamental insight.
AI might help us reach the next level. But that doesn’t mean we won’t understand anything. It could be we have periods of vast intricacy we cannot follow, punctuated by profound elegance we (or at least experts) relatively easily can. And then the scaffolding we needed to get there falls away.
The tools we built to replace muscles have mostly obsoleted raw strength for tasks like excavating earth.
There's no reason to think we can't do the same for brains. And then we'll never need to think for a living again. Some people may want to do it as a commercially insignificant hobby, of course, the way people lift and compete in strongman competitions today.
We'll have AI taking care of our needs, the way a good mother takes care of their children.
Math is not magic, a proof is just a series of applications of a set of rules on some axioms. A mathematician could understand any proof given enough time to study it; the only way for AI to make proofs that a human couldn't understand is by making really, really long proofs.
I think perfect rationality doesn't exist, because it is rational to reject something that you don't understand. So rationality of a given physical system will always be bounded.
Law and medicine are fundamentally harder fields to obtain decent training data for, and LLM results are therefore expected to be less powerful. Also, making mistakes in these fields is costly, but perhaps you were alluding to that already.
But otherwise, mathematical proofs are read and written by humans, and at the end of the day the relevant standard of proof is what other mathematicians will accept.
Occasionally, mathematicians don't agree. For a prominent example, you can read about Shinichi Mochizuki's claimed proof of the so-called ABC Conjecture:
I think this bubble has given a lot of people software brain and are trying to apply it to fields it is wholly inappropriate for, though. Law is about argumentation and rhetoric. It is about providing a persuasive argument. This is how it is taught. The actual legal code is a way to formalize parts of it, but increasingly I see people angrily insisting that the only thing that matters is the text.
As you might imagine, I find textualism a load of applesauce, but I don’t think the vast majority of people making this argument even understand textualism as jurisprudence. It seems to stem from Crypto bros and the whole “code is law” argument which is just codswallop.
I actually always thought it was the opposite for me – I had to figure out how to work things out from scratch because I could never remember anything.
Source: the post-it notes, ALL OF THEM.
However I can only guess that this is important, I'm not absolutely certain. They're at risk of being an economic disruptor just by being extremely stupid (by how much they need to study) faster than us to the same ratio we jog faster than continental drift.
I guess whether he will eventually fix those gaps and resolve the issues remains to be seen.
Most of us turn out like the rescued exotic bird which turns out to be a seagull covered in curry.
And at this level, while there are anecdotical exceptions, mathematicians have always been pretty decent (with their conferences, workshops, paper publications, international collaborations, ...).
So, it does not mean "teaching the subject", it means "creating a human network of people that share the understanding". LLM can be useful at telling a human, but you still need a human. The point of Tao is not that LLM is not good at providing explanations, it is that "providing explanations" is not the contribution to science, "the human network" is. It's like saying "LLM are great cook, they generate tons of food in space", but the point of having cooks is so that people can eat food and not die. Having LLM generating mathematical proofs is as useless as having LLM generating food that no one can access: the point was never to "generate proofs" or "generate food", the point was "creating a shared human understanding" or "eating the food so human can survive".
I get that there is a cultural benefit to keeping it alive. Just like we ideally want the languages represented at the universities.
But keeping humans in the loop does not appear to be necessary in order to call it science, and certainly not in order to have progress or dessiminate that progress.
I don't have a problem with people doing math. As long that we don't idiomatically hold on to that way of doing things.
I do, however, find it hard to belive that individual humans will play a big role from here and forward, in any scientific desciplines.
I agree, but as a software engineer this gives me pause because I keep trying to insist on coding standards but I’m unable to come up with a compelling reason why it matters. Ostensibly the reason we cared about things like DRY and code quality was so that it would be easy to understand and easy to maintain and easy to make changes to later. But it now seems like a shared understanding of the codebase is less important than ever, and it’s more about shoveling requirements in without breaking any existing functionality.
Is a well tested slopfest better? That seems to be the conclusion for mathematics, so why not software too?
We already have countless examples of such filling up the arXiv, written by hacks long before LLMs started writing proofs. No one cares about them. You might as well build a box blasting radio static into the void. You could save a lot of electricity that way.
That's the part where LLMs are used as tools. Which there are plenty of places where they are useful.
Also, do you know what turning completeness is? Why are you bringing that up here?
The crowd that AI psychosis has brought to HN is interesting. But not in the "I'd love to learn more" kind
> But not in the "I'd love to learn more" kind
I hope you are able to see the problem in your own communication here.
Computation classes are interesting because they say something about fundamental capabilities.
Two machine that are Turing complete are in theory able to carry out the same computations. They are isomorph mediums of computation.
Regardless. Please keep it sober. If you think you know something, enlighten us. But don't just propagate out lies.
Turing completeness is not exactly a high bar, and it's genuinely confusing as to why you bring it up. Your C++ precompiler is exactly as intelligent as whatever is your favorite agentic workflow with whatever harness you're referring to. Both might be Turing complete. Neither are intelligent. But one of them seems to be fooling you to think otherwise.
There have been many times that the C++ precompiler produced some output I couldn't understand. I might even at some point thought it was trying to tell me something profound I was too dumb to comprehend. Turns out it was just a missing semicolon.
How would you not care? Are you a robot?
They can say random stuff with the goal of increasing their shareholder value. Things they spit out do not have to be true. It is not easy to verify things they say, therefore, everything they say should be taken with a huge grain of salt.
So that’s what I’m doing here. For what it’s worth I find a lot of the AI people’s worldview very consistent. They believed AI would be the most important technology of our life times and committed their work to it. Some of these same people are total liars so yeah I won’t really hang onto their every word.
For this to actually work in a way that benefits our species, humans will need to become something else/next through their interaction with the technology.
Anyways, sipping wine on the beach and doing puzzles when I feel like sounds nice.
The human brain is exceptionally efficient.
Would you personally vouch, at your job, for the importance of proper assembly coding standards?
That's the marketing pitch.
A plumbing robot doesn't need to be humanoid, an octopus shape may well be better for all the awkward corners. A robot police officer could be the municipality itself for sensory nodes (essentially the sales pitch of Flock etc.), plus some drones or robot dogs to perform arrests*.
The robot vacuum cleaners and lawnmowers we already have are nothing like a human. A robot taxi driver can be just the car. Robot dogs are already used for maintenance and security sweeps.
If you've got wheelchair access, you've got wheeled robot access. If you've got guide dog access, you've got access for Boston Dynamics' Spot.
* this may be a bad idea with current robotics, but I aver it's not improved by making those robotics humanoid.
In the last 100-200 years, that has been proven wrong at every single step.
No people. If you want something with fine motor control and dexterity, it's easier to make that the robot and then have another robot bring the workpiece to the arm than it is to build a single robot that can walk around and do it. There are compromises in human features because we're generalists.
There is no reason to believe that that you can not fully simulate intelligence in a C++ precompiler.
The precompiler can be simulated by human intelligence, and human intelligence can simulate a c++ precompiler.
Again, you are the one who arrogantly say they llms can not be intelligent without supplying any argument for such.
Not really. You've provided the arguments yourself, just now. But, you don't understand them. Which, brings me back to the initial remark, as to why this engagement is bound to be unproductive. I'm off to bed. Have a good one.
It depends on what exactly you mean by "commercial value commensurate with the costs involved" but I'd volunteer the 3G/4G/5G specifications and the other documentation required to implement the mobile network protocols. 5G is currently sitting at over 50,000 pages and it's one of the reasons Qualcomm/Broadcom/Apple are the only ones who can realistically make a mobile radio.
I don't think there is a single human to whom more than a few thousand pages would be comprehensible at a time except for the occasional genius.
If there exists a text which only one person can understand, that person can communicate their understanding to others, even if that doesn't help them with the original text. That dissemination of knowledge is what provides the value, not the mere existence of the text. If that person forgets or dies before they can share their knowledge then it will be lost.
We have many examples of this from history: ancient texts written in a lost language. These texts provide us with no value until the day they can be deciphered, unless you count linguistic puzzle-solving as a virtue.
How's your understanding of Schroedingers "An Undulatory Theory of the Mechanics of Atoms and Molecules"? You seem to be using the results of it as applied to semiconductor engineering just fine. And, I promise you, most semiconductor engineers haven't read it in full, they just accepted the results as passed on by several layers of teacher.
I have a paper on routing algorithms, which I have attempted to read to my cat. I don't think my cat retained much, but they seem to be enjoying the cat food that got delivered using the results.
I'd suggest that we're going to be a lot closer to the cat than the author of the paper when AI takes off.
All the stuff you've listed is understood by some person, and that understanding is the source of its value.
Now that we've cleared that up, can you furnish an example that satisfies the original claim of incomprehensibility and value?
"Not many people understand some things fully" is so massively different from "the human mind is incapable of understanding some things that AI will understand for us"
Paul Krugman (1998): predicted the internet’s economic impact would be no greater than the fax machine’s.
The 1876 Western Union memo dismissing the telephone as having too many shortcomings, and the banker telling Horace Rackham not to invest in Ford because the automobile was a novelty.
We are in good company!
When the flath-earth craze started I've been trying to at least get that bit actually personally verified. Haven't managed to do it to this day, though. So I'll just keep parroting various things without properly understanding them.
C'est la vie.
And for the inevitable critics of Sabine...maybe Leonard Susskind is good enough for you: https://youtu.be/2p_Hlm6aCok
You can take any of the theories that Hossenfelder would spend time on instead of the ones she does not like, and you would find (basically) a similar percentage of physicists saying it's a mistake to continue in this direction.
In other terms: for each physics theory, on 100 physicists, you have 5 physicists saying it is a mistake to continue working on it (number made up for illustration, and there is probably some variations, but you get the gist). You took one theory and found few physicists saying it is a mistake to continue working on it. You conclude, incorrectly, that it means this theory is fundamentally differently treated as any other theories.
(on top of that, it is unfortunate that Hossenfelder later screw up her image by doing way too much mistakes that someone reliable would not do)
The best I can do is things that are incomprehensible to nearly everyone, but still provide value. There's a small leap of imagination to consider an author that understands it and can show others how to leverage results without understanding be mechanical rather than biological.
That makes sense, although I'd argue that at least in the realm of HEP theoretical physics has extraordinarily expensive kit compared to what scientists make. See: The LHC.
If there exists a text which only an AI can understand, that AI can communicate their key conclusions to others, even if that doesn't help them with the original text.
The only difference here is the amount of meat involved. Perhaps tossing a few steaks on the server racks could help with that.
Because, again, I can point to hundreds of examples of texts where nobody but the author understands it, and they're only giving summarized "commandments" that you should follow if you want good results.
If I told you "don't use spin locks, call futex instead", do you think have gained an understanding of the Linux scheduler?
Anyways, people benefitted greatly from Newton's laws of gravity, even though we still don't have a quantum-compatible set of laws for it. The laws of gravity are still incomprehensible for people, but the approximation that we've observed is still immensely valuable.
You keep falling back on "incomprehensible for some people" but that wasn't the claim. It was about a text which is incomprehensible in principle; that is, utterly impossible for any human to ever understand.
Every writing must be comprehensible to at least the author, regardless of whether it has commercial value or not. If I hit the keyboard a few times, I've created writing, but it doesn't mean anything. It is just gibberish and without meaning, so there is nothing to try to comprehend. So if there is something to be comprehended, then at minimum the author should know it.
Therefore what you keep claiming is the only refute of your argument of "an [...] incomprehensible [...] writing" is actually a paradox, and cannot be disproved itself. However "humans comprehending things" is not a paradox, which means that your specific request to beat your paradox is not actually related at all.
His examples disproving the non-paradox version of your challenge (writing incomprehensible to folks other than the original authors) are sufficient to disprove your statement, as he gave examples of both people not comprehending human made and 'God' made writing (the universe/gravity)
If you want examples where nobody but the author understands it, examples are a dime a dozen.
This will shift your argument--that doesn't count! etc., to the point where it's by construction unsatisfiable and vacuous. And it doesn't matter: an LLM might e.g. break some cryptographic algorithm in a way utterly unintelligible to humans, but the fact that it works would be sufficient on its own to make all of us choose to abandon that algorithm and choose different ones.
an LLM might e.g. break some cryptographic algorithm in a way utterly unintelligible to humans, but the fact that it works would be sufficient on its own to make all of us choose to abandon that algorithm and choose different ones.
No, that is the entire point. If it is an algorithm which accomplishes something useful, then it is intelligible as such. That which is incomprehensible cannot be understood even in part, so it provides no value as a bit of knowledge (unless you're looking for a strong random number source, I suppose).
His original claim amounts to creating an AI that takes its place above humans as some kind of electronic God, delivering edicts to humanity that we cannot comprehend, but which somehow have value to us. It's unskeptical, pseudo-religious nonsense.
Understanding == Value
If a mathematician produces something incomprehensible then it has no value. It's meaningless. Indistinguishable from random noise.
An AI which produces incomprehensible text is producing no value. We didn't need to spend trillions of dollars on LLMs to figure that out. Markov chains can do that job perfectly well.
Again, do you believe that there are documents, of any value, that humans don't understand?
Maybe an LLM could help you notice what I was saying, since it's clearly beyond at least one human's comprehension!
There is no value in an undeciphered document until understanding is achieved, just as a lode of gold ore in some asteroid orbiting a distant star has no value until we can fly there and extract it.
If an LLM can help us understanding something then it was not incomprehensible, by definition.