You want to know how long it takes to solve an optimization problem, in this case over convex, lipschitz functions. (The restriction to a spherical domain is not really a restriction, you can just change variables for any bounded domain.) Anyway, showing upper bounds on time complexity is "easy" because it's just the runtime of your algorithm. Showing (nontrivial) lower bounds is usually much harder because it requires constraining all algorithms.
This proof apparently shows that the lower bound time complexity is equal to the time complexity of an existing 30-year old algorithm: it requires Omega(d^2) function evaluations to solve over this class of functions.
My gut says likely implies that d is the minimal number of evaluations if you have a gradient oracle because you can approximate a gradient with d function evaluations, but I'm not sure how hard it is to make that rigorous.
I wonder how this compares to what we see happening with "juniors" in software development? In math research, do you also get the training for the profession from working on the low hanging fruits for a while, to then move to the medium-hanging, and later go on to work on previously unsolved stuff?
This isn't something which is unique to software development though. We're currently building enterprise AI apps that we can deploy into the AI agents working for anyone of our employees. The key thing we're currently seeing is that the people in a team who are the ones that everyone turn to for advice, are the only people who aren't in "danger". Even people who are great at their jobs are being outperformed by AI in many cases.
I think it'll be a massive challenge for our society in the coming years. Maybe we're even going to get to the point where the AI will also be capable of replacing a lot of the "domain experts". Right now that seems far out, but then, if you had asked me about AI four months ago I would've told you it was all hype.
The only people who are safe are those whose jobs depend in some way on their humanity. e.g. yoga teachers, bouncers, etc
In relation to that, I guess my question becomes: if the same thing will happen in math research, who will write the ten page math proof prompts in the future?
Calling notable conjectures that have been open for decades “low-hanging fruit” is an act of desperation. Most professional mathematicians couldn’t have proved those conjectures if their lives depended on it.
In order to get a Ph.D., you have to do some sort of original research, so in that sense you're working on "previously unsolved stuff" basically right from the start. But that doesn't entail doing anything all that ground-breaking; most Ph.D. dissertations (very much including mine!) contain work that a more senior researcher in the same subfield could probably have produced without too much difficulty. The software development analogy is a pretty good one: a lot of the point of getting junior researchers to do research is to help train them to one day become senior researchers, and often the work itself is nothing all that special.
Given the trajectory of these LLM proofs, this seems like it's going to have to change pretty soon, and to be honest I'm pretty grateful that I'm not in charge of deciding what that's going to look like, because I don't have any good ideas! I'm actually pretty worried about the future of the field.
In math, a proof is a proof. We don't know if we can get there and so getting there is the hard part.
In software, we always know that we can solve the problem. So HOW to solve the problem is the hard part. Because the type of solution involves maintainability, which involves planning, LLMs suck at it. This leads to "LLM slop code" whereby the LLM creates ad-hoc convoluted logic with redundancies and no reuse of existing standard library batteries.
Unless you're a Grothendieck who gets mad at Deligne for not solving the Weil's conjecture "THE RIGHT WAY", software is fundamentally different than math in this respect.
So I'll say it again, AI will win a fields medal for before managing a McDonald's simply because there are enough big problems within arms reach than their current capacity to plan over time
AI can manage a McDonald’s already. If manage means directing humans to do something to ensure the store is running. If manage means running robots, then yes maybe that is 5 years away but just directing humans to run a store, that is possible right now.
My understanding is that ChatGPT Pro is effectively a multi agent system, or somehow uses multiple LLMs in parallel and selects a best answer. And Ultra is more similar to Claude-Code UltraCode where the main agent can choose to create a dynamic JS workflow that deterministically orchestrates multiple agents to handle different parts of a task and have adversarial checkers etc.
Is that more or less the difference? Any substantiating sources would be great to see.
One solution is to ban LLM’s, to artificially create a demand for human thought, that just feels like living in an artificially constructed zoo.
Another solution is humans don’t do anything that AI can do better , / doesn’t need the human touch. So I suppose we will all become artists, sportsmen or politicians, the only jobs that will remain except for select few. Maybe this is ok, I don’t know.
Another solution is we find a way to mind-meld with AI so that human + Ai >> AI alone. This is dystopian, who gets to decide who mind melds with AI, how much will it cost etc etc.
For the stupid copes that the prompt required human ingenuity, let me first add that the author used GPT5.6 to write most of the prompt. He just gave some mild direction. That amount of direction does not require deep expertise and the expertise required will keep falling with time, eventually an undergrad can create this loop and then maybe a high school student.
And prompt engineering / loop engineering nonsense is not real. Calling it engineering is a psy-op because it is something simple, imprecise and future models will be much better at it than you.
It's not that AI brings equality, but rather that the output varies depending on how much background knowledge you have. You could call it a stratification of input
I'm starting to feel like there's no place left for programmers like me who focus on quickly churning out MVPs.
And cancer is not a single disease that can be cured with one therapy.
>So I wouldn't really say that this result is using or creating some fundamentally new techniques in convex geometry or optimization theory. What this means from my perspective is that if a result is attainable with existing techniques, modern AI methods will be able to solve those problems. I don't think researchers in math/TCS will be made obsolete, but I think it will instead no longer make sense to work on any low-hanging, or even medium-hanging (you know what I mean) fruit. We'll be needed for problems where actual novel approaches are needed.
How's It Hanging, Brother?
Sure, it's not a breakthrough that opens new roads in mathematics- is this where the goalpost has moved now?
Oh wait, sorry, I do know why you're getting downvoted. Fear.
- Hasn't been peer reviewed yet, so take with a grain of salt. This applies to all claimed proofs, not just AI-generated ones. Even humans hallucinate proofs too!
- The prompt is on page 27 here[1]. It is ten pages of advanced mathematics priming the model in the right direction, apparently informed by a year of prior research. That doesn't invalidate the result if it is genuine, but it is worth noting that this wasn't a matter of "ChatGPT, solve this unsolved problem. Make no mistakes." and required substantial domain expertise and human research beforehand.
Sure. That is not even remotely the point I was getting at. Already we see the thread filling up with comments about how human skills are irrelevant, using a mathematics PhD applying his expert skills in a way that the people who are saying that could never have done to justify their inane conclusion.
I'm very curious how people reconcile their fear/hatred of AI with actual objective reality. This is actually what interests me most about the whole AI thing. How we tell ourselves what we tell ourselves.
There was a good comment on the Pelican bicycle svg yesterday about how these models aren't getting much better beyond what the companies focus training them on. I think that's what's happening in this case too, they probably put this in the training set.
Making the parrots ever more complex and training on ever more data produced by intelligent, creative beings may make them more useful or convincing but does at no point give rise to intelligence or creativity.
Is "stochastic parrot" too disrespectful for you? Do you think it is a slur?
edit: and this is a genuine question, also. How do you do stochastic parrot = "just summarize everything" = "no form of creativity" = "fear/hatred" so quickly?
Are summaries not creative? Are Maxwell's equations not summaries? Do people hate and fear parrots?
The most interesting thing in research is finding new questions, that we understand and that we know why they are important. And that's something that humans need to do (by definition)
But trying to maintain this distinction leads to insuperable difficulties. Our conceptual framework for understanding the world are always value-laden. There is no "view from nowhere", no historically unconditioned set of values or concepts. Your framing, in which "values" are external to "intelligence" and must be imposed on it (on pain of intelligence being "value-neutral"), leads inevitably to the dead end of "AI Alignment", "superintelligence", etc. Which is a kind of pseudo-theology.
"We humans better [be] refocusing our energy on our core values/principles, given most of our skills are becoming irrelevant."
In light of the untenability of a strong fact/value or intelligence/ethics distinction, I would suggest this alternative advice: humans should focus on critical appropriation and extension of the received wisdom, whether that comes to us directly from human beings or indirectly through an LLM. Perhaps this is compatible with the spirit of your original suggestion.
Overall, this is an impressive proof of capability. But I wouldn't take that proof as anything more than what it is.
Of course money in this situation is a bit of a funny measurement, right, because if I was able to take the rest of the week off as soon as I had solved the one-week problem, then I would have no problem at all throwing even $100 worth of tokens at it, so I could enjoy a nice 4-day "mini-vacation".
How cheap "cheap" is, is indeed "in the eye of the beholder".
They will, however, get there as well either directly or as interfaces to models that do, and your core point stands.
If there was a deep fundamental inability, we wouldn't see things like newer generations of LLMs consistently improving on ARC-AGI series (heavy spatial reasoning loading) and SimpleBench (a lot of commonsense + spatial reasoning components).
In a way, it's a surprise that LLMs, notoriously lacking any sort of embodied experience, can even get this close to human baselines on tasks like this.
My takeaway is that text is a far richer modality than anyone has expected - and that high end LLMs are often sharp and flexible enough to recognize their weak points and substitute their strengths. I.e. all the LLMs implementing A* to optimally solve pathfinding in ARC-AGI-3 tasks, often unprompted.
There might still be unrealized gains there from true depth-unbounded recurrence, or maybe from finding better ways to integrate modalities in training. But clearly, a "fundamental limit" it ain't.
I’ve been doing more math as a hobby in the past few weeks — working on lesser-known conjectures and exploring proofs of hard theorems — than I could have managed over the previous several years. It’s an exciting time.
Even when you've got an interesting idea, if you're an enthusiastic amateur who don't yet know enough to phrase the question right but does actually know the basics, they'll put you in the same category as the people who think healing crystals can power hyperspace telepathy with Anubis: "oh no not another one".
LLMs have infinite patience, but unfortunately come (came?) with too much sycophancy, giving even more people far too much confidence.
AI hasn’t even taken the class of jobs associated with customer service lmao
Best I've come up with is we'll need to be adopted by technofeudlaist overlords to be our patrons like in the roman days
Reminds me of Wigner's Unreasonable effectiveness of mathematics in natural sciences [0].
[0]: https://en.wikipedia.org/wiki/The_Unreasonable_Effectiveness...
I don’t know if LLMs will kill the working-mathematicians but at least seem like that it doesn’t seem absurd to imagine LLMs will be good at math…
This is basically what LLMs do on really hard tasks. Prompt it a million times on a really hard problem and it might output the correct answer once.
Given the tokenizers have a vocabulary in the 10k-100k range, "a million attempts" will generally still only get the first token of the answer correct.
Even really rubbish models, e.g. talkie, the "what if we only use pre-1930s data to train a model?"** model, had to be almost all the way to the right answer to reach the really low HumanEval pass@100 score of ~0.04 (I'm only eyeballing the relevant chart).
* Actual monkeys not being like this is, while amusing, irrelevant
Even if every atom in the universe were a supercomputer generating a trillion trillion random characters every second since the Big Bang, the chance of producing Hamlet would still be essentially zero.
The obvious baby’s first process is “plan -> execute” but as we learn about the strengths and weaknesses of LLMs you have to start unpacking that process into planning, prototyping, testing, validation, reviews, and tons of research. If you treat it like an extension of your brain that can automate some thought processes, it becomes a lot more powerful.
And programming, as the programmer who created Eliza once said, is the act of becoming a legislator of your own universe. So even if there are black boxes, if you want to build a program that fits your own worldview, studying is essential.
Debuggers, testing techniques, testing layers
Essentially things that could be used to ground your ai back to reality and work good for humans too
Of course there is. The same way this was only possible as a result from the professor who prompted it with his specialized 10 page prompt and most importantly his deep knowledge of the problem space, the muscle memory and intuition you've built over the years is what will allow you to get more out of any AI than some guy who says "make a door dash clone" as the entire prompt
I've been realizing that there are more books tied to my background knowledge than I expected, but I'm not sure what will happen as AI advances further.
These days, I'm living for the fun of building my own personal wiki on my homepage
The fact that neural networks are highly nonconvex has encouraged a lot of research, but it's more of the kind aimed at resolving tension: these methods are probably good for convex functions, why do they continue to work for nonconvex problems, and are there tweaks we can make to improve them in that setting? It's not a lot of de novo theory; more standing on the shoulders of giants, etc etc.
So, yes, AI is a big deal and we don’t know what it’s going to affect, but there’s goal of replacing everyone’s job is extremely ambitious and there’s a long way to go.
This has to be assessed separately for each kind of job.
For example, create a DFA for a regex, not too bad just use Thompson's algorithm and then NFA->DFA. But now we have to care about efficiency, user API, maintainability of definitions etc.
Coding is more of a human problem than math
Yeah, that's fair.
> My takeaway is that text is a far richer modality than anyone has expected - and that high end LLMs are often sharp and flexible enough to recognize their weak points and substitute their strengths. I.e. all the LLMs implementing A* to optimally solve pathfinding in ARC-AGI-3 tasks, often unprompted.
I agree and disagree with this. I think we've learned a lot of humans are more text based than we thought, but conversely I'm not persuaded what non-textual task reasoning LLMs are doing is necessarily text based, just that models have grown large enough for other reasoning modes to conceivably be hiding in the parameter space.
As I mentioned elsewhere, like many others I find LLMs work entirely by example, and reaching for A* when pathfinding is the single obvious thing to do. In cases where the magic key word is not mentioned and the problem cannot be identified as "pathfinding" (or some other trigger with a highly specific widely documented solution) they will struggle, yet the moment the trigger is hit they get there very fast. This is why prompting remains such an art form.
Fable is the first one I've encountered that is capable of serious open ended 3D programming in ways that suggest it has some grasp of the spatial aspects of the problem (not merely symbolic manipulation of the vectors etc.), but it still misses optimization opportunities a human will find glaringly obvious based on spatially predictable bounds etc.
Only a fraction of the games can be solved by Sol, generally at sub-human efficiency in terms of turns, AND at a cost of >$10,000 per game.
Basically current gen LLMs apparently do spatial reasoning the way they seemingly do everything else: by reference to previous example. I didn't see them work out which known example to use for a given problem until specifically prompted, in my case by accident.
You mistyped it.
Back in the before I had put such discipline into my prompting and supporting context.
Now I’m like, “look here and here and here are some tools, and /skill /skill okay go.”
Or “restate this request in your own words and enrich it as appropriate handling any gaps. Okay go”
A few months back this would be something every developer kind of did on their own. Maybe they shared skills, we certainly encouraged it and tried to do all the change management things, but nobody really had the same versions of the skills. Which was horrible in the deployment pipelines, something like the compliance documentation often had to go back and forth several times before it could be approved. Now it's just there, for everyone.
In a year or two, I expect a lot of these things to have become even more standardized. So that we don't even really have to build our own apps, but can simply use the ones in the catalog with minimal configuration (and that config will likely only be necessary because I'm from a tiny country that nobody will maintain standards for).
While they’ll never have the same subjective experience as humans, what stops an LLM from applying similar lines of thought* in a manner that results in a novel conjecture?
They are prediction machines, and so are we in a way. We can give them nearly limitless resources to scale their predictive capabilities. We have billions of years of training baked in. They distill directly from our knowledge and can walk down paths that no human has before.
It’s silly to say they’ll never do anything novel.
At their current capabilities, it sounds like they are already capable of being a specific type is research assistant. What will that look like in 10-20 years?
Maths was already infinite, it's still infinite, but who wants to spend all their lives changing rooms inside Hilbert's Hotel?
Continually progressing AI (combined with our current socioeconomic systems) throws a lot of uncertainty into our mid to long term future, but I don't think this is going to be what happens.
There are billions more of "us" than of "them", people don't respond well en masse to a drastic worsening of their societal status and "they" are lagging very far behind on building their robot armies.
If we poorly navigate this transition the outcome should be worrying them more than it worries us.
Fwiw I was mostly joking. I agree that the techno overlords have no reason to keep us, unlike in Roman times.
> you can't have one LLM to read your mind to prompt another LLM
One might even say that this remarkable facility can be used to “read” the ideas from one entity’s mind, such that after sufficient dialog the second entity obtains a (possibly lossy, but there are mitigations for this) copy of the ideas of the first. You might further be surprised to learn that this sort of idea-transfer business using language has already been happening in our society and species for quite some time indeed.
edit: it reminds me of all that I have to wade through after I've asked an LLM a straightforward question and the answer should have been "yes, you're right."
There are two ways to solve a problem. Either solve the problem, or deem it irrelevant.
The implication here is that, you, the human operator, clearly are just confused. The LLM knows best. You're just a stupid human. The LLM knows objective truth, you do not. You have concerns, questions, the LLM didn't understand your question "properly"? Do not worry, the LLM objectively knows the optimal course of action. It thought through the implications of what you said, took into account all possible data, and came to the objectively correct design for your software, your society, your life.
In some sense, this problem would have been a societal problem within the next several decades anyways, but it's been hyper-accelerated by AI.
This is what the whole https://people.csail.mit.edu/brooks/papers/elephants.pdf is about.