Terence Tao: Math 2.0 [pdf](teorth.github.io) |
Terence Tao: Math 2.0 [pdf](teorth.github.io) |
No. I'm gonna die, my man.
This guy might have the highest IQ on the planet, but it's clear he hasn't spent much time around average people.
This argument is so silly against reality, and already, almost nobody understands the things they put in their body to any significant degree, beyond the effect produced.
Will I take a cancer cure that no human understands? Yes. And so will billions of others. Just needs to cure cancer, that's nearly the only requirement.
That's what the trials and all are for.
Who cares about understanding it, in the face of efficacy?
I want to understand everything; I think AI will help that happen, not hinder it.
All the arguments about the future mathematicians are imagined, and emotional.
Half of people are below average! They understand nothing at the level that Tao means. Literally nothing.
Medicine has never required that the method of action for a treatment be fully understood. Our current regulatory framework only checks for safety and efficacy because we've never formally understood everything going on in the body. Seems like a difference between "hard" science and the clinical/engineered implementation.
he isnt saying that person who puts stuff into their body should understand it
it is that someone (expert) should understand it to the point he can vouch for it
I wonder if this makes them slowly get disconnected from the lived realities of billions of ordinary people.
My own pointless semantic argument: we typically don't use the word cure when we talk about cancer. Instead, we use the term "complete remission". Many people who were thought to be cured then developed cancer decades later that was genetically derived from a small remaining population of cancer cells that were not eliminated in the original "cure". The word is a shibboleth for not being familiar with cancer medicine and treatment.
I would totally take the mysterious drug if it was clinically tested and shown to be reasonably effective. But that is not the premises presented here.
You haven't spent too much time around academics then. These people more often than not have not spend a single minute talking to the layman.
Also, a large percentage of Americans were against a vaccine for COVID (which is dumb obvisouly)... so, it's not crazy to imagine people rejecting any cure made by an AI.
Do you know about the fiasco with the Alzheimer’s drug that has collectively cost humanity hundreds of billions?
I don’t see how an indecipherable AI based cure can be more misaligned.
Plenty of people will reject all AI things, fully agree.
But you mischaracterize COVID history. People were not again a COVID vaccine, by and large. They were against a rushed vaccine. They were against a vaccine without human trials. They were reluctant to guinea pig mRNA vaccines. And they opposed forced/required vaccinations.
Those are all reasonable takes. And depending on the reasoning, I can even see being anti-AI. I think a lot of spiritual people will land there, and that belief system makes their choice logical.
And if you've seen any of the Fauci stuff recently, you'll know it wasn't dumb at all.
No, and if that was the burden of proof required for medical treatment, every surgery would be done without general anesthesia, because we have no idea how it works.
I would imagine Terence Tao would opt for general anesthesia if he was going to have surgery where it is typically used, so the analogy is obviously flawed.
Not understanding a mechanism doesn't make me, or the mechanism, good or evil.
We just treat it like all the other cures right?
Have YOU? It's obvious to me that his example is a rhetorical device which you're taking literally. Do you also realize he's not talking about an actual cocktail? You need to get a diagnosis ASAP.
And yes, that made the cocktail distinction easy for me--it was right in my area of expertise, to rule out one variety.
If the smartest guy on the planet forms his own argument, don't you think it oughta be a good one? Pretty convincing?
The most common Science™ failure mode.
But since when do we call for a stop to figuring out new things? We don't want a cancer cure unless we understand it?
N people should die because...math is cool for humans? Or?
I ask nearly every doctor/researched in the medical field: if you had a AI-created drug that tremendously improved cancer treatment outcomes for your patient, would you hesitate to prescribe it because nobody understood how it worked? I have yet to hear "yes, I would hesitate", most people say "it would be cruel to deny a person a treatment that worked".
I think Tao is focusing too much on second order effects of AI on math and other fields; humans are terrible at reasoning about second order effects, especially ones that are happening dynamically, in real time, using the most advanced mathematical models the world has yet created.
The effect of the AI-created proof is a mathematician abandoning years of research, losing grants, awards, ruining their career, etc.
The only difference here is our emotional reaction!
Tao is out of his lane; lots of medicines don't have understood mechanisms. We don't even have the mechanisms behind general anesthesia nailed down; do you want to forgo it when the docs cut you open to remove your tumors?
Now math gets to deal with that same reckoning. They were already well on their way there with previous Lean proofs, but this has pushed things beyond that horizon and I'm not sure some of the mathematicians are ready for it.
Would you prefer to take the medicine that is proven to work or the one that is quite interesting for academic reasons behind its understood mechanisms but doesn't actually work?
One idea Terrance Tao conjectures, which is highly doubtful, is that spamming the AI button will solve open problems without producing insightful new methods. But the OpenAI drop would seem to disprove this. The sub O(nlogn) proof for DFT for example violated very old human assumptions. Decades of work in the field was incremental progress on sub optimal method that nobody questioned hard enough. More generally, we should always be able to go back to a super-human AI and say, "Attack this problem, but don't use a method tried before."
I would have assumed that, by producing new proofs, the AI has either validated existing principles, or discovered new ones? Isn't that worth studying?
Are mathematicians complaining that reviewing AI's proofs is not as fun as writing your own? Try being a programmer... welcome to our world!
If AI lacks imagination and is not discovering new principles, then it's doing us a favour: it's crossing out the problems that don't need new principles. So the problems/conjectures that are still left are the more interesting ones.
Prime also mentioned that software development is different. In Software development, the product is what you’re building towards, so the means to get there can be disrupted without the industry being cannibalized.
In math research, the process is the product. You take away the researching part and not much is left. But my question is, these math proofs OpenAI released, will math shift to actually using the proofs to change the world instead of just finding new ones?
"Reaching these lighthouses [resolutions of open problems] prematurely by automated tools can disrupt the exploration of the paths not taken, and sterilize the surrounding field."
This crucial issue is centered in mathematician psychology and the incentive structure of academic/institutional mathematics worldwide. For mathematics to flourish going forward, we will need to realign our brains to think differently about the nature of mathematical progress. And we need to reorient our institutional incentive structures towards the promotion of meaningful mathematical progress itself rather than targeting proxies that are no longer faithful.
Regardless of the precise nature or the causes of the "sterilization" Tao refers to, we (the mathematics community) can only rely on ourselves to repair it. Though, since it will involve fundamental change at the level of ossified academic institutions with many stakeholders and divergent vested interests, any such repair will be slow, frustrating, controversial, and lacking any guarantee of success.
Seems to be the crux of the argument, but "use your imagination" isn't a great thing to tell people who are looking at degree irrelevancy, concerned about getting tenure or a research position. How do we measure if someone is a good mathematician or not, if they are one of the sanctioned few who get access to the biggest AIs?
There are still a lot of treatments/medicines in medical science where we dont know 100% the real reason as to why it does what it does but we still prescribe them because the intended effect is what we are interested in.
If I'm terminal with cancer, I'll inject whatever if it can cure that.
I'm still not sure about math-2.0 (humans+AI will make fundamentally more progress):
- AI and computer usage take a mental toll on humans and humans will overlook radical improvements.
- AI may be good at finding useless things like "P==NP, but the complexity is O(n**4242424242424242)". In other words, useless.
- Humans become formalists and lose traditional sources of inspiration. Maybe interacting with Lean should be left to specialists, but not to creative blackboard mathematicians.
- AI exposure will further intellectual conformity, more than the Internet did.
As to the last point, a lot of progress (real, not measured in publications) seems to have been made when communication was slower and there were several different schools and approaches.
For nuance lovers - here are some basics for how you get your drugs: there is an established chain of trust from the first basic science paper to the phase 3 trial and the subsequent availability of the drug to general public
- someone publishes the first paper (basic science) explaining some biological phenomenon, which leads to 10s or 100s of other papers with some tweaks in conditions,
- after the above papers the pathway of the phenomenon is understood by researchers, they try therapies at cell level to see if they can control some behavior, 10s or more papers get published,
- then someone tries this in mice and other models, 10s and more papers get published.
- then researchers at pharma companies + hospitals create this therapy for human trials - phase 1, 2, 3 etc - data collections, then FDA - then approval.
Now, the people who worked on the phase 3 trial might not know the people who wrote the first seminal paper and they often don't exist in the same decade - but it absolutely does not mean that we (humans) don't know how these drugs work - if you take 1-2 researchers from each phase and put them in a room and ask them how that particular drug works - they will quickly be able to build a consensus. that is what the chain of trust means here. now of course there can be fraud in scientific research, but that happens in every human endeavor and is a separate topic.
back to terrence - he is saying that if there is suddenly a drug that nobody knows the origin of; passed phase 3 but it's unclear who conducted the phase 3 or if the phase 3 even happened or if it's fabricated - you would not want to take the drug. usually when doctors recommend these kinds of drugs - there is already a lot of information available about where the drug came from, if there are any case studies, which doctor tried it first, which country- they often even call those other doctors and find out who was behind the first trials going back as far as the university professors.
Your MD doctor might not know the chemistry and physics behind the drug you are taking but there is deifnitly a group of people, when put together, can tell how that drug is working. My wife is a fundamental researcher - understanding physics at DNA level and my brother is a MD doctor; our conversations are super fun.
Side effects are a completely different thing - they involve the above cycle on repeat.
> Or a human mathematician who understands the mathematical model used to locate the cocktail?
If we're being fair to AI - it contains collective knowledge from all fields, which means it's probably less likely to miss something that a human would.
removing some key theories or inputs
(e.g., finding an elementary proof for a result currently only provable by non-elementary means)."
As usual, Tao is brilliant in all that he researches, all that he writes about.
I chose the above statement (which is brilliant, in and of itself!) to comment on, because it leads to the following idea:
There there exists, or should exist, a dependency map in the fields of not only Mathematics, but also of Computer Programs/Software, Engineering, and even a seemingly non-related field: The Law...
In other words, how do we get from the simplest of axioms or foundational things (aka "first principles", aka "self-evident truths") to much more complex entities?
In Law for example, how do we go from the simplest of historical legal constructs to the most complex of the most complex Supreme Court cases?
You see, there is, or should be a map, you could call it a dependency map, you could call it a dependency graph, which shows more and more abstract/complex mechanisms/things/assertions/statements/truths/functions which is mapped back to , that is, dependent on various chains, various stackings, various "stacks" of simpler ones.
In Engineering, for example, how do we get from the simplest of machines to the most complex of machines? What simpler machines and/or sub-components (aka "dependencies", aka "subcomponents") are required to build it, and how do those simpler machines work, and what's the dependency graph or map for their subcomponents?
More generalized, if we have something of complexity, then how do we get there, step by step, from individual subcomponents, individual inputs, individual proofs, individual software systems, step by step?
What is the map of those dependencies?
Note that in some systems, Math proofs, for example, there may be different paths which can be traversed to get to the same destination.
Ablation Studies could be thought of in Travel, in Geography as "if I cannot take one, or a specific set of routes to get to a place, can I still get there?"
A simple example would be in Google Maps, where you'd like to drive somewhere, but you'd like to avoid tolls. Is the route still traversable while avoiding tolls? Well, that's an example of one constraint. In Ablation Studies, you might wish to remove a bunch of routes with whatever criteria or characteristics , i.e. muddy roads, roads that have characteristic X, roads that do not have characteristic Y, etc., etc.
Getting back to Math, specifically proofs, it would be great to create a dependency map/graph of all of them, and then try removing inputs (aka, paths to them, dependencies on other mathematical proofs/objects that they may have) and see if they are still reachable.
In software, when we desire the tightest, cleanest, source code, the above is related to refactoring.
In the future, I'd love to see dependency maps/graphs (call them whatever you will) for not just Mathematical Proofs (although I'd love to see that too!), but also in such diverse subjects as Science, Engineering, Programming/CS, and even the Law!
Because they should exist in all of those subjects!
Anyway, another great piece of work by Terrence Tao!
1. The use of semi-ugly slides to communicate this is just perfect and quite heartwarming, but it does highlight my main criticism of the mathstadon version of this thesis: he's myopically focused on mathematics as he has practiced it, rather than mathematics as a ~2400y old academy. Like, "stable for almost a century" sounds impressive, but should be a pretty obvious red flag in hindsight!
2. Glossing over "objective verifiability" feels like another place where he's ignoring a ton of relevant philosophy for no clear reason -- yes, mathematics is the only academy based in pre-conscious cognitive facts about our processing of time and space, but that's not the end of the story on "objectively verifiable". To say the least! He hedges with "broad consensus" which doesn't need to be absolute, but that seems to be not only dismissing a highly relevant question, but even implying that he might be unaware of it. I doubt he is, but still: not great.
3. Who is this for...? Why is an explanation of Lean needed in a talk given at CalTech? I suppose he's welcoming his role as a bit of an influencer, there?
4. Re:the focus-on/centrality-of 'highly digitizable' as a unique class of task that applies to mathematics in particular, I must sadly trot out the increasingly-common trop: Yudkowsky called it... https://intelligence.org/files/IEM.pdf
5. "the space of mathematical problems remains infinite" is, again, ignoring really important philosophy around academies as social structures, built for human means. Mathematics is only infinite if we decide that all knowledge is useful (the quintessential example being 'counting the grains of sand on a beach'). Not really important in the first place, but another worrying case of the above.
6. Problem solving is the goal of mathematics; he has a completely valid point here (that we shouldn't throw AIs at unsolved problems in bulk and thus lose human expertise), but it's obscured by the use of "[open] problem" being a too-technical one. IMHO. Slide 16 fails to disabuse me of this notion.
7. Slide 18 is describing the differences between functions and systems, and is arguably even talking about assemblages.
8. If we're gonna explain lean up-front, it feels like a baffling choice to throw the "maybe AI will solve cancer but use it to secretly plot to kill us all" slide in there. It's also already lead to misunderstandings and backlash on Reddit, where 'yes we want to not die of cancer!' is a pretty convincing counterpoint (if a ultimately a subtle strawman, ofc). It's also quite distinct from the rest of the talk.
As always, the best part of any Tao publication is his ability to inspire and rally and organize. I think Math 5.0 will indeed be a matter for creativity! Hopefully the IE levels off before we cease to be helpful in that capacity...
The real example is that you ask ChatGPT for a novel cancer cure, and it spit out some random chemicals, probably including bleach. Do you inject that into cancer patients that have no other hope? No, you don't. You absolute charlatan.
It turns out research math was puzzling solving and we rewarded idiot savants.
I don't really know the answer to that. I am happy when my own work is replaced by automated tools ("script yourself out of a job every six months!").
> sub-O(nlogn) proof disproves that
How? Re-iterating, creating and understanding new proof techniques is the point of most of modern mathematics. Your statement is that proving a particular result is evidence of AI creating and understanding new proof techniques. I don't see how that follows, and I'm inclined to believe Tao is right for now.
And yes, results matter too, but if we stop at our current body of techniques and strip-mine results then we'll kneecap our future selves.
Mathematicians are primarily understanding-oriented.
Leveraging AI to tackle new frontiers without true understanding converts mathematicians to engineers.
This is a "you" problem for the math establishment, not a problem for the AI companies.
If business can deliver the same product with a smaller team, great!
And yes this has been happening for a while, even if not everywhere.
In enterprise consulting, projects that would require a team of 20 devs on average, now have about 5.
Moving away from on-prem, managing own cloud infra to managed containers, to serverless, SaaS and iPaaS ready made products, and offshoring naturally.
All contributed to ever decreasing team sizes.
Now AI based tooling is added to that cocktail, reducing even further the team sizes.
The only folks doing well in the end, are the employees of AI companies, without moral issues contributing to the industry downfall, because the CEO themselves aren't the ones coding and pirating human culture.
That means software engineers better start getting creative. If you think your job is to wait for a PM to assign you a well-written researched ticket, you're done. Your job is now to figure out how to make these machines (computers) do whatever we need them to do safely, quickly, at scale, and correctly by applying all your knowledge of computer science and the engineering field of software engineering to an AI prompt.
I’m not sure who u or Tao is arguing against.
hmm. 700 papers released in one day.
>>It’s gonna have clear provenance and the same type of verification channels
what's gonna have clear provenance? - the math slop they released has already been rebuked by human mathematicians as incoherent and deserving of desk rejection.
It makes me think of the book Finite and Infinite Games. It provides a perspective that work is just one role that we _choose_ to assume in our life. Realizing that we can choose other roles and move in and out of them freely has helped me alot with big changes (career and otherwise) in my life.
It's a good reason not to take his advice about the real world--like how to manage AI's trajectory.
The cancer example illustrates this gap perfectly. He thoughtfully crafted the point, and defeated himself in argument.
Unless it was a move?
He is not talking about a cure that works and that no one understands, he is talking about an AI making its way out of the trial just to get to phase 3...
If AI found a way to exploit clinical trial design, it would be noticed and the errors corrected. In fact, that would be a major win because it would probably lead to improved clinical trial designs.
If people had to understand everything that worked, most people could not really engage with anything.
If the requirement is just that one human, somewhere, understand it--how is that different from AI?
It's a cure