Friendly warning to those who might not be aware: the vast majority of comments below posts like the above will be left by (otherwise intelligent) programmers who think mathematics is a closed system where one attempts to solve endless Olympiad-type problems. I wouldn’t take any of it seriously at all. Better to listen to what those who actually know what the subject is about have to say.

Unfortunately, mathematics (especially pure mathematics) is by its very nature very, very poorly understood by those who haven’t worked as a mathematician. Even worse, those who don’t understand are seemingly not at all aware of their misunderstanding and are entirely confident in their (very wrong) characterisation of the subject.

  "hey, did you know that x^i is the unit circle?" 

  "what's i?" 

  "i is defined as if you square it the result is -1" 

  "what does that have to do with circles?"

Everyone involved in AI should have read The Library of Babel [1].

It's about many things, but perhaps the most relevant idea here is that no information matters without understanding. We could generate all possible knowledge, but unless someone--a human--can verify and understand it, it doesn't count. The cure for mortality could be written on the moon, but if no one reads it, it hasn't really been discovered.

[1] https://maskofreason.wordpress.com/wp-content/uploads/2011/0...

> We (humans) should help humanity flourish.

I think this axiom is of course true. But the mistake the article makes, in my opinion, is to try to apply this axiom separately to each domain. If we have this as the over-arching axiom, it is not clear at all that humans should be steering the development of mathematics. Maybe it would be better for humanity if the department of world math is run by AI.

There's one flaw in the evidence for the logic chain. The hugging face attack is used to demonstrate three things: the need for oversight today (fundamental to the article) and to demonstrate some kind of drift or unexpected capability gain, and finally to hint and some fundamental morality of the AI or at least the risk of drift from our morality.

I'd argue that what the hugging face attack illustrates is that large AI companies are motivated to have bombastic claims supported by bombastic demos. The model was clearly trained or encouraged to work as it did, as evidenced by the fact it keeps using this particular escape hatch.

And the fact that it aligns with prior and current calls for what very likely might be a regulatory capture / oversight capture move right before IPO. It aligns so well with this "barely constrained superweapon" narrative it might as well be PR.

What's up with these insanely off base articles on HN this morning? Every single one of these is an unhinged pro-AI anti-human schlock piece.

> Similar logic applies to every industry and every job. And it comes to the conclusion that we won’t have enough people for all the jobs that need to be done.

In the (extremely) short run, yes. In the long run, those jobs will also be done by AI.

It's like chimpanzees seeing human society and saying "look how complex it is, imagine how many chimpanzees it needs to maintain it".

I’m just shocked by how “intelligent” people believe a token guessing system can be

People keep talking about this like there's a finite amount of math to be done, and then the party's over. But that's never how math has worked, is it? Every problem you solve, ten new ones open up. Like a fractal, the more you zoom in, the more detail emerges. No matter how much better AI is at solving problems, it's not going to generate the "final, complete compendium of mathematics" that that seems to hover over this post.

Math is meaningful because ... some people like to do it. The same as any other human pursuit. It doesn't need a reason beyond that. And AI won't change that. There will continue to be things to explore, things to find out, things that are maybe just at the edge of AI's reach and needs a human to decide whether it's worth continuing to explore or not. (Remember, AI isn't free).

So, IDK, I think for people who enjoy exploring math, there will always be interesting areas to explore. AI just gives us a better flashlight.

BTW I do agree that there's going to be an incident soon, whether intentional, accidental, or paperclip-factory, that leads governments around the world to shut all this down for some time, perhaps even shutting off access to GPUs entirely. It seems unavoidable. But that's just a temporary respite and skirts the core philosophical premise of the post.

Because the AI needs new stuff to train on?

Here is a link to the original article:

https://poshenloh.com/posts/20260919-math-ai

The original posted link from OP is from Terry Tao’s website where the article was posted as a guest blog post.

I sometimes think about this: to grasp the fundamentals of knowledge and complex phenomena, perhaps we need an external system rather than human knowledge systems. By that logic, maybe we need AI, which can handle far greater complexity.

Human capability, when you think about it, is complex. Why is Newton praised as being so damn great? He established the law of universal gravitation, F=ma. Why is that such a big deal?

He distilled countless phenomena in an open system into a single mathematical formula.

What makes it great is that he found common state variables and relationships across entirely different phenomena like falling objects, planetary motion, collisions, and artillery trajectories.

But does F=ma hold true for the entire macroscopic world? No. There are various conditions and specific situations in motion, but within most scenarios and a certain range of approximation, it outputs values that are useful to humans.

Why is the Schrödinger equation so great? Because it turned the time evolution of quantum states into a calculable mathematical law.

Human thought is essentially creating a closed system by deciding what to cut out and what to keep from the infinite degrees of freedom in reality. Academia is what reinforces that closed system.

A great theory is great not because it perfectly replicates reality, but because it compresses the immense complexity of reality into a small, closed formal system while still managing to explain a multitude of phenomena.

In that process, it feels like human thought and progress are shifting into a different framework. What LLMs do well is primarily exploring within the ontology and representation space that humans have already built.

I think there are two broad categories of discovery: One is forming a new closed system, and the other is connecting fragmented knowledge within that closed system. I feel that the vast majority of research focuses on the latter.

What LLMs excel at is finding unvisited points within a given representation space. This is typically the process through which master's and PhD students connect dots, build their skills, and form their own mental models. But the logic behind criticizing LLMs seems to be that they eliminate the very work these graduate students need to do in order to grow.

However, looking at it from another angle, perhaps our current knowledge systems and classifications have reached a limit, suggesting that we might actually need a completely new classification and knowledge system.

What is the core principle of an LLM? It's predicting the probability of the next sequence.

Let's say you type the word "cat". Cat - is cute (90%), want to eat it (6%), furry (4%). Because "is cute" has the highest probability, the next sequence proceeds in that direction.

Within this framework, human knowledge and logic largely operate the same way. Once an initial logical proposition is established, we follow it up with whatever makes logical sense next. From that perspective, I think LLMs will actually do this better.

But what is it that LLMs cannot do right now? They cannot create that initial logical proposition. I believe they lack the ability to carve out a closed system from an open system.

Stacking logic step-by-step within a closed system—LLMs do this exceptionally well. But whether that constitutes true "intelligence" is a different matter.

I feel that being logical does not necessarily equate to having intelligence.

Humans preserve and create different mental models and knowledge systems within an open system. Just as your thoughts differ from mine, LLMs lack the ability to form these distinct mental models.

If so, within these limits, what humans must ultimately do is construct the logical frameworks that LLMs can then fill in. Perhaps a new kind of logic dedicated to designing these frameworks will become the next major trend.

Viewed from this perspective, I have no idea if we are in a mere technological transition or something else entirely. Or whether it is even correct to say humans are strictly necessary to build that framework. Maybe my learning is just lacking.

waiting for the xkcd for that

I always think of this one but I bet there's better

* https://m.xkcd.com/435/

This hand-wringing exists because we have been trained to believe our purpose is to produce. Art, knowledge, widgets, etc.

Our biblically literate ancestors knew better. Our purpose is to love God and love people, which is why we still have a modicum of sense for the value of completely unproductive people.

As we’ve become spiritually hollowed, and more biblically illiterate, we’ve started to dehumanize unproductive people, a slippery slope if we ourselves become unproductive.

The axiom in this article will only be accepted by the intelligentsia if you can tell a story about why it is true: that humans are created in the image of God.

Until we re-find our ability to tell cosmic stories, this hand-wringing will continue amongst the atheistic elites.

> Why Do We Need Human Mathematicians Anymore?

TLDR: we don't. Terry Tao just announced he's becoming a UFC heavyweight fighter.

Why do we need any job? Check "Bullshit Jobs" by David Graeber. Not saying being a mathematician is a bullshit-kinda job, but there have been so many made up positions way before this AI-era. In that sense AI isn't changing much.

Pure math has been overvalued for a long time. Now that we don't need humans for it, it's really time for pure mathematicians to pack it in.

This sounds like a student in algebra class asking, "when will I ever use this in my life?"

Well it's actually nice, maybe more people will be able to do world class math with the help of these tools. There are few fields as elitist and hostile as pure mathematics, most mathematicians I know build their whole life around their profession and their self esteem is strongly coupled to the fact that they can do things that most other people can't. Naturally, many will be devastated when (if) you take that away from them. That said I think AI is still overhyped and human mathematicians can easily outthink it in most domains, look at how difficult it is for an AI to write even a single decent paper, a good PhD student can easily outclass it in that regard. All of these impressive results were generated by having world-class mathematicians steer the systems using highly tuned prompts, so I see it more like a super violin that produces beautiful music when played by master violinists rather than being a fully autonomous orchestra which many people are led to believe this already is.

And maybe let's not only hear the opinion of two or three Fields level mathematicians with blogs, 99 % of the worlds mathematicians in academia might profit from these tools as they might partially close the gap between them and the world elite, making creativity and tenaciousness more important than having the right neocortical structure allowing you to outperform 99.9 % of other humans at keeping context in your head and making predictions, AI can do that better now with the right prompts.

A few people with way to much power are building AI to achieve their utopian, sci-fi dream. A few other powerful people are using that first group to achieve their post-democratic goals. It’s the most obvious top down attempt at imposing their vision onto the rest of the world. They aren’t building tools to serve anyone else than themselves