I think people like author are positive about us, humanity, being able to build AI or something being very close to that.

I am not.

From the energy efficiency perspective human brain is very, very effective computational machine. Computers are not. Thinking about scale of infrastructure of network of computers being able to achieve similar capabilities and its energy consumption... it would be enormous. With big infrastructure comes high need of maintenance. This is costly and requires a lot of people just to prevent it from breaking down. With a lot of people being in one place, there socioeconomical cost, production, transportation needs to be build around such center. If you have centralized system, you are prone to attack from adversaries. In short I do not think we even close to what author is afraid of. We just closer to beginning to understand what is the need to actually start to think about building AI - if ever possible at all.

I love the language of this article :-)... it may be florid, but that's quintessentially human.

About the substance, I agree that there are fair grounds for concern, and it's not just about mathematics.

The best case scenario is rejection and prohibition of uses of AI that fundamentally threaten human autonomy. It is theoretically possible to do so, but since capital and power are pro-AI[^1], getting there requires a social revolution that upends the current world order. Even if one were to happen, the results wouldn't last for too long. Unless said revolution were so utterly radical that would set us in a return trajectory to the middle ages (I have something of the sort published somewhere, check my profile!).

I'm an optimist when it comes to the enabling power of AI for a select few. But I'm a pessimist otherwise: if the richest nation on Earth can't educate its citizens, what hope is there that humans will be able to supervise and control AI for long? Given our current trajectory, if nothing changes, we are set for civilization catastrophe.

[^1]: Replacing expensive human labor is the most powerful modern economic incentive I know of. Money wants, money gets.

The camera didn't kill painting. Neither the bicycle nor the motor-car killed running. There are already subfields of mathematics where it's believed that all the interesting discoveries have been found and no-one is looking except for the occasional amateur - and other subfields where to even have a hope of doing cutting edge research you would need to both do multiple years of postgraduate study and then get accepted onto one of a small number of close-knit teams that are pushing that cutting edge on an industrial scale.

So I don't see any reason to worry about the impact of AI. Unlike most fields with AI worries, mathematical research isn't even a significant employment area, and people with jobs doing it could almost certainly be doing something else for more money.

The terrifying thing about lean and machine learning is not the idea that we will train computers on human written proof, but that we won't have to. With the rules of chess easily computed, a computer can use the bellman equations and self play to learn a policy that is vastly superior to human play, at least on the critical path.

The state space of mathematics is pretty different from chess, but I think ultimately, mathematicians are just running something like A* on the space of propositions, with a custom heuristic that is learned by approximating the result of running A* with that heuristic. where your error is just the difference between the actual and predicted length of proof.

Considering that mathematics is, at its core, a language for defining relationships between quantities, and then relationships between those relationships, so on and so forth, I think it's fair to assume that the possible number of such relationships are infinite. Some of these relationships will obviously be useful in the real world, but they don't always have to be. I too, suspect that we can keep on building theorems on top of theorems with increasing complexity, until a point is reached that it becomes just too tedious (but not impossible) for a human being to work through the proof.

"perhaps advanced ML research models will be more analogous to improvements in climbing gear, aiding the development of mountaineering as a sport, than an intrusion of corporate control into our minds". I would argue that GPS and Satellite-connected phones are already poisoning the wilderness experience.

Test-driven development is not about tests. It's about writing code. The tests are there just in order to keep the bugs away.

Mathematics is just proof-driven development. For an spectator it might look like mathematics is about writing proofs, but that's not different than seeing a software developer write a lot of tests. The proofs are the best tools against insidious logic bugs that the society of mathematics has come up with in the last few hundred years. Mathematicians would welcome automating all the proofs, just like software engineers are happy for code assistants to take over the task of writing tests.

We'll know that the time draws nearer when an AI confirms or refutes Mochizuki's proof of the abc conjecture. As of right now, I don't think they're capable of doing that. And, as they can't even check a very (very!) complex proof, they won't be able to conjure any inhumanly complex proofs de novo.

Also:

> To expand: what if the practice of mathematics becomes completely determined by the diktats of a vast capitalist machinery of proprietary machine learning models churning out proof after proof, and theory after theory, conjured from the aether of all possible true statements?

I don't think that this is possible even in theory, as computational resources are limited and "the aether of all possible true statements" is incomprehensively vast. (There's a massive orders-of-magnitude difference in size between true-seeming-yet-false statements and the number of elementary particles in the visible universe. More statements than particles.) You can't brute force it.

I don’t share the author’s concern about a corporate takeover of mathematics. Most mathematics isn’t of commercial interest. Even when it is, it seems like there would often good reason to share it, like any other source code. Is Lean so different from other programming languages?

Such libraries would need documentation, or nobody would know when to use them, and then sharing is pointless.

If corporations build them, they would have to decide what to contribute to the commons and what to keep private. But that’s no different than any other language.

When does math become recreational for people?

This article is written in an unnecessarily extravagant style, IMO.

Also, I appreciate anonymity, but, to my point

> I live by myself in a remote mountain cave beyond the ken of civilised persons, and can only be contacted during a full moon, using certain arcane rites that are too horrible to speak of.

Okay.

A recurring problem I see is that people have absolutely drunk the koolaid of the utilitarian worldview. They've been marinating in it for so long, with no exposure to anything else beyond this parochial and base existence, that they have no idea they've been marinating in it. Everything is reduced to economic exchange. Everything is reduced to economic output. Learning is reduced to something that has value only if it is "marketable", only if it results in more units of toilet paper. The human being disappears from the picture as a mere instrument of the economy, and economy that exits for its own sake, that hovers over us like a god.

Given that kind of picture of reality, it is little wonder that AI seems like such a profound threat to so many people (putting aside for the moment the distinction between the aspirations of AI companies and the actual affordances it possesses). If being human is to be an economic instrument, then any AI that could eliminate the economic value of human beings is something akin to extinction. The god of economics has no further need of you. You may die now.

But this utilitarian view of the world reeks of nihilism. It is the world of total work, of work for work's sake. We never inquire about the ends that are the very reason for work in the first place. We never come to an understanding that economies exists for us, that we create them for mutual benefit. And we never seem to grasp that the economic part of human life is only part of human life, that it exists for the sake of those parts of life, the more important and most important parts of life, that are not a matter of economics. We have come to view life as meaningless, so we run into the embrace of the god of economics, losing ourselves in its endless churn, its immediate goals, truncating our minds so that we do not conceive of anything else, longing to escape the horror of the abyss that awaits us outside of its dreary confines...

The point of studying something in a theoretical capacity is to understand it, not to produce something of economic value. Each person must come into understanding from a state of not understanding. Homo economicus does not comprehend this. Homo economicus lives to eat and shit and cum and to accumulate things.

> The camera didn't kill painting

But it did. Painter used to be a trade where you could sell your painting skills as, well, a skill applicable for other than purely aesthetic reasons, simply because there were no other ways to document the world around you. It just isn't anymore because of cameras. Professional oil portrait painter isn't a career in 2025.

> nor the motor-car killed running

Is running an art/vocation comparable to photography and/or painting? We no longer have mailmen who run the length of the country afaik.

But running did heavily contribute to sedentary lifestyles in western countries, along with a bunch of other things.

> mathematical research isn't even a significant employment area

I agree, I think it will move from mathematicians "doing" math, to managing computerised system that do it instead. I'm sure we already have such systems.

I think far more important to humanity is improving mathemetical-literacy. From my perspective, math is made for mathematicians - it could be more accesible. As "pure" amth matures, there is still plenty opportunity in "applied" math (however you might define it).

> we can keep on building theorems on top of theorems with increasing complexity

This is a somewhat bleak picture of math. We also have the other phenomena of increasing simplicity. Both statements and proofs becoming more straightforward and simple after one has access to deeper mathematical constructions.

For example : Bezout's theorem would like to state that two curves of degree m, degree n would intersect in mn points. Except that you have two parallel lines intersecting at 0 instead of 1.1 =1 point, two disjoint circles intersect at 0 instead of 2.2=4 points, a line tangent to a circle intersecting at 1 point instead of 1.2=2 points. These exceptions merge into a simple picture once one goes to projective space, complex numbers and schemes. Complex numbers lead to lots of other instances of simplicity.

Similarly, proofs can become simple where before one had complicated ad-hoc reasoning.

Feynman once made the same point of laws of physics where in contrast to someone figuring out rules of chess by looking at games where they first figure out basic rules(how pieces move) and then moves to complex exceptions(en passant, pawn promotion), what often happens in physics is that different sets of rules for apparently distinct phenomena become aspects of a unity (ex: heat, light, sound were seen as distinct things but now are all seen as movements of particles; unification of electricity and magnetism).

Of course, this unification pursuit is never complete. Mathematics books/papers constantly seem to pull a rabbit out of a hat. This leads to 'motivation' questions for why such a construction/expression/definition was made. For a few of those questions, the answer only becomes clear after more research.

> the possible number of such relationships are infinite

I think you need to be careful taking about "infinite" in the context of math. If the number of quantities, relationships etc is finite, so are all their combinations. Even things like the infinit-ude of available numbers might have fixed patterns that render their relevant properties effecively finite, and lead to further distinctions e.g finite vs countable, etc.

Personally, I feel like math has a bit of a legacy problem. It holds on to the conventions of an art that is very old, with very different initial assumptions at its conception, and this is now holding it back somehow. I lack the background to effectivly demonstrate this other than "Things I know/understand seem less intutive in standard mathenatical terms" e.g. generating functions and/or integrals feel easier to understand (to me) when you understand the, to be software-like 'loops'.

In fact, the idea of "constructivist math" seems (again, to me) to beg for a more algorithmic/computational approach.

> a language for defining relationships between quantities

Could you expand on this? I don't see maths as a language for quantities specifically (i.e. what does symmetry have to do with quantities).

> just too tedious (but not impossible) for a human being to work through the proof.

Already happened with the four colour theorem arguably.

just the zfc axioms alone are already infinite. It's an axiom schema ranging over an infinite number of actual statements. That's just statements, without even considering symbols as you're saying.

> I think it's fair to assume that the possible number of such relationships are infinite

That’s easily proven to be true. “Two plus two equals four” is a theorem, so is “three plus three equals six”, etc.

> We'll know that the time draws nearer when an AI confirms or refutes Mochizuki's proof of the abc conjecture. As of right now, I don't think they're capable of doing that. And, as they can't even check a very (very!) complex proof, they won't be able to conjure any inhumanly complex proofs de novo.

I agree, but... Spend time formalising a large part of existing mathematics and proofs, train a bunch of sufficiently powerful and generative models with that, and with cooperative problem solving and proof strategies, and give them access to proof assistants and adequate compute resources, and something interesting could happen.

I suspect the barrier is finding a business model that would pay for this. Turning mathematics into an industrial, extruded-on-demand product might work, but I dont know who (except maybe the NSA) would stump-up the money.

I am not suggesting models will be capable of generating 'all' proofs - that is clearly impossible. Merely that they will get better at doing so, and there is no clear reason at the moment to believe they will never reach a human level of competence. If you have one model functioning at such a level, it is presumably trivial to have a million of them, none of which will need to be paid, housed, or sleep etc.

Why would an AI confirmation or rejection be more convincing than the proof itself?

also why would capitalists expand resources on churning out proof after proof when mathematical proofs are not patentable?

I wish I had a nickel for every time an AI hypester complained about goalposts. I thought it was clear that I was talking about AGI, or a general purpose intelligence on the level of humans. You know, what the hypesters have been saying is just around the corner if only they can have another trillion dollars. Sorry for the confusion

> That only demonstrates the possibilities yet to be explored. biology has millions-of-years head start; what's possible today could be balked out a few centuries ago by the same argument as yours.

Yes, we may only have a few centuries left to go before AGI. I was going with a few decades, but now that you mention it, a few centuries is more likely given we are running into Moore's Law limits with transistor technology.

> At a fundemental level, what holds back biology is all the other things it does (ala staying alive) and the limits imposed (e.g. heat etc) that a purpose-made device can optimise on.

You don't honestly believe that AGI will not have to deal with continuity, reliability, and heat dissipation issues that living things have to deal with, do you? All the more reason megawatts vs handful of watts is relevant. You just pointed out that it's not just an algorithmic optimization problem, but a much more complex problem of which we are barely scratching the surface.

> Seems we are arguing different things. I went back through the thread, and believe the proposition is: "us, humanity, being able to build AI or something being very close to that", which I translate as a comment on our literal species. I took your statement "From the energy efficiency perspective human brain is very, very effective computational machine" as being in that scope, and not just a reference to the current era (or Decade!).

I was replying to a literal statement about increased mechanical efficiency over biological efficiency. Which, in the case of AGI is completely inverted. Biological systems are so much more efficient that the comparison is embarrassing.

Also, I was saying our species is at least 3 decades from in-silico AGI. That doesn't mean we'll have some wild new tech that no one thought of next year. But the chances are so slim you might as well be saying we will genetically engineer flying pigs.