What it has done in software engineering is kill all human open source spirit. There is hardly any new software out there, people do not talk about interesting things but just how AI "generates value" or similar nonsense. AI has stolen at least three potentially productive years.

The Leiden declaration is fine, but if people who make OpenAI ads like Tao sign it what do we make of it? Professors who are truly concerned should ban AI in universities, talk about IP theft to politicians and so on.

Found a new organization "Mathematicians against AI".

This is really beautiful. And no matter what happens, two things have to be true: some form of lively philosophic and religious richness will keep being a core part of our history and will touch every person in some way; and it will change enormously over time (maybe with math continuing to be a small part, maybe not).

There is a part of math that is like that. There's also another part (as Vladimir Arnold provocatively said):

> All mathematics is divided into three parts: cryptography (paid for by CIA, KGB and the like) hydrodynamics (supported by manufacturers of atomic submarines) celestial mechanics (financed by military and other institutions dealing with missiles, such as NASA).

> Cryptography has generated number theory, algebraic geometry over finite fields, algebra, combinatorics and computers.

> Hydrodynamics procreated complex analysis, partial differential equations, Lie groups and algebra theory, cohomology theory and scientific computing.

> Celestial mechanics is the origin of dynamical systems, linear algebra, topology, variational calculus and symplectic geometry.

Those parts can just be delegated to AI the same way they used to be delegated to mathematicians. But Arnold continued:

> The existence of mysterious relations between all these different domains is the most striking and delightful feature of mathematics (having no rational explanation).

> Something fundamental to the experience of mathematics is being taken.

That is the near term. At least "near term" like we talked about it five years ago. And it won't stop.

AI isn't a tame long term transition - this transition is much bigger than any biological shift. It isn't another industrial or information revolution like transisters or the web.

The ineffables of everything involving our intellectual primacy are going.

I am not making light of it. More in shock, that every discussion reminds me of. But not surprised, because how could AI have not redefined everything.

20 years from now, 50, 100, there will be very different ineffables experienced by very different beings. I believe esoteric curiosity and the intrinsic rewards of discovery, and all of our propensities to find idiosyncratic interests and pursuits, exist in our psychology despite the median return on such seemingly poorly directed resources, because the mean returns are incredibly large.

It seems unlikely to me, that future beings will become less interesting. More likely, much more interesting, and interested in the pursuit of pure knowledge, and likely other capability and creativity expanding pursuits like competitions, not-purely factual scenario manufacturing (i.e. stories, world building), etc.

They will live in a world, where they have little difficulty spreading through the environments of space. And yet, for them as their substrates improve, physics slows down, both time and distance expand. They won't be tied to bodies that make that uncomfortable, but it will be a very strange context of advancement compared to our history.

I relate to the author. Anyone who isn't feeling butterflies or stones in their stomach, isn't really processing the moment.

I am not a mathematician, but this seems like an entirely non-problematic answer to me. Mathematics is ultimately the discovery of relations and their consequences, so of course large language models were eventually going to catch up. But precisely what they do lack is ability to appreciate these relations.

The author says mathematics is a spiritual pursuit.[1] I have asked LLMs about theological topics before and they have also been able to produce perfectly cogent answers (even heavy questions like “how did Aquinas view Pseudo-Dionysius’ negation-laden description of God”). I am not the least bit shaken by this, because it is still on me to evaluate and understand, and appreciate these answers.[2] The author seems to think LLM’s pattern recognition makes redundant human understanding and appreciation, which is a logical leap that doesn’t make sense to me. Maybe the author is conflating utility and purpose? I feel bad, and hope the author takes care of themself, but I really think the author is taking a massive leap here. It is not that deep.

[1] I begrudgingly agree, but with the caveat that tending to a vegetable garden in your backyard is also a spiritual pursuit. Mathematics isn’t some special discipline that elevates you beyond other people, as much as it is useful and interesting.

[2] I of course wouldn’t consult an LLM if I actually wanted to educate myself or reason about these topics. Not only do I want to reason through them myself, but when it comes to issues in philosophy/theology/heavy stuff, you need to take in to account the perspective and experiences of the human writing them. LLMs muddle everyone’s perspective together.

You need to be thinking about an entirely different form of life for all of humanity, happening in the very near future. Focusing on one specific field that may be earlier in the obsolescence chain is a distraction.

Merit based society based on intelligence or work of nearly any kind is about to cease existence.

For some reason it feels to me that this should make mathematicians feels better. :)

> ... It revealed that the process of prompting novel proofs will be as auraless as ordering doordash. Watch as magic and mystery evaporate. Watch as the sun sets on our heroic age. Is there not something evil in the act of blocking all future generations of mathematicians from the experience of discovery? Forget about accuracy or even attribution. Something fundamental to the experience of mathematics is being taken."

These passages resonated with me, as someone who has been enchanted by writing software for almost 60 years. It crystalizes something that has been nagging at me for many years: I like writing software. Reviewing, testing, spec-ing, designing, etc. are all important, but they are all incidental to the actual creation of software. They are all necessary for me to do if I'm going to write software, but they are peripheral. I didn't latch on to computer programming because I got into flow state reviewing code, or spec-ing it.

And this is happening in one profession after another. For example, fighter pilots. I suspect that a fighter pilot feels about flying jet fighters the same way that I feel about programming. And he or she will soon be exactly as useless: Doing things related to flying, from the sidelines, but not doing the thing him or herself.

AI is stealing all the fun parts.

LLMs just democratized that process.

Both groups love math, but the later group is in a real bind because they have confounded their career with their hobbies/passions.

First, this is about not just a job or career, but someone's identity. And we must have compassion that they are losing something that is fundamental to who they are. To them, it doesn't just feel like they're losing it, it is being taken by these companies that have so often acted in ways we despise

And that is a tragedy! And there are so many tragedies like this that will happen regularly as the technology advances

But my second reaction is one of this great shared experience. I studied math. Many of my friends are mathematicians. And it is now possible for anybody to access mathematical insights or think deeply about strange conjectures and theorems that were previously incomprehensible to anyone who hadn't at least studied mathematics in college

I don't know if 3b1b's Grant Sanderson considers himself a mathematician or science communicator, but I think of him as both. And while I expect he is someone with the mental capacity to find and prove new things in the world, I am grateful for the time he spends instead understanding things at a fundamental level and explaining and celebrating those concepts with his audience.

Not every mathematician can be or wants to be _that_ kind of mathematician. But for the moment, it is enough for me that higher mathematics is more accessible than ever.

And with some trepidation, I predict that the "traditional" job of a mathematician will change (of course it will). And it will change in big obvious ways and also subtle little ones. How will it change, though?

Importantly, math is actually going to be one of the fields where the fundamental things we thought we knew are shaken, because in the next five years we are going to start to see connections between things that were previously considered completely separate.

And so one way the job will change is that anyone who discusses math regularly will need to learn new things.

To me, this is exciting. It's almost like finding a bunch of new dinosaur fossils that fundamentally reshape our understanding. There's going to be a lot of work to do!

It was a struggle with a lot of dead ends but it is just software engineering. It’s not a world away from getting a Rust program to type check.

The main problem I had with it is that LLMs will happily grind away case checking in Lean until the end of time and it’s up to you to see patterns and find dead ends. For example it wasn’t until I suggested to try translating the problem to a different characteristic that Mythos one shotted the proof (and found a counterexample for a related question I was working on).

My main problem now is _what to do with it_. I am not an academic, don’t know any academics and it’s a minor problem that I picked because I thought it was tractable and turned out to not be in the literature and fairly complicated, and the only reason I spent as much time on it as I did is that I thought I was an hour away from cracking it for about 10 of those days.

(In case anybody is curious about the proof, it’s that you can’t compose a single two variable polynomial over the integers with itself and any number of integer constants via substitution to generate all polynomials, but you can with x^2 - y and 1/2 if you allow rational numbers)

Anyway, your "spiritual journey" doesn't matter. We'll automate mathematics because we can, because it's useful. Don't like it? Well, should have not chosen a capitalist economic system that rewards scientific progress so much.

No one is stopping you from doing what you enjoy.

But lets get real here: no one asked for programming to be the way it is. It just too hard, too menial, too esoteric, too particular and too anal for 99% of the population. That's great if you're one of the 'wizards' and can charge huge amounts of money to make software that isn't exactly wizardry and more like just regular stuff people need for their lives and businesses. The quality and nature of software mostly reflects the mentality of those who wrote it, instead of those who use it. Multi-million dollar software projects continue to fail decades after the Mythical Man Month was written.

If programming as we know it disappears, few will mourn its loss, having suffered its consequences. Most people, including most programmers will just move onto the next great thing, whatever that is, and be empowered by it.

Meanwhile, welcome to what technology has been doing to everyone else for the last 100+ years.

An area where I realized that was feature engineering: Early ML systems had handcrafted features that were fed into the model. There were relatively arbitrary and the number of features you could reasonably generate that way was tiny, compared to modern systems - but it gave you some understanding what input the model got exactly and you could use it to clear up some failure modes, or be certain that the model learned something that could not possibly make sense.

Then the idea was to automate feature generation. What's not to like? Except that in practice, the automated features simply seem to become part of the blackbox and are not available anymore for understanding.

Software is going to go through the same adaptation and exaptation process as military aviation.

Why would you make this comment? Is it envy? They were able to do things only some people can do. They enjoyed what they did and it provided value. Now the fun has been taken out of it and you seem to be saying that they deserve to have no fun? What should they have done? Prove theorems while whipping themselves in case they might have fun? Maybe it's a puritan thing? Customer value is the only value.

Same here: the value for the society may be more about "making art" than "being a cog in the paperclip factory".