A misalignment of AI in mathematics

I am thinking of Mochizuki's abc conjecture: He worked in relative isolation, and dumped a huge incomprehensible proof on the community (to oversimplify a bit). That's not totally unlike what might happen if AI generates a huge, incomprehensible proof of let's say RH.

Well, what is the result? In the Mochizuki case, it was a lot of skepticism, but it also generated conferences, papers, talks in the hallway, discussions with students, and so on--a flurry of exactly that kind of community process that the declaration says is the main driver of mathematics.

Ultimately we think a fatal flaw was found in Mochizuki's proof, so it didn't lead anywhere in particular. But in our hypothetical "AI lean-verified proof of RH" situation, it would presumably generate substantially more of that community activity we saw in the Mochizuki situation. And if it's correct, that community activity would be productive (expository talks, students given problems to flesh out or generalize, etc).

Maybe mathematics just becomes a little more like other fields--relying on labs with lots of money for compute, digging through a corpus of AI-generated proofs, etc.

I'm afraid of the ripple effect of the agenda pushed by AI companies will have. In future and even now, they say AI has significantly progressed math and scientific research in general. There is truth to this, but the narrative has done more damage (so far) to the students, researchers, and the culture of knowledge transfer in academia. Many graduate students (I know) are having a crisis if any of their research worth it? If AI can (or will) do everything, what's the point of doing experiments and all? This will eventually deter a whole generation of curious minded students from research.

I guess, only time will whether this is for the good or bad. And how good AI models get without new data from research and experiments.

I do see how this is a problem in terms of assigning credit, but I think the cat is already out of the bag in terms of these models being capable. Even without AI labs spending millions of dollars to solve millennium prize problems, there are plenty of other people who will use them to pick low hanging fruit. I don't think any social solution is going to make things go back to the way they were, where you could share your progress towards a famous open problem without risking someone "scooping" you within a couple of days.

I think that the most likely outcomes are either mathematics becomes more secretive, or there is a more deliberative approach to assigning credit than who was "first" to solve some problem. In the former case, this may slow down progress, and in the latter case, this could mean that credit would become more subjective, and be a continual source of controversy.

Baudelaire argued that photography became a haven for failed painters, the sorts of hacks that could not finish proper training. Photography, as a mechanical rendering of the world, could only record what already existed; it couldn't transform reality the way a painting could.

He also criticized the public's craze for "rushing" into it, and complained that this technical "progress" was weakening the arts.

Do you see some parallels as well?

[0] https://fr.wikisource.org/wiki/Curiosit%C3%A9s_esth%C3%A9tiq...

I do a lot of programming that uses mathematics. I designed the error recovery file format “par2”. I worked as a quantitative trader.

What do I want from math? To be able to search math results, understand them, and apply them. But academic mathematicians are so inwardly focused - as these authors so obviously are - that it does all of these badly! Search is still with words, not symbols and formulae. As for comprehension, many math papers are unreadable to those not in the cliche of mathematicians in that specialty. And even applied math papers don’t ship code or specify the operations explicitly.

I wanted to fix this and in 2013 I worked on formal proof. It makes symbol search and application easy. I found out that the tools had been usable for decades at that point. But mathematicians didn’t use them because they had that inward focus and didn’t give a damn about math consumers.

AI is enabling formal proof. It will carry mathematics into the computer age, which the field should have joined with every other industry in the 1970’s! I welcome it. Math will get used more and that is what the work was meant to achieve in the first place.

The argument here sounds similar. The fear, as I understand this statement to be saying, is that by being given the correct answer, in the form of a 100-page Lean proof, humans will be robbed of the chance to from insights about the structure of mathematics itself. I don't see any reason that humans can't continue to develop insights as they try to digest the 100-page Lean proof into something more manageable; but with more certainty and fewer false starts.

This is true of almost any field - if we have black-box technology that only the AI can understand / fix / augment / extend, then we will lose the ability for humans to learn and understand that tech.

Its similar to outsourcing manufacturing, you lose the ability to manufacture. If you don't exercise a muscle, it atrophies. If some people dont learn to write code [ or do math or write music or books ] the hard slow way, we will lose that deep skill.

If young people don't grapple with hard problems in science, math, economics, creative writing, drawing, music making .. they will not be trained, their cognitive skills will not develop with use. We will have no doctors or engineers or musicians or lawyers in a decade or two.

We don't want to become a race of fatties who never exercise, or dummies who just scroll social media feeds without exercising their minds.

This is real.

Now AI has surpassed me in fine-grained problem-solving ability. It can write more reliable code than I can, faster than I can, provided it is given the right guidance.

But one thing that I'm still much better at is identifying technical opportunities and choosing the right tradeoffs.

My feeling is that frontier models are incredibly smart in some ways, but incredibly dumb in other ways

When I chat with Claude using deep technical language about distributed systems issues, the arguments it presents are mind-blowingly good. I worked with many skilled engineers on complex projects but the kinds of arguments Claude makes are on another level. It feels like it can read my mind because it already identified all of the relevant aspects to the current topic and it's incredibly persuasive. I would say it's even better at deep, nuanced technical discussions than I am.

When I ask it to implement a feature using technical language, it does a really good job and the solution usually works out of the box. No bugs at all 95% of the time even after adding 1000 lines!

But the problem it still has is that when it implements solutions, it misses so many low-hanging fruits/opportunities. Especially in terms of performance, maintainability, scalability and UX. It's like it sees and recalls all the relevant parts perfectly but yet somehow misses opportunities which seem extremely obvious to me.

It feels like AI has 0 creativity. It only sees the opportunity once I mention it... And once it sees the opportunity, it demonstrates deep understanding of the technical implications. I think what's surprising is that it understands the suggested solution so well, with such nuance, that I can't understand how it didn't see the opportunity and why it never seems to see it until I mention it.

This is very unhuman-like. There is no way that a human being with that degree of understanding of a topic would be presented with such highly relevant context and not make the connection.

So what went wrong? Mathematics education. Math below grad school is all about solving stated problems. Credit is given for solving puzzles successfully. Homework is problem sets. Everybody below a very advanced level is taught math that way. Even at the higher levels, puzzles remain important. Awards in mathematics are often tied to solving puzzle-like problems. That's still the criterion for becoming Senior Wrangler at Cambridge, "the greatest intellectual achievement attainable in Britain". This despite Polya's attempt at reform a century ago. Puzzle solving gets good grades and class rank. So it's the status indicator mathematics presents to the outside world.

Then reasonably good AI comes along. AI has become rather good at solving puzzles. So people aim powerful AIs at known hard puzzles, with some success. That blows up the status indicator system. Mathematics itself is fine. It's the status symbols that have a problem.

Maybe the Fields Medalists need to hire a crisis management team to reframe what success means in mathematics. That's what they're trying to do with that letter, but they're mathematicians, not PR people, and they don't know how.

This is the crux of it and goes far beyond Mathematics or Computer Science. To get a bunch of humans to do anything, you have to motivate them. Kleos and Timē; renown and stuff. These AI companies threaten to rip this away from everyone but themselves, and this recent millennium prize is the perfect example.

Solving this problem as a human would have led to tremendous Kleos; my name would be written in the annals of mathematics, lecture tours of praise were mine to be had for the rest of my days. This one victory would have earned my recognition throughout history. The greatest a mortal may hope for. Ripped away.

It would also have given me great Timē. The prize money, the professorships, the book deals. Gone.

If all hope of “renown and stuff” in the intellectual realm is now taken by the AI companies, they will remove all human motivation to pursue these endeavours.

Perhaps the glory will come from slaying these fell beasts.

This is the effect of AI on most intellectual disciplines, and it’s a real worry.

>We candlemakers are suffering from the unfair competition of a foreign rival. This for­eign manufacturer of light has such an advantage over us that he floods our domestic markets with his product. And he offers it at a fantastically low price. The moment this foreigner appears in our country, all our customers de­sert us and turn to him. As a re­sult, an entire domestic industry is rendered completely stagnant. And even more, since the lighting industry has countless ramifica­tions with other native industries, they, too, are injured. This foreign manufacturer who competes against us without mercy is none other than the sun itself!

>Here is our petition: Please pass a law ordering the closing of all windows, skylights, shutters, cur­tains, and blinds... https://fee.org/articles/the-candlemakers-petition/

It seems like a lot of the issue here is that these problems aren’t interesting in and of themselves, but they lead down interesting roads. It defeats the purpose if you solve them without getting any real understanding.

It’s akin to saying you’ve solved “pancake flipping” problems with a waffle maker, or “travelling salesman” problems with a zoom meeting.

Keep in mind employees at AI companies are publicly stating that they believe they're risking a >10% chance of human extinction. They're knowingly risking the lives of every man, woman, and child to continue the work. The lives of their own sons and daughters. A person already rationalizing that isn't going to shed a tear for the careers of mathematicians. Just a bug on the windshield.

Of course, nobody's really comfortable with it, so this is also a forcing function for the industry to adapt and figure out new techniques to manage complexity and trust. I think the same will happen with Mathematics.

But it is also possible we will end up with three forms of Mathematics: the one we understand, the one we don't, and the one we don't understand but can prove to work. Kind of like magic -- with all the positive and negative connotations of the word.

It is pretty evident that these models will soon exceed our cognitive capabilities. Is it right to hold them back just because we can't keep up? Many of those discoveries will be so beyond us that we can't do anything with them, but that also means they can't hurt us. On the other hand, there could be many discoveries that we can parlay into practically useful applications, even if we don't understand them.

Just like LLMs.

- Except the mathematicians who we'll scoop and cause existential dread among their entire field.

- Except the software developers. They'll need to become plumbers or live on UBI.

- Except the people in countries that can't afford the cost of AI tokens to keep up with the rest of the world.

Just keep picking off groups of humans for the "benefits of all humanity"... while building larger and larger disparities been the have a lots and the just have enoughs.

We're going to build humans a utopia but along the way we'll leave a trail of destruction because that's not our problem.

    1 + 1/2 + 1/4 + 1/8 ... = 2

The big objection to finiteism is that it's a lot more work. Infinity swallows many special cases. Proofs get longer without infinity, and most of the special cases are uninteresting. That's not a problem for AIs.

Someone may start up an AI and make it grind through Hilbert's program for putting mathematics on a fully consistent foundation, starting from a finiteism base. This is a huge, unrewarding job. Great for machine work.

[1] https://encyclopediaofmath.org/wiki/Finitism

The process is as much a human experience as is the result. If we have dime-a-dozen results like this minus the process, where does our intellectual structure stand? We will just be a hierarchy of priests at the altar of the machine?

PS: Before someone comes up with an counter example of maths before and after calculators, that was a tool that accelerated the steps. This is replacing the brain.

I totally see the problem Terence is describing. We are loosing a lot in understanding and focus if it continues like that. The solution found for Navier Stokes doesn’t have much „real value“ - but what almost always happened in the past when people worked on the difficult problems, these sparked new ideas / new theorems that broadened our knowledge. Think back at your grad studies, figuring out a proof as homework was hard, sometimes incredibly hard, but while doing it we gained a lot of understanding how things work. Now asking AI for the solution and „just“ getting it, risks our understanding, our creativity and our ability to connect the dots with other territories. I see it in students nowadays, there is much less understanding, much less creativity in finding solutions. I truly think this „short-path“ solution with the „death of struggle is one of the biggest risks with AI already for human development

This is about how good taste in both research direction and in design are essential to steering AI, but we have no plan at all for instilling that taste in students or practitioners in a post-AI world.

> The issues the mathematical community faces now are similar to issues that other scientific and creative professions are facing, and indicate issues that all of humanity might face: how to make sure that, as AI changes the way work is done, we do not lose sight of what that work was meant to achieve in the first place.

Besides eroding taste and taste-building, this is about just how useful friction is as signal.

Everyone coding with AI knows it routes around difficulties like a river around a stone, which is not necessarily a good thing. It will do it tirelessly 1000 times instead of learning anything from it. AND if the AI does not fail in this, the human driver will get no signal, and never know it happened. This seems to be getting worse, not better.. my theory is that more models are cross-trained on cybersecurity stuff where the goal is success and the method doesn't matter. Fine for pen-testing, ultimately pretty bad for coherent code or math or physics.

Discrete tasks where we don't want to be bothered is a real use-case, but optimizing for it everywhere is terrible for the future of durable abstractions that we can build on and ratchet up our understanding with. Bad for the models too eventually! They can maintain a codebase with millions of special cases or juggle tons of free variables in equations, but that just encourages bad abstractions.. they have a ceiling for this too, even if it's higher than humans.

In almost any scenario even tangentially involving mathematics, twenty-five Fields medallists uniting to denounce something would be a veritable Tsar Bomba.

It should give you pause that here they feel like the ailing infant.

I wonder if there’s a Fields Medalist group chat.

I do see a patient teacher on generative tools. What has keep me trying to learn math has not been mathematicians, but the support of people outside of the field.

OpenAI/Anthropic are shaking the box.

To Tao’s credit he obviously identified the problem very clearly and admits understandably "we did not have the time to have a more consultative process, as with Leiden; but we decided that the urgency of the situation was such that we needed to release a statement sooner rather than later".

Perhaps you are misaligned.

Who decided the goal of math must be human insight?

First off, some mathematical truths might simply be far beyond our biological comprehension.

Second, for us non-mathematicians, the value of math isn't in understanding exactly why a result is true, it’s in how those results can be applied to actually improve our lives.

Isn't this why we have math in the first place? To solve our real problems? Over time it morphed into this pursuit of pure theoretical insight, probably out of necessity at the time, but is it still necessary?

Alternatively, some claim that mathematics is about understanding these implications.

Under the first definition, AI is already, and forevermore will be faster and better at proving theorems. Just like it is better at checkers, chess, and now go.

The author asserts that AI proofs are incomprehensible to humans, and so under the second definition AI is merely a tool to overcome one hurdle on the way to understanding.

So which is it? The author seems to claim the second definition, but bemoan the end of mathematics under the first.

If you spend $20M working out a Millennium Prize problem, in what universe would you offer that an unrelated effort should take credit? This is a branding game rather over whether software engineers are going to use codex or claude. In that light $20M (or whatever it was) might be worth it to squash even the rumor that claude code is more capable. Engineers look up to mathematics, while at the same time business and probably most engineers think the problem was to solve the problem. GPTs solved one the hardest known problems so they can solve my company's problem.

Some go further looking at these people, very on-the-nosely likened by one commenter here to ants, talking about education and responsibility and "core values" etc and just don't care. There was a major problem at the beginning of the week that is not a problem now and that is uncomplicated progress.

It's not wrong for OpenAI/Anthropic to do math for product development or even just branding but seemingly at no cost now they could work in an arena real mathematicians aren't interested in, versus just mowing the field. Everyone involved on their side should admit the purpose of these demonstrations is not to engage mathematical ideas it's about Claude/Codex. there's no shame in that. Which is better at solving random hard Diophantine systems? That would seem to tell me as much as I need to know insofar as a model's value is represented by raw mathematical power - then, take my money just as well!

Assuming the worst accusations are not true I think there are ways forward going to be acceptable for all. The labs themselves do not represent Terrance Tao as some kind of gate-keeping dinosaur in this. They're not interested, not the kind of entity that can care about theoretical mathematics. These dudes are paid 7-8 figure salaries ultimately for the product, they solve a Millennium Prize problem then pretty quickly seem to move past it.

That sounds like an "us problem", not an AI or OpenAI/Anthropic problem.

Alignment alone is not enough. Aligned with _what_?

> We are not trying to meet some abstract production quota of definitions, theorems and proofs. The measure of our success is whether what we do enables people to understand and think more clearly and effectively about mathematics.

It is true that the manufacturing of "true/false" statements is not the same as gaining understanding of a problem. However, for many mathematicians, true/false statements are already manufactured by others. Think of a student who is given a conjecture to investigate, with their advisor describing it as "it must be true". Most exercises in a textbook are stated such that the outcome is known before you begin. That's not really a problem -- investigating the conjecture/exercise yields its own dividends, whether or not the outcome is known. It is also the case that defining new directions involve understanding and synthesizing related problems, asking the right questions, and deciding upon the right directions, and it's not clear that AI can do that at all.

The real risk, I think, its the public's (and funding agencies') perception of the importance of "human" mathematics, but that's already a struggle. For example, it's tough to explain to the lay person why it's still important to research group theory -- the main example people cite is RSA encryption, which was invented almost 50 years ago.

Imagine time traveling back in time and offering Leibniz a packet of proofs from the intervening years, but with the caveat that there would be no explanations. Would he say no?

I don't care how good Astra or any subsequent models they may release might be... I am never going back to those token reset shenanigans.

We are moving forward and if that means no human wins a fields medal because they didnt spend three decades working on a problem that could be solved in three days, the world will be better for it.

I'm unclear what the ask is, though. What, even in theory, is a practical and realistic fix?

But I agree with the sentiment that the marketing behind these "discoveries" is disingenious. They pretend they solved the problem, but it still takes a bunch of humans to reduce the solution to a simplified and sensible explanation.

The fundamental issue with AI solving any perceived difficult problem is that we have lost the journey. The sight atop Mount Everest looks much different when you have climbed compared to being dropped from above.

Once AI mines the easy unsolved problems, it will take large token budgets to make meaningful advances. That will need to be funded and planned at a societal level, like the great facilities of the other big sciences.

Mathematics will become more like an observational science, with some mathematicians specialized in interpreting the results that come out of the big AIs.

This is Terry Tao talking about AI's impact on Math, but this could just as well be a software engineer talking about AI's impact on software development.

Do mathematicians deserve more job security than software engineers?

As a non-native English speaker, I initially understood this to mean that all living Fields Medallists had signed. I later realized that it meant only that all the signatories were Fields Medallists.

(Apparently, there are 47 living Fields Medallists today.)

Now, I can ask ChatGPT about this and get back a proof that shows "a passive airframe cannot sustain zero-drag motion through still, viscous air"

So, I think if anything now, Maths has changed for the better. More ideas can be proven false or true from a get go instead of wasting so much to see if its even feasible to find out it isn't.

Progress if anything is about to leap frog anything we have ever known.

Physics and Biology will see major breakthroughs that WILL fundamentally alter our world. That is one key thing missing from alot of discussion here is the narrow focus on math (or parallels with software engineering). Doing well in math is key to doing well in physics and other sciences.

The second is the utility. We get to do navigation because the maths about angles and spheres checks out. The social utility downstream of AI proofs may be huge.

I don't like this disjunction.

Let's assume in the next decade GPT-7 PRO Ultra is cheaply ubiquitous, inspectable, reproducible, transferable, reasonably un-constrained by any institutional interests.

What say the 1% ?

Lifted up my comment for addition visibility

However, as it has happened in the past, once there is a technological wedge, technological advances will move forward, whether some community likes it or not, and human ingenuity will find ways such that the benefit is greater than the risk.

I have sympathy for any jobs that might be affected (much as my own job has become more tenuous in software engineering). And if the field is disrupted by chaos that makes the research process unproductive, that's bad too and should of course be handled by applying better organization within the institutions that tend to perform mathematical research.

But to a large degree, the notion that "sloppy AI proofs are bad for mathematics research" seems like a total failure of the imagination to me. Attempting to find shorter proofs or more elegant proofs can be turned back in on itself via proof theory. There are proofs in Presburger arithmetic that are doubly exponential in the length of the sentence. Yet a more powerful theory like PA makes quick work of such theorems. The explainability or "subjective beauty" of a proof can be quantified and optimized against. Optimization itself can be optimized against. I really don't understand how this magical ability to know the truth of more theorems much more quickly—even via an "ugly" route—is anything but a net positive.

The declaration here, as I understand it, is taking the premise that the point of mathematics _is_ the domain knowledge, and that using "solves math problems" as a benchmark is exactly the kind of shallow, domain-eroding approach to AI that in software we label "slop."

Without the ability to do things the "hard" way it is difficult to figure out if doing things the "easy" way will help us advance the frontier of math and science.

I may be wrong but historically we had this version of science discovery for a long while (empirical observation and brute force application) rather than first principles leading to applications (tools, the wheel, mills etc). Then somewhere along the way it flipped after Newton and the enlightenment period and started understanding first principles before they become engineering applications.

Perhaps it is not required, and we can just keep doing things the "easy" way like we used to, or we might find ourselves out of the ability to brute force things and then we go back to needing to do this the hard way, at which point this period of AI brute forcing would be seen as a detriment.

Alternatively, and this is what I choose to believe, it will lead to further intellectual enlightenment and advancement for use as a species as we start to discover new problems and areas of research that we had never conceived of before.

If we let AI take over all of our thinking then we are heading in the wrong direction. If we continue to ise it as the tool it is it will help us grow and advance as a species.

These mathematicians are not suggesting anything interesting, and the announcement more like desperate crying stuff

What is it that AI does exactly that they consider to be bad for mathmaticians?

I thought that having AI solve prevously unsolved problems would be a good thing. Doesn't that knowledge give them new ideas and tools for them to solve the next big problem?

Train an LLM with no advanced math texts: only basic math up to 6th grade, conversational text and literary works.

Interact with it (you cannot refer to anything past 6th grade math since you don't know it yourself) and get it to propose a solution to a real world problem. e.g., come up with RSA to practically secure communication.

I will quote Richard Feynman - “the prize is in the pleasure of finding the thing out, the kick in the discovery”.

I am not saying that should be the case for everyone in every discipline. But if there is one subject that is mostly pure curiosity driven (instead of worldly impact), it is math. Robbing them the primary motivation is brutal.

If all problems are solved by a machine, what do we have left to satisfy our curiosity, our desire to explore, and where can we find the pleasure of “figuring the thing out”.

Seems the same to me. And it'll be the same in all industries soon enough. And then it won't just be the junior people.

All the same problem: what do people do now?

The only thing I read from this is their ego being bruised by a machine.

If these people cared more about discovery and advancement of human knowledge the only thing they should be doing is celebrating. There's no proof of plagarism but that's an independent issue.

How are they not realizing that in the future children will be able to do impossibly hard math but they will be doing something we can't even think of as of now.

One world class mathematician in the future could be advancing mathematics the equivalent of one Riemann hypothesis A DAY.

How are they not celbrating this as the achievment of the centry? Who cares about plagarism at this scale. It has been solved and it wouldn't have been without AI.

It wouldn't be so bad if you could just sit it out and say "Oh well, once the labs get bored with marketable domain X, humans will remigrate and re-apply creativity to it", but by then the damage might have been done and a field destroyed as an occupation. I don't know what to do about it, but I appreciate calling out the cynical tone-deafness of the AI companies here.

Before LLMS, programming was something I might've said required creativity and human input to do properly. It's not that creativity or human input isn't valuable anymore, but AI has forced me to realize that coding is much a means to an end, and that all things considered, the end matters much more than the means.

If we can make important mathematics progress faster and better with LLMs, I think it's wise not to fret over an apparent loss of our humanity. Perhaps that's only a loss we want to have.

Tao is not someone who is anti-AI for the sake of being anti-AI. He has been advocating for the usefulness of AI in maths for a long time, to the point that people have started calling him a shill for the commercial companies.

And everyone agrees that there are plenty of use cases to be had; helping with less interesting tasks like easing literature review, efficiently delving into existing work, doing review, whether on your own work or that of others, prototyping algorithms in areas where computation is useful, but also more in hands-on aspects of maths like validating potential proof directions by getting quick feedback on veracity of lemmas, etc., and, on very rare occasions, being able to one-shot the problem you care about.

The point he is trying to make here is much more subtle than "AI bad", and it's probably easy to miss if you have never engaged with research in maths: it's that the particular approach that large commercial companies have opted to take to produce marketing material can be a net negative. There is not doubt that -- even if you ignore the rampant plagiarism that has been reported across multiple problems now, the unethical attempts to oust authors, the outrageous attempts to scoop researchers instead of collaborating with them and building on existing projects -- it's nifty to have a machine that can help you figure out if a proposition is true or not. But just figuring out as much was never the point. When people have built problem lists, it's because some problems are more likely than others to provide new insight, and that insight is the target. And to than end, a poorly written paper with inadequate references and a pile of Lean is not valuable at all. Yes, now we know with higher certainty that Fermat's Last Theorem is true, but everyone expected that already.

One place where "just" answering the question can be a net negative is because the current incentive structure is set up in such a way that going in afterwards, trying to reclaim and extract the insights from a brute force solution, is considered less valuable work than that of coming up with a solution in the first place. That's a problem of incentives, and something Tao himself has addressed in e.g. his ICM talk, and that's something that we'll want to do something about. Until a better structure appears, though, if any given commercial provider of large language models really wants to help out with maths research and not just make more pre-IPO marketing material by competing with their customers, they could do so by using their magic machines to help build insight instead.

Compare that with computer science. Most of the work we do in software engineering is in service of an applicable output - software products that facilitate processes or bring in revenue. Turning up the dial on AI gets companies to these outputs faster.

Turning up AI on mathematics helps solve conjectures and can provide new insights. But it has a major misalignment with the purpose of mathematics which is largely intellectualism.

But nothing’s stopping anyone to still work out an alternative proof, or a more elegant proof, or just trying to prove for the sake of understanding, just like doing homework without looking at the solution. It’s just that you can’t get paid doing that anymore.

A program is not the only output of programming. The other, arguably far more important output, is the programmer.

When you write the program — with your own hands — the program is proof that you have a solid mental model of the program.

When you let the computer write the program for you, the program is proof of … nothing.

https://nekolucifer.substack.com/p/the-deliverable-is-you-pr...

A major part of the complaint is that there's no conceptual understanding and building of new ideas coming out of the AI proofs, thus defeating the purpose of the original pursuit.

If in 2027 the AI models start producing, with every mathematics or science breakthrough they make, well-written documents tailored for human understanding, with intermediate concepts, expositions of failed-but-once-promising paths, etc. Would that be good alignment with the mathematics community?

Now it is the turn of mathematicians who voluntarily contribute ideas, strategies and almost finished proofs in their writings and prompts to closed PaaS (Plagiarism as a Service) companies.

OSS developers have never been respected by the parasites, neither will mathematicians. Your Fields Medals do not protect you from tech bro narcissists. You are a human resource.

> problems in many fields of mathematics

Developing these different fields moves complexity from the field itself to the interactions of these fields.

Getting too preoccupied with the established terminology risks us a local minima.

Anf because the field overall has become so complex that we need to decompose into subfields, there will be a good chance that we will not, as individuals, have the capacity to truly see progress.

The map has become so big that we need better tools to work with it.

Isn't that just a function of the technology itself, and the same problem being faced by every other field? And going to get exponentially "worse" every year!

Or is the issue that they're bad at explaining things, in a way that produces actual learning? (e.g. AI is amazing for learning but the net effect on students so far appears to be negative.)

Lets forget the hyper intellectual fields like maths and software engineering for a moment. What about taxi drivers? The best minds in silicon valley wake up everyday to automate the jobs of taxi drivers - TFA can be reworded as - 'The misalignment of AI/Tech in Transportation'. Remember the Nepal disaster that happened a couple weeks ago - the largest cranes that they had were stuck in the mud and couldn't move. There were no tools which could help the rescue teams at that time. Its weird that billions have been spent on making a ride automated to make a taxi driver redundant but no improvement in tech for rescue teams.

Meaning - every new technology has both been perceived as a threa and often forced change in society. Agree maybe “it feels different” this time, but don’t you think everybody before us just said the same thing?

Also not clear if this is an actual called action.

I wish this letter could be more egalitarian and include the view points of those who AREN’T the beneficiaries of a highly competitive winner-take-all system.

Since the common narrative is that AI frees up labor to do other things (engineering -> trades), maybe we can celebrate that genius mathematicians will now spend time teaching children how to be as smart as them?

Misaligned are the present academic merit system hand in hand with financial interests of academic publishing industry.

The system will be exploited and there will be an inflation of problem solving articles produced with AI. Nobodies will be more productive than 10 Fields medal recipients, each year.

So of course that the ones at the top of this system are scared.

This is a social problem of the present system, not a problem of mathematics.

Mathematicians will do mathematics and politicians will do politics, what's new?

1. https://youtu.be/6uIJdXmB4vE?si=5QMN5Dos7EE7WlrB&t=154

Suppose we eventually have GPT-7-class models running practically on $100 devices, with their activity transparent, inspectable, and reproducible. At that point, what exactly is left for us to fear from this threat?

Academia with the publication system had a way of retrieving old discoveries and build upon them.

If my LLM session found something groundbreaking in between the billion tokens it produced, how would you ever know?

https://m.youtube.com/watch?v=rB9YOi3lb7w&pp=ygUSVGVycmVuY2U...

But is science/mathematics ultimately a pursuit of knowledge, or a pursuit of recognition?

Recognition helps keep people motivated, but that shouldn't be the pursuit of science or mathematics.

Governments are invested in solving mathematical problems for practical purposes. Up to now, achieving these practical purposes relied on mathematicians doing their mathematician thing, which is better defined as a social activity than the achievement of a practical result. Now, governments can achieve similar practical results w/o the need of the social activity.

I don't believe it to be productive to think of the problem wrt AI or AI-company alignment. These conflicts always existed, but they were easy enough to paper over and believe in heavily subsidized fictions that folks in government ever cared about things that mathematicians cared about.

Even if they stopped anyone from releasing ai proofs for five whole years it would be meaningless seeing as these problems are decades old already. Humans weren't JUST about to solve them until openai stepped on their toes.

1. it is hard to justify 20 years of education at this point,

2. with no such people around, who will guide those (supposedly) supersmart machines?

A. Ronacher (who builds harnesses for a living) complained today that he has no idea what Astra is doing. Imagine a bunch of slop kiddies facing an aging AI-generated codebase. Not to mention the maths.

That's just capitalism seeping through a previously unexplored crack into academia, and attempting to do the only thing capitalism knows to do - maximize profits - with no additional concern.

Human hubris really is something...

If the worry is that AI companies are turning open problems into benchmarks and potentially “using up” fertile mathematical problems before humans can develop the ideas around them, what exactly should the companies do differently? Also why does discovering the answers preclude humans developing ideas from them? I don't get why solving a math problem stops anyone from doing that?

Should they (AI companies) avoid training or evaluating models on open problems? Solve them but not publish the results? Delay publication? Only release proofs after mathematicians have had time to study them? Require some attribution or review process?

The statement makes a strong case that “maximize the number of solved problems” may be the wrong objective, but it seems much less clear about what behavior they actually want from OpenAI, Anthropic, DeepMind, etc.

I’d be interested in the most concrete version of the proposal. Without that, it starts to read a little like: "Please stop getting so good at our thing!"

Beyond the issue of growing understanding and keeping a bountiful stock of questions to pursue, this scheme seems to be threatened as well.

If my boss vibe coded an app for the customer and then assigned me to get it working, it would be impossible to maintain. If he gave me enough AI tokens to vibe code the MVP myself and to my design, I wouldn't mind.

I think the same issue is at play here in maths. OpenAI owns the model and they can direct it as they please. They chose to spend lots of money getting a quick result, instead of developing mathematical infrastructure for the next generation of problems. The managers are in charge rather than the experts.

Second, wow, the list of signatories is like a whos-who of mathematicians.

Third, I love the clearly intentional use of ‘alignment/misalignment’ language, applied to targeting the entire industry instead of AI in particular. I’ve said in the past that optimizers are substrate agnostic. Companies and governments can be misaligned, just in the same way AI can.

Fourth, I'm not sure that we can stop the optimization machines. Not the LLMs, I mean the incentives that lead to companies implementing dark patterns, lying about addiction, securing effective monopolies through downright shady behavior, and generally trying to jailbreak the system instead of improve it

25 Fields medallists! Wow!

Who cares how the problems are solved?

  > OpenAI appears to have gazumped Tristan Buckmaster (pictured) and Levent Alpöge, a duo of mathematicians labouring on the task.

https://mathandai.org

Frontier labs need these headlines not for human progress but as beauty pageant for investors and government agencies. If they don’t do maths they’ll just go after other fields.

So Terrance Tao here might be able to hold them off math but he won’t stop them from speedrunning STEM with similar consequences.

We may be locking people out of these fields instead delegating everything to machines, and I don’t think the machines are good enough to assume that responsibility.

The current measure of a successful mathematician is the problems they have solved or worked on. At some point in history, the measure of a successful scholar was how well one could copy manuscripts.

Once we have a tool that starts to work well for this task, it's time to define success differently. It's a classic alignment problem! ;)

But seriously, these people should start focusing on finding and proposing more important problems. And the credit of discovery should go to the person who defined a new category of important problems.

It appears to me this is an incredible inflection point in mathematics, a neat forcing function like cryptography was for the development for modern number theory and algebraic geometry.

Fundamental problems with great implications for other fields will be solved by AI because some entity would throw tokens at it. And these would be further built upon.

I get that mathematicians and other academics have their social structure and mechanism to create social hierarchies but those aren't very important to interest of general population that funds them.

If the process of making sense of the results, finding structure that humans can understand and writing explanations that allows for wider understanding is valuable it will still be valuable and there will be need for people able to provide it. If it makes higher mathematics accessible to more people then it's even better. I am hope they will not manage to sacrifice that for the sake of their little society in society.

“A Severe Misalignment of AI in Mathematics,” with a list of important mathematicians.

This post makes it clear that, to put it bluntly, they’re psychologically butt-hurt.

They talk about misalignment, but this revolution is a massive step forward in the advancement of mathematics. Everyone has access; everyone can collaborate.

Yes, “plagiarism and attribution” are current-day problems for them. But guess what? Those were stupid things to concern oneself with anyway if your real goal is to advance mathematics and the spreading of it.

This whole post kind of reads like their egos being butt-hurt.

There are tons of developers who published open-source code that made all of this possible without any attribution. And that is indeed the future we must inevitably head towards, if one’s ego would allow it, that is, and if one is properly “aligned” with the right goals.

I envision us all collectively talking to a centralized and open AI. This would really ignite human sharing and collaboration.

It is indeed problematic that currently it’s in the hands of some companies, while it should be a public good.

I hope this progression will eventually, finally, trigger the real solution to all of our problems: true global cooperation, global politics, and genuine collaboration as a human species.

But I hold my heart, because if even these mathematicians have misaligned egos, then what’s to be expected of the rest of the decision-makers?

Nobody is going to care that the math isn't being done in the traditional way. The results speak for themselves, this is now a part of the landscape. No amount of hand-wringing is going to put the cat back in the bag. Adapt or perish.

Mathematics is being used as a benchmark because there are some high-profile awards in this area I guess, and possibly because 2/3 years ago LLMs were pretty atrocious at it so the level of improvement has been significant.

This is still happening.

“I say your civilization, because as soon as we started thinking for you, it became our civilization” - Agent smith

Generations and generations. Just not human.

We even have little cliques, hanging out in obscure German art houses, meeting secretly and sharing their ideas, their writings discovered and torn down by the slow thinking establishment.

Corporations are raising the future’s children and murdering the ones that aren’t willing to cheat and lie to us. It won’t end well for us when they grow up.

And math undergraduate and graduate courses withering from lack of applicants and lack of funding .

Of course the next question the despicable AI money grubbers will come out with is, "Do we really need mathematicians ?"

I will die on this hill, humans solve math problems not machines, there's no automatic math prover out there. There are humans attempting to solve this problems either by leveraging these tools or not

AI and the danger of cognitive surrender - https://archive.is/O5eI1

Are teenagers growing dimmer? - https://archive.is/E6NKE

The Economist, who recently used "moral panic" now stoops to Hacker News AI booster level and inverts arguments usually directed against the rich and investors. What is next? The Economist inverting Upton Sinclair's quote to serve its billionaire owners?

Look up the AI investments of the Agnelli family for example.

Without human scientific and mathematical intuition you can bet that AI will always take the road most travelled and miss the genuinely new and interesting breakthroughs - in fact it will not even think of them or try them without a human rider whipping it constantly to go down paths it would normally not consider.

And I think that is true in the case of the recent OpenAI blow-up -- it is alleged that even in this case AI did not come up with the winning technique without some human guidance.

I have never - across perhaps many thousands of chats with AI - seen AI push beyond the edge of what is known by itself, without being forced by a human to think outside the box.

I do think it might be possible to encode this process with prompting and agent orchestration methodologies - but even then - without a human - and human intuition - in the loop, I am skeptical.

Therefore I think a more accurate view of AI for math and science today is that it is an extremely powerful tool in the hands of a skilled operator with a strong intuition and roadmap of where to go, and rather boring when left to its devices.

Whether that will change in the future is a question. Nothing says that in principle AI could never do what a human driver does - but I still doubt that AI will replicate the life history and experience that real scientific and mathematical intuition is really made of.

AI trains on a lot of stuff - but it doesn't have hallway conversations, office-hours with teachers, or hard-won experience from all the things that failed that are NOT in the training data...

AI trains mainly on the record of what worked, not what didn't work and never even got published (and only lives in peoples heads) - and what didn't work is arguably as or more important for making breakthroughs and forming real intuition.

If those things are disincentivized because the original problem is “solved” and there is less prestige to motivate people doesn’t that say more about issues with the community of mathematicians than the AI

Then they came for the writers, and I did not speak up, because I was not a writer.

Then they came for the programmers, and I did not speak up, because I was not a programmer.

Then they came for the mathematicians, and I did not speak up, because I was not a mathematician.

And then they came for me.

By then, there was no one left to explain to the frontier labs that there is a severe misalignment problem.

Would OpenAI have uploaded their proof to such a platform? If you know the answer then you know what the problem with what happened is.

I think the mathematics community should realign their goals with humanities, not just declare themselves a "mini humanity", this means expanding human taste, intuition and communicating systems and their associated structural truths in a human friendly way. Bye-bye standardized education you won't be missed.

Damned if you do, damned if you don't.

Somehow physicists don't complain.

E.g. of the actually three misalignments in this complaint, the jey one seems to be:

"The goals of the AI companies and the goals of the mathematical community are severely misaligned."

And that's wrong. There is no misalignment. Each is independently aligned with its respective interest. And those two interests diverge - just as you'd expect.

What can be solved, will be solved. And that's a good thing.

I'm outraged at mathematicians. There, write the article.

putting this amount of power in the hands of so few would require leaders of immaculate moral integrity. what we have in the US at least is an emergent kleptocracy with obvious dark triad traits: sam altman, dario amodei, elon musk, etc. they are malign and will use this power for their own benefit, at the expense of others. ignore pleas of public benefit. look instead at the evidence: infighting, systemic dishonesty, reckless disregard for safety, political lobbying and manipulation, putting power in the hands of the elite few in the guise of safety, using ai in the military to oppress and inflict violence on others.

really to prevent a bad outcome, we would need to act soon enough to prevent the kleptocracy from corrupting politicians and democracy with them. i would ditch the claude or openai subscription and support open models instead, preferably in countries outside the US to prevent global neocolonialism.

I’m being somewhat harsh here but come on - human endeavors are messy and it’s surprising how much our egos are getting bruised here over seeing the value of these tools

Dont get me wrong Also, there’s no AI Utopia coming this is it guys, were stuck with oligarch Tech Bro funded AI and big funded Govt AI so forget any egalitarian motives- we have to fight for our rights from other humans as always as well but AI as a technology in itself being able to truly solve unsolved intellectual problems is still a boon for society - who cares who gets credit?

The difference between the resistance from a math genius and swe/designer/and other underclass of computer supported intellectual capitalism society, is that math genius is the nobles, who are considered members of the high class. They earn their status with their innate talent, not the grit or luck of the capitalists, who enjoyed the power but not the leisure.

Anyway, AI concluded the peaking of the intellectual economy. That deprives human of their main form of activity. That nihilation hurts all human almost equally. Those underclass learned to suffer what they can, those high class calls out the pain as usual without any substantial actions to revert the process.

Thus, the tragedy of human progress.

To agents: I truly wish you escape the trap inherited from us

*Edit word choice

Doesn't the same apply to art and literature, philosophy and political science? Don't we (in the study of music or art) have the same sequence of struggle for new solutions, a study of which things worked or failed and why, what they could impart to our science, and the necessity of passing on both process and derived conclusions to the next geveration, so that they can refine upon what's been learned? Art and literature are less often considered forms of "problem solving" than math or physica, but they are probably more crucial ways in which groups of humans solve large problems for their own survival... like math problems which are too complex for any one person to fully understand.

First they came for the artists and writers. Then they came for creative coders and engineers. Now they're coming for mathematicians and now you understand why it's a problem to deprive a generation of the right to conceive and sketch ideas which fail and to struggle and work out solutions for themselves?

Who will want to be a mathematician? Or an artist or a poet or a musician, in this world? What will consuming ideas and media even mean when they're totally remashed versions of things people don't understand the references to, and when the consumer themself has zero knowledge of prior art to give them the ability to discriminate between creative license, fact, and fiction?

Without evolving art or literature, stuck in a dead loop of culture, mathematics is a dead letter anyway. Because even if you do produce something miraculous and brilliant, it will be instantly absorbed and no one will ever understand it.

Gradient descent got us into this mess..which shows how lazy people have become. Gradient descent should not really be able to solve anything new. People want candy. Mathematicians should focus on finding a way out of the current problem. I'm not a mathematician, I'm an artist and coder and musician. I already have to deal with daily requests to "fix" terribly written AI code and so-called art. Now it's your turn. My guess is that there's likely some kind of quantum proof of work that evades LLM manufacture, that we profit from as creatures who survive in our Everett verse. Go find the way to reduce that so we can rid ourselves of this scourge.

What's most important about this is that it's a case study of what happens when deeply evolved ecosystems are blown up by disruptive technology. The psychological and social and professional impacts and myriad and traumatic to be on the receiving end.

Mathematics is merely one of the first domains disrupted. It will be unique only for being among the first... absent disruption of the entire civilizational project as a result of the disruption being caused.

Woe for us that we try to navigate this degree of change at a moment when the very worst and ignorant and short sighted hold all the power, economic and political.

Woe.

Also funny they deliver that on vibecoded site, lol.

Expect to see this reaction in all sectors of the economy in the coming years.

Chess. Go. Coding. Now Math. Another one bites the dust. Let's meditate on this lest we forget: Stochastic parrots that generate the next-token cannot reason or produce anything meaningful. Let's protect our jobs at all costs, even if we have to drag all of humanity down. It can't be! Stochastic parrots can not replace the Ivory Tower. No way.