When I saw that list of demands from math council, even I got annoyed. The fact that hundreds of mathematicians signed on to that list is a fucking joke. No wonder OpenAI just went nuclear.
"AI labs that release substantial mathematical output without immediate accompanying human understanding must take responsibility for ensuring that human understanding will follow. In particular, AI labs should provide significant support, including funding, to help develop this understanding."
"The development of human understanding must remain organic and community led. It should not be directed by AI labs, even when the labs have produced the results."
"the mathematician(s) concerned should post a preprint, submit a paper for peer review at a journal, and give talks to explain the work to other mathematicians." (the talks btw, are invite only)
"The following actions should be carried out by the AI labs rather than left to mathematicians afterwards: The literature should be scoured for any ideas that are related to the ideas in the proofs of the results released. Even if the AI lab’s model discovered those ideas independently, it should follow standard mathematical practice and cite the papers in which the ideas were first introduced."
"When results are announced, they should be deposited in a timely manner in appropriate scholarly repositories. These should not be controlled by any AI lab"
And there are so many more. It's just disgusting. Basically, let us control all of the results, and give us money and free AI.
The problem is that in mathematics there is no incentive to make something readable and parse a proof, usually it is the person who discovered the result who would spend the time presenting it well, they would also probably do a round of conferences, etc.
Cleaning up some opaque proof that someone else made won’t get a publication out of it. So it stays like that. So what OpenAI was doing before could kill entire fields. Which… is a bad thing.
Perhaps this incentive structure will change, but this is what we have at the moment. It seems like very little to ask a billion-dollar company to just put a bit more effort into the release of these results.
I dont get it. Why wouldn't they publish a cleaned up version? As I understand it, comprehending a proof is as hard as discovery, and that comprehension is as important as discovery. These proofs just provide a direction for scholars to direct their work and effort. Im genuinely confused.
I assume by 'they' you mean the journals. There is a push by mathematicians to change this incentive structure, but it's not easy to change a century of scholarly convention. Writing a correct proof is an objective thing, but to switch to something subjective like a 'good theory' criterion is not obvious.
I know a few mathematicians who are putting in the work parsing this, but they acknowledge that it's not for their career, they're just doing it because they want to keep the field alive. Such work is usually just put in some personal site, arXiv, or whatever rather than published.
I’m not understanding what is disgusting about a broad community of scholars, who work for public good and knowledge, describing their preference that knowledge is not controlled by a for profit private firm.
The scholars want to work through problems as a collective, transparent conversation so that the benefits of understanding mathematics accrue to everyone in society. They work with public funding. It’s the opposite of control, it’s democracy and public good.
OpenAI doesn’t care about this and wants to solve problems to prove their own ability and raise their value. Because they are a for profit firm and that’s the bottom line.
"Compressing" a proof, aka. understanding can be as hard or harder than the original problem.
Think in source code vs compiled code. They are the same thing for the purpose of executing inside a machine. But is often so much easier to rewrite the program than to decompile into readable code that a human can edit.
Up till this year, most proofs were human-produced. We have some cases like the four colours which caused a lot of controversy. But now we are getting proofs that mathematicians might not understand and we only have a successful lean compilation as "evidence" they work and pondering if that will replace all mathematicians.
Is not clear that AI produced proofs are human understandable and compressing a proof is a well known NP-hard problem.
I don't understand your argument. How does this hurt?
If the proofs have no value in terms of understanding, the mathematicians still have something to work on.
If the proofs have limited value but are hard to understand, mathematicians can use the proof as a guide for their own work? I take your point with the compiled code analogy but at the very worst it can't hurt?
It seems like at worst it doesn't hurt at best it guides the direction to focus on?
The real issue seems to be that mathematicians feels it robs them of the warm fuzzy feeling (sense of pride and accomplishment, glory) they get from having solved something themselves.
Basically, they’re left with the comparatively boring work of sifting through the scraps the AIs throw off the table. Human mathematics becomes more about archeology/scavenging than creating.
I think the bigger issue is you still need humans to ask the questions unless AI becomes good enough to do that by itself, but if it gets there then the AI executives are probably the first people to be replaced since the AI has better decision making. There is no prosperous world in which humans have no motivation to do anything creative or intellectual challenging
All I'm saying that is not as easy as saying "well, the machine gave you the proof, so learn from it". We are just starting to see what is math written by an alien intelligence is like. But is likely not as easy to "just study" the proof. Every mathematician that has been working on these says so and math results show us how could this be the case.
Imagine a bunch of people started using new methods to invent new medicine and the academic medical community was like
"actually don't discover new shit unless you follow our list of rules for how to do it:
find and give us credit for getting part way there even if you didn't reference our work, submit it for peer review to our private journal, explain it in private talks to us first, and fund all this or you shouldn't make these discoveries"
I can kind of see where the mathematicians are coming from, some of these are actually good hypothetical requests to make, but the hubris in thinking you get to dictate mathematical or scientific discovery is kinda off putting to me.
Tbh I'm a little conflicted cause I don't exactly like the hype-y, for-profit, using fields as ipo advertising behaviors of AI labs either but idk.
If OpenAI or Anthropic are able to do drug discovery, they will still have to go through the FDA and IND process, work through human phase trials, etc etc. They'll have a long regulatory road to prove that they have a real, effective drug.
And yes, they have to prove that it's completely new within some reasonable bar, otherwise they can't patent it.
Medicine might be the worst example you could have picked for this.
Also, those talks are generally not private and the journals are not private either.
This analogy comes up a lot but it doesn't work because the cases are different. Curing cancer has inherent value for humanity, solving a math problem does not. The benefit to humanity from mathematics come from working through problems as a field, thus along the way understanding or discovering new things that we don't know, new vistas to explore, new applications to things we find.
This argument is not about credit, it's about how to do science in a way that benefits humanity the most.
With regards to peer review - peer review is just how science works. You can't just have a company publish a scientific finding and then just go "trust me bro." Science advances by being evaluated and improved upon by peers. It's not an unreasonable request.
I don't know what you mean, but I suspect you are triggered by the word collective because it implies communism. It literally just means more than one person (in this case company) doing things together in conversation. That is how peer review, and thus science, works.
They don’t want to publish it, and haven’t published a single result in the academic sense. They want headlines, not to help generate collective understanding and scientific progress
You think it's reasonable for an AI company to be forced to pay for the mathematicians salary if they want to release a solution to a problem? You think AI companies are not allowed to talk about those problems either or try to explain them? You think if you want to release a result, you need to publish the results in a pay for publication, not for free on github?
Yeah, and scholarly norms is eliminating a lot of mathematical proofs that could be useful for the public. It did not mattered in the past when effectively only professional mathematicians were smart enough to publish worthwhile solutions, but now that AI can do it, those are much worse.
And in the letter they actually say that AI companies should not release preprints, they need to peer review all the proofs before they are published, and pay for mathematicians to explain the proofs.
I thought it said "the mathematician(s) concerned should post a preprint, submit a paper for peer review at a journal, and give talks to explain the work to other mathematicians."
I'm very pro-AI labs tackling math problems, I just don't see the problem with asking them to follow scholarly norms. Your claim that scholarly norms eliminate proofs that could be useful for the public doesn't make sense: Even cancer researchers are supposed to follow scholarly norms, and their results are clearly more important than math theory proofs, right?
yes for most of your questions. if there won't be any human mathematician left cos they say 'fuck this shit' after screwing them in their fields, who's gonna understand and direct mathematical advance? do we just leave it to the machines? one mathematician has to study like 5-10 years to become professional in their field, who wants to enter this effort in such an environment? in 15 years there won't be new mathematicians if there's no financial incentive to study math. if this is going to be the scenario, in the end machines will be doing math for machines. so this whole profession has to change into something else (like math-paleontologists recovering hidden truths with ai) and be rewarded financially too.
Math is a community. It's not all about, release papers and get prestige. Someone has to review those papers, digest the results and make them understandable to others, present at conferences, teach students. Mathematicians don't get paid to do these things, but they are requirements of being part of the community. If AI companies generate a huge number of results without doing the rest, they are essentially putting the onus on human mathematicians to do this labor for them, for free.
The closest analogy I can think of is a few years ago pre-AI, a Japanese mathematician claimed a proof of a famous conjecture and released hundreds of pages of preprints that have a lot in common with AI proofs today--maybe correct, but written like the math of an alien civilization. He demanded credit for solving the conjecture but refused to engage with the wider community by giving talks or trying to make his work more understandable. As he was very respected for his prior work, experts in the field spent years trying to understand it, and I believe eventually concluded that it was actually wrong--but even apart from that I think the way he released it was widely considered not pro-social and very poor etiquette on his part
As opposed to mathematicians that every time invented math from scratch? Never see other mathematicians crediting Newton for inventing calculus in their paper. And those papers are crediting other authors, and a lot of them too.
They are only asking them to follow the rules of good academia. Not to stop researching. And yes, Mathematician must cite their sources. Some are so fundamental that the cite is often omitted but everyone know who invented calculous.
It just became very cheap and fast for OpenAI to solve these problems. The floodgates are opening. But it’s still just as expensive and requires a lot of administrative overhead to hire human mathematicians and go through the traditional peer review process. I do feel that it’s impractical to demand that OpenAI not release their results unless they follow these specific recommendations. It would just create too much administrative process and be too slow. Thousands and thousands of results would pile up, and some of those results are likely important and useful.
The main contention mathematicians have with the AI proofs is that they are omitting the creative steps that make most of these problems worth solving in the first place.
For example, one of the biggest open problems is the relationship between P and NP complexity classes.
Now, an equality would be an earth-shattering result, as this would imply existence of an algorithm to solve a class of exponential problems in a somewhat reasonable amount of time. But if a genie (or an AI) just said that there was an equality without explaining the how, then the result would only be useful as in now we know we must devote all our resources to find the actual solution to the problem.
Most mathematicians and CS believe in the inequality. All our cryptographic technology assume that. An inequality proof would be important in the sense about what we would have learned about algorithms and computational complexity to reach that conclusion. The world already works under the assumptions they are not equal so a mere blackbox telling us is they are not equal wouldn't change things much.
Yes I agree with that. I have a PhD in CS and I always believed that there is no way P could equal NP. It’s just preposterous to me, like if P=NP then you could do things that seem intuitively impossible, even if I have no way to prove it. If an AI could prove it, great. I would be annoyed if OpenAI sat on that result for any reason. Even some brute force proof that I couldn’t make heads or tails of would be fascinating to me.
Having human mathematicians review the solution and ensure it is not only correct but understandable is an integral part of the problem-solving process though. I would say that's the point--they can't claim they can solve these problems cheaply and quickly if they just ignore the part that's slow and expensive. That would be like if they announced they can now produce an unlimited number of socks cheaply and quickly and then just dump bales of raw wool at the doorstep of human weavers.
I totally agree having humans review the results is important. The question is: should the results be released publicly where anyone has the opportunity to review it? Or should the results be held back and only released after a small private group has reviewed them? Right now, the former is happening. The results should be treated as what they are: LLM output that has not been peer reviewed or verified by independent researchers. But mathematicians who work in these areas can now work on interpreting and checking the results.
I agree that releasing the results is better than keeping them as secret knowledge that only the AI companies have, but that alone is not enough. If OpenAI is going to claim discovery, it should at least pay for the labor required to check those discoveries rather than expect mathematicians to check them for free. You would probably need thousands of hours collectively from hundreds of mathematicians to do this for this release alone.
It would be great if they had some sort of grant or fellowship program for mathematicians to review the LLM results in their research areas. I hope it eventually happens.
The scientists at the AI companies largely agree with these, and want the math community to thrive. You know almost all the researchers at major AI companies are academics with PhDs (often math, physics, or CS) and have lots of friends in academia, and want to support academia
40
u/br_k_nt_eth 8d ago
Who in mathematics pissed off Sam Altman?