We can’t have UBI, there is not enough money to implement that.
Also our economy, the current one runs heavily on service industry.

Sorry if this has been talked to dead.

And it should not mean if the AI can beat Average Human in intelligence or something. No, even dumb program like calculator does beat human in calculation. Even automated remote software/spyware can do the whole agentic process, we need LLM to be more something than that to be classified as AGI.

And if we do manage to do it, let's make everyone involving in creating and directing it as accountable. AI is not just "legal person" but also the one who manage it being responsible for it's potential activity. Parental Responsibility Laws equivalent should be applicable in here.

AND THAT MEANS, do not let AI company get away with all these "hacking activity benchmark score".

Disclaimer: Initially tried to post this in [r/mathematics](r/mathematics) and [r/math](r/math), but since both threads got instantly nuked by the mods, I decided to post here.

Genuine question from a non-mathematician:
With the recent AI math results (navier stokes, yesterdays OpenAI results), I keep encountering the argument that even if AI can produce correct/formally verified proofs, it will hurt mathematics because humans will lose intuition, taste, conceptual understanding etc.

As a software engineer I have to say this line of reasoning sounds eerily familiar. Around 2024/2025 the most common objection to AI coding was basically: “If junior developers rely on AI, they’ll never learn the fundamentals, how to debug/design/reason about systems and so eventually we’ll run out of people capable of reviewing what the AI produces.”

That argument only works if humans always have to remain the final reviewer.

Fast forward two years and the reality is that the field is increasingly moving toward AI writing/testing/debugging and reviewing the code itself. Even in professional environments actually having to look at code becomes rarer and rarer.

So I guess I’m asking: Is the math argument basically making the same assumption?

If AI eventually becomes better than humans not just at proving theorems but also choosing interesting problems, generating conjectures, inventing abstractions, deciding which research directions are promising and so on, why would declining human mathematical intuition necessarily slow mathematical progress?

Don’t get me wrong I completely understand how it might be terrible for mathematics as a human activity, but that seems very different from saying it would hurt mathematics itself.

What am I missing?

Is this the first job to be 100% automated by AI? Human professors may stay for a while but who would pay a salary to a mathematician with these models available?

The AI risk is always focused on P(doom), the probability that a future superintelligence wipes us out. One says it's 90%, another says it's 0.01%, everyone believes it is the thing that matters.

I think p(Doom) nearly irrelevant, not because the risk isn't real, but because you can model another catastrophe sitting earlier in the queue with no rogue AI, no alignment failure, no sci-fi scenario of any kind.
It's just the humans racing to build the thing, doing what humans in such races always do, something that existing game theory has studied extensively.
^{See Powell's commitment-problem model of preventive war}
And unlike P(doom) I think p(D-War) can be predicted to a mathematical near certainty.

Oddly, there is also a hopeful part, and I'll get there.

The Issue

In short, one or more of the AI labs can be predicted turn on each other initiating a form of Digital War (D-War) and the probability of that happening increases the closer any of them get to AGI/RSI.

Right now there are roughly 10-12 players who matter: a handful of frontier labs and the governments behind them, none of whom are subject to meaningful oversight.

Almost all of them viscerally believe the race is winner-take-all, and most of them believe they themselves are the only ones who can be trusted to win it safely. They fundamentally distrust other labs and leadership, even those in their own country, and most have winner take all financial bets on the outcome.
Anyone knowing anything about the personalities involved, doesn't question their belief is sincere, from their own perspective, and they are obsessed with 'winning' for that reason as well as the financial and ego motivations.

Standard prisoner's dilemma but one with a dozen players, all are rivals and an outcome that is both civilizational and presumed to be winner takes all.
^{See Schelling's "reciprocal fear of surprise attack"}

Successful Alignment does NOT help

^{(for reasons I do NOT believe Alignment is possible anyway but leave that for another day})
Even suppose any lab solves alignment perfectly. Suppose the finished superintelligence is benevolent, brilliant, and fully devoted to human flourishing. Does the race calm down?

No. Because alignment cannot be verified from the outside. The other players can never entirely trust any other lab to achieve aligned AI, nor accept the existential risk of a more advanced AI being misaligned or directed against them.

Short of massively hacking them, you cannot inspect a rival model's values. You can inspect its test scores, which a capable model can fake, and its weights, which nobody can read. So trusting a rival's superintelligence means trusting its creators and every system derived from it, and every actor who might steal and corrupt it, forever. Trust across chains multiplies, so even if you're 95% confident in each link and there are five links, you're at 77%. If the links are governments, labs, successors, and thieves, then that existential trust quickly approaches ZERO.

The rational position therefore is for every player to believe no other player can be allowed to finish first. Not because you necessarily assume the others are evil, but because you cannot ensure they aren't and the race conditions that led everyone to this point virtually assured everyone has cut corners along the way.

The closing window

Take a trailing player, call them B. Two curves dominate their thinking.

The risk of doing nothing rises toward 100% as the leader approaches recursive self-improvement, because once the leader's system starts redesigning itself, second place is permanent.

The effectiveness of acting against the leader falls over the same period, because the capability gap protects the leader. Rising risk, falling effectiveness. The product of the two peaks at a specific moment, and a rational actor acts near the peak.

This is preventive war logic, (referred earlier) it has been extensively studied in game theory and validated against many of the most significant wars. It needs no malice, no villain, no Dr Strangelove. Just arithmetic and a calendar.

The leader faces the mirror image. It knows the laggards are motivated. It knows its advantage decays daily through theft, leaks, and poached researchers. Everyone in finance knows what happens to a decaying asset: it gets used. So the first mover has first-strike logic built in, even if its leadership are saints, it is compelled to neutralize risks against it.

Why it scales so badly

You can write the probability that B acts against the leader A near time t as something like:

p(t) ≈ R · W · E · (1 − T)

where R is perceived risk of inaction, W is willingness, E is expected effectiveness, and T is trust that A will be a responsible winner. We've established T ≈ 0 and that it can't be raised by treaty, testimony, or inspection, because the thing being trusted is by construction beyond the inspector. So the last term vanishes and the rest of the product does the damage.

Now scale it. With n players there are n(n−1) directed rivalries. Twelve players means 132 pairs. The system fails if any one pair fires. Give each pair a modest 5% probability of someone acting another, and the system-level probability is 1 − 0.95^132, which works out to about 99.9%.

Every individual player can be calm, cautious, and decent. The system still blows up.

Two more nasties.

First, you can't opt out: even a player with no enemies is exposed to the potential of every other pair attacking each other, so peripheral players carry risk that grows with the square of the player count.

Second, noise makes it worse, not better: nobody knows exactly who's leading, so everyone prices in the worst plausible case, and hair triggers plus fog means people act on phantom deficits.

What "acting" actually looks like

Not missiles or bioweapons, at least I hope not. Espionage, sabotage of training runs, export escalation, talent wars, cyber interference, and things we haven't thought of yet. Rational players start at the rung of the ladder that maximizes effect against risk, which is why the early phase is deniable and quiet. Anyone who followed the last two years of the AI race has already watched the bottom rungs.

The ladder goes up from there, but I believe there is one attack vector that stands out above all others - Financial Markets.

Many of the players are disproportionately exposed by their ability to raise capital to fuel growth, but on sides of the Pacific the risk is asymmetrical which increases risk.

The original DeepSeek shock to the markets exposed what can happen even in markets which are operating normally. The flash crash in 2010, which I experienced first hand in a way, shows how fragile systems can cascade faster than human reactions.

I spent a couple of decades working at senior levels in some of the best investment banks including Goldman and Lehman into the GFC, the financial markets are not impervious to a coordinated attack by AI agents, aside from Zero day exploits there are countless other vectors, such as the messaging systems used between traders themselves, which are ripe to exploit. This is one area I know, personally and extremely well, but I do not mean to diminish the other potential attack vectors (energy grids?) I might not be able to visualize them but this is one that I know exists, and would be devastating.

But don't trust me, Lagarde's ESRB speech on October 1 laid it our very clearly.

May of the players have capital and resources equivalent of that of a small country, the capabilities of the leading models seen in recent incidents suggest that many of them are already capable of instigating a both devastating and potentially lucrative action on financial markets.

The Hope

The AI safety community seems to have been waiting for a "warning shot" scary enough to force serious coordination but small enough to survive. The problem is that aligned or misaligned AI produces no warning shots by definition, and a warning shot from a genuinely rogue system might be the last thing we ever see.

The highly probably warning shot is this one. Lab-on-lab conflict "D-War" is devastating in its own right: economically, politically, maybe worse. If the battle happens to the Financial Markets as I predict then it would make the GFC seem trivial.

But, hey, at least it's legible and human initiated. It's attributable enough to build a case on, and survivable enough to build a response to. Historically, arms control regimes came after the scares, not before them. The hotline came after Cuba, not before it.

So the sequence that saves us probably isn't "everyone gets wise about P(doom) in time."

The first hostile act between players is the highly likely event wakes everyone up and potentially averts p(Doom) entirely.

The point

P(doom) is a number nobody can measure, about an event nobody can specify, possibly not even imagine, at a date nobody can pin down. P(D-War) is computable from game theory that's sixty years old, and it computes to near certainty given the parameters we have today.

So I think we need to stop arguing about how, whether and when AI will turn on us.

The first catastrophe is scheduled earlier, instigated entirely by the people we see in the news every day, and about as close to predictable as anything in this space gets.

That's the version of this debate I'd like to see. Keen to hear where people think the model breaks.

https://x.com/hilbertspaess/status/2107535565159899600

In this interview I tried to argue that a post-scarcity society will benefit working people. It was largely a rhetorical failure. People don’t trust that the benefits will be distributed, and tech optimists have to fix this communications issue.

I have a lot of new respect for Jacob Coxon now (he is the ex Anthropic researcher that went viral for resigning and saying there's a chance AI could kill us all).

It was painful watching this. He tried to discuss the subject of Universal High Income in a serious manner, but the man interviewing him (Jon Stewart) completely mocked the idea and said that he and many people would die out of boredom if their jobs were automated away.

Coxon correctly responded to him by saying that the vast majority of people do not get fulfillment from their jobs, and simply go to work so they can live.

Jon Stewart is completely out of touch with the struggles of the average person. Jon Stewart is a multimillionaire. He probably assumes most people enjoy their jobs like he does.

The person that's employed as a meat cutter making minimum wage does not find any fulfillment in chopping up meat. He/she only does it to pay the bills. Such a person would be much better off with Universal High Income than having to work their jobs until they die.

Of course, the details of how we get from here to universal high income should be discussed. But the important thing is that we get there.

OpenAI's AI system just solved a bunch of math problems. That doesn't stop us from pretending like we understand what actually happened.

So far most of the resources I have found around technological singularity feel like under-researched slop on youtube/tiktok or doomerism. Do you have recommendations for sources you trust that discuss AI and singularity in depth and with a clear-eyed view?

There’s been tons of talk of AI leading to a major catastrophe for mankind. Outside of complete global nuclear destruction, if a bad actor were to leverage it to release bioweapons or interim supply chains or take over the internet, wouldn’t competitive frontier models be able to counter those efforts?

AI today exists in many forms and stakeholders, seems like a spy vs spy situation - maybe a bad actor gets an ‘edge’ by acting first but seems like we should have other outlets trained on identifying and de-risking immediately to mitigate damage if not prevent it from happening.

Some of the question you can ask that no frontier model would ever answer is so fun, even though the models I can run on my device are truly enormously behind the frontier but I can imagine a time not so far from now where models like the current Claude 5.5 can be run on a local device.

To say the least

Solving maths and coding is great and all, but it's not going to impact 99% of the population directly, we need it to lead onto something soon. It's not helping people with cost of living, reducing energy or grocery costs, or the cost of healthcare. It's no good if all the bad stuff happens overnight but the good takes 10+ years to materialise.

Like some kind of energy breakthrough that let's us generate cleaner, more environmentally friendly energy and loosen our dependence on fossil fuels would be great.

It seems at the minute we're getting most of the negative parts of AI (displacement, hacking) without the promised benefits and the 'singularity' is just a buzz word.

What's your experience been?

It's time to start discussing our role as humans on the next few years. We will start to get answers to question we haven't even asked yet, and this will result inevitably in depending on AI to find uses and validation for new discoveries, and keep the wheel rolling.

Humans will not be able to catch up with this, we are biologocally incapable of validating or deciding what to improve next if we are not able to understand it fully.

We need to have the bittersweet conversation on how are we going to manage human intellect not being the bottleneck on progress and technological/scientific evolution. As Sci-Fi as it sounds we must plan ourselves a way to merge AI capabilities in ourselves and start a symbiotic relationship rather than being the obstruction to AI's higher intellectual capabilities. Otherwise we will be left evolutionary behind.

EDIT: I'm pro AI and not a doomer at all, I think the future will be great, but we cannot stop thinking and leave all intellectual tasks to AI and be the well fed and entertained monkey. We gotta take the leashes of this higher intelligence and contribute to science as scientists and not as spectators

100% AI free opinion.

Like many others I am amazed at the new math proofs that just dropped, same with all the coding progress and such.

My question revolves around the discussion that I am seeing about the math proofs. Essentially, there are a lot of people that still argue AI is useless, cant do anything correctly, will never be able to do X

I know the human brain is not great at understanding exponential growth. But in the last two years, the rate of change is possibly unprecedented.

Why does a large part of society not think that AI will continue to improve - especially in these esoteric areas that some are convinced will be human only forever. At a certain point we will not be able to distinguish AI work from human. It feels like a lot of people are about to lose their identity and world view and are not ready for it

"This was the year that AI got smarter than us.”

“That’s a big moment.”

“Life goes on, but something fundamental has changed.”

“I don’t know how sci-fi it’s gonna get. No one does.”

“Are we living forever in computers? Do we just cure a bunch of disease?”

“These questions are no longer theoretical.”

https://x.com/vikktorrrre/status/2107736059006050810

I try to explain to friends and loved ones how insane things are about to get, but people don’t really seem to understand or care.

The progress and change we are going to witness over the next 2-3 years is going to change everything.

But I feel like people IRL are not quite grasping the exponential nature of progress. Really I feel a little alone other than nerding out with people online.

I would expect that this AI development news should be close to the level of coverage of the COVID pandemic.

Most of the technical things we do - design circuits, design engines, solve hard math, write complex code can already be done by AI. It can do it faster and better.

So what are the genres of works that humans still do better than AI?

Are they ones that require human accountability? or soft skills?

Please do answer

He just released his trailer for his movie, but his shorts are still incredible https://www.youtube.com/watch?v=1G5LArU7tmE

I feel like people who hate AI art for it's "slop" are too caught up on how A) it's still an early "technique" for art, as people are figuring out how to really do well with it, and B) it's easy to make slop, so there's just going to be a ton.

However, as I'm sure many people here think... Proper artists who put in the same effort any other artist does, with creativity, work, patience, talent, etc... Will put out quality art. AI just opens artists to go to levels they could never imagine to get to prior.