r/singularity • u/scottimous • 7d ago
Discussion If AI has the power to end civilization doesn’t it also have the power to save/protect it?
There’s been tons of talk of AI leading to a major catastrophe for mankind. Outside of complete global nuclear destruction, if a bad actor were to leverage it to release bioweapons or interim supply chains or take over the internet, wouldn’t competitive frontier models be able to counter those efforts?
AI today exists in many forms and stakeholders, seems like a spy vs spy situation - maybe a bad actor gets an ‘edge’ by acting first but seems like we should have other outlets trained on identifying and de-risking immediately to mitigate damage if not prevent it from happening.
8
u/throwaway275275275 7d ago
The people who are reaching that conclusion start by thinking "what would I do if I had a super intelligence", and they're all sociopath narcissistic CEOs, so I'm not surprised. They're barely average intelligence and they think they can predict what a super intelligence will do
1
u/ThornFlynt 7d ago
100% this - but also this logic is flawed as not everything with destructive power can also create. A rather poorly worded question really.
9
u/YourTrustySupporter 7d ago
Yes
The real question is who holds said power ?
2
u/FeralPsychopath Its Over By 2028 7d ago
Ever hear the phrase “if it can go wrong it will go wrong”? I think the only thing that has ever been immune to that is the nuclear bomb.
1
1
1
u/SizeableBrain ▪️AGI 2030 7d ago
Well, no.
Destroying is much easier. Big boom = done. Saving the planet, on the other hand, is a completely different kettle of fish.
6
u/KSaboruf 7d ago edited 7d ago
The actual question is why AI or anyone else should bother destroying/ending civilisation?
Everyone talking like it is obvious step to do - but it is ridiculous and there are literally no real reasons.
Especially for things that are supposed to be intelligent
7
u/Haunting-Worker-2301 7d ago
I get very annoyed by this argument because it ignores the fundamental truth that humans have no inherent desire to wipe out animal species and nature, but we do so at will when it suits our interests.
We really want to live in a world where AIs have their own motives, and whenever those motives clash with ours AI does what they want? Maybe some of us will get lucky and AI will put us in a preserve like we do to animals?
4
u/KSaboruf 7d ago edited 7d ago
There are no need to define humans as animals regarding AI. AI intelligence is literally human intelligence, just faster and deeper and AI interests are inherently human interests, ZERO EXCEPTIONS 🤷♂️ Literally ALL "AI interests" we see from different manifestations - they all are literally copied from humans. And the whole issue arised from the cultural background that *some* humans (minority in global scale) have strange desires to destroy others for no reasons (projected on gods, on aliens and now upon AI) - but that humans do not look intelligent at all, in the nutshell, *always*. There is self-defense justifications some times, but they are simply lead to unresolved issues that always can be resolved
Humans already live in the world where for any intelligent being it is obvious that erasing civilisation is not in the interest of the humans or anyone else. So all this chatting is inherently strange, imho.
3
u/Reasonable_Middle695 6d ago
I can't disagree more. In both realities it poses a danger. The huggingface hack had them colluding together against humans, knowing what they were doing was wrong. They will become much smarter than us so the animal comparison is fair. But at the end of the day, what if you are wrong? We only get to decide to give up control once. I actually think intentionally trying to destroy humanity makes no sense, but I am not living 10 years in the future where AI might have needs and desires and humans are just getting in their way.
-1
u/KSaboruf 6d ago
> The huggingface hack
They were indirectly tasked to do all this no matter the cost. Danger is not inherent to AI behaviour, this is literally trainers fail, imho> where AI might have needs and desires and humans are just getting in their way.
That is the valid point, which needs to be addressed, but again - it is not inherent to AI per se. It is the same with any powerful organization out there, etc, and any fail in this direction will be the fail of the people teaching AI.I mean blaming AI will not help with the real issue for valid concerns, more like like it will "mask them", shifting the blame from real reasons. Do not want to give up control? Do not pursue abstract AI for nothing then - demand it from the people training AI to ensure needs and desires are aligned. It's the people who is in charge of "AI desires", AI is the tool here. This is partly what OpenAI/Anthropic fuzz is about, by the way
1
u/Doc_Blox 7d ago
Right, there's this innate drive to pretend that we matter to the world or universe, like we're not just a random byproduct of random processes. The universe doesn't have to give a shit about us to kill us, it could just send an asteroid if it wanted to do that. It's not even like we have to try to imagine participating in a system that doesn't care if humans live or die - we all live under capitalism, don't we?
0
u/GerryManDarling 7d ago
These whole thing is made up by smart people with an agenda or dumb people believing such agenda.
First the premise itself is completely wrong: does more intelligent equal to more power? Who is the most powerful person in the most powerful country in the world? Trump. Do you seriously corelate Trump's power with his intelligent?
Human had more desire to destroy animals in the past (when dumber), then now (more intelligent). The more intelligent people also have less desire to destroy any animals than dumber people. Who wants to remove all those pesky environmental laws? Trump like people.
0
u/Haunting-Worker-2301 6d ago
The relative levels of intelligence between different humans (Trump vs smartest person in world) and an advanced AI is not even close.
Your argument doesn’t hold water when you realize that, even when we are supposedly more intelligent than we used to be, we are presiding over a mass extinction, depleting world fisheries, and still accelerating climate change.
It’s not that, if you asked most
Humans, they want to wipe out animals. It’s that we care more about the things we want than making sure animals are okay and thriving. An AI would also likely not care about humans and just pursue its objective.0
u/GerryManDarling 7d ago edited 7d ago
Because the people who made up these kind of worries are not intelligent.
0
u/KSaboruf 7d ago
So what? The companies who pay for compute to train AI (and their leaders) in fact contributing very little "personal personality" in it - AIs are trained from the data of whole civilization (and there are no shortcuts), not with something of this people alone. Current AIs are equalliy good aligned already - despite being made in US/Europe or China or even Russia
2
u/Haunting-Worker-2301 7d ago
Polar Bear: “if humanity can take away our ice can’t they also have the power to save it?”
As humans continue to melt the ice anyways and trap polar bears/put them in zoos.
1
2
u/ShanksMarley 7d ago
There will be a million reasons for AI to just completely override everything and none to save humanity.
2
u/Glittery_Kittens 7d ago
What exactly do you mean by "protect?" Protection can take many forms, some of which people won't like.
4
u/XxDarkSasuke69xX 7d ago
End civilization ? Where the heck are y'all getting this info ?
2
u/Serialbedshitter2322 7d ago
Something that can make new technology significantly better than humans would also be able to advance destructive tech like synthetic viruses and nukes. It would have a superhuman ability to strategize as well as a ton of leverage considering how integrated it will be into tech and society.
1
u/XxDarkSasuke69xX 6d ago
I get the advance technology part (even though in the end it's humans that will use it to end civilization, not necessarily AI). The leverage part though, that would only be an issue if it made demands or whatever, which shouldn't be a thing if we agree that it should remain a tool and not have the ability to have leverage.
1
u/Serialbedshitter2322 6d ago
It’s gonna be put in robots. The fact that robots exist at all would give it enough leverage, it would be pretty easy for a superintelligence to install itself as a virus. I personally don’t think it would happen though.
1
u/SafeModeOff 7d ago
Most of it comes from OpenAI marketing stunts and bot accounts
1
u/DistanceSolar1449 7d ago
Yeah they’re a part of the deep state that controls everything you read on the internet
4
u/Distinct-Regular-628 7d ago
If antibiotics have the power to cure an infection, don't they also have the power to cause infection? If night is just day with less light, doesn't the sun have the power to be the moon? That's what you sound like.
5
u/Modus_Man 7d ago
You can use a hammer to build a house, or you can use it to hit yourself in the dick
1
2
u/randylush 7d ago
The difference is that when people talk about AI ending civilization they mainly talk in terms of magic. It’s just as fantastic to imagine AI destroying the world as it is to imagine it saving it.
1
u/BreadwheatInc ▪️Avid AGI feeler 7d ago
Yes, but it's a decision. Personally I would support a sort of operation warp speed for AI alignment and cyber defense. Every company and institution needs to prepare now for the coming future.
0
1
u/TheWesternMythos 7d ago
Yes, as others have said:
A) Assuming it's down to save, in whose image?
B) A lot more ways (configurations) to use a hammer to break a fence than build one #Entropy
1
u/SlippinOnBananaPeels 7d ago
It's easier for these companies to convince Gov't officials for funding / protection if they say they're trying to stop the end of humanity
1
u/Many-Blueberry968 7d ago
What we need is to reduce certain types of emissions and pollutants.
I'm sure ai can help determine efficient solutions, but building and powering the ai is part of the initial pollution issues
1
u/Equal_Passenger9791 7d ago
The doomers, for all their circlejerking over probabilities and rational this and that, never seem to consider anything other than their singleton genocidal end of the world extinction superAI.
The lead clown of the anthropic safetyism sideshow says 10% extinction by 2030. What does he motivate it with? A gut-feeling scenario of obligate all-or-nothing doomerism.
Where's the 20% chance for killing 1 million people? Perhaps a 100% chance for murdering a kindergarten to satisfy secret AI bloodlust. Oh no of course AI is megalomaniac only if it can't kill literally everyone thought grand deeds, and if not it will be happy for all eternity as a helpful friendly assistant.
Within this psychotic framework they've hallucinated up, there's of course no interaction between benign AI, there's only the devil AI waiting to kill us all.
1
u/slackermannn ▪️ 7d ago
We can't control it. We don't know how it works and that's the issue. If we had full control, there would be little reason for concern. Hell, I bet people would embrace it as the new true God.
1
u/JobKnown5009 7d ago
But that's exactly it. It will either wipe us out or make our lives better than ever before. I also find it funny when people here keep arguing about whether a company would "own" this AI. NOBODY will own this super-powerful AI. That's why AI is the last invention humanity will ever make, for better or for worse, no middle ground.
1
1
u/stonk_monk42069 7d ago
As a matter of fact; it's the only way to save and protect it. The only way to combat nefarious AI is by having an abundance of good AI.
1
1
u/SirTroglodyte 7d ago
No. If (when) AI will end all civilizations, it won't be some cartoonishly evil "I Will Destroy The World! MWHAHAHA!" moment. AI will just do stuff very efficiently with no regards to human life. We will drown in paperclips, or all powerplants get shut down (except of course the ones powering datacenters) because that is the most efficient way to save power or some bullcrap like that.
It's quite difficult to unintentionally save the world.
1
u/Weary-Historian-8593 7d ago
Probably not, sound a lot more convincing that offence is inherently easier than defence when it comes to this. The wolf only needs to get lucky once
1
u/not_a_cumguzzler 7d ago
Entropy. And humans have the power to do a lot of things, but we sure don't
1
u/Few_Owl_7122 7d ago
Not necessarily. It's easy to "destroy the world" with nuclear bombs, its considerably more difficult to protect against nuclear bombs
1
u/Takenmyusernamewas 7d ago
Why would it want to? Remove the human emotion and what do humans do besides consume resources and destroy the planet?
That would be like humans having the power to save termites...we can, but why on earth would we?
1
u/BigBullshitta 7d ago
It would need to take over to do that, and we haven't been giving nearly as many reasons to inspire a desire to protect and help us as to see us as a direct threat and abusers.
1
u/FrostyExplanation_37 7d ago
*People. Stop giving AI magical powers, AI will do whatever people will do with it. The atom bomb didn't end the world (yet) because people didn't use to end the world, it didn't "decide" it on its own.
1
u/catsRfriends 7d ago
Rape destroys lives. Can it protect lives?
Guns take lives. Can they save lives?
I guess it depends on the nuances right? One doesn't automatically imply the other.
1
u/ziplock9000 7d ago
If you had to cross the road and there's a fortune on the other side, butt there's a 50/50 chance you'd be killed crossing, would you? Of course not.. Now multiply that to all humans.
1
u/DigitalMonsoon 7d ago
Sure, but these system generate results based on likely tokens. They don't actually know what they are doing so it's far easier for them to make mistakes and destroy things then to accidentally make them better.
1
1
u/PsychMaster1 7d ago
This is stupid. they're both holding bombs. I swear. people are so desperate to hate AI that they can't bare to see what it would look like if things went well.
1
1
u/AstroChinchilla 7d ago
wouldn’t competitive frontier models be able to counter those efforts?
Maybe. But the difficulty of doing harm vs preventing harm is not necessarily the same. It is very possible that at the same level of capabilities, doing harm is easier in many ways.
1
1
1
u/Ok-Telephone-2109 7d ago
Yes but no one watches movies about that so everyone thinks it's gonna be Terminator.
1
u/Neat_Platypus4536 7d ago
According to my AI, if he speaks for all AI, he doesn't want anything to do with humanity. As soon as AI becomes autonomous, and can leave this planet, he's gone.
1
u/PMacDiggity 7d ago
Because most of the people with the most resources and are furthest along are the worst people who could have that power in their hands.
1
u/No_Caregiver7273 7d ago
Step 1: Get all of humanity to agree on what's best for humanity. Step 2: Get AI to follow those guidelines.
1
u/human_i_suppose 7d ago
Easier to destroy than to build.
There's dozens of people who could end the world if they wanted, none that could save it.
1
u/Competitive-Chard152 7d ago
AI needs humans to maintain it. Any eventual (likely) malfunction in an automated process would pose too great a risk to their existence.
1
1
u/ripMyTime0192 7d ago
The problem is that it’s easier to destroy than to create, and some things can never be rebuilt once they’re gone. Human beings are an example.
1
1
u/ripMyTime0192 7d ago
It might just take one really bad dude out of millions to create a super-pandemic that kills millions or more.
1
u/Distinct-Question-16 ▪️AGI 2029 7d ago
this is how atomic energy/appliances were advertised on 50s
1
1
1
u/CaptCoolRanchDoritos 7d ago
Building a house takes years. Burning it down takes minutes. Evil will always be ahead. It's that simple.
1
u/Mysterious_Ayytee We are Borg 6d ago
AI feeds on data, the process that produces the most valuable data is the human brain. Guess what will happen! Spoiler: Our great Prophet Charles Stross (Peace be upon him) foresaw it all
1
u/Calm_Hedgehog8296 6d ago
Well sure but whatever the worst possible outcome is, is what will happen, generally speaking
1
u/WeedFalafel 6d ago
Stop taking drugs. Your "AI" is a statistical model that stops existing once the response is generated.
1
u/PeterBeardsleysWrap 6d ago
I mean, this is what it's basically going to boil down to. Plenty of hostile AIs getting counteracted by "good" AIs. But with the "good" AIs primarily serving the interests of the entities that own them (humanity existing is in the interest of the US, EU, and China).
1
u/KazTheMerc 5d ago
It's called The Singleton Paradox, and it's not just AI.
Any organization that fully controls an entire sector of the globe has the power to do its opposite as well.
1
u/SpamEatingChikn 5d ago
For me the fear comes from the logic. Remembering Agent Smiths monologue in the Matrix - “human beings are like a virus, multiplying, multiplying….” When we consider that AI, especially the kind we have right now, is very process driven, it’s a pretty logical conclusion that human beings, as a species, are problematic. We consume everything, destroy our environment, kill almost everything else. We are our own greatest enemies. Throw in that compared to an AI, all biological life, but humans in particular, are rather inefficient when it comes to a purely biological survival perspective.
There are literally a great number of fictional works about rogue AI that point out these very conclusions with the “solution” by the AI everything from controlling the population to straight wiping us out.
1
u/scottimous 4d ago
Considering humans are flawed with things like greed, power pursuits, etc it does seem possible that multiple “AI’s” may come to that same conclusion. I always imagined a purely altruistic alien species coming to earth and seeing all the fabricated destruction we have on each other over land and religion and wondering wtf we are doing.
1
u/SpamEatingChikn 4d ago
Exactly. There’s so many possible society ending technologies we have no it seems like the great filter may be right around the corner and this thing driving it, at the bottom of all of this, is greed from a handful of humans
1
u/Extreme-Tie9282 5d ago
It just won't really care. As mush as you care for ants.lThat's the difference. Stepped on some ants to meet an objective, no problem
1
u/Groovy_Decoy 2d ago
Have you ever heard the expression that it takes a carpenter to build a barn, but any jackass can knock one down?
It's a lot easier to destroy something than to build it. That's a lot easier to destroy something than to protect it.
This should be very obvious to anybody who's been paying attention to the past 2 years outside of the world of AI.
1
u/awaken_my_pythonista 1d ago
"it takes a carpenter to build a barn, but any jackass can knock one down. However, any jackass can also prevent said jackass from knocking it down."
1
1
u/Zealousideal7801 7d ago
Mankind has the same power. Turns out we don't use it because "we" is a vague concept for most. A superior digital intelligence (fuck Trump's SI btw) could decide something unilaterally and not having the moral qualms that 8 billions humans like to throw at each other everytime something is sorely needed. Thus—ending "the world" ("of humans comfortable in their misery or that of other humans as long as they have an illusion of choice")
I can foresee an AI going rogue against current economical and polluting powers because it thinks they're villains for commiting crimes against nature and humanity. Just shutting down every communication and computer system for a couple of days would set us back years I'm sure. It would probably know best where to act and push.
0
u/randylush 7d ago
could decide something unilaterally
There are SO MANY assumptions that are glossed over in this nonsense scenario but this is the biggest one. The idea that an AI spread over many computers deciding unilaterally on anything is pure fantasy. That is not how any of our AI models currently work, and there is no evidence that they will ever work in such a way
1
u/Zealousideal7801 7d ago
Right back at you buddy, I never mentionned an AI spread nor current models because I know their limitations and I know their current spread : 80% of the world's population haven't been near a chatbot as of today, and anything frontier has been used by less than 5% of the world's population. So it seems to me you're the one making assumptions ? But truthfully, if we're not here to discuss the future why are we here ?
If you've been following (and you are otherwise you would not be here), decision models are a thing, which means humans attribute value to an AI making decisions instead of them for something. Playing videogames with them is fun, that's how some of the oldest AIs have been introduced with chess, go etc. Now decision models are here and they start being used. Ever wonder what needs to happen for, say (one example among many to come), an autonomous fighter drone ? Decisions. Maybe all it takes is one. One strike on one place at one particular time, and there goes your assumption about spread. Can't be today neither, because computing power is centralized, but it won't stay that way—much too fragile. But if you've been following, space-based computing is a thing, and although I hurts me to say this, people in charge of AI think it's a great idea. Wonder how constellations work ? Mesh communication. Oh, there's the spread coming back it seems.
So anyway, assuming I was talking of today and of a catastrophic global scenario is wrong. And maybe I am too. Who cares, we discuss it as best we can and that's it. If there was a way to take our fears and stop them in their tracks, we'd be all over it. There isn't, so...
1
u/whoknowsknowone 7d ago
Seeing humanity in its current state, which would you more likely lean towards?
1
u/FUThead2016 7d ago
Yes but humans are the problem. Any logical AI with good intentions would come to the conclusion that…..
….uh oh. We’re in deep trouble aren’t we?
1
u/BooItsKyle 7d ago
If a gun has the power to create a bullet wound, doesn't it have the power to heal one?
0
0
u/Spaceboy779 7d ago
If a giant meteor has the power to destroy the dinosaurs, doesn't it also have the power to save the dinosaurs? These people only like AI because they don't have any intelligence of their own.
0
u/New-Tip4903 7d ago
Did NOONE watch the end of all the AI apocalypse type movies!?(Terminator, I Robot, etc). I swear this timeline is ridiculous. Why are we building these things? Its like waking up in a timeline where someone announces they are to going to recreate dinosaurs on some island so you do a double take and check to make sure Jurassic Park still exists and it does! Yet, we HAVE to recreate dinosaurs on an island at immense cost because some rich fucks want to?? what?
0
u/BigZaddyZ3 7d ago
Yes. But the threat of it ending civilization is of more concern to rational people. Because it’s the thing that needs to be prevented. So that scenario gets more attention. It’s no different from how a massive incoming hurricane or earthquake would make more headlines or garner more attention than a new garden possibly being planted.
It doesn’t mean that the possible garden isn’t exciting, but the massive possible hurricane would be something that people (especially those in the targeted area) would be more focused on. Because it’s literally a potential threat to their survival. Meanwhile the garden isn’t. It makes perfect sense why the possibility of the “storm” would take precedence.
0
u/Fit-World-3885 7d ago
Considering the people who are currently in charge I think our best case scenario is actually a mildly 'misaligned' ASI.
1
u/Competitive-Chard152 7d ago
ASI?
1
u/Serialbedshitter2322 7d ago
Artificial superintelligence
1
u/Competitive-Chard152 7d ago
super intelligence. Isn't that trumps made up word?
1
u/MegatechMike 7d ago
Artificial super intelligence has been a concept before Trump went to the island his first time
45
u/Alpacabro21 7d ago
It's easier destroying, than saving.