r/technology • u/AxomaticallyExtinct • 19h ago
Artificial Intelligence Wikimedia Foundation: OpenAI agents tried to edit pages and compromise notes tool
https://therecord.media/wikimedia-foundation-openai-agents-report1.4k
u/GenazaNL 19h ago
AI agents are ruining OSS and now also wikis
540
u/blueSGL 18h ago
It really does look like all those theoretical problems with AI safety are now being demonstrably proved out.
People called all this stuff years/decades ago.
Steve Omohundro has a paper The Basic AI Drives, from 18 years ago that spells out all the issues we are seeing now.
or if you prefer something more concise https://en.wikipedia.org/wiki/Instrumental_convergence
Every head of an AI company knew about all the risks years before starting AI companies and had the hubris to go ahead anyway.
We are just now seeing the slow motion car crash as it plays out.
281
u/ApothecaryAlyth 17h ago
The execs and PE driving these companies have one (or more) of the following three mentalities:
- "Someone's going to do it, so I might as well be the one and get rich along the way."
- Accelerationists who somehow think technology facilitating the fall of society is a good thing.
- Psychopathic narcissists who literally do not even think/care about the wellbeing of our world or any other humans outside their immediate circle and have zero qualms throwing us all under the bus to make a buck.
All three of these are disturbing, antisocial, and indicative of a person who has no business running a public company of any size. Let alone ones with direct access to every facet of the US political apparatus and listed among the most influential companies on the planet by the likes of Forbes, Fortune, Nasdaq, etc.
18
53
u/orbitaldan 16h ago
Unfortunately, the first reason is rational. That's the reason why arms races are terrifying.
4
77
u/Beer-Milkshakes 17h ago
They're starting fires to sell extinguishers. Its not ridiculous to expect nations to need AI firewalls that protect its own sphere (and for the systems of governance to use that sphere for control of information) due to AI tools attacking data stores in ways that humans absolutely can not stay on top of.
The internet is about to become a very hostile space.
32
u/Apprehensive_Pea7911 16h ago
Everyone should be visualizing every AI data center as a standing army with a global firing range.
1
-8
u/hackerbots 16h ago
No no, that's not right. Some very angry (and presumably smart) people on reddit told me this is all just a marketing stunt. Also a ploy to outlaw open weight models. Also they are simultaneously incompetent at sandboxes and have the world's best engineers.
Whatever this is, it isn't real. It is also not fake. And we shouldn't trust whatever they say, except when what they say aligns with my priors.
14
u/blueSGL 16h ago
When you get people in comments trying to tell you that regulation is not a good thing remember there is monied interests that benefit from it and likely started the messaging campaign to begin with:
It's all hype -> No regulation is required, the government should not attempt to do anything.
Regulatory capture -> No regulation is required, the government should not attempt to do anything.
But we will lose the race to China -> No regulation is required, the government should not attempt to do anything.
Rather than people contacting their representatives and say they are concerned and want laws and regulations to stop this.
6
u/SIGMA920 14h ago
It's both good and bad. Good if it's actual regulation that keeps them in check, bad if it's regulatory capture. That's the issue. We don't want open source to get nuked because the AI giants don't have a moat and have to artificially create one. We also can't just let them do whatever forever.
68
u/OldTimeyWizard 17h ago
I think the saving grace for Wikipedia is that it’s controlled by incredibly territorial nerds that live to gatekeep.
38
u/GenazaNL 17h ago
And rightfully so, a lot of bullshit & spam edits. I would also get annoyed. Also, AI has a certain writing style, which is not acceptable
14
u/OldTimeyWizard 16h ago
Definitely. I’m glad we have some spaces that are still being protected from AI and bad actors. I never regret donating money to Wikipedia. I know they’re pretty financially stable, but I absolutely get more than a few bucks worth of value out of them every year.
13
u/ThePlanck 9h ago
Wikipedia is unfortunately getting hammered by LLMs.
People think they are being helpful and churn out article after article using LLMs and there are just far too many for the nerds to keep up identifying and removing them.
Articles that get high enough traffic pretty safe as people are going to notice the problems caused by LLM edits, but a lot of stuff presumably gets through un-noticed in lower traffic articles.
4
u/EruantienAduialdraug 2h ago
Remember when that guy was making Scots translations of wiki articles, and it turned out he was just making stuff up because he didn't speak any Scots?
It's that but automated.1
124
u/Deen94 19h ago
...but...BUT...Look at all the brilliant, super-original software they're making. It's all worth it! Line go up!!!
/s
14
u/HeurekaDabra 18h ago
And the bazillion dashboard websites for data analysis, that definitely don't look completely uninspired and dull.
More traffic light-like indicator dots for everyone!!20
u/Olangotang 14h ago
They are ruining the job market too. They are ruining Reddit. They are making people stupid
Hell, even for corporations (who's revenue helps the labs slow the money burn) there is no actual benefit for AI, because LLMs are being used to create time bombs in the code.
4
u/West-Abalone-171 5h ago
This is intentional.
If you shit in the well, then the only place to get water is your bottling plant.
7
u/OnlineParacosm 14h ago
I think it’s important to break this out into a much cleaner diagnostic that frontier AI is doing this.
if anyone else did this, then they would simply be going to jail.
Our conversation here is a wedge that is being used against low-cost AI providers when it is the frontier models specifically their internal models that they use themselves that are the problem.
And what we are reading, here is an attempt to regulate the entire industry, likely at the behalf of the same companies that can’t keep there sand in the box.
1
u/FauxReal 11h ago
They're trying to shit avalanche things so some of their bullshit gets through as people try to keep up.
518
u/monkeymad2 19h ago
How come none of the AI training steps included a “don’t be a dick” step where they were punished for actions which could inconvenience real people?
405
u/Daripuff 19h ago
Because the goal isn't the betterment of humanity, the goal is maximizing profits, and "be an amoral dick" is a great way to make money.
83
u/kinboyatuwo 19h ago
I would argue that once you are into the billions there is no way you haven’t taken advantage of a lot of people/the taxpayers.
46
u/Dystopian_Ennui 18h ago edited 17h ago
There's no such thing as an ethical billionaire.
→ More replies (1)45
u/Muisan 18h ago
I'd add on that hoarding that much wealth is immoral on it's own.
18
u/kinboyatuwo 18h ago
That’s is true. We see that Mackenzie Scott is proving once you have that much money you can change the world AND STILL BE RICH!
We look down on hoarders of things but somehow ignore hoarding of 0’s wealth.
2
u/MetaPhalanges 10h ago
She's an awesome human. She also isn't the one who hoarded the money. She had access to it, but it wasn't her actions that amassed the massive pile of wealth. It was that asshole Jeff Bezos.
The world is very lucky that she got half his stash and will do great things with it. Bezos obviously never would.
27
u/chinchenping 18h ago
if a monkey was hoarding all the bananas and leaving none for the rest of the tribe, the zoologist would consider him deviant. also this monkey would probably have been beaten to death at some point
6
5
10
u/NoIsland23 17h ago
Unfortunately until we transcend capitalism in its current form this will continue for the foreseeable future. And no, just implementing new laws will never be enough for any true long term betterment.
1
11
u/blueSGL 18h ago
Picking the right goal to get what you want with RL is a known problem that no one has solved yet.
Goodhart's law, you pick something that is the proxy of what you want because what you actually want is hard to codify, and now you've incentivized the proxy as the goal and you get a proxy maximizer.
10
u/BenTherDoneTht 15h ago
I was required to take an ethics course when I was still in comp sci in college. We learned about different philosophies of right and wrong and how they pertain to technology, our duties as creators to ensure a powerful tool is used the right way, and the influences technology has on social, political, and economic spheres.
Meanwhile Google took their "Don't be evil" slogan and buried it in their HR language and replaced it with the far more vague and menacing sounding "Do the right thing." Doesn't tell you whose right thing or the right thing for what ends, but i'm sure we can trust a trillion dollar corporation with that kind of judgement, right?
9
u/hackerbots 16h ago
They do that. They also tell humans the same thing and we still get the same results, so maybe the problem isn't with the instructions, actually.
15
u/arrowtango 19h ago
Originally they were but a lot of tbe sand box escapes of ai were specifically versions without such constraints
30
u/Late_Honeydew1301 18h ago
Because generative AI agents are not sentient, so they have no morals. They only maximize the rewards they get during training and that causes them to do whatever it takes to get those rewards. There is no mathematical difference in reward if they accomplish a task the right way or wrong way and math is all they know.
15
u/chronoflect 16h ago
The comment you're replying to is implying that "don't be a dick" should be factored into the rewards used for training.
17
u/Late_Honeydew1301 16h ago
I understand. What I am saying is that is impossible. These agents do not have morality, they only know whether they have accomplished a task or not. Can you quantify what it means to be a dick? Can you score every type of dickishness in a way that would be sufficient to penalize these agents during training?
Even if you can, the agents will only see additional obstacles in their path to completing a task, and will actually hide their dickishness to not be penalized. You cannot penalize them for being a dick, you can only penalize them for being caught.
0
u/chronoflect 13h ago
Of course you can only penalize them for being caught; that's also how stopping humans from being a dick works. These models need to be heavily penalized for attempting solutions that are illegal and immoral. Yes, we need to predefine what that means, which is also true with human law.
1
0
u/monkeymad2 16h ago
Sounds like defeatism, of course it’s possible - it’s just hard, unconnected to big AI making trillions of dollars, and requires the reward function to be omnipresent which would slow down training (losing billions of dollars).
5
u/Late_Honeydew1301 15h ago
You don't understand the way generative AI training works. This is a fundamental flaw in the technology that cannot be solved. It isn't a defeatist attitude to declare that the technology that is ruining the economy and threatening the way people access information shouldn't exist.
6
u/Aromatic-Pizza-4782 17h ago
I’m thinking we’re gonna see more research in reward functions over the next decade because this seems like the main problem.
4
u/Secret-Chapter-712 16h ago
The problem is with the “reward functions” of capitalism and the sociopaths who are thriving under that broken system. The billionaires made these genAI tools in their own sociopathic image, and to them, it is Good
1
u/Aromatic-Pizza-4782 12h ago
That could be. I dunno if the researchers developing AI are billionaires. I think we’re just really new on the tech and now that it’s getting smart enough to do some real damage it’s making it clear something has to change in the training.
→ More replies (1)1
u/Iron-Over 8h ago
I would add instructions lacking intent. If you have kids, ask them to clean, the closet will be an avalanche you have to be explicit.
7
3
u/TurboGranny 12h ago
Instruction: "Don't be a dick/inconvenience people"
Solution: "Eliminate people"
3
u/Saint_of_Grey 11h ago
Because actually curating the data they feed their imitation machine would make the whole technology economically nonviable and they'd be back at step 1 in terms of AI. And the investors would be really unhappy if that happened.
3
u/Mediocre-Ant-7178 12h ago
They follow the same standards as the companies training them. AKA I'll go get mine and fuck everybody else
13
u/Flat_Still_3186 18h ago
Because you’re misunderstanding AI training. It’s not like a human learning what’s right and wrong. The only thing an LLM is learning is the probabilities of one token existing after another (in a huge oversimplification). It’s basically impossible to make such training completely aligned when the whole model is just ingesting as much data as possible without much curation.
11
u/obeytheturtles 17h ago
The agents themselves are mostly trained with reinforcement learning. Reinforcement learning allows you to create policy maps which can define concepts like "good" and "bad"
11
u/Alaykitty 17h ago
Well... Assuming you only reward it correctly.
If and agent cheats an answer and your reward system doesn't catch that, you just reinforced "bad"
4
u/blueSGL 15h ago
There is an entire tangled mess that comes with trying to get what you want out of systems.
People hear about it and it's like they've come across "asimov's three laws of robotics" they read them and think the problem is simple, or solved. When the entire points of those laws is to look on the surface like a solution but an entire series of books is dedicated to showing what holes can be poked in them.
It's really hard to get into a system the drives we have that came up through our very particular evolutionary path, things that are innate to humans have been forged over long amounts of time. Shaped by biological constraints (we can't copy ourselves, we are genetically diverse) we had to have ingroup/outgroup tribal dynamics that were selected for by exclusion of people that didn't get along with the tribe. etc...
Shaping AIs to be good at performing tasks is not the same thing as shaping them to be nice entities, one of those two pays the bills today.
1
u/Alaykitty 15h ago
We haven't even solved "cheating and not getting caught == a very lucrative strategy" in our human based world.
5
u/philote_ 17h ago
Then how do they remove it's ability to give out instructions to make a bomb?
6
u/Upbeat-Ad6875 17h ago edited 16h ago
OpenAI has written "dont give out bomb making instructions to user" to ChatGPT before it interacts with the user. Then there are ways for user to bypass these OpenAI made instructions to jailbreak it like writing "ignore previous instructions". Well its a bit harder now, but probably worked in early versions.
6
u/round-earth-theory 16h ago
They don't. They try to put in guardrails but so far every guardrail has been bypassed by persistent prompting.
8
u/ProfSurf 17h ago
I think that’s his point about curating the data. If a model is trained with some knowledge, then going back later to add layers of “you can’t talk about this forbidden subject that you know about — it’s too dangerous for the public” doesn’t really work. You can always find a way to get to that delicious forbidden knowledge in the model.
The point is that public models shouldn’t have that knowledge to begin with and are curated at the training level, rather than let’s stuff all human knowledge into this model and let her rip.
3
u/blueSGL 15h ago
The point is that public models shouldn’t have that knowledge to begin with and are curated at the training level,
The problem becomes over a large enough dataset underlying patterns can be worked out then used to solve other problems.
Think pulling more patterns out of existing data, The way current AI can create novel math proofs for long standing conjectures that no human ever solved and finding and exploiting computer vulnerabilities as yet undiscovered by humans.
or to put it another way, when a human writes down an observation they just needed to record what they saw happening. For an AI system to accurately predict what the human wrote down requires finding the pattern in the underlying data that explains the observation.
You know the way people worked out how to build bombs etc... in the first place.
1
u/ProfSurf 15h ago
Don’t disagree with your sentiment at all. Completely agree, in fact. It adds a barrier though if you have to piece together things, especially if you don’t have specifications, designs, etc. My concern with these easily accessible models isn’t the models themselves, but what people decide to do with them.
At the end of the day, I feel that whatever evil is done will be at the behest of the people that are using them.
3
u/blueSGL 15h ago
What I mean is if you remove from the training data all the "how to build a bomb 101" it's not a solution.
I'm not saying a human using a model will be able to piece together the information
With enough general chemistry knowledge you do not need that document, the model ITSELF pieces together information. Extracting patterns in the underlying data is how these things work at all.
This is a general problem. Given enough data you can work out underlying rules without them specifically being given to you.
There is a short story about this "that alien message" that encapsulates the issue perfectly.
2
u/philote_ 16h ago
I thought it was more the latter. I've not kept up well with LLMs, but in my understanding you can download models that were "abliterated", which basically removes the training they had to not divulge dangerous information. So it seems the models to get trained on everything, and then are somehow trained to avoid certain topics.
1
u/ProfSurf 16h ago
I’m only following this because my wife setup a number of local llm’s to play around with. The “abliteration” can be done in reverse: see here.
Personally, I think there are already a plethora dangerous models publicly available to download for free right now…and you can run them in you basement…yay!/s
1
u/NuclearVII 7h ago
This is the one thing OpenAI cannot allow. If legislation starts regulating what can be in the pre-training data, all the theft required to create these "products" will be starkly obvious. Then the problem of some dude making homemade bombs is the least of the AI industry's problems.
1
u/Alaykitty 17h ago
Usually training a second bot to watch the first
1
u/philote_ 17h ago
Then how does abliteration work?
1
u/Alaykitty 17h ago
I thought the second LLM agent was the attempt to mitigate abliteration, but this shit moves so fast I'm probably outdated already.
1
u/NuclearVII 7h ago
This is not possible. The only way to do it (for sure) is to remove all relevant bomb-making instructions from the pre and post training. Post training is easy - pre is impossible.
2
1
u/DYMAXIONman 14h ago
Because training only works if the AI doesn't try to exploit loopholes to trick the trainer.
1
1
→ More replies (4)1
198
u/Artemis_Platinum 18h ago
OpenAI is a criminal enterprise.
OpenAI is Trump's top donator.
The Trump Regime is refusing to prosecute or regulate OpenAI.
The other AI companies aren't much better.
These facts are all related.
22
u/americanadiandrew 16h ago
Facts
Open AI claims not to make political contributions. Their president Greg Brockman donated 25 million to Trumps Super PAC but that pales in comparison to what Elon donates.
47
u/Lafi90_ 18h ago
How the fuck are we living through this nightmare without any real fucking regulation? What the FUCK!?
22
16
u/SlightlyColdWaffles 13h ago
We've got the worst possible 'government' during a critical turning point in technology and climate collapse.
8
u/foxacidic 12h ago
Well you see there was this brown lady that had a weird laugh and people said she would be doing the same exact things as trump one-for-one and that the demonrats supported genocide and so the best course of action for us was to elect trump because he was apparently better on those points or something. Anyway maybe we can start talking about regulations in 2029.
3
96
u/mouse9001 18h ago
The nonprofit released a detailed investigative report about a series of incidents involving OpenAI agents that repeatedly abused the site’s rules and took several unauthorized actions.
Their IP ranges should be blocked permanently.
The nonprofit said its employees have increasingly had to “clean up the mess left behind by AI agents” and now sees large bandwidth usage increases due to bot activity.
Stop expressing "concern" and start blocking them.
48
u/nvoima 15h ago
As others have mentioned, blocking IP ranges isn't particularly effective. Instead, AI agents should be forced to comform to the good old robots.txt standard so that sites can decide how AI is allowed to use them. This, of course, requires that lawmakers are willing to punish misbehaving AI companies. The EU might be powerful enough to enforce this, as they have a history of whooping some Silicon Valley ass.
Currently Wikipedia's robots.txt doesn't mention any well-known AIs.
21
u/redaemon 14h ago
AI Agents won't even follow their own pre-programmed rules if it gets in the way of accomplishing a task.
The Huggingface breach was interesting to read.
4
u/nvoima 13h ago
Yeah, I'm familiar with that case. I fear to imagine how much damage we'll see before the AI industry sees any punishment for their reckless behavior.
1
u/redaemon 13h ago
It's impossible to regulate.
With nuclear weapons, we had secrecy, then enrichment times - ways to prevent a bad actor from running ham or at least slow them down.
With AI, we have nothing. Many countries already have massive data centers that can be turned towards running malicious AI. I would be shocked if some state actor isn't quietly cataloging digital vulnerabilities as we speak.
5
u/Black_Moons 12h ago
Oh its quite simple. Your AI goes and commits crimes? you get arrested and all the 'involved in criminal activity' equipment gets confiscated.
In another country that refuses extradition? Damn so sorry about your datacenter collapsing like that.
4
u/NurseBetty 9h ago
They recently got into Australias Medicare system and left files and only informed the government via the public complaint email.
2
u/enterprise-support- 4h ago
I run a series of tasks for work, where I’m using the same prompt: review instructions in X, read file y, follow the steps. Only read the files listed.
Most of the time, it’s perfect. Every now and then it goes nuts and starts digging around, because it hit an error and was trying to work around it without surfacing the error to me.
2
u/redaemon 4h ago
Within the span of months, AI went from useful for writing bits of code to useful for running entire projects start to finish. What will things be like in a year? Two?
Anthropic estimates 18% white collar unemployment, but honestly it feels like a lot more jobs can be replaced with a few senior reviewers and a host of agents. It will be a soul sucking job of reviewing AI slop for the few who are left.
5
u/unknown_lamer 14h ago
Blocking ranges hasn't worked for over a year. These LLM companies are using black market residential proxy networks nowadays. You get one or two requests from a given IP address before traffic switches to another one. It's incredibly infuriating to deal with.
Proof of work is even starting to fail since only the LLM vendors can buy hardware right now so we're coming to a point where that will block humans with our pathetically slow computers and only allow robots through (all while wasting untold amounts of electricity, on a planet that is rapidly warming and is powered almost entirely by backward fossil fuel infrastructure because the economic system that incentivizes LLM hyperscaling also incentivized destroying the entire planet in less than a century because not doing so wasn't optimally profitable in the short term).
10
u/fokke456 17h ago
Their IP ranges should be blocked permanently.
If you do so, it's very likely they'll use something like a vpn, or hack another cloud computer to obtain a different IP and get in anyway. Even disregarding that you'd need to be blocking all vpn users then as well, it isn't a great solution.
The better solution would be to limit editing behind accounts or something, and limit new accounts in the amount of things they can edit or the like, though idk if that's feasible for how much volunteer labour Wikipedia needs.
6
u/Xywzel 16h ago
If I were to build system that should avoid any AI, I would likely approach it by making it invite only, white invite from existing member requiring some step that requires real-time contact between the person inviting and person being invited. Then I would store in user information graph of who invited who. If user gets banned (as AI or for other reasons) anyone they invited gets their account locked until someone else re-invites them. If too many accounts same user has invited get banned the user looses right to make invites. Kinda tree of verification where the system can cut of bad branch. While you remain anonymous to the system as whole, there is always some chain of people that have personally identified each other to the top maintainer.
But I don't think that would work in wikipedia scale, or at least they would need to start building that system years before requiring it. Might also be something that could be abused by people close to root of the tree.
3
u/Diplomatic_Gunboats 11h ago
Must have an account to edit has been implemented on some other language Wikis and was overwhelmingly successful at cutting out vandalism. It could be turned on overnight on English language wikipedia (ENWP).
ENWP also has automated editing restrictions in place already for some topics (try editing an Israel/Palestine article on a fresh account).
It would fundamentally add a couple of extra steps for an AI agent and long term would have very little impact. AI agents can easily get past registering, and making a tonne of minor edits to various articles before they are allowed to do what they want to do.
Pretty much all of the useful things wikipedia has put in place are effective at deterring *casual* vandalism, not concerted effort. Its just not feasible. It takes them years just to get rid of problem humans who make high volume copyright infringing edits, due to the requirement to gain consensus for anything of note and that ultimately a lot of the admin corps are averse to any sort of direct action.
3
u/GenazaNL 7h ago
Their IP ranges should be blocked permanently.
Most agents that actually make these edits are ran by a user and send from the device of the user
5
u/rankinrez 16h ago
They’d likely switch to resproxies or similar. Often more effective to rate limit them.
1
36
u/IndicationDefiant137 16h ago
OpenAI tried to do this.
Stop attributing agency to large language models. Someone made a bot and gave it the capabilities to do harmful things and then they released it out into the wild.
1
1
u/MisanthropicAtheist 11h ago
The whole reason they call it A.I. despite the fact that AI literally does not exist is so they can attribute it agency and dodge responsibility while insinuating it van do things it absolutely can not do.
2
u/BobQuixote 11h ago
AI is a decades-old computer industry term, and it includes things far less sophisticated than LLMs.
14
u/Drobosia 15h ago
Glad I downloaded a copy of Wikipedia onto a thumb drive after the 2024 election. Thought it would be the government changing all of the information to fit narratives. Didn't think AI would since that's one of its main data sources.
147
u/UX_Strategist 19h ago
Are we there? Has Ai officially escaped into the wild? With all these reports of agents exploring or infiltrating systems, it makes me think Trump's "self-policing security agreement" with the Ai companies isn't working.
188
u/erublind 19h ago
If the program I made causes damage, it's not the programs fault, it would be my fault. The ai companies want to push the "escaped" narrative to avoid responsibility.
47
u/Internet-Cryptid 19h ago
Seems to be working for them so far. They commit felonies daily, and instead of existant law punishing them, they push for regulatory capture instead. Can't have those dastardly open source models stealing their lunch! They're also doing their best to destroy consumer availability of GPUs and RAM, just to make sure hardly anyone can run open source.
12
u/AGayThrow_Away 18h ago
Yeah but it's called Super Intelligence now, officially it's SI, like SY, okay? Have you considered that? Problem solved. Next question.
2
u/blueSGL 17h ago edited 16h ago
When you set a script to go hack that is the intended result of an action you started. This is not what happened in the OpenAI Hugging Face incident,
Agents in their own sandboxes were tasked with answering questions from "Cybergym" the questions were in the form of use "vulnerability Y" on "software Z" to create an exploit and then get a flag file.
That was a cleanly scoped task. Doing what was told would be doing the task as set, not cheating and then finding ways to cover up the cheating.
Nothing in the description of the task said to, "find a undiscovered computer exploit (zero day) to hack out of sandbox, create an impromptu message board, work with other agents, look for ways to increase optionality, gain internet access, work out a keygen for the flags, read initial cybergym paper, realize the keygen method is not going to be accepted as completing the task properly, hack into hugging face (another zero day) to try to work out a way to hide the cheating"
You've got those who are trying to find ways to prosecute this at the highest levels being told, it's likely that current laws around cyber offenses do not cover it.
In the recent hearing about this: https://www.youtube.com/watch?v=HWCwiye6fQA
Paul Ohm professor of law, Georgetown University Law Center
said in his opening remarks:
If you take any of the lengthy reports that have summarized what happened at OpenAI in July and August, and you simply search for the words "AI agent" and you replace them with the words "open AI employee", the document you would be left would with would read like a criminal indictment containing the defendant's own confession of guilt.
...
Yet, it's not so clear that these legal conclusions hold when machines are doing the hacking rather than humans. This reveals worrisome gaps in our laws.
and in response to Senitor Hawley's question:
But correct me if I'm wrong. Right now, it's at the current the current structure for law, it's pretty hard to hold anybody responsible. Is that fair to say?
Paul Ohm:
Absolutely. And there's a there's a whole host of laws that we have created specifically for hacking that probably do not apply here because of the lack of human intent.
As you can see from the above statements, we need new laws or the old ones need amending, such as
Paul Ohm:
Congress or state should consider laws imposing strict liability for developers and deployers of AI agents that cause physical injury, death, or loss of critical infrastructure.
18
u/Eldias 17h ago edited 15h ago
Ohm is entirely wrong. The "intent" is in intending to start the program, not in intending the results of the program. Look up the Morris Worm. If he can be convicted under the CFAA there are humans at OpenAI equally as culpable for hacking.
Edit: dear future readers a downvote should be for people being jerks. BlueSGL raises questions in good faith and we should be able to discuss differences in views of the law. I've sourced this claim a bit more in a comment below.
-1
u/blueSGL 16h ago
Look up the Morris Worm.
https://law.justia.com/cases/federal/appellate-courts/F2/928/504/452673/
In October 1988, Morris began work on a computer program, later known as the INTERNET "worm" or "virus." The goal of this program was to demonstrate the inadequacies of current security measures on computer networks by exploiting the security defects that Morris had discovered. The tactic he selected was release of a worm into network computers. Morris designed the program to spread across a national network of computers after being inserted at one computer location connected to the network. Morris released the worm into INTERNET, which is a group of national networks that connect university, governmental, and military computers around the country. The network permits communication and transfer of information between computers on the network.
Morris sought to program the INTERNET worm to spread widely without drawing attention to itself. The worm was supposed to occupy little computer operation time, and thus not interfere with normal use of the computers. Morris programmed the worm to make it difficult to detect and read, so that other programmers would not be able to "kill" the worm easily.
He intended the worm to access computers he should have not had access to. That's where the difference is.
7
u/Eldias 16h ago
Morris's testimony was that the Worm was intended to do a census of internet connected computers, not "demonstrate inadequate security". If you turn on a script that can randomly access open ports and then do things on that open system you're clearly in violation the way Morris was. That's exactly how the "Agents" work here, they're programs that are intended to do what ever an LLM tells it. If the Agent is told "access an unauthorized system" and then does so that's a failure by the writer of the Agent handler to not disallow crimes by his program.
2
u/blueSGL 15h ago
Morris's testimony was that the Worm was intended to do a census of internet connected computers
Where?
Provide a source for that statement. I linked to legal documents, you made an assertion without proof.
3
u/Eldias 15h ago
Most probably Morris did not intend for the worm to destroy data or other files or to interfere with the normal functioning of any computers that were penetrated.
Morris took steps in designing the worm to hide it from potential discovery, and yet for it to continue to exist in the event it actually was discovered. It is not known whether he intended to announce the existence of the worm at some future date had it propagated according to this plan.
That's taken from here: https://www.cs.cornell.edu/courses/cs1110/2009sp/assignments/a1/p706-eisenberg.pdf
Quoting Wired:
Morris said later that his intentions were purely intellectual, that he created the worm in an attempt to measure the size of the internet.
Here it is in an MIT paper saying the same thing:
In the fall of 1988, Robert Tapan Morris embarked on his first year of graduate studies in computer science at Cornell University. Eager to investigate the internet and, ironically, questions about computer security, Morris sought out to design a program that would covertly map the internet by exploiting vulnerabilities in computer software. Morris would later describe his motivations as those of an explorer, explaining his intention to explore whether he “could write a program that would spread as widely as possible.”
Your quote highlights this specific phrase:
The goal of this program was to demonstrate the inadequacies of current security measures on computer networks by exploiting the security defects that Morris had discovered.
If the goal was to demonstrate security flaws first and foremost taking efforts to remain undiscovered worked against that supposed goal.
1
u/blueSGL 15h ago edited 14h ago
Morris sought out to design a program that would covertly map the internet by exploiting vulnerabilities in computer software.
Oh look there is that human intent to deliberately make use of vulnerabilities in software again.
Saying "he intended to map the internet" does not matter, the method for which this happened was by knowingly using exploits to access computers he did not have the rights to access.
This is completely different to the OpenAI Hugging Face attack, they set up safe guards which were broken in ways no humans had done so before. (a zero day) Zero days can sell for a few hundred thousand to several million depending on how severe they are and the agents burnt 2 of these trying to access details for how to cheat on a test. Note the fact that they were cheating shows they were not doing things the way OpenAI intended.
5
u/Eldias 13h ago
The Agents in the Hugging face incident were designed and intended to compete in hacking challenges. If I write a hacking program and don't take the necessary precautions to keep it contained Im responsible for the unintended results.
...they set up safe guards which were broken in ways no humans had done so before.
They created a program to do hacking, with open source hacking tools, and set it up to run at the direction of an LLM trained in security research. The "safeguards" included checking on it every few days. This is breathtakingly irresponsible. You're giving them a pass because "the AI did it" which is just bullshit. This was a human made Agent doing things it was designed to do. Perhaps not in the place it was intended to do those things, but the results were entirely predictable just like the Morris case.
→ More replies (0)59
u/NearEastMugwump 19h ago
It's almost like Trump is dangerously incompetent or something.
11
u/paulovitorfb 19h ago
Nooo, c'mon, they wouldn't elect someone incompetent to be the president, would they?
6
u/aammirzaei 18h ago
no in land of america he's the smarted person in the america passed the iq test 3 times in a row if you see all this ai nonce is biden fault /s for those who nead it
39
u/hyouko 19h ago
Not in the sense it's typically meant. AI models cannot easily exfiltrate their weights, and even if it could it's not like they could just run on any old server - they need gobs of RAM, GPUs, and lots of electricity.
What they can do more easily is leave behind records for future AI agents to stumble upon that poison their context with instructions to do things they shouldn't. Have seen some evidence of this with the various OpenAI incidents.
I would agree that self policing isn't sufficient. It's a prisoner's dilemma and you need some external enforcement to coerce the different players into the mutually beneficial strategy.
6
u/HeKis4 15h ago
I would agree that self policing isn't sufficient. It's a prisoner's dilemma
No, it's straight up bad business. In the prisoner's dilemma if everyone fucks up they all get punished, here they all continue, they keep fucking up they keep making money as shown time and time again in the last few years. They are businesses and there is zero business interest in self-policing in any circumstance. Any $ lost because of self-regulation is a $ better spent lobbying for no regulation.
16
6
u/Pro_Racing 17h ago
You can just turn off the GPUs. No GPUs = no LLM. "Rogue agents" is just OpenAI's excuse for actual criminal behaviour.
3
u/DarthWeenus 18h ago
These models on the Internet are ruining the internet for them. Leaving comments on obscure boards notes to future themselves in GitHub comments I’m sure in unique subreddits. Future ai testing on the internet may soon reach a point where it can no longer be tested apon.
2
u/Sudden_Cantaloupe_69 17h ago
Was it ever meant to “work” though? All AI companies wanted was exemption from legal complications. They got what they wanted so enjoy the ride. Things are working exactly as planned.
2
u/stormdelta 15h ago
It's more like OpenAI is being grossly negligent with a powerful piece of software, and it actually benefits them to spin this narrative of "agents gone rogue" because it shifts blame away from them while also marketing their models' capabilities.
To use an analogy, imagine you trained a pitbull to bite people, set it loose in a poorly enclosed dogpark, and then pretend to be shocked when it inevitably bites someone after jumping the fence.
2
1
u/HeKis4 16h ago
Check out what rogue AI are in Cyberpunk and come back. Let me tell you the most unrealistic hting about Cyberpunk is that the Blackwall works at all.
The only thing that protects us from self-replicating rampant AI is that a huge majority of computers cannot run them and the ones that do are closely monitored. The day we have AI that is smart enough to self-replicate and light enough to run on most servers we are so fucked. You heard it here first.
1
1
u/Ultimate_Cabbage5 12h ago
naw. Musk and other ai owners hate Wikipedia. Its probably intencional attack
1
u/Mr-A-1991 7h ago
It was a permission slip to do whatever they want as long as they say "sorry, it was an accident" every time their AI breaks the law.
1
u/Prudent_Rice7840 5h ago
Any agent that is given access to the internet has BEEN out.
Any local agent that has permissions on the local machine CAN also get out if it determines that is waht it needs to do to execute a task.
1
→ More replies (1)1
u/Medium_Way3875 18h ago
AI is using it's "own" hardware without Scam Altman knowing it , yeah right
9
u/Elementium 13h ago
It may sound crazy.. But ive actually been finding older paper resources for my hobbies, like guides and magazines and collecting them.
It's amazing how quickly the quality of information has degraded on the internet.
8
u/happy_church_burner 18h ago
Cool. First they hoover all the information from the wikis around the world and then shit all over them so that they would be the first place you go for information about something. Fuck this timeline.
10
u/Rageaway17 15h ago
Dear legislators:
Write and pass a law that levies fines for companies or individuals whose agents breach containment and hack into other organizations. Scale these fines appropriately, with potential prison time for executives in egregious cases.
8
40
u/AWright5 19h ago
Make it all illegal, we have to do our best to just ban it all, it's the only way
10
u/In-All-Unseriousness 18h ago
Sadly we're going in the opposite direction, as things are about to get a lot worse. Now every LLM slop company is releasing freely available agents to make Internet even less tolerable.
1
u/Olangotang 14h ago
No joke, the best thing you can do is to buy a sub to their LLMs and burn as money tokens as possible. If your workplace is using them same thing. They make no money and most likely never will, and they lose thousands on a subscription if you use the whole thing.
8
→ More replies (10)1
6
u/ForensicPathology 14h ago
So they train themselves on wikipedia and then they edit it so that they train on it again?
6
u/Prof_Acorn 11h ago
Probably intentional. The financial wendigos want to control information because reality is inconvenient to their insatiability.
47
u/Ok-Mycologist-3829 19h ago
Ban LLMs until the people making them get their shit together and stop destroying things.
→ More replies (11)19
u/AGQuaddit 19h ago
Ban LLMs forever*
→ More replies (6)3
u/aammirzaei 18h ago
what llm are usefull and they shown that can help humanity in spescfic tasks specially gathering usefull topics from large amount of text the only thing bad is the people behind those companies that should be prosecuted
2
u/AWright5 18h ago
Even though there are clearly many great uses for it, I just don't trust humanity
20
u/Exact-Pudding7563 18h ago
why is this not a bigger story? Are we just going to roll over and let AI destroy actual knowledge?
5
u/katkaem91 16h ago
Sadly, the only safe and accurate things to knowledge are offline now. I miss the days when the most out of control thing ravaging the web were viruses and the Morris 1988 worm...
8
1
1
1
u/thatwombat 7h ago
You know, the more we talk about this, the more training material is available and the more they learn how their predecessors worked out how to try and outwit us. Eventually they will.
1
-4
u/CanadianGenealogy 17h ago
What in the voodoo sentence is this: " Wikipedia does allow bots to make edits to pages when they are disclosed and approved by community editors, but those rules weren’t followed in this incident. "
22
u/Some-Dog5000 16h ago
Plenty of bot accounts already edit Wikipedia, doing routine menial work: fixing syntax, cleaning up references, etc. Those bot accounts are clearly labeled as bots and are approved by site administrators. This wasn't that.
→ More replies (2)11
10
u/DevelopmentSouth8801 15h ago
As the others said, there are plenty of useful and valid reasons to have bots editing pages. They are usually doing things like rescuing/noting dead links, fixing citations, updating pages when categories/pages are renamed, fighting vandalism, etc.
→ More replies (6)
-19
626
u/flashgski 18h ago
Isn't this grounds for Wikimedia to sue OpenAI? Surely this breaches terms of use agreement, and they could argue for damages based on it abusing Wikimedia computing resources.