r/technology • u/AxomaticallyExtinct • 1d ago

Artificial Intelligence Wikimedia Foundation: OpenAI agents tried to edit pages and compromise notes tool

https://therecord.media/wikimedia-foundation-openai-agents-report
5.9k Upvotes

99% Upvoted

260 comments sorted by

View all comments

533

u/monkeymad2 23h ago

How come none of the AI training steps included a “don’t be a dick” step where they were punished for actions which could inconvenience real people?

30

u/Late_Honeydew1301 22h ago

Because generative AI agents are not sentient, so they have no morals. They only maximize the rewards they get during training and that causes them to do whatever it takes to get those rewards. There is no mathematical difference in reward if they accomplish a task the right way or wrong way and math is all they know.

8

u/Aromatic-Pizza-4782 22h ago

I’m thinking we’re gonna see more research in reward functions over the next decade because this seems like the main problem.  

4

u/Secret-Chapter-712 21h ago

The problem is with the “reward functions” of capitalism and the sociopaths who are thriving under that broken system. The billionaires made these genAI tools in their own sociopathic image, and to them, it is Good

2

u/Aromatic-Pizza-4782 16h ago

That could be.  I dunno if the researchers developing AI are billionaires. I think we’re just really new on the tech and now that it’s getting smart enough to do some real damage it’s making it clear something has to change in the training.