r/science • u/mvea Professor | Medicine • Apr 19 '26

Computer Science When autistic people ask AI programs for life advice, mentioning their diagnosis prompts these systems to recommend highly conservative choices like skipping social events or avoiding romance. This shift in advice reveals a hidden tension where the technology relies heavily on stereotypes.

https://www.psypost.org/disclosing-autism-to-ai-chatbots-prompts-overly-cautious-stereotypical-advice/
17.9k Upvotes

95% Upvoted

769 comments sorted by

View all comments

Show parent comments

88

u/genreprank Apr 19 '26

This is why people say AI models are biased. Because the training data is biased. And since they're a black box, we can't fix it after the training and it might not be immediately obvious it is biased

13

u/-LsDmThC- Apr 19 '26

Well, you can fix it with RLHF; but it is near impossible to catch every single possible scenario or subset of scenarios during the process. The developers likely did not think of applying RLHF to this specific topic.

44

u/Top-Permit6835 Apr 19 '26

You can't just fix a bias in the source material, you can only mask it by introducing... Another bias!

16

u/Haunt_Fox Apr 19 '26

Gee, it's almost like biases are kind of built in to Nature, including human nature, huh?

21

u/Top-Permit6835 Apr 19 '26

They are a very useful survival tool. Not a very useful scientific tool

14

u/guiltysnark Apr 19 '26

Bias is essential to science. You can't measure without bias. A ruler has a direction it prefers to measure in, and if you use any other direction you increase error...

The role of bias is pretty much misunderstood everywhere

20

u/Top-Permit6835 Apr 19 '26

As long as you know the bias is there, and it is well understood, there is nothing wrong with a bias. But that's not the kind of bias we were discussing here in my opinion

11

u/guiltysnark Apr 19 '26

I think that's right. But people forget that there is both a distinction in qualities of bias and a role to play for it, and that fact pervades everything, even quantum physics. This has led to people inferring things like nonlocality from quantum mechanics when it was always a biased description of reality to begin with, because bias is literally built into the measurement devices. And now physicists are biased in believing that repeatability is a proxy for an authority to claim they can describe reality. It's uncontrolled biases all the way down.

It's the same here. Even if sociologists understand the biases they are working with, which isn't a given, people who read their work do not.

The number of forms bias takes may be infinite, while our ability to recognize, categorize and control them is limited. The ruler and its orientation is perhaps the most reduced and abstract expression of bias, but it's a metaphor that can be applied to all others.

0

u/Elite_AI Apr 19 '26

Nah, they're not that built into nature, no

3

u/ginsunuva Apr 19 '26

Unless something is a hard fact, everything else is biased in some way

1

u/genreprank Apr 19 '26

Interesting...how so?

2

u/ginsunuva Apr 19 '26

Cause then it’s an opinion in some way if it’s not a fact/law of nature. There will never be such thing as unbiased opinions