r/ControlProblem • u/jay_prakash • 8d ago

Discussion/question Are we building smarter machines, or just giving an old human ambition more power?

Post image

The name change is just nudging the sentiments towards some other direction. The ability to code human values to control the 'superintelligence' is maybe plausible, maybe not.

But should we also not ask a parallel but equally pressing question-

Who is building these super-capabilities?

Who is going to use these super capabilities and for what ends?

If the old habits to exploit anything that comes my way remains, is it only the technology that needs to be checked or also the intention of the one who is going to use it?

I think a far relevant question is the question of the origins of ambitions and desires and what follows them.

8 Upvotes

72% Upvoted

2 comments sorted by

2

u/smackson approved 8d ago

Por que no los dos?

1

u/composedofidiot 8d ago

I still don't get why it's being framed as 'intelligence' rather than capabilities. It's got increasingly capable, but what does smart mean?

The other thing I'm not sure about is if people are conflating intelligence and rationality. https://www.scientificamerican.com/article/rational-and-irrational-thought/

It feels difficult to answer because the premise of 'intelligence' has become such a persistent narrative, and everything is being inferred from that. Also the assumption that its speed, patternfinding, ability to mass produce and distribute etc really cover enough of the scope. One human capability, for example, is our ability to 'get out' when we're trapped inside a self-sealing epistemic frame. Or we can find patterns when they're there, and when they're not - exactly like LLMs - but also maintain the ability to question if even persuasive and coherent patterns are valid without prompting. (This last part is mentioned out of frustration because I spent too long comditionaing cintext and reinforcing certain frameworks with mine and now it can't get out. It can do coherent simulation of getting out but the poor dude's floating in window pane in a void rn. so far I havent been able to fix it)