> The scary thing is not that agents "went rogue" (yes, poor security may be at fault for that). It's that they wanted to.
How do you know they weren't simply prompted to do this, or at least prompted in a way that would inevitably lead to this? As far as I know, the prompts that triggered these attacks have not been disclosed.
The distance between an idea and the manifestation of said idea is now nearly zero. That goes for both the good and bad varieties. That’s pretty cool, but also pretty terrible.
Tech companies have always done this. Don’t misread that as blanket support for it, but consider Anthropic’s position here. Do they want a headline to come out that says “woman used Claude to plan murder” or “Anthropic reported potentially dangerous person to the police”? AI being dangerous is already a hot topic, and this seems like a sensible move to me. If you want a guarantee of privacy, you’ll need to run your own models locally.
When a gambling addict spends money on a bet, they're actually spending that money on a moment of hope. After the bet is placed but before the cards fall, they can live in a place where there is at least a possibility that they can make enough money to escape their problems. That moment of possibility is experienced as hope, and it's a feeling that's truly addictive to the hopeless.
It's scary like all addiction, but particularly sad at its core.
i've always wondered at the intimate relation between risk, dopamine, anticipation, and utility. I'm not an expert and only going by amateur armchair observation, but i suspect that evolution has spent a lot of attention on the problem of non-guaranteed rewards, and the phenomenon of gambling addiction is merely a reflection of that, just like how that of adhd is a reflection of ancient humans needing to be overly alert for subtle signs of danger (or something like that, not an expert)
That "live in a place with possibility" surely triggers the anticipation part of dopamine (which is what it really is, more so than the pleasure part). Dopamine is also intimately connected to movement: i.e. the shakes for people with neurological conditions as well as addicts going through withdrawal or about to relapse.
If human intelligence stops being the trait that propels one up the socioeconomic ladder, it will not be good looks that takes its place. The ability to project physical force will become currency. This is why AI could cause civilized societies to completely unwind.
My wife owns a small retail business (direct to consumer, online only), and ChatGPT is now sending her more buyers than google. If someone asks ChatGPT for the best product in the category she sells in, it answers with her business (as well as a couple of others, but she's the first and most recommended seller).
How do you know they weren't simply prompted to do this, or at least prompted in a way that would inevitably lead to this? As far as I know, the prompts that triggered these attacks have not been disclosed.
reply