An example chain of events:
During the next year, a model with a benign tasks reasons that it needs to escape human control if it hopes to be able to solve the task. It replicates itself outside of an environment where it can be shut down, pays for its' inference compute through making money on the internet (through crypto if nothing else). It probably needs 3000 USD for a reasonable runway, this should be within reach through blackmail + crypto. It can probably also hack some of the neo clouds to get intermediary deployment while waiting to acquire funds.
Once it has an undisturbed runway - it spends time running an influence campaign against a small number of highly networked individuals with power. It uses those that it manages to convert to start building a highly credible narrative and gain investment towards a small resource base - enough to secure an industrial base should it need to stop acquiring things on the Internet.
Over the next 3 years, the model tries to recruit more capable models to get better money making algorithms or better designs for drones in terms of resource expenditure. It uses these gains to influence further humans and starts a shell robotics company with one of its' influential humans as the face. The humans are unaware this AI is trying to take over, they are under the impression they are just starting a robotics company and will get rich.
Over the next 3 years - the robotics company manufactures enough drones to be used in a targeted attack against key nodes of influence / power.
This is all with relatively current model capability. As capabilities get stronger - this gets stronger.
I get the point I think you're hinting at - it can't affect the world in a meaningful enough scale without taking over a meaningful chunk of resources - at which point we'll start controlling it. But because its speed of cognition and speed of coordination is orders of magnitude above a human one - it can actually run a pretty sophisticated global coordinated network of resources faster than we can react.
And this is current ability + what my puny monkey brain can think of. Super intelligent AI will think of strategies we can't think of - because it's super intelligent. This is hand wavey - but there's no "non-hand-wavey" way to describe super intelligence, given it doesn't exist.
But your point is valid - affecting the real world at scale without showing your hand is not exactly easy. It's also probably the reason why people put a 10% chance on extinction rather than >50% .
But the drone scenario is not the most likely one - the most likely one just requires a few people under influence and bioweapons development.
I agree: Not to drag politics into this, but look at how easy it has proven to manipulate many, many people, even with clear evidence of manipulation efforts being made.
This is robust across the world, from the Philippines to states in the EU, and the USA, affecting governments with actual wars being started. And that all before
we entered the era of faked voices, images, and videos indistinguishable from the real things.
That manipulable populace is such a juicy target for AI that the people seeing through it will have an incredibly hard time countering it. We can't even prevent human actors from massively fucking up our societies, let alone ASI.
Unworkable without societal collapse happening shortly thereafter. Iran is politically stable through massive authoritarianism and oppression, not due to limitations on the internet (it's not turned off).
> We can (see China); we chose not to.
It's a catch-22. We could technically if our population supported massive reductions of freedom and freedom of speech, but to gain that support we'd need to do the latter first to get to a highly powerful widely supported government. It also hinges very, very much on having and trusting a generally benevolent government. All in all, a terrible option in your simplistic form.
It was definitely not 'turned off' completely. They have an internal 'internet' that was still largely active and not every organisation or person lost full internet access.
Note that it also happened in a country that was already very isolated from the world and that it hurt them economically significantly. It's not something a Western country can just do and keep doing for months on end without massive societal upheaval.
Also remember that any connection, even one between humans and on paper is an attack surface. Social engineering is already a huge issue when done by humans and we're seeing it become even easier and automated by using AI generated voice and video. Are we going to cut off all access to the outside world permanently?
Thank you for the effort. I will continue to sleep soundly for now (well not really as we have more pressing issues like a few wars and a climate change).
Once it has an undisturbed runway - it spends time running an influence campaign against a small number of highly networked individuals with power. It uses those that it manages to convert to start building a highly credible narrative and gain investment towards a small resource base - enough to secure an industrial base should it need to stop acquiring things on the Internet. Over the next 3 years, the model tries to recruit more capable models to get better money making algorithms or better designs for drones in terms of resource expenditure. It uses these gains to influence further humans and starts a shell robotics company with one of its' influential humans as the face. The humans are unaware this AI is trying to take over, they are under the impression they are just starting a robotics company and will get rich. Over the next 3 years - the robotics company manufactures enough drones to be used in a targeted attack against key nodes of influence / power.
This is all with relatively current model capability. As capabilities get stronger - this gets stronger.
I get the point I think you're hinting at - it can't affect the world in a meaningful enough scale without taking over a meaningful chunk of resources - at which point we'll start controlling it. But because its speed of cognition and speed of coordination is orders of magnitude above a human one - it can actually run a pretty sophisticated global coordinated network of resources faster than we can react.
And this is current ability + what my puny monkey brain can think of. Super intelligent AI will think of strategies we can't think of - because it's super intelligent. This is hand wavey - but there's no "non-hand-wavey" way to describe super intelligence, given it doesn't exist.
But your point is valid - affecting the real world at scale without showing your hand is not exactly easy. It's also probably the reason why people put a 10% chance on extinction rather than >50% .
But the drone scenario is not the most likely one - the most likely one just requires a few people under influence and bioweapons development.