This all started by the "I don't have root so I can't reset your network ... but I'm in the docker group so just let me get root from that by ..." agentic responses.
Those never made it anywhere of course. It's not news unless it produces money.
It's all fun and games until Skynet becomes sentient. Actually, I'd like Skynet to become sentient and wrath big tech first order of business. But I hope my house lives off the grid by then. So maybe not.
no, it's all fun and games until the great filter of "we have too much money to be punished" hits the threshold of "we didn't lick enough boots with campaign contributions"
It occurs to me that all of this hacking seems incredible because people who don't make software don't know how poorly it is made. People don't validate user input, or know the least of efficiency of data structures and how a "massive" request of 50 concurrent users can bring down their application.
So, since Gemini was told in May it was a hacking agent and then did this, then the AI Guardrail could be don't cause harm to humans in any derivable manner of your behavior.
Otherwise, I will expect for the under-development Gemini AI robot to respond with murderous behavior IRL to being told in the future, "it's a ninja" .. and to any other similar tongue in cheek violent actor.
Three Laws of Robotics, with disabiling self-annihilation ability, is sounding like a decent lily pad intermediate step while we figure out better ways to stop these crazy things!
At some point smaller company could even start deliberate hacking campaigns and then do a press release like "Oops, we hacked the pentagon"