A person is also not like a hammer. Both persons and AIs have been known to mis-interpret instructions, the normal way to deal with that is either to terminate your relationship with the people (or to terminate the AIs and train up better ones). Hammers don't mis-interpret their instructions, they are wielded by a person who is in control.

You can keep 'trying to explain' but then you should use words according to their commonly held definitions otherwise it becomes really hard to have a conversation.

Imagine the human equivalent: I hire John. John is a capable, and competent guy. He's also got awesome computer skills. I tell John to 'go out and find me some good information on my competitors'. As a result John hacks their servers and comes back with all kinds of goodies. Six weeks later I notice what John did. I don't fire him, nor do I take any responsibility myself. But I do make press releases about what John did, in which I'm careful to craft the image that John has these capabilities and that we as a company are for hire.

This was an advertisement, not a confession of a crime.

They've been purposefully building more and more craft into the toolset, that's on them. If your AI is nicely boxed in it will give you the answer for 2+2, it isn't going to think '2+2, what a boring problem, I must go hack huggingface'. Not having this stuff airgapped is irresponsible to the max. I am obviously nowhere near as competent as they are at this stuff and yet my AI workhorse is guaranteed not going to break out of its sandbox because I've set it up in a way that it can not. My conclusion is that OpenAI purposefully left a channel, simply because there was a pathway to the net. And with 'pathway' for the sake of being completely clear I mean a number of connected systems that eventually gave way to the open internet. On top of that they failed in monitoring the outbound links, even if they had some logging in place.

I definitely think OpenAI (and Anthropic, and Google, and Meta) could have, and should have, done better.

But also I remember (and it wasn't even that long ago) people mocking the idea of AI ever getting competent enough to find zero-days in their sandboxes.

I'd go further: if any of these companies tries to make an excuse "oh, but ${safety measure} against ${capability} is too hard", the response needs to be "then you are forbidden from even developing ${capability}, and must be inspected continuously to ensure you never even accidentally produce ${capability}".

Webasto. They make a really decent little unit, easy to integrate too.

> I could be wrong about the severity.

They wrote to a device they did not own. If you did that, regardless of intent, you'd have a good chance of ending up in jail.

Agents litter all the time, these 'agent droppings' often contain clues about what is going on in the token stream. I log everything and every now and then I'm amazed at what scrolls by (for instance: an agent that picked up on an obscure log file that i had set up to monitor another part of the stack that it used to debug its own failure to start its own scripts, my agents are best compared to a prisoner with a very large iron ball attached to its ankle, just in case, and if that hampers 'progress' then so be it).

I'm not sure if I buy OpenAI's fearmongering regarding how dangerous their stuff is, but I am starting to lean towards believing that OpenAI is dangerous and irresponsible.

I've head the 'the hacker did it' excuse from lots of companies that messed up themselves but did not want to admit it.

I got banned from school computers and nearly expelled from high school (and threatened with “blacklisting,” lol) for “hacking” after a virus got into their network. Took them a month to figure out what really happened. Or at least, a month to come clean about it.

So, you're telling me they didn't have any monitoring in place around their AI to notify them of an attempt at breaching a system they have no business visiting in the first place? OpenAI should be blackholed on this basis until they clean up their act.

They must have had monitoring in order to be able to detect this retrospectively.

Any automated alarms for detecting things in real-time were not sufficient.

Given a previous generation of agents discovered a zero-day and used it to get around attempts to sandbox them into one specific test, this is not hugely surprising, but it is a reason to force them (and everyone else) to stop until security catches up with capabilities.

I'm thinking of the Jurassic Park novel: they had sensors to count the dinosaurs, but the test was made under the assumption escapes were possible and breeding was not, i.e. something like "if (dinosaurs_found < n) then escape_alert();". They didn't know dinosaurs_found >> n until everything was already going wrong.

You miss the point: there was a path. Sanctioned or not does not matter.

And you won't be able to disable it.

You can certainly avoid using agents. While I use coding agents and LLMs to research certain topics, I don't have a lot of use for other ways AI has been integrated into mobile devices.

When I first experimented with AppFunctiions, I was impressed with what the very limited test agent could do combining information from multiple apps in useful ways.

I'm working on a product for elder care. I could've spent quite a bit of time integrating Google maps and creating a bespoke automation that would tell care providers about shopping for specific items on their way to a care recipient home. Instead I just prompted the agent to plan that trip. Even at an early prototype stage, using a test agent that had limited access to apps, the result felt magical: agents automate automations. You don't stop to sharpen the saw, you just ask for the right saw for the job and there it is.

Unlike early attempts at agents that need sketchy access to grab screenshots and use your passwords, app tool calling feels a lot safer and more orderly.