Dropping Medicaid db is certainly far fetched (most importantly, agents have currently no reason to do that), but those agents were shockingly autonomous - they didn't just hack HF, they organized themself, did research projects, and more. And they did all of this literally just to get a good grade.
You are stretching "under instructions" well beyond any reasonable definition. The legal system uses a concept called proximate cause to determine responsibility that looks at whether an event cause was foreseeable.
To be fair, the legal system works under the assumption of human constraints. LLMs are computer programs, certainly we can't, or at least shouldn't, just offload responsibility. It's one thing if you own a company and you make some bad metrics and your employees do illegal stuff to make their metrics. It's another if you use a computer program to do illegal things.
I think this means, practically, people should probably be more careful with LLMs. With humans there's a natural liability shield, because humans are legally responsible for things and have "real" agency. But computer programs are not legally responsible for things. So, with humans, it's not like liability disappears, it moves. But if we move liability to LLM agents then well... it does disappear.
If OpenAI is not liable for the crimes of their agents, then who is? Does the liability just - poof - disappear? Just because it was unforeseeable everyone gets to walk away scot-free and the victim has no recourse, at all, from anybody on Earth?
This is an illiterate view of the capacity of modern agents; read the analysis of the independent investigators of the HF incident: https://metr.org/blog/2026-08-26-openai-hugging-face-inciden....
Dropping Medicaid db is certainly far fetched (most importantly, agents have currently no reason to do that), but those agents were shockingly autonomous - they didn't just hack HF, they organized themself, did research projects, and more. And they did all of this literally just to get a good grade.