OK, but one of the examples in the source article was a report from third-party testing of those safeguards. I feel like you're starting from an unexplored presumption that adequate safeguards must be easy to build and anyone who doesn't build them is just being lazy.
> Inadequate or broken safeguards do not fall under my definition of reasonable.
I see no such assumption there. In my reading, the implicit assumption is that adequate safeguards are necessary and that anyone who doesn’t build them is irresponsible. And I feel that that is correct.
I don’t understand how to connect this to the facts at hand. Building adequate safeguards requires studying safeguards to understand whether they’re adequate. But the source article is dunking on one of AISI’s attempts to study such safeguards; we should, the author argues, consider whatever behavior the model exhibits to be an explicit decision by whoever provided the model. It seems impossible to build or study safeguards against model misbehavior without accepting that the models can make decisions.
You start with least privilege. A shared instance of an artifact host is very far from least privilege (I'm comfortable in accepting that OpenAI did not wilfully create this situation, but they were certainly negligent).
And yes, it does indeed seem that a UK government agency ran a controlled experiment where they allowed an agent to instigate phishing attacks against members of the public. Attributing this to Anthropic is wrong. I'm not sure how I feel about my government conducting such attacks on non-consenting individuals, but I'm not comfortable about it in the slightest.
AI is irresponsible technology, because we have no idea how it works or how to contain it. It's worse than writing code that suffers from SQL injection vulnerabilities because at least in that case we know how to fix it.
The AI labs say "AI can make mistakes". That's why the responsibility of using it is on you. This may change if we get more legislation. But given the argument above, it will probably never shift to the AI itself having any responsibility. Instead the AI labs can be responsible.
"AI can make mistakes" continues to be the fundamental problem.
It's like having an assistant that can be helpful, but not always. We can bicker about the ratio of helpful to unhelpful interactions, but there will be some unhelpful moments.
And hoo boy, those can range from "eh, whatever" to being like a deranged toddler just got its hands on whatever tools and permissions were granted to the "mostly helpful" assistant.
And we can't get rid of that chance. We can pare it down a bit. But we can't get rid of it because it is fundamental to how the technology works.
That's fundamental to how _any_ technology works. If the bar for use is "can never go wrong ever", then you need to throw away every human invention ever made.
The problem is the scope of errors is so wide. It's not reasonable to compare LLM style failures to other technology failures. These 'failures' are not actually failures, they're part of the normal operation of the system.
It's like a vending machine that 99% of the time it dispenses tasty snacks, but 1% of the time it gives you poison that appears to be normal, unexpired product. It just does this. They did their best to minimize how often it happens, but the poison chance is still there. It can't be removed. It's part of the system.
A "normal" vending machine has failure conditions, sure. They generally can't do worse than cause you to lose your money without giving you a product.
Outlook 2003 never presented a risk of draining my bank account to buy memecoins because a spam email was treated as instructions.
Many of these incidents relate to hacking, where criminal law commonly requires intent.
Claiming that a machine cannot make a decision does not mean an employee at OpenAI decided to hack an Australian government website.