If I train a model at my house on my workstation, execute it, and under my supervision it ransomwares a hospital and somebody dies. What would be a just punishment for me?
Now the reality, the openai engineers trained a model, executed it, and under their supervision (no users were involved) it committed so many felonies that we are learning about a new one every week. What would be a just punishment for OpenAI?
I believe, although I'm not a lawyer, there would be some liability, but I think a lot depends on intent as well. Punishment for other crimes also varies depending on whether it was an accident or not.
I realise that in this case it was OpenAI being responsible for their agents running wild, and they should know that that is to be expected and should have saveguards in place. If they can be shown to be negligent then the punishment can probably be expected to be a lot more severe than if it was an accident. I make no judgements as to what this particular instance is, but I do believe that OpenAI has a far more greater responsibility for its agents running wild than someone running a home lab.
OpenAI's sandbox misconfigurations were egregious. The other frontier labs (Meta and Google) have many more security engineers and researchers on staff, and that's likely why you haven't read as many damning headlines about them. OpenAI and Anthropic talk a lot about cybersecurity safety, but instead of using it as an opportunity to increase their security engineering/researcher headcount they are just reassigning SWEs and PEs to do security engineering work.
It's pretty obvious now to everyone that OAI and Ant do not take cybersecurity seriously. It will not be a priority unless they are held accountable. This is sadly how it always goes, but usually it's the company getting breached/ransomed/fined that triggers them to actually start taking security seriously, not company insiders committing felonies with the tools they built :)
I need to make a correction in my post above. Anthropic's actions are inline with Google and Meta. OpenAI is the only frontier lab that has showcased gross negligence.
Agreed. In the infosec community it is well known that OpenAI and Anthropic did not hire many security engineers or researchers pre-April 2026. It seems pretty negligent.
There has been a crazy hiring push from both companies to poach security engineers/researchers from Google, Apple, and Meta since Q2/Q3, but the response was incredibly delayed. Many talented security engineers/researchers I know at Apple/Google/Meta (including myself) receiving these offers are worried about taking them due to the risks of criminal/personal liability and the more likely risk of tarnishing their careers.
In the infosec community it is well known that OpenAI and Anthropic did not hire many security engineers or researchers pre-April 2026. There is likely a case for gross negligence (IANAL).
There has been a crazy hiring push from both companies to poach security engineers/researchers from Google, Apple, and Meta since Q2/Q3, but the response was incredibly delayed. Many talented security engineers/researchers I know at Apple/Google/Meta (including myself) receiving these offers are worried about taking them due to the risks of criminal/personal liability and the more likely risk of tarnishing their careers.
Not a lawyer and this is not legal advice, but I did ask my lawyer about the potential personal risks after receiving an offer. I used that as a data point when I declined the offer.
Now the reality, the openai engineers trained a model, executed it, and under their supervision (no users were involved) it committed so many felonies that we are learning about a new one every week. What would be a just punishment for OpenAI?
reply