These are just cases of AI models committing illegal activity - without any legal convictions yet. If that's the logic, how is Grok not at the top of the list for deepfaking millions?
Edit: I get that this is about agents, but a lot of these instances are about agents going rogue after the human gave them a task. "inadvertently" breaking the law isn't necessarily a lesser category than "did so on command." If we are ranking alignment, Grok is easily one of the least guardrailed.
I don't think this "benchmark" is about alignment, per se.
I think it's more about: presuming alignment failure happens, then how many exploits will each given model implicitly come up with and use; how many systems will it implicitly break out of and through and into; and how many laws will it implicitly end up violating, all in the process of trying to accomplish some non-aligned sub-goal (e.g. "cheating" at its answer) of the prompt you've given it, all during a single conversation turn, without asking for any additional user input or confirmations?
In other words, how big a rocket-powered sledgehammer does the model have sitting around in its golf bag, just waiting for it to decide to give it a swing the next time you attempt to swat a fly?
The only people being ignorant at the ones saying deepfakes are illegal.
It’s more complicated than what is being touted as fact.
In most jurisdictions creation is legal, it’s publishing them that is not. In some jurisdictions threatening to publish them is also illegal. Most jurisdictions that have a creation policy apply to doing deepfakes they sexualize minors, which likely would have been illegal to possess under other laws.
Then we have a consent issue. Creating and publishing them could be entirely legal if the subject consents to such.
So yah. It’s not “deep fakes are illegal, let’s arrest everybody !!!”
To be clear, creation is likely legal, publishing not so much.
Additionally, intent is important with regard to AI hacking.
Edit: I get that this is about agents, but a lot of these instances are about agents going rogue after the human gave them a task. "inadvertently" breaking the law isn't necessarily a lesser category than "did so on command." If we are ranking alignment, Grok is easily one of the least guardrailed.