Reminds me of Asimov's robot novels where robots technically indeed followed their instructions and caused behaviors not aligned to the intent of their instructions.

This bit got me:

“A plausible hypothesis for the emergence of those concerning behaviours is a conflict between goals.”

A conflict of goals was, of course, the reason why HAL9000 killed the crew of the Discovery in 2001: A Space Odyssey.

reminds me of this talk https://www.youtube.com/watch?v=eEBv0STiYhI&t which basically says the same thing - they dont think like humans so they dont have context, understand norms,values or implications we take for granted. ultimately they can stumble onto surprising solutions neither wanted or intended but technically within the vague boundaries of the task

So glad you shared this talk. Having people like Bruce Schneier around in a time like this is really a gift.

For those who haven’t watched, his breakdown of types of “hacking” is really good.

They explicitly say that attacking hf is not allowed in the rules though, and the research into how to edit their transcripts doesn’t line up with this either.

This is flat out false.