You assume using network to solve the eval problem should be viewed as a security violation. From the perspective of the response team who let it continue, this was just a poorly written eval problem, where the solution unexpectedly involved some network hacking (not unreasonable given it's a cyber security eval).
I don't think your assessment that the response team did something wrong is fair, unless you have seen unreleased documentation about what sensitive resources were within the impact radius.
> You assume using network to solve the eval problem should be viewed as a security violation.
Actually, that's the part that I'm most ok with. LLMs cheat at tests, we know and expect that, and it's harmless during training (except for the scores).
My problem is that the response team found out that the AIs had 1) hacked their infrastructure, and 2) displayed an emergent swarming behavior (with no defections!). Either one of those should have made the team press the big red button that stops everything, but they didn't.
> unless you have seen unreleased documentation about what sensitive resources were within the impact radius
The on-call team didn't know that either! Once your infrastructure is hacked like this, you should be questioning everything you see while carefully following all threads, which takes time. Before you finish this process you don't know how bad things are, and therefore it's irresponsible to keep the AI running. The same goes for strange emergent behaviors.
Unless getting hacked by your own AIs is a normalized occurrence there, in which case it's a different kind of disregard for safety.
Source is unclear. Self-reported by candidates? Huge bias then, since people may or may not want to say whether they got / didn't get an offer, and whether they accepted / rejected an offer, and this split may vary by company and by year very significantly.
Why would it change by year? I get it could happen by company if some companies are much more secretive than others but not sure why it would change year over year.
No matter whether it's AI generated (I can't tell) or correct (it's not), it makes no sense to flag the response by the author.
reply