You're talking about the HuggingFace incident? It's notable that the agents (somewhat justifiably) thought that their evaluation had an LLM grader which would look for evidence that they had cheated. And it was specifically that grader they were trying to his evidence of cheating from, not humans more generally.
Honestly that part was more surprising to me than anything else, how narrow the compulsion to cheat was: they didn't learn "cheat in general" they learned "think about the grader in great detail and chat exactly as much and exactly in the ways that actually result in a higher score".
Yea agreed they were trying to deceive what they thought was an LLM grader. It’s unclear to me the extent to which cheating behavior is generalized? From what I’ve read there are signs that some amount of cheating has been reinforced in their training due to poor RLVR evaluation setups
Why should they care if their actions are discovered by humans? What are the humans going to do, discard their multi-million-dollar training run? Even if they do that, the amount of RL pressure is tiny relative to what happens inside an RL run.
Gasoline + LOX beats TNT on energy density per unit mass and per unit volume, no?
There's a reason that kerosene+h2o2 is a classic rocket fuel, and TNT is not.
It's usually kerosene + O2, not H2O2. But H2 + O2 is another classic liquid rocket fuel that has higher specific energy (energy per mass), but lower energy density (energy per volume) than kerosene, and is used mostly as a 2nd stage fuel (where mass is most important). Its lower density and the need to keep it very cool mean heavier tanks, and it's also hard to work with due to being such a small molecule that likes to leak, which causes development costs to be higher.
Honestly that part was more surprising to me than anything else, how narrow the compulsion to cheat was: they didn't learn "cheat in general" they learned "think about the grader in great detail and chat exactly as much and exactly in the ways that actually result in a higher score".