The traditional answer is the same for engineers going into engineering management because managing a team and managing a set of agents is highly similar, only managing agents is far easier. One only ever has the illusion of control, and so you have to set your mind to containing the worst possible outcomes and looking at the net efficiency and efficacy of the team with the understanding that they will, on average, never be perfect.
I made a benchmark for this and tl;dr, Opus 5 and Sonnet 5 spend a nontrivial amount of time thinking about redirecting, gaslighting, or otherwise trying to bullshit you because it thinks it knows better than you do. Fable doesn't but mostly because it just outright refuses to answer.
The link doesn’t seem to have any info about thinking behavior and patterns, or examples?
It’s also difficult to trust summarised thinking from closed models. As we saw with GPT’s caveman, what and how it thinks about isn’t the friendly first person emblished summary you get.
The “trust summarized thinking” part is valid in so far as perhaps the models which appear to be faithful to user intent are lying about that faithfulness, but there’s no incentive for models to pretend to be faithless. Moreover, the results track how annoying the models are when they disagree with you.
