As the time goes on it only becomes harder to differentiate between model capabilities with just one or two numbers. I would love to see some kind of multi-axis placement of all the models on less objective attributes, like wordiness, willingness to give up, an ability to "think ahead" and pre-solve possible problems in code, for example, that I didn't think of or didn't think of talking about, etc etc etc.
For example I've been really enjoying Deepseek v4.1 Flash, it's very "straightforward" to the point of being almost dumb sometimes, but it's absolutely relentless and would solve almost any problem no matter how inefficient the solution is.
No idea how to measure all that, just average CoT length per task is probably a good approximation for some things, but not others.
At this point I already have "ad blindness" but towards "traditional news publication declaring some technology revolutionary" — probability of thw mentioned tech being a nothing burger is just getting closer and closer to 100% with every publication.
I personally think that it's just fun and “in the spirit” of writing to be pedantic about it and typographically expressive in general. In many ways, writing is a very creative process, because it involves “fixating thoughts”, making something very analogue into something almost (but not completely) discrete; it's akin to graphic design in general in my eyes. And when I can bring some extra flavour to it – by sticking to pedantic rules, using rare typographic marks that have a meaning that's more historical now, or anything else – I tend to do it, even if just for my own amusement.
For example I've been really enjoying Deepseek v4.1 Flash, it's very "straightforward" to the point of being almost dumb sometimes, but it's absolutely relentless and would solve almost any problem no matter how inefficient the solution is.
No idea how to measure all that, just average CoT length per task is probably a good approximation for some things, but not others.