Has there ever been any measurement of this, of any sort? Honest question. I frequently see a plural of anecdotes to that effect, but I've not seen a concrete statement of fact or measurement that could be scrutinized or tested in any way.

If so, please share. This should be measurable, and I'm glad this project is measuring it.

Answers in the form of additional anecdotes, stated with even greater passion but still lacking a statement that could be tested and falsified, would validate my exact concern.

Official tweet from Tibo from OpenAI confirming they ran experiments tweaking “juice” (reasoning effort mapping values) and have reverted them: https://x.com/thsottiaux/status/2076495156757577895

As he confirmed, for at least a period of time, and for some users, “Sol xhigh” was actually “Sol high”, etc.

prove it

Literally the point of the linked GitHub repo.

No there isn't.

You mean to write YOU haven't experienced this. Many have indeed experienced this.

I'm not sure "experienced" is carrying much weight here. This is why there are benchmarks, anecdote doesn't mean very much.

This is a great omen for the future of AI

Where is the evidence? HN is no better than the "I've done my own research" on Facebook.

No you didn't.

The claim was that there's an experience of a model losing power. Your claim amounts to "No, you are not experiencing what you say". That's quite a claim for you to make with no data and no argument.

Or "what you're experiencing is irrelevant because perception is fickle"

who is paying you to say that?

Mr. Dario himself