Does the context size really matter that much if a “dumb zone” starts around 120k-150k anyway?

Things change fast! For Fable 5 it definitely feels past at least 272k

hmm, what about other models?

Not sure if this is still up to date (2023), but https://arxiv.org/abs/2307.03172 shows that performance degrades mostly in the middle of the context.

Anecdotally I've been stuck in that situation of being at 400-500k tokens and "just one more prompt bro" will get the task done, and I appreciate not having to wait through a compaction. If anything, keeping the bloated context helps with accuracy at the expense of speed in these cases.