I think I read somewhere (and in any case it matches my experience) that the first user will catch 50% of the problems and that with 5 users you can catch close to 90%. So human testing is not just invaluable, it's pretty cheap for the benefits it brings.
At every UX job I've had we generally ran usability tests with 6 participants. This was before the online usability testing tools proliferated and we had a lab in our office (one way mirror with observation room and everything). Every UX researcher I worked with said based on the available research and personal experience it is as you said: more than 5-6 participants would hit diminishing returns pretty hard and wasn't really worth it.
You want to hire your target audience, which may be vastly different from yourself, and suddenly you discover that the language is throwing them off, the color scheme brings different meanings, and they just don't understand the flow which felt completely natural for you.
It's more like the first user catches all of the critical bugs and a handful of users catch all of the high impact bugs. Cause that's kinda the definition of the impact of critical and high impact bugs.
I was also going to post "this sounds like usability/user research".
But I guess we all re-discover things when we need to.
The biggest issue I've seen with tech docs is often expert users of a given tool or product are enlisted to write the docs, because of their expertise.
But then ironically they end up writing those docs for an audience that shares their level of expertise, rather than for the intended audience.
So you end up with lots of assumptions or leaps of logic in the docs that the intended audience can't follow.
I'd wager this is interesting, seems there are a lot of users here building things on the side (or main projects) that need clear UX if they're anything like me, they don't have that UX background to know the best practices. I know I just clicked your link anyway...
There are thousands of great games which are not hand holding. Many studios are putting out amazing work that does not deserve to be lumped together with modern trash.
I'm wondering if this might not be addressing the actual problem? A game such as Rimworld, with much lower details in graphics / looking less realistic, feels much more like it has a "soul".
Can highly recommend Tynan Sylvester's book "Designing Games" which also goes a bit into why Rimworld looks like it does.
I'd rather more games move into more abstract directions for portraying game elements, the majority of games should not try to look "realistic".
I have also been using 5.6 Sol instead of 6. I found 6 to burn through my usage incredibly quick, making it somewhat unusable because I wouldn't be able to get anything done.
My results with 5.6 Sol were quite similar to 6, although I haven't tested it that much.
If this continues to be a thing ...one might actually imagine a good system where we have AGENTS.md with all generic agent instructions and a bunch of harness specific files like CLAUDE.md, GEMINI.md, and CODEX.md that all add whatever nuance and nudging each harness seems to individually require
I like this, but can we not have any of this in the root of the repo cluttering it up with the harness specific instructions for a half dozen plus different harnesses. Something like a dot agents dir
I'm working on a self-tracking app and an Astro Web Starter Template! :)
The app is very customizable, local-first and is built to adapt to what you want. Goal is to combine habit tracking, health logging and journaling into one fast and easy to use interfact.
Have been working on it for almost years now and it's making great progress.
My Astro Web Starter is built for building well-designed, performant and accessible websites. It contains everything a high-quality website needs (In my opinion). I built it so I have one good base on which to start my web projects on. Using basic HTML and CSS that can scale, without bloat.
As I already mentioned in another comment, I recommend this post for people who want to go a bit deeper: https://www.nngroup.com/articles/usability-testing-101/
reply