There’s no reason to trust a human any more than an LLM for this. Humans are not 100% accurate and very capable of making crucial factual errors in documentation. Ask me how I know.
There’s a _very_ good reason. Humans have a much richer mental mental model of the codebase, design decisions, external factors, etc.
Of course Humans can make errors when translating that to English. But if you’ve read any amount of Claudish it should be obvious that interpolating the code to documentation with an LLM typically produces a significantly worse result in almost every way. Except for superficially “sounding” correct.
That’s just not true especially when the author designed the language with the LLM like he says in the end (the AI is very good at forcing you to be specific and make decisions on ambiguous terms): all decisions can be included in the repository in a way the computer can follow and understand, which is a pattern called specification graphs. You can just document things in the docs as well if you want a more basic method.
If you are claiming this language is worse in every way than others that were made by humans alone, could you be specific?? What are the bad things you’ve found? What part of the docs are incorrect? Or are you just commenting without having looked at any of that?
Above all, I think I like human docs because then I not only know someone has a deep understanding of it, but also that someone can actually be queried about it. With Claude, there is always the accountability problem. If something seems off, do I even have someone other than Claude that understand it all and who is accountable ?
That is not necessarily true if the codebase is written by AI unless the human chose to be involved. That was also not necessarily true before AI either. Human skill level is wildly variable
It's probably possible to prompt an LLM into writing well, but most LLM writing that I've seen is poorly structured (e.g., the paragraphs in this post feel like strings of punchy marketing phrases), and the LLMs are too good at sounding like they're saying a lot while saying very little.
Plus, if a human puts the effort into writing something, that signals to me that it's worth something, because human time is much more scarce than LLM time.
I recently saw a language posted that caught my interest. I went into the docs and read the page "How X compares to Y", and Y is my favorite language
Within the first couple of paragraphs and bullets it made a completely false claim that is the opposite of how the language operates, something any person that knows the essentials of it would know. Language Y was listed in the front page as an inspiration (and I don't doubt them) but clearly they either did not read the docs the LLM wrote or did not use the language for long enough to know it has zero structural typing
