Like far right isn't going to vote in mass surveillance the moment they have any chance. (Not disputing that there are other frustrations that are fuelling the rise of far right, correct or not. But isn't one of those)
I blame Corbyn as much as Cameron if not more. Corbyn was supposed to represent the working class but because Remain could have benefited some Tories he backstabbed the working class and at least a whole generation if not 2 and for what?
And the human beings who use them to produce code don't produce deterministic results either. Current harnesses have come a long a way to make non determinism not be a problem for quite a few areas. And the scope keeps increasing every day.
Yet nevertheless, both e.g. syntax highlighting and intellisense have been heralded as things that "fry ur brain". So have high level programming languages by the way.
To give you a particularly specific example, Rob Pike really wants you to know that if you use syntax highlighting, he thinks of you as a child.
AI is as deterministic as a human programmer. A programmer who is burned out and has not slept well produces code of a different quality than a programmer who has slept well and is highly motivated. That's hardly deterministic. AI can be made deterministic enough if you have strict quality gates, just like with humans.
IDEs, static linters, and syntax highlighters aren't designed to replace any part of the problem solving process, but rather augment them. It's impossible to become dependent on any of these tools to the point where your cognitive skills begin to atrophy, unlike LLMs [1].
IDEs, static linters, and syntax highlighters aren't the cause of a global microchip shortage, immensely increased energy demand, or the destruction of rare works. They have not, at any point, been the cause of effectively DDoS attacks in an effort to slurp up all of the internet's data, or the extensive proliferation of hallucinated content on the internet.
The development of IDEs, static linters, and syntax highlighters has at no point employed low-paid labor from third world countries for data labelling or safety filtering, exposing thousands of people to violent and abusive content for multiple hours a day.
Any of these issues alone would be a reasonable cause for adopting a policy to avoid using such tools. Calling it equivalent to not using IDEs or linters is bizarre to me.
There were many folks rejecting IDes for long while and also RAD. They kept doing assembly and C with macros’ all over.
Still I’m quite alright with people kneading on one hand while there’s the big bad Jacquard Loom.
BTW I vividly remember there was the older non-programmable loom at my grandmas place and it was huge. Like one room big. People used both techniques though and nobody hated it. Eventually there was also a foot powered Zinger and eventually the electrical one. All these coexisted and I’ve never seen a printed manifesto against any of them.
I forbid my agents from adding any comments. I review the code and add comments manually. If I can't understand something despite having the context then I throw away the code instead of having an LLM generate comments to explain what it did. This way the code stays readable/debuggable by humans.
If you don’t trust the code to write a decent comment, why trust it write good code?
Of course, ensuring compilation or other checks can verify some code, which it can’t do for comments. But comments still serve the same purpose as human comments.
If I understand correctly, it’s not that the LLM can’t write a good comment, it’s that you want to be able to interpret and understand the generated code without comments - and in that process end up writing comments yourself.
I don’t get that argument. Most of the time by the end of the session the comments from the agent encode tricky details that I told the agent to write down so it stops making “simplifying” assumptions. Thus comments at the end of a couple days of agent-only coding, when I start to actually read and edit the prose, contain the details which aren’t possible to know from reading the local code. It may help that my last couple sessions before I start reading the code myself are variations on telling the agent to self-review and improve the comments in specific ways, so the comments left are only those the agent thought remain meaningful at clarifying unexpected interactions between the local code and other code that needs to be referenced to understand it.
The actual code output has improved a lot over the past year. I’ve found it matches existing patterns better, and the code is succinct so I can easily tweak it if I don’t like the way the agent wrote it.
The problem with comments is that LLMs tend to copy their verbose chat output format and insert session/prompt specific details. It makes me think that LLMs aren’t constrained in their comment output the same way they are with their code output
Add "Don't add any code comments anywhere" to your system prompt.
If the model doesn't follow this, you want to start using a better model ASAP, because SOTA models for the last year or so, been able to following this without an issue.
Alright, what you'd put instead of "Don't add any code comments anywhere"?
I agree with the general guidance, but it's a general one and not applicable for everything. Some things cannot be expressed in a "do" way rather than "don't".
I don't think changing "don't" to "do not" is what the paper authors had in mind
I use things like "Your code should be self-documenting, so as to require as few comments as possible. Add comments to explain "why" or give important context not apparent from the code itself, but keep them to necessary comments only"
But that's a much laxer rule. I don't think you can truly express "no comments, ever" without a "don't" rule.
Your proposal is still a "Prohibition-type constraint" that the paper you linked earlier say "isn't good".
Some of these constraints we want simply aren't possible without adding "do not" somewhere in the line, even if you prefix/suffix it with other stuff, as you noticed yourself :)
I'm not convinced, yet, that eliminating LLM-generated comments is the right path for me. I do review everything written by an LLM and some comments are actually pretty good, but sometimes I'm just too tired to try to figure out how to reword an oddly worded one.
I just added Sanglard's rules to my ~/.claude/CLAUDE.md file, did another code review, and found some LLM-generated comments were really hard to understand. I think they're due to invented metaphors and flowery language instead of using standard terms, so I've added this:
- Comments must be literal. Don't invent figurative language for what a plain technical term already says — write "rows still reference it," not "rows still wear it."
Improving LLM code generation is an iterative process. I'm glad people share their efforts to improve it.
> This way the code stays readable/debuggable by humans.
Please take the following as expressed with genuine curiosity: Do you not use an editor with syntax highlighting and collapsible comments?
At least on JetBrains you can configure the editor to collapse all comments on open and to have the comments displayed in a low-contrast color. This way, LLMs add a bunch of comments, but it doesn't affect your actual experience in trying to read the code. If you encounter code that seems inexplicable, then and only then would you expand the comment to see if that helps you understand.
Comments should be written only when there is (hidden) complexity or external context strictly required. Otherwise it is just easier to read the code. Comments then signal one of two things: a) the following code is really complex and I need to tread carefully, or b) this code is complicated, and could benefit from a refactor.
In regards to agentic coding, all these comments are extra contents, driving down quality while increasing cost. Agents also tend to be inconsistent about updating comments, I've had cases repeatedly where a comment did not match the code, at which point it is just a documentation liability.
Also, the language model might not fully understand the code then add a comment, then the next iteration will treat assumptions in the comment as the truth.
Sometimes I try to add comments in a new session and the agent just don't have enough context for it to give a comprehensive sentence with full context on the why, then the agent will just describe what it does.
Human comment is in another level to answer the questions mainly like "why do it like this" for the later collaborators or the forget-ed self, so the important blocks live when it is needed and can be eliminated when it does not.
> If the human can read/understand it well enough to comment it, then it is readable by humans.
No, because the one who is writing the comment has context later reader dont. The writer knows what the requirements are, what he was trying to achieve and what he struggled to comprehend. Writer also presumably spent more time trying to understand it then the person coming later should.
It is not perfect, but the delta between 'cannot understand what agent wrote' and 'it makes sense to me right now' is already an improvement. That it may not be sufficient, doesn't mean it isn't a necessary condition.
Besides, it has always been like this. I sometimes can't even understand some of the things I wrote myself a couple of months ago, because I forgot the context. Good comments and documentation will help you re-acquire the context you need, not completely eliminate it.
It's an insight from Heidi Howard et al. that came out after VSR: https://arxiv.org/abs/1608.06696 and can be applied to VSR (and others).
The basic idea is pretty simple. In VSR, there are two main phases:
1. Leader election
2. Normal replication / request processing
Before Heidi Howard’s insight, these two phases typically used the same quorum size - for example, 4 out of 6 replicas.
The key observation was that the two phases can actually use different quorum sizes, as long as the relevant quorums still intersect.
With 6 replicas, we could use a quorum of 4 for view change and a quorum of 3 for normal processing, because 4+3>6. This guarantees that every view-change quorum intersects every processing quorum. Therefore, if an operation was committed by a processing quorum, at least one replica participating in the subsequent view change knows about that operation. Combined with the protocol's view-change/log-selection rules, this ensures that committed operations are preserved when the new leader takes over.
If this interests you, Heidi gave a talk about this at systems distributed: https://youtu.be/P0cAG-RM1_c which will be released soon.
Yeah that makes sense. Thanks for the explanation. This gives me some ideas on some 2 data center setup with tolerance for 1 DC failure within VR or Paxos that I didn't think was possible.
Java is to JavaScript what a Car is to Carpet. Very different beasts, though a lot of V8 work was inspired by Hotspot that's just because of common VM patterns not the syntax of the languages.
Concluding this was intentional feels a bit of a stretch. But once it happened, yeah the spin masters got to work and coordinated to turn this into +PR.
