Lately, I've seen a couple responses to my comments which inquire about metrics. They seem strange and I wonder if they are bots. I just noticed this inquiry is from an account that is 11 days old. How does one benefit by adding bot comments in a forum like this?
I can understand normies being AI skeptics, but at this point it is a pretty insane position for any developer/nerd. It's like being schizophrenic and completely out of touch with reality.
Asking for proof of improvement is a very rational reaction to the last three to five years of constant AI hype cycling.
It's a simple question that seems to make AI cheerleaders really mad.
"How are you measuring improvement."
I use AI for coding every day and see it fail all the time, I'm a seasoned developer and early tech adopter just like everyone else on HN. I'm still very skeptical because of how often these systems just miss. It's gambler's ruin on a very large scale, we remember the hits and forget the misses.
What kind of metrics are you looking for? No one answers that question. It's like asking for clinical trial results about whether or not a parachute works and if you should use one when jumping out of a plane.
I don't expect AI to be perfect. Are people perfect? I leave room for corrections, but at this point they are usually minor and acceptable.
If this is true, then you are not saving a lot of time. Because most of the time is spent evaluating various options and ways to implement the functionality. Even when you are reviewing, you ll have to do that. (With LLMs, this is even more feasible, because now you can actually implement some of the variants, and evaluate them).
But on the other side, you are saving from typing the code. So if you are really reviewing everything, then you are not saving much time. The alternative is that you settle for some local maximum during each review, that in long term won't necessarly translate to a globlal maximum or even a global "good enough" position...
I'm saving time. It's implementing features in a few hours that would have taken me weeks to implement. I can review code much faster than I can create well reasoned, implemented and tested solutions.
This faux socratic method flavor of AI skepticism is so cringe at this point, and reeks of your own insecurity in your beliefs. We get it, you don't think AI is useful for the type of coding people claim it is. Why on earth would anyone try and convince you at this point?
This is not a binary thing. It could be useful at the same time detrimental in some manner. Look at smartphones. I am just raising the possibilities if one use LLMs indiscriminately to generate code.
Probably because it's utterly pointless to try and convince someone in 2026 that LLMs are useful for coding. Why waste time trying to talk to someone with their head in the sand?
Kindly share the metrics by which you evaluate the changes.