Cutting-edge models are capable of much more than generating CRUD apps and understanding the details is orthogonal to whether you wrote the lines yourself or not.
>understanding the details is orthogonal to whether you wrote the lines yourself or not
Reading the code does not give you the knowledge of the 10 different approaches you would have tried and failed before coming up with that code. Why exactly a piece of code is the way that it is cannot be determined by just reading the code.
Reading a mathematical proof does not give you any meaningful understanding of it.
You haven't considered that I probably already discussed 10 different approaches with my agent before authorizing a particular implementation. Or that I didn't ask about alternatives during review. You're confusing physically coding with thinking. You can still think just fine without writing code yourself.
If you are experienced, attentive, curious, willing to explore, etc., that doesn't change after an LLM allows you to deeply discuss any concept at will, quickly try different prototypes and zone in on the correct implementation. You will use these tools to their full extent.
> Why exactly a piece of code is the way that it is cannot be determined by just reading the code.
If you maintained a good commit history and your code is self-documenting and you're not capable of understanding and navigating code you read, you can still literally just ask the LLM to explain the code within the context of the codebase. They are extremely good at that exact task. They will give you as much as you give them. If you want to know, and can understand, the nitty gritty, you can do that and no one is stopping you.
> Reading a mathematical proof does not give you any meaningful understanding of it.
Again: these things can function as personal tutors. Run the proof through an LLM and ask it what you care about. Furthermore, there are plenty of elegant proofs you can read which confer "meaningful" understanding.
Before just assuming that everyone telling you these tools are useful is less capable or experienced than you, it's worth considering if it's actually you who needs to maintain an open mind and attempt to learn from others.
Yeah, I disagree on the latter point by a lot :-). Really, is it truly "orthogonal"? Not really, you will understand the details better by writing them yourself. On your first point, they are pretty good at a lot of stuff, given that they have seen it before. A lot of the time, I'm writing code that no LLM has seen before. That sounds super smug, but it's "da tru tru".
I welcome you to prove this conjecture, otherwise it's just vibes.
> A lot of the time, I'm writing code that no LLM has seen before
I hear this tired point over and over from people who cannot fathom that others who use LLMs successfully could possibly also be working in a specialized domain. Frontier models are excelling at difficult, long-horizon tasks now. I write all sorts of esoteric stuff, and I can confidently hand a frontier model specification for a language it's never even seen before, working in a domain it's never encountered, and likely get good results, provided I have the knowledge and experience to guide the model.
The reality is that this "they are only good at things they have 'seen before'" talking point that often gets parroted is vaguely defined and largely based in opinion. Obviously, models perform worse when the input or expected output are out of distribution.
But this was much more true a couple years ago than it is today; the gap has closed considerably, and those who are learning to think deeply with these tools will be better positioned than those who arrogantly think that their process cannot be augmented by the most intelligent systems ever created.
This is HN, most of it is vibes. Lol, conjecture, touch grass man. I do PL research, it's not that good at that stuff, do you think I'm not trying using it?
You're claiming that one cannot understand the details of well-documented and well-written code which they did not write themselves, and I'm pushing back asking for proof. We can move on from this though, I'm more interested in where you currently feel they fall short doing PL research. I think the state of frontier models today in this area is a lot better than it was even six months ago and I think there's still room for improvement.
