By definition an LLM is unable to build what has never been built before because you are going outside the realm of pretraining/RL. You might be able to combine components that already exist in a unique way, but even then you will be fighting against the mediocre making effect of LLMs
That understates what LLMs are capable of doing. They can and do build things outside of their training dataset. As long as you can get there by following existing concepts, they can do it.
Perhaps outside their training set in a strict sense of “this thing itself was not specifically in the training set” (although only perhaps - users often iteratively coax frontier models into achieving things just beyond their envelope, and those achievements could well be folded into the next model’s training set without you or I being aware of that). But even then, it is not directionally outside the training set: it steps outside not in the direction of the unknown, but in the direction of another part of its training set. An evocative metaphor would be dashing from cover to cover in the rain, because it lacks the umbrella or raincoat or simply the fortitude to tolerate getting wet that it would need to reach an uncovered destination.
That seems nonsensical. Consider a selection of even just 8 types of components or patterns that might be in a training set (or available to be found over Google and fed into a prompt by a harness, for that matter). There are 256 distinct ways to use 1-8 of them together.
And of course the number of libraries, known algorithms, etc, is much larger than 8. There's any number of ways to make something new by combining well-understood parts.
I don't think non-expert prompting is pushing the envelope on what can be built because I think the current generation of harnesses and models pushes to using well-understood patterns in well-understood ways but it's certainly moving the line for what has been built. And with more skilled prompting you can build pretty crazy stuff + extensive validation for it.
The problem where the centrepiece of the controversy was OpenAI admitting the potential of using private work happening by mathematician users as inputs?
Your argument is that advanced mathematics, that did not solve navier stones, were in the weights and that enabled them to solve it. And you think it butresses the argument that the llms can’t do _coding_ tasks they haven’t seen before?
> Your argument is that advanced mathematics, that did not solve navier stones
Do you think playing dumb is making you look credible or something? The argument isn't "was maths in the dataset" the controversy was if someone with potentially working approaches narrowed the search space enough for the LLM not to fail
> And you think it butresses the argument that the llms can’t do _coding_ tasks they haven’t seen before?
This isn't an argument being made. At least try to respond to what's actually written rather than making yourself look foolish and trying to wordplay your way to a point.
Well being a different person seems quite key. Not that I 100% disagree with the full quote rather than your selective quoting.
> By definition an LLM is unable to build what has never been built before because you are going outside the realm of pretraining/RL. You might be able to combine components that already exist in a unique way
In most threaded Internet forums when you respond to a thread you are implying a continuation of the previous conversation.
It seems you weren’t following that norm and we were talking past each other. That’s fine and I’m happy to leave it at that.
Perhaps in the future you can recognize the local norms before resorting to personal attacks even if those norms are different than the parts of the internet you normally hang out in.
I see I didn't make this aspect very clear in my comment, but I believe that what I described is only possible when a human is in tight control. I agree that an LLM on its own will generate pure slop 9 times out of 10. As an extension of your own creativity, I believe it will push the boundary.
There is a hundred-mile gap between those two places. "Write a register-based VM with a zig-style flat AST" is a lot less work than actually writing it. That's the kind of direction it needs.
There are decisions you care about, and ones you don't. Things like naming conventions or formatting probably belong to the latter category, and in that case having the AI arbitrarily decide isn't going to matter.
The author proposes to reinvent the wheel and depend on stale dependencies, which are EOL. Neither of that is an acceptable solution with LLM-based Agents being able to produce exploit(ExploitGym) chains in minutes from known bugs. The other issue is that Rust's forces the user to provide more information and APIs are usually kept generic for systems programmer, so standardizing things is not as straight-forward compared to Go where you can assume a memory-management, a virtual thread runtime and mostly ignore dynamic dispatching.
I don't think anyone is saying that you should depend on stale dependencies. You should not hand off your responsiblity to untrustworthy package managements services though. Evidence of the failure of automated package management is the near constant shai-hulud attacks as well as the originating post here.
The fact is that your dependencies are your responsibility. You should be staying up to date on what you depend on and their vulnerabilities. Most vulnerabilities affect specific code paths which may not be present in your code base and do not require mitigation. These are decisions you can make by understanding what your dependencies are instead of offloading to automated package management systems.
NPM has been the posterchild of these types of issues but the dependency stack I saw everytime I compiled even simple programs in Rust made me think Rust would be impacted next. Personally I think third party modules that are effectively universally used (axum, tokio, serde) should be integrated directly into the language.
Well I think the main important argument here is you shouldn't use the package management. A third party package manager would be even more concerning to use imo. I generally try to stay away from the typescript world so unfamiliar with nuances of NPM.
Man good motivating schedule. Are you married as well? I notice this is missing the wife time. I feel like I really have to wreck my sleep to get worktime between wife and kids.
This article feels slightly incoherent. You want high quality precise writing and to use an LLM to generate it? Feels like those are diametrically opposed
Exactly. The watermark is proportional to how much text is AI generated. Either the AI really just “fixed some typos” (not enough AI content to hide a watermark) or the AI did most of the writing (enough AI content to hide a watermark).
This feels like the inevitable outcome of a STEM-only education system. Now people think there's a mathematical formula for picking the "best" words, instead of having to be thoughtful and creative.
I was also getting confused. No obvious marker to close card so I instinctively pressed back twice and got back to HN. Would try to keep exploring but I'll definitely forget and do it again.
reply