Not from my experience. Missing 10% were caused by medium effort and that agent was not able to do play tests. So it is just testing and issuing additional prompts. Nothing hard. No thinking required.
It would be nice benchmark for models. Something like generate web browser from web standards in rust, c, and a language of your choice. An automatic judge would score the results by amount of found vulnerabilities and functionality. We could also see if generating code by LLM in rust makes it safer or if language doesn't matter anymore. And you know that it will work eventually, because AI labs need models capable to write new fully working models themselves to reach AGI or whatever you call it.
The narrative has a point. The LLMs are getting very good at this task. I got 95% of functionality by using LLM to port our Qt desktop app to thin web client and web server. Astra medium, two prompts, 2 hours of LLM processing. Each of two LLM plans had about ten steps. The missing step for porting of these apps is to test against the original (ex. Ms office) app in the plan.
Very interesting article and great job! But does it also provide enough information so that someone else can replicate your results or is that a secret sauce? The article mentions some prompts, tweaks, skills, etc.
The majority of gamers do not care because the games being released today aren't worth saving. They are all symptoms of the same root cause. Quality went down immensely in the 2010s.
This has happened to other media, and it was inevitable that it would happen to games. What eras of movies or music do you think of when you consider those collections? Why would gaming be different?
The point the article is making is that people assume that a game having physical media means its fully functional - but that's often not the case. Most games made recently rely on access to centralized servers to make the game playable in a concrete way. Emulation will help - but unlike previous generations having physical media won't be enough.
No, the opposite, the only app which is save is the app you never ship (stays in a cloud). No binary or source code available to search for exploits by ai.
reply