I can’t really speak directly to your question because I don’t have LLMs write react, but I get great results from svelte.
At this point I don’t think it’s about training examples, once there is sufficient training data it becomes a problem of verifiability, ie how easily can the model check its work for success.
Because they’ve essentially exhausted pre training scaling and are looking to post training to expand capabilities, which is really just optimization via reinforcement learning against specific tasks aka bench maxing.
seems self evident if you read the new. You can Google it yourself, but here's the results from my googling - and this is just for the hardware. Double that to add personnel and corporate infrastructure
"To build or purchase the physical hardware required to store tens of petabytes of data and train a State-of-the-Art (SOTA) frontier AI model, you are looking at a capital expenditure (CapEx) ranging from $320 million to well over $1 billion."
Programming languages are what you build creative and interesting things from, building in safety into your tools is a good thing.
I wouldn’t want a nail gun that shoots in the wrong direction when you apply the incorrect amount of pressure. Software engineering over indexes on an obsession with the tools themselves over the things you make with them.
I’ve been landing on go for a similar reason, which was the realization that LLMs are in many ways just massive cargo culting machines. People will call it “quality training data”, but really LLMs will just reflect the norms and behaviors of whatever ecosystem they’re using.
I like to think go’s lack of magic and standard library will lead to lower maintenance burden over time, but that’s mostly just vibes so far.
reply