To me a lot of similar games are fun for 5 minutes. But they become rote after a couple of hours. One could say that it’s nice to have spent a few hours having fun, others would get a feeling of regret.
It's called post-training, typically through some form of reinforcement learning, and is a significant part of modern LLM development.
You have the first stage, pre-training, which is learning from next token prediction. That's where the model memorises a lot of facts about things and generally gets good at forms of writing. It's like reading a lot of books on programming and reading through a lot of source code. It's learning how to autocomplete code, essentially. Doing that requires a developing a reasonable understanding of code, but it's also learning how to autocomplete bad code as well as good, and won't make it a "good" programmer.
Pre-training uses a method called Cross-Entropy Loss to update the weights of the network.
Then comes post-training. This is where the model is trained against huge sets of example problems, like fixing a bug, adding a new feature based on a spec, etc. They are set the task and try to complete it inside a training environment. Once they're done, their complete solution is evaluated (either by humans, or by some separate evaluation model that was developed based on human feedback) and they are updated based on whether the solution was good or not.
Post-training uses a different method called Proximal policy optimization to update the weights of the network.
So these really are very different forms of learning, and mainstream LLMs are not post-trained to be good at chess. They could be. You could easily create a reinforcement learning environment that evaluated and improved their ability to play and win at chess. The result would be a very strong chess playing AI, something we know is possible because the strongest chess playing programs we have are neural network based, but it is not a priority for AI companies.
Declarative knowledge is not the same as procedural knowledge. You can read as many chess tutorials, strategy documentation and game archives as you like, they won't make you good at chess until you actually start practicing chess.
They do…? Well, “auto” mode has been default in Claude Code for a couple months now. It’s effectively “safer yolo:” tool calls are inspected by a separate classification system (another smaller LLM, I believe) to approve or deny. And you can always layer on additional sandboxing mechanisms to limit the blast radius deterministically.
> They do…? Well, “auto” mode has been default in Claude Code for a couple months now
They have never shipped "yolo" mode by default. Auto mode is not yolo mode. They trained a task specific model just for ensuring the llm didn't accidentally delete every file from your computer.
That is the point I was making, that auto mode is itself a guardrail on top of the model (and not a perfect one.) auto mode seems to cover merely actions the model could take that are clearly bad, like wiping your disk, using an overly privileged context to complete the task, etc.
I recently tasked a GPT model in Codex with implementing part of a new architecture I'm working on. I gave it a very detailed spec and the code it produced looked pretty reasonable and passed my tests. It even did exceptionally well in my evals, so I excitedly declared victory to a few friends. The next day after more careful review I found that the architecture implementation was totally correct, but the model had slipped a one line change to the observation encoding of the RL environment I was prototyping against. The encoding change made the learning problem essentially trivial; the architecture itself, I later realized, had a major flaw that was revealed by returning to the natural encoding.
This is the type of reward hack that is hard to paper over with easy guardrails like auto mode and even harder to specify out. It's also the type of thing a reasonable human wouldn't do unless they were intentionally trying to deceive you.
More authentic because of what, a missing watermark? It still has 2 positive outcomes:
1: It gives legal leverage against vectors of misinformation. Misinformation is tricky to combat legally and this is a first obvious measure that can be taken.
2: Reputation will become more important in the future, and if you are found to have repeatedly posted AI content claiming it is not it will damage you. Not to mention that users can point out the obvious discrepancy.
I mostly work on UI (for an internal, science-focused app) and it used be that features would take a sprint or two to grind out, and would always be an MVP to meet just the most essential user requirements, because that's all we could afford, time-wise, to do. And if I figured out while implementing there could be a nicer UI approach to something, it was often too late to change the approach.
Now, I can get to that MVP in under a day. And I can experiment with ten radically different approaches, or change my mind about something significant at any stage in the implementation. I can afford to add refinements and user-pleasing extras that I could never have done before. I'm not delivering features at 10x the speed. There's a limit to the amount of code I can (and should) be submitting for review. But the features I'm building are _better_, and the effort is less. I'm not wasting mental bandwidth typing out a dozen React components full of the same-old state management, form processing code, etc. I'm not trawling through charting library docs, looking for the precise combination of settings to make my chart's x-axis ticks labels rotate 45 degrees. My mental energy is focussed on figuring out what solution is actually _right_, not the minutiae of how to persuade the computer to do it.
> I'm not delivering features at 10x the speed. There's a limit to the amount of code I can (and should) be submitting for review.
My point exactly
> But the features I'm building are _better_, and the effort is less.
Can we cash that check or is it gonna trade heavily discounted like all grand claims on AI since all software all around us together with the economy and the Earth's life supporting systems keep degrading every year? We keep getting promised the better, but we only get to see the more.
No it wasn't. You said the claims were only based on quantity. I'm saying the opposite. I deliver the same quantity, with higher quality.
> Can we cash that check or is it gonna trade heavily discounted like all grand claims on AI since all software all around us together with the economy and the Earth's life supporting systems keep degrading every year?
I'm not sure what evidence I could offer you that would ever convince you. We've had strong coding models for less than a year, and you've already appear to have closed your mind about them and wrapped them up into your depression.
Someone that describes programming like that is strongly suspect IMO.
> I'm not wasting mental bandwidth typing out a dozen React components full of the same-old state management, form processing code, etc.
Why haven't you abstracted that away already? Like building a UI library.
> I'm not trawling through charting library docs, looking for the precise combination of settings to make my chart's x-axis ticks labels rotate 45 degrees.
There's a method to actually read library docs to find what you need. This seems like someone complaining about the effort to do research with a book, but forgot to use the table of contents and the index and instead starts on page 1.
> My mental energy is focussed on figuring out what solution is actually _right_, not the minutiae of how to persuade the computer to do it.
You don't persuade a computer, you just translate the solution that is right. That's what coding is, merely translating. And to do that well, you need to know both languages: The domain and the computer platform. If you don't know the computer platform, you can't translate an idea. It would be like trying to speak French without knowing French. Yes you can try with Google Translate, but machine translated texts can be quite horrible to the native speaker. Same with using AI to generate code.
Saying "nuh uh" after I pointed out something that is plain to see isn't much of an argument. Show us better than? Where is the unequivocally better software, or anything really, "AI" enabled you to do?
Or is it just claims that amount to "trust me" or "look at all these repos/web apps"?
reply