I don’t know why you’re getting downvoted. One of my coworkers who was constantly touting Chinese models a few months ago now only uses opus 5.5. It is a huge difference.
Unless new data has come out, we don’t know if they are losing money on subscriptions. We know that there is a big difference between subscription and API costs per token, but that alone doesn’t mean much. Subscriptions may be underutilized, or margins on API tokens may be very high. Both are probably true.
This debate conflates two senses of the word deterministic:
1. Repeatable: the same inputs always lead to the same outputs
2. Algorithmic: the answer is arrived at by a set of steps that can clearly be shown to produce output with some desired property.
(2 is not a textbook definition of deterministic that I’ve ever heard, but I think it’s what people mean here)
LLMs don’t usually satisfy definition 1. They could, in principle, but it wouldn’t matter if they did. It wouldn’t tell you anything about whether the outputs are trustworthy.
What people seem to want is 2, but 2 is, I think, a contradiction. There cannot be an algorithmic process that can turns a natural language prompt into a working program, because natural language is too ambiguous to specify an output.
Nevertheless I agree with the main point of the article. Non-algorithmic processes (like intelligence) can produce algorithmic processes (like software). After all that’s what human programmers do!
Yeah, 1, while the correct meaning of "determinism", is just not an interesting property. It's unfortunate that people use the word to mean 2.
2 is an interesting property and I agree that it's not possible for LLMs to possess it because it's a property of natural language itself, not the model. Not possessing it doesn't mean that LLMs can't be used to write software (obviously), but it does mean that using them isn't like using a compiler, which does have property 2. This is usually brought up in response to people claiming using LLMs is just another step in the process of increasing abstraction that led from punch cards to assembler to low level languages to high level languages*. That point stands: LLM use is qualitatively different to those steps because LLMs aren't "deterministic" in the sense of property 2.
There must be a better word for this, but I can't think of one right now.
Moreover, 1 is already guaranteed by a few providers and local stacks, for example DeepSeek v4 has batch-invariant inference. It actually could matter for reproducibility in certain workflows and I'm sure more providers will give these guarantees if there's actual demand. Keeping it reproducible beyond model/snapshot lifetime is harder, though.
It means that NNs don't (as previously thought) smoosh unrelated concepts together into the same neurons, which would have made interpretability very difficult.
For instance, if you had a single neuron that fires for "puppies" and "murder", it would be much more difficult to tell if the model is thinking something nice or something not so nice.
No evidence other than the fact that this has been happening steadily in all areas for many years?
You might have a point if the goal was to have LLMs that spit out a correct proof without chain of thought or tool use. LLMs + agent harnesses are more than capable of self verification and course correction.
Today’s current problems were all hypothetical several years ago. At that time people claimed that the “real pressing problems” were misinformation and DEI issues. If we pretend that hypothetical problems can be safely ignored because there’s “no evidence” that they are real, we will keep getting surprised.
As far as I can tell, neither of the frontier US labs have referred to distillation as "stealing", but someone please provide a link if I'm wrong.
They do claim that it violates their ToS, which we can assume is simply correct, since they get to put whatever they want in their ToS.
Given all that, I don't know what the fuss is. Are they supposed to not use the advances that were openly published by Chinese labs? The entire industry is built on a discovery made at Google, which was published openly. Should Chinese labs therefore not use transformers? Should US labs not try to prevent distillation of their models?
I don’t think there is fuss, just the author sharing the information and mentioning how they find it a bit ironic that US labs expenses can be reduced drastically thanks to the Chinese companies they continuously frame as adversaries
You’re technically correct, but the vast majority of software has never been built to the kinds of standards you are describing. LLMs are not displacing that kind of work!
The person they responded to refers specifically to "healthcare, finance, automotive, defense, power plans, aviation, manufacturing", areas where I'd at least hope that we aspire to understand what the code does.
"
SQLite is built using a DO-178B-inspired process. The testing standards for SQLite are among the highest for commercial software.
SQLite is open-source but it is not open-contribution. All the code in SQLite is written by a small team of experts. The project does not accept "pull requests" or patches from anonymous passers-by on the internet.
"
I worked briefly in heath care and the code is so brittle and so poorly understood that almost everyone is afraid to touch anything and instead it's just layers and layers of stuff trying to patch around existing code.
I know this is true. But I don’t think “Some parts of important codebases are black boxes. Therefore it’s fine if all of that code base becomes a far bigger black box” sounds like a good argument.
Also, there was probably some human at some point that had some understanding of what they were trying to do and why. The black boxes generally get programmed around after they long left but at the time they had bugs ironed out over decades. (Yes I know sometimes true slop is done over a short period of time and the programmer leaves. But I’ve generally seen the black box built over decades instead).
In many codebases when you see this happening it is because someone reported a bug and the dev fixed the narrowest instance they could. Typically if(some specific case) do_fix() else do_normal(). Over time this fragments meaning in the source and makes it harder to reason about until it is a giant pile of slop.
The thing the dev should do that often doesn't get done is to take the information about the undesired behavior back up to the design/spec level and rework the definition to account for it. This doesn't normally get done because it takes longer and it is conceptually more difficult than the patch.
This is something that LLMs can accelerate significantly and the result is a shallower gradient of technical debt over the long term.
I worked briefly in aviation and the part I've seen was very understandable and easy to extend and modify in understandable way. Some parts were hard, but by necessity. We also had very good tests. But maybe that one software was just a good exception.
I worked for a reinsurance company and their main pricing tool is a huge brittle excel file full of spaghetti VBS code. And yet, they manage to underwrite billions.
When the derecho came through Iowa and many businesses were out of power for up to multiple weeks, the very large organization I was working for at the time in the insurance space got to discover just how many of their processes relied on machines sitting under people’s desks. Business critical servers and processing just chilling on a PC under someone’s desk. Also tens of billions of dollars in revenue a year.
yeah, the llm approach is incredibly wasteful wrt pretty much everything.
Performance, RAM, Development (Tokens).
But it does give you surprisingly stasble rube-goldberg machines.
And thats basically what 95-99% of enterprises want from their software.
It annoyed me to no end when i began my career, but at this point ive accepted it and can definitely still have fun developing software with llms. As a matter of fact, as my perfectionism approach to software in my earlier years was never really appreciated... So i dont really mind the new MO.
I still occasionally hand write though, esp. at the dayjob where ive got super small token budgets while continuously being told to use more AI. But that's normal, employers usually give off bipolar vibes with multiple stakeholders wanting to advance each of their bonus package KPI of any given quarter
The vast majority of software is not that important. I don’t really care about easytag (which I use for flac metadata), but I do care about xterm and tmux.
reply