I didn't downvote, but the idea of nationalizing the second largest publicly held company on earth is ridiculously extreme. It's very very far outside of the American overton window, to the point of not being a realistic outcome.
If you were actually involved in that work pre-LLM, you know it didn't go in a circle. Most projects died at training data and problem definition. That isn't the case with LLM applications which can ship with zero data collection and labelling effort.
Now you can iterate on your 20 ticket categories in days against real workloads and not commit to training an efficient model until you've proven the prototype in production.
although in our experience, synthetic data != real data. You need to be quite careful about the distribution (or just try to get public real data, or label stuff yourself)
In this case the training data is having the expensive model do the task on your real data set and using its output to train your small decision model.
The reality here is even more idiotic: someone criticizing a company for a terrible strategy they aren’t using that wouldn’t work if they did. According to the critic!
I don't think it's profit, I think it is genuine functionality. I think that they build features for their internal users while having no token concerns. And this mindset of using LLMs to solve product problems leads to useful features (e.g. agentic auto-approval). Then there is a bit of an issue when they try to ship a genuinely useful feature but now token cost matters.
Of course it is for profit. Tokens cost money. Just the energy cost alone.
I don't understand this "analysis". Why would it not be about profit?
"Features for internal users" makes no sense here, as it does not apply to the case. "Useful features" is also debatable - LLMs providing user data upstream to Anthropic is a use case for Anthropic, but less so for downstream users and token burners.
I don’t feel like reality is so interesting that CC devs are sitting in a room theorizing how to extract the most profit from their users.
I’m much more inclined to believe they just don’t _care_ about cost, and profit is an ancillary benefit to their functionality above all else.
Just recently they released auto idle-compaction for large sessions about to miss the cache window. If they were so profit driven I can’t imagine they’d have built this to begin with, that’s free tokens from your perspective.
A smart company would A/B test changes and choose based on lifetime value vs. average session value. The compaction is going to happen anyway, does idle compaction increase the length of a session or the amount of sessions per user?
You’re making a straw man argument out of the nature of what jobs are for, of course they are to help a company make a profit.
I don’t think the CC’s teams have hidden motives to optimize for token inefficiency as the OP I replied to implies.
A reality where a secret cabal of CC engineers are trying to find the most token inefficient ways to build the best harness on the market sounds quite interesting to me, but not realistic.
Is doesn't even make sense, the limiting factor for Anthropic is GPU count, not profitability. They want more adoption at fixed GPU count which demands higher token efficiency.
I highly doubt it's the case that it was purely driven by profit motives. I think it's much more likely that Anthropic engineers treat tokens in a similar manner to how we treat memory and compute.
Older programs were MUCH more memory and compute conscious because memory and compute were actively limiting factors. Anthropic engineers probably don't feel the token efficiency requirements nearly as much as we do because they are working with the equivalent of a super beefy dev machine.
Not about profit in the sense that nobody sat down to say "let's add a feature that looks useful and burns more tokens so we make more money". For status update thing I can believe that.
> Of course it is for profit. Tokens cost money. Just the energy cost alone.
That only counts if people wouldn't find some other way to spend tokens anyway. I regularly have to pace my usage to avoid hitting the weekly limit on the 20x plan; I have plenty of things I could automate.
Jeavon's Paradox means that until all possible uses are saturated, the cheaper frontier AI becomes, the more of it they can sell.
Hahaha good one. Profit is the whole reason of existence for for-profit companies. Sure, companies might start with some ideologic grain of thought. But as soon as you reach a certain size, all that goes flying out of the window.
I agree at the top, and it certainly trickles down to some degree. Still, a lot of the everyday engineers who actually make the stuff at every place I’ve ever been just care about making things that are good.
You're taking that phrase out of context. If you look at the parent and grandparent comments of the one you originally replied to, it's pretty clear that "the answer is profit" was pointing to the insinuation that Anthropic is deliberately causing Claude Code to waste lots of tokens for no reason other than short-term financial gain, and that's what solidasparagus was disagreeing with.
even if their motive is "make our product better", that is still just the profit motive with extra steps. the reason to make your product better is to make it more competitive so that... it makes more profit long term.
But would you argue that it is neither about short term nor long term financial gain? Cause that was my point. It really always is about profit one way or another, in my experience, with barely any exception I can think of.
Neither are compilers? But you still trust your JIT compiler. Determinism is unimportant, likelihood of correctness is what matters. Plus agents are deterministic, no?
Even if we accept your premise that the likelihood of correctness of more important I still don't think that the two scenarios are similar. I have seen more incorrect outputs from LLMs in the past week than I have seen incorrect outputs due to compiler bugs in my entire career. And without determinism there is no way to be sure that ratio will ever change.
I'm not sure. That is also my experience, but I'm not completely sure that isn't my fault and the compiler comparison would be totally valid if I just used the tool better. When I actually write fully fleshed out specifications and have the LLM handle the implementation, I rarely encounter bugs.
I don't think I've seen a Sol 5.6 or later model fail at a task with a verifiable outcome.
It's when I try to do too little of the defining that I get LLM bugs. In which case maybe programming hasn't changed that much, LLMs really can be reliable compilers, and we're all just being lazy.
You can debug a compiler. If an LLM writes code you don't like, how would you debug that? You wouldn't. You'd just ask again and hope it does a better job.
Not 1 in 1000 software engineers has the ability to debug a compiler. And you can absolutely debug agent failures, but similarly almost no does. In both cases for the mast majority of engineers, they are black boxes.
Could you go into more detail about how you'd debug LLM output?
I don't really care about what "most engineers" do. I did phrase it lazily but I am trying to point out something fundamental about LLMs and compilers.
You look at the inputs to the model at the trace level to root cause context construction vs model deficiency. You collect examples of the failure cases, build eval sets, and run ablations. That's black box debugging, but systematic debugging nonetheless.
I mean debugging within the model, not the harness. Harnesses are software like any other.
The model itself is more or less a black box - if tokens come out that you don't like, it seems like there's no equivalent to "this LLVM release fixed a code generation issue on platform X relating to situation Y".
I'm not sure why people are so motivated to say these are the same thing when they are clearly not.
And yes, rigorous approaches can be taken to systems built on and with LLMs, as you say. That doesn't make them the same as compilers.
Why is an LLVM release any different than a frontier model release? Well known, concrete issues get identified, evaluated against, RLVR'd on, and a new release comes out that doesn't have that issue. Or you can do that process yourself on an open-source model if you have the resources.
LLMs are not magic things that are out of your control, they are simply complicated. But there are concrete issues that are identified, people sit down and decide to fix it, build evals, run ablations, and they produce a model without that deficiency. It happens every day.
Would you reject the characterisation of model weights as a black box then?
I suppose if LLVM's source code has a similar size in bytes then it would be similarly difficult to work with. However, it would still remain source code readable by a human, not binary data with no semantic content when examined plainly.
Complete comprehension has never been a thing. Unless you were working solo with no dependencies, you never completely understood what was being shipped. I don't think an npm import vs an LLM implementation of a well-scoped, well-abstracted library is any different.
At agentic coding velocity, tech debt becomes obvious much more quickly. Empirical evidence is a bit of a ridiculous bar to hold for software development - we are as quantitatively blind now as we have ever been - but at least for me the anecdotal evidence is very clear. Today's agents hill-climb and their software architecture is pretty reliably bad without a human in the loop.
And it's debatable how much better they will get quickly. Verifiable rewards for maintainability of software is, I think, possible but very hard.
You should read the report, it is interesting technically if you ignore the fear-mongering. It distributes the model's layers over multiple compromised devices.
And considering typical WAN latencies of 20+ ms, what’s the expected inference rate, 1 token per month?
Maybe I did something wrong or unaware about the tricks to make it fast, but last time I tried to play with exo and distribute processing over a few home LAN computers to see if I can run models that don’t fit my 3090, the penalties were brutal.
Ultimately AI spend has been one of the largest infusions of funding into the small academic mathematics community in years. Just raw token spend alone probably dwarves whatever research budget increases have gone to that field in the past 5 years. That should tell you something about the economic value of that line of work
Like in any job market: generally you can easily get jobs most people don't want. Problem is that usually the reasons most people don't want the job are also the reasons you don't want the job. Sometimes (rarely?), it's not like that, and that's when you hit the jackpot.
But you've just said that people want to coast toward tenure. My point is that it's extremely competitive position, not something you can coast to or that it's job people don't want. Something tells me that you actually DO talk about stuff you don't know about.
reply