> Seems to me like the big winners will have know-how and some money to throw at AI.
What I find interesting about this is that Anthropic should be better equipped in terms of talent and token accessibility than any other company. Yet, it seems that this doesn't actually provide a fundamental advantage in terms of writing software yet. If I compare Claude Code with Pi, then I personally would prefer Pi over Claude Code, but even if one sees it the other way, I don't think there are many serious people who would say that Claude Code is by far a better harness than Pi. But Pi has a handful of people working on it with, from what I've gathered from reading Mario's and Armin's tweets, Claude Max subscriptions. So it doesn't seem as if throwing tokens at a problem is a major differentiator in creating software as of today.
He's talked about this before. I don't remember exactly what those apps were, but they sounded like reasonable tools one would want. Thomas is a competent guy, but I think he's vastly overestimating the agency and competence of normal people. They don't have a product-oriented mindset; they might get annoyed by a bug or slow loading times, but they're not good at coming up with features or entire apps that they would want to use. Even if there are people who can do this, they don't want to spend their weekend thinking about a detailed spec for an app they want. They really just want to consume.
People could already vibe-code simple products today, but there's hardly any non-programmers that do. This is even true for businesses. For the last year people have been boasting about canceling their SaaS products, but nothing seems to have changed. SaaS stocks got hit by the ai hysteria in January, but many are back to previous levels and some even higher. There's an Indiehacker on Twitter whose product is a simple stage timer. It's not a hardware product. It's literally just the software for a stage timer. One would expect his business to have died already, but he's apparently doing completely fine.
Does anybody actually know whether there's a limit to the complexity LLMs are capable of dealing with in a codebase? It's very obvious that they don't write code that is suitable for people to understand it (and it's gonna get worse and worse the more RL is used to train these models), but if there isn't a point at which LLMs also struggle due to the complexity they introduce, then I'm not sure it really matters anymore for a large part of non safety-critical software. I really hope there is, because steering them is, I feel, one of the last competencies through which I can still add value, but is there actually evidence that these models struggle more with poorly maintained code?
> I really hope there is, because steering them is, I feel, one of the last competencies through which I can still add value
Something as simple as output length is a hard linear floor for productivity, even putting aside the obvious context problems that you're intuiting, and it's far from being the biggest cost that arises from steering skill. Learning to make a smaller, faster model do the same work with less tokens is a technical domain that a lot of people don't seem capable of learning. I'm not just talking about "context engineering", but learning how to fine tune, post-train, create better harnesses, design inference setups, etc. If we're both using AI, but I'm beating you to market every single time and with a better product, what is your AI usage actually buying you? Yes, competency and skill is this meaningful right now, and it's highly technical. Not the least of which because you know how to describe the problem in way that gives it a smaller solution and requires less iteration.
Most of the labor who understand the technology enough to do those things lives at the companies selling you these services, but you can absolutely learn to do these things yourself right now. It's actually really fun! A hell of a lot more fun than fucking prompting that's for sure.
Where we're at, I would equate it to the early mainframe era where the programmers came with the computer. I'm placing calls that we follow a similar track and the two will end up decoupling, that "model engineers" are going to move in-house. OpenAI will have a ring to it like IBM does today.
Yeah, exactly my experience. Especially ever since Fable came out, I felt paralysed, many times, about how many decisions I have to hand it. Which is 10x harder when you handed it the reins to build the PR and you practically only knew the compressed initial problem statement. Then it stands up a draft PR and says: "Decisions you owe me: [insert bulleted list with 8 items each a paragraph long]" and you are like: "...oh shit".
It becomes a very heavy and difficult exercise of it walking you through the implementation and the judgment calls it had to make along the way. VERY exhausting.
There's a silver lining though: you do get to gradually clarify a proto project spec and various requirements, but boy does it take time and energy to re-contextualise when the bot tells you that you should make decisions.
Well, of course Anthropic employees would say that, since they likely do the same. Claiming that your primary competitor doesn't engage in a certain malicious practice is supposed to make it look as if there's no way you would too. If somebody even says that about their competitor, then surely there must be truth to that, otherwise you would never give credit to someone you're opposed to.
Is it really interesting though? It's essentially just vibe-coded by people who are unqualified for this kind of work. One of the authors claimed that what qualified them was having worked on a large-scale postgres cluster; they never actually worked on databases or compilers.
Have you considered the possibility that you're simply not as good as the experts, and that your experience of LLMs being capable of performing your work up to your standards doesn't imply that experts are necessarily in denial?
Being fairly at expert level in a "solved" domain (for some definition of solved) but also having near-expert level proficiency in a non-technical as-yet "unsolved" domain and watching the process repeat there (and watching how people react as inroads are made progressively deeper) is basically my own standpoint. But I am intentionally using "solved" and "unsolved" very loosely here: you can get stuck in the thicket of arguments about verifiable domains, what it means for a domain to be solved, whether a set of evals can tell us something has been solved or not, and so on. One way to avoid that (as my original comment pointed to) and focus on what matters for us is to look instead at the effects being produced in the work process and on the division of labor as a whole.
What I find interesting about this is that Anthropic should be better equipped in terms of talent and token accessibility than any other company. Yet, it seems that this doesn't actually provide a fundamental advantage in terms of writing software yet. If I compare Claude Code with Pi, then I personally would prefer Pi over Claude Code, but even if one sees it the other way, I don't think there are many serious people who would say that Claude Code is by far a better harness than Pi. But Pi has a handful of people working on it with, from what I've gathered from reading Mario's and Armin's tweets, Claude Max subscriptions. So it doesn't seem as if throwing tokens at a problem is a major differentiator in creating software as of today.
reply