I haven’t used open weight models yet, but this would match my intuition given I haven’t experienced a noticeable increase in code quality this calendar year. If open weight models are already on par with January frontier models that’s already enough for me to automate most of the manual parts of my work (I.e. not “architecting”).
And once frontier models can architect (turn business requirements into engineered systems) then I guess no one needs a job because that’s the digital singularity.
Idiot directors and VPs don’t realize that the independence is what leads to the games being different enough to justify separate existences. Put the same teams on Halo and CoD and they’ll converge and cannibalize each other.
Or maybe they do realize this and are willing to sacrifice the IP at the altar of Moloch.
TBH it seems like the Halo teams have been trying to copy COD for a while (going as far as adding COD mechanics to the Halo:CE remake).
If they're under the same roof, my optimistic hope is that they'll try to focus on how to differentiate the IPs. But realistically we'll probably just end up with even more micro transactions
Even talented teams given lots of independence and a large budget can coast and meander during development, and more importantly not release the interesting stuff in a timely manner (see GTA6).
At least with the Netflix model they fund every director available, including independent and C-list directors, and release everything quickly, using quantity to mask quality issues. Bigco Gaming is addicted to the 5yrs+ in-development blockbuster model and it's not producing many hits and the public notices that sort of thing.
Eh, a lot of these studios have not performed well at all over the last 10-15 years. It seems like Phil Spencer was the type of boss that let these teams run wild and be creative, and all he got back was a bunch of late, over budget, underwhelming games.
Halo Infinite from 343/Halo Studios stands out in particular. They thought that game was ready to ship in late 2020 and showed it off on their summer showcase, where it triggered such an intense backlash they delayed the game for a year and it still came out with minimal content.
Halo and Rare going under Activision could be a good thing. But if you want to be cynical, they are moving all of these studios out of "Microsoft Gaming Studios" and into Activision, Bethesda, King, etc. to make it easier to sell each of those individually or spin off the whole thing altogether.
Yeah and they’re typically fired for incompetence, or their startup fails, or their product flounders but that’s okay because the company doesn’t need the product to actually be successful.
Meanwhile, the people who work on actual products that matter have a valid criticism that can’t be dismissed with “some humans don’t even manage to do that.”
Yes, including myself in the past. I’m grateful to be out of that situation.
Probably a vast majority of software written never actually gets interacted with by a user. A vast majority of VC-funded software probably never gets interacted with by a user.
Feels pretty great to work on a product that actually has a user, and that user isn’t actually the product (and that user isn’t being manipulated or exploited). Tiny, tiny minority of the software industry I work in.
That does not sound right, I would guess the most code is in enterprise and business applications and people have to use it no matter how bad it is. I would guess only a tiny fraction of code - besides hobby projects - ends up in the trash without ever having users.
I’m coming from the angle that the majority of software written is internal or B2B rather than B2C, and that most critical B2B software features are hammered out in the first couple of years of development while the remaining development cycles are for niche requests to maintain specific clients who don’t actually know what they want and then don’t end up resubscribing / even using the feature.
If you don’t think about it as a per-software criteria, but per-feature criteria, the vast majority of software features developed are probably completely ignored or even untouched compared to the “critical few”.
Very, very long tail distribution. The most executed 100 lines of code (maybe some Java class init snippet, or some Linux kernel snippet) is probably executed more per day than the per-day bottom 80% of all code combined. Maybe the bottom 99.9% of all code combined.
But that is generally true, most things in the real world have a few common cases and a lot of rare exceptions, so every software dealing with the thing will have a hot path and many additional paths that are rarely or maybe never executed. Or look at large applications like Excel, Photoshop or Blender, they have a million features but most user will probably never use even a single percent of them.
If you would fire all incompetent [1] software developers, there would be almost nobody left.
[1] There is probably a more fitting word, I am just reusing incompetence here, but that is not a really fitting description, I think. I would maybe say carelessness or something like that, but a single word is not going to capture the issue accurately.
Same with pretty much any other profession. Hunting for reliable and reasonably-priced HVAC, plumbing, handymen, etc. is a part-time job.
But the point OP was trying to make is that because LLM harnesses output code that is at least better than the worst, say, 20% of developers then we should be fine with it. Meanwhile, I’ve worked in shops where you have to be way above the worst 20% in order to keep your job or even be hired in the first place. And it isn’t some crazy lose-sleep-over-it, stressful requirement. It’s just that literally if you’re not good enough your work will be identified as a liability and you’ll be let go. It’s only happened to 3 people fwiw.
Does writing table schemas count as writing code? Is that architecting? Both?
I personally don’t trust coding agents to have enough context to write domain-specific table schemas, and I don’t have the patience to transcribe all of the context into a natural language prompt. If I ask it to, it’ll write something for sure, and maybe that can be a jumping point for me, but at some point I have to physically write what the columns will be.
You should definitely try it if you haven't already. Most of the models are deeply familiar with domain specific areas in ways that most of us aren't. In fact, I would say the problem is generally the opposite. If model size and effort is is high, it will over architect what an app needs. I find myself reigning in a giant email notification system with subscriptions and "channels" and other such nonsense when sometimes you just want a simple one off.
Yes, the LLM knows the definitions of the words in my domain, that’s clear and obvious.
No, the LLM doesn’t know our product strategy and why certain things matter and certain things don’t. That’s very much what I get paid to do. There’s not even agreement within our team about what path we should take through the domain-product space, there’s no chance in hell that the LLM will choose a profitable random walk through that domain-product space.
If it were able to do that, then AGI would have already been achieved and we’re only compute power away from OpenAI or Anthropic making the marginal utility of any piece of code $0.
Yeah, it's not just over-engineering, it is just that generic systems that look like everything else usually don't solve a business problem, but I think I would be less dismissive of its world model in general.
LLMs are not a "random walk", they take in information and they explore the space according to the way they've been instructed.
Humans aren't magical, so if people can figure out what maintainable code is and how to write it, so can a non-human. Furthermore, humans are rather unlikely to be the optimal form of intelligence for software architecture and engineering, so it's almost certainly the case that if humans can figure it out, some non-human could do it better. That it's not yet solved by models suggests that it's not easy and maybe even not practically achievable with current tools. But that's not the same thing as not solvable.
> Furthermore, humans are rather unlikely to be the optimal form of intelligence for software architecture and engineering
I like this framing. Humans invented software (and engineering in general) as a means to solve problems with methods that work best for us. There may be entirely different, and parallel, problem-solving methodologies outside of human best-practices.
I have not found it difficult to get these systems to write maintainable code. You're probably correct that most people don't care about this so they don't bother to do so. But IMO, this doesn't seem like a problem that is crying out for a solution.
> Yes, the average distance per run may be higher for the 5 companies, but it won't be 5x.
You contradict yourself at the end of the comment. Yes, it could be estimated a linear equation (mx + b) where m is the cost per house and b is the cost per route, roughly. And then you could have a system of which equations, where each row in the system’s matrix corresponds to an additional company.
The optimal result is probably not 1 company, but it’s probably not 1 company per house either
To be fair, after visiting some trade shows recently, a lot of these “parasitic” LLM startups are not very convincing from a business-use case perspective. (They’re not really parasitic, more like a remora attaching itself to a shark, the shark being OpenAI or Anthropic).
At IMTS at Chicago the skepticism amongst the visitors towards AI and pure-AI startups was at an all-time high. More than once I heard, “Oh yeah I visited that AI booth and they couldn’t explain what value they would add to my business.”
So in that sense, while MCPs are “winning”, I also see them as a significant part of the AI “bubble”, specifically solutions looking for problems backed by VC money.
And once frontier models can architect (turn business requirements into engineered systems) then I guess no one needs a job because that’s the digital singularity.
reply