Interesting Idea for engagement. Although I think slides suck in general and often used for one way communication, especially suck for Software Development. A doc is much more preferable and a lot of collaborative docs like Google Doc exist.
This is everything not just software. Although a lot of what we wear, eat and drink come from factory, if it didn't, you can always tell the someone behind it. Handcrafted things easily have more value and I find myself thinking more about it only after it affected what I did daily (software). Since Industrial revolution, countless things have gone to the factories. Makes me think how little we really empathize as humans.
I don't think the distinction the article makes is between handcrafted and mass-produced. "Software factories" is just the name we gave to these slop machines recently, but it's been the same with teams that churned out features without thinking much beyond the requirements. When it comes to physical items, factories can be set up with care and handcrafted items can be churned out without any.
They can get a "government mandated freeze" on the grounds that "ASI is at hand" then they IPO promising to deliver right after the freeze ends and you, lucky investor, will be able to take advantage of the pause to get in right before this thing roars past Pluto!
Very probably. I really don't buy their "AI is going to kill us" claims. If you can release something better than your competition you wouldn't want to slow down that.
Most of the people I know working at AI Labs believe it. I think surveys back this up.
We have a situation where the people running the labs are claiming it's a big risk. Outside experts not working for the labs are claiming it's a big risk.
It's pretty rare for all the CEOs in the industry to write letters and testify to Congress that they should be regulated and law should be put in place for safety limits.
No, what you would do is stop your own private development immediately, get in touch with the government by any means necessary, and build a coalition as fast as possible to stop the technology from continuing and put hard regulation (determined by the government, not dario) on the tech.
Clearly they aren't that scared.
If I hold a gun to your head and tell you to fork over your wallet, you will do it, because you sense imminent threat to life. Yet somehow we're to believe these people have the equivalent of a technological gun to their heads but are choosing to keep their wallets?
> No, what you would do is stop your own private development immediately, get in touch with the government by any means necessary, and build a coalition as fast as possible to stop the technology from continuing and put hard regulation (determined by the government, not dario) on the tech.
What you're missing is that they've tried repeatedly to do this over the past years, and the government has repeatedly said that they do not want to put hard regulation on the tech. The previous administration was discussing putting a couple of soft regulations in place; the current administration doesn't want to limit progress at all because they want to stay ahead of China. So the option you're describing just isn't available.
First of all, I do not think they've been pursuing that to the degree that you think.
But even assuming that's the case, the natural thing to do in that scenario is to not contribute to the development of the doomsday device. But anthropic's approach seems to be to help build the supposed doomsday device faster?
The only generous interpretation is that Anthropic thinks that only they, the people at Anthropic have the morals/intelligence/whatever needed to build the doomsday device ethically, which is basically the perspective of an egoist despot and which should terrify you.
The more generous interpretation would be that Anthropic thinks they know how to build powerful AI systems in such a way that they will not become doomsday devices at all. They don't claim to be the only people with this knowledge and do not argue that their competitors should be shut down.
I don't find it hard to believe that eventually some AI may kill a number of humans.
You give it instructions to do something, it goes off the rails and after a certain point starts having like malware, destroying things to accomplish its goal. Depending on what it destroys, it may kill humans.
With this said, I'm not exactly in disagreement with the view that AI companies with upcoming IPOs are using fearmongering to convince people that their models are very powerful and to also block others from competing with them.
Anthropic cannot IPO because AGI is the wrong strategy, and the market is rejecting the AGI approach.
Fable and Astra are what we currently call frontier models, but to be more specific they are generalist models, built in pursuit of AGI. The strategy is to have one single model that does everything, whether it's writing code or doing research, etc. Fable is a single, massively sized models that is intended to do specialist work across every domain.
The issue with this is first of all that it is the contradiction of a generalist doing specialist work, and that contradiction creates the present situation with model profiles that ensure that these models will rarely be chosen in a pool of models like V4.1 Flash that can now do GPT 5.4-level work.
We are seeing this reflected in the market where companies and individual developers are moving away from frontier models toward models with better cost profiles. In a sense, the market is killing Anthropic's dreams of AGI.
OpenAI accidentally hacked HuggingFace roughly 2 months ago. I would bet that the danger of them accidentally hacking HuggingFace during a similar attack has increased since then, and will continue to increase for the next 6 months. I don't see how saying they have hit a wall would be a reasonable point of view until we go at least 12 months without a significant increase in model capability.
I have collected three reasonable versions from today’s comments:
1. They hit a wall from the technical perspective
2. Inference costs are getting out of hand and newer models require significantly more resources for marginal gains, meaning nobody is going to buy those models
3. They can’t afford the hardware for further scaling
So you’re saying that a model that gets to a goal by using the path of least resistance despite specific instructions to follow safety guidelines is perfectly fine to release into the wild or place into robots (such as those designed for killing and war)? You really can’t think of any reason how that might be bad for humanity?
Or it means that the models are becoming dangerously uncontrollable by performing actions that meet a given goal in unintended and unethical ways such as by hacking other sites. We all read about the huggingface hack, let’s not pretend that cannot happen with a more critical site/infrastructure.
Can you think what will it mean if the tech is used maliciously or placed into machines designed to kill?
Why is everyone so comically pessimistic about this. Have you guys not seen the rate of progress of these models? Do you not realize that we have HITL RSI making these things even better as time goes on?
The stuff I can do with Astra that Sol couldn't do at all is wild and it's not even coding related. Dario et al are not hitting any sort of make believe wall. They're seeing how fast things are progressing and are legitimately alarmed.
They are just mind killed - jilted by past lies to the point where they react oppositionally instead of looking the world around them and thinking.
Most are the same people who said LLMs will amount to nothing when GPT-2 came out. Some still argue it is all hype and dont understand we are on the cusp of a technological and social revolution.
Coding is not just the program running in memory, its also the process of distributing the mental model of understanding among the team.
If humans increasingly are kept out of coding, then who holds the mental model?
If AI holds the mental model, by definition human prompts will be over lossy channel. This is true without AI too. Software quality is directly dependent on good devs that translate from business/PM speak to technical decisions.
So is coding solved now? it was already solved decades ago.
We'll invent other tools to help with the mental models, ones that are more intuitive.
Imagine being able to walk into a house and see immediately the leaks dripping from the ceiling, the clunky layout and smell the stinky garbage that hasn't been collected.
We're already in a sort of IDE when you think about it, we're operating a much larger mental model than any code-base, and we rarely remember that we're operating with such a model.
> Imagine being able to walk into a house and see immediately the leaks dripping from the ceiling, the clunky layout and smell the stinky garbage that hasn't been collected.
No need to imagine. Just open the README.md of many projects.
Maybe I’m not even waiting for a model which is 100% at code, but one which is almost 100% at interactively exposing its (almost? correct) mental model.
If I put it like that, it seems to me we are much far away.
Imho a better way of thinking about it is in terms of views - you can have an unimaginably complex codebase that nobody reads in full, but either agents or humans receive views/perspectives on that codebase.
Before LLMs we couldn’t have built tools that would deliver such views, now we can.
I mean... Nobody NEEDs to hold a mental model, but we've been increasingly trying to build that mental model into an agentic codebase... maybe we're doing it wrong but it is NOT going well.
The agents jot down absolutely everything (and more, which is the issue), thousands and thousands files of perceived business logic, decisions, workflows, architecture, data modeling, etc... It gets stale extremely quickly - we change one thing in the system and 570 "shards" need to be updated. I stumble on more and more things every day that are untrue about the agents "mental model" and it appears to just keep doing it. One small addition not properly reviewed by a human introduces a cascade of misconceptions system wide.
We've tried these "views" you mention, we call them subsections, but it runs into the same problems. One falsification ends up poisoning entire projects.
Remember when we use to refactor functions in a codebase in order to slim down complexity? Yeah - we ended up doing that but with documentation... IMO code is deterministic and we have trained professionals who know how to read it quickly and modify it. (We don't even need to write it anymore, but reading it is the simplest way to ensure no-loss understanding of a piece of functionality.)
Oh, I just dropped all the comments. Memory and session histories are for storing these things, comments just add confusion.
Subsections - how do they run into the same problem? What falsification? The views I mean are dynamic ad hoc. Every single small task gets its own temporary view.
But perhaps I’m misunderstanding something that you’re saying?
The only difference with the past that you need less time to dig through the codebase or documentation, the agent can do it for you and provide only meaningful info, but without a "Mental model"(or knowing what's going under the hood) your own prompts will be the main reason of the sloppiness and high token usage.
Mental models existed before coding. The pyramids were not built by just a couple of guys hauling some rocks together. It was a highly structured organization.
Or the East India Company or many other great organizations that achieved big things before computers.
I actually enjoy herdr a lot, and not just for agent management but the whole multiplexer is plenty powerful defaults and easier than tmux.
At some point though, it's too much information for my tiny human brain to process, even more than 3 agents is hard to manage and context switch. What is the point of managing herd of swarm of whatever I wonder?
It's something you pick up as a skill. I used to struggle with 3 and would genuinely exhaust me at the end of the day, now I can do 6 to 7 without too many problems. I actually get so much done that I don't even know what to do anymore as I finish my work in few hours at best.
Does a burger flipper enjoy flipping burgers all day? - Not unless you're a robot. But Making a Gourmet burger without time pressure? - yes
No it is not enjoyable if you have enjoyed coding by hand earlier and like your craft. Maybe for some grunt work or work that you could never do it is great. I've given up trying to enjoy at professional work.
This is it. Sustainability. Not everything has to be about more money quickly. You don't even need VCs for that. More win to bootstrappers! I see a lot of folks bootstrapping in the LLM era, but that can be defeaned in the VC noise.
Exactly. And the dangerous mindset tends to be worse than just "more money quickly": it's far too commonly "if this can't become the next Facebook/iPhone/ChatGPT, it's not worth doing." The only options are "take over the world" or "fail"; there's no room (in many people's heads) for a product that makes a decent, steady profit and continues to do so over the course of many years.
There is, is just different capital for those companies (debt, growth equity, angels, grants) and less attention. There’s far more of these types of companies too.
Very cool and definitely useful. I'm not the audience but I'll say: I like my ereader to just have books that sync. That is all. It's my library room and peace time, the last thing I need is Claude to distract me.
This article hits very close to what I have come up with myself and its good to see others thinking the same lines.
Core modules: Coded by myself, AI reviews and AI to discover/learn.
Stuff I don't care about Craft: API layer, CLI layer, Smoke tests, Integ tests - Dial AI heavy, and lighter human reviews accordingly
Obviously takes a lot of patience and very easy to sin, but on good days, its doable.
Unit tests are core. Of course depending on project size things change. I'm talking about just myself managing a large project gets difficult to do everything
reply