Everybody is converging on similar ideas, but the execution isn't super clean in a lot of "software factories" popping up. Coordinating through shared written state becomes obvious once you build something complex enough. Anyways, I have been building Pullboard (open source). The original concept came from needing agent coherency on a very complicated project.
3. Inspection: checking what we built against the blueprint (the proofs)
This research makes the inspection close to perfect. But an inspection only checks the blueprint, not our vision, so a house built from the wrong blueprint can still pass a perfect inspection. The other half is making sure the blueprint matches our intent and stays that way while agents build. I'm working on that, so I think this is complementary:
The author is wrong, and a software factory isn't a distraction.
People are thinking about this from a quantity angle, but the key is quality, and achieving a self-maintaining project at home. I built a distributed trading system that runs multiple strategies in US options and MUST be correct. So my goal wasn't to build huge quantities of software and pump out garbage, but to ensure correct behavior and proper fixes in the codebase with agents.
That's just a non-sequitor though - "this isn't a distraction because I built a thing" isn't actually an argument from anything but personal satisfaction.
This is why code is going to be automated, and the software engineer is a link between organizational intent and the code. This is what I’m working on.
Finding the holes in your specs under automation: time spent is returned 10x
Writing code: Human 15 lines per minute. LLM: 600+ lines per minute
Functional Validation: same time per iteration. Fewer bugs with LLM when properly specified => fewer iterations. Increased awareness of LLM escaping specification boundaries feeds back into increasing quality of spec writing => fewer bugs.
The difference is that instead of spending half your time understanding the business requirements and half your time writing code, you'll spend half your time understanding the business requirements, 5% of your time typing these requirements into a chatbot, and the rest of your time doing... ????
Some of the rest of the time can be spent making it look like your ratios are the same but in reality you just take the saved time and do whatever you want with it. The decision making and reviews aren’t happening any faster anyway so just take it easy and coast a bit
I guarantee there will not actually be saved time - the bullshit will increase to more than compensate, as everyone actually fixing things will be drowned out by bullshitters just pretending to fix things and creating more actual problems.
Do they matter? Yes. Should you use them? No. (Mostly)
This is going to be opinionated from an extreme "vibe code" viewpoint, but I think the future is "GAD" Governed Agentic Development. Designing constraints that agents must follow, and therefore different forms of frameworks. What the agents need is lines in a coloring book, the human is responsible for drawing those lines. So frameworks of the future should be focused on constraining an agent to develop properly, or designing what is and is not permitted.
> So frameworks of the future should be focused on constraining an agent to develop properly, or designing what is and is not permitted.
Frameworks have always done this. Make it easy to do the right thing, add friction to doing the wrong things. Swapping s/programmers/agents/ doesn't make this some brand new earth shattering idea.
If anything agents make the job of designing frameworks or leading architecture easier because they at least have the patience to read the docs even if they don't always have the working memory to follow it all. As opposed to most humans who both don't have the working memory and don't even bother to read.
Most days I am capable of running 15-30 agents across 5 projects; if I had the budget it would scale. Pullboard is more of a methodology, and looking at the components alone does not reveal why it works. It's not just "orchestration" it's more like a coherence, outcomes and judgement over time.
I'd instead say that for both it's really at 100%, and not any number in between.
Problem is that we can't define precisely what is good enough or when software is "finished": I mean, we are trying to do that with human languages, so it should not be a surprise.
Yes, LLMs are now similarly aware of the average context a human would be aware of, but for anything specific to the situation a human has better chances of resolving the ambiguity.
Been using the Pullboard workflow to build software that MUST be correct, and recently open sourced it. There was another HN link today involving agents role-playing and I can't take that seriously, no offense. (Love fun projects like this, seriously, it's cool.)
From my perspective, agents are great, they code better than most of us. But like all of us, they also make many mistakes or can't see how their changes impact the codebase. But that is a test-driven development mitigation along with some sort of scrutinizing process, "prove it". I'm actively working on this, and by actively working on it, I mean building serious apps that hit correctness walls, figuring out why Pullboard didn't catch it. To me, this is all about the process.
Tests are inadequate to catch bugs. Tests are like putting thin net in some little sections of a window, but property based testing is like something more like a decent net but not a solid barrier. And agents will lie, misunderstand or hallucinate things when asked to "prove it." And none of this fights off the bloat, performance issues, and decrease in maintainability. People keep thinking the process will save them from actually having to think through and put things together correctly. It's too bad that people are missing the satisfaction from personally building solid, correct things. And users are getting increasingly worse software because of this trend.
Entirely agree, also things can be logically correct and well tested but not the behaviour the user intended.
Even if that's down to a bad initial prompt, or lack of data for the agent to notice the edge case I don't see how you can ever engineer a better agentic solution unless as a human you're monitoring the output.
> I don't see how you can ever engineer a better agentic solution unless as a human you're monitoring the output.
I also totally agree with your response. But my point was that you can't engineer a better agentic solution. I'm arguing to never use agentic development because it is fundamentally flawed and inferior.
You are correct that it can all be correct in one sense, and wrong in the product (“what the user wants”) sense. Of course you need to monitor the output, that’s a given. You need to question the entire system. That is the frontier for engineers.
Not standing on the factory floor reviewing PR widgets.
reply