I've been getting into emacs somewhat recently and really enjoying it so far, though I do worry a bit about how much it can do and whether it will lead to any sort of "oops, you opened a project that had a thing that autoexecs or exploits a parser and now you're pwned". I know that's not a unique problem to emacs, but it feels like the enormous amount of things emacs can do gives it a bigger attack surface for that (and I don't know my way around it well enough yet to be confident that I've looked at all the places that might happen)

Maybe the solution is that I should just be running the entire thing in a sandbox in any case.

Actually, Emacs might be handling it pretty well. Safe eval of local variables is a thing. Of course, in your own code, anything goes.

I'll be very interested to read any followups about the part you described at the end about getting it to do more "teaching," both from the perspective of how we can use this to help newer devs out but also from the perspective of helping ourselves understand the software being built.

One thing I've been worrying about recently is the notion of comprehension debt and making sure the humans can still understand the system (so as to be able to do incident response or something if the LLMs are down). The slower, more methodical workflow described here should definitely help by allowing checks like proper docs whenever a piece of the architecture changes for instance, but I'm curious what else could be done to help the humans understand as much of the implications of what the AI is doing as possible.

Edit: having the code author and reviewer be completely separate like you described below is also probably pretty helpful for making sure changes are comprehensible from the outside.

Yeah I don't entirely follow why having had it generate an integration test you'd just throw it away afterwards.

It's for chiseling out the design, not verifying the functionality.

Very cool! I love reading about details like this.

I'll definitely check both of these out! I've been working on a design doc with my assumptions, threat model, and required security controls for an agent sandbox so that I can use it as a baseline when comparing different offerings.

Just a few days ago there was an article about how they lost the bluebird logo and the word "tweet," so it seems possible!

Between this and the nix/guix stuff, Farid is definitely becoming one of my favourite mad computer scientist bloggers, right up there with Justine.

(Or computer mad scientist? Whatever (mad scientist) && (computer scientist) is)

The way I've done it has been when there's a longer task, I give it a markdown document that has a plan with numbered steps, and then for each step start a fresh context window and tell it to update that doc as it goes with any decisions taken or deviations from the plan, or other context needed for future steps. If it gets close to the end of the context I tell it to summarize the current state and anything a fresh context would need to know.