I've found that current models will almost automatically sketch out some things in Python (or another suitable language for the type of task, or an implementation language you've already stipulated) while proposing a design, and you can go through a conversation of "what if we did xyz instead?" and at some point "okay, let's try writing a full prototype" (perhaps tacking on a bunch of decisions here so that things are resolved enough to work with).
But that's at the scale of <1kloc. Might be different if you're trying to do a bigger feature all at once.
I suspect that's a problem both with human read/written code, and LLM written code -- larger features should be chunked up into more manageable to understand and reason about functions, and assembled at higher levels.
Guiding the LLM (or humans structuring/guiding juniors) to shape the abstractions help make the overall feature and reliability of the code much better.
But that's at the scale of <1kloc. Might be different if you're trying to do a bigger feature all at once.