This LLM as compiler analogy comes up here and there and it’s not hard to poke a hole in it.

If I generate a piece of code with LLM, and verify it to be correct; can you trust the LLM output, with the same prompt, in a new session based on my verification of my code, with the same certainty that you’d trust LLVM output for the same source code in a similar environment, ran independently? If no, why?

Yea, maybe one day in the future, "the LLM prompts" will be considered the source code. But a lot has to happen first, namely the same prompt needs to deterministically create the same intermediate code. I wouldn't check in source code that compiles a different binary every time, and we should not treat prompts as source code while they produce different code.

I agree with you on this one; the compiler output remains the same if run the next day, but AI code might not. That is why you need a strong, independent reviewer who reviews the AI code and freezes it after verification, so that next time you do not have to ask the AI to regenerate the code.

That would be compiler writers, which seems unlikely.

Anyone can look at compiler output and verify its correctness.

That's a big assumption. We don't even have formal specifications for many popular high level languages so there's nothing to verify against.

The question wasn’t who can, but who does.

kek