When comparing it to math, this always felt to me like a clear limitation in philosophy, though. Just look at how philosophers try to argue about "logic", compared to mathematics' elegant descriptions.
That would be one definition of "source code". But then there's also the definition by the Free Software Foundation as "the preferred form of the work to making modifications to it". As prompting LLMs becomes the preferred form of making modifications, I wonder if a lot of open-source repos will unwillingly become "closed (prompting) source" over time, with intricate legal consequences.
I still don't know if I'd agree with you. I think:
> the preferred form of the work to making modifications to it
Implies what you point to when you want to know where to make changes.
That wouldn't be a prompt - you might use a prompt to make changes, but the thing you're making changes to, certainly at present, is your java/python/rust++ /whatever codebase.
Lots of projects, even fully vibecodes ones, don't push and include prompts in their repos. But they all push the java code (sticking with java as an example for the sake of argument).
I can't imagine that working on a project where the java code is omitted, and people make changes to the software just by editing the prompts would go well. It's an interesting thought experiment, but I think you'd find it very hard to work on any kind of team where that was the practice.
> Implies what you point to when you want to know where to make changes.
Where do I point to when I want to know this? On a large and mostly LLM-generated codebase, I might dig through the code myself to find what needs to be changed where, but I would probably prefer to ask an LLM to do this.
Now if I had access to all previous prompts that were used in generating the codebase, it might very well be the case that I would prefer to work by changing those prompts and generating the whole thing again. In that case, the LLMs would basically turn into a compiler and the prompts would indeed become the new "source code". This is clearly not (yet?) a common approach to software development, but I wonder if it may become. Or should.
If one uses LLMs to generate the code, and others need LLMs to read the code, isn't this awkwardly close to someone publishing only compiled binaries and others then having to reverse-engineer these binaries in order to know where and how to patch them? Maybe the handicap is less grave, as the difference in strength of LLMs understanding code vs writing them isn't as huge as of decompilers decompiling vs compilers compiling code - but I think this difference still exists. It's still an advantage to have knowledge of the "secret" prompts that were used in generating the code.
I recently had to fight slop responses for months to get a simple bug report through on a public services website. Couple of years ago, I'd have received a "thank you" mail after a day, and the bug would've been fixed by the end of the week.
They have a well-specified task and a not-so-well specified expectation of some actions being "wrong". And training data that likely contains more examples of extreme stubbornness than cases of politely giving up. What to expect?
Agreed, "alignment" is also anthropomorphizing too much. What would be a better term? "Do what I want you to, but what I don't care to specify" (because doing so would miss the point of using AI in the first place)?
fun fact: the word compile also had the same trajectory - the same trajectory you complain about here with LLMs.
here's the reality: words change meaning. they are fuzzy. and when the words take on new meaning while just requiring a simple extension (in this case a computer doing a remarkably similar thing), complaining is just futile and childish.
I don't think the best path towards interoperability is to run after those who based their entire business model on making interoperability a problem in the first place.
If I were to create an office suite that cares for interoperability, it would be based on markdown and CSS, viewable in the terminal, one could `grep` and `sed` and `git diff` through it, LLMs could read and write it natively, ... But I'm afraid it'll take another 40 years until we're there.
reply