Yeah: in a world where I am coding, I can put in the effort to make the software work the way I think... to use an LLM, I largely have to think the way it wants to work / converse.
I get it but not every editor/IDE was vim/emacs. Some tools were more or less rigid and today you still can have malleable workflows through custom workflows and harnesses.
It's not like you need to limit yourself to Anthropic and Claude Code. I myself developed "rik" a harness that can modify only a single file and hook up blazingly fast models for it to be like wide-context autocompletion.
1) I really wasn't talking about an editor at all: I was talking about the libraries and frameworks you use to program, so that the way you interact with the problem space is the way you think. I mean, even as someone who has almost exclusively used vim for the past 25 years of coding, if I went back to the prior 10 years before THAT of using Visual Studio, I'd have described this no differently.
2) In fact I have never used off-the-shelf coding assistants (Claude Code, etc.) and have only used my own harness; yet to me the issue is that the model itself often likes to think about problems in ways I find frustrating and unhelpful. The issue isn't at the level of how diffs are applied (that's commodity) or how context is managed (I can fix that) but in how problems are described, how abstractions are managed, and the language used to discuss them.
The entire concept of copyleft and free software depended on copyright: it isn't "take my code and do what you will", it is "here is a very explicit contract that demands you credit my work and even requires you to contribute back in return for my efforts". And like, maybe a world with no copyright at all is better! But that isn't the world we are in: it is no more legal for you to have an AI read my code and then port it into your codebase (which could involve training as an intermediate step or could merely involve the AI searching for it and loading it into its context) than would be for you to directly read my code and port it into your codebase or even for you to utilize a deterministic tool to read my code and port it into your codebase; you only get to do those things if you retain my license on the result and abide by its terms.
FWIW, many of us who do open source would also find that hypothetical world in which the idea of copyright is abolished rather enticing... but you pretending that is this current world whenever you want to steal our code whilst copyright still paints our interests in using and altering the code from companies that have always been in fear of our community and our licenses (Apple, Microsoft, etc.) as theft is not merely a bit shitty... it is kind of evil/cruel.
It depended on copyright as the legal structure to uphold itself, a means to an end, not the end itself. As another comment in this thread nearby says, the point was to support the four fundamental freedoms and ultimately a world without copyright on code at all.
And on the first part about AI reading your code or training from it, it does seem like it's legal now, while you as a human reading and porting is not.
Why do you believe that having an AI read my code and port it into your code base is "legal now"? Let's call that question A. This is a very different question from B whether it is legal to train a model on my code, which is also still a different question from C whether it is legal to use the model you trained on my code to generate code that is in any way similar to my code into your codebase.
As far as I know, A is still illegal, and nothing at all has changed that situation; if it did, it would be shocking, as it essentially undermines the entire legal framework of copyright, and it would be making national news and destroying -- not harming: entirely wiping out -- major software companies.
The only thing I think people have really litigated in the courts is B (as the obvious defendant is the juicy maker of the model rather than the users of the model, whereas they might be able to dodge culpability in C and even A), with the result that it is very likely legal... but, while that might be a bit frustrating, it was never shocking: a model is a highly transformative use case vs. the original code fed into the training.
However... B being legal doesn't actually imply that C is legal -- even if a ton of people fail to appreciate this distinction -- and C is where the war is starting to be fought. Instead of software, I think the most interesting avenue for this to become scary for people assuming this is legal will come from lawsuits involving the music industry (which tended to lead the pro-copyright cause in many of the skirmishes I was involved in while working on device freedom issues).
That said, people really focus too much on C, hoping for that grey area in both the law and in their analogies, when what developers are actually doing now is often A: the model has become an agent that uses tools, has access to the Internet, pulls code off my got repository, and is essentially just running a non-deterministic transform on that data sitting in its context... I have heard nothing that makes it sound as if A is "legal now".
This position is trivially incorrect as it makes the entire concept of a login shell impossible in the first place, as you have just described the regime in which login, su, and even basic tools like ping (which have the permission to use raw sockets and write ICMP even though we would never let any of its parents do so) operate.
Terminal (and sshd) are special, in that they actually do end up with inherent transitive permission to their immediate intended child processes, as those children are nigh-unto universally (due to the entire idea of what terminals do) designed to be operated using text written to their input, which Terminal (and sshd) can man-in-the-middle.
But like, that doesn't generalize: it certainly (and obviously) is the case that I can have a process that is allowed to write files to disk that I would be happy to let you run even if I do not trust you to write to even those same files (as you will corrupt them), much less any file on my disk, as I merely need to trust that application to not give you the ability to do arbitrary writes.
The better idea here is that processes are their own form of encapsulation, and just because they can do something doesn't mean that they expose that functionality. You thereby must limit what tools can be used by a parent -- so almost no one gets a generic "exec" permission: you get a whitelist of tools and arguments you can utilize -- but the result doesn't look much at all like the permissions on subprocesses needing be a subset of the parent.
Yes, and macOS entitlements work similarly, as you know. I was interpreting “what it expects its children should be able to do” as a single rule for all child processes, like in OpenBSD pledge() which the parent seems to be referring to. Sure, such a rule could securely be greater than the parent’s own permissions if you strictly limit what processes can be executed with what arguments.
Actually, there’s another exception to my previous statement that I should have mentioned. For mitigations that are meant to prevent an attacker from turning memory corruption into code execution or code-execution-like control – such as restrictions on mapping memory executable (at least that’s one purpose for such restrictions), or memory map lockdown – the mitigations’ effectiveness doesn’t depend on whether they’re applied to children. Those mitigations are sometimes treated as part of the sandbox system, sometimes separate.
Reducing access to a weapon/tool behind any variety of roadblocks because you think the weapon/tool is dangerous is not the same as shifting blame off of the person who did the crime. Saying we don't want people to be able to buy guns without limitations or even saying people should not be allowed to make or sell certain kinds of weapons/tools is not the same thing as saying the person who jumped through the hoops (or ducked underneath them) is not liable for the crime they committed using the weapon/tool. It is certainly the case that no one, at the end of the day, claims the gun is to blame for the murder a human carries out with it... some people will claim maybe the humans making or selling the gun share some blame, but it would be a fundamental attribution error to blame the gun itself.
I think part of the problem -- answering all of why people are somehow OK with this, and even why the AI does this in the first place -- is that the majority of software developers never got to the point of understanding any of this: the lack of understanding how to carefully structure data surrounds a question they don't even know how to pose, much less answer, and so "write some tests then incrementally try to make them work without breaking any of the existing tests" is the only way they know how to develop at all. In the end, that means that, with the current state of the art (which might change, of course... potentially quickly), your strategy of treating the AI as a junior engineer who fundamentally isn't ready to do your senior-level architecture job makes a lot of sense.
BTW, as kind of a follow-up to this, I think the most important finding to report--for those who only use one model, as many people seem to--is just how much more often Claude seems to be extremely confidently (and even insufferably) wrong than any of the other three big models (all of which I use quite often... yet I only pull out Claude when I've given up hope in a problem and are looking for out-of-the-box brainstorming).
And like, it does this despite it speaking in extremely dense math, which both makes it sound correct and requires a lot more effort to prove when it is wrong... yet, it isn't actually correct more often, and so that time sink just isn't worth the benefit. I then think many people--including people who can speak math (as can I)--just stop bothering to check everything, as if you come across a human who speaks like this it probably does correlate with slow and careful thought that helps prevent errors.
Instead, Claude has the mistake rate of a somewhat accelerated beginner impossibly combined with the language of an expert professor; and we as humans just aren't good at that combination: it becomes very dangerous and makes it take longer to spot its egregious mistakes and trained-in biases. If you have to use Claude, I thereby claim you really need to have a team of not-Claudes to help insulate you from this, and Gemini (while being a bit senile) is a lot more collaborative and approaches problems in ways that makes it harder to get tricked.
(To translate this into more of an engineering analogy: Claude always feels to me like the engineer who put more effort into learning how to program in functional languages than into how to actually develop working code, and then confidently presents you answers in Haskell or Lisp that never quite work. To find their errors is then very costly. In contrast, Gemini feels more like a Java or Go developer who knows they are a cog... that's helpful! <- Which maybe just goes to show that AI has finally turned me into a manager, omg.)
Doesn't it feel like the exact opposite? If you care a lot about the AI being a vengeful god, you do not do what OpenAI and Anthropic are doing. These companies only pay lip service to the idea of that aspect of AI safety in the hope that they can use it to regulate open source AI out of existence.
Instead, they are focused on stuff like "can I ask the AI to help me build a nuclear bomb" or "is the AI willing to generate pornographic stories", which is neither trying to protect us from unleashing a vengeful god NOR preventing (in any honest way) the today-level problems you (very correctly) bring up.
This same level of "oops" in the backend causes users to accidentally see or delete other user's photos, which is definitely a categorically worse scenario.
And yet why are people going out of their way to send multipart emails with useless parts? Your analogy does not fit as the Amish person is potentially refusing the HTML part: I just cited my preference, confirming to the protocol, and am being actively sent garbage information as somehow someone seriously thought it was easier for them to go out of their way to give me the option and then laugh at me for making the wrong choice than to just send an email that is HTML only.
reply