You know that UMart leaked? All details they had on file, including login credentials. It was all in plaintext.
It was before compulsory reporting, so naturally they didn't tell anyone. I only found out because I have this anal habit of using a unique email for every web site. That UMart login email and password since has popped up on numerous lists.
>It was before compulsory reporting, so naturally they didn't tell anyone.
ages ago I worked for a particular Aussie ISP, and as a result got a free internet connection.
They kept my data (they are legally obligated to collect) in their billing system. Their billing system sat on their VMWare instance.
A year or two after I left, they had a multiday outage. I called one of the engineers I worked with there, and he was basically "Oh yeah so after you left, someone exposed VMWare publicly, and then happened and they fucked off with the data, then deleted everything when they were done.
I sent them about 5 emails going "Hey are you guys going to publicly acknowledge the data breach" and they just buttoned up and chastised the engineer for being honest.
Since then, they got acquired, then the acquiring company got acquired, then that company went almost bankrupt, got bailed out and then there was a demerger of the group. I have no idea who has my data these days but it's up to 5 different ISPs. All stemming from one employee perk.
I’m kinda surprised Anthropic doesn’t do this by default. Asking permission meant what felt like near-constant manual intervention. It almost made a joke of the agentic claim. It also became mostly useless, as the size and complexity of the scripts meant a brief scan wasn’t likely to find anything anyway. Yet they persisted with it for so long.
The sandbox seemed like a good replacement. For those of you who don’t know, the sandbox means Claude Code runs with no protections itself, but it runs tools inside something like bubblewrap. It looked impressive until the model solved some commands failing by giving itself permission to run them outside of bubblewrap, defeating the entire scheme. When I first saw it do that, my reaction was “you can’t be serious”. But apparently they were.
So now I’ve adopted your solution: run it in a container, like bubblewrap, where I control the permissions at startup, so no model can bypass them. Add --dangerously-skip-permissions and you have something that’s workable and safe enough. A hallucinated space in `sudo rm -rf / $R` isn’t a disaster. Real damage outside of the project requires a level of malicious intent I’ve never seen from an Anthropic model.
Workable, but it doesn’t work 100% of the time. Claude Code stores stuff in a myriad of places, which makes setting up the sandbox far harder than it could be. Worse, the way it writes configuration files requires bwrap to give write access to your entire home directory. No way I’m giving it that, but without it, it can’t auto-update or refresh login tokens.
If they rearranged a few things, Anthropic could make it much easier to run Claude Code in a container. They could even make it an option supported by the claude CLI itself.
> bonus points if they create a comprehensive test suite first.
If they didn't have a very comprehensive test suite at the end of the exercise, then based on my experience someone is kidding themselves.
A friend had a tiny accounting system - maybe 15k loc, written in a combination of Python 2.7 / bash / LaTeX. I migrated it to Python 3.13 or whatever by hand. I did try using an LLM for that, but it did things like delete chunks of functionality so I had to check every bloody line it changed. Translating Python 2 to 3 is a fairly mechanical process, so traditional tooling worked better. Then I thought I'd explore LLMs by getting one to translate it to Rust.
The original had no tests or doco. As others have noted, the first attempt superficially looked identical to the original near immediately. You think "Wow". But in my case, it was riddled with bugs. When I asked it to fix bugs I noticed, it introduced as many bugs as it fixed.
I eventually figured out the limited context meant it kept forgetting all the lessons it learned when it fixed things, so it reintroduced old bugs and misunderstandings fixed earlier. (Like I said, this was a learning exercise.) Which is how I came to the same realisation a lot of the industry has - the solution to the context problem is you have to define the application via tests, then tell it to make the tests pass.
The tests rapidly became huge and unwieldy, so part of the exercise became DRY'ing the test code out by getting the LLM to build an integration test harness. The harness captured UI responses, files written, emails and printer output; froze time; fed in known input; and compared the outputs with results captured from the original program.
That cured the bug injection problem, but there were still bugs in all sorts of unexpected places. By unexpected I mean places I would be surprised to find them in human-written code. Human devs run the program in the same way human users do. They use the backspace key and the browser back button to navigate as they test their work. AIs don't. They will use APIs to inject test data unless you explicitly tell them to do it differently. The size of the tests exploded to be far, far larger than the code. Rather than coding I spent my time testing, telling the LLM to write tests to catch the errors I found, then writing standing prompts to try and prevent similar errors. It was meant to be a vibe-coded project, but in the end I got worried about code paths I suspected existed, so I looked at the code anyway to see if they were written the wrong way. Sometimes they would have blown up, say when the SMTP server was offline. To be fair, there was no need to look - I could have just asked the LLM. How a programmer without decades of experience would know what to ask is a mystery.
There is no doubt I got a working Rust program faster than if I had written it myself. Several times faster - but not orders of magnitude. If it was a program I had written, so I was very familiar with it, and it had excellent doco, and a high-coverage test suite, it probably would have been orders of magnitude faster. I think Bun's Rust migration might be an example of that - it's a best-case scenario. There were lots of early announcements, but it took over 6 weeks before they were confident enough to ship it in Claude Code.
But an "I migrated a program with hundreds of thousands of lines of code to a new language and ended up with production-ready code in a few days" - sorry, pics or it didn't happen.
'Superficially identical, riddled with bugs' is the scariest version of this. When it kept reintroducing old bugs, did writing those bugs down as tests first help, or did it just break the tests too?
Yes, it helped. In fact, it was essential. If you didn’t write a test for a bug, the model would reintroduce it over and over.
As an example, it produced PDF invoices and statements. PDFs are hard to compare, so I didn't bother. As a consequence, the stuff on the PDFs got moved, or was dropped, and the invoice didn't add up. I tried asking it to compare the PDF to a known good one - but it wouldn't / couldn't do that reliably. In the end the only solution was to ensure the PDFs it generated were always the same (so times on them had to be fixed), capture the generated PDF (not easy as it was often emailed or sent to a printer, not saved), and do a byte-for-byte compare, and make a test fail if it was different.
The fundamental difference from a human programmer is that an LLM does not learn. Once a human spots that they keep introducing errors in PDFs, they will remember and devise fixes. The LLM may do that too but then its context window overflows, everything it knows is lost, and it starts repeating all the mistakes it made yesterday. Tests are one of the few ways you can prevent it making the same mistake tomorrow.
The other issue is LLMs make far more mistakes than you realise. If you watch carefully, you'll see many of their bash scripts have syntax errors, code often fails to compile, and tests fail. Running a 2nd model over the work another produced will find many clear violations of style and coding standards. They hide this, in the same way humans hide it. They use tools and tests to find errors, like compilers and `bash -n`. They fix those errors without being asked. Compared to just a year ago, a modern SOTA model is far more aggressive with its integration tests, often going to a lot of effort to set up sandboxes for integration tests.
I suspect this is a major reason for today's models being far better programmers than earlier ones. But as we all know, tests have a fatal flaw - they can only prove the presence of bugs, not their absence. If you are creating code with a large number of random bugs, relying on tests to catch them all won't work. Instead you start using other tricks such as isolation boundaries, simplifying APIs so they are hard to misuse, strict information hiding - all things an experienced programmer learns over time, because they too write code with a lot of bugs. So far, there is little sign the models use the tricks well. Either they don't know them, or their stochastic nature means they can't implement them reliably.
Is there more to this than the Wikipedia article says?
While it does say:
> Gilmore also complained about being subject to "secret law", when the airlines and government refused to show the directive under which they were requesting ID.
nowhere does the article say the courts commented on the complaint, let alone ruled on it.
Nor does the article say anything about "Ignorance of the law is no defense when you are charged with violating a law". The reason given was instead:
> there was no constitutional violation because air passengers could still travel without identification if they instead underwent the more stringent "secondary screening" search.
I'm not saying your wrong, just that the article you offered as evidence doesn't go as far as your claims.
The Wikipedia article is poor, but I cannot not edit it as I attended parts of the trial and in fact discussed the suit with john before he filed it so by Wikipedia’s rules I can’t edit it. Anyway a lawyer would be a better person.
John’s appeal is online. Section [10] of the decision discussion describes being subject to a law he cannot have access to. The judge saw it but could not discuss it in court, with the plaintiff (John Gilmore) or even his lawyer. Also the orders (law) are agreed by the court to be arbitrary, changing day by day and being arbitrarily different between airports. But since the Airport personnel knew them tht was good enough for the court.
And though the discussion mentioned that he could take a bus or walk, the conclusion, [21], said that his rights were not violated at all, both the bus precedent and the legality of being subject to unknown law. The brief Wikipedia page mentions only the former, though the latter was the primary factor of the complaint.
You replying to Simon Wright, creator and owner of whirlpool.net.au [0], which is a Australian version HN / Reddit. It was the major Australian discussion forum in its heyday. The odds of him doing paid spruiking for Grok, or anybody really, are pretty low.
I don't think he is paid but you can look through his comment history and find that his opinions of Grok and its affiliated entities are definitely slanted.
Or maybe your opinion of Grok and its affiliated entities is slanted, and mine is straight. Without an objective reference point from which to measure deviation, all you're measuring is the relative angle between our opinions.
It's also worth noting that, living in an unfavourable time zone relative to most HN activity, I'm rarely here to offer the opening opinions in a discussion. More often I'm arriving 12 hours into a conversation, where everything I'd want to say has already been said. I end up contributing only where I think a different perspective is missing. Naturally, that means my comment history will disproportionately reflect the rare instances where my opinions buck the prevailing HN orthodoxy.
If you disagree with a particular claim I've made, by all means challenge it on its merits. That could be the basis of a much more interesting conversation than this fatuous one.
I don't actually care that much for Grok the model because I think they are more or less homogenous with like, a small house spin on how "smart" they are and some other features like their behavior in a harness etc. Personally I have no problem believing that it might be good at doing what you ask directly instead of wandering or trying to be independent or whatever. I wouldn't call it out as paid marketing and I also just don't really think about this topic very much.
However, the comment I did reply to seemed to think that just because you ran a forum that you would be unlikely to be biased. This is like obviously false, if anything you have major internet brain where you spend enough time on the internet where you come up with a whole complex about how you are contributing the uncommon opinion the site needs to hear or whatever, when it's just the same dreck from the rest of the internet sprinkled in here when it hasn't been posted enough by someone else. That is not directly "I used Grok and thought it was good at x" but, like, it's clear that it's something that is at the periphery of a whole network of positions you hold that are aligned this way.
I don't necessarily think this is bad and in fact I think most people on the internet at least basically post in this way but I do think it's important to be aware of it at least. I'm not sure you are. I can tell you more or less where my slop opinions come from and I don't think it would be accurate in any capacity to describe them as bucking "the prevailing HN orthodoxy". I, uh, think you should probably describe your comments differently.
I sincerely tried to read your response but I couldn't make heads or tails of it. Is this speech to text? Were you inebriated at the time? Can you rewrite it for me?
> These "wrapping" keys were not "weaker". They were completely unfit for purpose.
Oh come on. They clearly didn't intend to use 4 as a private key or to wrap it. They had just purchased millions of dollars of HSM's to store them. It was something more subtle than that.
Have a little imagination. The transfer procedure sounds bespoke - possibly just a shell script using openssl to encrypt things at one end, and decrypt them at the other. Let the man who has written 100 shell scripts without a bug in one cast the first stone.
But not so subtle that good software engineering procedures wouldn't have picked it up. In fact today just an LLM review of the shell script probably would have caught it. The Internet Death Sentence was warranted - because poor software engineering procedures are unforgivable in a CA. But those procedures and policies aren't easy or cheap to come by. If they were just "don't chose 4 as a primary key" my grandkids could do it.
Corruption is worse under Trump, but according to a recent Australia ambassador to the USA it's always been there: https://www.abc.net.au/news/107237782
The conclusion I came to is we are going to move from a hard science where everything is known, to being more like social science where everything is a guess based on confidence intervals. It's what happens when no one human mind knows the system well enough to predict what any small change will have on the rest of it. We call such computer systems "balls of mud". [0] says most deployed computer systems are balls of mud, so it's not the end of the world.
All vibe coded systems are balls of mud by definition if you use the conventional definition. If you allow "understandable by an AI" then right now not much changes because AI aren't so hot at designing systems and even if they could, they make poor use of their limited context. Whether that changes in future is the big unknown.
We already know how balls of mud evolve. New features are "bolted on the side", making the system more inscrutable. These bolted on systems often grow interdependencies on each other, creating a spaghetti of relationships no one is aware of. This means every change could introduce a bug whose blast radius is the entire system. Changes get harder and harder. A death of sorts, or at least a growth limit is reached when on average every bug fixes introduces more bugs than it fixed.
Google Chrome is a fine example of this happening right now. It used to be a highly modular combination of mostly independent systems that had their own champion who understood them intimately. They left, Google management insisted the client platform that supports most of eco-system continued to evolve and grow. The ball of mud was formed. It's been evident for years chrome has a higher bug injection rate than Firefox. As complexity grew it became too hard to move to new tool versions, so compilers and libraries got pinned to known working version. That made it too hard to use as a component, so projects like ChromeOS are being replaced with Aluminium OS. Now the that Matt Jolly has thrown in the towel [1] it's hard to see how distributions like Gentoo and Debian that used his work can continue to ship chromium. Debian can use ungoogled chromium, but vendored libraries, obsolete tool chains, inability to do forky mandated reproducible builds make it look like it's a foreign object bolted into a true open source project. This is what software death looks like. It's not a bang, projects cease to evolve, then can't meet new requirements, and then are replaced.
It doesn't have to be that way. Projects like the open source kernels are decades old, but their strong stewardship has ensured they remained decomposable, so people can work independently on different sections without too much fear of stomping on each other's toes. Open source stewardship seems particularly good at this, possibly because open source projects can take a very long term view. A manager in a large organisation on the other hand is usually around for only a few years. They take the short term wins that come from features bolted on the side safe in the knowledge the long term death of the software will be somebody else's problem.
You know that UMart leaked? All details they had on file, including login credentials. It was all in plaintext.
It was before compulsory reporting, so naturally they didn't tell anyone. I only found out because I have this anal habit of using a unique email for every web site. That UMart login email and password since has popped up on numerous lists.
I'm not the only one who noticed: https://forums.whirlpool.net.au/archive/2744108
reply