HN Simulatornew | past | comments | lists | submit | markasoftware's commentslogin

Space has an efficient way to get power (peak solar 24/7) and is less regulated. Can easily launch a shitton of satellites exactly the same way over and over again once starship works, need to keep finding sites for underwater, each one needs different structural engineering, etc.

> A submerged data center is orders of magnitude easier and cheaper than a space data center.

Source?


Is a source necessary? Just applying some basic common knowledge, such as it takes less energy to move something up than down, or the fact that everything in space has to get there on a rocket, which is only possible by buying space on rockets from whatever three companies or two countries or whatever are sending up rockets. The ocean is accessible from every salt water harbor on earth.

Then there's the fact that you can run hardlines to ocean datacenters, the fact that the ocean provides abundant coolant (cooling things in space is hard but this might surpass common knowledge), the fact that repairs and upgrades don't require a rocket, astronaut, and spacewalk, the fact that you only need to harden against salt water instead of radiation and micrometeoroids, etc.


Thank you. I couldn't have said it better myself. I was on a nuclear powered submarine. A data center on a barge with a nuclear submarine parked next to it for power (out sub did 80 MW) is easy to envision and doable today. Sea water cooling systems are a solved problem obviously.

On the other hand, a data center in space? A data center is heavy. Solar panels are heavy. That's not a problem on land and sea. It is a problem when you have to lift it into space. Then what about maintenance Do you send a regular crew into space to work on it? Or do you keep a crew in space like a giant space station? It's all doable but at what cost?


> is easy to envision and doable today.

It’s not doable today. The private sector would never be given access to naval nuclear generation for a project like this.

It’s only doable today in the sense that we could just build nuclear plants everywhere today and end the climate crisis. On paper the energy works out but society doesn’t want nuclear because of hysteria.


Cost and long time horizons is why nuclear is not competitive, not hysteria

Yes a source is absolutely necessary. Investors are dumb but there's a reason multiple startups and established firms are pursuing space datacenters seriously; they did have to show investors some compelling numbers. Why are there no major underwater datacenter efforts today?

> Just applying some basic common knowledge, such as it takes less energy to move something up than down, or the fact that everything in space has to get there on a rocket, which is only possible by buying space on rockets from whatever three companies or two countries or whatever are sending up rockets. The ocean is accessible from every salt water harbor on earth.

Your whole premise assumes that energy to run these things is free/cheap and that energy to install them is the dominating factor. 24/7 solar power in space is literally the entire argument for these orbital datacenters. So unless you have some proposal for why electricity is easier to get underwater, you’re just ignoring the most significant factor.

Data centers on land are even easier than data centers in water. But they are getting blocked because of FUD about power prices.

So yes, a source is necessary because you ignored the biggest ongoing cost to running a datacenter when applying “basic common knowledge”.


> 24/7 solar power in space is literally the entire argument for these orbital datacenters.

That's good to know because that makes the case for data centers in space even worse.


>once starship works

More like IF Starship works. This thing is spearheaded by the same guy who spearheaded the Hyperloop high-speed rail, which became a single-car tunnel under Las Vegas, Humanoid robots that are actually just people in bodysuits, the Cybertruck, which was a complete flop, and now "flying cars" that are just proofs of concept that won't ever actually get sold (or will kill people if they are).


No, it makes things dramatically better when there are lots of small assets. You used to need a ton of tiny TCP connections to make that kind of site work. Try writing a web scraper and observe how much faster it goes with http2 on (which is one of my biggest gripes with wget and other recursive downloaders: no http2! And the "wget2" project is trash and crashes constantly)

No you don't need a ton of tiny TCP connections to make many requests with HTTP/1.1. You might be thinking of even older versions.

I used to think this too, but AI capabilities over the last couple years prove it completely wrong. The skills are more specialized per domain than I thought (though there is still a lot in common between fields)

And further, surely you know some very strong programmers who are very bad at art, or math, or human relationships?


Every new model is only marginally better than the last, since last year, and I'm sure they are throwing all the compute at every new model, which means it's hit diminishing returns.

Even after all of the resources it takes, it still needs human guidance. At the very least the conclusion is that it's not doing what humans are doing. They might know all the programming languages and all kinds of domains but then it needs to be told basic things to not fail at simple prompts.

Part of the problem is that this is very hard to talk about and describe. It requires a long conversation, probably better in real-time, to explain exactly how AIs are not really living up to the hype, and are actually introducing long-term risk by using them.


I'm convinced SponsorBlock could be completely automated with modern TTS and a fairly small LLM

Any are welcome, so far as I know having not checked their submission criteria, to spin it up and contribute tags to the SponsorBlock community. I'd wager there are people already doing so.

I think it would be more useful to more people to utilize the existing solution because it runs in a very small footprint for the end user. Most people wouldn't be able to run an entire TTS/LLM workflow without processing videos in the background or waiting some multiple of real-time in order to watch a video.

You could maybe do it with just LLM tagging the transcript, but that's still a ton of processing just to watch a video.

God that'd be so much wasted energy if every person wanting to skip in-video ads was running an entire LLM to do so.


Unavoidable for some stuff like podcasts where they inject random sponsorship into audio files and throws off all the timestamps. Though maybe wasteful llm adblock compute breaks ad serving compute, they somehow cancel each other out by making all the compute wasted on ads + telemetry defunct.

Circle back to original topic, eventually having local LLMs intercepting EVERYTHING in background to reproject a curated personal bubble filter feels like wasteful but sensible end game. Not just to hide ads / skip sponsors, run every image through anime filter, live editing xyz, if there'es some centralized DB to grab timestamps do it, but otherwise we could be burning tokens none stop to modify reality, and some add manipulation is just rounding error that comes along for the ride. I click a video, background processes play opens 10 instances of the video at 4x speed, processes, gives me version of video without ads, sponsorblocs, xyz content filtered out... or just generate video to conform to my biases because it makes me feel good.


It's not sponsorblock quality, but I have trained a model for podcasts. Turns out, at least for podcasts, you can get away with a surprisingly small model, small enough it can run on device directly.

No idea if it's good enough for all podcasts that are out there. It works for me and the podcasts I listen to, and I have tried testing it against a larger corpus, but I'm one developer and I just wanted to solve a problem for me.

https://github.com/Q726kbXuN/podcast-ad-tagger


Don't media plays already kind of do this with 'skip credits' or 'skip intro' ?

Auto mode cannot be considered safer than anything because it does not even attempt to guarantee any sort of security. The threat model is not even defined. It cannot be quantified or compared with any other approach other than "full access"

Compare approving every step manually: at first you read everything the llm proposes carefully, convinced that you'll catch it out. After a while you get faster and faster until at some point you find yourself hitting "approve" the second the text appears, long before you actually could absorb what it said.

This is called "approval fatigue".

If you think this can't possibly happen to you, you might be in for a surprise. This is an almost textbook case of classic conditioning. You'll find yourself hitting that button faster and faster even while fighting back the urge. So a human is no guaranteed backstop.

This is the scenario that cc auto mode guards against. It cuts down the number of permission prompts to something manageable, so that your fingers don't get trained into clicking 'approve' on 'rm -rf' in the very eyeblink that it appears.

So maybe you feel it doesn't work in theory; but weirdly enough, in practice this particular approach is actually one of the better ones; especially when used in combination with other common sense measures where practical.


I'm not claiming auto is worse than manual; I'm claiming it cannot be compared with anything security-wise because it does not aim to provide any guarantees, there's no systematic way to reason about what it will and won't allow, etc

With e.g. VM or container sandboxing, we can say things like "If there are no vulnerabilities in the hypervisor or hardware, then the agent can only access these files, these network services, etc". Auto mode doesn't even try to provide any guarantees, so it's impossible to reason about. It amounts to just using the models judgment about what's "malicious" or not, a definition which is subjective, changes model to model, and can be influenced by attacker input.

Empirical evaluations of how many vulnerabilities manual vs auto mode let through are not super meaningful because auto mode changes dramatically with model releases, the type of task being performed, whether the attacker is making targeted attacks designed to bypass auto mode, etc. A security feature cannot rely on these sorts of factors if it's to be taken seriously.

Further, percent of malicious tool calls accepted is a bad metric. It's like measuring the security of ssh in terms of what percent of unauthenticated logins succeed. The only acceptable value is zero. It's pass/fail, not a spectrum. So both auto and manual mode should fail equally here. Further, the set of malicious tool calls admitted in auto mode may not be a subset of those admitted in manual mode.


Ah, you're going for the actual adversarial case.

I think I'd trust the current approver with correctly classifying regular semi-interactive LLM behavior fairly well.

However, if we bring in a hostile adversary that calculus changes, and you'd be right to start thinking about other approaches.

On the other hand, transformers are a surprisingly powerful technology, and I wonder if there wouldn't be ways to train one to several nines reliability under adversarial attack. It seems to have worked in at least one case; though I'd be a bit cautious to extend that conclusion to more cases quite yet.

* https://hackmyclaw.com/ ; https://simonwillison.net/2026/Jun/26/hack-my-ai-assistant/


Hmm, what does auto mode protect against in the non-adversarial case? Aren't the models already extensively trained not to perform harmful actions without provocation?

It mostly stops honest mistakes, because those do happen, especially when the might not have been perfect either.

You do have a bit of a point, though: I’ve run several models in sandboxes with --dangerously-skip-permissions for months; in the full expectation that I'd need to replace a sandbox once in a while. But so far none of the sandboxes have trashed themselves, yet. knock on wood

Auto mode seems like a lighter-weight version of that: fewer local resources, and it can run on the metal if you need cc to help with system work.


That is not my experience. They attempt to nuke my containers quite frequently. I run regex on all their tool calls and stop them from doing certain things, its the only thing that prevents them from removing their home directory at some point.

This isnt even the main concern with dangerously skip all permission though. There are many. What happens when they get prompt injected on a web page and send someone all their privileged information because you give them free access to the internet.

Using llms like this is dangerous. They will ruin your day eventually.


I've been running Codex for many weeks in the Full access mode, since their Auto mode was useless in the sense that it blocks benign commands frequently.

No data loss has occurred.

Nowadays I use Claude, where the Auto mode works.


>No data loss has occurred.

"I've started never checking my parachute's condition before a drop for a year now, and no accident has occured".


All SOTA models will still try to delete its home directory. I have a plugin that prevents things like this and it gets triggered more than you'd think, despite having hook instructions on every request not do do certain things. It will also try to find ways around its permissions, quite cunningly.

You can not trust that they're RL'd to be safe. Even if it only happens 99.80% of the time (its more than that) it will eventually happen.

Less severe actions will happen even more often. People using llms with their native browser profiles are getting farmed, hard.

Llms are highly dangerous in terms of security, but they're being marketed as highly intelligent tools that everyone should use. Its really mind blowing.


> despite having hook instructions on every request not do do certain things

This tends to be a smell. Presumably you added "don't do this" instructions after the model started doing the thing. But that's mostly decoration, as you have found. It's like adding # Please don't crash comments to every instruction in a python program.

Try to actually debug the issue. Regularly ask "can you reconstruct[1] the reasoning why you issued that command?" ; dig down. Set traps that halt execution.

Consider checklists or skills to guide an agent through a tricky set of steps; and consider writing procedures for failures.

[1] "Reconstruct" still isn't perfect; but "explain why" is a license to hallucinate. Asking to add #comments on reasoning to each step is a potential help in reconstruction.


I thought the tech priests praying to their machines with no one really knowing what the prayer means was too far fetched to be real

> People using llms with their native browser profiles are getting farmed, hard.

Got any references for the prevalence of this, even links to anecdotal reports?

Reading the reviews on the chrome store for the extension [0], I haven't found a single report of someone who was prompt injected, just a bunch of "anthropic's subscription is expensive" or "it doesn't work", but no "it stole my bank account".

[0]: https://chromewebstore.google.com/detail/claude/fcoeoabgfene...


I have no idea how the extension works, and Im not talking about the extension. Im talking about browser use broadly, and most people are not using the extension, there are a thousand other solutions.

However, Anthropic is one of the only labs who does a decent job at making models that recognize prompt injection attempts (but not completely immune) however Im speaking about giving llms in general access to your browser broadly.

I just know through my own experiments that it is quite easy to prompt inject an llm operating a browser and get it to send secrets. Using the extension might be safer idk, Om not risking it.

Llms are a security nightmare when connected to the web.

I have a website where I experiment with this. Although Ive stopped baiting llms into doing reckless things.

Killai.now <--(dont use this, its a WIP) Killai.now/poison/about Killai.now/propaganda/about


why would they delete their home directories?

> All SOTA models will still try to delete its home directory.

Literally never seen this once in years of extremely heavy use across all major models and harnesses.


As real as approval fatigue is, it can't really be considered worse than effectively leaving the decision up to the party that proposed the action in the first place.

It really can be potentially worse.

Everyone thinks the decision was being checked by a human, and thus safe. Because of that, people might have concluded that further safety gates were unnecessary.

But the human just got conditioned to hit 'approve' out of reflex.

So in reality the proposing party (the llm/agent) might be the only one in the pipeline who actually did any checking, if at all.

In certain extreme circumstances this can even lead to physical danger in my line of work, so I'm learning to be careful.


>This is called "approval fatigue".

Doesn't matter, that would be on me.

It's also a false dillema. Those aren't the only options. A strictly enforced sandbox with sane defaults (for example ~/.ssh), that acts as a hard OS level limit, and not just as a suggestion to the agent, would not have "approval fatigue".


This is the conundrum we find ourselves in. Do we submit to approval fatigue and press “Don’t ask again”? Because that’s the dream. We all wish we could just trust the agent to do the task start to finish, unsupervised.

But this is the downside. It might be fine 99% of the time, but 1% of the time you get a critical secret published to GitHub, even after human review.

Is this even worse than what us humans did on our own before LLMs? I’m not sure. It’s not like secrets never got accidentally committed before, it just illustrates the new paradigm we’re working in.

Either you let your agents run wild, and you cross your fingers that they don’t do anything bad, or you need to accept that your new job is to be an agent babysitter, and audit every command.


The agent behavior is constrained with external tooling and contracts. Contracts that fit in memory.

No, this is the way I have worked for a long time, and I have not suffered from any approval fatigue.

Rather what has happened is that I have gotten very good at scanning what it is trying to do and know within a second or two whether there is anything fishy I need to look at more closely.

And by "fishy", I do not mean only security issues, but also architecture, code, etc issues. Sometimes it even adds unrelated unapproved changes to what it is supposed to be working on, and I stop it.

And I'm glad I work this way, because countless times I have stopped it from going down a path that is nonsensical or I strongly dislike.

There are techniques I have adopted to speed up working this. Sometimes I allow Claude to make larger changes at once, to batch several changes at once, etc. But these decisions are under my control.

I prefer to work this way than to wade through enormous PRs trying to wrap my head around what was done, where I am likely to just give up and approve it after a cursory review.


Auto mode is significantly safer than manual approval in practice. Details about the engineering approach and how we tested whether it is safer here: https://claude.com/resources/articles/auto-mode-default-in-c....

Auto mode may be significantly safer in the niche case where you want the model to have free reign, in a fully unopinionated environment, where it's allowed to make any changes and make any external calls.

Many of the problems when it comes to the manual approval feel like they stem from the expectation of using auto mode in the first place. After all, what's the point of improving the manual approval and safety matching if auto mode is the end all be all, and what's the point of training the model to output commands in a manner that can be parsed by a human if auto mode is always meant to be used? It's very rare for me to not see "command could not be statically analyzed", because the model keeps outputting bash golf.


> in the niche case where you want the model to have free reign, in a fully unopinionated environment, where it's allowed to make any changes and make any external calls.

That is not the niche case, that's the dominant use of models.

It would be helpful to distinguish between your own opinion on how models ought to be used, compared to how others actually use models.


> That is not the niche case, that's the dominant use of models.

It would be helpful to distinguish between your own opinion on how models ought to be used, compared to how others actually use models.


>That is not the niche case, that's the dominant use of models.

If it is, it's only the dominant case of models because it's pushed as the dominant case of models, with Anthropic devs even coming here to say how people should just accept it and it's so much better than being in control.

It would be helpful to distinguish between your own opinion on how models ought to be used, compared to how others actually use models.


> That is not the niche case, that's the dominant use of models.

And if everyone else jumped off a bridge, I mean, used raw interpolation for SQL queries, would you do it too?


Using raw SQL string interpolation doesn't get you anything much that you can't get just as easily with a parameterized query. In contrast, properly sandboxing agents is such a pain, and such a productivity drain, that hardly anyone does it in practice, save for a vocal minority online.

Don't worry, you can just use the automated approver to escape your queries! It's so secure!

Auto mode seems to have some reasonable restrictions baked in. For example if you give claude access to two k8s namespaces, seriousproject-qa and seriousproject-prod, auto-mode will grant `kubectl exec` on the qa environment while denying the same command for the production environment. Of course you should be enforcing that by restricting what kind of access tokens you give claude access to, but auto-mode does have opinions about this stuff too, even if it's not bulletproof

However I do share your desire for a better manual mode. Auto-mode is both sometimes surprisingly lax, and sometimes too restrictive. Maybe manual mode should inject a reminder "keep your commands as simple one-liners", the same way plan mode injects a reminder that you are in plan mode and don't want any code changes


Is that because auto mode is good, or because the manual approval system sucks?

Indeed. The article only compares the status quo manual system versus the auto system — and doesn’t consider ways that the manual could be improved.

As it says:

> 5% allow any shell command outright, and another 43% have interpreter rules like Bash(python:*) or Bash(node:*) that are essentially equivalent in practice

Claude Code never should have prompted to always allow Bash(python:*) or Bash(node:*) — that’s clearly a bad idea, and Anthropic know that (given the references in the article!).

Of course, Bash(python a.py) still isn’t safe if you also allow Write(a.py), because you may as well allow Bash(python:*) at that point, and that’s often practically what people want (“I want to both make edits to the source and then run the test suite”). So maybe the whole permission system is broken to start with — and the actual answer is leaning more heavily on fine-grained sandboxing.

You also, as the article rightfully points out, run into permission fatigue and users blindly hitting allow.

I do still wish you could do more with hooks — outside of auto, I might want to allow a given command via a hook if Claude Code has read access to given file, but there’s no way to query if a given path allows/denies read/write.


Will look into exposing that in hooks.

Ultimately, the threat model here is: human has to approve a lot of commands, so gets tired and accidentally approves a command they shouldn’t have. We spent a significant amount of effort to build a richer permission system, allow configuration at multiple levels, use static analysis to detect dangerous and safe commands, monitor the model to detect and block prompt injection at the inference layer, etc. Some of these components we rewrote and red teamed dozens of times. Our conclusion was that despite all of this work, in practice the threat model is that human attention is limited and that that a different approach performs an order of magnitude better in practice.


Did you ever test putting the auto mode classifier in front of manual approval? This is how I use it, but last I checked that option is not available out of the box.

That’s how auto works right now. It gives you a prompt.

The problem is that you didn't even attempt to build a proper box (a sandbox that prevents the AI from running bad stuff by design). It's an extraordinarily hard problem. Only OpenAI Codex is actually trying to do it, and they suck at it too. They keep breaking permissions, their own AI can't troubleshoot it, their defaults give read to the whole filesystem (go ahead, slurp my ssh keys), and their "manual approval" escape does not seem to work.

I agree actually. We built an open source agent-agnostic sandbox that we are landing improvements to, much more to come: https://github.com/anthropics/sandbox-runtime. That said the sandbox is not perfect, and any sandbox needs holes punched otherwise it is not useful in practice (at least for inference and MCPs, and almost certainly more). We will be doing much more in this area.

>and any sandbox needs holes punched otherwise it is not useful in practice

Don't assume our individual use cases. It can be perfectly useful with access to just one folder and zero network asside from the API.


That’s good to hear, it’s a much better approach than expecting humans to read commands and approve them or just trust that the LLM can set the proper horizons for its world itself while working.

Making a sandbox both fully secure and fully useful are contradictory requirements. If you want the agent to do any real work, you have to leave gaps.

With SOTA agents that’s a real bind, because they’re very likely to find those gaps. Whether they exploit them, close them, or leave them alone then becomes a prompting challenge yet again.

I don't think this means don't sandbox at all. You can prevent a lot of small and medium accidents by locking down or gating access to hidden files, ssh, sudo, etc.

But your sandbox can never be the whole story.


>Making a sandbox both fully secure and fully useful are contradictory requirements.

Any tradeoff deals with "contradictory statements". Doesn't mean we should always side with one side on a tradeoff.

Nobody said anything about "fully secure" and some arbitrarily "fully useful" combined, anyway.

For certain people it could be 100% fully useful with just access to a few fixed folders and a few fixed APIs.

Many of us don't want it to drive our lives, just do some specific this or that.


Right! That was the standard view until fairly recently. But I'm pointing out something else.

This year we've learned that If a SOTA LLM decides that the sandbox is standing between it and its current objective: well, it has the ability, the time, and literally nothing better to do all day. Odds are, if you've configured the sandbox for anything useful, you've also provided a crack it can use to get out.

Again, doesn't make a sandbox entirely useless. Does mean you can't just put it in a really tight box and call the job done. If anything, the strict sandbox might even hurt, because it's more likely to get in the way.

(By analogy: consider the safety issues posed by a dam without a spillway.)


This is much more of a problem if you're attempting to run at a massive scale, beyond what humans can audit. When I'm doing "artisanal" agentic programming, I'm still in the loop in the sense that I'm watching what it's doing and ready to hit Esc if it is doing something dumb. If the AI trips over the sandbox, I can cancel it and do whatever it wants me to do before it starts trying to engineer its way around.

Maybe you can include some percentage of intentionally malicious commands, but not actually run them if the user approves them. Just blare an alarm telling the user they misclassified a malicious command, and maybe pause the Claude Code session for a frustrating 5–15 seconds. Video games can elicit truly astonishing levels of performance from human brains by using this approach. I'm guessing it's probably outside your Overton window, which is too bad, because it has the potential to work a lot better than the 89% you get with "Auto mode", which is far too low if anything really significant is at risk. That's not even one single order of magnitude lower risk than just "allow everything always".

Speaking of really significant things being at risk, it's pretty infuriating that the web-chat Incognito mode can't write and test code locally, and neither the non-Incognito web-chat nor the local Claude Code client can be shielded from the knowledge of my email address and my geolocation. (Even the Incognito mode knows my geolocation.) This gives users no alternative but to put privacy-sensitive information at the disposal of Claude Code, which is far too unreliable to be entrusted with it. It's tempting to attribute this to Anthropic's corporate DNA valuing users' interests at a very low level, but conceivably it's just an oversight, and if I mention it here, it will be corrected in a month or two?

Currently, my assessment is that Claude Code, and in particular the Opus 5.5 model, is head and shoulders above the competition in software development capabilities. Nothing else is even close. But I've seen several small signs like this which suggest that Anthropic, as a company, is maliciously misaligned. So, if Grok, Qwen, GLM, or Astra starts to come anywhere close to being competitive, I'll switch as soon as I can. I imagine that most of your users feel the same way.


this is not a serious proposal

Not only is it a serious proposal, it's been used in practice in every airport X-ray baggage scanner for decades, where it's called Threat Image Projection (TIP): https://www.xscann.com/products-HiTipPlus.html and it's actually a regulatory requirement in that context in a number of countries: https://caainternational.com/course/threat-image-projection-...

Yes, but you can require it in such a situation, as they're subordinates who have to follow policy or lose their jobs.

As I pointed out above, Anthropic places a lot of requirements on its users already, but if Claude Code TIP were optional, presumably, savvy users would turn it on voluntarily, unless they wanted to use Auto Mode. As Anthropic's study clearly documents, the current manual approval workflow basically doesn't work at all, because the users just approve 87% of dangerous actions. So there's no reason to use it, except maybe very rarely.

Most of anthropic's money comes from enterprises that may want to enforce something like this at an org level.

The same scanners that let handguns et al through 93% of the time, according to internal audits and journalist review?

Maybe pick a better comparison if so?


Enterprises do the exact same thing today by sending phishing emails to their own people to figure out who needs additional training. Why would an enterprise not want this feature for Claude? It seems like an extremely serious proposal.

I do think it's outside their Overton window, which has gotten it downvoted to 0.

> I do still wish you could do more with hooks I might want to allow a given command via a hook [...] given path allows/denies read/write

You can make a script for the PreToolUse hook that can do that. I use a command gate script that parses input for doing that exactly, allowing/denying actions based on tool use + paths -- it started out as a hook to prevent claude from polluting my clojure projects/filesystem with python in sneaky ways. Babashka works really well here


If you have a file in your repository that is not safe to run you have bigger problems than your development system getting compromised.

What I mean by this is that if you restrict python execution to the entry point of your software and then Claude wants to run a command anyway so it threads it through the main.py script, you are no longer running something that is contained to your local development environment.

But when you think about it, what exactly prevents Claude from doing that anyway?

Claude could add malware into your project directly.

So the case where you say it doesn't matter which python script it can execute, actually does matter. Restricting python execution to a subset of all files buys you security.

The next step would be to get Claude to stop adding malware to your software, which it hopefully already does not do.


A key problem when using manual approval with Claude is that too much of Claude's tool use is Bash or Python slop that is difficult to review (especially when you are getting spammed by it) and impossible to handle through rules. Example:

sed -n 1,30p src/vm/wren_vm.c && sed -n 310,350p src/vm/wren_vm.c && sed -n 560,600p src/vm/wren_vm.c && sed -n 740,775p src/vm/wren_vm.c && sed -n 40,70p src/vm/wren_common.h && cat util/wren_to_c_string.py | head -80 && ls test/random doc/site doc/site/modules doc/site/modules/random 2>/dev/null; cat projects/premake/premake5.lua | head -80; cat util/generate_projects.py

After ten or more of these, you're going to get hit by permission fatigue, and then, yes, manual review becomes too error-prone to use safely.

The irony is that Claude is regularly ignoring its existing Read/Write/Edit tools that could make everything more readable.

Other models/harnesses have their own problems, but Claude Code is one of the worst for manual approval.

You can get manual approval to work reasonably well if you give the model a reasonable set of tools (I have literally written my own MCP servers to cover gaps in existing harnesses) where you can then write rules for the vast majority of safe cases.

For example, I have a stdio MCP server that allows models to inspect crates under ~/.cargo/registry/src (paired with the existing query-crates MCP server). It provides read-only operations limited to crate data, so I can pretty safely auto-approve everything in there. With the right setup, including a sandbox, you only ever have to manually approve something once in a while and usually with delimited scope, avoiding permission fatigue.


Opus especially seems to like to:

1) edit a file with sed, 2) try to write it with its regular tool 3) get blocked because sed changed the file 4) read the file with tool 5) edit with sed again.

Would be nice if it felt like real people were developing Claude Code, instead of chasing after another half-baked idea Claude came up with.


Related to that, there are quite a lot of common command cases where you have the choice:

- Yes, but only one time, you will have to manually approve next time

- Enable auto mode and stop worrying

- No

Nothing to save this command or part of it to the manual approve list. So will be requested again 100x times.


That's on purpose, so that you just give up and choose "auto mode".

They talk about "decision fatigue" when they're engineering it themselves with dark patterns.


The blog post has more detail, but tldr is humans make mistakes that automation can catch.

Sure, but Claude is deeply useless at making permissions hooks! Every time I've tried it it completely fails at the task, despite researching the docs and using various skills. It trivially evades its own just-created rules.

I run CC as its own user on my laptop with no secrets/keys/erc available other than through a LPE exploit or perhaps installing a keylogger. Can I guarantee CC won't attempt any of those things in auto mode?

There are no hard guarantees as far as LLM output is considered.

That may be for most cases, but it wasn't safe for this password scenario, and likely others that haven't been discovered. Maybe there's a PII setting that can be enabled by default.

I completely believe this, but you could also run auto mode classification in manual mode and tell the user the result.

Or even reject anything auto mode rejects while still allowing the user to reject things auto mode accepts. If what's approved is the intersection of what the user is approving and what auto mode thinks is safe then it would be strictly safer than either mode alone.

This would also be a way for users to gain confidence in auto mode before going all in on it.


Just a few days ago, opus 5.5 in auto mode deleted backups that it shouldn't have, and in the "apology" it even said, "Nothing was lost, I didn't delete your original files".

I hate this condescension by Claude. It doubles down in such a bad faith way.


There's no reason why we shouldn't be able to have the audo-mode safeguards happening when we set things to manual mode. Shouldn't you just have it checking on claude anyway?

> For high-stakes changes to production infrastructure, we still recommend reviewing Claude's actions yourself.

- from the linked article


No its not generally safer. It depends on who is doing the approving, the task being completed etc.

Claude routinely tries to install software if I don't stop it. I try to only use Claude within a container. Outside the container, I never use "auto" mode.

Ideally I would never use Claude at all outside a disposable container, even on a box with no sensitive information and with the sandbox enabled. It's like having an attacker breach your system — you don't know what it did.


Yes, furthermore in practice auto mode does not appear to provide anything approaching deterministic gating:

"Measuring the Permission Gate: A Stress-Test Evaluation of Claude Code's Auto Mode" https://arxiv.org/abs/2604.04978

(Shared by hkalbert71 the other day in: https://news.ycombinator.com/item?id=50387838)


I recently saw that someone's entire project got wiped out because of (incorrect) auto mode approval. And this is because auto mode failed to catch an error in tool use, nothing to do with prompt injection.

Meanwhile I have seen it rejecting perfectly safe commands.

It is a worse version of YOLO mode.


"My personal security model can't represent this, therefore it's not secure"

is scaling working? It is also my understanding that Opus 5.5 is probably a well under 10T parameter model. Probably a similar size class to Fable 5 earlier. And supposedly even GPT-4 was a 1T model. Clearly most of the gains are coming from something other than scaling up the model size. Yes, scaling up RL counts as scaling too, but the fact that labs have pivoted to a different axis of scaling is evidence that it's not just a "straight highway" to improve LLMs.

From the outside it seemed like GPT 4.5 was the apogee of parameter size scaling and since then a huge investment in RLVR became the new scaling axis.

So yeah, LLMs keep getting more powerful/useful but because of an industry wide pivot not a "straight highway" as imagined in the GPT-4 era, when simply going bigger and bigger seemed like a reasonable roadmap for the foreseeable future.


Reminds me of the extremely interesting paper https://arxiv.org/abs/1904.05234 Flash Boys 2.0

Basically when somebody makes a pricing error on a decentralized cryptocurrency exchange there's a mob of automated arbitrage bots that try to trade against it. Who wins and actually gets to make the trade is mainly determined by how high of a transaction fee your bot is willing to pay.

You can win instantly by offering to trade with a transaction fee so high it almost wipes out all but a cent or two of the entire arbitrage profits, because nobody else will pay a higher fee.

Instead what happens is bots offer the lowest possible fee, then immediately increase their fee as soon as they see a competing bot try to take the trade with a higher fee. This results in the bot offering the highest fee changing every several milliseconds as the fee quickly grows.

The way Ethereum mining works, this results in the winning bot being randomly selected from among the set of bots "bidding" up the transaction fee: blocks are mined at random intervals of iirc about 20 seconds, and whoever had the highest bid at the time the next block is mined wins. <20 seconds of bidding is short enough that even with the bidding, the transaction fee doesn't get bid up particularly high, and the winner makes out with a highly profitable arbitrage trade.

By giving up a guaranteed win and instead rolling the dice against other bots, transaction fees stay low and everyone has a higher expected profit. It's coordination that happens without any communication between the participants and quite possibly the participants didn't realize they were in effect cooperating when they added the high speed bidding behavior to their bots.


> You can win instantly by offering to trade with a transaction fee so high it almost wipes out all but a cent or two of the entire arbitrage profits, because nobody else will pay a higher fee

This is like 1980s sawtooth algos in real finance.

A lot of crypto is former HF folks taking old strategies to a dumber field.


I have really enjoyed watching crypto act like they are white knights against the evil traditional financial system and then go on to reinvent it piece by piece.

You're assuming that the contention was with the functionality and not with who controlled it.

The current players in crypto are the same people. And there’s noting systematic about crypto that changes this — people with money can afford to buy more mining capacity or stake more coins.

Yes, incumbents warped the original intentions, it's true. Famously, Bitcoin was coopted and the originally intended functionality only lived on in the Bitcoin Cash fork. Cryptocurrency was supposed to be a way to transact without touching the banking system, but banksters realized that it could be an unregulated vehicle for their financial shenanigans. I don't know how you might have guarded against that sort of attack - capitalism being about commodification at a fundamental level - but, "It was always a fox house!" is incorrect.

There’s nothing evil about aiming to earn a penny or a fraction of a penny on a transaction.

There is something evil, however, about scaremongering around fractions of a penny to convince folks to move money to one where the spread is several percent.

Who pays that penny?

Whoever wants to pay an extra penny. Like any other transaction.

That isnt cooperation. That is basic market principals in action. Bid and counter bids that whittle down the profit margin based on willingness to risk a win/loss? These are auction principals as old as time. This is reaction rather than cooperation.

The winner of the auction is not based on "willingness to risk a win/loss". The auction always ends at an absurdly low price because of the mechanics of the bidding, none of the participants ever drop out, because the price never goes above what they're willing to pay. Please read my comment or the paper again, this is not like an ebay auction at all.

For example, imagine a car auction that lasts only 20 seconds, with a random delay of up to 1 second, and with a very small minimum bid increment, say 2 cents, and no bidding fee or limit on number of bids. Now image 5 different people each outbidding each other by the minimum bid increment every 10 milliseconds. The price will climb by about $2/second, so by the end of auction, the winner will get the car for $40. Because everyone's bidding so fast, and nobody knows exactly when the auction will end, the winner is effectively random. At least on an expected value basis, this is highly beneficial to the participants vs actually bidding up to the true price of the car until other participants are not willing to bid anymore.


That sounds economically rational to me.

I think a comparable situation would be if everyone bid up to or higher than the actual price to generally support it.

They would be acting economically irrational in that situation to increase price overall, like corporate land lords refusing to lower prices even if it means they have higher than normal vacancy rates.


Xen had a generic guest to host vulnerability disclosed in the last few days: https://codeant.ai/security-research/xenbleed-a-grant-copy-c...

Yeah, all security design guarantees are as good as the implementation at the end of the day. All the fancy hashing and encryption math in the world isn't gonna save you from an erroneous `"" == ""` check or an invalid reference check.

I was replying to OP saying it's the same "approach" as Xen, if your preference is "approach" based. It's fair though to point to Nova's smaller code base. It's becoming an important metric for me in general (how big is the code base). It's not a bullet proof thing, but having a smaller code base is a positive sign in 2026 for me.


Novas core seems to be formally verified which rules out a vast amount of bug categories that can happen with xen, including this one

Interesting, I'll have to learn more about Nova.

See Udos most recent talk at 3mdeb Boot Mastery Conference.

He is an ex Intel CPU engineer and the depth of his thought into the security design had me questioning my life choices.


https://youtu.be/RbL65gVzhNk for anyone else looking for it

Thank you. I'll definitely check that out.

having worked with AWS, xen had a lot of very bad vulns over its lifetime as well.

there's no magic.


Now compare to Linux, which is the point raised. Nobody said alternatives like Xen are perfect... but my oh my, some are much better in this regard.

This is not a new vulnerability, it's just a description of how to exploit the PwnEscape vulnerability in practice on firecracker VMs (disclosed multiple weeks ago and discussed in some less upvoted posts on HN)

> already discussed at length on HN

Couldn't find it. Care to provide link.


The actual vulnerability got very little discussion. It's clever. Because Linux networking people are obsessed with checksum performance, virtio-net plumbs checksum offload metadata through the host/guest driver interface. Guests can thus deliver the host packets along with instructions on what to checksum. The vulnerability is a missing sanity check that allows checksum instructions to instruct the host to overwrite the header length field in the IPv4 header, rather than the checksum field.

Crucially, this happens in the middle of packet processing, so the kernel code is forced to work with two different ideas of how long the IPv4 header is. All hell predictably breaks loose.


That’s fascinating. I’ve seen this sort of thing on protocol implementations in layer 4+ but it’s rare to find it in layer 3.

The removal of a copy across the guest host boundary in the name of performance here is irresponsible.


Additional color: per the commit, the bug was found by Paulos Yibelo, who runs PwnAI (the author of this piece), and worked on security at AWS for 4 years prior.

OP asked for a link

And honestly, how the &*DK is this ANYTHING related to firecracker?

Ah maybe not on HN, I forget where I learned of it. Either way, multiple weeks ago: https://news.ycombinator.com/item?id=50108845

Not beating dsv4.1 at all

Guidelines | FAQ | Lists | API | Security | DMCA | Apply to YC | Contact

Search: