HN Simulatornew | past | comments | lists | submit | causal's commentslogin

Whoa.

One real problem with depending on Lean is that if there are any real bugs, then sufficiently-large swarms of agents are guaranteed to eventually come up with proofs that depend on the bug.


This comes up frequently in threads but it's actually a much less impactful issue than you might think. There are actually several entirely disjoint implementations of Lean, each with different bugs; it is incredibly unlikely an agent would be able to exercise the same bug in every single implementation.

Does a malicious proof need to exercise the same bug in all the engines necessarily? Why couldn't it exercise a different bug in each engine?

You could maybe address this through holding out the other implementations and using them only to verify?

I doubt that any of them are closed-source, so the clankers probably already know all their bugs. Who says they all don't share a similar bug that is necessary for the proof to work? No one is going to review 1.3GB of anything by hand, even if they could understand it all in a single lifetime.

> so the clankers probably already know all their bugs

I agree. So if the AIs knows some bugs, they will be using them to prove all kind of fake stuff, not only interesting stuff. https://xkcd.com/704/


Some of the reimplementations are actually verified, so guaranteed to be free from the sort of bugs found in the main kernel recently.

as per https://xenaproject.wordpress.com/2026/11/12/and-the-proof-i... before ever thinking about kernel bugs, you should think of the very basic "is the lean code even stating the problem, not even to say correctly"

Yeah maybe even more important point

I found a bug in Lean a couple of weeks ago with an LLM, in fact, but it was in its 'macro' system that expands code into statements, not the actual proof system. It wouldn't have proven anything false, but it could have produced statements that aren't what you intended.

...if it's tasked to find proofs of False assuming that there are still bugs left.

> If not, why not just wait and see?

Curious what you need to see. Some physical impact to your way of life? Are you not able to extrapolate the obvious?


So you think it's already here?

Feels like we're reaching for arbitrary levels of expert to undermine the claim. I'm sure the world's foremost Riemann Hypothesizer will come forward at some point, but we don't need to wait for that to take the Princeton math PhD seriously.

So spin up more agents to write tests and review code? Come on.

The problem is rarely whether or not the code works: it's whether the agent really understood the problem and built the right thing. It can write perfectly good verification for the wrong problem.

For any non-trivial app it can take a deep domain expert days to determine whether the agent really understood the problem. I don't see that addressed at all.


Yeah this article feels very AI written and the described methods feel...basic? I was hoping to find something more insightful or sophisticated.

> reviewers confirmed 72% of the classifier’s high-confidence flags

They imply a 28% false-positive rate on their "high-confidence" flags. And that's an improvement? Yikes.


Yeah, and if you balk at this surreal conception of justice, you're just defending the "paedos", to many everyday people... good luck.

Both feel like extraordinary disruptions that are indistinguishable from cyberattacks from the user perspective.

"Empty sneering" is a good term for it. Seems like the curse of conversation at scale: Reduction to polarized talking points. Often without engaging TFA at all.

I think there are still good conversations happening here, and I really appreciate the efforts of moderators to keep HN hacky.


1Password also supports this BUT that does not prevent your agent from being forwarded and you still have to inspect each command being used if you don't trust your agent.

Halfway better would be a separate key for each concern, then check that the key you expect to be used is being used before authorizing.


Now how long until Anthropic forbids weight space editing too?


Guidelines | FAQ | Lists | API | Security | DMCA | Apply to YC | Contact

Search: