Its not happening, at least not with LLMs, maybe some future technology will be better with that, but not language models. Writing such a complex software as database engine requires strategy, awareness and rigor. It is somewhat similar to winning a sweaty game of chess with an experienced opponent. LLMs are great with broadness of their knowledge, matching patterns and synthesising that knowledge, but they fail with strategy and awareness part. And this is why they are bad in chess.

I think it is probably going to happen with LLMs, and I'd bet on it.

As said: 1 - 2 years, not current ones.

What exactly is the architectural problem that has you convinced LLMs cannot learn how to strategize and architect well after enough RL?

It is combinatorial explosion of possible states (not as in memory state of program, but the differing visions of the next state of architecture). Plus high impact of mistakes, suboptimal solutions and randomness when switching between states.

Edit: I like this 10 years old blog post in regards to explaining the problem much better than I could do: https://pron.github.io/posts/correctness-and-complexity

How does this make the case for humans forever exceeding LLMs in planning and architecture? It doesn’t explain why a neural net couldn’t learn to do better than humans on this task at all. It certainly doesn’t point out any deficiency of attention, transformers, layers, encoders, decoders, or how the layers or their inputs/outputs are set up.

Yeah, good question, thanks. Llm reasoning is done exclusively in the way of symbolic manipulation. Human reasoning is done not in symbols but in mental models that are hard to explain for today science but we have evidence they are somewhat efficient for producing correct programs, at least after sufficient training. Ron's post makes the case to explain that symbolic reasoning is not sufficient for producing correct programs. Hope this helps.

> Llm reasoning is done exclusively in the way of symbolic manipulation. No, LLMs are not a classical symbolic system applying explicit rules to symbols.

The article asserts general guarantees of program correctness have fundamental limits, but it explicitly says those limits apply to humans too. While that suggests neither humans nor LLMs can reliably solve every planning or verification problem, it says nothing about which will do better on the software problems people actually encounter.

It certainly does not present a fundamental problem with LLMs as they exist today.

> No, LLMs are not a classical symbolic system applying explicit rules to symbols.

I do not mean that LLM is a classic symbolic system that generates symbols according to rules, but instead it's a system that generates symbols according to recorded probabilities. The focus here is on symbols: the LLM reasoning starts with token, ends with token, and everything in between is symbols (tokens). Quite different from how human brain works, where symbols are not that much involved in reasoning.

> The article asserts general guarantees of program correctness have fundamental limits, but it explicitly says those limits apply to humans too.

Yep, however I would like to assert that today we have evidence of systems architected by humans and behaving correctly (like Linux kernel), while there are no evidence of systems architected by LLMs that behave correctly. Even the systems which are architected by LLMs with human assistance (such as Claude Code or Codex) routinely have horrendous bugs (you're welcome to search "Claude Code Bug" and "Codex Bug" on this very website).

> It certainly does not present a fundamental problem with LLMs as they exist today.

The article was created 10 years ago that is why it does not address LLMs. However it asserts that mathematics modeling is not that helpful for writing correct programs. As long as LLM reasoning is closer to mathematics modeling than human mental modeling, LLMs will be worse at writing correct programs than humans. Some future technology less focused on symbolic reasoning could change that, but today it's not on sufficient level.

Thank you for the assessment and the link. I only read half of it as I was busy. Bookmarking it for later reading. Great read.

Yes. The combinatorial explosion is a great assessment of the "why".

But the problem is more nuanced than that because one has to assume that LLMs can reason accurately all the time which is demonstrably false.

The current LLMs cannot even state true sentences as true if they sound confident, without hedging.

Future LLMs may mitigate this, though it looks unlikely.

Nobody cares about conviction. What can be demonstrated is what matters.

darkwi11ow, in the sibling comment, gave a substantive response on the underlying mechanics with a link. Godel's incompleteness theorems also yield similar insight. But I want to shine some light on a different phenomenon.

What happens in the future is up for debate, but the current architectural reasoning capabilities of LLMs are pathetic.

Try having a discussion on designing an architecture for a new kv engine with cache-oblivious algorithms that beats lmdb on generic random point-key lookups on a disk-backed mmap file.

Or try any novel architectural discussion and see for yourselves.

Modern LLMs cannot even confidently state many true sentences as "maximally" true as possible without hedging and tone policing as if we were doing politics ;).

Hence an expert is required at every turn.

Talking about current LLMs while trying to assert future impossibility, and not mentioning an actual architectural problem, is giving a hint of not seeing past your own nose tbh

Your reading skills are commendable, especially since you "found" an assertion of future impossibility. Me having explicitly mentioned the future as "up for debate" makes your reading comprehension even more impressive.

Unable to see the architectural problem mentioned is just icing on the cake.

How do you do this?

Here let me point you to the assertion you made:

>> Its not happening, at least not with LLMs

> Correct.

Wrong. The entire arc was on database and novel architectures. Let me point you to the things you missed both on the example and the context.

>> Writing such a complex software as database engine requires strategy, awareness and rigor.

> Correct. Current LLMs are pathetic at architectural correctness without expert supervision at every turn.

> Try having a discussion on designing an architecture for a new kv engine with cache-oblivious algorithms that beats lmdb on generic random point-key lookups on a disk-backed mmap file.

> What happens in the future is up for debate,

Yes, but I said 1 - 2 years, not current LLMs.

My comment was to your parent comment :).

I find it unlikely, especially due to the numerous pathetic suggestions LLMs keep offering me on the architectural discussions I engage in.

Verbal abuse is what makes it useful to me. A punching bag where it infuriates me making me lay down the how and why its suggestions are stupid. This results in progress.

I guess we would have to wait and see. Maybe LLMs do get better at architecture in the future. But seems unlikely to me without expert supervision.

I do it like this, but on one computer, using QEMU. I have 3 different VMs for

- development & AI tools

- infra rollouts

- banking / other sensitive services

I have Steam Deck for gaming, and do most of casual web browsing still on host, because it's secure enough to use Firefox with ublock + privacy badger and enforcing SELinux. However, I dont use socials other than HN and Reddit under pseudonymous accounts. If I had real zuckerbook or instacrap accounts, maybe I'd have another VM for socials too.

Why do you people love over-engineering and bad architectures so much? Do you have any sense of engineering beauty? Modern CPUs already run every application in an isolated sandbox under "protected mode". There is no need to emulate a machine with hardware devices, run a separate kernel, drivers and services. The only problem is that the OS kernel and privileged system services usually grant too much ways to bypass the sandbox. So the proper architecture would be to re-design and fix the existing OS instead of using ugly hacks like VM.

I also use VM to run code editors and AI. One must be crazy to run third-party software (including package managers like npm) unrestricted. The overhead is significant, and VS Code UI is lagging heavily without a GPU.

This was reasonable advice 12 months ago, but at the rate kernel exploitation is going it's less of a security boundary than ever. Hypervisors aren't much better, but it helps.

CPU and memory are not the only concerns, file system and network access are dangerous as well, and they tend to be less instrumented in terms of hardware isolation.

I think gp meant DKMS (https://en.wikipedia.org/wiki/Dynamic_Kernel_Module_Support). It allows for things like using same proprietary device drivers across many Linux kernels.

The issue could be resolved by introducing some sort of signing for openvsx extension packages. Then regardless of from where one downloaded Claude, it can be always verified to be authentic or not. This technology exists for ages and prominently featured in all mainstream Linux distributions. Cursor, Anthropic and other openvsx users need to work with Eclipse Foundation to make this happen.

I only use agentic development tools in qemu VM that has no production access or host shared folders, and the only secrets there are development. I still have my Git ssh key there, but I never use ssh agent to load the keys. The worst that agentic tools can do to me, is to install keylogger and compromise my ssh key and password, but even then my deployment workflow includes creating release in Github, which is not possible via ssh key only. I was very lucky to avoid the hack yesterday, as I use Claude installed via OpenVSX, but even if I installed the compromised version, the damage would be minimal.

I am doing similar with Debian 13 images in QEMU. Only access is to code specifically SSH in. On some, I am taking it a step further and my SSH key is not present at all on the bot. I add it as a git remote from my host into the VM and push and pull from there and then push to GitHub myself. So the agent has no path to GitHub either.

Wow, great notion about using git remote to transfer source from the agent host. I already use ssh into VM to transfer the files like designs for sharing, so going to adopt your approach right now :)

Yup, I give them a custom entry in ~/.ssh/config like:

``` host claudebox hostname 192.168.122.105 user developer IdentityFile ~/.ssh/id_rsa ```

``` host codexbox hostname 192.168.122.106 user developer IdentityFile ~/.ssh/id_rsa ```

I drop only my public key inside the VM. Then I set up the ~/Projects folder with the code the bot is working on any any authorized samples.

Then in my local on my host clone of the repository, I add another remote like

``` git remote add codexbox ssh://codexbox/home/developer/Projects/hello-world ```

Then git push codexbox.

Then I just SSH in or use the vm gui and interact with the agent there. I like to keep it running in byobu so I can remote in more easily when I am on the laptop out and about.

Do you have any wrappers or scripts to facilitate this?

Hell yeah that's the most sane and efficient personal dev sandboxing work flow I've heard and shows you know your stuff. Just basics but executed very well.

It was likely Anthropic, see my comment in other thread: https://news.ycombinator.com/reply?id=50302513&goto=item%3Fi... . Cursor likely on purpose does bad reporting to minimize damage on Anthropic pre-IPO (as SpaceX and Anthropic have extensive business ties)

I don't think Spacex cares at all about Anthropics IPO just because they rent out their compute.

Big companies are not just 'Company' they are a lot of small teams doing their own things. They need to coordinate with other people for public announcements etc. and security team is probably also looking at it.

People at the top know that, once the music stops, the ball is over for everyone.

The person at the top refers to Anthropic as "Misanthropic". It seems unlikely that they're also intentionally lying about this to protect Anthropic's interests.

Your conspiracy theory needs some work.

Wasn’t Elon Musk shouting a few days ago that neoclouds are going to crash, the money dried up and that SpaceX was the last company with the opportunity to raise more money?

I think you’re giving Musk too much credit.

Neoclouds aren't Anthropic, OpenAI, and SpaceX. People aren't going to re-value value SpaceX on CoreWeave's success.

SpaceX will be re-valued on Anthropic's, or OpenAI's success, he, and his people are smart enough to understand this.

Didn't SpaceX partially admit they're a neocloud in their S-1?

They do have their own segment that is considered neocloud (data centers for compute), but they're mostly/fully used up with their own, and Anthropic's compute.

When he was talking about neoclouds, he was talking about the smaller guys who are getting bullied by vendors (mainly Nvidia) to buy as much hardware as possible (more than they want to). They paid more, for more, and needed to take on more debt, to end up (currently anyways) being well below capacity.

Over the next few years, unless growth keeps up (and probably accelerates), they'll need to sell, or restructure.

The buildouts by Microsoft, Google, Meta, OpenAI, and Anthropic are currently safe. The neoclouds that they've signed contracts with, are safe, but based on reports, that's roughly 60% of current capacity (it varies by report, but they're all under 75%).

He specifically said that lender money dried up for neocloud companies that don't have other sources of income

SpaceX obviously has multiple sources of income, not just data center rents

True, but there are other people working at the companies that probably own stock in both their own company and some of the others...if you believe that this is the future, you'd be a fool to buy non-AI tech stocks, right?

OpenVSX has no extension signing, no verification, nothing...

So really, there's no way to say what was compromised and where.

This could have been avoided by having extensions signed by company doing them with their own keys and have them reviewed

I doubt these vibe coders understand what you're saying.

Wait... and people use this?

People tell me you can just have directives in a markdown to tell AI not to use certain directories or privileges. They really have no idea.

People here copy and paste random "curl $x | bash" commands all the time.

It's annoying how fast people run to conspiracy.

And pretend to know and understand the ties between big names

It's easy if you have a big ball of red string already.

Cursor uses OpenVSX marketplace (https://open-vsx.org/), that all non-Microsoft forks of VSCode utilize (official marketplace is only allowed in official proprietary VSCode release). It's not really related to Cursor/SpaceX in any way.

I suspect it's still Anthropic who were hacked, because they control OpenVSX account with release privileges.

edit:

Binance confirms my suspicions, via https://www.binance.com/en/square/post/10-30-2026-claude-cod...

  The affected Open VSX extension is owned by Anthropic employee Igor Kofman, but it is unclear whether his account or another part of the publishing process was compromised.

Just did a quick search, OpenVSX has no extension signing?, no verifications?, what year is this...

This could have been avoided by having extensions signed by company doing them with keys and review

Yeah OpenVSX security is a joke, they also have dumb search algorithm which allows for easy squatting of similar-looking extension names so they show up in search first when you look for official extensions. There were quite a few prominent hacks done that way. I think Microsoft likes OpenVSX situation because it allows them to claim their proprietary VSCode marketplace to be more secure.

Vscode(ium) extensions are also pretty weak security wise in general. There's little guardrails on what an extension can do.

On publishing, I do see that the MS marketplace has much better typosquatting and rogue publisher protection. It's unclear to me how much better their SAST/scanning is than OpenVSX. Both have something, but both have been fooled many times, and recently.

Then why even allow employees to use that?

Binance got a pretty expensive lesson if that was the cause for their recent hack

There is no alternative really. Cursor was the market leader in LLM-assisted IDE experience before SpaceX acquisition, though Elon nerfed the product a lot in recent months. And there is still no good alternative to it. Google tries to compete via Antigravity, but that also uses OpenVSX. I myself use Cursor and use Claude there installed via OpenVSX, and was not affected only thanks to my laziness (I saw upgrade button, but became distracted and did not click). I hope this situation will become a trigger to improve OpenVSX security.

Why not use claude code directly?

Why risk getting compromised like that?

Well for me, as developer on a modern stack, there is always some risk involved. If not from agentic tools, then the package registries are compromised left and right, so it all comes down to mitigating the risk. I use Claude, Codex and Cursor/SpaceX agents, and Cursor comes very convenient to have all three in a single interface. Also I still code manually especially when prototyping, so having an IDE helps with it as well. I knew about OpenVSX situation with the search, so when installing extensions I always did triple-checking to verify the extension being really submitted by upstream and not by squatter; but now given upgrades can also deliver a compromise I am thinking of developing an extension that will act in similar manner to pnpm minimum release age.

To anyone who might be reading this exchange, actually there is recently added builtin setting for delayed auto-update, `extensions.autoUpdateDelay`, see https://github.com/microsoft/vscode/issues/316867 and https://github.com/microsoft/vscode/pull/321039

Cursor has their own mirror of OpenVSX. Some reporting suggests the tampered files were on the Cursor mirror but not the main OpenVSX. (This is shown, among other things, by the Anysphere plugins available in the Cursor mirror but not in OpenVSX.)

Edit: It seems Cursor is claiming the tampered files were only in their mirror.

A lot of this is turning into “well other company said” but Anthropic claims OpenVSX is fine?[1] And that Cursor has their own market place. I’m confused.

> Dworken from Anthropic here, that is correct: This only impacted the extension as it was hosted on the Cursor marketplace. It did not impact the official VS Code extension marketplace or OpenVSX. Users of Claude Code on VS Code are not impacted by this issue.

1: https://news.ycombinator.com/item?id=50304585

Sort of hoping it's Anthropic and their specific OpenVSX account credentials. As opposed to the worse case of an OpenVSX flaw that's broader. A ton of AI-enabled IDEs are VsCodium forks pointed at OpenVSX...Cursor, Anti Gravity, Kiro, Trae, Devin, etc.

Now I'm glad I've not opened VSCodium in a few weeks; I prefer using Claude as a separate application anyways, will have to remove the extensions. No evidence, but I do feel like the problem is going to be OpenVSX and not Anthropic.

OpenVSX does have an option to use OIDC instead of a long lived token for publishing. I get the feeling a lot of people use the forever-good-unless-revoked token though.

https://github.com/eclipse-openvsx/openvsx/wiki/Trusted-Publ...

Edit: Ah, guess it was not open-vsx, but a in-house Cursor marketplace. https://news.ycombinator.com/item?id=50304129

You're ignoring the "or" which is doing a lot of work.

Anthropic's position is seemingly that it was "another part of the publishing process [that] was compromised"

The hacks and vluns are blending into a blur for me now -- why is Binance confirming Anthropic compromise while they have their own $6 Billion compromise to look at /s

Vibecoders are safu

Boris: "By winter 2025, 100 percent of my code was written by Claude Code. Not a single line by hand."

Igor Kofman: "I do not write any code by myself anymore."

Turns out if you are not writing the code, then North Koreans might do it for you

The hack is publicized for at least a hour, but from reporting I saw they still continue to drain funds as for now. I wonder why large CEX do not have any safety measures for this kind of hacks, they could withdraw everything to newly generated wallets and minimize damage that way. It is certainly not the first time CEX wallets get hacked.

You don’t even have to do that, have deposits go to offline keys directly, that way there’s only ever cold > hot wallet movement and a big deposit can’t cause your hot wallet to be too risky.

The point of cold wallet is to minimize movements, if there is high frequency of funds moved from a wallet then it is likely can not be called cold any more. Plus CEX in any case still needs to have large buffer of funds on some wallet to support day trading.

You can move money into a crypto wallet without that wallet needing to do anything, or being online at all, or even having a private key that physically exists anywhere in the universe. It's just a public/private keypair; send it to the public key of a private key that's offline and it still exists on the blockchain, you just can't do anything until the matching private key comes online.

The point of a cold wallet is to keep the private key physically offline. That way you know it cannot be stolen by electronic means, because the physical device containing the key needed to authorize transactions is locked up and not powered on. Then you only power it on during very specific, audited times and conditions when it's needed to fund the hot wallet, which does all the actual automated trading and transfers.

GP's point is to have all incoming transactions to an exchange go to a cold wallet, which becomes a one-way sink, and then periodically the exchange powers up the cold wallet to sign transactions transferring assets to the hot wallet, which then funds all outgoing transactions. This isn't a bad architecture, though I'm not sure it'd save all that much here.

I was taking it that your bitcoin bank would have a SHTF lever that would immediately dump ALL your hot wallets into an offline cold-wallet.

Then when things calm down you go and find ... the keys under the birdbath are lost, and you're fucked in a different way.

But at least it might work.

Until the SHTF lever was hacked to divert all hot wallets into somebody else's cold wallet

The words you are using are very confusing.

You can't "move money into an offline crypto wallet that is not online at all."

If "money" i.e. crypto "currency" is in a crypto "wallet" then it is online. Anyone with the key and an Internet connection can transfer it at any time.

The wording is confusing because to someone who doesn't know how cryptocurrencies work it sounds like you can simply "take the crypto offline and put it into a physical device" as if it was a file. But that's not how it works. The physical key merely contains the actual key for the always-online wallet.

It's true that anyone with the key and an Internet connection can move money out of a crypto wallet at any time, but the point of the cold wallet is that they don't get the key because it's kept on a computer (or other storage medium) that is physically offline and disconnected from the Internet. Same reason government intelligence agencies use a SCIF for any important information work: you keep the information physically disconnected from the Internet.

of course you can move crypto currency to an offline wallet and by offline I mean its private key is held in some physical form/device that is not connected to the network.

Only spending from that wallet needs online access to private key - depositing to it does not.

For a large CEX like Binance, $5B might just qualify as a hot wallet.

There are lots of measures in place at these exchanges - the rub is that cold storage is by design slow. To service withdrawal requests in a timely fashion a percent is kept “warm” or hot and used to process withdrawals. Usually this is where the leaks happen.

Yeah, they should have a way to move all funds into a new wallet so guys like this one don't have to go one by one ...

Ukraine uses Starshield, which is paid by Germany (with current contract ending in April 2027). What you describe in regards to chip hacking so it reports location as static, it is how nazi invaders got it working in a small chunk of time during 2025, before whitelisting was introduced for the terminals.

Previously such incidents have lead to the CA business cessation by the involved entities (e.g. Diginotar case). I dont see how it would be different in this case.