You mention a lot of ethics concerns; I am totally uneducated on medical ethics but I think those are pretty important questions. It doesn’t sound like you’ve had a lot of support from this startup.

I might suggest to you that you publish this on a website - GitHub pages comes to mind as something Claude or codex could do for you easily - it would let you segment this information in a way that someone who wanted to help with the medical questions could get right to that stuff, and someone who wanted to help with ethics could also jump to what they were interested in.

Either way, thank you for the write up! Your curious and open attitude is lovely.

https://www.sciencealert.com/here-s-what-happens-to-spiders-... (with video)

The problems of inference proof for a closed model provider are more difficult than an open model provider -- you can get a probabilistic proof much more cheaply out of four independent inference providers running the same non-batched prompt than this 90-100x compute hit.

Anthropic and others also have a significant problem with these proofs in that they have a lot of incentive internally to have some portions of the pre-prompt secret; I haven't read this newest scheme closely enough to determine if they have a way to hide parts of the prompt with the proof. But if they don't, they are almost certainly not going to want to send these proofs out in the general case.

Last, I'll take this moment to soapbox and say for the researchers out there -- provable inference is nice. Provable training corpus information would be spectacular. It would be very, very, very nice to provide a set of model weights and a proof that they had only been trained on a dataset. To be clear, this is a really hard problem, but it would change fundamentally what levels of trust we could have about a given set of weights.

Is it hard? Seems like it isn't that hard if you use NVIDIA's remote attestation/enclave support instead of trying to do everything with ZKPs which have always been something of a dead end. CC overhead 2%, ZKP 100x+ and gigabyte sized proofs - just for inference. It's never been competitive.

I'm very skeptical anyone would care about this for training though. I mean, if there's someone out there who genuinely really really wants proofs of the dataset used to train model weights, let's talk? But it seems the value is mostly for open weights inference providers. They can prove what weights they're serving and they can prove they can't see people's prompts. It would take a lot of the risk out of using a no-name LLM cloud service.

Unfortunately, after many years of this stuff I have become old and cynical. The stated/revealed preference mismatch is huge. The majority of people say they want privacy in theory but in practice won't trade anything off to get it, and especially not convenience.

I’d love to read a plan that lets someone trust a pile of weights that comes with some sort of provenance proof. I’ll happily call it and you brilliant even if it was really easy to write and come up with. All my ideas for it look really difficult.

Here's the architecture I'd use. I'm going off of NVIDIA's current recommended stack / what seems most mature, if I understand correctly after a quick chat with GPT.

Basically I'd start with a bunch of Xeon 6s (Granite Rapids) connected to Blackwells. You could do it with AMD too but I trust their stack less. That gives you TDX as a starting point, which is pretty mature by now. I'd prepare a very minimal Linux image in which the OS boots straight into running a training server. You need the usual stuff like dm-verity or IMA to ensure the disk image is covered by the kernel image. The image also contains NVIDIA's driver and attestation SDK. Boot it all inside a TDX VM with GPU passthrough. I believe that currently uses encrypted bounce buffers so there'd be some perf loss, but supposedly there's some PCIe extension that can be used to eliminate that (I haven't investigated).

The trick is to combine NVIDIA's RA with Intel's. The best way to do that is just hash the evidence from the GPU attestation process into an intermediate data structure which also contains the usual stuff (e.g. ECC pubkey, nonce), that itself is hashed into the user slot of the TDX RA and then bundled all up together. After doing this you have a single über-data structure which can be used to prove that the given public key was generated inside a genuine Xeon TDX VM connected to a genuine NVIDIA GPU running in CC mode. I'd probably just use a zip file to collect all these bits of data together, for convenience when implementing the verifier. It might be worth including an RFC 3161 timestamp over all that as well. The RA protocols for something like this are always very tricky: I'd need to do some careful design work before committing to anything so don't over-read this comment.

Now the TDX RA covers the training server (indirectly, via the dm-verity root via the kernel args+hash). What is this server? It's training code (e.g. PyTorch) combined with a server that accepts document batches and - for more advanced training - maybe stuff like AllReduce traffic. Each time a document batch is accepted each doc is hashed into a sorted Merkle tree. The weight updates are then computed on the GPU and the results are hashed into a second accumulator.

Each node creates a blockchain-like log in which each record is signed by the static key embedded into the TDX RA. The records point at the hash of the previous records, and contain the current value of the Merkle root and weight accumulator. A verifier can rebuild the roots by iterating over the training set themselves in the right order and just hashing the text. I won't go into the details as you know how all this stuff works already.

I'm handwaving a lot of details around parallelism here. For a large scale training run with many nodes you'd need to solve a bunch of details like making the training servers verify each other's remote attestations before they join the cluster, etc. During the AllReduce step nodes would need to verify the data comes from a authenticated TDX VMs before accepting the gradients.

Anyway from time to time the weight updates have to be applied and the log also records the hash of the full sets of weights at that checkpoint. The final model consumer can be given the weights, along with the signed logs generated during the training run. They verify the chained RAs, verify the signatures on the logs, and if they wish, verify [a subset of] the hashes of the documents in the open training set.

Because the document Merkle tree of each batch is sorted, you can build both inclusion and exclusion proofs. Verifying an exclusion proof is more expensive as you'd have to check each batch versus a quick Merkle branch check for inclusion, but exclusion checks can be done in parallel. Normally in ML documents are shuffled rather than sorted, but the training server can sort the set for building the log entry Merkle tree, and then submit to the GPU in a shuffled order.

The bigger issue for exclusion proofs is that SHA256 is too rigid. You might need some sort of simhashing scheme to run along side it. Dunno, needs more tightly specified requirements.

Most do not even realize they have a choice and are not educated enough to make the one in their own best interest. Fast food wins as a staple for many people over a healthy home cooked meal.

Creating healthy and safe choices and making them the default is some of the most important work there is.

If some lenders go bankrupt on a datacenter note post-buildout and the capital structure is reworked, lowering the costs by removing capitalization costs from the cashflow of the inference provider, that is all to the good of people who did not do the initial lending.

I think that is extremely unlikely, but it's good for datacenter doomers to think through these scenarios IMO.

I am not sure whether the dot-com boom was a net-positive, and the AI boom may be less productive.

For reference this 'massive build-out' resulted in, according to a Berkeley study I found, 500MW of total datacenter power in the US across 320 locations at end of 2020. This wasn't too much for very long: as of 2005, according to Tier1 there were 511 datacenters. And it was likely only really a year or two: Dupont's VA3 utilization numbers published it at 50% utilization in 2003, 89.4% in 2004, then full thereafter.

Meanwhile, these infrastructure improvements allowed, for example, things like the iPhone to be built and deployable -- there was 3g, there were a LOT more web devs that could create the early (web only) apps the iPhone needed to compete. This in turn allowed another round of investment to be built on top of all of that.

On the capital side, investors were also able to buy Amazon at around $1 / share. AAPL at iPhone launch was $7/share. I haven't split adjusted either of these, this is just from memory.

I don't think a bust was necessarily good but the boom undeniably built out infrastructure, a small fraction of what the world is currently using, and did it pretty quickly, and then post bust that infra got bought out in a capital efficient manner. I think that's the bear case here.

Put another way, the Dotcom one boom was wrong about two things: timing was too fast, and the predicted scale of change was way, way, way too small. They undersold the future.

In that scenario the GPUs are a loss. But really at this point inference is so profitable it's hard to imagine these data centers not being able to at least cover operations & rent.

Capital waste is only capital waste if the physical goods that were built are no longer useful for anyone. Otherwise, it's only a waste for the people who lent money.

They were underpants gnomes who had no realistic way of making money.

The Google and antrhropics of today are the ISPs of back then. They didn't go anywhere. It was more the services on top of that that crumbled because they had nothing real to offer. I see the same here.

In 2001 society wasn't ready. People didn't yet want to order their groceries online or read books off a screen. The computers weren't ready, they were slow, heavy and clunky. The smartphone still needed to be invented. Mobile networks needed to come around, the miniaturisation of GPS, cameras. These things can't be rushed and that realisation popped the bubble. In the end they did happen, but in their own time.

I see so much of that now. We must have AI in our product! How, why, what value does it add? Who cares! It must be AI or we're out of the race!! That's really the hallmark of a bubble.

Here again, the usecases aren't fleshed out yet (except coding). The tech isn't finished, it's all very hit and miss, hallucination, loss of control (agents hacking) etc. The people certainly aren't ready, we need time to adapt society to new technology and a couple years isn't going to cut it. But these investors aren't investing this much to see a return in 2035. They want it much sooner. Sooner or later that will dawn on them and they start pulling out.

no it absolutely does not

Like, trains are an incredible technology which transformed the world, but very few people go around counting their lucky stars that they live in a world that permits such wonders. It's a train, it gets things where they need to go, and by the way isn't it so annoying that sometimes it doesn't quite arrive when the schedule said it would?

I'd like to imagine that we'll get something similar here sometime in the next ten years -- I understand there are (tens of?) thousands of these scrolls still to be scanned much less unrolled. Just an incredibly cool thing to be doing.

> SINCE THE EARLY 2010s, QUILICURA HAS BECOME A MAJOR HUB FOR A.I. DATA CENTERS THAT USE LARGE AMOUNTS OF WATER TO COOL DOWN SERVERS. BECAUSE OF THAT, COUPLED WITH AN ONGOING CLIMATE CRISIS, CHILE IS FACING A SEVERE DROUGHT EXPECTED TO LAST UNTIL 2040.

I do not believe that data centers (Were there AI datacenters in 2010? No) are causing a severe drought in Quilicura that will extend until 2040. I believe climate change could be causing such a drought, and I believe a drought is a terrible thing.

Estimates of water used for cooling are about 0.3mL of water per chatbot query; the 25,000 prompts that this really awesome and fun project answered resulted in 7.2 liters of water "saved" from use as cooling. I would imagine these 40 people drank significantly more than that during the day.

I put "saved" in quotes because no water is destroyed during datacenter cooling. Some (the parts sent to open evaporative coolers) goes back in the atmosphere, and I imagine disperses over a pretty wide region. The rest of it is recycled locally, whether into the water table, or I would guess more likely back into cooling another round of queries.

Anyway, it's fun. But I guess I'd prefer something that is more honest about the actual uses and cycles here -- it doesn't do a local community (or a global one) any good to misinform them about the situation; to their point the situation locally sounds like it could be pretty dire. Ultimately I think it risks falling between performative and exploitative.

I'll probably post a follow up if I get anywhere with this!