Agreed, I don't know much about JEPA but i wish this article had explained / motivated the memory graph more. "3. Fill gaps in memory with transformed memories" in particular seemed really really damning for the models—when you removed the examples from the demonstration that were near the goal state, the model completely broke. It seems like the only way you could make the model work again would be to manually flip and transform the memories it did have so there was something in the vicinity of the goal location.
That seems like a really massive failure of the model to generalize? This doesn't seem like a new technique, it seems like you've discovered a pretty big issue with the fundamental setup
I think most people (rightly or not) feel pretty negatively about the concept of a project having hundreds of unused dependencies. If you've lost track of your codebase to such an extent as that, you should probably expect a little bit of social pushback from people who pride themselves on understanding their code and being responsible for it. It's just a fundamentally different and in some ways upsetting mindset to the average programmer.
Furthermore, I have also noticed the error. I can still see it happening. Sandboxed agent, no access to global git ignore. Creates a new one after putting the secret in `.claude`. Same developer, same machine, fresh sandbox. No access to the original sandbox's global gitignore. Secret already on the project. Commit and push. Secret leaked.
What sorts of processes related to secret storage were you able to shift off of team computers? From my perspective, if I need to get e.g. an API key from Twitter and put it into my prod environment, i have to be doing that with my computer somehow. There's still millions of different types of secrets that need meat puppets to shuffle them around.
Yes, the developer documentation used to publish the string until May 2026, when Anthropic removed it and removed the feature from the API. Sonnet's explanation was correct, just out of date. https://web.archive.org/web/20260202050057/https://platform....
That's the only known impact so far. But we don't know the details of how these keys were exposed, so it's not entirely clear why or how "only keys" might have been affected
Okay, and when someone on a laptop can run an open model that can infect and take over IOT devices and then that model uses its pattern recognition to do so and then set up its own harness loop, what would you call that except replication?
Is that any different than any worms developed in the past decades? If you want to call them worms/virus/replicate (ie using real world names for their digital behavior) I'm not trying to police that. I just don't see the difference between a worm/virus created today with an LLM vs one 'yesterday' without an LLM.
Sure, the difference is that the cost of making and maintaining a worm has gone from "many skilled developers working full time" to "$20/month + an internet connection" and, at current rate of progress, probably to "a local open source model on a laptop" within 6-9 months
You can't even imagine a future where the LLM is the worm.
No, we've entered a new sci-fi future where intelligent agents will be able to do the fraud without human monitoring and input. This in itself is set to massively increase losses to digital theft and fraud alone before all the other problems.
It isn't just restricted to the exploits it is programmed with (and without human intervention to load additional exploits).
Of course, it remains to be seen what several thousand 27B models running at sub 1 tps speed can achieve. (The IoT device mechanism proposed by Anthropic).
I suspect a more likely path is compromise of the billing and provisioning systems of fly-by-night neoclouds.
It would still be running on that original laptop, no? So what did it replicate then? It can only possibly run on multiple thousands of dollars of compute and memory, and certainly not on an IoT device. The idea that AI is going to spread by itself is science fiction until compute and memory are so cheap and power efficient that my fridge can run it.
... No, that was the entire point of Anthropic's report on replication that GP was discussing. Did you read it? https://www.anthropic.com/research/self-replicating-ai Anthropic was able to get a large open-source model running (slowly, but running) across 5 networked consumer IOT devices ($50 TV boxes running Android). Not a stretch to say we could see this capability become much easier for to most agents in the next 12 months, especially as companies are able to distill smaller and more efficient versions of their models.
That seems like a really massive failure of the model to generalize? This doesn't seem like a new technique, it seems like you've discovered a pretty big issue with the fundamental setup
reply