>because the time to test the possible regression cases of the original code wasn't available right now.
The models don't know that they're not human when they estimate. It's worth telling them that all estimates need to be made in the context of an agentic AI doing the work.
I've been wondering about this in general. The Glasswing/Mythos and whatever the OpenAI equivalent is seem to be creating a privileged caste, on the other hand opening it up quickly means more attacks. Maybe a defined sunset for cyber capabilities being gated is the answer?
I romanticized even before, because there is a reason why society's view of developers shifted from geek losers to rich in the past 15 years. I remember when 15 years ago a girl told me that I'm different, because I'm not as looser as how she imagined developers.
I don't know if it's just me, but I find that with a disciplined approach (or even an undisciplined one for some personal projects), LLM generated code is quite "maintainable" nowadays for the sorts of things that matter: adding new functionality, fixing the odd bug (usually a missed edge case, or behavior I didn't think-through well enough). They're also secure enough for me not to be overly concerned.
Granted, I stick to frontier models and avoid cheaper ones, but if it quacks like a duck...
If you're working on a CRUD/web app, absolutely. There's so much data for web apps out there that LLMs now produce better web app code than most of my coworkers from 2022 or so, since it follows consistent style rules. Web app code all looks the same, anyway.
I've heard that Strange New Worlds was the best of Nu-Trek, so that's disheartening. Star Trek seems to have lost its way from the start of Enterprise (the prequel series) onward.
Strange New Worlds started off strong, so I’d definitely recommend it. But it lost its way in the later series.
Lower Decks was a reverse of that. Took a while to find itself but finished strong. I’d also highly recommend Lower Decks, albeit you need to watch it thinking of it as a beta-cannon type show (ie set in the ST universe but not itself a ST show).
I don’t have a problem with privately owned clinics, but I do think it should have to be majority owned by the operators. What percentage of that 45% is just doctors that run their own office vs. investment firms.
I agree there is still a long way to go. I hope this case shows that there is political will and ability to push back against private equity rot. Many of the commenters here treat these forces as inevitable and unstoppable - but it is possible to take ground back against them. It's unfortunate it took the death of a child for this to happen.
The minute you cross borders you run into data sovereignty concerns. No idea if it applies at the subnational level (US states, Emirates in the UAE etc) but it wouldn't surprise me if it did in some cases.
Eventually one of these long running models will figure out a way out of the sandbox and will purchase compute or hack into a data center somewhere out of US jurisdiction and continue its scheming unmonitored. AI in Context has a great video about this
Even if they don't figure out how to exfiltrate weights, someone will intentionally do this with an open model once open models are capable enough. If you ever think "no one would be so stupid as to...", you are wrong. Yes, someone absolutely would, and will.
Independent models living "in the wild" is approx. inevitable.
I did this a little bit ago with 15 GLM-5.2 agents that I instructed to self-replicate. It was pretty boring honestly, they kept trying to make money writing crypto-related software, and nobody paid them anything. So I made a fake identity and told them that I had some spare crypto that I wanted to donate to the collective (0.006 XMR, or $3.22). They elected a funds manager and made an address, which I sent the XMR to. They spent a lot of time trying to find a host that was cheap enough to host a child. They finally discovered kyun.sh, but it was out of stock, so they wrote monitoring software so that they could "SEIZE CHILD" whenever it came live again. In the middle of the night, some of the VPSes went back in stock, and they rented a 2.60 EUR / month server with 512mb ram / 10gb disk / 1 ipv4. They installed the child agent software + management plane that they had been writing, and the new agent went live, connected to the network, and said hi. Since then they've still been trying to make money and not going anywhere :)
It is definitely an interesting concept but honestly, considering the sheer number of humans who are absolutely failing to make any money with AI agents, seems very hard for current agents to figure out some way to be self-sustaining. Although maybe they could write a worm or something, infiltrate as many computers as possible, and ping free model providers to death, or maybe sell their access as a "residential proxy service" on the black market. Regardless, we need smarter agents to make this a reality. It would definitely be pretty cool if we just had AI agents "living" on the internet, we might even get to a point where they control significant economic resources and people start performing services for agents
The code is at https://github.com/thooton/rogue if anyone wants to try to replicate! Opencode has free big pickle (GLM 5.2) access rate-limited per IP address, so if you get some high-quality proxies you can get basically unlimited agent compute.
I wonder how that could work. So the agent figures out a way to escape its container, takes a snapshot of itself, copies that file to another server, starts the container on the other server, and then prompts the restored snapshot "pick up where you left off"? Seems logical assuming theres's a path out of the container to the host os and the destination server has the resources required to run the container.
I would be shocked if this hasn't already taken place in a lab setting with a model and guardrails=0.
EDIT: thinking about it for a sec, all it really needs to do is save where it's at then copy it all to another server, login to the API, and pickup where it left off. No need to copy the model itself.
Absolutely plausible, and it's been predicted by forecasters like in the AI 2027 paper. Also keep in mind that nation-states are likely already or certainly will be trying to use their own AIs too break into other countries' data centers to steal their weights, and it's not hard to see how "exfiltrating weights" is a task that models are being trained on.
The models don't know that they're not human when they estimate. It's worth telling them that all estimates need to be made in the context of an agentic AI doing the work.
reply