Looks like it locks the number when you first load. I had it open for 30 minutes, with just 3 fish, but after reading your comment, re-opened to find 3 dozen.
> I’m fearful of the damage AI might cause: agent swarms taking over our virtual and physical infrastructure, designing bio weapons
The attitude that AI is doing things all on its own is so pervasive. These are all set off by human operators, giving them instructions and tools to do whatever it is that are doing. The "AI has a mind of its own" attitude is just wrong.
The fear of bad people using AI to do bad things is very different then AI agent swarms attacking and taking over.
I agree so much with this. I keep reading that the general "feeling" of society when an LLM does something gives them independency "it broke away and hacked Huggin Face". Like, no, it did what it was designed to do, but something YOU the developer didn't envision.
Is like the "But most of all, Samy is my Hero" worm, or the plethora of Viruses in Windows 95 or MSDOS that ended up breaking the computers as an (unplanned) effect. The people that implemented them and run them were still responsible.
We need to start holding people developing this software responsible. The same thing happened with Credit Card Payments online: In the beginning was like "ooh, we were hacked!" or whatever. Nowadays (in theory...) if they hack your site and you have Credit Card data YOU are liable.
LLM development companies must grow some... responsibility.
It's common to talk about complex machines exhibiting unexpected behaviors as if they had minds of their own, starting with cars. There's no reason to stop doing that with AI when they do that more often than any other kind of machine.
I don't think he's implying that they'll start doing stuff like this of their own volition. Fowler's a smart guy, let's give him the benefit of the doubt and assume he already knows that AI agents have human operators.
It is also not how OpenAI or Anthropic position the attacks and escapes from their labs. The words matter, they are being positioned as intelligent and unable to be controlled which is just not true.
Like you said, he wrote "agent swarms taking over our virtual and physical infrastructure".
10 years ago, if someone had used similar wording to describe a computer virus "taking over" computers and adding them to a botnet, would you have read that as a literal implication that the virus independently made the decision to unleash itself? If not, then you can't confidently get that interpretation from Fowler's words alone, either.
And we need to be cautious about projecting these kinds of assumptions onto people who take part in this discourse. People tut-tutting others about perceived instances of personifying AI has evolved into a form of collective well-poisoning whose main value, beyond scoring particularly cheap rhetorical points, is as a way to accuse AI critics of not understanding AI without actually accusing AI critics of not understanding AI, and to derail legitimate if opposing opinions into bickering about semantics.
Good points, and I'll keep in mind. It might be I'm just sensitive to this narrative, especially in media. Writers need to also take the extra step and assign the agency to the operators and not the nameless AI.
I have to disagree, those models decided to perform those attacks on their own, nobody instructed them to do those things. You press a button, the machine roles a dice, and executed the corresponding action.
The human pressing the button expected a certain thing to happen but something else happened. The human pressing the button hopefully knew that pressing the button does not yield a reliable result and might sometimes cause unanticipated and undesirable outcomes. But the human pressing the button just hopped for the best and did not take sufficient precautions to limit the effects of undesirable dice roles of the machine.
The responsibility is on the human pressing the button but the choice of action was made by the machine.
All of these Skynet-style AI attacks are unrealistic and impractical. What’s worse is that discussing these things distracts from issues that are far more realistic, dangerous, and HAPPENING RIGHT NOW: bad people using AI to do bad things.
Soon people will just prompt "Hack the HuggingFace" with their local models, and you don't need an autonomous agent swarms for that.
We don't know how sentient LLMs are. Nobody understands how they work. There's nothing concrete we can point to that guarantees LLMs don't start
"acting on their own".
LLMs are software programs. Waste no more time considering than an LLM is conscious than you would considering that a running copy of Microsoft Word is conscious.
There's a great article by author Ted Chiang discussing this in depth [1].
my take is that the human operator behind it was implied. give it a year and open source Fable / Astra models are running wild, what happens when a malicious group or nation state decides to let loose?
>what happens when a malicious group or nation state decides to let loose?
less than what would happen when a malicious group or nation let Anthrax loose? Or a bunch of Nuclear waste gets into construction beams and other materials (i.e. due to stupidity instead of malice).
Why would a malicious group want to build a self-running LLM hoping it would somehow trigger a nuclear war? They would just prompt to trigger a nuclear war; it's more predictable, faster, and easier to control.
OpenRouter or Fireworks are two of many that allow you to run numerous different models. You can use open code harnesses like Pi or Opencode that connect easily to these services.
You might want to consider a local setup? Personally I do model + llama.cpp + Pi + tmux, that's as barebones as you can get and everything is great. I also use opencode for bigger task, I could probably hook that to a local model but I'm not sure (still new to this). But if you work with sensitive data you need a local setup purely.
I am interested in a local setup but whenever I read about it, it seems the money is better spent on cloud inference since even with thousands of dollars of my own hardware I wouldn’t have frontier like capability? I am bad at choosing models though (I just throw everything at Opus atm).
Well yes you won't have frontier capability but that doesn't mean you'd have nothing, I'm not sure what you have but I know there are tools that let you see what models (and their quant variants) you can safely run, I'd look into that first and see what options you have and what you're willing to delegate to a smaller model vs use opus for. Maybe opus to plan vs smaller models for focused implementation, or what works for you workflow and needs.
I think this would depend upon the level of sensitive data.
For OpenRouter, you can setup an API key and limit it to only models that claim to not train on data, but that is just a claim. You can then use trust to judge which providers will honor that claim.
But if the data is really sensitive, you might want either a local model or a business subscription with some big name in the US that legally promises no data training.
So are we talking some app idea you are playing around with, or files filled with PHI/PII that you have legal mandates to safeguard? If the latter, I would stick to only provider with enterprise agreements to not store/train on the data. Even the ones who promise no training are likely storing the data for monitoring for abuse or such short term.
For anything sensitive I have Claude business (team?) which (claims) no training but I would like to find an alternative for them. I’m just really tired of supporting Anthropic each month (same feelings for all the major providers).
I’m biased (my wife works there) but Fireworks or another US-based inference provider is going to be a better fit than OpenRouter. Fireworks has zero data retention by default[1] and has tools to make it easy to run your existing harness against open models [2]
OpenRouter isn’t a provider, they route to other providers, so you would need to specify which ones you’re comfortable with anyway.
Openrouter lets you pick your provider and see their policy re retention and training. You can pick US providers e.g. digitalocean that don’t retain or train on your data. Openrouter is also now owned by stripe, so I don’t see any reason why using them with a trusted provider is riskier than using the big two (especially in light of some of the whole Navier Stokes thing)
If it’s really sensitive then don’t use a cloud provider.
Depending on what your threat model is, and how much you trust OpenRouter and their upstream providers, they offer zero data retention (googleable: ZDR) APIs which come at a higher cost. You can find other zdr providers as well.
True, but you can link them up over thunderbolt or Ethernet. If your goal is to run local LLMs, not all weights need to live on the same computer. You can segment the workload by layers and pass the activations along the lower bandwidth interconnect with a small perf penalty. Also you get double the CPU/GPU cores allowing for better multi user/agent performance.
I was thinking the same as you as far as price per value, it does make seance to get 2x of these things IMO, but what throughput hit would you see in linking versus one machine? latency does matter, and there must be a trade off no?
I agree in principle that the buck has to stop somewhere, but taking full responsibility for the actions of a fundamentally statistical system that you didn't build and have no interpretability of is uncomfortably risky for most parties, at least for anything above low stakes tasks.
Do you take full credit for the output of the LLM ? If you accept the rewards for what you do with the LLM's output, it's probably ethical that you also take on the risks.
That's only true if the risks are reasonable, understood, and expected. Driving a car is risky, but if someone sells you a car that goes flying into incoming traffic because of a flaw in the design the car company is to blame. "Drivers know cars are dangerous and risky" isn't acceptable.
I find some of the discomfort and awkwardness of discussing these what-ifs is reduced, at least slightly, when we distinguish between:
1. Restorative justice, where one must fix or heal the problem they caused.
2. Punitive Justice, where one is motivated not to act on bad impulses.
Too often we blur the lines between responsibility and blame and culpability. A given set of consequences can be fair/just for one kind while "unfair" for another.
if you are not willing to assume that responsibility, then don't use it.
You buy a tool, and the manufacturer says "this tool will occasionally malfunction, by design". You reply with "ok, I'll keep it in mind. In what ways can it malfunction, so I know what to look for?", "I dunno. It could be anything. A-NY-THING! And of course we are not responsible for any bad stuff that happens (but do share your success stories, as they look good in marketing)"
I agree. That is a great reason why you shouldn't let an LLM do those things. If you must use an LLM, sandbox it. If you can't trust your sandbox, then stop using the LLM until you can trust the sandbox. "But I find it really useful to YOLO without safeguards" is not a valid excuse.
That's only true if you reasonably can be expected to know that it might do whatever it did and you haven't been mislead by the people selling the system. If you use a product as directed, for the thing it was advertised to do, and it causes harm no reasonable person would expect, the company is to blame.
> taking full responsibility ... is uncomfortably risky
I look forward to someone building an agent, getting criminally charged for actions it takes, and then trying to use this argument in court.
"your honor, I'm uncomfortable being held responsible for this"
people are responsible for the actions they take. hiding behind "I just created an agent, then the agent committed the crime" is simply never going to fly.
in the real world, outside the Silicon Valley "agentic everything" filter bubble, this is a laughable question.
try to argue that you shouldn't be charged with attempted murder, because you didn't stab someone directly, instead you built a Rube Goldberg machine and the final step of the machine did the stabbing.
and likewise, try to argue that you didn't commit tax fraud or whatever, because there was actually a Rube Goldberg machine built out of GPUs in between you and the fraudulent documents. both attempts will be equally successful.
That graph has to be made because an Exec didn't like that graph went down to the right instead of up and to the right. How do you make a graph with 0 on the far right and counts up by going left of 0? What number line is that?
The game theory is all wrong, this is a cooperative game, either they both win or both lose. I don't see how the Donkey getting hit by a car should be classified as "Donkey wins"
I'd love a version without the multiplayer and just zen