I think the better analogy is an insane nuclear power plant manager deciding it wants to buy pens to use as neutron-flux regulator rods — because after all, a pen is functionally a pencil and a pencil is made of graphite.
Then the pen manufacturer hears about this and says “Our pens are not made of graphite and are not suitable to be used in nuclear reactors”, to which the reactor owner says “it’s fine, they fit in the graphite rod holes, and we’re just using until the next generation of pens come out which will do an even better job”, and then the pen manufacturer says “I’m not going to sell you any pens until you agree that they will be used only writing.”
The previous power plant manager had, of course, already signed a contract agreeing to those terms; the power plant is subsequently under new management and upset about being bound by this contract, so they designate that none of their suppliers of parts and fuel are allowed to have any commercial dealings with the pen manufacturer or use these pens for any purpose, even writing.
> The previous power plant manager had, of course, already signed a contract agreeing to those terms
It's even more perplexing in this case though, because the original DoD contract was signed in July 2025. It was the same "plant manager" just arbitrarily changing their mind in the middle of the contract they signed.
And why would the manager ban the manufacturers pens even for writing, except out of spite to punish the manufacturer? And coming back to the real case, is the supply chain risk designation supposed to be a punishment or only there to actually protect the supply chain?
It turns out that a gun is usable for both legitimate and illegitimate reasons, and so it’s rational to both be concerned about the presence of a gun and work very hard to make sure only reasonable people are ever in a position to use it, and that’s not a contradictory position with the notion that one might actually need a gun under certain circumstances.
And if the military needs guns to both shoot at humans and animals and a gun factory says “these guns are for shooting small animals only, using them against humans would be inhumane” and refuse to sell them to the Navy Seals, they would be designated a supply chain risk?
Why not just stop buying their guns and let contractors still use those guns if they need guns to shoot animals?
That’s the same way to do it, and in earlier eras that’s what we would have done. Unfortunately, we elected a gangster who hears “I won’t let you do this” as a personal affront. This, again, is why we need to be very, very careful before we create guns and put them in places, because if the only thing we’re relying on is that only good people are going to pick up the tools we create, we’re going to be very, very disappointed.
If you have pens in the building, and staff need a pen, they will use the pen that is available, even if there’s a sign saying “not for purpose x”. If you want to stop that happening, you need to not have limited pens available to your staff at all.
Ah so a defense contractor won’t be allowed to use Claude to create their website because they might later want to do some target selection for military strikes using AI and then they wouldn’t be able to use Claude?
Close enough - if they’re using Claude for anything, the DOD can’t trust that staff didn’t also use it in product development, so they’d have to keep track of everything downstream of that component to ensure it was never used for target selection.
It could also be as simple as "anything tainted by the Claude non-targeting-conditions can't be used in target selection, and we don't want to leak classified information about which components we use in target selection by requiring specific items to be developed without Claude, so we can't allow it anywhere".
That's not accurate. DOD suppliers weren't banned from having "any commercial dealings" with Anthropic. They were banned from using Anthropic for their work with the DOD.
That's true legally, but the government's public statements did initially very explicitly present this as a ban on any dealing.
"Anthropic a Supply-Chain Risk to National Security. Effective immediately, no contractor, supplier, or partner that does business with the United States military may conduct any commercial activity with Anthropic."
Presumably, according to the defenders of the designation in this thread, this was an innocent mistake that doesn't indicate any political intent.
The way things work, the CIOs and CSOs of these companies just implement blanket bans, so it really is effectively a ban for any use. Source: I am at one such company.
That analogy still makes this sound like a textbook designation. In that situation the pen manufacturer is expected to sell pens without a theory of what will happen next. The person who buys a pen might use it to kill someone or commit horrific legal/moral crimes and that has nothing at all to do with the pen manufacturer.
I can see how I might reasonably refuse to deal with that pen manufacturer. The pen manufacturer is unreasonably pushy and they're going to be massive trouble to deal with because they're all up in my business in a paranoid way and disrupting my use of their product. Companies have made my never-again list for far smaller slights.
I think you might have misread the thread so far. Do you think the US military is buying atomic bombs from a supplier who is going to try and stop them from bombing things on demand? Obviously the bomb makers have to make peace with the fact that their customers may, of their independent initiative, one day detonate the bombs.
It's been very commonplace in robotics/AI research for vendors to sell robots to the DoD that are explicitly contractually not to be used for killing people.
It's that guy with the long beard and sunglasses that looks like ZZ Top's Billy Gibbons, walking out the Wal-Mart door with your suitcase nuke ... the clerk up front near the door was so stunned he didn't even ask to see his receipt
This isn't like using pens in a nuclear plant. AI obviously has tremendous military applications. And it's completely unsurprising that the U.S. military--whose entire doctrine for the last century has been about having the most advanced technology--demands to use frontier models without non-military people imposing restrictions.
Have you seen the movie Oppenheimer? The U.S. military did not care about the moral opinions of the nuclear scientists--they were going to get nukes first. This is like Truman telling off Oppenheimer for being upset after the U.S. used the bomb: https://www.biography.com/political-figures/a44361438/why-ha... ("Blood on his hands; damn it, he hasn’t half as much blood on his hands as I have. You just don’t go around bellyaching about it,” Truman said.... He called Oppenheimer a “cry-baby scientist” and said, “I don’t want to see that son of a b–– in this office ever again.”).
Machine learning needs a lot of examples to get good. Wars and wargames (including digital ones with unbounded self-play time) and all the textbooks together will at best, on the basis of what we see in other fields, get the equivalent of a new officer fresh out of training… and then, given current observed agentic AI behaviour, "follow orders" rather than "follow lawful orders".
> Yeah, but also we can be reasonably confident they're not good enough for many of them yet
What does “not good enough mean?” Who decides what is “good enough” and what tradeoffs are acceptable? Obviously it’s the military’s job to decide that.
In the context of my comment, "not good enough" includes "committing war crimes" and "would have started a war with China if there hadn't been a human in the loop".
We can also infer from other domains, vendbench or attempts to have an LLM play Star Craft, that these models are currently extremely mediocre at management roles.
> Who decides what is “good enough” and what tradeoffs are acceptable? Obviously it’s the military’s job to decide that.
In practice, in the case of the USA, Trump and Hegseth.
The US military, so far as I can tell, are rather less thrilled with these adventures.
Right—and deciding what constitutes “war crimes” isn’t within the expertise of AI developers. They can explain the abilities and limitations of their models to the military leadership. But the actual decision about how to use AI is up to the military through its chain of command. That starts with Trump as the duly elected president and Hegseth as the duly appointed civilian head of the military. They are the democratically accountable actors. And all this is being done in the open, so if the people have a problem with it they can hold it against Trump and his party.
That decision assuredly does not belong with people like Amodei, who are not democratically accountable.
> That decision assuredly does not belong with people like Amodei, who are not democratically accountable.
Has Amodei been conscripted?
If not, he's free to say "no", just as I'm free to refuse to work for e.g. a defence contractor if I don't like what's going on or if I don't feel competent for the role.
He, like me, is allowed to not want deaths on his conscience if he feels his own business' best product isn't good enough.
Oppenheimer was the guy who was pushing that opinion just as much as the military folks. In the case of AI, my understanding is that Amodei is not trying to beat the enemy to it.
What I don't understand is why the U.S. military is turning to a for-profit corporation when they could fund and pursue the development themselves. Why are they trying to "outsource" it to industry?
JAVA TECHNOLOGY IS NOT FAULT TOLERANT AND IS NOT DESIGNED, MANUFACTURED, OR INTENDED FOR USE OR RESALE AS ONLINE CONTROL EQUIPMENT IN HAZARDOUS ENVIRONMENTS REQUIRING FAIL-SAFE PERFORMANCE, SUCH AS IN THE OPERATION OF NUCLEAR FACILITIES, AIRCRAFT NAVIGATION OR COMMUNICATION SYSTEMS, AIR TRAFFIC CONTROL, DIRECT LIFE SUPPORT MACHINES, OR WEAPONS SYSTEMS, IN WHICH THE FAILURE OF JAVA TECHNOLOGY COULD LEAD DIRECTLY TO DEATH, PERSONAL INJURY, OR SEVERE PHYSICAL OR ENVIRONMENTAL DAMAGE.
OP makes a category error already by ignoring the part where the US military/government wants to commit crimes (against humanity and whatnot) and pretends, questioning that would be a crime.
You here make another one by pretending, AI was just some random tool, like a screwdriver or a programming language.
AI can make decisions autonomously that would otherwise require a human. Like selecting people to be murdered by drone strike based on arbitrary criteria. A programming language by itself, JAVA, cannot do that.
An if/then statement or random number generator in Java can make decisions autonomously. Heck, the US military has deployed fully autonomous weapons since at least 1979 (long before LLMs existed).
There is nothing stopping any plain old algorithm being used for the same. The difference is that people decide, for whatever reason, to use the tool or to not use it.
AI can make decisions all it likes. I if you don't integrate it with the drone strike MCP server it's as impotent as a JVM without any code to execute.
That is an analogy between one thing and another thing, but the topic of this thread is the designation of Anthropic as a supply chain risk, which is not involved in your analogy. It is traditional that comments refer to the topic of discussion in some way.
For example, one might extend your analogy to "The nuclear power plant then declares that it will not buy the pens after all, and the pen manufacturer (who had banned them as a customer) sues them to force them to buy the pens after all."
This is the most accurate parallel as far as I can tell. I've not seen any evidence that Anthropic deliberately built automated systems or escape hatches that literally prevent the US Government from using Claude to do these things. Other than the basic AI Safety mechanisms in place for everyone/general use. It's almost the reverse, where they explicitly don't want to strip away guardrails that say things like "don't kill people".
The intent and framing matters a lot here. Refusal to remove safety features is a LOT different than deliberately building mechanisms to sabotage would-be operations.
It doesn't seem like you understand what a supply chain risk designation is for.
It's to ensure that companies that the DoW relies on don't build on top of a product that isn't fit for DoW work.
You seem to be agreeing with this designation in a way. The DoW essentially said "Pen company won't let us use their pens as neutron-flux regulator rods, so anyone building power plants for us isn't allowed to use these pens as neutron-flux regulator rods".
They've essentially said "pen company won't let us use their pens as neutron flux regulator rods, so anyone building power plants for us isn't allowed to use these pens as pens"
(and yeah, there might be a completely legitimate reason to stop your subcontractors from using HuaweiSpyPens, but determining that ACME Pens is just as dangerous because they refused your neutron flux regulator order looks like a punitive measure, not practical risk reduction)
Congress can do that, can't it? Why can't it? It could declare all competing religions as terrorist organisations.
Obviously it'd be immoral, unorthadox and impractical but I expect Congress has a long history of promoting and stamping out ideologies and they can do it. What little I know about the civil rights era springs to mind.
Legally, Congress cannot do that, as long as it is "respecting" some religion. It's true that the US Constitution is more used as an excuse than upheld, but at least the language is pretty plain.
In theory. The issue is we've seen exactly what happens in a Congress v. constitution showdown once the Congresspersons put their collective feet down. Congress eventually redefines the law in a way that favours them. Plain language won't stop them if they get serious; they aren't supposed to be regulating the US economy the way they do either and a lot of stuff the military-industrial complex does looks like it should be unconstitutional (like the spying and the secret courts).
On the one hand there is a clear and comforting consensus in the US that they aren't going to force religious mores on anyone. So I don't see how the Congress could or would try. But if we ever end up in a situation where the US Congress decides it wants to go straight Catholic I really don't see how it would fail. They've got far too much power and they'll eventually rotate in a compliant judiciary and then redefine what words mean so they can do it.
It is a bit difficult when dealing with such an out-there scenario, but if Congress decides the US military should do something wild then eventually the military is going to do it. The realistic protection is that by virtue of good system design there isn't a plausible scenario where Congress would want to impose Catholicism on the US population by force.
The fact that Congress is currently abdicating its responsibility does not mean that it is actually powerless. Technically, Congress has tremendous powers if it chooses to wield it. SCOTUS can try to rule a law is unconstitutional but Congress can impeach and remove justices at any time. Of course, we hope Congress members will pay the price when it comes time for reelection but at the time of this decision, as long as they can somehow remain united, Congress reigns supreme.
> Congress can impeach and remove justices at any time.
The impeachment process requires a majority of the House and two thirds of the Senate, and then any replacement would have to be nominated by the President rather than Congress. Meanwhile the existing decision stands, and even the dissenting Supreme Court Justices (if there were any) should follow it as a result of stare decisis, and might even be inclined to because removing their colleagues under those circumstances could make them uneasy.
> Do you think in the military you can just be like “Yeah, I don’t feel it today, I’m going to take s sick day”?
What do you think happens if they refuse to do it?
Either nothing, or if they get sued or prosecuted for not doing it then they get brought in front of a court, and a judge who is doing their job would say that they were right not to do it.
> What do you think happens if they refuse to do it?
In war time they can be shot on the spot for disobeying orders.
> Either nothing, or if they get sued or prosecuted for not doing it then they get brought in front of a court, and a judge who is doing their job would say that they were right not to do it.
I think you confuse the military with corporate environments. Worst thing that can happen in the private sector is that you get fired.
In the military you can spend the rest of your life in prison if “you did not feel like obeying orders”. The military is built ob discipline and followimg orders, not Gen Z mood swings.
It actually is, and this is why there are separate judicial, executive and legislative branches to put those checks and balances on each other instead.
Your argument is that if Congress passes a law that says the military can summarily execute non-Catholics, the military should have the ability to do that, i.e. the courts or the public shouldn't stop them?
> Congress shall make no law respecting an establishment of religion, or prohibiting the free exercise thereof; or abridging the freedom of speech, or of the press; or the right of the people peaceably to assemble, and to petition the Government for a redress of grievances.
> No person shall be held to answer for a capital, or otherwise infamous crime, unless on a presentment or indictment of a Grand Jury, except in cases arising in the land or naval forces, or in the militia, when in actual service in time of war or public danger; nor shall any person be subject for the same offense to be twice put in jeopardy of life or limb; nor shall be compelled in any criminal case to be a witness against himself, nor be deprived of life, liberty, or property, without due process of law; nor shall private property be taken for public use without just compensation.
Example chosen specifically for its unambiguous unconstitutionality.
No, not at all. How did you get "the courts shouldn't stop them" from "that's why there's a separate judicial branch to impose the checks and balances"??? That's literally the opposite of what I said.
> The DOW should (and does) have ability to do anything and everything that Congress has said they can do.
> Me: That's not how it works.
> You: It actually is
Moreover, regardless of what the courts do, the military still shouldn't execute a law like that, because they too have the ability to refuse to violate the constitution even if Congress tells them to.
I chose to read the comment charitably. The natural understanding of "everything X says they can do" is limited by external restrictions on what X is supposed to be able to authorize.
That reading would make it a meaningless tautology. They're allowed to do anything they're allowed to do.
Which seems especially unfavorable in context, since "the ability to perform mass surveillance on American citizens" is naturally the sort of thing they ought not be allowed to do.
Then the pen manufacturer hears about this and says “Our pens are not made of graphite and are not suitable to be used in nuclear reactors”, to which the reactor owner says “it’s fine, they fit in the graphite rod holes, and we’re just using until the next generation of pens come out which will do an even better job”, and then the pen manufacturer says “I’m not going to sell you any pens until you agree that they will be used only writing.”