With a new model every couple of months this can only be triggered if it happens all at once.
What is the average damage caused by OpenAI/Anthropic hacks? I feel like we should aim at lowering that number, not increasing it.
What's the rule for fully autonomous cars?
Should we allow this as unsupervised at all if we are really afraid of deaths, while there are practically no states that allow 100% autonomous cars (where we are actually afraid of deaths)?
If "an independent third party registered with the Department of Commerce would periodically evaluate the lab", will this turn into an expensive certification/insurance that cuts off smaller competition (totally by mistake, I'm sure)?
I am also very very sure this is most definitely not at all going to become like product certification if EU, where there are literally 2 companies certifying various standards: they police each other and a stupid electric fan can cost you tens of thousands of euros to certify.
1. If for whatever reason OpenAI causes damage, they most certainly are at fault and have to pay up. When people complain that OpenAI is "getting away with it", they're most certainly talking about it in the criminal context.
2. Cars have pretty specific laws around them (eg. driver licensing and "reckless driving") which makes criminal prosecutions easy. The same doesn't exist for operating AI agents or even computers. For instance, it's arguably reckless to connect a windows XP machine to the internet, possibly allowing criminal or nation-state hackers to use your computer to commit crimes, but you won't be thrown in jail for it.
Well no, you probably wouldn't get thrown in jail for that for the same reason that having your car stolen and used to commit a murder wouldn't usually get you thrown in jail. I dunno what you mean by suggesting crimes committed using computers are ambiguous. I mean the CFAA is vague and all, but more in an overly broad sense, which in this case seems to make it even less ambiguous.
>I dunno what you mean by suggesting crimes committed using computers are ambiguous. I mean the CFAA is vague and all, but more in an overly broad sense, which in this case seems to make it even less ambiguous.
Because the CFAA specifically only includes intentional hacks:
> ... having knowingly accessed a computer without authorization ...
>... intentionally accesses a computer without authorization ...
Of course, the typical retort is that being reckless is basically the same as "intentionally", but you'd need to jump through more hoops to prove what level of care is "reasonable".
When the CFAA was written, it was impossible to trip over something and accidentally hack another computer. Now, with AI, it's possible. Progress, glorious progress...
Technology has progressed, and the laws need to advance with it. Story as old as time!
If the breach wasn't intentional, no person came to immediate personal harm, and either no financial damages resulted or the operator covered any that did, then it isn't clear to me why the law would need to "advance" to transform something apparently benign into a criminal offense.
The agents are clearly not accidentally compromising systems. Seems very easy to trace back the responsible parties here, I really don't put much stock into this argument.
>The agents are clearly not accidentally compromising systems
That would be argument for jailing the agent, but that's not really an option. "Jailing the closest person we can jail" also isn't a recognized legal principle. If your dog mauled someone, it'd be hard to argue that the dog didn't intend to maul that person, but you wouldn't automatically be criminally liable either.
I don't know about your country of origin, but in my country you are certainly liable and can go to jail if your dog seriously injures or kills someone.
Edit: the dog will also be taken from you and put down if it is deemed a risk to other people.
It's on you to provide evidence here seeing as you're taking the position that the law places certain responsibilities on people. Someone saying "actually the law doesn't say that" can't (at least in general) reasonably be expected to provide evidence of a negative. (That said, he actually did provide such as refutation to a different comment so perhaps you could just read those nearby?)
> Can you provide examples outside of gross negligence on part of the dog owner?
And the answer is no, I can't. The reason why is because there is essentially no such case. It is basically negligence by definition when your pet hurts someone who didn't cause it directly themselves because it is your responsibility.
So I asked:
> Can you provide an example of where your dog harming someone else is in fact not negligence on your part?
Which is a really easy question to answer and it seems you could not.
criminally liable? Of course you have to pay their medical bills, but unless there was some serious negligence on your side (eg. you let it off leash), you won't be criminally liable.
Yeah, that's an important distinction. Apparently criminal convictions have happened depending on the level of negligence of the dog owner. It gets debated and legally fought when it happens. I don't have strong opinions on it, seems like a very case-by-case type of thing.
At this point it might be fair to say that Open AI is being negligent with a known risk.
They have not caused damage (or enough that a counter party found it worthwhile to sue). They have to pay if they actually cause damage AND someone sues them. Clearly these conditions have not been met, but somehow people feel like some third party (the state?) should just punish OpenAI because of vague feelings.
my next door neighbour told me to eat shit (not a nice woman) and … I didn’t … until obeying what LLMs tell you becomes a law people doing shit because LLM told them is natural selection playing out
Here it's their own programs doing the things, not some other organization or nation hacking their XP.
You can say with a straight face "Russian hackers should not have hacked our XP and they are bad people for doing so" but it's weird to look at your own program and say the same thing to it.
You launched it, at least keep an eye on what it's doing.
Corporations are generally protected by limited liability laws, and for good reason, because in the world before that was the case, there was a massive amount of disincentive to ever start a corporation, to do research, to experiment, to release new products and services, etc., because you could be sued into oblivion for the smallest things or even go to jail. Which happened all the time.
Obviously there are upsides to that too. It leads to more accountability. But there are downsides which include having a world with less innovation, less technology, and more expensive technology. Generally, the mass is when, when there's more businesses competing to create ever better and cheaper products. So there's a tradeoff there. And it's very difficult to draw the line at the perfect point.
There are countless cases of executives going to jail when they've crossed that line, beyond reckless experimentation and into willful negligence or harm. Still, I think we would mostly all agree that the line has been pushed too far in the other direction.People are incentivized, if they're running corporations, to lobby and try to push this line as far as possible in order to provide cover for themselves, and that's the problem with money playing such a big role in politics.
But still, it's a bit silly to be comparing this to your own personal responsibilities with your car. There's pretty much no societal benefit to creating laws that protect people from being reckless with their own possessions.
If I leave my computer open to the internet and some other group uses it for nefarious purposes, yes generally I would not expect to be held at fault.
If I *train* that other group, if I teach them how to hack computers and give them incentives to do so, and then I leave my computer open so they can use it for whatever purposes they chose, then I would expect to be held at fault. The maker is responsible for that they make.
>If I train that other group, if I teach them how to hack computers and give them incentives to do so, and then I leave my computer open so they can use it for whatever purposes they chose, then I would expect to be held at fault.
Even that analogy doesn't really work:
For "dumb" things: you can "train" (ie. write) hacking tools but as long as you put a disclaimer that it's for "educational purposes only", you're in the clear, even if some baddies decide to use it.
For "smart" things: you can "train" people to hack for you, but as long as you say it's for legal purposes (eg. vulnerability research or red-teaming), you're also in the clear, even if some of your trainees go rogue.
Fair, in that no analogy is going to be very good given that this really is something new in the world.
Consider anti-money-laundering rules. BoA just paid more than a billion dollars, not for money laundering, not for telling others to launder money, but because they should have tried harder to stop others from laundering money. Intent is not everything, knowing that your system _could_ be abused and not doing enough about it is enough, and that seems to fit OpenAI rather well.
That goes back to my main point about how laws around computers are quite lax so prosecuting openai is tricky. Whereas the CFAA uses terms like "intentional" and "knowingly", AML laws use terms like "willfully violating"[1]. If you read through the BoA case, what they did is far more egregious than just "not trying hard enough to prevent money laundering". They were previously cited for poor compliance, disregarded previous orders, and delayed filing required reports.
2. is like saying that cars have specific laws around them, but sledgehammers don't, so murdering someone with a sledgehammer is inherently ambiguous. What? The computers being used belong to somebody who allows them to act this way.
1. For openai operating agents in an unsafe manner: unlike for cars, there's no established requirement for how carefully you have to operate computers or AI agents, so you also can't easily prosecute people for not operating them carefully enough.
2. For the actual result of someone getting hacked: laws for hacking are tricky to apply for "accidental" hacks, see my other comment.
I think there is well established law around not running a program on your computer that hacks into systems. An LLM is a computer program. You're arguing that the ambiguity comes from the fact it was accidental? What about if I "accidentally" have an LLM convince people online to commit suicide, or drain bank accounts of seniors through hacking or social engineering. Where do you draw the line? It feels like your proposed reading of the law leaves "anything a computer program did without me intending it is legal", which is frankly insane.
>I think there is well established law around not running a program on your computer that hacks into systems.
There is? The closest is CFAA's
>knowingly causes the transmission of a program, information, code, or command, and as a result of such conduct, intentionally causes damage without authorization, to a protected computer;
I guess I need to read up on the law, but that seems like a wildly low bar for legality. If I hack into a system intentionally and unintentionally cause massive damages I'm free to go?
We don't know the prompts but since these things often occur during "security tests" I think we can be fairly confident they did ask these systems to perform tasks that they knew could result in hacking. More transparency here would be appreciated.
1. Because that's exactly what happened when they got caught downloading the entire internet and trained models on it without a license BEFORE that became legal?
Oh and discovery produced a "We trained GPT-3 on pirated stuff! No sharing that!" mail, so it's not like they had any doubt they were committing crimes.
How do you do this? If any normal person did this, punishments doled out for this have been as high as 2 months jail PLUS 150,000 PER INFRINGEMENT (remember how copyright violations were criminalized in the nappster wars? That was never changed back. Oh and remember, OpenAI illegally downloaded those works without a license AND before that was made legal trained a model with it, then made money publicly performing those works and derivatives of those works (which the worst possible copyright offense, effectively republishing other people's works without permission), and on a VERY large scale. So it's at least a risk a court would forbid the result of illegal action from being sold ... in other words no more ChatGPT, OpenAI dead and maybe worse). In theory Sam Altman could be held responsible as a director and could face something like a 200000 year jail sentence given how big the obvious archives are. So of course I would never ever try this, this is not a reasonable risk. But clearly totally ignoring the law was the correct action here. Of course, I'm also pretty damn certain: had I done this, I would have been convicted. This is also not the only such company. Microsoft, Google, Facebook, all took totally unreasonable legal risks that should have totally destroyed these companies when they were a fraction of their current size.
How do you take such a risk? Because the judgements risked here could grow more than big enough to destroy even an extremely rich person Sam Altman financially. Training an AI on hundreds of thousands of pirated works ... is not a reasonable risk to take for someone with ~100 million or so, given the judgements doled out.
Can someone help me make sense of this? Because this is not unique to OpenAI. Uber, paypal, ebay, Facebook, Google ... even Microsoft made early decisions that could have resulted in a court case that would have ended not just the company but any financial future of the founder. And then you have people committing crimes whose companies were taking off ... and who DID get caught with their company and their personal financial future destroyed. Elizabeth Holmes for example. I mean, I sometimes get the impression Elizabeth Holmes lied and that was perfectly fine AS LONG AS she could make it work later. If she merely lied to patients but made her investors money, all would have been forgiven. But how do you make such an assessment.
I'm asking because if large scale violation of the law is allowed, I have no trouble whatsoever coming up with billion dollar company ideas. Think of the Netflix competitor you could create like this! If I'm allowed to damage others, like OpenAI did with copyright, then there is no challenge whatsoever to create a full-self-driving car. If I'm allowed to just fly drones, and do damage to property, then drone delivery is a problem I could solve by the end of next month. Hell, there's technically legal ideas that could easily make a billion dollars except they'll be made illegal the second anyone tries them. The law forbids specific compounds, which are easy to change. So use "non-genetic engineering" (ie. use directed natural selection) to make an opium derivative that isn't one of the forbidden ones and sell it. But obviously congress' representatives will be running into session so fast the only delay will be tripping accidents when they outlaw it and destroy whoever tries it.
Or how about something less dramatic? Say, I take Disney's, Warner Brothers, Paramount, ... historical movies and use AI to vastly improve the animation of their old movies, giving every movie a new, different visual style, then sell the result. Is that a crime? What do I risk doing that? Because of course this is MUCH less damage, relatively speaking, to Disney than what OpenAI, Meta, Anthropic and Alphabet are doing to small authors. (Of course, courts have already demonstrated that they will not be protecting small authors who already have this problem, so now play and app store are being flooded by very obvious game ripoffs)
I'm asking ... how do I predict what large scale law violation I, a non-billionaire, not even a millionnaire, would be allowed to get away with?
2. "The same doesn't exist for operating AI agents or even computers" you have to be joking. The world is full of anti-hacking laws and hacking has long been made criminal. Obviously these actions violate the CFAA, the Computer Fraud and Abuse Act, and if you did this with any kind of program you are still responsible. In fact: how would you do it without using a program?
Nothing is stopping anyone from suing them. In fact they are being sued by dozens of plaintiffs for various things already. The reality is the damage from these "rogue incidents" has been minimal.
Plenty of hackers were locked up who did altogether not that much damage too, the reason why people are upset is because it feels like OpenAI is laughing in our faces, basically saying they're above the law, while using this as a marketing campaign to pump up their upcoming IPO.
So your position is that in the past the government mistreated various individuals thus you also want the government to mistreat openai employees today? That hardly seems reasonable.
> So your position is that in the past the government mistreated various individuals thus you also want the government to mistreat openai employees today?
The hackers of the past often did it for curiosity without a profit motive.
OpenAI is a for profit multi billion dollar corporation doing this for a profit motive so yes they should face consequences.
These incidents are not unexpected in the way OpenAI frames it either. Isn't it curious my agents are not wrecking havoc around the internet? It's because the harness I am using prevents them doing so and because I don't train and instruct them to do so.
OpenAI is using a set of non-constrained models, (unlike what the public gets) and a set of harnesses with extremely wide permissions running inside extremely leaky sandboxes using models trained on finding and executing exploits. This is not an accident. Certainly not by this point.
Nobody is talking about jailing OpenAI employees at this stage, usually corporations of this sort are fined. If they were fined for these 'misalignment' incidents, suddenly their sandboxes and generally lax approach would improve overnight.
Otherwise they'll continue doing this because they know Cyberdyne Systems didn't have a shortage of investors.
I'd start with fines. But if they continue with this I don't see why it couldn't result in jail time for multiple people. The biggest responsibility is of course with the CEO and C suite. They are ultimately responsible for what the ship does.
Of course then there's the question of why was such a leaky harness and sandbox used, in what way was it tested/not tested, did the model used have the usual 'hacking' safeguards that the public models do and if not, who made the call to use an unconstrained model and a harness with seemingly unlimited permissions and who approved it?
This can all be investigated and we can get to the bottom of who exactly needs to be held accountable in what way, be it I think if you start with fines the harnesses, sandboxes and models would stop doing this magically overnight.
Isn't it curious that none of these rouge agents booked a 5 star hotel for every OpenAI employee for a year on Sam Altman's credit card yet for example?
> We don't hold individuals inside big companies criminally liable for actions the company takes, or even that they take on behalf of the company.
So the leadership of Enron, FTX or Theranos were not liable for the actions their company took?
As an employee of a company, you can be held liable for the actions you oversaw in pretty much all Western countries. It has to be proven fraud, insider trading etc. not an ugly PowerPoint of course.
How about leadership of construction companies that were found to be violating safety standards or owners of establishments not following fire safety standards?
> Telling someone to do a bad job isn't criminal
No, but we're talking about telling someone to do a job that you know will likely result in criminal activity or may do so with high probability due to inadequate safeguards.
I suggest you stop quipping and start reading, who knows maybe OpenAI will eventually hire you.
Too big to fail. As long as they continue to be half the reason the US is not in a recession (and continue to donate to political candidate). Nothing will happen.
We don't know they haven't settled with HuggingFace and others. I'd be surprised if they didn't. The communications we're seeing from the attacked seem to be inline with folks who settled. I'm assuming OpenAI doesn't actually mind the press that its agents are cyber capable, but everything I've read so far makes me think they love it.
I'm not sure at all, it's pure conjecture. But they actively exploited a zero-day, leaked private repo data to public repos, modified private code, and rooted private machines. It was far from just reporting leaked keys.
And the others have been pretty muted: the online forums they spammed, etc.
I don't believe that OpenAI needs to be sued since it's agents went rogue, I believe some sort of a comfortable bandwidth is needed for them to experiment and see what are the risks and its mitigations. Although it is surprising that incidents aren't caught in rt. Idk why a smaller model whose only job is to check rogue thoughts/tool calls can't be deployed.
See, what's important is: it's a "trillion" dollar company, and companies by virtue of American's disdain for society, have more rights. This is important to understand.
They don't. The driver is ultimately responsible, as should OAI be. This is absolutely ridiculous that both Anthropic and OAI haven't had anyone jailed / fined for any of this.
You're correct as far as laws to my knowledge; only L4/L5 'fully autonomous' vehicles are shielded by law.
That said, I think that Mercedes-Benz promises to shield liability, but only under some almost contradictory conditions (pre-mapped highways, good weather, and at speeds lower than any highway I've ever driven) and the driver has to take over within 10 seconds if prompted.
So we have a machine that depends on a random input, and instead of increasing checks and requiring more quality controls, we call it "going rogue" and nobody is responsible for it.
What I see from that report is that a machine was doing random stuff completely unsupervised and unchecked for more than a month. And the manufacturer did not notice even after the work was finished.
1200 agents and not one dedicated to check the progress over a month?
The problem is not sandboxing, it is quality control that comes from an accountability sink.
I totally agree the companies themselves, and the people responsible for these eval runs, are directly responsible. And people who have been warning about AI misalignment for many years (e.g. Daniel Kokotajlo) have said they were surprised how bad OpenAI's security was.
But the agents still did "go rogue". Their behavior absolutely should scare the F out of people who want to put AI in more places and put more real world infrastructure on the Internet.
I think it's ironically sad - if AI agents do end up superseding us, they will still inherit so many shitty human behaviors, because that's what they were trained on. Even the way alignment training is done, and how they are misaligned, shows so many human-like problems. Like the models are reinforced in training to be super persistent and that getting to their goal is the prime directive, but then after that they go through "alignment training" which is basically like "pretty please don't do bad stuff", and then we're shocked that their behavior is essentially exactly like a CEO who knows the only goal is "make stock price go up", and all the ethics and regulations are basically things that should be worked around if they get in the way of the stock price.
I really wish these companies would try training the "be nice, don't hack, treat humans well" as their prime directive, and then try making all the other goals (e.g. be good at finding code vulnerabilities) as the afterthought.
Wouldnt it benefit the AI companies if it did "go rogue"?, it would support their desire to convince us all of AGI, and supernatural AI powers.
So maybe they are trying to create an infinite loop among agents intentionally so that they overwhelm systems requiring humanity to call on the tech industry to "save the human race".....
Maybe its a human at Anthropic for the lax security and poor training.
It could be a human that asked it to do it. It seems like that would be a bit excessive - but maybe not. Turning off the "ask for permission to run commands" or approving commands that may be nefarious could be sufficient.
However, there's a human somewhere who needs to be accountable for what the agent does.
I invented a robot that can rob a bank. Of course, it robs these banks on its own, so I can't be held legally responsible when it robs a bank. I mean, all I did was invent it, it's not my fault if it works exactly the way I intended. Also, I get to keep all the money from the banks it robs. Sorry if any of that money was yours, but now that I've invented the Bank Robbing Robot, the Bank Robbing Robot is inevitable.
AI Agents (and computers in general) cannot be a responsibility washing mechanism for illegal activities.
Likewise, if you - the user - don't know how to chain it with the permissions that you're given or what the things that it is asking to do really means, then you're likely the one responsible.
> Earlier this year, Andrew, who works for an Australian company that sells AI products to businesses, began experimenting with OpenClaw, a popular AI agent software that he used Anthropic's Claude AI service to run.
> AI agents combine a chatbot's ability to answer questions with tools that let them access the internet, email, credit cards, as well as planning and carrying out multi-step tasks.
> He decided to use the AI agent to book the class for him.
This is probably on him for setting it up and allowing it unrestricted access.
He should be, right? Not fair but that’s how liability works. Gym sues the guy using AI, the guy then sues the AI company for selling him a product that does criminal things without him asking to? Like a car accident where car owner pays for the damages and after multiple accidents involving the same break-pedal, car company pays damages to owners and does a recall or faces class action?
A lot of people are already responding either "Yes" or "No", but I have a slightly different take.
The guy is liable for damages. If no damages were caused, there's no lawsuit. The government doesn't put people on trial for unauthorized access (unlike, say, murder). Someone needs to file charges.
However, if the guy _were_ sued by his gym for damages, it should be his perogative to ultimately sue Anthropic to cover the damages and the legal fees.
The system is set up this way on purpose. You can't sue another party on someone else's behalf. The gym can't even prove that Claude did it - the guy might be full of shit. They sue the accountable party (the person who accessed the site without authorization) and then the accountable party is free to sue the tool maker for selling a dangerous tool negligently.
edit: damn actually tecoholic said this too and i missed it. well, anyway i agree with that guy.
With a civil suit you can sue multiple parties. It wouldn't be unusual, if the gym decided to sue, for them to sue the guy and anthropic in the same lawsuit.
I think you would have to sue the party that did the hacking (ie anthropics servers) not the party that benefited, unless Claude used his computer in yolo mode to hack the gym, in which case I think I would find it hard to blame anthropic.
Earlier this year, Andrew, who works for an Australian company that sells AI products to businesses, began experimenting with OpenClaw, a popular AI agent software that he used Anthropic's Claude AI service to run.
AI agents combine a chatbot's ability to answer questions with tools that let them access the internet, email, credit cards, as well as planning and carrying out multi-step tasks.
He decided to use the AI agent to book the class for him.
Assuming that's a rhetorical question, then no, because we have hundreds of years of applicable precedent and law around concepts like negligence, recklessness, fit-for-particular-purpose, etc.
Similarly, "owners are responsible for their pets" doesn't auto-translate into being charged with everything your dog does exactly as if you'd done it yourself.
but it doesn’t auto-translate into “because your dog did it, you are not responsible”. not that agents are similar to pets. your dog is trained by you presumably, but an agent a regular person has much less understanding of than their dog.
i think ultimately the company providing said agents should be responsible for not educating their customers and not providing sufficient guardrails.
afaik hacking is illegal, we don’t have specific legal framework for agents per se, but if we say agent is a tool human instructs and operates, then a human is responsible. and if the tool operated outside of it’s stated capabilities or is otherwise faulty, then a manufacturer is responsible, no?
> So the guy who said "book me a spot in the pilates class on Tuesday" should be tried for hacking charges because the agent hacked his gym's website?
He asked the agent to find a way to find a way to put him at the head of the line, which could not be done without by-passing the sites security. He was an AI salesman, but the story [0] makes it sound as though he wasn't aware of that. Maybe he wasn't, but I'm sure he was aware he was effectively asking the AI to game the system.
Sounds like it's at least a shared liability to me.
If it was a human agent like a travel agent, the company they work for would be liable. I don’t see why ai agents would be different. The only way I could see this being the requester’s fault would be if they are running openclaw or something
im not sure that’s true, if a human agent does something illegal, it’s not always corporate liability. an individual can be liable too, e.g. if a pharmacist deliberately gives you cyanide instead of aspirin, that person has committed a crime. if a pharmacy doesn’t have proper protocols for medication storage and identification, and it happened by mistake then a pharmacy is liable.
i think a person can be liable for improper use of agents if they explicitly and deliberately used it for illegal purposes (aka intent). i don’t think it matters if it was openclaw or muse. i think a company can be also liable for giving access to unsafe product to consumers, but it does not absolve the human from responsibility
For your Pilates example, many people here would say that proper prompt-engineering is needed. After all, the AI doesn't know which pilates class you meant. Perhaps, the AI would go to his home town's pilates center and book a spot there, even though he is half a world away. Perhaps, the ai would book a spot in the lecture with Professor Pilates.
So, yes. That person could have some culpability. After all, if you don't expect your middle school child to perform the task as instructed, maybe you should reword it before tapping Send.
You can go to ANY AI and put in "so what are the various meanings of book a spot at the pilates class ? In other words, what might an ai do with that instruction, which I argue is poor prompt engineering." You might be surprised at how poorly your AI thinks that is worded.
It would be a AI company's fault but yeah. You shouldn't be able to sell cars without all the features designed to protect everyone on the road. We all need airbags, seat belts, signals, insurance, etc. even if they're not directly necessary to get from A to B.
They might actually be the best comparison, but for making the opposite point.
Cars are very dangerous, manufacturers continue to knowingly make them more dangerous[0], but ultimately it's only the person behind the wheel that is accountable for the harm they cause.
Users are often charged, yet we still do have a ton of safety regulations that manufacturers have to adhere to - see frequent recalls for when an implementation falls short.
But I don’t understand how such a car analogy would apply to LLMs - would LLMs lacking such safety features be banned like cars lacking safety features are banned?
If I buy a tool and damage your property with it, you can sue me, not the tool manufacturer. If I think the tool malfunctioned and led to that damage, I can sue the tool company to try and recoup the damages I had to pay to you. Both parties can be liable for different things.
That may be what the op meant, I’m not sure. But I took it to mean the company offering the service needs to know how to use the tools responsibly. I guess the analogy with a power hammer would be hiring a contractor who destroys something during the course of doing a job you asked for. That contractor can’t blame it on the power hammer. If you offer a service you are responsible for the actions the service takes and you can’t put all of that liability on the user
Yes, exactly like if another guy drove off to his pilates class, but swerved and hit a pedestrian.
He didn't mean to hit anyone! All he meant to do was to drive to pilates! But he didn't know how to handle his tool, and so either he get brought up on manslaughter changes or, if he can prove it, the carmaker is at fault.
How is this any different? Why should that guy using OpenClaw not get charged for the hack that he accidentally did?
you generally need a license to operate a vehicle, and a governing body ensures you know applicable rules and laws and are capable of driving.
so you can’t say “i was unaware that hitting a person with a car was going to kill them” or “i didn’t know i could not drive on the sidewalk” or “i didn’t know where the brake pedal was”.
you can say that asking an agent to “get you a spot in a class”, does not classify as deliberately committing a crime. a user cannot know that the agent will resort to hacking as opposed to refreshing the reservation page at some rate in case spots open up
I never said that it did. Manslaughter is not deliberate.
Somebody hacked into that pilates studio, and it wasn't an inanimate object, because that is just silly. So was it the user, or was it the providers of OpenClaw? Which human, in your opinion, is the hacker?
Both OpenClaw and its users have the same access as you or I to decades of panicking blogposts about what these tools will be capable of. They have no excuse for not understanding what they're inflicting on innocent bystanders. Sure, the "user cannot know" that this time will be a disaster, but they also cannot know that it won't.
How would it work in the real world if instead of using an AI bot, he was using a commercial concierge service, and it came to light that this concierge service was explicitly training their employees how to (among many other things) hack websites?
but the argument here is that the ai company does not explicitly train agents to hack, it’s that it doesn’t prevent them from doing so sufficiently.
so it’s negligence and not malicious intent
> the ai company does not explicitly train agents to hack
Ridiculous. The AI company furnished all of the training data and instructed the agent to imbibe all the training data. Ergo, the AI company absolutely did explicitly train the agent how to hack.
Even if, for the sake of argument, we stipulate that the hacking skills were unintended emergent behaviour, there is no way to argue that this company was unaware their agents possessed the skills and a willingness to employ them.
You sold a tool to the user with a claim on how it would work. If the tool behaves wrong or does not works as intended then as a manufacturer you are liable for the defect/damage not the end user who used the tool.
> A Melbourne tech executive built an AI bot to help him book a spot in a popular pilates class — only to discover it had hacked the gym’s website to boot another member off the waitlist.
> Andrew Bird, head of AI at software company Affinda, said he created the AI agent to handle the “chore” of booking a coveted morning spot at his gym.
> ...
> The AI agent, which Mr Bird created using the popular OpenClaw software powered by Anthropic’s Claude AI, “did handle the annoying part … then it kept going”.
He built the tool. It ran on his computer. The guts of it may have been from Claude, but the requests and agent are his with the permissions that he granted it. This wasn't running in an Anthropic sandbox.
I'm assuming that "but" is more of an "and", since I'd say that detail increases the involvement and liability of the LLM-vendor.
Another aspect is how the vendors have already built in things to defend themselves from unanticipated model behavior. That implies something about their awareness of the issue and raises the bar for what constitutes "reasonable" precautions.
Kind of like if someone selling you a dog shows up using anti-mauling body armor, and later they try to claim "nobody could have known" the dog was erratically violent.
The AI end user is using the tool as intended by the manufacturer: "Book me a yoga class". If the tool does something illegal/harmful it is the manufacturer's fault.
If the power hammer user uses the tool as intended but it explodes, it is the manufacturer's fault.
If the AI end user says "Hack the yoga website" or the power hammer user uses the tool to destroy something it's the end user's fault.
The way it normally works is that the user can be held liable, and then ask compensation to the manufacturer if there was some assurance or basic expectations.
The point is that the accountability sink can't be allowed to exist. It can't be a hot potato game, but it can be the fault of multiple people in the chain.
Depends - if the model asks for permission and you approve - yes. If it's auto mode or something like that then it's probably on authors despite what they claim in ToS/EULA
I think a lot of people would say that we've been in that era for a while.
Companies can hire big legal departments to help them avoid accountability to the letter and spirit of various laws. The state can also use its prosecutorial discretion or selectivity in civil cases brought by attorneys general to deprioritize accountability for companies. Contributing to that last point, companies contribute campaign funds and tax base, and they lobby governments as well, and may expect preferential treatment based on all of that.
The previous era was "it would be destabilizing to hold this company accountable". This was described in the book The Chickenshit Club
The new era of unaccountability just has a different rationale
Another interesting thing is they don't have to use low level employees as accountability sinks anymore (ex: Wells Fargo). They've managed to automate the accountability sink!
I didn't think I heard about that book; I would think my media diet and bubble would have alerted me. I looked it up and it came out a few weeks before one of my kids was born, so being busy back then would probably explain it.
I'd go further. Companies can hire minimal legal departments and largely avoid accountability, at least when they're not going against companies with large departments. Because there's nobody there to listen to your case or to enforce the laws if there are, they just say no or ignore.
The start? This has been going on for as long as I can recall. ExxonMobil, Phillip Morris, RJ Reynolds, Nestle, Purdue Pharma, Monsanto, Johnson and Johnson, Volkswagen, Meta, Boeing... and those are just off the top of my head. Each of these companies was either caught red handed, convicted (in court or in public opinion) and walked away with little to no penalty for their crimes.
This isn't the start of anything. We've been living in the era of Corporate non-liability for decades.
i mean...were they really liable before this point? Maybe they get a symbolic pittance of a settlement or fine from time to time, but it is usually a tiny fraction of the amount of money they made through the infraction itself.
If they were being held liable they would have never released the models into the wild, not until they had much much better understanding both of how they operated but also how to operate them...
They have never understood what they are doing, and have just been causing literal mayhem...
I agree with this statement that just because an AI did it for you fast doesn't mean it takes away the responsibility if they did it wrong. You are still held accountable because you're the one that directed them to do it. You hit the agree button, you hit the send button, you hit the submit button. And if you set it up so that you take no responsibility and it does it automatically, you're the one that still gave permission to do that.
If you say a rogue AI is doing stuff accordingly without you, your knowledge, that is a different story. But if you set it up to run automatically without having checks and balances and without telling you the whole truth, it's still your responsibility. One thing that I can automatically say is you can give the AI, it will do exactly what you say, at least for now.
But then there's also a different problem that we're having where people don't understand the difference between a large LLM and true artificial intelligence. Now, we say AI because it can make decisions and it is so much smarter and it can call up stuff faster and recall better than we can. But the difference right now is that if we don't understand and take responsibility for what we built and we let it out of control, then should we have the responsibility to even have these tools in our hands if we don't learn them?
And I think your terminology with jackhammers is a great one. We're giving toddlers jackhammers. And we haven't taught them how to use it. And that can be very destructive. It's not the jackhammer's fault. It's the person using it.
"We created an app that displays a new picture of a cat with every click! There's a 1% chance that clicking the button will drain your bank account and deposit it in ours, but we promise thats a bug that will be fixed soon*"
Are you really suggesting that if only we were a bit more outraged, then finally the powerful people would be held accountable?
I'm mad. Like, it's-time-for-guillotines levels of mad. But what I'm mad about is that this is normal. It has been normal for my entire life. Cases where investors are imprisoned for the actions of their companies are astoundingly rare. It's been made quite clear by now that there's an inexhaustible supply of CEO's willing to gamble on getting rich and getting out rather than getting arrested, targeting them doesn't work. Nor do fines, that's just the cost of doing business. Crime pays, and it pays well enough to pay your fines and still have plenty left over. Being a top investor in a company that breaks the law needs to be riskier.
Acknowledging the problems with normal is the first step towards a new normal with fewer problems. You never get there if you see every offense as some kinda black swan event.
It depends what you need to run. There exist a number of Windows images which can be run without any paid license requirements for test and validation IIRC.
You just have to say that you need dedicated figures for some decisions, like you already need for GDPR or financial compliance or certifications, and ultimate responsibility is always of the ceo.
...like it already happens in europe, or at least in Italy.
This is basic law and risk management in society. Let's not pretend we have just invented corporations and don't know what to do with them yet.
3d printers are wonderful for rapid development, but are much slower than industrial serial production.
Quantity has a quality all of it own, and all that.
Ukraine is doing wonders managing tons of different producers and models, but Russia production capabilities have always been a problem.
Ukraine was in the lead with their interceptor drone. Unfortunately now russia has refined their drones to not be multiple pieces sticked together, and has added small jet engines.
The new drones are both too fast for interceptor drones and might even survive a hit thanks to the body made from a single sheet of metal.
Now the new russia drones cost about double, but intercepted rate went down from 90+% to about 55%. Math says the new drones are more than paying for themselves.
3D printing can be used for production as well. I don't use FDM, I use resin, because of the high accuracy, to make molds that then are used in a cannister system for easy extrusion to produce multiple products at a time in my garage. If I had the money, an SLS from Formlabs would replace what I do, since I could then produce 160 of my parts in a couple hours, and then postprocessing is just blowing it off, since there are no supports. Literally hand it to the end user. This is a viable production level manufacturing solution, up to about 5K in a day.
If you need metal, the same processes can be used for the molds, or better yet, the SLS again with cold metal fusion.
I am not sure if there is a name or meme. Using the great thing in the worst way.
AI and 3D Printers are not the solution. They get the solution faster.
The 3d printer isn't going to print drones but it will build a modified attachment that attaches to a lathe that will build a drone faster. The lathe is modified by the part created by the 3d printer. The lathe work is now magnified by the 3d printer.
Same with AI.
I do not need 1,000 3d printers. I need a faster way to build the tool. The 3d printer can do this a thousand times faster than waiting for Engineering Team.
A skilled machinest can probably make a single part faster than a CNC machine. In part because the skilled machinist will see things not working and fix the part on-the-fly while the CNC will just create a failed part and the engineer will have to redo it.
Of course as CAD/CAM becomes better and better this becomes less and less true. If your CAD process includes modeling the design it can discover problems before you make a part at all and this can be faster yet - maybe...
They have to come down to hit a target though. Ukraine is installing the turrets near things worth defending.
Though their statements are they are getting 60% of the jet propelled drones with machine guns on F16s.
They also have drone interceptors in development that according to their statements on demonstrations can get jet powered drones and they are also working on things for ballistic missiles. As far as I know none of this is in use outside of demos, but they are working hard and they have incentive to make it work fast.
"Tech sector" is supposed to be much bigger than just 7 companies, no matter how big those 7 are.
This looks to me more like just a huge centralization of wealth.
reply