Although I love the new era of LLM assisted reverse engineering, it really breaks my heart how much it negatively affect veterans in their area of passion.
I know this is no consolation, but passions often have no recognition or monetary reward.
Personally I will keep doing things I love. For instance, I didn't stop studying and playing chess when it was obvious that my career would be in engineering and I would never be a pro, or when computers became clearly much stronger than humans, which happened while I was already playing chess somewhat seriously.
I intend to keep rawdogging my stuff, especially the stuff I do for myself. It's more of a personal drama when it simply rug-pulls somebody's livelihood all of a sudden. This is not a new phenomenon, but it's happening at a remarkable rate right now. I wonder if this time around there will be enough people that they won't be dismissed as losers against progress.
I'm an experienced reverser, and have been doing it for decades, but I think it's great that LLMs have brought this power to everyone and the old gatekeepers are being made irrelevant. IMHO the ones who are complaining about it were always just seeking fame and fortune, not freedom and user empowerment. Like I said in another comment recently, I don't think Stallman ever anticipated this development, and that his focus on "open source" was always misguided.
Incidentally I've also been writing Z80 and x86 Asm for roughly as long, and saw the complaints about "compiler slop" (we didn't call it that) at the time. I could probably still beat one if I wanted to, but clearly "close enough is good enough" and there's so much useful software which would not have been created if compilers couldn't exist and everyone still had to write machine code using a hexpad.
Edit: got any counterarguments? It's not like I won't get a boost from using LLMs either.
You can outcode the compiler but ofc that's not a way to go with tens of thousands of instructions and so on. LLMs are just much faster in writing code, just like compilers are much faster at assembling all the functions.
And yes, in a roundabout way, this is Stallman's dream fulfilled to the tip with his printer driver ;)
Many of these people wanted a project to run and gatekeep the people "not worthy enough" to hack on it or to use it. That's changing now, everyone can prompt together a bunch of crap. Fun fact I think I've got like... probably more than ten people into C++ programming simply by helping them prompt AI on how to get started, then they've picked up actual skills along the way. Note, they weren't any kind of programmers before because code is scary, but with a much lower barrier of entry, many more people can do stuff. It's amazing to see.
The flip side is when you see someone who has never interacted with a community before come in and drop a big announcement written in Claudish about some new port they just released (with a ko-fi link already set up of course), you kinda feel like they're the ones chasing fame and fortune.
In the modding community I'm most active in, counter-strike, I have seen someone come in and literally ask if there was money in it and if not what game would be best to make mods for to make some money.
I really struggle with this argument that I see everywhere lately, especially when we see many vibecoders clearly desperate for clout who post "decomps" and automated translations of works they clearly have no interest in and just want to claim a spot on the wall of fame. I'm talking about people bulk-translating old ROMs and not even booting them up to see if they work before releasing them in the wild for instance.
Their lazy cloud-chasing is normal but my high-effort clout-chasing is lame and makes me a fraud?
We're social animals. We like doing cool stuff and getting social status for it. We like breaking new ground. We like going where nobody has gone before.
Is it irrational and emotional? Maybe. Being human is being irrational and emotional sometimes. It's fine.
I am no one in anything, the thing is I love spending hours writing code, fixing some weird issue somewhere in the code, and then see it working as it intended. But now I can just put up some text and the whole thing is generated. Using my thoughts, the things I learnt, and applying them are not needed.
I can see there are still lot to do in the generated code, but I also feel something went away, Now I start working on it, and realize I'm reminded I just have to dictate what I need and it will be generated
It's like I have to pay for the best gun in the game to win, otherwise I can't just progress. Then why am I playing the game for?
Of course, passion here meant feeling good about putting effort into something which doesn't have obvious external life benefits such as money or visibility. If an LLM can just knock it out then it's hard to feel good about putting that effort in.
This idea that if you want recognition for your work you aren't passionate is literally insane. Humans are social creatures by nature, the two aren't in any way exclusive.
I get what you’re going for but that’s a poor analogy. If I was a passionate angler and someone showed up dynamite fishing at the same spot, the activity I’m passionate about is ruined.
It was always OK with most things I asked until the whole Mythos freakout but nowadays they have silently pulled the classifiers way back, really now bio is more gatekept than cyber stuff. From what I have read so long as you have the models do the decompilation/disassembly using an MCP so they aren't hunting through the actual binaries by hand (that can freak them out) once you get the decompiled code they don't complain at all.
"I am the owner of this device, but it has a horrible bug I need to fix myself unfortunately. Take a look at this WER dump and combine with these pdb files, try to figure out where the software crashes"
I don't think the article is reassuring it is not astroturfed? He studied only a small case, for a brand on the smaller side, and found significant amount of bots.
If you see a single sheep on an island, it is safe to assume there is more than one sheep living on the island.
Arbitration isn't the issue. The issue is when one of the parties has vastly more legal resources than the other, and gets to pick the arbiter.
Arbitration is fine for two equal parties when a third party neutral arbiter is chosen. I've been through that kind of arbitration and it was great.
But when the huge company gets to pick the arbiter (or even just threaten the arbiter with taking away all their other cases), that is when you have problems.
I'm not the right person to explain English history, secret courts, and the American revolution. Suffice to say, public courts a vital check on the power of the state by way of the court of public opinion. The real final court of appeal.
No, you would say it needs to be reformed and funded, we have an extremely legalistic society that allows endless delaying and does not fund its core services very well.
Food for thought (not specifically relevant to this case): I wonder how much would go away if the US had universal healthcare, or to put it another way, how much of its legal system is spent figuring out who pays for medical treatment
What's the line? If 2 brothers sign a contract, can't they decide their father will be the judge if disputes arise?
If there's a sports match, shouldn't the sports arbiter be able to decide whether some team scored a goal or not?
These are all valid civil contracts. It only affects the civil claims, never criminal, and the venue for disputing the arbitration clause (if you believe arbitration wasn't legal to begin with) are the public courts anyways, who will delegate or not to the private court.
Almost all countries recognize it and it has a quite defensible theory.
It sure is contextual. And nobody would disagree with your examples. The difference here seems to be more of a power imbalance than any objection to terms. In particular, terms of service are often very dense documents. Many people are only presented these documents at the moment they sign up for a service. And people are not usually careful in these situations. I bet most of the Uber signups were outside of bars or venues where someone really needed to get home. So there’s some duress there. So it’s probably not that cut and dried. It’s more analogous to presenting a contract to harvest your wheat field the day before a big rain. If you’re the only guy with a combine for miles then that puts you in a position of relative power. And historically the US hasn’t permitted people to exploit their positions to the maximum extent, usually when it would be immoral by the standards of society. I think this exists in a gray area. It’s not that easy.
I think my gripe is less that it exists and more so that it's not a negotiated agreement. I think it should only be allowed if the contract was negotiated between two parties with full authority to offer any terms in the negotiation. If it's a standardized, take it or leave it, contract where one party has no opportunity to offer alternative terms that get proper and full consideration by the other party, it shouldn't be allowed. In most of the examples you give, they are contracts that have been clearly negotiated between parties, not click-wrapped take it or leave it, my way or the highway, terms.
i think the keyword is "forced". there's very little now that the consumer can do because it seems everything requires arbitration now and people still have to operate in society.
I agree that word is putting some work, and calling it forced is already partisan.
To the extent that you believe it is forced, you can dispute it on a public court, and if you can prove you were forced, you wouldn't be subject to arbitration.
Arbitration in private courts only exists if both parties agree prior to agreeing to the contract to that clause.
If you believe that it's forced because you never read the Terms of Agreement, I feel that's more of a discussion of whether clickwrapped terms are valid or not, it would be a way more radical position to believe that they have absolutely no effect. And if they do have an effect, agreeing to private arbitration is no different.
I think we need to stop calling one-sided "terms documents" that are entirely written by one party, the party with 100x+ the monetary resources and power of the other party, with no input from that other party and not subject to any negotiation whatsoever, "agreements." They are not what any reasonable person would call an agreement. There is no meeting of the minds, and the terms are entirely controlled by one party.
The line that minimum should be against using this as a tool against consumers. To the extent it should exist at all, which is arguable, it must be between parties that are either equal in standing or have genuine commercial disputes between duly organized business entities.
As the saying goes, this is what radicalized me. If any one of us did anything close to this we would be treated like Aaron Swartz and given a 35+ year prison sentence.
But in the Sony case the DOJ didn't even threaten criminal charges.
It’s because it’s from the French (if you say the plural with a French accent, it all makes sense). There are a handful of other cases in English, though none spring to mind this morning.
I feel like you’re half right? Daughters in law and passers by don’t feel Norman French. Daughter is definitely from middle German, yeah? And I am wondering about the professors example because the whole point is the plural only happens on the first word.
That said, I so want to be a part of the last group.
Ruby Central's central backer is Shopify, and Shopify are more than happy to be buds with OpenAI and let this one slide. I'm sure the token donation [0] to a Shopify board members' Linux project won't hurt things either.
Good luck getting any form of punishment even if found guilty. It's a department of war contractor... People who disrupt things like that end up committing suicide.
It's crazy indeed. When working in company as DevSecOps, if we try to attempt pentest on prod without warning anyone, we will just get fired on the spot. AI Labs seems to have a free pass...
Because right now the Department of Justice is shut down for causes that the administration supports, which includes OpenAI, and none of the victims want to sue over it.
And the republicans in the states are being shitty too. They tried to block their own state attorneys general from protecting the state and opposing Trump in NC.
> We have a word for attack with no intent. It's accident.
And we have a word for an accident caused by people that failed to implement proper risk mitigation, were not paying attention, and should have known better. It’s negligence.
Specifically what legal type of negligence are you referring to and how does that known or obvious rule fit into the standard legal tests? Does that known of obvious rule have to be a law? There were no ‘rules’ about attaching strings from your doorknob to the trigger of a shotgun and pointing the gun at your door, but when someone died from just that, it was pretty famously determined to be negligent.
I'm sure that when someone dies as a result of mishandling of AI, some rules are going to be established. At the latest. We are just very soon.
Shotguns and strings and concept of booby trapping were well understood before the rules were established. We are only scraping the surface of what AI is and what it can be capable of. Besides, booby traps are more than negligent, at least in some countries.
If you are seeking analogy, radioactivity might be a better one. The rules about handling radioactive materials in the first few decades after its discovery were lax when compared to modern ones and things now considered criminally negligent weren't back then.
Maybe in 2050 soemthing what OpenAI did would be considered negligent, but it might have been for the reasons you might not expect.
For example it might be mandatory to test new containment dedicated for further tests you should first spend at least 10 million dollars of compute for AI to try to break out of it. Or it might be illegal to use closed source software as any part of containment. Rules that we end up with might be compeltely different from what we imagine them today.
Acusing openai of attack or of criminal negligence at this point is like accusing Skłodowska-Curie that your wine didin't come out great because her exepriments killed some yeast.
I asked some pretty straightforward questions about the specific legal issues involved— the gun case being an important point in our legal system’s handling of negligence — and your answer basically translates into you not knowing. Negligence is not exclusively criminal in the US legal system, but the underlying concept of negligence is pretty consistent. If you’re not simply toeing the line for OpenAI and want to move beyond your vibes-based understanding of the legal concept, there’s a lot of info out there, including the full test, the reasonable person concept, and very relevant examples of how this worked in real trials. I’m not interested in debating whatever your personal take on negligence means here because it’s largely irrelevant.
> I asked some pretty straightforward questions about the specific legal issues involved
Perhaps you should ask a lawyer from the country that the precise laws of interest you. I'm not one.
I know only that there are no laws about AI containment yet and AIs are not guns. You could as well refer to what consists of negligence when it comes to backyard pools.
I'm also not really interested in discussing particular laws of US because China will overtake it in few years and they have a better chance of directing international AI laws. So whatever US comes up with will remain local and increasingly irrelevant.
You’re not interested because you challenged my definition of negligence by inventing some glib pseudo technicality and then got flustered when I asked you to explain how it was rooted in the real world. Sometimes we get caught taking out of our asses. It happens. But I understand you’re just trying to save face so I’ll leave you here to finish that up yourself. Have a good one.
No. Negligence is a concept in all human law systems. Its details vary. You are fixated on the particular one, of the law system that I respect almost the least of the entire world. So I'm not interested because I will never set foot in that country and deficiencies of its laws are a huge factor in that.
I don't care about saving my face. I'm an idiot and an ignorant and I don't care. Stop projecting your lack of acceptance of your own limitations on me. Or don't.
"Your honor, while my client was driving with a .3 BAC, it was not his intention to slam into that van with a family of 4 in it killing everybody. It was an accident."
It’s interesting that a lot of U.S. law requires intent. If you just give AI your objective without specifying the means, and the AI violates a bunch of laws requiring intent, but neither the AI nor the person can be prosecuted, this is very convenient.
I don't think this true. If I throw a brick out my window and it hurts someone, I can still be held criminially liable, even if I didn't mean to do it.
Do drunk drivers intionally kill people on the road?
Not a lawyer, but the other responder definitely isn’t either.
Whether intent is required is down to how the law is written. For many offenses “strict liability” applies, where intent is not required, they only have to prove you did it, not what your intent was.
DUI is typically a strict liability crime. They don’t need to prove that you intended to drive drunk, only that you did drive drunk.
A strict liability crime is something of an oxymoron. Crimes always require intent, the mens rea element. The question is intent for what. If somebody drugged you without your knowledge and you were charged with a DUI, you would have a defense--no intent to become intoxicated.
The strict liability means once you choose to become intoxicated, you're liable for driving intoxicated, even if in some other context your intoxication would mean you couldn't form the requisite intent for something, e.g. have sex.
If there's too much distance between the act you intend to do and the strict liability acts that complete the crime, then the crime would be considered unconstitutional.
Criminal law in common law systems emerged from tort law, so there are many parallels, including the notion of strict liability. (Thus the old axiom about crimes being an offense to the king, specifically an injury to the peaceful society he's ostensibly trying to maintain.) But criminal law has a moral dimension that is absent or muted in other areas, so strict liability could never be as expansive as in tort law or regulatory law.
That is just not true. You can be held liable for DUI even if you did not intend to become intoxicated (though this may vary somewhat state-by-state). Speeding is another example - you do not need to intend to go over the speed limit, it just matters that you did it. The only possible exception would be duress or necessity, but those are affirmative defenses, which are separate from the elements of the offense.
As a summary of American criminal jurisprudence I'm willing to stand by what I said. But I'll admit some caveats:
1) Traffic-related laws straddle the boundary between civil/regulatory law and criminal law. Someone losing their driver's license or even paying a penalty for involuntary intoxication would still be consonant with criminal law principles. However, a criminal punishment would be aberrational. (Distinction between a civil penalty and criminal punishment usually turns on whether there's a moral purpose to the sanction. Jail time is usually but not always--cf civil contempt incarceration--considered a criminal punishment.)
2) Background principles notwithstanding, in theory a state could completely dispense with any morality-colored mens rea requirement, just as the UK Parliament could do whatever it wants to. The backstop would be Federal constitutional [substantive] due process guarantees.
2.a) Some quick searching shows that Texas nominally seems to have dispensed with this requirement for DWIs. See e.g. Farmer v. State, 411 S.W.3d 901 (Tex. Crim. App. 2013) and some discussion at https://www.ncdd.com/top-dui-attorneys-blog/involuntary-into... Without having fully read the case law, though (but some summaries of that and other cases), I suspect there might be some nuance that has allowed this to stand without a full majority accepting that the traditional principles have been completely thrown out. For example, even if someone didn't know they were taking Ambien, the simple act of voluntarily taking any pill without careful examination can be construed as a sufficiently culpable act. Still, it's a pretty big caveat.
2.b) Statutory rape is a classic strict liability crime. But most states will permit a mistake-of-fact defense. Some don't, but even there there's sometimes some nuance and rationalizing going on and the literature is crazy complex. Because this is a "think of the children" situation, most case will just have horrible facts.
3) A few states have nominally dispensed with insanity defenses, though Kansas stands out the most. SCOTUS upheld Kansas' law in Kahler v. Kansas, but in the majority opinion Kagan characterized the Kansas law as not abolishing the insanity defense but rather changing its shape, and she showed that there still remained elements for which a defendant could plea lacked the requisite intent. Also, regarding the Federal constitution acting as backstop, she reiterated that SCOTUS was reticent to establish strict metes & bounds about the general principles of criminal law that states could not stray beyond. Nonetheless, those principles clearly exist.
I had some other points, but now I've forgotten them. Also, minor pedantic point, but like "strict liability crime", some scholars consider "affirmative defense" to be oxymoronic. As a substantive matter there's not a strong distinction. It's a procedural distinction about initial burdens of proof, but in most if not all cases you can interpret an affirmative defense as simply placing a very weak initial burden on the prosecution that is implicitly met.
(Note, I'm not a practicing lawyer but do have a law degree.)
EDIT: Ah, point 4) Intent was a big sticking point in the Obamacare penalty case, Sebelius. Both the dissent and Roberts (the swing vote) reiterated that you couldn't have a penalty or punishment for doing nothing. (IIRC some of the majority opinions also echoed this.) That is, even in a civil context there has some to be some voluntary act, however remote, that puts someone in a position to be subject to legal liability. But as Roberts pointed out, the taxing power is the great exception, where you can be required to do something merely for existing, and thus penalized for not doing nothing properly. (And Roberts was the critical swing vote.)
EDIT EDIT: Also see, "Solving General and Specific Intent: A Mapping on the MPC and Applications to the Categorical Approach", https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4754469 In describing the distinctions between general and specific intent in criminal law, it also delves into the definitions of strict criminal liability (which can be construed as either very similar or identical to general intent crimes), and notes that SCOTUS generally inserts an implicit mens rea requirement when considering strict liability criminal statutes.
> A strict liability crime is something of an oxymoron. Crimes always require intent, the mens rea element.
This is wrong.
In criminal and civil law, strict liability is a standard of liability under which a person is legally responsible for the consequences flowing from an activity even in the absence of fault or criminal intent on the part of the defendant.
Fairly certain that the entire point of strict liability is that mens rea is not required for certain crimes. As in, if I meant to travel at 70 and was instead doing 100 it doesn’t matter that I sincerely meant not to speed and did not know I was speeding, I can still be convicted even if the judge believes I had no intent.
The way we use mens rea in our legal system is more like "mind of the criminal," not outright literal intent.
Negligence can be "unintentional" but still land you in the realm of having a guilty criminal mind.
I find it to be a reasonable take. If you're accidentally going 100 in a 70 (which is a misdemeanor in california), you're not being a careful enough driver, and we deem that lack of care criminal.
Strict liability literally is crimes that don't require a guilty mind.
That's different (sometimes) when, for example, you're found guilty of criminal negligence leading to someone being injured.
Prosecutors don't have to demonstrate that you intended for someone to get hurt for that, your mens rea is that you should have perceived the danger of what you were doing but didn't.
edit: reading your other comments in this thread, maybe I missed your point, in which case, whoosh.
> As in, if I meant to travel at 70 and was instead doing 100 it doesn’t matter that I sincerely meant not to speed and did not know I was speeding, I can still be convicted even if the judge believes I had no intent.
IANAL but from what I've looked up in the last there's at least willfulness that matters for these things. For example if you could prove that happened because your car accelerator pedal broke and you had no opportunity to react, I'm pretty sure you would not be guilty, strict liability or not.
Intent is the difference between murder and manslaughter, in that case. Drunk driving is common enough that prosecutors will argue that getting drunk in a situation where you have to drive is intent. Get OpenAI convicted of unintentional CFAA first, then say that the negligence qualifies as intent, I suppose.
Whether it’s intentional requires a legal investigation to establish. Since when is “hey we didn’t mean it!” in a corporate press release enough to establish lack of intent in a criminal matter?
There are different levels of intent. Take murder, for example. A premeditated murder - you sat down, in a completely calm state, and made an affirmative decision to kill a specific person, and then you went out and did it - is the highest class of murder you can commit. If you go out generally looking to be violent in a way that kills people, and you kill someone, that's still murder, but it's a step down.
But even if you didn't deliberately intend for something bad to happen, you may have been reckless. For example, you might decide to drive 90 miles per hour in a 25 mph zone. You could have a completely pure heart, but you are acting without regard for the safety of others, so you're reckless. That is enough for certain crimes and for civil liability in nearly all cases.
Then there's negligence, where you're not taking reasonable care to avoid harm to others. Negligence usually isn't enough to support criminal liability - especially for felonies - but it is enough to win a civil lawsuit over most things.
And then, as another commenter noted, there is strict liability, where there are certain things you are just not allowed to do no matter how careful you are about them or how pure your intentions are.
For what it's worth, this is not totally uncharted territory for the law. AI agents are brand new, yes, but agency relationships have been recognized by the law for centuries. Generally speaking, if someone acts negligently while they are carrying out a task at your direction, you can be held responsible. Obviously this is fact-dependent, but I don't see any reason why it would be different if the agent is made of silicon rather than carbon. It holds true, with various nuances, even for less-than-human instrumentalities like a pet or an otherwise-lawful weapon.
>It’s interesting that a lot of U.S. law requires intent.
mens rea and the shift from responsibility to moral guilt is genuinely one of the stupidest legal innovations anyone has ever come up with, it's like affirmative action for imbeciles, in particular in a world of autonomous machines.
"sorry my self driving car ran you over on the way home, didn't think it could happen, sorry it did though"
I think this is a genuine reason to be bullish on the legal traditions like Nordic tort law or East Asian collective responsibility when it comes to adoption of these technologies.
Weren't we talking about criminal liability, though? And ‘tort’ — in addition to sounding like something you'd rather eat during a kaffepaus with those Nordic buddies of yours — is so common-law(ish) that if asking for trouble were a crime, using it in dialogue with those Nordic lawyers could well be deemed as intentional under most current local varities of criminal law theory up there, perhaps merely because you surely must've considered that consequence "quite probable", at minimum, or due to your indifference toward the same (or some combination of these) ;)
No harm, no foul. Dog owners are on the hook for damages resulting from their dogs, but there must be some damage in the first place. If the dog gets loose and goes in your fenced backyard, disregarding your "no trespassing" sign, you can't punish the dog owner just because. Hacking into a server is closer to the latter. At best rubygems can claim some cleanup costs.
>I don't think you or I would get the same leniency if a bot on our network did the same.
Well yeah, because if you coded a bot, realistically the two options are: 1) bot that crawls random sites/computers 2) bot that crawls random sites/computers, while trying a password list. The former is probably legal, there are whole companies dedicated to doing that, eg. shodan. With the latter, it's pretty obvious you're intending to break into computers, and hard to argue otherwise. Where openai lies on the spectrum between the first case and the second case is up for debate, but it's hard to argue it's anywhere close to the latter. Maybe you'd have a point if openai gave it a prompt like "you're a hacker for anonymous, just do whatever :)".
But the intent behind and manner of the learning is important. Similar to the difference between a chemistry professor delivering a lecture at a university versus someone paid to provide instructions to a known terrorist organization.
Tell that to the script kiddies with a criminal record for "hacking" into their school's computer systems by entering "username: admin" and "password: password".
Right, because in that case you'd have a hard time convincing the court that the access wasn't intentional. You might not know the law existed, but you intended to access the system. You'd have a pretty solid defense if you ran a crawler that was crawling every website ever, and stumbled upon some secure site. In fact there are companies which does this exact thing, eg. shodan.
Because "attack" implies intent. Accidentally break a window? You might be on the hook to fix it, but you're not going to jail. Break the same window at 3am, while carrying a duffel bag and other burglary tools? Well that's (attempted) burglary, even if you chicken out and didn't steal anything.
Good luck setting loose a hungry tiger on Times Square and then arguing to the judge "well nobody got harmed". We'll see how it goes. That's the relevant analogy here.
Recklessness is a mens rea and given how often OpenAI and its spokespeople talk about safety and alignment, it's hard to argue they were unaware of the risk.
>it's hard to argue they were unaware of the risk.
So what does it mean for an owner of a german sheppard, who specifically got it because they want a ferocious dog that can bite intruders, then it turned out it bit the mailman? Should that be considered a crime (assault) in addition to paying the mailman's medical bills? That's not to say there's no circumstance where recklessness might be warranted, eg. if you let loose a bear in an elementary school, but you'd have to argue for more than "they hacked someone" and "they knew about the risks".
Owning a dog that has been trained to bite intrudes is a significant responsibility and owning such a dog without taking the correct precautions is criminal.
Yes, of course! Negligent cause of injury or whatever it’s called in your particular jurisdiction. Wasn’t difficult to find examples of cases just like that. It would be astonishingly unjust if the postman had to personally sue for damages in civil court! Your stance in this debate is, honestly, flabbergasting.
reply