I am pretty sure the distro packagers all know where to get the real code from, and its packaged for everything that runs X so minimal risk.
Also assuming something on github is the official repo is dangerous anyway. There might be multiple repos with the same name, there might be a more actively developed fork and the official one might be unmaintained, etc.
Because I do not think making github a way of verifying things is a good idea. It encourages people to assume that whatever they find on github is safe.
As for "happens every day". How often, really? JWZ says "I strongly recommend that you install a binary package rather than compiling it yourself." I would guess very few end users compile xscreensaver themselves.
Warning: it persists somehow, so you can't click on a link from HN, realise the problem, then copy+paste to correct it. Looks like you need to open an incognito window, use a different browser, or maybe otherwise clear your session.
Every play made by these companies aligns with what you’d do if you have a $1T valuation to protect against an industry that seems increasingly commoditizable.
I’m still… hopeful… that “good enough” models will be abundantly available to be run anywhere as to limit the low water mark for market capture by a few big ones.
If that was the move, you'd expect they'd be actively fear-mongering against open models specifically: not just their training, but their usage and deployment. They'd want to paint open models as dangerous weapons for terrorists, etc, and that only the big firms are capable of responsible engineering and "guardrails".
Instead, we've seen the opposite: all the big scary stuff has been from their own models (in a few notable cases, also demonstrating that they're actually quite irresponsible in their monitoring and security).
"We need to outlaw single-prop planes, they're a risk to the public: only Boeing has the ability to produce planes safely. How do we know single-prop planes are dangerous? Because Boeing planes keep falling out the sky."
I don't know if that analogy holds. You don't normally blame Toyota for car crashes, but the driver. If you must use a car analogy, consider the 2010 Toyota recalls rather than just normal car crashes.
That is to say, anthropic should be considered responsible, not (or not just) the users.
Who takes the blame when someone ask the blackbox to complete a task and it then breaks the law in order to complete that task without the user having any knowledge of what it was doing or intent for the machine to accomplish the task in an illegal manner?
The situation as provided is missing too many details for anyone to make a meaningful judgement call, but remember, more than one party can be found liable/guilty (maybe of different things) in a situation. Overall, the law usually protects people who are acting reasonably, but does not protect people who act negligently.
I'm not a lawyer, but I imagine there's not a lot of precedent or law around this, so it'll be up to the court to decide on the finer details until we do have meaningful laws/regs. Anthropic will probably have better lawyers than the user.
This is why it's important to get ahead of issues like this, and anthropic should not be involved in discussions around regulating itself...
When it's an open weight, locally run blackbox, you blame the user, since in that case it's very much possible for the user to restrict what the blackbox can actually do. They have full control over it, so it follows they have full responsibility.
None of that requires trying to impose limits on open models, unlike what Anthropic is aiming for.
> We have seen repeatedly that users are not able to meaningfully restrict the blackbox.
We have seen these companies repeatedly ship coding harnesses without any attempt at real sandboxing, and encourage people to let them run free. Seems that if they've somehow solved coding, they'd be able to solve making it easy for people to sandbox agents, and to be fully aware when they've enabled some LLM the ability to hack companies. Why don't they put 10,000 agents on that instead of ripping off Navier-Stokes?
Because its not profitable and that would shift responsibility onto the company that claimed it was safe, rather than the consumer who doesn't know better because the media has sold LLMs as a panacea.
Aren't able to or have just failed to do so? Regardless, that doesn't exempt them from the responsibility. It's ultimately software they're running on their own machine, there is no one else to responsibilize for it.
I wasn't trying to imply you were advocating to restrict them, just that that is what Anthropic is doing. Legislating away all competition is worth whatever small concessions they would need to make.
Sure, I agree. I just think there comes a point when the industry has a whole should be held accountable, and in the past we have done everything possible to shift that responsibility onto consumers how generally have very little choice in the tech they use.
Well, you shoehorned it into a completely unrelated conversation, so forgive me.
If that were the intent, from the OAI/Anthropic perspective, you'd think we would have seen at least one of the scary incidents (either real or simulated) being performed by Qwen or DeepSeek, rather than from their own models.
If anything, their public messaging has been to entirely pretend that open models don't exist, that they're mere toys or curiosities, that only the big dogs with eleven zeros of capex have capabilities which could be considered dangerous.
You could certainly argue that the various "regulate AIs like airplanes" bills might shut down some open model development (domestically; they're certainly not calling for international weapons treaties). But the big firms haven't exactly been jumping backflips in support of Sanders' Ban Artificial Superintelligence Act.
There are zero laws against the design, manufacture, sale, or possession of cars with spinning blades and/or flamethrowers.
Such laws attach to the human operators of such cars, if they recklessly endanger others by driving them on shared, public roads.
But they're perfectly legal to own (and operate off-road). And, there's no technical barrier stopping people from illegally driving them on public roads: just fear of laws.
(There's entire subcultures of people who modify cars to do insanely dangerous things, on private race tracks. Perfectly legal).
If we were to apply Amodei's safety standard to cars—do they have the theoretical risk of killing over 50 people—well, cars empirically kill tens of thousands of people in the US annually, so we'd have to categorically ban them. That standard is bad-faith and absurd.
No. Half the point of LLM's is having them research topics on the internet as your agent. It's totally harmless in the usual (tool-based search) workflow.
(Would we also need a real ID to operate curl(1)?)
Regulations should attach only at the point humans willfully or recklessly cause their agents to commit crimes.
But keeping with the analogy, a car manufacturer would be liable for a faulty design that caused wrecks, no? It doesn't need to be willful by the user.
Yes, sure. I'd be open to the idea of regulating commercial vendors of inference tools to consumers (for suitability). You can separate the concepts of commerce and speech (and US law does that extensively).
What Amodei's trying to do is outlaw free, open-weight models. That's two distinctions: (1) non-commercial, and (2) abstract, intangible mathematics. Pure speech. Same as Phil Zimmerman and PGP.
Even the EU's draconian open-source regulations (which HN criticizes endlessly) exempt free software which is purely non-commercial.
Yet we don't regulate computers so that they're rendered incapable of running software that can be used to hack into systems. Or said software, by the way. All we do is punish the people who use these tools to cause harm to others.
Technically no, but you can distribute software, especially for free, that happens to break copy protection for one or more valid reasons, according to the law.
And, more importantly, we certainly don't try to limit the capabilities of computing that might be used to do those things. Given how AIs are very general purpose tools, the fact that they can be used to do something potentially harmful is too narrow a threat.
Nobody is trying to push for cars that automatically stop themselves from running over someone either, which would make for a more comparable analogy than cars with flamethrowers.
That assumes AI is designed to be malicious, "the spinning blades". AI is not; the public version given to you is the sanded down one, in the same way that cars don't have spinning blades but can still be dangerous when misused.
If a person closes his eyes and hits the gas pedal on the car and runs over someone, the person is to blame, not the car. AI companies do the same. Your eyes may be closed but you're still operating/hosting the car/AI.
There being no third party checks is the point, and it's not bad.
Keep in mind that those open source/weight models are not as powerful as whatever closed source Claude/GPT version exists behind the scenes that hack companies. As per the Hugging Face hack - open weight GLM saved the day as closed source ChatGPT refused to help Hugging Face fix their systems. An example of my point that "sidestepping any third-party checks that could be present" IS good.
Part of point of open source/weight AI is that it allows what those model providers can't: uncensored models for whatever reason you may need it. You are responsible for being legal with it.
If your AI model develops the capability to manifest itself in the physical world as spinning blades and flamethrowers I'm fully open to re-evaluating the need for regulation.
Until then, I propose we use the 'unplug the server' rule. Can [insert bad thing] be prevented by unplugging the server? Then these things are fully in our control and no, we don't need to re-build hundreds of years of legal precedent from the ground up.
Liability in the event of harm is already something we have a pretty good handle on from living through the industrial revolution 150 years ago, a time when much more activity that companies did existed in the real world with real harms, as opposed to digital companies doing imagined "mental" harm to soft desk chair people who work at email jobs.
> If your AI model develops the capability to manifest itself in the physical world as spinning blades and flamethrowers I'm fully open to re-evaluating the need for regulation.
No, only metaphorical AI spinning blades and flamethrowers should be regulated. Actual AIs with spinning blades and flamethrowers will be subsidized and unleashed in your town.
But we still require cars to have seatbelts, turn signals, ABS, etc, etc. AI companies have a responsibility to build AI which is safe for normal purposes; users are responsible for not misusing them.
Open source models are objectively dangerous. You are delusional if you think otherwise. I am not really arguing for banning them (seems impossible to enforce), but also denialism of the real dangers involved is pretty naive.
A sizable part of the country thinks it’s okay to regulate firearms. Under your logic, people should be responsible for who they shoot, don’t blame the gun when a human pulled the trigger. While gun control advocates have that pesky second amendment hampering their efforts, AI/LLM systems have no such protection.
Regulatory capture is certainly something to be wary of, and obviously Anthropic is only going to be in favor of regulation that helps them in some way. But some form of regulation is coming. There is nothing that can stop it, unless you somehow make more money than these guys do and fund a massive counter-lobby. Spoiler alert: the only guys that have more money than Anthropic and OpenAI have all invested in them.
A much less sizable part thinks it's okay to limit local computing. There isn't a widespread push to make it so local hardware is limited in ways that they're not allowed to, for instance, run programs that can hack systems. These programs are very deliberately allowed, and it's people misusing them that are held responsible.
Regardless of anyone's instance on gun control, it's probably wiser to compare AIs with something that's a little more similar to it, and try to reach a conclusion from there.
> Regardless of anyone's instance on gun control, it's probably wiser to compare AIs with something that's a little more similar to it, and try to reach a conclusion from there
While the point is well taken, it might be well to consider that according to some of the major rhetoric, the AIs are perhaps (far?) more dangerous than guns.
And so it may be an interesting position to adopt or consider in argumentation.
In the sense of: "ok if we concede the danger for sake of argument, then how shall we treat this?"
Ok, so the plan is to stop z.ai, DeepSeek, and all other non-American labs from developing models entirely, full stop? How do you plan to enact this large scale system of international coordination? Asking politely?
I fully agree that companies are doing this, not their "agents", and that companies are responsible for the damages they do.
But I don't think the average person with their $20 subscription to whoever is the primary source of revenue. Enterprises fund most of it by buying API keys and making those fancy chatbot buttons on their sites that nobody clicks, as well as some internal business automations if I had to guess.
If it was just average people with $20 subscriptions funding all this (keep in mind, most people using AI services do not pay for it), by now, all these AI companies would have gone bankrupt.
Except enterprises aren't funding it with API usage either.
These businesses lose unbelievably large amounts of investor money. Their AI-related revenues don't get even close to paying for their AI-related outlays.
End user (invididual, small business, large business) clients are still complicit not because they are funding it, but because they are signalling to investors that this is something people/businesses want.
Nobody gets off the hook here. It is every bit as much end user nihilism as it is AI company nihilism.
I think we know today that blaming everyone and their plant doesn’t actually solve the problem. The formula that does is: find the ultimate beneficiaries (shareholders) and make them lose a lot of money to, then their delegates (CEOs) and punish them with jail time. There are lots of opinions about this, but I don’t see how this does not work.
It doesn't work because the people who are supposed to be leading us are on the payroll - sometimes actually - of the shareholders and CEOs, or are in fact in that same group.
The fact that there is no effective path to doing that is another conversation. My point is that doing that solves the problem. And the problem is not complicated. We just gave away our power to solve it.
Your end users are divided into multiple categories though.
AI spending is a bit like Pokemon games - most of your players are free to play, only earning you traffic and maybe some more players after recommending it to their friends - that's the average AI user. Around 5-10% of players pay a bit to the game - maybe to unlock some extra characters - those are the $20-$200 AI subscription users, with these you are not going to fund the whole business but it's some side cash. Then you have the 1℅, which spend massive amounts of money onto the game, and those are the ones primarily funding it - that's the business spending and where most of the money actually comes from.
You can only blame the average person for using the cloud service and recommending it to others.
I got sick of Pokemon's (Game Freak's, Nintendo's) exploitative practices back when they just tried to con you into buying two of every game. Anyway I don't know how bad pokemon is these days, but I think your analogy would make more sense if you thought about the AI companies like they're in a big hole in the ground. There's something down there, no doubt about it, but they don't know what it is or what to do with it, they just know that it loves munchin' down on cash. Sometimes Jensen Huang is down there and he also loves munchin' down on cash and if you don't feed him he gets mad and says he wasn't born a loser. OK, now forget about all the different users and enterprise customers because they're insignificant by orders of magnitude, a bit like how you focused on the whales in your analogy but we're zooming out further, until we can see the even bigger fish: the venture capital investors! They're the ones with the wheelbarrows of cash that they're tipping into the void.
Anyway, the only "whales" funding the AI companies is venture capital. The reason it's not a viable business (yet, so far, whatever) is because they spend more money than they make.
>End user (invididual, small business, large business) clients are ... complicit ... because they are signalling to investors that this is something people/businesses want
Erm, you appear to have missed the last 100+ years of updates to Western Capitalism.
People don't want it, you make the demand with hundreds of millions of spend on advertising (brainwashing), influencers, and social proof.
The capital holders don't sit back and see which company wins so capital can be directed to the best solutions, they buy up the competition, they pay corrupt politicians, they use current v market placement to embed their products name into billions of workplace computers and spread stories about how it's going to 'increase productivity but never at the cost of jobs', etc.
Yes, ultimately a dollar is a vote, but I don't think you can blame the people being manipulated "every bit as much" as those controlling the manipulation to feed their avarice and/or megalomania.
> Erm, you appear to have missed the last 100+ years of updates to Western Capitalism.
Not really.
> Yes, ultimately a dollar is a vote, but I don't think you can blame the people being manipulated "every bit as much" as those controlling the manipulation to feed their avarice and/or megalomania.
Oh but I do.
Collective social guilt is a thing. e.g. who is to blame for drug violence? Not only drug dealers.
This is exactly why I can't understand the AI mental health stuff about it being demotivating. I find it exciting to build apps in the same way, I don't pump out apps like a machine gun, but I do manage to make high quality apps I can maintain myself, as I've discovered. It just takes effort, which I'm willing to put in with AI. Never understood the hate around that. It's possible to have "soul" or anything that normal, "artisanal" apps have if you put in effort, and AI made it easier, at least for me, to put in effort.
You could also say the same with desktops. Desktops abstracted CLI commands like mv or ls. People who are taught on desktops will get scared when they see a terminal.
Yet programmers will still know commands.
Abstractions can make things easier but it doesn't require losing skills.
Desktops don't abstract CLI commands. Both are different interfaces on the same thing underlying. Both coreutils and Windows Explorer are "abstractions" (as you call them, but they're actually interfaces) on filesystems.
Not CLI commands but system calls. Technically, in that sense, both commands and desktops abstract system calls. It's just that desktops handle a lot more than individual commands, aiming to make it easier for the user to use the computer.
I call it an "abstraction" because that's what it is. Instead of you writing commands manually into the terminal, you use a program that calls it for you and does the hard work. That's what an abstraction is, you're abstracting away something by making an interface around it.
All interfaces, as a matter of fact, are abstractions.
Except Guitar Hero is a game, and is marketed as such. Guitar Hero is like Bitcoin mining, you are doing useless computations to make some number go up, in the case of Bitcoin, your money, and Guitar Hero, your score.
But AI actually does something unlike Guitar Hero. There is a real output that can be used tomorrow even if AI "goes away".
You say AI helps you learn no skills, essentially. I don't know about you, but AI helped me understand Linux better in a way I didn't feel confident I would be able to before AI. I have created code with AI that I can maintain, and understand without it.
AI can go in conjunction with learning manually and can be helpful. Just because many examples exist of AI being unhelpful does not mean the whole technology is unhelpful.
Guitar Hero is a "rhythm game", perhaps calling it a "guitar simulator" is a bit incorrect. A rhythm game is one where you, for example, push buttons in tune with a song, exactly what Guitar hero is.
So you are good at different things. Maybe not at a real guitar but perhaps at reaction times.
I say this as someone who had one of those plug-in guitars but never knew how to use it, hah.
It's not impossible to skew an LLM into a specific style of writing, one to reflect the author's intentions.
I think that LLMs made everything easier, as in, lowered the barrier. With that, comes a natual inscentive to be lazy, and not implement your style of writing into it. This will mean the production of more "slop" articles but also a chance for great novels with the help of LLMs. As I say, both human and AI slop exists, as well as truly great things from both. Similarly to how before AI, we had "little to no code!" promises, "make your app with no experience", as well as the endless WordPress/"PHP scripts" promising "SEO optimization" and likes from everyone.
Except all these pre LLM methods were limited and could easily be ignored; now the machine can truly get "smarter" and output new content to the point where beginners can truly become something better (or output beginner quality work at mass) with little to no work, exactly as promised before.
What strikes me about (some, or most of, as I see nowadays) Anti AI is that you never really cared about the human before AI, it was all about quality. But only now do you decide to care about the human? If you truly did, won't you have cared about the human before?
I think it might be a good idea to pause updates on our Android devices, as horrible as it is for security just to prevent this from happening. My OEM has a lot of bloat (Facebook and other junk apps you can't uninstall, useless system services, etc) that mostly do nothing except provide some functionality I don't use along with analytics every minute so they are uninstalled.
"Disabling" an app does not do that much, the OS can re enable it with an update or prevent you from disabling it.
I doubt there is true full privacy on a phone anymore with a stock ROM.
Every single reboot on my lower end Motorola phone, they have decided it fit to re-enable com.glance.lockscreenM every single boot and on every single update.
It's horrible. I don't think I can trust them to produce a Graphene phone at all, really, despite all of everyone's assurances.
At least you can disable it; on some phones (especially Chinese phones) you cannot do anything to the system services, they intercept all app requests, essentially what Google did but with their system. The only recommendation I can give you is to look into Hail (device-owner app that can "freeze" apps, a "lower level" form of disabling, see https://f-droid.org/en/packages/com.aistra.hail/) which will mean even if the app gets reinstalled on system update, the device owner app will prevent it from running.
I think GrapheneOS might have issues in the future considering that Android will be getting more and more locked down. People say GrapheneOS is the better OS when it just is hardened Android; it is doomed purely because of the Android base.
Perhaps a Linux based edition of GrapheneOS would do well, even if it will not have a lot of app compatibility. There's only so much patching you have to do to Android before it just becomes more work than good.
Despite it's name it can also block IPs and specific apps from connecting to specific servers. I use it to block all manufacturer and OEM related apps that I cannot uninstall through adb or disable. Very handy for blocking constant Chinese phone telemetry pings, as well as additional network level blocklists for ads (Pi-hole like), completely locally so you can trust it unlike a shady VPN.
I checked out your link; it seems to be a mailing list for a hosts file for telemetry? Why is there no option to just see the file without giving in my email? Not the most private.
You're right, our Telemetry Blocklist is a mailing list for a hosts file. I can offer you the list directly, no email required. Just visit https://raw.githubusercontent.com/lightinthedarksolutions/te... and copy the file to your device. This will block common telemetry servers without any external dependencies.
I love rethink but stopped using it on my up-to-date GOS. It randomly stops and no app can reach the internet (or me), randomly it doesn't obey the split tunneling rules or doesn't forward the connections to the tunnel at all (thus blocking all internet if wg proxy is enabled), and couple of other things.
I'll check Rethink again because all in all it's a fantastic software, just doesn't work very stable for me.
But this is an economy / capitalism problem, not an AI problem? AI isn't at fault that you can't get a job, your boss is for replacing you with AI.
That's like saying we should ban cars because millions of people who would upkeep horses for others to use just lost their jobs.
You're essentially making your life purposely harder because you want uninterrupted money. Perhaps look at the issues of the system that does this?
> What great things...
Aside from the medical AI stuff (detecting breast cancer, etc) that's mostly covered, AI can be helpful if you have to do a boring task that you don't want to do yourself. This is pretty much what every technology does - your phone is created so you don't have to run home and use the landline to make a phone call - it's making an already existing task easier.
>...and why would I get access to those things?
Why not? Seriously, why not? It's a bit like asking why should I get access to a car instead of keeping my horse around. You can keep your horse or car. Nobody's pressuring you to replace your horse with a car.
> Nowadays people hardly answer unknown numbers or might ignore inconvinient calls.
So your phone does make calls and it is useful but you don't want to use it nowadays NOT because the technology is useless but because of phone spammers and because phones may have potentially made it easier for phone spammers to exist?
Which, once again, is the exact point I raised with AI. You don't like it NOT because the technology is trash - but because the current capitalist system makes businesses profit less when they have to pay you, so they work hard to use technology to replace you in hopes of paying less.
Which is why I say, this is less an AI problem and more a capitalist one. If there was no benefit to replacing people with technology then people would not be replaced. But currently the current system tries to give you benefits for not having to pay people.
So even if we all fight AI and get it banned it won't solve the underlying problem which will mean that it will repeat with a different thing.
reply