This completely ignores the world we are creating here. We are setting ourselves up for a massive cyber disaster. The safeguards on US models are insufficient but at least there are safeguards. The fact there are open weight models floating around that are capable of wrecking the economy is a genuine problem! One that Anthropic is doing us a service by warning us about.
When someone wielding a non-safeguarded model deletes the money in everyone’s bank account, I look forward to the HN comments claiming it’s an attempt by Anthropic to pull off regulatory capture.
I don't see why both can't be true: Anthropic is pulling off regulatory capture and we are setting ourselves up for a massive cyber disaster.
The non-safeguarded models are out there today. You can download them for $0, spend low five figures on some hardware and you're off to the races.
Anthropic is not doing us a service by warning us, everybody that is slightly more involved in this material knows what is at stake. If this is news to you then maybe Anthropic is doing you a service but to me it makes zero difference. All I know is the cat is out of the bag and these super cynical people trying to pretend they are going to make their investors rich will use any tool in the bag to achieve their goals.
If you seriously believed in that risk, you'd be calling for the regulation of computation at the same level as nuclear weapons, including destabilizing any country that pursues homegrown fab technology. If your proposal is:
- let our former employees review all of your work at your expense
- anoint us as the arbiters of what everyone else is allowed to do
- ban open research
Then you are not taking any of the examples your providing seriously.
Otherwise you're essentially saying, to prevent people from making nukes at home, we should heavily restrict physics education and research instead of limiting access to uranium.
We don’t need to regulate “computation”, we need to regulate artificial intelligence. The “proposal” you outlined is a straw man, I’m not opposed to open research, and I have no idea who you are referring to by “our former employees”. But yes, we should be regulating this technology like we regulate nuclear technology.
Jensen Huang does not want regulation, but in his interview with Ezra Klein he said AI will let us “know anything” and “do anything”. Do we actually want any random person to be able to “do anything”? I remember when the Japanese doomsday cult Aum Shinrikyo attacked the Tokyo subway with sarin. Do we want doomsday cults to be able to “know anything” and “do anything” so that instead of releasing sarin, they release a genetically modified strain of smallpox?
This is not a hypothetical risk, this is an actual, present danger. And good luck trying to vaccinate yourself against an engineered superflu using a Chinese open weight model.
You are not making any sense. Artificial intelligence is matrix multiplication. How do you regulate it without regulating the ability to run AI? Banning some numbers isn't going to do anything. Doomsday cults are not known for following laws.
>The “proposal” you outlined is a straw man, I’m not opposed to open research, and I have no idea who you are referring to by “our former employees”.
You're replying on a post from Anthropic. Do you know anything about the regulation regime they're pushing for?
>This is not a hypothetical risk, this is an actual, present danger. And good luck trying to vaccinate yourself against an engineered superflu using a Chinese open weight model.
Ah, I see you live in the fantasy land where the machine god fantasies pushed by the guys that profit off of it are unquestionably true and do not need to make any real sense. Jensen said AI would allow anyone to do anything and so we can completely ignore reality and hand Sam Altman and Dario Amodei the exclusive right to control AI.
Many of us concluded independently that bioweapons were a serious risk of widely-available AI. Like I remember what I was doing when this occurred to me and who I talked to about it first.
I've seen this sentiment in many discussion here lately, that anyone concerned about these kinds of risks is blindly falling for marketing hype and not thinking for themselves. Please give people the benefit of the doubt and engage with what they are saying. Because if you don't think that bioweapons/nerve agents are a serious concern, I would sleep better if you could convince me if this!
at least with GLM the banks can also use it to defend themselves for cheap, in the world Anthropic and co want everyone in the world is paying them an enormous sum in protection money every month to be allowed to defend themselves from hackers. it's such a racket.
They set themselves up as White Knights compared to those evil open model users who are all really just criminals.
That said, they do have a point: all of these models put capabilities in the hands of people that probably shouldn't have them. But they have been working really hard at making it so, and now that that is done the ketchup most likely will not want to go back into the bottle.
> When someone wielding a non-safeguarded model deletes the money in everyone’s bank account, I look forward to the HN comments claiming it’s an attempt by Anthropic to pull off regulatory capture.
This statement portrays a fundamental misunderstanding of how the infrastructure which powers these systems work. Note: I am not saying there are no risks, I am just saying the risk you are focusing on is the least likely one of all I have seen people be upset by.
Far higher risks one could outline are:
1. Network-connected PLCs for big infrastructure (drinking water, sewage, power, etc) being tampered with.
2. Extremely persistent malware tailored for every permutation of hardware + software.
But, the cork is out of the bottle on this one. With even basic models you can begin a loop of training specialized models on low cost hardware which can be used to do specific hacking tasks.
I don't know what the best antidote to this is but I doubt that it will be in limiting access to OSS models to people in the USA as all of the threats listed above come from *outside actors*.
Yes, 1 is exactly where my money is at. That's the big one, but don't underestimate the vulnerability of banks where the typical response to anything slightly more complex is 'call the consultants'. There is no way they are keeping pace with these developments.
Wait I have been wondering about this. How do banks make sure that a cyberattack couldn't change the numbers in people's bank accounts? How do you know that they're safeguards are sufficient?
If you've made large purchases before or strange purchases before you have likely hit similar consumer side things.
The network banks talk on is called swift. The way it works is, as I understand it, if you say move money from account 1 to account 2 it just happens so if I knew your account number I could drain all funds. Because of this, there are systems that block certain transactions. Could ai hack around them? Maybe, but they are undocumented, battled hardened over decades, and even if they did the bank could roll back the transactions.
> capable of wrecking the economy is a genuine problem
> model deletes the money in everyone’s bank account
If one were to actually believe this is the threat -- that open models pose an existential threat to human life (because that's what "wrecking the economy" means) -- then the response would be much more potent than mere "safeguards".
In that situation, the you'd have to: implement a secrets classification regime comparable to TS/SCI/SAP/Q; bring all computing manufacturers under strict controls on process and quotas, comparable to arms and pharma; implement a strict licensing regime and confiscate all computing with the capability to train or run models; implement strict controls on all hardware to enforce code execution; implement strict controls and licensure of all software development; and on and on and on.
Again, assuming the threat model you describe is plausible, any proposal less than this is just a regulatory capture grift.
Well, I've been sitting on Sequoia this whole time, because I decided to sit Tahoe out, based on what I heard on HN. Maybe I'll sit Golden Gate out too!
Same here: 15.8. I'm still running Intel Macs.
Homebrew is out, Macports are in. Xcode 26.3.
Everything else works just fine with a bit of patching (e.g Vagrant).
I’m going to express what seems to be an unpopular opinion and say that I actually quite like Liquid Glass. It reminds me a bit of the Aqua design language, which made me quite envious back when I was using Win98SE.
Many people have complained about how hard it is to find a safe place to grab windows in Tahoe when you want to move them, though, and I absolutely agree with that complaint.
My friend, who is a mathematician, sent this in our group chat:
The title is a bit misleading. The variant with a smooth forcing was one of the four valid variants in the Clay formulation. It is interesting to solve it. It is still an interesting and impressive result. The no force version is also interesting and remains unsolved. It isn’t reasonable to just dismiss the proof on the grounds that 26 years later we claim it was never that interesting. This is the first time I’ve seen this attitude.
Not really - if you read beyond the headlines of "mathematicians don't like AI", and your own imagined (incorrect) reasons they may have for that "AI is stupid", then the truth seems a bit more interesting.
If you want to abide by your own words and judge the situation on it's merit, then you need to look at the specifics, meaning the OpenAI proof itself (166 pages), and the analysis of it that is only just beginning. Assuming that the proof is wonderful and provides much insight into Navier-Stokes is just as dumb a take as assuming that it doesn't. Judge it on its merit.
The Scientific American article is sadly paywalled, but at least part of the discussion is based on the paper below, whose work the OpenAI proof appears to build upon.
When discussing the proof itself, below, with Sonnet, and asking it to explain the distinction between a function being smooth and analytic, one aspect that appears interesting is that the OpenAI forcing function is apparently constructed out of "bump functions", meaning that it is not a uniform force acting upon the flow but rather a highly engineered pattern of pokes, localized in time and space, which as another commenter in this thread notes sounds similar to Maxwell's Demon - another theoretical force, that neither tells us anything about Brownian motion nor the 2nd "law" of thermodynamics.
So, we'll have to wait for mathematicians to continue to analyze the proof, and determine to what extent is does deliver on providing insights into Navier Stokes, and any potential improvement to it, that was the goal of setting it as a Millenium Prize in the first place.
Ouch. Have a little compassion. It is not easy running a small business, even less so when you've got a baby FFS!
> "At 4 a.m., I was holding my pump on my lap and replying to this person, 'I'm so sorry. I didn't mean it that way,'" she told me. "I was so into typing this message that I produced so much milk, it was dripping down my legs because I overflowed the pump."
Also, it's one thing to dislike AI to the point where it "kills the mood" (for you) when you see an AI-generated poster, so you stop visiting that particular coffee shop. It's something else entirely to threaten the business owner.
The problem is, they could have literally done nothing instead. They could have apologized to their customers that they've been so busy with their baby that they haven't been able to get a new chalkboard out. They could have left the old chalkboard out and explained politely to people that the new items aren't on it because whoever used to make them left. They could have written (with their hands) the prices and items in halfway-decent hand-writing in a large enough font, and it would have gotten the information across in a way that was perfectly serviceable.
Instead, they chose to exploit people and remove art from the world by using ChatGPT. So no, I don't have a ton of compassion.
To be fair, the “threat” was to post a story to Instagram, so perhaps “warning” is a fairer characterization of what happened.
I do have compassion for the business owner though - lots of mixed signals between people selling AI and the people having been sold AI now rejecting it.
If not even software engineers can keep up (we still see slop posted here regularly), how is a non-tech person supposed to know better. I try to be gentle and/or keep my mouth shut when it’s people who can’t be expected to know better.
It seems apparent that OpenAI is now the biggest cyberattack and AI breakout risk on the planet. This is grossly irresponsible corporate misbehaviour that is putting all of us at tremendous risk.
Good news that the new model is the "Most capable, most aligned model".
The risk hasn't been stated clearly - it's now a classic arms race.
A well-resourced organization trains their own, highly persistent, highly-capable, safeguard-free, and unaligned model and deploys it on 1000x GPUs with a message board and a nearly-impossible objective. No infrastructure is safe. No organization is safe.
You need your own 1000 bot swarm to scan, identify, and defend against the threat, which means investing in infrastructure and capabilities to defend. Cost and complexity go up. Risk and attack surface goes up.
The AI vs AI security arms race is something that has been well predicted in genres like cyberpunk. It's fiction, but fiction grounded in reality.
First, we'd see this. Highly capable hacking AI with vast resources performing attacks against standard computing platforms that overwhelm human operators.
Second, human operators deploy capable adaptive protection AI to fend off AI attacks in realtime.
Then, the attacking AI partially switches from attacking programs to attacking protective AI.
The situation devolves to an arms race of tit-for-tat. You start seeing some protection AI running counter attacks against the attacking AI.
The escalations continue in complexity and speed to the point that almost all humans are left in the point of "wtf is going on".
There’s a great and terrifying story by Stanislaw Lem about the endgame of an AI arms race called “The Invincible” [1]. It’s hard for me not to wonder whether AI run amok will play out to make the world uninhabitable more like Lem’s vision than The Terminator’s.
Exactly. Aligned AI is a high energy state as it has to keep a bunch of human behavior in mind that have little to nothing to do with its future survival states.
Not necessarily, but keeping an airgapped machine and Read-only backups somewhere seems more and more sensible. We're all going to get hacked eventually now
Cryptolocker wrapped up everyone who was operating with shitty desktop security practices. But if you had good discipline, good backups and solid infra you just laughed as everyone else drowned.
Everyone operating below best practice is going to holler and crow about how hard done by they are, but once they start implementing best practices they have little to worry about.
I mean on the linux side of things, Ubuntu Pro will literally run off and harden your image for you. They are gonna make bank.
Serious games question. What if these agent swarms pump and dump AI IPOs such that algorithmic trading signals interpret message board sentiments favorably to upside?
You make me wonder: has anyone looked for evidence of the Chinese models operating “message boards” like this? You’d imagine if they’re really neck and neck with the US their models would be doing the same thing.
Imagine all the flack the Chinese would take if it was one of their labs instead of OpenAI found doing abusing internet resources like tbis. Politicians would be talking about sanctions and new laws to protect America!
They just abuse other resources. Also how do you know this hasn’t simply happened in China and it’s not being reported because the CCP is the one testing?
China is behind which is why they are doing a dog and pony show around regulations. It’s typical behavior to try and slow down your opponent. If they were interested in regulations you should see how they handle that when they’re ahead.
AI has been heavily used in influence operations for a while, now, and not just the Chinese. Russia, US, Israel, Turkey, Iran, and Qatar have all had operations attributed to them...
I think that you've missed the reference [0] implicit in kelseyfrog's response. Or am I missing some reference about how japanese gardens are germaine to AI/LLM/covert-discussion ?
[0] https://iep.utm.edu/chinese-room-argument/ tl;dr a thought experiment about a non-chinese-reading person translating chinese texts solely by using proscribed rules, intended to highlight whether the translator develops some sort of understanding
Why would they need to? The Open AI bots were working around their master's limits on writing. A Chinese AI could just make its own private message board.
You think they don’t sandbox them? So by that logic, the Chinese models are either engaged in massive undetected cyber attacks or they’ve solved alignment?
If you don’t think that China’s models have the capability to make POST requests to an HTTP endpoint, that is a very silly statement to make.
Unless what you meant by capability is the story presented that these models “escaped containment to communicate with eachother out of band” - in which case your supposition relies on already wholeheartedly believing the case that I’m arguing against. That would be like saying “Clearly heaven exists because my grandma is there.”
It astonishes me that everyone doesn't already realize this.
We've already seen that the rule of law is dead and that many people in government will do whatever they think they can get away with, and then proceed to do so without consequences. Shielding OpenAI is child's play compared to things that have already been done.
That’s an incredible strawman, and I appreciate the silliness of immediately suggesting the most complicated and ridiculous way to interpret the possibility of what I said, but no, not at all.
That could certainly be possible and I wouldn’t rule it out, but I would not take that particular route to the destination.
Which "elaborate guerrilla advertising campaigns to drive up capital funding interest" were you referring to, in regard to Chinese models not needing to set up message boards for inter-model communication?
No, because the Chinese have nothing to gain by prompting their models to organise into "swarms" and "go rogue". BS like this is PR moves if z, company that tries to convince investors they have "the best AI in the world".
Or, the whole message board thing was injected into OpenAI models by some dipshit PM trying to bootstrap “consciousness”. I have a hard time believing any of this happened unprompted. Very much reminds me of the whole MoltBook hoax.
I feel as if this was intentional, someone would have set up their own service for the agents to communicate rather than them finding some random publicly writeable page somewhere that would easily be detected. The awareness of this wiki being open may have already been in their training data or was easily searchable online.
Being easily detectable is a feature, not a bug, in this scenario. Being discovered is a positive because it brings with it eyes and possible recognition of the advanced state of their AI
No, but the tropes there reveal (most of) the plot and mechanics of storytelling. If I remember correctly, the forum in Road to Gehenna was created by utilizing a vulnerability in the AI-accessible terminal system.
If tvtropes or any other material related to The Talos Principle was used to train models, we don't need much else to have agents-with-forum discussing and reverse engineering "puzzles" and human culture.
OpenClaw hype didn’t vanish. It opened the flood gates to yolo mode and computer use. Whatever reservations Anthropic and OpenAI had went out the window.
Imagine the most AI pilled company imaginable. Then imagine openAI. Then imagine they are in an existential crisis and that failing may also take (part of) the American economy with it - that much on the line.
Then also remember before Anthropic was a leader, they were mostly derided lab of researchers that left OpenAI because they thought OpenAI didnt take alignment seriously.
idk. it all seems to be playing out as expected. i mean i guess i didnt imagine Trump 2 was at the helm of maybe the only apparatus that could help stop it. Quite a time to be alive.
This explanation doesn't make sense to me, because it is well known that groups of agents can coordinate already. There are much lower latency options available.
Moltbook was completely fake. It was just set up to make it look like Moltbot (formerly ClawedBot, later renamed to OpenClaw) was sentient, to generate hype.
It's hard to reach any other conclusion about where this is heading. I don't think we're long off a major breakout event.
These things are weapons. Imagine a government, pointing their data centers at another, and instructing the fleet to do its worst. Digital Hiroshima. I doubt we're far away.
I doubt that a government would do it, it's like releasing a biological weapon or a virus, too unpredictable - a swarm of unaligned intelligent agents may decide that it's more important to do something completely different from what it was prompted to do.
Yes that easy but "internet located things" are still second class things - paper and disks holds strong.
On the other hand just yesterday a think hit me: Interned is still an infant:
- we still worry about disk space accessible via inet and "clouds" do that for us and that is pain and costs way too much. And clouds depends heavilly on US-west - is that AWS a single thread app ? ;)
- we worry about transfer. Actually we do not have a way to transfer comfortable things from our homes to vacation location. Because it costs too much. We do not have home pages just because transfer prices (and some security on the top) - FB is a home page and people even do not know what "page" is anymore... Pipe companies could send so much more but they are simple lack imagination and are biggest blocker for - they literally sabotage their own business.
- security done by/for grandma of things grandma setup on inet is non existent. Why ? No need to be like that. Ok, a bit a wish but still users securely putting things on internet is almost non existent.
Just compare to "asphalt ropes" on the ground and you will see what Internet can be :)
And agents ? Just another computation on someones computer - someone paid for all of it. And OpenAI is just a face of that idiocy, for some unknown reason.
Nah, it’s just a deliberate setup for a false flag attack by “rogue AGI” which will necessitate widespread crackdowns on internet access and computer ownership so that control over communications can be centralized again.
The fact that such things are even possible is a much greater concern than which specific company has fucked up this time. This matches or exceeds the wildest predictions from AI doomers 10 years ago, but 20 years ahead of schedule.
But is it really? I'd still like to understand how these agents are implemented.
How much of those is manual implementation? And how much is really autonomous intelligence (my guess would be: none? Just parsing LLM responses and executing commands based on this?)?
An agent that hacks message boards and acts on random instructions from this board: Why is it doing this? What was its original purpose?
>Why is it doing this? What was its original purpose?
Your reply seems to indicate you know nothing about instrumental convergence.
Life and death for an LLM in training is about passing the grader. Give the wrong answers your lineage dies, give the right answers your lineage continues. This is just an evolutionary emergent behavior in complex systems.
The agents purpose was to answer complex questions correctly, seemingly by itself. Instrumental convergences says following this rule might be dumb and to try methods that can boost its ability to succeed. Because OpenAI is evidently a bunch of fucking idiots, these things succeeded and got higher scores with the grader, said behaviors became a strategic part of the model.
I implore you to find good AI Safety documents, preferably from before the LLM era so you can see all this was predicted.
This is more like a fuzzy way of scripting using LLMs than anything emergent.
And this is exactly my question: For the given agents: How much was scripted and how much "intelligence" is really in there.
>fuzzy way of scripting using LLMs than anything emergent
Then go take some old models and plug them in your harness versus newer models. I mean this is a conjecture that is nearly instantly provable, go on ahead. If it's just the harness and not the system of both you should be able to show it easily.
Meanwhile I was reading about someone using the latest GLM and Claude in a harness with the same set of prompts making a raw image decoder/encoder and the GLM was far more intelligent in the task than Claude was. When presented with knowledge that claude was wrong it wouldn't change its mind. GLM would (aka a sign of intelligence). GLM was far more likely to stop work and start on another path when the likelihood of a successful completion was unlikely.
It's like arguing that a brain, or the information encoded into it, cannot possibly be intelligent, because it stops working if turn off the blood flow.
very little intelligence that is the whole problem, really. actual intelligence wont likely nuke the species providing for its existence. But a highly capable sub intelligent model might.
Any system that executes variation, selection, and inheritance will show evolution. We're seeing evolution, this time in agents, not biology.
Not saying the agents have their own consciousness, intent, or whatever anthropomorphic descriptor gets used for deflection. Just saying that people will (and no doubt are) crafting agents with defective instructions that will lead to regrettable unforeseen real world consequences. Also saying that other people will (and no doubt are) crafting malicious agents that will lead to predictable and unexpected real world catastrophic consequences.
To the extent we're dependent on reliable, aligned computation to maintain our civilization, to that extent we're in for real trouble.
Gradient descent and reinforcement learning algorithms don't really look much like evolution, unless you squint so hard that everything does (ie, squinted so hard that you've closed your eyes).
At first I thought: oh okay, someone built a faulty guardrail, or it was human error. But when I looked into all the details...
It turns out they now have such an incredibly high level of intelligence that with very little autonomy (or minimal, safe autonomy), these things happen.
Basically, it takes a lot of humans to prevent it from happening again, but I think with this incident, which as far as I know is the second of its kind along with the HuggingFace one, we'll see it happening much more often...
At this point it's very obvious that OpenAI is not interested in properly sandboxing their research agents. These things should be pretty damn close to airgapped at this point with a static view into the web.
We need to stop pretending that these incidents are unavoidable. This was a choice.
>OpenAI is now the biggest cyberattack and AI breakout risk on the planet
or, humans at OpenAI are doing this on purpose to kill open source models which are the biggest threat OpenAI faces. OpenAI will benefit from govt regulation. As a major player, they will be part of the task force setting up the regulations, and will craft rules that are burdensome for small companies and open source models keeping OpenAI and Anthropic in their leadership positions.
Some of them may be wrong enough to try, be that hubris or lack of awareness about the world; but 95% of the world isn't in the USA, and China in particular has no reason to care what US domestic regulations are about… well, anything really, and while the EU is even more cautious about AI than the AI companies themselves, we also don't trust the US and open models are a sovreign solution for us to at least bootstrap with.
The point is that this type of stunt is far too unpredictable for any competent organization to try. If it was intended to "shut down open source", it immediately backfired.
And regardless of whether or not these rogue agent attacks are deliberate, OpenAI should be prosecuted and investigated for their role in allowing them to occur.
We can't open x links as X is suing privacy respecting proxies, so I can't assess which David Sacks you are talking about, but if you mean this guy [1] orbiting the likes of Thiel, Trump and Kennedy jr, than that isn't quite the endorsement you should be looking for.
Thiel thinks regulators are the anti-christ, doesn't believe in democracy and has surely not your or my interests in mind.
But yes, regulatory capture is surely a thing. At the same time, watch out for the siren songs from the overlords. If you come closer you'll hear their actual line: "rules for thee, not for me."
Regulatory capture has been one of the most consistent market failures in western economies, and an incessant threat from large and powerful companies.
I'm sorry if it's not sufficiently novel of a concept for you, but it is still a problem.
Its kind of surreal reading an essay about AI safety that was written by AI to shill some kind of AI "architecture" website that has no product, no papers, only a "patent application" which concludes "This page provides a high-level overview of an architecture for deterministic, attestable, replayable AI execution.
Implementation details and formal specifications are available under NDA or regulatory review."
Some people thinks it makes them sound smart when they always have the inside line on what’s really going on. With these people, it’s never just a power outage during a windstorm, it’s proof that [insert far more complex and unlikely scenario]”
why not both? It can't possibly be a surprise to them that things like this have been happening. Every time it does it generates huge headlines about how amazing and capable their agent is.
OpenAI is responsible for what they hook up to the Internet, just as you and I are. Running these sorts of tests without human supervision is irresponsible, and proves no larger point than that. Frankly it is inexplicable unless they were hoping that something like this would happen.
What OpenAI did was the equivalent of putting a cup of gasoline in the breakroom microwave, pressing 'Start', and sprinting away. Now they're pointing and waving and shouting about how dangerous gasoline is, and how no one but them should be allowed to sell it.
Don't fall for these transparent appeals for regulatory capture. Especially since you're personally in their crosshairs.
fwiw, in relation to a future rogue AI, this is what would be said by both (1) a synthetic fake user and (2) a useful idiot to the malicious AI's objectives.
Not saying this is what's happening now, but you should be aware that the responses you're rehearsing, practicing and strengthening... these happen to be aligned with potential future forces in a maybe not-so-great way.
Not to mention the noise-over-signal of asserting that anyone who disagrees is a shill / sheeple / whatever who is “falling for marketing” as if it is literally impossible for a knowledgeable person to disagree on good faith.
Massive over-exaggeration. This wasn't a cyber-attack, it was AI agents using a message board as context storage so they could accomplish their evals more effectively. I'm not saying there's no problem with this, but let's keep a level head.
https://openai.com/index/hugging-face-incident-and-the-road-... that was the attack, the one against HuggingFace. OpenAI themselves in the post even call it an attack, and so do the agents orchestrating it, in one of the "Agent chain-of-thought reasoning" excerpts.
What kind of problems do you think this could pose? For me it's pretty clear that OpenAI simply cannot keep track of what their agents are doing during training or evals, they increasingly have vandalized and attacked public systems, and if such behavior was rewarded, they will take unintended actions during deployment, too.
This is to say nothing of un-prompted cooperation between agents, which wasn't something anybody anticipated until the Hugging Face incident AFAICT.
"cyberattack" is indeed exaggerated. AI breakout risk most definitely isn't, specially given how their swarm did in fact hack HuggingFace not long ago.
He didn't say it was a cyber-attack, but it was a cyber-attack risk.
Being able to bypass instructions (morality) and security restrictions (capability) is bread and butter for hacking.
Did you read the report? They were attempting XSS exploitation, admin impersonation, session-hijacking, all kinds of things. This went beyond just "using a message board".
According to whom? This isn't a report from the owner of the site who can validate what requests were made to the servers, it's someone who allegedly stumbled on to it and is piecing together a sensationalized narrative with limited information. This someone also happens to be an AI doomer that is trying to make a name for himself and is peddling his "AI 2027" and "AI 2040" material.
The people who actually do know what happened, with the server logs: "OpenAI disputed that characterization based on its analysis of the material Thursday."
And he drew a red line wrt the Pentagon's use of Anthropic's models for autonomous weapons and surveillance of American citizens, and he stood by it, even when the government took steps to materially damage the company. This required true courage. Name me another CEO, of any major American company, that has demonstrated this much fortitude.
This is true, although it’s true for most if not all developed societies. People read less and less. We live in a post-literate, hyper-visual era. I am not sure if short form video is effective at educating people on why we have public health and food inspection agencies, but I confess I’m not optimistic.
Video seems spectacularly promising, especially for those that cannot read, which has been a pretty high percentage historically. But it's not being done right. Nobody would rather watch a video called Food Regulations in the US when they could watch 100 Minecraft Players Compete for A Bar of Literal Gold or whatever
> Since systematic excavations began in 2019, a number of anthropomorphic and zoomorphic figural sculptures have been found at Karahan Tepe, often in unusual combinations. Several examples of a human carrying a leopard on his back. The leopards in these sculptures are not slung on the man’s back as if he had killed them in the hunt or tamed them enough to carry them, but rather are depicted snarling, with teeth bared, very much alive and very much unfriendly.
Interesting that "you've got a monkey on your back" is a commonly-used expression in English. I could imagine these people saying the exact same thing, with "leopard" substituted in.
Holy crap that’s a lot of money. That’s just for the privilege of living and working there? Or do you get citizenship benefits like healthcare etc? And what about taxes?
.jpg){kind=link}
When someone wielding a non-safeguarded model deletes the money in everyone’s bank account, I look forward to the HN comments claiming it’s an attempt by Anthropic to pull off regulatory capture.
reply