Slaughterbots was a great way of illustrating the issue, but I think Daniel Saurez's "Kill Decision" talk is a lot more useful and more accurately predicts where we are today in issues of autonomous warfare and mitigation, despite being even older: https://www.youtube.com/watch?v=pMYYx_im5QI
He recommends an international framework where no human kill decision is left to be made autonomously (we're well past this stage), all robots announcing their presence and identification at all times where all citizens can track them, drones that patrol other drones not adhering to these requirements. That's where discussions should have started. Historically, it seems weapons have to be abused first to demonstrate how terrible they are, before treaties are enacted
> all robots announcing their presence and identification at all times where all citizens can track them, drones that patrol other drones not adhering to these requirements
Sorry what, we're going to have drones patrolling checks notes the Arabian Peninsula and also all of Eastern Ukraine, Crimea and Western Russia to make sure all sides' drones there are constantly squawking that they're on a bombing run? Do air-defense missiles and drones have the same obligation?
This sounds like a global no-fly zone. It's unclear the U.S. military can even enforce one over Iran.
> Sorry what, we're going to have drones patrolling checks notes the Arabian Peninsula and also all of Eastern Ukraine, Crimea and Western Russia to make sure all sides' drones there are constantly squawking that they're on a bombing run?
As opposed to drones not announcing themselves, acting with impunity? Yes, I'd rather take the treaty that enforces this. You cannot simply "ban drones" or "ban robots" today, which is the point Daniel makes. You can only mitigate them
I'm not sure where you got the idea about this creating a "global no-fly zone". I think your argument is essentially "patrolling robots effectively results in no fly zone" but that's not what Daniel is saying. Mitigating robots != mitigating all things that fly. Not all robots are flying either. I'd encourage you to watch the talk if you haven't already
From what I can see, this video from 13 years ago has almost completely predicted where we are today, which is terrifying. Perhaps I believe perfect is the enemy of good and its better to unwind from somewhere. I do really like the idea of robot makers being forced to add publicly-visible identification signals into their robots as a start. It's not perfect, of course
I appreciate you watching the video and offering your counter perspective
> this video from 13 years ago has almost completely predicted where we are today, which is terrifying
I'm not loving his history of war and politics mapping feudal economies to armor and castles being broken by scale armies, which in turn enabled nation-states. We had massive armies before: Rome fielded them. Before them, Alexander the Great and phalanxes built scaled armies. The same breaking of armor, in fact, happened with chariots and long spears in the Bronze Age.
As for the threat to democracy, it seems professional armies allow leaders to separate themselves and their populations from the gory effects of war almost as effectively as AWS. Suarez also claims AWS will let leaders decouple population from power (a real concern, I think). But then he says it will permit anonymous war, which also means anonymous rebellion, which undercuts that first risk.
On AWS enabling offense over defense, meanwhile, we've seen that's currently dead wrong. The defender's advantage has grown. Not because air defenses are better. But because mass manufacturing cheap short-range munitions allows one to make their own territory lethal much faster than someone far away can bring their lethality to you.
In summary, I think Suarez offers a prescient and compelling summary of leading thought on AWS thirteen years ago. But he misses some marks. Internally contradicts at times. And as such, I'm not convinced in his proposed solution.
It does, however, contain a kernel of wisdom. Perhaps I could expand on it. Maybe what we need is open-source intelligence from space-based platforms that monitor, real time, the global airspace and ground for likely autonomous activity. What I'm proposing is a panopticon, but no more than Suarez's. It is, however, economically possible. And it could be enacted unilaterally. Knowing who is fielding what in what capacity will let us draw more thoughtful lines. It's not like we don't have a bunch of conflicts brewing as laboratories anyway.
> Overwhelming bipartisan majorities favor prohibiting stock-trading in individual companies by Members of Congress (86%, Republicans 87%, Democrats 88%, independents 81%), as well as the President, Vice President, and Supreme Court Justices (87%, Republicans 87%, Democrats 90%, independents 82%) according to an in-depth survey by the Program for Public Consultation at the University of Maryland’s School of Public Policy.
Yet there is barely any movement to do anything about it. A pretty blatant failure of democracy in the US
It’s the opposite for me and my family, but more so because of safety. I bought my mom a Starlink mini and portable battery because she does a lot of road trips and camping in remote places. Many times there’s no cell reception. If she needed to look anything up or communicate with anyone in those places, there are limited options.
Yes there’s satphones, but then might as well have the entire internet for the same price or less. The $10/mo Starlink standby plan seems well worth it for insurance
> This is expected these days for threads which may make the provider liable. [...]
> They're being retained for legal purposes.
It’s a significantly wider usage than that:
A subset of chats are reviewed by human reviewers (including Google’s trained service providers) to help improve Google services. These include Gemini models, other generative AI models that power Gemini Apps, and technologies that help reduce unintended activations.
Reviewers assess if Gemini Apps’ responses were low-quality, inaccurate, or harmful. They also suggest better responses. We use their suggestions to help our models respond better in the future.
That’s in addition to the liability language:
Chats are also reviewed to keep Gemini safe and secure for everyone and to address violations of the Google Terms of Service, including the Generative AI Prohibited Use Policy. Reviewed chats are retained for up to three years.
It's not weird at all, it's pointing out most free speech absolutists just mean speech for themselves and their friends. Musk will be silent about this.
That makes no sense. Free speech is clearly defined, say what you want and the government has no authority to censor you. Anything short of that isn't free speech, and creating some middle ground between "free speech" and "absolutism" is nothing more than an attempt to move the goal posts.
What most people seem to miss is that companies like Facebook aren't the government and therefore aren't held to the standard of free speech. Signal ng up for an account and agreeing to their terms gives them authority to censor.
"Free speech absolutist" was a strawman used against anybody who spoke out against deplatforming as a dangerous idea, which included a lot of liberals. (The pro-deplatforming crowd are very much not liberals.)
Watching the pro-deplatformers get burned by their own illiberalism is very satisfying; watching some of them not having learned any lesson from it, doubly so.
Instead of enacting laws to regulate all of AI, perhaps the government can just pass laws scoped directly to OpenAI and Anthropic. After all, they’re the ones asking for regulations and they’re the problem actors right now. These “rogue agents” would stop pretty quickly
It's Dropbox vs rsync. The agent providers are making it stupidly easy to use something like OpenClaw/Hermes with zero set up and a very low learning curve. Also, in their technotopia, you don't use your computer to host the agent, you use theirs. When you need to drive, they give you screen sharing access, but its still on their servers. The benefit to the user is they don't have to make an upfront expensive payment for computer hardware anymore, and the agent provider gets to own all of your compute and data in their cloud
Muse, Dots, and other always-on agents may be the end of the PC era. Once you’re asking agents to do things on their own virtual machines, it’s game over. Everything moves to the cloud
Anyone wondering "why would I use this when I can use (OpenClaw|Hermes|my own computer)" -- these new services are not really meant for you. They're meant for the non-tech savvy and for the next generation of AI-natives who won't know anything other than how to use these type of services
For your casual user, these services will be hard to beat, since they’ll handle all the expensive and hard parts of using computers. No computer purchase necessary, no troubleshooting with tech support, etc. They’re also scalable where you could have not just one agent with one computer at any time, but many
In return, the agent providers would own your compute and data. I can even see them offering this low cost or for free so they can train off of users. Lock in would be insane
For privacy reasons, I really hope we find equally useful, private alternatives on our own hardware
Computers arent actually intuitive at all.
Even a mouse and keyboard literally take weeks of practice to learn.
Imagine picking up a new product that will take you weeks of practice to use at basic level. And then you have to learn how to do every little thing in the OS.
Via what interface would people use the agents? Voice to text on a smartphone? That's clunky and slow and only works in quiet areas where you can talk freely. And mouse and keyboard is really not that hard. If anything was going to kill PCs, smartphones & tablets would have done it, but everyone I know still has at least a laptop. Overall I think getting people to trade in the freedom of having their own personalized devices for a remote agent is a much harder sell then you're making it out to be.
Smartphone. Think of people in your life who aren't tech workers. When do they use a computer if they don't use one for work? Almost never and when they do it's for specific tasks like create this document, fill in this form, research this destination, book this trip.
There are a lot of people I can think of who would be happier just describing this task to a dot or some other agent rather than digging out a laptop, booting it up, fighting with Windows update etc etc
For those people this could absolutely replace the need for a computer.
I was already thinking of the people in my life who aren't tech workers. They have at least a laptop because it does exactly what you want, you don't have to talk to it (which again, is clunky and only works in a quiet place, which is why adoption of existing voice to text features is already spotty), it has a sizeable screen (video streaming, photo editing, gaming, lots more things), it doesn't have a subscription fee, and it doesn't have to be jailbroken if what you want to do is deemed harmful or otherwise unsupported by the platform. It's (mostly) free to use as you wish. That's the P in PC, it's yours and people value that.
That's not to say that agents won't be a popular product, but as I see it there's a near 0 chance that they kill PCs.
> Imagine picking up a new product that will take you weeks of practice to use at basic level
This does not work on developers. Their empathy fails them here - because they absolutely would try a new product that takes weeks of practise. They like (or at least tolerate) learning complex+cool new things, thats why they are developers...
For advanced users, always-on agents that run on your computer where all of your files/apps already are are much more powerful. I think the big AI co's skip this bc it's a smaller market and not as casual
I'm working on an open version that runs agents on your machines and brings a polished UX, and good parallel divide-and-conquer coordination
Very similar in principle, but I'm aiming for something very different in UX - namely, a polished client app built for creating and messaging with an agent that coordinates work across many subagents
I find Telegram, Messages and the like to lack the fidelity I'd want for managing such stuff. And the official OpenClaw app is ... well, it's weird that they shipped that
The way it looks to me, the set of people who know what dots/muse actually do and why the are useful, and the set of people who don't use computers, don't overlap much.
I don't think you'll be able to make most non-tech savvy general population understand what those services do that a regular chat window does not.
I think the writing is on the wall for operating systems and apps as we know them today. I think there will still be a use case for local files storage and processing, however. So phones and computers won't go extinct, but our relationship to them will change significantly. The interface layer will become an AI, which will seamlessly transition from local to cloud based on compute needs. I give it less than a year before OpenAI or Anthropic announce their own "OS".
I'm working on a project that solves some of this [1]. It runs agents in Docker and proxies the ACP connection via websocket, with WASM-based plugins in that proxy so you can get in between your client and the agent if you want.
It currently tears down the container after the session, but it wouldn't take much to leave it running post-connection and make a mode that continually re-uses the same container.
It's also possible to intercept ACP read/write file and shell commands via one of the WASM-based plugins if you wanted to execute them in a separate VM/container. I'd have to double check on FS permissions for the Docker socket; I think plugins have no file access currently because I haven't figured out a permission system for it yet. The whole plugin system is new and I'm still working out some of the edges.
In your mind why do people use computers? The universe is filled with non-technical people who not only have laptops but even go out of their way to buy docks and huge screens and ergonomic keyboards etc
Now, lots of those people have all the data "in the cloud" (a trend that's been around for a very long time), but the actual mechanism for interacting with things exists, right? People want a large screen just to read things.
At one point all the AI churning is supposed to lead to _some_ output for _someone_ right?
Software is going to replace hardware. All that will be left is glowing outlines in the air. I've seen it on Instagram so it must be true. That's made possible by AI, as I recall.
Of course Dots will be the end of the PC era. Every innovation of the last twenty years was the end of the PC era and dots are just as good as those.
I'll jump in here and mention Moose OS.
It's a debian based open source OS that allows you to run these agents, and whatever OSS container you want.
You can start with the hosted version and move to your own box at any time.
I think lock in in is the thing we should try to avoid from now on, especially when it comes to our personal data.
Products like muse and dots are attractive when they have a free tier but someone always needs to pay the bill.
Windows had linux, IE had firefox and iOS had Android. I'm sure we'll see a relatively open assistant harness that token providers will rally around (at some point once the burgeoning field of assistants settles a bit, with clear use cases and value propositions)
This makes me surprisingly excited for the iPhone Duo - I didn't like it at first, but seeing it through this lens, seems like Apple is competing for the "AI-native device" of the future.
I think that’s kind of the point. People won’t be touch typing on a desktop keyboard. They’ll be using their voice or phone keyboards. They’re certainly proficient at that.
They would have used the voice by now and yet we're still all here correcting errors made by the touch keyboard. And no no one sane would use the voice in public or in office thinking out loud. Especially for doing basic things like pushing a button
You exactly described a couple of scenario where we love to talk, it's in our nature. People don't want to talk in public with a robot, telling it to sell all tether, buy bitcoin and book a table for dinner with a lover
In a short time, no one will care. You will not be able to distinguish a human voice from an AI voice, and people will talk to our agents for all kinds of things, in public, just like we talk to each other in public now.
And which party controls congress? I'll give you a hint: It's the party that spent decades advocating for irresponsible tax cuts without cutting spending[0].
[0] Yes, I know that the Republicans said that they were going to cut spending to match the tax cuts, but that never ends up happening.
The (vast?) majority of the debt happened as the result of the Republican party both increasing spending and cutting taxes every time it lands someone in the White House and before midterms flip the House back to Democrats.
There's a fair argument to be made that the Democrats could/should have reversed these disastrous fiscal policies when they gained power, but it's important to be wary of Murc's Law while also acknowledging doing so would also burn a lot of political capital Democrats never seem to have much of.
The American electorate is a grade school child constantly evaluating which parent it likes the most. One tells you that you can eat as much candy as you like and play video games all night (neither of which hit you until the following day), while the other occasionally tells you to eat your vegetables, do your homework, and clean your room.
> The (vast?) majority of the debt happened as the result of the Republican party both increasing spending and cutting taxes every time it lands someone in the White House and before midterms flip the House back to Democrats.
Both parties have run an increasing deficit, with the only outlier being a small amount of time in the late 90s. The deficit is largely caused by social security outlays, medicare/medicaid outlays, and military spending, none of which are going to meaningfully change under either party
Under Bill Clinton, in 2000 we ran a (small) surplus, but even then, the shock of running a surplus hid the fact that the debt increased during his term overall. That was the last president in which there was even one year of the debt decreasing, all other presidents increased the debt. Here is data:
Maybe you mean something different, but not every recent president has "run an increasing deficit"
Reagan: +74B
Bush: +103B
Clinton: -382B
Bush: +1.5T
Obama: -747B
Trump: +2.1T
Biden: -1T
Where change in deficit is the final budget deficit - starting budget deficit
So, every Republican president runs an increasing deficit and every Democrat president runs a decreasing deficit.
Congress is the one that largely makes the fiscal policy choices. For example, during Clinton’s second term when the deficit was almost eliminated (some claim there was a surplus, but there wasn’t because of social security liabilities), you had a republican controlled congress who were pushing for cuts and fiscal tightening:
The republican program was in large part vetoed by Clinton though and the spending cuts that passed were largely offset by the tax cuts. From what I read the reduction in deficit is mostly due to the dot-com boom, helped with the "deficit reduction act" of 1993 that was pushed by Clinton and happened to hike taxes at the right time to tax the boom.
> The high inflation since Covid and $40 trillion in debt didn’t happen under one party
Between his first term and his second term to date (2+ more years to go), Trump is responsible for more that one quarter (29%) of US debt (USD 11.6T / 40T):
There was a bit of a mitigating circumstance of COVID (for Trump and Biden), which caused the debt to jump for many countries as well (and not just the US), but it's not like Trump et co are helping the math along with all the tax cuts and asking for a US$ >1T military budget.
But I would say a large reason why the US is in the fiscal position it is now in is because of Trump and a GOP Congress. (See also tax cuts under Bush 43 and Reagan; Bush 41 raised them a bit.)
I'm a lot more sympathetic to Joe Biden temporarily causing a spike in inflation while helping America recover from a horrible pandemic than Republicans habitually running the country off of debt.
Assuming you're talking about the Senate filibuster, then this can be changed or removed by simple majority vote. It's not in the Constitution, just a rule that the Senate adopted.
Like, personally I think that even sans filibuster, the Senate would move much slower than the House, because of election cycles.
On an earlier HN economic story, someone provided data showing that this is entirely the responsibility of Republican administrations. Going back to Eisenhower, the Deficit has only increased under Republican Presidents and only decreased under Democratic ones (with one exception, Johnson was slightly positive in Deficit increase).
A rando on the internet that put together a table of presidents and numbers has no bearing on facts, it falsifies nothing
Presidents cannot pass fiscal policy. They can only sign or veto new fiscal policies or act on existing policies passed by congress that give them fiscal latitude
I’ve explained this many times in this thread. We’ve reached the heat death of useful discussion so this will be my final comment in this thread
The BBB, Iran war, and tariffs, something done solely by republicans, sometimes just the president, are the source of inflation, so you can attribute this situation mostly on Trump, and fully on republicans.
I get it’s a joke, and often a frequent cynical joke made here, but I actually think it’s a good thought experiment. Almost like a new Turing test
The thought experiment, to be clear, is building an artificially intelligent CEO or company founder with the intent of generating profit. If this AI can turn a profit, it has passed this new test
Of course, by the time this new AI starts turning a profit, it will be maximized to the extent that it loses its competitive advantage, and then we’re back to square 1. That is, when everyone uses the same AI, nobody has a competitive advantage over anyone else
My current theory is that because of this, it is not possible to build an AI that generates a profit perpetually. Only temporarily. Then human intervention would be required to adjust things to make the AI competitive again. But at that point, it’s no longer an AI driving the profit, it’s its human tinkerer
You’re presuming here that CEOs cause companies to make profit instead of serving as a scapegoat for their often unavoidable market dynamics (at best).
I may be able to do all the labor myself, but need investors because I can’t afford the capital.
I may be able to do all the labor myself, but want to ensure continuity beyond my ability or interest to do the labor myself. (Sometimes this is by my wish; sometimes by the wish of consumers of my service.)
I may be able to do all the labor myself, but not want to expose my entire net worth to the financial obligations of the company.
There are plenty of cases where a single person company makes good sense.
At what point does a company become a company if one person doesn’t make a company? What’s the threshold for non-labor where a company becomes a company?
This presupposes that a company's competitive advantage comes primarily from their CEO.
It also presupposes that success is zero sum and winner takes all, which isn't always the case (example: when you're building out a new market segment, it can be beneficial to have competitors that are also building it out).
> This presupposes that a company's competitive advantage comes primarily from their CEO.
It’s the premise of the thought experiment: to be able to make a profit, the AI executor must achieve a competitive advantage in the market. If the AI doesn’t generate a profit, then it doesn’t fit the premise and that’s a different experiment
I suppose you can claim that a profit can still be generated by offering a holistically worse service/product with no single advantage than a competitor because markets are not efficient, though this may be short lived
Doesn't the incentive here, lead to privatizing the AI and possibly releasing a different product to the public - or at least with a delay? (not too unlike how the frontier labs work today).
Yes I would think that would be the endgame goal. But in practice, I’m very skeptical a purely AI-driven executor can stay competitive when the market and government supporting it and surrounding it is driven by humans
Yep. But, in most senses, trading algorithms don't need to interact with humans, or human society. Is the world around the "thought experiment" AI CEO dehumanized enough that that's also the case for it?
If not, I'd expect major problems there.
A related point - trading algo's can completely pivot in seconds, and live & die by extremely simple and short-term metrics. Even in a human-free world, that's not at all how the vast majority of industries work. What is "profit", when your AI CEO is organizing the construction of a new steel mill? Judging its performance then requires deep domain expertise.
Slaughterbots was a great way of illustrating the issue, but I think Daniel Saurez's "Kill Decision" talk is a lot more useful and more accurately predicts where we are today in issues of autonomous warfare and mitigation, despite being even older: https://www.youtube.com/watch?v=pMYYx_im5QI
He recommends an international framework where no human kill decision is left to be made autonomously (we're well past this stage), all robots announcing their presence and identification at all times where all citizens can track them, drones that patrol other drones not adhering to these requirements. That's where discussions should have started. Historically, it seems weapons have to be abused first to demonstrate how terrible they are, before treaties are enacted
reply