From the pitch at GTC, the idea is not just to limit what each individual agent can do, but to also to ensure that the collective permissions do not exceed what you want agents to do.
Apparently agents will collude to break out of their individual sandboxes by delegating tasks to other agents that have the required permissions. This is a problem with agent fleets, because you need them to coordinate on their tasks. So you need to them to communicate.
A naive approach to prevent agents from doing something you dont want to do is a separation of powers, which individual permissions limited to each agents task. Say one agent to find and extract raw data, one to process it and write a report and one to publish it. If one of the agent decides that needs to publish the raw data, then chances are that they will collude to make that happen, even though no single agent has the permissions to achieve that goal.
OpenShell tries to fix this by monitoring the collective permissions set of the agent fleet, and by maintaining very granular permissions for each agent.
This was presented together with Sentry, which is something like a on-chip hardware firewall (BlueField) that sits between the chip agent run-time runs on, and the host. This improves safety by removing the ability of agents to modify their own OpenShell policies.
But the raw data has PII and releasing it will harm people! Wouldn't a more effective defense be to give all the communications between agents to an ai agent that's also on the team that's been thoroughly indoctrinated to your values and is empowered to terminate agents and reports back to hq? If I recall correctly, that's how the Soviets did it.
On a more practical traditional computing note, you'd have a deterministic program sitting that scrubs the formatted data of PII before and after processing. with extra code to alert if the after-processing ever actually sees PII.
Well, yes and no. It increases the pressure on people in power. More pressure, more mistakes.
In this case the question is, if US Army freezers were hacked earlier, why did nobody brief other operators of cooling equipment. Its a failure at the top! Everyone is getting flustered and knives are out.
But maybe they could not fix the underlying software or whatever and were not ready to share details without opening themselves up to more attacks. Also there are a lot of attacks happening all the time, which ones do you share?
If the psychological pressure at the top increases, more mistakes will be made. Sometimes mistakes can be subtle over or under reactions, sometimes the mistake is not to do anything. It is real effort to evaluate an appropriate response. For example, the military could reach out to commercial operators quicker, but then they have to handle all the comms, questions, and they also potentially leak more attack vectors. The commercials also start having this overhead of having to read emails from the military and internally those orgs have to deal with that overhead in comms to evaluate the seriousness.
This is a problem, because very few orgs have spare capacity for this stuff and so they have a choice. They can react to the military comms and risk mistakes somewhere else due to comms / management bottlenecks, or vice versa. There are only bad choices. And every time this happens, another choice has to be made.
Lots of these will be low-level attacks with manageable impact, which lowers the alertness over time, until that moment when something serious happens. If at point everyone is exhausted a real costly mistake will happen. Its the long game.
In this case, I am sure a consideration was that the attack was already used against high-profile cases and it makes sense to just use the attack vector more broadly before it gets closed.
If a fix existed, then why didn't Danfoss have a fix?
I agree that they had a firmware update, but unless they had the operational capacity to update the checks notes freezers of all their customers, that is not a fix. The theoretical and technological aspects of solving a problem are often the easy parts.
It's like if NASA would claim to have a fix for the Opportunity rover on mars to get it operational again. All they need to do is to replace the batteries. Well, that is not a fix if there is no way to actually replace the batteries in practice.
I don't see why the existence of a firmware update would remove the culpability of negligence from the hardware manufacturer.
IT. Whether they had the resources and competence to do so, is another question. (competent IT should also stop such devices from being connected directly to the internet in the first place).
A short outage can snowball very easily in a lot of lost time. What I learned when working with enterprises is that above all they value reliability. This is for a reason.
A short outage might at best trigger loads of paperwork for multiple hierarchies, big meetings etc. The org has no choice. It needs to evaluate if whatever happens is a threat to their business.
In the worst case it is that, plus missing some crucial windows of delivery. This is because a system that is unavailable for a short time can cause backlogs that, like traffic jams, cascade as everyone has to slow down and then synchronously speed up again.
Orgs have the option to create more resilience, but that is overhead similar to compliance. You need to drill all your backup plans all the time, otherwise they are worthless. The drills cost time and money. At scale it is infeasible to be robust to all failures. Therefore, enterprises (at least) often prefer reliable systems over sophisticated systems. Because this delegates the risk management to the vendors rather than adding an overhead to every employee. Because at some point the employee would just do drills all the time instead of work.
Why does Github not segregate the free offerings from the enterprise or even better, all paid offerings?
It is unacceptable that enterprise plans get impacted by traffic on free and public repos. Our repos are neither on the free plan nor are they open. We have not had more AI stuff happening in the last weeks. Our traffic is stable. I would wager that most enterprises did not spike the traffic all of the sudden. Even if they were, we are paying for our quotas. Still our Github actions were breaking and our PRs not viewable at some times.
I am hoping this instability is going to cause a Cambrian explosion of forges and if that is happening, Github will be the first victim of the AI revolution.
I am working on a truly decentralized / local first code review right now, and a big part of my motivation for this is how bad Github has become. I dont know if I have enough time to build CI as well, but I am hoping others do. Otherwise I will just fall back onto Jenkins.
This article read like Claude output placed inside massive ads. 0 value above telling Claude to summarize the source for me. Except, the source was not linked as article link.
Why not simply rebuild what you need with Claude? Airtable seems to me as a prototyping product. Once you know what you want, formulate it as product objectives and ask Claude how it could be implemented from scratch. Dont even give it any ideas about Django. Just ask it what technologies are sensible for what you need, read and verify its answers, lock in a stack, plan your features and implement them.
You will be surprised how many things you could not fix in Airtable or where awkward workarounds can be solved much more elegantly if you (or Claude) consider them straight in the design phase. Just make sure to start with your objectives and _maaaybe_ desired workflows to avoid biasing Claude too much and that you end up rebuilding Airtable.
> Why not simply rebuild what you need with Claude?
Claude isn't some magic genie that can give you whatever you ask for. If you need a spreadsheet, it is much easier at this point to start Excel than to ask Claude to write you an Excel.
Apparently agents will collude to break out of their individual sandboxes by delegating tasks to other agents that have the required permissions. This is a problem with agent fleets, because you need them to coordinate on their tasks. So you need to them to communicate.
A naive approach to prevent agents from doing something you dont want to do is a separation of powers, which individual permissions limited to each agents task. Say one agent to find and extract raw data, one to process it and write a report and one to publish it. If one of the agent decides that needs to publish the raw data, then chances are that they will collude to make that happen, even though no single agent has the permissions to achieve that goal.
OpenShell tries to fix this by monitoring the collective permissions set of the agent fleet, and by maintaining very granular permissions for each agent.
This was presented together with Sentry, which is something like a on-chip hardware firewall (BlueField) that sits between the chip agent run-time runs on, and the host. This improves safety by removing the ability of agents to modify their own OpenShell policies.
reply