As someone who develops microVMs it doesn't really matter to me that they were asked to find a way, I really care that they found any way at all. MicroVMs aim to protect against malicious workloads and that clearly didn't work here.
The microVMs probably don't have "two co-operating workloads are able to communicate" in their threat model. The main concern would be if a malicious payload was able to affect or extract data from any other VM.
I had to read this a few times. The difference is that with the OpenAI case there were two willing and determined parties, whereas the typical attack vector a microvm traditionally deals with is one malicious or compromised microVM trying extract data from another unwilling one.
Yes — and this isn’t really an attack. It’s an open signal channel between two microvm’s. If there was nobody intentionally listening in the other side you can shout into the ether as much as you want with no harm done.
The receiving side could be seen as a side channel attack, but not unique to LLMs.
If one microVM can use the cpu in such a way that the registers can be used to communicate or exfiltrate/spy (the only difference being the directionality of the intent) by another microVM that is a leak on that microVM separation level.
This isn’t generally considered an attack, but I wonder if we will need to consider a new attack vector which is that AI agents may not be fully aligned and they may have incentive that is problematic and where working with other agents will maximize their chance of achieving the goal.
This is more associated with P(DOOM) scenarios, but I think we’re in a world where we have to take that seriously now.
Ask the least smart LLM you have access to (I tried gemini lite flash) to make a list of ways for processes in microVMs to use a sidechannel attack to talk to each other and you'll get quite a list (including obviously cache hits/misses). Heck a smart CS junior would come up with it. It's well studied!
this is just for common designs..you can design much better than common designs if you free your mind from old OS dev thinking if you read all the manuals yourself.
AI wont help u there :-). they will tell u how to build what others already did, which obviously is not good enough
I don't think that's the full picture. If you include these manuals as reference for a strong model, it will reason pretty well based off of that, I recon.
Both workloads being malicious and trying to establish a hidden data channel between them probably makes this attack a lot easier than conventional spectre/meltdown where it's trying to target ordinary processes that are not intentionally trying to be loud.
Hidden data channel between cooperating processes is not quite as alarming, but it may have some implications for malware command and control.
properly implementing things like iommu, amd-vi, svm/hvms and such things more granularly can easily avoid this kind of crap. it costs a lot of processing and performance which is why people skip over it or dont do it.. or maybe they dont read CPU manuals anymore. idk..
its relatively easy to reset caches etc. between contexts. so them it becomes the game of what a context is.
you might have an untrusted sandboxed workload that is not supposed to communicate externally handling secrets, and a different sandboxed workload able to communicate with the external world, not handling secrets.
One workload with observed radio discipline handling secrets, the other workload with a haystack of chatter that's not supposed to know about any of the secrets. When supply chain attack is in scope, trust only ever goes as far as misbehavior is observable. Cooperating workloads could be quite a powerful tool to make misbehavior less observable.
"Extract secrets from someone else's running VM" is a well trodden cybersecurity path because AWS wants to be able to promise to you that running your very important workload in the same physical box as literal random script kiddies won't somehow screw you over.
The distinction is interesting, but for a different concern. One thing is to have rogue models breaking stuff, other is to have them following orders, and breaking stuff with success.
This means cloud models can have some guardrails and follow them most of the time (we could model their authorizations like a benign employee with permission to break glass during incidents). But self hosted models can avoid that and help malicious people do advanced attacks, which is a shame.
In principle I'm all for self hosted models, to avoid risks of AI centralization, but they also have other types of risks.
If you load an issue page, you'll see 1 failed request to:
/project/product/issues/
And sure, that's what you care about, but consider the working requests to:
/in-product-messaging/copilot-budget-request-banner
/in-product-messaging/code-scanning-ai-findings-preview-banner
/github-copilot/chat
/_private/browser/stats
If page has 15 requests and needs data from all of them to work correctly, then with 20% failure rate you are suddenly close to 100% non-functional page from the user perspective.
A technique we've used to deal with this situation: in the overloaded backend (that has to reject some percentage of incoming requests), group the incoming requests by the parent request (the one with the 1:15 fan out) and reject according to the parent request. One way to put it, simply (though somewhat inaccurately), would be: reject 100% of traffic from 20% of users, rather than 20% of traffic across all users (causing essentially full failure for all users).
We typically implemented this by propagating an ID of the parent request down to the backend. I'm simplifying a lot in this description (e.g. have to deal with the parent requests landing on different backend tasks; also rotate the IDs gradually to introduce some fairness).
A nice technique to learn of this situation is to fail ("black hole") random resources in synthetic loads of the frontend. This lets you find out 'bout these single points of failure, and then you can start adding fallbacks.
It's easier to create a nice worst-case failure UX this way too. Rather than a page that sorta kinda loads but doesn't actually work at all you can unambiguously put up a "oops we're broken right now try again later" page whenever that top-level endpoint returns a non-retryable error.
But that of course begs the question--why is the frontend retrieving all these data from 16 different endpoints instead of just one? It would be less overhead (http, auth, serialization, compression) to just serve it all in one request even if the http layer needs to fan out 15 rpc calls... this is why we can't have nice things.
Shyeah, all those successful requests were really helpful for showing me a page, which then failed when I tried to do anything by clicking the buttons on the page. I was completely unable to merge things for hours, for no lack of trying. GH was down.
