Doesn't the article state the opposite? The low end was not the problem. It was high end that was the problem.
I like ragging on Microsoft as much as the next guy, but honestly this problem sounds complex & hard to predict. AI or not, this feels like the type of bug that would have slipped by me as well.
The response clearly could have been stronger though.
Seems like the issue happens readily at high load on a system with many cores. Once the conditions were right it seemed to impact a large proportion of their fleet. They just didnt have that combination in their test/validation pipeline. They likely have some sort of load test in their pipeline, but it's running on machine that's much smaller than their production machines... so it didn't trigger the lock contention bug.
The opposite is true running in production for the last several months. They were runnign on big production machines with enough cores, but never quite had enough load to trigger the lock contention. But once both conditions were true, big machines and high load, due to some unrelated failover elsewhere suddently bringing more load, we then had the magic combination.
It's really not unreasonable to expect an enormous hyperscaler like Microsoft to do load testing on hardware that exactly matches their production systems.
I read "more than one low end" as "more testing environments than one low end" rather than "more low end environments than one". It's a snarky way of saying they should have been testing high end environments for no more reason than they had the money to do it, rather than saying they should have been testing the low end more.
> Now, are Microsoft going to offer any compensation for either lost business or additional accrued costs? I highly doubt it, even though it is literally their fault and they do still have the cash reserves, despite increasingly wasting it on loss-making AI adventures.
Aren't they? I know that they, and other big techs, have clauses in their terms that say if they don't meet service requirements they will offer refunds/credits. I've never been in a position where I was the one requesting the refund but I always presumed they actually did it.
The real answer here is nobody knows, and it will likely be up to SCOTUS. Everyone saying it needs a constitutional amendment doesn't understand what NPVIC is. The whole point of it is to do an end-run around the constitutional amendment process - a hack if you will.
Whether this hack is legal is a complicated question (https://en.wikipedia.org/wiki/Constitutionality_of_the_Natio...) that realistically can't be answered by anyone but SCOTUS. If states pull the trigger on NPVIC then I would bet good money that it ends up in SCOTUS, but who knows how they will rule.
And if the sentiment toward such sleight of hand is as negative as the sentiment toward Trump last Tuesday, then the pendulum could swing as swiftly to the right as it did to the left.
This was pretty clearly a backlash against the high gas and groceries prices. Oh and data centers. The average voter doesn't care about things they think are too complicated.
Not saying this is the case, but the way this unfolded, and announcement reminded me of the crypto scams that were popular a few years ago (ponzy schemes), with the final rug-pull being "oh no, our platform was hacked, all your funds were lost, sorry!", i.e. "coscoin"[0][1] I know this is not the case here, but the way they announce the hack is funnily similar to how those fake platforms did their final announcement.
according to Binance's own account so far, the first statement said it “may involve a compromised DevOps engineer’s laptop” that gave the attacker access to a service container to submit withdrawals. then on Friday, Richard Teng said the attacker “did not simply use one engineer’s permissions”, that they’d found additional points of breach, and malware built to destroy itself and its traces.
they also say there’s no evidence any private key leaked yet their own signing system signed the transfers.
That doesnt mean the key leaked - its likely in an HSM for example, but there is an API to get it to sign stuff. Access to use keys is less bad in many ways than leaking keys, but still pretty useful.
A quirk here- tags are not actually immutable in git. I agree with the premise, and use tags myself for releases, but if you rely on them being immutable then you have loaded a footgun
Honestly I don't agree with this. A few reasons why I feel like I will always want a plan mode:
1. I want to know whats going to happen, at least at a high level, before changes are actually made.
2. Plan mode helps me flesh out the missing details of my plan before being mid-execution
3. In situations where I have a limited budget for AI usage I will often times use a high powered model like Opus 5.5 or Fable to make a detailed plan, then scale down to a cheaper model for implementation. I feel like this saves cost in the end.
I get plan mode is basically just a small hidden prompt. I get that I can basically just preface my prompts with "make a plan only, don't make actual changes." Maybe this is just a UX trick, but it works well for my brain.
I think the US does the same? At the very least it doesn't make sense to include retired people in unemployment. By similar logic I think it doesn't make sense to count people who are not actively looking for employment.
I like ragging on Microsoft as much as the next guy, but honestly this problem sounds complex & hard to predict. AI or not, this feels like the type of bug that would have slipped by me as well.
The response clearly could have been stronger though.
reply