What do you mean, "not acceptable"? Given the specifics of the situation, I think 15 hours is defensible. Where do you suggest putting the blame for the implementation and deployment of the fix?
What's not acceptable to me is a bug of this impact going undetected in production for 6+ months.
I believe that even less than I believe the pardons won't be issued. The world has expected a lot of things from the US, only to see the exact opposite happen in last years.
Lock contention is when multiple threads try to access the same (memory) resource. More cores == more threads accessing the resource, and faster execution == less time between resource access for each thread. Both factors increase the time spent waiting on said lock, in relative and in absolute terms.
To put in a car analogy: if you have a ferry that can carry X cars per crossing, increasing either the number of lanes to the ferry (more cores) or increasing the maximum driving speed (faster cores) will only result in longer queues at the water's edge.
> Which leads to my suspicion it was internal load that brought the system down
IIRC Microsoft posted a comment around 10pm where they discussed rate-limiting external connections, and many reactions on last week's HN thread were like "wait, you so didn't even drop load before now?" So internal load may have been a contributing factor, but it's by no means certain given that they didn't stop serving clients.
> IIRC, I only had to put back the old apt source entries then apt update and apt dist-upgrade...
The problem with rollbacks is not the packages themselves, that usually works without issues. The problem is with configuration and data items: if a package changes configuration format on upgrade (e.g. move from pkg.conf to pkg.conf.d), a downgrade will not restore the old version (e.g. rebuild pkg.conf from the .d snippets).
Also, apt will not downgrade packages without prodding. At the very least, you would need --allow-downgrades.
This might be the first case in history that an american uses socialist when they mean communist, instead of the other way round. It is communism that steers towards goverment ownership of companies; socialism would involve ownership by trade unions.
Microsoft employs enough people that they can afford to have one team scan for unusual ingress/egress traffic patterns while others work on troubleshooting. If not, they should not be shedding 60k employees right now.
What's not acceptable to me is a bug of this impact going undetected in production for 6+ months.
reply