Hello, security engineer that has installed CrowdStrike on thousands of computers.
A thread on the outage and what is exactly happening.
Here’s a quick explainer on what *seems* to be the cause of the CrowdStrike outage and why it happened so quickly.
If I'm reading this correctly modern AVs work by looking for patterns in software behavior that resemble the actions of currently circulating viruses to try to stay ahead of the rapid proliferation of new viruses and threats. So if program A.) behaves in some way like known virus 1.) the software will shut down program A.), not because it's a known threat, but because it behaves like a known threat. So if I'm following this guy something in the stream of behavioral information Crowdstrike sends to all it's client computers in real time flagged some core windows process or something as a threat and began attacking it. This resulted in BSOD bootloops across their network of clients.
Short version; Computer auto-immune disorder, the immune system is attacking the body because it's incorrectly identified some part of the body as a threat.
But for some cloud systems though, such as AWS, booting to “safe mode” is not even possible so this fix can’t be applied. Virtual servers need to be shut down, their disks cloned, attached to another server, edited to remove the offending files and then finally reattached to the original server.
Lol
BUT, if you’re protecting your data properly you would have used BitLocker for disk encryption and so you need to manually decrypt the disk with a BitLocker Recovery Key, which is probably - for most companies - stored digitally on one of the servers that is currently booting over and over 🫠
AHAAAAAHAAHHAHAAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAH HEY FUCKWITS MAYBE PUTTING ALL YOUR SHIT ON REMOTE ALWAYS ONLINE SERVERS OVER WHICH YOU HAVE NO CONTROL WAS A CRITICAL STRATEGIC FAILURE TO RETAIN CONTROL OF YOUR CORE ASSETS YOU DUMB FUCKS YOU GAVE AWAY THE KEYS TO THE VAULT TO A GUY IN BLANK WHITE MASK WITH "TRUST ME" WRITTEN ON IT HAHAHAHAHAHAHAHAHAHAHAHAHA
The cloud was always an obvious, utterly inexplicable mistake of astonishing proportions and it's hilarious that capitalism drove everyone to turn their systems in to dumb terminals over which they have little if any control. I'll just be here basking in my "I called it" from well over a decade ago.
anyways, it sure is great to grant kernel-level access to a program so it can better protect you from viruses by, uh... using its kernel-level access to break your entire system?
No, they actually just pushed out a bugged driver that they use to hook into the windows kernel. Turned out to be nothing to do with the realtime A/V feed. Which is honestly funnier because there is NO reason to push that type of update out worldwide in one go, it should be done in stages to catch bugs like this before they go global...
The OP thread talked about it just below the "Load More Replies" fold
If I'm reading this correctly modern AVs work by looking for patterns in software behavior that resemble the actions of currently circulating viruses to try to stay ahead of the rapid proliferation of new viruses and threats. So if program A.) behaves in some way like known virus 1.) the software will shut down program A.), not because it's a known threat, but because it behaves like a known threat. So if I'm following this guy something in the stream of behavioral information Crowdstrike sends to all it's client computers in real time flagged some core windows process or something as a threat and began attacking it. This resulted in BSOD bootloops across their network of clients.
Short version; Computer auto-immune disorder, the immune system is attacking the body because it's incorrectly identified some part of the body as a threat.
Lol
AHAAAAAHAAHHAHAAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAH HEY FUCKWITS MAYBE PUTTING ALL YOUR SHIT ON REMOTE ALWAYS ONLINE SERVERS OVER WHICH YOU HAVE NO CONTROL WAS A CRITICAL STRATEGIC FAILURE TO RETAIN CONTROL OF YOUR CORE ASSETS YOU DUMB FUCKS YOU GAVE AWAY THE KEYS TO THE VAULT TO A GUY IN BLANK WHITE MASK WITH "TRUST ME" WRITTEN ON IT HAHAHAHAHAHAHAHAHAHAHAHAHA
The cloud was always an obvious, utterly inexplicable mistake of astonishing proportions and it's hilarious that capitalism drove everyone to turn their systems in to dumb terminals over which they have little if any control. I'll just be here basking in my "I called it" from well over a decade ago.
What if we took all of our extremely fragile eggs and put them all in single unstoppable basket
Capitalists are obligated to do this if it's the most profitable thing to do. We should use this against them.
I love software engineering
anyways, it sure is great to grant kernel-level access to a program so it can better protect you from viruses by, uh... using its kernel-level access to break your entire system?
bruh they invented computer cancer lmao
BRING ME JOHN MACAFEET. That whalefucker is the only one who can unfuck this whale of a problem.
Biden begging Xi right now to call Kim and ask him to use Juche Necromancy on John McAfee and save the world
No, they actually just pushed out a bugged driver that they use to hook into the windows kernel. Turned out to be nothing to do with the realtime A/V feed. Which is honestly funnier because there is NO reason to push that type of update out worldwide in one go, it should be done in stages to catch bugs like this before they go global...
The OP thread talked about it just below the "Load More Replies" fold