How The Microsoft CrowdStrike Crash Happened – A Deep Dive into Kernel Drivers and Testing

How a Software App Can Crash Entire Systems: the CrowdStrike Case

Read the video transcript below.

Use Speedify to Increase Your Upload and Download Speeds: Combine Wi-Fi, 4G / 5G Cellular, Ethernet, Starlink and Other Satellites

Image

Speedify combines Wi-Fi, 4G / 5G cellular, Ethernet, Starlink, and other satellites for faster internet uploads and downloads

Speedify is the only software app that combines Wi-Fi, 4G / 5G cellular, Ethernet, Starlink and other satellites at once for secure, faster, and more reliable internet uploads and downloads so you stay online without interruptions.

Speedify will automatically detect and start using any available Internet connections on your device while intelligently distributing your online traffic between them for optimal performance. If you need help we have quick start guides available for most common set ups.

Image

Speedify combines multiple personal hotspots for faster internet upload and download speeds

Speedify's Pair & Share feature enables you to connect to multiple hotspots at the same time for faster upload and download speeds and more reliable internet for everyone. Speedify's Pair & Share feature allows you to wirelessly share 4G / 5G cellular connections back and forth between multiple Speedify users on the same local network when live streaming from an event, calling from the commute or sharing from the field.

Speedify is the only app that allows you to share 4G / 5G cellular data between PCs, Macs, iPhones and Androids. Use multiple iPhones and Android phones as hotspots for internet access and get faster upload and download speeds and mobile failover for all paired devices.

Use Speedify to combine...

Get started with Speedify today!

Speedify combines Wi-Fi, 4G / 5G cellular, Ethernet, Starlink, and other satellites for faster internet uploads and downloads.

Image

Kernel Module Corrupt Data Files: the Root Cause for Many System Crashes

Ryan: How did Crowdstrike allow this to happen? Is there something different about Windows that makes it more vulnerable to this kind of stuff?

Alex Gizis: Windows does allow third-party companies like Crowdstrike to install kernel module drivers right into the the kernel the base of the operating system. This lets them do really powerful things, but it also means if your code crashes there, you crash the whole computer - blue screen to death. The rule is that you try to do as little as you can in the kernel, because you don't want to be the guy who crashes people's computer.

If you look at Windows Defender, Microsoft's product, they've got this kernel driver that when it sees suspicious things sends it out to a user space service that then has a whole database outside the kernel, and that's the same way Speedify works. We actually, in Windows, have a kernel driver too. We just have the tiniest little bit of code in the kernel necessary to do the job.

With Crowdstrike there's something really interesting: the file that crashed the kernel was a data file of what threats to look for that accidentally somehow got filled with zeros. Which means they've actually got the database in the kernel module. I would call that a no no. I would say that is too much.

How to Avoid Causing Vulnerabilities as a Software Developer

Ryan: How did an issue like this get through testing? Do you think they have a testing process?

Alex Gizis: So the story I'm hearing is that they do have a testing process and they did test it and there was no problem with it. But that they test somewhere in the middle of their continuous integration pipeline. And after they were done testing, then they compressed it and encrypted it and did all these other things the file that changed it. And there was a bug there that erased the contents this file and filled it with nulls.

Ryan: So the moral is test again.

Alex Gizis: You're going to have bad bugs.

Ryan: Has Speedify pushed out bugs before?

Alex Gizis: Speedify has pushed out bugs before.

Lessons for the Industry: Test and Have Recovery Procedures

Ryan: Is this Microsoft's fault?

Alex Gizis: No, they put out an interesting little statement on Friday that, due to their settlement with the EU in 2009, they have to give competing security companies the same level of access to Windows, including the kernel, that they themselves have. So Microsoft says they they're not even legally allowed to lock down the computer. And to be honest, I don't want them to.

I'm not quite sure what's going on there that Microsoft says that if you reboot the computer 15 times in a row it'll stop loading their kernel module and you'll be able to proceed.

Ryan: Like some hidden protection in Windows? The latest news is that as the outage persists, the affected computers that haven't been fixed yet are vulnerable to hacking. Is that true?

Alex Gizis: Obviously a computer that's currently blue screen and crashed is not vulnerable at all, it can't do anything. But if all you've done is delete that .sys file that was causing it to crash, all hackers need to do to crash computers like that now is get that sys file back. There's a magic spot, you put a file and anyone with Crowdstrike - you blue screen them. So Crowdstrike has to get an uncrashable version of that kernel driver out to everyone. The short-term fix to delete that file is not enough.

Image

Connectivity Tech Discussions

Our Connectivity Tech Discussions Between Two Palms video series shines the spotlight on Alex and technical guests, diving deep into caonversations about the latest Internet technology, including Starlink satellite, WiFi 7, Apple, fiber optics, new routers, remote connectivity, and networking protocols.

Join us and let's talk tech!

Watch on YouTube