Windows BSOD Chaos - CrowdStrike's Faulty Update

Introduction What's up everyone? Obviously you're not here because your flight was grounded. I hope everyone today is doing well with that. The massive outage that's actually happening today is not something that's actually affecting. I'd love to know how you're all doing with this outage and if it's affect. I know roach hit me up earlier saying that he's facing some minor issues and it hasn't really affected me at all. but I'm curious to see if it's affected any of you guys. let me know in the comments below. Today on July 19th, software update from CrowdStrike. Basically wreaks havoc. I will. Be willing to widespread. Outages. Although the incident isn't actually a cyber attack, it's. Hopefully a wake up call to switch hosting from Windows to Linux. If nothing else, I think it really shows the significant issues within Microsoft's actual ecosystem, which is nothing new. The hacking group L0pht also known as lost heavy industries Microsoft's Historical Issues kicked to Microsoft in the teeth through a reasonable disclosure for almost ten years. They formed this hacking. Group in 1992. That was 32 years ago. It really goes to show how long Microsoft has been just completely screwing up, and multiple ways. And back then they were just totally refused even acknowledge the fact that these guys had actually found vulnerabilities, which is nuts. Now, keep in mind, this isn't some little rinky dink go through. One of their primary affiliations was, cult of the Dead Cow, which, if you've ever been in the hacking scene, then you probably very well familiar with, which is another infamous hacking group essentially from back in the day. The New York Times did a featured article on them in 1999, and they also testified in front of Congress. So some kind of, kind of a big deal. shout out to Mudge. I the group ended up. Shutting down in 2000, I'm not a fan of Microsoft myself. Unfortunately, until I switched to 20 resolve, I'm pretty much stuck with Adobe products, which really sucks. And and, you know, forces me to stay inside of the trash windows ecosystem for my production. Anyways, my daily driver is Linux Mint. Confirmed. My point with all this is that for a long time we've seen Microsoft. Get caught with their pants down, and today was just another great example of a complete Microsoft fail by a company that has been excelling in failures for 49 years now. It. CrowdStrike's Role and Relationship with Microsoft So it leads us to this other company, CrowdStrike. Many of you already are very familiar with the Microsoft operating system, and in fact, probably, you know, have to be living under a rock if you haven't actually heard about it or haven't used it as an adult. CrowdStrike, on the other hand, is a completely different thing altogether. CrowdStrike is basically just a cybersecurity company. Its most well known product is called. Falcon Platform, and this platform essentially tries to catch bad actors who are trying to infiltrate various computer systems or servers or whatever. Typically you see both CrowdStrike Falcon and Microsoft working together as a team. Many people, including threat actors I've interviewed, have stated that they feel that Microsoft Defender is ample as a defense for your average end user, whether working, shopping, streaming, gaming, or browsing. That said, it's not always enough and sometimes it can, you know, completely miss fairly sophisticated attacks. And that's exactly where CrowdStrike is supposed to come in. CrowdStrike uses, AI and things. Like, you know. Accurate time monitoring to try to catch stuff that Windows. Defender might overlook or might be invisible actually to it. Impact of the Outage The impact of the outage is actually pretty severe. The CrowdStrike update triggered. Numerous windows systems to. Have the blue Screen of Death or Sad, as it's as it's typically called. At. The impact of the outage is pretty, pretty intense. The CrowdStrike update triggered numerous windows systems to BSOD or. Have the blue screen of death as where they freeze up and they basically shut down. This problem cascaded into massive disruptions across various sectors, from airline operations where airlines such as, American Airlines and Easyjet experienced severe delays and. Ultimately. Cancellations as their systems. Depended on things like. Microsoft services. Built airports had to. Revert to manual check in processes. Leading to really long queues and frustrated passengers, which is completely understandable. In addition. Several other airlines have actually requested FAA assistance. With technical issues, According to the FAA news. Health care services, many health care providers. In the UK lost access to patient. Records. appointment systems and prescription services. This forced medical staff to use manual methods compromising patient care and basically just delaying treatments. Grocery stores were affected to. ATMs, and other financial systems also faced a bunch of outages, disrupting transactions and leading. To widespread inconvenience and issues for all their end users. Retail stores, regular, small and medium large companies all were affected and really most. Concerning were some emergency services. The entire scope. Of the. Outage was really, really large, affecting approximately 24,000 customers that. We know off the off off hand. the number of individual computers actually impacted. No one really actually knows at this point. as many of the. Affected customers were actually large organizations. CrowdStrike serves nearly 60% of fortune 500 companies, and more than half of the fortune 1000 saw this error rippled through. a significant portion of the global business infrastructure. So one thing I wanted to get across is that, you know, this isn't really a isolated incident, though the recent CrowdStrike. Microsoft's Recent Issues Induced outage. Is part of a larger pattern of problems that. I feel plagues Microsoft products just in general on a daily basis. Frequent and disruptive outages are definitely something that. Microsoft customers are. Fairly familiar with. and the things that have. Occurred fairly recently, like fairly recently, there's, Microsoft Teams outage where earlier in January. Microsoft Teams faced a major outage that affected users globally, causing various issues like login. Problems and message delays and connectivity issues, which. Obviously very, very bad if if you're actually depending on that. For business communication, the root cause was a networking. Problem within. Microsoft's actual infrastructure. And another issue we saw was with Microsoft Windows updates, the June updates, in 2004, obviously with Windows 11, caused significant issues, including things like. Taskbar malfunctions and driver conflicts, which forced. A ton of users to actually rollback to previous versions. Another instance of Microsoft, you know. Just screwing everything up was another set of buggy updates and. Makers and software instability. Another example of why is some. Buggy. Updates and software instability. That related to our, new. Outlook 2024 was. Severely criticized. I just o gotten a. Lot of forums for reducing functionality and performance issues. Users had reported difficulties accessing things like emails and managing. Attachments. Which is kind of the point of outlook. Like it's like, imagine trying to open up your web browser and it just doesn't connect to the internet and everything's set up. That's kind of the issue. It's just it's it's absolutely. Insane that the email product that doesn't even do its main function, it's not. Securing vulnerabilities. we saw a ton of. Vulnerabilities lately with Microsoft. Products that are, you know, more often than not. Definitely the the main. Targets of cyber attacks due to their. Ridiculous amount of vulnerabilities that they actually have, which, you know, also date back to, you know, what we're. Talking about a lot earlier, L0pht blowing up their spot, looking at modern issues like, the May 2024 patch, addressed like 61 flaws. including three. Zero D vulnerabilities across various Microsoft services, which is no small number. actual issues. Yeah. Especially for a company that is as well versed in having vulnerabilities as Microsoft OT system attacks, these high profile of cyber attacks such as the one on the. Aliquippa star I probably totally butchered that water plant. exploited vulnerabilities in. Microsoft's operation technology systems, which, again, caused significant disruptions. So as my point with all this, Microsoft sucks is, is basically. Switching to Linux You know, my point, like, now would be a great time for anyone. Who's actually running a server or has, you know, this software is affected by it to seriously start doing some investigation. Into mitigating over to a linux server. And I know, like I'm being totally one sided in this, given the fact that, like, crowdsource is the one that actually put out the update, you know, like they they're the ones who basically did this. But, I mean, I just hate Microsoft, I really do. also the the level of suck is. Evident in the fact that, you know, if you. Had this on Linux or Mac, like you were running CrowdStrike on Linux or Mac, and you were hosting. Like you, you would have no. Issues. So like the CrowdStrike outage didn't impact Mac or Linux systems. So for me, this shows a pretty important difference in the stability and security. Of these. Operating systems. Compared to windows. And like, we're not even getting into the telemetry. Or the the fact that windows is just basically. Spyware or at best, you know, it has a ton of bloatware. both. Mac OS in Linux are often praised for things like now there's security features, which make them a lot less susceptible to. Such. Widespread disruptions. And I cover why windows sucks. And you know what makes Linux better? And another video that I actually. The the training of this video spawned. The creation of that video. Which basically just talks about the benefits of Linux over, over windows. So the actual fix. How to Fix the Issue The actual fix from what we're. Seeing from what they put out, is fairly. Simple to fix this. Issue from CrowdStrike, if you actually have. It. and it's causing this blue screen of death, it's pretty simple. You just boot windows into safe mode or windows recovery environment or WRE as it's called. and. So basically you boot windows in to see if like what it is, if. It's a diagnostic mode. Of windows. That starts the computer with a minimal set of drivers and services that are actually running. Basically, this mode helps you troubleshoot and fix issues by. Loading only the. Most essential of components. And I've had you. Use this mode extensively in the past that probably why, you know. Microsoft saw fit to built it into the systems to begin with. But it's fairly easy to enter safe mode just press and hold the shift. Key while clicking restart from the power menu. This initiates the. Restart process. Into Windows Recovery Environment. Navigate to troubleshoot, then to Advanced. Options, then Startup settings. press F4 key to boot and to save after the eco system restarts. An alternative method, if. You. Can't access the actual login screen, is to turn off your computer, then turn it on and turn it off three times. On the third attempt, windows will. boot into WRE Follow the same steps as above to enter safemode Navigate to the CrowdStrike. Driver folder, which is. Located in Yellow Seed bank on where you store things, but see windows. System32 drivers. CrowdStrike is the typical location for it. Locate and delete the problematic. Driver file, which again is which again is in the CrowdStrike folder. And should. Be “C-00000291*.sys” This is the problematic driver file that's actually causing all those issues. Then reboot the system after deleting the file. Reboot the system. The system should now boot without encountering the blue screen of data set. after that. Consider migrating your operations to a Linux host because windows is trash. The timeline as of the creation of this video is as follows. Timeline of the Outage At 3:55 a.m. on July 19th, the first occurrence happened where Microsoft said it determined the underlying cause and recovered the majority of services. However, the company said that some customers may still experience issues in the region, which has to. Be the biggest blanket statement. I've heard in this whole. Thing. It's like, you know, saying, hey, we fix the issue, but you. Still might have issues. Well, that tells us absolutely nothing. It's a effectually, Saying nothing to the people who. Are saying something. CrowdStrike rolled back the update after the major outage. And the London Stock Exchange at that time said that they were investigating Global Impact of the Outage technical issues. An hour later, at 4:55 a.m.. CrowdStrike Microsoft shares strike, and Microsoft shares lost ground. Microsoft said services were continuing to see improvements and KLM suspended most of its operations due. To global computer outage. Saying that the outage has made it impossible to handle flights. It said this they said this in the statement. We realize that especially given the summer vacation, this is extremely inconvenient for our customers and we're working hard to resolve the problem. Sky news was temporarily. Unavailable to. Broadcast at 5:55 a.m., Britain's Public Health Services said that Most general practitioner and services were suffering disruptions. India's I.T minister was, quote, in touch with Microsoft and the FAA halted Delta, American Airlines and United departures because of this outage. German hospitals canceled elective procedures. Resolving the Outage American Airlines said that it resumed operations, and the CrowdStrike CEO said that the company Was resolving a defect. Swiss National Cyber Security Services said that system failures were caused by CrowdStrike and Blue Screen of Death error messages affected Microsoft. Users. Globally. At this point. National Security Involvement At 6:55 a.m., the United States National Security Council became aware of the incident. German institutions were working with partners to actually resolve the I.T outage. at 7:55 a.m. I.T outage. Predictions and Assessments Might not have had a incredibly straightforward fix. Some quote unquote experts were saying ongoing tech disruptions were set to be the largest I.T outage in history. and security researchers were quoted as saying UK researchers were impacted by the I.T outage. a British hospital declared a critical incident. In the wake of the I.T outage at 855 a. Public Apologies and Statements M, the CrowdStrike CEO. Made an apology, saying we're deeply sorry. The MTA in New York. City's transit system said they were good. And completely operational. At 8:56 a.m., mass. General Brigham canceled non-urgent surgeries and hospital visits. Restoration of Services Two South African banks said services were fully restored, and at 9:21 a.m., the US markets were operating on. At 9:23 a.m.. CrowdStrike CEO said they were working with every customer. Well. 9:41 a.m. Department of Homeland Security was working with CrowdStrike and others to assess and address the outage. Finally, at 9:42 Biden woke up and was finally briefed on the. Outage, six hours after the original incident actually occurred. This was no doubt after yet his morning oatmeal and Metamucil. And, you know, I mean, these dentures at 9:43 a.m., health systems across the United States were facing outages. So Fedex said they were substantial disruptions from the outage. And UPS also said they were impacted. Come 9:44 a.m., Amazon Web Services said that there would be some connectivity issues and those. Connectivity issues they attributed to the CrowdStrike, which. At 9:49 a.m. Union Pacific was facing processing delays while other railroads were operating. Absolutely none. Final Remarks and Conclusion It will definitely be interesting to see how this progresses. I hope everyone's doing well out there. Staying safe. Thanks for watching. Please like and subscribe and I'll see you in the next video.