Friday, July 19th 2024

Faulty Windows Update from CrowdStrike Hits Banks and Airlines Around the World

Jul 19th, 2024 05:04 Discuss (234 Comments)

A faulty software update to enterprise computers by cybersecurity firm CrowdStrike has taken millions of computers offline, most of which are in a commercial or enterprise environment, or are Azure deployments. CrowdStrike provides periodic software and security updates to commercial PCs, enterprise PCs, and cloud instances, with a high degree of automation. The latest update reportedly breaks the Windows bootloader, causing bluescreens of death (BSODs), and if configured, invokes Windows Recovery. Enterprises tend to bulletproof the bootloaders of their client machines, and disable generic Windows Recovery tools from Microsoft, which means businesses around the world are left with large numbers of machines that will each take manual fixing. The so-called "Windows CrowdStrike BSOD deluge" has hit critical businesses such as banks, airlines, supermarket chains, and TV broadcasters. Meanwhile, sysadmins on Reddit are wishing each other a happy weekend.

Source: The Verge

Add your own comment

234 Comments on Faulty Windows Update from CrowdStrike Hits Banks and Airlines Around the World

Anoniem

Well, happy weekend other sys admins out there :) Glad we don’t use CrowdStrike services/solutions.

wolf

Better Than Native

Complete meltdown in Perth WA. Banks offline, grocery stores offline, a bloke couldn't get his zinger box from KFC after he knocked off. Triple J (radio) playing back to back bangers from a USB drive.

Feels like just about the only place not impacted was my workplace :fear:

N3utro

Good update management in a medium to large scale enterprise is to have multiple rings of deployment: the IT lab, the IT department users, non critical departments, critical ones. You never deploy windows updates directly, you manage your rings with WSUS. Bad management from these companies.

Assimilator

This doesn't just affect sysadmins, it affects anyone who uses a third party that uses Crowdstrike... ASK ME HOW I KNOW.

Microsoft's 365 platform is also having a bit of a wobble due to a seemingly unrelated issue with a configuration SNAFU in Azure.

N3utroGood update management in a medium to large scale enterprise is to have multiple rings of deployment: the IT lab, the IT department users, non critical departments, critical ones. You never deploy windows updates directly, you manage your rings with WSUS. Bad management from these companies.

The whole point of providers like Crowdstrike is that part of the service fee is for them to do that verification, so that you don't have to. Without such providers, small companies with minimal IT departments couldn't exist.

N3utro

AssimilatorThis doesn't just affect sysadmins, it affects anyone who uses a third party that uses Crowdstrike... ASK ME HOW I KNOW.

Microsoft's 365 platform is also having a bit of a wobble due to a seemingly unrelated issue with a configuration SNAFU in Azure.

The whole point of providers like Crowdstrike is that part of the service fee is for them to do that verification, so that you don't have to. Without such providers, small companies with minimal IT departments couldn't exist.

"banks, airlines, supermarket chains," these are not small companies

cerulliber

Monday hirings QA at Crowdstrike and get 10x ROI. share prices increased. problem fixed, just don't be greed.

Crackong

Windows Update breaks things.

First time? . jpg

WonkoTheSaneUK

Pour one out for sysadmins, who have just learned that the fix is to log into each affected PC one at a time and delete the single bad file from each one.
It's going to be a loooooooooooooooooooooooooooooooooooooooong day for those in bigger organizations!

Chaitanya

MS needs a good screwing over and Windows needs to be offline like good old Win 7(and older).

#10

AnarchoPrimitiv

N3utroGood update management in a medium to large scale enterprise is to have multiple rings of deployment: the IT lab, the IT department users, non critical departments, critical ones. You never deploy windows updates directly, you manage your rings with WSUS. Bad management from these companies.

Wouldn't be surprised if they fired a bunch of the people responsible for that to increase quarterly earnings.

#11

Onasi

CrackongWindows Update breaks things.

ChaitanyaMS needs a good screwing over and Windows needs to be offline like good old Win 7(and older).

I love to bag on MS just as the next guy, but it isn’t about them this time around.

#12

JcRabbit

Judging by the chaos out there, this is what the Y2K bug could have been, but wasn't (because we made sure on time that it would not turn into anything like this - and thus it became a non event).

#13

ErikG

Solution:

Boot Windows into Safe Mode or the Windows Recovery Environment
Navigate to the C:\Windows\System32\drivers\CrowdStrike directory
Locate the file matching “C-00000291*.sys”, and delete it.
Boot the host normally.

#14

Dazz023

CrackongWindows Update breaks things.

First time? . jpg

Just clickbait headline to attract anti-MS crowd.

#15

the54thvoid

Super Intoxicated Moderator

Dazz023Just clickbait headline to attract anti-MS crowd.

Not really - the header implicitly says,

Faulty Windows Update from CrowdStrike

It's blaming Crowdstrike.

#16

Assimilator

N3utro"banks, airlines, supermarket chains," these are not small companies

But they are capitalist companies, and capitalism is all about maximising profits, and that means buying as little equipment as possible and hiring as few people as possible.

OnasiI love to bag on MS just as the next guy, but it isn’t about them this time around.

You're expecting the anti-Microsoft crowd to be capable of basic reading comprehension...

the54thvoidNot really - the header implicitly says,

It's blaming Crowdstrike.

It still implies it's somehow to do with Windows Update, though. A better headline would be "Windows security vendor releases faulty product update, hits banks and airlines around the world".

#17

Shou Miko

ErikGSolution:
Boot Windows into Safe Mode or the Windows Recovery Environment
Navigate to the C:\Windows\System32\drivers\CrowdStrike directory
Locate the file matching “C-00000291*.sys”, and delete it.
Boot the host normally.

I already heard this didn't work for everyone including the registery fix.

I checked serveral Windows 10 installations at work incl. the one I use at work and I haven't found anything and a lot of my customers are running Windows 11 so hope they are more safe than Windows 10 users.

the54thvoidIt's blaming Crowdstrike.

Correct.

#18

Kn0xxPT

and then MS wants people move to Cloud Based systems .....
... lets ruin everyones day because of a "online bug" ....

#19

Chomiq

AssimilatorBut they are capitalist companies, and capitalism is all about maximising profits, and that means buying as little equipment as possible and hiring as few people as possible.

You're expecting the anti-Microsoft crowd to be capable of basic reading comprehension...

It still implies it's somehow to do with Windows Update, though. A better headline would be "Windows security vendor releases faulty product update, hits banks and airlines around the world".

Yeah, pretty much every news headline has MS logo in it but yeah, this is a CrowdStrike issue.

#20

DeathtoGnomes

Someone at CrowdStrike is getting bent over the desk.

I bet IT people will be creating better recovery drives/discs after this blunder. I wonder if they heard of a thing called, QA.

"this wouldnt have happened if AI did all the work"

#21

katzi

CrackongWindows Update breaks things.

First time? . jpg

Imagine blaming microsoft/windows update, for a 3rd party security software bricking windows.

#22

Onasi

Kn0xxPTand then MS wants people move to Cloud Based systems .....
... lets ruin everyones day because of a "online bug" ....

*sigh* It’s not MS. It’s not about cloud based OS. It’s about a separate, non-MS affiliated cybersecurity firm pushing out a scuffed update for their endpoint enterprise solution that corrupted Windows boot-loader for their clients. It literally doesn’t affect anyone on a consumer level personally.

#23

efikkan

Running Windows on critical systems, especially connected to the Internet and receiving automatic updates causing a global outage of numerous services, who saw this one coming? (trigger warning: sarcasm)

Having client PCs go offline may not be surprising, but seeing banks, traders, airlines, media companies etc., having their central services being offline from an update, that's just ridiculous. Come on guys, it's not 1995 any more, this level of incompetence isn't excusable. If you're making billions you can afford having properly trained staff and a properly managed tech "stack" with whatever appropriate failovers, backups, recovery images/procedures, etc. is needed to ensure reliability and security.

And yes, Microsoft certainly deserves blame for how easily their systems break, and for how tedious it is to roll back.

Thanks to CrowdStrike for accidentally revealing which companies who doesn't know how to handle their tech.

#24

Daven

Is it just me or do others think critical IT and society infrastructure services need to switch from Windows to Linux?

I don’t want this to be last thing I see before I die.

#25

ZoneDymo

katziImagine blaming microsoft/windows update, for a 3rd party security software bricking windows.

So you feel MS is in no way to blame? arnt they the ones who have a contract with this firm? is it not up to MS to check and verify this crap before letting it through?

This seems the mentality that lets CEO's that make a complete hash of it, the very thing they are paid WAY TOO MUCH for, leave with a "golden handshake".

Add your own comment

Faulty Windows Update from CrowdStrike Hits Banks and Airlines Around the World

234 Comments on Faulty Windows Update from CrowdStrike Hits Banks and Airlines Around the World

Latest GPU Drivers

New Forum Posts

Popular Reviews

Controversial News Posts

Faulty Windows Update from CrowdStrike Hits Banks and Airlines Around the World

Related News

234 Comments on Faulty Windows Update from CrowdStrike Hits Banks and Airlines Around the World

Latest GPU Drivers

New Forum Posts

Popular Reviews

Controversial News Posts