Friday, July 19th 2024

Faulty Windows Update from CrowdStrike Hits Banks and Airlines Around the World

A faulty software update to enterprise computers by cybersecurity firm CrowdStrike has taken millions of computers offline, most of which are in a commercial or enterprise environment, or are Azure deployments. CrowdStrike provides periodic software and security updates to commercial PCs, enterprise PCs, and cloud instances, with a high degree of automation. The latest update reportedly breaks the Windows bootloader, causing bluescreens of death (BSODs), and if configured, invokes Windows Recovery. Enterprises tend to bulletproof the bootloaders of their client machines, and disable generic Windows Recovery tools from Microsoft, which means businesses around the world are left with large numbers of machines that will each take manual fixing. The so-called "Windows CrowdStrike BSOD deluge" has hit critical businesses such as banks, airlines, supermarket chains, and TV broadcasters. Meanwhile, sysadmins on Reddit are wishing each other a happy weekend.
Source: The Verge
Add your own comment

234 Comments on Faulty Windows Update from CrowdStrike Hits Banks and Airlines Around the World

#1
Anoniem
Well, happy weekend other sys admins out there :) Glad we don’t use CrowdStrike services/solutions.
Posted on Reply
#2
wolf
Better Than Native
Complete meltdown in Perth WA. Banks offline, grocery stores offline, a bloke couldn't get his zinger box from KFC after he knocked off. Triple J (radio) playing back to back bangers from a USB drive.

Feels like just about the only place not impacted was my workplace :fear:
Posted on Reply
#3
N3utro
Good update management in a medium to large scale enterprise is to have multiple rings of deployment: the IT lab, the IT department users, non critical departments, critical ones. You never deploy windows updates directly, you manage your rings with WSUS. Bad management from these companies.
Posted on Reply
#4
Assimilator
This doesn't just affect sysadmins, it affects anyone who uses a third party that uses Crowdstrike... ASK ME HOW I KNOW.

Microsoft's 365 platform is also having a bit of a wobble due to a seemingly unrelated issue with a configuration SNAFU in Azure.
N3utroGood update management in a medium to large scale enterprise is to have multiple rings of deployment: the IT lab, the IT department users, non critical departments, critical ones. You never deploy windows updates directly, you manage your rings with WSUS. Bad management from these companies.
The whole point of providers like Crowdstrike is that part of the service fee is for them to do that verification, so that you don't have to. Without such providers, small companies with minimal IT departments couldn't exist.
Posted on Reply
#5
N3utro
AssimilatorThis doesn't just affect sysadmins, it affects anyone who uses a third party that uses Crowdstrike... ASK ME HOW I KNOW.

Microsoft's 365 platform is also having a bit of a wobble due to a seemingly unrelated issue with a configuration SNAFU in Azure.


The whole point of providers like Crowdstrike is that part of the service fee is for them to do that verification, so that you don't have to. Without such providers, small companies with minimal IT departments couldn't exist.
"banks, airlines, supermarket chains," these are not small companies
Posted on Reply
#6
cerulliber
Monday hirings QA at Crowdstrike and get 10x ROI. share prices increased. problem fixed, just don't be greed.
Posted on Reply
#7
Crackong
Windows Update breaks things.

First time? . jpg

Posted on Reply
#8
WonkoTheSaneUK
Pour one out for sysadmins, who have just learned that the fix is to log into each affected PC one at a time and delete the single bad file from each one.
It's going to be a loooooooooooooooooooooooooooooooooooooooong day for those in bigger organizations!
Posted on Reply
#9
Chaitanya
MS needs a good screwing over and Windows needs to be offline like good old Win 7(and older).
Posted on Reply
#10
AnarchoPrimitiv
N3utroGood update management in a medium to large scale enterprise is to have multiple rings of deployment: the IT lab, the IT department users, non critical departments, critical ones. You never deploy windows updates directly, you manage your rings with WSUS. Bad management from these companies.
Wouldn't be surprised if they fired a bunch of the people responsible for that to increase quarterly earnings.
Posted on Reply
#11
Onasi
CrackongWindows Update breaks things.
ChaitanyaMS needs a good screwing over and Windows needs to be offline like good old Win 7(and older).
I love to bag on MS just as the next guy, but it isn’t about them this time around.
Posted on Reply
#12
JcRabbit
Judging by the chaos out there, this is what the Y2K bug could have been, but wasn't (because we made sure on time that it would not turn into anything like this - and thus it became a non event).
Posted on Reply
#13
ErikG
Solution:
  1. Boot Windows into Safe Mode or the Windows Recovery Environment
  2. Navigate to the C:\Windows\System32\drivers\CrowdStrike directory
  3. Locate the file matching “C-00000291*.sys”, and delete it.
  4. Boot the host normally.
Posted on Reply
#14
Dazz023
CrackongWindows Update breaks things.

First time? . jpg

Just clickbait headline to attract anti-MS crowd.
Posted on Reply
#15
the54thvoid
Super Intoxicated Moderator
Dazz023Just clickbait headline to attract anti-MS crowd.
Not really - the header implicitly says,
Faulty Windows Update from CrowdStrike
It's blaming Crowdstrike.
Posted on Reply
#16
Assimilator
N3utro"banks, airlines, supermarket chains," these are not small companies
But they are capitalist companies, and capitalism is all about maximising profits, and that means buying as little equipment as possible and hiring as few people as possible.
OnasiI love to bag on MS just as the next guy, but it isn’t about them this time around.
You're expecting the anti-Microsoft crowd to be capable of basic reading comprehension...
the54thvoidNot really - the header implicitly says,


It's blaming Crowdstrike.
It still implies it's somehow to do with Windows Update, though. A better headline would be "Windows security vendor releases faulty product update, hits banks and airlines around the world".
Posted on Reply
#17
Shou Miko
ErikGSolution:
  1. Boot Windows into Safe Mode or the Windows Recovery Environment
  2. Navigate to the C:\Windows\System32\drivers\CrowdStrike directory
  3. Locate the file matching “C-00000291*.sys”, and delete it.
  4. Boot the host normally.
I already heard this didn't work for everyone including the registery fix.

I checked serveral Windows 10 installations at work incl. the one I use at work and I haven't found anything and a lot of my customers are running Windows 11 so hope they are more safe than Windows 10 users.
the54thvoidIt's blaming Crowdstrike.
Correct.
Posted on Reply
#18
Kn0xxPT
and then MS wants people move to Cloud Based systems .....
... lets ruin everyones day because of a "online bug" ....
Posted on Reply
#19
Chomiq
AssimilatorBut they are capitalist companies, and capitalism is all about maximising profits, and that means buying as little equipment as possible and hiring as few people as possible.


You're expecting the anti-Microsoft crowd to be capable of basic reading comprehension...


It still implies it's somehow to do with Windows Update, though. A better headline would be "Windows security vendor releases faulty product update, hits banks and airlines around the world".
Yeah, pretty much every news headline has MS logo in it but yeah, this is a CrowdStrike issue.
Posted on Reply
#20
DeathtoGnomes
Someone at CrowdStrike is getting bent over the desk.

I bet IT people will be creating better recovery drives/discs after this blunder. I wonder if they heard of a thing called, QA.

"this wouldnt have happened if AI did all the work"
Posted on Reply
#21
katzi
CrackongWindows Update breaks things.

First time? . jpg

Imagine blaming microsoft/windows update, for a 3rd party security software bricking windows.
Posted on Reply
#22
Onasi
Kn0xxPTand then MS wants people move to Cloud Based systems .....
... lets ruin everyones day because of a "online bug" ....
*sigh* It’s not MS. It’s not about cloud based OS. It’s about a separate, non-MS affiliated cybersecurity firm pushing out a scuffed update for their endpoint enterprise solution that corrupted Windows boot-loader for their clients. It literally doesn’t affect anyone on a consumer level personally.
Posted on Reply
#23
efikkan
Running Windows on critical systems, especially connected to the Internet and receiving automatic updates causing a global outage of numerous services, who saw this one coming? (trigger warning: sarcasm)

Having client PCs go offline may not be surprising, but seeing banks, traders, airlines, media companies etc., having their central services being offline from an update, that's just ridiculous. Come on guys, it's not 1995 any more, this level of incompetence isn't excusable. If you're making billions you can afford having properly trained staff and a properly managed tech "stack" with whatever appropriate failovers, backups, recovery images/procedures, etc. is needed to ensure reliability and security.

And yes, Microsoft certainly deserves blame for how easily their systems break, and for how tedious it is to roll back.

Thanks to CrowdStrike for accidentally revealing which companies who doesn't know how to handle their tech.
Posted on Reply
#24
Daven
Is it just me or do others think critical IT and society infrastructure services need to switch from Windows to Linux?

I don’t want this to be last thing I see before I die.
Posted on Reply
#25
ZoneDymo
katziImagine blaming microsoft/windows update, for a 3rd party security software bricking windows.
So you feel MS is in no way to blame? arnt they the ones who have a contract with this firm? is it not up to MS to check and verify this crap before letting it through?

This seems the mentality that lets CEO's that make a complete hash of it, the very thing they are paid WAY TOO MUCH for, leave with a "golden handshake".
Posted on Reply
Add your own comment
Nov 21st, 2024 04:16 EST change timezone

New Forum Posts

Popular Reviews

Controversial News Posts