Hospitals around the world are struggling with the aftermath of IT
GBMC HealthCare Down Due to the CrowdStrike Crisis: All Hands On-Device Detection, Help, and Support
The extent of the disruption varied between healthcare systems. “Our hospital is fully down due to #Crowdstrike issue,” Dana Chandler, a nurse at GBMC HealthCare, posted on X. “No phones, no computers, no safety nets. It’s an all hands on-deck kind of day. I hope our patients are safe. Rosenberg says that at Michigan Medicine, where he was awake since 1 am dealing with the crisis, anywhere from 15 to 60 percent of the computers were not working, depending on the unit.
CrowdStrike’s mission is to find and prevent security breeches, so it bills itself as having the “fastest mean time” to detect threats. Since its launch in 2011, the Texas-based company has helped investigate major cyberattacks, such as the Sony Pictures hack in 2014, as well as the Russian cyberattacks on the Democratic National Committee in 2015 and 2016. As of Thursday evening, CrowdStrike’s valuation was upwards of $83 billion.
How quickly can systems get afloat after a big IT outage? Lukasz Olejnik, researcher, consultant, and author, writes in The Verge
The update in question appears to have installed faulty software onto the core Windows operating system, causing systems to get stuck in a boot loop. While showing an error that says it looks like windows didn’t load correctly, systems are giving users the option to restart the PC. Many companies, including this airline in India, have resorted to the good old-fashioned way of doing things by hand.
“Our software is extremely interconnected and interdependent,” Lukasz Olejnik, an independent cybersecurity researcher, consultant, and author of the book Philosophy of Cybersecurity, tells The Verge. “But in general, there are plenty of single points of failure, especially when software monoculture exists at an organization.”
It won’t be easy to get things up and running after CrowdStrike deploys a fix. Olejnik tells The Verge that this issue could take “days to weeks” to resolve because IT administrators may have to have physical access to a device to get them working again. The size and resources of a company’s IT team have an effect on how fast that happens. “Some systems in certain specific circumstances may be unrecoverable, but I assume that the majority will be recovered,” Olejnik adds.
Around the globe, doctors, nurses and hospital administrators were going into panic mode as they raced to manage the consequences of the largest IT outage in history. Mass General Brigham, one of America’s biggest healthcare systems, cancelled all non-urgent surgeries, procedures, and medical visits. In the UK, Royal Surrey NHS Foundation Trust declared a critical incident affecting the systems used to deliver radiotherapy treatments. Hospitals in Canada, Germany, and Israel announced issues with their digital services, while the 911 emergency service in some US states was reported to be down. A WIRED reporter found both Baylor hospital network, one of the largest non-profit healthcare systems in the country, and Quest Diagnostics unable to process routine bloodwork. The cause of the lack of internet was that calls were still being made, which meant officers had to be dispatched manually.
It soon became clear that this was not an isolated incident. A company called CrowdStrike made a routine update to its product which it utilized by a lot of companies. That update contained a bug, an error that caused all computers running the software on a Windows operating system to crash.
“The impact is massive,” he says. “It affects all aspects of modern digital health systems. Luckily in units where the computers are running the whole time, like the ICUs and emergency departments, the computers didn’t take the CrowdStrike application upgrade, whereas in areas of healthcare which are more episodic like operating rooms, the disruption is much greater.”
Several major healthcare systems in the US, UK and Canada have reported outages due to a bug in a Windows software update that was widely distributed by cybersecurity company CrowdStrike. Mass General Brigham and Women’s Hospital, one of America’s biggest healthcare systems, cancelled all non-urgent surgery, procedures and medical visits and declared a critical incident affecting systems used to deliver radiotherapy treatments.