Doctors, nurses and hospital administrators around the globe were going into panic mode last week as they raced to manage the consequences of the largest IT outage in history.
MedicalBrief notes that while the Microsoft outage, believed to be caused by a software update and which affected computers worldwide, had limited impact on the healthcare system in South Africa.
WIRED reports that Mass General Brigham, one of America’s biggest health care systems, cancelled all non-urgent surgeries, procedures and medical visits. In the UK, Royal Surrey NHS Foundation Trust declared a critical incident affecting the systems used to deliver radiotherapy treatments. Hospitals in Canada, Germany, and Israel announced issues with their digital services, while the 911 emergency service in some US states was reported to be down.
The extent of the disruption appeared to vary both between and within health care systems. “Our hospital is fully down due to #Crowdstrike issue,” Dana Chandler, a nurse at GBMC HealthCare in Maryland, posted on X. “No phones, no computers, no safety nets. It’s an all-hands-on-deck kind of day. I hope our patients remain safe.” Rosenberg says that at Michigan Medicine, where he was awake since 1 am dealing with the crisis, anywhere from 15 to 60% of the computers were not working, depending on the unit.
“The impact is massive,” he said. “It affects all aspects of modern digital health systems. Luckily, in units where the computers are running the whole time, like the ICUs and emergency departments, the computers didn’t take the CrowdStrike application upgrade, whereas in areas of health care which are more episodic, like operating rooms, the disruption is much greater.”
Rosenberg said that the areas of greatest disruption have been so-called “digital bottlenecks,” which require communication between multiple computer systems. He gives the example of the critical practice of cleaning, disinfecting, and sterilising medical devices and patient care supplies. This is monitored through digital tools across several computers, to ensure that best practices are followed and the risk of potentially lethal infections is minimized.
“If one of those computers is affected, suddenly all of your sterilisation procedures have to slow down or even stop, and then operations stop,” he said.
With large health care systems employing thousands of personnel and looking after vast numbers of patients, modern health care has become reliant on digitisation as a matter of necessity, from systems which relay communications between busy departments to electronic medical records, or EMRs, which store vital information about individual patients.
But in recent years, concerning reports have emerged about the potential consequences of those systems breaking down. Studies have shown that during electronic medical record downtime, laboratory testing results are delayed by an average of 62% compared with normal operations, while in the NHS, IT failings have been directly linked with cases of patient harm.
In April, Sofia Mettler, at the time a resident physician at Mount Auburn Hospital, published a paper in JAMA Internal Medicine in which she described a day where the hospital’s EMR system was down for a period of seven to eight hours. The disruption meant that samples for morning lab tests were unable to be collected because the phlebotomy team did not know which patient needed which tests, while the results of tests conducted before the downtime could not be disseminated, making it harder to assess overnight progress.
Mettler, now a pulmonary and critical care fellow at Brigham and Women's Hospital, said that experience paled in comparison to the consequences of the CloudStrike outage.
“This time, the extent of the system downtime is way more profound,” she said. “We are currently unable to use any software that relies on digital data transmission. For example, we are unable to review CT scans, because the radiology software is down as well. It is difficult to make clinical decisions without access to what has become the essential part of medicine. We are using bedside ultrasound machines, but it is not nearly as good as CT scans in telling us what is going on in the lungs.”
However, in the wake of this incident, Rosenberg believes that health care systems need to diversify the software they use, especially in critical areas, to make them as resilient as possible.
“I think more reliance on diverse cloud computing will help us, because I would suspect in a modern cloud environment, they would have different timings of upgrades, for just this reason,” he said. “So part of the trick would be to run your systems in cloud operators, and stagger upgrades, so when these things happen, you can still run your systems from a different data center while the other one is fixed.”
In the UK, doctors and pharmacies warned customers of delays in issuing prescriptions and “slower than usual” service as they cope with backlogs.
On Tuesday, patients waiting for medicines were being told that emergency cases were being prioritised, as the health service attempts to deal with the fallout from Friday’s chaos.
Nick Kaye, the chairman of the National Pharmacy Association, told The Telegraph: “As pharmacists recover from last week’s IT outage and catch up on the backlog of prescriptions, we expect service in some community pharmacies to be slower than usual today.
“They may still be prioritising emergency prescriptions from their local GP surgeries as well as experiencing increased demand as services return to normal.”
GPs have also warned of continued delays this week as a result of the backlog.
The British Medical Association (BMA) said doctors were forced to cancel appointments on Friday, which would have an impact into this week.
Mondays are normally the busiest days for doctors’ surgeries, and the start of the week would be “particularly stretched because many patients who should have been dealt with last week would now need attention”.
Dr David Wrigley, the BMA deputy chairman, said: “Friday was one of the toughest single days in recent times for GPs across England. Without a clinical IT system, many were forced to return to pen and paper to be able to serve their patients.
“While GPs and their teams worked hard to look after as many as they could, without access to the information they needed much of the work has had to be shifted into the coming week.”
‘NHS playing catch up’
The loss of the EMIS patient record system, in particular, had caused “a considerable backlog”, Wrigley added.
The outage was caused by a faulty software update produced by CrowdStrike, a cyber security company, for PCs running Microsoft Windows.
The problem was estimated to have affected 8.5m devices worldwide. On Sunday, Microsoft released a recovery tool to aid the repair of PCs that use Windows.
CrowdStrike has issued a fix for the bug in the software update, saying, however, it would take “some time” for systems to be fully restored.
The Telegraph article – Pharmacy prescriptions delayed after IT outage (Restricted access)
See more from MedicalBrief archives:
NHS warns 100,000 Scottish patients of health data hack
Cover-up claims over NHS data breach affecting thousands
UK pharmacists to treat for common ailments, cutting out GPs