Operating rooms are a precious resource. They may account for 50% of a US hospital's revenues and cost as much as $80 a minute. A four-year modelling study at the University of Washington improved the accuracy of surgical-case duration estimates from 30% to 50%.
Figuring out how much time to allot for a surgery is a challenge every hospital faces. "OR scheduling is a $5bn problem. To optimise the OR, you have to answer a fundamental question: How long does each surgery take?" said co-lead author Rajeev Saxena, an anaesthesiology resident at the University of Washington School of Medicine. "Under-utilisation means fewer patients get surgical care and the hospital has excess capacity. Over-utilisation results in cancelled operations and overtime expenses."
To try to improve the prediction of surgical-case duration, Saxena, in collaboration with physicians in surgery and anesthesia, scientists, and informatics experts inside and outside University of Washington, created machine-learning models for each surgical specialty and for individual surgeons.
The researchers collected data from over 45,000 surgeries over four years performed by 92 surgeons. Their surgeon-specific models were able to improve accuracy from 30% (based on a surgeon's estimate) to 40%. Among the top-third of surgeons, accuracy improved to more than 50%.
As more data is inputted, the model will improve over time, noted co-lead author Matthew Bartek, chief resident in general surgery at the UW School of Medicine.
"If we can improve the data, we can zero in on more accurate estimates," Bartek said. "This is just the first step."
To create their estimates, researchers gathered data from multiple electronic medical records on the patient, the type of surgery, the surgery personnel, and the surgery scheduling information. Only preoperative data was used.
Of all the data captured, the greatest variability was between surgeons. "Each surgeon has a unique approach to an operation and this data confirmed it," said senior author Bala Nair, former director of a UW School of Medicine technical centre that looked at improving care through informatics and technology solutions. "This work is bringing operating rooms into the 21st century by applying modern data science methods to improve operations," he said.
"For years, operating rooms have been relying on surgeon's estimates for operating times. But surgeons usually greatly underestimate the time of procedures," said senior author John Lang, clinical director of operative and perioperative operations at UW Medicine.
Lang said two hospitals in the UW Medicine system are rated by the Medicare Case Mix Index highly for complex surgeries. UW Medical Centre is rated No 3 and Harborview Medical Centre is rated No 13. Because of this, accurate prediction of surgical case duration is especially challenging and impactful.
Said Saxena: "You can change an entire organisational culture by taking a data-forward approach and engaging key stakeholders."
Other collaborators on this study included Stuart Solomon, an anaesthesiologist at UW Medicine and Christine Fong, an informatics engineer at UW Medicine. The work was a collaboration with Perimatics, a Bellevue, Washington-based data science company.
Background: Accurate estimation of operative case-time duration is critical for optimizing operating room utilization. Current estimates are inaccurate and prior models include data not available at the time of scheduling. Our objective was to develop statistical models in a large retrospective dataset to improve estimation of case-time duration relative to current standards.
Study Design: We developed models to predict case-time duration using linear regression and supervised machine learning (ML). For each of these models, we generated: an all-inclusive model, service-specific models, and surgeon-specific models. In the latter two approaches, individual models were created for each surgical service and surgeon, respectively. Our dataset included 46,986 scheduled surgeries performed at a large academic medical center from January 2014 to December 2017, with 80% used for training and 20% for model testing/validation. Predictions derived from each model were compared to our institutional standard of using average historical procedure times and surgeon estimates. Models were evaluated based on accuracy, overage (case duration > predicted + 10%), underage (case duration < predicted – 10%), and the predictive capability of being within a 10% tolerance threshold.
Results: The ML algorithm resulted in the highest predictive capability. The surgeon-specific model was superior to the service-specific model, with higher accuracies, lower percentage of overage and underage, and higher percentage of cases within the 10% threshold. The ability to predict cases within 10% improved from 32% using our institutional standard to 39% with the ML surgeon-specific model.
Conclusion: Our study is a notable advancement towards statistical modeling of case-time duration across all surgical departments in a large tertiary medical center. Machine learning approaches may improve case duration estimations, enabling improved OR scheduling, efficiency, and reduced costs.
Matthew A Bartek, Rajeev C Saxena, Stuart Solomon, Christine T Fong, Lakshmana D Behara, Ravitheja Venigandla, Kalyani Velagapudi, John D Lang, Bala G Nair