Predictive Analytics and Artificial Intelligence in Surgery—Opportunities and Risks

Predictive analytics and artificial intelligence (AI) are being applied in hospitals across the world to aid in clinical decision support, discuss risk of certain procedures with patients, and identify patients whose clinical status is deteriorating. Researchers are using massive amounts of data and recent advances in machine learning to improve surgical quality and patient outcomes. The Surgical Outcomes Club, a consortium of surgeons and health services researchers who advance the science of surgical outcomes research, convened a panel of 4 experts at the 2022 meeting who highlighted their research and experience using predictive analytics and AI in surgical research. Three main areas of AI in surgery were discussed: computer vision, digital transformation at the point of care, and electronic health records data. They discussed the opportunities and risks of these areas of AI in surgery, and in this Viewpoint, we expand on that discussion.

Computer Vision

Video is a by-product of minimally invasive and robotic surgery and is increasingly being collected for open procedures. The potential exists for real-time annotation of video streams by automated algorithms to track surgical performance, identify complex anatomy, and provide feedback to reduce technical errors. In parallel to the growth of video-based analysis by human reviewers, computer vision for surgery is also useful for surgical skills training¹ and review of surgeon behavior. Annotation of surgical phases and tracking of surgical tools and hands² will allow coaches to evaluate this skill with quantifiable metrics that would otherwise require tedious human video annotation and qualitative review. Furthermore, advances in statistical methods applied to video data, such as convolutional neural networks, have tremendously improved the capability of robotic procedures—essentially giving computers the ability to see.³

Surgical vision tools can augment the trajectory of surgeons in training and capabilities of human operators. However, the high difficulty of operations makes AI for surgical computer vision difficult. Simulated or toy examples where performance is high do not yet generalize to real-world complexity. Overcoming these limitations of generalizability is hampered by inconsistent video annotation and lack of large and diverse data sets. Institutional interests and data sharing limitations have prevented creation of a large open data set drawn from many institutions that would be necessary to train robust algorithms. Surgical video data from publicly available sites are being used for training models, but the quality of these video sources is variable. A recent consensus statement⁴ suggested that a 10-year time horizon is likely for real-time implementation of these algorithms, and retrospective training tools may be available in the next 2 years. Appreciating the nascent nature of surgical vision will allow it to develop as a collaboration that will ensure that tools work in concert with surgeons and ultimately transform surgical care.

Digital Transformation at the Point of Care

Advances in AI, including the ability to harness large data sets, have now made it possible to apply advanced analytics to a range of optimizing care provided in the intensive care unit and at the point of care in high-intensity environments, including the trauma bay. These time-pressured situations, where complex information streams are plentiful and the stakes high for making the right decision, have significant promise for the application of AI technology. Signal processing approaches can theoretically now convert multimodality data streams coming from monitors, point-of-care testing devices, imaging, laboratory, and real-time documentation by bedside clinicians into early warning signals. These signals can draw attention of the clinicians to patients who seemingly look well but are in fact, off trajectory. The net effect is to augment clinician decision-making through leveraging technology advances. Although still in its infancy, there are several demonstration projects that have been recently published showing AI assistance in detecting early respiratory deterioration, guiding fluid resuscitation, earlier detection of sepsis or critical illness, and identification of potentially life-threatening findings on radiographic imaging. Each of these requires careful implementation to avoid unintended consequences, ongoing monitoring when in clinical environments to ensure appropriate outcomes, and external validation before reaching clinical production deployment.

Electronic Health Records Data

Electronic health record (EHR) data are frequently used for development of AI in quality improvement and surgical outcomes research. Preoperative risk estimation using EHR data is currently implemented in some institutions to inform discussions between surgeons and patients prior to undergoing an operation. Moreover, given the high cost and resources required for postoperative complication surveillance and reporting, such as that of the National Surgical Quality Improvement Program and the National Health and Safety Network, some institutions have begun to replace manual medical record review with AI. This affords the opportunity to report complication rates at the level of the surgeon specialty and individual surgeon on a frequent basis—a task that would be impossible to accomplish without AI. There are many advantages to using HER data. They almost always use common data models and universal codes, such as International Classification of Diseases or Current Procedural Terminology. These universal coding systems allow researchers to produce generalizable results; however, due to their high dimensionality, they can be difficult to work with. Recently, mappings of these codes to larger disease classes, such as PheCodes,⁵ have greatly improved the ability to work with HER data and produce generalizable results. Additionally, when structured EHR data are limited, unstructured or narrative text data can be used to fill in the gaps. There are many examples of applying natural language processing (NLP) to the EHR notes, for example to identify postoperative complications or symptoms of a urinary tract infection. NLP methods have advanced tremendously over the last decade, and pretrained models for EHR data have proven to be extremely useful, such as clinical Bidirectional Encoder Representations From Transformers (BERT).⁶ However, there are some risks with using HER data for AI. The data can introduce bias in the AI system. If data are missing due to lack of access to care, poor recording, or lost to follow-up of a patient, conclusions will be incorrect and ungeneralizable. Furthermore, sicker patients tend to be included in hospital EHR data, so collider bias can introduce distorted associations between risk factors and clinical outcomes. It is therefore paramount to familiarize yourself with the source data prior to developing the AI. Moreover, the limitations of EHR data are relevant to most clinical data sources, including those used in computer vision and digital transformation at the point of care. These limitations should always be addressed in publications and the conclusions should be appropriately tempered. Finally, not all data from the HER may be used for development of AI, and there is some ethical debate about whether informed consent should be sought for the use of EHR data for AI.⁷

Conclusions

Predictive analytics and AI in surgery have improved processes and quality of care. Significant impacts have been in early detection of patient deterioration, reduction in costs and manual labor of surveillance and reporting of complications, and training algorithms to analyze surgical execution. However, because the development of AI in surgery relies heavily on limited data sets with inherent biases and limitations, a critical evaluation of sources of bias in the data should be carried out and monitored prospectively. Finally, conclusions should always reflect the precision in the findings when developing AI in surgical outcomes and health services research.