Predicting Patient-Reported Outcomes Following Surgery Using Machine Learning

Abstract

Patient-reported outcomes (PROs) enable providers to identify differences in treatment effectiveness, postoperative recovery, quality of life, and patient satisfaction. By allowing a shift from disease-specific factors to the patient perspective, PROs provide a tailored patient-centric approach to shared decision-making. Artificial intelligence (AI) and machine learning (ML) techniques can facilitate such shared decision-making and improve patient outcomes by accurate prediction of PROs. This article aims to provide a comprehensive review of the use of AI and ML models in predicting PROs following surgery through an overview of common predictive algorithms and modeling techniques, as well as current applications and limitations in the surgical field.

Introduction

Surgical quality has been traditionally evaluated using objective measures such as mortality, hospital readmission, and postoperative complication rates.^1,2 However, these measures are frequently not the best attribute to assess quality for specific procedures and are usually not sensitive enough to detect differences in patient outcomes.² Patient-reported outcomes (PROs), in which a patient self-reports their satisfaction and quality of life after a procedure, enable providers to identify differences in treatment effectiveness.³ PROs are also useful for assessing differences in patient outcomes when comparing surgical interventions with comparable objective data such as survival or complications rates.⁴ As they represent a shift from disease-specific factors to the patient perspective, PROs provide a tailored patient-centric approach to shared decision-making.

The ability of providers to predict PROs, notably the effectiveness of an intervention, satisfaction, and post-operative recovery, is the foundation of informed shared decision-making. Predictive modeling using artificial intelligence (AI) and machine learning (ML) techniques can facilitate such shared decision-making and lead to improved patient satisfaction by accurate prediction of perioperative outcomes.^5–8 It is critical for surgeons to understand the available predictive modeling approaches of PROs in order to adequately appraise and demonstrate the effectiveness of a chosen procedure. To this end, this article provides a comprehensive review of the use of AI and ML models in predicting PROs following surgery through an overview of common predictive algorithms and modeling techniques, as well as current applications and limitations in the surgical field.

Modeling Techniques

Model Development

Previous studies used several ML algorithms to predict PROs including the Cox Lasso, random forest, neural networks, k-means clustering, generalized additive model (GAM), and gradient boosted model (GBM). A brief description of the algorithms is shown in Table 1.^9–13 A detailed overview of the algorithms, model development, performance metrics, and interpretation is previously published by our research group.¹⁴

Table 1.

Description of Machine Learning Algorithms Used for Predicting Patient-Reported Outcomes.

Algorithm	Description
Neural networks	The algorithm comprises three basic layers: Input, hidden, and output layers. The input layer has several nodes (neurons) representing original predictors. The algorithm uses the information from the input layer to form one or more hidden layers with any number of nodes assigned a weight. Next, the algorithm transforms the input data when they pass through the hidden layer to the output layer.
Random forest	The algorithm comprises many de-correlated decision trees that jointly determine the final prediction. The algorithm builds many decision trees on randomly bootstrapped copies of the training data and then uses the aggregated predictions from all the trees as the final output.
Extreme gradient boosting	The algorithm combines the prediction power of several models. The algorithm starts with building a weak model and sequentially trains many models, with each model learning and improving by addressing the biggest mistake that its previous model makes.
LASSO regression	Regularized regression models use functions such as least absolute shrinkage and selection operator (LASSO), ridge regression, and elastic net (a linear combination of ridge and LASSO regularization). Aside from decreasing the risk of bias and increasing likelihood of generalizability to new data sets, regularized regression generates coefficients that can be interpreted for feature importance.
Generalized additive model	The algorithm uses a regression model that allows for non-linear relationships between predictors and the outcome. GAM models produce reliable results and accurately estimate the variance, particularly in large databases.
K-means clustering	The algorithm is an unsupervised learning algorithm that finds cluster structure in a data set based on the greatest similarity within the same cluster and the greatest dissimilarity between clusters.

Minimal Clinically Important Difference

The minimal clinically important difference (MCID) is a patient-centered measure that indicates the smallest amount that a PRO must change in order to make a meaningful difference to patients.^15,16 In most studies, the primary outcomes was prediction of whether a patient would experience either no improvement or better improvement in PROs at the final follow-up. Using these measures, studies often defined two types of outcomes: improved PROs, or stable PROs.

Applications in Surgery

Surgical Oncology

ML has been used in the field of oncologic surgery by Rossi et al¹⁷ to predict post-discharge lung and gastrointestinal cancer surgery complications via telemonitoring of PROs and patient-generated health data (PGHD). Post-discharge complications are known to pose significant cost and psychological burden to patients and providers.¹⁷ This group of researchers identified PROs as a useful tool in predicting post-surgical complications and aimed to utilize ML algorithms using PRO input to identify patients at risk of poor outcomes. ML algorithms were trained on data from 52 patients including clinical variables, PROs (MD Anderson Symptom Inventory [MDASI]), and PGHD (VivoFit). Their ML model was able to achieve an area under the curve (AUC) of .74 and predict post-operative complications with high accuracy utilizing PROs and PGHD.¹⁷ The ML models also demonstrated that a decrease in physical activity or a large increase in physical activity can lead to a higher complication risk after surgery. These findings will help in accurate monitoring and risk stratification of patients who are more or less likely to develop complications after oncologic surgery based on PROs.

Breast Reconstruction

PROs following breast reconstruction, such as satisfaction with breasts, are a vital part of measuring a patient’s quality of life and psychological well-being. Decision-making regarding breast reconstruction is currently reliant on population-based outcomes and provider preferences. In recent work,¹⁸ our group trained three ML algorithms utilizing data from 1921 women in North America who underwent mastectomy and breast reconstruction to accurately predict PROs at 1 year postoperatively. Changes in satisfaction with breasts were measured using the validated BREAST-Q tool. We found that all three ML algorithms predicted increased or decreased satisfaction with breasts with high accuracy for both the training (.81 [range = .73–.83]) and validation (.83 [range = .81–.84]) datasets. The models also achieved high AUCs of .86 (range = .83–.89) and .84 (range = .78–.85) for both datasets.¹⁸ At 1-year follow-up, 30% of patients in the study demonstrated dissatisfaction with their reconstructive outcome. These patients could have been identified by the ML algorithm before treatment and individually counseled on alternative reconstructive options.¹⁸ These findings will provide physicians with a tool to better inform patient-centered decision-making and offer individualized care for patients undergoing breast reconstruction.

In a subsequent study, our group evaluated the performance of the models on a cohort of 1553 women after mastectomy and breast reconstruction with a follow up of 2 years.¹⁹ Of study participants, 45.2% reported increased satisfaction with the appearance of their breasts, while 27.2% reported dissatisfaction.¹⁹ The three ML models were trained, tested, and validated on the additional data and achieved equally higher performance. Furthermore, the models demonstrated high accuracy for predicting both increased and decreased satisfaction with surgical outcomes (AUCs = .86–.87 and .84–.85), respectively.¹⁹ These findings demonstrated that ML can accurately predict long-term PROs after breast reconstruction.

Orthopedic Surgery

Huber et al²⁰ sought to augment shared decision-making by evaluating PROs after hip and knee arthroplasty. The group compared eight ML models trained on National Health Service data to predict PROs following hip or knee replacement. The EQ-5D-3 L visual analogue scale (VAS) and the Oxford Hip and Knee Scores (Q scores) were used to record symptomatic improvement. The most important predictors for PROs in hip and knee arthroplasty were found to be preoperative VAS, Q score, and specific instrument dimensions. Extreme gradient boosting produced the highest performance of the models and achieved AUCs of .87 (VAS) and .78 (Q score) for hip replacement and .86 (VAS) and .70 (Q score) for knee arthroplasty.²⁰ The authors also compared their ML model to the hip prediction tool used by NHS, which is a linear regression model. The findings demonstrated that supervised ML outperformed this linear model.²⁰ In a clinical setting, the ML models will encourage shared decision-making and provide patients with accurate predictions of relevant outcomes that affect their quality of life. Furthermore, these predictions will be individualized, which will discourage providers from relying on population-based outcomes.

Further applications include Stefano et al²¹ prospective trial predicting 6-week PROs using wearable sensor data after joint replacement. The group used ML to identify patients at higher risk of unsuccessful recovery after arthroplasty to better predict poor outcomes.²¹ Data was collected from fifteen patients using three different trackers from 4 weeks before to 6 weeks after surgery, as well as PROs before and after surgery. Three million data points were collected and three sets of features with the highest importance scores relative to PROs were identified. The k-means clustering algorithm helped accurately predict 6-week PRO data as early as 11 days after arthroplasty.²¹

While models have been developed to predict PROs based on objective data, ML algorithms have also been developed to predict outcomes based on PRO data. Martin et al²² utilized ML to predict subjective failure of anterior cruciate ligament (ACL) reconstruction. The model was developed to identify risk factors based on PROs that are associated with poor outcomes after ACL reconstruction.²² The ML model was trained on the Norwegian Knee Ligament Register (NKLR) to help develop a clinically useful tool for identifying patients at risk of subjective failure of ACL reconstruction. The 2-year follow-up questionnaire was completed by 11 630 patients, of which 22% reported subjective failure. Four ML models were trained on this data, including Cox Lasso, survival random forest, GAM, and GBM, that produced AUCs of .67–.68. The GAM model achieved the highest AUC (.68; 95%, CI: .64–.71) and was used to create an in-clinic, patient specific calculator to predict subjective failure of ACL reconstruction.²² The product of this study was an ML model that adequately predicted PROs after ACL reconstruction to better inform patients about individual expectations prior to surgery.

Spine Surgery

ML models have been developed in the field of spine surgery to predict PROs after lumbar discectomy. Discectomy is known to be successful in relieving pain from lumbar disc herniation, but 20–30% of patients still experience poor outcomes after surgery.²³ Staartjes et al²³ utilized ML to accurately identify patients at risk of experiencing poor outcomes. The study included 422 patients whose leg and back pain severity and Oswestry Disability Index scores were recorded at 1 year post-operatively. The group found that neural network algorithms demonstrated high accuracy (85, 87, and 75%) in predicting MCID with high performance (AUC = .87, .90, and .84).²³ Regression models were also tested and showed lower performance when compared to ML models. The ML models developed in this study allow providers to accurately counsel patients in the perioperative setting about the probability of poor outcomes after undergoing lumbar discectomy. This type of discussion fosters personalized patient care.

Limitations

There are several potential limitations to consider in studies that use ML to predict PROs. First, most studies used self-reported survey data, which can introduce recall bias and possibly imprecision. Second, the low response rate in previous studies may indicate selection bias, in which patients who are likely to report positive PROs, such as satisfaction after surgery, respond to the questionnaire. Non-responders may have significant systematic differences from respondents, limiting the models’ validity. Third, studies were non-randomized; the procedure of interest was chosen based on patient and surgeon preference, which limited generalizability and predictive accuracy. Finally, most studies included non-diverse patient populations with limited racial and ethnic diversity, which may introduce bias into the model and limit predictive performance in racially/ethnically diverse patients.

Conclusion

There has recently been a rapid increase in the use of AI and ML models in predicting PROs after surgery. Studies evaluating AI-driven predictive modeling techniques demonstrated high accuracy in predicting both short- and long-term PROs. These models have the potential to improve surgical outcomes by informing patients about their individual expectations and encouraging patient-centered decision-making.