Abstract
Objective
The aims of this study were to develop machine learning algorithms for preoperative prediction of prolonged opioid prescriptions after TKA and to identify variables that can predict the probability of this adverse outcome.
Methods
Five algorithms were developed for prediction of prolonged postoperative opioid prescriptions.
Results
The stochastic gradient boosting (SGB) model had the best performance. Age, history of preoperative opioid use, marital status, diagnosis of diabetes, and several preoperative medications were predictive of prolonged postoperative opioid prescriptions.
Conclusion
The SGB algorithm developed could help improve preoperative identification of TKA patients at risk for prolonged postoperative opioid prescriptions.
Keywords: Opioid, Machine learning, Prediction
1. Introduction
In the United States, opioid related drug overdoses nearly tripled during the period from 1999-2014.1 Almost 4 out of every 100 US adults misuse prescription opioids.2 The opioid epidemic is particularly pertinent to the field of orthopedics as these surgeons are the leading prescriber of opioids among surgical subspecialties,3,4 and prescribing postoperative opioid medications leads to increased susceptibility to long term opioid use after surgery.5,6 The following factors have already been identified as contributors to prolonged postoperative opioid use: preoperative opioid use, female gender, patients aged <50 years, greater length of stay, and worse health status.7, 8, 9
However, no preoperative algorithm exists that can serve as a tool to predict prolonged postoperative opioid prescriptions for total knee arthroplasty (TKA) patients. Karhade et al. developed an algorithm for this purpose in total hip arthroplasty (THA), however there remains a need for a similar preoperative tool in TKA patients.10 The utilization of machine learning, a subfield of artificial intelligence and statistics, can be used to develop predictive algorithms for such purposes and have been previously implemented in the field of orthopaedics.10,11 Thus, the purposes of this study were to (1) develop machine learning algorithms for preoperative prediction of prolonged opioid prescriptions after TKA and (2) to use the algorithm with the best performance to identify variables that can predict the probability of prolonged postoperative opioid prescriptions after TKA.
2. Methods
2.1. Data source
A retrospective study was conducted at a single health care system from January 1st 2000 to March 1st 2018 with institutional reviewed board approval. Inclusion criteria for the study were: (1) age 18 years or older, (2) operative indication of osteoarthritis, and (3) primary elective TKA. Exclusion criteria for the study were: (1) revision TKA procedures and (2) TKA performed for inflammatory arthritis, trauma, tumor, or infection.
2.2. Outcome
The primary outcome was prolonged postoperative opioid prescriptions. This was defined as continuous opioid prescriptions in the 30 days after surgery, opioid prescriptions in days 30–90 days after surgery, and opioid prescriptions between days 90–180 days after surgery. The list of opioids included is available for review in Supplementary Table 1.
2.3. Variables
The following variables were evaluated as candidate predictors for prolonged postoperative opioid use: patient demographics, disposition, laboratory values, insurance status, neighborhood (zip code) characteristics, medications, and comorbidities. Preoperative opioid use was represented by a binary variable of whether the patient had been exposed to opioids preoperatively or not. Medications were reviewed in the year immediately preceding the index procedure. The full list of variables is available for review in Table 1.
Table 1.
Baseline characteristics of total knee arthroplasty population, n = 12542.
| Variable | n (%) | median (IQR) |
|---|---|
| Demographics | |
| Age | 67.0 (60.0–74.0) |
| Female sex | 7559 (60.3) |
| Race | |
| Non-White | 1214 (10.0) |
| White | 10947 (90.0) |
| Ethnicity | |
| Hispanic | 268 (2.2) |
| Non-Hispanic | 11893 (97.8) |
| Marital-Status | |
| Married | 7575 (62.6) |
| Not-Married | 4527 (37.4) |
| Veteran | 1597 (13.6) |
| Disposition | |
| Inpatient | 11683 (93.2) |
| Outpatient | 859 (6.8) |
| Laboratory Values | |
| Hemoglobin (g/dL) | 13.6 (12.7–14.5) |
| White blood cell (x103/μL) | 6.90 (5.80–8.20) |
| Platelet (×103/μL) | 254.0 (213.0–301.0) |
| Creatinine (mg/dL) | 0.90 (0.76–1.05) |
| Medicaid | 529 (4.2) |
| Medicare | 6469 (51.6) |
| Neighborhood characteristics | |
| Median household income ($) | 80,375 (62,114–100,674) |
| Median age (years) | 41.9 (37.6–44.8) |
| High school education (%) | 24 (15–30) |
| Unemployment rate (%) | 5.7 (4.6–7.3) |
| Medications | |
| Angiotensin converting enzyme inhibitor | 1366 (10.9) |
| Angiotensin II receptor blocker | 655 (5.2) |
| Anti-depressant | 1191 (9.5) |
| Beta-2-agonist | 833 (6.6) |
| Beta-blocker | 1742 (13.9) |
| Benzodiazepine | 992 (7.9) |
| Gabapentin | 512 (4.1) |
| Immunosuppressant | 1431 (11.4) |
| NSAID | 2164 (17.3) |
| Opioid | 2631 (21.0) |
| Anti-psychotic | 205 (1.6) |
| Comorbidities | |
| Tobacco use | 174 (1.4) |
| Alcohol abuse | 85 (0.7) |
| Drug abuse | 76 (0.6) |
| Diabetes | 1182 (9.4) |
| Renal failure | 346 (2.8) |
| Depression | 803 (6.4) |
| Psychoses | 73 (0.6) |
| Myocardial infarction | 298 (2.4) |
| Congestive heart failure | 470 (3.7) |
| Peripheral vascular disease | 474 (3.8) |
| Cerebrovascular accident | 407 (3.2) |
| Chronic obstructive pulmonary disease | 936 (7.5) |
| Arrhythmias | 1506 (12.0) |
| Valvular disease | 612 (4.9) |
| Liver disease | 265 (2.1) |
| Malignancy | 551 (4.4) |
| Prolonged Postoperative Opioid Prescriptions | 1125 (9.0) |
2.4. Missing data
The rates of missing data were race 3.0% (381), ethnicity 3.0% (381), marital status 3.5% (440), veteran status 6.3% (795), hemoglobin 11.7% (1470), white blood cell 11.7% (1464), platelet 11.7% (1472), creatinine 14.7% (1847), median household income in neighborhood 1.6% (198), median age of neighborhood 1.3% (166), high school graduation rate in neighborhood 1.3% (155), and unemployment rate of neighborhood 1.3% (160).
2.5. Data analysis
An 80:20 split of the available population was created to develop the training and testing cohorts for model development. Recursive feature selection with random forest algorithms was undertaken to identify the subset of predictors used for final model development. Stochastic gradient boosting (SGB), random forest (RF), support vector machine (SVM), neural network (NN), and elastic-net penalized logistic regression (ENPLR) algorithms were developed for prediction of prolonged postoperative opioid prescriptions. The models were assessed on the testing set by discrimination (area under the receiver operating curve – AUC), calibration (calibration plot, calibration intercept, calibration slope), and overall performance (Brier score). The null model Brier score was calculated to establish a benchmark for the expected Brier score of an algorithm that predicted probabilities for every patient equal to the observed prevalence of the outcome in the study population. Decision curve analysis was undertaken for the best performing algorithm, and the expected net benefit of decisions based on the algorithm's predictions were compared to decisions made on the basis of preoperative opioid use alone and for the default strategies of making changes for all patients and for no patients. Relative variable importance plots were developed to examine the contributions of the predictor variables to the model outputs averaged across all patients. Local explanations were developed to explain predictions at the level of individual patients. Data analysis was conducted with the Anaconda Distribution (Anaconda, Inc., Austin, Texas), R (The R Foundation, Vienna, Austria), RStudio (RStudio, Boston, MA), and Python (Python Software Foundation, Wilmington, Delaware).
3. Results
3.1. Patient population
Overall, 12,542 patients underwent TKA for osteoarthritis and 1125 (9.0%) received prolonged postoperative opioid prescriptions. The median age was 67 years (interquartile range 60–74) and 7559 (60.3%) patients were female (Table 1). In the year before surgery, 2631 (21.0%) patients received opioid prescriptions.
3.2. Machine learning algorithms
Machine learning algorithms were developed to preoperatively predict prolonged opioid prescriptions after TKA, and cross-validation of the training set (n = 10034) was performed. The AUC, calibration intercept, calibration slope, and Brier score values were calculated (Supplementary Table 2). In the testing set (n = 2508), the AUC ranged from 0.54 (SVM) to 0.76 (SGB and ENPLR) (Table 2). The calibration intercept ranged from −0.80 to 0.18 and the calibration slope ranged from 0.27 to 1.09. The Brier score ranged from 0.073 to 0.082. On decision curve analysis, the model exhibited higher net benefit than the default strategies of changing management for all patients or no patients as well as decisions based only on preoperative opioid use status (Fig. 1B). Overall, the SGB model achieved superior performance across discrimination and calibration with AUC = 0.76, calibration intercept = 0.16, calibration slope = 1.08, and Brier score = 0.073 (Fig. 1, Fig. 2A).
Table 2.
Machine learning model performance assessment in the testing set, n = 2508.
| Metric | Stochastic Gradient Boosting | Random Forest | Support Vector Machine | Neural Network | Elastic-net Penalized Logistic Regression |
|---|---|---|---|---|---|
| AUC | 0.76 | 0.64 | 0.54 | 0.75 | 0.76 |
| Intercept | 0.16 | −0.80 | 0.04 | 0.14 | 0.18 |
| Slope | 1.08 | 0.27 | 0.92 | 1.09 | 1.09 |
| Brier | 0.073 | 0.082 | 0.080 | 0.073 | 0.073 |
(AUC): area under the receiver operating curve. Null model Brier score = 0.082.
Fig. 1.
These graphs show the (A) calibration plot and (B) decision curve analysis for the stochastic gradient boosting for prediction of prolonged postoperative opioid prescriptions in the testing set (n = 2508).
Fig. 2.
These graphs show the (A) receiver operating curve and (B) global variable importance for the stochastic gradient boosting for prediction of prolonged postoperative opioid prescriptions in the testing set (n = 2508).
3.3. Variables that predict the probability of prolonged postoperative opioid consumption
Variables determined for prediction of prolonged opioid prescriptions by SGB algorithms were age, history of preoperative opioid use, marital status, diagnosis of diabetes, and preoperative medications (antidepressants, benzodiazepines, nonsteroidal anti-inflammatory drugs, gabapentin, and beta-2-agonists) (Fig. 2B). The other investigated variables did not predict prolonged opioid prescription usage. An example of an individual patient-level explanation for the model predictions is shown in Fig. 3. For a 64-year-old patient with a history of opioid, antidepressant, and benzodiazepine use, the predicted probability of prolonged postoperative opioid prescriptions was 0.22. In this case, history of opioid, antidepressant, and benzodiazepine use increased the estimation for the likelihood of prolonged postoperative opioid prescriptions, whereas the lack of other preoperative medication use (gabapentin, beta-2-agonist, nonsteroidal anti-inflammatory), being married, prior history of diabetes, and age between 60 and 68 decreased the predicted probability.
Fig. 3.
This graph shows an example of individual patient-level explanation for prolonged postoperative opioid prescriptions.
4. Discussion
Opioids are routinely prescribed after elective orthopaedic surgery, although they are often not necessary to manage postoperative pain when used as prescribed. However, opioid prescribing habits have come under scrutiny in the past several years, as the mortality rate of opioid-associated overdose has increased four-fold from 1999-2011.4 The opioid crisis is of particular concern to patients undergoing TKA, as postoperative pain management with opioids has been linked to long term addiction and abuse.12,13 As orthopaedic surgeons are among the highest prescribers of opioids in medicine, there is a clinical need for a tool that can assist these physicians to preoperatively determine which patients are at increased risk for opioid dependence following TKA.4 The SGB model developed in this study aims to serve this purpose for TKA patients, as it performed well on discrimination, calibration, and decision curve analysis for preoperative prediction of prolonged opioid prescriptions following TKA.
Although this study provided valuable information, this study had several limitations. The means by which postoperative opioid prescriptions were measured was via examination of electronic health records and pharmacy records. We assumed that the refilling of a prescription after 90 days constituted prolonged opioid use; however, it is certainly possible that patients refilled their prescription without any intent of using the medication. Other studies have found that many arthroplasty patients do not use the majority of opioid pills prescribed, illustrating that opioid prescriptions do not necessarily correlate with opioid use.14,15 However, it was not feasible to distinguish between opioid prescriptions and opioid use. In addition, due to the study population being drawn from 5 hospitals affiliated with a single healthcare corporation, there is the possibility of clustering at the level of providers. Because the algorithms in this study were developed using a retrospective study design, there remains a need for these findings to be validated in external samples using a prospective study design. This study also only observed opioids attained from pharmacies or providers and did not account for opioids attained or consumed illicitly. Finally, it must be acknowledged that the prescribing of opioids in this study was a decision made by an orthopaedic surgeon based on clinical judgement; our study could not account for the variance in provider thresholds for prescribing opioids.
Although the development of algorithms to predict postoperative opioid use has been performed, to our knowledge, this is the first attempt to predict opioid consumption following TKA using machine learning. Karhade et al. developed an ENPLR model which similarly identified preoperative opioid use, medication use, and age as variables that were predictive of prolonged postoperative opioid use following THA; however their model also identified preoperative hemoglobin as a predictive variable, one that our SGB model did not identify.10 Within the field of arthroplasty, machine learning has been used to develop models that predict outcomes that this study did not investigate such as cost, length of stay, and complication rates.8,16,17
The factors identified in this study are consistent with other research identifying prolonged opioid use following TKA. Kim et al. studied a cohort of 338 consecutive TKA cases from a large academic medical center to examine opioid consumption following TKA, and found that consumption of at least 12 mg/d morphine-equivalents over the 3 months preceding surgery increased the incidence of postoperative chronic opioid use by six-fold.18 This correlation between preoperative opioid use and prolonged opioid consumption following TKA has been identified by multiple studies.7,8,19,20 Sun et al. studied a population of 641,941 opioid naïve patients who had undergone surgical procedures and found that age and preoperative use of certain medications, including anti-depressants and benzodiazepines, were associated with chronic opioid use postoperatively, which they defined as the filling of at least 10 prescriptions or having more than 120 days’ supply of an opioid in the first year postoperatively.21 Their finding is consistent with the results of this study. Using claims data from a US commercial health plan, Kim et al. similarly found that comorbidities such as diabetes mellitus were significant predictors of persistent opioid use following TKA.22 Finally, marital status, specifically being unmarried, has been previously identified as a preoperative demographic associated with increasing the risk of experiencing clinically meaningful acute pain following surgery, which could necessitate increased postoperative opioid prescriptions.23
There remains a need for further studies to be conducted to validate or refute the performance of these machine learning algorithms. The intent of this study is to provide providers with a resource that may assist their individual decision-making with regards to prescribing opioids to patients following TKA. This SGB machine learning algorithm was developed to preoperatively identify patients who potentially may be at risk for prolonged opioid use following TKA, and provide the opportunity for clinicians to counsel and educate patients. By identifying such patients preoperatively, clinicians could potentially tailor a patient's postoperative pain management by limiting postoperative opioid prescriptions less than 90 days and utilizing alternative pain management strategies to lower the burden of opioid use.
Disclosures/conflicts of interest/funding
One of the authors (HSB) has received funding from Smith & Nephew, Exactech, Zimmer Biomet, and Wolter Kluwer. Another author (AFC) has received funding from SLACK publishing, ACI, Stryker, bOne, OREF, Pfizer, Avanos, Irrisept, Convatec, 3M, Recro, Zimmer Biomet, Heraeus, American Medical Foundation, DePuy, and holds equity in Joint Purification Systems, Sonoran Biosciences, Graftworx, and Hyalex. The remaining authors have no competing interests or funding to declare.
Ethics statement
This study was approved by our institutional review board.
Location
The work was performed primarily at the Massachusetts General Hospital.
Footnotes
Supplementary data to this article can be found online at https://doi.org/10.1016/j.jor.2020.03.052.
Contributor Information
Akhil Katakam, Email: akhil.katakam@temple.edu.
Aditya V. Karhade, Email: aditya.v.karhade@gmail.com.
Joseph H. Schwab, Email: JHSCHWAB@mgh.harvard.edu.
Antonia F. Chen, Email: afchen@bwh.harvard.edu.
Hany S. Bedair, Email: HBEDAIR@mgh.harvard.edu, HBEDAIR@mgh.harvard.edu.
Appendix A. Supplementary data
The following are the Supplementary data to this article:
References
- 1.Rudd Rose A., Seth Puja, David Felicita, Lawrence Scholl. Increases in drug and opioid-involved overdose deaths - United States. MMWR (Morb. Mortal. Wkly. Rep.) 2010-2015 doi: 10.15585/mmwr.mm655051e1. 50–51 (December 30, 2016): 1445–52. [DOI] [PubMed] [Google Scholar]
- 2.Skolnick Phil. The opioid epidemic: crisis and solutions | annual review of pharmacology and toxicology. https://www.annualreviews.org/doi/abs/10.1146/annurev-pharmtox-010617-052534 Accessed October 9, 2019. [DOI] [PubMed]
- 3.Levy Benjamin, Leonard Paulozzi, Mack Karin A., Jones Christopher M. Trends in opioid analgesic-prescribing rates by specialty, U.S., 2007-2012. Am J Prev Med. September 2015;49(3):409–413. doi: 10.1016/j.amepre.2015.02.020. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Ringwalt Chris, Gugelmann Hallam, Garrettson Mariana. Differential prescribing of opioid analgesics according to physician specialty for medicaid patients with chronic noncancer pain diagnoses. Pain Res Manag: J. Can. Pain Soc. 2014;19(4) doi: 10.1155/2014/857952. 179–85. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Goesling Jenna, Moser Stephanie E., Zaidi Bilal. Trends and predictors of opioid use following total knee and total hip arthroplasty. Pain. June 2016;157:6. doi: 10.1097/j.pain.0000000000000516. 1259–65. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Lespasio, Michelle J., Guarino A.J., Sodhi Nipun, Mont Michael A. Pain management associated with total Joint arthroplasty: a primer. Perm J. March 28, 2019;23 doi: 10.7812/TPP/18-169. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Bedard Nicholas A., Andrew J. Pugely, Westermann Robert W., Duchman Kyle R., Glass Natalie A., Callaghan John J. Opioid use after total knee arthroplasty: trends and risk factors for prolonged use. J Arthroplasty. 2017;32(8) doi: 10.1016/j.arth.2017.03.014. 2390–94. [DOI] [PubMed] [Google Scholar]
- 8.Hadlandsmyth Katherine, Vander Weg Mark W., McCoy Kimberly D., Mosher Hilary J., Vaughan-Sarrazin Mary S., Lund Brian C. Risk for prolonged opioid use following total knee arthroplasty in veterans. J Arthroplasty. 2018;33:1. doi: 10.1016/j.arth.2017.08.022. 119–23. [DOI] [PubMed] [Google Scholar]
- 9.Politzer Cary S., Kildow Beau J., Goltz Daniel E., Green Cynthia L., Bolognesi Michael P., Seyler Thorsten M. Trends in opioid utilization before and after total knee arthroplasty. J Arthroplasty. 2018;33:7S. doi: 10.1016/j.arth.2017.10.060. S147-S153.e1. [DOI] [PubMed] [Google Scholar]
- 10.Karhade A.V., Schwab J.H., Bedair H.S. Development of machine learning algorithms for prediction of sustained postoperative opioid prescriptions after total hip arthroplasty. J Arthroplasty. 2019;34:2272–2277. doi: 10.1016/j.arth.2019.06.013. e1. [DOI] [PubMed] [Google Scholar]
- 11.Navarro Sergio M., Wang Eric Y., Haeberle Heather S. Machine learning and primary total knee arthroplasty: patient forecasting for a patient-specific payment model. J Arthroplasty. 2018;33:12. doi: 10.1016/j.arth.2018.08.028. 3617–23. [DOI] [PubMed] [Google Scholar]
- 12.Etcheson Jennifer I., Gwam Chukwuweike U., George Nicole E., Caughran Alexander T., Mont Michael A., Delanois Ronald E. Does the amount of opioid consumed influence how patients rate their experience of care after total knee arthroplasty? J Arthroplasty. 2018;33:11. doi: 10.1016/j.arth.2018.06.028. 3407–11. [DOI] [PubMed] [Google Scholar]
- 13.Agrawal Yuvraj, Smith R. Malcolm, Garbuz Donald S., Masri Bassam A. Opioids in arthroplasty: mind the gap between north America and the rest of the world. JBJS. December 19, 2018;100(24):2162. doi: 10.2106/JBJS.17.01422. [DOI] [PubMed] [Google Scholar]
- 14.Huang Philip S., Copp Steven N. Oral opioids are overprescribed in the opiate-naive patient undergoing total Joint arthroplasty. J Am Acad Orthop Surg. August 1, 2019;27(15) doi: 10.5435/JAAOS-D-18-00404. e702–8. [DOI] [PubMed] [Google Scholar]
- 15.Sabatino Matthew J., Kunkel Samuel T., Ramkumar Dipak B., Keeney Benjamin J., Jevsevar David S. “Excess opioid medication and variation in prescribing patterns following common orthopaedic procedures.” the Journal of Bone and Joint surgery. America. February 7, 2018;100(3) doi: 10.2106/JBJS.17.00672. 180–88. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Harris Alex Hs, Kuo Alfred C., Bowe Thomas. Prediction models for 30-day mortality and complications after total knee and hip arthroplasties for veteran health administration patients with osteoarthritis. J Arthroplasty. 2018;33(5) doi: 10.1016/j.arth.2017.12.003. 1539–45. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Ramkumar Prem N., Navarro Sergio M., Haeberle Heather S. Development and validation of a machine learning algorithm after primary total hip arthroplasty: applications to length of stay and payment models. J Arthroplasty. 2019;34(4) doi: 10.1016/j.arth.2018.12.030. 632–37. [DOI] [PubMed] [Google Scholar]
- 18.Kim Kelvin Y., Anoushiravani Afshin A., Chen Kevin K., Roof Mackenzie, Long William J., Schwarzkopf Ran. Preoperative chronic opioid users in total knee arthroplasty-which patients persistently abuse opiates following surgery? J Arthroplasty. 2018;33:1. doi: 10.1016/j.arth.2017.07.041. 107–12. [DOI] [PubMed] [Google Scholar]
- 19.Cook David J., Kaskovich Samuel W., Pirkle Sean C., Conti Mica Megan A., Shi Lewis L., Lee Michael J. Benchmarks of duration and magnitude of opioid consumption after total hip and knee arthroplasty: a database analysis of 69,368 patients. J Arthroplasty. 2019;34(4):638–644. doi: 10.1016/j.arth.2018.12.023. e1. [DOI] [PubMed] [Google Scholar]
- 20.Hernandez Nicholas M., Parry Joshua A., Mabry Tad M., Taunton Michael J. Patients at risk: preoperative opioid use affects opioid prescribing, refills, and outcomes after total knee arthroplasty. J Arthroplasty. 2018;33:7S. doi: 10.1016/j.arth.2018.01.004. S142–46. [DOI] [PubMed] [Google Scholar]
- 21.Sun Eric C., Darnall Beth D., Baker Laurence C., Mackey Sean. Incidence of and risk factors for chronic opioid use among opioid-naive patients in the postoperative period. JAMA Int. Med. 01 2016;176(9) doi: 10.1001/jamainternmed.2016.3298. 1286–93. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Kim S.C., Choudhry N., Franklin J.M. Patterns and predictors of persistent opioid use following hip or knee arthroplasty. Osteoarthritis Cartilage. September 1, 2017;25(9):1399–1406. doi: 10.1016/j.joca.2017.04.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Katz J., Poleshuck E.L., Andrus C.H. Risk factors for acute pain and its persistence following breast cancer surgery. Pain. 2005;119:16–25. doi: 10.1016/j.pain.2005.09.008. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.