A Novel Machine Learning Predictive Tool Assessing Outpatient or Inpatient Designation for Medicare Patients Undergoing Total Knee Arthroplasty

Abstract

Background

Removal of total joint arthroplasty from the inpatient-only list has created significant confusion regarding which patients qualify for an inpatient designation. The purpose of this study is to develop and validate a novel predictive tool for assessing who will be an outpatient vs inpatient after total knee arthroplasty (TKA).

Methods

A cohort of Medicare patients undergoing primary TKA between January 2018 and September 2019 were retrospectively reviewed. Baseline demographics and patient characteristics were obtained, and their distributions for outpatient (less than 2 midnights) and inpatient stay were assessed. Subsequently, a XGBoost machine learning model was trained using 80% of the TKA patients, and the remaining 20% of patients were involved in testing the model's performance in terms of accuracy and the average area under the receive operating characteristic curve.

Results

Eight hundred ninety-nine Medicare patients underwent TKA at our institution between January 2018 and September 2019. Of which, 625 patients had outpatients stays, and 274 qualified for inpatient designation. Significant associations were demonstrated between inpatient visits and the following factors: higher body mass index, increased age, better functional scores, multidimensional fatigue inventory, Charlson Comorbidity Index, American Society of Anesthesiologists score, female gender, cardiac history, and the Revised Cardiac Risk Index. The XGBoost model for predicting an inpatient or outpatient stay was 63.3% accurate, with area under the receive operating characteristic curve of 68.8%.

Conclusions

Using readily available key baseline characteristics, functional scores, and comorbidities, this machine-learning model accurately predicts the probability of an “outpatient” vs “inpatient” stay after TKA in the Medicare population. body mass index, age, VR12 functional scores, and multidimensional fatigue inventory scores had the highest influence on this predictive model.

Introduction

Nearly 10% of adults in the United States (U.S.) have been diagnosed with osteoarthritis [1]. This accounts for 30 million people, and one-third of these patients are older than 65 years [1,2]. Total knee arthroplasty (TKA) has been demonstrated to be a successful procedure for end-stage arthritis, and multiple studies have projected a continued increase in TKA [[3], [4], [5]].

The U.S. spent nearly one-fifth of the gross domestic product on health care in 2018 [6]. In an effort to decrease costs, alternative payment models, such as the Bundled Payment Care Improvement (BPCI) model, have been implemented [[7], [8], [9]]. The focus of this model is to reduce costs while simultaneously improving the quality of patient care. However, numerous studies have demonstrated that the BPCI initiative oversimplifies costs associated with total joint arthroplasty (TJA), consequently resulting in decreased reimbursements. Patients of older age and those with higher measures of frailty account for a significantly increased cost after TJA, which may subsequently deincentivize care to older and higher risks patients [8,9].

In addition, the Center for Medicare and Medicaid services (CMS) seeks to further lower costs, by removing TJA from the inpatient-only list [[10], [11], [12], [13], [14]]. TKA was removed from this list in 2018, and total hip arthroplasty was removed in 2020. There have been numerous consequences to removing TKA from the inpatient-only list [[10], [11], [12], [13], [14], [15], [16], [17], [18], [19], [20], [21]]. Providers and hospital administration face the added pressure in classifying most TKAs as an outpatient procedure, to avoid potential financially damaging audits. This subsequently results in increased costs for physicians, patients, and health-care systems. An unintended impact may lead to a disparity in care toward the elderly and frail patients, to mitigate financial losses for institutions due to the CMS two-midnight rule [8,9]. Elderly may be disproportionally affected, as providers have a decreased incentive in operating on individuals whose procedures will lead to net losses, rather than gains.

In order to alleviate the penalties and concerns brought on by the removal of TKA from the inpatient-only list, proper risk stratification and evidence for assigning individuals to inpatient vs outpatient setting must be established before the planned surgery. The Outpatient Joint Arthroplasty-Patient Selection Score was developed using comorbidities of TJA patients and has been demonstrated as a safe way of identifying which patients can safely elect to undergo outpatient TJA [22]. Multiple studies have been built off of this model and have used machine learning to predict length of stay (LOS) after TJA.

In order to alleviate the penalties and concerns brought on by the removal of TKA from the inpatient-only list, proper risk stratification and evidence for assigning individuals to inpatient vs outpatient setting must be established before the planned surgery. The aforementioned idea can be put in action with supervised machine learning. Supervised machine learning labels the data based on the outcome of interest (inpatient or outpatient designation). The machine learning model assesses the data through a stepwise approach, by associating patient demographics and characteristics with the aforementioned outcome of interest. Finally, an algorithm is generated, which allows prediction of inpatient vs outpatient designation. Machine learning models have used large national databases to predict LOS [[15], [16], [17]]. However, no studies have been performed applying machine learning to predict inpatient vs outpatient designation after TKA at a single institution. The purpose of this study is to develop and validate a novel machine learning model, using a single institutions data, to predict inpatient vs outpatient designation for individuals undergoing TKA.

Material and methods

A cohort of Medicare patients undergoing primary TKA between January 2018 and September 2019 were retrospectively reviewed. All patients’ surgeries were performed at a single large urban academic institution. This was a multisurgeon study that included TKA performed by both arthroplasty-trained surgeons and general orthopedic surgeons. Inclusion criteria for this study were patients older than 65 years who underwent a TKA and had a full set of demographics and documented LOS. LOS was considered to fall under the outpatient class if the patient spent less than 2 midnights in the hospital, per the CMS two-midnight rule. Patient demographics (age, gender, BMI), past medical history (rheumatoid arthritis, cardiac history, history of a venous thromboembolic event, diabetes mellitus, and other rheumatologic diseases), Charlson Comorbidity Index (CCI), American Society of Anesthesiologist Physical Status Classification (ASA), Revised Cardiac Risk Index, and the modified frailty index (mFI) scores were obtained through chart review. Preoperative functional scores were obtained during postoperative visits. The following preoperative function scores were included in the analysis: knee disability and osteoarthritis outcome score (KOOSJR), VR12 physical component (pcs), and VR12 mental component (mcs) scores. Baseline variable’s distributions for outpatient and inpatient stay were compared using two-sample t test for continuous variables and chi-square test for categorical variables. Significance was set at P < .05.

XGBoost (eXtreme Gradient Boosting) is a machine-learning tool which provides gradient boosting framework to build predictive models. XGBoost uses training data to predict a target variable. It is a decision tree ensemble, which consists of a set of classification and regressions trees. Classification trees were used for this study because of the binary nature of the outcome. Gradient boosting builds new models to predict errors of prior models, subsequently resulting in a strong final algorithm. Hence, the final predictive model is built in a stage-wise fashion. The XGBoost algorithm was used for the current article, as it proved to be the most accurate model. The authors ran 3 other models—a L1 penalized logistic regression, a support vector machine model, and a random forest model. The performance metrics of these models is demonstrated in Table 1.

Table 1.

Accuracy and area under the curve (AUC) for machine learning and regression models.

	XGBoost	L1-penalized logistic regression
Accuracy	63.3%	60.00%
AUC	68.8%	63.43%
	Support vector machine	Random forest
Accuracy	60.00%	60.36%
AUC	67.53%	65.85%

XGBoost proved to be the most accurate model with the highest AUC.

While preserving the proportion of outpatient and inpatient classes in the study population, 80% of patients were randomly selected to be used for training the XGBoost model, while the remaining 20% were used for testing. Stratified 5-fold cross-validation was conducted on the training set. Class weight was used to handle the imbalanced outcome. Feature importance, shown as F score, was obtained for each feature used in the model to see the relative importance of predicting the inpatient or outpatient setting. It is based on the number of times each feature was used to split the data during training of the XGBoost model.

Finally, the performance of the trained model was evaluated on the test set. The performance metrics were accuracy and the average area under the receiver operating characteristic curve (AUC) for outcome measures. The receiver operating characteristic curve illustrates the diagnostic ability of a binary classification system. This method was originally developed for operators of military radar receivers. The curve is created through plotting the true-positive rate against the false-positive rate, and the area under the curve represents the probability that a group of characteristics will accurately predict an outcome. Using this method, 0.5 represents chance performance while 1.0 represents perfect performance of a predictive model. If the resultant AUC is greater than 0.5, the model is considered providing useful information for predicting the specific outcome based on a set of inputs. All statistical analyses and the modeling process were conducted in Python, version 3.7.3, with the Jupyter Notebook interface.

Our hypothesis is that a machine-learning model, which uses patient demographics, preoperative functional scores, and comorbidities, will help predict inpatient vs outpatient designation for patients undergoing TKA.

Results

Patient baseline demographics and presurgery details

There were 899 Medicare patients who underwent TKA at our institution between January 2018 and September 2019. Outpatient stays (LOS, less than 2 midnights) accounted for 625 patients (69.5%) undergoing TKA, while 274 patients (30.5%) qualified for inpatient designation (LOS, 2 midnights or greater). Significant associations were demonstrated between inpatient visits and the following factors: higher BMI (P < .01), increased age (P < .01), worse VR12 scores (P < .01), increased mFI scores (P < .01), increased CCI scores (P < .01), higher ASA (P < .01), female gender (P < .01), and Revised Cardiac Risk Index (P < .01) (Table 2).

Table 2.

Patient demographics and presurgery details for the patients who had an outpatient TKA and the patient who had an inpatient TKA.

Patient characteristics	Outpatient	Inpatient	P value
Mean age (y)	73.2 ± 5.5	75.0 ± 6.0	<.01
% Female	406 (65.0%)	213 (77.7%)	<.01
Mean BMI (kg/m2)	30.8 ± 5.8	32.17 ± 6.8	<.01
Mean CCI	3.6 ± 1.5	4.1 ± 1.8	<.01
Diagnosis
Rheumatoid arthritis	486 (77.8%)	215 (78.5%)	.37
Osteoarthritis	138 (22.1%)	57 (20.8%)
Avascular necrosis	1 (0.2%)	2 (0.7%)
Mean ASA	2.5 ± 0.6	2.7 ± 0.5	<.01
Mean KOOSJR	49.4 ± 13.2	46.6 ± 12.6	.21
Mean VR12 pcs	32.7 ± 7.9	28.0 ± 6.9	<.01
Mean VR12 mcs	51.0 ± 11.6	45.4 ± 11.8	<.01
Mean mFI	1.1 ± 1.2	1.4 ± 1.2	<.01
Mean RCRI	0.1 ± 0.3	0.1 ± 0.4	.02
Cardiac history	140 (22.4%)	70 (25.5%)	.34
Venous thromboembolism	41 (6.6%)	24 (8.8%)	.30
Diabetes mellitus	76 (12.2%)	43 (15.7%)	.18
Rheumatology	140 (22.4%)	70 (25.5%)	.35

RCRI, Revised Cardiac Risk Index.

Comorbidities of cardiac history, diabetes mellitus, rheumatologic diagnosis, and a history of a venous thromboembolic event were not significantly differently distributed between outpatient and inpatient admission after TKA (Table 2).

The XGBoost model

The XGBoost model for predicting an inpatient or outpatient stay was 63.3% accurate with the AUC to be 68.8%, which is considered poor but useful. The importance of each characteristic in predicting the final model can be seen in Figure 1. Feature importance was represented by the number of times a feature was used to split the data across all trees, which demonstrated the contribution of each feature in discriminating inpatient vs outpatient stay.

Figure 1.

The feature importance of each variable, in the predictive model for inpatient designation after TKA.

The importance of each feature in the trained XGBoost model can be seen in Figure 1. The higher value means greater influence in predicting the outcome. The top 8 features that had the highest F scores are BMI, V12 pcs, VR12 mcs, age, KOOSJR, CCI, gender, and mFI, which indicated BMI, age, preoperative functional scores, frailty, and comorbidity indices are rather important in predicting the inpatient or outpatient setting. Among these features, being female; higher BMI, age, KOOSJR, CCI, and mFI, and lower V12 pcs and VR12 mcs increased the probability of an inpatient designation after TKA.

The mean LOS is 0.70 midnights for the outpatient cohort and 2.59 for the inpatient cohort.

Discussion

Among the 4 models developed in this study, the XGBoost model showed the highest accuracy and AUC compared with the other 3 traditional machine learning models. In addition, the XGBoost algorithm acts more robust if missing values exist for input features, which will provide flexibility when one implements this model on new patients. Therefore, this model can be conveniently used by a wide range of providers, administrators, and academic centers.

To our knowledge, this trained XGBoost model is the most accurate and applicable predictive model to date in determining inpatient vs outpatient designation after TKA, while using a single institution’s data. When using the readily available variables such as BMI, patient age, preoperative functional scores, ASA, mFI, CCI, and gender, the machine learning algorithm proved to be 63.3% accurate with the AUC to be 68.8%. No other study has used machine learning to build a predictive model of inpatient vs outpatient designation, after TKA, with a single institution’s data. Two studies have built predictive models assessing LOS after TKA using the New York State Department of Health’s SPARCS database [15,16]. Another study queried the American College of Surgeons National Surgical Quality Improvement Program database, when building their predictive model assessing TKA inpatient vs outpatient designations [17]. Unfortunately, these studies may not be as valid and reproducible to one specific institution.

Similar to this study, Gronbeck et al. demonstrated increased age, functional status, and female gender to determine length of inpatient stay after TKA [17]. In addition, the authors demonstrated bilateral TKA and a history of metastatic cancer to be predictive of a stay longer than two midnights. This model had an acceptable discrimination with a C-statistic of 0.66 and validation of 0.65. A study performed by Navarro et al. used 141,446 patients who underwent TKA from 2009 to 2016 in the SPARCS database [16]. Their model incorporated age, race, gender, and comorbidities to produce an AUC of 0.782. Unlike the present study, their model did not predict inpatient vs outpatient status in concordance with the CMS two-midnight rule, which may not provide enough evidence to justify assigning a patient to the appropriate preoperative designation [16]. The strengths of our predictive model, in comparison to the aforementioned one, include a single institution's data, the outcome factor being inpatient vs outpatient LOS as defined by the CMS two-midnight rule, and the assessment of only patients undergoing primary TKA. Furthermore, all patients included in our analysis underwent a TKA after 2018, when TKA was taken off the CMS inpatient-only list.

The aforementioned machine learning models build off the Outpatient Joint Arthroplasty-Patient Selection score, which uses 9 comorbidities (general medical, hematological, cardiac, endocrine, gastrointestinal, neurological/psychological, renal/urology, pulmonary, infectious disease) to predict which patients can safely undergo outpatient TJA [22]. This model is of upmost importance as many centers and practices seek to safely perform same-day TJA, often at ambulatory surgical centers. Our model differs as it focuses on inpatient vs outpatient prediction as designated by the CMS.

Spending on health care continues to grow in the U.S. In 2018, 3.6 trillion dollars were spent on health care, which represented a nearly 5% increase in spending in comparison to the previous year [7]. In efforts to decrease TJA cost, the CMS has removed TKA from the inpatient-only list [11,12]. The CMS defines an inpatient stay as less than 2 midnights at the hospital after a procedure [[11], [12], [13], [14], [15], [16]]. Removing TKA from the inpatient-only list has created significant confusion and concern as there are no accepted parameters that can be applied to determine which patients can be classified as inpatient vs outpatient [5,9,10,12,13].

Owing to the discontinuation of TKA from the inpatient-only list, a physician has 3 options on what to do in respect to patients who have already spent one night at the hospital: (1) provide appropriate documentation on why a patient requires the extra night stay; (2) discharge the patient, in which case the patient will fall under an outpatient admission; or (3) discharge the patient on postoperative day 1, in which case the stay can be considered a short-stay inpatient hospitalization [10]. Hospital administrators and financial directors need concrete evidence to support inpatient vs outpatient designations, or they may be susceptible to extremely damaging audits. In an effort to avoid these audits, many institutions and providers are assigning a majority, if not all, patients undergoing TKA to outpatient designations. In a survey of American Association of Hip and Knee Surgeon (AAHKS) members, 59.5% of respondents reported that their hospitals have instructed them that all TKAs be scheduled as outpatient procedures, after the decision to take the procedure off the inpatient-only list [17]. Predesignation of TKA as outpatient admissions for Medicare patients undergoing this procedure will beget significant hospital costs—our study has demonstrated that more than 30% of patients undergoing TKA had an LOS that qualified for an inpatient designation.

Clinical trends toward decreased LOS after TKA are in a clash with financial directors' aim to avoid audits that can result in significant loss [14]. In another AAHKS survey, Krueger et al. demonstrated that 81% of respondents noted an increase in their practice’s administrative burden after TKA was removed from the inpatient-only list [18]. More than half of surgeons surveyed needed to obtain preauthorization or appeal a denial for an inpatient TKA on a monthly basis. Within 2 years of the removal of TKA from the inpatient-only list, 10% of respondents had already been subjected to audits [18]. In a study published by Iorio et al., the authors demonstrated that their institution had already been subjected to 2 quality improvement audits since the 2018's inpatient-only removal of TKA [19].

Not only is the confusion of inpatient vs outpatient designation a burden to hospitals and providers, but nearly a third of surgeons stated that their patients have had additional personal costs after their TKA, due to the preoperative surgical classification being designated as an outpatient procedure [17]. In a subsequent survey conducted by AAHKS, 61% of patients contacted the primary surgeon’s office because of an unexpected copayment [18]. Iorio et al. further demonstrated the burden to patients, as Medicare part B has an annual deductible of $185 and a 20% co-pay [19]. Individuals designated as having an outpatient procedure are subject to increased postoperative costs with respect to equipment and medications [19].

Before taking TJA off the inpatient-only list, the CMS alternative payment model of the BPCI has resulted in significant hospital losses after orthopedic procedures [6]. Petersen et al. [9] demonstrated a loss of $1934 per patient, in those undergoing primary TJA who are between the ages of 85 and 99 years. As this age group continues to increase in size, the current BPCI initiative in TJA is expected to result in declining profits by 2030 [6]. Pepper et al. [8] similarly demonstrated an increase in cost of care for patients aged 72 years or older at the time of their TJA [5]. In addition to age, the authors found an increasing frailty score to significantly increase costs after TJA. This study has similar findings to the aforementioned literature, as increasing age and frailty are both significant predictors of requiring an inpatient stay after TKA [[21], [23]]. Curtin et al. [23] assessed CMS data for expenditures of TKA in a BPCI model. The authors projected that removal of TKA from the inpatient-only list could potentially remove up to 40% of individuals from the BPCI program, with remaining bundle patients to likely require home health care and subacute nursing facilities after discharge from TKA [21]. If orthopedic surgeons and hospital systems continue to classify all TKAs as outpatient procedures, the hospital losses will be significantly increased as the population continues to age and undergo TJA. Haas et al. demonstrated a 30% lower payment from Medicare for outpatient TKA [20]. Therefore, providers and institutions may be subsequently biased toward performing arthroplasty procedures in patients of older age and increased frailty, to mitigate the projected loss in profits.

In an effort to ease potential hospital losses, due to classifying all patients undergoing TKA as an outpatient, the authors have used machine learning to justify inpatient designation, without fear of potentially destructive audits. CMS has not provided details on how patients should be designated for admission status, and surgeons have been extremely cautious in categorization to decrease their chances of serious financial consequences [[18], [19], [20], [21]]. Krueger noted the aforementioned idea to make the decision as to which patients should receive an inpatient or outpatient designation more of an art than science [18]. Our model uses readily available preoperative characteristics based on factors that have significant associations with an inpatient stay after TKA. These factors are a higher BMI, increased patient age, worse preoperative functional scores, higher ASA, higher mFI, higher CCI, and female gender. Future studies should focus on validating the current model. Upon validation, this model can be used throughout the country for admission classification in patients undergoing TKA. Ultimately, the use of this model will likely reduce costs to patients and losses for hospital systems. Subsequently, this will allow for the continuation of performing this beneficial and ever-growing procedure on the increasing older and frail population, without bias and fear of economic losses.

This study is not without limitations. The design is a retrospective review in nature, and the bias that comes with this type of data cannot be avoided. Furthermore, this study uses a predictive model, which has the limitations of uncertain projections in respect to future trends in patient characteristics and CMS initiatives. Other limitations include a small patient cohort, poor predictive capability, and limited input variables. In addition, admission designation was determined using LOS. The data used in this study were from a single tertiary academic center, and our urban population may not adequately represent the population of the U.S. However, using data from a single institution is a significant strength when compared to studies using data from large-scale national databases. Another strength of this study was the machine learning program used in building this predictive model—XGboost algorithm has received numerous awards such as the John Chambers Award and High Energy Physics meets Machine Learning award. Based on this study’s accuracy and AUC, we conclude this is so far, the most reliable, reproducible, and most valid predictive model on assessing inpatient vs outpatient designation after TKA.

Conflicts of interest

The authors declare that there are no conflicts of interest.