Incorporating Expected Outcomes into Clinical Decision Making for Total Knee Arthroplasty

Abstract

Objective:

Expected outcomes (e.g., expected survivorship after a cancer treatment) have improved decision making around treatment options in many clinical fields. We evaluated the effect of expected values of 3 widely available total knee arthroplasty (TKA) outcomes (risk of serious complications, time to revision, and improvement in pain and function at 2 years after surgery) on clinical recommendation of TKA.

Methods:

The RAND/UCLA Appropriateness Criteria (AC) method was utilized to evaluate role of the three expected outcomes clinical recommendation of TKA. The expected outcomes were added to 5 established pre-operative factors from the modified Escobar AC. The 8 indication factors were used to develop 279 clinical scenarios and a panel of 9 clinicians rated the appropriateness of TKA for each scenario as inappropriate, inconclusive, and appropriate. Classification tree analysis was applied to these ratings to identify the most influential of the 8 factors in discriminating TKA appropriateness classifications.

Results:

Ratings for the 279 appropriateness scenarios deemed 34.4% of scenarios as appropriate, 40.1% as inconclusive, and 25.5% as inappropriate. Classification tree analyses showed that expected improvement in pain and function and expected time to revision were the most influential factors that discriminated among the TKA appropriateness classification categories.

Conclusion:

Our results showed that clinicians would utilize expected post-operative outcomes factors in determining appropriateness for TKA. These results call for further work in this area to incorporate estimates of expected pain/function and revision outcomes into clinical practice to improve decision making for TKA.

Introduction:

Total knee arthroplasty (TKA) is an effective treatment for advanced knee osteoarthritis (KOA) (1). Currently, nearly a million procedures are performed in the US per year, and population-based studies have projected increasing TKA rates (2–6). When patients consider TKA with their surgeons, the main indications for surgery are often pain and functional limitations, corroborated by radiographic findings and failed conservative treatment (7–9). While these factors are important, recent evidence shows that many patients with KOA either utilize the surgery too early or too late, suggesting the need to augment available information to improve decision making (10).

In recent years, availability of outcomes data, such as revision rates on thousands of TKA patients from national and international registries, were instrumental in identifying prosthesis failures (e.g., that of metal-on-metal), but we believe they could also further inform the decision to undergo surgery in other ways. In oncology, for example, the availability of expected survivorship to patients and clinicians, based on the patient’s demographic and cancer stage, has had a significant impact on treatment decisions (11, 12). Similarly, knowing a patient’s risk of revision within 5 years after TKA based on the patient’s demographic and KOA severity may provide patient-centered information that could inform the decision to undergo such procedure. To that end, a number of risk calculators (e.g., to predict risk of complications after TKA) have been developed, but little is known about how these expected risks inform the TKA decision making process, or which expected outcomes were more important than others (13–15).

Appropriateness criteria (AC) are established tools that can inform the decision to undergo elective surgical procedures by assessing whether the procedure is appropriate or not for a patient vignette with specific characteristics (16). AC are developed using standardized methods which allow expected outcomes to be included and ranked among these characteristics. Prior AC developed to assess appropriateness of TKA rely on commonly used clinical pre-operative factors (e.g., pre-operative pain, and X-ray findings) to determine appropriateness of patients for TKA (17–19). The primary aim of this paper was to develop new AC for TKA that potentially incorporate expected post-operative outcomes with more traditional preoperative data. Our secondary aim was to determine the extent to which incorporation of expected post-operative outcomes impacted final judgement of approporiateness.

Materials and Methods:

We adapted the well-established RAND Corporation and University of California Los Angeles (RAND/UCLA) Delphi methods to develop the new AC of TKA (20). The RAND/UCLA Delphi method utilizes a panel of clinician experts who rate the appropriateness of a surgical procedure for hypothetical patients according to defined protocols. Appropriateness ratings are classified as “appropriate,” (expected benefit exceeds risks by a sufficiently wide margin to justify the procedure), “inappropriate” (risks outweigh expected benefit) and “inconclusive” (unclear whether benefits outweigh risks by a sufficiently wide margin) (21). To develop the new criteria, we built upon the modified Escobar AC for TKA, which have been developed using similar methodology (18). The modified Escobar AC for TKA were chosen for their simplicity and because they are the only criteria validated against outcomes, whereby TKA patients classified preoperatively as inappropriate on average, improved little from pre-surgery to two years post-surgery, whereas appropriate and inconclusive groups improved by a clinically meaningful large margin over the same period (22).

The modified Escobar AC has 16 scenarios based on 5 pre-operative indication factors: patient’s age, knee stability, number of knee compartments involved, Kellgren and Lawrence (KL) grade, and symptoms, which are based on the combined Western Ontario and McMaster Universities (WOMAC) Osteoarthritis Index pain and function subscale scores (18). The 5 pre-operative factors of the initial modified Escobar AC are displayed in Table 1. The Northwestern University Institutional Review Board reviewed our protocol and deemed the study not human research.

Table 1:

Pre-operative factors used in the modified Escobar AC for TKA and the new proposed expected post-operative outcome factors

Indication Criteria	Levels
Modified Escobar pre-operative factors
Patient age	1: <55 years 2: 55–65 years 3: >65 years
Knee Mobility/Stability	1: Knee is stable and mobility is preserved (flexion contracture <5 degrees and knee has normal or minor medial or lateral gapping in the 20 degrees flexed knee) 2: knee is unstable and mobility is limited (flexion contracture >=5 degrees and knee has moderate or severe medial or lateral gapping in the 20 degrees flexed knee)
Knee compartments with X-ray evidence of osteoarthritis	1: Uni-compartmental 2: Bi/ tri-compartmental
Severity of knee osteoarthritis on X-ray (KL grade)	1: Slight (KL grade <3) 2: Moderate (KL grade=3) 3: Severe (KL grade = 4)
Knee Symptomatology	1: Slight (combined WOMAC pain and physical function scale score <= 11]) 2: Moderate (combined WOMAC pain and physical function scale items score from 12 to 22]) 3: Intense (combined WOMAC pain and physical function scale score from 23 to 33]) 4: Severe (combined WOMAC pain and physical function scale score >=34])
Proposed expected post-operative factors
Implant longevity/time to revision	1: <5 years 2: 5–15 years 3: >15 years
Rate of serious complications such as pulmonary embolism and infection	1: 0% 2: 1–2% 3: 3–5%
Improvement in pain and function	1: Significant improvement in pain and function by 2 years after surgery 2: Some improvement in pain and function by 2 years after surgery 3: Little or no improvement in pain and function by 2 years after surgery

To arrive at the new criteria, we implemented the following steps, in accordance with well-stablished RAND/UCLA methods:

1-Assembling a clinical panel of experts:

We assembled a multidisciplinary panel of 9 experienced clinicians from leading institutions in the US (co-authors: DR, PF, MB, MM, LR, LM, JM, MP, PS) that included 3 high volume orthopedic surgeons, 2 rheumatologists who specialize in treating KOA, 2 general internists, 1 physical therapist, and 1 orthopedic rehabilitation nurse practitioner. The panel members were chosen because they had substantial knowledge and experience in treating persons with osteoarthritis and TKA, and were demographically diverse. Four of the nine clinicians were women and one was African American.

2-Selecting and refining the expected post-operative outcomes factors:

We selected 3 expected post-operative outcome factors to include in our updated AC: estimated time to revision, complication rate, and improvement in WOMAC pain and function 2 years after surgery. These factors were chosen because they are the 3 most important clinical outcomes for patients undergoing TKA, and they have become widely available from national registries and large administrative datasets (23, 24). The factors were chosen with input from the three orthopedic surgeons on the panel (MM, MB, MP), who also reviewed and refined the categories for each of the predicted outcomes variables based on their extensive clinical knowledge and the existing literature. The final categories were unanimously agreed upon by all three surgeons. The three expected post-operative outcomes factors are displayed in Table 1. Improvement in WOMAC pain and function 2 years after surgery was categorized into significant, some and little or no improvement. Escobar et al. estimated baseline-adjusted minimum clinically important differences (MCID)s for TKA which have been previously used to operationalize these concepts (25, 26).

3-Adding the expected post-operative outcomes factors to the Escobar AC:

The 16 algorithm appropriateness scenarios developed by Escobar and adapted by Riddle et al. are described in detail elsewhere.(18) We provide a brief description of these variables in Table 1, and the combinations of these variables to form scenarios in Table 2. In scenarios where preoperative factors are missing, these factors did not contribute to appropriateness classification. We expanded each of the 16 scenarios in Table 2 by including expected post-operative outcome factors (expected time to revision, expected complications rate, and expected improvement of symptoms). The resulting scenarios list included all possible mathematical combinations resulting from the 27 possible combinations of the 3 additional expected outcome factors with the 16 modified Escobar scenarios (i.e., 27*16 scenarios). This resulted in a total of 432 possible scenarios.

Table 2:

Escobar modified acceptability criteria for 16 scenarios based on the 5 pre-operative clinical factors

Scenario	Age	KL grade	Knee symptoms	Knee compartments with X-ray evidence of osteoarthritis	Knee stability	Appropriateness rating
1		<=3	Slight/ Moderate			Inappropriate
2		4	Slight			Inappropriate
3	<55	4	Moderate			Inappropriate
4	>=55	4	Moderate	Uni-compartmental		Inappropriate
5	>=55	4	Moderate	bi or tri-compartmental		inconclusive
6	<55	<=3	intense/severe	Uni or bi-compartmental		Inappropriate
7	<55	<=3	intense/severe	Tri-compartmental		inconclusive
8	>=55	<3	intense/severe		Normal	Inappropriate
9	>=55	3	intense/severe		Normal	inconclusive
10	55–65	<3	intense/severe		Limited	inconclusive
11	>65	<3	Intense		Limited	inconclusive
12	>65	<3	Severe		Limited	appropriate
13	>=55	3	intense/severe		Limited	appropriate
14	>=55	4	intense/severe			appropriate
15	<55	4	intense/severe	Uni-compartmental		inconclusive
16	<55	4	intense/severe	bi or tri-compartmental		appropriate

4-Eliminating clinically implausible scenarios:

The expanded list of 432 possible scenarios was then reviewed by the 3 orthopedic surgeons on the panel to assure clinical plausibility. A scenario was considered clinically implausible (i.e., not possible for the surgeon to see a patient with this combination of factors) if 2 of the 3 orthopedic surgeons indicated it. An example of a clinically implausible scenario is one where the patient is expected to improve significantly by 2 years yet has high risk of complications and time to revision was less than 5 years. Scenarios that were considered not clinically plausible were excluded from further consideration. Of the 432 possible scenarios, 153 scenarios were considered not clinically plausible and were therefore excluded from further consideration.

5-Panel rating of the new AC for appropriateness:

The remaining 279 clinically plausible scenarios were then distributed to the 9 panel members with specific instructions describing the 8 indication factors, and how to rate each scenario. Following RAND/UCLA methodology, panel members scored each scenario on a 1 to 9 scale (1 being least appropriate and 9 being most appropriate). Panel members were asked to complete their scoring over a period of 2 months. Since the scenarios are hypothetical, the panel members were asked to provide a score for the scenario assuming that each of the 3 expected outcomes was highly accurate.

6-Scoring approach:

RAND/UCLA methods were used to both score TKA appropriateness by each panel member, and to determine final appropriateness rating for the entire panel. TKA appropriateness was rated by each rater on a 1–9 scale: 1 being the least inappropriate, and 9 being the most appropriate (20). Following RAND/UCLA methodology, the median of the 9 panel member scores for each scenario was calculated and assigned the appropriateness rating of inappropriate, inconclusive or appropriate based on the following median scores: inappropriate (score 1–3), inconclusive (score 4–6), and appropriate (score 7–9) for TKA. Since large variation may exist among raters, we also reported level of agreement between raters, according to the RAND/UCLA methodology. Agreement between the panel members’ ratings was defined as 6 or more panelists’ ratings were in the same rating category established by the median score. For example, if the median score for a scenario is 7 (i.e., appropriate) and 6 of the 9 panel member scores were in the 6–9 range, then the scenario is rated appropriate with agreement.

Statistical analyses:

We applied classification tree methodology to predict appropriateness ratings (appropriate, inconclusive, inappropriate) from the 8 indication factors (five pre-operative and three expected post-operative factors) defining each scenario. We had a total of 279 median ratings of the nine panel members for inclusion in the classification tree. This classification tree approach identified the strongest combinations across the 8 indication factors that correctly identify the appropriateness ratings. The classification tree structure is based on progressive binary recursive partitioning analysis. The optimal tree is identified by criterion of minimum classification error from cross validation methods and pruned within one standard prediction error (27, 28). Briefly, a large tree was grown using the entire data set. Next, ten-fold cross validation independently estimated the model misclassification rate at each node. The initial tree was then pruned back to the tree having minimal misclassification rate based on the cross validation estimates. The final optimal tree was selected as the most parsimonious tree within one standard error of the tree having minimal misclassification rate. Analyses used algorithms from Salford Predictive Modeler software version 8.0 developed for classification and regression trees (CART) (29).

Results:

The 9 panel members each rated the 279 scenarios. Based on the median score from the 9 panelist ratings for each of the 279 scenarios, 34.4%, 40.1%, and 25.5% were deemed appropriate, inconclusive, and inappropriate respectively. Agreement between the panel ratings was observed in 72% of the appropriate scenarios, 65% of the inappropriate scenarios, and 13% of the inconclusive scenarios. The algorithm in its entirety is available in the Appendix (Supplementary Table S1).

CART analysis results

The optimal classification tree shown in Figure 1 identified the three appropriateness ratings using only information from the expected outcome factors: expected improvement in pain and function and expected time to revision. The classification rules based on this model are listed in Table 3. Scenarios with significant expected improvement in pain and function and expected time to revision ≥ 5 years were classified as TKA appropriate. Scenarios either with some expected improvement in pain and function or with significant expected improvement but shorter expected revision time (< 5 years) were classified as inconclusive. Scenarios with little or no expected improvement in pain and function were classified by panel members as TKA inappropriate. This classification tree model had good overall accuracy of 74.6%. Accuracy for the separate appropriate, inconclusive, and inappropriate outcomes was 70.8%, 72.3%, and 84.5%, respectively, demonstrating more accuracy for determining who should not rather than who should have TKA (Table 4).

Figure 1.

CART analysis showing the most influential factors in determining appropriateness ratings. Classification accuracy was 74.6%.

Table 3.

Classification criteria based on CART model

Appropriate	“Expected significant improvement in pain and function by 2 years after surgery” and “Expected time to revision >= 5 years”
Inconclusive	“Expected some improvement in pain and function by 2 years after surgery”
	“Expected significant improvement in pain and function by 2 years after surgery” and “Expected time to revision < 5 years”
Inappropriate	“Expected little or no improvement in pain and function by 2 years after surgery”

Table 4.

Classification accuracy of the CART model

Scenario Classification	N	Scenarios Mis-classified	Scenaiors Correctly Classified	% Accuracy
Inappropriate	71	11	60	84.51
Inconclusive	112	31	81	72.32
Appropriate	96	28	68	70.83
Total	279	70	209	74.91

Discussion:

We undertook this study to assess the extent to which expected postoperative outcomes may influence judgements of TKA appropriateness when simultaneously considering more traditional pre-operative indications. Utilizing a panel of 9 experts, we developed new AC that included 279 scenarios with all possible clinically feasible combinations of the indication factors. When the pre-operative factors and the expected post-operative outcome factors were included in the CART analysis, the two key factors driving appropriateness were expected improvement in function and pain 2 years after surgery and expected time to revision. These two post-operative outcome factors were sufficient to characterize 3 of 4 scenario recommendations. Preoperative factors did not influence classification by the panel once postoperative data were considered. These findings support the inclusion of expected improvement in pain, function and revision risk outcomes when evaluating whether patients may substantively benefit from TKA.

We focused this paper on 3 widely available TKA outcomes. All three expected postoperative outcomes are monitored in large national and international registries. For example, the Swedish national registry has 20+years of national-level follow-up data on all TKAs performed in the country to assess revision rates. The US-based FORCE-TJR registry has estimated WOMAC scores (based on the KOOS) assessed preoperatively and 1-year postoperatively on nearly 25,000 TKA patients. Our CART results demonstrated that two expected post-operative outcome factors (expected improvement and expected time to revision) were most influential for assessing the appropriateness of undergoing TKA surgery from a pool of 8 candidates (5 pre-operative and 3 expected post-operative factors). Clinicians associated greater value with these expected post-operative factors to assess appropriateness for TKA. Expected risk of complications was not deemed as important in determining appropriateness, and this may be due to the generally low risk of complications after TKA, or exclusion of some high risk patients, e.g., those with BMI>40, from surgery. Deriving expected postoperative outcomes using risk calculators could help to quickly incorporate these outcomes in clinical practice.

We found that the scenario ratings of inappropriate for TKA were predicted with the highest level of accuracy. This higher accuracy for inappropriate scenarios suggests that the incorporation of expected postoperative outcomes into decision making and the AC may be especially important in helping patients and surgeons decide when it is not appropriate to perform the surgery. This finding of higher accuracy for determining inappropriateness versus appropriateness for a given scenario is echoed in recent work by Escobar et al. (2020) (30). In our recent work examining timeliness of TKA among KOA patients, we found that a quarter of patients undergoing the procedure do so prematurely, suggesting that the use of expected outcomes could be helpful to improve the timing of TKA.

Overall, the results of this study are very encouraging, because they demonstrate the importance clinicians attribute to predicted outcome factors, and that clinicians may use these predictors if introduced in the clinic as a decision support aid. However, this finding should not undermine the importance of preoperative factors in determining TKA appropriateness. Each of the preoperative factors included in the AC is independently associated with postoperative outcomes and is also needed to generate the postoperative outcome predictions (31–37). Thus, our results support the use of the expected postoperative outcomes factors in conjunction with preoperative factors to further inform the decision making process, just as, for example, survival data in cancer patients augment preoperative factors, e.g., creatinine levels, in informing clinical decision making.

The utility of our work is not only limited to demonstrating that expected outcomes could be useful clinically, but it also generated a tool that could be useful in assisting clinicians make more informed decisions. However, additional work is still needed before the current AC are ready for clinical use. In the current study, appropriateness has been determined using hypothetical expected post-operative patient-reported and revision outcomes data, but in clinical practice, this information should be obtained from actual registry and administrative databases using risk calculators to facilitate accurately matching patients with the correct appropriateness scenario. A number of calculators have already been developed to predict postoperative risk of complications, e.g., the American Joint Replacement Registry calculator (38, 39). Franklin and colleagues also published the first models predicting the poor outcome TKA patients using clinical and patient-reported outcome measures (40). Additional work is still needed to further refine the accuracy of these calculators/models. Validation of the developed AC against actual post-operative outcomes, both clinical and patient-reported, is also needed to instill confidence in the collective judgement of the panel. In validating the modified Escobar AC, Riddle et al. showed that TKA patient rated preoperatively as inappropriate on average, improved very little, whereas appropriate and inconclusive groups improved by a large margin (22). Our hope is that the current AC, with the addition of expected outcomes, will have more discriminatory power such that the inappropriate, inconclusive and appropriate groups will all have distinct trajectories.

While this is the first study incorporating expected outcomes, it is important to recognize study limitations. Our work builds on the modified Escobar AC, which has only 5 pre-operative factors. Whereas these indicators are important, there may be other informative pre-operative factors not included in the modified Escobar criteria. In addition, it is recognized some factors in the Escobar AC used arbitrary cutoffs (e.g., age<55 vs. 55–65 and KL grades of 3 vs. 4) that lack evidence-base and may not reflect contemporary practice (e.g., judgments of knee stability using lateral knee joint gapping). The Escobar system was published in 2003 and likely does not capture more contemporary patient-relevant preferences, particularly of younger and more active patients. Clinicians in our study rated the appropriateness with the perception that the expected post-operative outcomes factors were accurately predicted. Probability estimates, which will be used in clinical practice, may create greater uncertainty for the end user as compared to the highly accurate estimates we assumed in the current study (41). For example, work by Bayliss and colleagues demonstrated that for men, aged 50–55 years, the 5-year revision risk was approximately 17%. Estimates like those by Bayliss and colleagues, while still useful, create more uncertainty regarding the outcome as compared to our highly accurate estimates. Misclassification of scenarios in the CART model was not trivial, reinforcing the need for further validation before using these scenarios in clinical practice. The expert panel of orthopedic surgeons and specialists were a convenience sample and their views may not be representative of all clinicians’ views treating this patient population. Finally, unlike the RAND method, panel members only emailed their individual ratings, and were not convened to discuss these ratings in person. However, Tobacman and colleagues showed that there is substantial agreement between mail-in ratings and in-person ratings (42).

In conclusion, we have identified 2 expected post-operative outcome factors, patient-reported function and revision, that clinicians found valuable in making recommendations regarding TKA. Given the large number of TKAs being performed in the US, we believe this new approach of combining pre-operative indications and expected post-operative outcomes can aid clinicians in making more informed recommendations, and thus increase the timely and appropriate use of TKA among KOA patients.

Significance and Innovations.

• Recommending total knee arthroplasty to patients is informed by variables derived from the patient’s preoperative status.

• Availability of expected outcomes (e.g., expected survivorship after a cancer treatment) have improved decision making around treatment options in many clinical fields, but their use to inform TKA decision making has been limited.

• We used established appropriateness criteria methodology to investigate how clinicians, who treat patients considering having TKA, would use 3 expected outcomes (risk of serious complications, time to revision, and improvement in pain and function at 2 years after surgery) to inform their recommendation of TKA, and found that time to revision, and improvement in pain and function at 2 years after surgery were fundamental to these recommendations.

• Given that data on these expected outcomes are widely available, using this data in clinical practice has the potential to improve TKA decision making.