Assessing the Validity of a New Prediction Model for Patient Satisfaction After Total Knee Arthroplasty: A Retrospective Cross-Sectional Study

Abstract

Purpose

Previously, a simplified model using statistically selected questionnaires from various patients reported outcome measures (PROMs) was proposed to predict patient satisfaction after total knee arthroplasty (TKA). However, this simple and useful model needs to be validated across ethnic and cultural differences. The objective of this study was to evaluate the utility of this predictive model in Japanese patients.

Patients and Methods

Of all knees treated using primary TKA at our institution between August 2017 and June 2018, this study involved 50 knees of 48 patients (11 men, 37 women) to whom the predictive model was applied preoperatively and from whom PROMs were obtained at least 1 year postoperatively. To evaluate PROMs, patients completed the KSS and the Forgotten Joint Score-12. Correlations were analyzed between preoperatively predicted postoperative patient satisfaction and actual postoperative patient satisfaction, as well as each PROM.

Results

KSS satisfaction improved from 15.6 ± 6.1 preoperatively to 27.8 ± 8.3 postoperatively, with satisfaction reported for 41 knees (82%). The preoperatively predicted postoperative patient satisfaction score was 26.3 ± 4.6, with no significant correlation with actual postoperative score (r = 0.05, p = 0.72). The difference between preoperatively predicted patient satisfaction and actual postoperative patient satisfaction was positively correlated with the score for question 9 of the Pain Catastrophizing Scale, among other instruments constituting the predictive model.

Conclusion

Our data suggest that the predictive model had a low predictive value and that it had limited applicability to Japanese patients. The results also suggest that a tendency toward catastrophic thinking is associated with discrepancy between preoperatively predicted postoperative patient satisfaction and actual postoperative patient satisfaction. The predictive model has low utility and needs some modification.

Introduction

Total knee arthroplasty (TKA) is a widely accepted surgical technique that is effective for relieving pain and achieving functional improvement.^1,2 Patient-reported outcome measures (PROMs) are increasingly used for pre- and postoperative evaluation, including pain relief, functional outcome, and satisfaction.³ The 2011 Knee Society Score (KSS) is also used as a measure of patient satisfaction after TKA. Vigorous debate has surrounded patient satisfaction and the 2011 KSS has been recognized as an important tool for assessing patient satisfaction after TKA.^4–8 Various factors have been shown to affect postoperative patient satisfaction but it remains difficult to identify preoperatively those patients likely to report high postoperative satisfaction.^9,10 Van Onsem et al recently proposed and demonstrated the utility of a simple model for predicting postoperative satisfaction.¹¹ This model comprises a combination of selected items from several PROMs that have been statistically identified as useful for predicting postoperative patient satisfaction score of 2011 KSS: PROMs used in this prediction model include Knee injury and Osteoarthritis Outcome Score (KOOS), EuroQol 5 Dimension, and Pain Catastrophizing Scale (PCS). This prediction model comprises 10 factors including sex, age, 2 items from EQ-5D, 3 items from KOOS, 2 items from PCS and 1 item from KSS.

Preoperatively predicting postoperative patient satisfaction using this simple model would be helpful for patient selection and education and would ultimately contribute to improved overall patient satisfaction after TKA. However, ethnic and cultural differences may affect the ability to predict patient satisfaction. Therefore, the prediction model should be validated across many countries. The objective of this study was to evaluate the validity and utility of the predictive model in Japanese patients. Our hypothesis was that postoperative patient satisfaction predicted by the model would correlate well with actual postoperative patient satisfaction, thus, demonstrating the potential of this model as a helpful tool for preoperative patient education and selection.

Patients and Methods

Of all knees treated with primary TKA at our institution which was tertiary medical care center between August 2017 and June 2018, this study involved 50 knees of 48 patients (11 men, 37 women) to whom the predictive model was applied preoperatively and from whom PROMs were obtained at least 1 year postoperatively. Mean age at surgery was 74.6 ± 7.6 years. Of the 50 knees, 46 had primary knee osteoarthritis (all of them were Kellgren-Lawrence classification grade 3 or 4) and 4 had rheumatoid arthritis; 44, 4, and 2 were treated using the bicruciate stabilized, posterior stabilized, and mobile cruciate substituting designs, respectively. Surgeries were performed by two experienced surgeons using a tourniquet via a subvastus approach with a gap-balancing technique to achieve neutral mechanical alignment. Postoperative drainage was not used in all cases. Patients were allowed to ambulate with full weight bearing as pain permitted from the day after surgery and underwent rehabilitation without restriction to their range of motion.

To evaluate PROMs, patients were asked by the attending surgeon to complete the KSS and the Forgotten Joint Score-12 (FJS-12). For patients who underwent bilateral TKA, PROMs were assessed for each knee. Preoperative PROMs and predictive model questions were assessed at the same time and postoperative PROMs were assessed at the 1-year postoperative visit.

This simple useful model comprising a small number of items is expressed as follows:

Predictive model = 26.10 + 2.3*gender (Female 0, Male 1) + 0.13*age + 1.59*Q3 (EQ-5L: pain) – 1.40*Q4 (KOOS S6) – 1.08*Q5 (KOOS S2) + 0.75*Q6 (KSS I-3) – 1*Q7 (KOOS Q1) – 1.12*Q8 (EQ-5L: anxiety) – 0.88*Q9 (PCS Q9) – 1.10*Q10 (PCS Q13)

Q3 (EQ-5L:pain): I have no pain or discomfort = 1 point, I have slight pain or discomfort = 2 points, I have moderate pain or discomfort = 3 points, I have severe pain or discomfort = 4 points, I have extreme pain or discomfort = 5 points.

Q4 (KOOS S6): During the last week, how severe was your knee joint stiffness after first wakening in the morning? None = 0 points, mild = 1 point, moderate = 2 points, severe = 3 points, extreme = 4 points.

Q5 (KOOS S2): During the last week, did you feel grinding, hear clicking or any other type of noise when your knee moves? Never = 0 points, rarely = 1 point, sometimes = 2 points, often = 3 points, always = 4 points.

Q6 (KSS I-3): Does your knee feel “normal” to you? Never = 0 points, sometimes = 3 points, always = 5 points.

Q7 (KOOS Q1): How often are you aware of your knee problem? Never = 0 points, monthly = 1 point, weekly = 2 points, daily = 3 points, constantly = 4 points.

Q8 (EQ-5L: anxiety): I am not anxious or depressed = 1 point, I am slightly anxious or depressed = 2 points, I am moderately anxious or depressed = 3 points, I am severely anxious or depressed = 4 points, I am extremely anxious or depressed = 5 points.

Q9 (PCS Q9): When I am in pain I cannot seem to keep it out of my mind. Not at all = 0 points, to a slight degree = 1 point, to a moderate degree = 2 points, to a great degree = 3 points, all the time = 4 points.

Q10 (PCS Q13): When I am in pain I wonder whether something serious may happen. Not at all = 0 points, to a slight degree = 1 point, to a moderate degree = 2 points, to a great degree = 3 points, all the time = 4 points.

Correlations were analyzed between preoperatively predicted postoperative patient satisfaction and actual postoperative patient satisfaction and for each PROM. KSS scores ≥20 (out of 40) were considered indicative of satisfaction and scores of <20 dissatisfaction. For statistical analysis, Bland-Altman analysis was used for errors between two groups, and Spearman’s rank correlation coefficient was used for correlations between preoperatively predicted postoperative patient satisfaction and actual postoperative patient satisfaction, differences in preoperatively predictive and actual postoperative patient satisfaction (Δsatisfaction) and each PROM. All analyses were performed using JMP software (SAS, Cary, NC), with a significance level of p = 0.05. Statistical power analysis determined that 34 knees were necessary to certify a correlation of 0.25 with 80% power and an alpha of 0.05.

This study was a retrospective cross-sectional study and was conducted with approval by the ethics committee of our institution (Tokyo Women’s Medical University-4952). All procedures performed in this study involving human participants were in accordance with the 1964 Helsinki declaration. Informed consent for the study was obtained from all patients preoperatively.

Results

The results of pre- and postoperative PROMs and satisfaction prediction are shown in Table 1. KSS satisfaction improved from 15.6 ± 6.1 preoperatively to 27.8 ± 8.3 postoperatively, with satisfaction reported for 41 knees (82%). The preoperatively predicted patient satisfaction score was 26.3 ± 4.6, with no significant correlation observed with actual postoperative patient satisfaction (r = 0.05, p = 0.72; Figure 1). Bland-Altman analysis showed no clear systematic error between preoperatively predicted and actual postoperative patient satisfaction (Figure 2). Δsatisfaction showed a moderate positive correlation with the score for question 9 of the PCS (PCS Q9, “When I’m in pain I can’t seem to keep it out of my mind”), which is included in the predictive formula (r = 0.57, p <0.0001). Multivariate analysis showed that PCS Q9 was also moderately correlated with the pre- and postoperative difference in KSS pain (ΔKSS pain = preoperative pain score − postoperative pain score) (r = −0.49, p = 0.0004).

Table 1.

Pre- and Postoperative Patient-Reported Outcome Measures and Predictive Satisfaction Score

	Pre-TKA	Post-TKA
KSS satisfaction	15.6±6.1	27.8±8.3
KSS total	82.7±21.2	123.1±32.3
FJS-12	20.8±13.0	48.7±24.5
Predictive score	-	26.3±4.6

Abbreviations: FJS-12, Forgotten Joint Score-12; KSS, Knee Society Score; TKA, Total Knee Arthroplasty; FJS-12, Forgotten Joint Score-12; KSS, Knee Society Score; PCS, Pain Catastrophizing Score; PROMs, patient-reported outcome measures; TKA, total knee arthroplasty.

Figure 1.

One-year KSS satisfaction compared with the predicted score (r=0.01, p=0.92).

Figure 2.

Bland–Altman analysis.

Discussion

The most important finding from this study is the difficulty in predicting postoperative patient satisfaction using the previously reported predictive model. The results lead us to reject our hypothesis and suggest the difficulty of preoperatively identifying patients likely to report high postoperative satisfaction based on the predictive score.

Predicting postoperative satisfaction is increasingly important for successful surgical intervention.¹² Various factors have been reported to influence postoperative satisfaction. Bourne et al have identified the following factors as being associated with low postoperative satisfaction: disappointing results, low Western Ontario and McMaster Universities Osteoarthritis Index (WOMAC) scores, preoperative pain at rest, and postoperative complications requiring readmission.⁶ Kwon et al have reported that of all scores in the WOMAC, the function score was the most closely correlated with postoperative patient satisfaction.⁷ Aside from PROMs, Matsuda et al reported that advanced age and varus alignment also correlate with postoperative patient satisfaction.⁸ Several attempts have been made to identify preoperative patient factors associated with high postoperative satisfaction but without success.^1,9 In such circumstances, Van Onsem et al developed a simple predictive model.¹² This model consists of only eight questions, except for basic information such as age and gender, and was considered a highly useful preoperative tool for predicting postoperative satisfaction. In their study, severity of OA on preoperative AP standing radiographs was not related to the postoperative satisfaction. However, a supplementary study conducted by Calkins et al in the United States showed that the predictive value of this model was low because it may produce inconsistent results depending on patients’ nationality and cultural background.¹³ Our data also suggest the low predictive value of the model and its limited applicability to Japanese patients. With regard to the validity of the KSS, Nobel et al¹⁴ stated that the KSS needs to be validated in other countries in addition to the US and Canada. Similarly, evaluating the validity of the predictive score in each region or country is considered to be important.

In this study, there was no significant correlation between preoperatively predicted and actual postoperative patient satisfaction and Bland-Altman analysis showed only random error, but no clear systematic error. To decrease the impact of these random errors, a larger sample size is required. Further studies may nevertheless interpret the utility of the predictive formula differently.

To determine the cause of the discrepancy between preoperatively predicted and actual postoperative patient satisfaction, we investigated the association of Δsatisfaction with preoperative PROMs. The results suggest an association between a tendency toward catastrophic thinking and a discrepancy between preoperatively predicted and actual postoperative patient satisfaction. PCS score is generally understood to correlate with postoperative pain.¹⁵ Bierke et al demonstrated a correlation between PCS score and poor patient satisfaction after TKA.¹⁶ Our data showed a positive and moderate correlation between the PCS Q9 score and Δsatisfaction. Among the PCS questions, this question is originally intended to assess “rumination,” which represents the tendency to think repetitively about pain.¹⁷ Higher scores for the question appear to reflect higher satisfaction in patients who had undergone TKA that was effective in relieving pain. In fact, patients with higher PCS Q9 scores also showed greater improvement in KSS pain. In other words, “better than expected” outcomes experienced by patients might affect their way of thinking about pain. Recent evidence suggests an association between central sensitization and postoperative pain after TKA.¹⁸ Further studies are required to investigate how central sensitization affects postoperative pain and satisfaction using such measures as the Central Sensitization Inventory score.¹⁹

We had enrolled 48 patients and among them, 4 people had rheumatoid arthritis. No difference was observed regarding the percentage in the original study by Van Onsem et al.¹¹ They had enrolled 113 patients and among them, 4% of patients had rheumatoid arthritis. Therefore, we think it is proper to combine rheumatoid arthritis patients with this study. The current study did not take into account comorbidities in extremities, such as hip arthritis, trauma and fractures. Those comorbidities may affect the postoperative satisfaction. However, the original study by Van Onsem et al¹¹ did not include the status of those comorbidities in their prediction model. This might be another reason of that the validity of this model was not proved by other studies including us.

This study has a few limitations. First, satisfaction was not assessed at 3 months postoperatively, which was the original assessment time point for the predictive score. Instead, we assessed satisfaction at 1 year postoperatively. However, Calkin et al¹⁰ assessed scores at 3 months and 1 year postoperatively and found no difference. Furthermore, van Onsem et al originally stated in their report that 3 months postoperatively is a premature time for surveillance because it is a rehabilitation period and the patient is still recovering. They mentioned that the predictive score could be a good indicator of satisfaction at 1 year postoperatively. Second, a small sample size was used. However, postoperative patient satisfaction was similar to that reported previously, suggesting that the cohorts were similar. Third, predictive scores could not be collected from all patients undergoing primary TKA, which possibly resulted in selection bias. However, results from a cross-sectional study would not likely be affected by missing samples.

Conclusion

We found that Van Onsem et al’s predictive model for postoperative patient satisfaction was of limited utility in a cohort of Japanese patients. The model may need some modifications taking into account the possible effects of ethnic and cultural differences.

Disclosure

Ken Okazaki reports personal fees from Smith & Nephew, Zimmer Biomet, and Olympus Terumo Biomaterials. The authors report no other potential conflicts of interest in this work.