Key Points
Question
What are the minimal clinically important differences in the Hip Disability and Osteoarthritis Outcome Score and in the Knee Injury and Osteoarthritis Outcome Score for veterans undergoing total hip or knee arthroplasty?
Findings
In this cohort study, a wide range of minimal clinically important differences were obtained depending on method. Methods anchored on the Self-Administered Patient Satisfaction Scale are recommended because they showed good to excellent discrimination based on receiver operating characteristic curve analysis.
Meaning
Because patient-reported outcome measures and their minimal clinically important differences may be used to guide policy and reimbursement, accurate minimal clinically important differences derived with clearly defined methods are required.
Abstract
Importance
The minimal clinically important difference (MCID) in a patient-reported outcome measure (PROM) is the smallest change that patients perceive as beneficial. Accurate MCIDs are required when PROMs are used to evaluate the value of surgical interventions.
Objective
To use well-defined distribution-based and anchor-based methods to calculate MCIDs in the Hip Disability and Osteoarthritis Outcome Score (HOOS) and in the Knee Injury and Osteoarthritis Outcome Score (KOOS) for veterans undergoing primary total hip arthroplasty or total knee arthroplasty.
Design, Setting, and Participants
A prospective cohort study was conducted of 858 patients undergoing total joint replacement between March 16, 2015, and March 9, 2017, at 3 high-complexity Veterans Affairs Medical Centers.
Interventions
Patients undergoing total hip arthroplasty or total knee arthroplasty were administered HOOS or KOOS PROMs prior to and 1 year after surgery. The Self-Administered Patient Satisfaction Scale (SAPS) for primary hip or knee arthroplasty was administered at 1-year follow-up as an anchor PROM.
Main Outcomes and Measures
The HOOS and KOOS before and 1 year after surgery, change scores (difference between postoperative and preoperative PROM scores), and MCIDs for each measure. For anchor-based methods, receiver operating characteristic curve analysis was performed, including calculation of the area under the curve.
Results
The mean (SD) age of the 271 patients who underwent hip arthroplasty was 65.6 (8.3) years, and the mean (SD) age of the 587 patients who underwent knee arthroplasty was 66.1 (8.2) years. There were 547 men in the knee arthroplasty cohort and 256 men in the hip arthroplasty cohort (total, 803 men). There were significant improvements in the mean values of every PROM, with mean (SD) differences greater than 39 for HOOS Joint Replacement (JR) and every hip subscale (HOOS JR, 39.7 [20.2]; pain, 47.6 [20.5]; symptoms, 45.1 [21.5]; activities of daily living, 43.7 [22.1]; recreation, 49.2 [33.5]; quality of life, 50.3 [27.8]) and mean (SD) differences greater than 29 for KOOS JR and every knee subscale (KOOS JR, 30.4 [17.5]; pain, 38.0 [20.4]; symptoms, 29.5 [22.1]; activities of daily living, 34.8 [20.5]; recreation, 34.6 [31.1]; quality of life, 35.2 [26.8]). Different calculation methods yielded a wide range of MCIDs. Distribution-based approaches tended to give lower values than the anchor-based approaches, which gave similar values for most PROMs. Area under the curve values demonstrated good to excellent discrimination for SAPS for nearly all PROMs.
Conclusions and Relevance
Minimal clinically important difference estimates can be highly variable depending on the method used. Patient satisfaction measured by SAPS is a suitable anchor for the HOOS and KOOS. This study suggests that the SAPS-anchored MCID values presented here be used in future studies of total hip arthroplasty and total knee arthroplasty for veterans.
This cohort study uses distribution-based and anchor-based methods to calculate minimal clinically important differences in the Hip Disability and Osteoarthritis Outcome Score and in the Knee Injury and Osteoarthritis Outcome Score for veterans undergoing primary total hip arthroplasty and total knee arthroplasty.
Introduction
The US health care system is transitioning from a volume-based fee-for-service model to one that prioritizes high-value patient-centered care. Value can be defined as the benefit of an intervention divided by its cost. Patient-reported outcome measures (PROMs) can be used to quantify the benefits of care by assessing patients’ perceived changes in health status in response to a treatment. However, the clinical importance of a given change in a PROM score is frequently unclear. To address this issue, the concept of the minimal clinically important difference (MCID) was created. The MCID was originally defined as the smallest difference in score in the domain of interest that patients perceive as beneficial. There is no universally accepted method for calculating the MCID. For example, a review of lower extremity PROMs found 11 methods to calculate the MCID, with up to a 4-fold difference in reported values for the same questionnaire. Factors such as the population being studied, the treatment, and the timing of follow-up are associated with the variability in published MCIDs.
Payers, such as Medicare, have begun incorporating PROMs into the assessment of health care value, including surgical care. In 2014, Medicare spent more than $7 billion on hospitalizations for total hip arthroplasty (THA) and total knee arthroplasty (TKA). Given the costs of these lower extremity joint replacements (LEJR), the Centers for Medicare & Medicaid Services has implemented the Comprehensive Care for Joint Replacement reimbursement model. The Comprehensive Care for Joint Replacement model links hospital payment to the quality and value of care delivered. One voluntary aspect of quality measurement is the submission of PROMs before and after surgery. The chosen PROMs include the Hip Disability and Osteoarthritis Outcome Score (HOOS) subscales, the Knee Injury and Osteoarthritis Outcome Score (KOOS) subscales, the HOOS Joint Replacement (JR), and the KOOS JR. The HOOS consists of 40 questions within 5 subscales: symptoms (5 items), pain (10 items), activities of daily living (17 items), sports and recreation (4 items), and quality of life (4 items). The KOOS contains 42 items with 7 symptom items, 9 pain items, 17 activities of daily living items, 5 sports items, and 4 quality-of-life items. The HOOS JR consists of 6 items from the HOOS; the KOOS JR consists of 7 items from the KOOS. Both JR scores have been shown to be valid and reliable measures of joint health for patients undergoing LEJR. The JR PROMs and each subscale are scaled to 100 points, with 100 representing perfect joint health and 0 representing total joint disability. The total HOOS and the total KOOS can be defined as the sum of the 5 subscales (range, 0-500).
The primary goal of this report was to use published, clearly defined methods to estimate MCIDs for HOOS and KOOS PROMs for patients undergoing primary LEJR at Veterans Health Administration hospitals. The Veterans Health Administration is the largest integrated health care system in the United States and has begun to incorporate measurement-based care with PROMs into clinical practice. We used both anchor-based and distribution-based methods, which are the 2 most commonly used approaches to determine MCIDs. In the former approach, differences in a PROM score are compared with an external, related measure of change or “anchor.” In distribution-based methods, the MCID is calculated from a statistical property of the distribution of outcome scores. We also compared our findings with MCIDs derived from other patient cohorts because veterans differ demographically and medically from the general population.
Methods
Patients at 3 high-complexity Veterans Affairs Medical Centers who were scheduled to undergo elective primary THA or TKA from March 16, 2015, to March 9, 2017, were prospectively enrolled to complete HOOS or KOOS questionnaires within 30 days prior to surgery and 1 year after surgery. Demographic and comorbidity data were also collected. Race/ethnicity data as defined by participants were collected because these factors may affect responses to PROMs. Subscale scores and JR scores were calculated as previously described. For each PROM, the primary end point was the change score, which is defined as the difference between the postoperative score and the preoperative score (−500 to 500 possible range for total HOOS and KOOS scores and −100 to 100 possible range for JR and subscale scores). The protocol for this study was reviewed and approved by the Veterans Affairs Central Institutional Review Board. Written informed consent was obtained from all study participants.
The Self-Administered Patient Satisfaction Scale (SAPS) for primary hip or knee arthroplasty was chosen as an anchor questionnaire; it was administered at the 1-year follow-up on the same day as the postoperative PROMs. The SAPS contains the following 4 items: (1) satisfaction with results of surgery, (2) satisfaction with results of surgery for improving pain, (3) satisfaction with results of surgery for improving ability to do home or yard work, and (4) satisfaction with results of surgery for improving ability to do recreational activities. It is scored on a 4-point Likert scale, with responses consisting of very satisfied (100 points), somewhat satisfied (75 points), somewhat dissatisfied (50 points), and very dissatisfied (25 points). The SAPS total score is the unweighted mean of the individual items, ranging from 25 to 100.
We used 2 separate SAPS thresholds as anchors: SAPS total score greater than 50 (on average, better than somewhat dissatisfied for all items) and all 4 SAPS items greater than 50 (somewhat satisfied or very satisfied for all items). Using the Youden Index method and based on receiver operating characteristic curve analysis, we selected the MCID cutpoint as the improvement in PROM score that balanced sensitivity and specificity of PROM change thresholds for each SAPS anchor. The area under the curve (AUC) was calculated for each receiver operating characteristic curve. We used 2 previously published, distribution-based approaches to estimate the MCID. First, we used one-half of the SD of baseline (preoperative) scores. Second, we used one-half of the SD of change scores.
To assess whether participants who completed the 1-year follow-up significantly differed on baseline variables from those who dropped out, we conducted t tests for continuous variables (eg, age) and tests of independent proportions for categorical variables (eg, race/ethnicity); t tests were also used to assess for significant differences between preoperative and postoperative PROM scores. All P values were from 2-sided tests, and the results were deemed statistically significant at P < .05. Pearson correlation coefficients were calculated to evaluate the correlations between PROM change scores and SAPS question 1 (satisfaction with results of surgery). Analyses were performed using R, version 3.6.0 (R Project for Statistical Computing).
Results
Cohort Characteristics
A total of 1220 patients were approached for enrollment, and 858 patients completed this study (Figure 1); 308 patients undergoing THA and 637 undergoing TKA completed preoperative PROM questionnaires. Most patients were male (803), obese (mean [SD] body mass index: THA, 31.2 [5.4]; TKA, 33.3 [12.6]; calculated as weight in kilograms divided by height in meters squared), and white (751) (Table 1). Comorbidities were prevalent in both cohorts, with especially high rates of hypertension (THA, 168 of 308 [54.6%]; TKA, 392 of 637 [61.5%]), type 1 and 2 diabetes (THA, 61 of 308 [19.8%]; TKA, 166 of 637 [26.1%]), and depression (THA, 74 of 308 [24.0%]; TKA, 145 of 637 [22.8%]). Patients had low baseline PROM scores, with, for example, a mean (SD) HOOS JR score of 41.5 (13.8) and a mean (SD) KOOS JR score of 45.8 (13.0) of a possible 100 points. There was an 88.0% retention rate (271 of 308) for patients undergoing THA and a 92.2% retention rate (587 of 637) for patients undergoing TKA. Follow-up PROM and the SAPS anchor responses were collected at the same session. The mean (SD) follow-up was 380 (29) days, and the median follow-up was 379 days (range, 242-491 days). Follow-up patients and patients who dropped out of the study had similar characteristics. In the THA cohort, there were significantly higher percentages of African American and Asian patients among those who dropped out (African American, 6 of 37 [16.2%]; Asian, 3 of 37 [8.1%]) as well as a higher rate of depression (14 of 37 [37.8%]). There were no significant differences in preoperative PROM scores. In the TKA cohort, there was a significantly lower percentage of white patients among those who dropped out (32 of 50 [64.0%]). There were no significant differences in the prevalence of comorbidities. The only significant difference in preoperative PROM scores was a lower mean (SD) KOOS recreation score among dropouts (11.9 [17.0]). Despite the similarities between follow-up patients and patients who dropped out, it is difficult to assess the association of the latter group because they may have had significantly higher or lower postoperative PROM scores than those who remained in the study.
Figure 1. Study Cohort.
Table 1. Preoperative Characteristics of Patients at Baseline and Follow-up and of Patients Who Dropped Out of the Study.
| Variable | Hip | P value (follow-up vs dropped out) | Knee | P value (follow-up vs dropped out) | ||||
|---|---|---|---|---|---|---|---|---|
| Baseline (n = 308) | Follow-up (n = 271) | Dropped out (n = 37) | Baseline (n = 637) | Follow-up (n = 587) | Dropped out (n = 50) | |||
| Age, mean (SD), y | 65.4 (8.5) | 65.6 (8.3) | 64.3 (10.0) | .45 | 66.1 (8.3) | 66.1 (8.2) | 66.8 (9.1) | .67 |
| BMI, mean (SD) | 31.2 (5.4) | 31.3 (5.3) | 30.2 (5.8) | .31 | 33.3 (12.6) | 33.4 (13.1) | 32.6 (5.0) | .38 |
| Female, No. (%) | 15 (4.9) | 15 (5.5) | 0 | .29 | 42 (6.6) | 40 (6.8) | 2 (4.0) | .44 |
| Race/ethnicity, No. (%) | ||||||||
| White | 251 (81.5) | 225 (83.0) | 26 (70.2) | .10 | 500 (78.5) | 468 (79.7) | 32 (64.0) | .009 |
| African American | 27 (8.8) | 21 (7.9) | 6 (16.2) | .02 | 51 (8.0) | 45 (7.7) | 6 (12.0) | .28 |
| Hispanic | 19 (6.2) | 18 (6.6) | 1 (2.7) | .57 | 61 (9.6) | 54 (9.2) | 7 (14.0) | .27 |
| Asian | 5 (1.6) | 2 (0.7) | 3 (8.1) | .01 | 16 (2.5) | 13 (2.2) | 3 (6.0) | .10 |
| Native American | 6 (1.9) | 5 (1.8) | 1 (2.7) | .99 | 23 (3.6) | 23 (3.9) | 0 | .15 |
| Comorbidities, No. (%) | ||||||||
| Hypertension | 168 (54.6) | 148 (54.6) | 20 (54.1) | .95 | 392 (61.5) | 361 (61.5) | 31 (62.0) | .94 |
| Depression | 74 (24.0) | 60 (22.1) | 14 (37.8) | .04 | 145 (22.8) | 138 (23.5) | 7 (14.0) | .12 |
| Type 1 and 2 diabetes | 61 (19.8) | 52 (19.2) | 9 (24.3) | .46 | 166 (26.1) | 152 (25.9) | 14 (28.0) | .74 |
| Chronic pulmonary disease | 39 (12.7) | 35 (12.9) | 4 (10.8) | .72 | 85 (13.3) | 78 (13.3) | 7 (14.0) | .89 |
| Alcohol use disorder | 34 (11.0) | 30 (11.1) | 4 (10.8) | .96 | 58 (9.1) | 51 (8.7) | 7 (14.0) | .21 |
| Drug use disorder | 32 (10.4) | 25 (9.2) | 7 (18.9) | .07 | 34 (5.4) | 33 (5.6) | 2 (4.0) | .63 |
| Preoperative PROM scores, mean (SD) | ||||||||
| HOOS | ||||||||
| Total | 162.0 (73.1) | 162.7 (74.1) | 157.2 (66.7) | .65 | NA | NA | NA | NA |
| Pain | 38.6 (15.2) | 38.7 (15.6) | 37.4 (12.7) | .57 | NA | NA | NA | NA |
| Symptoms | 41.9 (17.7) | 42.1 (17.7) | 41.0 (18.3) | .73 | NA | NA | NA | NA |
| Activities of daily living | 40.2 (16.3) | 40.4 (16.6) | 38.1 (14.6) | .39 | NA | NA | NA | NA |
| Recreation | 21.3 (19.2) | 21.5 (19.5) | 20.1 (17.4) | .67 | NA | NA | NA | NA |
| Quality of life | 20.4 (17.2) | 20.1 (17.2) | 22.3 (17.7) | .48 | NA | NA | NA | NA |
| JR | 41.5 (13.8) | 41.4 (13.9) | 42.2 (12.7) | .75 | NA | NA | NA | NA |
| KOOS | ||||||||
| Total | NA | NA | NA | NA | 180.1 (71.9) | 181.7 (70.7) | 161.0 (83.4) | .09 |
| Pain | NA | NA | NA | NA | 43.9 (16.4) | 44.1 (16.1) | 41.2 (19.6) | .32 |
| Symptoms | NA | NA | NA | NA | 49.9 (19.1) | 50.3 (18.9) | 45.1 (20.1) | .08 |
| Activities of daily living | NA | NA | NA | NA | 47.5 (17.7) | 47.7 (17.4) | 44.4 (20.6) | .27 |
| Recreation | NA | NA | NA | NA | 17.6 (19.6) | 18.1 (19.7) | 11.9 (17.0) | .02 |
| Quality of life | NA | NA | NA | NA | 21.4 (16.0) | 21.6 (15.6) | 18.4 (19.4) | .26 |
| JR | NA | NA | NA | NA | 45.8 (13.0) | 46.1 (12.7) | 43.3 (16.1) | .26 |
Abbreviations: BMI, body mass index (calculated as weight in kilograms divided by height in meters squared); HOOS, Hip Disability and Osteoarthritis Outcome Score; JR, joint replacement; KOOS, Knee Injury and Osteoarthritis Outcome Score; NA, not applicable; PROM, patient-reported outcome measure.
Changes in PROMs After Surgery
There was a wide range in the differences in change scores (the difference between postoperative and preoperative scores) for each PROM, ranging from negative scores (indicating worse health status after surgery) to maximal or near maximal improvement (100 for subscale and JR scores and 500 for total scores; Table 2). There were significant improvements in the mean values of every PROM, with mean (SD) differences greater than 39 for HOOS JR and every hip subscale (HOOS JR, 39.7 [20.2]; pain, 47.6 [20.5]; symptoms, 45.1 [21.5]; activities of daily living, 43.7 [22.1]; recreation, 49.2 [33.5]; quality of life, 50.3 [27.8]) and mean (SD) differences greater than 29 for KOOS JR and every knee subscale (KOOS JR, 30.4 [17.5]; pain, 38.0 [20.4]; symptoms, 29.5 [22.1]; activities of daily living, 34.8 [20.5]; recreation, 34.6 [31.1]; quality of life, 35.2 [26.8]). The mean improvements in each category were greater for patients undergoing THA than for those undergoing TKA. In general, satisfaction with the results of surgery increased with increasing change scores, with Pearson correlation coefficients ranging from 0.35 to 0.53 (Figure 2).
Table 2. Change Scores for PROMa.
| PROM | Mean (SD) [range] | Median (IQR) [1st quartile to 3rd quartile] |
|---|---|---|
| HOOS | ||
| Total | 236.0 (53.8) [−121.0 to 490.0] | 256.7 (139.2) [168.2 to 307.4] |
| Pain | 47.6 (20.5) [−20.0 to 100.0] | 47.5 (22.5) [37.5 to 60.0] |
| Symptoms | 45.1 (21.5) [−20.0 to 95.0] | 45.0 (30.0) [30.0 to 60.0] |
| Activities of daily living | 43.7 (22.1) [−32.3 to 100.0] | 46.5 (27.8) [30.8 to 58.6] |
| Recreation | 49.2 (33.5) [−68.8 to 100.0] | 56.3 (50.0) [25.0 to 75.0] |
| Quality of life | 50.3 (27.8) [−31.3 to 100.0] | 50.0 (37.5) [31.3 to 68.8] |
| JR | 39.7 (20.2) [−19.1 to 100.0] | 40.6 (26.4) [27.0 to 53.4] |
| KOOS | ||
| Total | 172.2 (96.9) [−146.8 to 369.9] | 173.3 (125.3) [111.5 to 236.8] |
| Pain | 38.0 (20.4) [−38.9 to 88.9] | 38.9 (25.0) [25.0 to 50.0] |
| Symptoms | 29.5 (22.1) [−46.4 to 92.9] | 28.6 (28.6) [14.3 to 42.9] |
| Activities of daily living | 34.8 (20.5) [−39 to 100.0] | 35.3 (26.5) [22.1 to 48.6] |
| Recreation | 34.6 (31.1) [−75 to 100.0] | 31.3 (47.1) [12.9 to 60.0] |
| Quality of life | 35.2 (26.8) [−37.5 to 93.8] | 37.5 (37.5) [18.7 to 56.2] |
| JR | 30.4 (17.5) [−28.4 to 91.7] | 29.3 (21.4) [19.8 to 41.2] |
Abbreviations: HOOS, Hip Disability and Osteoarthritis Outcome Score; IQR, interquartile range; JR, joint replacement; KOOS, Knee Injury and Osteoarthritis Outcome Score; PROM, patient-reported outcome measure.
The change score is defined as the postoperative score minus the preoperative score.
Figure 2. Comparison of Change Scores With Satisfaction With Results of Surgery Measured Using the Self-Administered Patient Satisfaction Scale (SAPS).
A, Patients who underwent total hip arthroplasty. B, Patients who underwent total knee arthroplasty. A change score is defined as the postoperative score minus the preoperative score. SAPS question 1 is about satisfaction with results of surgery (with a score of 25 = very dissatisfied, 50 = somewhat dissatisfied, 75 = somewhat satisfied, and 100 = very satisfied). Pearson correlation coefficients (r) are shown. The ends of the boxes represent the 25th and 75th percentiles. The horizontal line inside the box indicates the median. The whiskers represent the upper and lower adjacent values. Outlier data are shown as circles plotted beyond the whiskers. ADL indicates activities of daily living; HOOS, Hip Disability and Osteoarthritis Outcome Score; JR, joint replacement; KOOS, Knee Injury and Osteoarthritis Outcome Score; PROM, patient-reported outcome measure; and QoL, quality of life.
Comparison of Distribution- and Anchor-Based MCIDs
The MCIDs derived from distribution-based approaches were generally lower than those from anchor-based approaches (Table 3). The use of 0.5 SD of preoperative scores gave smaller values than the use of 0.5 SD of change scores as well as smaller values than nearly all of the anchor-based estimates. Area under the curve values for both anchor methods were high, with values greater than or equal to 0.75 in all cases and greater than 0.9 in many cases (eFigure in the Supplement). Overall, anchoring to all SAPS items greater than 50 (all responses either slightly or very satisfied) gave higher AUC values than anchoring on SAPS total scores greater than 50 (mean of all responses better than slightly dissatisfied).
Table 3. Distribution- and Anchor-Based MCIDs From Current and Previous Studies.
| Patient-reported outcome measure | Distribution-based MCID | Anchor-based approach | Distribution-based approach in Blevins et al, 2019 (0.5 SD of change score) | ||||||
|---|---|---|---|---|---|---|---|---|---|
| SAPS total score (mean of all items) >50 | All SAPS items >50a | Paulsen et al, 2014 | Lyman et al, 2018 | ||||||
| 0.5 SD of preoperative scores | 0.5 SD of change scores | MCID | AUC | MCID | AUC | ||||
| KOOS | |||||||||
| JR | 6.4 | 8.7 | 17.5 | 0.83 | 20.8 | 0.86 | NA | 14 | NA |
| Total | 35.4 | 48.5 | 82.9 | 0.91 | 91.8 | 0.91 | NA | NA | NA |
| Pain | 8.1 | 10.2 | 22.3 | 0.83 | 25.0 | 0.88 | NA | 18 | 10.3 |
| Symptoms | 9.5 | 11.1 | 10.8 | 0.80 | 14.3 | 0.92 | NA | 7 | 12 |
| ADL | 8.7 | 10.3 | 22.1 | 0.84 | 24.6 | 0.82 | NA | 16 | 10 |
| Sport and recreation | 9.9 | 15.6 | 17.5 | 0.80 | 17.5 | 0.75 | NA | NA | 15.8 |
| Quality of life | 7.8 | 13.4 | 12.5 | 0.88 | 12.5 | 0.88 | NA | 17 | 13.2 |
| HOOS | |||||||||
| JR | 7.0 | 10.1 | 18.2 | 0.95 | 17.6 | 0.97 | NA | 18 | NA |
| Total | 37.1 | 53.8 | 77.1 | 0.94 | 77.1 | 0.99 | NA | NA | NA |
| Pain | 7.8 | 10.3 | 27.5 | 0.89 | 25.0 | 0.96 | 24 | 36 | NA |
| Symptoms | 8.9 | 10.8 | 17.0 | 0.77 | 20.0 | 0.99 | NA | 20 | NA |
| ADL | 8.3 | 11.1 | 19.1 | 0.94 | 19.1 | 0.98 | NA | 14 | NA |
| Sport and recreation | 9.8 | 16.8 | 16.7 | 0.90 | 16.7 | 0.92 | NA | NA | NA |
| Quality of life | 8.6 | 13.9 | 6.0 | 0.93 | 7.0 | 0.98 | 17 | 13 | NA |
Abbreviations: ADL, activities of daily living; AUC, area under the curve; HOOS, Hip Disability and Osteoarthritis Outcome Score; JR, joint replacement; KOOS, Knee Injury and Osteoarthritis Outcome Score; MCID, minimal clinically important difference; NA, not available; SAPS, Self-Administered Patient Satisfaction Scale.
Recommended values for a veteran population.
Discussion
The wide range of MCIDs that we found illustrates the difficulty of establishing a single “true” value for the MCID. Investigators must select from numerous anchor- and distribution-based methods. Other authors have recommended against the use of distribution-based approaches because they are reflective of statistical properties of samples of patient responses rather than factors that are clinically important to patients. For anchor-based approaches, an appropriate anchor questionnaire must be chosen as well as the anchor response or threshold that corresponds to a minimal important change. We used receiver operating characteristic curve analysis for our anchored methods because the AUC provides information on how well a PROM discriminates between patients who did and patients who did not achieve a clinically important improvement. An AUC value of 1 indicates that the PROM correctly categorizes all patients, while a value of 0.5 is equivalent to results obtained by chance. Anchoring to all SAPS items greater than 50 gave the best AUC values. We therefore recommend the use of the MCIDs calculated with this anchor in future studies of LEJR for veterans, with the acknowledgment that changes in a single aspect of satisfaction rather than all 4 aspects may also lead to clinically important differences.
Satisfaction and clinical improvement are overlapping but distinct domains. Although patient satisfaction can be significantly associated with outcomes, such as pain and function, it can also reflect factors not directly associated with health status, such as travel distance, educational level, and insurance status. The PROM change scores in our cohort were weakly to moderately correlated with satisfaction with the results of surgery (Figure 2). Some patients in our cohort reported clinical worsening after surgery but were very satisfied. Conversely, some patients improved clinically but reported dissatisfaction. We therefore recommend the collection of both PROM and satisfaction scores, as PROMs provide granular information about important clinical outcomes such as pain, symptoms, and quality of life that might not be fully captured by a satisfaction score.
To our knowledge, this report is one of the few publications reporting on MCIDs for HOOS and KOOS questionnaires after total joint arthroplasty (Table 3) and is the first report on a veteran population. Paulsen et al used anchor-based methods to calculate minimal clinically important improvements of Danish patients undergoing THA. They used a hip-specific anchor question (“Overall, how are the problems now in the hip on which you had surgery compared with before your operation?”) and used a mean change approach. Their value for minimal clinically important improvements for the HOOS pain subscale was similar to our value. In a study of Medicare patients undergoing THA or TKA at a single high-volume urban hospital, Lyman et al calculated MCIDs using both distribution- and anchor-based approaches. They used 0.5 SD of the change score as their distribution-based approach and noted that this method was invalid because it gave results smaller than minimal detectable changes. The quality-of-life satisfaction item from the Hospital for Special Surgery Satisfaction Survey was used as the anchor question (“How much did your surgery improve the quality of your life?”). Most of the anchor-based MCIDs from their work were smaller than the ones we calculated from our veteran cohort, although the estimate for the HOOS symptoms subscale was identical to ours. Because most male Veterans Health Administration enrollees have similar health and demographic characteristics as male Medicare beneficiaries, the differences between the results of Lyman et al and our results may be due to the male predominance of the veteran population compared with the Medicare population. Blevins et al used the same distribution-based approach to calculate MCIDs for patients undergoing TKA from the same institution as Lyman et al. Most of these values were smaller than the anchor-based values that we recommend. Berliner et al also used one-half the SD of change scores to estimate MCIDs for the HOOS and KOOS total scores for patients who underwent LEJR at a single academic hospital. However, they scaled the scores from 0 to 100, and it is unclear whether their MCID estimate of 9.1 for HOOS and 10 for KOOS should be multiplied by 5 to allow for comparison with the current findings.
Limitations
This study has some limitations. First, the difference in sex distribution in the veteran and general populations suggests that these findings may not be applicable to female patients. Second, our follow-up at 1 year is an early time point and may not reflect mid-term to long-term clinical outcomes.
Conclusions
We present MCIDs for the HOOS and KOOS subscale and JR scores for veterans who underwent LEJR. We recommend the values obtained using an anchor-based method. The high AUC values from receiver operating characteristic curve analysis suggest that the SAPS is an appropriate anchor for LEJR PROMs. Because clinical benefit measured by PROMs may be associated with policy and reimbursement in the future, it is critical to understand how MCIDs are derived. Use of too small an MCID will overestimate the number of patients who benefit from a procedure. Use of too large an MCID will underestimate the number of patients who benefit. Our results may be generalizable to male Medicare beneficiaries given their similarity to Veterans Health Administration patients. However, rather than using a single number, it may be preferable to present and consider a range of MCIDs. Based on this and other anchor-based studies (Table 3), the MCID would be 17.6 to 18 for the HOOS JR, and the MCID range would be 24 to 36 for the HOOS pain subscale, 20 for the HOOS symptoms subscale, 14 to 19.1 for the HOOS activities of daily living subscale, and for the HOOS quality-of-life subscale would be 7 to 17. The range for knee PROMs would be 14 to 20.8 for the KOOS JR, 18 to 25 for the KOOS pain subscale, 7 to 14.3 for the KOOS symptoms subscale, 16 to 24.6 for the KOOS activities of daily living subscale, and 12.5 to 17 for the KOOS quality-of-life subscale. There is insufficient information to recommend ranges for the other PROMs. These ranges may be more appropriate than a single MCID when applied to other populations; however, caution should be used when evaluating groups with markedly different health care contexts or characteristics. Because MCID data are limited, additional studies in diverse populations are required to guide clinicians, payers, and policy makers.
eFigure. Receiver Operating Characteristic Curves
References
- 1.Squitieri L, Bozic KJ, Pusic AL. The role of patient-reported outcome measures in value-based payment reform. Value Health. 2017;20(6):834-836. doi: 10.1016/j.jval.2017.02.003 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Jaeschke R, Singer J, Guyatt GH. Measurement of health status: ascertaining the minimal clinically important difference. Control Clin Trials. 1989;10(4):407-415. doi: 10.1016/0197-2456(89)90005-6 [DOI] [PubMed] [Google Scholar]
- 3.Copay AG, Eyberg B, Chung AS, Zurcher KS, Chutkan N, Spangehl MJ. Minimum clinically important difference: current trends in the orthopaedic literature, part II: lower extremity: a systematic review. JBJS Rev. 2018;6(9):e2. doi: 10.2106/JBJS.RVW.17.00160 [DOI] [PubMed] [Google Scholar]
- 4.Copay AG, Chung AS, Eyberg B, Olmscheid N, Chutkan N, Spangehl MJ. Minimum clinically important difference: current trends in the orthopaedic literature, part I: upper extremity: a systematic review. JBJS Rev. 2018;6(9):e1. doi: 10.2106/JBJS.RVW.17.00159 [DOI] [PubMed] [Google Scholar]
- 5.Centers for Medicare & Medicaid Services. Comprehensive care for joint replacement model. Updated January 14, 2020. Accessed July 3, 2019. https://innovation.cms.gov/initiatives/CJR
- 6.Centers for Medicare & Medicaid Services. Overview of CJR quality measures, composite quality score, and pay-for-performance methodology. Accessed July 3, 2019. https://innovation.cms.gov/Files/x/cjr-qualsup.pdf
- 7.Klässbo M, Larsson E, Mannevik E. Hip Disability and Osteoarthritis Outcome Score: an extension of the Western Ontario and McMaster Universities Osteoarthritis Index. Scand J Rheumatol. 2003;32(1):46-51. doi: 10.1080/03009740310000409 [DOI] [PubMed] [Google Scholar]
- 8.Roos EM, Roos HP, Lohmander LS, Ekdahl C, Beynnon BD. Knee Injury and Osteoarthritis Outcome Score (KOOS)—development of a self-administered outcome measure. J Orthop Sports Phys Ther. 1998;28(2):88-96. doi: 10.2519/jospt.1998.28.2.88 [DOI] [PubMed] [Google Scholar]
- 9.Lyman S, Lee YY, Franklin PD, Li W, Mayman DJ, Padgett DE. Validation of the HOOS, JR: a short-form hip replacement survey. Clin Orthop Relat Res. 2016;474(6):1472-1482. doi: 10.1007/s11999-016-4718-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Lyman S, Lee YY, Franklin PD, Li W, Cross MB, Padgett DE. Validation of the KOOS, JR: a short-form knee arthroplasty outcomes survey. Clin Orthop Relat Res. 2016;474(6):1461-1471. doi: 10.1007/s11999-016-4719-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Resnick SG, Hoff RA. Observations from the national implementation of Measurement Based Care in Mental Health in the Department of Veterans Affairs. Psychol Serv. Published online May 6, 2019. doi: 10.1037/ser0000351 [DOI] [PubMed] [Google Scholar]
- 12.Knee injury and Osteoarthritis Outcome Score (KOOS). Accessed August 15, 2019. http://www.koos.nu
- 13.Hospital for Special Surgery. HOOS, JR. and KOOS, JR. outcomes surveys. Accessed August 15, 2019. https://www.hss.edu/hoos-jr-koos-jr-outcomes-surveys.asp
- 14.Mahomed N, Gandhi R, Daltroy L, Katz JN. The Self-Administered Patient Satisfaction Scale for primary hip and knee arthroplasty. Arthritis. 2011;2011:591253. doi: 10.1155/2011/591253 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Youden WJ. Index for rating diagnostic tests. Cancer. 1950;3(1):32-35. doi: [DOI] [PubMed] [Google Scholar]
- 16.Norman GR, Sloan JA, Wyrwich KW. Interpretation of changes in health-related quality of life: the remarkable universality of half a standard deviation. Med Care. 2003;41(5):582-592. doi: 10.1097/01.MLR.0000062554.74615.4C [DOI] [PubMed] [Google Scholar]
- 17.Blevins JL, Chiu YF, Lyman S, et al. . Comparison of expectations and outcomes in rheumatoid arthritis versus osteoarthritis patients undergoing total knee arthroplasty. J Arthroplasty. 2019;34(9):1946-1952.e2. doi: 10.1016/j.arth.2019.04.034 [DOI] [PubMed] [Google Scholar]
- 18.Paulsen A, Roos EM, Pedersen AB, Overgaard S. Minimal clinically important improvement (MCII) and patient-acceptable symptom state (PASS) in total hip arthroplasty (THA) patients 1 year postoperatively. Acta Orthop. 2014;85(1):39-48. doi: 10.3109/17453674.2013.867782 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Lyman S, Lee YY, McLawhorn AS, Islam W, MacLean CH. What are the minimal and substantial improvements in the HOOS and KOOS and JR versions after total joint replacement? Clin Orthop Relat Res. 2018;476(12):2432-2441. doi: 10.1097/CORR.0000000000000456 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.McGlothlin AE, Lewis RJ. Minimal clinically important difference: defining what really matters to patients. JAMA. 2014;312(13):1342-1343. doi: 10.1001/jama.2014.13128 [DOI] [PubMed] [Google Scholar]
- 21.Tyser AR, Gaffney CJ, Zhang C, Presson AP. The association of patient satisfaction with pain, anxiety, and self-reported physical function. J Bone Joint Surg Am. 2018;100(21):1811-1818. doi: 10.2106/JBJS.17.00372 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Wong ES, Wang V, Liu CF, Hebert PL, Maciejewski ML. Do Veterans Health Administration enrollees generalize to other populations? Med Care Res Rev. 2016;73(4):493-507. doi: 10.1177/1077558715617382 [DOI] [PubMed] [Google Scholar]
- 23.Berliner JL, Brodke DJ, Chan V, SooHoo NF, Bozic KJ. John Charnley Award: preoperative patient-reported outcome measures predict clinically meaningful improvement in function after THA. Clin Orthop Relat Res. 2016;474(2):321-329. doi: 10.1007/s11999-015-4350-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Berliner JL, Brodke DJ, Chan V, SooHoo NF, Bozic KJ. Can preoperative patient-reported outcome measures be used to predict meaningful improvement in function after TKA? Clin Orthop Relat Res. 2017;475(1):149-157. doi: 10.1007/s11999-016-4770-y [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
eFigure. Receiver Operating Characteristic Curves