Abstract
Objective
Little is known regarding the minimal clinically important difference (MCID) and substantial clinical benefit (SCB) with regard to the Knee injury and Osteoarthritis Outcome Score (KOOS), International Knee Documentation Committee (IKDC) Subjective Knee Evaluation Form, Lysholm score, and Short Form 12 (SF-12) score of patients who undergo osteochondral allograft transplantation (OCA). We aimed to determine the MCID and SCB associated with those patient-reported outcome measures (PROMs) after OCA.
Design
We analyzed the data of 86 consecutive patients who underwent OCA and who completed satisfaction surveys at a minimum of 1 year postoperatively and had at least one repeated PROM. MCID was determined using an anchor-based method: the optimal cutoff point for receiver operative characteristic (ROC) curves. If an anchor-based method was inapplicable, distribution-based methods were employed. SCB was determined using ROC curve analysis.
Results
Based on the ROC curve analysis, MCID was 16.7 for KOOS pain, 25 for KOOS sports/recreation, and 9.8 for IKDC. SCB was 27.7 for KOOS pain, 10.7 for KOOS symptom, 30 for KOOS sports/recreation, 31.3 for KOOS quality of life, 26.9 for IKDC, 25 for Lysholm, and 12.1 for SF-12 physical component summary. No significant association was noted between SCB achievement and the baseline patient factors and baseline PROMs.
Conclusion
We demonstrated the MCIDs and SCBs of several PROMs in patients undergoing OCA. These results will aid the interpretation of the effect of treatment and clinical trial settings. Moreover, the SCBs will help surgeons in the counseling of patients, where patients expect optimal results rather than minimal improvement.
Keywords: minimal clinically important difference, substantial clinical benefit, Knee injury and Osteoarthritis Outcome Score, International Knee Documentation Committee, Lysholm score, Short Form 12, osteochondral allograft transplantation
Introduction
Articular cartilage lesions in the knee rarely heal spontaneously, and if left untreated may result in degenerative disease (osteoarthritis). Among the diverse treatment modalities available for symptomatic osteochondral lesions, osteochondral allograft transplantation (OCA) is used in the United States, especially for relatively large osteochondral and cartilage lesions in active and young patients. To assess surgical outcomes, patient-reported outcome measures (PROMs) are frequently used in clinical practice to evaluate the knee and include the Knee injury and Osteoarthritis Outcome Score (KOOS), International Knee Documentation Committee (IKDC) Subjective Knee Evaluation Form, Lysholm score, and Short Form 12 (SF-12). The minimal clinically important difference (MCID) and the substantial clinical benefit (SCB) are useful benchmarks for assessing improvement. The term MCID was first described by Jaeschle et al.1 in 1989 as “the smallest difference in score in the domain of interest which patients perceive as beneficial and which would mandate, in the absence of troublesome side effects and excessive cost, a change in the patient’s management.” The SCB was originally described for patients who underwent surgeries for lumbar degenerative disease, and is defined as the clinical improvement that represents a substantial clinical benefit.2 Therefore, MCID is defined as the smallest clinically meaningful change, whereas SCB is defined as the clinical value that the patient considers as substantial improvement. Determining both values would be useful for ascertaining the treatment effectiveness and patient’s perception, and for calculating the sample size for trials. Among the various methods employed to determine the MCID, 2 examples of general approaches are anchor-based methods and distribution-based methods. Anchor-based methods compare the change in a PROM with an external measurement of change, which serves as the anchor. Distribution-based methods compare the change in a PROM to some measurement of variability, for example, the standard error of measurement, the standard deviation (SD), the effect size, or the minimum detectable change (MDC).
Several studies have indicated that OCA is a promising procedure; however, there is a paucity of information regarding the MCID and SCB of KOOS, IKDC score, Lysholm score, and SF-12 score of patients who undergo OCA. This study aimed to determine the MCID and SCB of these PROMs in patients who undergo OCA.
Methods
Patients
The present study was approved by the institutional review board of the institution. Informed consent was obtained at the time of patients’ data entry into the registry. Between February 2007 and December 2016, 203 patients underwent OCA by a single surgeon for symptomatic osteochondral lesions in the knee. The indications for surgery included one or more full-thickness osteochondral lesions with symptoms in the defect location. Only patients who were resistant to nonoperative treatment, including physical therapy and injectable therapies, underwent surgery. Patellar maltracking and tibiofemoral malalignment >3° from the neutral mechanical axis into the involved compartment were corrected with a concomitant osteotomy (high tibial osteotomy, n = 13; tibial tubercle osteotomy, n = 18). Moreover, concomitant pathology, including anterior cruciate ligament (ACL) tears, medial patellofemoral ligament (MPFL) tears, and meniscal deficiencies, were treated with ligamentous reconstruction (ACL reconstruction, n = 2; MPFL reconstruction, n = 2) and meniscal allograft transplantation (n = 3). Eight patients underwent articulated juvenile articular cartilage (DeNovo NT, Zimmer Inc, Warsaw, IN, USA) transplantation in addition to OCA. Accordingly, those who underwent concomitant procedures at the time of OCA (n = 37) were not excluded from this study.
Patients who underwent OCA were prospectively evaluated, and those who completed the 1 year of follow-up were included. Patients without a 1-year satisfaction survey or those without at least 1 repeated PROM at 2 time points (preoperatively and at the 1-year follow-up) were not analyzed. Of the 203 patients who underwent OCA in the period, 86 were considered eligible on the basis of compliance and ability to follow up and enrolled in this study. We compared the baseline variables of the included (n = 86) and excluded (n = 117) patients. The baseline variables were not significantly different between the 2 groups ( Table 1 ). The mean age of the patients included in this study was 34.7 years (range: 14-58 years), and the population comprised 46 female patients (53.5%). The mean total surface area per knee was 3.9 cm2 (range: 0.7-13.9 cm2). A total of 46 knees (53.5%) had undergone at least 1 previous surgery in the same knee. Etiologies included traumatic chondral lesions (n = 70; 81.4%) and osteochondritis dissecans (n = 16; 18.6%).
Table 1.
Comparison of Study Group and Excluded Patients (n = 203).
| Variables | Study Group (n = 86) | Patients with Incomplete Data (n = 117) | P |
|---|---|---|---|
| Age at surgery, years, mean ± SD | 34.7 ± 10.8 | 35.1 ± 9.9 | 0.7695 |
| Gender, male/female, n | 40/46 | 65/52 | 0.255 |
| Body mass index, kg/m2, mean ± SD | 28.2 ± 5.3 | 27.6 ± 4.8 | 0.4623 |
| Multiple previous surgeries, n (%) | 46 (53.5) | 77 (65.8) | 0.083 |
| Total surface area of defect per knee, cm2, mean ± SD | 3.9 ± 2.5 | 4.0 ± 2.5 | 0.6953 |
Evaluation of PROMs
Preoperatively, patients were assessed with frequently used PROMs such as KOOS, IKDC, Lysholm scale, and SF-12. KOOS was designed to extend the Western Ontario and McMaster Universities Osteoarthritis Index for a younger and more active group of patients with knee injuries or osteoarthritis.3 KOOS has been validated for patients with cartilage injuries4,5 and comprises a 42-item self-reported questionnaire of subscales that include pain (9 items), other symptoms (7 items), function in daily living (17 items), function in sport and recreation (5 items), and knee-related quality of life (QOL) (4 items), which are scored individually from 0 (extreme knee problems) to 100 (no knee problems).
The IKDC score was developed to evaluate knee-specific measures, including symptoms, function, and sports activity; a score of 100 (maximum) indicates the absence of symptoms and limitations in performing daily activities. It is based on 18 items covering 3 domains: (1) symptoms (including pain, stiffness, swelling, locking/catching, and giving way), (2) sport and daily activities, and (3) current knee function and knee function prior to knee injury.6
The Lysholm score was originally designed to evaluate ligamentous injuries; it has an overall score of 0 to 100 and reports on eight domains including limping, locking, pain, stair-climbing, support, instability, swelling, and squatting.7
The SF-12 score, which is derived from the SF-36 score, comprises a 12-item questionnaire evaluating specific factors of general health-related QOL that are divided into the physical (PCS) and mental (MCS) component summaries. The mean score of the general population is 50, with an SD of 10. Higher scores demonstrate better health-related QOL.8
Calculation of Minimal Clinically Important Differences and Substantial Clinical Benefits
MCIDs were calculated using an anchor-based method or a distribution-based method if an anchor-based method was inapplicable. Regarding the anchor-based method, patients were supplied the anchor question at 1 year postoperatively: “Compared to before surgery, how would you rate each operated joint now?” The responses were assessed using a 5-point scale: “much better,” “somewhat better,” “about the same,” “somewhat worse,” and “much worse.” Patients who answered “about the same” or “somewhat worse” were classified into the no change group, while those who answered “somewhat better” were classified into the minimal change group. Patients with answers of “much better” or “much worse” were excluded from the MCID analysis because they had more than minimal change. A receiver operating characteristic (ROC) curve was used to define the cutoff point that best discriminated between the minimal and no change groups. The optimal cutoff point was estimated using the point that maximized both specificity and sensitivity. The area under the ROC curve (AUC) was calculated to assess reliability. An AUC value of 0.7 to 0.8 was regarded as acceptable and an AUC value of 0.8 to 0.9 was regarded as excellent.9 For distribution-based methods, an SD-based estimate and effect size–based estimate were used in this study. The SD represented the variation among groups of scores; a previous study found that an SD of 0.5 is equivalent to the MCID.10 The effect size is a standardized measure of change that is obtained by dividing the difference in scores from baseline to posttreatment by the SD of the baseline scores. Effect sizes of 0.2, 0.5, and 0.8 were considered small, moderate, and large, respectively.11 MCID was calculated by multiplying the SD of the baseline scores by 0.2 ( Table 2 ).12
Table 2.
An SD-Based Estimate and Effect Size–Based Estimate (Distribution-Based Methods).
| Calculation | |
|---|---|
| SD-based estimate | The SD of the baseline scores × 0.5 |
| Effect size–based estimate | The SD of the baseline scores × 0.2 |
SD = standard deviation.
Substantial clinical benefits were calculated using ROC curve analysis to define the cutoff point that best discriminated between the substantial (“much better”) and nonsubstantial (“somewhat better,” “about the same,” or “slightly worse”) improvement groups. Additional analysis was performed to (1) ascertain the presence of a significant difference in the patient demographics between these 2 groups and (2) clarify if the baseline PROMs had the ability to predict SCB achievement using ROC curve analysis with AUC. A calculated value was determined as significantly predictive based on an AUC value higher than 0.7.
Statistical Analysis
Differences in patient demographics and the characteristics of cartilage defects were compared using an unpaired t-test or the Mann-Whitney U test for continuous data, based on the distribution of data as determined by the Shapiro-Wilk test. Fisher’s exact test or Pearson chi-square was used for categorical data, as appropriate. The Wilcoxon signed-rank test was used to compare differences in the PROMs between the 2 time points. The Mann-Whitney U test was used to compare differences in the PROMs and the mean change in the PROMs between the different groups. The level of significance was set a priori at P < .05. All statistical analyses were performed with Stata (version 13; StataCorp LP, College Station, TX, USA).
Results
Patient Cohort
Significant improvements were noted for all postoperative functional scores ( Table 3 ). Compared to before surgery, 57 (66.3%) patients reported that their knees were much better, 18 (20.9%) somewhat better, 6 (7.0%) about the same, and 5 (5.8%) somewhat worse. No patients reported that their knees were much worse ( Table 4 ). Therefore, 11 (about the same and somewhat worse) and 18 patients (somewhat better) were assigned to the “no change” and “minimal change” groups, respectively, for the MCID calculation. Additionally, 57 (much better) and 29 (somewhat better, about the same, or somewhat worse) were assigned to the “substantial” and “nonsubstantial” improvement groups, respectively for SCB calculation.
Table 3.
Pre- and Postoperative PROMs in Patients Included in This Study.
| PROMs | n | Preoperative | Postoperative | P |
|---|---|---|---|---|
| KOOS score | ||||
| Pain | 86 | 57.4 ± 17.8 | 82.5 ± 15.1 | <0.001 |
| Symptom | 86 | 44.4 ± 12.5 | 54.6 ± 14.0 | <0.001 |
| ADL | 86 | 67.5 ± 18.4 | 89.1 ± 12.6 | <0.001 |
| Sport/Recreation | 85 | 29.4 ± 22.5 | 59.9 ± 25.0 | <0.001 |
| QOL | 86 | 23.9 ± 18.6 | 57.4 ± 21.5 | <0.001 |
| IKDC | 86 | 40.6 ± 14.9 | 66.8 ± 18.9 | <0.001 |
| Lysholm | 86 | 51.0 ± 18.4 | 78.4 ± 18.6 | <0.001 |
| SF-12 | ||||
| PCS | 86 | 38.2 ± 8.3 | 48.7 ± 7.7 | <0.001 |
| MCS | 86 | 50.9 ± 9.2 | 54.6 ± 6.0 | <0.001 |
PROMs = patient-reported outcome measures; KOOS = Knee injury and Osteoarthritis Outcome Score; ADL = activities of daily living; QOL = quality of life; IKDC = International Knee Documentation Committee Subjective Knee Evaluation Form; SF-12 = Short Form 12; PCS = Physical Component Summary; MCS = Mental Component Summary.
Table 4.
Anchor Questions.
| Question | n = 86, n (%) |
|---|---|
| Compared with before each surgery, how would you rate your operated joint now? | |
| Much better | 57 (66.3) |
| Somewhat better | 18 (20.9) |
| About the same | 6 (7.0) |
| Somewhat worse | 5 (5.8) |
| Much worse | 0 (0) |
Minimal Clinically Important Differences
We calculated the MCID of KOOS pain, KOOS sports/recreation scales, and IKDC score using an anchor-based method because the mean change in the “no change” versus “minimal change” groups was significantly different ( Table 5 ). The MCIDs for each PROM that were calculated via the anchor-based method or 2 distribution-based methods are shown in Table 6 . The anchor-based method provided MCIDs (percentage of achieving) for several PROMs (16.7 [76%] for KOOS pain scale, 25 [57%] for KOOS sports/recreation score, and 9.8 [78%] for the IKDC score). All AUCs defined by the ROC curve were greater than 0.7, indicating that the cutoff point was acceptable. The IKDC score had the highest AUC (AUC = 0.77). In the 2 different distribution-based methods, the MCID range was as follows: KOOS symptom scale, 2.5 to 6.3; KOOS activities of daily living (ADL) score, 3.7 to 9.2; KOOS QOL, 3.7 to 9.3; Lysholm score, 3.7 to 9.2; SF-12 PCS score, 1.7 to 4.2; and SF-12 MCS score, 1.8 to 4.6.
Table 5.
Pre- and Postoperative PROMs in the No Change and Minimal Change Groups.a
| PROMS | No Change (n = 11) | Minimal Change (n = 18) | P |
|---|---|---|---|
| KOOS | |||
| Pain | |||
| Pre | 56.3 ± 18.6 | 52.6 ± 14.9 | 0.4298 |
| Post | 64.9 ± 17.4 | 72.5 ± 14.9 | 0.2410 |
| Mean score change | 8.6 ± 21.5 | 19.9 ± 16.5 | 0.0245 |
| Symptom | |||
| Pre | 39.9 ± 17.0 | 43.7 ± 10.6 | 0.6518 |
| Post | 37.0 ± 17.0 | 49.0 ± 9.5 | 0.0286 |
| Mean score change | –2.9 ± 14.4 | 5.4 ± 10.0 | 0.1001 |
| ADL | |||
| Pre | 63.5 ± 17.7 | 61.3 ± 19.1 | 0.8926 |
| Post | 77.0 ± 15.3 | 79.9 ± 13.2 | 0.6687 |
| Mean score change | 13.5 ± 12.5 | 18.6 ± 18.8 | 0.3011 |
| Sport/Recreation | |||
| Pre | 28.6 ± 29.7 | 28.9 ± 23.7 | 0.8038 |
| Post | 31.8 ± 17.8 | 47.2 ± 19.6 | 0.0338 |
| Mean score change | 3.2 ± 23.8 | 18.3 ± 27.5 | 0.0474 |
| QOL | |||
| Pre | 21.0 ± 18.0 | 25.3 ± 11.2 | 0.3987 |
| Post | 31.8 ± 15.4 | 47.6 ± 13.4 | 0.0141 |
| Mean score change | 10.8 ± 18.1 | 22.2 ± 18.6 | 0.2099 |
| IKDC | |||
| Pre | 37.5 ± 14.5 | 38.7 ± 12.9 | 0.7701 |
| Post | 42.2 ± 16.5 | 55.0 ± 13.3 | 0.0152 |
| Mean score change | 4.7 ± 11.7 | 16.3 ± 13.0 | 0.0172 |
| Lysholm | |||
| Pre | 43.6 ± 16.8 | 52.2 ± 16.9 | 0.1846 |
| Post | 50.4 ± 19.7 | 70 ± 15.1 | 0.0111 |
| Mean score change | 6.7 ± 14.1 | 17.8 ± 20.5 | 0.0833 |
| SF-12 | |||
| PCS | |||
| Pre | 37.3 ± 7.0 | 36.9 ± 7.8 | 0.8222 |
| Post | 40.8 ± 7.6 | 43.9 ± 7.1 | 0.2807 |
| Mean score change | 3.5 ± 6.3 | 7.0 ± 9.5 | 0.2082 |
| MCS | |||
| Pre | 51.9 ± 8.6 | 49.5 ± 10.8 | 0.6210 |
| Post | 50.3 ± 8.4 | 55.1 ± 7.3 | 0.0963 |
| Mean score change | −1.6 ± 8.3 | 5.6 ± 9.0 | 0.0963 |
PROMs = patient-reported outcome measures; KOOS, Knee Injury and Osteoarthritis Outcome Score; ADL, activities of daily living; QOL, quality of life; IKDC, International Knee Documentation Committee Subjective Knee Evaluation Form; SF-12, Short Form 12; PCS, Physical Component Summary; MCS, Mental Component Summary.
Values are shown as mean ± SD.
Table 6.
The MCIDs for Each PROM.
| PROMs | Anchor-Based |
Distribution-Based |
|
|---|---|---|---|
| ROC Curve (AUC) | SD | Effect Size | |
| KOOS | |||
| Pain | 16.7 (0.75) | — | — |
| Symptom | n/a | 6.3 | 2.5 |
| ADL | n/a | 9.2 | 3.7 |
| Sport/Recreation | 25 (0.72) | — | — |
| QOL | n/a | 9.3 | 3.7 |
| IKDC | 9.8 (0.77) | — | — |
| Lysholm | n/a | 9.2 | 3.7 |
| SF-12 | n/a | ||
| PCS | n/a | 4.2 | 1.7 |
| MCS | n/a | 4.6 | 1.8 |
MCID = minimal clinically important difference; PROM = patient-reported outcome measure; ROC = receiver operating characteristics; AUC = area under the ROC curve; SD = standard deviation; KOOS = Knee injury and Osteoarthritis Outcome Score; ADL = activities of daily living; QOL = quality of life; IKDC = International Knee Documentation Committee Subjective Knee Evaluation Form; SF-12 = Short Form 12; PCS = Physical Component Summary; MCS = Mental Component Summary; n/a = not available.
Substantial Clinical Benefits
The SCB of all the PROMs was calculated, except for that of SF-12 MCS, using an anchor-based method because the mean change significantly differed between in the “nonsubstantial” and “substantial” improvement groups ( Table 7 ). The SCBs for each PROM that was calculated with the ROC analysis and the percentage of achieving for each SCB are shown in Table 8 . All AUCs except for KOOS ADL and SF-12 MCS were greater than 0.7, indicating that the cutoff point was acceptable. The IKDC score had the highest AUC (AUC = 0.8394), followed by KOOS sport/recreation (AUC = 0.7903). Overall, more than half of the patients achieved the calculated SCBs. Baseline variables were not significantly different between these two groups ( Table 9 ). Based on an AUC, no calculated value was determined to be significantly predictive for achieving SCB.
Table 7.
Pre- and Postoperative PROMs in the Nonsubstantial Change and Substantial Improvement Groups.a
| PROMs | Nonsubstantial (n = 29) | Substantial Improvement (n = 57) | P |
|---|---|---|---|
| KOOS | |||
| Pain | |||
| Pre | 54.0 ± 17.6 | 59.1 ± 17.8 | 0.1698 |
| Post | 69.6 ± 16.0 | 89.0 ± 9.6 | <0.001 |
| Mean score change | 15.6 ± 19.0 | 29.9 ± 17.1 | <0.001 |
| Symptom | |||
| Pre | 42.2 ± 13.2 | 45.5 ± 12.1 | 0.5148 |
| Post | 44.5 ± 13.7 | 59.7 ± 11.2 | <0.001 |
| Mean score change | 2.2 ± 12.3 | 14.2 ± 14.8 | <0.001 |
| ADL | |||
| Pre | 62.1 ± 18.3 | 70.3 ± 8.0 | 0.0747 |
| Post | 78.8 ± 13.8 | 94.3 ± 8.0 | <0.001 |
| Mean score change | 16.6 ± 24.0 | 24.0 ± 15.4 | 0.0276 |
| Sport/Recreation | |||
| Pre | 28.8 ± 25.6 | 29.7 ± 21.0 | 0.5700 |
| Post | 41.3 ± 20.1 | 70.2 ± 21.3 | <0.001 |
| Mean score change | 12.6 ± 26.8 | 40.4 ± 22.7 | <0.001 |
| QOL | |||
| Pre | 23.7 ± 14.0 | 24.0 ± 20.7 | 0.6125 |
| Post | 41.6 ± 16.0 | 65.5 ± 19.5 | <0.001 |
| Mean score change | 17.9 ± 19.0 | 41.1 ± 23.5 | <0.001 |
| IKDC | |||
| Pre | 38.3 ± 13.3 | 41.7 ± 15.7 | 0.4241 |
| Post | 50.2 ± 15.7 | 75.3 ± 14.3 | <0.001 |
| Mean score change | 11.9 ± 13.6 | 33.5 ± 16.5 | <0.001 |
| Lysholm | |||
| Pre | 48.9 ± 17.1 | 52.1 ± 19.2 | 0.5929 |
| Post | 62.6 ± 19.2 | 86.5 ± 12.0 | <0.001 |
| Mean score change | 13.6 ± 18.9 | 34.4 ± 19.4 | <0.001 |
| SF-12 | |||
| PCS | |||
| Pre | 37.1 ± 7.4 | 38.7 ± 8.8 | 0.4790 |
| Post | 42.7 ± 7.3 | 51.8 ± 5.8 | <0.001 |
| Mean score change | 5.6 ± 8.5 | 13.1 ± 9.9 | 0.0019 |
| MCS | |||
| Pre | 40.4 ± 10.0 | 51.2 ± 8.8 | 0.6777 |
| Post | 53.3 ± 7.9 | 55.3 ± 4.7 | 0.6249 |
| Mean score change | 2.9 ± 9.3 | 4.1 ± 7.9 | 0.5020 |
PROMs = patient-reported outcome measures; KOOS = Knee injury and Osteoarthritis Outcome Score; ADL = activities of daily living; QOL = quality of life; IKDC = International Knee Documentation Committee Subjective Knee Evaluation Form; SF-12 = Short Form 12; PCS = Physical Component Summary; MCS = Mental Component Summary.
Values are shown as mean ± SD.
Table 8.
The SCBs and Percentage of Achievement to SCB.
| PROMs | ROC | AUC | Sensitivity | Specificity | n (%), achieving |
|---|---|---|---|---|---|
| KOOS | |||||
| Pain | 27.7 | 0.72 | 0.63 | 0.79 | 46 (53.5) |
| Symptom | 10.7 | 0.73 | 0.53 | 0.76 | 45 (52.3) |
| ADL | 25 | 0.65 | 0.49 | 0.79 | 33 (38.4) |
| Sport/Recreation | 30 | 0.79 | 0.71 | 0.76 | 42 (48.8) |
| QOL | 31.3 | 0.79 | 0.72 | 0.76 | 43 (50.0) |
| IKDC | 26.9 | 0.84 | 0.67 | 0.86 | 43 (50.0) |
| Lysholm | 25 | 0.78 | 0.70 | 0.72 | 49 (57.0) |
| SF-12 | |||||
| PCS | 12.1 | 0.71 | 0.56 | 0.79 | 39 (45.3) |
| MCS | n/a | n/a | n/a | n/a | n/a |
PROMs = patient-reported outcome measures; ROC = receiver operating characteristics; AUC = area under the ROC curve; SCB = substantial clinical benefit; KOOS = Knee injury and Osteoarthritis Outcome Score; ADL = activities of daily living; QOL = quality of life; IKDC = International Knee Documentation Committee Subjective Knee Evaluation Form; SF-12 = Short Form 12; PCS = Physical Component Summary; MCS = Mental Component Summary; n/a, not available.
Table 9.
Comparison of the Baseline Demographics between the Nonsubstantial versus Substantial Improvement Groups.
| Nonsubstantial (n = 29) | Substantial Improvement (n = 57) | P | |
|---|---|---|---|
| Age, years, mean ± SD | 33.1 ± 6.8 | 36.1 ± 11.1 | 0.1731 |
| Gender, male/female, n | 13/16 | 27/30 | 1.000 |
| Body mass index, kg/m2, mean ± SD | 27.9 ± 4.8 | 27.5 ± 4.8 | 0.6924 |
| Defect size, cm2, mean ± SD | 3.9 ± 2.4 | 4.1 ± 2.5 | 0.7067 |
| Multiple previous surgeries, n (%) | 19 (65.5) | 27 (47.4) | 0.170 |
| Defect location | |||
| MFC | n = 15 | n = 30 | 0.256 |
| LFC | n = 7 | n = 16 | |
| Patella | n = 1 | n = 6 | |
| Trochlea | n = 6 | n = 5 | |
| Tibia plateau | n = 1 | n = 0 | |
| Concomitant procedure, n (%) | 10 (34.5) | 27 (47.4) | 0.357 |
MFC = medial femoral condyle; LFC = lateral femoral condyle.
Discussion
This study determined the MCID and SCB of 4 frequently used PROMs in patients that underwent OCA for symptomatic osteochondral defects in the knee. An anchor-based method (ROC curve analysis) provided several MCIDs (for the KOOS pain scale, KOOS sports/recreation, and the IKDC score) and SCBs (all except for SF-12 MCS). This study is the first to comprehensively demonstrate MCID and SCB in patients that underwent OCA for the treatment of chondral and osteochondral defects in the knee. We evaluated MCID and SCB in patients with a minimum of a 1-year follow-up because, from clinical experience, we believe this is the most clinically relevant time point when the majority of patients experience the greatest improvement after undergoing OCA, rather than the slower improvement seen with cell-based cartilage repair. The reporting of MCID is important for identifying the threshold of a clinically important minimal improvement. However, patients undergoing surgery, especially active and young patients, expect optimal results rather than minimal improvement. This prompted the decision to identify SCB in addition to MCID after OCA.
Two methods are commonly used to calculate MCID: the anchor- and distribution-based methods. In our present study, anchor-based, rather than distribution-based methods, were primarily used. Although the best approach for determining the MCID is unclear, distribution-based methods seem non-ideal because they are truly statistical, do not use clinically important questionnaires, and do not consider patient perspectives.13 Therefore, the value of MCID that is determined using anchor-based methods is more clinically relevant, while distribution-based methods can support anchor-based methods or provide an MCID when anchor-based methods are unavailable.14 Additionally, we used ROC curve analysis as this method can be consistently used for the calculation of SCBs in the same cohort, which allows the comprehensive reporting of both values.
We calculated both MCIDs and SCBs for KOOS pain, KOOS sports/recreation score, and the IKDC score, comprehensively, using ROC curve analysis. A notable difference was observed between MCID and SCB in KOOS pain (16.7 vs. 27.7) and the IKDC score (9.8 vs. 26.9), while a non-marked difference was seen in the KOOS sports/recreation score (25 vs. 30). These findings indicate that patients undergoing OCA require considerable change comparable to substantial improvement in the KOOS sports/recreation score in order to consider themselves as minimally improved. Moreover, this observation suggests that the degree of decreased knee pain (evaluated by the KOOS pain scale) largely contributes to a patient’s perception of “substantial improvement” rather than “minimal improvement”.
Baseline patient factors and PROMs were not associated with SCB achievement after OCA in the current study. Risk factors for the achievement of clinically important change after OCA have not been clarified. When assessing the PROMs, SCB is crucial because, when surgery is considered, the probability of SCB achievement seems more important to patients than MCID achievement. As there is no strong evidence regarding which cartilage repair procedures are superior to any other, considering the probability of favorable outcomes aids surgical decision-making and treatment selection. Thus, clarifying the probability of achieving SCBs, as well as the associations of baseline factors with SCB achievement, would be useful for comparing cartilage repair procedures in future studies.
Several studies have reported the MCID of the KOOS scale using various methods in patients who underwent cartilage repair procedures. Using an anchor-based method, Ebert et al.15 assessed 104 patients at 5 years after they underwent matrix-induced autologous chondrocyte implantation and reported that the MCID of the KOOS sport/recreation score was 40, which could accurately predict whether patients would be “very satisfied.” Therefore, the calculated value of 40 appears to be the SCB, which does not represent the “minimal change.” Indeed, the reported MCID of the KOOS sport/recreation score (40) was much greater than our calculated MCID (25) and rather closer to our determined SCB (30). Nevertheless, our finding confirmed that KOOS sport/recreation is one of the most reliable PROMs for predicting satisfactory patient outcomes, which was in line with the previous study.15 Making a direct comparison is challenging because of the different cartilage procedures and different timing of the evaluations (ours, 1 year; Ebert et al.,15 5 years). Our shorter follow-up might have resulted in a decrease in our calculated MCIDs and SCBs as a previous study demonstrated that an increased duration of follow-up was associated with larger estimates of MCID.16 A longer follow-up study will be of interest, with regard to determining the difference in MCID and SCB of each PROM among various cartilage repair procedures.
Greco et al.,17 using an anchor-based method, reported that the MCID of the IKDC score was 16.7 at 1 year after various cartilage repair procedures, which is greater than that reported in our study (9.8). Our value was confirmed using more homogenous patients who underwent OCA, with a minimum of 1 year of follow-up. Additionally, their study also comprised patients with “much better” responses in the minimal change group, which seems to make the cutoff point exceed ours. The present study determined the value of MCID and SCB for the IKDC score after OCA by identifying a subset of individuals who experienced minimal change and substantial improvement, respectively, in which the IKDC score was the most reliable PROM with highest AUC for both MCID and SCB.
Although the Lysholm scale is a validated system for evaluating cartilage damage,18 MCID and SCB of the Lysholm scale were indeterminate in this population. We could not calculate MCID for the Lysholm scale and SF-12 (PCS and MCS), and SCB for SF-12 MCS using the ROC analysis because there was no significant difference in the mean score of each scale between the minimal and no change groups. The relatively small sample size may have prevented the detection of significance. Moreover, the SF-36 MCS score is considered the least responsive outcome score after cartilage repair.15 Using distribution-based methods, our calculated MCID for the Lysholm scale and SF-12 was comparable to that reported after ACL reconstruction.19
This study has several strengths. First, we employed an anchor-based method (ROC curve analysis) or 2 distribution-based methods, as required, to calculate MCID and SCB. Comprehensive reports of MCID and SCB are useful for clarifying the surgical outcomes using the PROMs. Second, this study was a single-surgeon series of patients with the same indications, procedures, and postoperative courses.
However, the study has several limitations. First, we included only 86 of the 203 potentially eligible patients, possibly limiting the generalizability of our findings. However, no statistical differences in the baseline characteristics of the included and excluded patients were noted. Because the population was young and geographically mobile, it was challenging to follow-up all the patients. Second, considerable numbers of the patients in our study had concomitant surgeries at the time of OCA. However, comparable outcomes have been reported when OCA was performed with those concomitant procedures.20-22 Third, although anchor-based methods are superior to distribution-based methods in some aspects, they still have some limitations, in particular recall bias. Finally, we did not analyze the effect of several parameters that might have influenced the patient’s perception, such as age, sex, body mass index, defect location, defect size, and preoperative mental health status. Although it was not possible to do so in this study owing to the sample size, stratifying the groups according to these variables may have provided different SCBs. Nevertheless, our results are useful as a benchmark in the future since this is the first study to report the MCID and SCB of several PROMs in patients undergoing OCA.
Conclusions
Our results demonstrated the MCIDs and SCBs of several PROMs in patients undergoing OCA for symptomatic osteochondral lesions. The MCID of 9.8 and SCB of 26.9 in the IKDC score were the most reliable for understanding the patient’s perception of improvement. Analysis of MCID values will allow the interpretation of the effect of treatment in clinical practice and clinical trial settings. Moreover, the SCB values and the percentage of achievement for each SCB will aid surgeons in the selection of treatment strategies for different cartilage repair procedures. They will also provide guidance for counseling patients before surgery by advising them of the clinical explanation of expected effects, where patients expect an optimal result rather than minimal improvement.
Footnotes
Acknowledgments and Funding: The author(s) received no financial support for the research, authorship, and/or publication of this article.
Declaration of Conflicting Interests: The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Ethical Approval: The present study was approved by the institutional review board of the institution (2018P000361/PHS).
Informed Consent: Informed consent was obtained at the time of patients’ data entry into the registry.
Trial Registration: Not applicable.
ORCID iD: Jakob Ackermann 
https://orcid.org/0000-0002-4063-5815
References
- 1. Jaeschke R, Singer J, Guyatt GH. Measurement of health status. Ascertaining the minimal clinically important difference. Control Clin Trials. 1989;10(4):407-15. [DOI] [PubMed] [Google Scholar]
- 2. Glassman SD, Copay AG, Berven SH, Polly DW, Subach BR, Carreon LY. Defining substantial clinical benefit following lumbar spine arthrodesis. J Bone Joint Surg Am. 2008;90(9):1839-47. [DOI] [PubMed] [Google Scholar]
- 3. Roos EM, Roos HP, Ekdahl C, Lohmander LS. Knee injury and Osteoarthritis Outcome Score (KOOS)—validation of a Swedish version. Scand J Med Sci Sports. 1998;8(6):439-48. [DOI] [PubMed] [Google Scholar]
- 4. Bekkers JE, de Windt TS, Raijmakers NJ, Dhert WJ, Saris DB. Validation of the Knee Injury and Osteoarthritis Outcome Score (KOOS) for the treatment of focal cartilage lesions. Osteoarthritis Cartilage. 2009;17(11):1434-9. [DOI] [PubMed] [Google Scholar]
- 5. Engelhart L, Nelson L, Lewis S, Mordin M, Demuro-Mercon C, Uddin S, et al. Validation of the Knee Injury and Osteoarthritis Outcome Score subscales for patients with articular cartilage lesions of the knee. Am J Sports Med. 2012;40(10):2264-72. [DOI] [PubMed] [Google Scholar]
- 6. Irrgang JJ, Anderson AF, Boland AL, Harner CD, Kurosaka M, Neyret P, et al. Development and validation of the International Knee Documentation Committee Subjective Knee Form. Am J Sports Med. 2001;29:600-13. [DOI] [PubMed] [Google Scholar]
- 7. Lysholm J, Gillquist J. Evaluation of knee ligament surgery results with special emphasis on use of a scoring scale. Am J Sports Med. 1982;10(3):150-4. [DOI] [PubMed] [Google Scholar]
- 8. Ware J, Jr, Kosinski M, Keller SD. A 12-Item Short-Form Health Survey: construction of scales and preliminary tests of reliability and validity. Med Care. 1996;34(3):220-33. [DOI] [PubMed] [Google Scholar]
- 9. Copay AG, Subach BR, Glassman SD, Polly DW, Jr, Schuler TC. Understanding the minimum clinically important difference: a review of concepts and methods. Spine J. 2007;7(5):541-6. [DOI] [PubMed] [Google Scholar]
- 10. Norman GR, Sloan JA, Wyrwich KW. Interpretation of changes in health-related quality of life: the remarkable universality of half a standard deviation. Med Care. 2003;41(5):582-2. [DOI] [PubMed] [Google Scholar]
- 11. Cohen J. Statistical power analysis for the behavioral sciences (revised ed.). New York, NY: Academic Press; 1977. [Google Scholar]
- 12. Samsa G, Edelman D, Rothman ML, Williams GR, Lipscomb J, Matchar D. Determining clinically important differences in health status measures: a general approach with illustration to the Health Utilities Index Mark II. Pharmacoeconomics. 1999;15(2):141-55. [DOI] [PubMed] [Google Scholar]
- 13. Diaz-Arribas MJ, Fernandez-Serrano M, Royuela A, Kovacs FM, Gallego-Izquierdo T, Ramos-Sánchez M, et al. Minimal clinically important difference in quality of life for patients with low back pain. Spine (Phila Pa 1976). 2017;42(24):1908-16. [DOI] [PubMed] [Google Scholar]
- 14. Revicki D, Hays RD, Cella D, Sloan J. Recommended methods for determining responsiveness and minimally important differences for patient-reported outcomes. J Clin Epidemiol. 2008;61(2):102-9. [DOI] [PubMed] [Google Scholar]
- 15. Ebert JR, Smith A, Wood DJ, Ackland TR. A comparison of the responsiveness of 4 commonly used patient-reported outcome instruments at 5 years after matrix-induced autologous chondrocyte implantation. Am J Sports Med. 2013;41(12):2791-9. [DOI] [PubMed] [Google Scholar]
- 16. Tashjian RZ, Deloach J, Green A, Porucznik CA, Powell AP. Minimal clinically important differences in ASES and simple shoulder test scores after nonoperative treatment of rotator cuff disease. J Bone Joint Surg Am. 2010;92(2):296-303. [DOI] [PubMed] [Google Scholar]
- 17. Greco NJ, Anderson AF, Mann BJ, Cole BJ, Farr J, Nissen CW, et al. Responsiveness of the International Knee Documentation Committee Subjective Knee Form in comparison to the Western Ontario and McMaster Universities Osteoarthritis Index, modified Cincinnati Knee Rating System, and Short Form 36 in patients with focal articular cartilage defects. Am J Sports Med. 2010;38(5):891-902. [DOI] [PubMed] [Google Scholar]
- 18. Smith HJ, Richardson JB, Tennant A. Modification and validation of the Lysholm Knee Scale to assess articular cartilage damage. Osteoarthritis Cartilage. 2009;17(1):53-8. [DOI] [PubMed] [Google Scholar]
- 19. Nwachukwu BU, Chang B, Voleti PB, Berkanish P, Cohn MR, Altchek DW, et al. Preoperative Short Form Health Survey Score is predictive of return to play and minimal clinically important difference at a minimum 2-year follow-up after anterior cruciate ligament reconstruction. Am J Sports Med. 2017;45(12):2784-90. [DOI] [PubMed] [Google Scholar]
- 20. Cotter EJ, Frank RM, Waterman BR, Wang KC, Redondo ML, Cole BJ. Meniscal allograft transplantation with concomitant osteochondral allograft transplantation. Arthrosc Tech. 2017;6(5):e1573-e1580. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21. Hsu AC, Tirico LEP, Lin AG, Pulido PA, Bugbee WD. Osteochondral allograft transplantation and opening wedge tibial osteotomy: clinical results of a combined single procedure. Cartilage. 2018;9:248-54. doi: 10.1177/1947603517710307 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Wang D, Eliasberg CD, Wang T, Fader RR, Coxe FR, Pais MD, et al. Similar outcomes after osteochondral allograft transplantation in anterior cruciate ligament-intact and -reconstructed knees: a comparative matched-group analysis with minimum 2-year follow-up. Arthroscopy. 2017;33(12):2198-207. [DOI] [PubMed] [Google Scholar]