Minimal Clinically Important Differences and Substantial Clinical Benefit in Patient-Reported Outcome Measures after Autologous Chondrocyte Implantation

Abstract

Objective

We sought to determine the minimal clinically important difference (MCID) and substantial clinical benefit (SCB) associated with the Knee Injury and Osteoarthritis Outcome Score (KOOS), International Knee Documentation Committee (IKDC) Subjective Knee Evaluation Form, Lysholm, and Short Form–12 (SF-12) after autologous chondrocyte implantation (ACI).

Design

Ninety-two patients with satisfaction surveys at a minimum of 2 years postoperatively and at least 1 repeated patient-reported outcome measure (PROM) were analysed. The MCID was determined using 4 anchor-based methods: average change, mean change, minimally detectable change, and the optimal cutoff point for receiver operating characteristic (ROC) curves. If an anchor-based method was not applicable, standard deviation–based and effect size–based estimates were used. SCB was determined using ROC curve analysis.

Results

The 4 anchor-based methods provided a range of MCID values for each PROM (11-18.8 for the KOOS pain, 9.2-17.3 for the KOOS activities of daily living, 12.5-18.6 for the KOOS sport/recreation, 12.8-19.6 for the KOOS quality of life, 10.8-16.4 for the IKDC, and 6.2-8.2 for the SF-12 physical component summary). Using the 2 distribution-based methods, the following MCID value ranges were obtained: KOOS symptom, 3.6 to 8.4; the Lysholm, 4.2 to 10.5; and the SF-12 mental component summary, 1.9 to 4.6. SCB was 30 for the KOOS sport/recreation and 34.4 for the IKDC, which most accurately predict substantial improvement. No significant association was noted between SCB achievement and the baseline PROMs.

Conclusion

The MCID and SCB determined in our study will allow interpretation of the effects of treatment in clinical practice and trials. Given the varied MCID values in this study, standardisation of the most appropriate calculation methods is warranted.

Introduction

Current clinical practice evaluates the results of surgical interventions through patient-reported outcome measures (PROMs) that allow for a structured and standardised assessment of patient-perceived pain and function. The Knee Injury and Osteoarthritis Outcome Score (KOOS), International Knee Documentation Committee (IKDC) Subjective Knee Evaluation Form, Lysholm score, and Short Form–12 (SF-12) score are common tools to evaluate knee function. The minimal clinically important difference (MCID) is a useful benchmark to determine whether patients improve enough clinically to notice a difference. Therefore, defining an MCID value is important for the use of PROMs. The term MCID was first described by Jaeschle et al.¹ in 1989 as “the smallest difference in score in the domain of interest which patients perceive as beneficial and which would mandate, in the absence of troublesome side effects and excessive cost, a change in the patient’s management.” Additionally, substantial clinical benefit (SCB) is defined as the clinical value that the patient considers as substantial improvement.² Determining both values would be useful for ascertaining the treatment effectiveness and patient’s perception.

Articular cartilage lesions in the knee do not heal spontaneously, and if left untreated, may progress to degenerative disease (osteoarthritis). Among the various treatment options that are available for symptomatic cartilage lesions, autologous chondrocyte implantation (ACI) is an established treatment, leading to improvements in pain, function, and mental health over a long-term follow-up period.^3-9 Despite its durability and effectiveness, it requires 2 procedures, is inconsistent in reproducing hyaline cartilage,¹⁰ and is associated with a relatively high reoperation rate due to hypertrophy of grafts and the development of arthrofibrosis.¹¹ In the pursuit of further improving outcomes, new techniques and technologies are being developed, and the MCIDs and SCBs for a specific population and specific procedure are crucial for determining the effectiveness of the procedure and calculating the sample size for trial planning. However, little is known regarding the MCID of the KOOS, IKDC score, Lysholm score, and SF-12 score of patients who undergo ACI.

The purpose of this study was to determine the MCID and SCB of several PROMs in patients who underwent ACI, using 4 anchor-based methods or 2 distribution-based methods if anchor-based methods were not applicable.

Methods

Patients

This study was approved by the institutional review board of our institution. Informed consent was obtained at the time of patients’ data entry into the registry. A total of 267 patients underwent ACI performed by a single surgeon from June 2007 to November 2015. The indications for surgery included one or more full-thickness articular cartilage lesions of the knee, with symptoms consistent with the location of the defect. Surgery was only indicated in patients who were resistant to nonoperative treatment, including physical therapy and medical therapies, such as anti-inflammatories and injections. The contraindications included inflammatory joint disease, unresolved or recent septic arthritis, metabolic or crystal disorders, or deficient soft-tissue coverage. Patellar maltracking and tibiofemoral malalignment >2° to 3° from the neutral mechanical axis into the involved compartment were corrected with concomitant osteotomy and were included in this study. Exclusion criteria included concomitant anterior cruciate ligament reconstruction (n = 7), medial patellofemoral ligament reconstruction (n = 1), and meniscal allograft transplantation (n = 17).

Patients who underwent ACI were evaluated prospectively, and those who completed at least 2 years of follow-up were included. Patients without a 2-year satisfaction survey or those without at least 1 repeated PROM at 2 time points (pre- and postoperatively) were excluded from the analysis. Of 242 consecutive patients, 92 were enrolled in this study. We compared the baseline variables of the included patients (n = 92) and excluded patients (n = 150) (Table 1). All factors but body mass index (BMI) at the time of ACI were not significantly different between the groups. The statistical difference in BMI at the time of ACI did not seem to have a clinically meaningful difference. The mean BMI of the patients who were included in this study and were excluded from this study was 26.5 kg/m² (range, 18-38.2 kg/m²) and 27.7 kg/m² (range, 18.1-43 kg/m²), respectively (P = 0.0498). The population consisted of 48 female and 44 male patients. The mean number of treated lesions per knee was 1.6 (range, 1-5), with a total surface area of 6.9 cm² (range, 0.6-28 cm²) per knee. A total of 33 knees (36%) had undergone at least 1 previous surgery in the same knee. Etiologies included traumatic chondral lesions (n = 79; 86%) and osteochondritis dissecans (n = 13; 14%).

Table 1.

Comparison of Study Group and Excluded Patients (n = 242).

Variables	Study Group (n = 92)	Patients with Incomplete Data (n = 150)	P
Age at surgery, years, mean ± SD (range)	31.4 ± 9.5 (15-51)	34.0 ± 10.4 (14-58)	0.0530
Gender, male/female, n	44/48	77/73	0.691
Body mass index, kg/m2, mean ± SD (range)	26.5 ± 4.3 (18.0-38.2)a	27.7 ± 4.7 (18.1-43.0)b	0.0498
Worker’s compensation, n (%)	6 (6.5)	10 (6.7)	0.595
Multiple previous surgeries, n (%)	33 (35.9)	66 (44.4)	0.228
Total surface area of defect per knee, cm2, mean ± SD (range)	6.9 ± 4.6 (0.6-27.5)	7.5 ± 4.9 (1.0-29.9)	0.3555
Number of defects, mean ± SD	1.6 ± 0.8	1.7 ± 0.9	0.4398
Defect location, cm2, mean ± SD
Medial femoral condyle	4.5 ± 2.2 (n = 24)	5.2 ± 2.6 (n = 58)	0.2524
Lateral femoral condyle	4.0 ± 2.4 (n = 16)	4.3 ± 2.0 (n = 20)	0.6865
Patella	4.5 ± 2.3 (n = 62)	4.4 ± 2.5 (n = 94)	0.7707
Trochlea	4.8 ± 3.2 (n = 37)	4.9 ± 2.6 (n = 65)	0.9075
Tibia plateau	1.0 ± 0.4 (n = 4)	1.6 ± 0.9 (n = 2)	0.2898
Concomitant procedure, n (%)	62 (67)	91 (61)	0.337

SD = standard deviation.

^aData from 90 patients.

^bData from 144 patients.

ACI Surgical Procedure

ACI was performed as described in detail previously.^12,13 Briefly, after an arthroscopic cartilage biopsy was performed during the initial surgery, chondrocytes were cultured, cryopreserved, thawed, and re-cultured for definitive implantation. A secondary surgery was then performed for implantation with arthrotomy. A type I/III bilayer collagen membrane (Bio-Gide, Geistlich Pharma, Princeton, NJ, USA) was used to cover the defect. The collagen membrane was placed on the cartilage defect and secured with multiple 6-0 Vicryl sutures (Ethicon, Somerville, NJ, USA). The suture line was waterproofed with fibrin glue (Tisseel, Baxter Biosurgery, Deerfield, IL, USA), and autologous cultured chondrocytes were injected underneath the membrane. After February 2010, the technique was simplified, and the collagen membrane was seeded with autologous cultured chondrocytes. The seeded membrane was placed on the cartilage defect and secured with resorbable suture and fibrin glue.

Evaluation of PROMs

Preoperatively, patients were evaluated with established PROMs such as the KOOS, IKDC, Lysholm scale, and SF-12 scale. The KOOS was developed to extend the Western Ontario and McMaster Universities Osteoarthritis Index for use in a younger and more active group of patients with knee injuries or osteoarthritis.¹⁴ It is validated for use in patients with cartilage injuries,^15,16 and consists of a 42-item self-reported questionnaire of subscales including pain (9 items), other symptoms (7 items), function in daily living (17 items), function in sport and recreation (5 items), and knee-related quality of life (QOL) (4 items) that are scored individually from 0 (extreme knee problems) to 100 (no knee problems).

The IKDC Subjective Knee Evaluation Form was developed by the International Knee Documentation Committee to evaluate knee-specific measures, including symptoms, function, and sport activity; a maximum score of 100 indicates no limitation in performing daily activities and absence of symptoms. It is based on 18 items covering 3 domains: (1) symptoms (including pain, stiffness, swelling, locking/catching, and giving way), (2) sport and daily activities, and (3) current knee function and knee function prior to knee injury.¹⁷

The Lysholm score was originally designed to evaluate ligamentous injuries, has an overall score of 0 to 100, and reports eight domains including limping, locking, pain, stair climbing, support, instability, swelling, and squatting.¹⁸

The SF-12 score, which is derived from the SF-36 score, consists of a 12-item questionnaire measuring specific factors of general health-related QOL that are divided into the physical component summary (PCS) and mental component summary (MCS). The mean score of the general population is 50, with a standard deviation (SD) of 10. Higher scores demonstrate better health-related QOL.¹⁹

Calculation of the MCID

The MCIDs were calculated using 4 anchor-based methods or 2 distribution-based methods if anchor-based methods were not applicable. For the anchor-based methods, patients were given the anchor question at a minimum of 2 years postoperatively: “Compared with before surgery, how would you rate each operated joint now?” The responses were recorded using a 5-point scale: “much better,” “somewhat better,” “about the same,” “somewhat worse,” and “much worse.” Patients who answered “about the same” or “somewhat worse” were classified into the no change group, while those who answered “somewhat better” were classified into the minimal change group. Patients who answered “much better” or “much worse” were not included in the analysis because they experienced more than minimal change. Four anchor-based methods were used to calculate the MCID. The average change corresponded to the mean change in the score of the minimal change group. The minimum detectable change (MDC) approach defines minimal change as the smallest change that can be considered above the measurement error with a given level of confidence (95%). Therefore, the MCID is equal to the upper value of the 95% confidence interval for the average change in score that is seen in the no change group. The difference in change was defined as the difference in the average change in score between the minimal change and no change groups. A receiver operating characteristic (ROC) curve was used to define the cutoff point that best discriminated between the minimal change and no change groups. The optimal cutoff point was estimated using the point that maximized both specificity and sensitivity. The area under the ROC curve (AUC) was calculated to assess reliability. An AUC value of 0.7 to 0.8 was considered acceptable and an AUC value of 0.8 to 0.9 was considered excellent.²⁰

For distribution-based methods, an SD-based estimate and effect size-based estimate were used in this study. The SD represented variation among groups of scores; a previous study found that an SD of 0.5 is equivalent to the MCID.²¹ The effect size is a standardized measure of change that is obtained by dividing the difference in scores from baseline to posttreatment by the SD of the baseline scores. An effect size of 0.2 is considered small, 0.5 is moderate, and 0.8 is large.²² The MCID was calculated by multiplying the SD of the baseline scores by 0.2.²³

Calculation of the SCB

SCBs were calculated using ROC curve analysis to define the cutoff point that best discriminated between the substantial (“much better”) and nonsubstantial (“somewhat better,” “about the same,” or “slightly worse”) improvement groups. Additional analysis was performed (1) to ascertain the presence of a significant difference in the patient demographics between these 2 groups and (2) to clarify if the baseline PROMs had the ability to predict SCB achievement using ROC curve analysis with AUC. A calculated value was determined as significantly predictive based on an AUC value higher than 0.7.

Statistical Analysis

Differences in patient demographics and the characteristics of cartilage defects were compared using an unpaired t test or Mann-Whitney U test for continuous data, based on the distribution of data that was determined with the use of Shapiro-Wilk test. For categorical data, Fisher’s exact test or Pearson chi-square was used accordingly. The Wilcoxon signed-rank test was used to compare differences in the PROMs between the 2 time points. The Mann-Whitney U test was used to compare differences in the PROMs and the mean change in the PROMs between the “no change” and “minimal change” groups. The level of significance was set a priori at P < 0.05. All statistical analyses were performed using Stata (version 13; StataCorp LP, College Station, TX, USA).

Results

Patient Cohort

The 92 patients who were included in this study were evaluated at a mean of 2.3 years postoperatively (SD 0.6, median 2, range 2-4 years). All the postoperative functional scores improved significantly (Table 2). Fifty-six (60.9%) patients responded that their knees were much better than they were before surgery, 22 (23.9%) patients responded that their knees were somewhat better, 7 (8.7%) responded that they were about the same, 5 (7.6%) reported that they were somewhat worse, and 2 (2.2%) reported that their knees were much worse (Table 3).

Table 2.

Pre- and Postoperative PROMs in Patients Included in This Study.

PROM	n	Preoperative	Postoperative	P
KOOS score
Pain	85	62.8 ± 20	81.8 ± 17.8	<0.001
Symptom	87	47.2 ± 17.8	51.2 ± 14.5	0.0332
ADL	85	73.3 ± 18.4	88.9 ± 14.4	0.0332
Sport/Recreation	86	31.6 ± 22.1	59.5 ± 26.7	<0.001
QOL	87	28.7 ± 20.1	57.0 ± 24.0	<0.001
IKDC	86	42.8 ± 18.1	67.1 ± 21.6	<0.001
Lysholm	86	55.9 ± 20.9	75.9 ± 19.6	<0.001
SF-12
PCS	92	40.2 ± 9.6	49.2 ± 8.6	<0.001
MCS	92	51.2 ± 9.2	54.1 ± 7.6	0.0052

PROMs = patient-reported outcome measures; KOOS = Knee Injury and Osteoarthritis Outcome Score; ADL = activities of daily living; QOL = quality of life; IKDC = International Knee Documentation Committee Subjective Knee Evaluation Form; SF-12 = Short Form–12; PCS = Physical Component Summary; MCS = Mental Component Summary.

Table 3.

Anchor Question (N = 92).

Question	n (%)
Compared with before each surgery, how would you rate your operated joint now?
Much better	56 (60.9)
Somewhat better	22 (23.9)
About the same	7 (8.7)
Somewhat worse	5 (7.6)
Much worse	2 (2.2)

The MCID

Twelve (about the same and somewhat worse) and 22 patients (somewhat better) were assigned to the “no change” and “minimal change” groups for the MCID calculation, respectively. There was no significant difference in the follow-up periods between these 2 groups (P = 0.4543). There was no significant difference between these groups in terms of baseline characteristics (Table 4). We were not able to calculate the MCID of the KOOS symptom scale and Lysholm score and the MCS of the SF-12 score using an anchor-based method because the mean change in the no change and minimal change groups was not significantly different (Table 5). The MCIDs for each PROM that was calculated with the 4 anchor-based methods or 2 distribution-based methods are shown in Table 6. The 4 anchor-based methods provided a range of MCIDs for each PROM (11-18.8 for the KOOS pain scale, 9.2-17.3 for the KOOS activities of daily living [ADL] score, 12.5-18.6 for the KOOS sport/recreation score, 12.8-19.6 for the KOOS QOL score, 10.8-16.4 for the IKDC score, and 6.2-8.2 for the SF-12 PCS score). In the 2 different distribution-based methods, the MCID for the KOOS symptom scale ranged from 3.6 to 8.4, that for the Lysholm score ranged from 4.2 to 10.5, and that for the SF-12 MCS score ranged from 1.9 to 4.6. The results showed that the value of the MCID depended on the method that was applied. All AUCs that were defined by the ROC curve were greater than 0.7, which indicated that the cut-off point was acceptable. The KOOS sport/recreation score had the highest AUC (AUC = 0.8).

Table 4.

Baseline Demographics in the Patients Included in the Analysis.

	No Change Group (n = 12)	Minimal Change Group (n = 22)	P
Age, years, mean ± SD	35.4 ± 9.0	32.1 ± 8.5	0.2940
Gender, male/female, n	4/8	9/13	0.734
Body mass index, kg/m2, mean ± SD	27.2 ± 4.8	26.7 ± 4.0	0.7576
Follow-up, years, mean ± SD	2.3 ± 0.7	2.2 ± 0.5	0.4543
Defect size, cm2, mean ± SD	5.9 ± 2.6	7.7 ± 5.5	0.2864
Number of defects, mean ± SD	1.7 ± 0.9	1.6 ± 0.7	0.9151
Worker’s compensation, n (%)	1 (8.3)	2 (9.1)	1.000
Multiple previous surgeries, n (%)	7 (58)	9 (41)	0.475
Defect location, cm2, mean ± SD
Medial femoral condyle	3.9 ± 2.2 (n = 3)	5.6 ± 1.6 (n = 6)	0.2237
Lateral femoral condyle	n = 0	4.3 ± 2.2 (n = 6)	n/a
Patella	3.6 ± 1.7 (n = 9)	4.7 ± 3.1 (n = 16)	0.2655
Trochlea	3.9 ± 1.8 (n = 7)	4.9 ± 2.8 (n = 7)	0.4295
Tibia plateau	1.1 (n = 1)	1.1 (n = 1)	n/a
Concomitant procedure, n (%)	8 (66.7)	15 (68.2)	1.000

SD = standard deviation; n/a = not available.

Table 5.

Pre- and Postoperative PROMs in the No Change Group and the Minimal Change Group.^a

PROM	No Change Group	Minimal Change Group	P
KOOS
Pain
Pre	56 ± 21.0	55.8 ± 23.0	0.9701
Post	59.5 ± 18.4	74.7 ± 13.2	0.0075
Mean score change	3.5 ± 11.9	18.8 ± 18.8	0.0149
Symptom
Pre	40.5 ± 13.0	42.4 ± 18.4	0.7859
Post	36.3 ± 15.6	46.4 ± 12.7	0.0887
Mean score change	−4.2 ± 12.8	4.1 ± 16.1	0.0834
ADL
Pre	69.0 ± 20.7	69.5 ± 20.0	1.000
Post	70.1 ± 19.8	85.5 ± 10.5	0.0084
Mean score change	0.9 ± 12.3	17.3 ± 18.0	0.0139
Sport/Recreation
Pre	33.8 ± 24.8	27.1 ± 22.3	0.3458
Post	32.1 ± 18.8	44.1 ± 23.0	0.1636
Mean score change	−1.7 ± 22.3	16.9 ± 20.1	0.0042
QOL
Pre	25 ± 21.8	25 ± 20.1	0.9278
Post	31.8 ± 17.2	44.6 ± 17.1	0.0422
Mean score change	6.8 ± 14.0	19.6 ± 18.5	0.0256
IKDC
Pre	37.7 ± 19.7	39.6 ± 18.6	0.8570
Post	40.9 ± 18.2	56.0 ± 15.0	0.0279
Mean score change	3.2 ± 12.0	16.4 ± 16.8	0.0211
Lysholm
Pre	46.6 ± 19.1	53.5 ± 23.0	0.3298
Post	51.8 ± 20.6	67.1 ± 17.8	0.0365
Mean score change	6.5 ± 17.8	13.5 ± 21.9	0.3023
SF-12
PCS
Pre	34.3 ± 7.9	37.9 ± 10.2	0.3581
Post	36.4 ± 8.8	46.1 ± 6.4	0.0022
Mean score change	2.0 ± 7.0	8.2 ± 8.4	0.0306
MCS
Pre	49.2 ± 10.6	48.9 ± 8.5	0.8429
Post	51.9 ± 9.6	55.0 ± 6.1	0.4492
Mean score change	2.7 ± 5.9	6.1 ± 9.9	0.2960

PROMs = patient-reported outcome measures; KOOS = Knee Injury and Osteoarthritis Outcome Score; ADL = activities of daily living; QOL = quality of life; IKDC = International Knee Documentation Committee Subjective Knee Evaluation Form; SF-12 = Short Form–12; PCS = Physical Component Summary; MCS = Mental Component Summary.

^aValues are shown as mean ± standard deviation.

Table 6.

The MCIDs for the Various Scores.

	Anchor Based				Distribution Based
PROM	AC	Change Difference	MDC	ROC Curve (AUC)	SD	Effect Size
KOOS
Pain	18.8	15.3	11.0	11.1 (0.76)	—	—
Symptom	n/a	n/a	n/a	n/a	8.4	3.6
ADL	17.3	16.4	9.2	10.3 (0.77)	—	—
Sport/Recreation	16.9	18.6	12.5	15 (0.80)	—	—
QOL	19.6	12.8	15.6	18.8 (0.73)	—	—
IKDC	16.4	13.2	10.8	12.6 (0.74)	—	—
Lysholm	n/a	n/a	n/a	n/a	10.5	4.2
SF-12
PCS	8.2	6.2	6.5	7.2 (0.72)	—	—
MCS	n/a	n/a	n/a	n/a	4.6	1.9

MCID = minimal clinically important difference; PROMs = patient-reported outcome measures; AC = average change; MDC = minimal detectable change; ROC = receiver operating characteristic; AUC = area under the ROC curve; SD = standard deviation; KOOS = Knee Injury and Osteoarthritis Outcome Score; ADL = activities of daily living; QOL = quality of life; IKDC = International Knee Documentation Committee Subjective Knee Evaluation Form; SF-12, Short Form–12; PCS = Physical Component Summary; MCS = Mental Component Summary; n/a = not available.

The SCB

Thirty-four (somewhat better, about the same, and somewhat worse) and 56 patients (much better) were assigned to the “substantial” and “nonsubstantial change” groups for the SCB calculation, respectively. The SCB of all the PROMs was calculated, except for that of KOOS Symptom and SF-12 MCS, using an anchor-based method because the mean change significantly differed between in the “nonsubstantial” and “substantial” improvement groups (Table 7). The SCBs for each PROM that was calculated with the ROC analysis and the percentage of achieving for each SCB are shown in Table 8. AUCs for KOOS sport/recreation, KOOS QOL, IKDC, and Lysholm were greater than 0.7, indicating that the cutoff point was acceptable. The IKDC score had the highest AUC (AUC = 0.8171), followed by KOOS sport/recreation (AUC = 0.8170). Overall, nearly half of the patients achieved the calculated SCBs. All baseline variables, except for sex, were not significantly different between these 2 groups (Table 9). There was a significantly higher percentage of females in the nonsubstantial change group (P = 0.025). Based on the AUC, no calculated value was determined to be significantly predictive for achieving SCB.

Table 7.

Pre- and Postoperative PROMs in the Nonsubstantial Group and the Substantial Improvement Group.

PROM	Nonsubstantial (n = 34)	Substantial Improvement (n = 56)	P
KOOS
Pain
Pre	55.9 ± 21.9	67.3 ± 17.7	0.0225
Post	69.4 ± 16.7	91.6 ± 9.1	<0.001
Mean score change	13.2 ± 18.0	24.4 ± 15.8	0.0045
Symptom
Pre	41.7 ± 16.5	50.6 ± 18.1	0.0119
Post	42.9 ± 14.4	58.1 ± 10.7	<0.001
Mean score change	1.2 ± 15.3	7.1 ± 19.1	0.1841
ADL
Pre	69.3 ± 19.9	75.8 ± 17.5	0.1533
Post	80.1 ± 16.0	95.0 ± 8.5	<0.001
Mean score change	11.7 ± 17.9	19.7 ± 15.5	0.0321
Sport/Recreation
Pre	29.5 ± 23.1	33.4 ± 21.8	0.5345
Post	39.9 ± 22.1	74.2 ± 17.9	< 0.001
Mean score change	10.1 ± 22.5	41.2 ± 24.4	< 0.001
QOL
Pre	25.0 ± 20.4	31.1 ± 20.1	0.1344
Post	40.1 ± 18.0	70.0 ± 18.2	<0.001
Mean score change	15.1 ± 18.0	39.1 ± 23.0	<0.001
IKDC
Pre	38.9 ± 18.8	45.7 ± 17.7	0.1000
Post	50.7 ± 17.5	80.1 ± 13.5	<0.001
Mean score change	11.7 ± 16.4	34.2 ± 18.5	<0.001
Lysholm
Pre	51.2 ± 21.8	59.0 ± 20.3	0.0901
Post	61.7 ± 20.0	85.9 ± 10.3	<0.001
Mean score change	11.2 ± 20.6	27.2 ± 19.4	0.0013
SF-12
PCS
Pre	36.6 ± 9.5	42.3 ± 9.3	0.0054
Post	42.7 ± 8.6	53.7 ± 4.6	<0.001
Mean score change	6.0 ± 8.4	12.0 ± 9.9	0.0020
MCS
Pre	49.0 ± 9.1	52.5 ± 9.2	0.0696
Post	53.9 ± 7.6	54.8 ± 7.0	0.3620
Mean score change	4.9 ± 8.8	2.7 ± 9.0	0.3988

PROMs, patient-reported outcome measures; KOOS, Knee Injury and Osteoarthritis Outcome Score; ADL, activities of daily living; QOL, quality of life; IKDC, International Knee Documentation Committee Subjective Knee Evaluation Form; SF-12, Short Form 12; PCS, Physical Component Summary; MCS, Mental Component Summary.

^aValues are shown as mean ± standard deviation.

Table 8.

The SCBs and Percentage of Achievement to SCB.

PROM	ROC	AUC	Sensitivity	Specificity	% Achieving
KOOS
Pain (n = 83)	27.7	0.6852	48.0	78.8	45.8
Symptom (n = 85)	14.28	0.5854	37.3	76.5	37.6
ADL (n = 83)	29.4	0.6403	37.3	84.4	37.3
Sport/Recreation (n = 84)	30	0.8170	72.6	81.8	58.3
QOL (n = 85)	37.5	0.7941	62.8	82.4	50.6
IKDC (n = 84)	34.4	0.8171	62.0	91.2	47.6
Lysholm (n = 84)	29	0.7092	52.9	81.8	46.4
SF-12
PCS (n = 87)	14.7	0.6976	47.2	85.3	37.9
MCS (n = 87)	4.4	0.4462	47.2	61.8	48.3

PROMs = patient-reported outcome measures; KOOS = Knee Injury and Osteoarthritis Outcome Score; ADL = activities of daily living; QOL = quality of life; IKDC = International Knee Documentation Committee Subjective Knee Evaluation Form; SF-12 = Short Form–12; PCS = Physical Component Summary; MCS = Mental Component Summary; ROC = receiver operating curve; AUC = area under the ROC curve.

Table 9.

Baseline Demographics in the Nonsubstantial Group and Substantial Improvement Group.

	Nonsubstantial (n = 34)	Substantial Improvement (n = 56)	P
Age, years, mean ± SD	33.3 ± 8.7	31.3 ± 9.4	0.3382
Gender, male/female, n	13/31	30/26	0.025
Body mass index, kg/m2, mean ± SD (range)	26.9 ± 4.3 (19.3-34.2)	26.4 ± 4.4 (18-38.2)	0.5651
Follow-up, years, mean ± SD, y	2.2 ± 0.6	2.3 ± 0.7	0.4481
Defect size, cm2, mean ± SD	7.1 ± 4.7	7.4 ± 4.8	0.7813
Number of defects, mean ± SD	1.6 ± 0.8	1.6 ± 0.9	0.6026
Worker’s compensation, n (%)	3 (8.8)	3 (5.4)	0.669
Multiple previous surgeries, n (%)	16 (47.1)	17 (30.4)	0.121
Defect location, cm2, mean ± SD
Medial femoral condyle	5.0 ± 1.9 (n = 9)	4.3 ± 2.3 (n = 15)	0.4132
Lateral femoral condyle	4.3 ± 2.2 (n = 6)	4.6 ± 2.6 (n = 14)	0.7799
Patella	4.3 ± 2.7 (n = 25)	4.5 ± 2.0 (n = 34)	0.7498
Trochlea	4.4 ± 2.3 (n = 14)	5.5 ± 3.5 (n = 24)	0.3117
Tibia plateau	1.1 ± 0.7 (n = 2)	1.5 (n = 1)	n/a
Concomitant procedure, n (%)	23 (67.6)	37 (66.1)	1.000

SD = standard deviation; n/a = not available.

Discussion

This study determined the MCID and SCB of 4 frequently used PROMs in patients undergoing ACI for the treatment of symptomatic full-thickness cartilage defects. We evaluated the MCID and SCB in patients with a minimum of 2 years of follow-up because this is the most clinically relevant time point, when patients experience the greatest improvement after undergoing ACI.^7,8 Each of the 4 anchor-based methods for calculating the MCID yielded a significant range of threshold values. Moreover, a considerable improvement is required for achieving of SCB rather than MCID.

Two different strategies are commonly used to calculate the MCID: the anchor-based and distribution-based methods. In our study, anchor-based methods were used primarily rather than distribution-based methods.²⁴ Although there is no clear consensus on the best approach to determine the MCID, distribution-based methods have been criticized because these methods do not use clinically important questionnaires and do not consider patient perspectives.¹⁴ Therefore, the value of an MCID that is determined using anchor-based methods is thought to be more clinically relevant, while distribution-based methods can either support anchor-based methods or provide an MCID when anchor-based methods are not available.²⁴ Additionally, we used 4 different anchor-based methods because previous studies showed various MCID values with the use of different anchor-based methods.^25,26 In our study, the largest MCID was most often seen in the average-change approach, while the smallest MCID was seen in the MDC approach.

Several studies reported the MCID of the KOOS scale using various methods in patients who underwent cartilage repair procedures. Using an anchor-based method, Ebert et al.²⁷ evaluated 104 patients at 5 years after matrix-induced ACI procedures and reported that the MCID of the KOOS sport/recreation score was 40, which could accurately predict whether patients would respond “very satisfied.” However, they defined the MCID as the magnitude of the change in score that best discriminated patients who reported being “very satisfied” from those with the other 3 responses (“somewhat satisfied,” “somewhat dissatisfied,” and “very dissatisfied”). Thus, this does not represent a “minimal change,” and the reported MCID of the KOOS sport/recreation score (40) was much greater than our calculated MCID (12.5-18.6), but similar to our calculated SCB (30). Using a distribution method, Engelhart et al.¹⁶ showed that the MDC of the KOOS scale in a mixed cohort of patients who underwent autograft implantation of the cartilage or a microfracture procedure ranged from 7.4 to 12.1, with a follow-up period of up to 12 months. The MCID for the KOOS pain, ADL, and sport/recreation subscales ranged, but the KOOS QOL score (our study: 12.8-19.6 vs. Engelhart et al.¹⁶ 7.4-8.7), which was evaluated using anchor-based methods, overlapped with the MDC range that was reported in their study. Our study revealed that there was a higher value of the upper limit than that reported in this previous study. The MDC of their KOOS QOL score was clearly smaller than ours. With the use of anchor-based methods, we successfully calculated the MCID for the KOOS subscale in patients undergoing ACI.

Using an anchor-based method, Greco et al. reported that the MCID of the IKDC score was 16.7 at 1 year after various cartilage repair procedures,²⁸ which is comparable to or slightly greater than that reported in our study. Our value was calculated using a more homogenous patient cohort that underwent ACI, with a minimum of 2 years of follow-up. However, their study also included patients who responded “much better” in the minimal change group, which we believe artificially elevated the cutoff point. Our study identified the MCID for the IKDC score (10.8-16.4) after ACI by identifying a subset of people who experienced minimal change.

Using an anchor-based method, Clement et al.²⁹ reported that the overall MCID of the SF-12 PCS score was 5 after total knee arthroplasty in patients with a mean age of 70 years. Our calculated MCID was greater than that reported in the study by Clement et al. This difference indicates that the MCID may vary according to age and procedure; thus, it is important to determine the value of the MCID for specific surgical procedures in a homogenous patient population.

We were not able to calculate the MCID for the KOOS symptom, the Lysholm scale, and the SF-12 MCS score using anchor-based methods because there was no significant difference in the mean score of each PROM between the minimal change and no change groups. The relatively small sample size might have hindered the detection of significance. Thus, a further study with a larger sample size is necessary. Using distribution-based methods, our calculated MCID for the Lysholm scale and the SF-12 MCS were comparable to those in a previous study that used distribution-based methods in patients who underwent anterior cruciate ligament reconstruction.³⁰

Our results demonstrated that a considerable improvement was required to achieve SCB rather than MCID after ACI, which are consistent with previous studies in other orthopedic procedures.^31,32 Nearly half of the patients achieved SCB, which is useful to discuss between treating surgeons and patients before surgery as patients usually expect optimal result rather than minimal improvement. Moreover, our results showed that the percentage of female patients was significantly lower in the “substantial improvement” group, which is consistent with previous studies demonstrating less favorable outcomes in females.^33-35

This study has several strengths. First, we primarily used 4 anchor-based methods or 2 distribution-based methods for calculating the MCID, as indicated. Because there is no gold standard to determine the MCID, different methods are recommended to estimate its range.²⁴

Second, we reported the MCID and SCB comprehensively, which we believe is useful for understanding the effectiveness of treatment using the PROMs. Finally, this study was a single-surgeon series of patients with the same indications, procedures, and postoperative courses.

However, the study also has several limitations. First, we only included 92 of the 242 patients who were potentially eligible due to either lack of 2-year follow-up or incomplete pre- and postoperative data sets. Thus, the limited number of patients who were included in this study may potentially limit our ability to generalize our findings, although we did not find any statistical nor clinically meaningful difference in the baseline characteristics of the included and excluded patients. Second, although an anchor-based method has some superiority to distribution-based methods, this method still has some limitations, primarily recall bias. Third, we did not analyze the effect of several factors that might have influenced the patient’s perception, such as age, sex, body mass index, defect location, defect size, and preoperative mental health status. However, there was no significant difference in the baseline characteristics between the no change and minimal change groups. Stratifying the groups according to these variables may have provided different MCIDs and SCBs, although it was impossible to do so in this study because of the sample size.

Conclusions

In conclusion, our results successfully established the MCIDs and SCBs of several PROMs in patients undergoing ACI for symptomatic cartilage lesions. These results will allow interpretation of the treatment effect in both clinical practice and clinical trial settings and can be used to estimate sample sizes for future trials. A considerable improvement was required to achieve substantial improvement after ACI. Given the range of MCID values that were identified in this study, standardization of the most appropriate calculation methods may be warranted.