CORR Insights®: What is the Minimum Clinically Important Difference for the WOMAC Index After TKA?

Where Are We Now?

Too much emphasis has been placed on whether differences in orthopaedic studies are “statistically significant,” and not enough attention has been paid to whether the findings are large enough for patients to care about. Treatment-effect sizes in terms of the Minimum Clinically Important Difference (MCIDs) is a priority at this journal [4] since research that fails to focus on effect sizes can cause clinicians to adopt treatments that expose patients to risk (and the healthcare system to costs) when any benefits of treatment are too small to matter. Reliable estimates of MCIDs for the commonly used orthopaedic instruments have been published in summary form in Clinical Orthopaedics and Related Research® [5], which is handy for busy clinicians.

The authors of the current study [2] take this uncertainty as an opportunity, comparing MCID estimates using three methods to determine how much change would be both reliably detected and meaningful to the patient. They recommend using the smallest MCID estimate, based on regression, as the most conservative, given that a power analysis based on too large of an expected change may result in an underpowered study.

The authors of this paper [2] also explore uncertainty that arises from missing data. Missing data can bias study results if the patients who were unavailable for followup are different from the patients who were available. In the current study, one-sixth of their patients are lost to followup [2]. Multiple imputation is recognized as the preferred method for addressing the potential contributions of missing data [3] and is available in many established statistical packages. Multiple imputation generates a range of plausible values based on regression models of existing data. This approach can allow estimation of a statistical model that is based on all data, not just records where data is complete. These imputed models should be relatively free of bias. Clement and colleagues [2] report that the effect of multiple imputation increased each of the estimates of MCID, although by an amount that is smaller than the observed confidence intervals of the MCID estimates. They suggest this increase may in fact be due to bias introduced by the imputation, given that “no improvement” patients would be expected to have lower WOMAC than “little improvement” patients, making the net difference seem larger. Although the estimated impact of missing data on the MCID estimate here appears negligible, the fact that MCID studies may be missing data under the right conditions is important information for these studies moving forward.

Where Do We Need to Go?

First, we need to recognize that even well-validated outcomes tools are influenced by parameters other than the efficacy of a particular medical or surgical intervention. Patients’ responses to questions may depend on general physical health (age, obesity, physical fitness) or mental health (anxiety or depression). Sociodemographic features such as education or poverty—themselves linked with general health—may also affect responses. These, in turn, may affect whether we can consider a single MCID score appropriate for all patients taking a particular instrument. Future research needs to better characterize how such factors influence outcomes scores.

Next, how do we identify a meaningful change in a patient’s status? I believe the best approach to assessing the MCID is by using an anchor question [4, 5], and seeing how many points gained on a particular outcomes tool is required before patients start to answer the anchor question differently. These anchor questions may reflect their current status ( “I feel good today”) or describe a change (“I feel better today than I did yesterday” or “… than I did 3 weeks ago”). The resulting MCID would have different interpretations based on the type of anchor. But the inherent subjectivity in the anchoring process points to what should be a few obvious concerns: Are some anchor questions better than others? Are patient-reported measures equally reliable for identifying improvement within individual patients and for making comparisons between groups receiving different treatments? If we have the choice between smaller and larger MCID estimates, which one should we accept? The larger MCID may reflect better patient benefit and show greater statistical reliability but may also result in underpowered studies if the effect size is larger than what would actually be seen.

And finally, in the case where the MCID has a confidence interval, do we want to accept the “average” estimate or are we better off using the lower or the higher end of the range?

How Do We Get There?

The potential future development of the MCID and alternatives to the MCID have already been well discussed elsewhere [1, 4]. I would only add here my concern about how the question, “did the patient meet the MCID?” might become a rubber stamp for acceptability similar to the p value. The value of a published study is more than whether the p value met statistical significance, and the outcome for a patient should be judged on more than whether a questionnaire score met a threshold. Perhaps this is a cynical view, but there is the old line “once you have a hammer, everything looks like a nail.” We have all seen cases of being hammered by numbers.

The MCID is still a useful tool in population research, as it allows comparison between groups based on aggregate responses. Potential concerns about an MCID being appropriate to assess a single patient should be less relevant to a population-based estimate. The caveat here is that a study design based on simply comparing score averages between groups must consider that a handful of high improvers may have the same effect on the average as many patients with smaller score increases. A design based on comparing percentages of patients meeting MCID may be more robust, although it might also require a larger sample size.

The potential impact of missing data should be kept in mind, as well. At the very least, journal editors may want to consider requiring authors to assess whether missing data could introduce bias in study results; for example, whether certain groups of patients were more likely to submit incomplete responses. The potential effects of missing data could be addressed by taking existing data sets, randomly inserting missing values, and recalculating the MCID. A potentially useful technical point is that “mixed-model” regressions, which account for repeated longitudinal measurements on the same patient, will use all existing data without dropping a patient because they are missing an outcome at any time.