Editorial: What Should We Do if There is No Minimum Clinically Important Difference Defined for a Key Study Endpoint?

There are two ways one can report between-group differences in comparative clinical studies: The better way and the easier way. Unfortunately, too often, authors and the journals that publish their work choose the latter.

The easier way emphasizes p values and the idea that hitting a p value of a certain size is important. As we’ve said before [2, 3], patients and surgeons don’t perceive p values; they perceive changes to their health, which get summarized as treatment effect sizes when averaged in clinical research. The trouble sets in when trivial effect sizes—the “impacts” of which would be imperceptible, even to the most sensitive patients—get married to attractive p values because of large sample sizes. For this reason, we have suggested that authors derive their clinical messages not just from p values but also based on the magnitudes of the observed treatment effects.

This may seem like a walk into the methodological weeds, but don’t fret. It boils down to this: Surgery is the focus of most surgical journals, and surgery always carries risk [4]. This means any intervention a study wishes to recommend needs to deliver improvements in health that are large enough for patients to notice (the minimum clinically important difference [MCID]), to consider large (the substantial clinical benefit [SCB]), or to result in a level of residual symptoms that those patients find that they can live with (the patient acceptable symptom state [PASS]).

The challenge arises when there’s no anchor-based MCID, PASS, or SCB for an outcome on which an author group wishes to report, or when those metrics exist but for patient populations that are too different to apply in the study being reported. It doesn’t make sense to use, let’s say, the MCID for a pain scale derived from patients who had a joint injection when evaluating patients who underwent front-and-back spinal fusions—patients having big spine surgery should reasonably expect larger improvements than those who just had a shot. Somewhat analogously, PASS scores for metrics of physical function drawn from populations of sedentary retirees cannot reasonably be applied to professional athletes. What then?

There’s no universally accepted solution to this problem, but there are some reasonable options to consider, and the problem calls for our attention. An undue emphasis on p values over effect sizes resulted in aspirin being overused for prevention of fatal coronary events [5, 6], a recommendation that only much later was dialed back, because low-dose aspirin reduced the risk of nonfatal myocardial infarction without a reduction in cardiovascular deaths but increased the risk of major bleeding [1]. With so many large-database studies and large-center or national registry studies now being published in orthopaedic journals, we will see more and more studies that can find statistical differences; with that in mind, we need to keep laser focused on effect sizes and not just p values.

Until a better solution emerges, here is what we’ll ask from authors publishing in Clinical Orthopaedics and Related Research® when there is no MCID defined for a key endpoint in their study:

• In the Methods section, authors should indicate that there are no suitable, published, anchor-based benchmarks for MCID, PASS, or SCB. By “suitable,” we mean benchmarks that apply to the operation or treatment being studied and/or to the population in question.

• Authors then should suggest an effect-size difference (either between groups or before and after an intervention) that they believe would be large enough to matter to patients. This should be done using a primary study outcome variable (such as a pain score or another valid patient-reported outcomes metric), and the authors should present an argument as to why the difference has face validity.

• In the Results section, authors should present their findings in one of three ways, choosing the one that clearly addresses the issues of inference testing—whether probabilistic (p values), Bayesian, or other—as well as effect size (Fig. 1). Claims should be accompanied by data that include one or more point estimates, the uncertainty around those estimates (confidence or credible intervals), and statistics in support of the claim.

• In the “Limitations” portion of the Discussion section, authors should remind readers that, because of the absence of anchor-based MCID, PASS, or SCB metrics, they defined clinically important differences that they believed were large enough to matter, but that readers should consider entertaining alternative interpretations based on the possibility that those definitions may be either too permissive or too severe.

• In the Discussion section, authors should compare their findings to those of others in light of reasonable effect-size metrics.

Fig. 1.

How to present your findings in a way that appropriately conveys their clinical relevance.

Of course, where anchor-based estimates of MCID, PASS, or SCB have been defined for a population similar enough to the one being reported on (or in those uncommon circumstances where distribution-based estimates need to be used), authors should cite and use those metrics. In these circumstances, it may be relevant to report what proportions of patients achieved an MCID, PASS, or SCB in the groups evaluated, and, if it is a comparative study, the difference between the groups on this important endpoint. But in the absence of such available benchmarks, we believe the steps we’re proposing here make sense, and we believe they will help our readers avoid recommending unhelpful surgical interventions to their patients.

Finally, since not all journals focus on these issues, we will end with a few words of advice for readers. We suggest that you look carefully for effect sizes that make sense to you as a clinician, taking into consideration the confidence interval around the estimated treatment-effect size, which conveys the range or uncertainty around that treatment effect. Keep an open mind—it’s possible that your current practice includes approaches that are of questionable benefit or no benefit at all. But also watch out for (and be skeptical of) studies that imply or suggest the need for a major change based on a p value without demonstrating a clear clinical benefit that will be large enough for you or your patients to perceive. Instead, look for differences that would matter to you if you or a loved one were a patient. Lastly, and importantly, don’t just focus on possible benefits; think also about the potential harms of the treatment being considered.