Disagreement with Categorization of Clinical Trials included in Treadwell et al. Systematic Review and Meta-Analysis

We write to express our concern about “Behavioral interventions for migraine prevention: A systematic review and meta-analysis”¹ by Treadwell et al.¹ We disagree with categorizing our control groups as active interventions when designed as attention control groups.^{2, 3} We also disagree that our studies were categorized as comparative effectiveness clinical trials when designed as efficacy studies.

This important systematic review and meta-analysis summarizes the behavioral research in migraine to date. The authors of this study classified our control groups as active interventions. We disagree with this classification. Our control groups^{2, 3} were designed as attention control groups to control for time and attention, but were not designed to provide an active treatment intervention and were not expected to provide benefit.⁴ For example, as an attention control group, Wells’ et al.’s headache education control group² was specifically designed to provide content-based information without behavior change recommendations. Treadwell and colleagues themselves defined behavioral interventions as “nonpharmacological strategies intended to modify behavior and/or ways of thinking for adults, adolescents, and children with migraine.”¹ The goal of our inactive attention control group was to match the time and attention of the mindfulness intervention, and it was not behavioral or intended as an active intervention.

Further, our pre-registered, NIH-funded studies were designed as efficacy randomized controlled trials (RCTs) with an intervention and an attention control group,⁴ but Treadwell et al. categorized them as comparative effectiveness clinical trials.¹ Comparative effectiveness studies^5–7 compare two active interventions to find out which one works best. Our studies included only one active intervention that was expected to work. Further, the critical choice of study design has major influence on other elements of study conduct in addition to and beyond the categorization of the control group. For example, other features often seen in comparative effectiveness studies but not in our study include: a “standard of care” group, non-inferiority analyses that require much larger sample sizes, and a real-world setting rather than the controlled environment of efficacy clinical trials.^5–8 The mis-categorization of our studies resulted in exclusion from the mindfulness meta-analysis. As the largest fully powered, pre-registered RCTs in the field, excluding our studies significantly impacts the results of the meta-analysis.

In view of our disagreement, it would be helpful if the authors repeat the analysis, including our studies^{2, 3} with the efficacy mindfulness studies, to show readers how this would change the results. The Treadwell et al. publication¹ has the potential to shape future work, including potential Guidelines, insurance coverage, and research directions. The appropriate categorization of our work could change the findings and conclusions of the study. Thus, in our view, a way forward would be for the analyses to be redone with the appropriate categorizations of our studies as defined a priori, conducted, and published. In the interim this letter will alert readers to this matter.