Systematic review of outcomes and endpoints in acute migraine clinical trials

Abstract

Background/Objective

To review the acute migraine clinical trial literature and provide a summary of the endpoints and outcomes used in such trials.

Method

A systematic literature review, following a prespecified (but unregistered) protocol developed to adhere to recommendations of the Preferred Reporting Items for Systematic Reviews and Meta‐Analyses, was conducted to understand endpoints and outcomes used in acute migraine clinical trials. Predefined terms were searched in PubMed to locate clinical trials assessing acute migraine treatments. Final database search was conducted on October 28, 2019. Identified publications were reviewed against established inclusion and exclusion criteria to determine eligibility. Data related to general trial design characteristics, sample characteristics, and outcomes and endpoints reported in each publication were extracted from eligible publications. Descriptive summaries of design features, sample characteristics, and the endpoints and outcomes employed across publications were constructed. Outcomes are presented within four broad categories: (a) pain‐related outcomes (pain relief, pain freedom, etc.), (b) associated symptoms (nausea, photophobia, etc.), (c) disability/impairment/impact, (d) patient‐reported outcome measures (PROMs, general health and migraine/headache‐specific). Endpoint types were categorized within three broad categories: (a) change from baseline, (b) fixed timepoint, and (c) responder definitions (e.g., 50% reduction). This review focuses on a subset of recent (1998 or later) randomized and blinded publications evaluating drugs or medical devices.

Results

Of 1567 publications found through the initial search and reference section reviews, 705 met criteria and were included for data extraction. Inter‐rater agreement kappas for the descriptive variables extracted had an average kappa estimate of 0.86. The more recent, randomized and blinded pharmaceutical and medical device article subset includes 451 publications (451/705, 63.9%). The outcomes and endpoints varied substantially across trials, ranging from pain relief or freedom, freedom from or relief of migraine‐associated symptoms, use of acute or rescue medication, and various other PROMs, including measures of satisfaction and quality of life. Within the recent randomized and blinded article subset, most articles examined ≥1 pain‐related outcome (430/451, 95.3%). Of the publications that examined pain, outcomes most often used were pain relief (310/430, 72.1%), pain freedom (279/430, 64.9%), and headache recurrence (202/43,051, 47.0%) or rescue medication use (278/430, 64.9%). Associated symptoms such as nausea, photophobia, and phonophobia were more frequently measured (299/451, 66.3%) compared to most bothersome associated symptom (16/451, 3.5%), as it is a new addition to regulatory guidance. Over one‐third of eligible publications examined disability/impairment (186/451, 41.2%) or ≥1 PROM (159/451, 35.3%). The definition of the endpoints used (e.g., change from baseline, fixed timepoint comparisons, categorization of “responders” to treatment based on wide variety of “responder definitions”) also differed substantially across publications.

Conclusion

Acute migraine clinical trials exhibit a large amount of variability in outcomes and endpoints used, in addition to the variability in how outcomes and endpoints were used from trial‐to‐trial. There were some common elements across trials that align with guidance from the International Headache Society, the Food and Drug Administration and other regulatory agencies (e.g., assessing pain and associated symptoms, 2‐hour post‐treatment). Other aspects of acute migraine clinical trial design did not follow guidance. For example, multi‐item PROMs intended to measure constructs (e.g., scales) are rarely used, the use of pain‐related outcomes is inconsistent, some associated symptom assessments are idiosyncratic, and the timing of the assessment of primary endpoints is variable. The development of a core set of outcomes and endpoints for acute migraine clinical trials that are patient‐centered and statistically robust could improve the conduct of individual trials, facilitate cross‐trial comparisons, and better support informed treatment decisions by healthcare professionals and patients.

INTRODUCTION

Migraine is a highly prevalent, chronic, and often disabling neurological disease with pervasive negative impact on individuals with the disease, their families, and society as a whole. ¹ , ² The 2016 Global Burden of Disease analysis reported that migraine is the second leading cause of years lived with disability. ² Analyses of data from the United States, population‐based, American Migraine Prevalence and Prevention Study ³ , ⁴ , ⁵ found that approximately 12% of respondents, including 17.4% of females and 5.7% of males, met criteria for migraine and 0.91% met criteria for chronic migraine (1.29% of females; 0.48% of males). As outlined in International Criteria for Headache Disorders (3rd ed; ICHD‐3) criteria, a migraine attack is characterized by a set of pain features and associated symptoms in various combinations such as nausea, vomiting as well as sensitivity to light and sound. ¹ First published in 1988, the three iterations of the ICHD criteria have improved the reliability and uniformity of diagnosis in clinical trials. ¹

Therapeutic approaches to migraine fall under two broad categories: preventive and acute treatments. ⁶ Preventive treatments, which include both pharmacological and non‐pharmacological options, aim to reduce frequency, severity, and duration of attacks, improve response to acute treatment, and reduce attack‐related disability. ⁶ , ⁷ Acute migraine treatments aim to resolve migraine pain and symptoms when an attack occurs and return individuals to a “normal” level of functioning as quickly as possible. ⁸

Almost 30 years ago, the International Headache Society (IHS) first published guidelines to help improve the quality of acute migraine clinical trials. ⁹ These guidelines were updated in 2000 ¹⁰ (second edition), 2012 ¹¹ (third edition), and, most recently, 2019 ¹² (fourth edition). The IHS guidelines address subject selection (migraine definition, attack frequency, duration of migraine, age of onset), trial design (blinding, randomization, placebo‐control, number of treated attacks, rescue medication), evaluation of results (headache diaries, (co)primary endpoints, secondary endpoints, adverse events), and statistical analyses (hierarchy of endpoints, power analyses, alpha corrections, statistical analysis plans). The US Food and Drug Administration (FDA) has also provided guidance for the design and conduct of acute migraine clinical trials, including non‐binding recommendations for outcomes and endpoints to be assessed and response scales for assessing those outcomes. ¹³ Other related guidance documents have been published by organizations and agencies including the American Headache Society (AHS), the European Medicines Agency (EMA), and the US National Institute for Neurologic Diseases (NINDS) to help improve treatment research and clinical practice. ⁶ , ¹⁴ , ¹⁵

This review was the first step of a federally funded project with the ultimate goal of creating recommended sets of outcomes and endpoints developed with patient input for both preventive and acute treatment trials in migraine. Ongoing qualitative work involving people with migraine was built on the findings of this systematic review of the acute migraine clinical trial literature. Future psychometric work will also build on this work to support the development of patient‐centered endpoints and outcome measures for use in migraine clinical trials.

The goal of this systematic review was to provide an overview of the acute migraine clinical trial literature with respect to outcomes and endpoints used, which may aid in the selection of harmonization of future endpoints, outcome assessments, and facilitate the development of new and novel treatments. Outcome refers to the measured variable (e.g., pain intensity on a 4‐point ordinal scale), whereas endpoint refers to the analyzed parameter (e.g., the proportion of people with moderate or severe pain at baseline who are free of pain at 2 hours). Herein, we summarize the findings and provide a comprehensive picture of concepts, endpoints, and associated outcomes used in acute migraine clinical trials for adults (defined as 18 years or older) published in English in peer‐reviewed scientific journals.

METHODS

A systematic literature review was conducted to understand the range and frequency of use of specific concepts, endpoints, and associated outcome measures in clinical trials assessing acute treatments for migraine in adults. IHS guidelines and US Food and Drug Administration recommendations for the conduct of acute migraine clinical trials were consulted to identify target variables for extraction. ¹² , ¹³ The unregistered protocol developed for this literature review adhered to the Preferred Reporting Items for Systematic Reviews and Meta‐Analyses (PRISMA) guidelines which include consensus‐based recommendations for the development and execution of high‐quality systematic literature reviews. ¹⁶ The PRISMA checklist includes recommendations for the conduct of the literature search and review, including pre‐specification of eligibility criteria for located publications, the database to be used for the search as well as draft search terms, the standardized process used to review located publications including record tracking/data management systems to be used, the data planned to be extracted from each publication meeting inclusion criteria, and the plan for summarizing the extracted information.

Search strategy

PubMed (https://pubmed.ncbi.nlm.nih.gov/), a search engine maintained by the National Center for Biotechnology Information, at the US National Library of Medicine, located at the National Institutes of Health was used as the primary database queried to identify initial articles for review. PubMed filters were used to limit results to human clinical trials and to articles published in English. No time frame restrictions were imposed on the results and the date of the final search was October 28, 2019.

The PubMed search term used to identify the initial articles was as follows:

(((migraine[MeSH Terms]) AND acute AND Clinical Trial[ptyp] AND Humans[Mesh]) AND English[lang])

The title and abstract of each publication returned from the search were each screened by two of three methodologists (CH, JM, and RF), using the Covidence online systematic review tool (https://www.covidence.org/), for relevance to the stated goals. The inclusion and exclusion criteria as specified in the Covidence system, used to manage the article review process, are provided in Table 1.

TABLE 1.

Inclusion and exclusion criteria used in publication screening process

Study characteristics	Inclusion	Exclusion
Patient population	Interventional, adult acute migraine trial	Trials using only pediatric patients were excluded (mixed adult and pediatric trials were included)
	All migraine types, including subtypes (e.g., menstrual migraine; medication overuse if sample is specified as migraine patients) were included	Trials with ONLY healthy volunteers given an acute intervention (mixed healthy/migraine samples were included)
Interventions	Interventions can be pharmacological (e.g., pills, injections), devices, physical (acupuncture, massage, exercise, etc.), dietary, or other novel treatment intended to treat acute attacks	Preventive migraine trials are to be excluded (mixed trials with preventive and acute outcomes included)
Comparators	Any
Outcomes	Any
Study design/publication characteristics	Open‐label studies and Phase 4 trials	Observational studies, surveys (not Post‐Marketing Phase 4), epidemiological studies, etc.
	Pilot studies with migraine patients	Letters to the editor (including those describing trials), abstracts/papers from conference proceedings, case reports/studies
Language	English	Non‐English

Once the initial list of screen‐pass publications was compiled, a review of the references for each identified publication was undertaken to locate any potentially relevant publications that were previously undiscovered, based on a review of the title and journal in which it was published. Newly located articles from reference section reviews (n = 294 non‐duplicate additional articles) were added to the “initial” list and title and abstract submitted to the screening review (as detailed above) for inclusion/exclusion in the final version of the initial list.

Study selection

With the candidate reference list finalized, a brief review of each full publication was undertaken by two of three methodologists (CH, JM, or RF) to confirm the relevance of the article to the current goals. With an agreed‐upon positive assessment from the brief review, the publication was included in the final references list. All agreed upon negative reviews resulted in the exclusion of the publication from this list. Disagreements on the status of an article were reviewed by a two doctoral‐level study team (CH and JM), blind to previous votes, and a discussion among the reviewers determined the final status of an article regarding inclusion/exclusion in the final list of publications slated for extraction.

Data extraction

All articles in the final list of publications were fully reviewed by a doctoral‐level study team member (DB, CH, or TN) and, if information relevant to the goal of the review was found in the publication during data extraction, it was included in the literature synthesis section of this literature review report.

For all located publications included in the final list of publications, pre‐identified salient key features of each acute publication were extracted. This included extracting all available information related to year of publication, journal name, ClinicalTrials.gov identifier(s), trial name, phase of trial (I‐IV), ¹⁷ general description of the trial design, sample size, patient sociodemographic characteristics (age, sex/gender, race, and ethnicity), salient migraine subtypes (e.g., migraine with or without aura only, menstrual migraine or menstrually related migraine only, episodic vs. chronic migraine, etc.), and type of treatment investigated (pharmacologic/medication, medical device, alternative, multiple/other types). Older terms such as classic migraine (migraine with aura), common migraine (migraine without aura), and transformed migraine (chronic migraine) were recoded using the best approximation in current terminology (in parentheses), based on expert migraine researcher input (RL and DB). Extraction was also based on the language used by authors of each publication reviewed. With respect to migraine diagnosis, for example, “episodic migraine” was only tallied in the data extraction if the authors specified, “episodic migraine,” not historic migraine/headache days. Given the range of studies examined (variety of quality, designs, and years of publication) and lack of information provided (e.g., often vague/limited details on inclusion/exclusion criteria), it was not possible to reliably categorize migraine populations based on the current definitions of episodic versus chronic migraine. This “author language” approach also used in extracting scale type variables as well (e.g., authors stated using a visual analog scale (VAS) but reported methods/results showed a 5‐response category item was used; in such cases, the number of available response categories was extracted and the “scale type” was also extracted as a “VAS.”).

Data extracted from the articles included all outcomes reported in the results section of the publication (e.g., pain freedom, pain relief, disability/impairment outcomes, health‐related quality of life (HRQoL), as well as an “Other” categorization in which extractors provided descriptions of any outcomes or endpoints that did not conform to prespecified columns in the standardized Excel extraction sheet), all endpoints used, and any specific patient‐reported outcome measures (PROMs) included and the timing of their assessments. The protocol also specified extracting data on endpoint hierarchy in the clinical trial analysis plan (e.g., primary, co‐primary, secondary), but given the limited number of publications that reported this information, its collection was abandoned.

Data related to the descriptive trial information were extracted by trained research assistants (LO, RB, SH, JW and TN). A second research assistant independently extracted the same data for approximately 5% of candidate publications. Data related to the concepts, outcomes, and endpoints examined were extracted by one of three doctoral‐level methodologists (DB, CH, and TN) into a pre‐coded, standardized, structured Excel worksheet.

Quality assessment of included studies

Given that the purpose of this systematic review was to provide a summary of endpoints and outcomes used in acute migraine clinical trials rather than provide any sort of meta‐analysis related to results/estimates reported in the publications, an assessment of study quality was not undertaken.

Analyses

To assess the reliability of the extracted values across the researchers extracting information and publications, rater/extractor agreement kappas were calculated. ¹⁸

To synthesize the sizeable amount of information collected during the data extraction from the large number of articles on acute migraine treatment, numerous tables and figures were planned to present summary information in a digestible fashion. These included summary tables focused on the study design characteristics, demographics of study participants, and outcomes (pain, associated symptoms, disability, migraine‐related PROMs, endpoints [change from baseline, fixed timepoint comparisons, responder definitions]), and timing (30 minutes, 1 hour, 2 hours) used. Summary tables were constructed using statistics appropriate to the data being summarized (mean, standard deviation [SD], median, quartiles [Q1 and Q3], minimum/maximum [Min/Max] of continuous variables; counts [N] and percentages [%] of publications including specific design elements). Analyses were conducted in SAS 9.4.

Acute migraine clinical trial outcomes are presented within four broad categories:

1. Pain‐related outcomes: pain relief, pain freedom, general pain, headache recurrence or rescue medication use, and meaningful relief

2. Non‐pain‐associated symptoms: most bothersome symptom, nausea, vomiting, photophobia, phonophobia, aura, and other symptoms

3. Disability/impairment

4. PROMs

   1. Headache‐related PROMs (e.g., 24‐hour Migraine‐Specific Quality of Life Questionnaire ¹⁹ [24‐hour MSQoL], Patient Perception of Migraine Questionnaire‐Revised ²⁰ [PPMQr]).

   2. Non‐headache‐specific PROMs (e.g., treatment efficacy, treatment preference, and patient global impression of change [PGIC]).

These four broad categories were developed by the second author (JM), an experienced migraine researcher, after reviewing an initial set of seven publications and studying the types of information reported and the level of detail available. Review of these draft categories and outcomes placed within each was undertaken by two expert migraine researchers independently (RL and DB) to evaluate their face validity and make suggestions for modification, resulting in the reported categories. The outcomes listed in categories 1, 2, and 3 above are exhaustive of the outcomes included in those subgroups.

Outcomes were used to define endpoints in three broad ways:

1. Change from baseline

2. Fixed timepoint

3. Responder definitions: 50% reduction, 75% reduction, and 100% reduction.

Specific endpoint definitions/timepoints (e.g., 15 minutes, 1, 2, and 24 hours) were also captured for the respective endpoint types.

While the purpose of the review was to summarize the full body of the acute migraine clinical trial literature, studies varied widely in the level of description provided regarding study designs, patient selection, and the outcomes and endpoints used. Due to this, and to allow for a focus on the more recent acute migraine clinical trial literature (which is more relevant to understanding the endpoints and outcomes currently used), the decision was made to focus on the subset of manuscripts that were published in 1988 or later (following the ICHD‐1 publication) ²¹ that were also randomized and blinded (quality of randomization and blinding was not assessed), and included interventions that were pharmacological or related to a medical device. We refer to this group of studies as the “recent randomized and blinded” article subset and report those in this publication.

RESULTS

Study selection and extraction

Of the 1567 publications found through the initial search and reference section reviews, 608 were removed as duplicates leaving 959. In all, 210 publications were identified as irrelevant leaving 749 publications, of which an additional 44 were eliminated based on a range of exclusion criteria leaving a set of 705 publications included for full data extraction. The PRISMA flowchart presented in Figure 1 provides details on the review and selection process. A complete list of all publications located from the PubMed search or reference section reviews and their ultimate status regarding inclusion/exclusion in the final selection of articles is available in the supplemental online materials. The earliest included article was published in 1972 and latest articles were from 2019. Average inter‐rater agreement for the descriptive variables (age, sex, migraine characteristics, trial design features, etc.) extracted from the set of 705 publications had a kappa estimate of 0.86. Given the somewhat inconsistent nature of reporting in the examined articles and the varied age and quality of reporting in the publications, the observed level of inter‐rater agreement was well above the recommended lower bound of 0.6 and was considered strong. ¹⁴ , ²²

FIGURE 1.

PRISMA diagram of article flow through the systematic literature review of acute migraine trials

Primary publication set study characteristics

Of the 705 publications eligible for review, just over 60% reported on trials that were placebo/sham controlled (427/705, 60.6%), over two‐thirds were blinded and randomized (494/705, 70.1%) and over three‐fourths used one of the iterations of the ICHD criteria for migraine (560/705, 79.4%) (Table 2). Of note, many publications published prior to 1988 used the 1962 Ad Hoc Committee criteria for migraine ²³ which used criteria that were not operational but had some overlap with the ICHD criteria for migraine.

TABLE 2.

General publication characteristics (n = 705 total publications)

Study characteristic	Percent	N
Study purpose(s)
Efficacy assessed	96.6	681
Pharmacokinetic study	4.4	31
Study/design features
Study 1988 or later	94.0	663
Randomized	77.6	547
Blinded	73.3	517
Randomized and blinded	70.1	494
ICHD migraine criteria used	79.4	560
Placebo/Sham controlled	60.6	427
Crossover design	23.0	162
Intervention information
Drug/medical device	96.3	679
Multiple active treatments	36.7	259
Open‐label study	21.8	154
Multiple dose levels assessed	23.8	168

Of the 705 primary publication set, most publications examined at least one efficacy outcome (681/705, 96.6%). The majority of publications did not report the study phase (i.e., Phase I‐IV) (625/705, 88.7%; data not shown). In the primary publication set, 94.0% (663/705) of the manuscripts investigated pharmacological/medication treatments, 2.3% (16/705) examined medical devices (e.g., neurostimulation devices, dental devices), 1.6% (11/705) examined complementary and integrative therapies (acupuncture, osteopathic manipulation, herbal supplements), and 2.1% (15/705) examined other interventions or interventions from multiple categories.

Recent randomized and blinded subset

While the purpose of the review was to summarize the full body of the acute migraine literature, to allow for a focus on the more recent and relevant pharmacologic and device acute migraine clinical trial literature, from this point forward we focus on a subset (451/705, 64.0%) of studies as defined in the Methods. Results from the full sample of publications are available in the supplemental online materials.

Sample characteristics

Available demographic characteristics for the subjects from the contemporary subset of 451 publications (pooled over all treatment groups) were summarized. The median total sample size was 243 participants per study (25th percentile: 80; 75th percentile: 640; 410/451, 90.9% of studies reported on age). Of publications that reported age, gender, and/or race descriptives, the average age was found to be 39.1 years (SD = 3.9), with 82.9% of patients identifying as female (427/451, 94.7% of studies reported gender), and 85.1% of patients reported as White/Caucasian (178/451, 39.5% of studies reported on race). Of note in these demographic summary values is that publications conducted exclusively outside of the United States (e.g., China, India, or Iran) often did not report the breakdown of patients into race/ethnicity categories and, therefore, did not contribute data to the current racial/ethnic summary values; while the population of acute migraine trial subjects is predominately White/Caucasian, there is more racial/ethnic diversity than indicated by the reported values due to the lack of reported racial information in a non‐trivial number of non‐US‐based trials. ²⁴

The majority (410/451, 90.9%) of publications examined a general migraine population (unspecified/multiple types), with limited numbers of publications reporting on studies using specific subgroups of migraine patients, such as menstrual migraine (21/451, 4.7%), episodic migraine patients only (17/451, 3.9%), or chronic migraine/transformed migraine patients only (3/451, 0.7%); in interpreting these numbers, particularly the episodic migraine publications, it is important to remember that “author language” was used to determine this classification and “episodic migraine” was not a commonly used term at the time for many publications—it is likely that if inclusion/exclusion criteria had been reliably reported across publications to allow for extraction/coding, numerous publications would likely have been classified as “episodic migraine” judged against the current <15 monthly headache day standard. With regard to aura, 82.5% (372/451) of articles included migraine with and without aura, 6.9% (31/451) included migraine without aura only, 1.1% (5/451) included migraine with aura only, and 9.5% (43/451) articles did not clearly specify.

Outcomes and endpoints

Within the recent randomized and blinded article subset, 95.3% (430/451) of the publications examined ≥1 pain‐related outcome while 67.2% (303/451) examined ≥1 associated symptom or MBS outcome (Table 3). A little over 41.2% (186/451) of publications examined disability/impairment outcomes, and 35.3% (159/451) of publications examined PROMs. Of those publications using PROMs, 94.3% (150/451) used ≥1 non‐headache‐related PROM, and only 17.6% (28/451) of publications examined migraine/headache‐related PROMs.

TABLE 3.

Outcomes assessed across selected subset of publications (n = 451)

Outcome grouping	Percent	N
Pain‐related	95.3	430
Pain relief	72.1	310
Pain free	64.9	279
Rescue medication use	64.9	278
Headache recurrence	47.0	202
Pain general	34.0	146
Meaningful relief	8.6	37
Associated symptoms or most bothersome symptom	67.2	303
Associated symptoms (nausea, vomiting, photophobia, phonophobia, etc.)	98.7	299
Most bothersome symptom	5.3	16
Disability/impairment	41.2	186
Patient‐reported outcome measures (PROMs)	35.3	159
Non‐headache‐related PROMs	94.3	150
Headache‐related PROMs	17.6	28

In examining the various combinations of outcomes used (Table 4), over one‐fifth of publications (105/451, 23.3%) examined ≥1 pain‐related outcome combined with associated symptom outcomes alone. Under 20% of publications examined only pain‐related outcomes (81/451, 18.0%), and 17.7% (80/451) of publications examined pain‐related, associated symptoms, disability/impairment, and used ≥1 PROM. The 11 publications (11/451, 2.4%) listed as using none of the outcomes in our constructed groupings were primarily safety studies (examining only adverse events), pharmacokinetic studies (examining assorted laboratory‐provided values), or studies evaluating health economics outcomes.

TABLE 4.

Combinations assessed across publications (n = 451)

Pain‐related	Associated Symptoms/MBS	Disability/impairment	PROMs (headache‐related and non‐headache‐specific)	Percent	N
Yes	Yes	No	No	23.3	105
Yes	No	No	No	18.0	81
Yes	Yes	Yes	Yes	17.7	80
Yes	Yes	Yes	No	16.9	76
Yes	Yes	No	Yes	8.7	39
Yes	No	No	Yes	5.5	25
Yes	No	Yes	No	2.9	13
No	No	No	No	2.4	11
Yes	No	Yes	Yes	2.4	11
No	No	Yes	No	0.7	3
No	No	Yes	Yes	0.7	3
No	Yes	No	No	0.7	3
No	No	No	Yes	0.2	1

Pain‐related outcomes and endpoints

In assessing headache pain intensity, the IHS acute trial guidelines ¹² recommend three response scales: a four‐category ordinal scale, an 11‐point numerical rating scale (NRS), or a 100 mm VAS. FDA recommendations ¹³ suggest the use of the four‐category ordinal scale only. Of the 430 publications that assessed headache pain intensity in some manner, 89.1% (383/430) used an IHS‐recommended pain scale. Since the same outcome (e.g., a 4‐point ordinal pain scale) can be used to assess a variety of endpoints (2‐hour pain relief, 2‐hour pain freedom, 2‐hour change in pain intensity), we discuss below the specific pain endpoints widely used in acute migraine clinical trials.

There were several commonly encountered specific outcomes that we have classified under the overarching pain‐related outcome headings (Table 3): pain relief (310/430, 72.1%), pain freedom (279/430, 64.9%), general pain (146/430, 34.0%), headache recurrence (n = 202, 44.8%), and rescue medication use (278/430, 61.6%). Meaningful relief was also encountered as an outcome but given the limited number of publications that employed it (37/430, 8.6%), results are not reported here.

Pain relief

Pain relief in the context of clinical migraine trials refers to a reduction in headache pain that is not a complete resolution of the pain. The IHS acute clinical trial guidelines ¹² define Headache Relief using a 4‐point ordinal scale as a decrease from moderate (2) or severe (3) pain at baseline to pain that is mild (1) or absent (0), a definition first proposed by Pilgrim and colleagues. ²⁵ The majority of publications assessing pain relief defined relief in a manner consistent with this recommendation (275/310, 88.7%). Among all publications examining pain relief, a large majority (292/310, 94.2%) used an ordinal response scale. Other publications used a continuous response scale (such as a VAS or NRS) (14/310, 4.5%). Of the publications that stated they used a VAS and/or NRS, over 70% reported using a VAS (10/14, 71.4%). Of note, the term “VAS” was widely and loosely applied to a variety of ratings scales, ranging from true VAS (which asks subjects to provide a tick mark on a line, the position of which is then is measured), to NRSs (typically ranging from 0 to 10), to ordinal scales with as few as five response options.

With respect to the timepoints used when assessing pain relief, a wide variety of endpoints were defined. These ranged from 10‐minute to 24‐hour post‐treatment. The most commonly used timepoint was 2‐hour (269/310, 86.8%), with 1‐hour (199/310, 64.2%), 30‐minute (149/310, 48.1%), and 4‐hour post‐baseline (121/310, 39.0%) also frequently employed.

Several different outcomes were reported as elaborations of pain relief, otherwise some termed pain response. These included sustained response (defined as meeting the criteria for pain relief at a given point and having no headache pain increases or need for additional medication through a set later timepoint) and the consistency of obtaining pain relief across multiple attacks. Table 5 provides a breakdown of the outcomes that are variations of pain relief. Of the 310 publications that examined pain relief, one‐third (105/310, 33.9%) examined sustained pain relief and about 10% of publications examined consistency of pain relief across attacks (39/310, 12.6%) and time to pain relief (30/310, 9.7%). Of the 105 publications that examined sustained pain relief, 70.5% (74/105) examined sustained pain relief at 24 hours only, 8.6% (9/105) at 48 hours only, and 16.2% (17/105) at both 24 and 48 hours.

TABLE 5.

Additional outcomes derived from pain relief (n = 310)

Pain‐relief derived outcome	Percent	N
Sustained response	33.9	105
24 hours	70.5	74
48 hours	8.6	9
24 and 48 hours	16.2	17
Other	4.8	5
Consistency across attacks	12.6	39
Time to relief	9.7	30

Pain freedom

Pain freedom in the context of migraine clinical trials refers to a complete resolution of headache pain. The IHS acute trial guidelines ¹² define Pain Freedom as patients who become free from headache pain following treatment. As noted previously, multiple pain severity rating scales are allowed by IHS acute migraine trial guidelines; however, a comment in the guidelines document notes that the four‐category response scale described above is preferred. For pain freedom, therefore, we tracked the number of publications that used the four‐category response scale for rating headache intensity and also defined freedom as the complete absence of pain (e.g., a response of “None” on the four‐category scale). The large majority of the 279 publications that investigated pain freedom defined it in a manner consistent with the IHS recommendation (252/279, 90.3%). Of the publications focused on pain freedom, the majority (265/279, 95.0%) used an ordinal response scale. Of the publications that stated they used a VAS and/or NRS (8/279, 2.9%), 25% (2/8) used a NRS alone, 62.5% (5/8) used a VAS alone, and 12.5% (1/8) used both an NRS and VAS (based on the response scale name used in the articles, which, as noted previously, may not be reflective of the actual response scale used).

General pain

We use the term general pain to describe general assessments of headache pain that do not conform to pain freedom, pain relief, or meaningful relief. These measures may be derived from any of the pain scales discussed above. Given the somewhat broader nature of the category, more variability was seen in the response scales used to assess pain and the types of analyses used to assess pain. Continuous response scales, used in 59.6% (87/146) of the publications examining general pain were most often used in assessing general pain, followed by ordinal scales (57/146, 39.0%). For those publications using a continuous response scale, 70.0% (61/146) were either 11 or 100‐point scales and for those publications reporting an ordinal response scale, a large majority (51/57, 89.5%) used a four‐category response scale. About half of the 146 general pain publications (79/146, 54.1%) reported using either a VAS or NRS. The VAS was used in about three times more publications than the NRS (VAS: 57/79, 72.2%; NRS: 20/79, 25.3%; VAS and NRS: 2/79, 2.5%). However, it is important to note that response scales that were termed “visual analog scale” by authors covered a wide range of response scales, many that would not be considered a true VAS.

Headache recurrence

The return of headache pain after it was resolved is termed headache recurrence. Recurrence was previously defined as achieving pain freedom and then experiencing a return of moderate to severe headache pain. The most recent version of the IHS acute trial guidelines ¹² replaced the term recurrence with the term “relapse” which they define as the occurrence of a headache of any severity within 24 or 48 hours after the initial treatment; FDA recommendations suggest relapse should be assessed through 48 hours after treatment. Given the variability in the definition and the relatively recent publication of the new IHS “relapse” guideline recommendations, any publication, which specifically stated that they examined headache recurrence/relapse, is included here under the broad categorization of headache recurrence. Headache recurrence/relapse was examined in 44.8% (202/451) of the 451 examined target publications (47% of publications using at least one pain‐related outcome, see Table 3), with the majority of publications using a 24‐hour cutoff alone to define the recurrence window (165/202, 81.7%). Only 8.9% (18/202) of the publications that examined headache recurrence used the 48 hours window alone, which is preferred in the current IHS acute migraine guidelines (7/202, 3.5% examined both 24 and 48 hours).

Rescue medication use

The use of another acute rescue medication or additional doses of study treatment medication (combined) was also a commonly used outcome in acute migraine trials, with over 60% of the 451 examined target publications (278/451, 61.6%) and 64.9% of publications using at least one pain‐related outcome (see Table 3) tracking subjects’ use of additional acute medication to attempt to alleviate the migraine attack.

Associated symptoms and most bothersome symptom

Associated migraine symptoms were often examined in acute migraine trials, with over two‐thirds (303/451, 67.2%) of the publications examining at least one associated symptom or most bothersome symptom (MBS). Historically, the most commonly assessed associated symptoms of acute migraine attacks were nausea, vomiting, photophobia, and phonophobia. Early regulatory guidance required statistically significant effects on pain and these migraine‐defining cardinal‐associated symptoms. Associated symptoms have typically been measured as dichotomous variables (present/absent) but have often been measured on 4‐point ordinal scales similar to the pain scale. More recent regulatory guidance has recommended the use of MBS as an alternative co‐primary endpoint as described below. Some studies have evaluated treated benefits on other associated symptoms including allodynia, osmophobia, neck pain, or dizziness. Rarely have acute treatment effects on aura been evaluated.

Associated symptoms

Of the previously mentioned “core”‐associated symptoms, nausea was the most commonly assessed, 96% (287/299) of the publications that examined associated symptoms including an assessment of nausea. Photophobia was next most common (252/299, 84.3%) followed by phonophobia (230/299, 76.9%). With respect to the rating scales used, across associated symptoms the most commonly used response scale was binary (Presence/Absence; ranging from 45% to 86% across specific symptoms); the use of a binary response scale for associated symptoms aligns with FDA recommendations. ¹³

Most bothersome symptom

The FDA relatively recently recommended the measurement of the most bothersome migraine‐associated symptom, as a co‐primary endpoint in acute migraine trials. ¹³ The definition of MBS requires that subjects designate their most bothersome (associated) migraine symptom from the choices of nausea, photophobia, or phonophobia. The MBS may be designated prior to randomization (and subjects only then treat attacks with study drug or device in which the MBS is present) or at the start of a treated attack, prior to administration of study treatment. Given the recent acceptance of the concept of MBS as an endpoint, a limited number of publications assessed it (16/451, 3.5%). In those publications that assessed MBS, 93.8% (15/16) used a binary (Present/Absent) response scale. The majority of these 16 publications (10/16, 62.5%) used a fixed timepoint endpoint type for analyses while 37.5% (6/16) used a change from baseline formulation. All 16 publications that assessed MBS (16/16, 100.0%) used a 2‐hour post‐treatment endpoint definition, with publications also commonly investigating MBS at 1 hour (7/16, 43.8%) as well as 30 and 90 minutes (both 6/16, 37.5%).

Disability/impairment

Disability/impairment refers to the decrement in a subject's ability to function normally in a wide range of possible domains, such as daily life activities, self‐care, mobility, or in employment/work‐related contexts. The current IHS acute trial guidelines ¹³ recommend assessment of functional disability using a single item, “How well can you function right now?” with four possible response options: “No disability (i.e., able to function normally)” to “Severe disability (i.e., unable to perform most to all activities of daily living or requiring best rest).” IHS also recommends the use of select scales, such as the Migraine Physical Function Impact Diary, ²⁵ to assess functional disability. As the majority of publications assessing disability did so with a single item, we tracked the number of publications using the IHS‐recommended functional disability item or one very similar to it. Of the 186 publications that assess disability/impairment in some way, 67.7% (126/186) of them used the IHS functional disability item or one substantially similar.

With respect to the response scales used for assessing disability, a large majority used an ordinal response scale with four possible response categories (137/186, 73.7%). Within the continuous response scale category, there were continuous rating scales (such as NRSs or VASs) but this could also include such outcomes as time lost to disability or estimated efficiency (as a percent of normal capacity) at work (10/186, 5.4%).

The most common endpoint type was fixed timepoint analyses (96/186, 51.6%), 44.6% (83/186) of the publications examined change from baseline, 2.7% (5/186) examine both fixed timepoints and change from baseline and 1.1% (2/186) examined other endpoints. Endpoints/timepoints at which disability was assessed were relatively consistent across trials, although timepoints from 10‐minute up to 24‐hour post‐treatment were encountered. Of the 186 publications that examined disability, 83.9% (156/186) used 2‐hour post‐baseline (either as a fixed timepoint or change from baseline), 54.3% (101/186) used 1‐hour baseline, and 38.2% (71/186) used 30‐minute post‐baseline.

Patient‐reported outcomes

Combining all PROMs (migraine/headache‐related and non‐migraine/headache‐specific PROMs), 35.3% (159/451) of publications examined ≥1 PROM. Most of the 159 publications assessing a PROM examined ≥1 non‐migraine/headache‐specific PRO (150/159, 94.3%) and publications less frequently assessed ≥1 migraine/headache‐related PROM (28/159, 17.6%).

Headache‐related PROMs

Compared to the preventive literature, the use of headache‐related PROMs in acute migraine trials was much less frequent. Of the 28 publications that examined ≥1 migraine/headache‐related PRO, well over half (17/28, 60.7%) of them used the 24‐hour MSQoL and 17.9% (5/28) assessed the PPMQr. Of note, there were also six named migraine/headache‐related item/scales that were seen in two or fewer publications from the subset, highlighting that lack of standardization in outcomes in the acute migraine research literature. For the interested reader, a full breakdown of all encountered “named” migraine/headache‐related PROMs (from the full 705 article sample) is available in the online supplemental materials. Almost three‐quarters of the publications assessing ≥1 headache‐related endpoint used change from baseline (20/28, 71.4%) and the remaining publications examined fixed timepoints (8/28, 28.6%).

Non‐headache‐specific PROMs

Non‐headache‐specific PROMs are scales/items that are not directly related to headache or migraine and could be used in a variety of disease areas; 150 publications used at least one non‐headache‐specific PROM (see Table 3). The most commonly used non‐headache‐specific PROM scales/items seen in the examined acute migraine trials were related to treatment satisfaction (41/150, 27.3%), treatment efficacy (57/150, 38.0%), and treatment preference (43/150, 28.7%). Treatment satisfaction was often measured using an ordinal scale with four categories (8/41, 19.5%) or seven categories (17/41, 41.5%). Treatment efficacy was often measured using an ordinal scale with four categories (21/57, 36.8%) or five categories (25/57, 43.9%). Treatment preference was most frequently measured using a binary scale (26/43, 60.5%) or an ordinal scale with three categories (7/43, 16.3%) or five categories (5/43, 11.6%). There were several other non‐headache‐specific PROMs used in fewer than five publications. For the interested reader, a full breakdown of all encountered previously non‐reported “named” non‐headache‐specific PROMs (from the full 705 article sample) is available in the online supplemental materials. Like the migraine/headache‐related PROMs, the non‐headache‐specific PROM endpoint type was primarily based on fixed timepoints (142/150, 94.7%) and change from baseline was less often observed (9/150, 6.0%).

DISCUSSION

We conducted a systematic literature review of acute migraine clinical trials following PRISMA recommendations and found that outcomes used to define endpoints varied substantially across trials, ranging from pain relief or freedom, use of rescue medication, and various headache‐related and non‐headache‐specific PROMs, such as those related to the impact migraine has on the patient's life or more general HRQoL. The focal subset of 451 manuscripts published in 1988 or later were randomized and blinded, and included interventions that were pharmacological or related to a medical device. Participants in the subset of articles were largely around 40 years old (mean age across publications was 39.1) and tended to be white (mean percent across publications 85.1%) women (mean percent across publications 82.9%). These results align broadly with previously reported epidemiological studies of people with migraine. ³ , ⁴ , ⁵

This recent randomized and blinded subset of publications often examined ≥1 pain‐related outcome (95.3%), tended to focus on pain relief (72.1%), pain freedom (64.9%), and headache recurrence or rescue medication use (76.5%) as well as associated symptoms or MBS (67.2%). Over one‐third of eligible publications examined disability/impairment (41.2%) or ≥1 PROM (35.3%). Associated symptoms such as nausea, photophobia, and phonophobia were frequently measured (299 publications) at 2‐hour post‐treatment and MBS was assessed far less frequently (only 16 publications total), due to its recent introduction by the FDA as a co‐primary acute migraine clinical trial endpoint. Over one‐third of eligible publications examined disability/impairment and ≥1 migraine/headache‐related and non‐migraine/headache‐specific PROM (41.2% and 35.3%, respectively). Of the 159 publications assessing ≥1 PROM, 94.3% examined a non‐migraine/headache‐specific PROM (often single item measures of treatment satisfaction, efficacy, or preference) while only 17.6% examined ≥1 migraine/headache‐related PROM (most common measures used were the 24‐hour MSQOL and PPMQr). Both migraine/headache‐related and non‐migraine/headache‐specific PROMs were often evaluated using a fixed timepoint endpoint type: 71.4% (migraine/headache related) and 93.3% (non‐headache).

When examining if publications followed current acute clinical trial guidelines, ¹² of the 430 publications that assessed headache pain intensity 89.1% used an IHS‐recommended pain scale (a four‐category ordinal scale, an 11‐point NRS, or a 100 mm VAS). Of the 186 publications that assessed disability or impairment, 67.7% used the IHS guideline recommended single functional disability item or something similar. The majority (81.7%) of publications that assessed pain relief or headache relief defined it in a manner consistent with IHS acute clinical trial guidelines recommendations and of the 375 publications that studied pain freedom, 82.1% defined it consistent with guideline recommendations.

Despite these outcomes and endpoints that were commonly used, there was also a wide variety in study design, endpoints included, and how endpoints were measured across publications. The endpoints and outcomes used in acute migraine treatment trials, even when “common” outcomes are used, had inconsistent operationalizations across publications. As demonstrated in summaries of data extracted from the articles reporting on acute clinical trials, the outcomes used to define endpoints in such trials vary substantially, ranging from change in pain or associated symptoms (relief, freedom, sustained relief, etc.), patient‐perceived disability, and various migraine/headache‐related and non‐migraine/headache‐specific PROMs, such as those related to the impact migraine has on the patient's life or more general HRQoL. The definition of the endpoints used (e.g., change from baseline, fixed timepoint comparisons, categorization of “responders” to treatment based on wide variety of “responder definitions”) also differs substantially across publications.

Limitations of the current project include the possibility of publication bias of positive studies and the possibility of selective reporting of only those outcomes and endpoints that were supportive of efficacy (i.e., not publishing negative results on certain endpoints for a variety of reasons). It is likely that we also did not identify every acute migraine trial publication ever published; however, our sample of 705 fully reviewed and extracted publications is likely large enough to be representative of the field. Additionally, we use the term “publication” or “manuscript” throughout because in some cases two or more publications came from a single study (e.g., efficacy results related to pain outcomes are sometimes reported in separate publications from PROM or other results) and these possible dependencies were not controlled for when summarizing the frequencies of outcome and endpoint occurrence. The scope of the current project also did not allow for the investigation of several methodological and clinical topics that are relevant to headache research. For example, from clinical and trial design standpoints, there are often differences and confusion around how outcomes and endpoints are defined to address headache recurrence and relapse as well as rescue medication use and 2nd dosing. These are specific areas that could benefit from focused research in the future. Methodologically, future studies should investigate issues with specific item wording, response scales, and VAS anchoring, which are outside the reach of the current work.

CONCLUSION

Current trials of acute treatments for migraine exhibit a large amount of variability in outcomes and the ways in which outcomes are used to define endpoints. This systematic review of the acute migraine clinical trial literature showed that while there were some common elements across trials that often align with current recommendations from guidelines (e.g., assessing pain and associated symptoms, focusing on the 2‐hour post‐treatment timepoint, using IHS‐recommended scales), there are also aspects of acute migraine clinical trial design that do not generally adhere to recommended guidelines (e.g., very few PROMs are used, especially migraine/headache related) or are inconsistently applied (e.g., specific pain‐related outcomes, idiosyncratic‐associated symptoms assessed, and endpoint types/timing are variable).

Standardization of the outcomes and endpoints used in acute migraine trials will facilitate cross‐trial comparisons and facilitate communication with patients about the benefits of treatment alternatives. Endpoints in acute treatment trials are defined in various ways. Empirical work could help identify the approaches that optimize power and sensitivity to change. Novel approaches could include analysis of trajectories of change ²⁶ or time to event analysis. Using psychometric methods to scale multiple migraine features (intensity of pain, nausea, photophobia, phonophobia, disability, allodynia) may improve the reliability of measurement improving trajectory analysis or more robust definitions of responders. Additionally, future work should rigorously evaluate existing PROMs against prespecified criteria, and to explore other criteria domains, such as physical and cognitive function. Based on ongoing qualitative work and feedback from the FDA, the development of measures of cognitive function may also be informative.

CONFLICT OF INTEREST

CRH and TKN are full‐time employees of Vector Psychometric Group, LLC. JSM is a full‐time employee of Vector Psychometric Group, LLC and has received honoraria/payment/reimbursement from the journal Cephalalgia (biostatistics editor) and the American Headache Society (teaching course). JSM has also received research grants/support from Amgen, Inc. and the National Headache Foundation. RJW is a full‐time employee and partner of Vector Psychometric Group, LLC, a consulting company servicing numerous pharmaceutical and medical device companies. RJW has received research grant support from Amgen, Inc., the National Headache Foundation, and the US Food and Drug Administration. DCB is a part‐time employee of Vector Psychometric Group, LLC and has received grant support and honoraria from the Food and Drug Administration and the National Headache Foundation and grant support and honoraria from Allergan, Amgen, Biohaven, Lilly, Novartis and Promius/Dr. Reddys. She serves on the editorial board of Current Pain and Headache Reports. RBL has received grant support from the National Institutes of Health, the Food and Drug Administration, the National Headache Foundation, and the Migraine Research Fund. He serves as a consultant, serves as an advisory board member, has received honoraria from or conducted studies funded by Alder, Abbvie/Allergan, American Headache Society, Biohaven, Eli Lilly, eNeura Therapeutics, Lundbeck, Merck, Novartis, Pfizer, and Teva, Inc. He receives royalties from Wolff's Headache, 8th Edition (Oxford University Press, 2009). He holds stock options in eNeura Therapeutics and Biohaven. PJG reports, over the last 36 months, grants and personal fees from Amgen and Eli Lilly and Company, grant from Celgene, and personal fees from Alder Biopharmaceuticals, Aeon Biopharma, Allergan, Biohaven Pharmaceuticals Inc., Cerecin, Clexio, Electrocore LLC, eNeura, Epalex, GlaxoSmithKline, Impel Neuropharma, Lundbeck, MundiPharma, Novartis, Pfizer, Sanofi, Santara Therapeutics, Satsuma, Teva Pharmaceuticals, Trigemina Inc., WL Gore, and personal fees from MedicoLegal work, Massachusetts Medical Society, Up‐to‐Date, Oxford University Press, and Wolters Kluwer. DWD reports consulting: Amgen, Allergan, Abbvie, Eli Lilly, Novartis, Lundbeck, AEON, Clexio, Cerecin, Biohaven, Linpharma, Promius, eNeura, Impel, Theranica, WL Gore, Nocira, XoC, Zosano, Upjohn (Division of Pfizer), Pieris, Revance, Equinox. Honoraria: CME Outfitters, Curry Rockefeller Group, DeepBench, Global Access Meetings, KLJ Associates, Majallin LLC, Medlogix Communications, Miller Medical Communications, Southern Headache Society (MAHEC), WebMD Health/Medscape, Wolters Kluwer, Oxford University Press, Cambridge University Press. Research Support: Department of Defense, National Institutes of Health, Henry Jackson Foundation, Sperling Foundation, American Migraine Foundation, Patient Centered Outcomes Research Institute (PCORI). Stock Options/Shareholder/Patents/Board of Directors: Aural analytics (options), ExSano (options), Palion (options), Healint (Options), Theranica (Options), Second Opinion/Mobile Health (Options), Epien (Options/Board), Nocira (options), Ontologics (Options/Board), King‐Devick Technologies (Options/Board), Precon Health (Options/Board). Patent 17189376.1‐1466:vTitle: Botulinum Toxin Dosage Regimen for Chronic Migraine Prophylaxis.

AUTHOR CONTRIBUTIONS

Study concept and design: Carrie R. Houts, James S. McGinley, Tracy K. Nishida, Dawn C. Buse, R. J. Wirth, David W. Dodick, Peter J. Goadsby, and Richard B. Lipton.Acquisition of data: Carrie R. Houts, James S. McGinley, Tracy K. Nishida, and Dawn C. Buse. Analysis and interpretation of data: Carrie R. Houts and James S. McGinley. Drafting of the manuscript: Carrie R. Houts, James S. McGinley, Tracy K. Nishida, Dawn C. Buse, R. J. Wirth, David W. Dodick, Peter J. Goadsby, and Richard B. Lipton. Revising it for intellectual content: Carrie R. Houts, James S. McGinley, Tracy K. Nishida, Dawn C. Buse, R. J. Wirth, David W. Dodick, Peter J. Goadsby, and Richard B. Lipton. Final approval of the completed manuscript: Carrie R. Houts, James S. McGinley, Tracy K. Nishida, Dawn C. Buse, R. J. Wirth, David W. Dodick, Peter J. Goadsby, and Richard B. Lipton.

Supporting information

Supplementary Material