Variation in Endpoints in FDA Medication Approvals

Abstract

Background and Objective

To assess the characteristics and extent of variation of the endpoints used in trials supporting the US Food and Drug Administration (FDA) approval of medications treating migraine.

Methods

Using the Drugs@FDA online database, we identified novel prescription medications approved by the FDA between January 2001 and September 2022, for migraine with or without aura, for both acute and preventive treatment, and for episodic and chronic presentations. For each medication, we used the most recent FDA-approved labeling to identify indication, mechanism of action, mode of administration, manufacturer, approval year, number of pivotal trials, trial design, and primary endpoints.

Results

Sixteen FDA-approved medications for the acute or preventive treatment of migraine were supported by 45 pivotal trials. There were 5 primary endpoint types: (1) change in mean monthly migraine days from baseline; (2) change in mean monthly migraine attacks from baseline; (3) change in mean monthly headache days from baseline; (4) mild to no pain After 2 hours; (5) pain free at 2 hours. There were 3 combinations of coprimary endpoints: (1) Headache Pain Free at 2 Hours and Most Bothersome Symptom Free at 2 Hours; (2) Pain Free at 2 Hours and Sustained Pain Free from 2-24 Hours Postdose; (3) Pain Free at 2 Hours and 2–24 Hours Sustained Pain Free and 2-Hour Pain Relief. Of the 8 preventive migraine medications, the timing of endpoint measurement included the full double-blind period, segments of the double-blind period, and the final month of the double-blind period.

Discussion

Migraine medication trial endpoints were inconsistent within the same indication (episodic or chronic), mechanistic class, and route of administration, frustrating direct comparison among these medications. Furthermore, inconsistent definitions for the indications “episodic” and “chronic” migraine were also observed. Consistent endpoint selection for medications approved for preventive and acute migraine treatment would enhance the ability of patients, physicians, and payers to make informed choices among these medications.

Introduction

Medication approval in the United States requires evidence of efficacy and safety¹ based on clinical trials with endpoints chosen by manufacturers that may also be based on consultation with the US Food and Drug Administration (FDA) officials.² A trial's endpoints are selected based on factors such as clinical relevance, time required for measurement, and ability of the endpoint to differentiate the new product from older products and are tailored to each disease and indication.³ Endpoint variation can help clarify different types of medication benefits but also frustrates direct comparison and can imply advantages that have not been clearly established. Use of consistent endpoints can thus be essential to informed patient decision-making and can also help payers structure formularies to discourage the use of high-cost medications offering similar benefits.

Since 2001, the FDA has approved several medications for the acute and preventive treatment of migraine. To facilitate appropriate medication selection, patients, clinicians, and payers need information about the expected effect of each medication and how each compares with lower-cost generic treatments. Because comparisons cannot easily be made without consistent endpoints, we evaluated the variation in trial endpoints used for FDA approval of pharmacologic interventions for migraine.

Methods

Standard Protocol Approvals, Registrations, and Patient Consents

This review was reported according to the STROBE reporting guidelines. No human or animal experimentation was conducted, so no institutional approval, patient consent, or state registration was necessary.

Study Design and Population

Using the Drugs@FDA online database, we conducted this cohort study by identifying all prescription and over-the-counter medications approved by the FDA between January 1, 2001, and September 1, 2022, for migraine with or without aura, for both acute and preventive treatment, and for episodic (<15 migraine or headache days per month) and chronic (≥15 headache days per month, at least 8 of which are migraine) migraine.^4,5 We excluded dietary supplements, medical devices, analgesics, antiemetics,⁶ and products if the active ingredients were already approved in another migraine medication before 2001 (e.g., Depakote ER).^7,8 Clinical trials intended for other primary headache disorders were also excluded (e.g., galcanezumab cluster headache trial).⁹ We cross-checked our list with published literature⁵ and medication databases such as Drugs.com and RxList.com to ensure inclusion of relevant medications.

For each medication, we used the most recent FDA-approved labeling to identify indication, mechanism of action, mode of administration, manufacturer, approval year, number of pivotal trials, trial design, and primary endpoints. Secondary endpoints were not considered. We limited our inclusion of pivotal trials to only those that were discussed in Section 14 of drug labels, and we did not include additional studies that may be listed in other FDA approval documents. FDA clinical review documents and ClinicalTrials.gov were used to provide any missing information.¹⁰ Endpoints were classified based on type (e.g., “Mild to No Pain After 2 Hours”) and timing (e.g., 12-hour vs 24-hour measurement).

Data Availability

No patient data or related documents are shared in this study because no individual patient data were collected.

Results

Sixteen medications were approved for migraine based on 45 pivotal trials between 2001 and 2022 (Table 1). Of the 8 preventive mediations (group 1), 3 were indicated for only episodic migraine, 1 for only chronic migraine, and 4 for both indications. Of the 9 acute medications, none explicitly distinguished between episodic or chronic indications, but trial inclusion criteria for all 9 drugs effectively limited trial participants to only those with episodic migraine (i.e., excluded patients with more than 8 migraine d/mo). The number of pivotal trials per medication ranged from 1 to 8 (median: 2). All pivotal trials were randomized, double-blind, and placebo-controlled, and several had study extensions. The medications belonged to 7 mechanistic classes, the most common being anticalcitonin gene-related peptide monoclonal antibodies (“anti-CGRP monoclonal antibodies”) (4 medications) and selective serotonin receptor (5-HT1B/1D) agonists (“triptans”) (3 medications). There were also 3 CGRP receptor antagonists (“gepants”), 3 nonsteroidal anti-inflammatory drugs (“NSAIDs”), 1 serotonin 5-HT1F agonist (“ditan”), 1 SNARE neuromuscular transmission inhibitor (onabotulinumtoxinA), and 1 medication with multiple mechanisms (topiramate). Three of these classes are newer: (1) anticalcitonin gene-related peptide monoclonal antibodies (“anti-CGRP monoclonal antibodies”); (2) “gepants,” or calcitonin gene-related peptide (CGRP) receptor antagonists; and (3) “ditans,” or serotonin 5-HT1F agonists; the remaining medications generally belonged to 4 older classes: (4) “triptans,” or selective serotonin receptor (5-HT1B/1D) agonists; (5) NSAIDs, (6) one SNARE neuromuscular transmission inhibitor (onabotulinumtoxinA), and (7) one medication with multiple mechanisms (topiramate). Routes of administration included subcutaneous (e.g., monthly [erenumab, galcanezumab, fremanezumab] or quarterly [fremanezumab] injection with anti-CGRP antibodies), intravenous (quarterly infusions of eptinezumab), intramuscular (quarterly injections of onabotulinumtoxinA), and oral administration (tablets of NSAIDs, triptans, gepants, ditans, and topiramate).

Table 1.

Medications + Pivotal Trial Endpoints FDA-Approved 2001–2022

Of 45 pivotal trials, 17 addressed medications for migraine prevention (CGRP monoclonal antibodies, topiramate, onabotulinumtoxinA, atogepant, and one of 2 rimegepant studies) and 28 addressed medications for acute treatment (triptans, NSAIDs, lasmiditan, ubrogepant, and one of 2 rimegepant studies).¹¹

Primary Endpoint Types

As shown in Table 1, the 45 pivotal trials used 5 single primary endpoint and 3 coprimary groups of endpoints. The 16 drugs can be grouped into preventive medications (8 medications) or acute medications (9 medications), as rimegepant is approved as both a preventive and an acute medication. Group 1 (Table 2) trials assessed preventive medications and were based on endpoints related to mean monthly changes from baseline, including Change in Mean Monthly Migraine Days from Baseline (6 medications, 12 trials), Change in Mean Monthly Migraine Attacks from Baseline (topiramate, 2 trials), and Change in Mean Monthly Headache Days from Baseline (2 medications, 3 trials). One medication, fremanezumab, used both Change in Mean Monthly Migraine Days from Baseline (episodic migraine trial) and Change in Mean Monthly Headache Days from Baseline (chronic migraine trial). Generally, preventive trials published in the past 5 years were adherent to the applicable iteration of International Headache Society (IHS) trial design guidelines with respect to the primary endpoint selection, but not universally so (i.e., fremanezumab chronic migraine trial), and older trials were generally nonadherent (see Table 1).

Table 2.

Endpoint Timing Differences Among Drugs Using Change in Mean Monthly Outcome From Baseline Endpoint (Group 1)

Group 2 (Table 3) trials assessed acute medications and were based on endpoints related to short-term changes, including Mild to No Pain After 2 Hours (3 medications, 16 trials), Pain Free at 2 Hours (sumatriptan/naproxen, 1 trial), as well as the 3 groups of coprimary endpoints: Headache Pain Free at 2 Hours and Most Bothersome Symptom Free at 2 Hours (4 medications, 7 trials); Pain Free at 2 Hours and Sustained Pain Free From 2-24 Hours Post-dose (sumatriptan/naproxen, 2 trials); Pain Free at 2 Hours and 2–24 Hours Sustained Pain Free and 2-Hour Pain Relief (diclofenac potassium, 2 trials).

Table 3.

Acute Treatment Studies (Group 2)

All group 2 trials, those for medications indicated for acute treatment, did not explicitly distinguish episodic from chronic migraine, although all of them excluded patients with more than 8 migraine attacks per month, effectively making these episodic migraine trials. Group 1 trials, for medications indicated for prevention, included trials addressing episodic migraine (11 trials) and chronic migraine (6 trials).

The grouping of trials according to the primary endpoint generally followed a historical pattern based on commonly used endpoints during particular eras, as reflected in mechanistic classes, but this was not consistently the case. As an example, the triptan trials were not adherent with the applicable (i.e., second) iteration of IHS trial design guidelines for primary endpoint selection (primary endpoint was mild/no pain after 2 hours vs recommended endpoint of pain free after 2 hours).¹² By contrast, the newer acute intervention trials were all adherent with both the 2018 FDA guidance document on the design of acute migraine trials and with the applicable IHS guidelines (see Table 1).¹³ Overall, the anti-CGRP monoclonal antibodies used the endpoint of Change in Mean Monthly Migraine Days, defined similarly across trials, but the fremanezumab chronic migraine trial used Change in Mean Monthly Headache Days.

Primary Endpoint Timing

As shown in Table 2, among the group 1 (preventive) migraine medications, mean monthly outcomes were calculated differently between trials. Calculations were based on the entire treatment period (“full period”) for 9 trials, a portion of the treatment period (“partial period”) for 5 trials, or the final month of treatment (“final month”) for 3 trials (Table 2). Five of the full period trials lasted 12 weeks while the remaining 4 lasted 24 weeks. Two of the partial period trials based their mean outcome calculations on treatment weeks 1–12 (of 24 total treatment weeks), 2 on weeks 9–26 (topiramate), and 1 on weeks 16–24 (erenumab). Of the final month analysis group, 2 trials based calculations on weeks 8–12 (erenumab) and 1 on weeks 9–12 (rimegepant). Acute treatments were not considered here as all used a 2-hour endpoint (groups 2 and 3).

Primary endpoint timing of group 1 varied among pivotal trials indicated for episodic and chronic migraine. Among the episodic migraine pivotal trials, 5 involved full period results analysis, 4 were partial, and 2 were final month. Among the chronic migraine pivotal trials, 4 involved full period results collection, 1 was partial, and 1 was final month. Further, “episodic” and “chronic” were defined using different eligibility criteria across trials (Table 2). Route of administration also varied among full, partial, and final month periods.

Discussion

The pivotal trials supporting FDA approval of migraine medications exhibited important variation in endpoint characteristics for the type of response evaluated and, in the case of medications indicated for preventive treatment, in the period over which the response was measured. This lack of consistency was observed among medications with the same indication (episodic/chronic), the same mechanistic classes, and within the different endpoint timing groups (full, partial, and final month).

Although this is the first study to our knowledge that addresses variability in the use of endpoints in pivotal clinical trials for neurology drugs, other research on the nature of endpoints used in pivotal trials for new medications has revealed similar trends.^14-18 For example, reviews of oncology medication approvals have identified substantial variability in the endpoints used to support FDA approval.¹⁹ There are also inconsistencies in the endpoints used in adult and pediatric trials of the same medication and indication.²⁰

Apart from differences in endpoint types and in timing of outcome collection, there were inconsistencies in language that yielded further variation.²¹ First, evaluating mean monthly migraine days vs headache days (group 1) allows for inconsistent data analysis given that qualifying a headache as a migraine attack relies on stringent criteria relating to pain severity, associated symptoms, attack duration, and treatment. Second, the pivotal preventive trials also varied between using migraine/headache days vs migraine/headache attacks; the discrepancy results in variable durations of attacks eligible for the primary outcome. Among the medications indicated for acute treatment, the endpoints were either Mild to No Pain After 2 Hours, Headache Pain Free at 2 Hours and Most Bothersome Symptom Free at 2 Hours, Pain Free at 2 Hours and Sustained Pain Free from 2-24 Hours Postdose, or Pain Free at 2 Hours and 2–24 Hours Sustained Pain Free and 2-Hour Pain Relief. These endpoints may not be directly comparable because a medication that reduces but does not eliminate pain might score well on a mild-to-no pain measurement, but not on a pain-free endpoint. Using coprimary endpoints may also preclude useful comparison with medications that use only one endpoint.^17,22

The distinction between episodic and chronic indications, as used in several of these trial designs, establishes a health care ecosystem hospitable to the selection of differing endpoints.^23,24 This is problematic in the context of the current definitions of episodic and chronic migraine having unclear clinical value in distinguishing disease burden among patients,²⁵ as well as ongoing debates about how to best define chronic migraine. Comparison is difficult because medications currently approved with only a chronic migraine indication may also be effective against episodic migraine and vice versa. This frequency distinction also creates the opportunity for selective reporting.²⁶ There is also variability between trials in defining the eligibility criteria for participants with episodic migraine (e.g., lower limit of headache frequency 3 vs 4 d/mo) and for participants with chronic migraine (e.g., some trials use an upper limit for headache frequency and others not) (Table 2). Finally, some trials do not explicitly specify episodic or chronic; however, although this was observed in the acute trials, all of them did effectively exclude patients with chronic migraine by requiring fewer than 8 migraine attacks per month to be enrolled in the study. This is partly a reflection of historical changes in the definition of chronic migraine as reflected by updates to the International Classification of Headache Disorders; the formal definition of chronic migraine was not introduced until 2004, in the ICHD-2, because chronic migraine was not defined in the ICHD-1.^25,27 In addition, the ICHD-2R and the ICHD-3 have made changes to the definition of chronic migraine in 2006 and 2018, respectively.^25,28 There is also minor variation in the definition of “month” (e.g., fremanezumab's month was 30 days rather than 4 weeks).

Variability in choice of endpoint may arise from a number of factors.^29-32 Manufacturers may prefer incongruous endpoints to differentiate newer from older products, which can help imply superiority or frustrate cost-effectiveness comparisons.^33-37 A medication's half-life or route of administration, or new evidence about the effect of a medication on the body, may suggest that outcomes should be measured during a particular time window.³⁰ It is possible that pharmacokinetics may have affected the time at which relevant values were collected.³⁰ However, among the different preventive medications with different pharmacokinetic profiles, there was no connection between the endpoint timing groups and the analyzed period of the treatment phase data (final month, partial, or full period). Other reasons for endpoint variability may include endpoint monitoring costs, attractiveness of the endpoint to potential trial participants, and idiosyncratic investigator or corporation preferences.^30,33

In addition, as the newer mechanistic classes for migraine medications were discovered over the 21-year study window of this investigation (i.e., CGRP monoclonal antibodies, gepants, and ditans), endpoint selection seems to have followed a historical trend based on changes in FDA and international guideline recommendations around endpoint selection for migraine trials. The IHS has published several guidelines for the design of clinical trials for migraine interventions that have led to improvements in achieving standardized trial design including primary endpoint selection.^12,23,38-40 In addition, in 2017, the FDA issued a draft guidance stating that for some diseases, there are 2 or more different features that are so critically important to the disease that a medication will not be considered effective without demonstration of a treatment effect on all of these disease features and that in such cases, coprimary endpoints should be used.² In line with this, an FDA guidance document for developing medications for acute migraine treatment, issued in 2018, recommended 2 specific coprimary endpoints: (1) reduction in pain and (2) reduction in most bothersome symptom (e.g., nausea, photophobia, and phonophobia).¹³ This FDA recommendation has been reflected in trials published for the latest generations of acute migraine medications (i.e., lasmiditan and ubrogepant, approved in 2019, and rimegepant, approved in 2020) and is in contrast with the primary endpoints published for the triptan trials, which were based on a single primary endpoint. Overall, our data show that compliance with IHS trial guidelines and FDA guidance has improved over time but is still not complete. Although modernizing outcome measures can improve understanding of a given drug's benefit and may be particularly important to the integration of patient-oriented outcomes, these benefits must be considered in light of the lack of consistency the changes create and their effect on appropriate selection from among a broad range of potential treatment options. At minimum, health care providers should be made aware of the change in outcomes over time and how this limits comparisons between drugs from different eras.

Business motivations may also play a role in endpoint selection, including the desire to minimize price competition by differentiating products from those of competitors.^41,42 The use of separate endpoints can make direct comparisons challenging even if products would perform similarly when tested using the same endpoint. The use of different endpoints can also be leveraged to imply an advantage over a competitor, even if the advantage has not been established in a head-to-head trial. For example, in 2013, onabotulinumtoxinA (Botox) was advertised as “the first and only preventive treatment proven to reduce headache days every month” for patients with chronic migraine, reflecting its pivotal trial endpoint Change in Average Monthly Headache Days from Baseline. However, divalproex (Depakote, Abbott Laboratories) was approved prior, for migraine prevention, without regard to the distinction between chronic or episodic migraine, based on mean reduction in 4-week rate of attacks of migraine.⁷ Although the onabotulinumtoxinA claim was technically true, it could be perceived to imply superiority over divalproex.

The benefits of diverse endpoints must be weighed against their drawbacks.^43-45 Difficulty in comparison can undermine physicians' and payers' efforts to identify optimal, cost-effective options and lead to needless use of newer medications when older or less expensive alternatives would perform equally well. With consistent endpoint use for medications of the same indication, there would be greater clarity as to which medications perform better, which should be offered to patients, and perhaps even which should or should not be FDA-approved (although superiority to existing treatments is not an approval criterion).⁴⁶

Ideally, efforts should be made to standardize migraine indication pivotal trial endpoints. In 2019, as part of its Patient Focused Drug Development efforts, the FDA awarded grant funding to the Migraine Clinical Outcome Assessment System to develop a publicly available core set of migraine endpoints that incorporates input from people living with migraine.^47,48 This project is still in development. Nevertheless, this initiative, the International Headache Society guidelines, and the 2017 and 2018 FDA guidance documents may eventually help reduce variation in migraine trial endpoints.^23,38,39

Further guidance about the selection of endpoints from regulatory agencies such as FDA—and a commitment to establish such guidance early, changing it infrequently and only when necessary—could help to ensure that, over time, medications will become more comparable with newer treatment alternatives.^2,13 The use of common endpoints for similar medications at the time of approval could reduce the need to later fund comparative effectiveness trials. This consideration becomes increasingly important as the number of medications in a therapeutic category increases, necessitating multiple pair-wise comparisons that pharmaceutical manufacturers have little incentive to conduct.⁴⁹

This study has several limitations. Primary endpoints are not the only trial characteristics that can prevent comparison between clinically similar medications; we did not consider differences in strength, dosing schedule, formulation, trial population, disease severity, disease history, or comorbidities of trial participants. In addition, other specifications related to trial design and patient data collection may not have been captured in the scope of this study (e.g., whether data were collected using paper vs e-diaries, whether a “month” is specifically defined as 4 weeks, whether imputation procedures were used for missing headache/migraine days, and whether there were concerns around multiple hypothesis testing and “cherry picking” primary endpoints).

Recent innovations in the treatment of migraine demonstrate how new medications for similar indications can be approved by the FDA based on different pivotal trial endpoints. The absence of common endpoints for investigational medications leads to confusion in the marketplace and makes it challenging to conduct cost-effectiveness comparisons once medications are approved. Efforts to standardize trial endpoints could improve comparability and, therefore, promote cost-effectiveness evaluations for the benefit of patients and the health care system.

Glossary

FDA

The US Food and Drug Administration

NSAID

nonsteroidal anti-inflammatory drugs

Appendix. Authors

Study Funding

The authors report no targeted funding.

Disclosure

L.K. Sharpless work was supported by the Harvard Student Employment Office Faculty Aide 2020 Summer Grant and the Harvard College Research Program HCRP AY20 Grant. S.L. Orr work is supported by the Canadian Institutes of Health Research, the Cumming School of Medicine Department of Pediatrics, and the Alberta Children Hospital Research Institute. A. Kesselheim and J. Darrow work is supported by Arnold Ventures. The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication. Go to Neurology.org/N for full disclosures.