Understanding the challenge of comparative effectiveness research in focal epilepsy: A review of network meta‐analyses and real‐world evidence on antiepileptic drugs

Abstract

Objective

Head‐to‐head randomized controlled trials (RCTs) are the gold standard for assessing comparative treatment effects. In the absence of direct comparisons between all possible antiepileptic drugs (AEDs), however, clinical decision‐making in focal (partial onset) epilepsy relies on alternative evidence borne from indirect comparisons including network meta‐analyses (NMAs) and from real‐world evidence (RWE) studies. We review NMAs and observational RWE studies comparing AEDs in the adjunctive setting to compare the robustness of these methods and to formulate recommendations for future evidence development.

Methods

A literature review identified NMAs and RWE studies comparing AEDs for the adjunctive treatment of focal seizures published between January 2008 and October 2018. NMAs were evaluated for robustness using a framework based on guidelines from the National Institute for Health and Care Excellence Decision Support Unit and the International Society for Pharmacoeconomics and Outcomes Research. RWE studies were evaluated using the GRACE checklist.

Results

From a total of 1993 records, 11 NMAs and six RWE studies were eligible. Key limitations identified in the NMAs include nonsystematic selection of RCTs, unexplored heterogeneity between included RCTs in terms of study and patient characteristics, and selection of AEDs and AED doses or dosing strategies that are not reflective of clinical practice. The main limitations of RWE studies concern sample size, design, and analysis methods. Approximately 90% of comparisons between individual AEDs were nonsignificant in the NMAs. None of the RWE studies adjusted for baseline differences between comparator groups; therefore, they lack the validity to make comparative conclusions.

Significance

Current NMAs and RWE studies provide only nominal comparative evidence for AED treatments in focal epilepsy, and should be used with caution for decision‐making due to their methodological limitations. To overcome these hurdles, adherence to methodological guidelines and concerted efforts to collect relevant outcome data in the real world are needed.

Key Points.

• NMAs in focal epilepsy have mostly shown no statistically significant differences between AEDs for adjunctive therapy, but are methodologically limited

• Key limitations of these NMAs include nonsystematic selection of studies and between‐study heterogeneity

• RWE could complement RCT‐based evidence, but in focal epilepsy there have been few well‐performed RWE studies

1. INTRODUCTION

Drug efficacy and safety comparisons are required for health technology assessment (HTA), for example, by the National Institute for Health and Care Excellence (NICE) ¹ and the Institute for Quality and Efficiency in Health Care, ² by payers, and in some cases, by regulatory authorities. ³ Comparative evidence is also crucial for the development of treatment guidelines that support clinical decision‐making, such as guidelines from the American Academy of Neurology. ⁴

There are different approaches to generating comparative evidence, resulting in different levels of evidence; randomized controlled trials (RCTs) designed for registration purposes are often considered the gold standard. However, head‐to‐head RCTs directly comparing antiepileptic drugs (AEDs) as adjunctive treatment of focal (partial onset) epilepsy are not mandated by regulatory bodies, and given the expense and uncertainty over whether statistically significant differences can be demonstrated against an active comparator, few of these study types have been conducted in this setting. ⁵ , ⁶ Additionally, RCTs, which have high internal validity, may incorrectly estimate the benefits of treatment in clinical practice, ⁷ due to a lack of external validity, which often arises from strict eligibility criteria resulting in unrepresentative patient samples. ⁶ , ⁸

By pooling studies that share at least one treatment arm, network meta‐analysis (NMA) allows comparisons of treatments that have not been directly compared in RCTs (Appendix S1, Figure S1). In light of this, NMAs are increasingly used to compare AEDs for the adjunctive treatment of focal epilepsy. ⁹ NMAs have the potential to inform prescribing by providing a ranking of AEDs by efficacy and tolerability. They are also required by several HTA agencies in reimbursement submissions where comparisons versus placebo do not help inform allocative decisions that may lead to the displacement of well‐established treatment options. However, in addition to limitations inherent to RCTs, NMAs have clinical and methodological limitations, such as the handling of heterogeneity, and their results need to be interpreted accordingly.

Randomized controlled trials and NMAs are often complemented with real‐world evidence (RWE) studies, which employ data from routine prescribing to assess and compare outcomes. As these studies often have broader patient selection criteria and longer follow‐up, they aim to be more representative of real life than RCTs. However, comparative RWE studies require well‐characterized patient groups to apply the statistical adjustments for confounding and bias needed to mitigate the absence of randomization and are therefore not always possible.

Building on previous assessments of NMAs of AEDs as adjunctive treatment of focal seizures, ⁹ , ¹⁰ we review the methodological quality and robustness of recent NMAs and comparative RWE studies in this field and compare the strengths and limitations of these study types to establish how they can support decision‑making and to formulate recommendations for future comparative research.

2. MATERIALS AND METHODS

2.1. Search strategy and eligibility criteria

A literature search was conducted for NMAs or RWE studies (defined in this review as any observational, nonrandomized study) comparing AEDs for the adjunctive treatment of focal seizures. Searches were conducted in MEDLINE and MEDLINE In‐process via the PubMed platform, and the Cochrane Database of Systematic Reviews and the Database of Abstracts and Reviews of Effects via the Cochrane Library on the Wiley Online platform. The search terms used to identify records for the review are provided in Appendix S1, Tables S1‐S3. The bibliographies of included articles were manually searched, and further searches by hand of PubMed and Google Scholar were conducted to identify additional articles of interest. Given that there has been active development of novel AEDs over the past decade, including the approval of four new AEDs for adjunctive therapy of focal seizures (brivaracetam, eslicarbazepine acetate, perampanel, and retigabine), this review focused on recent articles published between January 2008 and October 2018, to better reflect current treatment practices.

Articles were reviewed against predefined eligibility criteria by one reviewer, with any uncertainties verified by a second reviewer (Table 1). Initially, all articles were reviewed based on title and abstract. Where the applicability of the inclusion criteria was unclear, the article was included at this stage to avoid inappropriate exclusion of relevant studies. Full‐text articles of all records identified in the first round of screening were evaluated in more detail against the same eligibility criteria in a second screening round. In cases where the information reported was inadequate to ascertain eligibility, the article was excluded.

TABLE 1.

Eligibility criteria for the literature review

Domain	Description: NMAs	Description: RWE studies
Population	Patients with focal (partial onset) epilepsy requiring adjunctive therapy	Adult patients with focal (partial onset) seizures
Intervention and comparator	Studies should include comparisons between AEDs	Studies should include comparisons between any AEDs used as adjunctive therapy
Outcomes	No limitations on study outcomes were applied	Studies should include any efficacy or safety outcomes collected in a real‐world setting, including but not limited to: Change in seizure rate Seizure freedom Adverse events
Study design and data source	NMAs, indirect treatment comparisons or mixed treatment comparisons of randomized controlled trials were eligible	RWE studies, defined as any noninterventional study, including (but not limited to) prospective or retrospective cohort studies, case‐control studies, and case report forms, using data from sources including (but not limited to) health record databases, insurance claims databases, patient registries
Other considerations	Only publications with full texts or abstracts in the English language were included Only publications on human subjects were included Only publications published after January 1, 2008 were included

Abbreviations: AED, antiepileptic drug; NMA, network meta‐analysis; RWE, real‐world evidence.

2.2. Analysis of results

Included NMAs were evaluated using the criteria provided in Appendix S1, Table S4. These were based on guidelines from the NICE Decision Support Unit and the International Society for Pharmacoeconomics and Outcomes Research and cover limitations of the component RCTs and NMA methodology. ¹¹ , ¹² These particular guidelines were chosen as they are focused on assessing NMAs in the context of HTA and clinical decision‐making. ¹³ , ¹⁴ There are 18 criteria in five categories: NMA methodology, availability of RCT data, RCT design heterogeneity, heterogeneity of patient characteristics, and NMA results. The quality of RWE studies was rated using the GRACE checklist (Appendix S1, Table S6), which was published by the Academy of Managed Care Pharmacy to improve the interpretation of noninterventional comparative effectiveness studies for their use in decision‐making. ¹³ It comprises 11 items, split into data and methods.

3. RESULTS

3.1. Included NMAs and reported outcomes

Eleven NMAs were identified in the literature review (Figure 1 and Table 2). Nine of the NMAs that investigated efficacy defined it as a ≥50% reduction in seizure frequency (≥50% responder rate), and eight also investigated seizure freedom. Safety was primarily assessed as rates of overall withdrawal and withdrawal due to adverse events. Across the NMAs that performed pairwise comparisons, there were few significant differences between AEDs for efficacy or safety outcomes (~10% of all comparisons; Figure 2). ¹⁴ , ¹⁵ , ¹⁶ , ¹⁷ , ¹⁸ , ¹⁹ , ²⁰ , ²¹

FIGURE 1.

Article flow diagram. CDSR, Cochrane Database of Systematic Reviews; DARE, Database of Abstracts and Reviews of Effects; NMA, network meta‐analysis; RWE, real‐world evidence

TABLE 2.

Key characteristics of included NMAs

Analysis type	Study	Included trials, n	AEDs investigated	Patients per AED	Comparisons	Outcomes	Authors’ conclusions
MTC	Costa et al  2011 16	63 PBO‐controlled RCTs and 8 H2H trials	ESL, GBP, LCM, LEV, LTG, OXC, PGB, TGB, TPM, ZNS	ESL n = 845 GBP n = 693 LCM n = 733 LEV n = 988 LTG n = 685 OXC n = 659 PGB n = 1300 TGB n = 490 TPM n = 851 ZNS n = 500	Each AED vs PBO Each AED vs pool of all other AEDs LTG vs GBP, TPM, PGB, LEV	≥50% reduction in seizure rate Seizure freedom Withdrawals for any reason Withdrawal due to AE	No definitive conclusions were made regarding clinical superiority of new AEDs vs old AEDs
ITC	Martyn‐St James et al  2012 18	20 PBO‐controlled RCTs	ESL, LCM, PGB, RTG, TGB, ZNS	ESL n = 760 LCM n = 829 PGB n = 953 RTG n = 815 TGB n = 494 ZNS n = 380	Each AED vs PBO RTG vs each other AED	≥50% reduction in seizure rate Seizure freedom Withdrawals for any reason Withdrawal due to AE Specific AEs included: ataxia, dizziness, fatigue, nausea, and somnolence	Risk/benefit for RTG was similar to that for comparator AEDs
MTC	Bodalia et al 2013 14	40 PBO‐controlled RCTs and 3 H2H trials	GBP, LCM, LEV, LTG, OXC, PGB, VGB, VPA, TGB, TPM, ZNS	GBP n = 428 LCM n = 643 LEV n = 1041 LTG n = 695 OXC n = 519 PGB n = 1182 VGB n = 311 VPA n = 107 TGB n = 283 TPM n = 383 ZNS n = 451	Each AED vs PBO Each AED vs each other AED	≥50% reduction in seizure rate Withdrawal due to AE	VPA, LEV, and GBP demonstrated the best balance of efficacy and tolerability
ITC	Gao et al  2013 24	15 PBO‐controlled RCTs	BRV, CAR, ESL, LCM, PER, RTG	BRV n = 156 CAR n = 1526 ESL n = 844 LCM n = 970 PER n = 138 RTG n = 812	Each AED vs PBO Each AED vs pool of all other AEDs	≥50% reduction in seizure rate Seizure freedom Withdrawals for any reason AEs (dizziness, headache, fatigue, somnolence, nausea, and ataxia)	BRV, followed by RTG, might be more effective than all other newer AEDs
ITC	Khan et al  2013 17	12 PBO‐controlled RCTs	ESL, LCM, PER, RTG	ESL n = 748 LCM n = 970 PER n = 654 RTG n = 812	Each AED vs PBO Each AED vs each other AED	≥50% reduction in seizure rate Seizure freedom Withdrawals for any reason ≥50% reduction in secondary generalized seizures	When PER was compared with other AEDs, results were similar and not statistically significant
ITC	Zaccara et al 2013 25	8 PBO‐controlled RCTs	ESL, LCM, OXC	ESL n = 855 LCM n = 944 OXC n = 519	Each AED vs each other AED	Withdrawal due to AEs Specific AEs included dizziness, ataxia/abnormal coordination, and diplopia	Taking into account dose‐effect, OXC may be associated with more frequent neurological AEs than LCM and ESL
ITC	Brigo et al  2016 15	17 PBO‐controlled RCTs	BRV, LCM, ESL, PER	BRV n = 1358 LCM n = 970 ESL n = 1264 PER n = 1173	BRV vs each other AED	≥50% responder rate Seizure freedom rate Occurrence of AEs Withdrawal due to AEs	Indirect comparisons did not demonstrate significant differences in efficacy outcomes between BRV and LCM, ESL, or PER; however, they did indicate a possible favorable tolerability profile of BRV compared with ESL and potentially also PER
ITC	Zhu et al 2017 20	25 PBO‐controlled RCTs	BRV, LCM, ESL, LEV, PER	BRV n = 1360 LCM n = 1307 ESL n = 1235 LEV n = 948 PER n = 1140	LEV vs each other AED	≥50% responder rate Seizure freedom rate Occurrence of TEAEs Occurrence of SAEs Withdrawal due to AEs	ESL, LCM, and BRV are not inferior to LEV in efficacy, but PER may not be as efficacious at the highest recommended dose; BRV had similar tolerability to LEV, whereas ESL, LCM, and PER had a worse tolerability profile
ITC	Zhu et al 2018 22	19 PBO‐controlled RCTs	BRV, LCM, ESL, PER	BRV n = 1719 LCM n = 1307 ESL n = 1235 PER n = 1140	Each AED vs each other AED	≥50% responder rate Seizure freedom rate Occurrence of TEAEs Occurrence of SAEs Withdrawal due to AEs	No significant differences between the AEDs were found in efficacy, but BRV may have a better tolerability profile
MTC	Zhuo et al 2017 21	31 PBO‐controlled RCTs and 1 H2H RCT	ESL, LEV, RTG, TGB, PER, OXC, LTG, TPM, PGB, ZNS, GBP	ESL n = 465 LEV n = 663 RTG n = 360 TGB n = 77 PER n = 1038 OXC n = 764 LTG n = 141 TPM n = 482 PGB n = 271 ZNS n = 612 GBP n = 119	Each AED vs each other AED	≥50% responder rate Dizziness Somnolence	TPM appears to be the most efficacious AED, and LEV demonstrates balanced effectiveness and tolerability
MTC	Hu et al 2018 19	73 PBO‐controlled RCTs and 3 H2H RCTs	BRV, CAR, ESL, LCM, LEV, RTG, TGB, PER, OXC, LTG, TPM, PGB, ZNS, VGB, VPA, GBP, RUF	BRV n = 803 CAR n = 786 ESL n = 1185 LCM n = 944 LEV n = 1964 a RTG n = 865 TGB n = 494 PER n = 1123 OXC n = 902 LTG n = 429 b TPM n = 655 PGB n = 1768 ZNS n = 678 VGB n = 479 c VPA n = 107 GBP n = 728 RUF n = 156	Each AED vs each other AED	Seizure freedom rate Withdrawal due to AEs	The newer AEDs—BRV, LEV, OXC, RTG, VGB, and TPM—were demonstrated to be as efficacious as the older AEDs (VPA) for the treatment of partial epilepsy, whereas OXC, RTG, and RUF had poorer tolerability; LEV showed the best efficacy and tolerability

Abbreviations: AE, adverse event; AED, antiepileptic drug; BRV, brivaracetam; CAR, carisbamate; ESL, eslicarbazepine; GBP, gabapentin; H2H, head‐to‐head; ITC, indirect treatment comparison; LCM, lacosamide; LEV, levetiracetam; LTG, lamotrigine; MTC, mixed treatment comparison; NMA, network meta‐analysis; OXC, oxcarbazepine; PBO, placebo; PER, perampanel; PGB, pregabalin; RCT, randomized controlled trial; RTG, retigabine; RUF, rufinamide; SAE, serious AE; TEAE, treatment‐emergent AE; TGB, tiagabine; TPM, topiramate; VGB, vigabatrin; VPA, sodium valproate; ZNS, zonisamide.

^aIncludes a crossover trial, ⁵¹ with patients switching between treatments and PBO. It is unclear how the authors of the NMA had arrived at the conclusion that LEV 1000 mg had 200 patients and LEV 2000 mg had 202 patients in their arms.

^bIncludes a crossover trial, ⁵² with patients switching between LTG and PBO. It is assumed that the “Placebo/LTG” arm (n = 26) and the “LTG/Placebo” (n = 30) arm were used as placebo and active control arms, respectively, in the NMA.

^cIncludes a crossover trial, ⁵³ with patients switching between VGB and PBO. It is assumed that the “Placebo/VGB” arm (n = 40) and “VGB/Placebo” arm (n = 40) were used as placebo and active control arms, respectively, in the NMA.

FIGURE 2.

Network meta‐analyses (NMAs) of antiepileptic drugs (AEDs) in focal epilepsy considering each pairwise comparison. AE, adverse event; BRV, brivaracetam; CAR, carisbamate; ESL, eslicarbazepine; GBP, gabapentin; LCM, lacosamide; LEV, levetiracetam; LTG, lamotrigine; OXC, oxcarbazepine; PER, perampanel; PGB, pregabalin; RTG, retigabine; RUF, rufinamide; TGB, tiagabine; TPM, topiramate; VGB, vigabatrin; VPA, sodium valproate; ZNS, zonisamide. Each dot represents a pairwise comparison that has been made between two different AEDs. The size of the dot represents the number of NMAs making that particular comparison. Unfilled and filled dots represent nonsignificant and significant differences, respectively. Partially filled dots indicate that some studies reported significant differences while others reported no significant differences. Significant differences between AEDs are reported in these figures based on published NMAs, and should be interpreted with caution bearing in mind the various limitations highlighted in this article. 9 NMAs reported pairwise comparisons for ≥50% responder rates, 8 reported seizure freedom rates, 4 reported rates of all withdrawals, and 8 reported rates of withdrawals due to adverse events

3.2. Quality of NMAs

Various limitations in the data included in the NMAs, as well as the methodology used to conduct them, were identified. These limitations are discussed in turn in the following sections; however, it should be noted that some limitations, such as the nonsystematic inclusion or exclusion of RCTs from NMAs or choice of appropriate statistical model, are expected to have a greater impact than others, such as use of data from single trials for some AEDs.

3.2.1. NMA methodology

When NMAs are conducted to inform clinical decision‐making, selection criteria should be chosen such that included studies represent clinical practice as far as possible. ¹² Moreover, it is important that selection criteria are adhered to, as nonsystematic inclusion or exclusion of studies can bias the direction of results. ¹² Relevant studies appear to have been erroneously excluded from seven of the NMAs from this review, whereas in four NMAs some RCTs were included despite not meeting the eligibility criteria (Appendix S1, Table S5). None of the studies acknowledged these discrepancies. Furthermore, six NMAs did not include all treatment options available for adjunctive therapy of focal epilepsy (Appendix S1, Table S5). In four cases, ¹⁷ , ¹⁸ , ²⁰ , ²² the authors focused only on newly approved AEDs despite more established AEDs being a key part of recommended treatment for the adjunctive therapy of focal seizures. ⁴ , ²³ The remaining two studies did not provide rationale for the AEDs selected for inclusion. ¹⁴ , ²¹

NMAs require the use of an appropriate model that accurately reflects, and is justified by, the data synthesized from the included RCTs. ¹¹ Commonly used models include random‑effects models, which attempt to accommodate unexplained heterogeneity as they assume that the true effect size may differ between included studies, and fixed‐effect models, which assume there is no variation in relative treatment effects across studies for a particular pairwise comparison. Inadequately justified model choices were seen in three NMAs; however, all of these opted for random‐effects models, which is the more conservative approach (Appendix S1, Table S5).

3.2.2. Availability of RCT data

Five of the NMAs evaluated AEDs based on data from single trials of brivaracetam, gabapentin, oxcarbazepine, retigabine, rufinamide, tiagabine, and valproate, resulting in a potentially imprecise estimated treatment effect (Appendix S1, Table S5). ¹⁴ , ¹⁹ , ²¹ , ²⁴ , ²⁵ Furthermore, although NMAs are known to be affected by bias against the publication of negative results in the scientific literature, only three of the NMAs covered in this review reported conducting a formal assessment of publication bias. ¹⁴ , ¹⁸ , ¹⁹

Certain RCT study characteristics are known to influence patient responses to AEDs and placebo (Appendix S1, Table S5). For example, placebo response tends to be higher in studies conducted more recently ¹⁰ and can vary across different regions. ²⁶ Furthermore, the use of last‐observation‐carried‐forward (LOCF), which is common in RCTs for focal epilepsy, is expected to artificially inflate seizure reduction and freedom rates. ⁶ Heterogeneity in study design between NMA RCTs could, therefore, mask true treatment effect differences or produce artificial ones. Within the NMAs identified in this analysis, there was substantial variation in the study year, location, and total duration of the titration, maintenance, and double‐blind periods of included RCTs.

3.2.3. RCT design heterogeneity

Decisions beyond appropriate model selection can affect how potential sources of heterogeneity are accounted for in an NMA. For example, pooling mixed doses into a single intervention increases the sample size but at the cost of increased risk of heterogeneity (n = 8/11 NMAs; Appendix S1, Table S5), an issue that is further compounded because some studies included unlicensed doses (n = 4/8 NMAs). Similarly, analyses may be restricted to the maintenance phase of RCTs, where AED doses usually remain fixed (n = 6/11 NMAs) to reduce the risk of heterogeneity, as opposed to using the full double‐blind period in which the variable‐dose titration period is also included (n = 5/11 NMAs), which is more reflective of clinical practice.

3.2.4. Heterogeneity of patient characteristics

Differences in patient characteristics between included studies may also impact the observed treatment effect (Appendix S1, Table S5). Pooling RCTs with patients of varying ages can be a source of heterogeneity, particularly when combining RCTs that include pediatric and adult patients (n = 10/11 NMAs; Appendix S1, Table S5). ¹⁴ , ¹⁵ , ¹⁶ , ¹⁷ , ¹⁸ , ²⁴ Current evidence suggests that high seizure frequency and longer epilepsy durations at trial baseline are associated with poorer AED responses. ²⁷ , ²⁸ , ²⁹ Furthermore, increasing epilepsy duration has also been associated with higher placebo response.30 A review of the included RCTs suggests there was substantial variation between studies in terms of baseline epilepsy duration and seizure frequency, number of lifetime AEDs, and number of concomitant AEDs taken during the study.

3.2.5. NMA results

With the tradeoffs involved in many of the methodological decisions taken when conducting an NMA, sensitivity or subgroup analyses should be conducted to explore their impact; these were absent from many of the NMAs (Appendix S1, Table S5). Three of the four NMAs with data from head‐to‐head studies tested for inconsistency as a potential measure of between‐study variation, and all three studies found no significant evidence of inconsistency. ¹⁴ , ¹⁶ , ¹⁹ Although this suggests there was little variation between studies across different AED comparisons in these networks, it does not account for potential heterogeneity between RCTs within a given AED comparison.

3.3. Included RWE studies

Six relevant RWE studies carried out between 2008 and 2018 were identified in the literature (Figure 1, Table 3). There was a high level of variation between the studies, both in terms of study design and outcomes measured, and the characteristics of included participants. For example, data were retrospectively collected from (electronic) medical records in some studies, and prospectively via a study‐specific case report form in others. Follow‐up ranged between 3 and 24 months.

TABLE 3.

Key characteristics of included real‐world evidence studies

Study	Study design	Follow‐up period	AEDs investigated	Patients per AED	Comparisons	Outcomes	Authors’ conclusions
Acar & Aras 2018 31	Single‐center, retrospective cohort	3 and 6 mo	LEV, LCM	LEV n = 30 LCM n = 28	LEV vs LCM at 3 and 6 mo Each AED at 3 and 6 mo vs baseline	Seizure frequency Adverse events (any)	Both LEV and LCM add‐on therapy result in a significant reduction in the mean number of seizures following 3 and 6 mo of treatment; there was no statistically significant difference between LEV and LCM
Brodie et al 2014 34	Single‐center, prospective audit study	Not reported; patients were kept under observation until one of the endpoints was reached	LCM, LEV, PGB, TPM, ZNS	LCM n = 160 LEV n = 136 PGB n = 135 TPM n = 135 ZNS n = 141	Each AED at study follow up vs baseline	≥50% reduction in seizure frequency rate a <50% reduction in seizure rate a Withdrawal from treatment (any reason) a Withdrawal from treatment (lack of efficacy) a Withdrawal from treatment (side effects) a	Good tolerability is a key factor in predicting the success of AED treatment; TPM, LEV, and LCM were better tolerated by patients in this study
Kurth et al 2017 35	Single‐center retrospective analysis	6 mo	LCM, PER	LCM n = 70 PER n = 70	LCM vs PER	Seizure freedom a ≥50% reduction in seizure rate Retention rate a	When compared with LCM, treatment with PER resulted in numerically higher responder and seizure freedom rates in clinical practice
Liguori et al 2018 32	Single‐center, case series	>3 mo	CBZ, LCM	CBZ n = 8 LCM n = 8	CBZ vs LCM Each AED at 3 mo vs baseline	Seizure freedom Seizure frequency (monthly) >75% reduction in seizure rate EpiTrack score	LCM is an efficacious AED and could have a lesser cognitive adverse effect profile than CBZ
Maschio et al 2017 33	Single‐center, prospective cohort and historical control group	6 mo	LCM, LEV	LCM n = 25 LEV n = 19	LCM vs LEV	Responder rate	LCM appears to be more effective than LEV, without impacting the mood or quality of life of patients
Viteva & Zahariev 2018 36	Single‐center, prospective cohort study	24 mo	GBP, LCM, LEV, LTG, OXC, PGB, RTG, TGB, TPM	GBP n = 18 LCM n = 12 LEV n = 135 LTG n = 73 OXC n = 82 PGB n = 47 RTG n = 6 TGB n = 43 TPM n = 120	Each AED vs each other AED	Reduction in seizure severity a	Newer AEDs have some similar characteristics regarding many aspects of their effectiveness; however, LEV improves seizure severity and frequency and has satisfactory tolerability; PGB improves seizure severity and frequency in patients with partial seizures but has low tolerability; OXC has good tolerability but is less efficacious

Abbreviations: AED, antiepileptic drug; CBZ, carbamazepine; GBP, gabapentin; LCM, lacosamide; LEV, levetiracetam; LTG, lamotrigine; OXC, oxcarbazepine; PER, perampanel; PGB, pregabalin; RTG, retigabine; TGB, tiagabine; TPM, topiramate; ZNS, zonisamide.

^aNo statistical analyses were reported.

All the included studies that investigated efficacy defined this as a reduction in seizure frequency; however, the exact definition used varied (Table 3). The majority of studies did not perform statistical analyses of pairwise comparisons; however, when performed, no statistical differences between AEDs in terms of efficacy were observed (Figure 3). ³¹ , ³² , ³³ For the comparisons with significance testing that overlapped with comparisons made in the NMAs, the findings were similar (Figure 3); both RWE studies and both NMAs comparing the effectiveness of levetiracetam and lacosamide found no significant differences.

FIGURE 3.

Real‐world evidence of antiepileptic drugs in focal epilepsy considering each pairwise comparison. CBZ, carbamazepine; LCM, lacosamide; LEV, levetiracetam; LTG, lamotrigine; OXC, oxcarbazepine; PER, perampanel; PGB, pregabalin; TPM, topiramate; ZNS, zonisamide. Number of RWE studies of AEDs in focal epilepsy considering each pairwise comparison available reporting seizure‐free rates (N = 3) or reduction in seizure rates (N = 5). Each dot represents a pairwise comparison that has been made between two different AEDs. The size of the dot represents the number of RWE studies making that particular comparison. Empty dots represent nonsignificant comparisons

Three included RWE studies reported safety data defined as adverse events or treatment‐emergent adverse events. ³¹ , ³⁴ , ³⁵ The most frequently reported adverse events included dizziness and fatigue. No pairwise comparisons between AEDs were performed, and it is therefore not possible to draw conclusions about their relative safety profiles.

3.4. Quality of RWE studies

The quality of the RWE studies identified in this review was generally poor, particularly for the items relating to study methods on the GRACE checklist. Assessments of the included studies are detailed in Appendix S1, Table S6.

3.4.1. Data

All studies appeared to make use of data sources containing exposure information and information on objectively assessed, disease‐relevant outcomes. However, sample sizes were low, with <100 patients in most treatment arms, and all studies were conducted at a single center. Therefore, it is unclear whether the patients included in each study were representative of the entire population. ³¹ , ³² , ³³ , ³⁴ , ³⁶ Additionally, many of the studies did not present detailed baseline characteristics and did not identify or provide the details of all potential confounders. ³¹ , ³³ , ³⁴ , ³⁵

3.4.2. Methods

There were several methodological limitations observed in the included studies. In some cases, patients in the different comparator groups were not recruited concurrently but at different times, ³³ , ³⁴ , ³⁵ and many studies did not statistically test baseline similarity between the comparator groups on identified confounders. ³² , ³⁴ , ³⁵ Moreover, when comparing outcomes between groups, none of the six studies adjusted for differences in potentially confounding baseline characteristics, possibly due to small sample sizes. None of the studies presented the results of sensitivity analyses.

4. DISCUSSION

4.1. Considerations around RCT and NMA data

Previous reviews have highlighted critical limitations of NMAs in epilepsy, ⁹ , ¹⁰ and it has been noted that NMAs should not play a large role in selecting AEDs for adjunctive therapy of focal epilepsy, largely because of issues with AED RCTs such as overly narrow eligibility criteria, short follow‐up durations, and use of fixed doses, which are carried through to the NMAs that use them. ³⁷ RCTs are typically viewed as the gold standard for establishing the efficacy of interventions, owing to tightly controlled internal validity that minimizes bias. However, this is achieved by enforcing strict eligibility criteria and treatment patterns, which may come at the cost of external validity, that is, applicability to real‐life clinical practice. For instance, populations studied in regulatory epilepsy RCTs tend to have more drug‐resistant epilepsy than is often observed in clinical practice. ³⁸ Additionally, the common use of fixed doses in RCTs may result in suboptimal outcomes in some patients compared to those that could have been obtained with individualized dosing in clinical practice. Conversely, studies with broader selection criteria and the use of flexible dosing can increase heterogeneity between trials, thus proving problematic in conducting like‐with‐like comparisons between AEDs in NMAs.

One of the main motives for conducting NMAs is to pool data from similar RCTs, increasing the overall sample size to improve precision around estimated treatment effects. However, for many AEDs there are few trials in the relevant patient population, and as there may be differences in doses considered between trials, these numbers may be smaller still for analyses comparing individual doses. ¹⁰ Consequently, confidence or credible intervals, particularly for secondary outcome measures, may be wide, meaning there is uncertainty over the true effect size, and a lower likelihood that statistically significant differences between AEDs can be identified. As seen in this review, most results from the NMAs were nonsignificant, a finding similar to what has been seen in the monotherapy setting. ³⁹

The endpoints that can be explored in NMAs are determined by which outcomes have been reported by their component trials. As the European Medicines Agency requires RCTs in epilepsy to report ≥50% responder rates, this is the main outcome in most RCTs. ⁴⁰ However, it has been argued that this is not a clinically meaningful outcome, because patients achieving a reduction in seizure frequency, but not seizure freedom, do not show an accompanying improvement in quality of life. ¹⁰ More meaningful outcomes such as seizure freedom are less widely reported, and studies are often not sufficiently powered to demonstrate statistically significant differences between AEDs given the relatively low rate of achievement. In addition, RCTs in epilepsy have only evaluated the efficacy of AEDs over the course of between 4 and 24 weeks. Because epilepsy is a fluctuating condition, the lack of longer‐term RCT data means that conclusions around long‐term efficacy from NMAs are limited. ⁴¹ , ⁴² On the other hand, NMAs of tolerability outcomes may be impacted by RCT limitations to a lesser degree than NMAs of efficacy outcomes. For example, tolerability outcomes are not heavily influenced by epilepsy duration or severity, or by the use of LOCF; thus, there may be more of a place for NMAs in assessing comparative tolerability than effectiveness of AEDs as adjunctive therapy in focal epilepsy. ²⁵

In selecting and analyzing RCTs via NMAs, limitations in the RCT data can be compensated for or amplified, depending on methodological choices. A major challenge for NMAs when combining the results of RCTs is the introduction of heterogeneity between the different RCTs. This is particularly challenging when individual RCTs include a highly selected, homogeneous population, but selection criteria between RCTs are different. Compared with study‐level meta‐analyses, individual patient‐level meta‐analyses may be better able to adjust for known sources of heterogeneity. However, these sorts of analyses are more resource intensive and require individual patient data to be available from the included trials. All of the NMAs identified in this review conducted study‐level analyses, and potential sources of heterogeneity, such as varying AED doses, differing numbers of concomitant AEDs, and interstudy differences in population age, trial duration, and severity of epilepsy, were not adequately assessed or accounted for in many of the NMAs identified. NMA quality can be improved by adhering to methodological standards. However, the limitations of the underlying data source (RCTs) will continue to hamper generalizability of results, an issue that will need to be addressed separately.

4.2. A complementary role for RWE

Real‐world data can come from a variety of sources, including patient registries, (electronic) health record databases, insurance claims databases, and study‐specific case report forms. The key difference with RCTs is that in most types of RWE study, patients are prescribed the AEDs of interest as part of routine clinical practice and are not randomized. Pragmatic trials bridge traditional RCTs and observational studies by attempting to assess patients who are representative of the real‐world patient population in a setting that resembles clinical practice and are therefore sometimes defined as RWE studies despite also randomizing patients to intervention arms. ⁴³ In this review, we considered only nonrandomized observational studies within the definition of RWE.

Beyond the obvious constraint that the AEDs of interest need to be available and used in clinical practice, evidence based on high‐quality real‐world data could avoid some of the challenges faced by NMAs; the possible follow‐up of patients is usually longer than in RCTs, sample sizes are less limited by recruitment and financial considerations, and no patients need to be excluded due to the risk associated with randomization to a certain AED. ⁴⁴ Therefore, it is easier to select a broader and thus more representative group of patients. Furthermore, certain outcomes, such as long‐term retention, dosing, adverse events, quality of life, and health care resource use and cost, are more relevant to assess in the real world than within the constraints of an RCT. ⁴⁵ , ⁴⁶

Nevertheless, RWE studies are also subject to various methodological constraints. First, data need to be available to select and characterize the patients of interest, and to construct meaningful outcomes. For newly launched treatments, for example, there is a delay before sufficient data on their real‐world use can be collected. There is also wide disparity between the contents and coverage of different data sources, and there are currently no standards that can be applied across real‐world data sources, hampering comparability. ⁴⁷ Current guidelines for observational research are largely operational and do not deal with the crucial question of data availability. ⁴⁸ Some key outcomes, such as seizure frequency and reason for treatment change, are often not recorded in clinical practice in an analyzable way. ⁴⁶

Second, comparability between AEDs is dependent on there being a sufficient overlap of use in the same type of patients (severity, treatment line, age, sex, etc). The very basis of RWE methodology is how to select medically comparable treatment groups and adjust for remaining baseline differences between the comparator groups, to account for biases and confounding, and increase internal validity. Patient and physician bias, for example, resulting from prior beliefs held about a drug's safety or efficacy profile, may influence which AEDs are selected as well as how the outcomes of treatment are judged. Although it may be possible to adjust for biases introduced by the lack of randomization, complete information about biasing factors might be unavailable; therefore, efforts should also be made to evaluate the sensitivity of RWE study results to unmeasured confounding. ⁴⁹ , ⁵⁰ Single‐center studies are, for example, more susceptible to such biases compared to larger, multicenter studies that involve a broader range of patients and physicians. The lack of or inability to control for biases and confounding can lead to incorrect conclusions and treatment recommendations. This sensitivity of RWE to prescribers’ AED selection is reminiscent of NMAs’ sensitivity to heterogeneity of patient selection in different RCTs.

In this review, we identified that there is a lack of good‐quality, relevant RWE studies that have made comparisons between AEDs for the adjunctive treatment of focal epilepsy. During review of the literature, the vast majority of articles were excluded, because they did not report on AEDs as adjunctive therapy for focal epilepsy or did not include a comparison between AEDs, reporting only on a single AED. Of the studies that were eligible for inclusion, only a minority conducted statistical analyses when comparing AEDs, with the results of these analyses not being statistically significant in all cases. Moreover, as none of the reviewed RWE studies adjusted their comparison between AEDs for confounding, comparative statements such as those made in the reviewed RWE studies are not only methodologically fraught, but misleading and possibly detrimental to prescriber decision‐making, highlighting a quality of reporting and reviewing issue. Therefore, there is currently a major evidence gap around the real‐world comparative safety and effectiveness of adjunctive AEDs for the treatment of focal seizures.

For RWE comparisons of adjunctive AEDs to become relevant to prescribing, regulatory, payer, and HTA decisions, hurdles need to be overcome. First, basic methodological standards, such as adjustment for confounding, need to be heeded in the execution of studies. Second, the peer review process should eliminate comparative statements based on objectively flawed studies. Third, sample sizes need to increase, to improve representativeness and generalizability to what is inherently a heterogeneous disease, and to be able to adjust for possible confounding between comparator arms. However, this currently comes at the cost of losing relevant outcome measures such as seizure freedom. Therefore, finally, for sufficient RWE data to become available, the measurement of relevant outcomes in the real world needs to improve, and these data should be made available for analysis. This applies particularly to seizure freedom, and additionally for side effects, quality of life, and other important considerations in the treatment of epilepsy.

4.3. Limitations of the paper

Limitations of this review include single‐reviewer screening and extraction of articles, restriction of searches to articles published from 2008 onward, and including only articles written in the English language. Furthermore, hand searches of proceedings from epilepsy meetings were not conducted, which may have led to studies presented at these meetings but not published in a peer‐reviewed journal being missed. Nevertheless, the NMAs and RWE studies discussed here are thought to be representative of the level of evidence available for the comparative efficacy and tolerability of AEDs for the adjunctive treatment of patients with focal epilepsy at the time the searches were conducted.

5. CONCLUSION

Data comparing AEDs for adjunctive treatment of focal epilepsy are available from NMAs and a limited number of RWE studies. However, the conclusions that can be drawn from these are considerably hampered by the size and quality of the studies. The emergence of several consortia and collaborations between major epilepsy centers, including EpiCARE and the Epilepsy Study Consortium, in the past decade promises an increasing role for RWE in complementing NMAs to shape clinical practice.

CONFLICT OF INTEREST

S.T., J.P., B.P., and S.B. were employees of UCB Pharma at the time this study was conducted. P.K. has served as a consultant for Abbott; has served on medical advisory boards for Alliance and Aquestive; was a consultant for Aquestive, Eisai, SK Life Sciences, Sunovion, and UCB Pharma; was a speaker for Aquestive, Eisai, Sunovion, and UCB Pharma; and has received research support from Lundbeck. M.B. has served as a paid consultant for Eisai, GlaxoSmithKline, SAGE, GWPharma, and UCB Pharma. S.S. is an employee of Costello Medical and has served as a paid consultant for UCB Pharma. We confirm that we have read the Journal's position on issues involved in ethical publication and affirm that this report is consistent with those guidelines.

Supporting information

Appendix S1

Thieffry S, Klein P, Baulac M, et al. Understanding the challenge of comparative effectiveness research in focal epilepsy: A review of network meta‐analyses and real‐world evidence on antiepileptic drugs. Epilepsia. 2020;61:595–609. 10.1111/epi.16476

DATA AVAILABILITY STATEMENT

All relevant data are within the article and its Supporting Information.