Multiple treatment comparison in narcolepsy: a network meta-analysis

Abstract

Study Objectives

Randomized controlled trials (RCTs) that compared the safety and efficacy of medical treatments for narcolepsy were analyzed using network meta-analysis.

Methods

The RCTs in narcolepsy were searched. Network meta-analysis compared efficacy and safety of multiple treatments, multi-arm studies, and multi-criteria treatment decisions, based on a random model that assumed heterogeneity between studies, with corrections for multi-arm studies.

Results

Fourteen RCTs, three drug treatments, and six doses were identified: sodium oxybate (6 and 9 g/d), modafinil (between 200 and 400 mg/d), and pitolisant (up to 20 and up to 40 mg/d). Significant heterogeneity (>50%) between studies was found in 12/14 studies for almost all endpoints, but between-design consistency was present. For ESS and MWT, sodium oxybate 9 g/d, modafinil, and pitolisant up to 40 mg/d had similar efficacy. Pitolisant 40 mg/d and sodium oxybate 9 g/d in two nightly doses had similar efficacy in reducing cataplexy. A good safety profile characterized by a TEAE incidence risk ratio (IRR) <1.5 was found for all the compared treatments, except for sodium oxybate 9 g/d. Although no significant difference was found, Pitolisant 40 mg was shown with the best P scores for the benefit/risk (BR) ratio.

Conclusions

Modafinil (200–400 mg/d), sodium oxybate 9 g/d, and pitolisant up to 40 mg/d had similar efficacy in reducing excessive day time sleepiness. Only sodium oxybate 9 g/d and pitolisant up to 40 mg/d were shown with a comparable beneficial effect on cataplexy. Overall, Pitolisant was found with the best P score on the BR ratio

Clinical Trial Registration

PROSPERO 2017 CRD42017054686. Efficacy, safety, and benefit-risk comparison of alternative treatments in narcolepsy: a network multiple comparisons of treatment meta-analysis. http://www.crd.york.ac.uk/PROSPERO/display_record.php?ID=CRD42017054686.

Statement of Significance

Currently, few comparative results are available on medical treatment interventions and optimal drug doses for patients with symptoms of narcolepsy. This study used network meta-data analysis of 14 randomized controlled clinical trials as an innovative approach to compare the efficacy, assessed by excessive daytime sleepiness and cataplexy, safety, and the optimal benefit/risk ratio of multiple treatments for patients with narcolepsy.

Introduction

Narcolepsy is a chronic and disabling neurological disorder mainly characterized by excessive daytime sleepiness (EDS), cataplexy, and rapid eyes movement (REM) sleep disorders. As a consequence of marked EDS, patients may exhibit psychosocial distress, as many aspects of working, home, and social life are impacted [1, 2]. Also, narcolepsy is associated with a high risk of comorbidities [3, 4]. The International Classification of Sleep Disorders (ICSD-3) distinguishes type 1 narcolepsy (with cataplexy) from type 2 narcolepsy (without cataplexy) [5].

Current guidelines do not provide unequivocal recommendations on how to choose a first-line treatment based on the patient’s primary phenotype and the compared medical benefit of existing interventions [6, 7]. Level 1 evidence is based on randomized controlled trials (RCTs) and meta-analysis and was reached for some interventions, including modafinil on EDS [8], and sodium oxybate on EDS and cataplexy [9, 10].

Pitolisant, the first of a new histamine H3 receptor (H3R) class of pharmacological agents, was recently granted marketing authorization by the European Medicines Agency (EMA) for the treatment of narcolepsy, with or without cataplexy [11]. Other psychostimulants (methylphenidate, amphetamines) [12] or antidepressants [13] empirically used to treat cataplexy did not provide any evidence through RCTs and were eliminated.

The aim of the present study was to conduct a systematic review of the literature to identify RCTs that compared the safety and efficacy of medical treatments for narcolepsy, and to perform network meta-analysis, using the current Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines [14].

Materials and Methods

Protocol and registration

The protocol of the systematic review of the literature and meta-analysis protocol conformed to the current PRISMA guidelines (Figure 1) [14]. Before starting the statistical analysis, the statistical analysis plan was locked and registered in the Prospective Register of Systematic Reviews (PROSPERO) [15] from the UK National Health Service (NHS) National Institute for Health Research (NIHR) database.

Figure 1.

The PRISMA [14] flow diagram used in this study.

Eligibility criteria

Patients were adults with or without cataplexy irrespective of gender and age. An intervention was any treatment with results in at least one RCT and a drug that had marketing approval in narcolepsy indication. The comparison was made between the identified treatments and placebo (considered as control treatment). However, the comparison between any pair of treatments was also sought. Outcomes included efficacy on EDS, symptoms of cataplexy, and drug safety.

Information sources and literature search

All articles, books, and abstracts related to the efficacy and safety of drugs in narcolepsy were searched in the literature, irrespective of language, and cited references were checked manually. Electronic searches were performed in the following electronic databases: PubMed/MEDLINE, the Cochrane Central Register of Controlled Trials (CENTRAL, Cochrane Library), the Database of Abstracts of Reviews of Effects (DARE, Cochrane Library), the Cochrane Database of Systematic Reviews (CDSR, Cochrane Library), World Health Organization (WHO) International Trials Registry Platform (ICTRP) search portal, ClinicalTrials.gov, the US Food and Drug Administration (FDA) website, and the EMA website. Public information was also collected for sodium oxybate from the EMA website (scientific discussion, March 2007) [16].

For the literature search on the databases, the following keywords were used: “modafinil” or “armodafinil” and “narcolepsy,” “pitolisant,” “sodium oxybate,” or “gamma-hydroxybutyrate (GHB)” and “narcolepsy,” “modafinil,” and “narcolepsy,” “amphetamines,” and “narcolepsy.” The selection of published RCTs included the treatment of adult patients with narcolepsy. Once the first list of abstracts was retrieved and reviewed, each publication that met the study inclusion criteria was independently reviewed in full by two reviewers.

Study selection

RCTs were selected that provided data on at least one of the following selected outcomes for both efficacy and safety: the Epworth Excessive Sleepiness Score (ESS), the Maintenance of Wakefulness Test (MWT), number of cataplexy (CTP) attacks during the treatment exposure, and safety reporting of adverse events (AEs) during the treatment exposure.

Data collection

All data from publications were systematically reviewed. Each publication was evaluated by the two authors of this study.

Data items

For each study, data collected included the publication year, description of the design (randomization and concealment procedures), sample size, patient disposition, intent-to-treat selection, endpoints, ESS, MWT, cataplexy, and reported safety data. Observed heterogeneity among trials was reviewed and discussed.

Geometry of the network

The network evidence graph is a specific tool for network meta-analysis [17]. For each endpoint, each node in the network is associated with treatment. An overlap (edge) for any two treatments represents a direct comparison, the degree of the overlap weighted by the inverse of the standard error of the treatment effect [18].

Risk of bias in individual studies

The methodological quality of selected studies was evaluated using three validity domains (internal, external, and statistical validity). Domain-based evaluations (17 items) were performed by two investigators. Any discordance between the reviewers was discussed and resolved via consensus and summarized by an internal validity score (IVS) that included seven items, an external validity score (EVS) that included five items, and statistical validity score (SVS) that included five items. For studies with at least four inadequate items, a sensitivity analysis was conducted with and without these items and results considered reliable when the two selections provided the same conclusions.

Endpoints

EDS was measured by ESS and MWT. Cataplexy was reported as the weekly rate of cataplexy (WRC). To provide a unique main endpoint and to reduce type 1 multiplicity in the analysis, the ESS and MWT were combined into the EDS mean Z score, to define the narcolepsy score (NS) as the mean of EDS and WRC Z scores (ESS and WRC used minus their values such that larger values indicated patient improvement). The NS was the main endpoint. However, each endpoint was separately analyzed.

Safety was estimated by the incidence of reported treatment-emergent adverse events (TEAEs) and divided into three categories: (1) central nervous system events: nervousness, anxiety, confusion, dizziness, sleep disorders, psychiatric disorders; (2) gastro-intestinal AEs: nausea, dyspepsia, dry mouth, vomiting, diarrhea, anorexia, abdominal pain, gastro-intestinal pain, constipation; and (3) other AEs: weakness, fatigue, headaches, infection, pain, pyrexia, asthenia, and hypothermia. The main safety endpoint was the overall safety score (OSS) defined as the TEAE incidence rate during the exposure period.

The benefit/risk (BR) ratio was used as a measure of the overall medical benefit, or patient utility and attempts were made to combine the efficacy (the NS) and safety (the OSS). Depending on a linear correlation observed between NS and OSS, the unit-less BR ratio was defined as the residual value of the linear fit between NS and OSS, or the simple ratio NS/OSS.

Summary measures

Continuous variables, including ESS, MWT, and Z scores, were compared by using the weighted mean difference, except for endpoints with heterogeneous noncombinable units. For example, cataplexy was reported in the reviewed studies as various nonconvertible statistics, for which the standardized mean difference (SMD) was used. For safety assessment, TEAE rates were compared using the incidence risk ratio (IRR).

Synthesis of results

A random-effects model was assumed to be most likely, where differences might be expected among studies, but the fixed model was performed for sensitivity purposes. A network meta-analysis is appropriate for multiple comparisons [16–18]. For the expected multi-arm corrections, correlated pairwise comparisons in multi-arm studies were corrected by the weight reduction approach [19], equivalent to the standard regression approach (dimension of the design matrix reduced until it is invertible). For the assessment of model fit, the generalized Cochran Q_t [20, 21] was split into Q_d, measuring the inconsistency between the net estimates, Dt, based on a full design-by-treatment interaction random effects model [22], and the direct differences Dd and Q_h evaluating the heterogeneity across studies.

Treatment ranking by P scores measured the extent of certainty that any one treatment was better than another treatment, averaged over all competing treatments [23], equivalently with the surface under the cumulative ranking curve (SUCRA) defined as the rank of treatment within the range of treatments [24]. Finally, all results were compared with an alternative statistical model containing different assumptions [21]. The statistical analysis was performed using R statistical packages (version 3.2.4) and the meta-library, Netmeta [25].

Before optimization, the heterogeneity of reports needed conversions, and so median values and quartiles were converted into mean values using a heuristic approximation [26]. The estimate of nonreported SDs was based on the knowledge of the mean changes and the observed t-value or p-value. Final values were assimilated to mean changes by assuming that the correlation between baseline and final values was R ≅ 0.5. Values not reported in tables in the reviewed publications were estimated from graphics. Crossover and parallel results were appropriately mixed and corrected under considerations of carryover effect [27].

Risk of bias across studies

The assessment of publication bias was made using by funnel plots in this multiple treatment comparison, the comparison was conducted for each endpoint and each treatment, in particular, based on direct comparisons. As each study considered placebo as control, we use placebo as the unique control.

Results

Study selection

The literature search for published RCTs on modafinil and armodafinil in the treatment of narcolepsy identified 384 articles, out of which 30 trials were retrieved.

Modafinil and armodafinil are close compounds (modafinil is the racemic compound while armodafinil is the pure R-enantiomer). Both have shown identical pharmacological properties: they bind to the dopamine transporter and inhibit dopamine reuptake and have wake-promoting actions the same level of effect on EDS. Armodafinil studies were pooled with modafinil studies, however a comparison between the two groups was conducted to confirm the relevance of our choice.

Ten studies were ultimately selected, and the others were excluded due to non-RCTs (5), not assessing at least one endpoint of efficacy or safety (5), retrospective studies (7), or not involving patients with narcolepsy (3).

A literature search for sodium oxybate resulted in a total of 263 citations (including seven RCTs), of which 217 were excluded as non-RCTs (197), no efficacy or safety results (17), companion paper or secondary publication of an already included study (3), finally four nonredundant results on RCTs were found for oxybate, including three studies comparing at least two dosages with placebo. Nine citations on pitolisant were found, leading to the selection of three RCTs. Two studies assessed a new compound (JPZ110) [28]. However, this new compound did not have marketing approval and was not yet clinically available [28].

In total, 19 RCTs were assessed, 14 RCTs were considered to be eligible for network meta-analysis (Figure 1, Tables 1 and 3 for multitreatment comparison) and all of them had a placebo arm. Ten, four, and three studies compared modafinil, sodium oxybate, and pitolisant with placebo, respectively. Eight studies compared only one treatment with placebo, whereas the six other studies [29,36–38,40,41] compared multiple treatments, respectively (three or four treatments, Table 3). For the four studies on sodium oxybate, the two studied dosages 6 and 9 g/d were compared in three studies [29,36,37], whereas the low dose (6 g/d) was only compared with placebo in one study [35]. Three studies assessed Pitolisant: two for the 40 mg [40, 41] and one study for the 20 mg dose (still unpublished, results available [11]).

Table 1.

Publications included in the network meta-analysis, following review of published RCTs on drug treatment for narcolepsy

Study	Tested drugs	Design	Treatment duration	Sample size	Endpoints of interest	Comments
Scrima et al. [29]	GHB 50 mg kg/ night in 2 doses Placebo	DB-RCT 4-wk crossover	4 wks	N = 20	Stanford scale instead of ESS Cataplexy, sleep attacks Safety	Safety data poorly documented
Billiard et al. [30]	Modafinil 300 mg/d Placebo	RCT, 2-way 4-wk, X-over	4 wks, 2 wk placebo washout (WO)	N = 50	MWT, cataplexy, sleep attacks, inadvertent naps	No ESS. Safety not documented. Selected only for MWT and cataplexy.
Broughton et al. [31]	Modafinil 200 mg/d Modafinil 400 mg/d Placebo	Double-blind, crossover RCT 3 × 2 wks	3 × 2 wks No WO period	N = 75	MWT (primary endpoint), ESS, sleep attacks, inadvertent naps. Safety = AE.	Safety data poorly documented
US-MDF [32]	Modafinil 200 mg/d Modafinil 400 mg/d Placebo	DB-RCT 3 parallel groups	9 wks	N = 283 n = 92 placebo n = 96 MDF200 n = 95 MDF400	20 min MWT and CGI (primary endpoint) ESS MSLT Sleep attacks on daily basis Safety AE	No data on cataplexy
US-MDF [33]	Modafinil 200 mg/d Modafinil 400 mg/d Placebo	DB–RCT 3 parallel groups	9 wks	N = 271 n = 93 placebo n = 89 MDF200 n = 89 MDF400	ESS 20 min MWT, sleep attacks, inadvertent naps CGI Safety AE	No data on cataplexy
Moldofsky et al. [34]	Modafinil 300–500 mg/d Placebo	DB, placebo- controlled, 2 wks after 16-wk MDF open label (OL)	16 wks OL. 2 wks DB.	N = 63	40 min MWT ESS Daily number of cataplectic attacks Number of periods of severe sleepiness, voluntary sleep episodes (naps), and sleep attacks	Study assessing the treatment interruption and withdrawal symptoms after 16-wk OL. Safety not documented. Study selected only for efficacy
Harsch et al. [35]	Armodafinil 150 mg/d Armodafinil 250 mg/d Placebo	DB-RCT 3 parallel groups	12 wks	N = 196 n = 64 /ADF 150 n = 67 /ADF 250 n = 63 /placebo	20 min MWT (primary endpoint) ESS Cataplexy CGI Cognitive tests (CDR) Fatigue inventory Safety	Safety only most frequent AE (>5%)
X-US-2002 US Sodium oxybate multi-center study group [36]	Sodium oxybate treatment without titration to 9 g/d. Subsequently, all doses were titrated to 9 g/d. Placebo	DB-RCT With four parallel groups	4 wks	N = 136 n = 34/placebo n = 34 (3g) n = 33 (4g) n = 35 (9g)	Cataplexy on weekly basis (primary endpoint) ESS, CGI. Safety: treatment without titration to 9 g. Subsequently all doses were titrated to 9 g.	Patients included with weekly cataplexy attacks. Safety only most frequent AE (>5%).
X-INT-2005 Sodium oxybate Inter- national Study Group 2005 (OMC-SXB-15) N3 [37]	Sodium oxybate 4.5 g/d Sodium oxybate 6 g/d Sodium oxybate 9 g/d Placebo	DB-RCT 4 parallel groups	8 wks	N = 228 n = 59 /placebo n = 64 (4.5g) n = 58 (6g) n = 47 (9g)	40 min MWT (primary endpoint) ESS Sleep attacks CGI Safety	TT N = 246 Safety only most frequent AE (>5%).
Black et al. [38]	Placebo Modafinil 200–600 mg/d Sodium oxybate 6–9 g/d -X 6–9 g/d + Modafinil 200–600 mg/d	DB- RCT 4 parallel groups	8 wks	N = 222 n = 55 /placebo n = 63/ MDF n = 50 /x n = 54 / x +MDF	20 min MWT (primary endpoint) ESS CGI Sleep attacks Safety	No data on cataplexy
Saletu et al. [39]	Modafinil fixed titration at 3 weeks (200 mg/d W1, 300 mg/d W2, 400 mg/d W3) Placebo	DB-RCT Placebo- controlled crossover	3 wks 1 wkWO	N = 16 matched with 16 control HV	ESS MSLT, EEG AE	Safety data poorly documented
HARMONY I [40]	Pitolisant up to 40 mg/d Modafinil up to 400 mg/d. Placebo	DB-RCT 3 parallel groups	8 wks	N = 94 n = 31/ pitolisant n = 30/ placebo n = 33/ MDF	ESS (primary endpoint), % of responders, 20 min MWT Cataplexy and Sleep attacks CGI Safety AE
HARMONY I BIS [11]	Pitolisant up to 20 mg/d Modafinil up to 400 mg/d Placebo	DB-RCT three parallel groups	8 wks	N = 165 n = 67/ pitolisant n = 33/ placebo n = 65/ MDF	ESS (primary endpoint), % of responders 40 min MWT Cataplexy and Sleep attacks CGI Safety AE
HARMONY CTP [41]	Pitolisant up to 40 mg/d Placebo	DB-RCT two parallel groups	7 wks	N = 105 n = 54/ pitolisant n = 51/ placebo	Weekly rate of cataplexy (primary endpoint) ESS, % of responders 40 min MWT CGI, Patient Global Opinion Safety AE	Patients included with at least three cataplexy per wk

Table 3.

Comparison of treatments and studied endpoints within studies

Study	Placebo	MDF	SX6	SX9	P20	P40	ESS	MWT	CTP	AE
Scrima et al. [29]	*		*						+	+
Billiard et al. [27]	*	*						+	+
Broughton et al. [31]	*	*					+	+	+	+
US-MDF [32]	*	*					+	+		+
US-MDF [33]	*	*					+	+	+	+
Moldofsky et al. [34]	*	*					+	+
Saletu et al. [39]	*	*					+	+		+
Harsch et al. [35]	*	*					+	+	+	+
X-US-2002 [36]	*		*	*			+		+	+
X-INT-2005 [37]	*		*	*			+	+	+	+
Black et al. [38]	*	*	*	*			+	+		+
Dauvilliers et al. [40]	*	*				*	+	+	+	+
HARMONY I BIS [11]	*	*			*		+	+	+	+
Szakacs et al. [41]	*					*	+	+	+	+

MDF = modafinil; CTP = cataplexy rate.

Because EDS is a frequent complaint in sleep disorders, ESS was commonly used in these studies with large patient sample sizes, and has been shown to be both a consistent and sensitive evaluation method [42].

The findings of this study identified three treatments and six doses: sodium oxybate 6 g/d (hereafter called “SX6”) and 9 g/d (“SX9”), modafinil 200–400 mg/d (“modafinil”), and pitolisant up to 20 mg/d (“P20”) and up to 40 mg/d (“P40”). Table 2 summarizes the RCTs that were excluded from the study, which included publications by Laffont et al. [43], Boivin et al. [44], and Besset et al. [45], which contained insufficient data. In the sodium oxybate (SX)-US 2004 study [47], patients were drawn from a long-term study on sodium oxybate and were assigned to either sodium oxybate (same dosage) or placebo, to assess the effect of abrupt interruption of sodium oxybate. Lammers et al. [46] only reported cataplexy in a short 4-week crossover study without the use of the ESS or the MWT, and a lower dose of sodium oxybate was tested.

Table 2.

Publications excluded from the network meta-analysis following review of published RCTs on drug treatment for narcolepsy

Study	Tested drugs	Design	Treatment duration	Sample size	Endpoints	Comments
Laffont et al. (MOD 024) [43]	Modafinil 200 mg/d Placebo	DB-RCT crossover 2 × 2 wks	2 wks	N = 10	No data on ESS, MWT, cataplexy. No data on safety reported.	Not published, only as an abstract. No data on ESS, MWT, or cataplexy. Safety not documented.
Boivin et al. [44]	Modafinil 300 mg/d Placebo	DB-RCT 4-week crossover	4 weeks	N = 10	PSG, EMG (Periodic Leg Movement index) EDS on 10 points VAS (no ESS) Cognitive test (FCRTT) Daily number of sleep attacks	No data on ESS, MWT or cataplexy. Safety not documented.
Besset et al. [45]	Modafinil 300 mg/d Placebo	DB-RCT 4-wk crossover	4 wks	N = 16	Stanford scale instead of ESS. Attention Safety (poor data) PSG (REM).	No data on ESS, MWT or cataplexy. Safety data poorly documented.
Lammers et al. [46]	GHB 60 mg/ kg/night in two doses Placebo	DB-RCT 4-wk cross-over	4 wks	N = 24	Cataplexy on daily basis. Sleep attacks on daily basis. PSG (REM).	No data on ESS, MWT. Safety not documented. MSLT only on 7 patients.
X-US-2004 U.S. Sodium oxybate Multicenter Study Group (N2, OMC- SXB-2005) [47]	X at established dose from 3 to 9 g/d Placebo	DB-RCT placebo controlled	2 wks	N = 55 N = 26/x N = 29/ placebo	Cataplexy on weekly basis only	Study testing rebound effect after abrupt cessation of sodium oxybate. No data on ESS or MWT. Safety data poorly documented.

Risk of bias within studies

Methodological quality was evaluated for each trial (Supplementary Table 15). Almost all the trials were acceptable for internal validity, external validity, and statistical concerns. In the study by Moldofsky et al. [34], the two study arms were selected after a 16-week open label treatment with modafinil, potentially penalizing the placebo group. In the study by Black et al. [38], patients were all treated with modafinil at the established dose until randomization, and the abrupt withdrawal from modafinil potentially created an artificially worsened placebo group when treatment arms were changed. In this study, the highest doses of sodium oxybate were given without previous titration, unlike as in other trials, and this may have penalized the drug safety profile.

Study characteristics

All the studies evaluated were of short duration, from 2 to 12 weeks. All other long-term studies were conducted as open-label studies and did not provide comparative data. The design and main characteristics of the selected studies are presented in Table 1, and the number of treatments and measured endpoints for each study are shown in Table 3.

Results of individual studies

Modafinil approval was based on two main 9-week double-blind (DB) placebo-controlled studies [32, 33]. No differences in the efficacy on EDS was found between the two doses of modafinil used [32, 33]. No effect on cataplexy was reported in either trial [32, 33], which was also confirmed in a previously published meta-analysis [8].

For sodium oxybate, a 4-week crossover pilot study [29], and a 4-week DB parallel group RCT, testing three doses of 3, 6, and 9 g/d compared with placebo in 136 patients with narcolepsy (SX-US-2002) [36] provided evidence of a beneficial effect of sodium oxybate in cataplexy and EDS with the highest dose (9 g/d), and on ESS with a lower dose (6 g/d). The effect on EDS was shown in another study using a dose of sodium oxybate of 9 g/d [37]. Black et al. [38] compared sodium oxybate and modafinil, individually and in combination with placebo for EDS in patients treated with modafinil at the established dose at inclusion. Two previously published meta-analyses of sodium oxybate confirmed its efficacy in cataplexy, EDS, and sleep architecture abnormalities [9, 10].

P40 [40] with EDS as the primary endpoint was found to be superior to placebo and similar to modafinil in its effects on the reduction of cataplexy. A second study [41], conducted in patients with cataplexy confirmed the efficacy of P40 for reducing cataplexy (primary endpoint) and also EDS. As a third study, P20 did not reach significant efficacy differences compared with placebo and modafinil. Results of this study were made public by the European Medicine Agency [11].

Synthesis of results: network configuration

The set of direct comparisons between treatments (network geometry) is illustrated by the network evidence graph (Figure 2). At each side of this comparison, P20 and P40 were compared with placebo and modafinil, and at the other side, the two doses of sodium oxybate were tested with the same comparators. No direct comparison was conducted between sodium oxybate and pitolisant doses.

Figure 2.

Multiple comparison network analysis. The colors indicate the existence of multiple treatments tested in the same study or design, for example, P40 versus placebo and versus modafinil in the same trial.

Synthesis of results: efficacy assessment

Forest plots (Figure 3), P scores, and heterogeneity/consistency tests (Table 4) are described with all details available in Supplementary Tables 2.1 to 14.8. For ESS (12 studies), only three interventions reached a significant mean difference when compared with placebo: P40 (−3.05) (95% confidence interval [CI] = −5.24% to −0.85%) (p < .001), SX9 (−2.94) (95% CI = −5.04% to −0.85%) (p < .001), and modafinil (−2.37) (95% CI = −3.41% to −1.32%) (p < .001), without statistical differences between them. Homogeneity across studies (p = .16), and slight between-design inconsistency (p = .02) were found.

Figure 3.

Forest plots for efficacy and safety on point estimate and 95% CI of each studied treatment. Placebo is arbitrarily fixed at 0. ESS: mean change (scale units), compared as mean difference; larger reductions are associated with patient improvement. MWT: compared as mean differences (measured in mean changes in minutes) is directly associated with patient improvement. Cataplexy: mean reduction, calculated as standardized mean difference, but provided as the weekly rate of cataplexy (after calibration). NS: calculated as mean differences, measured in Z standardized units; higher Z values denote patient improvement, with Z > .5 denoting a clinically significant value. OSS: calculated as the IRR compared with placebo (ratio of the incidence of TEAEs during treatment exposure); the IRR should be as small as possible. BR ratio: calculated as the residual value of the regression of NS by OSS and measured in Z-standardized units. Black squares represent BR ratio associated to the Narcolepsy Z-score (including cataplexy and EDS). Triangles represent the BR ratio associated with the EDS Z score (including only EDS and not cataplexy).

Table 4.

Characteristics and tests for each analysis

	ESS	MWT	Cataplexya	NSb	OSSc	B/Rd	EDSe	B/Rf
No. of studies	12	12	8	14	13	13	13	12
N pairwise computations	22	20	16	24	23	23	23	22
I 2	0.51	0.73	0.32	0.57	0.81	0.58	0.53	0.6
Tests
Overall Q	0.01	<.001	0.17	<.001	<.001	<.001	0.01	<.001
Within Qh	0.16	<.001	0.51	0.01	<.001	<.001	0.01	<.001
Between Qi	0.04	0.6	0.09	0.09	0.04	0.19	0.24	0.32
P scores
Modafinil	0.73	0.49	0.27	0.57	0.45	0.50	0.70	0.72
P20	0.19	0.38	0.31	0.29	0.55	0.30	0.32	0.33
P40	0.80	0.83	0.88	0.85	0.56	0.87	0.89	0.90
Placebo	0.12	0.15	0.18	0.09	0.86	0.12	0.15	0.19
SX6	0.36	0.48	0.57	0.48	0.49	0.51	0.25	0.29
SX9	0.78	0.67	0.79	0.73	0.07	0.61	0.68	0.58

Number of studies, number of pairwise computations, heterogeneity index I², and tests (Overall Cochran Q statistics measuring distinctions between net estimates and observed differences, split into within-design Q_h, measuring heterogeneity between studies, and between-designs Q_i, incorporating the concept of design inconsistency) for the following analyses: ESS, MWT, cataplexy, narcolepsy Z score, safety, and benefit/risk ratio.

MDF = modafinil, CTP = cataplexy rate.

^aWeekly reduction of cataplexy rate (CTP).

^bNarcolepsy score.

^cOverall safety score.

^dBenefit/risk ratio, calculated as the residual of the linear fit of the NS by the OSS.

^eEDS Z score.

^fBenefit/risk ratio based on efficacy limited to EDS, and calculated as the residual of the linear fit of EDS Z score by OSS.

The MWT (12 studies) measured the mean changes in time (minutes) from baseline. There was significant heterogeneity across studies (p < .001), and no between-day design inconsistency (p=.601) was found. Significant relative benefits when compared with placebo were found for P40 (4.88 min) (95% CI = 0.57% to 9.20%) (p = .009) and modafinil (1.85 min) (95% CI = 0.16% to 3.55%) (p < .001).

Cataplexy was reported in eight studies, and the difference was calculated by SMD converted by linear calibration into decrease of weekly rate of cataplexies (DWRC). Significant reductions were observed for two treatments: P40 (SMD = −.52) (95% CI = −.90% to −.13%) (p < .001) (DWRC = −5.9), SX9 (SMD = −.41) (95% CI = −.79% to .032%) (p = .023) (DWRC = −5.2). No marked or significant heterogeneity across studies (p = .51) or between-design inconsistency (p = .09) were found.

For the EDS Z score, (combining ESS and MWT) significant differences were found when compared with placebo for P40 (.54) (95% CI = .14% to .95%) (p < .001), followed by modafinil (.36) (95% CI = .18% to .55%) (p < .001) and SX9 (.35) (95% CI = .02% to .68%) (p = .048). A significant heterogeneity across studies (p = .01) and no between-design inconsistency (p = .06) were found.

The P scores for EDS Z score and corresponding BR ratio (Table 4) were very similar with the following ranking: P40 (P score = .90), modafinil (P score = .72), SX9 (P score = .58), P20 (P score = .33), SX6 (P score = .29), and placebo (P score = .19).

For the NS, synthesizing both EDS and cataplexy, significant differences were found when compared with placebo for P40 (NS = .56) (95% CI = .13% to .99%) (p < .001), followed by SX9 (NS = .36) (95% CI = .03% to .69%) (p = .03) and modafinil (NS = .32) (95% CI = .13% to .51%) (p = .012). A significant heterogeneity across studies (p = .01) and no between-design inconsistency (p = .06) were found.

The P scores of NS and corresponding BR ratio were similar (Table 4) and summarized by the values of NS in decreasing order: P40 (P score = .85), SX9 (P score = .73), modafinil (P score = .57), SX6 (P score = .48), P20 (P score = .29), and placebo (P score = .09).

Synthesis of results: safety assessment

Thirteen studies provided results on safety. The main safety endpoint was the OSS and its subdivision into three categories of TEAEs during treatment exposure, Forest plots (Figure 3), P scores, and heterogeneity/consistency tests (Table 4), All details are in Supplementary Tables 6.1 to 8.9. Placebo was confirmed as the safest intervention, followed by four treatments with similar p-values, including P20, P40, SX6, and modafinil, characterized by an acceptable IRR <1.5. Only SX9 was characterized by a higher IRR compared with placebo (IRR = 3.86, 95% CI = 2.32% to 6.4%) (p < .001). Supportive analysis conducted on safety subgroups (including the symptoms of headache, central nervous system, and gastro-intestinal symptoms) provided similar results. Significant heterogeneity was found across studies (p < .001), and a slight, but nonstatistically significant, between-design inconsistency (p = .04) was found.

BR ratio estimate

A linear relationship between NS (efficacy) and OSS (safety) was found (Intercept = −.101) (95% CI = .21% to .01%), (p = .12) (slope = .20) (95% CI = .04% to .39%) (p = .007), providing evidence of a direct association between efficacy (NS) and safety (OSS). A BR ratio was calculated as the residual of the linear fit of NS by OSS. The best ratio was found for P40 (score of .53) (95% CI = .10% to .95%) (p < .001), followed by modafinil (score of .29) (95% CI = −.09% to .49%) (p = .036) and SX9 (score of .22) (95% CI = −.13% to .57%) (p = .121) (Figure 3 and Table 4) (Supplementary Tables 9.1 to 9.9). The ranking of P scores confirmed that P40 had the optimal/BR ratio (p = .87), followed by modafinil (p = .50) and SX9 (p = .61). A highly significant heterogeneity across studies (p < .001) and between-design consistency (p = .19) were observed.

As NS combines both EDS and cataplexy, the benefit risk was also calculated on efficacy based on EDS only (EDS Z score). Results are very similar to NS (Table 4 and Figure 3).

Finally on this aggregate endpoint, we provided a statistical justification to our choice of pooling together armodafinil and modafinil: the two results on modafinil with and without the only armodafinil Harsch study [35] provided virtually unchanged results (Supplementary Section 19).

Multiple comparisons between compared treatments

SX9, modafinil, and P40 were found as the best alternatives and did not significantly differ between each other. The 95% CI of the difference between each pair (Table 5) was calculated for ESS and the NS global score, based on the fixed model.

Table 5.

Multiple comparisons between treatments

Endpoint	Treatments	95% CI	p Value
ESS	P40–MDF	−1.93% to 1.03%	.275
	P40–SX9	−2.47% to 1.76%	.381
	MDf–SX9	−1.60 to 1.80	.460
NS	P40–MDF	.51 to −.11	.090
	P40–SX9	.52 to −.22	.191
	MDF–SX9	.19 to −.29	.351

Multiple comparisons between the three best treatments, P40, modafinil, and SX9, based on the fixed model. The upper limit of the 95% CI corresponds to the observed non-inferiority limit. p Values correspond to a one-tailed superiority test of the first treatment followed by the second one.

Additional analysis

A principal components analysis (PCA) was conducted by analyzing the results of all the studied endpoints (Figure 4). The percentage of explained variance was 53% and 35% for the first two factors, revealing two clusters of strongly correlated variables: the efficacy endpoints (cataplexy, ESS, and MWT), as opposed to the safety endpoints for the first factor, confirming the inverse correlation between efficacy and safety. The mutual position of the treatments provided an integrated image of each profile in this efficacy/safety mapping. Placebo position predicted the best safety and worst efficacy, whereas the findings for modafinil suggested a balanced profile for safety and efficacy, except for cataplexy. Sodium oxybate dosages suggested good efficacy, but safety concerns particularly for SX9. P40 had predicted efficacy comparable with SX9 and modafinil, with a better safety profile.

Figure 4.

PCA of compared efficacy and safety profiles. PCA was conducted on the three treatments described by nine components, resulting in a BR ratio. The percentage of explained variance was 53% and 35% for the first two factors, showing two clusters of strongly correlated variables: the efficacy endpoints included the weekly cataplexy rate (WCTP), the narcolepsy index (NI), the MWT, and the ESS. The three TEAE safety IRR for gastro-intestinal serious adverse events (SAVG), central nervous system serious adverse events (SAVC). Other AEs included SAVG, and all AEs, including all serious adverse events (SAVALL) are inversely correlated with these (as opposed to efficacy endpoints on the first axis). The BR ratio appears as a compromise between efficacy and safety. This efficacy/safety mapping allows comparison of the treatments by their mutual position: placebo (PCB) confirmed with the best safety and the worst efficacy, and modafinil (MDF) as a balanced profile for both efficacy and safety; efficacy for SX has a higher incidence of AE, particularly for SX9. P40 shows a similar efficacy compared with SX9, but with a better safety profile and the best BR ratio.

For sensitivity purposes, a re-analysis of the ESS endpoint was performed by excluding the Black et al. [38] study (Supplementary Tables 10.1 to 10.9), resulting in lower effects of SX9 compared with the main analysis. By excluding the Moldofsky study [34], the results were unchanged (Supplementary Tables 11.1 to 11.9).

Risk of bias across studies

The assessment of publication bias was assessed using funnel plots.

The examination of all the funnel plots for each endpoint and each drug in particular compared with placebo does not provide suspicion of an asymmetrical distribution of the points representing the studies (Supplementary Section 16).

Discussion

This network meta-analysis was performed according to the current PRISMA guidelines [14], and compared six interventions based on 14 RCTs, investigated four efficacy endpoints and three categories of adverse reactions, and derived a BR score in adult patients with narcolepsy.

The study methodology included the evaluation of EDS, which was evaluated by two complementary scales, the MWT, and the ESS. EDS is a major complaint in sleep disorders and is commonly evaluated using the ESS, which uses eight items to produce a maximal score of 24 [42]. Although the ESS is simple to use, and has shown internal consistency and sensitivity, it is often used to evaluate large sample sizes, which may limit its use in clinical practice to determine the improvement of a patient [42]. Cataplexy was measured by the weekly rate of attacks during the treatment period. The composite NS synthesized the efficacy of these two major symptoms of narcolepsy for most patients [48]. The NS provides a unique endpoint and statistical test and quantifies clinical relevance, and values exceeding Z = .5 are considered as clinically significant according to Cohen’s rule [49]. The NS was used as the main endpoint; however, each endpoint was separately analyzed.

The network evidence graph (Figure 2) highlights the comparison between placebo and modafinil as our central evidence axis. Sodium oxybate and pitolisant were both compared with placebo and modafinil, but not between each other. Methodological issues exist for comparing sodium oxybate in the context of RCTs, as unlike the other drug treatments, sodium oxybate induces deep sleep and has multiple contraindications, two characteristics that are hardly incompatible with blinding. Despite the lack of direct comparison between SX9 and pitolisant, a network meta-analysis provided a rigorous indirect comparison through the synthesis of comparisons P40 and placebo and SX9, and P40, modafinil and SX9. The consistency between the net estimates and the direct comparisons tested by the between-design generalized Cochran Q_d [20, 21] (Table 4) was demonstrated for nearly all the analysis.

Efficacy and safety are often separately assessed, although they should normally be integrated following a BR ratio, constituting the best estimate of overall clinical utility of an intervention. The findings of this study confirmed the direct association between safety, measured by the OSS and efficacy, measured by the NS, justifying the residual of the linear regression of NS by OSS as a measure of efficacy and safety.

In terms of clinical comparisons between drug and placebo in the treatment of adult narcolepsy, modafinil was the most extensively investigated intervention, and was characterized by a significant improvement in EDS, based both on ESS and MWT, and had no significant effect on cataplexy, and an acceptable safety profile (IRR = 1.5) (95% CI = 1% to 2.27%). SX9 provided evidence of efficacy on both EDS (ESS and MWT) and cataplexy, but showed a significant increase in the number of adverse reactions (IRR = 3.86) (95% CI = 2.32% to 6.40%). However, in some studies [36], sodium oxybate dosages were given without titration, unlike as in other trials, and this may have penalized the drug safety profile.

P40 improved both EDS and cataplexy accompanied by a mild safety profile (IRR = 1.34) (95% CI = .59% to 3.04%). No significant efficacy effect was found for the other interventions, including SX6 and P20.

In terms of between-treatment comparisons, the network analysis provided pairwise comparison even in the absence of direct comparisons. SX9, modafinil, and P40 were found to have significant clinical effects when compared with placebo, but were not statistically or clinically different when compared with each other. For ESS (Table 5), the lower and upper confidence limits of the differences were less than the minimum clinically relevant difference when the ESS was 3. Only noninferiority results can be concluded. For instance, the upper limit of the 95% CI for the difference for P40 and modafinil was 1.03. Therefore, a difference of 1 (≅1.03) might be considered as the noninferiority margin of P40 compared with modafinil. For the NS Z score (Table 5), some may interpret a difference as large as .5 as found between P40 with modafinil and SX9, respectively as some slight sign of superiority [20], however this is not supported by statistical evidence

The analysis used p-values and P scores as values of statistical significance, and provided evidence of a very similar effect of three interventions (SX9, P40, and modafinil), confirmed by the absence of pairwise difference on individual p-values. Compared with p-values, the ranking of P scores has recently been increasingly used as its interpretation is more relevant to routine clinical practice in which a patient must receive active treatment [23]. Instead of the two type 1 (α) and type 2 (β) risks of classical statistics, minimizing the probability of selecting one drug when at least another is better (γ risk) is the relevant question. The ranking of the P score performs this minimization, with the P score being associated with an intervention that measures the extent of certainty that one treatment is better than another treatment, averaged over all competing treatments [23]. Thus, the ranking of P scores minimizes the gamma risk [23]. The BR ratio accounting for both efficacy and safety constitutes the best estimate of clinical utility.

However, efficacy, safety, and benefit/risk ratio must be separately discussed for patients with type 1 narcolepsy experiencing both EDS and cataplexy and patients with type 2 narcolepsy only experiencing EDS. For type 1 patients, the NS is appropriate for which highest P scores are obtained for Pitolisant 40 mg followed by sodium oxybate 9 g and modafinil (found in third position due to lack of effect in cataplexy). For type 2 patients, P40 remains with the highest P score, directly followed by modafinil and sodium oxybate. The BR ratio was both applied on EDS Z score and NS, results were virtually unchanged, P40 with the highest P score, followed by the two outsiders with similar performances.

For this endpoint, the P score of .87 associated with P40 outweighed the score of the alternative drugs.

The present study had several limitations. The analysis was based on a systematic review of RCTs that could be identified through our research, and relied upon the availability of the publications for review and their accessibility in the databases used.

We justify our research of unpublished results in meta-analysis that may help to overcome publication bias, which arises due to the lack of RCTs that published negative results [50].

We used aggregate endpoint (Z scores) to summarize efficacy and BR: combining variables like ESS and MWT to produce a Z score might appear as somewhat artificial as they measure different aspects of wakefulness, that is, a tendency to fall asleep during current activities for ESS and the ability to remain awake in a laboratory environment for MWT. For this reason, we also compared the tested treatments separately for each endpoint, whereas Z scores were mainly used to reduce inflation of statistical type 1 error.

For a meta-analysis of findings from the literature, summary statistical data is usually given in publications. The initial conditions of patients are sometimes accounted for by the mean change. However, adjustment for baseline cannot be conducted based on individual values. Therefore, approximations were needed, due to the necessary estimate of SDs that were not reported in many of the publications. Some approximate conversions were also needed when only medians were available. For cataplexy, only 8 out of the 14 studies provided values, data were sometimes reported as graphics, and heterogeneous values were provided in terms of days, numbers, or rates of cataplexy. Also, the assessment of efficacy on cataplexy by evaluating the rate of attacks is an oversimplification that does not reflect the high level of heterogeneity among total and partial attacks.

In the analysis undertaken in the present study, EDS and cataplexy were considered to be the main symptoms of narcolepsy, whereas other symptoms (e.g. hallucinations or sleep attacks) that are irregularly documented in trials were not analyzed. Safety was often poorly documented in the publications, in particular, for older trials. This profile corresponds to a majority of patients, in particular, patients with type 1 narcolepsy. However, for a significant proportion of patients, namely but not only patients with type 2 narcolepsy as defined in the ICSD 3 [5], only EDS is a major problem and the NS outweighs the relative importance of cataplexy. For this group of patients, ESS and MWT analysis are more appropriate, and for which consistent treatment ranking was found between the two endpoints. As in all meta-analytical multiple comparisons, due to differences between the available sample sizes, more statistical significant results are likely to be found for interventions investigated with a larger sample size, such as modafinil; the lowest sample size with pitolisant, which may have reduced the power in superiority tests.

Disturbed nocturnal sleep may constitute a significant problem not taken into account in the present analysis, since it was rarely documented in the clinical trials reviewed in this study.

Conclusion

A network meta-analysis based on 14 published RCTs compared the efficacy, safety, and BR ratio of medical treatments for adult narcolepsy. Three drug treatments at specific doses, modafinil (200–400 mg/d), sodium oxybate 9 g/d, and pitolisant up to 40 mg/d were found to have similar clinical efficacy and were significantly more effective than placebo for excessive day time sleepiness. Only sodium oxybate 9 g/d and pitolisant up to 40 mg/d were shown with a comparable beneficial effect on cataplexy. Overall, Pitolisant at a maximal dose of 40 mg/d was shown to have a slightly better safety profile and the highest BR ratio.

Work Performed: University of Melbourne, Faculty of Medicine Melbourne, Australia

Funding

Financial support was provided by Bioprojet Pharma (Paris, France) to conduct this study and for editorial assistance.

Conflict of interest statement. Professor Lehert and Professor Falissard are consultants for Bioprojet Pharma.