Efficacy and safety of biological agents for systemic juvenile idiopathic arthritis: a systematic review and meta-analysis of randomized trials

Abstract

Objective. To define the optimal biologic agent for systemic JIA (sJIA) based on safety and efficacy data from a randomized controlled trial (RCT).

Methods. Through a systematic literature search, sJIA RCTs evaluating biologic agents were identified. The primary efficacy outcome was defined as a 30% improvement according to the modified American College of Rheumatology Paediatric 30 response criteria (JIA ACR30). The primary safety outcome was defined as serious adverse events (SAEs). Outcomes were analysed by pairwise and network meta-analyses. The quality of evidence between biologic agents was assessed by applying the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) methodology.

Results. From the 493 citations originally identified, 5 RCTs were eligible for inclusion—one each for anakinra, canakinumab and tocilizumab and two for rilonacept: all vs placebo. While all were effective, the network meta-analysis indicated with low-quality evidence (due to indirect comparison and inconsistency) that rilonacept-treated patients were less likely to respond than those treated with canakinumab [odds ratio (OR) 0.10 (95% CI 0.02, 0.38), P = 0.001] or tocilizumab [OR 0.12 (95% CI 0.03, 0.44), P = 0.001]. Risks of SAEs were similar among the biologic agents (supported by very low-quality evidence) and not different from placebo.

Conclusion. Despite heterogeneous eligibility criteria and study designs across the five studies and different modified JIA ACR30 criteria, this meta-analysis of short-term RCTs presents empirical evidence that canakinumab and tocilizumab are more effective than rilonacept. Biologic agents in sJIA seem safe and comparable with respect to SAE risk in the short term.

Rheumatology key messages

• Rilonacept seems to be less effective compared with canakinumab and tocilizumab for treating systemic JIA.

• Canakinumab should mainly be considered in systemic JIA patients with high systemic involvement and limited joint involvement.

• Tocilizumab seems to be appropriate in systemic JIA patients with extensive joint involvement.

Introduction

JIA is a chronic inflammatory joint disease with onset in children <16 years of age [1, 2]. JIA is a heterogeneous disease that contains seven diverse categories in which the systemic JIA (sJIA) category includes systemic manifestations such as quotidian fever and rash. sJIA is considered to have a different etiology compared with other forms of JIA; it is more auto-inflammatory in nature [1, 3, 4]. The treatment strategies for sJIA are different from the other JIA categories [5, 6]. In general, with mild disease activity, treatment with an NSAID may suffice. However, in most cases, treatment with a systemic glucocorticoid or biologic agent is necessary, with or without conventional synthetic DMARDs (csDMARDs) such as MTX. Treatment strategies for sJIA need to be tailored according to patient-specific manifestations, primarily according to the degree of systemic features. In patients with active systemic features, treatment with a biologic agent can be part of the initial therapy [6]. The recommended biologic agents for sJIA in current clinical guidelines include anti- TNF agents (for patients with chronic arthritis, but not in the acute phase of sJIA), anakinra, canakinumab and tocilizumab [6]. However, only canakinumab and tocilizumab are approved by the US Food and Drug Administration (FDA) and European Medicines Agency (EMA) for the treatment of sJIA. Our overall objective was to determine the optimal biologic agent for use in sJIA patients who are candidates for biologic therapy, based on the relative efficacy and safety of these therapies using meta-analysis techniques including both direct and indirect evidence from randomized controlled trials (RCTs) [7].

Methods

Our protocol was registered in advance and is available from PROSPERO (CRD42013004736) [8]. The results and the manuscript are reported according to the recommendations given in the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement [9].

Types of studies and outcomes

Randomized trials comparing a biologic agent with placebo, csDMARD or another biologic agent in patients meeting the ILAR sJIA criteria [2], or equivalent, were considered eligible for inclusion. Both randomized withdrawal trials and parallel randomized trials were included but handled separately. Withdrawal trials randomize those patients with an initial response during an active open-label run-in period to either continue active treatment or placebo and therefore addresses a different population than the parallel trial design.

Two co-primary binary outcomes were considered: 30% improvement according to the American College of Rheumatology Paediatric response criteria (JIA ACR30) [10] for efficacy and serious adverse events (SAEs) as a proxy for harm [11]. JIA ACR30 is considered a validated measure of efficacy in JIA [10]. However, it was developed and validated in trials of polyarticular forms of JIA. In sJIA, modified JIA ACR responses are often used, requiring inclusion of systemic features (e.g. there must be an absence of fever and/or rash). We evaluated the JIA ACR30 responses with and without modifications, although the modified responses were considered more important as they account for the systemic nature of sJIA. We included the JIA ACR50, 70, and 90 as secondary benefit outcomes both with and without modifications because JIA ACR30 only reflects the minimal clinically important improvement for the patient.

As secondary safety measures, we included the number of patients with serious infections, any infections, any adverse events (AEs), withdrawals due to AEs and withdrawals for any cause. Post hoc analyses were carried out using the total number of events per total patient-days, because the placebo groups had high withdrawal rates in some trials and because the duration of the tocilizumab trial was longer than the others. These post hoc analyses were carried out for AEs and any infections (SAEs and serious infections were limited by very few events and by several zero-event treatment arms and were therefore not evaluated with this approach).

Literature search

A comprehensive search was performed in July 2014 using the Cochrane Central Register of Controlled Trials (the Cochrane Library, latest issue), Medline via PubMed (from 1950), Embase via Ovid (from 1980) and ClinicalTrials.gov. The search included keywords and text words related to RCTs, JIA and biologic agents (see supplementary data, search strategies section, available at Rheumatology Online). No language restrictions were applied. Reference lists in retrieved RCTs and systematic reviews in the field of biologics in JIA were scrutinized for further eligible trials. We also scrutinized relevant reports on the FDA and EMA websites and searched relevant pharmaceutical companies’ websites to identify unpublished trial data.

Data collection and risk of bias assessment

Study selection and data extraction were done independently by at least two reviewers. Disagreements were resolved by consensus with a fourth reviewer. Core outcome data in each study consisted of group size and the number of patients in each group who had an event for each outcome. The total number of events and the total number of patient-days were also abstracted. Furthermore, we collected study characteristics including important inclusion and exclusion criteria and essential baseline characteristics.

Quality assessment was performed using the Risk of Bias Tool from the Cochrane Collaboration (London, UK) [12]. Two reviewers (S.T. and G.A.) independently assessed whether the threats to the studies’ internal validity were adequately reported. Any disagreements were resolved by consensus with a third reviewer (R.C.).

Data synthesis

We conducted standard pairwise meta-analyses in Review Manager (version 5.3.3; Cochrane Collaboration) by applying inverse variance random-effects models to accommodate the anticipated heterogeneity among studies within each biologic [13]. In addition to reviewing forest plots, we statistically analysed heterogeneity among biologics using Cochran’s Q test [14] and the I² index for inconsistency [15] when two or more trials were identified for a given biologic agent. Fixed-effects models were used to support all random-effects models to assess the potential for any small-study bias [16]. For computational reasons, outcomes were expressed as odds ratios (ORs) with 95% CIs for each biologic vs placebo [17]. In cases where one study reported zero events in both treatment arms, we used a risk difference (RD) approach instead of expressing outcomes as ORs. Outcomes evaluated as total events per total patient-days were analysed as rate ratios [18].

To combine both direct and indirect comparisons, an arm-based approach was used to include the multiple comparisons in a network meta-analysis [19]. We performed mixed-effects logistic regression in a random-effects model within an empirical Bayes framework by using a generalized linear mixed model [20, 21]. Allowance was made for differences in heterogeneity of effects between different drugs by specifying that the linear predictor varies at the level of the drug and the drug varies across studies. In the network meta-analyses, we evaluated heterogeneity (i.e. between-study variance) in the network using T² (an estimate for τ²), which examines heterogeneity because of Study and Study × Drug interaction (i.e. smaller values indicate a better model per se). Outcomes were expressed as ORs with 95% CIs for each biologic vs placebo, as well as for all biologics mutually compared.

Evaluations of safety outcomes were limited because, in many cases, few or no patients experienced an event and/or because specific outcomes were not reported in some studies. Pairwise OR meta-analyses were possible for evaluating the risks of AEs, infections and withdrawal due to any causes, whereas an RD analysis was used for all other outcomes (including SAEs). Unfortunately, network meta-analysis was possible only for the risks of AEs and infections, as the other models did not converge. Meta-analyses were only conducted for the parallel trials due to several differences between the identified withdrawal trials. For completeness, both efficacy and safety results from the withdrawal trials were describe.

Quality of evidence

Evidence for the comparative effectiveness of various biologic agents for primary outcomes was assessed using criteria suggested by the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Working Group [22]. The GRADE methodology assumes that evidence from RCTs starts as high-quality evidence that can be downgraded to moderate, low or very low based on shortcomings in the body of evidence [23].

Results

Study selection

As illustrated in Fig. 1, five parallel trials of four different biologic agents (anakinra [24], canakinumab [25], rilonacept [26, 27] and tocilizumab [28]) and two withdrawal trials (canakinumab [25] and tocilizumab [29]) were eligible for inclusion. A search of ClinicalTrials.gov led to identifying all seven included studies and one unpublished etanercept sJIA parallel trial (NCT00078806), which was terminated early because of slow enrolment [30].

Fig. 1.

Flow diagram of the study selection process

Study characteristics

Study demographic and baseline characteristics are described in Table 1, study characteristics of the included trials are described in Supplementary Tables S1 and S2 and risk of bias assessment is included in Supplementary Table S3, available at Rheumatology Online. Four of the parallel trials had a 4-week placebo-controlled period, whereas the tocilizumab trial had a 12-week placebo-controlled period. To increase comparability we used JIA ACR data at week 4 reported in the FDA sJIA tocilizumab statistical review [31] (efficacy was consistent throughout the 12 weeks). Except in the anakinra trial, MTX background therapy was allowed during the studies. The canakinumab trial distinguished itself from the other trials, as it included patients having high systemic involvement and low joint involvement. Conversely, the tocilizumab trial included patients with more joint involvement compared with most other trials.

Table 1.

Demographic and baseline clinical characteristics of included studies

Drug/author/ type of trial	Total patients randomized (I/C)	MD global (VAS), median	Age, mean, years	Disease duration, mean, years	Females, %	Prior use of biologic agent, %	Patients on steroids, %	Number of joints with active arthritis, mean/median	Number of joints with limited range of motion, mean/median	CRP, mean/median, mg/l	Fever, %	Rash, %
Anakinra Quartier et al. [24] pRCT	24 (12/12)	6a	8.5	3.7	63	54.2	100	16.0/NR	17.0/NR	75/NR	38	NR
Canakinumab Ruperto et al. [25] pRCT	84 (43/41)	6.7	9.0	3.6	60	57.1	70	8.0/14.1	6.5/13.4	175/141	96	52
Rilonacept Lovell et al. [26] pRCT	24 (17/7)	5.5	12.6	3.1	67	50	71	NR/10.5	NR/7.0	72/NR	46	42
Rilonacept Ilowite et al. [27] pRCT	71 (36/35)	5.1	9.9	2.6	65	NR	66	11.1/8.2	NR/6.0	NR/44	23	42
Tocilizumab De Benedetti et al. [28] pRCT	112 (75/37)	6.7	9.7	5.2	50	82.1	90	15.2/19.9	16.2/19.8	166/99	58	18
Canakinumab Ruperto et al. [25] wRCT	100 (50/50)	0.0	9.1	3.2	55	57	NR	NR/0.0	NR/0.0	NR/5.3	0	1
Tocilizumab Yokota et al. [29] wRCT	43 (20/23)	NR	8.7	4.7	65	NR	100	NR/NR	NR/NR	<5	NR	NR

I/C: Intervention/control; MD global: physician global assessment of disease activity on a 10-point numerical rating scale (0–10 VAS, where higher scores indicate more active disease); NR: not reported; pRCT: parallel randomized controlled trial; VAS: visual analogue scale; wRCT: withdrawal randomized controlled trial.

^aValue given as mean.

The canakinumab withdrawal trial was an international multicentre study with a 12–32 week open-label active treatment lead-in phase that included a 20 week glucocorticoid tapering phase for patients on glucocorticoids. The duration of the placebo-controlled withdrawal phase was until 37 flares had occurred (88 weeks). The tocilizumab trial, undertaken in Japan, had a 6 week open-label lead-in phase and 12 week placebo-controlled withdrawal phase. MTX background therapy was only allowed in the canakinumab trial.

Efficacy

Primary efficacy outcome (parallel trials)

All trials reported a modified JIA ACR30 response with inclusion of systemic features. However, there were some differences among the trials concerning the way JIA ACR30 was modified (Supplementary Table S4, available at Rheumatology Online). Both rilonacept trials differed from the other trials: one trial required both the absence of fever and absence of rash, while the other trial required both the absence of fever and that patients on glucocorticoids at baseline have their dosage of systemic glucocorticoids tapered at least 10%. Fig. 2 presents results from the pairwise meta-analysis of modified JIA ACR30. All biologic agents were statistically significantly superior to placebo. However, significant heterogeneity among them was present (χ² = 15.65, P = 0.001), indicating one or more differences in efficacy across the four drugs. Table 2 presents the between-drugs results from the network meta-analysis (Supplementary Table S5, available at Rheumatology Online) and the quality of evidence assessments: canakinumab and tocilizumab were statistically significantly more effective than rilonacept, although with low-quality evidence, no difference was found between anakinra and rilonacept (very low-quality evidence) and no difference was noted among anakinra, canakinumab and tocilizumab (low-quality evidence).

Fig. 2.

Pairwise meta-analysis of modified JIA ACR30 (plus systemic features) responses

Table 2.

GRADE evidence profile

Quality assessment					No. of events/patients (%)		Effect		Quality
Comparison, drug 1 vs drug 2	Risk of bias	Inconsistency	Indirectness	Imprecision	Drug 1	Drug 2	Relative, OR (95% CI)	Absolute
Modified JIA ACR30 response
Anakinra vs canakinumab	No	No	Seriousa,b	Seriousc	11/12 (92)	35/43 (81)	0.55 (0.04, 6.83)	108 fewer per 1000 (from 665 fewer to 154 more)	Low
Anakinra vs rilonacept	No	Seriousd	Seriousa,b	Seriouse	11/12 (92)	26/52 (50)	5.73 (0.52, 62.88)	351 more per 1000 (from 158 fewer to 484 more)	Very low
Anakinra vs tocilizumab	No	No	Seriousa,b	Seriousf	11/12 (92)	57/75 (76)	0.69 (0.06, 8.18)	74 fewer per 1000 (from 600 fewer to 203 more)	Low
Canakinumab vs rilonacept	No	Seriousd	Seriousa	No	35/43 (81)	26/52 (50)	10.37 (2.60, 41.33)	412 more per 1000 (from 222 more to 476 more)	Low
Canakinumab vs tocilizumab	No	No	Seriousa	Seriousg	35/43 (81)	57/75 (76)	1.25 (0.28, 5.66)	38 more per 1000 (from 290 fewer to 187 more)	Low
Rilonacept vs tocilizumab	No	Seriousd	Seriousa	No	26/52 (50)	57/75 (76)	0.12 (0.03, 0.44)	485 fewer per 1000 (from 178 fewer to 673 fewer)	Low
Serious adverse events
Anakinra vs canakinumab	No	No	Serioush	Very seriousi	0/12 (0%)	2/43 (5%)	Not estimable	Not estimable	Very low
Anakinra vs rilonacept	No	No	Serioush	Very seriousi	0/12 (0%)	2/53 (4%)	Not estimable	Not estimable	Very low
Anakinra vs tocilizumab	No	No	Serioush	Very seriousi	0/12 (0%)	3/75 (4%)	Not estimable	Not estimable	Very low
Canakinumab vs rilonacept	No	No	Serioush	Very seriousi	2/43 (5%)	2/53 (4%)	Not estimable	Not estimable	Very low
Canakinumab vs tocilizumab	No	No	Serioush	Very seriousi	2/43 (5%)	3/75 (4%)	Not estimable	Not estimable	Very low
Rilonacept vs tocilizumab	No	No	Serioush	Very seriousi	2/53 (4%)	3/75 (4%)	Not estimable	Not estimable	Very low

^aEstimate based on indirect analysis.

^bMTX was not allowed in the anakinra trial, whereas it was allowed in all other trials.

^cPoint estimate indicates canakinumab is more effective than anakinra; however, CI includes both no difference and anakinra’s being more effective than canakinumab.

^dDefinition of modified JIA ACR30 in the rilonacept trials is different from definitions used in other trials.

^ePoint estimate indicates anakinra is more effective than rilonacept; however, CI includes both no difference and rilonacept’s being more effective than anakinra.

^fPoint estimate indicates tocilizumab is more effective than anakinra; however, CI includes both no difference and anakinra’s being more effective than tocilizumab.

^gPoint estimate indicates canakinumab is more effective than tocilizumab; however, CI includes both no difference and tocilizumab’s being more effective than canakinumab.

^hInterpretation based on the pairwise meta-analysis of serious adverse events.

ⁱNetwork meta-analysis was not possible. GRADE: Grading of Recommendations Assessment, Development, and Evaluation: JIA ACR: American College of Rheumatology Paediatric response criteria.

Secondary efficacy outcomes (parallel trials)

Modified JIA ACR50, 70 and 90 responses were not reported in one of the rilonacept trials [27], nor were they reported for the placebo group in the tocilizumab trial. Among the three eligible RCTs, modified JIA ACR90 was reported in only the canakinumab trial. Some variability was present among the three trials definitions of modified JIA ACR (Supplementary Table S4, available at Rheumatology Online). Results from the pairwise and network meta-analyses of modified JIA ACR50 and 70 supported the findings from the modified JIA ACR30 analysis showing that canakinumab was statistically significantly more effective than rilonacept and no differences existed between anakinra and rilonacept or canakinumab (Supplementary Figs. S1 and S2 and Supplementary Tables S6 and S7, available at Rheumatology Online).

JIA ACR30, 50, 70 and 90 responses (without systemic features) were not reported in the canakinumab trial and in one of the rilonacept trials [26]. Only JIA ACR30 was reported in the anakinra trial. JIA ACR90 response was reported in only the tocilizumab trial. Results from the pairwise and network meta-analyses of JIA ACR30, 50 and 70 supported to some extent the findings in the primary efficacy analysis (modified JIA ACR30). Tocilizumab was statistically significantly more effective that rilonacept for JIA ACR50 but not for JIA ACR70, and no differences were noted between anakinra and rilonacept or tocilizumab for JIA ACR30 (see Supplementary Figs. S3–S5 and Supplementary Tables S8–S10, available at Rheumatology Online).

Efficacy (withdrawal trials)

At the end of the open-label phase in the canakinumab trial, 100 of the 177 patients had at least a JIA ACR30 response plus the absence of fever and therefore qualified for randomization into the placebo-controlled withdrawal phase. Of the 50 patients randomized to continue canakinumab, 39 had no flare compared with 24 of 50 switched to placebo at the end of the withdrawal phase [OR 3.84 (95% CI 1.61, 9.16)]. The median time to flare was longer in the canakinumab group (>88 weeks) compared with the placebo group (38 weeks) (P = 0.003).

Of the 56 patients treated with tocilizumab in the 6 week open-label run-in phase, 44 met the response criteria (JIA ACR30 response and a CRP concentration <5 mg/l) and were randomized (tocilizumab 21, placebo 23). Of the 20 tocilizumab-treated patients included in the efficacy analysis, 16 (80%) maintained response (JIA ACR30 response and a CRP concentration <15 mg/l) compared with 4 of the 23 (17%) in the placebo group [OR 19.00 (95% CI 4.08, 88.38)]. The median time of maintained response was longer in the tocilizumab group (>12 weeks) compared with the placebo group (4.9 weeks) (P < 0.0001).

Safety

Primary safety outcome (parallel trials)

Because the anakinra trial reported zero SAEs in both treatment groups, the pairwise meta-analysis was evaluated as RDs. There was no statistical increased risk compared with placebo for any of the four biologics, nor was there any statistically significant heterogeneity among them (χ² = 0.87, P = 0.83) (Fig. 3). Because data were so sparse, a network meta-analysis was not possible (unable to converge). Overall, no difference between drugs for SAEs was judged to be present, although the quality of evidence was very low (Table 2).

Fig. 3.

Pairwise meta-analysis of SAEs

Secondary safety outcomes (parallel trials)

Tocilizumab statistically significantly increased the risk of AEs compared with placebo, whereas rilonacept decreased the risk (see Supplementary Fig. S6, available at Rheumatology Online). The network meta-analysis refined these data, showing that canakinumab and tocilizumab statistically significantly increased the risk of AEs compared with rilonacept and that tocilizumab statistically significantly increased the risk of AEs compared with canakinumab (Supplementary Table S11, available at Rheumatology Online). Post hoc analysis of AEs—evaluated as the total number of events per total patient-days (where anakinra was eligible for inclusion)—showed that rilonacept statistically significantly decreased the risk of AEs compared with placebo, whereas anakinra, canakinumab and tocilizumab did not differ from placebo (Supplementary Fig. S7, available at Rheumatology Online).

Both canakinumab and tocilizumab statistically significantly increased the risk of infections compared with placebo (Supplementary Fig. S8 and Supplementary Table S12, available at Rheumatology Online). When evaluated as events per total patient-days, however, the risk was not increased (Supplementary Fig. S9, available at Rheumatology Online). Differences between the pairwise meta-analyses results obtained with the total number of patients with at least one event vs the total number of events per total patient-days indicated that the high withdrawal in the placebo groups in the canakinumab and tocilizumab trials (Supplementary Fig. S10, available at Rheumatology Online) might have influenced the results, as no increased risk was seen in the latter approach. All other secondary safety outcomes did not indicate any differences among drugs (Supplementary Figs. S11 and S12, available at Rheumatology Online).

Safety (withdrawal trials)

In the canakinumab trial, six patients in each group had SAEs during the placebo-controlled phase. The total patient-days were 11 622 for the canakinumab group and 9045 for the placebo group, as more patients in the placebo group discontinued early (52% vs 22%). The rate ratio of SAEs was not statistical significant [0.78 (95% CI 0.25, 2.41)]. More patients discontinued due to AEs in the placebo group (12% vs 0%). The number of patients with AEs (canakinumab 92% vs placebo 82%), infections (58% vs 42%) and serious infections (4% vs 4%) did not differ between canakinumab and placebo. Accounting for exposure time, no differences between canakinumab and placebo were noticed.

In the tocilizumab trial, zero patients in each group had SAEs. More patients in the placebo group discontinued early (82% vs 20%) and one patient in each group discontinued because of AEs. The number of patients with AEs was 95% in the tocilizumab group and 100% in the placebo group. The total number of patients with infections was not reported.

Discussion

This is the first meta-analysis of RCTs to compare both the efficacy and safety of biologic agents in patients with sJIA. Five parallel trials comparing four different biologic agents with placebo were available, but no trials compared biologic agents directly. Our analyses were able to point out potential differences between some of the evaluated biologic agents. Specifically, despite the lack of data, canakinumab, tocilizumab and anakinra seem to produce comparable efficacy (and to some extent safety), whereas rilonacept seems to be less effective compared with canakinumab and tocilizumab in patients with sJIA. However, for a chronic disease like sJIA, the short trial durations limit the generalizability of our findings.

We considered analysing both withdrawal design studies and parallel group studies together, but due to the very disparate designs, they were handled separately. We included two withdrawal design trials, one each of canakinumab and tocilizumab, both vs placebo. A meta-analysis of these two trials was not done because the designs were quite different. In summary, these trials showed that both canakinumab and tocilizumab are effective in terms of preventing relapse and safe in the longer term. These two trials supported the findings from the meta-analyses of the parallel group trials for these two drugs.

Our finding regarding a modified JIA ACR30 response extended previous findings in a meta-analysis of RCTs [32]. They also found no differences among anakinra, canakinumab and tocilizumab based on a modified JIA ACR30 response. At the time their literature search was conducted (through January 2012), full publications of the canakinumab and tocilizumab trials were not available and there were no data from the rilonacept trials.

Our efforts in this area point the way to potential differences among RCTs in sJIA and are therefore of considerable interest. sJIA is presented clinically in two roughly defined categories. In the first category, patients present with mainly systemic features and relatively mild joint disease. In the second category, patients present with systemic features and severe debilitating arthritis [33]. We recommend examining these categories separately in future studies.

The work conducted by the Paediatric Rheumatology Collaborative Study Group (PRCSG) and the Paediatric Rheumatology International Trials Organisation networks in order to standardize outcomes in JIA clinical trials, in agreement with regulatory agencies such as the FDA and EMA, has led to advances in paediatric rheumatology [34]. However, the modified JIA ACR definitions varied greatly among studies. The tocilizumab trial and one of the rilonacept trials [27] prioritized investigating JIA ACR responses without inclusion of systemic features (only modified JIA ACR30 responses were adequately reported), although they enrolled both children with and without active systemic manifestations. In other trials the JIA ACR30 (without systemic features) responses were not reported, making comparison across trials difficult. Some of the observed discrepancies might be related to whether the investigator’s studies were under scrutiny by regulatory agencies. Currently EMA has recommend that fever should be added to the core set parameters of JIA ACR for sJIA [35]. A minimum uniform reporting requirement is recommended.

Head-to-head RCTs comparing biologic agents are subject to several problems. Given the small effect sizes when comparing two active medications, the trials often require large numbers of patients per group, which makes these trials very hard to complete and not feasible in sJIA [36]. EMA highlighted an alternative option with a three-arm study design that includes the test drug, an active comparator and a placebo arm. The placebo period could be short and the test and active-comparator arms could continue for a longer period [35]. We attempted a network meta-analysis to partially overcome the lack of head-to-head trials, but network analyses are subject to distortions. Although a network of randomized trials apparently permits inferences into the comparative effectiveness of interventions that may not have been evaluated directly against each other, certain methodological aspects are poorly understood [37]. As emphasized by our needing to downgrade the confidence in our estimates for indirectness. Treatment effects derived from network meta-analyses should be interpreted with due attention to their uncertainty if no head-to-head comparison studies have ever been performed [38]; although appealing, pseudo-direct comparisons can be misleading [39].

In summary, we were unable to define the single optimal biologic agent for the treatment of sJIA, as only limited evidence between evaluated agents was present. On the other hand, a few differences were noted. Rilonacept showed lower efficacy compared with other biologic agents and might not be considered as a first-line biologic agent (rilonacept is not approved for sJIA and is only available in the US for the treatment of cryopyrin-associated periodic syndrome). Our study also indicated that canakinumab should be considered mainly in patients with high systemic involvement and limited joint involvement, which also might be true to some extent for anakinra, whereas tocilizumab seems to be appropriate in patients with extensive joint involvement. We recognize that further data are needed before these conclusions can be interpreted with high confidence.