Dear Editor,
We thank Dr. Fakhouri et al. for their comments on our study. The comments note possible heterogeneity resulting from differences in study designs and patient characteristics among the trials included in our network meta-analysis (NMA), which are limitations common to all meta-analyses. In particular, Dr. Fakhouri raised concerns relating to cross-trial differences in prior exposure to biologic disease-modifying antirheumatic drugs (bDMARDs), number of prior conventional synthetic disease-modifying antirheumatic drug (csDMARD) failures, background dose of methotrexate (MTX), and placebo arm response rates among all the trials included in the network. In this response letter, we will discuss the approaches taken to mitigate and estimate the heterogeneity of trials included in the NMA and introduce supportive evidence to address the concerns raised by Dr. Fakhouri.
To minimize confounding differences between the trial populations in our NMA, the studies included in the NMA were required to meet the pre-defined selection criteria to be eligible for inclusion in the analysis. Specifically, the studies were required to be phase III randomized controlled trials evaluating Janus kinase (JAK) inhibitors among patients who had an inadequate response or were intolerant to at least one csDMARD (csDMARD-IR). In addition, patients were eligible for inclusion if the patient population was naïve to bDMARDs or if no more than 20% of patients in the trial had prior exposure to bDMARDs. These inclusion criteria were selected on the basis of prior publications, health technology assessment reports, and clinical input to reduce the heterogeneity between trials [1–3].
Additionally, a random-effects model was implemented to account for the potential between-trial heterogeneity in treatment contrasts and ensure proper statistical inference under such heterogeneity. The tau heterogeneity parameter, which is the precision parameter of the distribution of the underlying true effects across studies and quantifies the between-trial heterogeneity, was summarized from our model results. The posterior median estimate for tau in our random-effects model for American College of Rheumatology (ACR) outcomes is 200.1 for the 12-week network and 61.7 for the 24-week network. While the posterior medians for tau suggest heterogeneous treatment contrasts across studies, such heterogeneity was taken into account in the estimation of the model through the use of the random-effects model.
In regards to the first concern about the inclusion of studies enrolling patients with prior bDMARD exposure, the majority of trials of JAK inhibitors included a small proportion of patients (at most 20%) with prior exposure to bDMARDs. These trials have been widely used in prior indirect comparison studies or health technology assessments among the csDMARD-IR patient population [1–4]. Prior studies have specifically evaluated the impact of including trials with a small proportion of patients with prior bDMARD exposure and found negligible impact on meta-analysis results [1,3,5].
The second concern notes the difference between trials related to the number of prior csDMARD failures experienced by patients at baseline. We agree that the included trials vary in terms of number of prior csDMARD failures. However, we do not believe these differences will have significant impact on response to treatment especially as Kremer et al. note that “the response to baricitinib was similar across levels of disease duration and the number of prior csDMARDs used, suggesting that baricitinib is an equally effective treatment option for patients regardless of their previous treatment experience” [6].
Dr. Fakhouri also noted the potential for bias resulting from including trials with Asian patients, arguing that these patients may be exposed to lower doses of concurrent MTX treatment. To address these concerns, we ran two sensitivity analyses to control for the geographic regions in which the trials took place. First, we excluded trials that were conducted exclusively in Asian countries. This resulted in the exclusion of SELECT-SUNRISE, a phase III randomized controlled trial of upadacitinib conducted in Japan. Placebo-controlled data in SELECT-SUNRISE were available only at 12 weeks because of the trial design [7]. As such, the reported 24-week network is unchanged with the exclusion of SELECT-SUNRISE. The 12-week ACR results for the sensitivity analysis excluding SELECT-SUNRISE are reported in Table 1. Only minor differences in median ACR response rates are observed in the 12-week network, with the efficacy rankings of treatments remaining consistent across ACR20/50/70 and surface under the cumulative ranking curve (SUCRA) outcomes.
Table 1.
ACR outcomes and SUCRA scores at week 12 in the csDMARD-IR RA population excluding SELECT-SUNRISE
| Treatment | Median ACR20% (95% CrI)a |
Median ACR50% (95% CrI)a |
Median ACR70% (95% CrI)a |
SUCRAb |
|---|---|---|---|---|
| Week 12 networkc | ||||
| csDMARD | 35.7 (28.7, 43.2) | 13.8 (9.9, 18.6) | 4.2 (2.7, 6.4) | 0.001 |
| JAK combination therapiesd | ||||
| Upadacitinib 15 mg + csDMARD | 69.6 (58.9, 78.4) | 41.6 (30.8, 52.5) | 19.9 (12.8, 28.4) | 0.844 |
| Tofacitinib 5 mg + csDMARD | 66.6 (56.1, 76.1) | 38.4 (28.4, 49.4) | 17.7 (11.4, 25.9) | 0.663 |
| Baricitinib 2 mg + csDMARD | 65.1 (51.9, 76.8) | 36.9 (25.0, 50.3) | 16.6 (9.5, 26.6) | 0.563 |
| Baricitinib 4 mg + csDMARD | 64.8 (54.6, 74.0) | 36.6 (27.1, 46.8) | 16.4 (10.7, 23.8) | 0.528 |
| JAK monotherapy therapiesd | ||||
| Upadacitinib 15 mg | 66.6 (52.3, 78.7) | 38.3 (25.2, 52.8) | 17.6 (9.7, 28.7) | 0.642 |
| Tofacitinib 5 mg | 58.0 (42.6, 72.5) | 30.1 (18.1, 45.1) | 12.4 (6.1, 22.5) | 0.258 |
ACR American College of Rheumatology, csDMARD conventional synthetic disease-modifying antirheumatic drug, csDMARD-IR inadequate response to csDMARD, CrI credible interval, JAK Janus kinase, RA rheumatoid arthritis, SUCRA surface under the cumulative ranking curve
aMedians and credible intervals for ACR outcomes were estimated using a random-effects multinomial model. The distribution of means and credible intervals were sampled using Monte Carlo methods (150,000 posterior simulations per treatment after 50,000 burn-in, thinning parameter of 10, and 3 chains)
bSUCRA was calculated to assess the overall ranking of each treatment based on ACR20 outcomes. Higher SUCRA values (closer to 1) represent more favorable rankings
cAs a result of differences in trial design, ACR outcomes were used in the 12-week network if reported between 12 and 14 weeks
dJAK combination therapies and monotherapy treatments were analyzed together in the same network for 12-week ACR outcomes
An additional sensitivity analysis excluding both SELECT-SUNRISE and RA-BALANCE was run to further provide supportive evidence. RA-BALANCE was a global phase III randomized controlled trial of baricitinib conducted in China, Argentina, and Brazil. ACR results for the sensitivity analysis excluding SELECT-SUNRISE and RA-BALANCE are reported in Table 2. Similarly, this sensitivity analysis resulted in minor numerical differences in the median ACR response rates while the efficacy ranking of treatments in both networks again remained unchanged.
Table 2.
ACR outcomes and SUCRA scores at week 12/24 in the csDMARD-IR RA population excluding SELECT-SUNRISE and RA-BALANCE
| Treatment | Median ACR20% (95% CrI)a |
Median ACR50% (95% CrI)a |
Median ACR70% (95% CrI)a |
SUCRAb |
|---|---|---|---|---|
| Week 12 networkc | ||||
| csDMARD | 36.2 (29.1, 43.8) | 14.1 (10.1, 19.0) | 4.4 (2.8, 6.6) | 0.004 |
| JAK combination therapiesd | ||||
| Upadacitinib 15 mg + csDMARD | 69.9 (57.8, 79.5) | 42.1 (29.9, 54.1) | 20.3 (12.4, 29.9) | 0.827 |
| Tofacitinib 5 mg + csDMARD | 67.1 (55.7, 77.3) | 39.0 (28.1, 51.1) | 18.2 (11.3, 27.5) | 0.669 |
| Baricitinib 2 mg + csDMARD | 65.4 (50.0, 78.5) | 37.2 (23.5, 52.6) | 17.0 (8.8, 28.7) | 0.563 |
| Baricitinib 4 mg + csDMARD | 64.8 (52.2, 75.6) | 36.6 (25.2, 48.9) | 16.6 (9.7, 25.6) | 0.514 |
| JAK monotherapy therapiesd | ||||
| Upadacitinib 15 mg | 67.0 (50.7, 80.5) | 38.9 (24.0, 55.5) | 18.1 (9.1, 31.2) | 0.642 |
| Tofacitinib 5 mg | 58.6 (40.6, 75.3) | 30.7 (16.8, 48.6) | 12.9 (5.6, 25.4) | 0.282 |
| Week 24 networkc | ||||
| csDMARD | 35.0 (28.1, 42.6) | 18.9 (14.0, 24.8) | 7.7 (5.2, 11.1) | 0.016 |
| JAK combination therapies | ||||
| Upadacitinib 15 mg + csDMARD | 69.8 (41.5, 89.2) | 50.9 (23.9, 77.1) | 30.1 (10.5, 57.9) | 0.830 |
| Baricitinib 4 mg + csDMARD | 65.3 (43.6, 81.9) | 46.0 (25.6, 66.1) | 25.9 (11.5, 45.0) | 0.676 |
| Tofacitinib 5 mg + csDMARD | 62.1 (44.3, 77.8) | 42.5 (26.1, 60.6) | 23.2 (11.8, 39.2) | 0.520 |
| Baricitinib 2 mg + csDMARD | 60.4 (32.8, 82.7) | 40.8 (17.3, 67.3) | 21.9 (6.9, 46.2) | 0.458 |
ACR American College of Rheumatology, csDMARD conventional synthetic disease-modifying antirheumatic drug, csDMARD-IR inadequate response to csDMARD, CrI credible interval, JAK Janus kinase, RA rheumatoid arthritis, SUCRA surface under the cumulative ranking curve
aMedians and credible intervals for ACR outcomes were estimated using a random-effects multinomial model. The distribution of means and credible intervals were sampled using Monte Carlo methods (150,000 posterior simulations per treatment after 50,000 burn-in, thinning parameter of 10, and 3 chains)
bSUCRA was calculated to assess the overall ranking of each treatment based on ACR20 outcomes. Higher SUCRA values (closer to 1) represent more favorable rankings
cAs a result of differences in trial design, ACR outcomes were used in the 12-week network if reported between 12 and 14 weeks and used in the 24-week network if reported between 24 and 26 weeks
dJAK combination therapies and monotherapy treatments were analyzed together in the same network for 12-week ACR outcomes
As such, the incremental benefit of including evidence generated from these trials outweighs the potential for bias resulting from the geographic region in which the trials took place.
Finally, our model used an anchor-based approach which subtracts the placebo arm response from the response of the active treatment arm on a probit scale to inform comparisons between active treatments across different trials. To further address concerns regarding the impact of cross-trial differences in reference arm response, we conducted a sensitivity analysis adjusting for reference arm response as a trial-level covariate [8]. The results of the reference arm response-adjusted model are presented in Table 3. Once again, minor numerical differences are observed in the median ACR response rates for both 12-week and 24-week results. In the 24-week network, the efficacy rankings of baricitinib 2 mg + csDMARD and tofacitinib 5 mg + csDMARD switch between 3rd and 4th among the JAK combination therapies. Upadacitinib 15 mg + csDMARD remains ranked numerically highest across ACR20/50/70 and SUCRA outcomes in both the 12-week and 24-week networks.
Table 3.
Reference arm response-adjusted ACR outcomes and SUCRA scores at week 12/24 in the csDMARD-IR RA population
| Treatment | Median ACR20% (95% CrI)a |
Median ACR50% (95% CrI)a |
Median ACR70% (95% CrI)a | SUCRAb |
|---|---|---|---|---|
| Week 12 networkc | ||||
| csDMARD | 35.9 (28.9, 43.4) | 13.9 (10.0, 18.8) | 4.3 (2.7, 6.4) | 0.008 |
| JAK combination therapiesd | ||||
| Upadacitinib 15 mg + csDMARD | 71.5 (57.6, 82.2) | 43.8 (29.7, 57.9) | 21.4 (12.1, 33.1) | 0.848 |
| Tofacitinib 5 mg + csDMARD | 66.4 (49.4, 85.6) | 38.3 (23.0, 63.2) | 17.5 (8.4, 38.2) | 0.657 |
| Baricitinib 2 mg + csDMARD | 65.3 (48.0, 79.8) | 37.1 (22.0, 54.5) | 16.7 (7.9, 29.9) | 0.549 |
| Baricitinib 4 mg + csDMARD | 64.8 (50.7, 76.6) | 36.5 (24.0, 50.1) | 16.3 (8.9, 26.3) | 0.510 |
| JAK monotherapy therapiesd | ||||
| Upadacitinib 15 mg | 66.8 (45.7, 81.9) | 38.6 (20.3, 57.4) | 17.7 (7.1, 32.6) | 0.606 |
| Tofacitinib 5 mg | 58.3 (34.9, 82.2) | 30.4 (13.3, 58.0) | 12.5 (4.0, 33.1) | 0.322 |
| Week 24 networkd | ||||
| csDMARD | 34.8 (27.9, 42.4) | 18.5 (13.7, 24.3) | 7.4 (5.0, 10.7) | 0.025 |
| JAK combination therapies | ||||
| Upadacitinib 15 mg + csDMARD | 71.1 (45.3, 88.4) | 52.0 (26.7, 75.4) | 30.9 (12.1, 55.6) | 0.868 |
| Baricitinib 4 mg + csDMARD | 66.2 (47.2, 80.5) | 46.4 (28.2, 63.8) | 26.2 (13.1, 42.2) | 0.691 |
| Tofacitinib 5 mg + csDMARD | 55.2 (25.4, 85.6) | 35.3 (12.1, 71.1) | 17.8 (4.3, 50.5) | 0.386 |
| Baricitinib 2 mg + csDMARD | 62.4 (37.0, 83.0) | 42.4 (20.1, 67.2) | 23.0 (8.3, 45.9) | 0.530 |
ACR American College of Rheumatology, csDMARD conventional synthetic disease-modifying antirheumatic drug, csDMARD-IR inadequate response to csDMARD, CrI credible interval, JAK Janus kinase, RA rheumatoid arthritis, SUCRA surface under the cumulative ranking curve
aMedians and credible intervals for ACR outcomes were estimated using a random-effects multinomial model. The distribution of means and credible intervals were sampled using Monte Carlo methods (150,000 posterior simulations per treatment after 50,000 burn-in, thinning parameter of 10, and 3 chains)
bSUCRA was calculated to assess the overall ranking of each treatment based on ACR20 outcomes. Higher SUCRA values (closer to 1) represent more favorable rankings
cAs a result of differences in trial design, ACR outcomes were used in the 12-week network if reported between 12 and 14 weeks and used in the 24-week network if reported between 24 and 26 weeks
dJAK combination therapies and monotherapy treatments were analyzed together in the same network for 12-week ACR outcomes
We also calculated the deviance information criterion (DIC) for the reference arm response-adjusted model and compared the DIC with that of the reported model, shown in Table 4. DIC is often considered a measure of model fit, with lower values of DIC suggesting better fit [9]. The DIC for both models were similar, but slightly favor the non-reference arm response-adjusted model reported in the manuscript.
Table 4.
Deviance information criterion for reported model and reference arm response-adjusted model
| Model | DIC |
|---|---|
| Week 12 network | |
| ACR random-effects | 441.5 |
| ACR reference arm response-adjusted random-effects | 442.7 |
| Week 24 network | |
| ACR random-effects | 325.1 |
| ACR reference arm response-adjusted random-effects | 326.4 |
ACR American College of Rheumatology, DIC deviance information criterion
All analyses referenced in this article are based on previously conducted studies and do not contain any studies with human participants or animals performed by any of the authors. No institutional board review was required.
We appreciate the feedback provided by Dr. Fakhouri and the opportunity to further discuss the potential limitations of our study. We believe the discussions and additional results reported in this letter support the robustness of the findings in the reported NMA. Ultimately, continued research involving head-to-head randomized trials will be ideal to evaluate comparative efficacy among JAK inhibitors. In the absence of such data, we believe our network meta-analysis provides timely and clinically useful evidence in regards to the comparative efficacy among the different JAK inhibitors.
Acknowledgements
Funding
Financial support for the study was provided by AbbVie. AbbVie participated in interpretation of data, review, and approval of the presentation. All authors contributed to development of the presentation and maintained control over final content.
No Rapid Service Fee was received by the journal for the publication of this article.
Writing Assistance
The authors would like to thank Rochelle Sun, employee of Analysis Group, Inc., for assistance with the analysis and preparation of the response letter.
Authorship
All named authors meet the International Committee of Medical Journal Editors (ICMJE) criteria for authorship for this article, take responsibility for the integrity of the work as a whole, and have given their approval for this version to be published.
Disclosures
Keith A. Betts, Ella X. Du, Cynthia Z. Qi, Yan Song, and Patrick Tang are employees of Analysis Group, Inc., which has received consulting fees from the sponsor. Ruta Sawant and Namita Tundia are employees of AbbVie, Inc., and hold stock/options. Janet Pope has consulted and received honoraria from AbbVie, Amgen, BMS, Gilead, Janssen, Lilly, Merck, Novartis, Pfizer, Roche, Sandoz, Sanofi, UCB.
Compliance with Ethics Guidelines
This article is based on previously conducted studies and does not contain any studies with human participants or animals performed by any of the authors.
References
- 1.Stevenson M, Archer R, Tosh J, et al. Adalimumab, etanercept, infliximab, certolizumabpegol, golimumab, tocilizumab and abatacept for the treatment of rheumatoid arthritis not previously treated with disease-modifying antirheumatic drugs and after the failure of conventional disease-modifying antirheumatic drugs only: systematic review and economic evaluation. Health Technol Assess (Winchester, England) 2016;20(35):1–610. doi: 10.3310/hta20350. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Baricitinib for moderate to severe rheumatoid arthritis: Technology appraisal guidance. National Institute for Health Care and Excellence (NICE); 2017. Accessed 10 October 2020.
- 3.Upadacitinib for moderate to severe rheumatoid arthritis. National Institute for Health Care and Excellence (NICE); 2020. https://www.nice.org.uk/guidance/ta665. Accessed 10 Oct 2020.
- 4.Lee YH, Song GG. Relative efficacy and safety of tofacitinib, baricitinib, upadacitinib, and filgotinib in comparison to adalimumab in patients with active rheumatoid arthritis. Z Rheumatol. 2020;79:785–796. doi: 10.1007/s00393-020-00750-1. [DOI] [PubMed] [Google Scholar]
- 5.Fakhouri W, Wang X, de La Torre I, Nicolay C. A network meta-analysis to compare effectiveness of baricitinib and other treatments in rheumatoid arthritis patients with inadequate response to methotrexate. J Health Econ Outcomes Res. 2020;7(1):10–23. doi: 10.36469/jheor.2020.12273. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Kremer JM, Schiff M, Muram D, Zhong J, Alam J, Genovese MC. Response to baricitinib therapy in patients with rheumatoid arthritis with inadequate response to csDMARDs as a function of baseline characteristics. RMD Open. 2018;4(1):e000581. doi: 10.1136/rmdopen-2017-000581. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Kameda H, Takeuchi T, Yamaoka K, et al. Efficacy and safety of upadacitinib in Japanese patients with rheumatoid arthritis (SELECT-SUNRISE): a placebo-controlled phase IIb/III study. Rheumatology. 2020;59(11):3303–3313. doi: 10.1093/rheumatology/keaa084. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Signorovitch JE, Betts KA, Yan YS, et al. Comparative efficacy of biological treatments for moderate-to-severe psoriasis: a network meta-analysis adjusting for cross-trial differences in reference arm response. Br J Dermatol 2015;172(2):504–12. 10.1111/bjd.13437. [DOI] [PubMed]
- 9.Dias S, Welton NJ, Sutton AJ, Ades AE. NICE DSU Technical Support Document 2: A generalised linear modelling framework for pairwise and network meta-analysis of randomised controlled trials. London: National Institute for Health and Care Excellence (NICE); 2010. https://www.ncbi.nlm.nih.gov/books/NBK310366/. [PubMed]