The difference in all-cause mortality between COVID-19 patients treated with standard of care plus placebo and those treated with standard of care alone: a network meta-analysis of randomised controlled trials of immunomodulatory kinase inhibitors

Abstract

Objectives

The aim of this network meta-analysis (NMA) was to assess whether participants assigned to a placebo and standard of care (SoC) group had different major coronavirus disease 2019 (COVID-19)-related outcomes than those assigned to SoC alone.

Design

Frequentist model-based NMA.

Setting

We searched for randomised controlled trials (RCTs) of Janus kinase/Bruton tyrosine kinase inhibitors for the management of COVID-19.

Participants

Patients with COVID-19 infection.

Main outcome measures

The primary outcome was the 28-day all-cause mortality, and secondary outcomes were: (1) use of mechanical ventilation; (2) secondary bacterial infection; (3) acceptability (i.e. drop-out rate); and (4) safety (i.e. serious adverse events). We conducted an NMA using the frequentist model. Effect sizes were estimated using odds ratios (ORs) with 95% confidence intervals (95% CIs).

Results

We identified 14 eligible RCTs enrolling a total of 13,568 participants with COVID-19. Participants assigned to placebo plus SoC had a significantly higher risk of 28-day all-cause mortality than those receiving SoC alone (OR = 1.39, 95% CI = 1.07–1.79). This finding did not change substantially by subgroup analysis stratified by epidemiology factor, pandemic history progression and statistical methodologic consideration. In addition, none of the treatments investigated were associated with a significantly different risk of secondary bacterial infection, acceptability or safety compared with the SoC group.

Conclusions

This NMA suggested a higher all-cause mortality in patients treated with placebo plus SoC compared with those treated with SoC alone. However, caution is advised in interpreting these results due to the absence of a direct head-to-head comparison. Future research should critically evaluate the necessity of placebo administration in COVID-19 RCTs and consider alternative study designs to minimise potential biases.

Trial registration: The current study was approved by the Institutional Review Board of the Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan (TSGHIRB No. B-109–29) and registered in PROSPERO (CRD42022376217).

Introduction

To date, the coronavirus disease 2019 (COVID-19) pandemic has resulted in millions of deaths worldwide.^1,2 The immunomodulatory kinase inhibitors, including Janus kinase (JAK) and Bruton tyrosine kinase (BTK) inhibitors, are among the most widely researched agents to reduce mortality related to COVID-19. The rationale for the therapeutical benefit of immunomodulatory kinase inhibitors was based on the observation of immune dysregulation and an associated imbalance in JAK and signal transducer and activator of transcription pathways during severe COVID-19 infection. ³ Similarly, the major cause of mortality related to COVID-19 resulted from acute/subacute lung injury, which was linked to over-expression of BTK. ⁴ Recently, Ngamprasertchai et al. ⁵ published a network meta-analysis (NMA) investigating the efficacy of individual immunomodulators on reducing all-cause mortality in patients with COVID-19, which revealed that, among JAK/BTK inhibitors, baricitinib was the only regimen associated with a lower 28-day all-cause mortality compared with the control group (which included both placebo plus standard of care [SoC] and SoC-only groups). However, a difference in the efficacy between the SoC-only and SoC plus placebo controls on all-cause mortality was observed. Although the authors of the NMA did not address the possible reasons for the difference in all-cause mortality rates between patients with SoC-only or SoC plus placebo, we hypothesised that this difference in efficacy between the SoC-only and SoC plus placebo arms might have arisen for two possible reasons: (1) an actual harmful placebo effect; or (2) methodological weaknesses.

The use of placebo in clinical randomised controlled trials (RCTs) has been a subject of debate, especially with regard to the potential impact related to inadequate placebo application. ⁶ Application of placebo would contribute to favourable behavioural change ⁷ or unfavourable behavioural changes. ⁸ This potential impact by the use of placebo could be found in the major outcomes of certain diseases. In a systematic review of cardiovascular studies using all-cause mortality as the outcome, ⁹ the mortality rate in the placebo arm of an indicated study of peripheral arterial disease was increased in comparison with the average mortality in the ordinary peripheral arterial disease population. ¹⁰

Second, possible methodological weaknesses might have created bias in the previous NMA. ⁵ Specifically, the authors included RCTs without any events in one arm in their analysis. Although the authors did not specify their statistical continuity correction method, we could infer the possible continuity correction method used from the supplementary figures. There was debate about the risk of bias associated with this continuity correction method, ¹¹ which might result in a counterintuitive statistical result. ¹²

This study aimed to re-examine whether participants assigned to an SoC plus placebo group might have different major outcomes of COVID-19 compared with those assigned to an SoC-only group.

Methods

The detailed information of the method has been described in the supplementary material (Appendix: eMethods and materials). In brief, we followed the PRISMA guideline ¹³ (eTable 1) and focused on RCTs of JAK/BTK inhibitors in acute COVID-19 treatment. The detailed search strategies and keywords used in each database are listed in eTable 2. We selected 28-day all-cause mortality as our primary outcome. The reason for choosing 28 days as the period for ascertaining the primary outcome was that the most all-cause mortality occurs during the first month of COVID-19.^14,15 The secondary outcomes included: (1) use of mechanical ventilation; (2) secondary bacterial infection; (3) acceptability; and (4) safety profile. The odds ratio (OR) with 95% confidence intervals (95% CIs) was used as the measure of effect size. We used the frequentist model for NMA. ¹⁶ All analyses were performed using network suite for Stata version 16.0 (StataCorp LLC, College Station, TX, USA). ¹⁶ Our NMA model is a linear mixed model that combines direct and indirect evidence to compare multiple treatments. To enhance the clinical application of our results, we ranked the treatment probability for the analysis of each outcome to calculate the surface under the cumulative ranking curve (SUCRA). The SUCRA represents the cumulative probability of a specified treatment being superior to a hypothetical treatment, thus indicating the statistical superiority of the specified treatment. We evaluated potential inconsistencies between the direct and indirect evidence using a loop-specific approach, node-splitting method and the design-by-treatment interaction mode. The inconsistency test was another method to test the transitivity assumption. In addition, we evaluated the heterogeneity of tau values among the main results of this study. In the current NMA, we conducted subgroup analyses to eliminate the potential impact of confounding factors. We conducted subgroup analyses considering three aspects: epidemiological factors (age-stratification subgroup); pandemic history progression (subgroup of different publication years); and statistical considerations (subgroup of RCTs without any zero events and subgroup of RCTs with a placebo arm). Specifically, we performed subgroup analysis based on different age-stratification subgroups, that is, mean age less than 60 years versus more than 60 years. Moreover, to account for the potential fluctuation of SoC during the COVID-19 pandemic, along with the wide promotion of COVID-19 vaccine and newly developed anti-viral treatment strategy, we conducted a subgroup analysis according to the publication year of the included RCTs. We compared the difference between RCTs published in 2019 and 2020 versus those published in 2021, 2022 and 2023. This is because the global case fatality rate from COVID-19 dropped significantly since early 2021 ¹⁷ due to the widespread promotion of the COVID-19 vaccine and newly developed anti-viral treatment strategies in late 2020. ¹⁸ Furthermore, we conducted a sensitivity analysis by subgroup restricting to RCTs without any zero events. We conducted another sensitivity analysis by subgroup restricting to RCTs with a placebo arm. We assessed the transitivity assumption by visually evaluating the distribution of the effects of each treatment arm. We conducted an additional assessment of transitivity by comparing the average effects of the common reference treatment in different groups of trials (known as designs in NMA literature) using Comprehensive Meta-Analysis (version 3; Biostat, Englewood, NJ, USA). ¹⁹ If there was no statistically significant difference, the assumption of transitivity was considered unlikely to be violated.

Results

Eligibility of the retrieved studies and treatment arms

Figure 1 shows a flowchart of this NMA. After the initial screening procedure, 53 articles were considered for the full-text review, of which 39 were excluded for various reasons (eTable 3). Ultimately, 14 RCTs were included in this study. Figure 2 shows the overall network plot of treatment comparisons for all-cause mortality between JAK/BTK inhibitors, placebo plus SoC and SoC alone.

Figure 1.

Flow diagram of this network meta-analysis.

Figure 2.

Network plot of the primary outcomes: all-cause mortality. The lines between nodes represent direct comparisons in various trials, and the size of each circle is proportional to the size of the population involved in each specific treatment. The thickness of the lines is proportional to the number of trials connected to the network.

Characteristics of the included studies

A total of 13,568 participants (mean age: 58.1 years, range: 46.5–72.5 years; mean female proportion: 35.5%, range: 23.3%–45.6%) were included (Table 3). The mean treatment duration was 2.0 weeks (range: 2.0–4.0 weeks), and the mean overall study duration (treatment and follow-up duration) was 4.2 weeks (range: 4.0–13.0 weeks). All the included RCTs recruited subjects with treatment-naïve patients. With regard to the severity of COVID-19 infection among the recruited participants in the included RCTs, all the RCTs recruited their participants in moderate-to-severe severity of COVID-19 infection in their inclusion criteria.

Table 3.

Characteristics of the included studies.

Study name	Design	Diagnosis	Comparison	Subjects	Mean age (years)	Women (%)	Tx duration (day)	Primary outcomea	Country	Registry
Troseid et al. 20	Placebo-controlled	Eligible adults infected with severe/critical COVID-19	Baricitinib 4 mg dailyPlacebo + standard of care	139136	59.060.0	19.427.2	14	1418	Multiple countries	NCT04891133
Coutre et al. (iNSPIRE) 21	Placebo-controlled	SARS-CoV-2 infection confirmed by reverse transcription PCR test	Ibrutinib 420 mg once dailyPlacebo + standard of care	2224	48.554.5	27.333.3	28	10	USA	NCT04375397
Ely et al. (COV-BARRIER-NIAID-OS-7) 15	Placebo-controlled	SARS-CoV-2 infection, confirmed by PCR test	Baricitinib 4 mg dailyPlacebo + standard of care	5150	58.4 ± 12.458.8 ± 15.2	51.040.0	14	2029	Multiple countries	NCT04421027
Han et al. (RUXCOVID) 22	Placebo-controlled	SARS-CoV-2 infection confirmed by PCR test or another rapid test	Ruxolitinib 10 mg/dayPlacebo + standard of care	286145	56.456.9	43.649.7	14	93	Multiple countries	NCT04362137
Karampitsakos et al. 23	Open-label	Positive PCR test result for SARS-CoV-2	Baricitinib 4 mg dailyStandard of care alone	125126	73.072.0	40.841.3	14	4050	Greece	NCT05082714
Montejano et al. (PANCOVID) 24 no TDF/FTC part	Open-label	SARS-CoV-2 detected by PCR or antigenic test	Baricitinib 4 mg dailyStandard of care alone	7571	NA	NA	10	13	Spain	EudraCT: 2020-001156-18
Murugesan et al. 25	Open-label	18–65 y/o adults, positive for SARS-CoV2 infection by real-time PCR	Tofacitinib 20 mg/dayStandard of care alone	5050	47.046.0	24.028.0	14	00	India	NA
RECOVERY 26	Open-label	Clinically suspected or laboratory-confirmed SARS-CoV-2 infection	Baricitinib 4 mg dailyStandard of care alone	41484008	58.5 ± 15.457.7 ± 15.5	33.934.2	10	514546	UK	NCT04381936
Rein et al. (RUXCOVID-DEVENT) 27	Placebo-controlled	SARS-CoV-2 infection confirmed	Ruxolitinib 30 mg/dayRuxolitinib 10 mg/dayPlacebo + standard of care	778547	63.6 ± 12.963.6 ± 12.362.5 ± 13.3	19.536.827.7	14	394533	Multiple countries	NCT04377620
Wolfe et al. (ACTT-4) 28	Placebo-controlled	Adults with laboratory-confirmed SARS-CoV-2 infection	Baricitinib 4 mg dailyPlacebo + standard of care	516494	58.2 ± 14.358.5 ± 13.7	41.941.3	14	2730	Multiple countries	NCT04640168
Guimaraes et al. (STOP-COVID) 29	Placebo-controlled	SARS-CoV-2 infection as determined by PCR	Tofacitinib 20 mg/dayPlacebo + standard of care	144145	55.0 ± 14.057.0 ± 14.0	34.735.2	14	48	Brazil	NCT04469114
Kalil et al. (ACTT-2) 30	Placebo-controlled	18 y/o or older adults, laboratory-confirmed SARS-CoV-2 infection	Baricitinib 4 mg dailyPlacebo + standard of care	515518	55.0 ± 15.455.8 ± 16.0	38.135.7	14	2437	Multiple countries	NCT04401579
Marconi et al. (COV-BARRIER-NIAID-OS-5-6) 14	Placebo-controlled	SARS-CoV-2 infection, confirmed by PCR test	Baricitinib 4 mg dailyPlacebo + standard of care	764761	57.8 ± 14.357.5 ± 13.8	35.937.8	14	62100	Multiple countries	NCT04421027
Cao et al. 31	Placebo-controlled	COVID-19 diagnosed according to the Chinese management guideline	Ruxolitinib 10 mg/dayPlacebo + standard of care	2021	63.064.0	40.042.9	28	03	China	NA

COVID-19: coronavirus disease 2019; NA: not available; PCR: polymerase chain reaction SARS-CoV-2: severe acute respiratory syndrome coronavirus 2; Tx: treatment.

^a28-day all-cause mortality.

Primary outcome: all-cause mortality

Compared with the SoC group, only baricitinib was associated with a significantly lower all-cause mortality rate (OR = 0.88, 95% CIs = 0.78–1.00). In contrast, only placebo plus SoC was associated with a significantly higher all-cause mortality rate than SoC alone (OR = 1.39, 95% CI = 1.07–1.79) (Figure 3 and Table 1). According to the SUCRA, high-dose ruxolitinib was ranked best among all interventions (OR = 0.70, 95% CI = 0.33–1.50 compared with the SoC group) (Tables 1 and 2).

Figure 3.

Forest plot of the all-cause mortality in reference to standard of care. The indicated treatment was associated with a significantly lower 28-day all-cause mortality rate than the standard of care alone if the odds ratio is less than 1.95% CI: 95% confidence interval; ADHD: attention-deficit hyperactivity disorder; Bar: baricitinib; BTK: Bruton tyrosine kinase; COVID-19: coronavirus disease 2019; hiRux: ruxolitinib high dosage; Ibr: ibrutinib; JAK Janus kinase; lowRux: ruxolitinib low dosage; NMA: network meta-analysis; OR: odds ratio; Pla: placebo + SoC; RCT: randomised controlled trial; SoC: standard of care; SUCRA: surface under the cumulative ranking curve; Tof: tofacitinib.

Table 1.

League table of all-cause mortality rate.

			League table
hiRux			0.91 (0.49,1.69)		0.44 (0.20,0.94)*
1.03 (0.25,4.26)	Tof				0.49 (0.14,1.66)
0.79 (0.37,1.68)	0.77 (0.22,2.66)	Bar		0.88 (0.78,1.00)*	0.64 (0.51,0.80)*
0.85 (0.47,1.56)	0.83 (0.21,3.28)	1.07 (0.54,2.12)	lowRux		0.63 (0.23,1.72)
0.70 (0.33,1.50)	0.68 (0.19,2.37)	0.88 (0.78,1.00)*	0.82 (0.41,1.64)	SoC
0.50 (0.25,1.04)	0.49 (0.14,1.66)	0.64 (0.51,0.80)*	0.59 (0.31,1.13)	0.72 (0.56,0.94)*	Pla	0.29 (0.01,7.58)
0.15 (0.01,4.13)	0.14 (0.00,4.62)	0.19 (0.01,4.86)	0.17 (0.01,4.77)	0.21 (0.01,5.52)	0.29 (0.01,7.56)	Ibr

Pairwise (upper-right portion) and network (lower-left portion) meta-analysis results are presented as estimate effect sizes for the outcome of all-cause mortality rate. Interventions are reported in order of mean ranking of all-cause mortality rate, and outcomes are expressed as OR (95% confidence interval). For the pairwise meta-analyses, OR of less than 1 indicates that the treatment specified in the row got less all-cause mortality rate than that specified in the column. For the NMA, OR of less than 1 indicates that the treatment specified in the column got less all-cause mortality rate than that specified in the row. Bold results marked with * indicate statistical significance.

Bar: baricitinib; BTK: Bruton tyrosine kinase; hiRux: ruxolitinib high dosage; Ibr: ibrutinib; lowRux: ruxolitinib low dosage; Pla: placebo + SoC; SoC: standard of care; Tof: tofacitinib.

Table 2.

SUCRA of the all-cause mortality rate.

Treatment	SUCRA	OR (95% CI) in comparison with SoC
hiRux	76.4	0.70 (0.33,1.50)
Tof	70.0	0.68 (0.19,2.37)
Bar	64.1	0.88 (0.78,1.00) *
lowRux	63.6	0.82 (0.41,1.64)
SoC	43.0	Reference
Pla	16.6	1.39 (1.07,1.79) *
Ibr	16.3	4.73 (0.18,123.70)

Sorted by efficacy order (the former, the less all-cause mortality rate)

*Achieved statistical significance.

95% CI: 95% confidence interval; Bar: baricitinib; hiRux: ruxolitinib high dosage; Ibr: ibrutinib; lowRux: ruxolitinib low dosage; OR: odds ratio; Pla: placebo + SoC; SoC: standard of care; SUCRA: surface under the cumulative ranking curve; Tof: tofacitinib.

Regarding the cause of mortality, only a limited number of RCTs provided detailed information, which varied among those RCTs. Therefore, additional analysis of the cause of mortality was not feasible. The most frequently reported cause of mortality was ‘death due to adverse events’, which ranged from 4.0% to 11.1% in placebo-controlled RCTs, whereas the information of ‘death due to adverse events’ was not reported in any open-label RCTs (i.e. RCTs of SoC only). Rather, in the RCTs of SoC only, the most frequent cause of death was COVID-19 infection (12.6%).

Sensitivity analysis by excluding RCTs without zero events in any arm

Compared with the SoC group, only the placebo plus SoC group was associated with a significantly higher all-cause mortality rate than the SoC group (OR = 1.39, 95% CI = 1.07–1.79) (eFigures 1A and 2A, eTables 4A and 5A).

Sensitivity analysis restricted to RCTs with a placebo-control arm

Only baricitinib was associated with a significantly lower all-cause mortality rate than the placebo plus SoC group (OR = 0.64, 95% CI = 0.51–0.80) (eFigures 1B and 2B, eTables 4B and 5B).

Sensitivity analysis by subgrouping RCTs by published date

Because only one RCT was published between 2019 and 2020, we could not perform a subgroup analysis. The result of the subgroup analysis of RCTs published between 2021 and 2023 yielded the same findings that only placebo plus SoC was associated with a significantly higher all-cause mortality rate than SoC alone (OR = 1.39, 95% CI = 1.07–1.79) (eFigures 1C and 2C, eTables 4C and 5C).

Sensitivity analysis by subgrouping RCTs by mean age-stratification

We could not perform a subgroup analysis on those over the age of 60 years, as there were only four RCTs in this age group. However, the result of the subgroup analysis of RCTs with an average age of below 60 years yielded the same findings that only placebo plus SoC was associated with a significantly higher all-cause mortality rate than SoC alone (OR = 1.41, 95% CI = 1.08–1.83) (eFigures 1D and 2D, eTables 4D and 5D).

Assessment of transitivity assumption: comparing the efficacy of JAK/BYK inhibitors in studies with the placebo plus SoC or SoC as the control

The network plot of the primary outcome showed that the indirect comparison between placebo plus SoC vs. SoC only mainly came from the loops of ‘placebo plus SoC versus baricitinib versus SoC only’. Therefore, we will evaluate the transitivity assumption for this treatment in two RCT designs (eFigure 1E).

Specifically, the OR for all-cause mortality in the baricitinib groups was 0.098 (95% CI = 0.050–0.183) in the placebo plus SoC RCTs, and 0.130 (95% CI = 0.047–0.313) in the SoC-only RCTs. The difference in the ORs of these two baricitinib groups was very small and non-significant (p = 0.639) (eFigure 2E). Therefore, there is currently insufficient evidence to support a violation of the transitivity assumption.

To further assess the transitivity assumption, we created a scatterplot (eFigure 2F) illustrating the distribution of the odds of each treatment arm for all-cause mortality. The x-axis represents the study ID and the y-axis displays the odds on the natural log scale. Each treatment arm is denoted by a symbol whose size is inversely proportional to its sampling error. Red circles represent SoC, orange triangles represent placebo and green crosses represent baricitinib. The scatterplot revealed that the odds of all-cause mortality for baricitinib in the three trials (marked by BLUE rectangles) comparing it with SOC were similar to those for baricitinib in the five trials (marked by RED rectangles) comparing it with placebo. The only exception was study ID 10, which had a large sampling error and a small weight.

Sensitivity analysis by excluding RECOVERY trials

Because the RECOVERY trial had the largest sample sizes (8156 subjects) and the broadest COVID-19 severity range, ²⁶ we conducted a sensitivity analysis by excluding this trial. The odds ratio of SoC compared with other treatments for all-cause-mortality became non-significant (eFigure 2G).

Secondary outcome: new invasive or non-invasive ventilation usage

None of the investigated treatments were associated with significantly different risks of ventilation usage compared with the SoC group (eFigures 1F and 2H, eTables 4E and 5E).

Secondary outcome: secondary bacterial infection

None of the investigated treatments were associated with a significantly different risk of secondary bacterial infection compared with the SoC group (eFigures 1G and 2I, eTables 4F and 5F).

Secondary outcome: acceptability in terms of drop-out rate

None of the investigated treatments were associated with significantly different drop-out rates compared with the SoC group (eFigures 1H and 2J, eTables 4G and 5G).

Secondary outcome: serious adverse events

None of the investigated treatments were associated with a significantly different risk of serious adverse events (SAEs) compared with the SoC group (eFigures 1I and 2K, eTables 4H and 5H).

Risk of bias and publication bias

Of the 91 items, 63 (69.2%), 19 (20.9%) and 9 (9.9%) of the included studies had an overall low, unclear and high risk of bias, respectively. The unblinding and concealing procedures after randomisation were the main contributors to the high/unclear risk of bias (eFigures 3).

Funnel plots of publication bias and Egger’s test across the included studies (eFigures 4A–J) revealed general symmetry and no significance among the included studies in the NMA. The inconsistency test revealed no significant inconsistencies in this NMA (eTable 6). In addition, there was no significant heterogeneity, defined by the tau value, in the main results of this study (eTable 7). The results of the GRADE evaluation are listed in eTable 8. Briefly, the overall quality of the evidence ranged from low to medium.

Discussion

This NMA suggested that patients with COVID-19 treated with placebo plus SoC appeared to show a statistically significantly higher 28-day all-cause mortality odds than those receiving SoC alone. This difference remained consistent in subgroup analyses stratified by epidemiological factor, pandemic history progression and statistical consideration.

The most intriguing finding of this NMA is the higher odds of all-cause mortality in the placebo arm compared with the SoC arm. This is particularly noteworthy, given that all-cause mortality is considered an objective measure, but not a subjective outcome or minor symptoms, such as pain ³² and cough. ³³ Furthermore, this difference remained consistent in the subgroup and sensitivity analyses.

There were four possible explanations for this difference in mortality in RCTs on the treatment of COVID-19.

Alteration of the adherence related to placebo use

A previous meta-analysis ³⁴ found that the mortality of patients receiving placebo with good adherence was similar to that of patients receiving beneficial drug therapy. ³⁴ However, in our study, none of the included RCTs provided information regarding the association between adherence and mortality in individual patients. Furthermore, the acceptability of placebo plus SoC was similar to SoC alone, without a statistically significant difference.

Immunomodulatory effect by the placebo prescription

Placebo administration has been suggested to trigger endogenous neurotransmitter release, consequently mimicking pharmacological effects. ³⁵ This is supported by observations during the treatment of asthma ³⁶ and chronic cough. ³³ However, we did not observe the presence of such pharmacological mimicry, because there were no significant differences in some secondary outcomes (e.g. secondary bacterial infection) or safety profile (e.g. SAEs) between the SoC plus placebo group and the SoC group. Secondary infection and SAEs were considered the indicators of biophysiological harm related to active ingredients.

Behavioural modification associated with placebo administration

The application of placebo could affect the behaviours of the parents of children with attention-deficit hyperactivity disorder. ⁷ Further, the attendance in a clinical trial and reception of placebo would contribute to increased medical attention and result in participants’ behaviour change, which is the Hawthorne effect. ³⁷ The use of placebo in a statin trial might lead to the false belief that patients could ‘eat anything and stop exercising’, which resulted in worse outcomes than that of patients on the waiting list. ⁸ In one systematic review of cardiovascular study regarding outcome of all-cause mortality, ⁹ the authors confirmed that the mortality rate in the placebo arm of the study of peripheral arterial disease was increased in comparison with the average mortality in the ordinary population with peripheral arterial disease. ¹⁰ That is why, recently, some researchers recommended using the ‘best-available-therapy’ as the control group, rather than placebo. ³⁸

Alteration of prognosis because of assignment to SoC-only group

Some previous evidence on the preferences and performance of the patients in an SoC-only arm ³⁹ of a weight loss trial suggests that assignment to an SoC-only arm could disappoint patients and promote behaviour change that could subsequently affect their prognosis. Another trial of alcohol abstinence also showed the efficacy of ‘assignment to the waiting-list or SoC-only group’, ⁴⁰ which resulted in 20% decreased alcohol consumption.

Strengths and limitations

Our NMA has several strengths to be addressed. First, the distinction between SoC-only and placebo plus SoC groups was made, and this could not be done in a traditional pairwise meta-analysis. Second, we included only RCTs to reduce the potential biases associated with observational studies. Third, we undertook sensitivity analyses by different subgroup analysis to re-appraise the results of this study, which found no evidence of intransitivity.

This NMA also has several limitations. First, some comparisons in our NMA were underpowered because of the heterogeneity in the characteristics of the participants and the small number of trials. Second, when the RECOVERY trial, which had the largest sample size, was excluded from the sensitivity analysis, the difference in the odds of all-cause mortality between patients receiving placebo plus SoC and those receiving SoC alone became smaller and non-significant. This may have been related to disease severity because the RECOVERY trial recruited both moderately and severely ill patients. ²⁶ Finally, we distinguished between placebo plus SoC and SoC alone and found a significant difference in the mortality between the two treatment groups. However, this finding is based entirely on indirect evidence, because they have never been directly compared in a trial. Although the transitivity assumption did not appear to be violated, these results should be interpreted with caution.

Implications

This NMA of RCTs found a statistically significant difference in all-cause mortality between COVID-19 patients treated with SoC plus placebo and those treated with SoC alone.

Conclusion and interpretation

In this NMA, we found that COVID-19 patients treated with baricitinib showed a significantly lower 28-day all-cause mortality compared with those treated with SoC. We also found that patients treated with SoC plus placebo showed a higher mortality than those treated with SoC alone. Therefore, future RCTs of COVID-19 treatments should re-evaluate the use of placebo and consider alternative study designs to minimise potential biases.

Declarations

Competing Interests

None declared.

Funding

Brendon Stubbs holds an NIHR Advanced fellowship (NIHR301206, 2021–2026) for unrelated work. Brendon has received honoraria for co-editing a book on exercise and mental illness, as well as for advisory work from ASICS, ParachuteBH and FitXR, all unrelated to this article. The views expressed are those of the author(s) and not necessarily those of mentioned above, the NHS, the NIHR, the Department of Health and Social Care, the MRC or GSTT. Yu-Kang Tu is supported by a grant (NSTC 112-2314-B-002-210-MY3) from the National Science & Technology Council in Taiwan.

Ethics approval

The current study was approved by the Institutional Review Board of the Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan (TSGHIRB No. B-109–29).

Guarantor

P-TT and C-WH.

Contributorship

Ping-Tao Tseng, who contributed equally as one of the first authors, took the whole responsibility for the literature search, data curation, data analysis and manuscript draft. Bing-Syuan Zeng, who contributed equally as one of the first authors, took the whole responsibility for the literature search, data extraction, interpretation of analysis results and manuscript draft. Chih-Wei Hsu, who contributed equally as one of the first authors, took full responsibility for data extraction, data analysis, interpretation of analysis results and manuscript draft. Trevor Thompson contributed significantly to study design, concept formation, investigation, methodology designation, validation of data, resources/software and manuscript major revision. Brendon Stubbs contributed significantly to study design, investigation, methodology designation, project administration, validation of data, resources/software and manuscript major revision. Po-Ren Hsueh contributed significantly to study design, concept formation, data curation, formal analysis, investigation, methodology designation and manuscript major revision. Kuan-Pin Su contributed significantly to data curation, formal analysis, investigation, methodology designation, validation of data, resources/software and manuscript major revision. Yen-Wen Chen contributed significantly to study design, concept formation, data curation, formal analysis, validation of data, resources/software and manuscript major revision. Tien-Yu Chen contributed significantly to concept formation, data curation, formal analysis, investigation, methodology designation, project administration and manuscript major revision. Yi-Cheng Wu contributed significantly to study design, concept formation, data curation, formal analysis, investigation, validation of data, resources/software and manuscript major revision. Pao-Yen Lin contributed significantly to formal analysis, investigation, methodology designation, project administration, validation of data, resources/software and manuscript major revision. Andre F Carvalho contributed significantly to study design, concept formation, investigation, methodology designation, project administration and manuscript major revision. Dian-Jeng Li contributed significantly to study design, concept formation, data curation, formal analysis, investigation, methodology designation and manuscript major revision. Ta-Chuan Yeh contributed significantly to study design, concept formation, data curation, formal analysis, investigation and manuscript major revision. Cheuk-Kwan Sun contributed significantly to methodology designation, project administration, validation of data, resources/software and manuscript major revision. Yu-Shian Cheng contributed significantly to study design, concept formation, investigation, methodology designation, project administration and manuscript major revision. Yow-Ling Shiue contributed significantly to study design, concept formation, investigation, methodology designation and manuscript major revision. Chih-Sung Liang, who contributed equally as one of the corresponding authors, took full responsibility for communication, manuscript major revision, project administration, validation of data, resources/software and manuscript submission. Yu-Kang Tu, who contributed equally as one of the corresponding authors, took full responsibility for communication, manuscript major revision, data curation, formal analysis, validation of data, resources/software and manuscript submission.

Data availability statement

All the research data could be available upon reasonable request.

Acknowledgements

This article presents independent research. The views expressed in this publication are those of the authors and not necessarily those of the acknowledged institutions.

Provenance

Not commissioned; reviewed by Robin Ferner, Sun Jae Park and Julie Morris.