Skip to main content
PMC home page
Elsevier Sponsored Documents logoLink to Elsevier Sponsored Documents
. 2021 Aug;109:114–117. doi: 10.1016/j.ijid.2021.06.034

Is convalescent plasma futile in COVID-19? A Bayesian re-analysis of the RECOVERY randomized controlled trial

FW Hamilton a,b,⁎, Todd Lee c, DT Arnold d, R Lilford e, K Hemming e
PMCID: PMC8214317  PMID: 34157385

Highlights

  • •

    The RECOVERY trial reported that convalescent plasma had no effect on mortality in COVID-19.

  • •

    A re-analysis of the data using Bayesian methods suggests there is a real possibility of benefit.

  • •

    The probability of any benefit of plasma in seronegative patients in this trial was more than 75%.

Keywords: serology, convalescent plasma, antibody, COVID-19

Abstract

Background

Randomized trials are generally performed from a frequentist perspective, which can conflate absence of evidence with evidence of absence. The RECOVERY trial evaluated convalescent plasma for patients hospitalized with coronavirus disease 2019 (COVID-19) and concluded that there was no evidence of an effect. Re-analysis from a Bayesian perspective is warranted.

Methods

Outcome data were extracted from the RECOVERY trial by serostatus and time of presentation. A Bayesian re-analysis with a wide variety of priors (vague, optimistic, sceptical, and pessimistic) was performed, calculating the posterior probability for: any benefit, an absolute risk difference of 0.5% (small benefit, number needed to treat 200), and an absolute risk difference of one percentage point (modest benefit, number needed to treat 100).

Results

Across all patients, when analysed with a vague prior, the likelihood of any benefit or a modest benefit with convalescent plasma was estimated to be 64% and 18%, respectively. The estimated chance of any benefit was 95% if presenting within 7 days of symptoms, or 17% if presenting after this. In patients without a detectable antibody response at presentation, the chance of any benefit was 85%. However, it was only 20% in patients with a detectable antibody response at presentation.

Conclusions

Bayesian re-analysis suggests that convalescent plasma reduces mortality by at least one percentage point among the 39% of patients who present within 7 days of symptoms, and that there is a 67% chance of the same mortality reduction in the 38% who are seronegative at the time of presentation. This is in contrast to the results in people who already have antibodies when they present. This biologically plausible finding bears witness to the advantage of Bayesian analyses over misuse of hypothesis tests to inform decisions.

1. Introduction

Convalescent plasma (CP) – blood components from patients who have recovered from an infection – has been used for more than a century to treat infections, with widespread use in the 1920s and 1930s for pneumococcal infections and scarlet fever, before falling out of favour with the development of antibiotics (The Lancet Haematology, 2020). The principle is that of ‘passive immunization’, i.e., passing antibodies from those who have recovered from the infection to those naïve to it, thereby providing a degree of protection from that specific agent (Keller and Stiehm, 2000). It is therefore unsurprising that interest in the use of CP to prevent and treat coronavirus disease 2019 (COVID-19) has been widespread (The Lancet Haematology, 2020). Unfortunately, despite best efforts, most of this usage has occurred outside of randomized controlled trials (RCT), with >100 000 doses given in the United States alone (FDA, 2021).

Fortunately, the RECOVERY collaborative group have recently reported the largest RCT of CP in hospitalized patients with COVID-19 (The RECOVERY Collaborative Group et al., 2021). The authors concluded that CP provided no benefit, with the observed mortality equal in both arms: 1399 (24%) of 5795 patients allocated to CP and 1408 (24%) of 5763 patients allocated to usual care died within 28 days (rate ratio (RR) 1.00, 95% confidence interval (CI) 0.93–1.07; P = 0.95). They also concluded that there was no difference across pre-specified subgroups, including those with detectable severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) antibody tests at the time of randomization (seropositive group) (19% versus 18%; RR 1.06, 95% CI 0.94–1.19) and seronegative patients (32% versus 34%; RR 0.96, 95% CI 0.85–1.07), with test for interaction P = 0.23. In particular, they noted, on the advice of the Drug Safety and Monitoring Committee (DMC), that: “there was no convincing evidence that further recruitment would provide conclusive proof of worthwhile mortality benefit either overall or in any pre-specified subgroup”. In the United Kingdom, the data have been taken by the regulator as strong evidence of a null effect, leading the Medicines Health Regulatory Authority (MHRA (CAS-ViewAlert 2021); the UK medicines regulator) to recommend against the use of CP in patients hospitalized with COVID-19, effectively removing the therapy in the National Health Service (NHS), with many editorials agreeing with the authors that this proves no effect (Liu and Aberg, 2021).

Before accepting that CP is ineffective in hospitalized patients, it is important to recognize the clear distinction between patients who are likely to benefit and those who are not. The therapeutic mechanism of CP and monoclonal antibody (e.g., REGN-COV2) treatments is passive immunization – the gifting of antibodies. These antibodies (donated by recovered patients) develop in most people by 7–10 days, as part of the normal immune response. It is not surprising to think that the greatest (or any) benefit of CP would only occur in patients who present early or are seronegative, or conversely, that there will be little to no benefit in giving antibodies to those who already have antibodies or have developed their own immune response. The previous literature on severe acute respiratory syndrome (SARS) supports this distinction (Cheng et al., 2005; Yeh et al., 2005), as well as data clearly identifying a protective effect of monoclonal antibodies (manufactured antibodies, rather than donated) in early COVID-19 trials, with much weaker effects in hospitalized patients later in the disease course (ACTIV-3/TICO LY-CoV555 Study Group et al., 2021; Chen et al., 2021; O'Hare, 2021; Weinreich et al., 2021). Immunological data and cases of persistent infection show that failure of an early antibody response is associated with both severe disease and, in patients without any antibodies, the risk of persistent disease (Kemp et al., 2021; Sette and Crotty 2021). Others have also argued that seropositivity is a reason for failure of CP (Bajpai et al., 2020).

On that background, it is logical to analyse the data from patients who are seronegative (hypothesized more likely to benefit) separately from those who are seropositive (hypothesized less likely to benefit). Likewise, it is rational to analyse the data by time from symptom onset, given that the only positive trial of CP occurred with treatment given within 72 hours (Libster et al., 2021).

Although subgroup analyses can be complicated by chance imbalances, lower power, and issues of multiple testing, they are appropriate to generate hypotheses and could be used in support of the argument of not disregarding CP as a potential treatment too soon (Jones et al., 2011; Lee et al., 2021). Moreover, conflating absence of evidence for a small effect with evidence of no effect further risks discarding a therapy that could still have a meaningful benefit. It was therefore sought to undertake a Bayesian re-analysis to estimate the probability of (a) any benefit, (b) a small benefit (defined here as equivalent to a number needed to treat (NNT) of at most 200), and (c) a modest benefit (equivalent of a NNT of at most 100) for all patients and for both subgroups specified above.

2. Methods

The intention-to-treat results from the RECOVERY trial were extracted, both overall and for pre-specified subgroups: seronegative, seropositive, ≤7 days since symptom onset, and >7 days since symptom onset. These two subgroups (antibody status and time from onset) were selected on the basis of the scientific justification described above. No granular data were available to combine these two subcategories.

The ‘Bayes’ function Stata version 16 (StataCorp, College Station, TX, USA) was used to calculate posterior probabilities. The probabilities of (a) any benefit (Odds Ratio, OR <1), (b) a small but arguably clinically important benefit, estimated as an absolute risk difference of at least 0.5% (Number Needed to Treat,NNT ≤200), and (c) a modest benefit, which was defined here as a risk difference of at least 1% (NNT ≤100) were calculated. These risk differences were chosen after internal discussion between the study authors regarding what would be considered an important effect size considering the complexity and challenges in using CP. By nature, they are subjective, but reflect effect sizes that might be salient to patients, their families, and clinicians.

As suggested by a recent review on Bayesian re-analysis in COVID-19 (Zampieri et al., 2021), four probability assumptions were chosen to account for varying prior views: (1) vague (no information; mean risk difference (RD) 0, standard deviation (SD) 10 000); (2) optimistic (10% risk of harm; mean RD 0.01, SD 0.007); (3) sceptical (tightly around the null; mean RD 0, SD 0.007); (4) pessimistic (10% chance of benefit; mean RD –0.005, SD 0.0036). Posterior probabilities were computed from binomial regression models. Posterior density function graphs were produced for each prior assumption.

All code used to generate these figures is available in the Supplementary Material.

3. Results

Table 1 presents the posterior probabilities of benefit for each prior.

Table 1.

Estimated posterior probabilities of benefit for a variety of prior assumptions

Vague prior Optimistic prior Sceptical prior Pessimistic prior
Whole trial (n = 11 558)
Any benefit 64% 65% 64% 62%
Small benefit 43% 41% 40% 38%
Modest benefit 20% 19% 19% 18%
Seronegative subgroup (n = 3676)
Any benefit 86% 87% 86% 86%
Small benefit 79% 78% 77% 78%
Modest benefit 68% 68% 66% 67%
Seropositive subgroup (n = 5888)
Any benefit 20% 23% 21% 21%
Small benefit 9% 11% 11% 10%
Modest benefit 3% 4% 4% 4%
≤7 days since symptom onset (n = 4466)
Any benefit 95% 95% 95% 95%
Small benefit 90% 90% 90% 90%
Modest benefit 80% 82% 82% 80%
>7 days since symptom onset (n = 7086)
Any benefit 17% 17% 17% 17%
Small benefit 7% 8% 7% 7%
Modest benefit 3% 3% 2% 2%
Open in a new tab

Vague prior: N(0, SD = 10 000); optimistic prior: N(0.01, SD = 0.007); sceptical prior: N(0, SD = 0.007); pessimistic prior N(−0.01, SD = 0.0036). Small benefit defined as a risk difference >0.5% (equivalent to a NNT ≤200); Moderate benefit defined as a risk difference >1% (equivalent to a NNT ≤100). SD, standard deviation; NNT, number needed to treat.

Across the whole trial population, the estimated chance of any benefit was found to be around 65%, with little difference across all prior assumptions. The posterior probability of a modest benefit (preferring treatment arm) was found to be around 19% across all prior assumptions. The associated posterior density functions are available in the Supplementary Material, as supplementary figures.

In the seronegative subgroup, the estimated likelihood of any benefit was greater, at around 90%, across all prior assumptions. The estimated chance of a risk difference (modest benefit) of >1% was also high (more than 66% across all three priors), and varied little between prior assumptions. This contrasts with the seropositive arm, where the estimated chance of any benefit was only 20%, and with a very small (3%) chance of a modest benefit (NNT ≤100).

These results are mirrored in the early treatment subgroup, with an around 95% chance of benefit in patients treated within 7 days of symptom onset. The chance of a modest benefit (NNT ≤100) was about 80% across all prior assumptions. However, in patients who presented after 7 days, the chance of CP providing any benefit was small (17%), with a very low chance (~2%) of a modest benefit (NNT ≤100).

4. Discussion

The RECOVERY trial has been a paradigm for a rapid pragmatic approach to trialling new therapies in a pandemic. Good practice requires a firm pre-specified analysis plan with a clear pre-defined subgroup analysis (Schulz et al., 2010). However, the conclusions drawn by the RECOVERY collaborative group and the MHRA with respect to CP risks conflating absence of evidence of a small effect with evidence that there is no benefit. Re-analysis of the original data using Bayesian methods yielded a small probability (>15%) of an effect with an NNT of 100 across the whole trial, with even higher probabilities of 90% and 75%, respectively, in patients presenting within 7 days of symptoms and patients antibody-negative on presentation.

Patients in the population who presented early were easy to identify (from history alone) and constituted more than a third of the whole trial population. The estimated chance of a benefit with a NNT of 100 changed from ~7% in those presenting late to ~90% in those presenting early.

Many clinicians, patients, and their families might consider benefits in the region of one life saved for every 100 or 200 people treated as meaningful benefits. From a societal perspective, the treatment would need to achieve a mean of only one quality-adjusted life year to justify a £20 000 treatment cost. However, it is not our intention to prove that CP is a cost-effective treatment – at heart that is a value judgement. We wish only to show that the conclusion that the treatment is ineffective is unlikely to be true for people who have not developed immunity at the point where the therapeutic decision is made. It is always important to consider the literature in the round when making policy recommendations.

Previous trials have been small and underpowered, with a recent meta-analysis of evaluations of CP in COVID-19 identifying less than 2000 patients across all RCTs prior to RECOVERY (Janiaud et al., 2021). Only one previous trial of high-titre CP reported data based on antibody status (Simonovich et al., 2021), with 34/105 deaths (Rodionov et al., 2021) in the seronegative placebo arm and 65/228 deaths in the CP arm, a RR of death with CP of 1.12, but with very wide confidence intervals (95% CI 0.51–2.43). A further trial, which stopped early due to declining case numbers, recruited older adults within 72 hours of symptoms (Libster et al., 2021). In that study, 13/80 patients (16%) who received CP developed severe disease compared to 25/80 (31%) in the placebo arm, giving a RR of 0.52 (95% CI 0.29–0.94) in favour of CP. In a recent case series of 14 immunocompromised patients with COVID-19 who had no detectable SARS-CoV-2 IgG, transfusion of CP was associated with clinical improvement and the degree of clinical improvement correlated with the IgG titre post transfusion. Finally, the largest observational study in hospitalized patients (n = 3082), a US registry study (published after RECOVERY), identified a lower risk of death in patients transfused early with higher levels of SARS-CoV-2 IgG antibody. Taken in the round, the literature supports the present re-analysis of the RECOVERY data, showing a benefit of CP among immunologically naïve patients with COVID-19 (Joyner et al., 2021).

Yet further support for our conclusion can be found from secondary outcomes in the RECOVERY trial that we would expect to correlate with the primary outcome if the hypothesis that CP is particularly effective in immunologically naïve patients. Both secondary outcomes in the original study, i.e., discharge home by day 28 and invasive mechanical ventilation or death, showed heterogeneity with respect to serological status and intervention effect, with impressive P-values of 0.008 and 0.01, respectively, in favour of CP. Although we have not focused on this to avoid accusations of ‘cherry picking’ the data, this is entirely consistent with and supportive of a causal path by which CP reduces mortality, and both of these are critical outcomes relevant to both patients and clinicians.

It is recognized that there may have been chance imbalances in age or comorbidity within the seronegative or early subgroup of patients, since randomization was not stratified on serological status or time from presentation (The RECOVERY Collaborative Group et al., 2021). However, they were pre-specified subgroups and made up a substantial proportion of all participants as stated above. It is also recognized that both of these groups double counted a number of patients for the reason given above. An alternative approach would have been to create four non-overlapping groups: both (serologically positive and late presentation), neither, and two either groups. However, these groups could not be constructed because we did not have the raw data. In any event, both serological negativity and early presentation tap into a shared pathway concerning the development of immunity, even if there are other pathways involved.

In conclusion, the RECOVERY trial for CP reported no benefit. Recognizing the changing literature since the trial started and using a variety of priors, we suggest the reporting of no effect may be premature. It remains plausible that CP has a small but clinically important effect on mortality in those who have not already developed an antibody response or who present early. It is clear that any effect is likely small, but we would argue that clinicians, scientists, and government agencies should review all trial data in totality, rather than regarding the null result as fixed.

Acknowledgments

Funding

No formal funding was used for this article, but FH's time was covered by the GW4 CAT Wellcome Doctoral training scheme. RJL is funded by the NIHR ARC West Midlands. This research was funded in whole, or in part, by the Wellcome Trust. For the purpose of Open Access, the author has applied a CC BY public copyright licence to any Author Accepted Manuscript version arising from this submission.

Ethical approval

No formal ethical approval was required for this work, as it is was a re-analysis of published data.

Conflict of interest

None of the authors have any conflict of interest to declare.

Footnotes

Supplementary material associated with this article can be found, in the online version, at doi:10.1016/j.ijid.2021.06.034.

Appendix. Supplementary materials

mmc1.docx (190.9KB, docx)

References

  1. ACTIV-3/TICO LY-CoV555 Study Group. Lundgren J.D., Grund B., Barkauskas C.E., Holland T.L., Gottlieb R.L. A Neutralizing Monoclonal Antibody for Hospitalized Patients with Covid-19. N Engl J Med. 2021;384(10):905–914. doi: 10.1056/NEJMoa2033130. Mar 11. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Bajpai M., Kumar S., Maheshwari A., Chhabra K., Kale P., Gupta A. 2020. Efficacy of convalescent plasma therapy compared to fresh frozen plasma in severely ill COVID-19 patients: A pilot randomized controlled trial.http://medrxiv.org/lookup/doi/10.1101/2020.10.25.20219337 [Internet]. bioRxiv. medRxivAvailable from: [Google Scholar]
  3. Chen P., Nirula A., Heller B., Gottlieb R.L., Boscia J., Morris J. SARS-CoV-2 Neutralizing Antibody LY-CoV555 in Outpatients with Covid-19. N Engl J Med. 2021;384(3):229–237. doi: 10.1056/NEJMoa2029849. Jan 21. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Cheng Y., Wong R., Soo Y.O.Y., Wong W.S., Lee C.K., Ng M.H.L. Use of convalescent plasma therapy in SARS patients in Hong Kong. Eur J Clin Microbiol Infect Dis. 2005;24(1):44–46. doi: 10.1007/s10096-004-1271-9. Jan. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. FDA . 2021. Convalescent Plasma COVID-19 Emergency Use Authorization.https://www.uscovidplasma.org/ [Internet][cited 2021 Apr 1]. Available from: [Google Scholar]
  6. Janiaud P., Axfors C., Schmitt A.M., Gloy V., Ebrahimi F., Hepprich M. Association of Convalescent Plasma Treatment With Clinical Outcomes in Patients With COVID-19: A Systematic Review and Meta-analysis. JAMA. 2021;325(12):1185–1195. doi: 10.1001/jama.2021.2747. Mar 23. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Jones H.E., Ohlssen D.I., Neuenschwander B., Racine A., Branson M. Bayesian models for subgroup analysis in clinical trials. Clin Trials. 2011;8(2):129–143. doi: 10.1177/1740774510396933. Apr. [DOI] [PubMed] [Google Scholar]
  8. Joyner M.J., Carter R.E., Senefeld J.W., Klassen S.A., Mills J.R., Johnson P.W. Convalescent Plasma Antibody Levels and the Risk of Death from Covid-19. N Engl J Med. 2021;384(11):1015–1027. doi: 10.1056/NEJMoa2031893. Mar 18. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Keller M.A., Stiehm E.R. Passive immunity in prevention and treatment of infectious diseases. Clin Microbiol Rev. 2000;13(4):602–614. doi: 10.1128/cmr.13.4.602-614.2000. Oct. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Kemp S.A., Collier D.A., Datir R.P., Ferreira I.A.T.M., Gayed S., Jahun A. SARS-CoV-2 evolution during treatment of chronic infection. Nature. 2021 doi: 10.1038/s41586-021-03291-y. [Internet]Feb 5Available from: [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Lee T.C., McDonald E.G., Butler-Laporte G., Harrison L.B., Cheng M.P., Brophy J.M. Remdesivir and systemic corticosteroids for the treatment of COVID-19: A Bayesian re-analysis. Int J Infect Dis. 2021;104:671–676. doi: 10.1016/j.ijid.2021.01.065. Mar. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Libster R., Pérez Marc G., Wappner D., Coviello S., Bianchi A., Braem V. Early high-titer plasma therapy to prevent severe Covid-19 in older adults. N Engl J Med. 2021;384(7):610–618. doi: 10.1056/NEJMoa2033700. Feb 18. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Liu S.T.H., Aberg J.A. Convalescent plasma in patients hospitalised with COVID-19. Lancet. 2021;397(10289):2024–2025. doi: 10.1016/S0140-6736(21)01064-3. May 29. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. O'Hare R. 2021. Blood plasma treatment has limited effect for sickest COVID-19 patients.https://www.imperial.ac.uk/news/211493/blood-plasma-treatment-limited-effect-sickest/ [Internet][cited 2021 Apr 1]. Available from: [Google Scholar]
  15. Rodionov R.N., Biener A., Spieth P., Achleitner M., Hölig K., Aringer M. Potential benefit of convalescent plasma transfusions in immunocompromised patients with COVID-19. Lancet Microbe. 2021;2(4):e138. doi: 10.1016/S2666-5247(21)00030-6. Apr. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Schulz K.F., Altman D.G., Moher D., CONSORT Group CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. BMJ. 2010;340:c332. doi: 10.1136/bmj.c332. Mar 23. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Sette A., Crotty S. Adaptive immunity to SARS-CoV-2 and COVID-19. Cell. 2021;184(4):861–880. doi: 10.1016/j.cell.2021.01.007. Feb 18. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Simonovich V.A., Burgos Pratx L.D., Scibona P., Beruto M.V., Vallone M.G., Vázquez C. A Randomized Trial of Convalescent Plasma in Covid-19 Severe Pneumonia. N Engl J Med. 2021;384(7):619–629. doi: 10.1056/NEJMoa2031304. Feb 18. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. The Lancet Haematology The resurgence of convalescent plasma therapy. Lancet Haematol. 2020;7(5):e353. doi: 10.1016/S2352-3026(20)30117-4. May. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. The RECOVERY Collaborative Group. Horby P.W., Estcourt L., Peto L., Emberson J.R., Staplin N. 2021. Convalescent plasma in patients admitted to hospital with COVID-19 (RECOVERY): a randomised, controlled, open-label.http://medrxiv.org/lookup/doi/10.1101/2021.03.09.21252736 platform trial [Internet]. bioRxiv. medRxiv;Available from. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Weinreich D.M., Sivapalasingam S., Norton T., Ali S., Gao H., Bhore R. REGN-COV2, a Neutralizing Antibody Cocktail, in Outpatients with Covid-19. N Engl J Med. 2021;384(3):238–251. doi: 10.1056/NEJMoa2035002. Jan 21. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Yeh K.-M., Chiueh T.-S., Siu L.K., Lin J.-C., Chan P.K.S., Peng M.-Y. Experience of using convalescent plasma for severe acute respiratory syndrome among healthcare workers in a Taiwan hospital. J Antimicrob Chemother. 2005;56(5):919–922. doi: 10.1093/jac/dki346. Nov. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Zampieri F.G., Casey J.D., Shankar-Hari M., Harrell F.E., Jr, Harhay M.O. Using Bayesian Methods to Augment the Interpretation of Critical Care Trials. An Overview of Theory and Example Reanalysis of the Alveolar Recruitment for Acute Respiratory Distress Syndrome Trial. Am J Respir Crit Care Med. 2021;203(5):543–552. doi: 10.1164/rccm.202006-2381CP. Mar 1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. CAS-ViewAlert [Internet]. [cited 2021 Apr 1]. Available from: https://www.cas.mhra.gov.uk/ViewandAcknowledgment/ViewAlert.aspx?AlertID=103152

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

mmc1.docx (190.9KB, docx)

RESOURCES