Natural unblinding of BCG vaccination trials

Graphical abstract

Abstract

There is significant public and clinical interest in the potential for Bacillus Calmette-Guérin (BCG) vaccination to protect against type 2 Severe Acute Respiratory Syndrome Coronavirus (SARS-CoV-2) induced COVID-19. This question could be best answered by blinded and placebo controlled clinical trials. However, a skin reaction occurs within days at the site of BCG injection, making it rather challenging to blind this vaccination. Here, we examined registered clinical trials in ClinicalTrials.gov on BCG against COVID-19 by October 9th 2020, and found that 94.7% of such trials were listed as placebo controlled (all with normal saline as placebo), and single to quadruple blinded. The mode of overcoming the natural unblinding by the BCG induced skin reaction was not clarified on the website in either of the trials. We conclude that detailed description of the strategy towards overcoming the BCG vaccination induced skin reaction associated unblinding hurdle will be important for the interpretation of the theoretically blinded COVID-19 directed clinical trials.

1. Introduction

A relatively large number of off-label Bacillus Calmette-Guerin (BCG) vaccination trials are registered, which aim to prevent type 2 Severe Acute Respiratory Syndrome Coronavirus (SARS-CoV-2) induced COVID-19. These trials recently received support by Evangelos J Giamarellos-Bourboulis et al. [1], who performed an intriguing clinical study on BCG vaccination against off-target (i.e. non-tuberculous) infections. The research group examined the effects of BCG (strain 1331; Intervax) vaccination compared to placebo (0.1 ml normal saline) on time interval to the first infection post hospital, and overall incidence of infections during a 12 month follow-up period in recently hospitalized elderly (>65y old) patients. Significant protection against respiratory tract infections of probable viral origin (hazard ratio 0.21, p: 0.013) was observed in the BCG group. The investigators connected these findings with plausible protection against SARS-CoV-2 by BCG vaccination, which exponentially heightens the impact of their results due to the ongoing COVID-19 pandemic. Such conclusion has been supported by Tomita, et al. [2], who, through predictive modelling of human leukocyte antigen (HLA) class I‐binding peptides, observed that cross-reactive T cells against SARS-CoV-2 could be generated by BCG vaccination.

Historical BCG vaccination and COVID-19 epidemiology derived conclusions about the long term, off target protection from this vaccination against SARS-CoV-2 are ambiguous [3]. Clinical observations and controlled trials, however, preceding the work of Giamarellos-Bourboulis, et al. indicated that acutely delivered BCG vaccination could potentially provide short term protection against off target bacterial and viral infections, especially in infants and children (reviewed in [4]). However, there was only one placebo controlled trial published prior to 2020, which examined BCG from this off-target antiviral immunity perspective. Leentjens, et al. [5], in a randomized double-blind placebo controlled trial, found that in 20 young adults, BCG vaccine significantly (p = 0.041) enhanced the immunogenicity of the 2009 pandemic influenza A (H1N1) vaccine, which administered 14 days after BCG vaccination, compared to 20 young adults who received normal saline as placebo. The group sample sizes, however, were low in this study allowing for false positive (type‐1 error) and false negative (type‐2 error) findings, which could have been corrected for with trial sequential analysis, for example.

Importantly, neither of the two publications on the placebo controlled trials listed above explained how double blinding (i.e. of patients and research personnel) could be achieved by using normal saline as placebo for BCG. Typically, a skin reaction occurs in 10–14 days at the site of the BCG injection and a permanent small scar develops in 3–6 months after the immunization in more than 85% of adults receiving the vaccine [6]. Therefore, it seems biologically not possible to blind BCG vaccination (especially with normal saline injection as placebo) in clinical investigations (see graphic abstract). Lack of appropriate blinding in randomized clinical trials may lead to significant bias [7], [8], [9], although this dogma has been recently challenged [7], [10], [11]. Based on these considerations, we decided to investigate registered clinical trials in ClinicalTrials.gov on BCG vaccination against COVID-19.

2. Methods

We searched publically available data on trials registered in clinicaltrials.gov with words “BCG” and “COVID-19”.

3. Results

As of October 9, 2020, there were 19 registered clinical trials, which aimed to examine the acute protective effects of BCG vaccination against COVID-19 morbidity and/or mortality, over an observation period of 6 months following injection, most commonly. Eighteen (18/19; 94.7%) were placebo controlled (Table 1 ), all with normal saline as placebo, and single to quadruple blinded per description. None of the trial narratives on the website clarified if and/or how exactly the researchers are planning to overcome the naturally occurring, dermal reaction induced biologic/physiologic unblinding of the subjects and/or research personnel.

Table 1.

Registered clinical trials in clinical ClinicalTrials.gov as of October 9, 2020 identified with “Covid19” and “BCG” search words, which examine acutely delivered BCG vaccination against Covid-19 morbidity and mortality. Only one (italic) was not placebo controlled. All placebo-controlled trials designate normal saline as placebo, and are described as single to quadruple blinded.

BCG Against Covid-19 for Prevention and Amelioration of Severity Trial (BAC to the PAST) https://ClinicalTrials.gov/show/NCT04534803

Reducing Health Care Workers Absenteeism in Covid-19 Pandemic Through BCG Vaccine https://ClinicalTrials.gov/show/NCT04328441

Performance Evaluation of BCG Vaccination in Healthcare Personnel to Reduce the Severity of… https://ClinicalTrials.gov/show/NCT04362124

BCG Vaccination for Healthcare Workers in COVID-19 Pandemic https://ClinicalTrials.gov/show/NCT04379336

Reducing COVID-19 Related Hospital Admission in Elderly by BCG Vaccination https://ClinicalTrials.gov/show/NCT04417335

Application of BCG Vaccine for Immune-prophylaxis Among Egyptian Healthcare Workers https://ClinicalTrials.gov/show/NCT04350931

Prevention Of Respiratory Tract Infection And Covid-19 Through BCG Vaccination In Vulnerable… https://ClinicalTrials.gov/show/NCT04537663

BCG Vaccine in Reducing Morbidity and Mortality in Elderly Individuals in COVID-19 Hotspots https://ClinicalTrials.gov/show/NCT04475302

Prevention, Efficacy and Safety of BCG Vaccine in COVID-19 Among Healthcare Workers https://ClinicalTrials.gov/show/NCT04461379

BCG Vaccination to Protect Healthcare Workers Against COVID-19 https://ClinicalTrials.gov/show/NCT04327206

COVID-19: BCG As Therapeutic Vaccine, Transmission Limitation, and Immunoglobulin… https://ClinicalTrials.gov/show/NCT04369794

Bacillus Calmette-guérin Vaccination to Prevent COVID-19 https://ClinicalTrials.gov/show/NCT04414267

Using BCG Vaccine to Protect Health Care Workers in the COVID-19 Pandemic https://ClinicalTrials.gov/show/NCT04373291

Using BCG to Protect Senior Citizens During the COVID-19 Pandemic https://ClinicalTrials.gov/show/NCT04542330

Efficacy of BCG Vaccination in the Prevention of COVID19 Via the Strengthening of Innate… https://ClinicalTrials.gov/show/NCT04384549

BCG Vaccine for Health Care Workers as Defense Against COVID 19 https://ClinicalTrials.gov/show/NCT04348370

Efficacy and Safety of VPM1002 in Reducing SARS-CoV-2 (COVID-19) Infection Rate and Severity https://ClinicalTrials.gov/show/NCT04439045

Study to Assess VPM1002 in Reducing Healthcare Professionals' Absenteeism in COVID-19… https://ClinicalTrials.gov/show/NCT04387409

Study to Assess VPM1002 in Reducing Hospital Admissions and/or Severe Respiratory Infectious https://ClinicalTrials.gov/show/NCT04435379

4. Conclusions

We conclude that the large majority of registered clinical trials examining BCG’s effect on COVID-19 are designed to be blinded, at least on the patient’s side. However, due to a naturally occurring dermal reaction and scar, most subjects are bound to be unblinded within 2 weeks following injection (see graphic abstract).

The COVID-19 test can be frequently positive in asymptomatic individuals during outbreaks in vulnerable populations [12]. This indicates potentially high false positive rates of a single nasopharyngeal swab test for COVID-19. There is lack of a “gold standard” test to which the results of the SARS-CoV-2 RT‐PCR can be compared [13]. Consequently, it can be rather challenging to discern between truly uninfected states, asymptomatic pass-through (i.e. presence of passing through virus on mucosal surfaces without infection of host cells), asymptomatic infection/carriage, preclinical infection, or even respiratory symptoms arising from other infections in an individual with a SARS-CoV-2 positive PCR test. Under such circumstances, significant detection bias [14] may occur in a trial, if a naturally unblinded (as in case of BCG) subjects seek testing or are recommended to be tested (by the unblinded research personnel) differently than the control group. In case of BCG vaccination against COVID-19, the unblinded placebo group would likely have a higher rate of testing than the BCG group, which is “intended” to be protected (intention bias and social desirability bias feeding into testing/detection bias). Additionally, the placebo group is likely to have a higher dropout rate from the study, leading to attrition bias [14].Utilization of an “ideal” nocebo (i.e., an immunologically inert substance that induces the same dermal reaction as BCG vaccination, in this case) could provide the means to truly blind BCG vaccination trials. We could not find, however, any BCG trials by an internet search on January 3rd 2020 (with search terms on Google: BCG, vaccination, trial, nocebo) where such nocebo control would have been used.

If the natural unblinding of BCG vaccination is recognized, then prospectively set clinical criteria for testing and defining an a priori “gold standard” set of tests (beyond a single nasopharyngeal PCR [as in [15], for example]) for making the diagnosis of COVID-19 could counteract detection bias. Such measures could significantly improve the quality of an openly unblinded prospective clinical trial studying BCG against COVID-19. In the current status quo, however, detailed description of the strategy towards overcoming the BCG vaccination induced skin reaction associated unblinding, and the secondary biases, will be required to interpret the theoretically blinded COVID-19 directed clinical trials.

Discussion of biases holds true for the interpretation of retrospective observations on unblinded acute/booster BCG vaccination induced protection against COVID-19, as well. An example for such is the yet unpublished work of Amirlak, et al. [16], which retrospectively found that 71 staff members of the Emirates International Hospital who received a BCG booster in an unblinded fashion were significantly (p = 0.004) protected against symptomatic COVID-19, compared to 209 staff members who did not receive the booster. It is unclear from the manuscript, however, how intention bias and social desirability bias in the booster group was controlled for (i.e., not seeking COVID-19 testing in case of mild symptomatology in the BCG booster group).

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.