COVID-19 Vaccine Trials—The Potential for ‘Hybrid’ Analyses

Abstract

Background:

Although several COVID-19 vaccines have been found to be effective in rigorous evaluation and have emerging availability in parts of the world, their supply will be inadequate to meet international needs for a considerable period of time. There also will be continued interest in vaccines that are more effective or have improved scalability to facilitate mass vaccination campaigns. Ongoing clinical testing of new vaccines also will be needed as variant strains continue to emerge that may elude some aspects of immunity induced by current vaccines. Randomized clinical trials meaningfully enhance the efficiency and reliability of such clinical testing. In clinical settings with limited or no access to known effective vaccines, placebo-controlled randomized trials of new vaccines remain a preferred approach to maximize the reliability, efficiency and interpretability of results. When emerging availability of licensed vaccines makes it no longer possible to use a placebo control, randomized active-comparator non-inferiority trials may enable reliable insights.

Methods:

In this article, ‘hybrid’ methods are proposed to address settings where, during the conduct of a placebo-controlled trial, a judgment is made to replace the placebo arm by a licensed COVID-19 vaccine due to emerging availability of effective vaccines in regions participating in that trial. These hybrid methods are based on proposed statistics that aggregate evidence to formally test as well as to estimate the efficacy of the experimental vaccine, by combining placebo-controlled data during the first period of trial conduct with active-controlled data during the second period.

Results:

Application of the proposed methods is illustrated in two important scenarios where the active control vaccine would become available in regions engaging in the experimental vaccine’s placebo-controlled trial: in the first, the active comparator’s vaccine efficacy would have been established to be 50–70% for the 4–6 month duration of follow-up of its placebo-controlled trial; in the second, the active comparator’s vaccine efficacy would have been established to be 90–95% during that duration. These two scenarios approximate what has been seen with adenovirus vaccines or mRNA vaccines, respectively, assuming the early estimates of vaccine efficacy for those vaccines would hold over longer-term follow-up.

Conclusions:

The proposed hybrid methods could readily play an important role in the near future in the design, conduct and analysis of randomized clinical trials performed to address the need for multiple additional vaccines reliably established to be safe and have worthwhile efficacy in reducing the risk of symptomatic disease from SARS-CoV-2 infections.

Introduction

While some vaccines already have been proven effective at preventing symptomatic disease from infection with the SARS-Cov-2 virus and have achieved regulatory authorization, it remains important to identify additional effective vaccines to address worldwide needs.^1,2 Newer effective vaccines, for example, might have improved scalability for mass vaccination campaigns. Placebo-controlled trials remain a preferred approach to maximize the efficiency and interpretability of results;³ however when randomization to a placebo control would no longer be proper, designs involving non-inferiority comparisons against known effective active comparator vaccines provide the potential to reliably evaluate candidate COVID-19 vaccines.⁴

This article considers the setting where, during trial conduct, a safe and effective vaccine becomes available in communities participating in a placebo-controlled trial of an experimental vaccine, leading to the judgment that the trial could only be continued if the effective vaccine would be used as active comparator regimen, replacing the placebo arm. ‘Hybrid’ methods are proposed to aggregate the evidence about efficacy of the experimental vaccine, by combining the placebo-controlled data during the first period of trial conduct, denoted Stratum ‘P’, with active-controlled data during the second period, denoted Stratum ‘A’. In particular, a test statistic and an estimate for the efficacy of the experimental vaccine are proposed that use the collective evidence from these two periods.

We consider the implementation of this hybrid approach under two possible settings, (see Figure 1):

1. The ‘platform’ trial has concurrent randomization to two or more experimental vaccines and a placebo vaccine (as potentially in the World Health Organization Solidarity Vaccines Trial⁵); at t^R, it no longer is proper to continue randomization to placebo, (for example, if there is persuasive evidence of efficacy of one of the vaccines). Randomization continues to that vaccine (now denoted the active comparator vaccine) and to the remaining experimental vaccines. Participants previously randomized to placebo continue on placebo and are followed until t^F, beyond which it wouldn’t be proper to do so. (This setting could arise if two experimental vaccines enter into the platform trial simultaneously, yet only one of these crosses a monitoring boundary early due to extreme benefit or, more likely, if one of the experimental vaccines enters into the platform trial at an earlier time than the other.)

2. The trial of the experimental vaccine begins as placebo-controlled (i.e., in Stratum ‘P’); at time t^R, it is no longer proper to continue randomization to placebo given availability of a different vaccine having persuasive evidence for efficacy established in a separate randomized placebo-controlled trial. Randomization is then continued (i.e., in Stratum ‘A’), between the experimental vaccine and that different vaccine (denoted the active comparator vaccine). Participants previously randomized to placebo continue on placebo and are followed, for as long as proper, until t^F.

Figure 1.

Settings are illustrated for use of the proposed hybrid analyses, where t^R denotes the calendar date when it is no longer proper to continue randomization to placebo, and t^F denotes the calendar date when it would no longer be proper for participants randomized to placebo to remain on placebo and in follow-up.

The solid lines denote calendar times when participants are being randomized to an arm, while the dotted lines denote when follow-up is continued for those randomize before t^R

Note, in either of the above settings, t^R and t^F may occur simultaneously. If participants in Stratum ‘P’ would be offered a vaccination at t^R, then methods in this article would apply by censoring the Stratum ‘P’ participants at t^R.

Methods

Trials of COVID-19 vaccines generally have primary endpoint, ‘virologically confirmed symptomatic COVID-19 disease’. ‘Vaccine efficacy,’ (VE), denotes the relative reduction in the rate of such primary endpoint events in a vaccinated group of participants compared to placebo controls. Suppose Cox regression analyses are used to estimate the hazard ratio (HR) or relative rates of primary endpoint events on the experimental vaccine versus comparator vaccine. Then, for a placebo-controlled trial, VE is estimated as 100 (1-HR).

Assume the placebo-controlled period of the trial, Stratum ‘P’, is designed to reliably assess the strength of evidence to rule out that true VE of the experimental vaccine is ≤ VE₀ ≡ 100 (1 – HR₀), where HR₀ denotes the experimental vaccine to placebo hazard ratio under the null hypothesis. In turn, assume the active comparator-controlled period of the trial evaluating the experimental vaccine, denoted Statum ‘A’, is designed using methods of non-inferiority trials to reliably assess strength of evidence to rule out a threshold for loss of efficacy, denoted by Δ, and referred to as the non-inferiority margin. Frequently, the non-inferiority margin is chosen to be the minimum threshold constituting an unacceptable loss of efficacy, denoted by Δ ≡ δ. In turn, ruling out that the true experimental vaccine to active-comparator vaccine HR is ≥ δ allows the conclusion that the experimental vaccine is ‘at least similarly effective to’ the active comparator vaccine in the non-inferiority trial. However, in the interest of increasing availability of safe vaccines with worthwhile efficacy to better meet worldwide needs, one could consider instead the criterion that the experimental vaccine be ‘at least similarly effective to’ an active comparator vaccine at the threshold of satisfing the World Health Organization and Food and Drug Administration criteria for success,^1,6 (i.e., the lower limit of the 95% confidence interval (CI) for VE is > 30%, or correspondingly the upper limit of the 95% CI for the active comparator vaccine to placebo estimated hazard ratio is < 0.70). This leads to the recently proposed modified non-inferiority margin⁴ denoted by Δ ≡ δ₀.

Note that when the non-inferiority margin is derived using the traditional ‘95–95’ approach with preservation of 50% of the active comparator’s effect,^7–9 then:

$$\begin{gathered} \mathit{\delta }={\left\{(95\text{\% CI upper limit of the HR})/{(95\text{\% CI upper limit of the HR})}^{\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$2$}\right.}\right\}}^{-1} \\ ={(95\text{\% CI upper limit of the HR})}^{-\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$2$}\right.}, \end{gathered}$$ (1)

and, in turn, that³:

$${\mathbf{\delta }}_{\mathbf{0}}={\{\text{(}95\text{\% CI upper limit of the hazard ratio)}/{(.70)}^{\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$2$}\right.}\}}^{-1}.$$ (2)

The proposed hybrid approach is based on the aggregation of two statistics, a statistic assessing the strength of evidence to rule out that the experimental vaccine to placebo hazard ratio is ≥ HR₀ ≡ 1 − (VE₀/100), using placebo-controlled evidence generated in the first period of the trial (Stratum ‘P’), and a statistic assessing the strength of evidence to rule out that the experimental vaccine to active comparator hazard ratio is ≥ Δ, using active-comparator-controlled evidence generated in the second period of the trial (Stratum ‘A’). Both of these statistics will use Cox regression analyses to estimate the HR, i.e., the relative rates of primary endpoint events on the experimental vaccine versus placebo or comparator vaccine. Based on a Cox partial likelihood, the Wald statistic for the estimate of (ln HR) is approximately normally distributed with true mean (ln HR) and, in a 1:1 randomization, with variance approximated by (4/L), where L is the number of primary endpoint events.¹⁰ If the true HR under the null or alternative hypothesis would be outside the range of (0.5, 2), a more accurate approximation to this variance is estimated by

$$\text{V}(L,\text{B})\equiv (4/L)\left\{{(1+\text{B})}^2/(4\text{B})\right\},$$ (3)

where B is the estimated hazard ratio.¹¹ The usual Wald test statistic then is the difference between the estimate of (ln HR) and its value under the null hypothesis, where this difference is divided by the square root of V(L, B). Under the null hypothesis, this test statistic would be approximately normally distributed with mean 0 and variance 1.

These insights can then be applied to evidence from Stratum ‘P’ and Stratum ‘A’. In Stratum ‘P’, where the goal is to test the null hypothesis that the true experimental vaccine versus placebo HR = HR₀, suppose L_P denotes the actual number of events obtained in Stratum ‘P’ and T_P denotes the Wald test statistic using data only from Stratum ‘P’. Then

$${\text{T}}_{\text{p}}=\left\{ln{\text{HR}}_{\text{p}}-ln{\text{HR}}_0\right\}/{\left\{\text{V}\left(L_P,{\text{HR}}_{\text{P}}\right)\right\}}^{\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$2$}\right.},$$ (4)

where HR_P is the estimated experimental vaccine to placebo HR using data from Stratum ‘P’. Under the null hypothesis, T_P has a standard normal distribution. In Stratum ‘A’, where the goal is to test the null hypothesis that the experimental vaccine versus the active comparator vaccine HR = Δ, suppose L_A denotes the actual number of events obtained in Stratum ‘A’ and T_A denotes the Wald test statistic using data only from Stratum ‘A’. Then,

$${\text{T}}_{\text{A}}=\left\{ln{\text{HR}}_{\text{A}}-ln\mathbf{\Delta }\right\}/{\left\{\text{V}\left(L_A,{\text{HR}}_{\text{A}}\right)\right\}}^{\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$2$}\right.},$$ (5)

where HP_A is the estimated experimental vaccine to active comparator vaccine HR using data from Stratum ‘A’. Under the null hypothesis, T_A has a standard normal distribution.

Aggregated evidence from Stratum ‘P’ and Stratum ‘A’ regarding efficacy of the experimental vaccine could be based on the statistic, T ≡ {(w)^1/2 T_P + (1-w)^1/2 T_A}, for any weighting, w, between 0 and 1. This statistic could be used for formal hypothesis testing that would protect the false positive error rate under a joint null hypothesis that the true experimental vaccine to placebo vaccine hazard ratio is HR₀ and that the true experimental vaccine to active comparator vaccine hazard ratio is Δ. By choosing HR₀ and Δ in a consistent manner, these two null hypotheses can be closely aligned, such as in the setting of considerable interest where HR₀ ≡ 0.7 and Δ ≡ δ₀, since both represent the hypothesis to be rejected in order to satisfy the criteria put forth by the World Health Organization and US Food and Drug Administration for worthwhile vaccine efficacy. When the statistics T_P and T_A are (approximately) independent, it follows immediately that the aggregated statistic, T, would be approximately normally distributed with mean 0 and variance 1, enabling the generation of p-values for formal hypothesis testing of experimental vaccine efficacy.

If this hybrid approach is used for formal hypothesis testing, then the targeted numbers of primary endpoints in Stratum ‘P’ and in Stratum ‘A’ could be approximated to achieve proper experimental power under specified alternative hypotheses. If Stratum P were a stand-alone placebo-controlled trial designed to have 90% power, under the alternative hypothesis that the experimental vaccine to placebo HR = HR₁, to rule out the null hypothesis that HR = HR₀ when using a Wald test statistic with a 2.5% false positive error rate, then, applying equation (3), the number of events needed, L_0P, is¹¹:

$$\left[{\left(1+{\text{HR}}_0\right)}^2/\left(8{\text{HR}}_0\right)+\left\{{\left(1+{\text{HR}}_1\right)}^2/\left(8{\text{HR}}_1\right)\right\}\right]\times {(3.242)}^2/{\left[0.5\left\{\ln {\text{HR}}_1-\ln {\text{HR}}_0\right\}\right]}^2$$ (6)

In turn, suppose the active comparator vaccine had been established to have vaccine efficacy VE*, and denote HR* = 1− ( VE* / 100). If Stratum ‘A’ were a stand-alone non-inferiority trial designed to have 90% power, under the alternative hypothesis that the experimental to active comparator HR = HR₁/HR*, to rule out the null hypothesis that HR = Δ when using a Wald test statistic with a 2.5% false positive error rate, then, using equation (3), the number of events needed, L_0A, is¹¹:

$$\left[{(1+\mathbf{\Delta })}^2/(8\mathbf{\Delta })+\left\{{\left(1+{\text{HR}}_1/\text{HR}\ast \right)}^2/\left(8{\text{HR}}_1/\text{HR}\ast \right)\right\}\right]\times {(3.242)}^2/{\left[0.5\left\{\ln \left({\text{HR}}_1/\text{HR}\ast \right)-\ln \mathbf{\Delta }\right\}\right]}^2$$ (7)

Suppose further that L_P, the number of primary endpoint events actually obtained in Stratum ‘P’, is meaningfully less than the targeted L_0P primary endpoints, thus necessitating the generation of L_A primary endpoint events in Stratum ‘A’. Under these specified alternative hypotheses, we anticipate that setting L_A = L_0A {1-(L_P / L_0P)} will lead to reasonable experimental power, though further exploration is warranted.

This hybrid approach also allows estimation of the experimental vaccine efficacy based on HR_P, the estimated experimental vaccine to placebo HR using Stratum ‘P’ data, and HP_A, the estimated experimental vaccine to active comparator vaccine HR using Stratum ‘A’ data. A logical estimator of ln HR for the experimental vaccine vs placebo, for some weighting, w, would be:

$$\begin{gathered} \left\{w\right\}\ln {\text{HR}}_{\text{p}}+ \\ \{1-w\}\left\{\ln {\text{HR}}_{\text{A}}+\ln {\text{HR}}^{\ast }\right\} \end{gathered}$$ (8)

where VE* is the previously established vaccine efficacy for the active comparator vaccine, and where HR* ≡ 1− (VE* / 100). Exponentiating this yields the estimated experimental vaccine to placebo HR, and in turn the estimated VE = 100 (1-HR), under the constancy assumption that VE* is an unbiased estimate of the true VE of the active comparator vaccine in the setting of Stratum ‘A’. If, as above, (L_A / L_0A) ≈ (1−(L_P / L_0P)}, then a logical choice for w is L_P / L_0P, for both inference and hypothesis testing. The properties of this choice of w should be compared to the properties for other weightings, such as an inverse variance weighting.

For illustration, suppose L_0P = 150, L_P= 50, HR_P = 0.42, HR* = .30, and HR_A = 1.2. Then the experimental vaccine has 58% estimated efficacy in the placebo-controlled Stratum ‘P’, the active comparator vaccine has 70% estimated efficacy (similar to the adenovirus vaccines^{12, 13}), and the experimental vaccine appears slightly less effective than active comparator in Stratum ‘A’, with estimated efficacy of 64% in that stratum. Setting w ≡ L_P/L_0P, the overall estimate of efficacy for the experimental vaccine, pooling the data from Strata ‘P’ and ‘A’, would be 62.1%.

The assumption of independence of statistics T_P and T_A clearly would be valid, as in Setting #2 of Figure 1, when there are three separate cohorts of participants, one to derive the non-inferiority margin, the second constituting Stratum ‘P’ and the third constituting Stratum ‘A’. If participants who received placebo are crossed-over to receive an active vaccine, these participants are censored at this point to reduce risk of confounding. This would be the most likely setting to apply the proposed hybrid methods.

The assumption of independence of the statistics T_P and T_A might be violated in Setting #1 of Figure 1, where multiple vaccines are simultaneously randomized against placebo control. In this setting, let Stratum ‘P’ include follow-up to time t^F for all participants who were randomized to the experimental vaccine or placebo prior to time t^R (see in Figure 1), while Stratum ‘A’ would include all participants randomized to the experimental or active comparator vaccines after t^R. Importantly, these are separate, hence independent, cohorts. Given the independent increment structure of the Cox partial likelihood, the lack of correlation between Stratum ‘P’ and Stratum ‘A’ would be maintained even if we allow that Stratum ‘A’ also include the follow-up after t^F for patients randomized to the experimental vaccine vs active comparator vaccine before t^R. Importantly, the risk for some dependence of statistics T_P and T_A in Setting #1 would arise when there was dual use of patients randomized before t^R: to be specific, the follow-up to t^R of those patients randomized to the experimental vaccine vs placebo would be used in T_P, while the follow-up to t^R of those patients randomized to the active comparator vaccine vs placebo would be used to formulate the non-inferiority margin in T_A. In this hybrid situation, the number of events in concurrently randomized experimental vaccine and placebo participants occurring before t^R likely would not be large, since the hybrid approach would not be needed if the events from that comparison up to t^F would provide adequate statistical power. Further, the information about the experimental vaccine vs placebo and the active comparator vaccine vs placebo comparisons obtained before t^R would be uncorrelated with comparative information obtained after t^R. It follows that any potential correlation between statistics T_P and T_A, even in this setting of a potential dual use of information before t^R should be small.

Results

In this article, two scenarios are considered regarding the active control vaccine that could become available in regions engaging in the experimental vaccine’s placebo-controlled trial: in the first, denoted Scenario I, the active comparator’s vaccine efficacy would have been established to be 50–70% for the 4–6 month duration of follow-up of its placebo-controlled trial; in the second, denoted Scenario II, the active comparator’s vaccine efficacy would have been established to be 90–95% during that duration. These two scenarios approximate what has been seen with adenovirus vaccines^12,13 or mRNA vaccines,^14,15 respectively, assuming the early estimates of vaccine efficacy for those vaccines would hold over longer-term follow-up. While the emerging availability of the mRNA vaccines creates particular interest in Scenario II, we begin with Scenario I given it directly parallels the most frequently used design for placebo-controlled trials of COVID-19 vaccines.

Scenario I

An experimental vaccine is being evaluated to assess whether it meets or exceeds the recommended standard for worthwhile efficacy. An effective vaccine becomes available as the active comparator in the second period of the trial, with estimated efficacy in the range of 50% to 70% over 4 to 6 months of follow-up.

The World Health Organization- and Food and Drug Administration-recommended standard for ‘worthwhile’ efficacy for an experimental vaccine is having a point estimate of ≥ 50% vaccine efficacy with a 95% CI lower bound of > 30% efficacy.^1,5,6 Covid-19 vaccine trials often have been designed to provide 90% power, when the true efficacy is 60%, to rule out the null hypothesis that vaccine efficacy ≤ 30%, preserving a 2.5% false positive error rate. By equation (6), assuming 1:1 randomization, such a trial requires L_0P = 150 primary endpoint events, where the threshold for statistical significance is an estimate of ≥ 50% vaccine efficacy. Hence, the goal of such placebo-controlled trials, in essence, is to address whether the experimental vaccine satisfies this standard for worthwhile efficacy. During trial conduct, if it becomes necessary to replace the placebo control by an active control vaccine having vaccine efficacy in the range of 50% to 70% over 4 to 6 months of follow-up, it would be logical for the resulting non-inferiority trial to be designed to rule out the non-inferiority margin, δ_o, in equation (2), ensuring that the experimental vaccine would be ‘at least similarly effective to’ an active comparator vaccine having efficacy at the threshold of satisfying this standard criteria for ‘worthwhile’ vaccine efficacy.

Suppose there were only L_P = 50 primary endpoints in the placebo-controlled Stratum ‘P’. Hence, the proposed hybrid approach would be based on the aggregation of two statistics, where the statistic T_P in Stratum ‘P’, defined in equation (4), would be:

$${\text{T}}_{\text{P}}=\{ln{\text{HR}}_{\text{P}}-ln0.7\}/{\left\{\text{V}\left(\mathit{50},{\text{HR}}_{\text{P}}\right)\right\}}^{\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$2$}\right.}$$

To compute the statistic, T_A, in Stratum ‘A,’ assume that the non-inferiority margin is Δ = δ_0. Assume the active comparator regimen had estimated VE* = 70%, (so HR* = 0.30), from a placebo controlled trial with 400 events, (similar to numbers of events achieved in early generation placebo-controlled Covid-19 vaccines trials.¹³) It follows from equations (2) and (3) that δ₀ = 2.21 and from equation (7) that L_0A = 180; in turn L_A = 120. Hence, by equation (5),

$${\text{T}}_{\text{A}}=\left\{ln{\text{HR}}_{\text{A}}-ln2.21\right\}/{\left\{\text{V}\left(\mathit{120},{\text{HR}}_{\text{A}}\right)\right\}}^{\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$2$}\right.}.$$

Scenario II

Partway through a trial conducted to evaluate an experimental vaccine, an established effective vaccine becomes available as an active comparator with estimated vaccine efficacy in the range of 90% to 95% over 4 to 6 months of follow-up. There is reason to expect the experimental vaccine may have similarly high efficacy.

Suppose the goal is to address whether the experimental vaccine indeed would be highly effective, with sufficient evidence to rule out VE ≤ 80%. A high bar for efficacy such as this could be set based on potential weaknesses of the experimental vaccine other than efficacy, such as having cold chain constraints and potential deficiencies for large-scale manufacturing, or could be set by the sponsor of the experimental vaccine if it was thought that their vaccine could only compete with a highly effective vaccine if it were shown to be similarly highly effective.

In this scenario, in Stratum ‘P’ containing the placebo-controlled evidence from the trial’s first period, by equation (6), L_0P = 80 events would provide 90% power when HR₁ = 0.05, in order to rule out HR₀ ≥ 0.2 when using a Wald test statistic having 2.5% false positive error rate under HR = HR₀. Suppose, however, that the actual number of events, L_P, obtained in Stratum ‘P’ is less than 80. The Wald test statistic using data only from Stratum ‘P’, is

$${\text{T}}_{\text{P}}=\left\{ln{\text{HR}}_{\text{P}}-ln0.2\right\}/{\left\{\text{V}\left(L_P,{\text{HR}}_{\text{P}}\right)\right\}}^{\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$2$}\right.}$$

In turn, Stratum ‘A’ contains the evidence from the second period of the trial for whether the estimated hazard ratio for the experimental vaccine vs the highly effective active comparator vaccine would rule out the non-inferiority margin, Δ = δ. Since L_A events are obtained in Stratum ‘A’, the Wald test statistic is

$${\text{T}}_{\text{A}}=\left\{ln{\text{HR}}_{\text{A}}-ln\mathit{\delta }\right\}/{\left\{\text{V}\left(L_A,{\text{HR}}_{\text{A}}\right)\right\}}^{\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$2$}\right.}$$

In this setting, if the placebo-controlled trial evaluating the active comparator vaccine, after extended follow-up, had 300 events and yielded HR* = 0.05 then by equations (1) and (3), δ = 3.428. It follow, by equation (7), that L_0A = 34 events would be required for a stand-alone non-inferiority trial to have 90% power to rule out the non-inferiority margin δ, when the true experimental vaccine to active comparator hazard ratio = 1. In turn, L_A should be 34 {1-(L_P / 80)}.

Discussion

‘Hybrid’ methods are proposed addressing settings where, during a placebo-controlled trial, the placebo arm is replaced by an active comparator vaccine due to emerging availability of one or more effective vaccines in regions participating in that trial. These hybrid methods enable aggregation of evidence about the efficacy of the experimental vaccine, by combining placebo-controlled data during the first period of trial conduct with active-controlled data during the second period. Given the increased interpretability and efficiency of placebo-controlled trials, reasonable efforts should be made to maximize the proportion of the information provided in placebo-controlled Stratum ‘P’, such as through continued follow-up of participants on their originally randomized intervention in Stratum ‘P’ for as long as proper.^1,3

This article’s Scenario I addresses the classic setting where evidence in Stratum ‘P’ would be from a placebo-controlled trial evaluating whether an experimental vaccine, hypothesized to have true 60% VE, satisfies the accepted standard for having worthwhile efficacy, and where evidence in Stratum ‘A’ would be from a comparison with an active-comparator vaccine having 50% to 70% VE, assessing whether the experimental vaccine is ‘at least similarly effective to’ a vaccine at the threshold of satisfying this same standard. In contrast, Scenario II illustrates a setting assessing whether the experimental vaccine has high VE, and where the evidence in Stratum ‘A’ would be for a comparison of the experimental vaccine with an active comparator vaccine having very high 90% to 95% VE. One variation of this setting could be where an 80-event placebo-controlled trial would be designed to have high power to rule out a null hypothesis of 30% VE when true VE is ≥ 0.70 and then, in a nested manner, would also be well powered to rule out a null hypothesis of 80% VE if true VE is ≥ 0.95. In turn, suppose in Stratum ‘P’ that the estimate of VE is ≥ 80% after 50 events. Then the null hypothesis of 30% VE would be ruled out by the O’Brien-Fleming boundary, yet if an active comparator vaccine with 95% VE became available at that time, the hybrid design could be implemented to provide additional information relevant to whether the null hypothesis of 80% VE could also be rejected.⁴

Given the potential in Covid-19 vaccine trials for behavioral changes if participants knew they had been vaccinated, (i.e., less masking, less social distancing), blinding is important to obtaining unbiased estimates of efficacy.^3,16 Hence, the evaluation of the experimental vaccine against the active comparator in Stratum ‘A’, like the evaluation against the placebo control in Stratum ‘P’, should be conducted in a blinded manner. To further enhance the reliability and interpretability of the proposed hybrid approach, samples should be obtained from breakthrough cases at diagnosis visits, enabling viral genotyping to assess the influence of viral variants of concern on vaccine efficacy. If such assessments also were performed in the placebo-controlled trial of the active comparator regimen, then methods in this article could be generalized to enable unbiased assessments about variant-specific efficacy of the experimental vaccine, in turn, increasing the likelihood of the validity of the constancy assumption.

Reliably evaluating whether multiple additional vaccines are safe and have worthwhile efficacy in reducing the risk of virologically-confirmed symptomatic disease would have indisputable value in addressing the COVID-19 pandemic. The proposed hybrid methods could play an important role in this process.

Table 1.

Summary of Notation in Equations for Statistical Methods

CI	Confidence Interval
L	Number of primary events in a Cox regression analysis
V	Variance of the estimated log Hazard Ratio in a Cox regression analysis
HR	Hazard ratio
VE	Vaccine efficacy
Δ, δ, δ0	Non-inferiority margins used in Stratum ‘A’
TP, TA	Test statistics using data from Stratum ‘P’ and Stratum ‘A’, respectively
LP	Observed number of events in Stratum ‘P’
L0P	Number of events needed in Stratum ‘P’ alone to achieve 90% statistical power
LA	Observed number of events in Stratum ‘A’
L0A	Number of events needed in Stratum ‘A’ alone to achieve 90% statistical power
HR0, HR1	Experimental vaccine to placebo hazard ratios, under the null and alternative hypotheses, in Stratum ‘P’
VE0	Vaccine efficacy of the experimental vaccine, under the null hypothesis, in Stratum ‘P’
HRP	Estimated experimental vaccine to placebo hazard ratio using data in Stratum ‘P’
HRA	Estimated experimental vaccine to active comparator vaccine hazard ratio using data in Stratum ‘A’
VE*, HR*	Estimated vaccine efficacy and hazard ratio from the placebo-controlled evaluation of the Active Comparator Vaccine
tR	Calendar date when it is no longer proper to continue randomization to placebo
tF	Calendar date when it is no longer proper for participants randomized to placebo to remain on placebo and in follow-up