Estimands and inference in cluster-randomized vaccine trials

Summary

Cluster-randomized trials are often conducted to assess vaccine effects. Defining estimands of interest before conducting a trial is integral to the alignment between a study’s objectives and the data to be collected and analyzed. This paper considers estimands and estimators for overall, indirect, and total vaccine effects in trials, where clusters of individuals are randomized to vaccine or control. The scenario is considered where individuals self-select whether to participate in the trial, and the outcome of interest is measured on all individuals in each cluster. Unlike the overall, indirect, and total effects, the direct effect of vaccination is shown in general not to be estimable without further assumptions, such as no unmeasured confounding. An illustrative example motivated by a cluster-randomized typhoid vaccine trial is provided.

1 |. INTRODUCTION

Vaccines are integral to combating a variety of infectious diseases. Quantifying a vaccine’s effects is vital to determining its benefits, which can then guide public health policies aimed at reducing the burden of disease. Cluster-randomized trials are often conducted to quantify the effects of a treatment or intervention such as a vaccine. In cluster-randomized trials, individuals are grouped together based on certain characteristics (eg, neighborhood of residence), and the entire cluster is randomized to treatment or control. The process of randomization ensures that the treatment and control groups are exchangeable. Cluster-randomization is useful when it is impractical or infeasible to randomize at the individual level.¹ Comparisons between randomized clusters can be used to assess the overall impact of an intervention on the population, which is particularly important in settings where an intervention may have indirect (or spillover) effects.² For example, in the infectious disease setting, whether one individual is vaccinated could affect the outcome of another individual. Moulton et al³ describe a cluster-randomized trial in the White Mountain Apache Reservation and the Navajo Nation wherein approximately 9000 infants within 38 clusters were randomized by cluster to the vaccine of interest (Streptococcus pneumoniae conjugate vaccine) or control (a meningococcal C conjugate vaccine). Diallo et al⁴ present a cluster-randomized trial of an inactivated influenza vaccine in Senegal in which approximately 7800 enrolled, age-eligible children within 20 clusters were randomized by cluster to the influenza vaccine or control (an inactivated polio vaccine). Sur et al⁵ describe a cluster-randomized trial of a typhoid vaccine in India, with approximately 38 000 individuals within 80 clusters randomized by cluster to the typhoid vaccine or control (hepatitis A vaccine).

Because the cluster-randomized trial is a common study design for evaluating vaccine effects, it is important to carefully define the estimands, that is, parameters of interest, in these trials. Careful definition of the effects of interest prior to the study can aid in study planning and can ensure that the study’s goals are achieved.⁶ Recently, there has been increased interest in defining estimands in clinical trials. The International Council on Harmonization (ICH) has published an addendum to the E9 guidelines detailing the use of estimands in clinical trials.⁷ This addendum aims to describe the necessity of defining the target estimand before the design and analysis of trials to avoid misalignment of the trial goals and the data, as well as to ensure that estimation of the estimand is possible without relying upon dubious assumptions.⁸

Leuchs et al,⁶ Koch and Wiener,⁹ Permutt,¹⁰ and Phillips et al¹¹ discuss examples of estimands of interest in regulatory clinical trials. Target estimands specifically for cluster-randomized trials have been previously considered for certain designs. Wu et al¹² consider estimands for matched-pair cluster-randomized trials. Hudgens and Halloran¹³ consider estimands of the direct, indirect, total, and overall effects of treatment assuming a two-stage randomization scheme. In this design, clusters are randomly assigned to a treatment allocation program, and individuals within the clusters are randomly assigned to treatment based on the cluster-level assignment. In some cluster-randomized trials, individuals may not comply with their randomization assignment or may choose not to participate in the study.^3,5,14,15 Frangakis et al¹⁶ consider clustered encouragement designs, which allow noncompliance, where individuals belong to one of three principal strata: always-takers, compliers, and never-takers. Kang and Keele¹⁷ also consider cluster-randomized trials with noncompliance. Like Frangakis et al,¹⁶ they consider the setting where there are the three principal strata mentioned above and also the special case where there are no always-takers. Even for this special case, they show the total and indirect (spillover) effects are not identified because principal strata membership is unknown for some individuals.

In this paper, we consider cluster-randomized vaccine trials where individuals choose whether to participate in the trial or not. As illustrated by the examples described above, it is common in cluster-randomized vaccine trials for the control to be another vaccine which is not expected to affect the outcome of interest. For simplicity, below the control vaccine will sometimes be referred to just as a control. Here, we consider the particular case where a control vaccine is employed and individuals are blinded, that is, unaware whether their cluster is randomly assigned to the vaccine of interest or to the control vaccine. In this setting, it is reasonable to assume individual participation behavior is unaffected by randomization, such that there are only two principal strata: always participators and never participators. Thus, our setting is similar to the special case considered by Kang and Keele.¹⁷ However, because it is assumed an individual will participate or not in the trial regardless of randomization assignment, principal strata membership is known for all individuals; this allows for identification and estimation of overall, total and indirect effects.

Sur et al⁵ provides a motivating example of a cluster-randomized vaccine trial where individuals self-select whether to participate. In this trial, clusters of individuals were randomized to either a typhoid vaccine or a control vaccine (for hepatitis A). The presence of a control allowed study blinding, so individuals in the clusters did not know which assignment their cluster received. While some individuals chose not to participate in the trial, outcome data were collected on all individuals. This allows inference about different effects of the vaccine, as described below.

The outline of the remainder of this paper is as follows. In Section 2, notation, estimands, estimators, and effects of interest are described. In Section 3, the Sur et al⁵ cluster-randomized typhoid vaccine trial is considered. Finally, Section 4 concludes with a discussion.

2 |. METHODS

2.1 |. Notation and potential outcomes

Consider a cluster-randomized vaccine trial with n clusters (or groups) of individuals where each cluster is randomly assigned to vaccine or control. For i = 1, …, n, let A_i = 1 if cluster i is assigned to vaccine and A_i = 0 otherwise. Let $Y_i^{a=1}$ denote the potential outcome if cluster i is assigned vaccine, and let $Y_i^{a=0}$ denote the potential outcome if cluster i is assigned control. For example, $Y_i^a=1$ could denote the proportion of individuals in cluster i who would develop typhoid within 1 year after randomization if, possibly counter to fact, cluster i were assigned to vaccine. For now, we leave the particular outcome associated with $Y_i^a$ unspecified. Different specifications of $Y_i^a$ will correspond to different vaccine effects, as described below. Let Y_i denote the observed outcome for cluster i, such that $Y_i=Y_i^{a=1}{A}_i+Y_i^{a=0}\left(1-A_i\right)$. Below, the subscript i is sometimes dropped for notational convenience.

In cluster-randomized vaccine trials, one individual’s vaccination status may affect another individual’s outcome, that is, there may be “interference” between individuals.¹⁸ For instance, if one individual receives a typhoid vaccine, this could affect whether another individual develops typhoid or not. Throughout this paper, it is assumed that there is no interference between individuals in different clusters, that is, there is “partial interference.”¹⁹ Under this assumption, the outcome Y_i for cluster i depends only on the treatment assigned to cluster i. No assumption is made regarding the form of interference within clusters.

2.2 |. Estimands and estimators

Vaccine effects, that is, the causal effects of vaccination, can be defined by contrasts in the expected values of the potential outcomes Y^a=1 and Y^a=0. Assuming the n clusters in the trial are randomly sampled from an infinite super-population of clusters, the average treatment (vaccine) effect is generally defined by

$$\theta =E\left[Y^{a=1}\right]-E\left[Y^{a=0}\right]$$ (1)

where E[X] denotes the expected value of X in the super-population of clusters. In words, Equation (1) is the difference in the average outcome in the super-population when a cluster receives a = 1 compared to when a cluster receives a = 0. Alternatively, the n clusters could be considered the finite population of interest and E[X] defined instead to be $n^{-1}\sum _{i=1}^n{X}_i$. The super-population perspective is adopted in this paper, but similar considerations to those provided here apply if the finite population approach is utilized instead. Likewise, estimands other than Equation (1) could be considered. For example, for binary Y, the risk ratio E[Y^{a = 1}]/E[Y^{a = 0}] = Pr[Y^{a = 1} = 1]/Pr[Y^{a = 0} = 1] might be of greater interest than the risk difference [Equation (1)]. For instance, Y might be an indicator of whether or not at least one individual in a cluster gets infected.^20,21 More generally, causal effects can by defined by g(E[Y^{a = 1}], E[Y^{a = 0}]) for some contrast function g(x, y) where g(x, x) = 0; for example, g(x, y) = x − y corresponds to Equation (1). Below, estimands of the form in Equation (1) are described, but similar considerations apply for other contrasts.

A few aspects of defining causal effects bear mentioning. First, causal effects are typically defined by contrasts in expected values of the potential outcomes over the same set of units.^22,23 In many settings, the unit is defined to be an individual; for example, a unit could be a participant in a randomized controlled trial. Here, we consider the clusters to be the units since randomization is at the cluster level. Note that contrasts in average potential outcomes between different sets of units do not have a causal interpretation. For example, suppose a cluster-randomized vaccine trial is conducted in schools, where students within the same school constitute the clusters. A comparison of the average Y^{a = 1} among clusters (schools) in rural areas to the average Y^{a = 0} among clusters in urban areas is not a causal effect. Also note that causal effects are contrasts in the expected value of the same outcome under different counterfactual scenarios. Contrasts in different outcomes are not causal effects. For example, a comparison of the average incidence of typhoid when clusters receive vaccine with the average incidence of cholera when clusters receive control would not be a causal effect. We will revisit this point below when discussing direct effects.

The average treatment effect can be estimated by the difference in sample means:

$$\widehat{\theta }=\frac{1}{n_1}\sum _{i=1}^n{Y}_{i}I\left(A_i=1\right)-\frac{1}{n_0}\sum _{i=1}^n{Y}_{i}I\left(A_i=0\right)$$ (2)

where $n_a={\sum }_{i=1}^{n}I\left(A_i=a\right)$ for a = 0, 1. This estimator is consistent and unbiased under commonly used randomization schemes, such as a completely randomized experiment where the number of clusters assigned vaccine (treatment) is fixed.^24–26 The SE of $\widehat{\theta }$ can be estimated and 95% Wald confidence intervals can be constructed in the usual manner for the difference in means. Equivalently, Equation (2) can be obtained by computing the least squares estimate of the slope parameter of simple linear regression of Y on A. A generally more precise estimator can be obtained by regressing Y on A and Z where Z is some vector of baseline covariates. For simplicity, only estimators of the form in Equation (2) are considered below; see Tsiatis et al²⁷ for further discussion on using baseline covariates to improve efficiency. Note also that Equation (2) utilizes only cluster level data and thus avoids the complexities associated with inference based on statistics constructed using individual level data, which require accounting for possible within-cluster correlation (eg, using mixed effects models or generalized estimating equations).

2.3 |. Overall, indirect, and total effects

In this section, the general approach above is used to define estimands and estimators of the overall, indirect, and total effects. The outcome of interest will depend on the context of the vaccine trial, such as the infection or pathogen of interest, the target population, and so forth. Here, the outcome of interest is generically referred to as disease.

The overall effect compares the average disease outcome among all individuals when a cluster is assigned vaccine vs when a cluster is assigned control. This quantity may be the most relevant to public health policy because all individuals within clusters are used in the comparison. As it is likely that populations of interest will include a mixture of individuals who would and who would not choose to be vaccinated, the overall effect may be valuable for public health officials and policy makers in assessing the overall impact of a vaccine at the population level.

The overall effect estimand and estimator can be defined in terms of individual level outcomes as follows. Let m_i denote the number of individuals in cluster i. For individual j in cluster i, let Y_ij = 1 if individual j develops disease, and let Y_ij = 0 otherwise. Let $Y_{ij}^{a=1}$ indicate the outcome that would have been observed for individual j if cluster i is randomized to vaccine, and define $Y_{ij}^{a=0}$ analogously for control, such that $Y_{ij}=Y_{ij}^{a=1}{A}_i+Y_{ij}^{a=0}\left(1-A_i\right)$. For the overall effect, the estimand Equation (1) can be expressed in terms of individual potential outcomes by defining $Y_i^{a=1}=\sum _{j=1}^{m_i}{Y}_{ij}^{a=1}/m_i$, and $Y_i^{a=0}=\sum _{j=1}^{m_i}{Y}_{ij}^{a=0}/m_i$ for cluster i. The overall effect estimator can likewise be expressed in terms of the observed individual-level outcomes by letting $Y_i=\sum _{j=1}^{m_i}{Y}_{ij}/m_i$.

The indirect effect quantifies the effect of vaccination on individuals who chose not to participate in the trial and, therefore, have no chance of receiving the vaccine. This effect is defined as a contrast in the average outcomes among non-participants when their cluster does or does not receive vaccine.²⁸ Because the indirect effect is defined only among individuals who never receive the vaccine, this effect (if present) is solely due to interference. Thus, indirect effects are a type of spillover or peer effect.¹⁹ Quantifying indirect effects may be of interest from a public health policy perspective because vaccinating some, but not all, individuals within a cluster can still provide benefits to those who are unable or choose not to be vaccinated.

Like the overall effect, the indirect effect estimand and estimator can be defined in terms of individual level outcomes. To do so, first define the potential outcome $S_{ij}^{a=1}$ where $S_{ij}^{a=1}=1$ if individual j in cluster i would choose to participate in the trial if, possibly counter to fact, cluster i were randomized to vaccine and $S_{ij}^{a=1}=0$ otherwise. Define $S_{ij}^{a=0}=0$ analogously. Denote the observed participation outcome for individual j in cluster i by S_ij, such that $S_{ij}=S_{ij}^{a=1}{A}_i+S_{ij}^{a=0}\left(1-A_i\right)$. Assume $S_{ij}^{a=1}=S_{ij}^{a=0}$, that is, an individual’s decision to participate is not affected by whether their cluster is assigned vaccine or control. This assumption may be reasonable in cluster-randomized trials where individuals are blinded, such as the typhoid vaccine trial described in Section 3, because in such settings, randomization is not expected to have an effect on an individual’s decision to participate in the trial. As mentioned in the Introduction section, Frangakis and Rubin²³ and Kang and Keele¹⁷ utilize the principal stratification framework when considering non-compliance in cluster-randomized trials. Under the assumption $S_{ij}^{a=1}=S_{ij}^{a=0}$, all individuals belong to one of two principal strata: always participators, that is, individuals where $S_{ij}^{a=1}=S_{ij}^{a=0}=1$, and never participators, that is, individuals where $S_{ij}^{a=1}=S_{ij}^{a=0}=0$. Fortunately, unlike the setting considered by Kang and Keele, here the principal strata membership of each individual can be inferred directly from the observed data because $S_{ij}=S_{ij}^{a=1}=S_{ij}^{a=0}$.

The indirect effect is the effect of vaccine in the non-participator principal stratum. The indirect effect has the general form [Equation (1)], with $Y_i^{a=1}$ now defined to be $\left\{\sum _{j=1}^{m_i}{Y}_{ij}^{a=1}I\left(S_{ij}^{a=1}=0\right)\right\}/\left\{\sum _{j=1}^{m_i}I\left(S_{ij}^{a=1}=0\right)\right\}$ and $Y_i^{a=0}$ defined to be $\text{{}\sum _{j=1}^{m_i}{Y}_{ij}^a=0_{ij}^{a=0}=0\text{)}\}/\left\{\sum _{j=1}^{m_i}I\left(S_{ij}^{a=0}=0\right)\right\}$. This estimand compares the average disease outcome among non-participators when a cluster is assigned vaccine vs when a cluster is assigned control. Similarly, the indirect effect estimator can be expressed by Equation (2) with Y_i defined to be $\left\{\sum _{j=1}^{m_i}{Y}_{ij}I\left(S_{ij}=0\right)\right\}/\left\{\sum _{j=1}^{m_i}I\left(S_{ij}=0\right)\right\}$.

The total effect measures the effect of treatment in the always participator principal stratum. Because always participators receive the vaccine if and only if their cluster is assigned vaccine, the total effect encompasses both the individual effect of receiving the vaccine as well as the effect of other individuals in the cluster being vaccinated. The total effect estimand and estimator have the same form as the indirect effect estimand and estimator described above, but with $S_{ij}^{a=1}=0$ replaced by $S_{ij}^{a=1}=1$, $S_{ij}^{a=0}=0$ replaced by $S_{ij}^{a=0}=1$, and S_ij = 0 replaced by S_ij = 1. The total effect quantifies the difference in the average disease outcome among always participators when a cluster is assigned vaccine vs when a cluster is assigned control. The total effect is often the effect of primary interest in this type of trial. An illustration of the overall, indirect, and total effects is given in Figure 1.

FIGURE 1.

Cluster counterfactual comparisons. The left circle represents a cluster if, possibly counter to fact, assigned to vaccine (A = 1). The right circle represents a cluster if, possibly counter to fact, assigned to control (A = 0). Within each circle, S indicates which individuals chose to participate in the study (S = 1 indicates participation, S = 0 otherwise). The overall, indirect, and total effects are contrasts in average potential outcomes over different sets of individuals within the clusters

There are a few special cases of note. In the scenario where all individuals in the population are willing to participate in trials (ie, there are no non-participators), the indirect effect is not well-defined, and the total and overall effects are equivalent. In some trials, only a subset of individuals may be eligible to be randomized for vaccination. For example, in Sur et al,⁵ individuals were eligible if they were at least 2 years of age, were not pregnant or lactating, and did not have an elevated temperature when the vaccine was given. Indirect effects, analogous to that defined above for non-participators, can be defined and estimated in these individuals if their outcome of interest is measured.

2.4 |. Direct effect

The overall, indirect, and total effects each describe an effect of treatment (vaccination) which is at least partially due to interference, if present. The effect of treatment that is not attributable to interference may also be of interest. Such an effect is sometimes referred to as a direct effect. This section describes why it is not possible in general to estimate the direct effect of vaccination in a cluster-randomized trial with self-selection of participation without additional assumptions, such as no unmeasured confounding. Informally, the direct effect compares the average outcome when an individual is vaccinated to the average outcome when an individual is not vaccinated, holding fixed the proportion of other individuals vaccinated.²⁸ Several formal definitions of the direct effect estimand have been proposed; for example, see Hudgens and Halloran,¹³VanderWeele and Tchetgen Tchetgen,²⁹ Liu et al,³⁰ Eck et al³¹ and Sävje et al.³²

To develop intuition behind the lack of identifiability of the direct effect, consider the following naive approach. Suppose the proportion of vaccinated individuals with disease is compared to the proportion of unvaccinated individuals with disease in clusters assigned to vaccine by

$$\frac{1}{n_1}\sum _{i=1}^n\frac{\sum _{j=1}^{m_i}{Y}_{ij}I\left(S_{ij}=1\right)}{\sum _{j=1}^{m_i}I\left(S_{ij}=1\right)}I\left(A_i=1\right)-\frac{1}{n_1}\sum _{i=1}^n\frac{\sum _{j=1}^{m_i}{Y}_{ij}I\left(S_{ij}=0\right)}{\sum _{j=1}^{m_i}I\left(S_{ij}=0\right)}I\left(A_i=1\right).$$ (3)

By the law of large numbers, Equation (3) converges to

$$E\left[Y^{a=1}\right]-E\left[{\tilde{Y}}^{a=1}\right]$$ (4)

where

$$Y_i^{a=1}=\left\{\sum _{j=1}^{m_i}{Y}_{ij}^{a=1}I\left(S_{ij}^{a=1}=1\right)\right\}/\left\{\sum _{j=1}^{m_i}I\left(S_{ij}^{a=1}=1\right)\right\}$$

and

$${\tilde{Y}}_i^{a=1}=\left\{\sum _{j=1}^{m_i}{Y}_{ij}^{a=1}I\left(S_{ij}^{a=1}=0\right)\right\}/\left\{\sum _{j=1}^{m_i}I\left(S_{ij}^{a=1}=0\right)\right\}.$$

Unfortunately, the estimand Equation (4) is not a causal effect, as it comprises a comparison of different cluster-level outcomes, namely $Y_i^{a=1}$ and ${\tilde{Y}}_i^{\widehat{a}=1}$. As noted above, for an estimand to have a causal interpretation, the same outcome must be compared under different counterfactual scenarios.

It is conventional, although not incontrovertible,³³ to define causal effects only for a treatment or exposure that is manipulable, that is, there can be “no causation without manipulation”.³⁴ If this convention is followed, then in cluster-randomized trials with non-participation, the direct effect of vaccination would only be considered well defined in always participators. Otherwise, to define the relevant potential outcomes would require considering a counterfactual scenario where non-participators receive vaccine. However, for the study design under consideration, always participators receive vaccine if and only if other always participators in their cluster also receive vaccine. Thus, it is not possible to observe both (a) a vaccinated always participator and (b) an unvaccinated always participator, while holding fixed the proportion of other individuals who are vaccinated in the cluster; hence the direct effect is not identifiable without additional assumptions.

On the other hand, if the “no causation without manipulation” convention is not adopted, there are other complications that may arise with estimating the direct effect. In particular, in cluster-randomized trials with non-participation, vaccine coverage within a cluster is dictated by the collective level of individual participation in the study, which is not under the investigator’s control. Factors associated with participation may also be associated with the outcome of interest, creating the potential for confounding. Thus causal inference methods for observational studies, such as those assuming no unmeasured confounding, would in general be necessary to draw inference about direct effects. To be concrete, consider the counterfactual scenario (or policy) where individuals independently receive vaccine with probability α. Let A_ij denote the vaccination status of individual j in cluster i, and let $A_i=\left(A_{i1},A_{i2},\dots ,A_{i{n}_i}\right)$. The random vector A_i takes on values a_i in the set $A\left(n_i\right)={\{0,1\}}^{n_i}$. Let Y_ij(a_i) denote the potential outcome for individual j (in cluster i) corresponding to a_i. The potential outcomes Y_ij(a_i) may also be expressed as Y_ij(a_i,−j,a_ij) where a_i,−j denotes the vector of treatment indicators for all individuals except individual j and a_ij is the treatment indicator for individual j. Define the average outcome for individual j when vaccinated under policy α by

$$Y_{ij}(1;\alpha )=\sum _{\mathbf{b}\in \mathcal{A}\left(n_i-1\right)}{Y}_{ij}\left({\mathbf{a}}_{i,-j}=\mathbf{b},a_{ij}=1\right)P_{\alpha }\left({\mathbf{A}}_{i,-j}=\mathbf{b}\right)$$

where P_α denotes the probability under policy α. Define Y_ij(0;α) analogously such that Y_ij(0;α) is the average outcome for individual j when not vaccinated under policy α. Then define the direct effect under policy α to be

$$E\left[{\overline{Y}}_i(1;\alpha )\right]-E\left[{\overline{Y}}_i(0;\alpha )\right]$$ (5)

In the cluster-randomized trial setting considered in this paper, individuals self-select whether to participate such that it would be dubious to assume treatment received is independent of an individual’s potential outcomes. However, in some settings, it might be reasonable to assume there exists some vector of baseline covariates, say L_i, such that the set of potential outcomes for individuals within cluster i are conditionally independent of the treatment selected given these covariates, that is, Y_ij(a_i)⊥A_i|L_i. This is a cluster level version of the usual no unmeasured confounders assumption. Under this assumption, inverse probability weighted estimators have been proposed which are consistent for Equation (5).^35,36

3 |. TYPHOID VACCINE TRIAL

A cluster-randomized study was conducted to investigate the effectiveness of a Vi polysaccharide typhoid vaccine in Kolkata, India, over 2 years of follow-up from 2004 to 2006.⁵ The control in this trial was an inactivated hepatitis A vaccine. Geographic mapping and a census that characterized and counted all people and households in the study area were used to define 80 clusters. For purposes of randomization, clusters were stratified by ward (an administrative unit of Kolkata) and by the number of residents in certain age groups. Overall, 40 clusters were assigned to Vi vaccine and the other 40 to control. Because data from the typhoid trial are not publicly available, a simulated data set was constructed (see Data S1). The data were simulated to match exactly the cluster level summary statistics from the actual trial shown in Table 1.

TABLE 1.

Summary statistics of a cluster-randomized study in Kolkata from 2004 to 2006 of a Vi typhoid vaccine vs a hepatitis A control vaccine⁵

	Typhoid vaccine	Control
Number of clusters	40	40
Mean ± SD of people per cluster	777 ± 136	792 ± 142
Mean ± SD of participants per cluster	472 ± 103	470 ± 104
Number of participants	18 869	18 804
Number of non-participants	12 206	12 877
Number of events in participants	34	96
Number of events in non-participants	16	31

Sur et al⁵ measure vaccine effects in terms of hazard ratios. However, causal interpretations for hazard ratios are difficult because hazard ratios can depend on time and have an inherent selection bias.³⁷ In particular, time-specific hazard ratios compare different subsets of subjects and, as noted above, estimands have a causal interpretation only when comparing potential outcomes between the same set (or subset) of units. Due to these issues, instead of using the hazard ratio to determine the vaccine effects as in Sur et al,⁵ the risk difference of typhoid over 2 years is calculated here to quantify vaccine effects.

The overall, indirect, and total effects were estimated using Equation (2) with the Y_i definitions provided in Section 2.3. The effect estimates, estimated SEs, and 95% Wald confidence intervals (CIs) are shown in Table 2. For example, the overall effect estimate was obtained by taking the difference in the average number of cases of typhoid per 1000 individuals between Vi clusters and control clusters. In particular, Vi clusters had 1.61 cases of typhoid per 1000 people, while control clusters had 4.10 cases of typhoid per 1000 people. Thus, the overall effect estimate is −2.49 cases per 1000 people. The SE of the overall effect estimate was calculated by ${\left\{{\widehat{\sigma }}_0^2+{\widehat{\sigma }}_1^2\right\}}^{1/2}$ where ${\widehat{\sigma }}_a$ denotes the estimated SE for clusters assigned a = 0 (control), 1 (Vi). Finally, a 95% Wald CI was estimated in the usual manner with a result of (−3.41, −1.58). The overall effect estimate has a straightforward interpretation which may be of interest to public health officials such as epidemiologists. In particular, the number of cases of typhoid per 1000 persons over a 2 year period is estimated to decrease by 2.5 on average when a cluster receives the Vi vaccine compared to receiving control.

TABLE 2.

Estimates of overall, indirect, and total effects, SE, and 95% Wald confidence intervals (CI)

Effect	Estimate (SE)	95% CI
Overall	−2.49 (0.47)	(−3.41, −1.58)
Indirect	−1.29 (0.56)	(−2.38, −0.19)
Total	−3.30 (0.67)	(−4.61, −1.99)

Note: Effect estimates are differences in typhoid cases per 1000 people per 2 years.

Both participants and non-participants appear to benefit from the Vi vaccine. In particular, over the study period, on average, there were 1.85 cases of typhoid per 1000 participants in Vi clusters, and 5.15 cases of typhoid per 1000 participants in control clusters. Thus, the total effect estimate is −3.30 (95% CI −4.61, −1.99), indicating that assigning a cluster to Vi vaccine causes 3.3 fewer cases of typhoid per 1000 participants compared to assigning a cluster to hepatitis A vaccine. Likewise, Vi clusters had 1.29 cases of typhoid per 1000 non-participants on average, while control clusters had 2.58 cases of typhoid per 1000 non-participants on average over the study period. Taking the difference between these values gives an indirect effect estimate of −1.29 (95% CI −2.38, −0.19). The indirect effect estimate suggests that assigning a cluster to the typhoid vaccine results in 1.29 fewer cases per 1000 non-participants; as non-participants never receive the vaccine, this indicates an indirect (or herd immunity) effect of the typhoid vaccine.

On the other hand, the naive direct effect estimator Equation (3) equals 0.56 (95% CI −0.44, 1.55). Although not statistically significant, this point estimate implies that the average number of cases of typhoid per 1000 people is higher in vaccinated individuals compared to non-vaccinated individuals in clusters randomized to the Vi vaccine. However, as described above, this estimate cannot be interpreted as an effect of the vaccine as discussed in Section 2.4. For example, perhaps individuals at higher risk of typhoid chose to participate in the trial, or those who participated tended to have different health care seeking behavior. Moreover, the average number of cases of typhoid per 1000 people was also higher in participants compared to non-participants (2.57, 95% CI 1.19, 3.96) in the control clusters, providing direct evidence of confounding. Sur et al⁵ reported similar results, with incidence of typhoid higher in participants compared to non-participants, both within Vi vaccine clusters and within control clusters.

4 |. DISCUSSION

Randomized controlled trials are the gold standard in vaccine trials since randomization ensures that the vaccine and control groups are comparable. Carefully defining estimands in clinical trials is vital to ensure accurate interpretation of the resulting treatment effect estimates. Because cluster-randomized trials can be large and expensive to conduct, it is important to formally characterize estimands for use in these trials. This paper considers causal estimands in cluster-randomized trials where interference may be present within clusters. An illustrative example is provided motivated by a recent cluster-randomized typhoid vaccine trial demonstrating inference and interpretation of the overall, total, and indirect effect estimands. These types of analyses can be used to inform public health policies regarding vaccination.

In cluster-randomized trials with self-selection, estimators of the direct effect must account for possible confounding. As described at the end of Section 2, a standard method to adjust for confounding is to condition on covariates and assume that conditional on these covariates, participants and non-participants are exchangeable. A possible indirect way to adjust for confounding could involve comparing outcomes between participants and non-participants in the control clusters as an estimate of the confounding bias, if present, similar to negative control approaches described in Lipsitch et al³⁸ and Tchetgen Tchetgen.³⁹ Alternatively, two-stage randomized designs could be considered to eliminate possible confounding when drawing inference about the direct effect. In two-stage randomized experiments, clusters are first randomly assigned to a treatment allocation program, then individuals within those clusters are assigned to treatment or control based on their cluster’s treatment allocation program.¹³ Randomization eliminates possible confounding at the cluster and individual level, such that direct, indirect, total, and overall effects can be estimated.^13,40,41 However, it may not always be feasible to conduct two-stage randomized trials. In addition, the effects estimated by a two-stage randomized experiment are not equivalent to the effects estimated in cluster-randomized trials with participation self-selection and may have less public health relevance.^42,43

Estimated effects may have greater real-world relevance depending on the estimands of interest and characteristics of individuals in the trials, such as the level of participation. Westreich⁴⁴ provides several examples of population intervention effects defined by contrasts in average potential outcomes under different possible interventions on the distribution of treatment. These population intervention effects may be more germane to real-world policy than the traditional approach of defining causal effects by comparing average outcomes when all individuals in the population receive treatment vs when no individuals receive treatment. The estimands described here for cluster-randomized trials with self-selection are examples of population intervention effects, to the extent that the participation rate in the trial approximates vaccination uptake should the vaccine under evaluation become widely available to the public. For example, in Sur et al,⁵ about 60% of individuals on average chose to be vaccinated in both Vi and hepatitis A clusters; thus, the overall, total, and indirect effect estimates approximate the effects of vaccinating 60% of the population. Such effect estimates could potentially help inform public health policy decisions regarding vaccination.