Propensity score methods for observational studies with clustered data: A review

Abstract

Propensity score methods are a popular approach to mitigating confounding bias when estimating causal effects in observational studies. When study units are clustered (eg, patients nested within health systems), additional challenges arise such as accounting for unmeasured confounding at multiple levels and dependence between units within the same cluster. While clustered observational data are widely used to draw causal inferences in many fields, including medicine and healthcare, extensions of propensity score methods to clustered settings are still a relatively new area of research. This article presents a framework for estimating causal effects using propensity scores when study units are nested within clusters and are nonrandomly assigned to treatment conditions. We emphasize the need for investigators to examine the nature of the clustering, among other properties, of the observational data at hand in order to guide their choice of causal estimands and the corresponding propensity score approach.

1. INTRODUCTION

Observational studies are commonly used to evaluate the effect of an intervention or exposure (hereafter collectively referred to as “treatment”) in the medical and healthcare literature, because random assignment to treatment conditions may be unethical or impossible. Unlike in randomized control trials (RCTs), where the treatment effect can be estimated unbiasedly by simply comparing the outcome between treatment groups, in observational studies, the contrast in outcome may not accurately reflect the treatment effect due to a potential lack of comparability between groups on pretreatment characteristics associated with the outcome (eg, prognostic factors). This distortion of the treatment effect is often referred to as “confounding bias.” ¹ While regression adjustment is frequently used to adjust for confounders, over the past two decades, propensity score methods have seen increasing application in observational studies as a less parametric approach to reducing confounding bias compared to regression adjustment. ² , ³ Propensity score methods are less parametric in the sense that they are “design‐based,” in which one attempts to create a sample with treatment groups balanced on baseline characteristics, thus mimicking an RCT at least with respect to the observed characteristics, and confirming that balance is obtained, before analyzing the outcome data. ⁴ , ⁵

Many of the settings in which treatment effects are being estimated are multilevel (eg, students nested within classrooms or schools, patients nested within hospitals or health systems), and this structure is common in many disciplines, such as education, psychology, epidemiology, and medicine. However, extensions of propensity score methods to multilevel settings are still a relatively new area of research, and even defining the causal estimands in this context is not always straightforward. Several articles have provided a review of using propensity score methods with clustered data, ⁶ , ⁷ , ⁸ but most of these are geared toward an education or social sciences audience. This article aims to bring attention to this topic among clinical and public health researchers by providing an up‐to‐date guide for estimating causal effects with clustered observational data using existing propensity score methods. For simplicity, we will focus on two‐level hierarchical structures and binary treatment assignment at the individual level (ie, individuals are nested within clusters and are nonrandomly assigned to treatment conditions), but the ideas can be easily extended to higher level structures. The different implications of cluster‐level treatment assignment (ie, when the treatment is administered at the cluster level) will be briefly discussed.

The article proceeds as follows. The remainder of this section discusses two different contexts that involve clustered observations and their implications for the basic ideas and assumptions underlying the use of propensity scores for causal inference. Section 2 discusses models for the estimation of propensity scores with clustered data. Section 3 reviews three common uses of the propensity score—matching, stratification, and weighting—and their extension to clustered settings, each followed by estimation of the treatment effect. Lastly, Section 4 emphasizes key points, provides a brief discussion on cluster‐level treatment assignment, and suggests directions for future research.

1.1. Two perspectives on clustering

Before conducting propensity score analysis in the context of clustered data, it is important to recognize the nature of the clustering. Here we provide a brief review of two perspectives on clustering detailed in Thoemmes and West ⁸ : clustering as a central vs incidental feature of the design. In cases where clustering is a central feature, treatment implementation and the assignment mechanism (ie, the selection of individuals into treatment conditions) may vary across clusters. ⁸ For example, patients' insurance status may strongly predict clinical decisions in some but not other hospitals. Thus, the propensity score analysis in these cases seeks to approximate a multisite randomized trial where random assignment is implemented within clusters. ⁸ Achieving balance on pretreatment characteristics within each cluster is particularly important if cluster‐specific treatment effects are of interest to the investigator. ⁸ The overall treatment effect can then be estimated by pooling the cluster‐specific treatment effects, for example, with weights proportional to cluster sizes. On the other hand, in cases where clustering is an incidental feature, treatment implementation and the assignment mechanism are presumably identical across clusters; the propensity score analysis thus seeks to approximate a single‐level randomized experiment where individuals are grouped simply by coincidence (eg, by being in the same room at the same time when treatment is assigned). ⁸ In these cases the estimand of interest is the treatment effect in the entire study population, and thus achieving covariate balance over the entire sample is the primary goal. ⁸ With incidental clustering, cluster‐level covariates are generally not as meaningful and important as individual‐level covariates in affecting causal conclusions. Hence, most of the issues related to clustering discussed in this article will generally be less of a concern for studies with incidental clustering and more prominent for the first case.

Being explicit about the clustering nature of the data can help guide the choice of treatment effect estimands (overall and/or cluster‐specific treatment effects), important covariates to adjust for (individual‐level and/or cluster‐level confounders), and appropriate propensity score strategies. In this article we thus adopt Thoemmes and West's ⁸ perspectives on clustering and will discuss in further detail the additional nuances and suitable approaches for each type of clustering.

1.2. Notations, assumptions, and propensity score analysis with clustered data

We now provide an introduction to defining causal effects and using propensity scores for causal inference in observational studies while raising potential issues and complexities specific to clustered settings. For a two‐level structure, we use $h$ to index clusters and $k$ to index individuals within a cluster ($k=1,2,\dots ,n_h$, where $n_h$ is the number of individuals in cluster $h$), with the assumption that cluster membership is fixed for each individual. Let $N=\sum _h{n}_h$ be the total number of individuals in the study population.

The treatment effect is defined based on Rubin's causal model, ⁹ , ¹⁰ including the stable unit treatment value assumption, or SUTVA for short. ¹¹ SUTVA has two components: (i) an individual's outcome is unaffected by the treatment condition received by other individuals; (ii) there is only one version of each treatment condition. Under SUTVA, each individual has two potential outcomes associated with a binary treatment: $Y_{\mathit{hk}}(1)$ (potential outcome under treatment) and $Y_{\mathit{hk}}(0)$ (potential outcome under the control condition). SUTVA is more likely to be violated in multilevel settings compared to single‐level settings, especially when clustering is a central feature. An example of violation to component (ii) is when there are variations in a particular surgical procedure (treatment) across hospitals (clusters). Component (ii) can be relaxed to allow these variations in treatment but for each individual they should all have the same effect on the outcome; that is, each individual should have a well defined outcome under “treatment.” ¹² Component (i) is questionable in settings where individuals in the same cluster are highly connected, for example, students in the same classroom, such that their outcomes may be affected by the treatment received by their peers. In their study on the effect of kindergarten retention, Hong and Raudenbush ¹³ developed a causal model that relaxes this component of SUTVA by allowing school assignment and the retention rate of a child's school to affect the child's potential outcomes. Specifically, they modeled peer effects on a child's potential outcomes as operating through a scalar function that characterized the school retention rate as low or high. Hence, each child had four instead of two potential outcomes, depending on whether the child was retained and whether the child attended a low or high‐retention school. ¹³ Most of the existing propensity score methods in the multilevel context, however, are still carried out assuming SUTVA. Therefore, the concepts and methods discussed in this article will be based on Rubin's potential outcomes framework and with SUTVA assumed unless otherwise noted.

The individual treatment effect is defined as the difference between the two potential outcomes, $Y_{\mathit{hk}}(1)-Y_{\mathit{hk}}(0)$, which can never be directly observed because only the potential outcome corresponding to the treatment condition received is observed for each individual. Inference thus is often interested in treatment effects averaged across individuals. The average treatment effect (ATE) in the population is defined as the expected value of the individual treatment effects; here we average across individuals such that each individual receives equal weight:

$$\mathrm{ATE}=E\left[Y_{\mathit{hk}}(1)-Y_{\mathit{hk}}(0)\right].$$

Another treatment effect estimand that is commonly of interest is the average treatment effect on the treated (ATT):

$$\mathrm{ATT}=E\left[Y_{\mathit{hk}}(1)-Y_{\mathit{hk}}(0)\right|Z_{\mathit{hk}}=1],$$

where $Z_{\mathit{hk}}$ denotes the assigned treatment condition (1 if treated and 0 otherwise). ATT is estimated instead of ATE when interest lies in the treatment effect among those who would receive the treatment in reality or for whom the treatment is intended. ¹⁴ , ¹⁵

As mentioned earlier, cluster‐specific ATEs or ATTs may be of interest when clustering is a central feature; these are defined as:

$${\mathrm{ATE}}_h=E\left[Y_{\mathit{hk}}(1)-Y_{\mathit{hk}}(0)|H=h\right]\text{or},$$

$${\mathrm{ATT}}_h=E\left[Y_{\mathit{hk}}(1)-Y_{\mathit{hk}}(0)\right|Z_{\mathit{hk}}=1,H=h],$$

where $H$ denotes cluster membership. In some cases it may also be useful to estimate the between‐cluster variance of the ATEs or ATTs to examine treatment effect heterogeneity.

Confounding can occur at different levels in multilevel settings. Let $\mathit{X}$ and $\mathit{V}$ denote a set of observed, and supposedly correctly measured, individual‐level and cluster‐level covariates, respectively. We note that cluster‐level covariates may be aggregate versions of the individual‐level covariates. For unbiased estimation of the treatment effect using propensity scores, the following identifying assumptions, which jointly define the strong ignorability condition, are assumed ¹⁶ :

1. Unconfoundedness:

   $$\left(Y_{\mathit{hk}}(1),Y_{\mathit{hk}}(0)\right)\perp Z_{\mathit{hk}}\mid \left({\mathit{X}}_{\mathit{hk}},{\mathit{V}}_h\right).$$

2. Positivity or overlap:

$$0<\mathrm{Pr}[Z=1|\mathit{X}\mathbf{,}\mathit{V}]<1.$$

Unconfoundedness states that treatment assignment is independent of the potential outcomes conditional on a set of observed covariates. For estimation of the ATT, only $Y_{\mathit{hk}}(0)\perp Z_{\mathit{hk}}\mid \left({\mathit{X}}_{\mathit{hk}},{\mathit{V}}_h\right)$ is needed. Violation of this assumption—specifically, the omission of cluster‐level confounders in the analysis (ie, if $\mathit{V}$ does not include all cluster‐level confounders)—is referred to as the “unmeasured context” problem by Arpino and Mealli, ¹⁷ and is a focus of much of the existing literature on propensity score methods with clustered data. ³ , ¹⁷ , ¹⁸ , ¹⁹

A variation on the unconfoundedness assumption in the multilevel causal inference literature is “latent unconfoundedness” ¹⁸ , ²⁰ :

$$\left(Y_{\mathit{hk}}(1),Y_{\mathit{hk}}(0)\right)\perp Z_{\mathit{hk}}\mid \left({\mathit{X}}_{\mathit{hk}},{\mathit{V}}_h,{\alpha }_h\right),$$

where ${\alpha }_h$ is cluster‐specific and captures the effect of all unobserved cluster‐level confounders, meaning that this modified version of (i) does not require all cluster‐level confounders to be observed and included in $\mathit{V}$. As discussed further below, ${\alpha }_h$ may be considered as a fixed parameter for each cluster $h$, or a random variable drawn from a distribution. ¹⁸

Positivity implies that there exists both treated and control individuals for all combinations of the observed covariates. For estimation of the ATT, positivity is only needed for combinations of the observed covariates that are present among the treated individuals. If these assumptions hold, we say that treatment assignment is strongly ignorable given $\mathit{X}$ and $\mathit{V}$. When estimating cluster‐specific treatment effects, only the individual‐level covariates $\mathit{X}$ are required since $\mathit{V}$ is constant within clusters. That is, to estimate ${\mathrm{ATT}}_h$ (effects within cluster $h$) we only need to assume that treatment assignment is strongly ignorable given $\mathit{X}$. However, as will be further discussed in subsequent sections, it is often difficult or impossible to estimate cluster‐specific treatment effects with small clusters; one extreme example being when all individuals in a cluster receive the same treatment condition, which is more likely to happen when the clusters are small.

The propensity score will be a key tool for estimating treatment effects in the settings we consider. For individual $k$ in cluster $h$, their propensity score is their probability of receiving treatment conditional on the values of their observed covariates:

$$e_{\mathit{hk}}=\mathrm{Pr}\left[Z_{\mathit{hk}}=1|{\mathit{X}}_{\mathit{hk}}\mathbf{,}{\mathit{V}}_h\right].$$

Rosenbaum and Rubin ¹⁶ showed that the propensity score is a balancing score, meaning that conditional on the propensity score, the distribution of the covariates entered in the propensity score model would be balanced across treatment groups, and that the treatment assignment is strongly ignorable given $e_{\mathit{hk}}$ if it is strongly ignorable given $\mathit{X}$ and $\mathit{V}$. Given these properties, the propensity score alone can be used to equate the treatment groups on observed covariates ($\mathit{X}$ and $\mathit{V}$). As will be seen in Section 2, to alleviate the unmeasured context problem (ie, if $\mathit{V}$ fails to include all cluster‐level confounders), the propensity score model can also include a cluster‐specific term (${\alpha }_h$), based on the latent unconfoundedness assumption described earlier:

$$e_{\mathit{hk}}=\mathrm{Pr}\left[Z_{\mathit{hk}}=1|{\mathit{X}}_{\mathit{hk}}\mathit{,}{\alpha }_h\right]\text{if}{\alpha }_h\text{is considered}\mathrm{as}\mathrm{a}\text{fixed parameter},\text{or}$$

$$e_{\mathit{hk}}=\mathrm{Pr}\left[Z_{\mathit{hk}}=1|{\mathit{X}}_{\mathit{hk}}\mathit{,}{\mathit{V}}_h\mathit{,}{\alpha }_h\right]\text{if}{\alpha }_h\text{is considered}\mathrm{as}\mathrm{a}\text{random variable}.$$

Propensity score analysis for causal inference generally involves the following key steps: (1) estimate the propensity score; (2) induce covariate balance between treatment groups using an appropriate strategy, including matching units on propensity scores, grouping units into strata with similar propensity scores, or applying propensity score weights; (3) check covariate balance with metrics such as the standardized mean difference. This may be done both within and across clusters when clustering is a central feature; the thinking behind this is articulated further below. Researchers may repeat steps (1)‐(3) to re‐estimate the propensity scores using a different model until adequate balance is achieved; (4) estimate the ATE(s) or the ATT(s); (5) perform sensitivity analysis to assess the impact of unobserved confounders on treatment effect estimates. We will discuss step (1) in the next section and steps (2)‐(4) in Section 3. Many sensitivity analysis methods (step 5) have been developed in the single‐level context, ²¹ , ²² , ²³ but sensitivity analysis for clustered data is an area in need of more research. For a review of propensity score methods in a single‐level context, see, for example, Austin ²⁴ and Stuart. ²⁵ Those interested in carrying out the above in a multilevel context may consider the R package multilevelPSA, which is specifically developed to implement propensity score analysis for multilevel data, including estimating a separate propensity score model for each cluster, assessing covariate balance and summarizing or visualizing results at different levels. ²⁶

2. PROPENSITY SCORE ESTIMATION WITH CLUSTERED DATA

For binary treatments, the following logistic regression models are often used to estimate propensity scores in a multilevel context, depending on assumptions of the assignment mechanism and specifically, whether unconfoundedness holds conditional on the observed covariates (which may include covariates at any level) only or additionally on cluster membership (${\alpha }_h$ in the latent unconfoundedness assumption).

2.1. Single‐level models

Single‐level models include the observed individual‐level covariates ${\mathit{X}}_{\mathit{hk}}$ and cluster‐level covariates ${\mathit{V}}_h$ (the index $k$ in the subscript is omitted since $\mathit{V}$ is constant within clusters) as predictors:

$$\text{logit}\left(e_{\mathit{hk}}\right)=\alpha +{\mathit{X}}_{\mathit{hk}}\mathit{\beta }+{\mathit{V}}_h\mathit{\gamma }.$$

$\alpha$ is an intercept, $\mathit{\beta }$ and $\mathit{\gamma }$ are vectors of coefficients for the individual‐ and cluster‐level covariates, respectively. Interaction (either same‐level or cross‐level) and polynomial terms may be included for more flexible modeling of the propensity score. Single‐level models assume that no confounding remains after conditioning on the observed covariates (${\mathit{X}}_{\mathit{hk}}$ and ${\mathit{V}}_h$), which is unlikely in reality. ³ These models can be a reasonable choice if the assignment mechanism is independent of cluster membership, which is more plausible when clustering is incidental in the observational study. ⁸ The propensity scores estimated from single‐level models are used to achieve balance on the observed covariates across clusters but not necessarily within clusters. ²⁷

2.2. Fixed effects models

Fixed effects models assume unconfoundedness conditional on the observed individual‐level covariates and on cluster membership, thus the inclusion of cluster indicator variables or equivalently, cluster‐specific intercepts:

$$\text{logit}\left(e_{\mathit{hk}}\right)={\alpha }_h+{\mathit{X}}_{\mathit{hk}}\mathit{\beta },$$

where ${\alpha }_h$ is a fixed intercept term for cluster $h$. Fixed effects models do not include cluster‐level covariates ${\mathit{V}}_h$ because the cluster‐specific intercept captures the effects of both observed and unobserved cluster‐level covariates, thus protecting against the unmeasured context problem. ³ , ¹⁷ , ²⁸ However, the number of parameters to be estimated increases considerably when there are many clusters, especially when interactions between cluster membership and the individual‐level covariates are considered, which is equivalent to allowing the slopes of the individual‐level covariates to vary across clusters ⁸ :

$$\text{logit}\left(e_{\mathit{hk}}\right)={\alpha }_h+{\mathit{X}}_{\mathit{hk}}{\mathit{\beta }}_h.$$

In these cases the clusters need to be large enough and have a reasonable number of individuals in each treatment condition in order to support estimation of the model. ³ , ⁸

2.3. Random effects models

An alternative to fixed effects models that reduces the number of parameters are random effects models:

$$\text{logit}\left(e_{\mathit{hk}}\right)={\alpha }_h+{\mathit{X}}_{\mathit{hk}}\mathit{\beta }+{\mathit{V}}_h\mathit{\gamma },{\alpha }_h\sim N\left(0,{\sigma }^2\right).$$

The main difference between fixed and random effects models is that the intercept term ${\alpha }_h$ is considered fixed for each cluster in fixed effects models (by including cluster indicator variables), whereas ${\alpha }_h$ usually follows a normal distribution in random effects models. In addition to a random intercept, random slopes can also be specified:

$$\text{logit}\left(e_{\mathit{hk}}\right)={\alpha }_h+{\mathit{X}}_{\mathit{hk}}{\mathit{\beta }}_h+{\mathit{V}}_h{\mathit{\gamma }}_h,$$

$${\alpha }_h\sim N\left(0,{\sigma }^2\right),{\mathit{\beta }}_h\sim N\left(0,{\mathrm{\sum }}_{\mathit{\beta }}\right),{\mathit{\gamma }}_h\sim N\left(0,{\mathrm{\sum }}_{\mathit{\gamma }}\right).$$

Kim and Seltzer ²⁹ found that random intercept and slopes (RIS) models yielded improved covariate balance within clusters compared to random intercept only (RI) models when the effects on treatment assignment of certain individual‐level covariates vary across clusters. The choice of whether to include random slopes, though, should hinge upon feasibility as well as the (presumed) degree of robustness of the model to omission of random slopes, which may vary by study. ⁷ One caveat of random effects models is that the observed covariates and the unobserved cluster‐level covariates are assumed to be independent, ¹⁹ and because the random effects shrink toward zero, there may be more residual imbalance as compared to the fixed effects approach. ³

Both fixed effects and random effects models acknowledge the potential existence of unobserved cluster‐level confounders and follow the latent unconfoundedness assumption with the inclusion of ${\alpha }_h$. ¹⁸ Several studies ³ , ⁸ , ¹⁷ have compared the two models with single‐level models for propensity score estimation. Although these studies have very different simulation setups and use the propensity scores in different ways (matching, stratification, or weighting), they all showed that fixed and random effects models outperformed single‐level models (which include observed individual‐ and cluster‐level covariates) in reducing bias, due to their ability to capture remaining cluster heterogeneity.

Fixed effects models with cluster‐specific slopes and RIS models mimic the assignment mechanism in a multisite randomized trial, and the estimated propensity scores are not comparable across clusters because of the different regression slopes. ⁸ As a result, these models aim to achieve balance within clusters rather than across clusters and thus are preferable when ${\mathrm{ATE}}_h$ or ${\mathrm{ATT}}_h$ may be of interest.

Fixed effects models with cluster‐specific slopes are generally the ideal approach when clustering is a central feature, both because they acknowledge the potential heterogeneity in the assignment mechanism across clusters, and because they have the advantage of automatically controlling for all cluster‐level covariates so that only the individual‐level covariates are needed. However, in reality cluster sizes are often too small to support propensity score modeling within clusters. ²⁷ A few strategies have been proposed to overcome the issue with small clusters by combining similar clusters into larger “groups” or “classes.” ³⁰ , ³¹ , ³² , ³³ For example, Kim and Steiner ³⁰ identified homogeneous classes of clusters with respect to the treatment assignment model using latent class analysis; Lee et al ³³ formed cluster groups with similar treatment prevalence. A distinct propensity score model is then constructed for each group, or class, of clusters. These strategies were shown to be effective in reducing bias when cluster sizes are too small to estimate fixed cluster effects.

3. THE USE OF PROPENSITY SCORES AND TREATMENT EFFECT ESTIMATION WITH CLUSTERED DATA

The previous section discussed how propensity scores can and should be estimated in clustered settings; we now turn to how those propensity scores can be used in this context. In particular, this section reviews the extension to clustered data of three common methods that use propensity scores—matching, stratification, and weighting—and the estimation of treatment effects when using these methods. We remind the reader that the use of propensity scores can be viewed as the preprocessing stage to induce comparability between treatment groups on observed covariates in the data, which sets the basis for the outcome analysis stage where treatment effects are estimated with the preprocessed data. ²⁵ , ³⁴

3.1. Matching

Propensity score matching is the process of creating matched sets of treated and control individuals with similar values of the propensity score. It is generally used to estimate the ATT ³⁵ ; hence we focus on the estimation of ATT in this subsection. Two common algorithms for forming matched sets are greedy matching and optimal matching. ³⁶ For more details on these matching algorithms, we refer the reader to Austin, ²⁴ Stuart, ²⁵ and Gu and Rosenbaum. ³⁷ Here we focus on greedy k:1 nearest neighbor matching, which is one of the most common matching methods. ²⁵ In k:1 nearest neighbor matching, each treated individual is matched with one or multiple controls whose propensity score values are closest to theirs; unmatched controls are then discarded once all treated individuals have found their matches. To ensure quality matches, researchers may impose a prespecified caliper distance (eg, 0.2 of the SD of the logit of the propensity score) on the absolute difference in the propensity scores of matched individuals. ³⁸ Treated individuals with no controls falling within the caliper are discarded, which will then estimate a restricted ATT.

The considerations for matching in general apply to both single‐level and multilevel settings. With clustered data, researchers need to also choose between matching within or across clusters, or a combination of both (“hybrid” matching strategies).

3.1.1. Within‐cluster matching

Within‐cluster matching requires that matched individuals belong to the same cluster, which controls for all cluster‐level covariates automatically. Treatment effect estimates will then be free from the bias caused by unobserved cluster‐level confounders, which is desirable when strong cluster‐level confounding is likely. Hence, cluster‐specific propensity score estimation (ie, using a fixed effects model with cluster‐specific slopes) followed by within‐cluster matching is the ideal approach when clustering is a central feature and the aim is to approximate a multisite randomized trial. This achieves balance within clusters and could yield cluster‐specific effect estimates if the clusters are sufficiently large. ⁸ However, this approach may not always be feasible: with small cluster sizes, there may be insufficient propensity score overlap between treatment groups within clusters and many unmatched individuals may be discarded, leading to a more limited estimand, or the quality of the matches may be relatively low in terms of their similarity on the individual‐level covariates.

3.1.2. Across‐cluster matching

The other extreme is across‐cluster matching, which selects matches using the entire study population, irrespective of cluster membership. Therefore, across‐cluster matching requires a joint propensity score estimation model so that the propensity scores across clusters are comparable. ⁸ This achieves balance on the observed covariates in the entire matched dataset but may not produce balance on the individual‐level covariates within each cluster. ⁸ Hence, across‐cluster matching is most suitable under incidental clustering and when the estimand is the overall ATT. ⁸ Across‐cluster matching may retain a larger sample size compared to within‐cluster matching. However, it relies on stronger identifying assumptions than within‐cluster matching, as it requires the correct measurement and modeling of cluster‐level covariates. ³⁰ , ³⁹ The choice between within‐ and across‐cluster matching thus exemplifies a bias‐variance tradeoff and depends on how important it is to achieve balance on cluster‐level covariates.

3.1.3. Hybrid matching strategies

In many ways the choice of matching approaches comes down to how important it is to get balance on individual‐level vs cluster‐level covariates, often based on how important each set are in terms of predicting the outcome. Thus, alternative matching strategies have been developed to combine the benefits of both within‐ and across‐cluster matching. The general idea is to prioritize within‐cluster matches, but not at the expense of poor balance on the individual‐level covariates. Rickles and Seltzer ⁴⁰ proposed a two‐stage propensity score modeling (2SM) approach, which is an extension of Stuart and Rubin's ⁴¹ matching method with multiple control groups to the multilevel setting. The 2SM approach first attempts matching within clusters. For treated individuals without an acceptable match (ie, too dissimilar in terms of the individual‐level covariates), matches (“nonlocal” controls) are then searched across clusters but within a preidentified group of clusters that share similar characteristics as the cluster to which the treated individual belongs. ⁴⁰ Because across‐cluster matching may lead to poor covariate balance if it is based on propensity score models that vary across clusters, for each treated individual looking for nonlocal matches Rickles and Seltzer ⁴⁰ recalculated the propensity scores of nonlocal controls using the same model as the treated individual to ensure comparability of the propensity scores. In addition, Rickles and Seltzer ⁴⁰ addressed any potential imbalance in cluster‐level covariates by adjusting for between‐cluster differences in each nonlocal control's observed outcome before estimating treatment effects. Another popular hybrid matching strategy is “preferential” within‐cluster matching, proposed by Arpino and Cannas. ⁴² Like the 2SM approach, preferential within‐cluster matching prioritizes matching within clusters and searchers for matches across clusters only if no acceptable within‐cluster match can be found, but does not restrict across‐cluster matching to a group of similar clusters. ⁴² The R package CMatching provides functions that implement within‐cluster matching and preferential within‐cluster matching. ⁴³

3.1.4. Treatment effect estimation after matching

After a matched dataset with adequate balance is formed, researchers can proceed to the outcome analysis stage by comparing outcomes among the matched sample. In practice, a “doubly robust” approach, in which the ATT is estimated using a (multilevel) outcome model that includes both the treatment condition and observed covariates as predictors, is recommended. ⁴⁴ Note that weights may be applied if necessary, for example, when matching with replacement. In essence “doubly robust” means that when the propensity score and outcome models are used in combination, only one of the two models is needed to be correctly specified to provide a consistent estimate of the treatment effect. ⁴⁵ Treatment effect heterogeneity across clusters can be evaluated by allowing the slope associated with the treatment variable to vary by cluster.

A less ideal (because of the potential for residual confounding after matching) but simpler alternative is to estimate the ATT nonparametrically as follows:

$${\widehat{\tau }}^{\mathrm{ATT}}=\frac{1}{N_1}\sum _{h,k}\left(Y_{\mathit{hk}}-{\widehat{Y}}_{\mathit{hk}}(0)\right)Z_{\mathit{hk}}.$$

${\widehat{\tau }}^{\mathrm{ATT}}$ estimates the average of the individual treatment effects among the treated, since $Z_{\mathit{hk}}=0$ for controls and here $N_1$ is the number of treated individuals in the matched dataset. ${\widehat{Y}}_{\mathit{hk}}(0)$ is the estimated potential outcome under the control condition, which can be estimated by averaging the outcomes of all controls matched to individual $k$ in cluster $h$. If the ATT is estimated within each cluster, one can obtain an overall ATT estimate by pooling the cluster‐specific ATTs with weights proportional to the number of treated individuals in the cluster. The SE estimate of the nonparametric ATT estimator can be obtained via cluster bootstrapping (resampling the clusters rather than individuals). However, Abadie and Imbens ⁴⁶ found that bootstrap SEs were not valid for nearest‐neighbor matching with replacement and a fixed number of matches. An alternative is to simply regress the outcome on the treatment variable in the matched dataset, and obtain SEs that account for the within‐cluster dependence in the outcome due to shared cluster‐level characteristics. ⁴²

3.2. Stratification

Propensity score stratification involves dividing the sample into mutually exclusive strata based on the quantiles of the estimated propensity scores. Therefore, the distribution of the observed covariates should be roughly similar between the treatment groups within each stratum. Recent studies on the choice of the number of strata can be found in Neuhäuser et al ⁴⁷ and Orihara and Hamada. ⁴⁸ Another method that is closely related to propensity score stratification is optimal full matching (OFM). ⁴⁹ Unlike the matching methods discussed in Section 3.1, which may discard unmatched individuals, full matching forms matched sets that contain at least one treated and at least one control individual using the entire sample. ⁴⁹ , ⁵⁰ OFM is optimal in the sense that the mean of the within‐stratum distances in the propensity score between treatment conditions is minimized. OFM can thus be viewed as a form of propensity score stratification where the number of stratum and the grouping of individuals are optimized to minimize the propensity score differences within strata.

Like matching, stratification (or OFM) can be done either within or across clusters in multilevel settings, with the same considerations as with within‐ vs across‐cluster matching. However, stratification across clusters is more commonly done in applied studies ²⁷ , ⁵¹ as stratification within clusters is often hindered by small cluster sizes.

3.2.1. Treatment effect estimation after stratification

After a stratified sample is formed, treatment effects (or cluster‐specific treatment effects if stratification is done within clusters) can be estimated by taking the weighted average of the stratum‐specific treatment effects. Similar to matching, stratum‐specific treatment effects can be estimated nonparametrically (eg, by simply comparing the difference in the mean outcome between treated and control individuals irrespective of cluster membership) or more ideally, using a doubly robust approach (eg, by regressing the outcome on the treatment variable and all observed covariates, and incorporating fixed or random cluster effects). Stratum‐specific treatment effects are then weighted by the size of each stratum for estimating the ATE, or by the number of treated individuals in each stratum for estimating the ATT.

Another way to estimate the overall treatment effect, as demonstrated in Leite et al, ⁷ is by applying the “marginal mean weights” provided by Hong, ⁵² which are weights based on stratum membership and treatment assignment:

$$w^{\mathrm{ATE}}=\frac{N_s}{N_{z,s}}\mathrm{Pr}[Z=z],$$

$$w^{\mathrm{ATT}}=z+(1-z)\frac{N_{1,s}}{N_{0,s}}\times \frac{1-\mathrm{Pr}[Z=1]}{\mathrm{Pr}[Z=1]},$$

where $N_s$ is the total number of individuals in stratum $s$ and $N_{z,s}$ is the number of individuals with treatment level $z$ within stratum $s$. Investigators can then construct a weighted outcome model (preferably a model the accounts for the cluster structure) to obtain the ATE or ATT. ⁷

3.3. Weighting

Propensity score weighting is a technique that directly uses the estimated propensity scores to adjust for confounding bias. In multilevel settings, weights are constructed in the same manner as in single‐level settings. For estimating the ATE, propensity score weights, or “inverse probability of treatment weights” (IPTW), are: $w_{\mathit{hk}}=\frac{1}{{\widehat{e}}_{\mathit{hk}}}$ for treated individuals and $w_{\mathit{hk}}=\frac{1}{\left(1-{\widehat{e}}_{\mathit{hk}}\right)}$ for control individuals, where ${\widehat{e}}_{\mathit{hk}}$ is the estimated propensity score. For estimating the ATT, treated individuals receive a weight of 1, while the weight for control individuals is $\frac{{\widehat{e}}_{\mathit{hk}}}{\left(1-{\widehat{e}}_{\mathit{hk}}\right)}$. Similar to single‐level settings, the weights are applied to create a pseudo‐sample balanced on the observed covariates. In the case of estimating the ATE, both treatment groups are weighted to resemble the combined group in terms of the observed covariates, whereas in the case of estimating the ATT, control individuals are weighted to resemble the treated group.

Extreme weights can often lead to inflated variance estimates. To prevent extreme weights, one may consider the use of stabilized weights ⁵³ : $w_{\mathit{hk}}=\frac{P(Z=1)}{{\widehat{e}}_{\mathit{hk}}}$ for treated individuals and $w_{\mathit{hk}}=\frac{P(Z=0)}{1-{\widehat{e}}_{\mathit{hk}}}$ for control individuals when estimating the ATE. In multilevel settings, one may replace the across‐cluster average probability of treatment (or no treatment) in the numerator of these “marginal stabilized weights” by the cluster‐specific probability $P(Z=z|H=h)$ to create “cluster‐mean stabilized weights.” ⁵⁴ Schuler et al ⁵⁴ found that cluster‐mean stabilized weights outperformed marginal stabilized weights in the context of a continuous treatment.

3.3.1. Treatment effect estimation after weighting

The ATE or the ATT can simply be estimated by the difference of the weighted means of the outcomes between the two treatment groups (“marginal” estimator), ³ ignoring the cluster structure:

$${\widehat{\tau }}_{\mathit{ma}}=\frac{\sum _{h,k}{Z}_{\mathit{hk}}{Y}_{\mathit{hk}}{w}_{\mathit{hk}}}{\sum _{h,k}{Z}_{\mathit{hk}}{w}_{\mathit{hk}}}-\frac{\sum _{h,k}\left(1-Z_{\mathit{hk}}\right)Y_{\mathit{hk}}{w}_{\mathit{hk}}}{\sum _{h,k}\left(1-Z_{\mathit{hk}}\right)w_{\mathit{hk}}}.$$

On the other hand, a cluster‐weighted estimator first estimates the cluster‐specific treatment effects (${\widehat{\tau }}_h$), then takes a weighted average of these to estimate the overall treatment effect (${\widehat{\tau }}_{\mathit{cl}}$) ³ :

$${\widehat{\tau }}_{\mathit{cl}}=\frac{{\sum }_h{w}_h{\widehat{\tau }}_h}{{\sum }_h{w}_h},$$

where

$${\widehat{\tau }}_h=\frac{{\sum }_{k=1}^{n_h}{Z}_{\mathit{hk}}{Y}_{\mathit{hk}}{w}_{\mathit{hk}}}{{\sum }_{k=1}^{n_h}{Z}_{\mathit{hk}}{w}_{\mathit{hk}}}-\frac{{\sum }_{k=1}^{n_h}\left(1-Z_{\mathit{hk}}\right)Y_{\mathit{hk}}{w}_{\mathit{hk}}}{{\sum }_{k=1}^{n_h}\left(1-Z_{\mathit{hk}}\right)w_{\mathit{hk}}}\text{and}{w}_h={\sum }_{k=1}^{n_h}{w}_{\mathit{hk}}.$$

Li et al ³ recommended using the bootstrap (resampling the clusters) to estimate the SEs of these nonparametric estimators.

More recently, Yang ¹⁸ and He ¹⁹ proposed alternative propensity score estimation methods to improve the marginal estimator (${\widehat{\tau }}_{\mathit{ma}}$) such that it is robust to unobserved cluster‐level confounding: Yang ¹⁸ proposed a calibration method that involves defining the weights in a way that satisfies two constraints: (i) for each observed covariate, the sum in the full sample equals the weighted sum in each treatment group; (ii) in each cluster, the sum of the propensity score weights in each treatment group equals the size of the cluster. He ¹⁹ proposed conditional propensity score estimation, which utilizes a sufficient statistic for the unobserved cluster‐level covariates, for example, a function of the treatment allocations in the cluster. Both methods were shown to reduce bias in ${\widehat{\tau }}_{\mathit{ma}}$ due to omitted cluster‐level confounders under certain conditions, though Yang ¹⁸ cautioned that the robustness of the calibrated estimator may not hold if some individual‐level confounders were omitted.

In terms of parametric estimation, doubly robust estimators for the ATE and the ATT have been formulated as follows ³ , ⁵⁵ , ⁵⁶ :

$${\widehat{\tau }}_{\mathit{dr}}^{\mathrm{ATE}}={\sum }_{h,k}\frac{1}{N}\left\{\left[\frac{Z_{\mathit{hk}}{Y}_{\mathit{hk}}}{{\widehat{e}}_{\mathit{hk}}}-\frac{\left(Z_{\mathit{hk}}-{\widehat{e}}_{\mathit{hk}}\right){\widehat{Y}}_{\mathit{hk}}^1}{{\widehat{e}}_{\mathit{hk}}}\right]-\left[\frac{\left(1-Z_{\mathit{hk}}\right)Y_{\mathit{hk}}}{1-{\widehat{e}}_{\mathit{hk}}}-\frac{\left(Z_{\mathit{hk}}-{\widehat{e}}_{\mathit{ij}}\right){\widehat{Y}}_{\mathit{hk}}^0}{1-{\widehat{e}}_{\mathit{hk}}}\right]\right\};$$

$${\widehat{\tau }}_{\mathit{dr}}^{\mathrm{ATT}}={\sum }_{h,k}\frac{1}{N_1}\left[Z_{\mathit{hk}}{Y}_{\mathit{hk}}-\frac{\left(1-Z_{\mathit{hk}}\right)Y_{\mathit{hk}}{\widehat{e}}_{\mathit{hk}}+\left(Z_{\mathit{hk}}-{\widehat{e}}_{\mathit{hk}}\right){\widehat{Y}}_{\mathit{hk}}^0}{1-{\widehat{e}}_{\mathit{hk}}}\right].$$

${\widehat{Y}}_{\mathit{hk}}^1$ and ${\widehat{Y}}_{\mathit{hk}}^0$ are the fitted outcomes based on the outcome model in the treated and control groups, respectively. ³ As with the nonparametric estimators, the SEs of the doubly robust estimators can be estimated via cluster bootstrapping. ³ Alternatively, researchers can fit an outcome model on the propensity score weighted sample to obtain the treatment effect and cluster robust SEs.

Due to the direct use of propensity scores, weighting‐based estimators are particularly sensitive to incorrect specifications of the propensity score model. Potential solutions include using a doubly robust approach (combining the propensity score and outcome models), or the calibrated propensity score weighting estimator proposed by Yang, ¹⁸ which was shown to be robust to propensity score model misspecification.

4. DISCUSSION

For clustered observational studies in which individuals are nonrandomly assigned to treatment, propensity score methods should be modified to incorporate the cluster structure for two main reasons: (1) to account for within‐cluster correlation in both the propensity score and the outcome due to shared cluster‐level characteristics; (2) to account for potential unobserved cluster‐level confounding. While these may be less of an issue when clustering is incidental (and thus individuals in the same cluster may not share much in common), taking account of the cluster structure when clustering is a central feature not only reduces bias, but is also necessary for valid SE estimation.

Regardless of the propensity score method used, it is unanimously agreed in the literature that taking account of the cluster structure in either the propensity score estimation (eg, using a multilevel model to estimate propensity scores) or the outcome analysis stage (eg, using the cluster‐weighted IPTW estimator or a multilevel outcome model) can significantly reduce bias. Incorporating cluster information in both stages yields the least bias. ³ , ⁵⁷ In addition, Li et al ³ found that the choice of the outcome model may be more important for reducing bias than the propensity score model. Specifically, their simulation results suggested that failing to include the cluster structure in the outcome model may lead to larger bias compared to failing to include it in the propensity score model when unmeasured cluster‐level confounding is present. ³ More recently, Suk et al ⁵⁸ extended the causal forest algorithm in Athey and Wager ⁵⁹ to multilevel observational settings, showing promising performance when propensity scores estimated from multilevel models were used.

This review focuses exclusively on clustered observational studies where treatment is assigned to individuals. However, some research questions involve treatment assignment at the cluster‐level, such as an intervention at a whole school or community level. In this case, it is thus more important to control for cluster‐level covariates than to control for individual‐level covariates ⁶⁰ ; note too that the cluster‐level covariates may be aggregate versions of the individual‐level covariates. The outcome of interest can be measured either at the individual or the cluster level. For studies that measure the outcome at the cluster level, the ideas discussed under the usual unstructured data context can be directly applied, as the clusters themselves will be the units for analysis. ⁶¹ On the other hand, propensity score methods require some modifications for studies where interest is in the cluster‐level treatment effect on individual‐level outcomes, which is common for many health care interventions, for example, interventions on the health systems. These studies may approximate a clustered randomized control trial (clustered RCT), where randomization of treatment occurs at the cluster level. ⁶⁰ In terms of matching strategies for studies that emulate a clustered RCT, some have suggested to match the clusters only or to match clusters first and then match individuals within matched clusters. ³⁰ , ⁶² Recent work by Zubizarreta and Keele, ⁶³ however, proposed an optimal matching strategy in which individuals are first matched across all possible combinations of treated and control clusters, and then clusters are matched considering the individual‐level matches. Other methodological work that examines the effect of cluster‐level treatment on individual outcome includes Leyrat et al ⁶⁴ and Balzer et al. ⁶⁵

In the remaining of this section, we provide suggestions on areas for future research in the realm of multilevel causal inference.

4.1. Violation of SUTVA

As mentioned in Section 1, the SUTVA assumption is likely untenable in multilevel settings where cluster members are interconnected and influential to one another. However, most of the existing literature reviewed in this article have proceeded under SUTVA, both within and across clusters, without an explicit discussion on the violations to the assumption. A few studies have adopted a modified version of SUTVA that allows “partial interference”—that is, that there is interference between individuals in the same cluster, but no interference between individuals from different clusters. ⁶⁶ Examples include a kindergarten retention study in Hong and Raudenbush ¹³ and a study on the effect of students' eating habits and supportive school environments on obesity in Eckardt. ⁶⁷ Both studies capture the interference within schools through a binary variable indicating the school's treatment pattern which, together with the student's own treatment condition, defines the potential outcomes. This approximates a two‐stage randomization where schools are first randomly assigned to treatment allocation plans, and students are then assigned to treatment conditions within schools. Other examples can be found more broadly in the social networks and infectious disease literatures, in which the spillover effect (the effect of treatment received by others) is commonly of interest. For more propensity score‐based causal inference methods in these contexts, see, for example, Tchetgen Tchetgen and VanderWeele ⁶⁸ and Forastiere et al. ⁶⁹ Causal inference with interference is a highly active area of research, and propensity score methods allowing different interference patterns between individuals in multilevel settings is an important direction for future work.

4.2. Nonparametric propensity score estimation

In Section 2, we presented logistic regression models that are frequently used for the estimation of propensity scores with clustered data, but researchers can also use nonparametric techniques to estimate propensity scores; such an approach can be especially beneficial when the number of covariates is large. Nonparametric techniques, such as generalized boosted modeling, random forest, and neural networks, were shown to be promising alternatives to logistic regression for estimating propensity scores in single‐level settings. ⁷⁰ , ⁷¹ A recent study by Han and Kwak ⁷² showed that propensity scores estimated by a conditional inference tree produced better covariate balance than did propensity scores estimated by multilevel logistic regression models with two‐level clustered data, but the comparison was evaluated in an example with fairly large clusters (24 countries with 144 to 1532 schools per country). The performance of nonparametric propensity score estimation has yet to be investigated with different machine learning techniques and in different multilevel settings.

4.3. Insufficient overlap

Insufficient overlap (ie, violation of the positivity condition), either within or across clusters, between the treated and control groups is another issue that could be further addressed in future works. For example, the preferential within‐cluster matching approach introduced by Arpino and Cannas ⁴² was shown to be effective when the propensity score distribution showed sufficient overlap in the full sample, but its performance in situations where overlap is poor remains unclear. ⁴² Propensity score weighting generally does not discard individuals, but insufficient covariate overlap between treatment groups can lead to extreme weights, which may in turn result in bias and large variability of the treatment effect. ⁷³ Excluding individuals with propensity scores falling outside a pre‐specified range, or reducing large weights to a pre‐specified maximum value are common solutions to the insufficient overlap problem. ⁷⁴ , ⁷⁵ However, the methods for resolving insufficient overlap were developed under single‐level settings and it is unclear how they would work or how they should be modified in the multilevel context.

4.4. Variance calculation

Variance estimation remains a somewhat open issue in this literature. Two common ways to estimate the variance of the ATE/ATT with clustered data are to calculate a cluster robust variance, or to perform cluster bootstrapping (resampling clusters from the original sample) ⁶ ; the key is that the within‐cluster correlation induced by the clustered structure should be taken care of in some way. Because robust (or “sandwich”) variance estimators do not account for the uncertainty in estimating propensity scores, some have recommended the bootstrapping approach in the single‐level context (resampling individuals from the original sample, and reestimate propensity scores in each bootstrap sample). ⁷⁶ However, even in the single‐level context, more research is needed to examine the performance of bootstrap variances—and what the resampling units should be (eg, individuals or matched pairs)—under different settings and with different propensity score methods. For example, as mentioned in the previous section, bootstrap SEs are invalid if propensity score matching with replacement was done, ⁴⁶ but Austin and Small ⁷⁷ found that bootstrap SEs performed well if matching was done without replacement. A multilevel structure further complicates the operation of bootstrapping; Li et al ³ suggested cluster bootstrapping to calculate the variance of IPTW‐based estimators, but the validity of bootstrapping in the context of matching or stratification with clustered data remains unclear and understudied. Moreover, it is unclear how variance should be estimated if across‐cluster matching or a hybrid matching approach was done, given that the variance estimation would potentially need to account for the clusters as well as for the matched pairs/sets, but the matched pairs/sets would not be nested within clusters. Further work is needed to understand the best approaches for variance estimation for the various contexts and methods.

4.5. Sensitivity analysis

While there are plenty of sensitivity analysis methods in the causal inference literature, especially those that assess the sensitivity of results to an unobserved confounder, these methods have not yet been well established in multilevel settings. Recently, Scott et al ⁷⁸ extended the sensitivity analysis approaches in Imbens, ²² Carnegie et al, ⁷⁹ and Middleton et al ⁸⁰ to two‐level settings, allowing the evaluation of robustness to both individual‐ and cluster‐level unmeasured confounding. Further development and application of sensitivity analysis methods for unmeasured confounding at multiple levels could be an important area for future research.

This review presents a framework for examining treatment effects when study units are clustered in some way. Choice of the strategies and modeling approaches presented in this review should be based upon the study objective, study design, and assumptions of how cluster‐level characteristics may impact causal inferences. With the increasing use of large‐scale, multilevel data for clinical and public health research, it is important for investigators to be aware of such structure and explicate its implications for drawing causal conclusions.

Chang T‐H, Stuart EA. Propensity score methods for observational studies with clustered data: A review. Statistics in Medicine. 2022;41(18):3612‐3626. doi: 10.1002/sim.9437

DATA AVAILABILITY STATEMENT

Data sharing not applicable to this article as no datasets were generated or analysed during the current study