Commentary: Right truncation in cluster randomized trials can attenuate the power of a marginal analysis

Introduction

Cluster randomized trials (CRTs) provide a powerful and flexible experimental design to evaluate interventions delivered at the group level.¹ The clustered design, however, creates an analytical issue as the outcomes of participants in one cluster tend to be more similar to each other than those in different clusters, creating a positive within-cluster correlation that reflects extra variation attributable to each cluster. Power calculations for CRTs often account for such extra variability via the intracluster correlation coefficient (ICC) or the coefficient of variation (CV).

Motivated by a vector control trial with a count outcome (i.e. number of episodes of clinical malaria), Mwandigha et al.² examined the impact of right truncation on the statistical power of CRTs. The authors found that several factors in this setting result in the attenuation of statistical power. For this reason, existing sample size formulae may severely underestimate the required sample size when outcomes are truncated. In addition, because closed-form sample size formulae for truncated counts are lacking, they proposed a simulation-based approach to calculate power.

The results of their study are important and of practical value for designing CRTs. In this commentary, we seek to expand on their findings. Specifically, their current results are restricted to the conditional model (generalized linear mixed model), which carries a cluster-specific interpretation of the intervention effect. In contrast, the marginal model, estimated by the generalized estimating equations (GEE), is a commonly used alternative where the corresponding intervention effect parameter carries a population-averaged interpretation. We discuss the connections and differences between a conditional and marginal analysis of clustered counts and explore the effect of right truncation on the marginal analysis of CRTs.

Conditional model

Let $Y_{ij}$ represent the observed number of malaria episodes recorded for individual $j$ ($j=1,\dots ,m_i$) in village $i$ ($i=1,\dots ,N)$. Mwandigha et al.² assumed a random-effects model for the event rate ${\lambda }_{ij}$

$$\mathit{log}\left({\lambda }_{ij}\right)={\beta }_0+{\beta }_1{X}_i+u_i$$

where $X_i$ is the binary intervention indicator and $u_i\sim N(0,{\sigma }_c^2)$ is the cluster random effect reflecting the extra variability. The observed count follows a Poisson distribution subject to right truncation at $T\ge 1$, with the probability mass function (PMF)

$$P\left(Y_{ij}=y|0\le Y_{ij}\le T\right)=\frac{{\lambda }_{ij}^{y_{ij}}}{y_{ij}!Q_T\left({\lambda }_{ij}\right)},\mathrm{ }y=1,\dots ,T$$

where we define $Q_t\left(\lambda \right)={\sum }_{k=0}^t{\lambda }^k/k!$. The expectation of the truncated distribution is ${\lambda }_{ij}\frac{Q_{T-1}\left({\lambda }_{ij}\right)}{Q_T\left({\lambda }_{ij}\right)}$, which is no larger than the expectation of the original distribution, ${\lambda }_{ij}$.

In the absence of truncation such that $T=\mathrm{\infty }$, the above PMF corresponds to the standard Poisson because $Q_{\mathrm{\infty }}\left(\lambda \right)=e^{\lambda }$. In this case, $e^{{\beta }_0}$ is the baseline event rate conditional on the random effect $u_i=0$. The actual baseline event rate in each cluster is $e^{{\beta }_0+u_i}$, which differs from the overall baseline event rate $e^{{\beta }_0+{\sigma }_c^2/2}.$^3,4 The intervention effect $e^{{\beta }_1}$ is the rate ratio (RR) conditional on the latent random effect $u_i$. In the presence of right truncation, however, $e^{{\beta }_1}$ remains the conditional RR associated with the unobserved Poisson distribution had there been no truncation. Therefore, it is important not to mistakenly interpret the effect measure as the RR associated with the observed distribution.

Marginal model

In contrast, the marginal approach models the population-averaged RR corresponding to the observed distribution. The observed event rate is modelled by

$$\mathit{log}\left({\mu }_{ij}\right)={\gamma }_0+{\gamma }_1{X}_i$$

where $e^{{\gamma }_0}$ is the average baseline event rate, and $e^{{\gamma }_1}$ is the population-averaged RR that corresponds to the observed distribution. Without truncation, it is known that the conditional model is collapsible such that ${\gamma }_0={\beta }_1+{\sigma }_c^2/2$ and ${\gamma }_1={\beta }_1.$³ In this case, the interpretation of treatment parameters from both models is equivalent. Right truncation, however, leads to non-collapsibility. Marginalizing over the random effect in the conditional model, we obtain the marginal expectation in the intervention group as ${\mu }_1=e^{{{\beta }_0+\beta }_1}{E}_u\left[e^u{Q}_{T-1}\left(e^{{\beta }_0+{\beta }_1+u}\right)/Q_T\left(e^{{\beta }_0+{\beta }_1+u}\right)\right]$; the marginal expectation in the control group ${\mu }_{\left(0\right)}$ is similarly calculated by replacing ${\beta }_1=0$ in the expression for ${\mu }_1$. Clearly, the marginal RR is $e^{{\gamma }_1}={\mu }_{\left(1\right)}/{\mu }_{\left(0\right)}$, which approaches null as the truncation point $T$ moves towards one.

For clustered count outcomes, the Poisson GEE specifies the Poisson variance function and a working correlation model. While the variance estimator of the random-effects model is only valid when the likelihood is correctly specified, the Poisson GEE automatically accounts for misspecification through the sandwich variance method.⁴ In other words, $e^{{\gamma }_1}$ correctly estimates the marginal RR for the observed distribution even when the Poisson variance and the correlation model are both misspecified. In particular, the Poisson GEE (or modified Poisson regression) is a viable alternative to the log-binomial regression for analysing clustered binary data.⁵ This scenario is identical to the extreme case of right truncation where $T=1$.

Validity of marginal analysis

Since the marginal model is agnostic to the correct likelihood, a natural question is whether such a model is valid under right truncation. To empirically verify the validity of Poisson GEE, we replicate the simulations in Mwandigha et al.² with a single cohort. We use their conditional model to generate correlated counts except that the intervention effect is set to be null ($e^{{\beta }_1}=1$), and analyse each dataset with the Poisson GEE. In this simulation, both the variance and correlation model are misspecified, but the sandwich variance accounts for such misspecification. The statistical literature suggests that the performance of the GEE sandwich variance can be improved by finite-sample corrections.⁶ Our own research articles in CRTs also recommended the use of a $t$-test coupled with the Kauermann and Carroll bias-corrected sandwich variance⁶ for better control of test size.^7–10 The bias corrections have now been implemented in PROC GLIMMIX in SAS, the geesmv package in R, and our recent Stata package xtgeebcv.¹¹ Because the empirical results are similar between the independence and the exchangeable working correlation model in the simulations, we only present the latter for brevity. From Table 1, it is evident that the Poisson GEE has adequate control of the type I error rate regardless of truncation. These empirical results validate the marginal analysis of CRTs with truncated counts, when appropriate finite-sample corrections are used for the sandwich variance.

Table 1.

Empirical type I error rates of the Poisson GEE analyses with the exchangeable working correlation model. The results are based on 1000 simulations. The $t$-test is constructed based on the Kauermann and Carroll bias-corrected sandwich variance and $N-1$ degrees of freedom. The log-linear random effects model is used to generate outcome data with and without right truncation

Parameters for DGPa				Empirical Type I error ratesb %
$(e^{{\beta }_0},\mathrm{ }{e}^{{\beta }_1})$	${\sigma }_c^2$	$n$	$\stackrel{-}{m}$	T = ∞	$T=6$	$T=3$	$T=1$
(1.25, 1)	0.05	30	15	5.3	5.6	4.7	6.0
	0.10	30	45	5.3	5.2	5.3	5.4
	0.20	60	35	6.3	6.1	5.6	5.5
	0.30	90	40	6.0	5.9	4.8	5.9
	0.40	110	40	5.0	4.9	4.3	4.5
(2.70, 1)	0.05	25	10	5.6	5.0	5.2	4.4
	0.10	30	30	6.9	6.9	6.3	5.9
	0.20	55	20	6.1	5.5	5.4	5.5
	0.30	80	30	6.1	5.5	4.8	5.2
	0.40	110	25	5.8	5.4	5.7	4.8

^aThe first four columns are values used in the data generating process (DGP): the conditional baseline event rate $e^{{\beta }_0}$, conditional RR $e^{{\beta }_1}$, between cluster variance in the log scale ${\sigma }_c^2$, number of clusters $n$ and average cluster sizes $\stackrel{-}{m}$.

^bThe last four columns correspond to results under different truncation points.

Impact of truncation on marginal analysis

To inform the design of CRTs based on marginal models, we further investigate the impact of right truncation on marginal analysis, and compare with the patterns observed in Mwandigha et al.² We maintain their data-generating process and set the true conditional RR as $e^{{\beta }_1}=0.7$. Because the Poisson GEE models the marginal RR that corresponds to the observed distribution, the true effect size $e^{{\gamma }_1}$ decreases as $T$ approaches 1. For example, when the conditional baseline event rate is 1.25, while the conditional RR corresponding to the unobserved Poisson distribution is always $e^{{\beta }_1}=0.7$, the marginal RR corresponding to the observed counts $e^{{\gamma }_1}$ becomes 0.72, 0.82 and 0.9, as $T$ equals 6, 3 and 1. Table 2 presents the empirical power of the Poisson GEE analysis with the bias-corrected variance. Only the results based on the exchangeable working correlation are presented, because those based on the independence working correlation are nearly identical.

Table 2.

Empirical power of the Poisson GEE analyses with the exchangeable working correlation model. The results are based on 1000 simulations. The $t$-test is constructed based on the Kauermann and Carroll bias-corrected sandwich variance and $N-1$ degrees of freedom. The log-linear random effects model is used to generate outcome data with and without right truncation

Parameters for DGPa				Empirical powerb %
$(e^{{\beta }_0},\mathrm{ }{e}^{{\beta }_1})$	${\sigma }_c^2$	$n$	$\stackrel{-}{m}$	$T=\mathrm{\infty }$	$T=6$	$T=3$	$T=1$
(1.25, 0.7)	0.05	30	15	81.6	81.4	75.6	38.0
	0.10	30	45	77.1	76.5	74.3	56.6
	0.20	60	35	79.3	79.6	78.8	66.6
	0.30	90	40	79.0	79.8	80.0	73.3
	0.40	110	40	73.7	76.8	78.9	73.3
(2.70, 0.7)	0.05	25	10	79.4	77.1	58.2	20.6
	0.10	30	30	79.5	78.9	71.4	44.8
	0.20	55	20	76.0	76.1	71.1	47.5
	0.30	80	30	76.0	78.6	76.8	59.7
	0.40	110	25	74.3	78.6	76.5	65.2

^aThe first four columns are values used in the data generating process (DGP): the conditional baseline event rate $e^{{\beta }_0}$, conditional RR $e^{{\beta }_1}$, between cluster variance in the log scale ${\sigma }_c^2$, number of clusters $n$ and average cluster sizes $\stackrel{-}{m}$.

^bThe last four columns correspond to results under different truncation points.

The following patterns are observed. First, the Poisson GEE has slightly lower power than the random-effects model. This is expected because the Poisson GEE is agnostic to the true data-generating process and is not likelihood-based. Second, right truncation can similarly attenuate the power of the marginal analysis, and the effect is more pronounced (i) when the number of clusters is not large, (ii) when the baseline event rate is higher and (iii) when the truncation point approaches one. That is to say, the patterns observed in Mwandigha et al. apply to marginal analysis, indicating the necessity to account for right truncation in the sample size calculation for marginal analysis. However, an interesting curvilinear relationship between the power and truncation point also emerges with marginal analysis when the number of clusters is at least 80. In those cases, the power of the Poisson GEE first increases as $T$ becomes smaller but then decreases when $T$ moves closer to 1, suggesting right truncation does not monotonically reduce the power. This result arises due to slight differences in operating characteristics between conditional and marginal analysis of truncated counts, and motivates us to more carefully extend the message in Mwandigha et al.² to alternative models. Importantly, although we recommend accounting for truncation in sample size calculation for both conditional and marginal analyses, we realize that the effect of right truncation on power depends on the pre-specified analysis model and the number of clusters, as well as the truncation point.

Summary

Conditional models and marginal models are two commonly used approaches for the design and analysis of CRTs. We support the findings of Mwandigha et al.² and interpret their findings in the context of marginal analysis of CRTs. We conclude that right truncation generally attenuates the power of both the conditional and marginal analyses, with a few exceptions when a marginal analysis is applied on a large number of clusters. The simulation-based approach enables researchers to explore the effect of truncation on alternative modelling strategies, and, as we demonstrate, can also be useful to provide accurate power estimates for marginal analysis of truncated counts.

Funding

F.L’s work is supported within the National Institutes of Health (NIH) Health Care Systems Research Collaboratory by the NIH Common Fund through cooperative agreement U24AT009676 from the Office of Strategic Coordination within the Office of the NIH Director and cooperative agreement UH3DA047003 from the National Institute on Drug Abuse. M.O.H’s work is supported by the United States National Institutes of Health, National Heart, Lung, and Blood Institute (R00 HL141678). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Conflict of Interest

None declared.