Analysis of Randomised Trials Including Multiple Births When Birth Size Is Informative

Abstract

Background

Informative birth size occurs when the average outcome depends on the number of infants per birth. Although analysis methods have been proposed for handling informative birth size, their performance is not well understood. Our aim was to evaluate the performance of these methods and to provide recommendations for their application in randomised trials including infants from single and multiple births.

Methods

Three generalised estimating equation (GEE) approaches were considered for estimating the effect of treatment on a continuous or binary outcome: cluster weighted GEEs, which produce treatment effects with a mother-level interpretation when birth size is informative; standard GEEs with an independence working correlation structure, which produce treatment effects with an infant-level interpretation when birth size is informative; and standard GEEs with an exchangeable working correlation structure, which do not account for informative birth size. The methods were compared through simulation and analysis of an example dataset.

Results

Treatment effect estimates were affected by informative birth size in the simulation study when the effect of treatment in singletons differed from that in multiples (i.e. in the presence of a treatment group by multiple birth interaction). The strength of evidence supporting the effectiveness of treatment varied between methods in the example dataset.

Conclusions

Informative birth size is always a possibility in randomised trials including infants from both single and multiple births, and analysis methods should be pre-specified with this in mind. We recommend estimating treatment effects using standard GEEs with an independence working correlation structure to give an infant-level interpretation.

Many neonatal and perinatal trials include infants from both single and multiple births,^1,2 which makes the statistical analysis challenging. Whereas outcomes of infants born to different mothers can usually be considered independent, outcomes of infants from the same birth are likely to be similar due to shared genetic and environmental factors.^3,4 Multiple births therefore create clustering in the data, where the mother is the cluster and her infant(s) are the cluster member(s).³

Methods for analysing clustered data are widely available, and their performance has been investigated in studies including infants from both single and multiple births.^1,3–8 It is now well established that clustering due to multiple births should be taken into account in the analysis,^1,3,7–9 especially when the multiple birth rate is not low.^4,5 Failure to account for clustering due to multiple births can increase the chance that an ineffective treatment is found to be effective,⁷ which could lead to inappropriate recommendations for clinical practice. Generalised estimating equations (GEEs)¹⁰ are the most popular analysis approach for handling clustering due to multiple births.^1,2

Informative cluster size (ICS) is a common problem in clustered data. It occurs when the outcome of interest is related to the size of the cluster, conditional on the covariates in the analysis model.¹¹ For randomised trials including infants from both single and multiple births, the cluster size is the birth size (i.e. the number of infants per birth), and ICS is likely to arise in two main ways. First, the average outcome may differ between singletons and multiples. For example, multiples have lower average birthweights¹² and increased risk of mortality and cerebral palsy.¹³ Second, the average effect of the intervention may differ between singletons and multiples. For instance, antenatal corticosteroid therapy for preventing respiratory distress syndrome in preterm infants may be more effective in singletons than in twins.¹²

When ICS is present, GEEs do not necessarily estimate the treatment effect of interest.¹⁴ Failure to account for ICS could therefore lead to biased treatment effect estimates and incorrect conclusions regarding the effectiveness of treatment. Analysis methods based on GEEs have been suggested for handling ICS,^15,16 and these have been used to account for informative birth size.^1,17 However, their performance has not been formally investigated in this setting, and it is unclear when these methods should be applied. The aims of this article are to (1) study the performance of GEE methods for handling ICS through simulation and analysis of an example dataset; and (2) provide recommendations for their application in randomised trials including infants from both single and multiple births.

Methods

Statistical methods

Three GEE methods were considered for estimating the marginal effect of treatment (i.e. for a continuous outcome, the difference between the average outcomes in treated and untreated infants; and for a binary outcome, the difference between their log odds/risks) when randomisation is performed at the mother level. First, the cluster weighted GEE (CWGEE) approach uses an independence working correlation structure and weights equal to the inverse of the birth size (i.e. weight 1 for singletons, 1/2 for twins, and so on).¹⁵,¹⁶ This gives equal weight to each mother in the analysis, irrespective of the birth size, and produces a treatment effect estimate with a mother-level interpretation when ICS is present: it is the difference between the average outcomes (or log odds/risks) of a randomly chosen infant from a randomly chosen mother assigned to treatment, and a randomly chosen infant from a randomly chosen mother assigned to control.^14,15,18 Second, the GEE independence (GEE_ind) approach uses an independence correlation structure without specifying weights. This gives each infant equal weight in the analysis, and produces a treatment effect estimate with an infant-level interpretation when ICS is present: it is the difference between the average outcomes (or log odds/risks) of a randomly chosen infant assigned to treatment and a randomly chosen infant assigned to control.^14,15,18 Second, the GEE independence working correlation structure without specifying weights. This gives each infant equal weight in the analysis, and produces a treatment effect estimate with an infant-level interpretation when ICS is present: it is the difference between the average outcomes (or log odds/risks) of a randomly chosen infant assigned to treatment and a randomly chosen infant assigned to control.^14,15,18 Third, the GEE exchangeable (GEE_exch) approach uses an exchangeable working correlation structure without specifying weights. This method weights infants in a way that minimises the variance of the treatment effect estimate.¹⁹ Although GEE_exch is not recommended in the presence of ICS as it does not necessarily estimate a treatment parameter of interest,¹⁴ we include it here to investigate what can go wrong with it when ICS is present. All methods were implemented using the GENMOD procedure with empirical sandwich variance estimation in SAS version 9.3 (SAS Institute Inc., Cary, NC, USA).

Simulation study

A simulation study was conducted to evaluate the performance of CWGEE, GEE_ind, and GEE_exch when ICS is present. Simulation scenarios were chosen based on an example dataset (described below), and 10 000 datasets were generated for analysis in each scenario. Mothers were randomised to the intervention or control group (300 per group), and independently assigned to have a single birth with 80% probability or a twin birth with 20% probability; higher order multiples are rare in practice and were not considered. This produced an expected total sample size of 720 infants with 33.3% from a multiple birth, which is typical of preterm populations.² Outcomes of infants from the same birth were positively correlated, with an intracluster correlation coefficient (ICC) of 0.1, 0.5, or 0.9. Additional simulations were performed for an ICC of 0.5 while varying the probability of a twin birth from 5% to 95% by 5%.

Continuous outcomes were randomly generated from the model

$$Y_{ij}={\beta }_0+{\beta }_1{X}_{1i}+{\beta }_2{X}_{2i}+{\beta }_3{X}_{1i}{X}_{2i}+a_i+e_{ij},$$ (1)

where Y_ij is the outcome for the jth infant from the ith mother, X_1i is the randomised treatment group (1 = intervention, 0 = control), X_2i is the multiple birth status (1 = multiple birth, 0 = single birth), $a_i\sim N\left(0,{\sigma }_a^2\right)$ is a random mother effect, and $e_{ij}\sim N\left(0,{\sigma }_e^2\right)$ is a random error. Under model (1), ICS occurs whenever the outcome and/or the effect of treatment on the outcome depends on birth size (i.e. whenever β₂ and/or β₃ are non-zero); more general definitions of ICS are discussed elsewhere.^14,18,20 Variances were chosen to give a total variance of ${\sigma }_a^2+{\sigma }_e^2={15}^2$ and produce the desired ICC according to the equation $ICC={\sigma }_a^2/\left({\sigma }_a^2+{\sigma }_e^2\right)$. The mean outcome for singletons in the control group was set to 100 (β₀ = 100), since outcomes of many developmental assessments follow an N(100, 15²) distribution in the population, such as the Mental Development Index (MDI) standardised score from the Bayley Scales of Infant Development.²¹ The me an outcome for singletons in the intervention group was chosen to be 104 to produce a treatment effect of 4 among singletons (β₁ = 4), since the example trial was designed to detect a 4-point improvement in the MDI. The mean outcome for twins in the control group was set to 97 (β₂ = −3), since a 3-point reduction in the mean developmental outcome for twins compared with singletons is plausible based on the example dataset. The mean outcome for twins in the intervention group was chosen to be 101, 99, or 103 (β₃ = 0, −2 or 2) to produce a treatment effect among twins of 4 (to match the singletons), 2, or 6, respectively, in order to explore the effect of ICS in the absence or presence of a treatment group by multiple birth interaction. Simulation methods for binary outcomes are described in the Supporting information.

Treatment effects were estimated for each simulated dataset based on the unadjusted model µ_ij = β₀ + β₁X_1i, and the model adjusting for multiple birth status as a main effect µ_ij = β₀ + β₁X_1i + β₂X_2i, where µ_ij = E[Y_ij] is the mean outcome, since both unadjusted and adjusted estimates are commonly presented. An interaction model µ_ij = β₀ + β₁X_1i + β₂X_2i + β₃X_1iX_2i was used to test for evidence of a treatment group by multiple birth interaction but not to estimate treatment effects, since these should primarily be estimated from main effects models.²² Simulation results were summarised by averaging the treatment effect (β₁) estimates and their estimated standard errors. Monte Carlo (simulation) standard deviations were very similar to the average estimated standard errors and are not reported. The power to detect a treatment group by multiple birth interaction was calculated as the percentage of simulated datasets where the interaction term was statistically significant (P < 0.05).

Example dataset

To illustrate the impact of choosing different GEE methods in a real trial where ICS may be present, we consider a trial of high-dose vs. standard-dose docosahexaenoic acid (DHA), a source of omega-3 fatty acids, for preterm infants.²³ Consenting mothers of infants born less than 33 weeks gestation were randomly assigned to receive capsules rich in DHA or placebo capsules, and infants received the treatment through breast milk. There were 545 mothers and 657 infants included in the trial, of whom 33.6% were from a multiple birth (30.4% in the high-DHA group, 36.7% in the standard-DHA group). The primary outcome was neurodevelopment of the infant at 18 months, as measured by the MDI, while significant mental delay (MDI < 70) was a key secondary outcome. These outcomes were reanalysed for the 584 infants from a single or twin birth who remained after excluding infants with missing outcomes, along with nine sets of triplets for comparison with the simulation results. Treatment effects were estimated based on a linear model for the MDI, and both a logistic and log binomial model (see Supporting information) for significant mental delay.

Results

Simulation study

The simulation results for a continuous outcome are given in Table 1 (see also Supporting information). Average unadjusted treatment effect estimates were very similar for all methods and ICCs when the treatment effect was the same for singletons and twins, but varied otherwise. When the true treatment effect among twins was 2, average unadjusted treatment effect estimates were close to the true overall mother-level treatment effect of 3.60 for CWGEE, which is the average of the true treatment effects for singletons and twins, weighted by the expected proportion of mothers with single and twin births [i.e. (0.8 × 4) + (0.2 × 2) = 3.60]. For GEE_ind, estimates were close to the true overall infant-level treatment effect of 3.33, which is the average of the true treatment effects for singletons and twins, weighted by the expected proportion of singleton and twin infants [i.e. (0.667 × 4) + (0.333 × 2) = 3.33]. Likewise, when the true treatment effect was 6 among twins, average unadjusted treatment effect estimates were around 4.40 and 4.67 for CWGEE and GEE_ind, respectively (see Supporting information). As the ICC increased, average unadjusted treatment effect estimates remained stable for CWGEE and GEE_ind. For GEE_exch, estimates were similar to GEE_ind when the ICC was low but similar to CWGEE when the ICC was high. Independently of whether the treatment effect differed between singletons and twins, average unadjusted standard errors increased with the ICC for all methods. CWGEE produced the largest standard errors when the ICC was low, whereas GEE_ind produced the largest standard errors when the ICC was high, although differences between methods were fairly small. Adjusting for multiple birth status only slightly reduced the standard errors and made little difference to the treatment effect estimates on average. All methods produced identical results for each simulated dataset using the correct interaction model, although the power to detect an interaction was very low, ranging from 10.3% to 13.6%. Similar results were obtained for binary outcomes (see Supporting information).

Table 1.

Simulation results for a continuous outcome with 20% twin births

Treatment effecta (Singletons/twins)	ICC	CWGEEb		GEEindb		GEEexchb
		Unadjusted	Adjustedc	Unadjusted	Adjustedc	Unadjusted	Adjustedc
4/4d	0.1	4.00 (1.17)	4.00 (1.17)	4.00 (1.14)	4.00 (1.13)	4.00 (1.14)	4.00 (1.13)
	0.5	3.99 (1.20)	3.99 (1.19)	3.99 (1.21)	3.99 (1.20)	3.99 (1.19)	3.99 (1.18)
	0.9	4.02 (1.22)	4.02 (1.22)	4.02 (1.28)	4.02 (1.27)	4.02 (1.22)	4.02 (1.21)
4/2e	0.1	3.62 (1.17)	3.62 (1.17)	3.35 (1.15)	3.35 (1.13)	3.41 (1.15)	3.40 (1.13)
	0.5	3.61 (1.20)	3.61 (1.19)	3.34 (1.22)	3.34 (1.21)	3.52 (1.19)	3.51 (1.18)
	0.9	3.59 (1.22)	3.59 (1.22)	3.33 (1.28)	3.34 (1.27)	3.58 (1.22)	3.58 (1.22)
4/6f	0.1	4.41 (1.17)	4.41 (1.17)	4.68 (1.14)	4.68 (1.14)	4.64 (1.14)	4.64 (1.13)
	0.5	4.39 (1.19)	4.39 (1.19)	4.66 (1.21)	4.65 (1.21)	4.49 (1.19)	4.48 (1.18)
	0.9	4.39 (1.22)	4.39 (1.22)	4.66 (1.27)	4.65 (1.27)	4.41 (1.22)	4.41 (1.22)

^aTrue difference in mean outcome (intervention minus control) for singletons and twins.

^bAverage treatment effect estimate (average estimated standard error) over 10 000 simulated datasets.

^cResults adjusted for multiple birth status.

^dTrue overall mother-level and infant-level treatment effect is 4.00.

^eTrue overall mother-level treatment effect is 3.60. True overall infant-level treatment effect is 3.33.

^fTrue overall mother-level treatment effect is 4.40. True overall infant-level treatment effect is 4.67.

The impact of varying the multiple birth rate on the average unadjusted treatment effect estimate for a continuous outcome is shown in Figure 1. When the multiple birth rate was low, estimates were similar between methods and close to the true treatment effect of 4 for singletons. As the multiple birth rate increased, estimates approached the true treatment effect of 2 (Figure 1a) or 6 (Figure 1b) for twins. The relationship was approximately linear for CWGEE but exponential for GEE_ind, with GEE_exch estimates falling in between. The difference between methods increased as the percentage of twins moved away from 0% or 100%. A similar pattern was seen for adjusted treatment effect estimates (data not shown) and binary outcomes (see Supporting information).

Figure 1.

Average unadjusted treatment effect estimate for a continuous outcome with an ICC of 0.5 by varying percentage of mothers with a twin birth when the treatment effect is 4 for singletons and (a) 2 or (b) 6 for twins.

Example dataset

Treatment effect estimates for the DHA trial are given in Table 2. For the MDI, treatment effect estimates were somewhat different between methods, but none produced sufficient evidence to support the hypothesis that high DHA increases the mean MDI. For significant mental delay, unadjusted odds ratios ranged from 0.47 for GEE_ind to 0.52 for CWGEE. This relatively small difference between methods is consistent with the results from the simulations where a treatment group by multiple birth interaction was present. Subgroup analyses produced an estimated odds ratio of 0.60 among singletons and 0.31 among twins, but there was little evidence to suggest that the effect of treatment varied by birth size (P > 0.4 for all GEE methods). If an infant-level interpretation of the odds ratio is of interest, the GEE_ind results suggest that treatment reduces the odds of significant mental delay by 53% for a randomly selected high-DHA infant compared with a randomly selected standard-DHA infant. If a mother-level interpretation is desired, the CWGEE results suggest that treatment reduces the odds of significant mental delay by 48%, comparing a randomly selected infant from a randomly selected high-DHA mother with a randomly selected infant from a randomly selected standard-DHA mother. Relative risks can be interpreted similarly. High DHA was shown to be effective for reducing both the odds and risk of significant mental delay using GEE_ind (P = 0.03) and GEE_exch (P = 0.04), while the evidence in favour of the intervention was less convincing using CWGEE (P = 0.06).

Table 2.

Treatment effect estimates for outcomes from the high-docosahexaenoic acid (DHA) vs. standard-DHA trial

Outcome	Analysis method	Unadjusted treatment effect	Adjusted treatment effectd
MDI standardised scorea	CWGEE	1.38 (−1.43, 4.20)	1.30 (−1.51, 4.10)
	GEEind	1.52 (−1.31, 4.34)	1.39 (−1.41, 4.19)
	GEEexch	1.41 (−1.40, 4.22)	1.29 (−1.50, 4.08)
Significant mental delayb	CWGEE	0.52 (0.26, 1.02)	0.52 (0.26, 1.03)
	GEEind	0.47 (0.24, 0.93)	0.48 (0.24, 0.94)
	GEEexch	0.50 (0.26, 0.98)	0.50 (0.26, 0.98)
Significant mental delayc	CWGEE	0.54 (0.29, 1.02)	0.55 (0.29, 1.03)
	GEEind	0.50 (0.26, 0.94)	0.50 (0.27, 0.95)
	GEEexch	0.53 (0.28, 0.98)	0.53 (0.28, 0.99)

^aTreatment effects are estimated difference in means (95% confidence interval) comparing high DHA with standard DHA.

^bTreatment effects are estimated odds ratio (95% confidence interval) comparing high DHA with standard DHA.

^cTreatment effects are estimated relative risk (95% confidence interval) comparing high DHA with standard DHA.

^dResults adjusted for multiple birth status.

Comment

We have explored the problem of ICS in randomised trials including infants from both single and multiple births. We considered scenarios where ICS arises due to differences in average outcomes between singletons and multiples (no interaction), and differences in average treatment effects between singletons and multiples (interaction). Treatment effect estimates were obtained from main effects models only, as recommended for randomised trials.²² Our simulation results indicate that treatment effect estimates are only influenced by ICS in the latter scenario (Figure 2), in which case different GEE methods are expected to produce different treatment effect estimates, although the differences we found were relatively small. Whether these differences are of practical importance will depend on the context. Our example dataset illustrates the potential for the strength of evidence supporting the effectiveness of treatment to vary according to the GEE method chosen.

Figure 2.

Flow chart summarising simulation results and analysis recommendations.

When treatment effect estimates differ between methods, GEE_exch produces estimates similar to GEE_ind when the ICC is low and similar to CWGEE when the ICC is high. This makes sense intuitively, since GEE_exch weights infants in a way that minimises the variance of the treatment effect estimate.¹⁹ When the ICC is low, a set of twins provides almost as much information as two singletons, and the variance is minimised by giving each twin a weight close to one, similar to GEE_ind. In contrast, when the ICC is high, a set of twins provides little more information than a singleton, and the variance is minimised by giving each mother a weight close to one, similar to CWGEE.

Since CWGEE, GEE_ind, and GEE_exch are expected to produce different treatment effect estimates when a treatment group by multiple birth interaction is present, a method of analysis could be chosen after interactions have been investigated, according to Figure 2. If there is insufficient evidence of an interaction, treatment effects could be estimated using any GEE method. GEE_exch may be preferred for maximising efficiency, although efficiency gains associated with this method were minimal in our simulation study due to treatment assignment at the mother level.^24,25 If evidence of an interaction is found, treatment effects could be estimated using CWGEE or GEE_ind, depending on whether a mother-level or an infant-level interpretation is preferred. GEE_exch should not be used in this case, since it fails to estimate a treatment parameter of interest.¹⁴ The alternative to this data-driven approach is to pre-specify a method of analysis that remains appropriate across a range of scenarios. Since interactions are often plausible, CWGEE or GEE_ind would be chosen for analysis, depending on the desired interpretation. We prefer this approach in the randomised trial setting, since statistical methods should be pre-specified before issues such as ICS can be investigated in the data and interaction tests are typically underpowered.²⁶

The choice between GEE_ind and CWGEE will depend on the context. GEE_ind estimates the effect of treatment for a typical infant, while CWGEE estimates the effect of treatment for a typical infant from a typical mother.¹⁴ Since the former interpretation is most relevant for describing the impact of treatment on the total burden of disease and the demand for specialised child health or education services, we recommend using GEE_ind to estimate treatment effects in general. This method has appropriate type I error and coverage rates when birth size is uninformative,⁷ and has the advantage of producing unadjusted treatment effect estimates that are consistent with the raw means or percentages for each treatment group. If the mother’s perspective is actually of primary interest, this should be justified in the trial protocol and CWGEE can then be used for analysis.

It may be argued that any GEE method can be chosen when the multiple birth rate is low, since differences in treatment effect estimates between methods are small in this case. Figure 1 suggests this may be a reasonable strategy when the multiple birth rate is 5% or less, as would be expected in trials recruiting from the general population of pregnant women. However, this strategy may be problematic for treatments that have very different effects in singletons and multiples, where larger differences between methods are expected, or for outcomes where small changes would be considered clinically important. The safest approach is to pre-specify a method of analysis that acknowledges the possibility of ICS and produces treatment effect estimates with the desired interpretation, irrespective of the multiple birth rate.

Adjusting for multiple birth status as a main effect had little impact on average treatment effect estimates but led to small gains in efficiency. Whether adjustment should be made for multiple birth status in practice depends on the trial. Adjustment is problematic when there are few multiples in a trial, since the outcome may be the same for all multiples, and when multiple birth status is determined after treatment commences, since fetal resorption may be influenced by treatment group. However, adjustment can be useful in other settings. Adjustment is recommended when multiple birth status is used as a balancing factor in the randomisation,²⁷ and has the benefit of correcting for chance imbalance in the multiple birth rate between treatment groups otherwise. Adjustment can also increase efficiency if multiple birth status is associated with the outcome.²⁸ Our results indicate that adjustment does not eliminate ICS when treatment effect estimation is of interest and the effect of treatment varies according to cluster size.

Randomised trials including multiple births differ from most settings where GEE methods for handling ICS have been investigated previously,^{11,15,16,29–32} due to the small cluster sizes and focus on treatment effect estimation. Small clusters were considered in a recent simulation study comparing GEE_ind and GEE_exch to within-cluster resampling,¹¹ which is asymptotically equivalent to CWGEE,¹⁵ in a non-randomised setting. The authors concluded that all methods performed well for estimating covariate effects, but that within-cluster resampling should be preferred if intercept estimation is of interest.³³ Others have noted that when the covariate effects are the same regardless of cluster size, ICS often has little impact on parameter estimates aside from the intercept,³⁴ which is of limited interest in randomised trials. Our findings indicate that treatment effect estimates are also influenced by ICS when the effect of treatment varies according to cluster size. As such interactions are often plausible, ICS is a serious concern for trials including multiple births.

ICS can arise in neonatal and perinatal trials whenever clustering is present and cluster sizes vary, which may occur for reasons other than multiple births. Our findings can reasonably be extended to settings where siblings from different births are present, while further research is needed to understand how methods for handling ICS perform in longitudinal settings. ICS is rarely a concern when analysing outcomes that are measured on the mother, since each mother can usually be considered independent.

A limitation of this study is that only randomisation at the mother level was considered. This approach is necessary for interventions given to the mother, and is often preferred by parents otherwise, making it the most common choice in practice.^1,2 If randomisation is performed at the infant level, choosing CWGEE or GEE_ind over GEE_exch in the absence of an interaction is expected to result in greater efficiency losses than those observed in our study.^24,25 A further limitation is that only GEE methods for addressing ICS were examined. Clustered data are also commonly analysed using mixed-effects models, and methods for handling ICS in this context have been discussed previously – see14 and references therein. Such approaches may be of limited use in randomised trials including multiple births, since GEEs are more popular^1,2 and perform well in this setting.⁷

In conclusion, informative birth size is always a possibility in randomised trials including infants from both single and multiple births, and analysis methods should be pre-specified with this in mind. If a treatment group by multiple birth interaction is present, different GEEs are expected to produce different treatment effect estimates with different interpretations. We recommend estimating treatment effects using standard (unweighted) GEEs with an independence working correlation structure to give an infant-level interpretation.

Supporting information

Additional supporting information may be found in the online version of this article at the publisher’s web-site: