A Comparison of Regression Approaches for Analyzing Clustered Data

Abstract

Objectives. We used 3 approaches to analyzing clustered data to assess the impact of model choice on interpretation.

Methods. Approaches 1 and 2 specified random intercept models but differed in standard versus novel specification of covariates, which impacts ability to separate within- and between-cluster effects. Approach 3 was based on standard analysis of paired differences. We applied these methods to data from the National Collaborative Perinatal Project to examine the association between head circumference at birth and intelligence (IQ) at age 7 years.

Results. Approach 1, which ignored within- and between-family effects, yielded an overall IQ effect of 1.1 points (95% confidence interval [CI]=0.9, 1.3) for every 1-cm increase in head circumference. Approaches 2 and 3 found comparable within-family effects of 0.6 points (95% CI = 0.4, 0.9) and 0.69 points (95% CI = 0.4, 1.0), respectively.

Conclusions. Our findings confirm the importance of applying appropriate analytic methods to clustered data, as well as the need for careful covariate specification in regression modeling. Method choice should be informed by the level of interest in cluster-level effects and item-level effects.

Public health research depends increasingly on clustered data study designs to evaluate the effectiveness of programs, interventions, and policies. Clustered data consist of data points recorded on multiple items within a cluster, such as those that arise in family studies (multiple siblings within a family), longitudinal studies (multiple time points within a person), school-based research (multiple students within a school), and complex surveys (multiple respondents within a geographic region or neighborhood). Although nonclustered (i.e., independent) data can easily be analyzed via standard statistical methods, clustered data can offer significant advantages over independent data; these include the ability to separate out family effects from sibling effects, time effects from person effects, school effects from student effects, and regional or neighborhood effects from respondent effects. To succeed in these objectives, data analysts must take care to choose techniques that (1) appropriately account for correlation among items within the same cluster and (2) effectively separate and characterize cluster-level effects and item-level effects.

The issue of intracluster correlation has been considered by numerous authors.1^–3 These accounts have noted that if we presume independence when analyzing data that are cluster-correlated, then biased estimates of standard errors are likely to result. Typically, positive intracluster correlation may cause standard errors to be underestimated when the exposure of interest is fixed for the cluster and overestimated when the exposure varies within cluster. Use of inaccurate standard errors can lead to invalid test statistics and confidence intervals, and ultimately, misleading inferences on, say, the effectiveness of a new behavioral therapy for medication compliance or the success of a new smoking-prevention strategy. To avoid incorrect inferences, data analysts are encouraged to use statistical methods that account for intracluster correlation in an effort to preserve the validity of resulting conclusions. Fortunately, many of the most commonly used statistical software packages—e.g., Stata (StataCorp LP, College Station, TX), SAS (SAS Institute Inc, Cary, NC), and SPSS (SPSS Inc, Chicago, IL)—now incorporate state-of-the-art methods for analyzing clustered data.

The separation of cluster-level and item-level effects of a particular exposure has received relatively less attention in the statistical literature, although it has not been entirely ignored.4^–8 The social sciences literature has long referred to the partitioning of “individual” versus “contextual” effects, which are analogous to item-level and cluster-level effects, respectively. Other near-synonyms in the statistical literature have included person-specific versus population-averaged effects, and within-cluster and between-cluster effects. It is easy to argue that many researchers are interested in both levels of effect; for example, a sociologist may be interested in both the neighborhood influences (e.g., socioeconomic level, as reflected by average income in the neighborhood) as well as the individual-specific factors (e.g., household-specific income) that modify smoking risk, to design the most effective interventions to prevent smoking among adolescents. We discuss different approaches for capturing both levels of effect in clustered designs.

Our purpose was to review several different approaches for analyzing clustered data, with the previously mentioned objectives in mind. Although many earlier papers have emphasized the need for taking intracluster correlation into account, far fewer have highlighted methods for and advantages of distinguishing among cluster-level (or between-cluster) and item-level (or within-cluster) effects via careful model and covariate specification in regression. Separation of effects via careful modeling has the further advantage of reducing confounding by cluster; that is, the distortion of item-level effects by cluster-level correlates associated with exposure and outcome.

The statistical approaches we focused on included random effects regression analysis of all items across many clusters, with and without adjustment for the cluster-averaged exposure covariate, and ordinary regression analysis of independent items consisting of differences between sibling pairs randomly selected from different clusters. For each approach, we also devoted attention to the proper interpretation of the exposure effect (i.e., whether it represents a “within-cluster” or “between-cluster” phenomenon), and use of purposeful covariate selection to discriminate between cluster-level and within-cluster effects. Note that we assume in what follows that the exposure measurement varies among items within a cluster (such as an individual student’s gender in a school-based study); hence, the methods and recommendations proposed do not apply to exposures that are fixed within a cluster (such as in cluster-randomized trials, in which treatment is completely synonymous with clinic or location).

METHODS

The different statistical strategies we consider here were applied to data on head circumference at birth and intelligence quotient (IQ) at age 7 years among participants in the National Collaborative Perinatal Project (NCPP). We describe 3 different regression techniques that are appropriate for analyzing clustered data. We also describe use of adjustment variables to distinguish between cluster-level and item-level effects.

Notation

Suppose that the letter i = 1, 2, . . ., n indexes the cluster (e.g., the school identifier in a school-based study), and j = 1, 2, . . ., m_i indexes the item within the cluster (e.g., student within a school). Then let Y_ij denote the individual response for item j in cluster i, and let X_ij represent the individual exposure measurement for item j in cluster i. As noted earlier, we assume that X_ij varies from item to item within a cluster. Finally, let X̄_i denote the average exposure measurement for all items within cluster i, referred to below as the “cluster-averaged exposure” measurement.

Modeling Approaches

Approaches 1 and 2 make use of all the available data and apply a mixed-effects linear regression model with a random intercept term for each cluster. This means that in sibling studies, for example, there is a unique intercept for each family and in school-based studies, there is a unique intercept for each school. These intercept terms are assumed to be random, drawn from a normal distribution with mean and variance to be estimated from the data. Conceptually, the random intercept represents each cluster’s tendency to have a different mean response measurement. In this way, the random intercept also takes into account the correlation among observations within a cluster, resulting in appropriate estimates of the standard errors for the estimated regression coefficients.

Approach 1 involves modeling response variable Y_ij as a function of X_ij as the sole predictor variable:

(1)

where u_0i is the random intercept parameter unique to each cluster (i ) in the sample (assumed to follow a normal distribution), and e_ij is the random error term (also assumed to be normal and independent of the intercept terms).

In approach 2, we model Y_ij as a function of both X_ij and X̄_i :

(2)

The model for approach 2 is very similar to the model that corresponds with approach 1, except for the addition of the term for X̄_i, the cluster-averaged exposure measurement. This latter approach allowed us to separate out the item-level effect (as captured by the estimated regression coefficient for X_ij ) from the cluster-level effect (as captured by the estimated regression coefficient for X̄_i ), as recommended by a number of other authors.4^–8

In approach 3 we conditioned on family by choosing 2 siblings at random from each family and regressing IQ difference between siblings on the head circumference difference. This approach reduced the data to 1 row of observations per family, so that we could use an ordinary regression model for independent data. The model is written as:

(3)

Note that this model lacks an intercept term because we would generally not expect non-random differences in IQ score between randomly chosen siblings. Unlike approach 2, approach 3 only provides an estimate of the item-level or within-cluster effect. This will become clear in the example that uses data from the NCPP.

Childhood IQ and Head Circumference at Birth

In searching for neonatal correlates of intelligence, one of the more consistent findings is the association between IQ in childhood or adulthood and head circumference at birth. We briefly review a small handful of these findings here and refer the reader to a more comprehensive list of references provided by Ivanovic et al.9 Interest persists in this question because head circumference at birth is a simple and reliable indicator of possible cognitive defects in childhood and adulthood. Furthermore, even a modest association between head circumference and IQ at the individual level could have more dramatic implications for the population distribution of IQ.10 If fetal growth is related to neurodevelopmental outcome, then this might suggest an area for possible public health intervention.

Nelson and Deutschberger’s11 analysis of data from the NCPP demonstrated a positive relation between head size at 1 year of age and IQ at age 4 years, with larger head circumference associated with higher IQ. This finding was supported by Broman et al.12 In a different sample, Hack et al.13 established that very-low-birthweight infants born with subnormal head size whose heads did not reach normal size by 8 months were at higher risk of lower IQ scores at age 8 years. Strauss and Dietz,14 who used data from the NCPP, argued that there were no IQ differences between siblings with and without intrauterine growth retardation, except for those with intrauterine growth retardation coupled with small head circumference (resulting in lower IQ). The report from Bergvall et al.15 showed that Swedish boys born with head circumference small for their gestational age were at risk of lower intellectual performance in adulthood. In a similar analysis of adult men in Sweden, Lundgren et al.16 identified reduced intellectual performance among conscripts who were born small for gestational age, with a particular association between small head circumference at birth and performance on tests of logical capacity.

Several studies, including those by Nelson and Deutschberger11 and Petersson et al.17 demonstrated a general increase in IQ with increasing head circumference at birth, with some diminution of this effect at very large head circumferences, possibly indicating a negative effect on intelligence from primary megalencephaly (defined as head circumference above the 98th percentile) because of brain enlargement. Brennan et al.18 claimed that there was no “clinically meaningful” association between head circumference at birth and intelligence at age 7 years, based on an analysis of NCPP data, although readers may disagree on the definition of “clinically meaningful.” They found a difference of approximately 2 IQ points for infants categorized as having “relatively small” heads versus those without small heads.

We also examined this question by using the different analytic methods proposed for clustered family data. Specifically, in one approach, we reevaluated this question in a model that separates within-family from between-family effects by adjusting for family-averaged head circumference, in the sense of the “contextual” effect described earlier. Although we did not argue that siblings’ head size has a direct impact on individual IQ level, it is conceivable that family-averaged head circumference serves as a proxy for other important family-level factors (genetic or environmental) that would directly affect IQ—for example, nutritional status or degree of prenatal care. Adjustment, therefore, could lead to reduced confounding by these other factors and a better estimate of the within-family effect.

In this example, we used data from the NCPP to study the association between head circumference at birth and childhood intelligence, as measured by the Wechsler Intelligence Scale for Children administered at approximately 7 years of age.19 Broman20 gives a full description of the study design and demographic characteristics of study participants. The complete NCPP data set comprises 59 391 children born to 48197 mothers, who were recruited from 12 different medical centers throughout the United States. We restricted attention to singleton births, and our sample comprised 55 740 children born to 45 593 mothers. We then excluded those with missing data on IQ at age 7 years (n = 16021) or on head circumference at birth (n = 4076). Then, to make our results as generalizable as possible, we decided a priori to exclude potentially nonrepresentative children: those with IQ less than 40, indicating an incomplete administration of the test (n = 100); those born at a gestational age less than 37 weeks (n = 9660); and those with birthweight less than or equal to 2500 g (n = 7666). The resulting sample after these exclusions consisted of 32 051 children born to 26924 mothers.

Because our aim was to illustrate and compare methods for analyzing clustered data and making within-cluster comparisons, we further reduced the sample by including only those families with 2 or more children enrolled in the study, yielding a final sample of 9147 children born to 4170 mothers.

RESULTS

We compared results obtained after applying the 3 approaches to the NCPP data. In this sample, IQ ranged from 40 to 150 with a mean of 97 and standard deviation of 14. Head circumference ranged from 26 cm to 40 cm with a mean of 34 cm and a standard deviation of 1.3. Figure 1 ▶ plots IQ as a function of head circumference at birth for the entire sample of 9147 children. The plot clearly demonstrates increasing IQ as a function of head circumference, with a slight diminution of effect at the highest measurements of head circumference, as noted by previous authors.

FIGURE 1—

Intelligence quotient (IQ) at age 7 years as a function of head circumference at birth among 9147 children: National Collaborative Perinatal Project.

Note. Circles represent those observations that exceed the length of the range multiplied by the length of the interquartile range of the box edges. The two outlying horizontal lines are median values for 26 cm and 40 cm, respectively. There was not as much spread in distribution for these values as there was for the others.

Table 1 ▶ presents results from fitting the various regression models discussed. Approach 1 (N = 9147) specifies a mixed-effects linear model, with a random intercept for each family, modeling IQ as a function of head circumference. This approach yielded an estimated regression coefficient of 1.09 for head circumference (95% confidence interval [CI] = 0.88, 1.30), suggesting that for every increase of 1 cm in head circumference, IQ score increased, on average, by about 1 point. This result is consistent with some of the earlier reports in the literature; it does not, however, separate out the individual head circumference effect (or within-family effect) from the cluster-averaged head circumference effect (or between-family effect).

TABLE 1—

Unstandardized Parameter Estimates (and 95% Confidence Intervals) From 3 Approaches for Analyzing Clustered Data: National Collaborative Perinatal Project

	Analytic Sample
Approach	Full Sibling Data Set (N = 9147), b (95% CI)	Sibling Pairs (N = 8340), b (95% CI)
Approach 1: b̂1(1)	1.09 (0.88, 1.30)	1.18 (0.96, 1.40)
Approach 2
b̂1(2)	0.61 (0.35, 0.87)	0.69 (0.41, 0.97)
b̂2(2)	1.36 (0.93, 1.80)	1.25 (0.81, 1.70)
Approach 3: b̂1(3)	. . .	0.69 (0.41, 0.97)

Note. The National Collaborative Perinatal Project data was used to analyze the association between head circumference at birth and intelligence quotient at age 7 years. Approach 1 was a mixed-effects regression of intelligence quotient on head circumference. Approach 2 was a mixed-effects regression of intelligence quotient on head circumference and family-averaged head circumference. Approach 3 was an ordinary linear regression of paired intelligence quotient differences on paired head circumference.

The effect for individual head circumference changed dramatically under approach 2, which adjusted for family-averaged head circumference. The within-family head circumference effect from this model was estimated as 0.61 (95% CI = 0.35, 0.87), indicating an increase of only about 0.6 IQ points for every 1 cm increase in head circumference. Thus, adjustment for family-averaged head circumference resulted in a 44% reduction in the estimated effect of individual head circumference. At the same time, an increase of 1 cm in the family-averaged head circumference resulted in an increase of 1.36 IQ points, on average. This finding would be consistent with the notion that the cluster-averaged effect can play an important role as an adjustment variable, representing either the “contextual” effect of the exposure variable or a surrogate for other important cluster-level (family-level) factors.

Under approach 3, we found an expected difference of 0.69 IQ points for a 1-cm difference in head circumference between siblings. This is very similar to the estimate for the within-family difference that we found using approach 2, which adjusted for cluster-averaged exposure measurement. Thus, approach 2 closely mimics the findings from a within-family, paired-difference analysis. For comparison, we also applied approaches 1 and 2 to the 4170 sibling pairs (8340 children total) from this reduced sample. Approach 1 yielded an estimated regression coefficient for head circumference of 1.18, and approach 2 gave 0.69 for within-family effect (the same estimate obtained under approach 3) and 1.25 for the family-level effect. Both samples, therefore, were seen to give consistent results.

DISCUSSION

We applied 3 approaches for analyzing clustered data to sibling data on IQ and head circumference from the NCPP. Approach 1 likely represents the most common technique for analyzing clustered data. By incorporating a random intercept term, it accounts for correlation within cluster. By including only a single parameter, however, to capture the within- and between-cluster effects, it precludes the possibility of distinguishing among these effects. Raudenbush and Bryk6 considered this question, and demonstrated that the regression coefficient from approach 1, b₁⁽¹⁾,equates to a weighted average of the within-cluster and between-cluster parameters from approach 2 (b₁⁽²⁾ and b₂⁽²⁾, respectively). They argue convincingly that approach 1 makes sense only if one is prepared to assume that the within-cluster and between-cluster effects of exposure on outcome are identical.6 In our example on head circumference and IQ, this would require assuming that the influence of individual head circumference in IQ is exactly the same as the influence of family-averaged head circumference (a proxy for family-level characteristics) on IQ. To consider another example, imagine a study that looks at effect of IQ on test performance in the classroom setting. Approach 1 requires us to assume that the effects of individual intelligence and classroom environment on test performance are identical. There may be times when this is sensible, but we believe that most often this assumption will not be justified.

For the example of IQ and head circumference, we saw confirmation of Raudenbush and Bryk’s model assertions.6 From approach 1, we observed an estimated head circumference effect of 1.1 points, which lies between the estimated within-family and between-family effects of 0.6 and 1.4 obtained via approach 2. These findings suggest that the individual-level and family-level effects are not the same; hence, approach 1 gives a summary measure of effect that may not be defensible.

The example presented here supports the argument for careful modeling when analyzing clustered data, as results and interpretation vary by analytic method. We found strong support for recommending approach 2, a mixed-effects (random intercept) regression analysis of the item-level predictor adjusted for average predictor value over the cluster. Approach 2 makes use of all available data in clusters and permits us to exploit the correlated nature of the data to full advantage. Approach 3 has the advantage of simplicity, but sacrifices a portion of the data, as well as the ability to discriminate between within-cluster and between-cluster effects. Although approach 1 accounts for the within-cluster correlation, approach 2 is superior to approach 1 in that it allows separation of within-cluster and between-cluster effects, leading to a better understanding of the data set and its implications.

Interestingly, the analysis by Bergvall et al.15 presented separate estimates for within-family and between-family effects, using a somewhat different strategy. They specified individual head circumference along with the difference between family-averaged and individual head circumference as predictors of low intellectual performance. As such, their within-family effect is directly comparable to the one we present, but their between-family effect is an amalgam of the within- and between-family effects (see Begg and Parides4 for a full discussion).

In the example we present, the interest was in estimating the individual-level effect of head circumference on IQ. There may be situations, however, when the cluster-level effect is also of interest. For example, again consider a study in which individual IQ and classroom-averaged IQ on test performance are analyzed. Approach 2 would be appropriate for obtaining the individual-level IQ effect as well as the classroom-averaged IQ effect (which would represent something about the classroom environment). In a model that adjusts for both individual intelligence and “classroom intelligence,” the coefficient corresponding to individual IQ would represent the effect of individual ability on test performance, and the coefficient for class-averaged IQ would represent the effect of classroom environment on test performance. Unlike approaches 1 and 3, the model under approach 2 allows estimation of both cluster-level and item-level effects. In general, the model choice should be informed by the level of interest in the cluster-level and item-level effects.

Finally, there is another regression modeling approach that is commonly used in the analysis of clustered data: the generalized estimating equation technique.1 Like the random effects modeling approach, the generalized estimating equation approach is a valid method for analyzing cluster-correlated data. It differs in formulation, correcting for intra-cluster correlation after regression coefficient estimates are obtained, whereas the random effects approach explicitly models the intra-cluster correlation along with the regression coefficients. We applied both approaches to the data in the NCPP example, specifying an exchangeable correlation structure under the generalized estimating equation model; results from both models were indistinguishable. We decided to present the results of the random effects modeling only for simplicity; we endorse both modeling techniques in the clustered data setting. For a more detailed discussion of the similarities and differences of these approaches, see the excellent text by Diggle et al.1