Assessing differential effects: Applying regression mixture models to identify variations in the influence of family resources on academic achievement

Abstract

Developmental scientists frequently seek to understand effects of environmental contexts on development. Traditional analytic strategies assume similar environmental effects on all children, sometimes exploring possible moderating influences or exceptions (e.g. outliers) as a secondary step. These strategies are poorly matched to ecological models of human development which posit complex individual by environment interactions. An alternative conceptual framework is proposed that tests the hypothesis that the environment has differential (non-uniform) effects on children. A demonstration of the utility of this framework is provided by examining the effects of family resources on children’s academic outcomes in a multisite study (N=6305). Three distinctive groups of children were identified, including one group particularly resilient to influence of low levels of family resources. Predictors of group differences including parenting and child demographics are tested, the replicability of the results are examined, and findings are contrasted with those using traditional regression interaction effects. This approach is proposed as a partial solution to advance theories of the environment, social ecological systems research, and behavioral genetics in order to create well-tailored environments for children.

Developmental scientists widely endorse the premise that individual children differ in how they respond to the contexts in which they live, learn, and play. A fundamental ecological axiom is that environments differ in their effects on individuals as a function of individuals’ characteristics such as age, gender, temperament, and genes (Bronfenbrenner, 1979; Friedman & Wachs, 1999; C. T. Ramey, Ramey, & Lanzi, 1998; Von Bertanlanffy, 1975). Empirical studies are frequently framed within an ecological model, and increasingly examine interaction effects as a way of understanding individual differences. Many theorists elaborate that children’s experiences of a particular environment depend on a combination of their prior and current experiences as well as their genetic and biological profiles (Bechtel & Churchman, 2002), thus theories suggest testing interactions between individual and environmental aspects which are more complex than those typically assessed by empirical research. Closely related ideas about variations in how children respond to environments are captured in the conceptual paradigm that emphasizes the need for person-centered versus variable-centered analyses (Bergman & Magnusson, 1997; Richters, 1997; von Eye & Bogat, 2006). In urging that empirical research do more to understand individual differences, these theorists emphasize equifinal functions in which individuals can achieve the same outcome through different developmental pathways and multifinal functions in which multiple and differing pathways can lead to a similar outcome (Cicchetti & Rogosch, 1996; Richters, 1997). From a quite distinctive vantage point, behavioral geneticists also affirm complex differentiated responses of individuals to their environments (Gottlieb, Wahlsten, & Lickliter, 2006). There is scant disagreement that children bring different genetic and developmental profiles to an environment, which in turn results in individual variation in how the environment shapes future development. Another example of the pre-eminence of differential environmental effects is the rapid rise in research investigating resilience and vulnerability – that is, inquiry into why children who share demographic similarities show a wide array of outcomes even when exposed to environments considered to be inadequate, harmful, or toxic (Datcher-Loury, 1989; Masten & Coatsworth, 1998; Masten et al., 1999).

Despite decades of recognizing the importance of documenting and interpreting differential environmental effects, there are remarkably few examples of investigations that systematically and thoroughly test for differential effects, beyond including gender, age, and ethnicity as variables that should always be considered (Boyce et al., 1998). Many developmentalists (Collins, Maccoby, Steinberg, Hetherington, & Bornstein, 2000; Meehl, 1990; C. T. Ramey et al., 1998; Richters, 1997; Sameroff, 1983) have recognized a mismatch between sophisticated theory guiding developmental inquiry and the analytic frameworks used to test theory. One reason for this mismatch is that there are serious limitations in the existing methods typically used to study environmental influences, including limited power associated with testing for multiple interaction terms and the necessity of an a priori identification of moderators (Boyce et al., 1998). In fact, statistical textbooks have long recommended against testing for complex interaction effects because of the difficulty in interpreting the multiple parameters needed to evaluate these effects (Cohen, Cohen, West, & Aiken, 2003). Studies that report complex interactions that show different patterns at different ages often contribute to a sense that results are too complicated to interpret or that every child will require a totally individualized environment, limiting generalizability of results and having limited implications for intervention and prevention (S. L. Ramey, 2005).

This paper proposes and then tests the utility of a relatively novel adaptation of finite mixture models, known as regression mixture models, as a potentially powerful alternative to examine differential effects of contexts. We propose a broad theoretical framework for more vigorous investigation of differential effects of environments and then demonstrate the use of these models to assess differences in the effects of family resources on achievement.

Conceptual Framework for Differences in Contextual Effects

Current approaches to evaluating environmental effects follow a process which typically emphasizes testing main effects and potentially a small set of moderators. In this framework we propose the reverse. We begin by testing the hypothesis that there are differential environmental effects. If this hypothesis is affirmed we proceed by examining the plausible reasons for these differences. This conceptual framework has wide applicability and benefits from adapting regression mixture models and other analytic approaches to overcome well-recognized limits of traditional interactions.

Our framework is illustrated (see Figures 1a and 1b) with reference to the example used in the current study, evaluating differences in the impacts of family resources on achievement and receptive language. The conceptual model posits that there is a relationship between developmental contexts (including family resources) and children’s outcomes. The next proposition is that contextual effects are not the same for all individuals. We propose a general moderating factor (see Figure 1a), which captures differences in the effects of family resources on outcomes, illustrated by the dashed arrows from the moderating factor to the effects of family resources. Given that evidence for a moderating factor is found, the conceptual framework emphasizes understanding reasons for the observed differences by bringing in theoretically relevant individual and contextual predictors; in this case, characteristics of individual children and their family (see Figure 1b).

Figure 1.

Figure 1a. Conceptual model: evaluating differential effects

Figure 1b. Conceptual model: explaining differences in effects

The most important aspect of this conceptual framework is that it relaxes the assumption of structural homogeneity (Richters, 1997) which states that if there are differences in contextual effects, all of the moderators responsible for those differences are included in the model (typically using one and two -way interactions). This framework calls for empirical identification of multiple patterns in the relationship of the environment with outcomes. Subsequently, predictors of those differences should be tested simultaneously to understand complex processes which lead to these differences. This approach is flexible and may be applied to a broad array of research areas.

Family Resources and Child Outcomes

Our conceptual model for evaluating differential effects is illustrated by examining the effects of family resources on children. The development of children in very low resource or at-risk family contexts is of pressing interest to scientists, practitioners, and those who shape and implement public policy (cf. Shonkoff & Phillips, 2000). Developmental scientists have studied the effects of poverty and other aspects of family resources and the pathways through which these family characteristics alter developmental trajectories. The majority of research about the effects of family resources has concentrated on children living below or just above the poverty line. In the United States, poverty is strongly correlated with parental education and is over-represented among racial and ethnic minorities. This research consistently reports major effects of family resources on child outcomes, especially language and academic outcomes (e.g. Brooks-Gunn & Duncan, 1997; McLoyd, 1998; Sirin, 2005).

In addition to income and socioeconomic status variables, there are many more specific dimensions of family resources that are hypothesized to influence children directly. Substnatial variation among families living in poverty has long been recognized and the social transactional features of family environments warrant a more specified conceptualization than income clustering alone (see C. Ramey, Ramey, & Lanzi (1998) for example, for an empirical typology of families living in poverty). The adequacy of time for caregiver and family interactions has been identified as one type of resource which is important for promoting positive outcomes (Dunst & Leet, 1987; Kim, 2004; Van Horn, Bellis, & Snyder, 2001). Coleman (1988) discusses the importance of social capital, or the resources available in social structures that individuals can use to meet their needs and achieve their goals. Within a family, social capital is represented by “the relations between children and parents” (Coleman, 1988, p. S110). Thus, the interaction between parents and children is essential for successful intellectual development. Within the family unit, social capital depends on both the physical presence of the parents and the attention provided by the parents to the child. When parents spend more time with their children, they are likely to provide more cognitive stimulation and impart knowledge and skills, which in turn facilitates their child’s intellectual development (Coleman, 1988; Kim, 2004). Increased time spent with parents, even simply increased time eating meals as a family, is associated with improved academic performance in addition to fewer problem behaviors (Cooksey & Fondell, 1996; Hofferth & Sandberg, 2001). Furthermore, when other types of family resources are held constant, decreased time with parents is associated with a greater likelihood of dropping out of high school (Coleman, 1988).

In addition to financial status and family time, the ability of a family to meet their basic needs such as housing, food, and clothing is also an important family resource. Although similar, the relationship between perceived ability to meet basic needs and reported financial resources is only moderate. One study found a correlation between the two of only .47 (Van Horn et al., 2001). An expanded definition of family resources would include recognition that the subjective perceptions of the adequacy of resources may contribute independent effects beyond those associated with objective measures of family resources. Supporting this tenet is research showing that perceptions of family resources may mediate the effects of more objective measures such as parental education on children’s cognitive processes (Brody & Flor, 1997, 1998). The present study includes an examination of individual differences related to subjective ratings of family resources on children’s achievement in a sample of low income families.

Differential Effects of Family Resources

While research on the average effects of disadvantage, especially economic and parental education disadvantage, consistently demonstrates negative effects on child outcomes, research which examines individual differences within those experiencing environmental disadvantage is quite limited. Most prior research relies on the assumption that the effects of family resources can be described as one average effect for all children. However, research findings are consistent with the conclusion that the relationship between family resources and academic achievement varies as a function of certain student characteristics, such as grade, minority status, and location of the school (Sirin, 2005); further, there is wide variation in the school performance of children from similar family socioeconomic backgrounds (Datcher-Loury, 1989; C. Ramey, Ramey, & Lanzi, 1998). Recognition of the possibility of differences in how family contexts impact children implies delineating and then understanding the moderators which may reduce the impacts of poverty for some children.

Another perspective with similar implications emphasizes understanding resiliency (Gutman & McLoyd, 2000; Stewart, 2006). This perspective focuses on identifying those children who succeed despite coming from low resource families. Many children show remarkable resilience in that they are able to overcome adversity to have healthy development despite significant biological and environmental risks (Masten & Coatsworth, 1998). Even in severe, ongoing adverse circumstances, some individuals still exhibit positive academic, behavioral, and social outcomes (Masten, Hubbard, Gest, Tellegen, Garmezy, & Ramirez, 1999). The implication of this approach is that there are factors which moderate the effects of family resources such that some children show a resilient response to adversity. A critique implied in much of this work is that quantitative studies that focus on average effects of poverty fail to capture the complex processes that lead some children to succeed while others struggle.

A few studies examine moderators of the effects of low-income or poverty on children’s development. The most commonly examined moderators are children’s sex (Chatterji, 2006; Huston et al., 2001; Ma, 2005) and ethnicity (Chatterji, 2006; Orr, 2003). Some of these studies simply report effects for each group without specifically testing for differences (Baharudin & Luster, 1998), while others report statistical interactions but fail to find evidence of moderation (Chatterji, 2006). Few studies have found significant interactions between ethnicity and low income. However, Pungello, et. al (1996) examined the math aptitude percentile scores of four groups of children: European Americans in a low income group, European Americans not in a low income group, African Americans in a low income group, and African Americans not in a low income group. The authors reported that the magnitude of difference between the scores of the two European American groups was greater than the difference between the two African American groups. Stronger evidence for differential effects of poverty comes from experimental trials that aimed to increase families’ incomes; two different interventions found benefits of effects of these programs for boys but not for girls (Huston et al., 2001; Leventhal, Fauth, & Brooks-Gunn, 2005). Beyond these demographic variables there is some suggestion that family and parenting may be related to differential effects (R. M. Clark, 1983; Gutman & McLoyd, 2000). In one study examining families in poverty, Gutman and McLoyd (2000) interviewed parents of high and low achieving children. They found that parents of high achievers were more supportive and positive than parents of low achievers, who were found to focus more on negative behaviors, suggesting that parenting practices may moderate the effects of poverty on academic outcomes. In sum, there is some empirical and theoretical rationale for examining differences in the effects of family resources on children’s development in order to demonstrate our broader conceptual framework for evaluating differential effects.

Methodological Challenges in Assessing Differential Effects

Differential effects are typically evaluated through the use of interaction terms which assess whether the effects of one variable on another are moderated by a third variable (Baron & Kenny, 1986). Empirical research using interactions often lags behind the theoretical rationale supporting these effects. This is partly due to the fact that few methods exist which are efficient in evaluating moderation that is more complex than two- or three- way interactions (Boyce et al., 1998). The typical method for testing these hypotheses is to identify potential moderators and then to examine whether those moderators interact with the main effects of interest, one limitation of this approach is that it can require a large number of parameters. For example, in the present study we examine whether the effect of family resources on children’s achievement and language ability differs amoung ethnic groups which requires thirty six interaction terms. With this large number of interactions the results are often difficult to interpret. An additional problem with this approach is that it involves testing a large number of null hypotheses, requiring the consideration of a Type I error rate adjustment (which could result in considerably under-powered tests of those interactions (Cohen et al., 2003).

The current study uses regression mixture models to capture differential effects. Regression mixture models involve the identification of latent (unobserved) groups of respondents who differ in the effects of predictor variables on their outcomes. While these models have been used in the marketing literature to identify groups of consumers who differ in the values they place on aspects of products (Desarbo, Jedidi, & Sinha, 2001), we know of only one methodological paper which has used this approach in the behavioral sciences (Kaplan, 2005). Regression mixture models are an extension of finite mixture models (McLachlan & Peel, 2000), a general class of statistical models which includes Latent Class Analyses, general growth mixture models (GGMM; Muthen & Shedden, 1999), and semi-parametric trajectory models (Nagin, 2005). What makes regression mixture models different is how the latent classes are defined. In most applications of mixture models latent classes are defined based on the means, variances, and sometimes covariances of observed or latent variables. In GGMM a growth curve model is estimated and latent classes are identified which differ in the means, variances, and covariances of the intercept and growth parameters. While this allows for the examination of heterogeneity in trajectories over time, it does not assess individual differences in the effects of predictors of those trajectories. For example, GGMM could identify groups of individuals who differ in their achievement trajectories and then to assess whether family resources predicts which group an individual is in. However, this is not the same thing as assessing whether the effects of family resources on trajectories vary amoung individuals since the effects of resources on class membership are constant for the entire sample.

In contrast, in the regression mixture model, means, variances, and covariances of the outcomes, and the effect of predictors (which may be either continuous or categorical) on outcomes, can all vary between latent classes. Thus, these alternative models permit assessing differences between groups in the effects of family resources. What makes these analyses unique is that, as in our conceptual framework, they allow for different effects of a predictor on an outcome to be examined without the need to include a moderator in the analyses. Analyses empirically identify latent classes of individuals who differ in the effects of predictors on outcomes. Once these classes are identified and characterized, then the influence of child and family factors on the latent classes can be assessed. GGMM and the semi-parametric models relax the assumption that individuals all follow the same basic developmental trajectory over time. Regression mixture models relax the assumption that the effects of some predictor on an outcome (either longitudinal or cross-sectional) are constant across individuals. Because regression mixtures do not require that moderators be included in the analyses, or that differential effects are due to just a small number of moderators, they offer potential to contribute to a better understanding of individual differences.

The ability to detect differential effects empirically comes at a cost. One cost is that the differences found are data driven rather than theory driven. This should be acknowledged when using these methods. Theories developed based on the results should be subjected to further testing. Another cost is that while regression mixtures may be used to examine effects that are in theory causal, we do not believe that these methods allow for making causal statements. The causal effects of a predictor (Y) on an outcome (X) for a population can be defined as the difference between the mean value of Y if everyone in the population were exposed to X and the mean value of Y if nobody were exposed to X (Maldonado & Greenland, 2002). However, regression mixtures rely on the relationship of X with Y to define separate populations; this seems to preclude making causal claims about results of regression mixtures.

Research questions

This study aims to demonstrate our conceptual framework for assessing differential effects using regression mixture models to test differences in effects of family resources on academic achievement and receptive language in a sample comprised primarily of former Head Start children. Because previous research suggests evidence that effects of low resources may be gender-specific and may depend on parenting practices, we include child sex, parenting, and ethnicity as predictors of differences in effects of family resources. This paper aims to demonstrate the utility of this statistical model to an area where there is rationale for expecting differential effects, but with limited previous research examining those differences.

This paper has five research questions: 1) Can a moderating factor which represents individual differences in the relationship of family resources and child outcomes be identified? This involves determination of the optimal number of latent classes that comprise the moderating factor and interpretation of differences in regression weights and intercepts between classes. Existing literature provides little basis for making specific hypotheses. Based on our literature review we make some broad hypotheses that lower family resources will, on average, be associated with poorer outcomes, yet there will be a group of children who will be more resilient to the negative effects of low resources. 2) If a moderating factor is identified, is it related to children’s sex and ethnicity? We use these variables as predictors of the moderating factor because these are the most commonly assessed interactions in the literature and they provide a good beginning for understanding individual differences. 3) Do parenting practices predict differences in the relationship between family resources and outcomes? We use parenting practices as predictors of the moderating factor because prior findings suggest that family processes may moderate the effects of family resources (R. M. Clark, 1983; Gutman & McLoyd, 2000). 4) Are these results stable and replicable? We examine the likelihood that the results would be replicable in another sample from the same population by comparing results from 300 bootstrapped samples from the original dataset. And, 5) Are there differences in the results obtained using regression mixture models to assess differential effects of ethnicity and those obtained using regression analyses with an interaction term? Because far fewer parameters are used to assess differential effects of ethnicity in the regression mixture model, we hypothesize that regression mixtures will be more efficient in demonstrating differential effects.

Methods

Data are from the National Head Start-Public School Transition Demonstration Study, a 30-site, five-year, longitudinal intervention study (for a full description see C. T. Ramey, Ramey, & Phillips, 1996; S. L. Ramey et al., 2001). The intervention evaluated by the larger Transition Study demonstrated no effects on children’s academic or receptive language outcomes or the family environment (S. L. Ramey et al., 2001); therefore, treatment condition is ignored for these analyses. The data for this study are publicly available for research purposes. The Transition study included two cohorts of families of kindergartners who were followed through third grade. Data were collected from 1992 through 1997. Because relatively little is known about differential effects of family resources and because longitudinal analyses are more complicated, this study uses cross sectional analyses to examine differential effects of family resources in the third grade data (collected in 1996 for cohort I and 1997 for cohort II).

Participants

The sample enrolled children who were formerly in the Head Start program and their peers from the same classrooms when they entered kindergarten. Data from 6305 third grade students and their families are included in most analyses; however, because local sites were given the option to administer the Parenting Dimensions Inventory (PDI), data for analyses including this as a predictor include 5426 students. Children in this study were 50% girls, 33% African American, 48% White/non-Hispanic, and 6% Hispanic. An additional 13% described as “other” racial or ethnic group. Because the outcome measures are known to have ethnic differences in their means (Wiig, 1985), ethnicity was used as a covariate in all analyses. Average family income was below the federal poverty line, and median parent education level was a high school diploma (31% did not have a diploma).

The Transition study included 30 sites in different states ranging from Alaska to Georgia. All major geographical regions were represented except Hawaii. Urban, suburban, and rural school districts are represented across the sites. Although the data contain a nested data structure, this study focuses on individual level processes and both predictor and outcome variables have low school-level intraclass correlation coefficients. The design effect (the multiplier by which standard errors are increased to account for clustering) due to schools in this study ranges between 1.04 and 1.13, calculated using the formula from Neuhaus and Segal (1993), justifying the use of individual level analyses.

Data Collection

With the exception of the child outcomes, data used in this study were collected via a family interview that took place in the respondent’s home. Child outcome measures were administered at school sites by a trained assessor.

Measures

Family resources were assessed with the Family Resource Scale (FRS; Dunst & Leet, 1994; Dunst, Leet, & Trivette, 1988), a measure designed to assess resources and needs of families of high-risk children. FRS data were collected in kindergarten and third grades. The FRS measures four aspects of family resources: ability of families to meet basic needs; adequacy of financial resources; amount of time spent together; and amount of time parents have for themselves (Van Horn et al., 2001). Internal consistencies for the basic needs, financial resources, time for family, and parent personal time subscales ranged from .72 to .84. Validity of the subscales has been demonstrated through relationships with other measures of family resources, including poverty level, education, and work status (Van Horn et al., 2001). In the current study, the four subscale scores for the FRS were created by standardizing and averaging items on each subscale, which implies that the items on each subscale should be weighted equally, an assumption supported by previous research (Van Horn et al., 2001). Generally, when assessing interactions, it is preferable to center predictor variables to reduce non-essential multicolinearity (Cohen et al., 2003), all analyses use Z-scores which are centered by definition.

The abbreviated form of the Parenting Dimensions Inventory (PDI; Slater & Power, 1987) was included as a predictor of class membership. The PDI comprises26 items assessing parenting practices on four dimensions: nurturance (emotional quality of relationships), responsiveness (willingness of the parent to value the thoughts and feelings of the child), nonrestrictive attitude (extent to which the parent grants the child freedom to express ideas and try new things), and consistency (predictability or uniformity of parents’ behaviors). The PDI was developed based on several existing measures of parenting, and its factor structure has been verified through confirmatory factor analysis. Internal consistencies for the nurturance, responsiveness, nonrestrictive attitude, and consistency scales were .76, .54, .70, and .79 respectively in one study (Slater & Power, 1987), and ranged from .65 to .88 in a subsequent study (Kelley, Power, & Wimbush, 1992).

Student reading achievement was measured with the broad reading and broad math scales from the Woodcock Johnson achievement test (Woodcock & Johnson, 1990). The reading and math tests both consist of two subtests: Passage Comprehension and Letter-Word Identification and Calculation and Applied Problems. The Woodcock-Johnson Achievement Test is nationally normed and standardized yielding Rasch-Wright scores. Receptive language skills were measured with the Peabody Picture Vocabulary Test-Revised (PPVT, Dunn & Dunn, 1981). The PPVT is a good predictor of school performance among low-income children (McLoyd, 1998). Because of mean differences between ethnic groups on the PPVT (Wiig, 1985), child race/ethnicity was included as a covariate in analyses. Thus, reported effects of FRS on achievement and receptive language scores control for differences in outcomes associated with child race/ethnicity.

Analytic method

Because general growth mixture models (GGMM) and the semi-parametric trajectory models proposed by Nagin (2005) have recently been presented in Developmental Psychology (Schaeffer, Petras, Ialongo, Poduska, & Kellam, 2003; Shaw, Gilliom, Ingoldsby, & Nagin, 2003), regression mixtures are compared to these approaches and differences are highlighted.

Finite mixture models

The label finite mixture model refers to a broad family of statistical models that utilize empirically-derived latent subgroups or classes to approximate an unknown overall population distribution of univariate or multivariate outcomes that can be categorical or continuous, manifest or latent. For the general form of a multivariate mixture model of observed continuous variables, consider a sample of n individuals measured on a set of m continuous variables, Y=(Y₁, Y₂,… Y_m) where y_im is the observed value on variable Y_m for subject i. In our example Y includes the two Woodcock Johnson subtests and the PPVT scores. The multivariate probability density function of Y, f(y,φ), is modeled as a mixture (weighted sum) of a finite number of probability densities, f_k(y,θ_k), corresponding to the outcome distributions for K subgroups (latent classes), with subgroup membership represented by a latent categorical variable, C, where C = 1, 2,…, K. The value of K is specified a priori, but the mixing weights (class proportions), π₁,π₂,…,π_K, are included in the set of model parameters to be estimated. The unconditional probability density function of Y is then expressed by

$$f(\mathbf{y},\phi )=\sum _{k=1}^K{\pi }_k{f}_k({\mathbf{y}}_k,{\mathbf{\theta }}_k),$$ (1)

where φ=(π,Θ) denotes the vector of all unknown parameters to be estimated; π=(π₁,π₂,…,π_K); and Θ=(θ₁,θ₂,…,θ_K). If we assume that each subgroup has a multivariate normal distribution, then we could express the outcome, Y, conditional on membership in latent class k, as

$$\begin{array}{l}{\mathbf{Y}}_{i\mid k}={\mathbf{\mu }}_k+{\mathbf{\epsilon }}_{ik},\\ {\mathbf{\epsilon }}_{ik}~N(\mathbf{0},{\mathbf{\sum }}_k),\end{array}$$ (2)

where μ_k is the vector of means, Σ_k is the variance/covariance matrix for class k, and θ_k=(μ_k, Σ_k).

To illustrate the use of this model, consider the example in the present study where Y is the vector of three outcome variables: reading achievement, math achievement, and receptive language. Population heterogeneity in the joint distribution of the outcomes can be modeled using a latent class model. For example, the population of students may derive from a mixture of two subgroups such as a group of general education students and another group of students with a reading disability. In this case, there would be two latent classes (K=2) and μ₂ would differ from μ₁ in that the mean for reading achievement would be lower whereas the means for math achievement and receptive language might be more comparable for the two classes.

GGMM models are an extension of Equation 1 where the response vector Y is comprised of the intercept and growth parameters from a latent growth model (Muthen & Shedden, 1999). Thus, the GGMM model simply identifies respondents who differ in the means, variances, and covariances of the growth parameters that describe their developmental trajectories. The semi-parametric trajectory model (Nagin, 2005) is similar to the GMM model except that the variances of the growth factors are fixed at zero within class. While the finite mixture model in Equation 2 may be useful for modeling population heterogeneity in outcomes, it does not explicitly model heterogeneity in the effects of predictor variables such as family resources. For that, it becomes necessary to specify the distribution of outcomes conditional on a set of predictor variables.

Regression mixture models

Extending Equation 2 to include predictor variables, the outcome, Y, conditional on membership in latent class k and on a set of P observed covariates which may be either continuous or categorical variables, X=(X₁, X₂,…, X_P), can be expressed as

$$\begin{array}{l}{\mathbf{Y}}_{i\mid X,k}={\mathbf{\beta }}_{0k}+\sum _{p=1}^P{\mathbf{\beta }}_{pk}{x}_{ip}+{\mathbf{\epsilon }}_{ik},\\ {\mathbf{\epsilon }}_{ik}~N(\mathbf{0},{\mathbf{\sum }}_k),\end{array}$$ (3)

where β_0k is the vector of intercepts, Σ_k is the residual variance/covariance matrix for class k, and β_pk is the vector of regression coefficients for X_p in latent class k. In our example X is a vector of responses to the four family resource subscales. This formulation allows the effect of family resources on achievement to be different for the different (latent) subgroups of students. In fact, differential effects are parameters by which the latent variable is identified. This model differs from other mixture models, such as GGMM, in a subtle but important way which can be seen by comparing equations 2 and 3. GGMM models the joint distribution of Y, whereas regression mixtures model the joint distribution of Y conditional on X. Equation 2 does not include moderation, as groups differ in their growth parameters, whereas in equation 3 the latent class variable captures moderated effects by allowing differences in regression weights between classes to be a class specific parameter. Thus, we term the latent class variable in the regression mixture model a moderating factor. This model has been previously proposed and implemented in the field of marketing research (Desarbo et al., 2001; Wedel & Desarbo, 1994).

Latent class regression

It is also possible to specify a model for class membership prediction in this mixture model framework that can be estimated in the same model that estimates φ. Consider a set of Q covariates, where z_iq is the observed value on Z_q for individual i. The set of predictors, Z, can be related to class membership using a multinomial regression model, such that

$$Pr(c_i=k\mid {\mathbf{z}}_i)=\frac{exp\left({\alpha }_k+{\displaystyle \sum _{q=1}^Q}{\gamma }_{qk}{z}_{iq}\right)}{{\displaystyle \sum _{s=1}^K}exp\left({\alpha }_s+{\displaystyle \sum _{q=1}^Q}{\gamma }_{qs}{z}_{iq}\right)},$$ (4)

where latent class K is designated as the reference class with α_k=0 and γ_k=0 for identification. In our example, Z is comprised of sex, ethnicity, and the subscales on the PDI. In the case of regression mixture models, where the latent classes are derived not only from differences between individuals on the mean and variance/covariance structure on a set of outcomes variables, but also from heterogeneity in the population with regard to the effect of a set of exogenous variables, X, on the outcomes variables, Y, predictors of class membership may be viewed as moderators of the effects of X on Y. We note that regression mixture models, either with or without predictors of the latent classes are statistically identified based only on the constraining parameters for the reference class to 0, as is detailed above, and the distributional assumptions placed on ε_ik. Identification is not dependent on the inclusion or specification of covariates in the model.

Parameter Estimation

To estimate the parameters of the model, the maximum likelihood approach is used. Because of the large number of parameters, the necessary maximization is rather expensive computationally and, therefore, not straightforward. Instead of using a Quasi-Newton-method on the complete data likelihood, a modified EM-Algorithm is used.

First, the expected value is approximated by replacing the integral with a finite sum of points, m=1,…,M. Using the conditional independence assumption of the classes, then P(W_j = m, c_ij = t |Y_j, X_j), where W_j represents the probability mass for group j at a certain point. This can be computed directly without having to find the much more complicated P(W_j = m, c_j = t |Y_j, X_j) first. Notice that this general formulation allows W_j to come from any mixing distribution. A more detailed description of the modified EM-Algorithm can be found in Muthen & Shedden (1999) and Vermunt (2003).

Use of regression mixtures in this study

In the current study, we use regression mixtures to examine differences in the effects of family resources, as measured by the four FRS subscales, on math achievement, reading achievement, and receptive language using Mplus (L. K. Muthén & Muthén, 2006). The first aim of the study, is answered by finding the number of latent classes which best fit the data and by determining if those classes are differentiated by differences in the effects of family resources. The optimal number of classes is determined by estimating models with an increasing number of classes, K, and then comparing those models using fit statistics. We investigated models that include between one and five latent classes. This is the model depicted in Figure 1a, where ‘moderating factor’ represents the latent class variable. Since we believe that within-class heterogeneity, which is how much members of classes differ from each other, is likely to vary from class to class, we allow the residual variances for the outcomes (math, reading, and receptive language) to be different for each class. The means for the outcomes as well as the regression weights of the outcomes on each FRS subscale are allowed to differ between classes. Because race/ethnicity is used as a covariate in these analyses, its effects are held constant across classes. To reduce the chance that the results are due to local maxima (Hipp & Bauer, 2006; Nylund, Asparauhov, & Muthen, 2007), analyses were run with 100 different start values and, in most cases, 90% of the start values converged to the best likelihood value.

To determine the optimal number of classes, we examine fit indices, class proportions, classification efficiency, and the interpretability of each class. The AIC, BIC, and adjusted BIC, are used to determine the correct model by choosing the model with the lowest values for each. We also use the bootstrapped likelihood ratio test (BLRT) to determine the number of classes (McLachlan & Peel, 2000). This tests the null hypothesis that a given model fits no better than a model with one fewer class. Failing to reject this test provides evidence for the model with one fewer class. This test has been shown to work well in a variety of mixture model settings (Nylund et al., 2007). Classes are interpreted based on class-specific intercepts on the outcomes, residual variances, and regression coefficients. Because the multivariate distribution of Y is the foundation for identifying latent classes, the results are sensitive to deviations from normality (Bauer & Curran, 2004). We found that there were 53 cases (less than 1% of the sample) that were over four standard deviations from the mean on any of the outcomes. With those cases included, a stable latent class solution could not be identified. The results that we report exclude those extreme cases and are quite stable, with few substantively meaningful changes between models when predictors of class membership are included.

After addressing the first aim, we consider aims two and three in which different sex, ethnicity, and parenting practices are included as predictors of class membership using multinomial regression. The diagram for this model closely matches Figure 1b, where sex and race/ethnicity, individually, are used to explain class membership. One class is selected as the reference class so that each parameter can be interpreted as the change in log odds of being in a given class for a one unit increase of the corresponding predictor. To assess aim three, the four subscales in the PDI were included as predictors of latent class membership. This demonstrates how demographic variables and contextual variables can be included in regression mixture models, allowing these models to assess how multiple processes work together to “cause” differences in effects of family resources. Because these relationships are estimated simultaneously, the inclusion of predictors of latent classes in the model can change the meanings of classes. When this happens, it suggests a lack of stability in the classes (B. O. Muthén, 2003). We do not expect results to be identical as the model specification changes and as predictors are added to the model; we do expect that the overall interpretation of the classes remains stable.

Bootstrap resampling methods

In order to examine the extent to which the model results are a function of random sample fluctuation and provide validation for these results in the current population, we used a bootstrap resampling technique (Davison & Hinkley, 1997; Efron & Tibshirani, 1993). The technique is quite simple. We take our original dataset and randomly sample cases, with replacement, until we have a sample that is the same size as those in our analyses. Because replacement is used, each sample will be different with individual cases possibly being either repeated in each new dataset or absent altogether. Because each sample contains observed data points, we can see how sampling fluctuations within the population from which the original dataset was drawn will influence the results. We drew 300 samples using this technique and ran the 2, 3, and 4-class models for each sample. We report the percentage of those runs where we would have selected a 3 or 4-class model under each criterion and report whether the models converged to results similar to those seen in the original data. Of note, 5% of the 3-class models and 26% of the 4-class models failed to converge, despite the fact that 100 start values were used for each bootstrap sample. Because failure to converge is typical when the model doesn’t fit, we interpret the failure of the 4-class model to converge (when the 3-class model did converge) as indication that the 3-class model is the appropriate one.

Results

Identification of Latent Classes Representing Individual Differences

Our analyses begin by using regression mixture models to identify groups of children who differ in effects of family resources on their outcomes (see Figure 1a). We also allowed the intercepts and variances of the outcomes to vary across groups because constraining either the intercepts or variances imposes fairly stringent assumptions and it did not make a substantive difference in model interpretation. We simplified the model slightly by constraining residual covariance to be the same between classes. Because the FRS scores are centered around the mean, the intercepts can be interpreted as the expected scores on each outcome for a child at the sample mean of family resources (the mean is zero for all four subscales). The first task is to determine the number of latent classes which best characterizes the sample. Table 1 reports fit indices and estimates of the proportion of children in each class for models with 1 through 5 classes. The best model should have the lowest value on the penalized information criteria (BIC, ABIC), indicating that the 3-class model is best when judged by the BIC. The adjusted BIC plateaus at 3-classes, is 2 units smaller at 4, and then increases. Only the AIC does not support the 3-class solution; this is not surprising as simulation studies have demonstrated that the AIC typically overestimates the number of classes needed (Nylund et al., 2007). The bootstrapped likelihood ratio test (BLRT) is an empirical test for whether each model fits better than a model with one fewer class, supporting the 3-class solution¹. Finally, with the 4-class model, the smallest class contains less than 1 percent of the children, which is too few to reliably identify. With the exception of the AIC, the evidence supports the 3-class model which forms the basis for the rest of this study.

Table 1.

Fit indices for regression mixture models

	1 class	2 class	3 class	4 class	5 class
Log-Likelihood	−70881	−70322	−70228	−70174	−70129
Parameters	30	49	68	87	106
AIC	141821	140741	140592	140522	140470
BIC	142024	141072	141051	141110	141185
ABIC	141928	140916	140835	140833	140848
entropy	n/a	0.52	0.36	0.57	0.52
BLRT (p-value)	n/a	<.01	<.01	0.11	0.21
% class 1	100	27	23	1	1
% class 2		73	36	12	10
% class 3			42	37	10
% class 4				50	38
% class 5					41

Note: N = 6305, BLRT is the bootstrapped likelihood ratio test

While there is good support for the 3-class model, the entropy value (how well the model is able to classify individuals) is low at .36. Low entropy is expected since the classes are differentiated primarily based on the effects of family resources. We expected that low entropy would be caused mostly by the classes that differ primarily in regression weights because the overlapping regression lines make it very difficult to distinguish individuals as being in a particular class. This evidence was supported by an inspection of posterior probabilities. This might indicate that the classes are not stable, in which case, as predictors are added, we would expect the interpretation of the classes to change. It could also indicate that the classes are not well-separated based only on differential regression weights and intercepts, in which case the entropy should increase with the addition of predictors without greatly affecting interpretation of the classes. We do not believe that low entropy should play a large role in model selection in regression mixture models since with no predicators of class membership there is little individual level data for classifying a particular person and entropy would be expected to be low; however, it is important to examine model stability. Results, below, indicate that these findings were stable when covariates were included. If the model is not efficiently classifying individuals, it follows that posterior probabilities for each individual have limited value. Finally, classes may capture quantitative rather than qualitative differences in effects of family resources, in which case the classes should be interpreted less as an indication that there are subpopulations for which these effects differ than as a tool that captures a continuum of individual differences. We expect that the inclusion of predictors of class membership will help clarify these results.

The next step is to interpret the meaning of the three classes. The largest class (see Table 2) contained about 42% of the respondents². For this class it is clear that family’s reported ability to meet its basic needs is the strongest predictor of outcomes, and is related to higher achievement. Measures of effect size for each regression weight within each class are obtained using partial correlations (computed by standardizing the variances within classes). The effect sizes for basic needs range from .22 to .28 for the first class and are the strongest effects for any predictor across all classes. There is also an effect of the availability of money for this class such that more money relates to better achievement, although that effect is only significantly different from 0 for the receptive language outcome. In class 1 there is no unique effect for parent personal time and there is a consistent, small negative impact for time the family spends together. Possible explanations for this negative effect are discussed later, but we note here that the zero-order relationship of time for the family and outcomes in the entire sample is small, but positive. Because this class is characterized by the positive effects of basic needs, it will be called the basic needs class.

Table 2.

Parameter estimates, standard errors, and effect sizes for 3 class model

	Basic Needs (42%)			Resilient (36%)			Low acheivement (23%)
Reading	B	(SE)	r	B	(SE)	r	B	(SE)	r
Intercept	489.67	(0.55)		486.16	(0.85)		465.57	(1.09)
Basics	2.71	(0.86)	0.22	−0.86	(0.94)	−0.06	1.77	(0.93)	0.09
Money	1.37	(0.79)	0.11	2.24	(1.00)	0.15	3.70	(1.10)	0.18
Time for Self	−0.34	(0.69)	−0.03	−0.14	(0.84)	−0.01	−0.11	(1.21)	−0.01
Time for Family	−1.66	(0.84)	−0.14	−0.45	(0.78)	−0.03	−2.05	(1.22)	−0.10
African American	−1.97	(0.37)		−1.97	(0.37)		−1.97	(0.37)
White/non-Hispanic	4.57	(0.33)		4.57	(0.33)		4.57	(0.33)
Hispanic	−1.50	(0.62)		−1.50	(0.62)		−1.50	(0.62)
Residual Variance	116.52	(7.58)		209.98	(10.06)		399.57	(22.79)
Math
Intercept	486.09	(0.66)		489.95	(0.93)		472.82	(1.09)
Basics	2.52	(0.61)	0.22	−0.53	(0.65)	−0.06	0.49	(0.58)	0.04
Money	0.96	(0.72)	0.09	0.99	(0.62)	0.11	2.32	(0.81)	0.18
Time for Self	−0.48	(0.65)	−0.04	0.49	(0.55)	0.05	−0.06	(0.73)	0.00
Time for Family	−1.43	(0.64)	−0.12	−0.61	(0.53)	−0.06	−1.28	(0.83)	−0.10
African American	−1.79	(0.29)		−1.79	(0.29)		−1.79	(0.29)
White/non-Hispanic	2.50	(0.27)		2.50	(0.27)		2.50	(0.27)
Hispanic	−0.98	(0.50)		−0.98	(0.50)		−0.98	(0.50)
Residual Variance	112.15	(7.22)		82.42	(7.35)		168.11	(10.78)
Language
Intercept	99.60	(0.44)		102.46	(0.36)		97.84	(0.38)
Basics	2.37	(0.61)	0.28	0.14	(0.25)	0.02	0.75	(0.42)	0.09
Money	1.45	(0.47)	0.17	0.28	(0.38)	0.04	1.43	(0.42)	0.17
Time for Self	−0.65	(0.38)	−0.04	−0.08	(0.31)	−0.01	−0.28	(0.39)	−0.03
Time for Family	−1.18	(0.53)	−0.12	−0.27	(0.31)	−0.04	−1.17	(0.45)	−0.14
African American	−2.34	(0.17)		−2.34	(0.17)		−2.34	(0.17)
White/non-Hispanic	4.69	(0.16)		4.69	(0.16)		4.69	(0.16)
Hispanic	−1.24	(0.28)		−1.24	(0.28)		−1.24	(0.28)
Residual Variance	47.77	(3.54)		35.33	(1.76)		57.15	(4.80)

The second class, comprising 36% of the students, is characterized by the lack of a relationship between family resources and outcomes. There is some evidence that parent’s perception of greater adequacy of money relates to higher achievement, but this is significantly different form 0 only when predicting reading achievement. Students in this class have a slightly lower intercept than those in the basic needs class on reading achievement, but have somewhat higher intercepts on both math achievement and receptive language. In general, students in this class perform well. These students are only significantly affected by one of the family resource measures and for only one of the outcomes, with a relatively small correlation of .15. Because these students are relatively unaffected by a lack of family resources, we term this the resilient class.

About 23% of the students are classified as belonging to the third class which is distinguished by having low intercepts and positive effects of adequacy of money on achievement which are significant for all outcomes and slightly stronger than in the other two classes. Students belonging to this class are best characterized by being much lower than other students on the outcomes, especially reading and math achievement. This class is thus termed the low achievement class.

To illustrate how these classes represent different effects of family resources, Figure 2 depicts the relationship of FRS subscales with the reading outcome for each class. Level of family resources is on the x axis. “Low” is 2 standard deviations below the mean and “high” is 2 standard deviations above the mean. The lines represent the linear effects of each FRS subscale score for each class; they cross in the middle for each class because the FRS scores are centered at 0. In Figure 2 the differences in the intercepts are apparent; the intercept in the low achievement group is much lower. It is evident that the effects of family resources are not negligible. For example, a child with low basic needs in the basic needs class would be expected to read at a lower level than an average child in the resilient class. The difference in reading achievement for a child of low versus high basic needs is almost a full standard deviation. The extent to which the slopes of family resources vary between classes illustrates differential effects.

Figure 2.

Relationship of Family Resources to Reading Achievement

Predictors of Class Membership

In the second aim, multinomial regression is used to assess the relationship of sex and race/ethnicity with the latent classes (see Figure 1b). The resilient class is the reference class, therefore, the parameter estimates reported are the log odds of being in each of the other classes versus the resilient class. Results (see Table 3) indicate that girls and White/non-Hispanic children are about half as likely to be in the low achievement class than in the resilient class and that African American and Hispanic students are more likely to be in the low achieving than in the resilient class. The odds for African Americans of being in the low achieving versus the resilient class are 1.4 while for Hispanics they are almost 1.8. Females are two and a half times more likely than males to be in the basic needs class than the resilient class while African Americans are nearly half as likely as the average child to be in the basic needs class than the resilient class. This indicates that boys are less likely to be affected by basic needs than girls, and African Americans and White/non-Hispanics are less likely to be affected by basic needs than the average across all racial/ethnic groups.

Table 3.

Multinomial regression of class membership on race/ethnicity and sex

	B	(SE)	OR
Low Achieving vs. Resilient
Intercept	−0.472	(0.13)
Female	−0.61	(0.13)	0.54
African American	0.343	(0.12)	1.41
White/non-Hispanic	−0.674	(0.15)	0.51
Hispanic	0.565	(0.20)	1.76
Basic Needs vs. Resilient
Intercept	−1.402	(0.42)
Female	0.965	(0.24)	2.62
African American	−0.503	(0.24)	0.60
White/non-Hispanic	−1.528	(0.37)	0.22
Hispanic	0.312	(0.34)	1.37

Note: N = 6287

An important question in these analyses is how much the interpretation of the classes changes with the inclusion of predictors. Changes in the class specific regression weights would suggest that the results are not robust. The entropy increased from .36 in the unconditional model (with no predictors of class membership) to .51 in this model, indicating that the predictors increase the ability of the model to classify individuals. Next we looked at the parameter estimates for each class and found that the results, when sex and ethnicity were included, changed slightly³. The low achievement class remained unchanged and only one effect that was significant no longer was significant. The basic needs class remained relatively unchanged. The effects of basic needs remained strong and significantly different from zero. The negative effects of time for the family remained about the same size, but the standard errors increased and effects were no longer significant. Finally the effect of parental time for self on PPVT scores was now significant and negative. In sum, these results indicate that the effects of family resources vary across groups of children and that those groups differ in their make-up in terms of race/ethnicity and sex.

Parenting Practices and the Impact of Family Resources

The next analyses assess whether self-reported parenting predicts how children are affected by family resources. In these analyses four dimensions of parenting (nurturance, responsiveness, nonrestrictive attitude, and consistency) were added to the multinomial latent class regression⁴. Results indicated that children with more responsive parents were about half as likely to be in the low achieving rather than the resilient class (see Table 4). Further, children with more nurturing, responsive, and less restrictive parents were less likely to be in the basic needs versus the resilient class, with the odds ratios being about .50 for each of these effects. The entropy value in this model increased to .63, suggesting that adding parenting dimensions increased the ability of the model to classify individual children into classes. This finding provides some explanation for the observation that the basic needs class was positively impacted by the factors of basic needs and negatively impacted by time spent with the family. These results show that children who experience more negative parenting practices tend to be negatively impacted by the lack of basic resources and to be at increased risk for poor academic outcomes if they spent greater amounts of time in families where parents report higher negative parenting practices.

Table 4.

Multinomial regression of sex, race/ethnicity, and parenting on class membership

	B	SE	OR
Low Achieving vs. Resilient
Intercept	−0.642	(0.13)
Female	−0.524	(0.12)	0.59
African American	0.399	(0.15)	1.49
White/non-Hispanic	−0.375	(0.11)	0.69
Hispanic	0.29	(0.19)	1.34
Nurturance	0.002	(0.06)	1.00
Response to Child	−0.545	(0.09)	0.58
Nonrestrictive Attitude	−0.067	(0.07)	0.94
Consistency	−0.038	(0.06)	0.96
Basic Needs vs. Resilient
Intercept	−3.033	(1.00)
Female	0.865	(0.43)	2.38
African American	−0.587	(0.50)	0.56
White/non-Hispanic	−0.983	(0.65)	0.37
Hispanic	−0.311	(0.52)	0.73
Nurturance	−0.607	(0.29)	0.54
Response to Child	−0.722	(0.23)	0.49
Nonrestrictive Attitude	−0.864	(0.37)	0.42
Consistency	0.176	(0.14)	1.19

Note: N = 5426

Assessing Model Stability

All of the models reported for the first aim were rerun on 300 bootstrapped datasets with the goal of finding the probability that these results would be replicated. Given the sensitivity of these techniques to the shape of the outcome distribution, we didn’t expect that the results would always be the same, but we wanted to establish that, given the same population, the researcher would typically find the same results. The first indication of model performance is the percentage of models that converged to an acceptable solution; all 300 models converged for the 2-class solution, 95% of the models converged for the 3-class solution, and 74% converged for the 4-class solution. The poor performance of the 4-class model is likely caused by the very small 4^th class which is more sensitive to sampling variability than the others. Of those models for which the 3-class solution converged, 26% indicated the 3 over the 4-class solution using the AIC criterion, 51% indicated the 3-class solution using the BIC, 27% indicated three classes using the adjusted BIC, and 33% indicated the 3-class solution using the BLRT. This is somewhat consistent with the results reported above where the 3-class solution was indicated by the BIC and BLRT, and the 4-class solution was preferred using the AIC, the adjusted BIC was ambiguous. Of course, before selecting the 4-class solution the analyst would want to be sure that the 4-classes were meaningful and had reasonable representation. Just as in the analyses above in which the solution was rejected because of a low portion of respondents in one group, in our bootstrapped samples, when the 4-class solution converged the smallest class had on average 2% of the respondents, and in only 5% of the replications did the smallest class contain over 5% of the sample. Thus, we conclude that if the 4-class model were to converge, about 95% of the time it would be considered inadequate due to small class sizes and the 3-class model would be accepted.

In order to establish the stability of interpretation of the 3 class model, we sorted the results of each replication by the defining features of each class. Since the low achieving class is easiest to identify given its low intercept for reading achievement, we started by identifying that 94% of the replications were successful in obtaining a low achievement class with a reading intercept of less than 475. The next easiest class to identify is the Basic Needs class, which has a relatively strong impact of basic needs. These results had a few extreme outliers indicative of unacceptable solutions (in one case the regression weight for reading on basic needs was 32), and some other cases in which the results would have a different interpretation (the effect of basic needs was essentially 0). However, 73% of the replications had reasonably sized and significantly different from zero positive effects of basic needs such that the interpretation of this class would be similar to those in the analyses reported above. Finally, the resilient class is most easily distinguished by having no significant effect of basic needs; this third class had both a reasonable and non-significant effect of basic needs in 68% of the bootstrapped samples that converged.

In summary, based on results from the 300 samples meant to represent 300 draws from the same population that we started off with, we would have identified the 3-class model about 90% of the time, factoring in replications which didn’t converge and replications in which fit indices suggested a 4-class solution but in which one of the classes was small enough to discount. Of the 3-class models that converged, the low Achieving class was reliably identified, and we would have replicated the same substantive meaning of the other two classes about 70% of the time.

Comparing Regression Mixture Analyses and Traditional Interactive Models

The final analyses looked at how the results using regression mixtures compare to more traditional regression models. An advantage of regression mixture models is that they potentially provide a parsimonious explanation of complex interactions. Thus, it was important to compare results of the regression mixture model to those obtained using interactions in a linear regression model. We focused on interactions between family resources and race/ethnicity so that the results would be comparable to those in Table 3. There are four racial/ethnic groups in the present analyses, four predictors, and three outcomes, necessitating 36 parameters to examine the interaction of ethnicity and family resources in predicting reading achievement, math achievement and receptive language. Note that regression models are much more efficient at finding interactions when few parameters are involved (i.e. family resources and sex). The purpose of this analysis is to show what happens with more complex interactions.

Results from these analyses demonstrated that four of the 36 effects examined were significantly different from zero (α=.05) (see Table 5). The only effect that showed some consistency was the interaction between being African American and basic needs. As compared to the average, African Americans were less affected by a lack of basic needs. This is consistent with results from the second aim showing that African Americans are more likely to be in the resilient class than the basic needs class. More importantly, while five of the six effects of ethnicity were significant in the regression mixture models, only four of 36 were significant using traditional interaction methods. In general, the conclusion drawn from using linear regression models with interactions would be that there is no consistent interaction between ethnicity and family resources. Using regression mixture models, the opposite is found. While this is not a simulation study allowing us to compare results to a predetermined ‘truth,’ the contrast between needing 36 rather than 6 parameters to capture the interaction makes a compelling case for the efficiency of regression mixtures. Further, estimating 36 fairly highly correlated parameters reduces power due to multicolinearity.

Table 5.

Linear regression model of outcomes on FRS with race/ethnicity interactions

	Reading		Mathematic		PPVT
	B	(SE)	B	(SE)	B	(SE)
Intercept	482.68	(0.31)	484.63	(0.22)	100.48	(0.12)
Basic	1.32	(0.41)	0.90	(0.29)	1.17	(0.16)
Money	2.30	(0.49)	1.25	(0.35)	1.09	(0.20)
Time for self	−0.80	(0.50)	−0.51	(0.35)	−0.61	(0.20)
Time for family	−0.89	(0.47)	−0.63	(0.33)	−0.68	(0.18)
African American	−3.23	(0.42)	−2.27	(0.30)	−2.58	(0.17)
White/non-Hispanic	5.27	(0.40)	2.99	(0.28)	4.92	(0.16)
Hispanic	−2.32	(0.73)	−1.34	(0.52)	−1.35	(0.29)
African American*basic	−1.30	(0.54)	−0.55	(0.38)	−0.83	(0.21)
African American*money	0.33	(0.67)	−0.10	(0.48)	−0.04	(0.26)
African American*self	0.23	(0.67)	0.52	(0.48)	0.32	(0.27)
African American*family	0.24	(0.62)	0.09	(0.44)	0.11	(0.25)
White/non-Hispanic*basic	0.08	(0.58)	−0.31	(0.41)	−0.76	(0.23)
White/non-Hispanic*money	0.23	(0.62)	0.60	(0.44)	−0.06	(0.24)
White/non-Hispanic*self	0.49	(0.62)	0.46	(0.44)	0.47	(0.25)
White/non-Hispanic*family	−0.98	(0.62)	−0.92	(0.44)	−0.13	(0.25)
Hispanic*basic	0.53	(1.00)	0.37	(0.71)	0.02	(0.40)
Hispanic*money	0.16	(1.15)	−0.26	(0.82)	0.26	(0.46)
Hispanic*self	−1.88	(1.13)	−1.40	(0.81)	−0.19	(0.45)
Hispanic*family	1.13	(1.03)	1.30	(0.74)	0.08	(0.41)
Residual variance	309.90	(5.52)	157.46	(2.80)	48.57	(0.87)

Note: Parameters indicated in bold are significant at p<.05, N = 6287

Discussion

This study demonstrates the use of a new conceptual framework to investigate differences in environmental effects. In contrast to traditional approaches examining moderation, this study begins by testing for the presence of a moderating factor, an indication that the effects of family resources are not the same for all children. Using regression mixture models, a relatively novel statistical approach, we identified three latent classes of children which differed in intercepts of outcomes and relationship of achievement and language ability with family resources. One class was characterized by having low intercepts, especially on reading; this class includes children with learning challenges or functional disabilities. The other two classes differed in the effects of family resources, but had similar intercepts. After finding evidence for differential effects, multiple factors, suggested by previous research to be related to these differences, were examined.

The resilient or unaffected class confirms the hypothesis that not all students are negatively affected by a family’s relative lack of resources. A significant proportion of this former Head Start sample had relatively high levels of achievement and language ability and appeared to be resilient to effects of low family resources. As opposed to students in the basic needs class, and consistent with findings of a few other studies (Gutman & McLoyd, 2000; Leventhal et al., 2005; Pungello et al., 1996), these children are more likely to be female and to be in families where parents are more nurturing and responsive, and had a less restrictive attitude. The basic needs class is also substantively important; this is a group of children who, when at the average levels of basic needs and time with family, score about the same on the outcomes as their resilient peers. However, this class is strongly affected by basic needs. A child in this class who is low on basic needs will score almost a full standard deviation on all outcomes below a child whose basic needs are met. It is also notable that the effects of time spent with family are significantly negative in this class. Children in this class are differentiated from those in the resilient class by having parents who are lower on nurturing and responsiveness, and are more restrictive. One hypothesis for this finding is that for children in less positive family environments, time spent with family does not lead to positive outcomes, whereas, more positive parenting tended to promote resiliency.

This study adds to the small body of research looking for individual differences in the effects of family resources. This study is unique in that it looks at differences in the effects of perceptions of resources and includes sex, ethnicity, and parenting together, finding that all of these factors have some impact on individual differences. One of the powerful aspects of regression mixtures is that differential effects can be found empirically, however, because this approach is largely data driven it becomes critically important that these results be replicated, particularly with respect to the finding of a negative effect of time spent with family for the basic needs class. Thus, a take home message from these results is that more research is needed looking at individual differences in the effects of both poverty and the perceptions of family resources.

Whenever the substantive interpretation of results is driven by the data rather than theory, it is important to evaluate critically whether the results are just a function of random fluctuations in the data. We believe that this approach should be similar to methods used to assess validity in psychometric analysis (L. A. Clark & Watson, 1995; Cronbach & Meehl, 1955). This is a process which may involve replication of results in independent samples, cross-validation, and testing of specific interaction terms suggested by the models. Ultimately, we see regression mixtures as a useful tool for developing theories about differential effects of contexts; these theories should be tested using diverse approaches. In this study, we provide evidence for the replicability of our results using a bootstrapping technique which allows us to assess how often we would obtain similar results from a different sample from the same population. The answer was somewhat encouraging and instructive about the use of regression mixture models: in 5% of the cases we would not have been able to obtain results for a 3-class model, and in the remaining cases we would have found substantively similar classes 70% of the time. We take this to mean that the results we report are not a fluke of random sampling and that the results are reasonably sensitive to effects of random sampling on distributional properties of the outcomes. This supports the valuable role of these models in useful for developing theories rather than testing theories.

An additional aim of this paper was to compare the results of regression mixture models with those obtained using the traditional approach of testing interactions. One major difference is how the two methods approach the problem: using GLM, specific interactions are tested with product terms, whereas in regression mixtures, one first tests for evidence that the effects of the variable(s) of interest are moderated by other variables. This means that traditional approaches should be efficient at finding a specified interaction, but that they are limited in ability to test whether the effects of one variable on another are uniform across a population. The fact that traditional models require separate interaction terms for every interaction limits their ability to test complex moderation. This study demonstrated that ethnic differences in the relationship of family resources with achievement would not have been evident using traditional interaction models, a result which contrasts sharply with evidence from regression mixture models. The regression mixture models are advantageous because they require fewer parameters to estimate differential effects and do not start with the assumption that differences are due to a single moderator.

Regression mixtures are unique in their ability to identify differential effects empirically, even with cross sectional data. In a longitudinal context it is sometimes possible, using traditional methods, to model the effects of one variable on another as varying between individuals. However, this approach is limited to certain situations in which the predictor variable is measured repeatedly. Regression mixture models permit the finding of differential effects empirically through the identification of groups of individuals differing in the effects of one variable on another.

The results of this study provide support for the utility of an inductive approach to examining differential effects. However, this study has a number of limitations. First, cross sectional analyses were employed, making it difficult to determine the causal mechanism of this relationship. While it seems unlikely that children’s achievement affects family resources (perhaps with the exception of time with the family), effects not included in these analyses might account for the results. While longitudinal analyses may help clarify these results, the application of regression mixture models to longitudinal data is not straight forward and should be examined in additional methodological work. Additionally, these results only apply to the population of relatively low-income families. It is likely that the results would look different in other populations. The appropriate use of these techniques involves replication of the results over multiple studies. Further, this framework should not displace testing for theoretically important moderating effects using traditional methods. We advocate a global evaluation of differential effects. If particular moderators are the focus of an investigation, it is typically better to use traditional approaches. This is especially true if investigators have specific hypotheses about each parameter in the model.

Although regression mixtures are powerful and efficient, they also have disadvantages. The latent classes comprising differential effects are identified by making strict assumptions about the multivariate distribution of outcomes. The current study was nearly ideal in that the sample size was large and distributions of the outcomes were close to multivariate normal. In general, we believe that regression mixture models are best viewed as a large-sample technique, though further methodological research is needed before sample size guidelines are provided. We also believe that more theoretical and simulation work is needed to understand the performance of regression mixture models in different situations,(i.e. when outcome distributions are not multivariate normal). In our view, these models have potential to inspire new empirically based possibilities for assessing individual differences in much the way that structural equation modeling opened up possibilities for assessing complex mediation. Finally, we caution against making causal statements based on results of regression mixture models.

Individual differences are an important but under-tested component of many developmental theories. It is generally accepted that children respond differently to the same environment. However, too frequently, quantitative research ignores differences and focuses on average effects. This study provides strong evidence for differential effects of family resources and presents a compelling methodology for assessing these differences. More importantly, we hope that the framework and methodology used in this study will provide an impetus for developmental scientists to match theory, involving differences in environmental effects, with more congruent empirical tests.

Appendix A: Mplus code for Table 3

title: Regression mixture model for Table 3;
data: file is c:\data\frs2.dat;
variable:
names are newid READ_RW3 MATH_RW3 PPVT_RW3 BASIC_3
 MONEY_3 TIMES_3 TIMEF_3 childsex black hisp white;
usevariables are READ_RW3 MATH_RW3 PPVT_RW3 BASIC_3
 MONEY_3 TIMES_3 TIMEF_3 childsex black hisp white;
missing are all (-9);
classes=c(3);
analysis: type=mixture;
     starts=100 20;
model:
%overall% ! Outcomes are regressed on family resources and ethnicity
READ_RW3 on BASIC_3 MONEY_3 TIMES_3 TIMEF_3 white black hisp;
MATH_RW3 on BASIC_3 MONEY_3 TIMES_3 TIMEF_3 white black hisp;
PPVT_RW3 on BASIC_3 MONEY_3 TIMES_3 TIMEF_3 white black hisp;
! Using sex and ethnicity to predict class membership
C#1 on childsex white black hisp;
C#2 on childsex white black hisp;
%c#2% ! Seperate model for class 2
READ_RW3 on BASIC_3 MONEY_3 TIMES_3 TIMEF_3; ! These statments allow
MATH_RW3 on BASIC_3 MONEY_3 TIMES_3 TIMEF_3; ! regression weights to
PPVT_RW3 on BASIC_3 MONEY_3 TIMES_3 TIMEF_3; ! vary betwen classes
READ_RW3; ! These statments allow
MATH_RW3; ! residual variances to
PPVT_RW3; ! differ between classes
READ_RW3 with MATH_RW3 PPVT_RW3; ! covariances between outcomes
MATH_RW3 with PPVT_RW3;     ! are allowed to vary betwen classes
%c#3%
READ_RW3 on BASIC_3 MONEY_3 TIMES_3 TIMEF_3; ! These statments allow
MATH_RW3 on BASIC_3 MONEY_3 TIMES_3 TIMEF_3; ! regression weights to
PPVT_RW3 on BASIC_3 MONEY_3 TIMES_3 TIMEF_3; ! vary betwen classes
READ_RW3; ! These statments allow
MATH_RW3; ! residual variances to
PPVT_RW3; ! differ between classes
READ_RW3 with MATH_RW3 PPVT_RW3; ! covariances between outcomes
MATH_RW3 with PPVT_RW3;     ! are allowed to vary betwen classes
output:
sampstat standardized;