Introduction to symposium on unmeasured heterogeneity in school transition models

Abstract

Researchers have used models of school transitions for over 30 years to describe inequality of educational opportunity and have contributed a number of important refinements and extensions. School transition models have the complication that the estimated effects of family background on the probability of continuing in school are affected by differential attrition on unobserved factors at earlier stages of schooling. The articles in this symposium present a variety of useful approaches to unobserved heterogeneity in school transition models. Investigators who use these approaches should attend to several issues: (1) models for school transitions may be used both descriptively (and are not therefore subject to any well-defined “bias”) and as tools for causal inference. (2) The concept of bias presupposes an underlying experiment, structural model, or population model that would, in principle, define the corresponding unbiased parameters – yet these underlying models are difficult to specify for school transition models. (3) Unobserved determinants of whether individuals make school transitions may be both exogenous and endogenous with respect to the observed regressors in the model. Without a model of how unobserved heterogeneity arises, attempted “corrections” for unmeasured heterogeneity may yield misleading estimates of the effects of measured determinants of school continuation.

The articles in this symposium present a variety of perspectives on and approaches to the problem of “unmeasured heterogeneity” in the analysis of the effects of social background and other factors on individuals’ decisions to continue through school. They are revised versions of papers presented in a didactic workshop at the May 2010 meetings of the Research Committee on Social Stratification of the International Sociological Association (RC28) in Haifa. The workshop, led by Buis, was an important development for RC28 inasmuch as it recognized that work on school transition models remains a lively subfield of stratification research and has undergone substantial development during the past 30 years. The value of such workshops lies in publically consolidating our knowledge, transmitting the wisdom and technical knowhow of several generations of researchers to our newest members, and recognizing enduring problems. I hope that we have similar workshops on other areas of stratification research in future meetings. We are grateful to Buis and his colleagues for taking this initiative. In these remarks I place these articles in perspective through a short review of the motivation for and history of work on school transitions in social stratification research, the problem of unmeasured heterogeneity, and some of the remaining conceptual issues that ongoing research educational stratification should address. Some of this discussion revisits material that I present in a recent commentary (Mare, 2006), but my comments on the problem of unmeasured heterogeneity appear here for the first time.

1. The school transitions approach to educational stratification

Although the articles in this symposium generously credit my early work on this topic, its roots extend back at least 15 years earlier to important contributions by others. Duncan (1965, 1968) analyzed cohort trends in the United States in school progression ratios, the aggregate counterpart to microdata on school continuation decisions. She also pioneered the study of family background effects on educational attainment, albeit through the use of linear models of highest grade of school completed (Duncan, 1967). Spady (1967) analyzed the associations between father's educational attainment and school continuation ratios using published tabulations from the 1962 Occupational Changes in a Generation Survey. Boudon (1974) analyzed the role of education in class mobility, essentially taking a school transitions approach to education, albeit relying mainly on simulations and artificial data. Hauser (1976) criticized Boudon's work on a variety of grounds, including its use of methods that confounded associations between class origins and school progression with ceiling and floor effects of the marginal distribution of educational attainment. My early work (Mare, 1979, 1981a, 1981b) blended these strands of work by exploiting the 1973 Occupational Changes in Generation II Survey (Featherman & Hauser, 1975), Hauser and Featherman's (1976) cohort trend analyses of these data, new methods for the “regression analysis of life tables” (Cox, 1970), and developments in software and computing that made flexible forms of maximum likelihood estimation available to applied researchers. I applied logistic regression model for discrete time event histories (Allison, 1982) to the study of social background effects on school progression and showed the connection between this way of looking at educational stratification and what were, at the time, standard approaches based on analyses of number of years of schooling completed.

Building on these early developments, many researchers have extended these approaches to the study of educational stratification. This is not the place for a comprehensive review of this literature. In my estimation, however, some of the most important developments have included: (1) comparative research on educational stratification, notably the pioneering project organized by Shavit and Blossfeld (1993); (2) related models for education systems that have complex systems of early tracking and that cannot be reduced to a simple single sequence of irreversible transitions (Breen & Jonsson, 2000); (3) investigation of the effects of whether parents make key school transitions on their offspring's school transitions (Mare & Chang, 2006); (4) the use of panel data and time-varying explanatory variables in school transition models (Lucas, 2001); and (5) Cameron and Heckman's (1998) several contributions, including discussion of the crucial issue of scale identification in binary response models, a formal statistical treatment of dynamic selection bias including a semi-parametric (latent class) model for unobserved heterogeneity, and a behavioral model of educational decision-making.

The power of the school transitions approach lies in its recognition that individuals obtain their educational attainment through a cumulative, time-dependent process. For a number of societies, this process can be usefully viewed as a single sequence of irreversible transitions although it does not, even in the United States, capture all of the complexities of educational progression. But it does capture the essential aspects of educational mobility and stratification and permits a richer investigation of basic processes of intergenerational transmission than the study of summary measures of educational attainment such as highest year of school completed in a linear regression model or an ordered classification of attainment levels in a nonlinear (ordered) regression model. This approach has a number of advantages: (1) by treating educational attainment as a sequence of events, the analysis of transitions makes it possible to ask at which stages of the process is inequality the greatest. (2) Because school transitions are typically age-graded, it is possible to link school transition rates and their inequalities to characteristics of schools, labor markets, and political environments as well as to family circumstances. (3) It makes it possible to decompose educational stratification among socioeconomic or other types of groups into (a) the relative chances of groups of making a given school transitions, and (b) the composition of the population that is “at risk” of making a given transition although, of course, the population at risk to making a given transition is affected by differential rates of school progression at earlier stages. This is a valuable approach to the description of educational inequality, though it is clearly not the only approach. My own early writing on the subject perhaps contributed to sometimes fruitless arguments about the “best” way to measure inequality of educational opportunity. A variety of approaches, based on both school transitions and also other summary measures of school attainment, are needed to describe such complex phenomena. (4) Although my own primary interest in school transitions remains the apt description of educational stratification, by focusing on key decision points that individuals face as they go through school, these empirical models may be aligned with behavioral models of educational decision-making. Most efforts along these lines have assumed that school continuation decisions are made by students and their families (e.g., Breen & Goldthrope, 1997; Cameron & Heckman, 1998; Morgan, 2005), although it is possible to extend these models to represent the intersecting actions of multiple parties (families, counselors, teachers, admissions officers, employers, banks, etc.) (e.g., Gamoran & Mare, 1989; Manski & Wise, 1983).

2. The problem of unmeasured heterogeneity

Unmeasured heterogeneity is a problem for the analysis of school transitions because, for all transitions beyond the first, the population at risk to making a transition is typically a nonrandom subset of all observations in an entry cohort of students. This a special case of nonrandom “sample selection,” in which closely related processes govern both whether an individual gets to be at risk to the transition in question and also whether the individual makes this transition. Comparisons of the effects of socioeconomic background on grade progression, whether across transitions within a single cohort, across cohorts, or across populations, are made difficult by the confounding of true variation in stratification processes with variation in the pattern of selection of individuals into the risk sets for school transitions. Depending on the goals of one's study, variations in patterns of selectivity may be regarded either as part of an explanation for variation in socioeconomic background effects across transitions or as a source of “bias” that requires some kind of statistical “correction” so as to isolate other sources of variation in background effects.

In my initial work I was concerned the about heterogeneity effects on the estimated effects of social background, for interpreting both intracohort and intercohort variations in the estimated effects of socioeconomic background (Mare, 1980, 1981a, 1981b), though I was not aware of the emerging literatures on sample selection bias (e.g., Heckman, 1979) and unmeasured heterogeneity in survival models (e.g., Heckman & Singer, 1984; Vaupel & Yashin, 1985) – which provide a formal apparatus for understanding this problem. Nor did I attend to scale identification in binary response models, which also bedevils efforts to compare logit coefficients across transitions (Allison, 1999; Cameron & Heckman, 1998; Long, 1997; Mare, 2006; Mood, 2010; Winship & Mare, 1983, 1984). My own efforts were confined to illustrating how taking account of a major source of unmeasured heterogeneity (e.g., mental ability) can alter one's conclusions about how background effects vary across transitions (Mare, 1980) and using sibling data to control for unmeasured family level heterogeneity (Mare, 1993). These approaches illustrated the problem and suggested some ways of tackling it, but they were, by today's standards crude and unsystematic.

The articles in this symposium report state of the art efforts at describing, exploring the implications of, and taking account of unmeasured heterogeneity in models of educational stratification. Holm and Jaeger (2011) apply the bivariate probit model for sample selection to adjust the estimated effects of socioeconomic background on school transitions, and demonstrate its value in an analysis of the British National Child Development Survey. Buis (2011) proposes a method for analyzing the sensitivity of estimates of school transition models to alternative assumptions about unmeasured heterogeneity and applies the method to data for the Netherlands. Tam (2011) uses simulated data to evaluate Cameron and Heckman's (1998) latent class model for dynamic selection bias and shows how estimation of this model may be improved by incorporating measured indicators of heterogeneity into the latent class framework. Karlson (2011) employs a semi-parametric multinomial logit model with latent classes to take account of unmeasured heterogeneity in the Danish multi-path educational system. In their analysis of recent cohorts in the United States, Lucas, Fucella, and Berends (2011) employ an impressive arsenal of procedures to deal with the challenges of comparing the effects of socioeconomic background on school transitions within and across student cohorts, including time varying covariates, which are tantamount to including a much richer set of measured family background measures combined with exclusion restrictions; a larger set of measured covariates than are typical in school transition studies; bivariate probit models for modeling the joint process of selection into the risk set for a transition and the transition itself; careful attention to scale identification; and longitudinal observation.

Taken together, these articles illustrate a wide variety of tools for representing unobserved influences in school transition models. They show that a correct understanding of these processes may depend on the specific empirical context in question, the type of available data – including whether they are longitudinal or cross sectional and how rich a set of observed measures are available –, and what assumptions are plausible for estimating a model. They contain a good deal of wisdom about practical aspects of estimating and interpreting school transition models, yet also show that more research is needed on the properties of alternative estimation methods, on the relationship between measured and unmeasured heterogeneity, and on the robustness of conclusions to alternative methods.

3. When should we worry about unmeasured heterogeneity?

3.1. Description and causal inference

The strengths of the articles in this symposium notwithstanding, users of models for school transitions should also consider some basic issues before proceeding to “correct” for “bias” in their analyses of educational stratification. Some of these issues are usefully laid out by Xie (2011) in his contribution to this symposium. The first issue concerns the purposes of these analyses. Empirical work includes both description of empirical relationships and also attempts to estimate the (causal) effect of one variable on another. Either of these efforts may be in the service of evaluating social science arguments (whether explicit theories or more informal claims) because some of the empirical implications of these arguments may best be translated into descriptive statistics, whereas others imply explicit empirical models. Some forms of description bear only loose relationship to any specific argument or prediction, yet nonetheless have potential value for assessing arguments that may arise down the line. In stratification research, both description and the assessment of arguments are part of our business, though much of our work on social mobility and educational, occupational, and other forms of socioeconomic attainment is motivated by descriptive rather than explicitly causal questions. This observation applies to the analysis of bivariate occupational mobility tables and to multivariate analyses in which we estimate the partial associations between various social background factors and attainment.

That so many of our analyses of stratification processes are avowedly descriptive raises the question of whether and when we should be concerned about “bias.” So long as our measures and populations are well conceived, our samples well-drawn, and our rates of non-response are sufficiently low (or properly adjusted), we can learn much about stratification processes and how they vary between time and place. Although social scientists occasionally worry about “omitted variable bias” in assessing the effects of educational attainment on economic outcomes – most commonly in the econometric study of “ability bias” in estimates of the economic returns to schooling – we seldom work from established “structural” models and are usually satisfied with sum maries of observed data. Why then we should specially worry about bias in the analysis of school transitions? Evidently this concern arises because of the unique structure of transition data. At each school transition we seek to estimate the partial association between making the transition in question and each regressor, net of the effects of the other regressors in the model. As in other regression models estimated on nonexperimental data, our estimates almost always depend on what variables are included and omitted from the regression. Additionally, however, in school transition models, our estimates depend on the process of selection that yields a particular sample “at risk” to making each transition. The selection process depends on the volume of school attrition that has occurred in earlier transitions, the effects of measured variables on prior transitions, and the effects of unmeasured variables on those transitions. The problem of “unobserved heterogeneity,” therefore, is created by nonrandom selection on unobserved determinants of school transitions. If it is possible to “correct” for this unobserved heterogeneity, the correction is evidently to put our estimates of the effects of family background on school transitions on a similar footing to those that we obtain from other types of conventional analyses of the determinants of socioeconomic attainment. We do not seek unbiased estimates in any absolute sense because, for the most part, our goal is to achieve apt descriptive statistics rather than precise causal inference.

3.2. Experiments and other frameworks for thinking about bias

Even when the concern about bias is limited to the problem of selective attrition in school transition models, an estimated empirical relationship can be properly said to be “biased” only if there are clearly defined parameters for which an unbiased estimate is in principle attainable. These parameters must be based on an idealized model that specifies the conditions under which unbiased estimation can occur. One type of model is the randomized experiment, in which the (causal) parameter of interest is a contrast between a treatment and a control group. A second type of model is a “structural” model in which institutional setting, preferences, opportunities, and behavioral rules are well defined and (causal) parameters quantify the behavioral relationships that these definitions imply. A third type of model is a “population” model in which a sampling mechanism is specified and a sample (descriptive) statistic is an estimator for an underlying population parameter. In each of these cases parameter estimates may be biased for a variety of reasons, including sample selectivity.

The randomized experiment is perhaps the easiest of these to conceptualize, though the hardest to implement in practice. In studying the effects of social background on progression through school, we might hypothetically assign children at random to families that vary in socioeconomic standing (as measured by the educational attainments, occupations, etc. of the parents). For the first transition – that is, the transition at which all children are at risk (assuming universal initial school attendance), mean differences in transition rates among children with different types of families would be unbiased estimates of the population level effect of family characteristics. For the second (and later) transition(s), however, these mean differences would no longer be unbiased because school attrition at the first (and later) transitions would be the result of both measured differences in transition rates among family types and unmeasured heterogeneity in the families.

It is difficult to formulate an experiment that would correspond to efforts to “correct” for this unmeasured heterogeneity. One possibility is to imagine an intervention in which children who failed the first transition were (counterfactually) permitted to try to make the second (and later) transition(s). If such an intervention were possible, then the original randomization of children to families would allow us to use mean differences in success rates at the second transition as an unbiased estimator of family effects on that transition. But this hypothetical intervention eliminates the fundamentally conditional nature of school transitions by focusing on all children, rather than just those who remain at risk. Moreover, a real world implementation of this intervention would amount to a radical change in the institutional setting in which children acquire educational credentials. The selectivity of higher transitions is, at least in societies that organize schooling as a single sequence of academic hurdles, one of their defining features. It is questionable whether estimates based on the counterfactual assumption that later transitions are not selective can be properly regarded as unbiased estimates of effects for societies in which educational selectivity is a fundamental feature of the process.

An alternative interpretation of bias in school transition models and of methods that “correct” for bias is that school attrition alters the distribution of the student population on unmeasured traits, both considered by themselves and jointly with measured characteristics of students. The correction for bias amounts to reweighting the students who are at risk at the second (and later) transition(s) so as to produce the same distribution of measured and unmeasured family traits that were observed in the initial cohort of children. This interpretation has the virtue that it does not assume universal completion of the first transition. Instead, it is tantamount to estimating the effects of family background on making the second transition under the counterfactual assumption that attrition at the first transition occurred at random (conditional upon the measured family characteristics). Although this is a more satisfactory basis for comparing the effects of family characteristics on school continuation across transitions, one must still be cautious. By hypothetically imagining a second transition in which the eligible children were selected at random – rather than through a selective process – one is considering a different institutional setting than the actual one that we wish to study. Whether this hypothetical modification of the institutional setting – for the purposes of achieving a correction for selectivity – yields interpretable results, may depend on the institutional context and the specific goals of an analysis.

3.3. Where does heterogeneity come from?

Yet even recognizing the limited degree to which it is possible to correct for bias in school transition models (because true unbiasedness is seldom attainable or even a goal of standard models of mobility and stratification) and the difficulty of obtaining interpretable estimates (because of the lack of a plausible corresponding experiment or structural model to which a model that is free from unmeasured heterogeneity would correspond), there remains the problem of interpreting the unobserved heterogeneity itself. If we have unmeasured heterogeneity, is it predetermined with respect to the measured family background variables in the analysis or, in part, a function of those family variables? Progression through school is a function of not only measured social characteristics but also unmeasured personal characteristics, such as “ability.” Although some aspects of ability may be innate and thus predetermined with respect to measured independent variables in a school transition model, others aspects are surely affected by the families in which children are raised. Because families vary in large part along lines measured by socioeconomic background in models of school transitions, ability is often, at least in part, endogenous with respect to family background. This raises the thorny issue of whether all or only some forms of unmeasured heterogeneity should be taken into account. Just as in conventional linear models of the effects of a treatment on an outcome, one usually wishes to control for exogenous covariates of the treatment, but not endogenous variables that may represent mechanisms through which treatments exert their effects. A blanket control for all unobserved variables is likely to overcontrol for the effects of the treatment. In a nonexperimental context, this problem can only be addressed with an explicit model of the ways that unobserved variables combine with observed variables to affect the outcome. Similarly, in models of school transitions, it is important to distinguish, at least in principle, between unobserved heterogeneity that is simply correlated with measured family background and which thus should be taken into account, and unobserved heterogeneity that is in fact a consequence of measured family background and which thus should not be controlled. Again, outside of an experimental context, this problem can only be addressed with an explicit model of how unobserved heterogeneity works.

4. Conclusion

The analysis of schooling continues to be a central focus of stratification research because it occurs at the stage of individuals’ lives when their social standing comes to depend more on their own accomplishments and less on those of their parents. The analysis of school transitions continues to be a focus of educational stratification research because successes and failures in school transitions are the concrete behaviors through which educational inequality comes about. It is likely that progress in our understanding of school transitions will be best achieved through improvements in data and measurement – that is, through doing a better job of studying measured heterogeneity – as well in our behavioral and descriptive models. Some promising lines of work are shown in the articles presented in this symposium. Yet developing appropriate methods and models for unmeasured heterogeneity in school transition rates and probabilities remains an important part of the social science agenda because understanding unmeasured variation is essential to interpreting observed variation in educational success. But how we understand unmeasured heterogeneity depends crucially on the specific purposes of our analyses and the underlying models that we have in mind. The approaches presented in this symposium are a good foundation for further progress on these issues.