Another casualty of sibling fixed-effects analysis of education and health: An informative null, or null information?

Reducing educational inequalities in cardiovascular disease (CVD) will not be achieved by increasing educational attainment. That seems to be the conclusion of Madsen et al.’s discordant-twin study of the relation between educational attainment and cardiovascular disease (Madsen et al., 2014), and virtually all prior studies that used the same approach of comparing the health outcomes of siblings (or twins) with different levels of educationd-they also found that educational effects on health were substantially reduced or eliminated entirely when compared to general population estimates (Amin et al., 2013; Behrman et al., 2011; Fujiwara and Kawachi, 2009; Gatz et al., 2001; Gilman et al., 2008; Lawlor et al., 2006; Madsen et al., 2010, 2014, 2011; Naess et al., 2012; Osler et al., 2007; Sondergaard et al., 2013, 2012; Webbink et al., 2010).

Madsen et al. used data from the Danish Twin Registry to compare the risk of CVD between 32,432 monozygotic and dizygotic twins with different attained educations (either primary, secondary, or tertiary). Their “intra-pair” analyses yielded no statistically significant associations between education and heart disease. Importantly, Madsen et al.’s paper carefully reviewed the strengths and weaknesses of the within-sibling (i.e., sibling fixed-effects) approach to causal inference, strengths and weaknesses that have been commented on previously at great length (Donovan and Susser, 2011; Frisell et al., 2012; Gilman and Loucks, 2012; Kaufman, 2008; Kaufman and Glymour, 2011; Keyes et al., 2013; Lahey and D’Onofrio, 2010; Madsen and Osler, 2009; McGue et al., 2010; Susser et al., 2010). How definitive are Madsen et al.’s results? The answer hinges on the question posed in our title: does a null finding arising from a study using within-sibling comparisons prove the absence of a causal effect? And the answer to that question can be addressed from the perspectives of precision and bias.

1. Precision

From the perspective of precision, effect estimates from fixed-effects analyses are invariably less precise than corresponding general population estimates, and this was the case in Madsen et al.’s study. For example, the odds ratio (and 95% confidence interval) for the effect of secondary versus primary education on cardiovascular disease among female dizygotic twins was 0.88 (0.80–0.97) in the unpaired analyses, and 0.85 (0.71–1.02) in the within-pair analyses. The effect estimates were virtually the same, yet the smaller sample size in the within-pair analyses lead to reduced precision. Consider the parallel effect estimates for ischemic heart disease: 0.71 (0.59–0.87) in the unpaired analyses and 1.00 (0.69–1.44) in the within-pair analyses. Here, for the within-pair analyses, the odds ratio was completely attenuated (to the null value of 1), but as expected the confidence interval became wider along the way. This unfortunate state of affairs puts the reader in a difficult position: how is it possible to compare results of two studies that used different study designs (between-pair versus within-pair comparisons) and have different statistical power (a lot versus not much)? Any difference between them could be attributable to the design difference, the power difference, or both. Both differences are important, and should factor into evaluating whether or not the within-pair results are interpreted as informative nulls (strongly suggesting the absence of a causal effect), or null results that are estimated too imprecisely to be conclusive. We think the examples above illustrate both situations: the within-pair analyses of cardiovascular disease are indeterminate, whereas the within-pair analyses of ischemic heart disease provide strong evidence of no causal effect in twins.

There is an important corollary point about the precision of the within-pair analyses. That is, the precision depends on the number of discordant pairs for each high-versus-low education contrast. The overwhelming majority of discordant pairs in Madsen et al.’s study were pairs in which the discordancy was 1 educational category (primary versus secondary or secondary versus tertiary). In contrast, the absolute number of primary-versus-tertiary twin pairs was small: 462 male and 596 female dizygotic pairs, and 152 male and 208 female monozygotic pairs (from their Table 2). The number of cardiovascular events within these twin pairs was unfortunately not reported, but it must have been very low. Therefore, we have much greater confidence in Madsen et al.’s findings regarding the hypothesized causal effect of an intervention that increases education from primary to secondary, or secondary to tertiary, than an intervention that leaps individuals from secondary school through college.

2. Bias

From the perspective of bias, the numerous commentaries cited above cover the sources of bias that are most likely in a within-sibling approach to causal analysis, particularly confounding by non-shared factors. Where the exposure of interest is educational attainment, we are particularly concerned about factors that lead some individuals go farther in school than their brothers or sisters. Any factor that does, and that isn’t controlled for, is an important source of unmeasured confounding. Put another way, in Bill Clinton’s biography …

“How could two brothers be so different: the governor and the coke dealer, the Rhodes scholar and the college dropout, one who tried to read three hundred books in three months and another who at his most addicted snorted cocaine sixteen times a day, one who could spend hours explaining economic theories and another whose economic interests centered on getting a new Porsche?” (Conley, 2004; Maraniss, 1995)

The literature on sibling differences has identified a range of factors that are associated with the concordance of outcomes among children raised in the same household: for example, birth order (first born children tend go to school longer), socioeconomic status (children in higher socioeconomic status households have greater variability in educational outcomes), and family chaos (associated with a higher degree of parents’ differential treatment) (Atzaba-Poria and Pike, 2008; Conley et al., 2007; Reiss et al., 1995). Had Madsen et al.’s study found strong effects in their within-pair analyses, we would be suspicious of residual confounding by these factors (although less so in regards to birth order, given that Madsen et al. studied twins (Zajonc, 2001)). That they found no support for a causal effect of education on ischemic heart disease is perhaps more reassuring (from a causal inference perspective). We suggested previously that a null result from a sibling fixed-effects analysis is potentially more informative than a positive result (i.e., one that is significantly different from zero) (Gilman and Loucks, 2012). VanderWeele et al. recently articulated a similar point with respect to Mendelian randomization studies, and we think that their point applies in just the same way here:

“… the biases would have to align perfectly to move the effect estimate to zero when there is in fact a true effect. There is a whole range of nonzero values for false positives to take; there is only one zero value that a false negative may take. A null association, with narrow confidence interval, … may thus arguably provide more robust evidence of the negative conclusion of no or very little effect.” (Vanderweele et al., 2014, p. 434)

3. Should public health practitioners advocate for expanding educational opportunities?

The weight of the evidence from studies using quasi-experimental designs to investigate the association between education and health suggests that interventions that produce relatively small increases in educational attainment (i.e., increases within the range of valid inferences from the within-sibling studies, plus the studies using instrumental variable and propensity matching approaches (Loucks et al., 2012)) would have practically no health benefitd-and would not meaningfully reduce the dramatic educational inequalities in health that have persisted for generations (Pappas et al., 1993). That said, there is strong evidence that expanding educational opportunities produces economic growth at the societal level, and leads to robust individual returns in the form of earnings at the individual level (Ashenfelter and Zimmerman, 1997; Blundell et al., 2005; Card, 1999; Hanushek and Woessmann, 2012; Leonhardt, 2014). Therefore, providing a relatively small increase in schooling after years of exposure to cumulative disadvantage might not be sufficient for reversing the long-term health consequences of early life experiences (Knudsen et al., 2006).

4. What are the most likely (familial) causes of educational inequalities in health?

Reducing educational inequalities in health will require interventions that target the common causes of both, and there is evidence for the following candidates. Parental socioeconomic status strongly predicts offspring’s education (McLoyd, 1998; Sewell and Shah, 1968) and risk of cardiovascular disease (Galobardes et al., 2006; Loucks et al., 2009). Other aspects of the early childhood environment, both negative (childhood neglect and abuse) and positive (parental monitoring), are also associated with both educational attainment and health (Dong et al., 2004; Loucks et al., 2011; McLoyd, 1998; Midei and Matthews, 2011; Morton et al., 2014). This evidence is bolstered by randomized trials of early life enrichment programs such as the Perry Preschool and Abecadarian projects that have been shown to lengthen participants’ educational careers and to improve their adult profile of cardiovascular risk factors including health behaviors, blood pressure, and metabolic syndrome (F. Campbell et al., 2014; F. A. Campbell et al., 2002; Muennig et al., 2009). Moreover, these programs appear highly cost effective: a cost benefit analyses of the Perry Preschool program showed a return on investment of $7.16 to the public for every $1 spent on high quality early life education (Schweinhart et al., 1993). These trials are relatively small, and much work remains to understand the ability of early child interventions to reduce long-term educational inequalities in health. At the same time, it is very important to recognize that the overwhelming majority of studies reporting educational inequalities in health (Manrique-Garcia et al., 2011; McLaren, 2007), including studies like Madsen et al.’s that used quasi-experimental designs, focus on the putative causal effect of years of schooling. These studies do not address the potential health benefits of interventions that improve the quality of schooling and content of learning; the health benefits of such endeavors cannot be discounted by Madsen et al.’s or other similar negative findings. That said, despite the clear income returns to increasing years of education, epidemiologic evidence suggests that a substantial portion of educational inequalities in health may be addressed by investing in the social and economic conditions of early childhood (Conti and Heckman, 2010), which maximally promote early child development and as a result, benefit health over the life course.