Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2021 Jul 1.
Published in final edited form as: Psychol Med. 2019 Jul 1;50(9):1530–1538. doi: 10.1017/S0033291719001533

Internalizing and Externalizing Psychopathology in Middle Age: Genetic and Environmental Architecture and Stability of Symptoms Over 15 to 20 Years

Daniel E Gustavson 1, Carol E Franz 2, Matthew S Panizzon 3, Michael J Lyons 4, William S Kremen 5
PMCID: PMC6938573  NIHMSID: NIHMS1042518  PMID: 31258104

Abstract

Background:

Internalizing and externalizing psychopathology factors explain much of the covariance among psychiatric conditions, especially at the level of genetic risk. However, few studies have examined internalizing and externalizing factors in middle-aged samples, especially their ability to predict later symptoms across midlife. The goals of the current study were (i) to quantify the genetic and environmental influences on internalizing and externalizing psychopathology in individuals in their early 40s, and (ii) examine the extent to which these genetic and environmental influences predict self-reported measures of internalizing and externalizing symptoms 15–20 years later.

Method:

1484 male twins completed diagnostic interviews of psychopathology at mean age 41 and self-reported measures of anxiety, depression, substance use, and related variables at up to two time-points in late middle age (mean ages 56 and 62).

Results:

Structural equation modeling of the diagnostic interviews confirmed that internalizing and externalizing factors accounted for most of the genetic variance in individual disorders, with substantial genetic (ra=.70) and environmental (re=.77) correlations between the factors. Internalizing psychopathology at age 41 was correlated with latent factors capturing anxiety, depression, and/or post-traumatic stress symptoms at ages 56 (r=.51) and 62 (r=.43). Externalizing psychopathology at age 41 was correlated r=.67 with a latent factor capturing aggression, tobacco use, and alcohol use at age 56. Stability of both factors was driven by genetic influences.

Conclusions:

These findings demonstrate the considerable stability of internalizing and externalizing psychopathology symptoms across middle age, especially their genetic influences. Diagnostic interviews effectively predict self-reported symptoms and behaviors 15–20 years later.

Keywords: heritability, twin study, anxiety, depression, antisocial personality disorder, drug use, structural equation modeling

Introduction

Theories of psychopathology have emphasized the sources of common and specific variance among disorders, clustering disorders into internalizing and externalizing domains (Weiss et al., 1998, Lilienfeld, 2003, Kendler and Myers, 2014). The internalizing domain comprises anxiety disorders, depressive disorders, post-traumatic stress disorder (PTSD) and other related conditions (Weiss et al., 1998, Caspi et al., 2014, Kotov et al., 2017). Similarly, the externalizing domain comprises substance use disorders, antisocial personality disorder, conduct disorder, and attention-deficit/hyperactivity disorder (Weiss et al., 1998, Kotov et al., 2017). In contrast to narrowband-specific sources of variance that capture the features that are unique to particular disorders (e.g., physiological arousal in anxiety), internalizing and externalizing latent factors have been useful in capturing the shared variance among disorders in these two broad domains. These factors tend to explain much of the associations among individual disorders, as well as their associations with cognition and personality (Krueger et al., 2002, Whiteside and Lynam, 2003, Young et al., 2009, Hink et al., 2013, Tackett et al., 2013, Caspi et al., 2014, Gustavson et al., 2017).

Internalizing and externalizing factors have historically been studied in children and adolescents (Weiss et al., 1998, Krueger et al., 2002, Lilienfeld, 2003), especially in twin studies that inform their genetic and environmental architecture (King et al., 2004, Blonigen et al., 2005, Hink et al., 2013, Tackett et al., 2013, Gustavson et al., 2017). Genetic influences continue to account for the commonality between disorders at a pairwise level well into adulthood both within the internalizing (Chantarujikapong et al., 2001, Koenen et al., 2008) and the externalizing domains (True et al., 1999a, True et al., 1999b), and across the two domains (Xian et al., 2000, Fu et al., 2007). However, when middle-aged or older adults are included in studies of internalizing and externalizing psychopathology, they tend to have heterogeneous age ranges that included adolescents and/or young adults (Krueger, 1999, Kendler et al., 2003, Eaton et al., 2011, Kendler and Myers, 2014). Thus, it will be useful to demonstrate the genetic and environmental structure of internalizing and externalizing factors in middle age within a sample with a more homogenous age range.

Although genetic influences on individual disorders are relatively stable throughout the lifespan (Eley et al., 2003, Bergen et al., 2007, Bornovalova et al., 2009, Petkus et al., 2016), studies have not yet established the stability of internalizing and externalizing psychopathology across the lifespan. Because genetic influences on both psychopathology factors account for much of the genetic variance in individual disorders (e.g., Tackett et al., 2013, Kendler and Myers, 2014, Gustavson et al., 2017), internalizing and externalizing factors also likely demonstrate at least moderate genetic stability over time. Furthermore, the longitudinal studies on individual disorders noted above focus on early life (ages 0–30) or late life (> age 60), so it is still somewhat unclear the extent to which genetic influences on psychopathology demonstrate stability in midlife.

In the current study, we began to address these issues by utilizing data from a large study of middle-aged male twins. Participants completed diagnostic interviews of psychopathology symptoms in early middle age (~age 41) as part of the Harvard Drug Study of Substance Abuse (HDS; Tsuang et al., 2001). These twins also completed self-reported measures of relevant psychopathology symptoms at mean age 56 and/or 62 as part of the Vietnam Era Twin Study of Aging (VETSA), including measures of anxiety symptoms, depression symptoms, PTSD symptoms, and aggression (Kremen et al., 2006, Kremen et al., 2013a, b). They also answered basic questions about their current tobacco and alcohol use. We tested the hypothesis that latent internalizing and externalizing psychopathology factors (at age 41) would predict similar latent factors capturing internalizing and externalizing symptoms/behaviors in late middle age, and that much of this stability would be explained by genetic influences. Such evidence would be consistent with recent findings that genetic influences on psychiatric disorders are similar to those on quantitative variation of milder versions those same disorders (e.g., anxiety disorders vs. trait anxiety) in the general population (for review, see Martin et al., 2018), but would extend these findings by demonstrating that there is also longitudinal stability in this covariance.

Method

Subjects

Analyses were based on 1484 individual male twins (433 monozygotic [MZ] twin pairs, 308 dizygotic [DZ] twin pairs, 10 unpaired twins) from the Vietnam Era Twin Registry. All of the men underwent psychiatric diagnostic interviews in early middle age as part of the HDS (M=41.29 years, SD=2.42) and participated in at least one of the two waves of the longitudinal VETSA project at average age 56 (N=1291, M=55.90 years, SD=2.44) or average age 62 (N=1207, M=61.72 years, SD=2.45). Most twins completed both waves of VETSA (N=1014).

All VETSA participants were recruited randomly from the HDS and were not selected for history of any diagnosis. The only inclusion criteria for VETSA 1 was that both twins must have been between ages 51 and 59 at the time of recruitment, and that both twins in a pair agreed to participate in the study. All twins were invited to participate in the second wave of VETSA regardless of the participation of their co-twin. Individuals all served in the United States military at some time between 1965 and 1975 and are generally representative of American men in their age group with respect to health and lifestyle characteristics (Kremen et al., 2006, Schoenborn and Heyman, 2009). Nearly 80% did not serve in combat or in Vietnam, and lifetime PTSD rates (7.3%) were similar to that of the general population (Kilpatrick et al., 2013; see Table S1 for lifetime prevalence of other disorders based on DSM-III-R criteria at mean age 41). Table 1 displays basic demographic characteristics of the sample.

Table 1.

Demographic Characteristics of the Sample

Demographic Variable N M SD Range
Lifetime Education (years) 1483 13.80 2.11 5, 20
Lifetime Occupation 1477 5.47 1.87 0, 9
Race/Ethnicity (N=1478)
 White non-Hispanic (%) 1307 0.88 - -
 White Hispanic (%) 35 0.02 - -
 American Indian (%) 9 0.01 - -
 Pacific Islander (%) 3 0.00 - -
 African American (%) 75 0.05 - -
 More than 1 race (%) 40 0.03 - -
 Declined to answer (%) 9 0.01 - -
Age
 at diagnostic interview 1484 41.29 2.42 35, 46
 at VETSA wave 1 1291 55.90 2.44 51, 60
 at VETSA wave 2 1207 61.72 2.44 56, 66
Open in a new tab

Note: Lifetime education was the number of years of school completed. Lifetime occupation was based on the Hollingshead Four-Factor index (Hollingshead, 1975), from 0 (unemployed) to 9 (major professionals). A mean of 5.47 corresponds to a clerical worker, sales worker, technician, or semiprofessional. VETSA = Vietnam Era Twin Study of Aging.

Measures

Internalizing and externalizing psychopathology (Age 41).

Psychopathology symptoms were assessed using structured interviews based on the Diagnostic Interview Schedule, Version III Revised (DIS-3R; Robins et al., 1988), administered by telephone. Questions from the DIS-3R assess mental disorders according to the revised third edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-III-R; American Psychiatric Association, 1987). All measures used in these analyses were based on the count of lifetime symptoms, including both dependence and abuse symptoms.1

Because all summed symptom counts were positively skewed, measures were binned into ordinal variables for each disorder, thereby creating polychotomous measures. This method produces unbiased estimates of censored data in twin models, and is preferred to transformations in liability-threshold models such as those examined here (Derks et al., 2004). Bins were selected prior to analyses based on three a priori considerations, similar to our previous work using a different twin sample (Gustavson et al., 2017): (i) equalizing the n per bin as much as possible, (ii) maximizing the number of bins for each trait (i.e., to preserve as much variance as possible), and (iii) attempting to have at least 150–200 individuals per bin (i.e., to prevent empty bins after subdividing the sample based on twin pair and zygosity). The binning procedure for each disorder is described here and displayed in Table S2, but more information on the unbinned variables can be found in the supplement (Table S1).

Internalizing psychopathology was assessed based on symptoms of generalized anxiety disorder (GAD), major depressive disorder (MDD), and post-traumatic stress disorder (PTSD)2. For GAD, 3 bins were created based on symptom endorsement: 0 symptoms (n=1199), 1–6 symptoms (n=132), and 7 or more symptoms (n=151). For MDD, 5 bins were created: 0 symptoms (n=507), 1 symptom (n=331), 2 symptoms (n=213), 3–4 symptoms (n=218), and 5 or more symptoms (n=213). For PTSD, 4 bins were created: 0 symptoms (n=879), 1–3 symptoms (n=220), 4–7 symptoms (n=203), and 8 or more symptoms (n=174). PTSD symptoms were based on the number of unique symptoms reported across three events.

Externalizing psychopathology was based on symptoms for tobacco use disorder, alcohol use disorder, cannabis use disorder, and antisocial personality disorder (ASPD). For tobacco use disorder, 5 bins were created: 0 symptoms (n=513), 1–2 symptoms (n=263), 3 symptoms (n=234), 4 symptoms (n=215), and 5 or more symptoms (n=259). For alcohol use disorder, 5 bins were created: 0 symptoms (n=477), 1 symptom (n=228), 2 symptoms (n=233), 3–4 symptoms (n=318), and 5 or more symptoms (n=224). For cannabis use disorder, 3 bins were created: 0 symptoms (n=1166), 1 symptom (n=131), and 2 or more symptoms (n=182). For ASPD, 4 bins were created: 0 symptoms (n=438), 1 symptom (n=399), 2 symptoms (n=293), and 3 or more symptoms (n=351).

Internalizing symptoms (Age 56 and 62).

At mean age 56, self-reported internalizing symptoms were assessed using three scales: the 20-item Center for Epidemiologic Studies–Depression scale (CES-D; Radloff, 1977), the stress reaction subscale of the Multidimensional Personality Questionnaire factor-form NZ (MPQ; Tellegen, 1985, Krueger et al., 1996), and one item related to anxiety in the Short-Form Health Survey (Have you been a very nervous person? SF-36; Ware Jr and Sherbourne, 1992).

At mean age 62, self-reported internalizing symptoms were assessed using three scales: the CES-D, the 20-item trait form of the State-Trait Anxiety Inventory (STAI; Spielberger, 1983),3 and the 17-item self-report DSM-IV-based PTSD Checklist civilian version (PCL; Weathers et al., 1993). At both waves, all measures were log transformed to improve the distributional characteristics.

Externalizing symptoms/behaviors (Age 56).

At mean age 56, externalizing symptoms/behaviors were assessed with four measures: the Aggression subscale and Constraint factors of the MPQ, tobacco use, and alcohol consumption. The MPQ aggression subscale was included to capture antisocial tendencies and the reverse-scored MPQ constraint factor (including control, harm avoidance, and traditionalism subscales) was included to assess behavior disinhibition, similar to previous work (Krueger et al., 2002). Tobacco use was a categorical variable in which individuals were given a score of 1 if they answered yes to the question “Do you smoke cigarettes now?” or answered yes to similar questions for cigars, pipe tobacco, or chewing tobacco (n=385; see supplemental method for more information). Otherwise, they were given a score of 0 (n=906). Alcohol use was based on the total number of alcoholic beverages reported in the past two weeks, binned into 5 bins: no drinks (n=437), 7 or less drinks (n=127), 8–14 drinks (n=234), 14–28 drinks (n=215), and >28 drinks (n=233). We were unable to create a measure of externalizing symptoms/behaviors at age 62 because the MPQ was not assessed at this wave and there were no other comparable measures of personality (e.g., aggression, antisocial behavior). Although tobacco use and alcohol consumption data were available, the latent factor would not be identified with only two indicators.

Data Analysis

Analyses were conducted using Mplus Version 7.2 (Muthén and Muthén, 2010–2012). For all analyses involving polychotomized symptom count data, we fit liability-threshold models and accounted for missing data using pairwise deletion with weighted least squares, means and variance adjusted (theta parameterization). For models without ordinal variables (Figure 3c only), we used full-information maximum likelihood. Model fit was determined using chi-square values (χ2), the root-mean-square error of approximation (RMSEA), and the comparative fit index (CFI). Models had good fit if the χ2 value was less than two times the degrees of freedom, RMSEA<.06, and CFI>.95 (Hu and Bentler, 1998). Significance of parameters was evaluated with χ2 difference tests using the “difftest” option. We also provide 95% confidence intervals (95% CIs) using standard errors, but when significance tests based on standard errors disagree with χ2 difference tests, we only interpret the latter (see supplemental Data Analysis). Figures display standardized factor loadings, but the full unstandardized and standardized output of all models are displayed in the supplement (Tables S5–S9).

Figure 3:

Figure 3:

Open in a new tab

Model of the genetic (A), shared environmental (C), and non-shared environmental (E) overlap between externalizing psychopathology at age 41 and an externalizing factor at age 56 based on measures of aggression, constraint, current tobacco use, and current alcohol use. Variance explained by latent variables (ovals) can be computed by squaring these standardized factor loadings. Measured variables are displayed in rectangles. Significant factor loadings are displayed in bold with black lines (p < .05) and 95% standard error-based confidence intervals are displayed in brackets. This model fit the data well: χ2(285)=316.73, p=.102, RMSEA=.017, CFI=.990. ASPD = antisocial personality disorder, Cannabis = Cannabis use disorder, Alcohol = Alcohol use disorder, Tobacco = tobacco use disorder, MPQ Aggr = Multidimensional Personality Questionnaire – Aggression subscale, MPQ Constr = Multidimensional Personality Questionnaire – Constraint subscale.

Genetically-informative analyses were based on the assumptions of the classical twin design, which decompose variance in a trait (and covariance between traits) into three sets of influences. Additive genetic influences (A) are assumed to correlate at 1.0 for MZ twins and 0.5 for DZ twins because MZ twins share 100% of their alleles identical-by-descent while DZ twins share 50% of their alleles identical-by-descent. Shared environmental influences (C), which are influences that make twins in a pair more similar, are assumed to correlate at 1.0 in both sets of twins. Non-shared environmental influences (E), which are influences that make twins dissimilar, are assumed to be uncorrelated in both sets of twins. We also assume equal means and variances across twin number and zygosity. These standard assumptions for univariate models also apply to the multivariate common pathway models described here, including when phenotypic correlations are decomposed into their genetic (ra), shared environmental (rc), and non-shared environmental components (re). Genetic and environmental correlations were not estimated directly but were computed from statistically-equivalent Cholesky decompositions.

Results

Descriptive Statistics

Descriptive statistics for the self-reported measures at age 56 and 62 are displayed in Table 2. Phenotypic correlations among all variables are displayed in the supplement (Table S3).

Table 2.

Descriptive Statistics for Continuous Dependent Measures at Mean Ages 56 and 62

N M SD Range Skewness Kurtosis Reliability rMZ rDZ
Internalizing (Age 56)
 CES-Depression 1283 8.40 8.22 0, 52 1.78 3.89 .89 .42 .27
 SF-36 Anxiety Item 1284 1.85 1.09 1, 6 1.59 2.51 - .27 .14
 MPQ – Stress Reaction 1280 4.57 3.80 0, 14 .68 −.52 .89 .52 .27
Internalizing (Age 62)
 CES-Depression 1195 7.41 8.11 0, 53 1.93 4.47 .89 .32 .21
 State-Trait Anxiety Inventory 1196 31.60 9.80 20, 76 1.10 0.99 .93 .42 .20
 PTSD Checklist 1198 26.35 10.65 17, 84 2.03 4.94 .94 .37 .22
Externalizing (Age 56)
 MPQ - Aggression 1281 3.25 2.80 0, 17 1.23 1.86 .83 .42 .18
 MPQ - Constraint 1279 46.08 7.56 18, 63 −.63 .34 .75 .42 .16
Open in a new tab

Note: Reliability was computed with Cronbach’s alpha. All questionnaire measures (except MPQ-Aggression and MPQ-Constraint) were log-transformed in all analyses to improve their distributional characteristics (range in skewness: −.61 to 1.05, range in kurtosis: −.72 to .72). The final two columns display the cross-twin correlations for monozygotic (MZ) and dizygotic (DZ) twins. CES = Center for Epidemiologic Studies; SF-36 = Short-Form Health Survey; MPQ = Multidimensional Personality Questionnaire; PTSD = Post-traumatic stress disorder.

Internalizing and Externalizing at Age 41

The genetic and environmental model of internalizing and externalizing psychopathology symptoms at age 41 is displayed in Figure 1 (see figure captions for model fit). The estimated phenotypic correlation between the internalizing and externalizing factors was r=.65.

Figure 1:

Figure 1:

Open in a new tab

Model of the genetic (A), shared environmental (C), and non-shared environmental influences (E) on internalizing and externalizing psychopathology at age 41. Individual measures (in rectangles) were binned symptom counts based on dependence and abuse criteria for that disorder. Variance explained by latent variables (ovals) can be computed by squaring these standardized factor loadings whereas genetic correlations (rg) are computed by dividing the genetic covariance between internalizing and externalizing psychopathology (.75*.47) by the square root of the product of their genetic variances (√(.752*(.472 + .482)). Significant factor loadings and correlations are displayed in bold with black lines (p < .05) and 95% standard error-based confidence intervals are displayed in brackets. 95% confidence intervals for residual ACEs are displayed in the supplement (Table S4). This model fit the data well: χ2(234)=258.50, p=.130, RMSEA=.017, CFI=.993. ASPD = antisocial personality disorder, CAN = Cannabis use disorder, ALC = Alcohol use disorder, TOB = Tobacco use disorder, GAD = Generalized anxiety disorder, MDD = Major depressive disorder, PTSD = Post-traumatic stress disorder.

Variance in the internalizing factor was relatively evenly split between genetic influences, a2=.45, and non-shared environmental influences, e2=.55, with no evidence for shared environmental influences, c2 = .00. About half the variance in the externalizing psychopathology factor was also explained by genetic influences, a2=.56, which were strongly correlated with those genetic influences on internalizing psychopathology, ra=.70, χ2(1)=46.76, p<.001, 95% CI [.12,1.0], explaining 54% of their phenotypic correlation. Non-shared environmental influences explained an additional 28% of the variance in the externalizing psychopathology factor and were also correlated with the environmental influences on internalizing psychopathology, re=.77, χ2(1)=44.87, p<.001, 95% CI [.55,.99]. Shared environmental influences explained the remaining 18% of the variance in externalizing psychopathology, but were nonsignificant, c2=.17, χ2(1)=1.06, p=.303, likely due to low power to detect these influences even in large samples (Martin et al., 1978).

There was little evidence for significant genetic influences on the symptom counts for most individual disorders after accounting for those on the latent factors (range in unique a2=.00 to .16; see supplement Table S4 for 95% CIs). However, significant residual genetic influences were identified for tobacco use (a2=.43) and alcohol use disorders (a2=.04), both χ2(1)>17.13, ps<.001; for other disorders, all χ2(1)<1.49, ps>.222. Residual non-shared environmental influences were significant for all disorders (range e2=.15 to .56). All residual shared environmental influences were nonsignificant, and those influences that were estimated at zero (i.e., GAD, MDD, tobacco use disorder) were fixed to zero in all subsequent models, including the nonsignificant shared environmental influences on the internalizing psychopathology factor, to aid in model convergence. We also included two residual paths in this model (and all subsequent models): a path from the non-shared environmental influences on GAD to those for MDD, χ2(1)=10.15, p=.001, and a path from the genetic influences on alcohol use disorder to tobacco use disorder, χ2(1)=17.13, p<.001.4

Prediction of Internalizing Symptoms at Ages 56 and 62 from Psychopathology at Age 41

Next, we examined the genetic and environmental overlap between internalizing psychopathology at age 41 and later internalizing symptoms. Figure 2a displays the associations between internalizing psychopathology at age 41 and internalizing symptoms at age 56. Figure 2b displays the associations between internalizing psychopathology at age 41 and internalizing symptoms at age 62. Figure 2c displays the associations between internalizing symptoms at age 56 and age 62. We present separate bivariate models for ease of interpretation, but the results were nearly identical in a trivariate model that simultaneously examined all three time-points (supplemental Figure S1).

Figure 2:

Figure 2:

Open in a new tab

Model of the genetic (A), shared environmental (C), and non-shared environmental (E) overlap between internalizing psychopathology at age 42 and self-reported internalizing symptoms at age 56 (Figure 2a) or age 62 (Figure 2b). Figure 3c displays the overlap between self-reported internalizing symptoms at age 56 and 62. Variance explained by latent variables (ovals) can be computed by squaring these standardized factor loadings. Measured variables are displayed in rectangles. Significant factor loadings are displayed in bold with black lines (p < .05) and 95% standard error-based confidence intervals are displayed in brackets. All models fit the data well: χ2(161)=161.83, p=.467, RMSEA=.004, CFI=.999 (Figure 2a); χ2(161)=160.15, p=.504, RMSEA=.000 CFI=1.00 (Figure 2b); χ2(142)=220.52, p<.001, RMSEA=.038, CFI=.982 (Figure 2c). GAD = Generalized anxiety disorder, MDD = Major depressive disorder, PTSD = Post-traumatic stress disorder, CES-D = Center for Epidemiologic Studies – Depression, SF36 = Short-Form Health Survey, MPQ = Multidimensional Personality Questionnaire, STAI = State-Trait Anxiety Inventory, PCL = PTSD Checklist.

Genetic influences on internalizing psychopathology at age 41 (AInt) explained 22.1% of the total variance in internalizing symptoms at age 56, χ2(1)=26.91, p<.001. This resulted in a genetic correlation of ra=.66, 95% CI [.33,.98] that accounted for 61% of the total phenotypic correlation (r=.51). Non-shared environmental influences on internalizing psychopathology at age 41 (EInt) explained an additional 6.8% of the variance in symptoms at age 56, re=.43, χ2(1)=19.56, p<.001, 95% CI [.25,.60]. The remaining variance in internalizing symptoms at age 56 was explained by unique genetic influences (A56), a2=.29, χ2(1)=1.73, p=.189, unique non-shared environmental influences (E56), e2=.31, χ2(1)=24.40, p<.001, and unique shared environmental influences (C56), c2=.10, χ2(1)=0.40, p=.527. Although the unique genetic and shared environmental influences were nonsignificant, both could not be simultaneously removed without a significant reduction in model fit, χ2(2)=8.96 p=.011.

Similar results were observed at age 62 (Figure 2b). Genetic influences on internalizing psychopathology at age 41 (AInt) explained 21.2% of variance the internalizing symptoms latent factor at age 62, resulting in a genetic correlation of ra=.69, χ2(1)=30.33, p<.001, 95% CI [.34,1.0] that explained 70% of their phenotypic correlation (r=.43). Non-shared environmental influences on internalizing psychopathology at age 41 (EInt) explained an additional 2.9% of the variance in internalizing symptoms at age 62, re=.24, χ2(1)=8.20, p=.004, 95% CI [.08,.40]. The remaining variance was explained by non-shared environmental influences specific to age 62 (E62), e2=.49, χ2(1)=35.15, p<.001, and nonsignificant genetic influences (A62), a2=.23, χ2(1)=1.68, p=.195, and shared environmental influences specific to age 62 (C62), c2=.03, χ2(1)=0.05, p=.830.

In both models, we examined evidence for stability of residual genetic or environmental influences on individual disorders to later self-reported symptoms (e.g., between MDD at age 41 and CES-D at age 56 in Figure 2a). Based on these analyses, we included a path from the non-shared environmental influences on PTSD symptoms (age 41) to PTSD symptoms at age 62 (Figure 2b), χ2(1)=22.93, p<.001. All other paths were nonsignificant, χ2(1)<1.24, p>.265, and were excluded from the final models. Additional analyses displayed in the supplement (Figure S2) found no evidence that the genetic or environmental influences on externalizing psychopathology (at age 41) were significantly associated with internalizing symptoms at age 56 or age 62 after accounting for its covariance with the internalizing psychopathology at age 41.

As shown in Figure 2c, despite some measurement differences (i.e., different measures of anxiety, PTSD symptoms only at age 62), the genetic influences on the internalizing symptoms factors were perfectly correlated between ages 56 and 62, ra=1.0, χ2(1)=16.37, p<.001, 95% CI [.12,1.0], explaining 59% of the phenotypic correlation (r=.78). There was also a moderate non-shared environmental correlation, re=.54, χ2(1)=99.18, p<.001, 95% CI [.45,.63], but most of the non-shared environmental variance at age 62 was unique (explaining 37% of its total variance).

Prediction of Externalizing Symptoms/Behaviors at Age 56 from Psychopathology at Age 41

Figure 3 displays a similar model in which externalizing psychopathology at age 41 predicts externalizing symptoms/behaviors at age 56. Although factor loadings for externalizing symptoms/behaviors at age 56 were somewhat low, genetic influences on externalizing psychopathology at age 41 accounted for 72.3% of the variance in externalizing symptoms/behaviors at age 56. This resulted in a perfect genetic correlation, ra=1.0, χ2(1)=20.17, p<.001, 95% CI [.37,1.0], that explained 98% of their phenotypic correlation (r=.67). There were also some unique non-shared environmental influences on the externalizing symptoms/behaviors factor at age 62, explaining 25.0% of its variance, χ2(1)=7.14, p<.008.

The model displayed in Figure 3 also included residual genetic and non-shared environmental associations between tobacco use disorder symptoms (age 41) and tobacco use at age 56, explaining 50.4% and 4.0% of the variance in tobacco use at age 56, respectively, both χ2(1)>6.27, ps<.012. Similar paths were tested between alcohol use disorder (age 41) and alcohol use (age 56), but they were nonsignificant, both χ2(1)<3.73 p>.053. Finally, unlike the models of the internalizing factors, additional analyses suggested that the new nonshared environmental influences on externalizing symptom/behaviors at age 56 were explained by environmental influences internalizing psychopathology at age 56, e2 = .25, χ2(1)=4.53, p=.033, 95% CI [.10, .49], even after controlling for externalizing psychopathology at age 41 (supplemental Figure S3). Thus, most – if not all – of the variance in externalizing symptoms/behaviors at age 62 were explained by earlier psychopathology symptoms.

Discussion

We observed considerable genetic and environmental correlations between internalizing and externalizing psychopathology symptoms in these middle-aged adults not selected for psychopathology diagnoses. Both latent factors were also moderately to strongly correlated with self-reported measures of related symptoms and behaviors 15 to 20 years later. Internalizing and externalizing factors have been useful in understanding psychopathology’s associations with cognition and personality (Krueger et al., 2002, Hink et al., 2013, Tackett et al., 2013, Caspi et al., 2014), but studies have most often focused on children and adolescents, or adult samples with heterogeneous age ranges. Our results demonstrate that these latent factors show strong stability across midlife, even though the symptom/behavior measures at age 56 and 62 differed substantially from the diagnostic interviews in earlier middle age.

Although internalizing and externalizing psychopathology factors were explained roughly equally by genetic and non-shared environmental influences, genetic influences accounted for the majority of their phenotypic stability across the 15- to 20-year interval. Previous work has focused on stability of genetic influences on individual disorders (Eley et al., 2003, Bergen et al., 2007, Bornovalova et al., 2009, Petkus et al., 2016), but this is the first evidence, to our knowledge, that much of this genetic stability can be attributable to higher-order internalizing and externalizing psychopathology variance (at least, to the extent that the self-reported symptom/behavioral measures and symptom counts from diagnostic interviews tap similar internalizing/externalizing variance). In contrast, there was little evidence that genetic influences specific to individual disorders predicted later symptoms/behaviors (except for tobacco use disorder).

These results are relevant to ongoing genome-wide association tests (GWAS) of psychopathology. GWAS have also demonstrated high genetic overlap among psychiatric disorders (Bulik-Sullivan et al., 2015, Lo et al., 2016, Duncan et al., 2018). Adequately powered GWAS, however, require samples in the tens or hundreds of thousands (Visscher et al., 2017). Consequently, studies include individuals from across the lifespan. Although these results suggest that there is considerable genetic stability across middle age, to the extent that this stability is not perfect (e.g., for internalizing), this will hurt power to detect genes that are not currently influencing some individuals in the sample (e.g., those who are younger than the majority of the sample). Beyond gene identification, development of polygenic risk scores for internalizing and externalizing psychopathology may be useful in identifying individuals at risk for problems throughout the lifespan.

Finally, non-shared environmental influences also exhibited some stability across this 15- to 20-year span. Just as the genetic influences reflect the combined effects of many independent genetic influences, these non-shared environmental influences likely reflect a multitude of factors, each of which only accounts for a small fraction of the variance (Plomin et al., 2001, Friedman et al., 2016). Nevertheless, it will be useful to identify those that contribute to this persistence of psychopathology symptoms over time, as they may be crucial in future treatment and prevention efforts.

Strengths and Limitations

First, the sample comprised only men, so these findings may not generalize to women. Second, this community-based sample was not selected for psychopathology symptoms, and associations may differ in samples with higher rates of symptoms. Third, power to detect shared environmental influences was low, even in this large sample (Martin et al., 1978). However, significant shared environmental influences have been observed in some (Gjone and Stevenson, 1997, Burt et al., 2005), but not all previous work (Krueger et al., 2002, Gustavson et al., 2017).

Fourth, there were limitations to the available measures because VETSA was not designed as a study of psychopathology. For internalizing symptoms, the SF-36 anxiety measure at age 56 was based on only a single questionnaire item and PTSD symptoms were only assessed at age 62. Nevertheless, there were high factor loadings in both cases and the overall results were highly consistent at age 56 and 62 (e.g., as evidenced by the perfect genetic correlation between these internalizing symptom factors in Figure 2c). The factor loadings on the externalizing symptoms/behaviors factor were small, and it was only possible to examine externalizing symptoms/behaviors at age 56 due to the lack of inclusion of the MPQ at the age 62 assessment. The low factor loadings may reflect the cruder dependent measures (i.e., a dichotomous variable for smoking, only one question about alcohol use in the past two weeks) and the lack of measures of illicit substance use (e.g., cannabis use). Despite these limitations, most of the variance in externalizing symptoms/behaviors at age 56 was explained by externalizing psychopathology at age 41, and the remaining nonshared environmental influences were explained by internalizing psychopathology at age 41.

Finally, and perhaps most importantly, the internalizing and externalizing factors at age 56 and 62 were based on responses to self-reported questionnaires about symptoms and substance use rather than diagnostic interviews. Although this severely limits our ability to draw strong conclusions about the stability of internalizing and externalizing psychopathology over time, we nevertheless observed strong genetic associations between diagnostic interviews and symptoms measures 15 to 20 years later. The genetic and environmental correlations may have been even larger if assessments were identical. However, the fact that we observed strong associations despite measurement differences suggests that these psychopathology factors are highly stable across midlife.

Summary and Conclusions

Internalizing and externalizing psychopathology factors in early middle age are explained roughly equally by genetic and non-shared environmental influences. Both factors demonstrate substantial ability to predict self-reported internalizing and externalizing-related problems 15 to 20 years later, with substantial stability of both genetic and non-shared environmental influences. In fact, there was little evidence for new genetic influences on psychopathology symptoms. These findings highlight the persistence of psychopathology-related symptoms and behaviors across adulthood and middle-age.

Supplementary Material

Supplement

Acknowledgements

The content of this manuscript is solely the responsibility of the authors and does not necessarily represent the official views of the NIA/NIH, or the VA. The U.S. Department of Veterans Affairs has provided financial support for the development and maintenance of the Vietnam Era Twin (VET) Registry. Numerous organizations have provided invaluable assistance in the conduct of the VET Registry, including: Department of Defense; National Personnel Records Center, National Archives and Records Administration; Internal Revenue Service; National Opinion Research Center; National Research Council, National Academy of Sciences; the Institute for Survey Research, Temple University. Most importantly, the authors gratefully acknowledge the continued cooperation and participation of the members of the VET Registry and their families as well as the contributions of many staff members and students.

Financial Support

This research was supported by Grants R01’s AG050595, AG022381, AG054002, AG059329, K08 AG047903 and P01 AG055367 funded by the National Institutes of Health/National Institute on Aging.

Footnotes

Conflicts of Interest

None

Ethical Standards

The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008.

1

Although this is a relatively large population sample, the prevalence of psychopathology diagnoses was low, so we were unable to analyze the data at the level of diagnoses, especially using genetically-informative models (that split by zygosity and twin 1 vs. twin 2).

2

Researchers have identified overlapping genetic influences between PTSD and externalizing psychopathology (Xian et al., 2000, Logue et al., 2013), and some have proposed that there are internalizing and externalizing subtypes of PTSD (Miller et al., 2004). In the current study, we examined the possibility that PTSD would be an indicator of both internalizing and externalizing psychopathology. This alternative model found a small, nonsignificant factor loading of −.02 from PTSD to externalizing, χ2(1)=0.20, p=.887, but was otherwise nearly identical to Figure 1. Thus, we included PTSD as an indicator of internalizing psychopathology only, consistent with recent theoretical models (e.g., Kotov et al., 2017).

3

The anxiety-related SF-36 item was also available at age 62 but we elected to use the full STAI scale because it was based on many more items. Post-hoc analyses indicated that the primary conclusions of the study remained the same if the SF-36 item were used instead of the STAI.

4

These associations were identified based on modification indices from a preliminary phenotypic model, although the residual association between GAD and MDD was expected a priori based on evidence that anxiety and depressive disorders are more correlated with one another than either are with PTSD (e.g., Ginzburg et al., 2010). The residual association between tobacco and alcohol use disorders was not anticipated, but excluding this correlation did not impact the other results. When including these associations in the genetic model (Figure 1), both genetic and environmental paths were initially included, but remained in the final model only when significant.

Contributor Information

Daniel E. Gustavson, Department of Psychiatry, Center for Behavioral Genetics of Aging, University of California, San Diego

Carol E. Franz, Department of Psychiatry, Center for Behavioral Genetics of Aging, University of California, San Diego

Matthew S. Panizzon, Department of Psychiatry, Center for Behavioral Genetics of Aging, University of California

Michael J. Lyons, Department of Psychological and Brain Sciences, Boston University

William S. Kremen, Department of Psychiatry, Center for Behavioral Genetics of Aging, University of California, San Diego and Veterans Affairs San Diego Healthcare System

References

  1. American Psychiatric Association (1987). Diagnostic and statistical manual of mental disorders. 3rd ed, revised American Psychiatric Association: Washington, DC. [Google Scholar]
  2. Bergen SE, Gardner CO & Kendler KS (2007). Age-related changes in heritability of behavioral phenotypes over adolescence and young adulthood: A meta-analysis. Twin Research and Human Genetics 10, 423–33. [DOI] [PubMed] [Google Scholar]
  3. Blonigen DM, Hicks BM, Krueger RF, Patrick CJ & Iacono WG (2005). Psychopathic personality traits: heritability and genetic overlap with internalizing and externalizing psychopathology. Psychological Medicine 35, 637–48. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Bornovalova MA, Hicks BM, Iacono WG & McGue M (2009). Stability, change, and heritability of borderline personality disorder traits from adolescence to adulthood: a longitudinal twin study. Development and Psychopathology 21, 1335–53. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Bulik-Sullivan B, Finucane HK, Anttila V, Gusev A, Day FR, Loh PR, ReproGen, C, Psychiatric Genomics, C, Genetic Consortium for Anorexia Nervosa of the Wellcome Trust Case Control, C, Duncan L, Perry JR, Patterson N, Robinson EB, Daly MJ, Price AL & Neale BM (2015). An atlas of genetic correlations across human diseases and traits. Nature Genetics 47, 1236–41. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Burt SA, McGue M, Krueger RF & Iacono WG (2005). Sources of covariation among the child-externalizing disorders: informant effects and the shared environment. Psychological Medicine 35, 1133–1144. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Caspi A, Houts RM, Belsky DW, Goldman-Mellor SJ, Harrington H, Israel S, Meier MH, Ramrakha S, Shalev I, Poulton R & Moffitt TE (2014). The p factor: One general psychopathology factor in the structure of psychiatric disorders? Clinical Psychological Science 2, 119–137. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Chantarujikapong SI, Scherrer JF, Xian H, Eisen SA, Lyons MJ, Goldberg J, Tsuang M & True WR (2001). A twin study of generalized anxiety disorder symptoms, panic disorder symptoms and post-traumatic stress disorder in men. Psychiatry Research 103, 133–45. [DOI] [PubMed] [Google Scholar]
  9. Derks EM, Dolan CV & Boomsma DI (2004). Effects of censoring on parameter estimates and power in genetic modeling. Twin Research 7, 659–69. [DOI] [PubMed] [Google Scholar]
  10. Duncan LE, Ratanatharathorn A, Aiello AE, Almli LM, Amstadter AB, Ashley-Koch AE, Baker DG, Beckham JC, Bierut LJ, Bisson J, Bradley B, Chen CY, Dalvie S, Farrer LA, Galea S, Garrett ME, Gelernter JE, Guffanti G, Hauser MA, Johnson EO, Kessler RC, Kimbrel NA, King A, Koen N, Kranzler HR, Logue MW, Maihofer AX, Martin AR, Miller MW, Morey RA, Nugent NR, Rice JP, Ripke S, Roberts AL, Saccone NL, Smoller JW, Stein DJ, Stein MB, Sumner JA, Uddin M, Ursano RJ, Wildman DE, Yehuda R, Zhao H, Daly MJ, Liberzon I, Ressler KJ, Nievergelt CM & Koenen KC (2018). Largest GWAS of PTSD (N=20 070) yields genetic overlap with schizophrenia and sex differences in heritability. Molecular Psychiatry 23, 666–673. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Eaton NR, Krueger RF, Keyes KM, Skodol AE, Markon KE, Grant BF & Hasin DS (2011). Borderline personality disorder co-morbidity: relationship to the internalizing-externalizing structure of common mental disorders. Psychological Medicine 41, 1041–50. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Eley TC, Lichtenstein P & Moffitt TE (2003). A longitudinal behavioral genetic analysis of the etiology of aggressive and nonaggressive antisocial behavior. Development and Psychopathology 15, 383–402. [DOI] [PubMed] [Google Scholar]
  13. Friedman NP, Miyake A, Altamirano LJ, Corley RP, Young SE, Rhea SA & Hewitt JK (2016). Stability and change in executive function abilities from late adolescence to early adulthood: A longitudinal twin study. Developmental Psychology 52, 326–40. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Fu Q, Heath AC, Bucholz KK, Lyons MJ, Tsuang MT, True WR & Eisen SA (2007). Common genetic risk of major depression and nicotine dependence: the contribution of antisocial traits in a United States veteran male twin cohort. Twin Research and Human Genetics 10, 470–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Ginzburg K, Ein-Dor T & Solomon Z (2010). Comorbidity of posttraumatic stress disorder, anxiety and depression: A 20-year longitudinal study of war veterans. Journal of Affective Disorders 123, 249–57. [DOI] [PubMed] [Google Scholar]
  16. Gjone H & Stevenson J (1997). The association between internalizing and externalizing behavior in childhood and early adolescence: Genetic or environmental common influences? Journal of Abnormal Child Psychology 25, 277–286. [DOI] [PubMed] [Google Scholar]
  17. Gustavson DE, du Pont A, Hatoum AS, Rhee SH, Kremen WS, Hewitt JK & Friedman NP (2017). Genetic and environmental associations between procrastination and internalizing/externalizing psychopathology. Clinical Psychological Science 5, 798–815. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Hink LK, Rhee SH, Corley RP, Cosgrove VE, Hewitt JK, Schulz-Heik RJ, Lahey BB & Waldman ID (2013). Personality dimensions as common and broadband-specific features for internalizing and externalizing disorders. Journal of Abnormal Child Psychology 41, 939–957. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Hollingshead AB (1975). Four factor index of social status. Yale University: New Haven, CT. [Google Scholar]
  20. Hu LT & Bentler PM (1998). Fit indices in covariance structure modeling: Sensitivity to underparameterized model misspecification. Psychological Methods 3, 424–453. [Google Scholar]
  21. Kendler KS & Myers J (2014). The boundaries of the internalizing and externalizing genetic spectra in men and women. Psychological Medicine 44, 647–55. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Kendler KS, Prescott CA, Myers J & Neale MC (2003). The structure of genetic and environmental risk factors for common psychiatric and substance use disorders in men and women. Archives of General Psychiatry 60, 929–937. [DOI] [PubMed] [Google Scholar]
  23. Kilpatrick DG, Resnick HS, Milanak ME, Miller MW, Keyes KM & Friedman MJ (2013). National estimates of exposure to traumatic events and PTSD prevalence using DSM-IV and DSM-5 criteria. Journal of Traumatic Stress 26, 537–47. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. King SM, Iacono WG & McGue M (2004). Childhood externalizing and internalizing psychopathology in the prediction of early substance use. Addiction 99, 1548–59. [DOI] [PubMed] [Google Scholar]
  25. Koenen KC, Fu QJ, Ertel K, Lyons MJ, Eisen SA, True WR, Goldberg J & Tsuang MT (2008). Common genetic liability to major depression and posttraumatic stress disorder in men. Journal of Affective Disorders 105, 109–15. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Kotov R, Krueger RF, Watson D, Achenbach TM, Althoff RR, Bagby M & Zimmerman M (2017). The hierarchical Taxonomy of psychopathology (HiTOP): A dimensional alternative to traditional nosologies. Journal of Abnormal Psychology. [DOI] [PubMed] [Google Scholar]
  27. Kremen WS, Franz CE & Lyons MJ (2013a). VETSA: The Vietnam Era Twin Study of Aging. Twin Research and Human Genetics 16, 399–402. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Kremen WS, Franz CE & Lyons MJ (2013b). VETSA: the Vietnam Era Twin Study of Aging - ADDENDUM. Twin Research and Human Genetics 16, 403. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Kremen WS, Thompson-Brenner H, Leung YM, Grant MD, Franz CE, Eisen SA, Jacobson KC, Boake C & Lyons MJ (2006). Genes, environment, and time: The Vietnam Era Twin Study of Aging (VETSA). Twin Research and Human Genetics 9, 1009–22. [DOI] [PubMed] [Google Scholar]
  30. Krueger RF (1999). The structure of common mental disorders. Archives of General Psychiatry 56, 921–6. [DOI] [PubMed] [Google Scholar]
  31. Krueger RF, Caspi A, Moffitt TE, Silva PA & McGee R (1996). Personality traits are differentially linked to mental disorders: a multitrait-multidiagnosis study of an adolescent birth cohort. Journal of Abnormal Psychology 105, 299–312. [DOI] [PubMed] [Google Scholar]
  32. Krueger RF, Hicks BM, Patrick CJ, Carlson SR, Iacono WG & McGue M (2002). Etiologic connections among substance dependence, antisocial behavior, and personality: Modeling the externalizing spectrum. Journal of Abnormal Psychology 111, 411–424. [PubMed] [Google Scholar]
  33. Lilienfeld SO (2003). Comorbidity between and within childhood externalizing and internalizing disorders: Reflections and directions. Journal of Abnormal Child Psychology 31, 285–291. [DOI] [PubMed] [Google Scholar]
  34. Lo M-T, Hinds DA, Tung JY, Franz C, Fan C-C, Wang Y, Smeland OB, Schork A, Holland D & Kauppi K (2016). Genome-wide analyses for personality traits identify six genomic loci and show correlations with psychiatric disorders. Nature genetics. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Logue MW, Solovieff N, Leussis MP, Wolf EJ, Melista E, Baldwin C, Koenen KC, Petryshen TL & Miller MW (2013). The ankyrin-3 gene is associated with posttraumatic stress disorder and externalizing comorbidity. Psychoneuroendocrinology 38, 2249–2257. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Martin J, Taylor MJ & Lichtenstein P (2018). Assessing the evidence for shared genetic risks across psychiatric disorders and traits. Psychological Medicine 48, 1759–1774. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Martin NG, Eaves LJ, Kearsey MJ & Davies P (1978). The power of the classical twin study. Heredity (Edinb) 40, 97–116. [DOI] [PubMed] [Google Scholar]
  38. Miller MW, Kaloupek DG, Dillon AL & Keane TM (2004). Externalizing and internalizing subtypes of combat-related PTSD: A replication and extension using the PSY-5 scales. Journal of Abnormal Psychology 113, 636–45. [DOI] [PubMed] [Google Scholar]
  39. Muthén LK & Muthén BO (2010–2012). Mplus Version 7 user’s guide. Muthén & Muthén: Los Angeles, CA. [Google Scholar]
  40. Petkus AJ, Gatz M, Reynolds CA, Kremen WS & Wetherell JL (2016). Stability of genetic and environmental contributions to anxiety symptoms in older adulthood. Behavior Genetics 46, 492–505. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Plomin R, Asbury K & Dunn J (2001). Why are children in the same family so different? Nonshared environment a decade later. The Canadian Journal of Psychiatry 46, 225–33. [DOI] [PubMed] [Google Scholar]
  42. Radloff LS (1977). The CES-D scale: A self-report depression scale for research in the general population. Applied Psychological Measurement 1, 385–401. [Google Scholar]
  43. Robins L, Helzer J, Cottler L & Goldring E (1988). National Institute of Mental Health Diagnostic Interview Schedule: Version III Revised (DIS-III-R). Department of Psychiatry, Washington University Medical School: St Louis, MO. [Google Scholar]
  44. Schoenborn CA & Heyman KM (2009). Health characteristics of adults aged 55 years and over: United States, 2004–2007. National Health Statistics Report 16, 1–31. [PubMed] [Google Scholar]
  45. Spielberger CD (1983). Manual for the State-Trait Anxiety Inventory STAI (form Y)(“self-evaluation questionnaire”).
  46. Tackett JL, Lahey BB, van Hulle C, Waldman I, Krueger RF & Rathouz PJ (2013). Common genetic influences on negative emotionality and a general psychopathology factor in childhood and adolescence. Journal of Abnormal Psychology 122, 1142–53. [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Tellegen A (1985). Structures of mood and personality and their relevance to assessing anxiety, with an emphasis on self-report.
  48. True WR, Heath AC, Scherrer JF, Xian H, Lin N, Eisen SA, Lyons MJ, Goldberg J & Tsuang MT (1999a). Interrelationship of genetic and environmental influences on conduct disorder and alcohol and marijuana dependence symptoms. American Journal of Medical Genetics 88, 391–7. [DOI] [PubMed] [Google Scholar]
  49. True WR, Xian H, Scherrer JF, Madden PA, Bucholz KK, Heath AC, Eisen SA, Lyons MJ, Goldberg J & Tsuang M (1999b). Common genetic vulnerability for nicotine and alcohol dependence in men. Archives of General Psychiatry 56, 655–61. [DOI] [PubMed] [Google Scholar]
  50. Tsuang MT, Bar JL, Harley RM & Lyons MJ (2001). The Harvard Twin Study of Substance Abuse: What we have learned. Harvard Review of Psychiatry 9, 267–279. [PubMed] [Google Scholar]
  51. Visscher PM, Wray NR, Zhang Q, Sklar P, McCarthy MI, Brown MA & Yang J (2017). 10 years of GWAS discovery: Biology, function, and translation. The American Journal of Human Genetics 101, 5–22. [DOI] [PMC free article] [PubMed] [Google Scholar]
  52. Ware JE Jr & Sherbourne CD (1992). The MOS 36-item short-form health survey (SF-36): I. Conceptual framework and item selection. Medical Care 30, 473–483. [PubMed] [Google Scholar]
  53. Weathers FW, Litz BT, Herman DS, Huska JA & Keane TM (1993). The PTSD Checklist (PCL): Reliability, validity, and diagnostic utility. In The 9th annual meeting of the International Society for Traumatic Stress Studies. [Google Scholar]
  54. Weiss B, Susser K & Catron T (1998). Common and specific features of childhood psychopathology. J Abnorm Psychol 107, 118–27. [DOI] [PubMed] [Google Scholar]
  55. Whiteside SP & Lynam DR (2003). Understanding the role of impulsivity and externalizing psychopathology in alcohol abuse: Application of the UPPS impulsive behavior scale. Experimental Clinical Psychopharmacology 11, 210–7. [DOI] [PubMed] [Google Scholar]
  56. Xian H, Chantarujikapong SI, Scherrer JF, Eisen SA, Lyons MJ, Goldberg J, Tsuang M & True WR (2000). Genetic and environmental influences on posttraumatic stress disorder, alcohol and drug dependence in twin pairs. Drug and Alcohol Dependence 61, 95–102. [DOI] [PubMed] [Google Scholar]
  57. Young SE, Friedman NP, Miyake A, Willcutt EG, Corley RP, Haberstick BC & Hewitt JK (2009). Behavioral disinhibition: Liability for externalizing spectrum disorders and its genetic and environmental relation to response inhibition across adolescence. Journal of Abnormal Psychology 118, 117–30. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplement
NIHMS1042518-supplement-Supplement.docx (701.8KB, docx)

RESOURCES