Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2014 May 1.
Published in final edited form as: J Abnorm Child Psychol. 2013 May;41(4):667–677. doi: 10.1007/s10802-012-9701-0

Prospective Associations of Internalizing and Externalizing Problems and Their Co-Occurrence with Early Adolescent Substance Use

Craig R Colder 1,2,✉, Matthew Scalco 3, Elisa M Trucco 4, Jennifer P Read 5, Liliana J Lengua 6, William F Wieczorek 7, Larry W Hawk Jr 8
PMCID: PMC3640685  NIHMSID: NIHMS429712  PMID: 23242624

Abstract

The literature is equivocal regarding the role of internalizing problems in the etiology of adolescent substance use. In this study, we examined the association of internalizing and externalizing behavior problems and their co-occurrence with early adolescent substance use to help clarify whether internalizing problems operate as a risk or protective factor. A large community sample (N=387; mean age at the first assessment 12 years old; 83 % White/non-Hispanic) was assessed annually for 3 years. Externalizing problem behavior in the absence of internalizing problems showed the strongest prospective association with alcohol, cigarette, and marijuana use. A weaker, albeit statistically significant prospective positive association was found between co-occurring internalizing and externalizing behavior problems and substance use. Internalizing problems in the absence of externalizing problems protected adolescents against cigarette and marijuana use. Clarifying the role of internalizing problems in the etiology of adolescent substance use can inform the development of early intervention and prevention efforts. Our results highlight the importance of further considering the co-occurrence of internalizing and externalizing behavior problems in developmental pathways to substance use.

Keywords: Adolescence, Substance use, Problem behavior, Internalizing, Externalizing

Overview

Developmental models of adolescent substance use posit that etiological processes are evident before initiation (Dodge et al. 2009; Masten et al. 2008). Behavior problems represent pivotal risk factors in these models that often precede onset and escalation of adolescent substance use (Hawkins et al. 1992; Windle 1990; King et al. 2004) and as such may serve as early markers of risk. Both internalizing problems, which reflect a child’s internal distress (e.g., depression, anxiety), and externalizing problems, which bring a child into conflict with others (e.g., rule breaking, aggression), are included under the broad rubric of behavior problems. Much of the research examining adolescent substance use has focused on externalizing problems, and those studies examining internalizing problem have yielded equivocal results. Yet, externalizing and internalizing problems often co-occur in adolescents (Achenbach and Rescorla 2001; Angold et al. 1999), and surprisingly little research has examined the impact of such co-occurrence on the development of adolescent substance use. This is a critical gap in the literature because pathways to substance use may be different for children with co-occurring problems compared to those with high levels of problems in only one domain (internalizing or externalizing). Our goal was to examine the prospective influence of externalizing and internalizing problems and their co-occurrence on early adolescent substance use. Substance use in early adolescents is an important outcome of interest because it is associated with problem substance use later in adolescence (Grant and Dawson 1997; Gruber et al. 1996).

Behavior Problems and Substance Use: Risk vs. Protective Mechanisms

A large literature suggests that externalizing problems in childhood predict the early onset and rapid escalation of adolescent SU (e.g., Hawkins et al. 1992; Zucker et al. 2011), and as such, are risk factors for the development of substance use. According to developmental models, externalizing problems are thought to confer risk by increasing the likelihood of alienation from conventional institutions (e.g., school), exclusion from prosocial peers, and affiliation with deviant peers who provide opportunities and reinforcement for alcohol and drug use (Dodge et al. 2009; Tarter et al. 1999).

In contrast to the literature on externalizing problems, findings regarding the association between internalizing problems and adolescent substance use have been equivocal. Some studies have shown no association (Fleming et al. 2008; Hussong et al. 1998; Kaplow et al. 2001; Stice et al. 1998), and studies that do find associations with substance outcomes offer evidence for both risk (King et al. 2004; Shivola et al. 2008) and protective (Costello et al. 1999; Kaplow et al. 2001; Fleming et al. 2008) functions of internalizing symptoms.

Theoretical mechanisms have been posited for both risk and protective pathways associated with internalizing problems. The most commonly invoked risk mechanism is self-medication, which posits substance use is motivated by a desire to reduce emotional distress and is acutely negatively reinforced by the pharmacological consequences of use (i.e., Hussong et al. 2011; McCarthy et al. 2010). With regard to internalizing problems playing a protective role is the view that social withdrawal and fear of negative consequences of engaging in illicit activities protect children from affiliating with deviant peers and from engaging in substance use (Windle 1993; Colder et al. 2010). Indeed, Fite et al. (2006a) found that high levels of internalizing problems were associated with low affiliation with deviant peers, and low levels of alcohol use. Some have posited that a self-medication pathway may not emerge until late adolescence or early adulthood (Cloninger 1987; Colder et al. 2010; Hussong et al. 2011), suggesting that internalizing problems may be primarily protective against substance use in early adolescence.

The Co-occurrence of Behavior Problems and Substance Use

The literature documents a high degree of co-occurrence between externalizing and internalizing problems (Achenbach and Rescorla 2001; Angold et al. 1999). Yet most prior substance use research has focused on either internalizing or externalizing problems alone, and has not examined either in the context of the other. Failure to take into account the co-occurrence of behavior problems can yield erroneous conclusions because the nature and implications of co-occurring problems may be different then either problem alone (Caron and Rutter 1991). As a result, our understanding of the co-occurrence of these problem domains and substance use is not well understood. This is a significant limitation of the extant literature, as whether high levels of internalizing symptoms emerge as a risk or protective factor may depend on their co-occurrence with externalizing symptoms.

The impact of this co-occurrence on substance use outcomes has been debated in the literature. Some have argued that internalizing problems can exacerbate the risk associated with externalizing problems such that self-medication motives may operate additively with deviant peer affiliations to promote substance use (Capaldi 1991, 1992; Hussong et al. 2011; Miller-Johnson et al. 1998). This implies that the co-occurrence of internalizing and externalizing problems confers greater risk for substance use compared to internalizing or externalizing problems alone. On the other hand, fear of negative consequences and social withdrawal associated with internalizing problems may diminish risk associated with the externalizing pathway. This suggests that risk for substance use may be highest for youth experiencing only externalizing problems, lowest for youth experiencing only internalizing problems, and intermediate for youth experiencing both internalizing and externalizing problems. In this study, we examine internalizing problems alone and in co-occurrence with externalizing problems in the prospective prediction of early adolescent substance use. Of interest, is whether internalizing problems either alone or in co-occurrence with externalizing problems operate as a risk or protective factor.

How best to conceptualize problem behavior has also been the topic of some discussion. Problem behavior can be represented as continuous dimensions or as categories reflecting clinical levels of problems. Prior research examining the co-occurrence of problem behavior has typically created groups (e.g., internalizing problems only, externalizing problems only, both internalizing and externalizing problems, no problems) based on arbitrary cut-offs of continuous measures of problems (e.g., Dishion 2000; Loeber et al. 1999; Miller-Johnson et al. 1998). Using arbitrary cutoffs to create categories from continuous variables has long been known to reduce power and in some cases to create spurious associations (Cohen 1983; MacCallum et al. 2002). Moreover, a recent meta-analysis compared categorical and dimensional approaches to measuring psychopathology, and findings suggested that dimensional approaches provided more reliable and valid assessment (Markon et al. 2011). Others have argued that dimensional approaches are particularly useful in community samples with low base rates of problems because they are more reflective of symptom presentation (Hankin et al. 2005; Nottleman and Jensen 1995). Despite methodological advantages of construing behavior problems along a continuum, using a dimensional approach presents a significant challenge for considering the co-occurrence of problems because it is not clear how to translate multiple dimensions of problem behavior to co-occurring dimensions without imposing a categorical scheme. An important advancement in our study was to conceptualize internalizing and externalizing problem behavior and their co-occurrence as continuous dimensions.

Keiley et al. (2003) provided a novel methodology to account for the co-occurrence of internalizing and externalizing behavior problems using a dimensional approach. They used a complex confirmatory factor model to distinguish components of variance that are shared (co-occurring) and unique (pure) among symptoms of childhood problems. The model provided an excellent fit to the data, suggesting strong support for distinguishing pure and co-occurring dimensions of internalizing and externalizing problems in a non-referred community sample. Fite et al. (2006b) also found this factor analytic approach to be useful for distinguishing pure and co-occurring dimensions of aggressive behavior. These studies suggest that this factor analytic method is well suited for testing associations between pure and co-occurring dimensions of problem behavior and adolescent substance use. We utilized this factor analytic approach in our study.

The Present Study

Our goal was to shed light on the mixed literature regarding associations between internalizing problems and early adolescent substance use by considering the co-occurrence of internalizing and externalizing problems. We used a prospective design to establish direction of effects, and an unselected community sample to represent the full spectrum of behavior problems. Rather than creating groups representing different combinations of internalizing and externalizing problems, we conceptualized behavior problems as continuous dimensions and distinguished pure internalizing, pure externalizing, and co-occurring internalizing and externalizing problem dimensions. Given that self-medication may not be germane to early adolescent substance use (Cloninger 1987; Colder et al. 2010; Hussong et al. 2011), internalizing symptoms were expected to be protective in our early adolescent sample. We hypothesized that high levels of pure externalizing and high levels of co-occurring internalizing and externalizing problems would be prospectively associated with high levels of substance use. We also hypothesized that effects of pure externalizing problems would be stronger than those of the co-occurring factor. High levels of pure internalizing problems were hypothesized to be prospectively associated with low levels of substance use.

We also considered potential gender differences in the relationship between problem behavior and substance use. The literature provides some evidence for gender differences in the relation between behavior problems and SU, however the magnitude of these differences is small. Armstrong and Costello (2002) reviewed community studies to examine the relation between childhood psychopathology and adolescent SU outcomes, and concluded that “…the similarities between the sexes have been more remarkable than the differences.” The most systematic pattern was for internalizing problems (e.g., depression and anxiety) to be more strongly associated with SU outcomes for females, but such differences were modest. Accordingly, we examined potential gender differences, but anticipated small differences.

Method

Participants

Participants were taken from a 3-wave longitudinal study of the impact of behavior problems on substance use initiation. Participants were recruited utilizing a random-digit-dial (RDD) sample of telephone numbers from ASDE Survey Sampler, Inc., specifically generated for Erie County, NY. Adolescents were eligible for the study if they were between the ages of 11 or 12 at recruitment, and did not have any disabilities that would preclude them from either understanding or completing the assessment.

The sample included 387 families (a caregiver and child). The participation rate was 52.4 %, which is well within the range of that found in population-based studies requiring extensive levels of subject involvement (Galea and Tracy 2007). The sample was roughly evenly split on sex (N=213 females, 55 %). Mean age at Time 1 was 12 years (range was 11 to 13 years). Some youth had a birthday between recruitment and the first assessment, thus the upper range was 13. The majority were White/non-Hispanic (83 %), 9 % were Black, 2 % were Hispanic, 1 % were Asian, and 5 % were other (usually of mixed ethnic background). Median family income was $70,000 and 6 % received public assistance income. The majority of parents had completed college or some graduate/professional school (58 %). The sample compares well to the general population in Erie County NY across a diverse set of characteristics, including gender, race/ethnicity, income, and receipt of public assistance. The sample had somewhat more married couple families, fewer female-headed families, and higher levels of education than in the county from whence it came.

Total attrition for the study was 7.5 % (29/387). The Time 2 assessment occurred approximately 1 year after the initial interview. Families that did not complete the Time 2 assessment (N=14) were not reliably different on any demographic variables (i.e., adolescent gender, marital status, race, level of parental education, and family income) compared to families who did complete the Time 2 assessment. The mean age for adolescents at Time 2 was 13 years (range from 11 to 14 years). The Time 3 assessment occurred approximately 1 year after the Time 2 assessment. Families that did not complete the Time 3 assessment (N=20) did not differ on any demographic or study variables at Time 1 compared to families who completed the Time 3 assessment. The mean age for adolescents at Time 3 was 14 years (range from 12 to 16 years).

Procedure

The study was described to participants as an investigation of the transition into adolescence. All interviews were conducted at our university research offices. Before the interview, the caregiver and adolescent were asked to give consent and assent, respectively. After completing the consent/assent procedures, the caregiver and adolescent were taken to separate rooms to enhance privacy. All questionnaires were read aloud and responses were entered directly into a computer. Adolescents were also asked to complete several computer tasks and asked various questions on problem behaviors, peer victimization, and physical development. However, in this study we only consider problem behavior and substance use data. Interviews took approximately 2½hours to complete, and procedures were the same at each assessment. Families were compensated $75 for their time at Time 1, $85 at the Time 2, and $125 at Time 3. The institutional review board committees of all participating institutions approved all study procedures.

Measures

Problem Behavior

Problem behavior at Time 1 was assessed using the Youth Self Report (YSR) form of the Achenbach System of Empirical Behavioral Assessment (Achenbach and Rescorla 2001). Items from the rule breaking and aggressive subscales were used to measure externalizing problems (the two substance use items on the rule breaking scale were excluded given that our outcome measure is substance use), and items from the withdrawn/depressed and anxious/depressed subscales were used to measure internalizing problems. The YSR has been used extensively and has been shown to have good reliability and validity (Achenbach and Rescorla 2001). Items on the YSR are rated on a 3- point scale (0=not true to 2=very true or often true).

T-scores in our sample were similar to those reported by Achenbach and Rescorla (2001) for their non-referred sample. For example, Achenbach and Rescorla (2001) reported average T-scores for 11–18 year old males and females to be approximately 54 for anxious/depressed, withdrawn/depressed, rule breaking, and aggressive behavior scales. In the current sample, these T-scores ranged from 51 to 55 for males and females.

As would be expected in an unselected community sample, low endorsement was evident for some behavior problem items. When analyzing such data at the item level, low endorsement items can create analytic problems (e.g., nearly perfect correlations). Accordingly, items with very high correlations were combined based on their being from the same YSR scale, and when possible, based on conceptual similarity. The following pairs of items were combined: YSR item 81 with item 82, YSR item 67 with item 101, and YSR item 72 with item 96.

Substance Use

Items taken from The National Youth Survey (NYS) were used to assess self-reported lifetime and past year alcohol, tobacco, and marijuana use (Elliott and Huizinga 1983) at each wave of assessment. However, only substance use data from the first and third assessment are considered in this report so that we could test the prospective effects of problem behavior on substance use when prevalence of use was highest in our sample. Lifetime use of alcohol without parental permission, cigarettes, and marijuana were each assessed with a dichotomous item (0=no, 1=yes) at Wave 1. Past year use of each substance was assessed with open-ended frequency (e.g., “How many times in the past year have you used alcohol/cigarettes/marijuana?”) and quantity items for alcohol and cigarettes (e.g., “On the days you drink alcohol/smoked cigarettes, about how many drinks/cigarettes do you have/?”) at Wave 3. Given the young age of the sample and expected low levels of marijuana use, quantity of marijuana use was not assessed.

Rates of substance use are generally low prior to the age of 13 (Chen and Kandel 1995; Colder et al. 2002, 2001), and as expected given the age of our sample, rates of use were low, especially at Wave 1. For example, with respect to lifetime use at Wave 1, 4.15 % used alcohol without parental permission, 2.33 % used cigarettes, and no participants reported marijuana use. Dichotomous lifetime use variables were used for alcohol and tobacco use to provide statistical control for baseline use. Three variables were used to represent Wave 3 substance use, including quantity × frequency indices for alcohol and cigarette use representing total number of drinks/cigarettes in the past year, and frequency of marijuana use in the past year. These variables were highly skewed (skews ranged from 11.95 to 15.82), suggesting the need for remedial measures (Kline 2010). Accordingly, we created ordinal variables from our Wave 3 substance use indices based on visual inspections of the distributions. Although creating categories from variables that are continuously distributed is generally not recommended (Cohen 1983), this approach can be useful when observed variables are highly skewed (MacCallum et al. 2002). Moreover, other researchers have adopted the same strategy for handling adolescent substance use variables (e.g., King et al. 2004). For alcohol use in the past year, the ordinal variable represented no use (N=256, 70.91 %), a few sips to one drink (N=59, 16.34 %), more than one drink but less than 4 drinks (N=27, 7.48 %), and more than four drinks (N=19, 5.26 %). For cigarette use, three categories were created representing no use (N=332, 91.21 %), a few puffs to five cigarettes (N=17, 4.67 %), and greater than five cigarettes (N=15, 4.12 %). For frequency of marijuana use in the past year, three categories were created to represent no use (N=331, 90.93 %), use one to five times (N=19, 5.22 %), and greater than 5 times (N=14, 3.85 %). These prevalence rates of alcohol, cigarette, and marijuana use are similar to those found for 8th graders in the Monitoring the Future Study (Johnston et al. 2011).

Data Analysis

Items on the YSR as well as the substance use outcome variables were ordinal; therefore, the weighted least square mean and variance adjusted estimator (WSLMV) was used for both the CFA and the structural equation model. All models were estimated in Mplus version 6.1 (Muthén and Muthén 1998–2007). The Root Mean Square Error of Approximation (RMSEA), Comparative Fit Index (CFI), and Tucker-Lewis Index (TLI) were used to test model fit. Because some researchers have argued that setting specific cut-offs for assessing “good” model fit cannot be generalized across all models (Hu and Bentler 1999; Marsh et al. 2004), ranges were used. For the RMSEA, values > 0.08 were indicative of poor fit, values > 0.05 and < 0.08 were indicative of acceptable fit, and values < 0.05 were indicative of excellent fit. For the CFI and TLI, values < 0.9 were considered to indicate poor fit, values > 0.9 and < 0.95 were considered to indicate acceptable fit, and values > 0.95 were considered to indicate excellent fit.

Results

Measurement Model

In contrast to conventional factor analysis of the YSR that account for the strong correlation between externalizing and internalizing problem by allowing the factors to covary (e.g., Achenbach and Rescorla 2001), our approach handles this issue by specifying a third co-occurring factor and specifying each item to load on two factors. The advantage of our approach is that it yields a continuous dimension representing co-occurring problems. Each externalizing item was specified to load on both the “pure externalizing” and the “co-occurring externalizing and internalizing” factors, and each internalizing item was specified to load on both the “pure internalizing” and “co-occurring externalizing and internalizing” factors. Factor covariances were set to 0 because all covariance between internalizing and externalizing symptoms is modeled in the co-occurring factor. Thus, the model partitions variance of each symptom into three components, including variance that is unique to either internalizing or externalizing problems (the “pure” factors), variance that is shared between internalizing and externalizing problems (the co-occurrence factor), and error variance. As such the co-occurrence of internalizing and externalizing behavior problems, as well as pure internalizing and externalizing problems are represented by continuous latent factors. Convergence problems were encountered when the model was first estimated, which may have been a function of the ordinal indicators and the model complexity. Accordingly, latent factor variances were constrained to be equal.

The fit of the model was good, x2(945)=1187.85, p<0.0001; RMSEA=0.026; CFI/TLI=0.95/0.94. Standardized factor loadings above 0.30 were considered “substantial” (Tabachnick and Fidell 2007). As shown in Table 1, all factor loadings for the co-occurring factor were statistically significant and only one loading was below 0.3. Several factor loadings on the pure externalizing and pure internalizing factors were either not statistically significant or below 0.3. However, it is important to note that none of the items had factor loadings below 0.3 on both the co-occurrence and their respective pure factor. Moreover, R2 values were generally high (see Table 1). The variances for the latent factors were statistically significant. These CFA results provide support for the proposed problem behavior latent factors. That some items did not load on their respective “pure” factor has implications for interpretation of these factors. Accordingly, it is worth noting the highest loading items on the “pure” factors to aid in interpretation of these factors. These included items 90, 21, 57, 23, 37, and 39 for the externalizing factor, and items 33, 112, 75, and 71 for the internalizing factor. Although these factors are not identical to the internalizing and externalizing scales of the YSR because some loadings did not reach statistical significance, the content of items with strong loadings suggests reasonable correspondence. As such, we would expect our “pure” factors to correspond to clinical problems as found for the YSR (Achenbach and Rescorla 2001). That is, an association would be expected between elevations on the pure internalizing factor and diagnoses of depression and anxiety, and between elevations on the pure externalizing factor and diagnoses of Oppositional Defiant Disorder (ODD) and Conduct Disorder (CD).

Table 1.

Standardized factor loadings and R2 for measurement model

YSR item number Extern Intern Co-Occur R2
3 0.25 0.54 0.35
16 0.19 0.66 0.47
19 0.33 0.44 0.30
20 0.30 0.52 0.36
21 0.70 0.40 0.65
22 0.27 0.55 0.37
23 0.56 0.47 0.53
37 0.55 0.52 0.58
57 0.41 0.63 0.57
68 0.24 0.57 0.38
86 0.03ns 0.51 0.26
87 −0.21 0.74 0.59
89 0.08 ns 0.54 0.30
9 0.48 0.50 0.48
95 0.34 0.55 0.42
97 0.55 0.67 0.76
104 0.18 0.36 0.16
52 0.08 ns 0.31 0.10
28 0.42 0.53 0.46
39 0.56 0.36 0.44
43 0.34 0.63 0.51
63 0.45 0.35 0.33
81 + 82 0.29 0.70 0.57
90 0.63 0.40 0.55
67 + 101 0.54 0.79 0.92
72 + 96 0.40 0.49 0.40
14 0.32 0.53 0.39
29 0.30 0.36 0.22
30 0.06 ns 0.42 0.18
31 0.32 0.58 0.44
32 0.32 0.35 0.22
33 0.60 0.57 0.68
35 0.47 0.55 0.53
45 0.49 0.46 0.45
50 0.49 0.41 0.40
52 0.44 0.55 0.50
71 0.53 0.36 0.41
112 0.59 0.45 0.56
5 0.03 ns 0.52 0.28
42 0.36 0.27 0.21
65 0.46 0.35 0.34
69 0.42 0.42 0.36
75 0.67 0.28 0.52
102 0.11 ns 0.53 0.29
103 0.39 0.73 0.68
111 0.10 ns 0.54 0.30
Open in a new tab

Note. All factor loading are significant at p < .05, except those labeled

ns

(p > .10).

YSR = Youth Self-Report, Extern = Wave 1 pure externalizing factor, Intern = Wave 1 pure internalizing factor, Co-Occur = co-occurring internalizing and externalizing factor

Full Structural Model

Next we estimated a hybrid structural equation model that included the latent problem behavior factors, substance use observed measures, gender, and age. In this model, Wave 3 substance use variables were regressed on Wave 1 problem behavior latent factors, Wave 1 substance use variables, gender, and age. The model fit the data well, x2(1250)= 1505.01, p<0.0001; RMSEA=0.023 (C.I.=0.018 to 0.027); CFI/TLI=0.94/0.93. Standardized path coefficients are displayed in Fig. 1 and reported in the text. High levels of pure externalizing problems prospectively predicted high levels of alcohol (β=0.38, p<0.05), tobacco (β=0.45, p<0.05), and marijuana use (β=0.51, p<0.05). High levels of co-occurring problems prospectively predicted high levels of alcohol (β=0.19, p<0.05) and tobacco (β=0.20, p<0.05) use, but did not predict marijuana use (β=0.05, p>0.15). High levels of pure internalizing problems prospectively predicted low levels of tobacco use (β=−0.22, p<0.05) and marginally predicted low levels of marijuana use (β=−0.20, p<0.10), but did not predict alcohol use (β=−0.09, p>0.10). This pattern is consistent with the Time 1 exogenous correlations. Although two hypothesized effects were not statistically significant (e.g. co-occurring factor to marijuana use and pure internalizing factor to alcohol use), when there were effects, they were in the direction and magnitude that was hypothesized. For example, the standardized path coefficients suggest that the externalizing factor had the strongest effect on each substance use variable, followed by smaller positive effects of the co-occurring factor and tobacco and alcohol. Likewise, the effects from the internalizing factor to tobacco and marijuana use were in the negative direction indicating protection from substance use. To test whether these paths were equal strength, paths from the pure externalizing were constrained to be equal to the paths from the co-occurring factor. The nested test yielded a significant decrement in model fit (x2(3)=18.36, p<0.01), suggesting that paths from the externalizing factor were stronger in magnitude than those from the co-occurring factor.

Fig. 1.

Fig. 1

Open in a new tab

Full structural model for prospective effects of problem behavior factors on substance use outcomes. Note. Problem behavior factors were constrained to correlate with each other at 0. Standardized parameters are presented. Alc = alcohol, Tob = tobacco, * p<0.05 and ♦ p<0.10

Finally, we considered potential gender differences in the model. We initially tried to evaluate the full model (measurement model and path coefficients) for gender invariance, but this lead to convergence problems and Heywood cases likely due to the complexity of our measurement model (two loadings per observed measure with ordinal indicators). As an alternative, we used the factor scores from the measurement model of problem behavior, and then estimated a path model with age, Wave 1 substance use, and the three behavior problem factor scores predicting Wave 3 substance use in a multiple group path model. When the path coefficients were constrained to be equal across gender, the nested test was not statistically significant (x2(32)=26.46, p=0.74), suggesting no differences across gender.

Discussion

We used a novel factor analytic approach to examine internalizing and externalizing problems and their co-occurrence as prospective predictors of early adolescent substance use. Examining each domain of behavior problems in the context of the other impacted the nature of associations with substance use. High levels of both “pure” externalizing and co-occurring problems were prospectively associated with high levels of substance use, whereas high levels of “pure” internalizing problems were associated with low levels of substance use. These findings are consistent with our hypotheses, and suggest that consideration of co-occurring symptoms is important for understanding the implications of behavior problems for early adolescent substance use.

Pure Externalizing and Pure Internalizing Problems

As expected, the strongest and most robust effect was that “pure” externalizing problems were prospectively associated with high levels of alcohol, tobacco, and marijuana use. This is consistent with a large literature suggesting that the development of adolescent substance use occurs within a broader context of antisocial behavior (Hawkins et al. 1991; Zucker et al. 2011) and with theoretical models that emphasize behavioral deviance as a framework to understand the development of adolescent substance use (e.g., Dodge et al. 2009; Jessor and Jessor 1975).

We also found that “pure” internalizing problems were associated with low levels of substance use 2 years later, although this association was statistically significant for tobacco and marijuana use, but not alcohol use. This supports our hypothesis that internalizing problems in the absence of externalizing problems protect early adolescents from substance use. Others have also found that internalizing problems are associated with low risk for substance use (Costello et al. 1999; Kaplow et al. 2001, 2002; Fleming et al. 2008). Adolescent substance use is largely a social behavior that occurs in the context peers (Dishion and Medici Skaggs 2000; Kobus 2003; Oetting and Beauvais 1990), and social withdrawal that often accompanies internalizing problems may protect youth from social processes that promote substance use (Fite et al. 2006a). Fear and worry are also common features of internalizing symptomatology (Brodbeck et al. 2011), which may deter youth from illegal and risky behaviors, such as substance use.

Why the effects of “pure” internalizing symptoms were weaker for alcohol than cigarette and marijuana use is unclear. One possibility is that alcohol use is more normative and meets with more approval by youth (Johnston et al. 2011). Perhaps youth characterized by high levels of internalizing symptoms (in the absence of externalizing problems) in early adolescence are less inclined to engage in cigarette and marijuana use because use of these drugs is perceived as a more deviant and risky.

The “pure” dimensions from our latent variable model are not commonly considered in the literature and some items did not load significantly on these factors. This might raise questions about the validity of these factors. However, in prior work we examined the issue of validity by considering how these dimensions related to a variety of external criterion variables reflecting temperament and social adjustment (Scalco et al. 2012). As expected, pure internalizing symptoms were associated with inhibited, fearful, and avoidant behavior and pure externalizing problems were associated with disinhibited, sensation seeking, approach oriented behavior. Moreover, parent reported behavior problems corresponded to the youth reported pure dimensions as expected. Overall, these associations support the validity of the “pure” internalizing and externalizing factors.

Co-occurring Behavior Problems

An important contribution of this study was that we modeled the association between co-occurring problems and early adolescent substance use. High levels of co-occurring internalizing and externalizing symptoms were prospectively associated with high levels of both tobacco and alcohol use in our study, but these associations were weaker than those for “pure” externalizing behavior problems. This suggests that internalizing problems may diminish the risk associated with externalizing problems. Adolescents who engage in high levels of externalizing behaviors tend to affiliate with deviant peers who support substance use (Brook et al. 2011; Dishion et al. 1995). However, when youth high in externalizing symptoms also experience internalizing symptoms, they may be socially withdrawn and somewhat protected from such affiliations. Also, as suggested above, worry and fear that are common features of internalizing problems may result in more caution, and thus, curtailment of substance use perhaps as a result of concern about potential negative consequences of substance use.

Whereas the co-occurrence of internalizing behavior problems appears to be protective relative to externalizing problems alone in our normative sample, this co-occurrence appears to pose risk when contrasted with the effects of “pure” internalizing problems. Both interpretations suggest that co-occurrence does not operate additively in early adolescence and that consideration of the co-occurrence of internalizing and externalizing behavior problems may help explain the mixed findings in the literature regarding internalizing problems. Studies ignoring co-occurrence would yield either no association between internalizing problems and adolescent substance use or a weak relationship, the direction of which would probably depend on the degree of co-occurrence in the sample.

The literature on co-occurrence of internalizing and externalizing problems with respect to substance use outcomes has been sparse. The studies that have been conducted typically utilized high risk samples, and have relied on a categorical approach to identify youth experiencing high levels of both internalizing and externalizing problems that yield small (N<40) extreme groups. Our study utilized a normative sample and conceptualized co-occurring problems as a continuum. In contrast to our findings, some prior studies have shown that youth characterized by elevated symptoms in both domains are at greater risk for substance use than those characterized by elevation in only one problem domain (e.g., Capaldi 1991, 1992; Miller-Johnson et al. 1998; Ingoldsby et al. 2006). As such, these studies have suggested that internalizing and externalizing problems operate additively (or perhaps synergistically) to increase risk for substance use. Perhaps self-medication motives that stem from internalizing symptoms may lead to particularly high levels of substance use in a context that is broadly supportive of substance use (e.g., antisocial behavior and deviant peers) for some youth (Hussong et al. 2011). Differences in findings suggest that the implications of co-occurrence may depend on the severity of symptomology. Extremely high levels of symptoms that co-occur (as might be found in extreme groups created in high risk samples) may confer increased risk for adolescent substance use, whereas individual variability in a normative distribution may yield a protective effect relative to externalizing symptoms alone. These issues imply important caveats about generalizability of findings and the influence of sample characteristics. Our findings are generalizable to an unselected sample that represents a normative distribution of behavior problems and may not generalize to clinical or other high risk samples.

Limitations and Future Directions

Although this study makes an important advancement in the literature by considering the association between a normative continuum of co-occurring internalizing and externalizing problems and early adolescent substance use, there are some limitations to note. First, we examined correlates of substance use among early adolescents who are in the beginning stages of use. Our findings may not generalize to later ages or later stages of substance use. For example, substance use in adolescents typically occurs in social contexts (Hussong 2000), and age-related changes in the social context may result in shifts in the etiological role of internalizing problems. With age, substance use becomes increasingly normative (Johnston et al. 2011). Early adolescent substance users are likely to engage in a broad array of deviant behaviors (e.g., delinquency, violence, sexual promiscuity) that violate age norms in rebellion against authority and rejection of conventional norms (Jessor et al. 1991). Children high on internalizing problems may be disinclined to engage in such rule violating behavior, and thus are unlikely to find themselves in social contexts that promote substance use in early adolescence. As substance use becomes more normative, there is more opportunity to engage in use outside of deviant peer networks. Hence, youth characterized by high levels of internalizing symptoms (without externalizing symptoms) may be exposed to social contexts that support substance use in late adolescence. The result may be a later age of onset, but once initiated, substance use may quickly escalate as a means of coping with emotional distress. Indeed, there is evidence that internalizing problems confer risk later in adolescence (Sung et al. 2004). Such age-related changes suggest that long-term longitudinal studies are important to examine how the results of this study may vary across different developmental periods.

Second, internalizing and externalizing problems are broad second-order dimensions that include a variety of distinguishable first-order domains of behavior problems. Considering lower-order dimensions of behavior problems (e.g., aggression, delinquency, anxiety, depression) suggests many different forms of co-occurrence, and some may have different implications for adolescent substance use. Most forms of externalizing behavior increase risk for substance use (Iacono et al. 2008). However, some forms of internalizing problems may reduce risk for adolescent substance use (e.g., separation anxiety) whereas others may increase risk (e.g., depression, social anxiety) (Colder et al. 2010). Moreover, different forms of internalizing problems may operate through different mechanisms. For example,Hussong et al. (2011) speculated that depression may cause social isolation and later affiliation with deviant peers and that social anxiety might result in increases in substance use through a self-medication pathway. An important direction for future research will be to extend our work to a variety of other forms of co-occurrence, and to consider potential mechanisms that might account for risk or protection.

Third, problem behavior is dynamic. Not only can internalizing and externalizing problems influence each other over time (Patterson and Capaldi 1990; Wiesner 2003), but substance use can influence problem behavior (e.g., Marmorstein 2009; Mason and Windle 2002). Considering changes in problem behavior and potential reciprocal associations was beyond the scope of this study, but represents an important direction for future research

Finally, a full test of gender invariance could not be performed. Although, a multiple group path analysis suggested the associations between problem behavior and substance use were similar for males and females, this approach makes the assumption that the measurement model is the same across gender. Although prior work suggests that this may be a tenable assumption (Achenbach and Rescorla 2001), this remains a limitation of the study.

Conclusions

In this study, we used a novel factor analytic approach to examining the prospective influence of co-occurring internalizing and externalizing problem behaviors on substance use in early adolescence. Our study provides some important insights into the role of problem behavior in the development of early adolescent substance use, and provides evidence for the importance of modeling this co-occurrence in order to fully understand the prospective effects of both internalizing and externalizing in the context of one another. Given the high co-occurrence between internalizing and externalizing symptoms in adolescence, examining internalizing problems in the absence of externalizing will likely yield erroneous conclusions. Furthermore, simply pitting internalizing against externalizing problems in statistical models to see which one provides unique prediction of adolescent substance use is also problematic because internalizing problems seem to operate differently depending on co-occurring levels of externalizing problems.

An advantage of our factor analytic approach is that it allows for conceptualizing behavior problems as a continuum and therefore captures naturally occurring variation in the distribution of behavior problems. Accordingly, this approach may be particularly well suited to large community samples. Other approaches would also help advance our understanding of how internalizing and externalizing problems operate together in the etiology of adolescent substance use. Mixture models are useful for identifying classes or groups of youth that share similar patterns of internalizing and externalizing problems, however, these methods construe co-occurring and “pure” problems as categorical variables. Another useful approach would be a moderational model whereby one domain of problems moderates the impact of the other on substance use. However, such a model requires making a conceptual distinction between the moderator and independent variable (Kraemer et al. 2008). All of these approaches would be useful in moving the literature forward.

Finally, we did not test potential mechanisms of the link between problem behavior and substance use. This remains an important direction for future research. For example, understanding the mechanisms by which “pure” internalizing problems are associated with low levels of substance may inform interventions. If high internalizing problems operate through low likelihood of affiliating with deviant peers, then helping youth avoid such affiliations may be a useful prevention strategy.

Acknowledgments

This research was funded by a grant from the National Institute on Drug Abuse (R01 DA020171) awarded to Dr. Craig R. Colder.

Contributor Information

Craig R. Colder, Email: ccolder@buffalo.edu, University at Buffalo, The State University of New York, Buffalo, NY, USA; Department of Psychology, University at Buffalo, The State University of New York, Park Hall, Buffalo, NY 14260-4110, USA.

Matthew Scalco, University at Buffalo, The State University of New York, Buffalo, NY, USA.

Elisa M. Trucco, University at Buffalo, The State University of New York, Buffalo, NY, USA

Jennifer P. Read, University at Buffalo, The State University of New York, Buffalo, NY, USA

Liliana J. Lengua, University of Washington, Seattle, WA, USA

William F. Wieczorek, Buffalo State College, Buffalo, NY, USA

Larry W. Hawk, Jr., University at Buffalo, The State University of New York, Buffalo, NY, USA

References

  1. Achenbach TM, Rescorla LA. Manual for the ASEBA school-age forms and profiles. Burlington: University of Vermont Research Center for Children, Youth, and Families; 2001. [Google Scholar]
  2. Angold A, Costello EJ, Erkanli A. Comorbidity. Journal of Child Psychology and Psychiatry. 1999;40(1):57–87. [PubMed] [Google Scholar]
  3. Armstrong TD, Costello EJ. Community studies on adolescent substance use, abuse, or dependence and psychiatric comorbidity. Journal of Consulting and Clinical Psychology. 2002;70(6):1224–1239. doi: 10.1037//0022-006x.70.6.1224. [DOI] [PubMed] [Google Scholar]
  4. Brodbeck J, Abbott RA, Goodyer IM, Croudace TJ. General and specific components of depression and anxiety in an adolescent population. BMC Psychiatry. 2011;11:191. doi: 10.1186/1471-244X-11-191. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Brook DW, Brook JS, Rubenstone E, Zhang C, Saar NS. Developmental associations between externalizing behaviors, peer delinquency, drug use, perceived neighborhood crime, and violent behavior in urban communities. Aggressive Behavior. 2011;37:349–361. doi: 10.1002/ab.20397. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Capaldi DM. Co-occurrence of conduct problems and depressive symptoms in early adolescent boys: I. Familial factors and general adjustment at Grade 6. Development and Psychopathology. 1991;3:277–300. doi: 10.1017/s0954579499001959. [DOI] [PubMed] [Google Scholar]
  7. Capaldi DM. Co-occurrence of conduct problems and depressive symptoms in early adolescent boys: II. A 2-year follow-up at Grade 8. Development and Psychopathology. 1992;4:125–144. doi: 10.1017/s0954579499001959. [DOI] [PubMed] [Google Scholar]
  8. Caron C, Rutter M. Comorbidity in child psychopathology: concepts, issues and research strategies. Journal of Child Psychology and Psychiatry. 1991;32:1063–1080. doi: 10.1111/j.1469-7610.1991.tb00350.x. [DOI] [PubMed] [Google Scholar]
  9. Chen K, Kandel DB. The natural history of drug use from adolescence to the mid-thirties in a general population sample. American Journal of Public Health. 1995;85(1):41–47. doi: 10.2105/ajph.85.1.41. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Cloninger RC. Neurogenetic adaptive mechanisms in alcoholism. Science. 1987;236(24):410–416. doi: 10.1126/science.2882604. [DOI] [PubMed] [Google Scholar]
  11. Cohen J. The cost of dichotomization. Applied Psychological Measurement. 1983;7(3):249–253. [Google Scholar]
  12. Colder CR, Mehta P, Balanda K, Campbell RT, Mayhew K, Stanton WR, et al. Identifying trajectories of adolescent smoking: an application of latent growth mixture modeling. Health Psychology. 2001;20(2):127–135. doi: 10.1037//0278-6133.20.2.127. [DOI] [PubMed] [Google Scholar]
  13. Colder CR, Campbell RT, Ruel E, Richardson JL, Flay BR. A finite mixture model of growth trajectories of adolescent alcohol use: predictors and consequences. Journal of Consulting and Clinical Psychology. 2002;70(4):976–985. doi: 10.1037//0022-006x.70.4.976. [DOI] [PubMed] [Google Scholar]
  14. Colder CR, Chassin L, Lee MR, Villalta IK. Substance Abuse and Emotion. Washington, DC: American Psychological Association; 2010. Developmental perspectives: affect and adolescent substance use; pp. 109–135. [Google Scholar]
  15. Costello EJ, Erkanli A, Federman E, Angold A. Development of psychiatric comorbidity with substance abuse in adolescents: effects of timing and sex. Journal of Clinical Child Psychology. 1999;28(3):298–311. doi: 10.1207/S15374424jccp280302. [DOI] [PubMed] [Google Scholar]
  16. Dishion TJ. Cross-setting consistency in early adolescent psychopathology: deviant friendships and problem behavior sequelae. Journal of Personality. 2000;68(6):1109–1126. doi: 10.1111/1467-6494.00128. [DOI] [PubMed] [Google Scholar]
  17. Dishion TJ, Medici Skaggs N. An ecological analysis of monthly ‘bursts’ in early adolescent substance use. Applied Developmental Science. 2000;4:89–97. [Google Scholar]
  18. Dishion TJ, Capaldi D, Spracklen KM, Li F. Peer ecology of male adolescent drug use. Development and Psychopathology. 1995;7:803–824. [Google Scholar]
  19. Dodge KA, Malone PS, Lansford JE, Miller S, Pettit GS, Bates JE. A dynamic cascade model of the development of substance-use onset. Monographs of the Society for Research in Child Development. 2009;74(Serial No. 294):1–134. doi: 10.1111/j.1540-5834.2009.00528.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Elliott DS, Huizinga D. Social class and delinquent behavior in a national youth panel. Criminology: An Interdisciplinary Journal. 1983;21(2):149–177. [Google Scholar]
  21. Fite PJ, Colder CR, O’Connor RM. Childhood behavior problems and peer selection and socialization: risk for adolescent alcohol use. Addictive Behaviors. 2006a;31(8):1454–1459. doi: 10.1016/j.addbeh.2005.09.015. [DOI] [PubMed] [Google Scholar]
  22. Fite PJ, Colder CR, Pelham WE., Jr. A factor analytic approach to distinguish pure and co-occurring dimensions of proactive and reactive aggression. Journal of Clinical Child and Adolescent Psychology. 2006b;35(4):578–582. doi: 10.1207/s15374424jccp3504_9. [DOI] [PubMed] [Google Scholar]
  23. Fleming CB, Mason WA, Mazza JJ, Abbott RD, Catalano RF. Latent growth modeling of the relationship between depressive symptoms and substance use during adolescence. Psychology of Addictive Behaviors. 2008;22:186–197. doi: 10.1037/0893-164X.22.2.186. [DOI] [PubMed] [Google Scholar]
  24. Galea S, Tracy M. Participation rates in epidemiological studies. Annals of Epidemiology. 2007;17(9):643–653. doi: 10.1016/j.annepidem.2007.03.013. [DOI] [PubMed] [Google Scholar]
  25. Grant BF, Dawson DA. Age at onset of alcohol use and its association with DSM-IValcohol abuse and dependence: results from the National Longitudinal Alcohol Epidemiologic Survey. Journal of Substance Abuse. 1997;9:103–110. doi: 10.1016/s0899-3289(97)90009-2. [DOI] [PubMed] [Google Scholar]
  26. Gruber E, DiClemente RJ, Anderson MM, Lodico M. Early drinking onset and its association with alcohol use and problem behavior in late adolescents. Preventive Medicine. 1996;25:293–300. doi: 10.1006/pmed.1996.0059. [DOI] [PubMed] [Google Scholar]
  27. Hankin BL, Fraley RC, Lahey BB, Waldman ID. Is depression best viewed as a continuum or discrete category? A taxometric analysis of childhood and adolescent depression in a population-based sample. Journal of Abnormal Psychology. 2005;114(1):96–110. doi: 10.1037/0021-843X.114.1.96. [DOI] [PubMed] [Google Scholar]
  28. Hawkins JD, Von Cleve E, Catalano RF. Reducing early childhood aggression: results of a primary prevention program. Journal of the American Academy of Child and Adolescent Psychiatry. 1991;30(2):208–217. doi: 10.1097/00004583-199103000-00008. [DOI] [PubMed] [Google Scholar]
  29. Hawkins JD, Catalano RF, Miller JY. Risk and protective factors for alcohol and other drug problems in adolescence and early adulthood: Implications for substance abuse prevention. Psychological Bulletin. 1992;112(1):64–105. doi: 10.1037/0033-2909.112.1.64. [DOI] [PubMed] [Google Scholar]
  30. Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Structural Equation Modeling. 1999;6(1):1–55. [Google Scholar]
  31. Hussong AM. The settings of adolescent alcohol and drug use. Journal of Youth and Adolescence. 2000;29(1):106–119. [Google Scholar]
  32. Hussong AM, Curran PJ, Chassin L. Pathways of risk for accelerated heavy alcohol use among adolescent children of alcoholic parents. Journal of Abnormal Child Psychology. 1998;26(6):453–466. doi: 10.1023/a:1022699701996. [DOI] [PubMed] [Google Scholar]
  33. Hussong AM, Jones DJ, Stein GL, Baucom DH, Boeding S. An internalizing pathway to alcohol use and disorder. Psychology of Addictive Behaviors. 2011;25(3):390–404. doi: 10.1037/a0024519. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Iacono WG, Malone SM, McGue M. Behavioral disinhibition and the development of early onset addiction: common and specific influences. Annual Review of Clinical Psychology. 2008;4:325–348. doi: 10.1146/annurev.clinpsy.4.022007.141157. [DOI] [PubMed] [Google Scholar]
  35. Ingoldsby EM, Shaw DS, Winslow E, Schonberg M, Gilliom M, Criss MM. Neighborhood disadvantage, parent–child conflict, neighborhood peer relationships, and early antisocial behavior problem trajectories. Journal of Abnormal Child Psychology. 2006;34(3):303–319. doi: 10.1007/s10802-006-9026-y. [DOI] [PubMed] [Google Scholar]
  36. Jessor R, Jessor SL. Adolescent development and the onset of drinking: a longitudinal study. Journal of Studies on Alcohol. 1975;36(1):27–51. doi: 10.15288/jsa.1975.36.27. [DOI] [PubMed] [Google Scholar]
  37. Jessor R, Donovan JE, Costa FM. Beyond adolescence: problem behavior and young adult development. New York: Cambridge University Press; 1991. Problem-behavior theory and young adulthood; pp. 17–38. [Google Scholar]
  38. Johnston LD, O’Malley PM, Bachman JG, Schulenberg JE. Monitoring the future national results on adolescent drug use: overview of key findings, 2010. Ann Arbor: Institute for Social Research, The University of Michigan; 2011. [Google Scholar]
  39. Kaplow JB, Curran PJ, Angold A, Costello EJ. The prospective relation between dimensions of anxiety and the initiation of adolescent alcohol use. Journal of Clinical Child Psychology. 2001;30(3):316–326. doi: 10.1207/S15374424JCCP3003_4. [DOI] [PubMed] [Google Scholar]
  40. Kaplow JB, Curran PJ, Dodge KA The Conduct Problems Prevention Research Group. Child, parent, and peer predictors of early-onset substance use: a multisite longitudinal study. Journal of Abnormal Child Psychology. 2002;30(3):199–216. doi: 10.1023/a:1015183927979. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Keiley MK, Lofthouse N, Bates JE, Dodge KA, Pettit GS. Differential risks of covarying and pure components in mother and teacher reports of externalizing and internalizing behavior across ages 5 to 14. Journal of Abnormal Child Psychology. 2003;31(3):267–283. doi: 10.1023/a:1023277413027. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. King SM, Iacono WG, McGue M. Childhood externalizing and internalizing psychopathology in the prediction of early substance use. Addiction. 2004;99(12):1548–1559. doi: 10.1111/j.1360-0443.2004.00893.x. [DOI] [PubMed] [Google Scholar]
  43. Kline RB. Principal and practice of structural equation modeling. 3rd ed. NY: Guilford Press; 2010. [Google Scholar]
  44. Kobus K. Peers and adolescent smoking. Addiction. 2003;98(s1):37–55. doi: 10.1046/j.1360-0443.98.s1.4.x. [DOI] [PubMed] [Google Scholar]
  45. Kraemer HC, Kiernan M, Essex M, Kupfer DJ. How and why criteria defining moderators and mediators differ between the Baron & Kenny and MacArthur Approaches. Health Psychology. 2008;27(2):S101–S108. doi: 10.1037/0278-6133.27.2(Suppl.).S101. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Loeber R, Stouthammer-Loeber M, White HR. Developmental aspects of delinquency and internalizing problems and their association with persistent juvenile substance use between ages 7 and 18. Journal of Clinical Child Psychology. 1999;28(3):322–332. doi: 10.1207/S15374424jccp280304. [DOI] [PubMed] [Google Scholar]
  47. MacCallum RC, Zhang S, Preacher KJ, Rucker DD. On the practice of dichotomization of quantitative variables. Psychological Methods. 2002;7(1):19–40. doi: 10.1037/1082-989x.7.1.19. [DOI] [PubMed] [Google Scholar]
  48. Markon KE, Chmielewski M, Miller CJ. The reliability and validity of discrete and continuous measures of psychopathology: a quantitative review. Psychological Bulletin. 2011;137(5):856–879. doi: 10.1037/a0023678. [DOI] [PubMed] [Google Scholar]
  49. Marmorstein NR. Longitudinal associations between alcohol problems and depressive symptoms: early adolescence through early adulthood. Alcoholism, Clinical and Experimental Research. 2009;33(1):49–59. doi: 10.1111/j.1530-0277.2008.00810.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Marsh H, Hau K-T, Wen Z. In search of golden rules: comment on hypothesis-testing approaches to setting cutoff values for fit indexes and dangers in overgeneralizing Hu and Bentler’s (1999) findings. Structural Equation Modeling. 2004;11(3):320–341. [Google Scholar]
  51. Mason WA, Windle M. Reciprocal relations between adolescent substance use and delinquency: a longitudinal latent variable analysis. Journal of Abnormal Psychology. 2002;111(1):63–76. doi: 10.1037//0021-843x.111.1.63. [DOI] [PubMed] [Google Scholar]
  52. Masten AS, Faden VB, Zucker RA, Spear LP. Underage drinking: a developmental framework. Pediatrics. 2008;121:235–251. doi: 10.1542/peds.2007-2243A. [DOI] [PubMed] [Google Scholar]
  53. McCarthy DE, Curtin JJ, Piper ME, Baker TB. Negative reinforcement: possible clinical implications of an integrative model. In: Kassel JD, editor. Substance abuse and emotion. Washington, DC: American Psychological Association; 2010. pp. 15–42. [Google Scholar]
  54. Miller-Johnson S, Lochman JE, Coie JD, Terry R, Hyman C. Comorbidity of conduct and depressive problems at sixth grade: substance use outcomes across adolescence. Journal of Abnormal Child Psychology. 1998;26(3):221–232. doi: 10.1023/a:1022676302865. [DOI] [PubMed] [Google Scholar]
  55. Muthén B, Muthén L. MPlus Version 6.1. Los Angeles: Muthén & Muthén; 1998–2007. [Google Scholar]
  56. Nottleman ED, Jensen PS. Comorbidity of disorders in children and adolescents: developmental perspectives. In: Ollendick TH, Prinz RJ, editors. Advances in clinical child psychology. Vol. 17. New York: Plenum; 1995. pp. 109–155. [Google Scholar]
  57. Oetting ER, Beauvais F. Adolescent drug use: findings of national and local surveys. Journal of Consulting and Clinical Psychology. 1990;58(4):385–394. doi: 10.1037//0022-006x.58.4.385. [DOI] [PubMed] [Google Scholar]
  58. Patterson GR, Capaldi DM. A mediational model for boys’ depressed mood. In: Rolf J, Masten AS, Cicchetti D, Nuechterlein KH, Weintraub S, editors. Risk and protective factors in the development of psychopathology. New York: Cambridge University Press; 1990. pp. 141–163. [Google Scholar]
  59. Scalco MD, Trucco EM, Colder CR, Lengua LJ, Wiezcorek WF, Hawk LW. The prospective effect of co-occurring internalizing and externalizing symptoms on adolescent substance use; Poster presented at the Biennial Meeting of the Society for Research on Adolescence; Vancouver, Canada. 2012. Mar, [Google Scholar]
  60. Shivola E, Rose RJ, Dick DM, Pulkkinen L, Martunnen M, Kaprio J. Early onset depressive disorders predict the use of addictive substances in adolescence. A prospective study of adolescent Finnish twins. Addiction. 2008;103(12):2045–2053. doi: 10.1111/j.1360-0443.2008.02363.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  61. Stice E, Barrera M, Jr., Chassin L. Prospective differential prediction of adolescent alcohol use and problem use: examining the mechanisms of effect. Journal of Abnormal Psychology. 1998;107(4):616–628. doi: 10.1037//0021-843x.107.4.616. [DOI] [PubMed] [Google Scholar]
  62. Sung M, Erkanli A, Angold A, Costello EJ. Effects of age at first substance use and psychiatric comorbidity on the development of substance use disorders. Drug and Alcohol Dependence. 2004;75:287–299. doi: 10.1016/j.drugalcdep.2004.03.013. [DOI] [PubMed] [Google Scholar]
  63. Tabachnick BG, Fidell LS. Using multivariate statistics. Boston: Pearson Education, Inc.; 2007. [Google Scholar]
  64. Tarter R, Vanyukov M, Giancola P, Dawes M, Blackson T, Mezzich A, et al. Etiology of early age onset substance use disorder: a maturational perspective. Development and Psychopathology. 1999;11:657–683. doi: 10.1017/s0954579499002266. [DOI] [PubMed] [Google Scholar]
  65. Wiesner M. A longitudinal latent variable analysis of reciprocal relations between depressive symptoms and delinquency during adolescence. Journal of Abnormal Psychology. 2003;112(4):633–645. doi: 10.1037/0021-843X.112.4.633. [DOI] [PubMed] [Google Scholar]
  66. Windle M. A longitudinal study of antisocial behaviors in early adolescence as predictors of late adolescent substance use: gender and ethnic group differences. Journal of Abnormal Psychology. 1990;99(1):86–91. doi: 10.1037//0021-843x.99.1.86. [DOI] [PubMed] [Google Scholar]
  67. Windle M. A retrospective measure of childhood behavior problems and its use in predicting adolescent problem behaviors. Journal of Studies on Alcohol. 1993;54(4):422–431. doi: 10.15288/jsa.1993.54.422. [DOI] [PubMed] [Google Scholar]
  68. Zucker RA, Heitzeg MM, Nigg JT. Parsing the undercontrol–disinhibition pathway to substance use disorders: a multilevel developmental problem. Child Development Perspectives. 2011;5(4):248–255. doi: 10.1111/j.1750-8606.2011.00172.x. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES