Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2010 Mar 1.
Published in final edited form as: J Abnorm Child Psychol. 2009 Apr;37(3):363–373. doi: 10.1007/s10802-008-9269-x

Temperament Pathways to Childhood Disruptive Behavior and Adolescent Substance Abuse: Testing a Cascade Model

Michelle M Martel 1,3, Laura Pierce 2,3, Joel T Nigg 3, Jennifer M Jester 4, Kenneth Adams 2,4, Leon I Puttler 4, Anne Buu 4, Hiram Fitzgerald 3, Robert A Zucker 4
PMCID: PMC2659729  NIHMSID: NIHMS77995  PMID: 18787942

Abstract

Temperament traits may increase risk for developmental psychopathology like Attention-Deficit/Hyperactivity Disorder (ADHD) and disruptive behaviors during childhood, as well as predisposing to substance abuse during adolescence. In the current study, a cascade model of trait pathways to adolescent substance abuse was examined. Component hypotheses were that (a) maladaptive traits would increase risk for inattention/hyperactivity, (b) inattention/hyperactivity would increase risk for disruptive behaviors, and (c) disruptive behaviors would lead to adolescent substance abuse. Participants were 674 children (486 boys) from 321 families in an ongoing, longitudinal high risk study that began when children were three years old. Temperament traits assessed were reactive control, resiliency, and negative emotionality, using examiner ratings on the California Q-Sort. Parent, teacher, and self ratings of inattention/hyperactivity, disruptive behaviors, and substance abuse were also obtained. Low levels of childhood reactive control, but not resiliency or negative emotionality, were associated with adolescent substance abuse, mediated by disruptive behaviors. Using a cascade model, family risk for substance abuse was partially mediated by reactive control, inattention/hyperactivity, and disruptive behavior. Some, but not all, temperament traits in childhood were related to adolescent substance abuse; these effects were mediated via inattentive/hyperactive and disruptive behaviors.

Keywords: temperament, reactive control, disruptive behavior, substance abuse

Substance abuse has serious consequences for youth, including the potential for long-term addiction. A key application of a temperament framework to developmental psychopathology involves the identification of liability for substance abuse. An extensive literature suggests a linkage between temperament traits and substance abuse (Caspi, Moffitt, Newman, & Silva, 1996; Das Eiden, Chavez, & Leonard, 1999; Masse & Tremblay, 1997; Mun, Fitzgerald, Von Eye, Puttler, & Zucker, 2001). At the same time, another literature links behavioral problems such as antisocial behavior with subsequent substance abuse (Masse & Trembley, 1997). Yet these two approaches to identifying predictors of substance abuse have not been connected. Children at high risk for substance abuse disorders often exhibit behavioral manifestations of risk for psychopathology early on, whether conceptualized as temperament traits or disruptive behaviors (Zucker, Ellis, Bingham, & Fitzgerald, 1996).

Temperament and disruptive behavior compete for some of the same behavioral terrain, begging the question of conceptual framing in any study of this sort. As used herein, temperament is defined as early-appearing and relatively stable differences in emotional reactivity and regulation (Rothbart & Bates, 1998). To operationalize temperament traits for children, we drew upon Eisenberg et al’s (1996; 2003) model. In this model, reactive control is defined as a relatively automatic modulation of behavior in response to immediate incentive (Eisenberg, Fabes, Guthrie, & Reiser, 2000). Resiliency refers to the flexible modulation of control in response to context demands. Negative emotionality connotes the propensity to experience anxiety, sadness, and irritable anger.

As conceptualized in the DSM-IV-TR (APA, 2000), disruptive behavior problems are conceived as categorical syndromes (Attention-Deficit/Hyperactivity Disorder [ADHD], Oppositional-Defiant Disorder [ODD], Conduct Disorder [CD]). However, they can also be conceived as continuously distributed liabilities that express in the normal and pathological range. Factor analytic studies of child psychopathology identify behavior problems in the domain of inattention/hyperactivity and externalizing or disruptive behavior (Achenbach & Edelbrock, 1978; Pillow et al., 1998), including delinquency and conduct problems. Inattention/hyperactivity and disruptive behaviors serve as relevant indicators of risk for child ADHD and ODD/CD respectively (Achenbach, 1991a; 1991b; Lampert, Polanczyk, Tramontina, Mardini, & Rohde, 2004).

Potential overlap between temperament and psychopathology is likely, both at a conceptual and a measurement level. For example, temperament may predispose to psychopathology, interact with psychopathology, or be influenced by psychopathology (Watson, Kotov, & Gamez, 2006). Temperament and psychopathology may also reflect the same underlying process (common cause/spectrum model; Watson, Kotov, & Gamez, 2006). Work to date that has controlled for item overlap in questionnaire measurement of temperament traits and psychopathology has often, though not always, found similar relations as when not controlling for this overlap (e.g., Lemery, Essex, & Smider, 2002; Martel & Nigg, 2006). This work advances work on temperament-psychopathology relations by suggesting that relations found between these two domains are real and not entirely due to item overlap.

As early as infancy, apparent markers of temperament, including high novelty seeking and low harm avoidance, appear to predict adolescents’ onset of drinking (Caspi et al., 1996; Das Eiden, Chavez, & Leonard, 1999; Masse & Tremblay, 1997; Mun et al., 2001). Our previous work has established relations between low temperamental reactive control and low resiliency in childhood and disruptive behaviors and substance abuse in adolescence (Martel et al., 2007; Wong et al., 2006). Other work links novelty seeking, high physical activity, low behavioral control, and negative emotionality with alcoholism and other substance abuse (Caspi et al., 1996; Mulder, 2002; Wills & Dishion, 2004). Thus, early risk for adolescent substance abuse may be able to be identified in early childhood via temperament traits.

Childhood inattention/hyperactivity and disruptive behavior problems may mediate a developmental progression from vulnerable temperament in preschool to adolescent substance abuse. Substance use problems in adolescence are in fact frequently preceded in childhood by disruptive behavior problems (or diagnoses such as ODD and CD) as well as by inattention and hyperactivity (or a diagnosis of ADHD; Molina & Pelham, 2003; Tarter, Kirisci, Feske, & Vanyukov, 2007; Zucker, 2006). Children with high levels of hyperactivity, aggression, and delinquency also frequently go on to become adolescents with substance use problems (Eron et al, 1987; Hawkins et al., 1992; Hawkins et al., 1999; Jester et al., 2008; Masse & Tremblay, 1997).

Further, disruptive child behaviors appear to follow a developmental progression. It has been argued that ADHD predicts onset of ODD, and ODD predicts onset of CD, at least in some children (Burke, Loeber, Lahey, & Rathouz, 2005; Loeber, Green, Keenan, & Lahey, 1995). In this vein, Tarter et al (2007) reported that conduct problems mediated associations between hyperactivity and substance use disorders. Inattentive/hyperactive and disruptive behavior problems are associated with temperament traits including low effortful and reactive control, negative emotionality, and decreased agreeableness (Lynam et al., 2005; Martel & Nigg, 2006). Thus, these temperament traits may increase risk for substance abuse problems indirectly by increasing risk for childhood disruptive behaviors.

We therefore propose a cascade model that captures these supposed relations. This model begins with family, or genetic, risk which subsequently manifests itelf in certain temperament traits, such as low reactive control, resiliency, and negative emotionality. These, in turn, through interactions with the environment (Patterson et al., 1998), lead to symptoms of inattention/hyperactivity. Problems with caregivers, teachers, and peers increase likelihood of conduct problems and later substance abuse problems, with onset during adolescence (Molina & Pelham, 2003; Zucker, 2006). In the current study, we attempt to provide an initial, empirical examination of this cascade model. It was hypothesized that (1) risky temperament (e.g., low reactive control) would increase risk for inattention/hyperactivity, (2) inattention/hyperactivity would increase risk for delinquency and conduct problems, (3) disruptive behavior problems would increase risk for substance abuse, and (4) this sequence would mediate the relation of family risk status with adolescent substance abuse, shown in Figure 1.

Figure 1.

Figure 1

Open in a new tab

Hypothesized Model of Temperament Pathways to Adolescent Substance Abuse from Birth to Adolescence

METHOD

Participants

Participants were 674 children (486 or 72% boys) from 321 families who had completed at least one wave of data collection of an ongoing, prospective, multi-wave high risk study. Two percent of the sample was ethnic minority, primarily biracial, due to study design. This study, which started in the early 1980s, is following a community sample of families with high levels of substance use disorders and other psychopathology along with a community contrast sample of families drawn from the same neighborhoods but without the high substance abuse profile and with lower levels of psychopathology. This sample is at considerably enhanced risk for substance use disorders and disruptive psychopathology (Zucker & Wong, 2005). Multiple siblings were included per family when possible. These 321 families included 85 families with one child participant, 145 families with two child participants, 67 families with three child participants, 22 families with four child participants, and two families with five child participants.

These 321 families were drawn from an initial recruitment sample ascertained through the father’s alcoholism for the high risk group, and absence of substance use disorder for both parents in the contrast sample. The following criteria determined selection of men (fathers) into the high-risk sample (N= 161). [a] Men were initially identified through a network covering all courts in a four county wide area. All men with a drunk driving conviction involving a blood alcohol concentration of at least 0.15% if first conviction (or at least 0.12% if a previous drinking-related legal problem had occurred) were potential study candidates. In addition, they were required to [b] meet a Feighner et al. (1972) diagnosis for probable or definite alcoholism, [c] have at least one son between three and five years of age, and [d] be living with the child and his biological mother at the time of enrollment. The presence of fetal alcohol syndrome in any child was exclusionary. Maternal diagnosis was free to vary.

A contrast/control group of families who resided in the same neighborhoods as the alcoholic families, but with no alcohol or other substance abuse history in either parent, was recruited using epidemiological door-to-door canvassing (n=96). In addition, an intermediate risk group (n=64) was obtained by recruiting all families with an alcohol abuse/dependence diagnosis who were found during the community canvass. At later waves, all siblings within +/- 8 years of the primary male target child were also recruited. Additional description of study procedures, recruitment strategies, and eligibility criteria is found in Zucker et al. (1996) and Zucker et al. (2000). Full family assessments involving both parents and participating children occurred at three-year intervals, starting at baseline, wave 1 (ages 3 through 5). The present study reports on data from preschool (wave 1: child age 3.0 – 5.9 years), early childhood (wave 2: child age 6.0 – 8.9), middle childhood (wave 3: child age 9.0 -- 11.9 years), early adolescence (wave 4: child age 12.0 – 14.9 years), and late adolescence (wave 5: child age 15.0 – 17.9 years). Adolescent substance abuse was measured annually during early and late adolescence in addition to the three-year data collection (waves 4 and 5: child age 12.0 – 17.9 years).

Measures

Family Risk Status

At wave 1, parent alcoholism and antisocial personality disorder lifetime diagnoses were given on the basis of responses to the Diagnostic Interview Schedule (DIS), Drinking and Drug History Questionnaire, and short Michigan Alcoholism Screening Test. The DIS is a thorough, guided interview that covers mental and physical health information, including substance abuse. The Drinking and Drug History Questionnaire uses various items from three well validated measures of alcohol and drug use: 1978 NIDA Survey (Johnston, Bachman, & O’Malley, 1979), the American Drinking Practices Survey (Cahalan, Cisson, & Crossley, 1966), and the Research Questionnaire for Alcoholics (Schuckit, 1978). A trained clinician made diagnoses using DSM-IV criteria for each parent’s alcohol use and antisocial personality disorder. A family risk variable was created by summing the index scores for paternal and maternal alcoholism and antisocial personality disorder. The family risk variable ranged from zero (neither parent has alcoholism or antisocial personality disorder) to four (both parents have alcoholism and antisocial personality disorder).

Temperament traits

Child temperament was assessed at waves 1 through 4 by having a test administrator, who knew the child and the family after having spent essentially a full day of data collection with them, complete the California Child Q-sort (CCQ) common language version (Block & Block, 1980; Caspi et al., 1992). The administrator made use of all available data from the hours spent interviewing and assessing the child in completing their ratings so this measure is largely based on administrator observations of the child during testing, one-on-one interaction with the administrator, and parent-child interactions.

The CCQ consists of 100 cards which must be placed in a forced-choice, nine-category normal distribution. The administrator (staff person) described the child by placing descriptive cards in one of the categories, ranging from one (least descriptive) to nine (most descriptive). Three temperament scores were computed based on previously published Q-sort item scales for resiliency (Eisenberg et al., 2003; e.g., “is resourceful in initiating activities; uses and responds to reason”), reactive control (Eisenberg et al., 1996; e.g., “is restless and fidgety[R]”; “has a rapid personal tempo, reacts and moves quickly[R]”), and negative emotionality (Eisenberg et al., 1996; e.g., “has rapid shifts in mood”). Consistent with those reports, each scale achieved adequate internal reliability in our sample (α ≥ .91 for each). Average cross-wave inter-administrator correlations for the temperament scales were statistically significant though modest in magnitude (reactive control r=.28, p<.01; resiliency r=.27, p<.01; negative emotionality r=.19, p<.01).

Child problem behaviors

Inattention/hyperactivity was assessed at waves 1 through 5 via parent report on the Child Behavior Checklist (CBCL; Achenbach, 1991a) and via teacher report on the Teacher Report Form (TRF; Achenbach, 1991b). A composite of inattention/hyperactivity was generated as the mean of parent and teacher ratings at waves 1 through 5. Internal reliability for this composite was .79. Youth reports were not included in the composite due to evidence that they lack validity for these symptoms (Henry, 2006; Jensen et al., 1999).

Disruptive behavior (delinquency and conduct problems) was measured similarly using the externalizing behavior composite from the CBCL and TRF, but with a focus on the most knowledgeable rater. At wave 1 (ages 3-5), parent report was used; at waves 2 (ages 6-8) and 3 (ages 9-11), parent and teacher reports were used; at waves 4 and 5 (ages 12-17), parent, teacher, and adolescent reports were used. Internal reliability for this composite in the current study was .84 when averaged across all waves.

Adolescent Substance Abuse

Twenty-seven items pertaining to alcohol abuse and seventeen items concerning drug abuse inquired as to the presence or absence of substance-related problems in the past year. These questions were taken from the Drinking and Drug History Questionnaire (Cahalan et al., 1966; Johnston et al., 1979; Shuckit, 1978) administered in self-report form to adolescents at waves 4 and 5, as well as during annual wave collection from ages 11 through 17. If an item was ever endorsed, it was counted as endorsed, a procedure utilized in order to accurately assess low base-rate instances of substance abuse and substance-related problems (for validity of this clinical aggregation approach, see Piacentini, Cohen, & Cohen, 1992). High test-retest reliability of self-reports of substance use (e.g., r=.89-.99 for non-marijuana illicit drug use from ages 16 to 17) was evident in the current sample and are consistent with those reported in the larger literature (O’Malley, Bachman & Johnson, 1983). Alpha coefficient reliability for this substance abuse composite score in the current sample was .99, suggesting that the variables used to generate this composite were highly associated with one another.

Data Analysis Plan

Handling of Missing Data and Nonindependence

At wave 1, only male target children were initially recruited. Female target children and additional siblings were recruited into the study at later waves as funding permitted. Between 351 and 565 children completed data collection at each individual wave, and 204 children completed all waves of data collection, although most children completed more than one wave of data collection. A subset of families was not assessed at waves 2 and 3 when funding difficulties systematically precluded their evaluation. This missingness fits the description of data missing at random (Little & Rubin, 1987). Participants were included as long as they had at least one data point for child traits, child problem behaviors, or adolescent substance abuse. Missing-at-random data was handled by using the full information maximum likelihood (FIML) estimator available in Mplus (Muthen & Muthen, 2007), as recommended in the methodology literature (McCartney, Burchinal, & Bub, 2006; Schafer & Graham, 2002). The CLUSTER command was implemented in Mplus to take in account the non-independence of data (due to siblings living in the same family) which could otherwise artificially inflate the standard errors of the parameter estimates.

Covariates and Statistical Checks

In order to include other potentially important predictors, all analyses that did not include a family risk variable included lifetime paternal alcoholism, lifetime maternal alcoholism, socioeconomic status (SES; measured using the revised Duncan scores which take into account parental employment; Mueller & Parcel, 1981), sex of child, and age at wave 5 data collection (to control for differences in opportunity to develop drug or alcohol problems as a function of final reporting age) as covariates.

Strategy for Mediation Analyses

The model underlying the current work suggested that traits (e.g., reactive control) evolve into inattention and that inattention/hyperactivity in turn progresses into antisocial behavior in a cascade-like sequence (Patterson, Reid, & Dishion, 1998), consistent with a common cause/spectrum model of trait-psychopathology relations (Watson et al., 2006). The correct analysis for this question might be open to some debate. We chose not to conduct latent growth curve models because those assume that all constructs are distinct (i.e., that reactive control is a distinct construct from antisocial behavior). Rather, we approached this analysis in two rather simple and straightforward ways for descriptive purposes. First, to maximize power, regression analyses were conducted using composite scores averaged across all available data waves. However, since this strategy does not capitalize on the longitudinal design, we also examined models in which we chose trait or behavior measures from the ages at which they were theorized to be most important (temperament in preschool, psychopathological behaviors in childhood, and drug abuse in adolescence). This enabled us to rule out the possibility that overlapping wave data might be driving relations between temperament and psychopathology.

In order to test mediation, multiple, simultaneous regression analyses were conducted to test the path coefficients in the model (Baron & Kenny, 1986) and to evaluate whether full or partial mediation occurred by testing the significance of the indirect path (MacKinnon, Lockwood, Hoffman, West, & Sheets, 2002). In order to compare model fit in non-nested models, the Akaike information criteria (AIC) was examined. The model with the smallest AIC is the best fitting (Kline, 2005).

RESULTS

Descriptive Statistics on Sample

Descriptive statistics are presented in Table 1 Children from alcoholic families, including high and intermediate risk groups, had lower reactive control and resiliency, marginally higher levels of negative emotionality, higher levels of inattention/hyperactivity and disruptive behavior, and more substance abuse. A correlation matrix of key study variables are presented in Table 2 These preliminary analyses suggested that the pathways that had been hypothesized might be plausible, so testing of those paths proceeded step by step.

Table 1.

Descriptive Statistics on Sample

Alcoholic Non Alcoholic Total
N 493 181 674
N (%) boys 358(72.6) 128(70.7) 486(71.65)
Family SES 315.38(127.75) 370.04(132.96) 330.32(131.37)**
Age at wave 5 16.54(.94) 16.49(.96) 16.53(.95)
Reactive Control 4.94(.89) 5.20(.85) 5.01(.88)**
Resiliency 5.73(.72) 5.88(.69) 5.77(.72)**
Negative Emotionality 4.29(.92) 4.14(.89) 4.25(.91)+
Inattention/Hyperactivity 53.85(4.22) 52.58(3.36) 53.50(4.04)**
Disruptive Behavior 51.94(6.61) 48.58(5.65) 51.02(6.53)**
Substance Abuse 3.78(5.40) 1.65(3.53) 3.20(5.05)**
Open in a new tab

Note.

+

p<.10,

*

p< .05,

**

p<.01 for alcoholic vs. non-alcoholic group comparison by t-test or chi-square.

Table 2.

Correlation Matrix of Temperament Traits, Disruptive Behaviors, and Substance Abuse

RC NE Resil I/H DB SA
Reactive Control
Negative Emotionality -.12**
Resiliency .06 -.80**
Inattention/Hyperactivity -.38** .25** -.25**
Disruptive Behavior -.41** .29** -.25** .64**
Substance Abuse -.15** .06 -.01 .13** .40**
Open in a new tab

Note.

*

p< .05.

**

p<.01. RC=Reactive Control. NE=Negative Emotionality. Resil=Resiliency. DB=Disruptive Behavior. SA=Substance Abuse.

Question 1: Does inattention/hyperactivity mediate the relationship between temperament and disruptive behavior problems?

In order to evaluate whether inattention/hyperactivity (at ages three through 17) mediated the relationship between temperament traits (at ages three through 14) and disruptive behavior problems (at ages three through 17), three regression models were conducted, one for each temperament trait. As depicted in Figure 2, all individual pathways were significant (p<.01) in the expected direction. Age of child at wave 5 data collection and lifetime paternal alcoholism were significant covariates. As the figure suggests, significant partial mediation was observed in each instance (first panel indirect estimate= -1.25, p<.01; 99% confidence interval: -1.84- -.66; middle panel indirect estimate=.92, p<.01, 99% confidence interval: .45-1.38; bottom panel indirect estimate=-1.34, p<.01, 99% confidence interval: -1.83- -.86). Thus, the first portion of the conceptual model was supported. Each trait operated partially via inattention/hyperactivity in relation to disruptive behavior problems.

Figure 2.

Figure 2

Open in a new tab

Inattention/Hyperactivity Partially Mediates Pathway Between Traits and Disruptive Behavior

Note. *p < .05. **p<.01. Analyses controlled for sex, age at wave 5, paternal alcoholism, maternal alcoholism, and family SES. Indirect effect estimates: -1.25, .92, -1.34 respectively; all p<.01.

The significant indirect effects hold when one examines a model of non-overlapping wave data including traits measured at wave 1 (ages 3 to 5), inattention/hyperactivity measured at waves 2 and 3 (ages 6 to 11), and disruptive behaviors measured at waves 4 and 5 (ages 12 to 17; indirect estimates= -.62 for resiliency; .35 for negative emotionality; -.38 for reactive control; all p<.01).

Question 2: Does disruptive behavior mediate the relationship between inattention/hyperactivity and substance abuse?

As shown in Figure 3, disruptive behavior (at ages three through 17) fully mediated and, in fact, suppressed the positive relationship between inattention/hyperactivity (at ages three through 17) and substance abuse (at ages 11 through 17; indirect estimate=.52, p<.01, 99% confidence interval: .32-.73). Child sex (estimate=-.09, p<.05), age of child at wave 5 data collection (estimate=.08, p<.05), and lifetime paternal alcoholism (estimate=.11, p<.01) were significant covariates. The significant indirect effect (estimate=.31, p<.01, 99% confidence interval: .16-.46) held using non-overlapping measures of inattention/hyperactivity at waves 2 and 3 (ages 6 to 11) and measures of disruptive behavior at waves 3 and 4 (ages 9 to 14).

Figure 3.

Figure 3

Open in a new tab

Disruptive Behavior Suppresses the Pathway Between Inattention/Hyperactivity and Substance Abuse

Note. *p< .05. **p<.01. Analyses controlled for sex, age at wave 5, paternal alcoholism, maternal alcoholism, and family SES. Indirect effect estimate: .52, p<.01.

Question 3: Do inattention/hyperactivity and disruptive behaviors mediate the relationship between temperament and substance abuse?

Resiliency and negative emotionality were not significantly related to substance abuse (path estimate=.01, .04 respectively, both p>.05), so they were not considered in subsequent analyses. The focus of analyses therefore became reactive control, which was inversely related to later substance abuse (estimate=-.16, p<.01).

Inattention/hyperactivity was related to substance abuse problems at wave 5 (estimate=.11, p<.05), but failed to mediate the relationship between reactive control and substance abuse (indirect estimate=-.13, p>.05, 95% confidence interval:-.36 to .09). Age at wave 5 data collection (estimate=.12, p<.01) and lifetime paternal alcoholism (estimate=.15, p<.01) were significant covariates. On the other hand, disruptive behavior (at ages three through 17) was related to substance abuse problems (at ages 11 through 17; estimate=.39, p<.01), completely mediating the relation of low reactive control (ages three to 14) to substance abuse, shown in Figure 4 The direct path between reactive control and substance abuse dropped from -.16 (p<.01) to -.02 (p>.05); the indirect effect was significant (estimate=-.94, p<.01, 99% confidence interval: -1.4 to -.49). Child sex (estimate=-.11, p<.01), age at wave 5 data collection (estimate=.09, p<.05), and lifetime paternal alcoholism (estimate=.11, p<.01) were significant covariates. The indirect effect remained significant (estimate=-.13, p<.01; 99% confidence interval: -1.12 to -.3) when non-overlapping measures of reactive control (at waves 1 through 3; ages 3 to 11) and disruptive behavior (at waves 3 and 4; ages 9 to 14) were used.

Figure 4.

Figure 4

Open in a new tab

Disruptive Behavior Mediates Pathway Between Reactive Control and Substance Abuse

Note. *p< .05. **p<.01. Analyses controlled for sex, age at wave 5, paternal alcoholism, maternal alcoholism, and family SES. Indirect effect estimate: -.94, p<.01.

Thus, the major tenets of the hypothesized cascade model held. Reactive control was significantly related to child inattention/hyperactivity and disruptive behaviors, as well as to adolescent substance abuse. The relation between inattention/hyperactivity and substance abuse was mediated by disruptive behaviors, and the relation between reactive control and substance abuse was mediated by disruptive behavior. Now we move to the final step, to see if this chain mediates family risk transmission.

Question 4: Do child traits and behavior problems mediate the relationship between family risk and adolescent substance abuse?

As expected, family risk status was moderately related to substance abuse in youth at wave 5 (estimate=.21, p<.01 with SES, sex of child, and age at wave 5 data collection covaried). As shown in Figure 5, when substance abuse was regressed on family risk, controlling for reactive control (ages three to 14) and child problem behavior (ages three to 17), the direct path between family risk and adolescent substance abuse dropped from .21 (p<.01) to .09 (p<.05). The indirect effect was significant (estimate=.15, p<.01; 99% confidence interval=.03-.28). Child sex (estimate=-.09, p<.05) and age at wave 5 data collection (estimate=.09, p<.05) were significant covariates. We rechecked this model using non-overlapping data: reactive control measured at wave 1 (ages 3 to 5), inattention/hyperactivity measured at waves 2 and 3 (ages 6 to 11), and disruptive behavior measured at waves 3 and 4 (ages 9 to 14). The indirect effect estimate remained significant in this model (estimate=.04, p<.05; 95% confidence interval=.003 to .08).

Figure 5.

Figure 5

Open in a new tab

Traits and Child Disruptive Behavior Mediates Pathway Between Family Risk and Substance Abuse

Note. *p< .05. **p<.01. Analyses controlled for sex, age at wave 5, and family SES. Indirect effect estimate=.15, p<.01.

Data Checks

Models with reversed paths were examined, compared to the full model presented in Figure 5 (AIC=23453.33) in order to provide a more stringent test of the direction of effects between key variables. When the path between reactive control and inattention/hyperactivity was reversed, the AIC increased to 23606.15, indicating worse fit for this model. When the path between inattention/hyperactivity and disruptive behaviors was reversed, the AIC was identical (23453.33), suggesting equivalent fit.

Two-group analyses were also conducted to clarify potential gender differences in mediation models. All mediation effects were significant and in the same direction within boys and girls with one exception. The indirect effect in the final model, testing whether child traits and problem behavior mediated the association between family risk and substance abuse, was significant in boys (estimate=.17, p<.05; 95% confidence interval=.05 to .28) but marginal in girls (estimate=.06, p>.05; 95% confidence interval=- .04 to .16).

DISCUSSION

Early temperament dysregulation may increase risk for childhood behavior problems that in turn evolve into substance abuse during adolescence. This study sought to test a developmental model of substance abuse using a coherent cascade model in which it was hypothesized that (1) risky temperament (e.g., low reactive control) would increase risk for inattention/hyperactivity, (2) inattention/hyperactivity would increase risk for delinquency and conduct problems, (3) disruptive behavior problems would increase risk for substance abuse, and (4) this sequence would mediate the relation of family risk status to adolescent substance abuse. Three temperament traits were examined out of many that could be nominated. Only one, low reactive control, mediated risk for substance abuse in middle adolescence, while negative emotionality and resiliency appeared to increase childhood risk for disruptive behaviors without increasing later risk for adolescent substance abuse once family risk was controlled, consistent with previous literature (Auerbach et al., 2001). Low reactive control appears to predispose children to the development of inattention/hyperactivity. Hyperactive and impulsive behavior in turn mediated the effect of low reactive control on disruptive or antisocial behavior, consistent with previous work suggesting that inattention/hyperactivity predisposes to disruptive behavior problems (Burke et al., 2005; Loeber, Green, Keenan, & Lahey, 1995). However, it may be that disruptive behavior is a more powerful driver of later problem behaviors like substance abuse than inattention/hyperactivity since disruptive behavior suppressed the effect of inattention on substance abuse. This suppression effect also suggests substantial overlap between inattention and disruptive behavior. In line with these ideas, disruptive behavior partially mediated the relationship between low reactive control and substance abuse.

A key finding was that reactive control, inattention/hyperactivity, and disruptive behaviors partially mediated the pathway between family risk and adolescent substance abuse. Family risk, likely heavily influenced by genetic and biological factors, may increase risk for low levels of reactive control and high levels of childhood disruptive behavior problems via abnormalities or less efficient neural processing in limbic-frontal circuitry (Rothbart & Posner, 2006). In addition, family risk in the form of parent psychopathology may increase the likelihood of negative interaction patterns in the home, in this way increasing risk for disruptive and delinquent behaviors that are often associated with adolescent substance abuse.

Based on this study’s findings, one pathway to adolescent substance abuse appears to be through childhood behavior problems like ADHD and ODD. This finding is in line with previous work testing components of our cascade model. Children with disruptive behavior problems (or diagnoses such as ODD and CD) and/or inattention and hyperactivity (or a diagnosis of ADHD) appeared more likely to develop substance use problems in adolescence (Eron et al, 1987; Hawkins et al., 1992; Hawkins et al., 1999; Jester et al., 2008; Masse & Tremblay, 1997; Molina & Pelham, 2003; Tarter et al., 2007; Zucker, 2006). Childhood disruptive behaviors appeared to follow the developmental progression hypothesized in previous work such that inattention/hyperactivity predicted the onset of disruptive behaviors (Burke et al., 2005; Loeber et al., 1995), although this admittedly remains controversial. Further, disruptive behavior mediated associations between inattention/hyperactivity and substance abuse, consistent with Tarter et al. (2007).

A fully longitudinal analysis of these patterns faces several barriers. One is the fact that behavior in any domain at an early age could influence behavior in another domain later, and yet most of the behaviors discussed herein occur in some degree across all ages studied (the major exception being substance use, which for the most part only occurred in the latter years of the study). We were able to partially address this in secondary analyses that restricted measures to the age at which they were theorized to be most salient or influential. In addition, previous longitudinal research on trait-psychopathology relations, using observational temperament measures, suggests that maladaptive traits increase risk for psychopathology, rather than the other way around (Eisenberg et al., 2001). However, it should be noted that, although traits and psychopathology are often conceptualized as separate constructs and thus reported associations could be seen as evidence of a cascade effect, it is also possible that traits and psychopathology are both tapping into the same processes, namely trait expression, consistent with a spectrum model of trait-psychopathology relations (Van Leeuwen, Mervielde, De Clercq, & De Fruyt, 2007). The data presented in the current paper cannot rule out this possibility.

A key limitation was that temperament was measured using only one set of examiner ratings per wave. It might have been more ideal to have included laboratory-based observational measures of temperament at the early waves and multiple-informant ratings of temperament at the older waves, as well as measurement of additional temperament traits such as behavioral disinhibition. Only one measure of substance abuse, albeit a composite alcohol and drug measure, was used in the current study. However, separate analysis of alcohol and drug problems and alcohol and drug use yielded similar findings as those reported here, so the results did not depend on a particular way of defining alcohol/drug-related outcomes. The generalizability of these findings to more ethnically diverse samples is unclear. The sample was 72% male, thus providing some improvement on prior literature emphasizing risk solely in males. Nonetheless, caution should be used in generalizing these findings to girls. A final limitation was that we did not examine growth curve trajectories or lag models so we did not demonstrate that early levels of one trait predicted change over time in other traits and psychopathology. We did not analyze those models because the model being tested was a cascade model in which one behavior was hypothesized to develop into the other, not a model in which one trait influences a completely different trait. Alternative formulations of developmental relations among traits and psychopathology should be considered in future work.

The results of the current study have important clinical implications. First, prevention and intervention efforts directed at childhood inattention/hyperactivity and disruptive behaviors are important and could have the added benefit of decreasing adolescent substance abuse. Early assessment and intervention for childhood disruptive behaviors might include the development of enhanced prosocial alternatives to substance abuse. Additional research on temperament traits associated with later-developing childhood disruptive behaviors and adolescent substance abuse may be useful in identifying children at risk for adolescent substance abuse at earlier ages. Of course, as this cascade effect only partially mediated the pathway between family risk and later substance abuse, study of additional risks for substance abuse remain important.

In conclusion, temperament may mediate the relationship between family risk and substance abuse. However, that temperament effect is mediated by inattention and disruptive behavior problems. At a general level, our data suggests that a developmental cascade of behaviors links family risk and adolescent substance abuse via low reactive control, inattention/ hyperactivity, and disruptive behavior.

Acknowledgments

This work was supported by NIAAA grant R01-AA12217 to Robert Zucker and Joel Nigg, NIAAA grant R37-AA07065 to Robert Zucker and Hiram Fitzgerald, and NIMH grant R01- MH59105 to Joel Nigg. Martel was supported by 1 F31 MH075533-01A2. The authors thank the participants and Susan Refior, the long term MLS Field Director, whose steadfast commitment and support have made this study possible.

References

  1. Achenbach TM. Manual for the Child Behavior Checklist/4-18 and 1991 Profile. University of Vermont Department of Psychiatry; Burlington, VT: 1991a. [Google Scholar]
  2. Achenbach TM. Manual for the Teacher’s Report Form and 1991 Profile. University of Vermont Department of Psychiatry; Burlington, VT: 1991b. [Google Scholar]
  3. Achenbach TM, Edelbrock CS. The classification of child psychology: A review and analysis of empirical efforts. Psychological Bulletin. 1978;85:1275–1301. [PubMed] [Google Scholar]
  4. American Psychiatric Association. Diagnostic and statistical manual of mental disorders. Fourth. Washington, D.C: American Psychiatric Association; 2000. [Google Scholar]
  5. Auerbach JG, Benjamin J, Faroy M, Geller V, Ebstein R. DRD4 related to infant attention and information processing: A developmental link to ADHD? Psychiatric Genetics. 2001;11(1):31–35. doi: 10.1097/00041444-200103000-00006. [DOI] [PubMed] [Google Scholar]
  6. Baron RM, Kenny DA. The moderator-mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology. 1986;51(6):1173–1182. doi: 10.1037//0022-3514.51.6.1173. [DOI] [PubMed] [Google Scholar]
  7. Block JH, Block J. The role of ego-control and ego-resiliency in the organization of behavior. In: Collins WA, editor. Development of cognition, affect, and social relations: The Minnesota Symposia on Child Psychology. Vol. 13. Hillsdale, NJ: Lawrence Erlbaum; 1980. pp. 39–101. [Google Scholar]
  8. Burke JD, Loeber R, Lahey BB, Rathouz PJ. Developmental transitions among affective and behavioral disorders in adolescent boys. Journal of Child Psychology and Psychiatry. 2005;46(11):1200–1210. doi: 10.1111/j.1469-7610.2005.00422.x. [DOI] [PubMed] [Google Scholar]
  9. Cahalan D, Cisson I, Crossley H. American drinking practice: A national study of drinking and attitudes. New Brunswick, NJ: Publications Division, Rutgers Center for Alcohol Studies; 1966. [Google Scholar]
  10. Caspi A, Block J, Block JH, Klopp B, Lynam D, Moffitt TE, Stouthamer-Loeber M. A “common-language” version of the California Q-Set for personality assessment. Psychological Assessment. 1992;4:512–523. [Google Scholar]
  11. Caspi A, Moffitt TE, Newman DL, Silva PA. Behavioral observations at age 3 years predict adult psychiatric disorders. Archives of General Psychiatry. 1996;53:1033–1039. doi: 10.1001/archpsyc.1996.01830110071009. [DOI] [PubMed] [Google Scholar]
  12. Das Eiden R, Chavez F, Leonard KE. Parent-infant interactions among families with alcoholic fathers. Development and Psychopathology. 1999;11:745–762. doi: 10.1017/s0954579499002308. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Eisenberg N, Cumberland A, Spinrad TL, Fabes RA, Shepard SA, Reiser M, Murphy BC, Losoya SH, Guthrie IK. The relations of regulation and emotionality to children’s externalizing and internalizing problem behavior. Child Development. 2001;72(4):1112–1134. doi: 10.1111/1467-8624.00337. [DOI] [PubMed] [Google Scholar]
  14. Eisenberg N, Fabes RA, Guthrie IK, Murphy BC, Maszk P, Holmgren R, Suh K. The relations of regulation and emotionality to problem behavior in elementary school children. Development and Psychopathology. 1996;8:141–162. [PubMed] [Google Scholar]
  15. Eisenberg N, Fabes RA, Guthrie IK, Reiser M. Dispositional emotionality and regulation: Their role in predicting quality of social functioning. Journal of Personality and Social Psychology. 2000;78:136–157. doi: 10.1037//0022-3514.78.1.136. [DOI] [PubMed] [Google Scholar]
  16. Eisenberg N, Guthrie IK, Fabes RA, Shepard S, Losoya S, Murphy BC, Jones S, Poulin R, Reiser M. Prediction of elementary school children’s externalizing problem behaviors from attentional and behavioral regulation and negative emotionality. Child Development. 2000;71(5):1367–1382. doi: 10.1111/1467-8624.00233. [DOI] [PubMed] [Google Scholar]
  17. Eisenberg N, Qhou Q, Losoya SH, Fabes RA, Shepard SA, Murphy BC, Reiser M, Guthrie IK, Cumberland A. The relations of parenting, effortful control, and ego control to children’s emotional expressivity. Child Development. 2003;74(3):875–895. doi: 10.1111/1467-8624.00573. [DOI] [PubMed] [Google Scholar]
  18. Eron LD, Huesmann LR, Dubow E, Romanoff R, Yarmel PW. Aggression and its correlates over 22 years. In: Crowell DH, Evans IM, O’Donnell CR, editors. Childhood aggression and violence. New York: Plenum Press; 1987. pp. 249–262. [Google Scholar]
  19. Feighner JP, Robins E, Guze SB, Woodruff RA, Winokur G, Munoz R. Diagnostic criteria for use in psychiatric research. Archives of General Psychiatry. 1972;26:57–63. doi: 10.1001/archpsyc.1972.01750190059011. [DOI] [PubMed] [Google Scholar]
  20. Hawkins JD, Catalano RF, Kosterman R, Abbott R, Hill KG. Preventing adolescent health-risk behaviors by strengthening protection during childhood. Archives of Pediatrics and Adolescent Medicine. 1999;153:226–234. doi: 10.1001/archpedi.153.3.226. [DOI] [PubMed] [Google Scholar]
  21. Hawkins JD, Catalano RF, Miller JY. Risk and protective factors for alcohol and other drug problems in adolescence and early adulthood: Implications for substance abuse prevention. Psychological Bulletin. 1992;112:64–105. doi: 10.1037/0033-2909.112.1.64. [DOI] [PubMed] [Google Scholar]
  22. Henry DB. Associations between peer nominations, teacher ratings, self-reports, and observations of malicious and disruptive behavior. Assessment. 2006;13(3):241–252. doi: 10.1177/1073191106287668. [DOI] [PubMed] [Google Scholar]
  23. Jensen PS, Rubio-Stipec M, Canino G, Bird HR, Dulcan MK, Schwab-Stone ME, Lahey B. Parent and child contributions to diagnosis of mental disorder: Are both informants always necessary? Journal of the American Academy of Child and Adolescent Psychiatry. 1999;38(12):1569–1579. doi: 10.1097/00004583-199912000-00019. [DOI] [PubMed] [Google Scholar]
  24. Jester JM, Nigg JT, Buu A, Puttler LI, Glass JM, Heitzeg MM, Fitzgerald HE, Zucker RA. Trajectories of childhood aggression and inattention/hyperactivity: Differential effects on substancfe abuse in adolescence. Journal of the American Academy of Child and Adolescent Psychiatry. 2008;47(10):1–8. doi: 10.1097/CHI.0b013e3181825a4e. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Johnston LD, Bachman JG, O’Malley PM. Drugs and the class of ’78: Behaviors, attitudes, and recent national trends. Washington, DC: National Institute on Drug Abuse, Division of Research, U.S. Department of Health, Education, and Welfare; 1979. [Google Scholar]
  26. Kline RB. Principles and practice of structural equation modeling. Second. New York: Guilford Press; 2005. [Google Scholar]
  27. Lampert TL, Polanczyk G, Tramontina S, Mardini V, Rohde LA. Diagnostic performance of the CBCL-attention problem scale as a screening measure in a sample of Brazilian children with ADHD. Journal of Attention Disorders. 2004;8(2):63–71. doi: 10.1177/108705470400800204. [DOI] [PubMed] [Google Scholar]
  28. Lemery KS, Essex MJ, Smider NA. Revealing the relation between temperament and behavior problem symptoms by eliminating measurement confounding: Expert ratings and factor analyses. Child Development. 2002;73(3):867–882. doi: 10.1111/1467-8624.00444. [DOI] [PubMed] [Google Scholar]
  29. Little RJA, Rubin DB. Statistical Analysis With Missing Data. Wiley; New York: 1987. [Google Scholar]
  30. Loeber R, Green SM, Keenan K, Lahey BB. Which boys will fare worse? Early predictors of the onset of conduct disorder in a six-year longitudinal study. Journal of the American Academy of Child and Adolescent Psychiatry. 1995;34(4):499–509. [PubMed] [Google Scholar]
  31. Lynam DR, Caspi A, Moffitt TE, Raine A, Loeber R, Stouthamer-Loeber M. Adolescent psychopathy and the Big Five: Results from two samples. Journal of Abnormal Child Psychology. 2005;33(4):431–443. doi: 10.1007/s10648-005-5724-0. [DOI] [PubMed] [Google Scholar]
  32. MacKinnon DP, Lockwood CM, Hoffman JM, West SG, Sheets V. A comparison of methods to test mediation and other intervening variable effects. Psychological Methods. 2002;7(1):83–104. doi: 10.1037/1082-989x.7.1.83. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Martel MM, Nigg JT. Child ADHD and personality/temperament traits of reactive and effortful control, resiliency, and emotionality. Journal of Child Psychology and Psychiatry. 2006;47(11):1175–1183. doi: 10.1111/j.1469-7610.2006.01629.x. [DOI] [PubMed] [Google Scholar]
  34. Martel MM, Nigg JT, Wong MM, Fitzgerald HE, Jester JM, Puttler LI, Glass JM, Adams KM, Zucker RA. Child and adolescent resiliency, regulation, and executive functioning in relation to adolescent problems and competence in a high-risk sample. Development and Psychopathology. 2007;19(2):541–563. doi: 10.1017/S0954579407070265. [DOI] [PubMed] [Google Scholar]
  35. Massé LC, Tremblay RE. Behavior of boys in kindergarten and the onset of substance use during adolescence. Archives of General Psychiatry. 1997;54:62–68. doi: 10.1001/archpsyc.1997.01830130068014. [DOI] [PubMed] [Google Scholar]
  36. McCartney K, Burchinal MR, Bub KL. Best Practices in Quantitative Methods for Developmentalists. Monographs of the Society for Research in Child Development. 2006;71(3) doi: 10.1111/j.1540-5834.2006.07103001.x. [DOI] [PubMed] [Google Scholar]
  37. Molina BSG, Pelham WE. Childhood predictors of adolescent substance use in a longitudinal study of children with ADHD. Journal of Abnormal Psychology. 2003;112(3):497–507. doi: 10.1037/0021-843x.112.3.497. [DOI] [PubMed] [Google Scholar]
  38. Mueller CW, Parcel TL. Supplement to “Measures of socioeconomic status: Alternatives and recommendations”. Child Development. 1981;52:13–30. [Google Scholar]
  39. Mulder RT. Alcoholism and personality. Australian and New Zealand Journal of Psychiatry. 2002;36:44–52. doi: 10.1046/j.1440-1614.2002.00958.x. [DOI] [PubMed] [Google Scholar]
  40. Mun EY, Fitzgerald HE, Von Eye A, Puttler LI, Zucker RA. Temperamental characteristics as predictors of externalizing and internalizing child behavior problems in the contexts of high and low parental psychopathology. Infant Mental Health Journal. 2001;22(3):393–415. [Google Scholar]
  41. Muthen LK, Muthen BO. Mplus User’s Guide. Fourth Edition. Los Angeles, CA: Muthen & Muthen; 19982007. [Google Scholar]
  42. O’Malley PM, Bachman JG, Johnston LD. Reliability and consistency in self-reports of drug use. International Journal of Addition. 1983;18(6):805–824. doi: 10.3109/10826088309033049. [DOI] [PubMed] [Google Scholar]
  43. Patterson GR, Reid JB, Dishion TJ. Antisocial Boys. In: Jenkins JM, Oatley K, stein NL, editors. Human Emotions: A Reader. Malden, MA: Blackwell Publishing; 1998. pp. 330–336. [Google Scholar]
  44. Piacentini JC, Cohen P, Cohen P. Combining discrepant diagnostic information from multiple sources: Are complex algorithms better than simple ones? Journal of Abnormal Child Psychology. 1992;20(1):51–63. doi: 10.1007/BF00927116. [DOI] [PubMed] [Google Scholar]
  45. Pillow DR, Pelham WE, Hoza B, Molina BSG, Stultz CH. Confirmatory factor analyses examining Attention Deficit Hyperactivity Disorder symptoms and other childhood disruptive behaviors. Journal of Abnormal Child Psychology. 1998;26(4):293–309. doi: 10.1023/a:1022658618368. [DOI] [PubMed] [Google Scholar]
  46. Rothbart MK, Bates JE. Temperament. In: Damon W, Eisenberg N, editors. Handbook of child psychology: Social, emotional, and personality development. Vol. 3. New York: Wiley; 1998. pp. 105–176. [Google Scholar]
  47. Rothbart MK, Posner MI. Temperament, attention, and developmental psychopathology. In: Cicchetti D, Cohen DJ, editors. Developmental Psychopathology vol. 2: Developmental Neuroscience. Hoboken, New Jersey: John Wiley & Sons; 2006. pp. 465–501. [Google Scholar]
  48. Schafer JL, Graham JW. Missing data: Our view of state of the art. Psychological Methods. 2002;7:147–177. [PubMed] [Google Scholar]
  49. Schuckit MA. Research questionnaire. San Diego, CA: J.M Sattler Publisher; 1978. [Google Scholar]
  50. Tarter RE, Kirisci L, Feske U, Vanyukov M. Modeling the pathways linking childhood hyperactivity and substance use disorder in young adulthood. Psychology of Additive Behaviors. 2007;21(2):266–271. doi: 10.1037/0893-164X.21.2.266. [DOI] [PubMed] [Google Scholar]
  51. Van Leeuwen KG, Mervielde I, De Clercq BJ, De Fruyt F. Extending the spectrum idea: Child personality, parenting and psychopathology. European Journal of Personality. 2007;21:63–89. [Google Scholar]
  52. Watson D, Kotov R, Gamez W. Basic dimensions of temperament in relation to personality and psychopathology. In: Krueger RF, Tackett JL, editors. Personality and Psychopathology. New York: Guildford Press; 2006. pp. 7–38. [Google Scholar]
  53. Wills TA, Dishion TJ. Temperament and adolescent substance use: A transactional analysis of emerging self-control. Journal of Clinical Child and Adolescent Psychology. 2004;33(1):69–81. doi: 10.1207/S15374424JCCP3301_7. [DOI] [PubMed] [Google Scholar]
  54. Wong MM, Nigg JT, Zucker RA, Puttler LI, Fitzgerald HE, Jester JM, Glass JM, Adams K. Behavioral control and resiliency in the onset of alcohol and illicit drug use: A prospective study from preschool to adolescence. Child Development. 2006;77(4):1016–1033. doi: 10.1111/j.1467-8624.2006.00916.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  55. Zucker RA. Alcohol use and the alcohol use disorders: A developmental-biopsychosocial systems formulation covering the life course. In: Cicchetti D, Cohen DJ, editors. Developmental Psychopathology: Risk, Disorder, and Adaptation. Vol. 3. Hoboken, New Jersey: Wiley & Sons; 2006. pp. 620–656. [Google Scholar]
  56. Zucker RA, Ellis DA, Bingham CR, Fitzgerald HE, Sanford KP. Other evidence for at least two alcoholisms, II: Life course variation in antisociality and heterogeneity of alcoholic outcome. Development and Psychopathology. 1996;8:831–848. [Google Scholar]
  57. Zucker RA, Fitzgerald HE, Refior SK, Puttler LI, Pallas DM, Ellis DA. The clinical and social ecology of childhood for children of alcoholics: Description of a study and implications for a differentiated social policy. In: Fitzgerald HE, Lester BM, Zuckerman BS, editors. Children of Addiction: Research, Health and Policy Issues. New York: Garland Press; 2000. pp. 174–222. [Google Scholar]
  58. Zucker RA, Wong MM. Prevention for children of alcoholics and other high risk groups. In: Galanter M, editor. Recent developments in alcoholism, Vol. XVII: Alcohol problems in adolescents and young adults. Chapter 14. New York: Kluwer Academic / Plenum; 2005. pp. 299–319. [DOI] [PubMed] [Google Scholar]

RESOURCES