It is well known that children and adolescents present different problem behaviors as they grow into adulthood. Examples of externalizing and delinquent behaviors are persistent disobedience, stealing, aggression, vandalism, gang fighting, and homicide. Researchers have queried whether the successive development of these behaviors over many years is orderly rather than random, and what type of models best fit the fact that not all children escalate to the most serious externalizing acts. Recently, researchers have also queried whether some externalizing behaviors are precursors to internalizing behaviors. This paper reviews the state of research on developmental pathways in externalizing behaviors and their relation to the development of internalizing behaviors.
A developmental pathway is defined as the orderly behavioral development between more than two problem behaviors with individuals differing in their propensity to progress along the successive problem behavior represented by the pathway during development.1 Thus, pathways are a window into dynamic rather than static individual differences in youths' progression to serious problem behaviors. The study of developmental pathways also focuses on the need to identify the smallest number of pathways that best fit the development of externalizing behaviors in large numbers of youth (Loeber et al., 1993). The study of such pathways follows a tradition of study of developmental pathways in other areas of human functioning such as substance use (Yamaguchi & Kandel, 1984) and smoking (Flay, 1993; see also review by White, Jackson, & Loeber, 2009). The study of developmental pathways in externalizing behaviors is one of the methods towards the identification of steps that precede the emergence of very serious behaviors such as violence or costly forms of theft (see Loeber & LeBlanc, 1990, for a brief history of the study of developmental pathways in delinquency). It should be understood that the conceptualization of a developmental pathway consists of the formulation of probabilistic, testable model which does not have to be deterministic (i.e., stipulate that a given individual will become violent later in life).
The data on offense patterns show that by adulthood, delinquency outcomes of offenders are far from uniform. For example, a category of seemingly unspecialized violent chronic offenders emerge (Loeber & Farrington, 1998) as do more specialized adult-onset white collar criminals with little or no history of violent offenders, while a third group may consist of repeat property offenders without a history of violence but an onset in adolescence (Loeber, 1985). These diverse outcomes may reflect different underlying externalizing tendencies, expressed in distinctly different developmental pathways. However, some researchers proposed that development toward delinquency can be captured by a single pathway (e.g., Hirschi & Gottfredson, 1987; Jessor & Jessor, 1977; Osgood et al., 1988; Patterson et al., 1992; Robins, 1966) and have questioned whether there is a need to formulate and test several pathways. The answer to single or multiple pathways does not only depend on different outcomes as listed above but may also depend on whether externalizing behaviors are best represented along a single or multiple dimensions. We will come back later to this important issue.
Pathways differ from developmental types (e.g., early-onset vs. late-onset offenders, or life-course persistent vs. adolescence-limited offenders) in that pathways concern the successive development of problem behaviors in individuals, while developmental types concern categories of individuals with distinct developmental trajectories over time (Loeber, 1985; Moffitt, 1993). Developmental pathways have been studied by scholars from different disciplines, including psychiatry, criminology, and developmental psychology. Often these scholars studied the same individuals but at different developmental periods and by means of different kinds of nomenclatures and methods.
Developmental pathways can be thought of as a sieving process leading to the identification of those who develop a succession of specific problem behaviors versus those who develop fewer behaviors. Representations of developmental pathways tend to fall under the rubric of escalation models because less serious problem behaviors in developmental pathways tend to precede the more serious behaviors (Loeber & LeBlanc, 1990). Thus, developmental pathways chart how individuals progress from innocuous to very serious problem behaviors. Because violence as a rule emerges in middle to late adolescence, we are interested in the identification of individuals at risk for displaying violence years before this actually takes place. Thus, knowledge of developmental pathways by its documenting steps in pathways is highly relevant to assessment and preventive interventions. Rather than waiting for more serious problem behaviors to emerge, the documentation of steps within pathways, allows the identification at an early stage of those with some significant probability of escalating to serious problem behaviors. Assessment instruments based on knowledge of developmental pathways can also aid in the identification of those youth whose problem behavior is temporary vs. those whose problem behavior is likely to persist and escalate in severity (Loeber, Keenan, & Zhang, 1997; Loeber, Slot, & Stouthamer-Loeber, 2006).
We need to briefly review different methods for the identification of developmental pathways in increasing order from least to more empirical proof (Loeber & LeBlanc, 1990): (a) temporal difference in the average or median ages of onset of different behaviors; (b) prediction of the onset of one behavior by the presence of another, preceding behavior; (c) prediction of the change of one behavior by the change in another, preceding behavior; (d) demonstration that individuals' displaying one behavior have displayed one or more other behaviors earlier in life at a probability that is higher than chance (discussed in more detail below). Since scientific evidence for developmental pathways has not uniformly advanced to these criteria; we will also review less optimal levels of evidence. We also will review whether an early step in a developmental pathway is a necessary condition for a late step to occur (all or most cases who display behavior B have gone through behavior A at an earlier stage in life).
Also, it should be understood that the choice of measurements can reveal or conceal the presence of developmental pathways. For example, official records of delinquency are notorious underreporting of actual offending patterns (Elliott, Huizinga, & Egerton, 1985), and this may fail to indicate when onset of specific offense types takes place, thereby obfuscating developmental sequences between different forms of delinquency. Similarly, reliance on parent ratings of externalizing behaviors may lead to an underestimation of covert or concealing problem behaviors (as evident from juveniles' self-report), and this also become less valid indicators of onset and developmental sequences. For that reason, the best studies on developmental pathways rely on multiple sources and informants (e.g., Loeber et al., 1993).
This paper focuses on developmental pathways in externalizing and internalizing behaviors as studied by scholars in different disciplines, and addresses the following questions:
What are the developmental pathways between different diagnoses of Disruptive Behavior Disorders, including Oppositional Defiant Disorder (ODD), Conduct Disorder (CD), and Antisocial Personality Disorder (APD)?
What is a parsimonious model of pathways in externalizing problems that incorporates psychiatric symptoms of Disruptive Behaviors Disorders, other externalizing behaviors, and delinquent acts? And do the data best fit a single or multiple pathways?
Which pathways have been documented from externalizing to internalizing behaviors?
What are the limitations of past studies and what are the prospects of future studies on developmental pathways?
Developmental Pathways from Oppositional Defiant Behavior to Conduct Disorder and to Antisocial Personality Disorder
The Disruptive Behavior Disorders (DBD) as described in the DSM-IV (American Psychiatric Association, 1994) are a group of disorders characterized by behavior that is disruptive to the social environment and range from fairly benign annoying behaviors to the violation of rules or the infliction of harm on others. These disorders include Attention Deficit-Hyperactivity Disorder (ADHD), Oppositional Defiant Disorder (ODD) and Conduct Disorder (CD). Although they overlap in the sense that they involve impaired behavioral self-control, the disorders differ in their constituent symptoms and primary features. ADHD is reflective of impairments in impulse and behavioral control and in the marshalling of attentional resources. ODD describes a pattern of negativistic, hostile and oppositional behaviors in regards to authority figures and in social contexts. CD represents a pattern of rule- and norm-violating behavior, including status offenses such as truancy, as well as more severe behaviors such as assault and rape. The disorders have been demonstrated to be distinct from one another empirically, including in a number of factor analyses (see Burke, in press, for a review).
For this review we will highlight selected studies from the literature to illustrate the examination of pathways in the development of DBD, with reference to the methods of identifying developmental pathways described in the introduction. The core focus will be on the developmental pathway from ODD to CD to Antisocial Personality Disorder (APD). We will also introduce research on the role of ADHD in this pathway, as historically evidence tended to suggest a developmental sequence involving ADHD and CD specifically. There is no prescriptive developmental relationship between ADHD and the other DBD in the DSM-IV, but there is a strong developmental sequence described from ODD to CD, in part supported by the supposed progression from defiant and hostile interactions in ODD to more specific and severe theft, property destruction and physical assault among the symptoms of CD. Such a progression would be in accord with other models of the emergence and development of externalizing behavior described by disciplines outside of psychiatry and psychology (e.g. Loeber et al., 1993).
Evidence for the temporal difference in the ages of onset supports a developmental pathway from ODD to CD to APD. It is important to note that the criteria for these disorders in the DSM-IV (APA, 1994) are structured so as to introduce some developmental relationships (e.g. ODD cannot be diagnosed if the criteria for CD are met, since ODD symptoms are assumed to be present when CD criteria are met; APD criteria require some evidence of CD prior to age 15). The research we will describe here employs constructs created without these restrictions, in order to explore the nature of these relationships independently from diagnostic prescriptions.
Frick and colleagues (1993) found evidence that ODD symptoms largely clustered separately from CD symptoms in an overt, non-destructive quadrant in a two dimensional structure. They also reported evidence from parental retrospective report that the median age of onset in a clinical sample of these symptoms was earlier than CD symptoms. Using data from a large national survey of adults, Nock and colleagues (2007) reported that the onset of ODD typically precedes the onset of CD, and the onset of ODD is typically later than ADHD when the disorders co-occur for the same individual. Such retrospective data is useful to a point, but is subject to error due to potential biases in recall. This may be particularly true when self-report by adults of disorders with onset in childhood is sought, especially when such conditions may not be as readily recognized by the individual as others. Cross-sectional evidence from a large British community sample is consistent with an earlier age of onset and stable prevalence across ages of ODD, contrasted with low rates of CD in childhood and a marked increase with age into adolescence (Maughan, Rowe, Messer, Goodman & Meltzer, 2004). Prospective examination of the relative age of onset in a clinical longitudinal sample supported an earlier average age of onset of ODD compared to CD (Loeber, Green, Lahey, Christ & Frick, 1992).
Evidence from prospective studies of the predictive relationship between ODD and CD has tended to support a developmental sequence between the two (Lahey & Loeber, 1994; Lavigne et al., 2001; Loeber et al., 2000). While the majority of children with ODD do not progress to CD (Burke et al., 2002; Lahey et al., 1994), those with ODD are at higher risk for an onset of CD than children without ODD (Burke, Loeber & Birmaher, 2002; Loeber et al., 1995). In addition to predicting the onset of CD, changes in ODD symptoms predict corresponding changes in CD. In a clinical sample, even after accounting for the homotypic continuity of CD from one year to the next, higher ODD symptoms predict an increase in CD symptoms in the following year (Burke, Loeber, Lahey & Rathouz, 2005). Similarly, in a large representative community sample, ODD was a strong risk factor for later CD (Rowe, Maughan, Pickles, Costello & Angold, 2002), but for boys and not girls. On the other hand, using a large community sample of Dutch youth, van Lier and colleagues (2007) tested developmental trajectory models to examine relationships between ODD and CD. They found that 4% of boys and 2% of girls followed a high CD trajectory that started low in childhood and peaked in adolescence, whereas 6% of boys and 5% of girls followed a high ODD trajectory that started high in childhood and remained so throughout adolescence. In addition, about three-quarters of both boys and girls in the high ODD trajectory also followed the high CD trajectory.
When considering evidence of developmental pathways showing that individuals with CD have shown ODD in the past, some interesting methodological issues emerge. The DSM-IV suggests that when CD is present, ODD may be presumed to be present (APA, 1994), and in clinical studies, the proportion of youth meeting criteria for CD without concurrent or previous ODD is generally low (Burke, in press). However, studies based on community samples report higher proportions of youth with CD who do not have a history of ODD (Burke, in press). Substantial portions of youth with CD who have not met criteria for ODD suggests that ODD is not a necessary condition for CD, and that a developmental sequence from ODD to CD is pertinent only for a portion of cases of CD.
Among the disorders described in the DSM-IV, CD is unique due to its being expressly included in a pathway to a later personality disorder. APD requires evidence of CD before the age of 15 in order for criteria to be met, and is not itself diagnosed until age 18. Thus, a theoretical pathway from ODD to CD to APD is incorporated into the diagnostic schema of the DSM-IV. Given the criteria and descriptions of APD, mean age of onset cannot be studied as evidence for a developmental pathway. However, evidence from predictive studies has tended to support the DSM-IV developmental sequence: CD is a strong predictor of APD, whether measured in a clinical sample in childhood (Lahey et al., 2005) or adolescence (Loeber et al., 2002), or in a community sample (Copeland et al., 2009). The pathway to APD also appears to be typified by progression through CD; progression from ODD to APD independent of CD is rare (Burke, 2007; Copeland et al., 2009; Loeber et al., 2002).
The developmental relationship between ADHD and the other DBD is not expressly prescribed in the DSM-IV, and is not entirely clear in the literature. ADHD is clearly comorbid with ODD and CD (e.g. Angold et al,. 1999; Burke et al., 2002). ADHD has historically been regarded as a strong risk factor for CD, and many scholars continue to theorize that there is a direct relationship between ADHD and CD (e.g. Mannuzza, Klein, Abikoff, & Moulton, 2004). As researchers have more consistently and reliably considered ODD and CD distinctly within their analyses, evidence has accrued to suggest that the developmental relationship between ADHD and later CD is not direct, but relies on the presence of comorbid ODD (Burke et al., 2005; Lahey et al., 2000). More recently, Lahey and colleagues (2009), using a large representative sample, suggested that controlling for conduct problems at 4-7 years of age rendered negligible the prediction from both oppositionality and ADHD at 4-7 years of age to later conduct problems at 8-13 years of age. However, the constructs employed in that study were not DSM-based, and appeared to include behaviors more suggestive of oppositionality than conduct problems among the indicators of conduct problems. Thus, within that analysis, one could not rule out the possibility that early oppositionality remains predictive of later conduct problems over and above contemporaneous early conduct problems.
In summary, the evidence suggests that for ADHD, ODD, CD and APD, there are several developmental sequences (see Figure 1). However, although ADHD is often comorbid with, and may precede ODD, many children with ADHD will not develop ODD and conversely many children with ODD will not have had ADHD. The same is true for ODD and CD, even though they also show high levels of comorbidity, and when comorbid, show a clear developmental sequence. On the other hand, evidence suggests that most individuals with APD will have shown CD in the past, and that APD is a frequent outcome of CD.
Figure 1.
Developmental sequences in the Disruptive Behavior Disorders and APD.
Developmental Pathway from Minor Externalizing Behaviors to Serious Property Crime, Violence and Homicide
When one shifts from a few psychiatric diagnoses to a large number of specific externalizing problems, the identification of developmental pathways can become nearly overwhelming. For example, even with 20 different externalizing behaviors the number of possible combinations of developmental sequences is 2.43 × 1018. Only vast studies with ten thousands of participants studied over decades could possibly address such complexity. For that reason, it is essential to reduce complexity of possible developmental sequences by several types of data reduction (Loeber et al., 1993):
Find out whether factor analyses and meta-analyses of externalizing behaviors distinguished more than one factor. Evidence of two or more factors can then mean that the search for pathways should be one complex pathway or several distinct pathways.
Establish whether the age of onset curves for different behaviors are similar, providing a rationale for grouping such behaviors in a possible step in a pathway;
Establish whether age of groupings of behaviors differ in their age of onset, which then can lead to a preliminary formulation that there is a developmental pathway starting with behavior grouping 1 (called step 1) to behavior group n (called step n). One criteria for establishing a stage model is that transitions between stages follow an invariant sequence, but transitions within a stage can occur less systematically;
Compute back-ward probabilities between successive behaviors in developmental pathways. This means the calculation of the proportion of individuals who displayed problem behavior B by a certain age who also displayed problem behavior A at a younger age. If significantly more of those displaying behavior B also displayed behavior A at a younger age (compared to those displaying behavior B prior to behavior A), then one might speak of a developmental sequence from A to B (Loeber et al., 1993). Thus, backward probabilities (A given B) are important for the formulation of developmental pathways. Also, prospective, longitudinal data rather than cross-sectional or retrospective, longitudinal data are to be preferred for this step in the analyses of developmental pathways. Third, testing a model of developmental pathways should have a minimal number of individuals who skip steps in a pathway (e.g., behavior A is not followed by behavior B, but is followed by behavior C).
In the first stage of the analyses, Loeber and Schmaling (1985) undertook a multidimensional scaling of externalizing behaviors based on 28 factor analytic studies covering over 11,000 children. The main tool was to establish the extent to which two externalizing problems loaded on the same factor in factor analytic studies. The results showed a single dimension, one of overt, confrontational versus covert, concealing externalizing behaviors, with disobedience being situated at equal distance between overt and covert acts. Frick et al. (1993) repeated this analysis based on a larger sample of factor analytic studies, and confirmed the presence of the over-covert dimension of externalizing behaviors, but also documented a destructive vs. a nondestructive dimension. Since that time, the distinction between overt and covert externalizing behaviors has been accepted by researchers (e.g., Vassallo, Smart, Dussuyer, & McKendry, 2002).
Using the factor analytic evidence, we chose these overt and covert externalizing dimensions to create a pathway of different problem behaviors in each (called the Overt and Covert Pathways, which are discussed below). In addition, we thought it prudent to construct a pathway that concerned various forms of children's disobedience towards parents and teachers (called the Authority Conflict Pathway, which is discussed below). On the basis of research findings described below, we formulated three pathways in externalizing behaviors:
An Authority Conflict Pathway prior to the age of 12, that starts with stubborn behavior, and has defiance/disobedience as a second stage, and authority avoidance (i.e., truancy, running away from home, and staying out late at night) as a third stage;
A Covert Pathway starts prior to age 15 which begins with minor covert acts (shoplifting and frequent lying), has property damage as a second stage (i.e., vandalism and fire-setting), and moderate delinquency as a third step (i.e., fraud, pick-pocketing), and has serious delinquency as a fourth step (i.e., auto theft and burglary).
An Overt Pathway that starts with minor aggression (bullying, annoying others), has physical fighting as a second stage (physical fighting and gang fighting), and severe violence as a third step (rape, attack, strong-arm).
Analyses using longitudinal data from the Pittsburgh Youth Study (PYS; Loeber et al., 1993) for each of the three pathways showed that the development of externalizing problems in boys took place systematically rather than randomly, and best fitted three pathways rather than a single pathway. The boys typically followed an orderly progression from less to more serious problem behaviors from childhood to adolescence (Loeber, Keenan, & Zhang, 1997; Loeber, Wung, Keenan et al., 1993; Loeber, Pardini et al., 2005). Boys can be on each of the three pathways at the same time (thus, the pathways are not mutually exclusive). However, escalation in either the overt or covert pathway was often preceded by boys' escalation in the authority conflict pathway (Loeber, Wung et al., 1993). In other words, conflict with authority figures was either a precursor or a concomitant of boys' escalation in overt or covert acts. Also, an early age of onset of problem behavior or delinquency, compared to an onset at a later age, was more closely associated with boys' escalation to more serious behaviors in the Overt and Covert Pathways (Tolan et al., 2000).
The pathway model was evaluated in several ways. First, we computed the backward probabilities, that the extent to which individuals who had reached a later step in a pathway also had gone through the preceding steps (vs. reverse probabilities). The results showed that most of those who reached a higher step in a pathway also had manifested problem behaviors characteristic of a lower step in the pathway. This applied to each of the three pathways. Second, the pathway model accounted for the majority of the most seriously affected boys, that is the self-reported high rate offenders and court reported delinquents (Loeber, Keenan, & Zhang, 1997; Loeber, Wung et al., 1993). The pathways model has been documented in four other longitudinal data sets (Loeber, Wei et al., 1999; Tolan & Gorman-Smith, 1998; Tolan, Gorman-Smith, and Loeber, 2000), and has been features in publications of the Office of Juvenile Justice and Delinquency Prevention (Kelley, Loeber, Keenan, & DeLamatre, 1997) and the Centers for Disease Control (Dahlberg & Potter, 2001).
Experimenters and Persisters
Developmental pathways to serious forms of delinquency should be studied in the context of normal development, including the fact that many youth show some externalizing problems at some time in their early life. Several of the early steps in the pathways refer to the frequency and persistence of behaviors in such steps. An example is disobedience, which does not refer to rare incidents of disobedience but to a pattern of persistent disobedience. Therefore, in the evaluation of pathways, it is important to make a distinction between ‘experimenters’ (those who show a less serious problem at a low frequency) and persisters. Loeber, Keenan and Zang (1997) found that more persisters compared to experimenters entered a pathway at the first step. Also, fewer persisters compared to experimenters entered a pathway at a second or later step. Thus, the identification of pathways was better when the ‘the noise’ caused by experimenters was omitted from the data, and is improved by concentrating on persisters. This also means that interventions for persisters can be more reliably instituted at their entering the first step into a pathway.
Homicide as an Extension of the Violent Pathway
In the early stages of the development of the pathway model, very few of the boys in the PYS had committed homicide. Although we speculated that violence was a stepping stone toward homicide, this could only be demonstrated in recent years when the homicide rate among the young males in the PYS increased. Longitudinal analyses (Loeber et al., 2005) showed that 93.9% of the homicide offenders had been violent earlier in their life (as evident from self-report and official records of violence). Thus, homicide among these inner-city young males is an additional, fourth step in the Overt Pathway, with violence being a necessary condition for late homicide in most of the homicide offenders in this inner-city sample of young males.
Developmental Pathways in Girls' Externalizing Behaviors
The research reported thus far concerned males only. Do girls show similar pathways in externalizing behaviors? Gorman-Smith and Loeber (2005), using longitudinal data from the first five waves of the National Youth Survey of a nationally representative sample of girls. Similar developmental patterns of externalizing and delinquent involvement were found for girls as have been found for boys. Approximately 70% of involved girls followed most steps in each of the pathways.
The Role of Anger
Recently, Loeber et al. (in press) started to explore developmental pathways in girls from the Pittsburgh Girls Study (PGS) with a focus on trying to understand the pathway to anger in girls. Anger can occur with other externalizing behaviors but may also become manifest independent of externalizing behaviors. Averill (1983) proposed that different severity levels of anger, such as minor levels of annoyance (Averill, 1983) to which we add irritability, while anger is the most serious expression. Loeber et al. (in press) proposed that girls and women may inhibit the expression of anger, but that an accumulation of inhibited angry cognitions and emotions fuels a reservoir of angry emotions which eventually are more likely to become manifest in externalizing behavior.
The results of the analyses on data from the PGS documented a developmental pathway from irritability to anger. The majority of recurrently angry girls (88.8%) of those girls had gone through an earlier phase of highly irritable behavior. The reverse, irritable girls that have gone through an earlier period of anger were much less common (3.0%). In summary, irritability appears as a behavioral precursor to the onset of angry emotionality in a subsample of irritable girls. It is yet unclear why some girls and not others make the transition from irritability to angry emotionality.
Pathways to Early Externalizing Problems
Both the formulation of developmental pathways in externalizing problems for boys and girls started with data after preschool. This limitation of the preceding studies does not do justice to the fact that externalizing problems are often manifest during the preschool period (Goodenough 1931; Tremblay & Côté, 2005). Keenan and Shaw (2003) proposed a stage escalation model with two externalizing pathways for the preschool period. One pathway, called the pathway to reactive antisocial behavior starts with irritable behavior during infancy. Children displaying persistent irritability are thought to be at risk of developing emotional difficulties as toddlers (low frustration tolerance, over-activity and being demanding) who, in turn, are at risk of developing disruptive, angry behaviors as preschoolers, including reactive aggression, crying, whining and defiance. A second pathway, called the pathway to proactive antisocial behavior, starts with children displaying under-arousal (i.e., they are under-responsive to stimulation), who appear at risk of developing behavior difficulties as toddlers (as demonstrated by persistent unresponsiveness to punishment and a high level of sensation-seeking), who, in turn, are at risk of developing oppositional behaviors and conduct problems. The ‘Keenan and Shaw’ model may provide a link with the ‘Loeber’ pathway model in that the first two steps in the authority conflict pathway in the ‘Loeber’ pathway model (stubborn behavior and defiance/disobedience; Figure 2) may link to oppositional behaviors in the two ‘Keenan and Shaw’ pathways. Although the Keenan and Shaw pathways appear plausible, they need to be buttressed by empirical findings.
Figure 2.
Three developmental pathways in externalizing behaviors (Loeber et al., 1993).
Summary
Developmental pathways in externalizing behavior and delinquency and developmental sequences between different disruptive diagnoses share a conceptualization of escalation in the severity of externalizing behaviors with development in certain individuals but not in others. Thus, developmental pathways represent evolving individual differences with development. Along that line, developmental pathways represent selection processes in that increasingly smaller groups of youth become at risk for the more serious behaviors, comparable to a successive sieving process.
Development Pathways between Externalizing and Internalizing Problems
Much of the focus of the study of development from early behavioral problems has been on persistence of, or developmental sequencing of later antisocial or delinquent behavior (Loeebr et al., 1993; Loeber, Farrington, Stouthamer-Loeber, & White, 2008). However, less attention has been given to developmental pathways from early behavioral problems to other dysfunctional outcomes that are themselves impairing and harmful. Commonly occurring dysfunctions associated with impairment have often been roughly categorized as internalizing (causing difficulties primarily directed inwards, in terms of the individual's own functioning, such as depression or anxiety) or externalizing (causing difficulties primarily directed outward, such as conduct problems or aggression). Of course, these rough categorizations are in many ways inadequate. For instance, an individual's depression or anxiety also cause problems for others in a person's life, and youth with conduct problems suffer greatly themselves in addition to the problems they cause for others.
It has been observed that internalizing and externalizing problems often co-occur at greater than chance rates (e.g. Angold et al., 1999; Zoccolillo, 1992). This has led to a number of theories about how these types of problems may be related, including speculation that at least some co-occurrence reflected distinct diagnostic entities , such as Depressive Conduct Disorder (ICD-10, World Health Organization, 1992), a disorder that is not included in the DSM-IV (APA, 1994). However, much of this speculation arises from data collected in late childhood or adolescence, leading perhaps to an undue focus on conditions within the individual that manifest more prominently later in development. Additionally, a reliance on diagnostic definitions and diagnostic exclusion criteria may have limited the degree to which important etiological factors have been evident. Examining pathways from earlier in development without restriction to specific diagnostic requirements between internalizing and externalizing problems may provide alternative explanations and may clarify how some children with externalizing problems in childhood develop toward impairing internalizing problems in adolescence or adulthood. In fact some evidence for novel intriguing developmental models has already accrued from this type of approach.
Most of the intial work on this topic focused on depression and conduct disorder. In terms of temporal onset, findings from the Dunedin Longitudinal Study (Moffitt et al., 2001) indicated that Major Depressive Disorder typically emerges after the onset of CD in girls. Not only have the disorders been observed to frequently co-occur (e.g. Angold et al., 1999) but they have also been noted to fluctuate in synchrony (Lahey, Loeber, Burke, Rathouz & McBurnett, 1992). Although their co-occurrence was speculated to reflect a distinct diagnostic condition (Puig-Antich, 1982) which was codified as Depressive Conduct Disorder in the ICD-10 (World Health Organisation, 1992), the evidence has tended to argue against the existence of such a distinct disorder (Fergusson, Lynskey and Horwood, 1996; Zoccolillo, 1992).
It may be argued that adherence to diagnostic categorizations has hampered the ability to identify important developmental processes underlying the pathway from externalizing to internalizing problems. First, there has been a diagnostic prohibition against assigning both ODD and CD (APA, 1994), which often influenced researchers' operationalization of these constructs. However, testing ODD, CD and depression together has indicated that ODD, rather than CD, is associated with depression. In a clinical sample (Burke et al., 2005), assessed annually, increasing ODD was found to predict increasing depression over one year increments, but CD did not. This held after accounting for negative life events that are associated with conduct problems, such as removal from the home, being held back in school, being disliked by peers and other depressogenic consequences of disruptive behavior. Testing the reverse pathway found that increasing depression did not predict increasing ODD. The temporal ordering of the onset of ODD prior to the onset of depression and anxiety has been supported in other research (Boylan et al., 2007; Nock et al., 2007), and ODD has been found in a community sample to be a strong predictor of depression in young adulthood (Copeland et al., 2009).
Secondly, employing a dimensional approach rather than a strictly categorical diagnostic approach may help to further elucidate developmental processes. Specifically, if one conceptualizes ODD as strictly an “externalizing disorder of childhood”, one may obscure explanatory clues regarding outcomes other than externalizing problems. On the other hand, looking at the symptoms of ODD rather than the diagnosis reveals that some symptoms may describe more affectively oriented features. Burke (in press) found that in a longitudinal clinical sample, a dimension of negative affect (being touchy, spiteful and angry) within ODD was related to later depression and anxiety, whereas a dimension of oppositional behavior (arguing, being defiant, and losing one's temper) was related to later CD. In a community sample of girls (Burke, Hipwell & Loeber, 2010) concordant results were found in terms of the prediction from negative affect within ODD to later depression. Some differences emerged in the pathway from ODD symptom dimensions to CD; namely, for Caucasian girls, negative affective ODD predicted CD, whereas for African American girls, this was not the case. For all girls, the behavioral symptoms of ODD predicted later CD (Burke, Hipwell & Loeber, 2010). A largely overlapping construct derived from ODD symptoms and dubbed irritability has been examined by Stringaris and Goodman (2009), who found that irritability predicted later depression and anxiety in a community sample of boys and girls, and found that irritability indexed by ODD symptoms was also fairly specific in predicting depression and anxiety in adulthood (Stringaris, Cohen, Pine & Liebenluft, 2009).
In summary, examining developmental sequences involving ODD, CD and the internalizing disorders has helped to clarify the relationships between the disorders and to illustrate dimensions of ODD that may explain developmental sequences with depression and anxiety (see Figure 3). It appears that the links previously believed to exist between CD and internalizing disorders are not well supported, and that an affective dimension of ODD may instead precede depression and anxiety in a developmental sequence. Much work remains, however, to firmly establish these conclusions.
Figure 3.
Developmental pathway between ODD and anxiety and depression.
Early Markers for Emotional and Behavioral Dysregulation
The emerging evidence regarding dimensions of symptoms of ODD and their relationship to both internalizing and externalizing problems is important for a number of reasons. Among them is the possibility for improved prediction of later serious emotional and behavioral disturbance from an earlier developmental stage. This is particularly true in regards to predicting depression, which tends to onset later than ODD does. Keenan and Wakschlag (2002) have demonstrated that ODD and CD can reliably be distinguished from normative behavioral problems as early as preschool, and it has been shown that ODD during early childhood is predictive of persisting difficulties.
A separate area of research has a much more extensive history. Efforts to examine early childhood temperament have been ongoing for several decades. Early childhood temperament is believed to be the earliest developmental indicator of biologically driven individual differences in emotional and behavioral dysregulation (Lahey, Van Hulle, Keenan et al., 2008). Although a number of efforts have been made at linking childhood temperament to later psychopathology, firm evidence as to how temperament might best be understood in terms of later life-course emotional and behavioral dysregulation has been elusive. It remains unclear whether early temperament is best viewed as the first manifestation of later difficulties, or as perhaps as individual differences that interact with environmental factors to produce later distinct emotional and behavioral problems.
Lahey and colleagues (2008) demonstrated that it is possible to predict future conduct problems from measures of temperament taken as early as the first year of life. Specifically, infant fussiness, activity level and predictability during a child's first year predicted conduct problems measured across years 4 to 13. They further found that parenting in year 1 also predicted later conduct problems, and that interactions between maternal responsiveness and low infant fearfulness predicted later conduct problems. More specific evidence linking early temperament and later models of child psychopathology comes from work by Stringaris and colleagues (2009), who examined the prediction from measures of temperament at age 38 months to psychopathology measured at 91 months. There were specific links between temperamental emotionality and ODD comorbid with affective psychopathology, and between temperamental activity level and ODD comorbid with ADHD. In the context of previous evidence regarding the dimensions of ODD and their association with later behavioral or affective disorders, the findings from Stringaris et al., (2010) suggest that individual differences evident in the preschool years may provide even earlier distinguishing markers for diverging internalizing or externalizing dysregulatory pathways.
Limitations and Prospects
We described the state of research on developmental pathways in externalizing problems and their relevance for later serious forms of delinquency, but also for the development of internalizing problems. Pathways represent orderly development of a succession of different problem behaviors in a probabilistic framework. Pathways may help us to account for individual differences as they emerge over time. This knowledge, particularly the identification of youth along developmental pathways, can be helpful for the planning of interventions to curtail the escalation to more serious behaviors.
Problems and Issues
There are several limitations to the pathway models, which can be addressed in future research. One limitation is that only a restricted number of behaviors are included in several of the pathway models. For example, in the PYS, the reason is that the initial approach to identifying steps in the pathways depended on charting onset curves for different behaviors based on longitudinal data from the PYS (Loeber et al., 1993). Those behaviors that showed similar onset curves constituted the same step in the pathway. The onset of several externalizing behaviors was less systematic and, for that reason, was excluded from the formulation of the pathway model.
There are several issues that warrant more research in the future. Individuals' progression along developmental pathways does not appear driven by intra-individual characteristics only (such as impulsivity) but is influenced by the social environment as well (Haapasalo & Tremblay, 1994; Loeber et al., 2008), but research in this area can benefit from further empirical work explaining not only why some individuals progress on externalizing pathways and others less or not, and why individuals have different velocities of progression along developmental pathways.
The analyses of pathways in the PYS were inherently limited by the fact that the youngest boys in the study were age 6 at the beginning of the assessments. Although retrospective information from the parent was collected, this source of information often is incomplete and, therefore, prospective information from infancy needs to form the basis for future formulations of developmental pathways in externalizing behavior during the preschool period. Such work needs to take advantage from information about temperament as a precursor to externalizing behaviors (e.g., Frick & Morris, 2004; Keenan & Shaw, 2003).
Much more needs to be known about de-escalation models of change or inverse pathways, showing the order by which individuals shed problem behaviors as they improve or desist need to be researched (Loeber & LeBlanc, 1990). Pathways in externalizing problems should be seen in the context of incomplete development of social skills and competencies incompatible with externalizing problems. In addition, it is likely that developmental pathways overlap with poor moral development and cognitions favorable to rule and law breaking and victimization of others through physical harm and/or property loss.
Implications
We believe that knowledge of developmental pathways is relevant for assessment, preventive interventions, and the evaluation of interventions. This is the case no matter what child serving system is involved; schools, juvenile justice agencies and medical and mental health providers all struggle to find the most efficient ways to provide primary, secondary and tertiary modes of intervention. Assessments of juvenile externalizing behavior are justified when results are relevant for an appraisal of current behaviors and future risk. Knowledge of developmental pathways can help with the formulation of future risk of escalation by specifying two important parameters: (a) which behaviors are linked to later progression to serious outcomes; (b) which populations of youth are at risk of progressing to more serious acts. Ideally, assessments should take into account whether problems behaviors persist rather than be temporary (i.e., helping to make the distinction between experimenters and persisters).
Developmental pathways are also important for specifying targets in preventive interventions. For instance, if the presence of antisocial behavior or conduct problems as distinguished from more general defiance or oppostionality is key to the later emergence of CD, this would have dramatic implications for the search for reliable early markers of high risk for later severe externalizing behavior. That is, if severe antisocial behavior in late childhood or adolescence was predicted not by oppositionality or defiance but by more specific early antisocial or conduct problems, or by specific types of temperament or dimensions of dysregulation, it would be much easier to identify those in childhood likely to develop antisocial behavioral problems and avoid making false positive predictions. Accordingly, more well-targeted and specifically tailored prevention strategies could be delivered.
On the other hand, if antisocial behavior emerging later in development is not reliably predicted by early antisocial or conduct problems, but could develop from more generally oppositional and defiant behavior, or if it occurs frequently de novo in adolescence without any previous history of either conduct problems or oppositionality, reliable prediction of severe antisocial behavior becomes more complicated. These complications increase the rate of false positives when prevention is the goal. Furthermore, contributions to risk for antisocial behavior from contextual factors and transactions with others in the environment are likely increased if such problems emerge not in a direct homotypic fashion from early manifestations of antisocial behavior. This would also have the effect of complicating prediction, even though it could be regarded as providing for an increased ability to modify family and environmental factors to reduce the emergence and course of severe antisocial behavior.
Finally, developmental pathways can help to increase the ways by which intervention success is evaluated. Evidence suggests that developmental cascades, or related sets of developmental failures, describe individual pathways to undesirable outcomes. On the other hand, typically, an all-or-nothing formulation of treatment success is often used, but we argue that, instead, a relative change in individuals' propensity to escalate along a developmental pathway is a more refined and, possibly, a more developmentally informed intervention outcome.
Acknowledgments
This work is supported by grants from the Commonwealth of Pennsylvania (SAP 4100043365) to Drs. Loeber and Burke, the National Institute of Mental Health (MH 074148) to Dr. Burke, and (MH056630 and MH056630) to Dr. Loeber.
Footnotes
This definition is more appropriate than that given in Loeber, Keenan, and Zhang (1997), which defined a developmental pathway as the behavioral development of a group of individuals that is different from the behavioral development of other group(s) of individuals. A two-step developmental pathway usually is referred to as a developmental sequence.
References
- American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders. Washington, DC: American Psychiatric Association; 1994. [Google Scholar]
- Angold A, Costello EJ, Erkanli A. Comorbidity. Journal of Child Psychology and Psychiatry. 1999;40:57–87. [PubMed] [Google Scholar]
- Averill JR. Studies on anger and aggression. Implications for theories of emotion. The American Psychologist. 1983;38:1145–1160. doi: 10.1037//0003-066x.38.11.1145. [DOI] [PubMed] [Google Scholar]
- Boylan K, Vaillancourt T, Boyle M, Szatmari P. Comorbidity of internalizing disorders in children with oppositional defiant disorder. European Child & Adolescent Psychiatry. 2007;16:484–94. doi: 10.1007/s00787-007-0624-1. [DOI] [PubMed] [Google Scholar]
- Burke JD. The relationship between Conduct Disorder and Oppositional Defiant Disorder and their continuity with antisocial behaviors: Evidence from longitudinal clinical studies. In: Shaffer D, Leibenluft E, Rohde LA, editors. Externalizing disorders of childhood: Refining the research agenda for DSM-V. Arlington, VA: American Psychiatric Association; in press. [Google Scholar]
- Burke JD. Antisocial Personality Disorder. In: Bell A, Reinecke M, editors. Personality Disorders in Childhood. New York: Wiley; 2007. pp. 429–494. [Google Scholar]
- Burke JD, Hipwell AE, Loeber R. Dimensions of Oppositional Defiant Disorder as predictors of depression and Conduct Disorder in preadolescent girls. 2010 doi: 10.1097/00004583-201005000-00009. Manuscript submitted for review. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Burke JD, Loeber R, Birmaher B. Oppositional Defiant and Conduct Disorder: A Review of the Past 10 Years, Part II. Journal of the American Academy of Child and Adolescent Psychiatry. 2002;41:1275–1293. doi: 10.1097/00004583-200211000-00009. [DOI] [PubMed] [Google Scholar]
- Burke JD, Loeber R, Lahey BB, Rathouz PJ. Developmental transitions among affective and behavioural disorders in adolescent boys. Journal of Child Psychology and Psychiatry. 2005;46:1200–1210. doi: 10.1111/j.1469-7610.2005.00422.x. [DOI] [PubMed] [Google Scholar]
- Copeland WE, Shanahan L, Costello EJ, Angold A. Childhood and adolescent psychiatric disorders as predictors of young adult disorders. Archives of General Psychiatry. 2009;66:764–772. doi: 10.1001/archgenpsychiatry.2009.85. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Dahlberg LL, Potter LB. Youth violence. Developmental pathways and prevention challenges. American Journal of Preventive Medicine. 2001;20:3–14. doi: 10.1016/s0749-3797(00)00268-3. [DOI] [PubMed] [Google Scholar]
- Elliott DS, Huizinga D, Ageton SS. Explaining delinquency and drug use. Beverly Hills, CA: Sage; 1985. [Google Scholar]
- Flay RR. Youth tobacco use: Risks, patterns, and control. In: Orleans CT, Slade J, editors. Nicotine addiction: Principles and management. Vol. 1993. New York: Oxford University Press; 1993. pp. 365–384. [Google Scholar]
- Frick PJ, Lahey BB, Loeber R, Tannenbaum L, Van Horn Y, Christ MAG, Hart EA, Hanson K. Oppositional defiant disorder and conduct disorder: A meta-analytic review of factor analyses and cross-validation in a clinic sample. Clinical Psychology Review. 1993;13:319–340. [Google Scholar]
- Frick PJ, Morris S. Temperament and developmental pathways to conduct problems. Journal of Clinical and Adolescent Psychology. 2004;33:54–68. doi: 10.1207/S15374424JCCP3301_6. [DOI] [PubMed] [Google Scholar]
- Goodenough F. Anger in young children. Minneapolis, MN: University of Minnesota Press; 1931. [Google Scholar]
- Gorman-Smith D, Loeber R. Are developmental pathways in disruptive behaviors the same for girls and boys? Journal of Child and Family Studies. 2005;14:15–27. [Google Scholar]
- Haapasalo J, Tremblay RE. Physically aggressive boys from ages 6 to 12: Family background, parenting behavior, and prediction of delinquency. Journal of Consulting and Clinical Psychology. 1994;62:1044–1052. doi: 10.1037//0022-006x.62.5.1044. [DOI] [PubMed] [Google Scholar]
- Keenan K, Shaw DD. Development of conduct problems during the preschool years. In: Lahey BB, Moffitt TE, Caspi A, editors. Causes of Conduct Disorder and Juvenile Delinquency. New York: Guilford; 2003. pp. 153–181. [Google Scholar]
- Keenan K, Wakschlag LS. Can a valid diagnosis of disruptive behavior disorder be made in preschool children? American Journal of Psychiatry. 2002;159:351–358. doi: 10.1176/appi.ajp.159.3.351. [DOI] [PubMed] [Google Scholar]
- Kelley BT, Loeber R, Keenan K, DeLamatre M. Developmental pathways in disruptive and delinquent behavior. OJJDP Juvenile Justice Bulletin. 1997 www.ncjrs.org.
- Lahey BB, Applegate B, Barkley RA, Garfinkel B, McBurnett K, Kerdyk L, et al. DSM-IV field trials for oppositional defiant disorder and conduct disorder in children and adolescents. American Journal of Psychiatry. 1994;151:1163–1171. doi: 10.1176/ajp.151.8.1163. [DOI] [PubMed] [Google Scholar]
- Lahey BB, Loeber R. Framework for a developmental model of oppositional defiant disorder and conduct disorder. In: Routh DK, editor. Disruptive behavior disorders in childhood. New York: Plenum; 1994. pp. 139–180. [Google Scholar]
- Lahey BB, McBurnett K, Loeber R. Are attention-deficit hyperactivity disorder and oppositional defiant disorder developmental precursors to conduct disorder? In: Sameroff A, Lewis M, Miller S, editors. Handbook of developmental psychopathology. Second. New York: Plenum; 2000. pp. 431–446. [Google Scholar]
- Lahey BB, Van Hulle CA, Rathouz PJ, Rodgers JL, D'Onofrio BM, Waldman ID. Are oppositional-defiant and hyperactive-inattentive symptoms developmental precursors to conduct problems in late childhood? Genetic and environmental links. Journal of Abnormal Child Psychology. 2009;37:45–58. doi: 10.1007/s10802-008-9257-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lavigne JV, Cicchetti C, Gibbons RD, Binns HJ, Larsen L, DeVito C. Oppositional defiant disorder with onset in preschool years: longitudinal stability and pathways to other disorders. Journal of the American Academy of Child and Adolescent Psychiatry. 2001;40:1393–1400. doi: 10.1097/00004583-200112000-00009. [DOI] [PubMed] [Google Scholar]
- Loeber R. Patterns and development of antisocial child behavior. In: Whitehurst GJ, editor. Annals of Child Development. Vol. 2. Greenwich, CT: JAI Press; 1985. pp. 77–116. [Google Scholar]
- Loeber R, Burke JD, Lahey BB. What are adolescent antecedents to Antisocial Personality Disorder? Criminal Behaviour and Mental Health. 2002;12:24–36. doi: 10.1002/cbm.484. [DOI] [PubMed] [Google Scholar]
- Loeber R, Burke JD, Lahey BB, Winters A, Zera M. Oppositional defiant and conduct disorder: A review of the past 10 years, Part I. Journal of the American Academy of Child & Adolescent Psychiatry. 2000;39:1468–1484. doi: 10.1097/00004583-200012000-00007. [DOI] [PubMed] [Google Scholar]
- Loeber R, Farrington DP, Stouthamer-Loeber M, White HR. Violence and Serious Theft: Development and Prediction from Childhood to Adulthood. New York: Routledge; 2008. [Google Scholar]
- Loeber R, DeLamatre M, Keenan K, Zhang Q. A prospective replication of developmental pathways in disruptive and delinquent behavior. In: Cairns R, Bergman L, Kagan J, editors. Methods and models for studying the individual. Thousand Oaks, CA: Sage; 1998. pp. 185–215. [Google Scholar]
- Loeber R, Green SM, Keenan K, Lahey BB. Which boys will fare worse? Early predictors of the onset of conduct disorder in a six-year longitudinal study. Journal of the American Academy of Child and Adolescent Psychiatry. 1995;34:499–509. [PubMed] [Google Scholar]
- Loeber R, Green SM, Lahey BB, Christ MAG, Frick PJ. Developmental sequences in the age of onset of disruptive child behaviors. Journal of Child and Family Studies. 1992;1:21–41. [Google Scholar]
- Loeber R, Keenan K, Zhang Q. Boys' experimentation and persistence in developmental pathways toward serious delinquency. Journal of Child and Family Studies. 1997;6:321–357. [Google Scholar]
- Loeber R, Le Blanc M. Toward a developmental criminology. In: Tonry M, Morris N, editors. Crime and Justice. Vol. 12. Chicago: University of Chicago Press; 1990. pp. 375–473. [Google Scholar]
- Loeber R, Pardini D, Homish DL, Wei EH, Crawford AM, Farrington DP, Stouthamer-Loeber M, Creemers J, Koehler SA, Rosenfeld R. The prediction of violence and homicide in young men. Journal of Consulting and Clinical Psychology. 2005;73:1074–1088. doi: 10.1037/0022-006X.73.6.1074. [DOI] [PubMed] [Google Scholar]
- Loeber R, Schmaling K. Empirical evidence for overt and covert patterns of antisocial conduct problems. Journal of Abnormal Child Psychology. 1985;13:337–352. doi: 10.1007/BF00910652. [DOI] [PubMed] [Google Scholar]
- Loeber R, Slot NW, Stouthamer-Loeber M. A three-dimensional, cumulative developmental model of serious delinquency. In: Wikström POH, Sampson R, editors. The explanation of crime: Contexts and mechanisms. Cambridge: Cambridge University Press; 2006. pp. 153–194. [Google Scholar]
- Loeber R, Stouthamer-Loeber M, Hipwell AE, Burke JD, Battista D. Some Key Issues in the Early Development of Aggression in Girls. In: Pepler D, Craig W, editors. A Focus on Relationships: Understanding and Addressing Aggressive Behaviour Problems. 2009. in press. publisher to be announced. [Google Scholar]
- Loeber R, Wei E, Stouthamer-Loeber M, Huizinga D, Thornberry TP. Behavioral antecedents to serious and violent juvenile offending: Joint analyses from the Denver Youth Survey, Pittsburgh Youth Study, and the Rochester Youth Development Study. Studies in Crime and Crime Prevention. 1999;8:245–263. [Google Scholar]
- Loeber R, Wung P, Keenan K, Giroux B, Stouthamer-Loeber M, Van Kammen WB, Maughan B. Developmental pathways in disruptive child behavior. Development and Psychopathology. 1993;5:101–132. [Google Scholar]
- Mannuzza S, Klein RG, Bonagura N, Malloy P, Giampino TL, Addalli KA. Hyperactive boys almost grown up: Replication of psychiatric status. Archives of General Psychiatry. 1991;48:77–83. doi: 10.1001/archpsyc.1991.01810250079012. [DOI] [PubMed] [Google Scholar]
- Maughan B, Rowe R, Messer J, Goodman R, Meltzer H. Conduct disorder and oppositional defiant disorder in a national sample: developmental epidemiology. Journal of Child Psycholology and Psychiatry. 2004;45:609–21. doi: 10.1111/j.1469-7610.2004.00250.x. [DOI] [PubMed] [Google Scholar]
- Moffitt TE. Adolescence-limited and life-course-persistent antisocial behavior: A developmental taxonomy. Psychological Review. 1993;100:674–701. [PubMed] [Google Scholar]
- Moffitt TE, Caspi A, Rutter M, Silva PA. Sex differences in antisocial behaviour: Conduct disorder, delinquency, and violence in the Dunedin longitudinal study. Cambridge: Cambridge University Press; 2001. [Google Scholar]
- Nock MK, Kazdin AE, Hiripi E, Kessler RC. Lifetime prevalence, correlates, and persistence of oppositional defiant disorder: results from the National Comorbidity Survey Replication. Journal of Child Psycholology and Psychiatry. 2007;48:703–13. doi: 10.1111/j.1469-7610.2007.01733.x. [DOI] [PubMed] [Google Scholar]
- Rowe R, Maughan B, Pickles A, Costello EJ, Angold A. The relationship between DSM-IV oppositional defiant disorder and conduct disorder: findings from the Great Smoky Mountains Study. Journal of Child Psychology and Psychiatry. 2002;43:365–73. doi: 10.1111/1469-7610.00027. [DOI] [PubMed] [Google Scholar]
- Stringaris A, Goodman R. Longitudinal Outcome of Youth Oppositionality: Irritable, Headstrong, and Hurtful Behaviors Have Distinctive Predictions. Journal of the American Academy of Child and Adolescent Psychiatry. 2009;48:404–412. doi: 10.1097/CHI.0b013e3181984f30. [DOI] [PubMed] [Google Scholar]
- Tolan PH, Gorman-Smith D, Loeber R. Developmental timing of onsets of disruptive behaviors and later delinquency of inner-city youth. Journal of Child and Family Studies. 2000;9:203–230. [Google Scholar]
- Tremblay RE, Côté S. The developmental origins of aggression: Where are we going? In: Tremblay RE, Hartup WW, Archer J, editors. Developmental origins of aggression. New York: Guildford Press; 2005. pp. 447–464. [Google Scholar]
- Vassall S, Smart D, Sanson A, Dussuyer I, McKendry B. Patterns and precursors of adolescent antisocial behaviour. Melbourne: Australia: Crime Prevention Victoria; 2002. [Google Scholar]
- Van Lier PAC, van der Ende J, Koot HM, Verhulst FC. Which better predicts conduct problems? The relationship of trajectories of conduct problems with ODD and ADHD symptoms from childhood into adolescence. Journal of Child Psychology and Psychiatry. 2007;48:601–608. doi: 10.1111/j.1469-7610.2006.01724.x. [DOI] [PubMed] [Google Scholar]
- White HR, Jackson KM, Loeber R. Developmental sequences and comorbidity of substance use and violence. In: Krohn MD, Lizotte AJ, editors. Handbook on crime and deviance. Dordrecht, Netherlands: Springer; 2009. pp. 434–468. [Google Scholar]
- World Health Organization. International Statistical Classification of Diseases and Related Health Problem, Tenth Revision. Geneva: World Health Organization; 1992. [PubMed] [Google Scholar]