Putting Theory to the Test: Modeling a Multidimensional, Developmentally-Based Approach to Preschool Disruptive Behavior

Abstract

Objective

There is increasing emphasis on dimensional conceptualizations of psychopathology but empirical evidence of their utility is just emerging. In particular, while a range of multidimensional models have been proposed, the relative fit of competing models has rarely been tested. Further, developmental considerations have received scant attention. In this paper, we test a developmentally-based 4-dimension model of disruptive behavior theorized to represent the defining features of disruptive behavior at preschool age: Temper Loss, Noncompliance, Aggression, and Low Concern for Others.

Method

Model testing was conducted in two independent samples of preschoolers: Clinically-Enriched (N=336) and Epidemiologic (N=532). Tau-equivalent confirmatory factor analyses were used to test the fit of the Developmental Model relative to 3 leading competing models (DSM ODD/CD Model, “Callous” Model and an “Irritable/Headstrong/Hurtful” Model). Reliability of the 4 dimensions was also tested. Validity of the dimensions was tested by predicting multi-informant, multi-method ratings of disruptive behavior and impairment, and incremental utility relative to DSM symptoms.

Results

In both samples, the Developmental Model demonstrated a superior fit compared to the competing models within the full sample, and across key demographic sub-groups. Validity was also demonstrated, including incremental utility relative to DSM-IV disruptive behavior symptoms.

Conclusions

Critical next steps for achieving scientific consensus about the optimal dimensional model of disruptive behavior and its clinical application are discussed.

Dimensional approaches to psychopathology are an important complement to categorical classification systems^1-3. Categorical approaches have clinical utility but they necessarily reduce the complexity and heterogeneity of clinical phenomenology. Dimensional approaches are less parsimonious but have the advantage of identifying clinical patterns along a continuum of severity. This may be particularly useful in early childhood because emergent psychopathology may be milder, and distinctions from normative misbehaviors may best be captured as points along a dimension. Further, multidimensional approaches parse complex clinical phenotypes into distinct, component dimensions and allow for consideration of their pattern as clinical profiles. This enables identification of unique etiology and course, and provides critical information for targeted prevention.

The goal of this paper is to advance understanding of the phenotype of disruptive behavior in early childhood, one of the most common and earliest emerging developmental psychopathologies^4-6. We do so by testing a developmentally-based, multidimensional model in which: (a) core dimensions of disruptive behavior are conceptualized in terms of deviations from normative developmental processes in the regulation of emotion and behavior, and (b) behaviors are assessed in developmentally meaningful terms.

A number of studies have “parsed” the heterogeneity of disruptive behavior using categorical, subtype and dimensional approaches. Of course, the most commonly accepted approach is the DSM categorical distinction between Oppositional Defiant Disorder (ODD) and Conduct Disorder (CD). (This may not be a meaningful distinction at preschool age: psychometric evidence suggests a single disruptive behavior disorder (DBD) syndrome at this age⁷.) Seminal work by Frick and others has provided strong empirical support for a callous/unemotional sub-type of CD, including differentiation within early onset patterns^8-10. Most recently, Stringaris has applied a multidimensional approach to ODD and has demonstrated differential predictive utility of Irritable, Headstrong and Hurtful dimensions, a pattern replicated by others^11-15.

We are keenly aware that proliferation of models and labels can be vexing and confusing for the field. Thus, we introduce an alternative model here with some reluctance. Our rationale for doing so is that prior work has typically lacked a developmental conceptualization. While the proposed 4-dimension model draws heavily on prior work, its developmental framework is designed to characterize symptoms in a manner that can be meaningfully applied to young children, foster normative-atypical distinctions during this developmental period, and, ultimately, can be linked to underlying developmental processes that go awry in disruptive behavior. This approach reflects a core theoretical principle of the developmental psychopathology framework, i.e., disorder is viewed in terms of developmental deviation^16,17. The other organizing theoretical principle that undergirds this model is that clinical heterogeneity is important for characterizing meaningful phenotypic variation and differential etiologic pathways¹⁸.

Within our developmentally-based model, we theorize the four defining dimensions of DBDs as: (1) Temper Loss; (2) Noncompliance; (3) Aggression; and; (4) Low Concern for Others. Figure 1 provides a heuristic of this model illustrating the theorized developmental underpinnings of the dimensions and exemplars of their normative and clinical manifestations^19-22.

Figure 1. Heuristic Model of Disruptive Behavior Dimensions and Their Developmental Underpinnings.

The Temper Loss dimension reflects problems in regulation of overt anger including both temper tantrums and angry mood. The regulation of negative emotion is a core developmental task of early childhood and reflects the capacity to modulate the intensity and temporal features of emotional arousal in a goal-oriented manner^23-25. Whereas temper tantrums and transient irritability are common responses to frustration in early childhood, frequent, intense tantrums and pervasive negative mood are associated with clinically-significant problems at preschool age^26-28. The Noncompliance dimension reflects resistance to, and failure to comply with rules and social norms. Internalization of rules is a central developmental task of early childhood, including the capacity to shift behavior in response to environmental demands and to inhibit behavior in response to both internal and external controls ^19-22. Whereas noncompliance is a normatively expectable expression of autonomy at preschool age, its normative manifestations are goal-directed, flexible and tempered by a desire to please others^20,21. In contrast, clinical manifestations are characterized by recalcitrant defiance and deliberate rule-breaking ²⁹. The Aggression dimension reflects a tendency to respond aggressively in a variety of situations. Aggression emerges in the first year of life as a natural way of expressing anger and continues to be normative as a response to frustration and peer conflict throughout early childhood. Atypical forms include high frequency, hostile and proactive aggression, which is distinguishable from normative aggression by 18 months of age³⁰. The Low Concern for Others dimension reflects pervasive disregard of others’ needs and feelings. This dimension draws on extensive work on callousness in older youth³¹ but is conceptualized in terms of disruptions in the early development of empathy and conscience formation. While “self-centered” behavior is expectable in young children to some extent, concern over others’ distress and expressions of guilt when causing harm or displeasure to others are evident in the first years of life ^22,32-34. Atypical forms of Low Concern are hypothesized as intentionally causing others distress and purposeful provocativeness.

An additional limitation of prior work is that it has typically focused on validating a single model. This has contributed to a proliferation of unique models without a basis for identifying the best-fitting approach. We use rigorous psychometric modeling to test the relative fit of this proposed developmental 4-dimension model compared to the three leading models described above: a DSM ODD/CD Model, a “Callous” Model⁹ (distinguishing callous (Low Concern) from general disruptive behavior) and a model that parallels that originated by Stringaris (with Irritable, Headstrong and Hurtful dimensions)^12,13.

Specific aims of the study are to: (1) Test the comparative fit of the Developmental Model relative to these three alternative models in the Clinically-Enriched sample and replicate this in the Epidemiologic Sample. Establish whether model fit patterns are robust to variations in child sex, age, ethnicity and poverty; (2) Establish the reliability of the Developmental Model in the Clinically-Enriched sample and replicate it in the Epidemiologic Sample; (3) Examine the concurrent and predictive validity of the Developmental Model in the Clinically-Enriched Sample, using a multi-method, multi-informant approach. Replicate validity in the Epidemiologic Sample; and (4) Demonstrate the incremental utility of the Developmental Model relative to traditional DSM-IV DBD symptoms in the Clinically-Enriched Sample.

Ideally, these aims could be achieved via a study that: (a) employs developmentally-sensitive measures a priori designed to provide comprehensive coverage of each dimension; (b) utilizes rigorous psychometric methods to test the model fit and compare it to leading competing models: and (c) tests the incremental utility of the dimensions relative to a traditional DSM DBD diagnosis against a range of clinical and functional outcome measures at preschool age and beyond within a large representative sample.

To begin to test the developmental, multidimensional model, this study uses a secondary data approach drawing on two extant early childhood samples. The first sample is Clinically-Enriched and assesses the dimensions via data derived from a developmentally-enhanced clinical interview. The second is an Epidemiologic sample, with dimensional assessment derived from a developmentally-sensitive parent-report questionnaire about young children’s social-emotional problems. The use of secondary data departs from the ideal approach because of limitations of measurement and design inherent in the existing samples. First, although the existing measures provide reasonable coverage of key disruptive behavior constructs for early childhood, they were not designed to comprehensively assess the full spectrum of behaviors along the theorized dimensions. Second, the dimensions were derived from the available measures in the extant studies, which differed in both content and method of assessment. Third, the range of clinical outcome measures available varies across the samples and was limited in areas.

Despite these limitations, these two datasets provide a valuable opportunity to conduct a preliminary empirical test and replication of the theorized model. This will advance a developmentally-informed empirical knowledge base in three ways: (1) The use of psychometrically rigorous approaches serves an important integrative function because it enables testing the theorized model against key competing models in one sample and replicating it in a second independent sample; (2) Developmentally-sensitive measurement within early childhood importantly informs a developmental understanding of the dimensional approach; and (3) The samples provide preliminary data linking the dimensions to clinically-relevant outcomes concurrently and longitudinally, thereby providing initial evidence of their validity.

Method

Participants

Data were derived from two independent samples of preschoolers (for sample comparison see Table S-1, available online). The Clinically-Enriched Sample was comprised of 336 3-5 year old children (mean age= 4.47 years) recruited from clinics affiliated with two urban Midwestern universities for a study of preschool disruptive behavior in children from low-income environments.³⁵ This sample was predominantly African American (83.6%). Longitudinal follow-up was approximately 1 year post-baseline. A comparable test-retest sample of 31 preschoolers was also recruited. Participants for the Epidemiologic Sample were derived from a longitudinal study, where the sample was randomly selected from birth records.³⁶ The sample was ethnically diverse but predominantly Caucasian (74.7%). Because of the present study’s focus on disruptive behavior at preschool age, the analytic sample is comprised of the 532 children who were approximately age 3 at the second wave (mean age= 3.49 years), and who had complete data on the dimensional items. Longitudinal outcome data for this sample are derived from an early school age follow-up. This analytic sample was comparable to the excluded sample (n=687) in terms of child sex, parental education, poverty, and marital status but less likely to report minority ethnicity (30.1% vs. 39.3%, χ₁²=11.18, p<.001).

Measures

Disruptive behavior dimensions

In the Clinically-Enriched Sample, the dimensions were derived from ODD and CD symptoms assessed with the Kiddie-Disruptive Behavior Disorders Schedule (K-DBDS)³⁷. To derive the dimensions, a developmentally-enhanced approach was used including (a) exclusion of developmentally impossible and improbable CD symptoms, (b) the use of actual rather than subjective frequencies (ODD-0=never, 1=rarely, 2=a few times/month, 3=few times/week, 4=1-2 times/day, 5=Many times/day; CD-0=never; 1=once; 2=a few times; 3=weekly; 4=1x/day; 5=More than once/day), (c), developmental adaptation of wording, and (d) deconstruction of DSM symptoms to specify varying developmental manifestations in early childhood (e.g. “often defies” disaggregated into “ignores”, “breaks rules,” and “outright refuses”).³⁸

In the Epidemiologic Sample, the dimensions were derived from items on the Infant-Toddler Social and Emotional Assessment scale (ITSEA³⁹). The ITSEA is a developmentally-based parent-report questionnaire designed to assess socio-emotional problems and competencies in 1-3 year olds. It is a measure of developmentally relevant indicators of early problem emergence and competencies rather than a diagnostic measure. Dimensional items were selected from the ITSEA Externalizing Domain and Negative Emotionality scales and reverse coded items from the ITSEA Competence Domain. Items on the ITSEA were rated on a 3 point scale (0=Not True/Rarely; 1=Somewhat True/Sometimes; 2=Very True/Often) (see Table S-2 for composition of dimension by method, available online).

Validity measures

In the Clinically-Enriched Sample, three measures of criterion-rated validity were used: (1) parent- and teacher-rated impairment (baseline and 1 year follow-up); (2) directly observed disruptive behavior (baseline and 1 year follow-up); and (3) baseline clinical consensus diagnosis. Impairment was assessed by parent and teacher report on the non-clinician version of the Children’s Global Assessment Scale (CGAS)⁴⁰. Observed disruptive behavior was assessed during the Disruptive Behavior Diagnostic Observation Schedule (DB-DOS)^{26, 27}. The DB-DOS is an observational paradigm intended to distinguish the normative misbehavior of early childhood from disruptive behavior. To parallel CGAS ratings by parent and teacher⁴¹, observed disruptive behavior scores from the Parent- and Examiner-Engaged DB-DOS contexts were used. DB-DOS coders were blind to child disruptive behavior status.

A clinical consensus diagnosis method developed for this study was used to provide a standardized method for utilizing clinical judgment to integrate multi-source data about the child’s disruptive behavior at baseline (Shernoff, E., Hill, C., Danis, B., Leventhal, B., and Wakschlag, L. [2012]. Integrating assessment data across methods and contexts: A systematic approach to clinical decision making; unpublished data). Determination of a child’s clinical status (not disruptive, subclinical/subthreshold (has symptoms but not impaired or is impaired but does not meet symptom criteria) or disruptive (meets symptom criteria and is impaired)) were based on review of all salient measures obtained during the assessment (teacher data were obtained via questionnaire).³⁸ This included parent and teacher report of DBDs, other clinical problems and impairment, observed behavior in the clinic (DB-DOS and during developmental testing), standardized assessment of child developmental functioning, and family stressors. Clinical determination of DBD status was based on considering whether the behavior was outside of a developmentally expectable range, impairing, alternative clinical or developmental explanations (e.g. behaviors consistent with delayed language level), and using judgment to weigh discrepant information from varying sources. Inter-rater reliability was good (mean weighted kappa (Κ) = .84, range = .80 - .87).

In the Epidemiologic Sample, validity was tested at 3 years of age via parent report on the Child Behavior Checklist (CBCL) externalizing scales⁴² and the Family Life Impairment Scale (FLIS), a measure of the extent to which child behavioral/emotional problems limit the child and family’s participation in age-appropriate social occasions and daily-living activities by parent- and teacher report at early school age (Briggs-Gowan M, Horowitz S, Carter A. The Family Life Impairment Scale 1997. New Haven CT: Yale University; unpublished data). FLIS reliability (Cronbach alpha = .85) and test-retest reliability (r = .71) are acceptable.

Results

Aim I: Testing the hypothesized four-dimension solution

We employed tau-equivalent confirmatory factor analytic PROBIT models to test the relative fit of our hypothesized model to the three competing models. Tau-equivalent models test for the optimal number of dimensions by testing the comparative fit of models whose only difference is their dimensional structure⁴³. We tested our developmentally-based 4-dimension model (Model 1: “Developmental”) against: Model 2 (“DSM ODD/CD”)-a 2-dimension ODD/CD Model created by combining Temper Loss and Noncompliance as a single oppositional dimension and Aggression and Low Concern as a conduct dimension; Model 3 (“Callous”)-a 2-dimension model generated by combining Temper Loss, Noncompliance and Aggression in a single disruptive behavior dimension and retaining a distinct Low Concern (“Callous”) dimension; and Model 4 (“Irritable, Headstrong, Hurtful”-“IHH”) a 3-dimension model combining the Temper Loss and Aggression dimensions into a single “Irritable” Dimension, with Noncompliance (“Headstrong”) and Low Concern (“Hurtful”) retained as distinct dimensions.

The upper portion of Table 1 reports the relative fit of these models in the Clinically-Enriched Sample. In this sample, the Developmental Model fit significantly better than Model 2 (χ₇² = 60.8, p < .001), Model 3 (χ₇² = 35.1, p < .001), or Model 4 (χ₄² = 17.6, p < .01). Next, we replicated these model comparisons in the Epidemiologic Sample (lower portion of Table 1). In this sample, the Developmental Model also had a superior fit relative to Model 2 (χ₇² = 137.3, p < .001), Model 3 (χ₇² = 167.8, p < .001), and Model 4 (χ₄² = 112.8, p < .001).

Table 1. Model Comparisons.

Clinically-Enriched Sample
	Difference testsa
Model	X 2	df	p	Δ X 2	Df	p
Model 1: Developmental	385.6	180	< 0.001
Model 2: DSM ODD/CD	455.4	187	< 0.001	60.8	7	< 0.001
Model 3: DB/Callous	421.1	187	< 0.001	35.1	7	< 0.001
Model 4: IHH	401.5	184	< 0.001	17.6	4	< 0.010

Epidemiologic Sample (ITSEA Dimensions)
	Difference testsa
Model	X2	df	p	Δ X 2	df	X2
Model 1: Developmental	520.3	221	< .001
Model 2: DSM ODD/CD	764.0	228	< .001	137.3	7	< .001
Model 3: DB/Callous	847.5	228	< .001	167.8	7	< .001
Model 4: IHH	708.7	225	< .001	112.8	4	< .001

Note: Model 1: Temper loss, Noncompliance, Aggression, and Low Concern for Others Dimensions;

Model 2: Oppositional (ODD) Dimension (Temper Loss and Noncompliance) and Conduct (CD) Dimension (Aggression and Low Concern); Model 3: Disruptive Behavior (DB) Dimension (Temper Loss, Aggression, and Noncompliance) and Low Concern Dimension; Model 4: Irritable, Headstrong, and Hurtful (IHH) Dimensions (Temper Loss and Aggression, Noncompliance, Low Concern). ITSEA = Infant Toddler Social Emotional Assessment.

^aMplus Weighted Least Squares Mean Variance (WLSMV) estimator

We then fit tau-equivalent models across sub-groups defined by child sex, age, poverty status, and ethnicity as possible within the constraints of each of the samples. We tested whether the superior fit of the Developmental Model was due to sub-group differences. To do this, we constructed models in which the dimensional structure was held constant across subgroups. In these models, the factor variances were fixed and the factor loadings constrained to equality. Factor co-variances were allowed to differ between the sub-groups, as were factor means and item scale parameters.

Sub-group analyses also demonstrated a superior fit for the Developmental Model across sex and age in the Clinically-Enriched Sample and across sex, ethnicity and poverty status in the Epidemiologic Sample (data are shown in Table S-3, available online).

Aim II: Reliability and stability analyses

Descriptive statistics and correlations among the dimensions in both samples are reported in Table 2. In each sample, the correlations among the dimensions were of moderate and comparable magnitude (Clinically-Enriched Sample Mean r =.58, range= .52-.64; Epidemiologic Sample Mean r=.46, range =.38-.56).

Table 2. Descriptive Statistics and Correlations for Disruptive Behavior Dimensions.

	Descriptive Statisticsa
	Clinically- Enriched		Epidemiologic		Correlationsb
Dimension	Mean	SD	Mean	SD	1. Temper Loss	2. Noncompliance	3. Aggression	4. Low Concern
1. Temper Loss	1.67	1.33	.54	.37	-	.42	.56	.42
2. Noncompliance	2.45	1.32	.58	.39	.64	-	.46	.38
3. Aggression	.62	.71	.33	.35	.62	.59	-	.54
4. Low Concern	1.35	1.04	.37	.34	.57	.52	.57	-

^aNote:Means reflecting average scores of the items: Items on the Kiddie-Disruptive Behavior Disorders Schedule (K-DBDS) had possible scores ranging from 0-5 and on the Infant Toddler Social Emotional Assessment (ITSEA) from 0-2.

^bCorrelations for the K-DBDS Dimensions are on the subdiagonal and on the superdiagonal for the ITSEA Dimensions. All correlations are statistically significant (p<.001).

Table 3 reports on reliability and stability of the 4 dimensions. Internal consistency (Cronbach’s α) was high and comparable across samples (Mean α =.71 and .70 for Clinically-Enriched and Epidemiologic Samples respectively). One-week test-retest reliabilities in the Clinically-Enriched Sample were also high (Mean r =.87, range=.82-.92). Stability coefficients in this sample were all significant and of similar magnitude to the correlations (Mean r=.61, range .55-.66).

Table 3. Reliability and Stability of Dimensions.

			Test-Retest and Stability (Clinically-Enriched Sample)
	Internal Reliability (Cronbach’s α)		1 week test-retest (n=31)						1-year stability (n=307)
	Clinically- Enriched	Epidemiologic	Test		Retest
Dimension			mean	sd	mean	sd	p a	r b	r c
Temper loss	.68	.80	1.82	1.51	1.80	1.63	.854	.92	.59
Noncompliance	.77	.63	2.76	1.53	2.64	1.47	.400	.87	.66
Aggression	.80	.76	.79	.67	.72	.75	.384	.82	.62
Low Concern	.60	.60	1.48	1.39	1.50	1.28	.850	.86	.55

^aNote: p-value for test-retest comparison (paired t-test) indicating no significant difference.

^bPearson correlation coefficient between test and retest, p < .001 for each dimension.

^cPearson correlation coefficient between baseline and 1-year, p < .001 for each dimension.

Aim III: Criterion validity analyses

The final phase of the analyses involved testing the criterion-related validity of each individual dimension concurrently and longitudinally. Outcomes in the Clinically-Enriched Sample were parent- and teacher-reported impairment and observed disruptive behavior. In the Epidemiologic Sample, the outcome was the CBCL externalizing score, assessed by parent-report concurrently and parent- and teacher-report longitudinally and parent-report on the FLIS impairment scale at both timepoints.

In the Clinically-Enriched Sample, bivariate correlations between the dimensions and the validity measures at baseline and follow-up were generally significant and in the expected directions (see Table S-4, available online). For example, correlations between dimensions and CGAS scores were negative and significant (M=-.38, range = −.69-.09). Correlations between the dimensions and observed disruptive behavior were positive and generally significant (Mean r=.22, range .05-.36).

Results of regression analyses for the Clinically-Enriched Sample are shown in the upper half of Table 4. Each of the dimensions explained unique variance on at least one outcome in this sample, but findings varied by method/informant and wave. Temper Loss was consistently associated with parent- and teacher-reports of impairment but not with observed disruptive behavior. Noncompliance was consistently associated with parent-reported impairment and observed disruptive behavior with parent and examiner but was not associated with teacher-reported impairment. A generally similar pattern was observed for Aggression.

Table 4. Validity of the Four-Dimensional Model (Multivariate Regressions).

Clinically-Enriched Samplea
	Baseline				1-Year Follow-up
	CGASb		Observed DBc		CGAS		Observed DB
	Parent Report	Teacher Report	With Parent	With Examiner	Parent Report	Teacher Report	With Parent	With Examiner
Temper Loss	−.30***	−.34***	.02	.09	−.16*	−.30***	−.03	−.02
Noncompliance	−.25***	−.08	.24***	.18*	−.20**	−.04	.27***	.24**
Aggression	−.31***	−.02	.15*	.11	−.25***	.06	.27***	.19*
Low concern	−.06	.15*	−.09	−.15*	−.06	−.01	−.09	−.13+

Epidemiologic Samplea
	Age 3 Wave		Early School-Age Follow-Up
	CBCL Externalizing	FLIS Impairment	CBCL Externalizing		FLIS Impairment
	Parent report	Parent report	Parent report	Teacher report	Parent report
Temper Loss	.39***	.16**	.13*	−.06	.03
Noncompliance	.12***	.08+	.12*	.14*	.16**
Aggression	.32***	.26***	.32***	.25***	.14*
Low Concern	.06	.07	.04	.07	.05

Note: CBCL = Child Behavior Checklist; CGAS = Children’s Global Assessment Scale; DB = Disruptive Behavior; FLIS = Family Life Impairment Scale.

^aData are standardized coefficients from regression models including the four dimensions simultaneously

^bHigher score indicates better functioning.

^cCombined sums of Disruptive Behavior Diagnostic Observation Schedule (DB-DOS) Anger Modulation and Behavior Regulation Scores. Higher scores=more problems

⁺p < .10,

^*p < .05,

^**p < .01,

^***p < .001

Bivariate associations with Low Concern were in the expected direction. However, in multivariate models, Low Concern was positively associated with teacher-reported functioning and negatively associated with observed disruptive behavior with the examiner. This counterintuitive pattern suggests that the unique effect of Low Concern is negative relative to the shared effect of all dimensions. Thus, the bivariate associations are likely reflecting shared variance with the other dimensions⁴⁴.

Replication analyses with the Epidemiologic Sample (lower half Table 4) shows that bivariate correlations between the dimensions and parent- and teacher-reports of externalizing behaviors on the CBCL and parent report on the FLIS were significant and in the expected direction (Mean r=.35, range .16-.65). In multivariate analyses, Noncompliance and Aggression Dimensions were consistently associated with parent-reported externalizing problems and impairment. Temper Loss and Aggression were also uniquely associated with parent- and teacher-reported externalizing problems, respectively. However, Low Concern was not a significant predictor.

Aim IV: Incremental utility analyses

Finally, to test the incremental utility of the dimensional scores beyond traditional DSM DBD symptom measures, we conducted proportional odds logistic regressions in the Clinically-Enriched sample (model not shown). We first regressed the consensus diagnosis score (0=not disruptive, 1=subclinical/subthreshold, 2=disorder) on a dichotomous variable reflecting whether or not the child met DSM-IV DBD symptom criteria. This model had a −2 log L of 618.4. Next, the dimension scores were entered into the model. This full model returned a −2 log L of 476.4. This difference in −2 log likelihoods of 142.0 indicated a superior fit for the model which included the dimension scores and demonstrated that dimension scores significantly improved prediction (X₄²= 142.00, p < .0001). In particular, incremental variance in the clinical consensus diagnosis was explained by each of the dimensions over and above DSM-IV DBD symptoms. Higher scores on each of the dimensions significantly increased the odds of being in a clinical category. The four dimensional scores each follow a continuous scale with minimum of zero and maximum of five. Thus, odds ratios (OR) show the effect of a one unit increment in score on the odds of being in the next higher clinical category: Odds ratios (95% confidence intervals): Temper Loss OR= 1.6 (1.2, 2.1, p < .001), Noncompliance OR=1.8 (1.4, 2.3, p < .001), Aggression OR=3.5 (2.0, 6.3, p < .001), and Low Concern OR=1.4 (1.0, 1.9, p < .05).

Discussion

This study tested a theoretically-derived, developmentally-sensitive 4-dimensional structure for disruptive behavior: Temper Loss, Noncompliance, Aggression and Low Concern for Others. To our knowledge, it is the first to use robust psychometric methods to test competing dimensional models of disruptive behavior within an early childhood sample. Compelling regularity in the fit of this model was observed across independent Clinical and Epidemiologic samples, and across preschool age, sex, ethnicity and poverty status. While there are inherent limitations in testing a model utilizing data from extant samples, findings are an important first step for conducting more optimal tests of the model’s utility, using samples and measures explicitly designed for this purpose. On the other hand, the consistency of the patterns across samples and demographic subgroups provide a sound proof of concept for the fundamental robustness of the model. In particular, findings provide credible initial evidence that developmentally-based, multidimensional conceptualizations of disruptive behavior identify meaningful heterogeneity early in the emergence of disruptive behavior. Second, we demonstrated the validity of the dimensions, and provided important preliminary evidence of their “added value” in the clinical sample. In particular, each of the dimensions contributed unique variance in a clinician-derived DBD diagnosis, above and beyond DSM-IV DBD symptoms.

Dimensional approaches advance understanding of clinical phenomenology in two key ways. First, examining behavior dimensionally enables a more nuanced examination of patterns that span from normative misbehavior to impairing symptoms of disorders. Dimensionality is especially salient during early childhood, when the overlap of normative misbehavior and disruptive behavior pose substantial challenges to accurate identification. Because the present analyses utilized secondary data, our dimensional measurement was somewhat constrained. Development of dimensional measures that capture the full spectrum from salient normative misbehaviors to milder behaviors of clinical concern to frank clinical indicators is currently underway (Wakschlag, L., Briggs-Gowan., M., Choi, S., Hullsiek, H., Burns, J., McCarthy, K., Leibenluft, L., and Carter, A. [2012]. Defining the developmental parameters of temper loss in early childhood: Implications for developmental psychopathology; unpublished data). Second, developmentally-specified multidimensional approaches parse clinical heterogeneity into narrow-band component phenotypes that enable greater specificity in linkage to underlying mechanisms. For example, recently a Disregard for Rules dimension in early childhood demonstrated developmental stability and distinct genetic and environmental etiologies not evident when a broad-band disruptive behavior measure was used⁴⁵.

While the Developmental Model demonstrated a superior psychometric fit relative to leading alternative models, these findings are certainly not conclusive. Prior work on the leading models has been conducted primarily in older youth whereas the present models were tested in early childhood (we have previously demonstrated the superior fit of a parallel Developmental Model in an independent sample at adolescence).⁴⁶ Measurement limitations within the present study also constrain interpretation of findings. For example, although we have theorized that our Low Concern dimension mirrors prior work on callousness, this has not been empirically tested via validation in relation to established callousness measures. We have suggested that dimensional models will have greatest utility if they have lifespan coherence.⁴⁷ Establishing the optimal model requires systematic investigation across age periods, with psychometrically validated measures that have meaningful coverage of symptoms as developmentally-expressed at different stages of the lifespan.

The present findings are a “first look” at the fit and utility of a developmentally-sensitive, multidimensional model and are by no means the “definitive” test of its added value. A primary limitation of the present study is the use of secondary data analysis to derive the dimensions. Our outcomes were also limited in terms of their capacity to fully characterize clinically meaningful endpoints, particularly within the Epidemiologic sample. Further, the timespan covered in the longitudinal follow-up did not enable us to test predictive validity to older ages when clinical categorization is better validated. In addition, shared method variance resulting from mothers reporting on both the dimensions and some of the outcomes may have contributed to the higher correlations of the dimensions to parent-compared to teacher-reported outcomes. While most findings were in the expected direction, the pattern for the Low Concern Dimension was counter to that predicted in the Clinically-Enriched sample for observed and teacher-reported outcomes in multivariate analyses. While our consideration of Low Concern in these preschool samples is particularly novel (since it has been virtually ignored in prior studies of preschool DBDs), it was also the dimension least well covered with our extant measures because it was not a construct conceptualized when these measures were developed. Thus, investigation of the contribution of a developmentally-conceptualized Low Concern dimension is especially important for explicating its salience in emergent disruptive behavior. Replication and extension in large early childhood samples with measures explicitly designed for this purpose is needed to explicate these findings.

Despite these limitations, the robustness of these findings across samples and methods provides impetus for further investigation of this multidimensional approach. The present study provides measurement and predictive support for the multidimensional approach in general, and for our 4-dimension developmental model in particular. In early childhood, the most compelling evidence of the “added value” of a multidimensional approach would be (a) more precise characterization during this developmental period; (b) greater specificity as demonstrated via differential relation of various dimensions to discrete pathophysiologic and environmental mechanisms; and (c) identification of multidimensional profiles of disruptive behavior patterns that predict differential response to psychosocial and pharmacologic treatment. In terms of enhanced early identification, debates about revisions for DSM-5 highlight compelling questions about whether developmentally-specified symptoms have sufficient utility to merit the loss of parsimony that comes with a uniform symptom set across development^31,48. The present work provides a framework for empirically testing this question in early childhood. This should include testing the predictive utility of falling at varying points along the dimensions to establish a meaningful risk continuum. Minimally important differences^49,50 must also be established a priori to test whether incremental value is not only statistically, but also clinically significant. The answers to these questions, in turn, may serve as the basis for the next generation of novel interventions and early prevention efforts that specify targets based on developmentally-based multidimensional phenotypes of disruptive behavior.

A call to action

Dimensional approaches are receiving increasing prominence across clinical and clinical research arenas. In the domain of disruptive behavior, there is now substantial evidence that a dimensional approach has value for parsing clinical heterogeneity, specifying etiologic pathways and enhancing prediction of clinical course ^9,13,51. Combined with preparations for DSM-5¹ and the introduction of the Research Domain Criteria (RDoC)⁵² approach by the National Institutes of Mental Health, the field is now at a critical juncture for advancing a more refined, dimensional understanding of disruptive behavior phenomenology. As this body of work proliferates, however, the lack of coordination across proponents of alternative models ironically contributes to less, rather than more, clarity about what the defining dimensions of disruptive behavior are. The scientific community is now positioned to achieve broad-based, empirically-grounded consensus about the optimally informative and parsimonious dimensional model of disruptive behavior. While this is more easily said than done (as it requires scientists to transcend deeply held theoretical positions in the service of unifying the field)⁵³, the field is well-poised for this transformational leap. A key aspect of this transformation is catalyzing the transition of dimensional approaches from a “good idea” primarily employed in research, to a clinically-meaningful metric. This will require prioritizing the development of methods that maximize information gleaned from complementary categorical and dimensional approaches for clinical practice. Together these steps would ensure that the full potential of a dimensional approach can be realized for disruptive behavior syndromes in research and clinical practice.