Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2014 Nov 30.
Published in final edited form as: J Am Acad Child Adolesc Psychiatry. 2012 Dec 12;52(1):26–36. doi: 10.1016/j.jaac.2012.10.001

Psychosocial Treatment Efficacy for Disruptive Behavior Problems in Very Young Children: A Meta-Analytic Examination

Jonathan S Comer 1, Candice Chow 1, Priscilla T Chan 1, Christine Cooper-Vince 1, Lianna AS Wilson 1
PMCID: PMC4247988  NIHMSID: NIHMS414131  PMID: 23265631

Abstract

Objective

Service use trends showing increased off-label prescribing in very young children and reduced psychotherapy use raise concerns about quality of care for early disruptive behavior problems. Meta-analysis can empirically clarify best practices and guide clinical decision making by providing a quantitative synthesis of a body of literature, identifying the magnitude of overall effects across studies, and determining systematic factors associated with effect variations.

Method

We used random-effects meta-analytic procedures to empirically evaluate the overall effect of psychosocial treatments on early disruptive behavior problems, as well as potential moderators of treatment response. Thirty-six controlled trials, evaluating 3,042 children, met selection criteria (mean sample age, 4.7 years; 72.0% male; 33.1% minority youth).

Results

Psychosocial treatments collectively demonstrated a large and sustained effect on early disruptive behavior problems (Hedges’ g = 0.82), with the largest effects associated with behavioral treatments (Hedges’ g = 0.88), samples with higher proportions of older and male youth, and comparisons against treatment as usual (Hedges’ g = 1.17). Across trials, effects were largest for general externalizing problems (Hedges’ g =0.90) and problems of oppositionality and noncompliance (Hedges’ g = 0.76), and were weakest, relatively speaking, for problems of impulsivity and hyperactivity (Hedges’ g = 0.61).

Conclusions

In the absence of controlled trials evaluating psychotropic interventions, findings provide robust quantitative support that psychosocial treatments should constitute first-line treatment for early disruptive behavior problems. Against a backdrop of concerning trends in the availability and use of supported interventions, findings underscore the urgency of improving dissemination efforts for supported psychosocial treatment options, and removing systematic barriers to psychosocial care for affected youth.

Keywords: externalizing problems, preschool, early childhood, parent training, aggression

Disruptive behavior disorders and related difficulties—characterized by problems of conduct and oppositionality—constitute one of the most prevalent classes of problems affecting children less than 8 years of age.1–4 Estimates suggest that one in 11 preschoolers meets formal criteria for a disruptive behavior disorder: one in 14 meets criteria for oppositional defiant disorder (ODD), and one in 30 meets criteria for conduct disorder (CD).3,5 Early disruptive behavior problems are reported across cultures,6 exhibit considerable stability,5,7–13 are associated with profound disability, and confer risk for later life psychopathology, family dysfunction, and criminality.14–16 Effective early intervention is critical.

National service use trends raise concerns about the quality of care for young children with disruptive behavior problems. In recent years, the proportion of very young children prescribed psychotropic medications in outpatient care has steadily increased.17–20 For example, between 1995 and 2001 there was a fivefold increase in the use of antipsychotic medications in Medicaid-insured 2- to 4-year-olds.21 From 1999–2001 to 2007, the rate of antipsychotic medication prescriptions to privately insured 2- to 5-year-olds with disruptive behavior disorders roughly doubled. Importantly, controlled evaluations of the efficacy of antipsychotic treatment for early child disruptive behavior problems have not been conducted. Potential adverse effects of antipsychotic treatment in youth, including metabolic, endocrine, and cerebrovascular risks, have been well documented.22,23 Although consensus guidelines accordingly recommend that psychosocial interventions constitute first-line treatment for preschool disruptive behavior disorders,24 the proportion of 2- to 5-year-olds receiving psychotherapy significantly decreased in recent years.18

The decreasingly prominent role of psychosocial interventions in the management of early disruptive behavior problems, and the increasing proportions of very young children receiving unsupported treatment regimens for these difficulties, collectively bring a sense of urgency to quantitatively synthesize and clarify that which we have learned from controlled evaluations of treatment for early disruptive behavior problems. Meta-analysis provides a quantitative synthesis of a body of empirical literature. By summarizing the magnitude of overall effects found across studies, determining systematic factors associated with variations in the magnitude of such relationships, and establishing relationships by aggregate analysis, meta-analytic procedures provide more objective, systematic, and representative conclusions than qualitative reviews.25

Although controlled evaluations of psycho-tropic interventions for disruptive behavior problems are lacking outside of an emerging literature on the effects of stimulant medications for early attention-deficit/hyperactivity disorder (ADHD), there is now a substantial body of rigorous empirical work evaluating the efficacy of various psychosocial treatments relative to control comparisons. The present study used meta-analytic procedures to empirically evaluate the overall effect of psychosocial treatments on early disruptive behavior problems, as well as potential moderators of treatment response which delineate the conditions under which a given treatment is related to outcome—i.e., moderators identify for whom and under which circumstances different treatments have different effects.26 Cross-literature meta-analytic moderation testing is essential to optimally informing clinical decision making by suggesting which patients may be the most responsive to particular treatments, and for which patients alternative treatments should be pursued.

METHOD

Study Selection Criteria

Studies published before January 1, 2012 that satisfied seven criteria were included. First, the clinical trial had to have prospectively examined a defined psychosocial treatment intentionally targeting disruptive behavior problems—including symptoms of externalizing behavior, aggression, oppositionality/noncompliance, and/or impulsivity/hyperactivity. Accordingly, studies examining incidental disruptive behavior outcomes after treatments targeting other clinical problems (e.g., depression) without an intentional impact on disruptive behavior were not included. Studies were included, however, across clinical populations when disruptive behaviors were in fact specifically targeted by the intervention (e.g., a treatment specifically targeting aggression in youth with pervasive developmental disorders). Retrospective evaluations and chart reviews were also not included. Second, the mean age of study participants had to be less than 8 years at baseline. Third, the study had to have entailed a randomized, between-subjects, controlled comparison. Open trials, nonrandomized designs, crossover designs, and comparisons of active treatments in the absence of a control condition were not included. Fourth, the sample size must have been large enough to afford statistical analyses (i.e., five or more subjects/condition). Fifth, the study must have included quantitative (not qualitative) analyses. Sixth, the study must have provided specific statistical information or enough data for the authors to obtain additional information to calculate the effect sizes needed for meta-analysis. Finally, for quality control, the study had to have undergone peer review (dissertations and data in book chapters were not included). Figure 1 presents a description of the flow of studies included.

FIGURE 1.

FIGURE 1

Open in a new tab

Flow diagram of the study selection process.

Several strategies identified studies satisfying these criteria: (1) computerized searches were conducted in MEDLINE and PsycINFO using keywords for youth, crossed with keywords for disruptive behavior problems, crossed with keywords for clinical trials (a list of all search terms used is available upon request); the references of articles found via computer search were reviewed for unidentified articles; tables of contents for the past 2 years of the study inclusion frame in journals that typically include clinical trials were reviewed (a list of these journals is available upon request); and a search was conducted by author name, using the names of known experts in the area.

Variable Coding

Eligible studies were reviewed and coded for study methodology, treatment, and child variables, as well as disruptive behavior symptoms. Mean age, percentage of male participants, and percentage of racial/ethnic minority youth were coded for each study. Individual effect sizes were extracted or computed for the following: aggression and serious rule violations; oppositionality/noncompliance; impulsivity/hyperactivity; and general externalizing symptoms (referring to total externalizing scores that combine elements of several of the previous categories (e.g., Child Behavior Checklist [CBCL] Externalizing Problems or Eyberg Child Behavior Inventory [ECBI] Total Score). An effect size for overall disruptive behavior symptoms was also computed, in which effect sizes for aggression, oppositionality/noncompliance, impulsivity/hyperactivity, and general externalizing symptoms were averaged into one pooled effect size per study and then pooled across studies.

Study Methodology Variables

Timing of posttreatment assessment referred to the average interval (in months) between the end of treatment and subsequent evaluation. Analyses also compared studies evaluating acute response (i.e., outcomes assessed upon immediate conclusion of treatment) versus follow-up response (i.e., outcomes assessed at a follow-up evaluation). Assessment mode referred to the format for assessing the dependent variable, and included the following three levels: structured observation; parent-report; and teacher-report. For additional analyses, the second level was further broken down into mother-report and father-report. Control group referred to the condition against which active treatments were compared, and included the following: no active ingredients (including no treatment, waitlist, and pill placebo); education, support, and attention controls (ESA); and treatment as usual (TAU).

Treatment Variables

Mean treatment length was computed for each study. Regarding treatment orientation, analyses compared studies that evaluated behavioral treatments versus nonbehavioral treatments. Behavioral interventions included those based on learning theory that placed contingency management and operant conditioning procedures at the center of treatment. Non-behavioral treatments included family systems approaches that did not explicitly incorporate learning theory and contingency management, as well as nondirective counseling approaches. Analyses also examined child involvement in treatment, referring to whether children did or did not attend treatment sessions. Treatments not including child involvement consisted of group parenting programs, individualized parenting programs, and teacher intervention programs. Treatments directly targeting children in session, as well as treatments in which children came to sessions but all directive interventions were focused on parents, were coded as entailing child involvement. Delivery format referred to whether treatment included group elements, or was individually delivered to each child/family.

Procedure

Coders were a clinical psychologist and three doctoral candidates in clinical psychology specializing in pediatric disorders. Training included didactics, practice coding, trained-to-criterion testing, and random, unannounced reliability checks. Didactic training included a 3-hour presentation on clinical trials evaluating early disruptive behavior problems, and the categories to be coded, augmented by handouts and additional meetings to discuss coding-related issues. Trainees spent 5 hours together as a group to practice coding five studies from the adolescent disruptive behavior disorders literature. These studies were selected for their inclusion of variables likewise included in the current meta-analysis. Coders were then each assigned three studies from the adolescent literature to code independently, and met with the first author to address discrepancies. Coders were then assigned five studies from the adolescent literature to code independently. Coders obtained acceptable inter-rater reliability on these studies and were thus deemed “trained-to-criterion” (i.e., 80% or greater reliability on all study codings). Studies included in the present meta-analysis were divided among the coders, with a randomly selected 5% of studies assigned to multiple coders. Coders met as a group once per month to address coding-related issues and to prevent potential rater drift. Inter-rater reliability was strong for the overlapping studies (all intraclass correlation coefficients > .80).

Data Analysis

Random-effects (RE) methods were used in the meta-analysis. Whereas fixed-effects (FE) methods assume that studies being analyzed have homogenous population effect sizes, RE methods assume that population parameter values will vary from study to study. In addition RE methods assume that studies represent a sampling of all of the possible studies that might be conducted or exist on a topic,27–30 and thus minimize the chances of inflating type I error rates regardless of whether the population parameter values are homogenous or heterogenous.31,32 RE models have been recommended over FE models as more accurate/realistic, as real-world data are likely to have heterogeneous population effect sizes even in the absence of known moderator variables.33

Effect sizes were calculated using Hedges’ g and its 95% confidence interval. Hedges’ g is a variation of Cohen’s d that corrects for biases due to sample sizes.29 Only one estimate of effect size was used per construct per study. This decision allowed samples to remain independent, rather than using several effect sizes from one study for a construct (e.g., a study using multiple measures of aggression), which could have created dependent samples and violated the assumptions of statistical analyses. To address this issue, multiple effect sizes for a single construct within single studies were averaged before synthesis with effect sizes from other studies. The magnitude of each pooled Hedges’ g was interpreted as follows: small effect (g = 0.2), medium effect (g = 0.5), and large effect (g = 0.8). To assess the significance of pooled effect sizes, Z scores were calculated for each pooled effect by dividing the pooled effect size by the standard error of that pooled effect. Z scores express the pooled effect size in terms of standard normal deviations, and a significance value (i.e., the probability of obtaining a Z-score of such magnitude by chance) can then be computed. The homogeneities of effect sizes were assessed with the Q statistic,29 which is designed to test whether observed variability across effects is greater than that which would be expected due to chance. In addition, a χ2 value was calculated to reflect the percentage of total between-studies variation in effect sizes due to heterogeneity. Values of 25% are considered low, 50% are considered moderate, and 75% are considered high.34 Values of 0% reflect complete homogeneity, and values of 100% reflect complete heterogeneity. Heterogeneity across effect sizes was expected, given the range of methodologies, outcomes, children, and treatments evaluated across trials.

Potential moderators of effects were evaluated for categorical variables by computing QBetween tests, which evaluate the extent to which effects systematically vary across different levels of variables (i.e., between groups), with significance reflecting moderation. For interpretation of identified moderators, significant QBetween tests were followed up with comparisons of pooled effects across the different levels of the variable. Meta-regression was used to evaluate potential moderators that were continuous.

Moreover, studies with significant findings are more likely to be published. The “file-drawer effect” is the probability that unpublished null findings would eliminate the obtained results. If studies that do not find significant results are not appropriately represented, publication bias may result. To evaluate publication bias, we conducted a trim and fill analysis for our evaluation of overall disruptive behavior symptoms. The trim and fill method35 offers a conservative estimate of publication bias by the following: evaluating funnel plot asymmetry and assuming that such asymmetry is due to publication bias; trimming the asymmetric positive results from the pooled estimation to calculate a hypothesized “true center”; replacing (“filling”) the trimmed positive effects observed in the literature, along with negative “counterparts” equal in magnitude; and estimating a new pooled estimate and surrounding confidence interval based on the newly symmetrical distribution of observed and imputed scores. The trim and fill method offers a highly conservative test because it assumes that the hypothetical missing values are not simply nonsignificant values, but rather that they are equal in magnitude but in the opposite direction from the asymmetrical positive values observed in the published literature. We considered the scenario in which the confidence interval surrounding the trimmed and filled estimate overlapped with the confidence interval surrounding the observed pooled estimate as evidence that publication bias did not significantly affect the observed pooled effect size.

RESULTS

Characterizing Literature on Psychosocial Treatment of Early Disruptive Behavior Problems

A total of 36 studies evaluating 3,042 children were identified that met selection criteria.36–71 Table 1 presents summary characteristics of identified studies. Regarding outcomes, one-third of studies presented outcomes on aggression and serious rule violations (κ = 12), 44.4% presented oppositionality/noncompliance outcomes (κ = 16), 41.7% presented hyperactivity/impulsivity outcomes (κ = 15), and 72.2% presented general externalizing outcomes (κ =26). Regarding design, 75% of studies included a control group with no active ingredients (κ = 27), 8.3% included a TAU control (κ = 3), and 19.4% included an ESA control (κ = 7). Roughly two-thirds of the studies assessed acute outcomes (κ = 25), and roughly one-third (36.1%) assessed follow-up outcomes (κ = 13). Regarding assessment mode, 22.2% included structured observations of outcomes (κ = 8), 83.3% included parent-reports (κ = 30), and 22.2% included teacher-reports (κ = 8). Most treatments studied were behavioral interventions (i.e., only 11.1% studied nonbehavioral interventions), and roughly half of the active treatments studied included group delivery elements (κ =19). Only slightly more than one-fourth of clinical trials directly involved children in treatment (κ = 10).

TABLE 1.

Characteristics of Randomized Clinical Trials Evaluating Psychosocial Methods for Early Disruptive Behavior Problems (κ = 36, N = 3,042)

Mean (SD) Range
Child characteristics
 Age, y 4.7 (1.5) 2.0–7.7
 % Male 72.0 (1.3) 37.0–100.0
 % Minority 33.1 (29.0) 1.0–100.0
Study characteristics
 Sample size 84.5 (48.5) 28–237
 Treatment length, wk 14.5 (7.9) 3–32
 Follow-up interval, mo 2.7 (4.1) 0–12
Open in a new tab

Overall Effect of Psychosocial Treatments on Early Disruptive Behavior Problems

Overall, psychosocial treatments had a large effect on early disruptive behavior problems (Hedges’ g = 0.82, SE = 0.10, 95% CI = .63–1.01, z = 8.63, p < .0001). This result was highly robust against the file-drawer effect (Fail-Safe N [FSN] =2,533). In addition, the confidence interval surrounding the adjusted effect calculated through the more conservative trim and fill analysis (trimmed studies = 1) overlapped almost entirely with the confidence interval surrounding the observed pooled estimate from the literature (adjusted CI = 0.57–0.98). Findings were consistent after removing the two studies that targeted disruptive behavior problems among youth with developmental disorders (data not shown). Figure 2 presents a funnel plot of effect sizes across the 36 studies. Despite the large pooled effect demonstrated overall, there was considerable heterogeneity in the magnitude of effects across studies (Q = 299.3, ι2 = 88.3), supporting the search for moderators that may systematically explain variability across study effects.

FIGURE 2.

FIGURE 2

Open in a new tab

Funnel plot of standard error by Hedges’ g drawn from randomized trials evaluating psychosocial methods for early disruptive behavior problems (κ = 36, N = 3,042).

Moderators of Psychosocial Treatment Effects on Early Disruptive Behavior Problems

Table 2 presents the results of analyses examining potential moderators of treatment effects, as well as pooled effects for each level of putative moderators. Breaking down disruptive behavior outcomes across the four symptom classes explained a significant amount of heterogeneity across treatment effects. All four classes of outcomes showed considerable effects, but it was general externalizing symptoms that showed the largest response, followed by oppositionality/noncompliance. Symptoms of impulsivity and hyperactivity showed the weakest treatment response relative to the other symptom classes, although the magnitude of effect on impulsivity/noncompliance still fell in the medium range.

TABLE 2.

Results of Analyses Examining Potential Moderators of Response to Psychosocial Treatments for Early Disruptive Behavior Problems (κ = 36, N = 3,042)

Categorical Moderator/Subgroup κ Hedges’ g SE 95% CI Z Test of moderation
Disruptive behavior problem class QBetween = 15.54, df = 3, p < .001
 Aggression and serious rule violations 12 0.71 0.16 0.40–1.03 4.47***
 Oppositionality/noncompliance 16 0.76 0.14 0.48–1.04 5.33***
 Impulsivity/hyperactivity 15 0.61 0.15 0.32–0.89 4.16***
 General externalizing symptoms 26 0.90 0.11 0.68–1.13 7.89***
Control group QBetween = 20.97, df = 2, p = .000
 No active treatment elements 27 0.84 0.11 0.63–1.05 7.92***
 Treatment as usual 3 1.17 0.32 0.55–1.78 3.71***
 Education/support/attention 7 0.47 0.19 0.33–0.61 6.45***
Timing of post-treatment assessment QBetween = 43.09, df = 1, p = .000
 Acute outcome evaluation 25 0.92 0.05 0.81–1.00 17.3***
 Follow-up evaluation 13 0.73 0.06 0.60–0.85 11.7***
Treatment orientation QBetween = 23.84, df = 1, p = .000
 Behavioral therapy 34 0.88 0.09 0.70–1.06 9.68***
 Non-behavioral therapy 4 0.42 0.25 −0.07–0.90 1.69
Child Involvement in treatment QBetween = 3.42, df = 1, p = .06
 Yes 10 0.83 0.17 0.50–1.17 4.87***
 No 32 0.82 0.10 0.63–1.02 8.27***
Treatment delivery format QBetween = 2.31, df = 1, p = .13
 Group elements 19 0.83 0.14 0.56–1.09
 Individualized for each child/family 13 0.76 0.17 0.42–1.10
Continuous Moderator κ β SE 95% CI Z Test of Moderation
Treatment length, wk 33 0.002 0.01 −0.002–0.01 1.03 Q = 1.06, df = 1, p =.30
Child age, y 34 0.10 0.01 0.07–0.12 8.12*** Q = 65.97, df = 1, p < .001
% Male youth 33 0.003 0.001 0.001–0.01 2.16* Q = 4.67, df = 1, p = .03
% Minority youth 15 −0.001 0.001 −0.004–01 −1.50 Q = 2.28, df = 1, p = .13
Open in a new tab

Note:

*

p < .05;

***

p < .001.

Study design elements significantly moderated treatment response. Effects varied depending on the type of control condition used, with TAU comparisons yielding larger effects than comparisons with no active treatment elements, which, in turn, yielded larger effects than ESA comparisons. Timing of post-treatment assessment also significantly moderated identified effects, with larger effects found immediately after treatment than at subsequent follow-ups. This pattern was also found when considering the follow-up interval continuously (β = 0.76, SE = 0.02, 95% CI = 0.72–0.80, Z = 39.2, p < .001). Nonetheless, the magnitude of pooled effects at follow-up evaluations was impressive, falling within the medium to large range. Effects were robust across assessment modes, although among parent-reported outcomes, effects drawn from mother-reports (Hedges’ g =0.72) were somewhat smaller than effects drawn from father-reports (Hedges’ g = 0.86, QBetween[df = 1] = 3.89, p = .05).

Treatment orientation significantly moderated outcome. Specifically, whereas behavioral treatments collectively demonstrated a large effect on disruptive behavior symptoms, nonbehavioral treatments (e.g., family systems approaches, nondirective counseling) demonstrated only small to medium effects. Treatment length did not moderate response. Similarly, treatment response was consistent regardless of whether children were directly involved in treatment, and whether or not treatment delivery incorporated group formats.

Age and gender moderated treatment response, with treatments showing larger effects in samples of older youth, and with higher percentages of males. Treatment effects were consistent across samples of varying compositions of racial/ethnic minorities.

DISCUSSION

The present meta-analysis quantitatively synthesized the empirical literature on controlled evaluations of psychosocial treatments for early disruptive behavior problems, calculating the pooled magnitude of overall effects across studies, and determining systematic factors associated with variations in the magnitude of such effects. Overall, psychosocial treatment options collectively demonstrate a large and sustained effect on early disruptive behavior problems (Hedges’ g = 0.82), with the largest effects associated with behavioral treatments (Hedges’ g = 0.88), samples with higher proportions of older and male youth, and comparisons against treatment as usual (Hedges’ g = 1.17). These findings provide robust quantitative support for consensus guidelines24 suggesting that psychosocial treatments alone should constitute first-line treatment for early disruptive behavior problems. Against a backdrop of reduced reliance on psychosocial treatments in this age range,18 and increased reliance on pharmacological treatments in the absence of controlled safety and efficacy evaluations,18 the present findings also underscore the urgency of improving dissemination efforts for supported psychosocial treatment options, and removing systematic barriers to psychosocial care for affected youth.

Behavioral treatments—such as Parent–Child Interaction Therapy (PCIT), Incredible Years, Helping the Noncompliant Child, and the Triple P–Positive Parenting Program—have been the most frequently studied form of psychosocial intervention. These behavioral treatments target child problems indirectly by reshaping parenting practices72–75 with the goals to increase in-home predictability, consistency and follow-through, and to promote effective discipline. These treatments help families disrupt negative coercive cycles by training parents to increase positive feedback for appropriate behaviors, to ignore disruptive behaviors, and to provide consistent time-outs for noncompliance. Although non-behavioral approaches have been less frequently studied, research to date suggests that efforts to disseminate psychosocial treatments and to reduce barriers to care for affected youth may yield greatest success by maintaining a primary focus on behavioral interventions.

Despite the observed progress in supported programs, gaps regrettably persist between treatment in experimental settings and services available in the community. In fact, our analyses found the greatest separation between active and control treatments when the comparison was TAU (Hedges’ g = 1.17), relative to comparison treatments with no active elements (Hedges’ g =0.84) or comparison treatments designed specifically for study purposes to control for the education, support, and attention offered in any psychosocial treatment (Hedges’ g = 0.47). Several factors may contribute to inadequate psychosocial care in practice settings. Inadequate numbers of professionals trained in evidence-based psychosocial programs, particularly in rural regions, can impinge on availability of care. Roughly 50% of U.S. counties have no psychologist, psychiatrist, or social worker.76 Long wait lists at underfunded clinics slow service delivery speed. Primary care providers typically fill this gap, but lack training and time to adequately address mental health needs (Comer JS, Barlow DH. The occasional case against broad dissemination: retaining a role for specialty care in the delivery of psychological treatments. Unpublished data). Supported treatments are not widely disseminated, whereas widely used approaches rarely show support.77 Moreover, when effective programs are disseminated, they are rarely implemented with fidelity,78,79 particularly when delivered by minimally trained professionals in overburdened facilities with high turnover. Given the increased prominence, credibility, and empirical support for telepsychiatry methods for reaching affected youth,80 technological innovations such as secure video conferencing for the delivery of treatments to families in nontraditional mental health settings (e.g., primary care physician offices, directly to homes) may offer transformative methods with which to overcome problems of local mental health care workforce availability and quality (Comer JS, Barlow DH. The occasional case against broad dissemination: Retaining a role for specialty care in the delivery of psychological treatments. Unpublished data). Future work is needed to evaluate the extent to which the large effect observed for psychosocial treatments for early disruptive behavior problems is robust across technology-based delivery formats with the potential to transcend traditional barriers to care. With group delivery formats showing comparable effects to individualized formats in the present meta-analysis, an emphasis on group treatment may further expand access by aiding cost-containment for payers and optimizing provider resources.

Treatment effects were somewhat heterogeneous across classes of disruptive behavior symptoms. General externalizing problems and problems of oppositionality and noncompliance showed the largest response to psychosocial treatments, whereas impulsivity and hyperactiv-ity showed relatively weaker reponses. Controlled evaluations suggest that there may be a role for stimulant medications in the management of these latter symptoms in preschool-aged youth,81 although stimulant effects in preschoolers may be somewhat smaller than effects in older youth. Given that psychosocial treatments nonetheless demonstrated a medium-sized effect on early impulsivitiy and hyperactivity symptoms, an adequate trial of psychosocial intervention should still be considered first-line treatment for young children presenting with these symptoms.24

Several limitations merit comment. First, as with any meta-analysis, the present findings speak to the population of treatments and affected youth of which the availabile literature is representative. Currently, the literature on psychosocial treatements for disruptive behavior problems has focused predominantly on family-based interventions for neurotypic youth, and so the present findings may not speak to less studied treatment options (e.g., school-based approaches) or to less studied populations seeking treatment for disruptive behavior (e.g., children with autism-spectrum disorders). Second, a number of key variables not included in the present analysis may play important roles (e.g., comorbidity, parent psychopathology, treatment adherence, therapist competence). Regrettably, very few clinical trials in this area have evaluated these variables. Although the work on these potentially important variables is not yet ready for meta-analysis, future clinical trials would do well to systematically incorporate these variables. Third, although file drawer analyses and trim and fill methods suggest that publication bias was not a problem in the estimation of overall effects, it is possible that the lack of significant effects associated with nonbehavioral treatments was due in part to the small proportion of trials (11.1%) that specifically evaluated nonbehavioral treatments. Fourth, we included only those data that had undergone peer review. There is much debate about the utility of including unpublished data in meta-analysis.82,83 Although we quantitatively examined the potential for publication bias, it is possible that including unpublished data would have yielded different findings. Finally, given the very limited number of controlled evaluations of psychotropic interventions using a parallel between-groups design, it was not possible to include pharmacologic interventions in the present meta-analysis. As noted, controlled evaluations suggest there may be a role under well-monitored circumstances for stimulant medications in the management of preschool impulsivity and hyperactivity.81 More controlled evaluations in this area will be needed before meta-analysis can be used as a tool to determine the extent to which stimulant effects on early disruptive behavior symptoms are robust.

Despite these limitations, the present meta-analysis offers a rare statistical portrait of the pooled effects of psychosocial treatments for early childhood disruptive behavior problems. Given the robust support for psychosocial interventions documented across the clinical trials literature, continuing trends away from psychosocial interventions for early disruptive behavior problems and toward off-label prescribing raises difficult challenges for payers, who must balance needs for treatment access, clinical flexibility, and prescriber autonomy with concerns over safety, costs, and quality of care. Off-label prescribing is not inherently cause for concern, particularly when patients present with symptoms for which rigorous clinical trials have not been conducted to evaluate treatment methods. However, the present findings empirically document a considerable literature of controlled trials supporting the large effects of psychosocial treatments, particularly those using behavioral methods, for the management of early disruptive behavior problems. Such findings bolster recent concerns regarding off-label psychotropic prescribing to very young children,84 particularly as first-line interventions, and call for the development of innovative methods to expand the availability and accessibility of evidence-based psychosocial treatments for early disruptive behavior problems and to accelerate the flow of affected youth into appropriate care.

Acknowledgments

This work was supported by National Institutes of Health (NIH) grant K23 MH090247 (PI: J.S.C.) and by a Charles H. Hood Foundation Child Health Research Award (PI: J.S.C.).

The authors are grateful to Ty Sawyer of Boston University for her helpful comments on earlier versions of this work.

Footnotes

Disclosure: Dr. Comer, Ms. Chow, Ms. Chan, Ms. Cooper-Vince, and Ms. Wilson report no biomedical financial interests or potential conflicts of interest.

References

  • 1.Baillargeon RH, Sward GD, Keenan K, Cao G. Opposition-defiance in the second year of life: a population-based cohort study. Infancy. 2011;16:418–434. doi: 10.1111/j.1532-7078.2010.00043.x. [DOI] [PubMed] [Google Scholar]
  • 2.Birmaher B, Ehmann M, Axelson DA, et al. Schedule for Affective Disorders and Schizophrenia for school-age children (K-SADS-PL) for the assessment of preschool children—a preliminary psychometric study. J Psychiatr Res. 2009;43:680–686. doi: 10.1016/j.jpsychires.2008.10.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Egger HL, Angold A. Common emotional and behavioral disorders in preschool children: presentation, nosology, and epidemiology. J Child Psychol Psychiatry. 2006;47:313–337. doi: 10.1111/j.1469-7610.2006.01618.x. [DOI] [PubMed] [Google Scholar]
  • 4.Keenan K, Wakschlag LS, Danis B, et al. Further evidence of the reliability and validity of DSM-IV ODD and CD in preschool children. J Am Acad Child Adolesc Psychiatry. 2007;46:457–468. doi: 10.1097/CHI.0b013e31803062d3. [DOI] [PubMed] [Google Scholar]
  • 5.Egger HL, Erkanli A, Keeler G, Potts E, Walter BK, Angold A. Test-retest reliability of the Preschool Age Psychiatric Assessment. J Am Acad Child Adolesc Psychiatry. 2006;45:538–549. doi: 10.1097/01.chi.0000205705.71194.b8. [DOI] [PubMed] [Google Scholar]
  • 6.Rescorla LA, Achenbach TM, Ivanova MY, et al. International comparisons of behavioral and emotional problems in preschool children: parents’ reports from 24 societies. J Clin Child Adolesc Psychol. 2011;40:456–467. doi: 10.1080/15374416.2011.563472. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Briggs-Gowan MJ, Carter AS, Bosson-Heenan J, Guyer AE, Horwitz SM. Are infant-toddler social-emotional and behavioral problems transient? J Am Acad Child Adolesc Psychiatry. 2006;45:849–858. doi: 10.1097/01.chi.0000220849.48650.59. [DOI] [PubMed] [Google Scholar]
  • 8.Keenan K, Shaw D, Delliquadri E, Giovannelli J, Walsh B. Evidence for the continuity of early problem behaviors: application of a developmental model. J Abnorm Child Psychol. 1998;26:441–452. doi: 10.1023/a:1022647717926. [DOI] [PubMed] [Google Scholar]
  • 9.Lavigne JV, Arend R, Rosenbaum D, Binns HJ, Christoffel KK, Gibbons RD. Psychiatric disorders withonset in the preschool years: I. Stability of diagnoses. J Am Acad Child Adolesc Psychiatry. 1998;37:1246–1254. doi: 10.1097/00004583-199812000-00007. [DOI] [PubMed] [Google Scholar]
  • 10.Lavigne JV, Cicchetti C, Gibbons RD, Binns HJ, Larsen L, DeVito C. Oppositional defiant disorder withonset in preschool years: longitudinal stability and pathways to other disorders. J Am Acad Child Adolesc Psychiatry. 2001;40:1393–1400. doi: 10.1097/00004583-200112000-00009. [DOI] [PubMed] [Google Scholar]
  • 11.Shaw D, Gilliom M, Ingoldsby E, Nagin D. Trajectories leading to school-age conduct problems. Dev Psychol. 2003;39:189–200. doi: 10.1037//0012-1649.39.2.189. [DOI] [PubMed] [Google Scholar]
  • 12.Tremblay R, Nagin D, Se ′guin J, et al. Physical aggression during early childhood: trajectories and predictors. Pediatrics. 2004;114:43–50. doi: 10.1542/peds.114.1.e43. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Wakschlag LS, Hill C, Carter AS, Danis B, et al. Observational assessment of preschool disruptive behavior, part I: reliability of the Disruptive Behavior Diagnostic Observation Schedule (DB-DOS) J Am Acad Child Adolesc Psychiatry. 2008;47:622–631. doi: 10.1097/CHI.0b013e31816c5bdb. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Burke JD, Waldman I, Lahey BB. Predictive validity of childhood oppositional defiant disorder and conduct disorder: implications for the DSM-V. J Abnorm Psychol. 2010;119:739–751. doi: 10.1037/a0019708. [DOI] [PubMed] [Google Scholar]
  • 15.Copeland WE, Miller-Johnson S, Keeler G, Angold A, Costello EJ. Childhood psychiatric disorders and young adult crime: a prospective, population-based study. Am J Psychiatry. 2007;164:1668–1675. doi: 10.1176/appi.ajp.2007.06122026. [DOI] [PubMed] [Google Scholar]
  • 16.Kim-Cohen J, Moffitt TE, Taylor A, Pawlby SJ, Caspi A. Maternal depression and children’s antisocial behavior. Arch Gen Psychiatry. 2003;62:173–181. doi: 10.1001/archpsyc.62.2.173. [DOI] [PubMed] [Google Scholar]
  • 17.Olfson M, Blanco C, Liu L, Moreno C, Laje G. National trends in the outpatient treatment of children and adolescents with antipsychotic drugs. Arch Gen Psychiatry. 2006;63:679–685. doi: 10.1001/archpsyc.63.6.679. [DOI] [PubMed] [Google Scholar]
  • 18.Olfson M, Crystal S, Huang M, Gerhard T. Trends in antipsychotic drug use by very young, privately insured children. J Am Acad Child Adolesc Psychiatry. 2010;49:13–23. doi: 10.1097/00004583-201001000-00005. [DOI] [PubMed] [Google Scholar]
  • 19.Olfson M, Marcus SC, Weissman MM, Jensen PS. National trends in the use of psychotropic medications by children. J Am Acad Child Adolesc Psychiatry. 2002;41:514–521. doi: 10.1097/00004583-200205000-00008. [DOI] [PubMed] [Google Scholar]
  • 20.Patel NC, Crimson ML, Hoagwood K, et al. Trends in the use of typical and atypical antipsychotics in children and adolescents. J Am Acad Child Adolesc Psychiatry. 2005;44:548–556. doi: 10.1097/01.chi.0000157543.74509.c8. [DOI] [PubMed] [Google Scholar]
  • 21.Zito JM, Safer DJ, Valluri S, Gardner JF, Korelitz JJ, Mattison DR. Psychotherapeutic medication prevalence in Medicaid-insured preschoolers. J Child Adolesc Psychopharmacol. 2007;17:195–204. doi: 10.1089/cap.2007.0006. [DOI] [PubMed] [Google Scholar]
  • 22.Maayan L, Correll CU. Weight gain and metabolic risks associated with antipsychotic medications in children and adolescents. J Child Adolesc Psychopharmacol. 2011;21:517–535. doi: 10.1089/cap.2011.0015. [DOI] [PubMed] [Google Scholar]
  • 23.Correll CU, Manu P, Olshanskiy V, Napolitano B, Kane JM, Malhotra AK. Cardiometabolic risk of second-generation anti-psychotic medications during first-time use in children and adolescents. JAMA. 2009;302:1765–1773. doi: 10.1001/jama.2009.1549. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Gleason MM, Egger HL, Emslie GJ, et al. Psychopharmacological treatment for very young children: contexts and guidelines. J Am Acad Child Adolesc Psychiatry. 2007;46:1532–1572. doi: 10.1097/chi.0b013e3181570d9e. [DOI] [PubMed] [Google Scholar]
  • 25.Field AP. Meta-analysis in clinical psychology research. In: Comer JS, Kendall PC, editors. Oxford Handbook of Research Strategies for Clinical Psychology. New York, NY: Oxford University Press; in press. [Google Scholar]
  • 26.Kraemer HC, Wilson T, Fairburn CG, Agras S. Mediators and moderators of treatment effects in randomized clinical trials. Arch Gen Psychiatry. 2002;59:877–883. doi: 10.1001/archpsyc.59.10.877. [DOI] [PubMed] [Google Scholar]
  • 27.Field AP. Meta-analysis of correlation coefficients: a Monte-Carlo comparison of fixed- and random-effects methods. Psychol Methods. 2001;6:161–180. doi: 10.1037/1082-989x.6.2.161. [DOI] [PubMed] [Google Scholar]
  • 28.Hedges LV. Meta-analysis. J Educ Stat. 1992;17:279–296. [Google Scholar]
  • 29.Hedges LV, Olkin I. Statistical Methods for Meta-Analysis. San Diego, CA: Academic Press; 1985. [Google Scholar]
  • 30.Hedges LV, Vevea JL. Fixed- and random-effects models in meta-analysis. Psychol Methods. 1998;3:486–504. [Google Scholar]
  • 31.Hunter JE, Schmidt FL. Methods of Meta-Analysis: Correcting Error and Bias in Research Findings. 2. Thousand Oaks, CA: Sage Publications; 2004. [Google Scholar]
  • 32.Lipsey MW, Wilson DB. Practical Meta-Analysis. Vol. 49. Thousand Oaks, CA: Sage Publications; 2000. Applied Social Research Methods Series. [Google Scholar]
  • 33.Field AP. Meta-analysis of correlation coefficients: a Monte-Carlo comparison of fixed- and random-effects methods. Psychol Methods. 2001;6:161–180. doi: 10.1037/1082-989x.6.2.161. [DOI] [PubMed] [Google Scholar]
  • 34.Higgins JPT, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta-analysis. Br Med J. 2003;327:557–560. doi: 10.1136/bmj.327.7414.557. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Duval S, Tweedie R. Trim and fill: a simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics. 2000;56:455–463. doi: 10.1111/j.0006-341x.2000.00455.x. [DOI] [PubMed] [Google Scholar]
  • 36.Bagner DM, Eyberg SM. Parent-child interaction therapy for disruptive behavior in children with mental retardation: a randomized controlled trial. J Clin Child Adolesc Psychol. 2007;36:418–429. doi: 10.1080/15374410701448448. [DOI] [PubMed] [Google Scholar]
  • 37.Bagner DM, Sheinkopf SJ, Vohr BR, Lester BM. Parenting intervention for externalizing behavior problems in children born premature: an initial examination. J Dev Behav Pediatrics. 2010;31:209–216. doi: 10.1097/DBP.0b013e3181d5a294. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Barkley RA, Shelton TL, Crosswait C, et al. Multi-method psycho-educational intervention for preschool children with disruptive behavior: preliminary results at post-treatment. J Child Psychol Psychiatry. 2000;41:319–332. [PubMed] [Google Scholar]
  • 39.Behan J, Fitzpatrick C, Sharry J, Carr A, Waldron B. Evaluation of the Parenting Plus Programme. Irish J Psychol. 2001;22:238–256. [Google Scholar]
  • 40.Brestan EV, Eyberg SM, Boggs SR, Algina J. Parent–child interaction therapy: parents’ perceptions of untreated siblings. Child Fam Behav Ther. 1997;19:13–28. [Google Scholar]
  • 41.Drugli M, Larsson B. Children aged 4–8 years treated with parent training and child therapy because of conduct problems: generalisation effects to day-care and school settings. Eur Child Adolesc Psychiatry. 2006;15:392–399. doi: 10.1007/s00787-006-0546-3. [DOI] [PubMed] [Google Scholar]
  • 42.Feinfield K, Baker BL. Empirical support for a treatment program for families of young children with externalizing problems. J Clin Child Adolesc Psychol. 2004;33:182–195. doi: 10.1207/S15374424JCCP3301_17. [DOI] [PubMed] [Google Scholar]
  • 43.Hanisch C, Freund-Braier I, Hautmann C, et al. Detecting effects of the indicated Prevention Programme for Externalizing Problem Behaviour (PEP) on child symptoms, parenting, and parental quality of life in a randomized controlled trial. Behav Cog Psychother. 2010;38:95–112. doi: 10.1017/S1352465809990440. [DOI] [PubMed] [Google Scholar]
  • 44.Hoath FE, Sanders MR. A feasibility study of enhanced group Triple P—Positive Parenting Program for Parents of Children with Attention-deficit/Hyperactivity Disorder. Behav Change. 2002;19:191–206. [Google Scholar]
  • 45.Kapalka GM. Avoiding repetitions reduces ADHD children’s management problems in the classroom. Emot Behav Difficult. 2005;10:269–279. [Google Scholar]
  • 46.Larsson B, Fossum S, Clifford G, Drugli M, Handegård B, Mørch W. Treatment of oppositional defiant and conduct problems in young Norwegian children: results of a randomized controlled trial. Eur Child Adolesc Psychiatry. 2009;18:42–52. doi: 10.1007/s00787-008-0702-z. [DOI] [PubMed] [Google Scholar]
  • 47.Lavigne JV, LeBailly SA, Gouze KR, et al. Treating oppositional defiant disorder in primary care: a comparison of three models. J Pediatr Psychol. 2008;33:449–461. doi: 10.1093/jpepsy/jsm074. [DOI] [PubMed] [Google Scholar]
  • 48.Matos M, Bauermeister JJ, Bernal G. Parent-child interaction therapy for Puerto Rican preschool children with ADHD and behavior problems: a pilot efficacy study. Fam Process. 2009;48:232–252. doi: 10.1111/j.1545-5300.2009.01279.x. [DOI] [PubMed] [Google Scholar]
  • 49.McCabe K, Yeh M. Parent-child interaction therapy for Mexican Americans: a randomized clinical trial. J Clin Child Adolesc Psychol. 2009;38:753–759. doi: 10.1080/15374410903103544. [DOI] [PubMed] [Google Scholar]
  • 50.Morawska A, Sanders M. An evaluation of a behavioural parenting intervention for parents of gifted children. Behav Res Ther. 2009;47:463–470. doi: 10.1016/j.brat.2009.02.008. [DOI] [PubMed] [Google Scholar]
  • 51.Nixon RV. Changes in hyperactivity and temperament in behaviourally disturbed preschoolers after parent-child interaction therapy (PCIT) Behav Change. 2001;18:168–176. [Google Scholar]
  • 52.Nixon RV, Sweeney L, Erickson DB, Touyz SW. Parent-child interaction therapy: a comparison of standard and abbreviated treatments for oppositional defiant preschoolers. J Consult Clin Psychol. 2003;71:251–260. doi: 10.1037/0022-006x.71.2.251. [DOI] [PubMed] [Google Scholar]
  • 53.O’Neal CR, Brotman L, Huang K, Gouley K, Kamboukos D, Calzada EJ, Pine DS. Understanding relations among early family environment, cortisol response, and child aggression via a prevention experiment. Child Devel. 2010;81:290–305. doi: 10.1111/j.1467-8624.2009.01395.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 54.Pisterman S, McGrath P, Firestone P, Goodman JT, Webster I, Mallory R. Outcome of parent-mediated treatment of preschoolers with attention deficit disorder with hyperactivity. J Consult Clin Psychol. 1989;57:628–635. doi: 10.1037/0022-006X.57.5.628. [DOI] [PubMed] [Google Scholar]
  • 55.Schuhmann EM, Foote RC, Eyberg SM, Boggs SR, Algina J. Efficacy of parent–child interaction therapy: interim report of a randomized trial with short-term maintenance. J Clin Child Psychol. 1998;27:34–45. doi: 10.1207/s15374424jccp2701_4. [DOI] [PubMed] [Google Scholar]
  • 56.Scott S, Sylva K, Doolan M, et al. Randomised controlled trial of parent groups for child antisocial behaviour targeting multiple risk factors: the SPOKES project. J Child Psychol Psychiatry. 2010;51:48–57. doi: 10.1111/j.1469-7610.2009.02127.x. [DOI] [PubMed] [Google Scholar]
  • 57.Shaw DS, Dishion TJ, Supplee L, Gardner F, Arnds K. Randomized trial of a family-centered approach to the prevention of early conduct problems: 2-year effects of the family check-up in early childhood. J Consult Clin Psychol. 2006;74:1–9. doi: 10.1037/0022-006X.74.1.1. [DOI] [PubMed] [Google Scholar]
  • 58.Sonuga-Barke ES, Daley D, Thompson M, Laver-Bradbury C, Weeks A. Parent-based therapies for preschool attention-deficit/hyperactivity disorder: a randomized, controlled trial with a community sample. J Am Acad Child Adolesc Psychiatry. 2001;40:402–408. doi: 10.1097/00004583-200104000-00008. [DOI] [PubMed] [Google Scholar]
  • 59.Sonuga-Barke ES, Thompson M, Daley D, Laver-Bradbury C. Parent training for attention deficit/hyperactivity disorder: Is it as effective when delivered as routine rather than as specialist care? Br J Clin Psychol. 2004;43:449–457. doi: 10.1348/0144665042388973. [DOI] [PubMed] [Google Scholar]
  • 60.Stolk MN, Mesman J, van Zeijl J. Early parenting intervention: family risk and first-time parenting related to intervention effectiveness. J Child Family Studies. 2008;17:55–83. [Google Scholar]
  • 61.Taylor TK, Schmidt F, Pepler D, Hodgins C. A comparison of eclectic treatment with Webster-Stratton’s parents and children series in a children’s mental health center: a randomized controlled trial. Behav Ther. 1998;29:221–240. [Google Scholar]
  • 62.Thompson MJ, Laver-Bradbury C, Ayres M, et al. A small-scale randomized controlled trial of the revised new forest parenting programme for preschoolers with attention deficit hyperactivity disorder. Eur Child Adolesc Psychiatry. 2009;18:605–616. doi: 10.1007/s00787-009-0020-0. [DOI] [PubMed] [Google Scholar]
  • 63.Turner KMT, Richards M, Sanders MR. Randomised clinical trial of a group parent education programme for Australian Indigenous families. J Paediatrics Child Health. 2007;43:429–437. doi: 10.1111/j.1440-1754.2007.01053.x. [DOI] [PubMed] [Google Scholar]
  • 64.Van Zeijl J, Mesman J, Van IJzendoorn MH, et al. Attachment-based intervention for enhancing sensitive discipline in mothers of 1- to 3-year-old children at risk for externalizing behavior problems: a randomized controlled trial. J Consult Clin Psychol. 2006;74(6):994–1005. doi: 10.1037/0022-006X.74.6.994. [DOI] [PubMed] [Google Scholar]
  • 65.Verduyn C, Barrowclough C, Roberts J, Tarrier N, Harrington R. Maternal depression and child behaviour problems: randomised placebo-controlled trial of a cognitive-behavioural group intervention. Br J Psychiatry. 2003;183:342–348. doi: 10.1192/bjp.183.4.342. [DOI] [PubMed] [Google Scholar]
  • 66.Webster-Stratton C. Randomized trial of two parent-training programs for families with conduct-disordered children. J Consult Clin Psychol. 1984;52(4):666–678. doi: 10.1037//0022-006x.52.4.666. [DOI] [PubMed] [Google Scholar]
  • 67.Webster-Stratton C. Enhancing the effectiveness of self-administered videotape parent training for families with conduct-problem children. J Abnorm Child Psychol. 1990;18:479–492. doi: 10.1007/BF00911103. [DOI] [PubMed] [Google Scholar]
  • 68.Webster-Stratton C, Hammond M. Treating children with early-onset conduct problems: a comparison of child and parent training interventions. J Consult Clin Psychol. 1997;65:93–109. doi: 10.1037//0022-006x.65.1.93. [DOI] [PubMed] [Google Scholar]
  • 69.Webster-Stratton C, Kolpacoff M, Hollinsworth T. Self-administered videotape therapy for families with conduct-problem children: comparison with two cost-effective treatments and a control group. J Consult Clin Psychol. 1988;56:558–566. doi: 10.1037//0022-006x.56.4.558. [DOI] [PubMed] [Google Scholar]
  • 70.Webster-Stratton C, Reid M, Hammond M. Treating children with early-onset conduct problems: intervention outcomes for parent, child, and teacher training. J Clin Child Adolesc Psychol. 2004;33:105–124. doi: 10.1207/S15374424JCCP3301_11. [DOI] [PubMed] [Google Scholar]
  • 71.Whittingham K, Sofronoff K, Sheffield J, Sanders MR. Stepping Stones Triple P: an RCT of a parenting program with parents of a child diagnosed with an autism spectrum disorder. J Abnorm Child Psychol. 2009;37:469–480. doi: 10.1007/s10802-008-9285-x. [DOI] [PubMed] [Google Scholar]
  • 72.Forgatch MS, Patterson GR. Parent-management training—the Oregon Model: an intervention for antisocial behavior in children and adolescents. In: Weisz JR, Kazdin AE, editors. Evidence-Based Psychotherapies for Children and Adolescents. 2. New York, NY: Guilford Press; 2010. [Google Scholar]
  • 73.McMahon RJ, Forehand RL. Helping the Noncompliant Child, Family-Based treatment for Oppositional Behavior. 2. New York, NY: Guilford Press; 2003. [Google Scholar]
  • 74.Webster-Stratton C, Reid MJ. The Incredible Years parents, teachers, and children’s training series: a multifaceted approach for young children with conduct disorder. In: Weisz JR, Kazdin AE, editors. Evidence-Based Psychotherapies for Children and Adolescents. 2. New York, NY: Guilford Press; 2010. [Google Scholar]
  • 75.Zisser A, Eyberg SM. Parent-child interaction therapy and the treatment of disruptive behavior disorders. In: Weisz JR, Kazdin AE, editors. Evidence-Based Psychotherapies for Children and Adolescents. 2. New York: Guilford Press; 2010. [Google Scholar]
  • 76.National Organization of State Offices of Rural Health. Statement on reducing behavioral health disparities in rural communities. Sterling Heights, MI: National Organization of State Offices of Rur al Health; 2011. [Google Scholar]
  • 77.Ennett ST, Tobler NS, Ringwalt CL, Flewelling RL. How effective is drug abuse resistance education? A meta-analysis of Project DARE outcome evaluations. Am J Public Health. 1994;84:1394–1401. doi: 10.2105/ajph.84.9.1394. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 78.Sandler I, Ostrom A, Bitner MJ, Ayers TS, Wolchik S, Daniels DS. Developing effective prevention services for the real world: a prevention service development model. Am J Community Psychol. 2005;35:127–142. doi: 10.1007/s10464-005-3389-z. [DOI] [PubMed] [Google Scholar]
  • 79.Weisz JR, Sandler IN, Durlak JA, Anton BS. Promoting and protecting youth mental health through evidence-based prevention and treatment. Am Psychol. 2005;60:628–648. doi: 10.1037/0003-066X.60.6.628. [DOI] [PubMed] [Google Scholar]
  • 80.AACAP Work Group on Quality Issues. Practice paramenter for telepsychiatry with children and adolescents. J Am Acad Child Adolesc Psychiatry. 2008;47:1468–1483. doi: 10.1097/CHI.0b013e31818b4e13. [DOI] [PubMed] [Google Scholar]
  • 81.Greenhill L, Kollins S, Abikoff H, et al. Efficacy and safety of immediate-release methylphenidate treatment for preschoolers with ADH. J Am Acad Child Adolesc Psychiatry. 2006;45:1284–1293. doi: 10.1097/01.chi.0000235077.32661.61. [DOI] [PubMed] [Google Scholar]
  • 82.Cook DJ, Guyatt GH, Ryan G, et al. Should unpublished data be included in meta-analyses? Current convictions and controversies. JAMA. 1993;269:2749–2753. [PubMed] [Google Scholar]
  • 83.Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D, Stroup DF. Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement. Lancet. 1999;354:1896–1900. doi: 10.1016/s0140-6736(99)04149-5. [DOI] [PubMed] [Google Scholar]
  • 84.Egger HL. A perilous disconnect: antipsychotic drug use in very young children. J Am Acad Child Adolesc Psychiatry. 2010;49:3–6. [PubMed] [Google Scholar]

RESOURCES