Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2010 Sep 16.
Published in final edited form as: Am J Psychiatry. 2009 Nov 2;166(12):1392–1401. doi: 10.1176/appi.ajp.2009.09020233

Adjunctive Divalproex Versus Placebo for Children With ADHD and Aggression Refractory to Stimulant Monotherapy

Joseph C Blader 1, Nina R Schooler 1, Peter S Jensen 1, Steven R Pliszka 1, Vivian Kafantaris 1
PMCID: PMC2940237  NIHMSID: NIHMS182635  PMID: 19884222

Abstract

Objective

The purpose of the present study was to evaluate the efficacy of divalproex for reducing aggressive behavior among children 6 to 13 years old with attention deficit hyperactivity disorder (ADHD) and a disruptive disorder whose chronic aggression was underresponsive to a prospective psychostimulant trial.

Method

Children received open stimulant treatment during a lead-in phase that averaged 5 weeks. Agent and dose were assessed weekly and modified to optimize response. Children whose aggressive behavior persisted at the conclusion of the lead-in phase were randomly assigned to receive double-blind, flexibly dosed divalproex or a placebo adjunctive to stimulant for 8 weeks. Families received weekly behavioral therapy throughout the trial. The primary outcome measure was the proportion of children whose aggressive behavior remitted, defined by post-trial ratings of negligible or absent aggression.

Result

A significantly higher proportion of children randomly assigned to divalproex met remission criteria (eight out of 14 [57%]) than those randomly assigned to placebo (two out of 13 [15%]). Divalproex was generally well tolerated.

Conclusions

Among children with ADHD whose chronic aggressive behavior is refractory to optimized stimulant treatment, the addition of divalproex increases the likelihood that aggression will remit. A larger trial is necessary to specify with greater precision the magnitude of benefit for adjuvant divalproex.


Chronic aggressive behavior among preadolescents heightens their risk for unfavorable outcomes that extend well beyond youth into later life (1, 2). Ramifications are family strain and the child's extrusion from ordinary socialization experiences, leading to progressive social marginalization. High prevalence of these detriments to children's development and to those around them places aggression among the most frequent concerns for which children obtain mental health treatment (3).

Aggressive behavior may constitute a symptom, an associated feature, or complication of numerous psychiatric and developmental disorders (4). Because diverse factors influence the emergence of children's aggressive behavior (5, 6), practice guidelines recommend that clinicians plan pharmacotherapy in the context of the specific psychopathology thought to give rise to the behavior (7). Preadolescents who exhibit chronic aggressive behavior most often fulfill diagnostic criteria for a disruptive disorder (oppositional defiant disorder, conduct disorder) and for attention deficit hyperactivity disorder (ADHD) (8, 9). Stimulant medications for ADHD demonstrate strong efficacy for its core symptoms and often ameliorate comorbid disruptive disorders (10, 11). Accordingly, guidelines for the pharmacotherapy of aggressive children with ADHD propose that initial pharmacotherapy hew to strategies recommended for ADHD (12).

However, a substantial proportion of children with ADHD do not experience satisfactory reductions in aggressive behavior with stimulant treatment (13). High rates of multiagent treatment in the clinical care of children with ADHD and conduct problems (1416) also suggest that suboptimal stimulant response is common.

Chronic aggression among children with ADHD that is refractory to stimulant treatment therefore poses a common clinical challenge. Faced with insufficient response to stimulant monotherapy, one prevalent strategy is to, in effect, “layer” additional agents in pursuit of behavioral stability (1418). Prescription of antipsychotic and antimanic mood stabilizing medications for this purpose has surged in the past decade (19).

Controlled-trial evidence for these compounds as antiaggressive agents in youth comes chiefly from evaluations of monotherapy with the antipsychotic risperidone (2022), the antimanic/anticonvulsant divalproex (23, 24), and lithium (2527). Studies have involved patients with diverse psychopathology, including developmental disorders, disruptive disorders, and “explosive temper.” Some studies include aggressive behavior specifically as a treatment outcome, while in others, antiaggressive effects are inferred from evaluations of a broader spectrum of conduct or affective disturbances (e.g., irritability). Although studies with adults indicate the benefit of fluoxetine in treating intermittent explosive disorder (28), antidepressants currently do not have a major role in treating childhood aggression.

Although contributing important data, these studies do not specifically address the value of these agents for aggressive behavior in the large number of preadolescents with ADHD and a disruptive disorder who experience insufficient response to adequate first-line stimulant treatment. Two placebo-controlled trials of risperidone for children with mental retardation or borderline IQs permitted those receiving a stimulant before enrollment to continue the same regimen (20, 21). Among children with concurrent stimulant treatment, the addition of risperidone was more efficacious than the addition of placebo (29). However, stimulant doses were not titrated to insure that they represented the child's optimal dose.

In the present study, we examined the efficacy of divalproex in reducing aggressive behavior that was demonstrably refractory to stimulant monotherapy among children with ADHD. During the trial's development, guidelines proposed divalproex as an adjunctive option for aggression that was underresponsive to stimulant treatment (30). However, the magnitude of divalproex's benefit as cotherapy in this context was unexamined. Emerging data for divalproex's efficacy as monotherapy in reducing aggression in youths with diverse psychopathology (23, 24) support a randomized, controlled trial to fill this evidentiary gap.

Treatment began with an open lead-in phase of stimulant monotherapy along with family-focused behavioral management treatment. Systematic titration and monitoring were designed to optimize children's overall stimulant response. Children whose aggressive behavior persisted after this stimulant monotherapy phase were eligible for random assignment to adjunctive, double-blind treatment with divalproex or placebo for 8 weeks.

Method

Participants

Boys and girls between the ages of 6 and 13 years participated in the present trial. Eligibility required diagnoses of ADHD and either oppositional defiant disorder or conduct disorder. Other inclusion criteria were 1) severity of aggression surpassing an a priori threshold of 24 on the aggression assessment instrument used in this study (described below); 2) severity of parent-reported disruptive behaviors above a T score of 65, based on the aggressive behavior subscale of the Child Behavior Checklist (31); and 3) a T score ≥70 on the Conners' Global Index (32).

Children were ineligible if they had major depressive disorder, bipolar I disorder, Tourette's syndrome, any psychotic disorder, or an anxiety disorder that the evaluators judged to be the chief impetus for aggressive behavior (e.g., aggressive outbursts solely around separation from parents for a child with separation anxiety disorder). Pervasive developmental disorder and mental retardation were also exclusionary, recognizing that children with these conditions may have less robust responses to stimulants and perhaps to divalproex (33, 34). Health-related exclusions were medical factors contraindicating treatment with stimulants or divalproex, seizure disorders, pregnancy, or history of allergic hypersensitivity to the study medications.

Recruitment involved publicizing the study to families seeking service at the study's clinical sites and to community-based mental health and pediatric practitioners, through presentations to local school personnel, by word-of-mouth from other participants, and through local health-related publications.

Enrollment occurred between February 2004 and January 2007 at two academic medical centers (the first author participated in study visits at both sites). Each site's institutional review board approved the study and authorized continuations pursuant to annual reviews. Parents or legal guardians provided informed, written consent. Children over 8 years of age also provided their written assent.

Design and Procedures

Evaluation instruments and procedures

Diagnostic evaluation involved the Schedule for Affective Disorders and Schizophrenia for School-Age Children–Present and Lifetime Version (35), administered to the child's parent by a child and adolescent clinical psychologist. The other component of diagnostic evaluation was a clinical assessment by a child and adolescent psychiatrist who also interviewed the parent and child separately. Both assessors conferred to arrive at consensus diagnoses using all available data.

The Retrospective-Modified Overt Aggression Scale gauged the severity of aggressive behavior. This instrument is an adaptation of the Modified Overt Aggression Scale (24, 36, 37), replacing informants' open-ended estimation of each behavior's frequency with a Likert-type rating (e.g., 0 to 1 time, 2 to 4 times), scaled to reflect the greater severity of more harmful behaviors, even with a low frequency. The child's parent reported the frequency of 16 aggressive behaviors grouped into the following four areas: verbal aggression, physical aggression toward others, aggression toward one's self, and damage to or hostile misuse of property (e.g., throwing things). The four items within each category were also graded by severity. Items were weighted to amplify the seriousness of more harmful acts in the total score. The full text of this instrument and its scoring are available in the data supplement accompanying the online version of this article. Parents completed this aggression rating scale at baseline and at each weekly study visit. This measure displayed satisfactory psychometric properties, evidenced by internal consistency (Cronbach alpha=0.82), convergent validity (correlation with Child Behavior Checklist aggressive behavior subscale score=0.52), and divergent validity (correlation with Child Behavior Checklist internalizing subscale score=0.22).

The proportion of children in each treatment group who attained remission was the study's primary outcome measure. In addition to a reduction of ≥40% from a subject's prerandomization score on the Retrospective-Modified Overt Aggression Scale, we endeavored to identify an absolute score that would correspond to behavior that was markedly more manageable and did not necessitate further pharmacotherapy adjustments. We designated a total score of ≤10 for this criterion.

The Conners' Global Index–Parent Version quantified the severity of ADHD symptoms, yielding age- and gender-referenced T scores for its restless-inattentive subscale.

Parents completed the Barkley Behavior and Adverse Events Questionnaire–Modified (38) at baseline and each study visit to augment open-ended questioning. Items pertaining to tremor, bleeding abnormalities, disorientation, and menstrual changes were added to better represent potential adverse effects associated with divalproex. Weight, height, blood pressure, and pulse were obtained at each study visit. Safety assessments also were 1) examination of serum valproic acid levels, 2) complete blood cell count including leukocyte differential and platelets, 3) examination of liver enzymes, 4) examination of the pancreatic enzymes amylase and lipase, and 5) a serum pregnancy test for female participants of childbearing potential. Specimens for valproic acid were collected weekly during weeks 1, 2, 3, 4, 6, and 8. Phlebotomy for other analytes occurred at weeks 0 (baseline), 4, and 8.

Open stimulant optimization lead-in phase

Open treatment with stimulant monotherapy endeavored to identify the optimal agent and dose for each child. Titration began with a triphasic-release methylphenidate preparation (Concerta) given once daily, with daily dosage adjusted weekly based on the response of ADHD symptoms and tolerability to a maximum dose of 90 mg/day. Children who experienced adverse effects attributable to triphasic methylphenidate's lengthy duration of action (chiefly insomnia and appetite suppression) switched to a biphasically released methylphenidate product (Metadate CD) as clinically indicated, with a maximum daily dose of 60 mg. Children who gained little or no symptom improvement for ADHD with methylphenidate could switch to mixed amphetamine salts, administered once daily as a biphasic-release beaded preparation (Adderall XR), with a maximum daily dose of 35 mg.

Evaluation procedures involved weekly ratings obtained for each agent and dose administered to identify a well-tolerated regimen associated with the best symptom response. Parent ratings on the Conners' Global Index–Parent Version restless-inattentive subscale gauged ADHD symptom response. Stimulant “adequate response” was defined as a restless-inattentive subscale raw score that decreased by at least two standard measurement error units for the child's age and gender (32) and a Retrospective-Modified Overt Aggression Scale score <18, indicating that aggression abated sufficiently to leave little room for improvement with additional pharmacotherapy.

Children whose ADHD symptom ratings met the ADHD improvement criterion but whose aggressive behavior ratings remained above 18 were eligible to participate in the randomized, controlled trial. In addition to rating scale criteria, the decision to randomly assign a child to the cotherapy conditions required the corroborating judgments of the two clinicians (first author and study psychiatrist) and parental reconsent for this study phase.

Concurrent psychosocial treatment

All families had behaviorally oriented psychosocial treatment during the stimulant medication lead-in phase and the randomized, controlled trial phase. Advanced graduate students in clinical child psychology met with individual families weekly at regular study visits immediately after parents provided behavioral ratings and were seen by study clinician-evaluators. Treatment was based on the Community Parent Education Program (39), a family-based behavioral intervention. Core components are 1) goal setting; 2) increasing positive interactions with the child and rewarding cooperative behavior and composure; 3) judicious ignoring of low-level misbehaviors; 4) communicating directions and feedback for behavior; 5) a system to reward cooperation and improved frustration tolerance; 6) a school behavioral program in consultation with teachers; and 7) handling uncooperative and dyscontrolled behavior. The first author's weekly supervision of therapists at both clinical sites emphasized adherence to Community Parent Education Program principles and procedures as appropriate.

Randomization and titration of divalproex and placebo

The randomization schedule allocated children, in equal proportion, to treatment with either divalproex or a matching placebo. Each site had a separate order of assignment, randomly generated and stratified by gender using permuted blocks of four.

Children randomly assigned to divalproex received an extended-release once-daily preparation. Titration began with 250 mg each evening and was adjusted as needed by 250-mg increments to attain a daily dose of approximately 20 mg/kg, based on pre-randomization weight, by the end of week 2. Further dose adjustments followed a protocol based upon tolerability and behavioral response. A nonblind child psychiatrist assessed trough valproic acid blood levels and advised if dosage increases were permissible and if decreases were required. Children taking active divalproex could have increases as needed if valproic acid levels were below 80 mg/liter and reductions if levels exceeded 110 mg/liter. To maintain blinding, the nonblind psychiatrist ordered “sham” dose increases and decreases for an equivalent number of children taking placebo. The trial's final endpoint assessment occurred 8 weeks after randomization.

Data Analytic Approach

All children who completed at least one postrandomization assessment were included in intent-to-treat analyses. Preliminary analyses examined the distribution of outcome variables, checking for marked deviations from normality. Descriptive statistics characterized the sample's demographic features. Chi-square or t tests compared treatment groups on these factors.

The principal assessment of drug efficacy compared rates of aggression remission (Retrospective-Modified Overt Aggression Scale total score ≤10) between groups allocated to divalproex or placebo, with logistic regression incorporating the baseline score as a covariate. In addition, mixed-effects regression analyses compared time tendencies in aggression scale ratings by inclusion of a week-by-treatment interaction term. All data were utilized, with those unavailable assumed as missing at random. High variance and skew undermined the appropriateness of parametric tests using raw aggression scale scores. Logarithmic transformation normalized scores adequately to perform these analyses.

Effect size computation employed the approach to standardized change units for repeated-measures designs described by Dunlap et al. (40) to compute Cohen's d for correlated data.

The Barkley Behavior and Adverse Effects Questionnaire–Modified items were dichotomized to allow for scores >4, representing moderate severity, to count as positive for the adverse effect. Chi-square tests compared treatment groups with no correction for multiple tests and with the p value set liberally at 0.10 to err on the side of overdetection of adverse effects.

Results

Participant Sample Derivation and Characteristics

Figure 1 (CONSORT chart) illustrates the subject flow through the study. Of the 30 children eligible for random assignment, 15 were allocated to receive adjunctive divalproex and 15 to receive adjunctive placebo, with 14 and 13, respectively, completing at least one postrandomization assessment for inclusion in intent-to-treat analyses.

FIGURE 1. Trial Progression of Children With ADHD Refractory to Stimulant Monotherapy.

FIGURE 1

Table 1 contains demographic and clinical characteristics of children randomly assigned to divalproex and to placebo. The divalproex group had significantly higher Child Behavior Checklist total and externalizing T scores. Children allocated to divalproex were more likely to have had triphasic methylphenidate as their optimized stimulant agent than those allocated to placebo (64% versus 23%; χ2=5.32, df=1, p<0.03) (Table 2).

TABLE 1.

Baseline Demographic and Clinical Characteristics of Children With ADHD Refractory to Stimulant Monotherapy Randomly Assigned to Divalproex or Placebo

Divalproex Group (N=14)
Placebo Group (N=13)
Characteristic N % N %
Sex (male) 10 71 11 85
Race/ethnicity
 Caucasian 10 71 10 77
 Hispanic 1 7 0 0
 African American 2 14 1 8
 Other 1 8 2 15
Guardian's relationship to child
 Biological parent 11 79 12 92
 Adoptive parent 3 21 1 8
 Grandparent 0 0 0 0
Comorbid diagnoses
 Oppositional defiant disorder 11 79 13 100
 Conduct disorder 3 21 0 0
 Anxiety disorder 2 14 3 23
 Mood disorder 2 14 1 7
Special education services
 Regular 9 64 3 31
 Special 5 36 10 69
Prior treatment
 Medication at enrollment 10 71 6 46
 Individual psychotherapy 8 58 3 23
 Family-based treatment/parent training 2 14 1 8
Mean SD Mean SD
Age (years) 8.56 2.32 8.34 2.00
Child Behavior Checklist of behavioral symptom severity T score
 Totala 78.42 4.65 73.15 7.35
 Externalizingb 77.71 4.61 70.77 7.53
 Internalizing 68.92 9.81 65.69 9.29
a

Significant between-group difference (F=5.05, df=1, 25, p<0.05).

b

Significant between-group difference (F=8.49, df=1, 25, p<0.01).

TABLE 2.

Pharmacotherapy Response Ratings for Children With ADHD Refractory to Stimulant Monotherapy Randomly Assigned to Divalproex or Placebo

Divalproex Group (N=14)
Placebo Group (N=13)
Treatment N % Meana SD Range N % Meana SD Range
Stimulant
 Methylphenidate
  Triphasic release 9 64 66.00 22.50 36–108 3 23 57.00 18.7 36–72
  Biphasic release 0 0 2 15 30.00 0.0 30
 Mixed amphetamine salts-extended release 5 36 22.00 5.70 15–30 7 54 28.13 5.9 20–40
 Dextroamphetamine spansule capsules 0 0 1 8 10.00 0.0 10
Adjunctive
 Divalproex or placebo 14 100 571.00 301.00 250–1500 13 100 685.00 263.0 500–1000
a

Data represent dosage (mg).

Response of Aggressive Behavior to Divalproex Versus Placebo

At the end of the trial, children assigned to divalproex treatment had a mean daily dose of 567 mg (SD=291) that corresponded to 19.54 mg/kg (SD=5.37). Their mean total serum valproic acid level was 68.11 mg/liter (SD=21.26, range=29.7–103 mg/liter). Children randomly assigned to placebo took a mean daily amount of the placebo product that represented the active drug equivalent of 685 mg (SD=263).

To evaluate the success of treatment masking, parents indicated at the end of the trial which treatment they believed their child had received. Among those randomly assigned to divalproex, the parents of 28% (three out of 13) believed their child was receiving the active drug, while among those randomly assigned to placebo, 23% of parents believed their child was receiving the active drug. The lack of association (χ2=1.05, df=2, p=0.59) between the actual and suspected treatment arm was consistent with adequate masking of group assignment.

The proportion of children fulfilling criteria for remission of aggressive behavior at the end of the controlled trial was significantly higher within the group randomly assigned to divalproex (eight out of 14, [57.14%]) than within the group randomly assigned to placebo (two out of 13 [15.38%]). The odds of remission with divalproex treatment were seven times greater than that with placebo, although a small sample size rendered the confidence interval (CI) for this estimate especially wide (odds ratio=7.33, χ2=5.04, p<0.05; 95% CI=1.16–46.23). The 41.76% observed difference in remission rates had a 95% CI of 10%–74%. The corresponding expected number needed to treat to benefit one patient was 2.39. Within the divalproex-treated group, there was no significant difference in mean valproic acid levels at the trial's end between children who did and did not remit. Random regression analyses examined the effect of treatment group on logarithmically transformed Retrospective-Modified Overt Aggression Scale ratings over the trial's 8 weeks (Table 3, Figure 2). Adjusting for baseline aggression scale scores, the main effect for week was large (F=15.08, df=1, 25, p<0.001). The treatment-by-week interaction was smaller but attained statistical significance (F=4.23, df=1, 140, p=0.04), indicating the steeper rate and magnitude of reductions in aggressive behavior ratings associated with divalproex treatment.

TABLE 3.

Behavioral Response Ratings for Children With ADHD Refractory to Stimulant Monotherapy Randomly Assigned to Divalproex or Placebo

Divalproex Group (N=14)
Placebo Group (N=13)
Baseline
End of Stimulant Monotherapy Lead-In Phase
End of Randomized, Controlled Trial Phase
Baseline
End of Stimulant Monotherapy Lead-In Phase
End of Randomized, Controlled Trial Phase
Measure Mean SD Mean SD Mean SD Mean SD Mean SD Mean SD
Retrospective-Modified Overt Aggression Scale score 62.13 42.63 41.80 25.79 32.13 44.14 61.54 28.98 53.31 29.50 35.77 28.86
Logarithmically transformed (loge) score 3.93 0.68 3.62 0.52 2.48 1.72 4.04 0.47 3.87 0.49 3.36 0.74
Conners' Global Index T score 80.60 6.49 72.80 13.67 65.13 15.60 77.46 7.66 67.54 9.10 71.50 13.17

FIGURE 2. Weekly Aggression Ratings for Children Randomly Assigned to Divalproex or Placebo.

FIGURE 2

aLogarithmically transformed Retrospective-Modified Overt Aggression Scale score.

Because both the main effect of week and its interaction with treatment were significant, secondary simple-effects analyses tested whether aggressive behavior ratings within the placebo-treated group may have declined appreciably with time, even if to a smaller degree than that of divalproex-treated children. The divalproex-treated group displayed a large effect of week (F=13.12, df=1, 13, p=0.003), while that for the placebo group was not significant.

Inclusion of Child Behavior Checklist T scores and stimulant regimen as covariates in these efficacy analyses did not alter the results, assuaging concern that group differences on these variables confounded outcome.

The divalproex group also displayed a larger decrease in ADHD symptom ratings (according to Conners' Global Index restless-inattentive subscale scores) over the trial compared with the placebo group (week-by-treatment interaction: F=6.45, df=1, 140, p=0.01). To examine whether change in ADHD symptoms mediated the effect of divalproex on reduced aggression, random regression analyses included the difference in Conners' Global Index restless-inattentive subscale ratings in the model that tested the influence of treatment group, time, and baseline aggression on final aggression scale ratings. Because the addition of the ADHD symptom change term to this model did not diminish the significant week-by-treatment interaction, this analysis did not furnish evidence that improved impulse control mediated the effect of treatment on reduced aggression, nor was the main effect for ADHD symptom change significant.

Tolerability and Adverse Effects

Several adverse effects associated with stimulant treatment (anxiety, fingernail biting, and suppressed appetite) increased between baseline and the end of the stimulant monotherapy lead-in phase (Table 4). Children treated with divalproex tended to have higher rates of treatment-emergent sadness and trouble falling asleep than children treated with placebo. There were no significant associations between valproic acid levels and the odds of developing any adverse effect.

TABLE 4.

Adverse Effect Rates in Children With ADHD Randomly Assigned to Divalproex or Placebo

Baseline
End of Stimulant Monotherapy Lead-in Phase
End of Randomized, Controlled Trial Phase
Divalproex
Placebo
Divalproex
Placebo
Divalproex
Placebo
Adverse Effect N % N % p N % N % p N % N % p
Abdominal pain 1 7 2 15 0 0 0 0 0 0 0 0
Anxiety or nervousness 2 14 2 15 3 21 2 15 3 21 1 8
Enuresis 0 0 1 8 0 0 1 8 1 7 1 8
Fingernail biting 2 14 6 46 0.07 3 21 2 15 2 14 1 8
Bruises easily 1 7 0 0 0 0 0 0 0 0 1 8
Constipation 1 7 1 8 1 7 0 0 0 0 0 0
Crying 2 14 3 23 2 14 3 23 5 36 4 31
Insomnia (initial) 6 43 8 62 0.07 5 36 4 31 5 36 1 8 0.08
Decreased appetite 0 0 3 23 0.06 3 21 4 31 1 7 3 23
Insomnia (middle) 2 14 1 8 1 7 1 8 2 14 1 8
Dizzy 0 0 1 8 0 0 0 0 0 0 0 0
Dry mouth 2 14 0 0 0 0 0 0 0 0 0 0
Headache 0 0 1 8 0 0 0 0 0 0 0 0
Increased appetite 2 14 2 15 0 0 1 8 1 7 0 0
Irritability 6 43 8 62 4 29 5 38 5 36 4 31
Low energy 1 7 1 8 0 0 1 8 1 7 0 0
Nightmares 1 7 0 0 1 7 0 0 2 14 0 0
Other 1 7 0 0 0 0 0 0 0 0 0 0
Rash 0 0 0 0 0 0 0 0 0 0 1 8
Restlessness 0 0 0 0 2 14 0 0 0.04 2 14 2 15
Sadness 3 21 2 15 2 14 3 23 3 21 0 0 0.07
Shaking 0 0 0 0 0 0 0 0 1 7 0 0
Stares into space 4 29 0 0 0.04 2 14 0 0 2 14 0 0
Stomach ache 1 7 1 8 1 7 0 0 0 0 0 0
Heart races 1 7 0 0 0 0 0 0 0 0 0 0
Less talkative 1 7 1 8 2 14 0 0 1 7 0 0
Overly talkative 5 36 1 8 0.08 0 0 2 15 2 14 2 15
Tics 0 0 1 8 0 0 1 8 1 7 0 0
Tired 2 14 0 0 1 7 0 0 0 0 0 0
Trouble walking 1 7 0 0 0 0 0 0 0 0 0 0
Tremors 0 0 0 0 1 7 0 0 1 7 0 0
Lack of interest 1 7 0 0 3 21 1 8 1 7 0 0
Unusually happy 1 7 0 0 0 0 0 0 0 0 0 0
Early waking 4 29 2 15 1 7 1 8 2 14 0 0

Discussion

Divalproex cotherapy, adjunctive to stimulant treatment, was more effective than adjunctive placebo in promoting the remission of aggressive behavior among children who experienced inadequate reductions of aggressive behavior with stimulant monotherapy. The divalproex-treated group displayed a faster rate of decline in ratings of aggressive behavior during the trial. The divalproex-treated sample also experienced a rate of remission substantially larger than that of the placebo group (57% versus 15%). However, a small sample size embedded this estimate of divalproex's advantage within a wide confidence interval, making it tentative until larger samples can yield a more precise estimate.

Children whose difficulties with aggressive, volatile behavior did not diminish sufficiently by the end of the stimulant monotherapy lead-in phase appeared unlikely to gain further benefit by simply continuing this regimen for an additional 8 weeks, even when accompanied by weekly psychosocial behaviorally oriented treatment. This result suggests that prompt revisions to treatment would be indicated. In this context, our finding that the addition of divalproex led to appreciable reductions in aggression provides an empirically based option for the management of youth shown to respond insufficiently to stimulant monotherapy.

Serial assessment of both ADHD symptoms and aggressive behavior allowed us to examine the relationship between changes in these domains. Changes in ADHD symptoms during the controlled comparison of divalproex with placebo did not mediate the effects of treatment on reduced aggression. Further exploration might consider the role of changes in affective features that may accompany divalproex treatment as potential mediators of its efficacy (e.g., irritability, low frustration tolerance).

The range in trough valproic acid levels at the trial endpoint was wide, and the mean of 68 mg/liter was toward the lower end of the customary therapeutic range. We found that the long-acting divalproex preparation used in the present trial was difficult to dose to obtain blood valproic acid levels between 80 and 110 mg/liter. Concentrations exceeding 110 mg/liter are often well-tolerated among children, but the possibility of a higher incidence of dose-dependent adverse effects, even if safely managed, risked unblinding. Although we found no association between end-of-trial valproic acid levels and response, it is possible that higher doses might have enhanced the overall response rate. In the doses used in the present study, the combination of divalproex with stimulant medication was well tolerated.

Design of future trials to yield a more precise estimate of treatment effects for adjunctive pharmacotherapy depends on certain parameters that the present study elucidates. First, the rate of response to stimulant monotherapy, approximately 50% with our procedures, is essential to establishing enrollment targets. Second, a preliminary gauge of the effect of the adjunctive agent on monotherapy nonresponders—in the present case a 42% advantage in remission rates with divalproex over placebo—is a prerequisite for power analyses determining sample size. Third, the retention rate and overall feasibility of sustaining the participation of highly impaired children in an out-patient study that involved several phases were promising. Although we recognize the limitations of the present trial's modest sample size, the data yielded provide the basis for overcoming them in subsequent research.

Antipsychotic medications are now widely used to treat aggressive behavior among children, and response rates of 75% and upward are reported from large trials (41). However, these agents have not yet been evaluated in controlled trials involving children prospectively shown to experience insufficient response after attempts to optimize first-line stimulant treatment. Our findings indicate that divalproex would be a viable active comparator that has a generally more benign risk profile for this patient group. It seems important to evaluate whether adjunctive antipsychotic treatment exhibits superiority relative to divalproex among children demonstrably underresponsive to stimulant therapy. This issue assumes particular urgency in light of the propensity for these agents to incur adverse metabolic and endocrine changes, weight gain, and neuromotor abnormalities at higher incidence and severity among children relative to adults.

The present trial also contributes to the currently small literature in child and adolescent psychiatry on polytherapy strategies for symptoms or comorbid conditions that cause persisting impairments refractory to initial treatment (4244). Such multistage trials are inherently more difficult to complete, yet the prevalence of combined pharmacotherapy in clinical care demonstrates that rigorous research to probe the value and the liabilities of these regimens is essential to a practice of psychopharmacology that aspires to be evidence-based.

Patient Perspective.

“Freddy” was a 6-year-old boy whose mother related a long history of attention deficit hyperactivity disorder (ADHD) symptoms and behavioral dyscontrol. She characterized his present difficulties as being “verbally and physically abusive.” She stated that “he destroys property, defies authority, has no friends, hurts animals, and constantly annoys others.” He was in frequent conflict with family members, peers, and school authorities. When not acutely upset, he was an affectionate, articulate, pleasant boy. His ingenuity and engaging character attracted other kids initially, but subsequent interactions discouraged them, owing to his inflexibility in accommodating other children's desires over his own. His aggression rating on our Retrospective-Modified Overt Aggression Scale was 82 at screening, and his ADHD symptoms ratings were approximately three standard deviations above the mean for boys his age. He began the stimulant lead-in phase with low stimulant doses, ultimately titrated to 72 mg of triphasic methylphenidate. At a 54-mg dose, attention and appropriate participation in school improved, but aggressive outbursts persisted, including incidents of throwing his desk over in response to directions to conclude an activity. His mother said she could now see him trying to control himself before lashing out, but ultimately anger got the better of him. However, increasing to 72 mg daily was associated with worrisome changes in demeanor. He was very serious and irritable and seemed on the lookout for unfair treatment and things to criticize. He complained constantly about the behavior of peers and siblings, even when such behavior did not directly involve him. Dose reduction to 63 mg did not improve things, and thus he was returned to his 54-mg dose and deemed a partial responder (i.e., he gained some benefit from stimulant treatment for ADHD symptoms [Conners' Global Index T score declined from 82 to 65]), but affective volatility, explosivenesss, defiance, easy frustration, and aggression remained. He seemed a bit less tense than he was when receiving the higher methylphenidate dose but still more serious and withdrawn.

After 6 weeks in the lead-in protocol, Freddy started adjunctive medication determined by random assignment. Weighing 26 kg, his in itial titration was to 500 mg daily of extended-release divalproex by the end of the second week. The unblind psychiatrist directed that his dose be reduced to 250 mg per day, although blind clinicians did not know if the impetus to do so was a high valproic acid level or a “sham” change. By his fourth week after randomization, his mother reported that he was “great,” with “no problems at all.” She stated, “He shows a lot less anger and is a lot more relaxed.” The next month consolidated this pattern, and he showed broader affect and seemed happier at office visits as well. He was far more easygoing and handled with equanimity minor upsets that would earlier have precipitated sustained rage. Freddy had been taking active divalproex, and his blood level at the trial's end was 79.6 mg/liter.

Acknowledgments

Dr. Blader has received research support from Abbott Laboratories and consulting fees from Otsuka America Pharmaceuticals and Validus Pharmaceuticals. Dr. Schooler has received research support from Ortho-McNeil-Janssen Scientific Affairs and consultant fees or honoraria from Organon, Ortho-McNeil-Janssen, Schering Plough, H. Lundbeck A/S, and Eli Lilly. Dr. Jensen has received speaker's fees or consulting honoraria from Johnson and Johnson, Janssen-Ortho, U.C. Pharma, Neuroscience Education Institute, Continuing Medical Education Outfitters, and CMED. Dr. Pliszka has received research support from Ortho-McNeil-Janssen Scientific Affairs, Shire, Eli Lilly, AstraZeneca, and Cephalon and consulting fees or honoraria from Shire and McNeil Pediatrics. Dr. Kafantaris has received research support from Eli Lilly and Pfizer and donation of study medication and matching placebo from AstraZeneca, GlaxoSmithK line, Janssen, Eli Lilly, and Pfizer.

Supported by NIH grants K23 MH-064975 (principal investigator, Dr. Blader) and MO1RR10710 (Stony Brook General Clinical Research Center); a Young Investigator Award from the National Alliance for Research on Schizophrenia and Depression (principal investigator, Dr. Blader); and a grant (drug, matching placebos, and financial support) for investigator-initiated research from Abbott Laboratories (principal investigator, Dr. Blader).

The authors thank the following individuals: research coordinators Jennifer Puig, Marcela Torres, and Alisa Jahns; research psychiatrists Carmel Foley, M.D., Deyan Budimirovich, M.D., Jeffrey Sverd, M.D., Josefina Antonio, M.D., Malgorzata Relja, M.D., and Mira Sachdev, M.D.; research pharmacists Frank Albergo and Wayne Patterson; psychosocial therapists Esme Londahl-Shaller, Evelyn Flaherty, Alisa Hurwitz, and Erica Chin; and Gaye Carlson. The authors also thank the children and families who participated in the study.

Footnotes

ClinicalTrials.gov registry number, NCT00228046 (www.clinicaltrials.gov).

References

  • 1.Lahey BB, Loeber R, Burke JD, Applegate B. Predicting future antisocial personality disorder in males from a clinical assessment in childhood. J Consult Clin Psychol. 2005;73:389–399. doi: 10.1037/0022-006X.73.3.389. [DOI] [PubMed] [Google Scholar]
  • 2.Tremblay RE. In: Why socialization fails: the case of chronic physical aggression, in Causes of Conduct Disorder and Juvenile Delinquency. Lahey BB, Moffitt TE, Caspi A, editors. Guilford; New York: 2003. pp. 182–224. [Google Scholar]
  • 3.Connor DF, Steingard RJ, Cunningham JA, Anderson JJ, Melloni RH., Jr Proactive and reactive aggression in referred children and adolescents. Am J Orthopsychiatry. 2004;74:129–136. doi: 10.1037/0002-9432.74.2.129. [DOI] [PubMed] [Google Scholar]
  • 4.Blader JC, Jensen PS. In: Aggression in children: an integrative approach, in Lewis' Textbook of Child and Adolescent Psychiatry. 4th ed. Martin A, Volkmar FR, editors. Lippincott Williams and Wilkins; Baltimore: 2007. pp. 467–483. [Google Scholar]
  • 5.Connor DF, Carlson GA, Chang KD, Daniolos PT, Ferziger R, Findling RL, Hutchinson JG, Malone RP, Halperin JM, Plattner B, Post RM, Reynolds DL, Rogers KM, Saxena K, Steiner H. Juvenile maladaptive aggression: a review of prevention, treatment, and service configuration and a proposed research agenda. J Clin Psychiatry. 2006;67:808–820. [PubMed] [Google Scholar]
  • 6.Steiner H, Remsing L, AACAP Work Group on Quality Issues Practice parameter for the assessment and treatment of children and adolescents with oppositional defiant disorder. J Am Acad Child Adolesc Psychiatry. 2007;46:126–141. doi: 10.1097/01.chi.0000246060.62706.af. [DOI] [PubMed] [Google Scholar]
  • 7.Jensen PS, Youngstrom EA, Steiner H, Findling RL, Meyer RE, Malone RP, Carlson GA, Coccaro EF, Aman MG, Blair J, Dougherty D, Ferris C, Flynn L, Green E, Hoagwood K, Hutchinson J, Laughren T, Leve LD, Novins DK, Vitiello B. Consensus report on impulsive aggression as a symptom across diagnostic categories in child psychiatry: implications for medication studies. J Am Acad Child Adolesc Psychiatry. 2007;46:309–322. doi: 10.1097/chi.0b013e31802f1454. [DOI] [PubMed] [Google Scholar]
  • 8.Rey JM, Sawyer MG, Prior MR. Similarities and differences between aggressive and delinquent children and adolescents in a national sample. Aust N Z J Psychiatry. 2005;39:366–372. doi: 10.1080/j.1440-1614.2005.01583.x. [DOI] [PubMed] [Google Scholar]
  • 9.Nagin DS, Tremblay RE. Parental and early childhood predictors of persistent physical aggression in boys from kindergarten to high school. Arch Gen Psychiatry. 2001;58:389–394. doi: 10.1001/archpsyc.58.4.389. [DOI] [PubMed] [Google Scholar]
  • 10.Abikoff H, Hechtman L, Klein RG, Weiss G, Fleiss K, Etcovitch J, Cousins L, Greenfield B, Martin D, Pollack S. Symptomatic improvement in children with ADHD treated with long-term methylphenidate and multim odal psychosocial treatment. J Am Acad Child Adolesc Psychiatry. 2004;43:802–811. doi: 10.1097/01.chi.0000128791.10014.ac. [DOI] [PubMed] [Google Scholar]
  • 11.Multimodal Treatment Study of Children with ADHD Cooperative Group A 14-month randomized clinical trial of treatment strategies for attention-deficit/hyperactivity disorder. Arch Gen Psychiatry. 1999;56:1073–1086. doi: 10.1001/archpsyc.56.12.1073. [DOI] [PubMed] [Google Scholar]
  • 12.Pliszka SR, Crismon ML, Hughes CW, Conners CK, Emslie GJ, Jensen PS, McCracken JT, Swanson JM, Lopez M, Texas Consensus Conference Panel on Pharmacotherapy of Childhood Attention Deficit Hyperactivity Disorder The Texas Children's Medication Algorithm Project: revision of the algorithm for pharm acotherapy of attention-deficit/hyperactivity disorder (ADHD) J Am Acad Child Adolesc Psychiatry. 2006;45:642–657. doi: 10.1097/01.chi.0000215326.51175.eb. [DOI] [PubMed] [Google Scholar]
  • 13.Jensen PS, Hinshaw SP, Kraemer HC, Lenora N, Newcorn JH, Abikoff HB, March JS, Arnold LE, Cantwell DP, Conners CK, Elliott GR, Greenhill LL, Hechtman L, Hoza B, Pelham WE, Severe JB, Swanson JM, Wells KC, Wigal T, Vitiello B. ADHD comorbidity findings from the MTA study: comparing comorbid subgroups. J Am Acad Child Adolesc Psychiatry. 2001;40:147–158. doi: 10.1097/00004583-200102000-00009. [DOI] [PubMed] [Google Scholar]
  • 14.Connor DF, Ozbayrak KR, Kusiak KA, Caponi AB. Combined pharmacotherapy in children and adolescents in a residential treatment center. J Am Acad Child Adolesc Psychiatry. 1997;36:248–254. doi: 10.1097/00004583-199702000-00016. [DOI] [PubMed] [Google Scholar]
  • 15.Safer DJ. Changing patterns of psychotropic medications prescribed by child psychiatrists in the 1990s. J Child Adolesc Psychopharmacol. 1997;7:267–274. doi: 10.1089/cap.1997.7.267. [DOI] [PubMed] [Google Scholar]
  • 16.Blader JC. Pharmacotherapy and postdischarge outcomes of child inpatients admitted for aggressive behavior. J Clin Psychopharmacol. 2006;26:419–425. doi: 10.1097/01.jcp.0000227356.31203.8a. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Zarin DA, Suarez AP, Pincus HA, Kupersanin E, Zito JM. Clinical and treatment characteristics of children with attention-deficit/hyperactivity disorder in psychiatric practice. J Am Acad Child Adolesc Psychiatry. 1998;37:1262–1270. doi: 10.1097/00004583-199812000-00009. [DOI] [PubMed] [Google Scholar]
  • 18.Martin A, Van Hoof T, Stubbe D, Sherwin T, Scahill L. Multiple psychotropic pharmacotherapy among child and adolescent enrollees in Connecticut Medicaid managed care. Psychiatr Serv. 2003;54:72–77. doi: 10.1176/appi.ps.54.1.72. [DOI] [PubMed] [Google Scholar]
  • 19.Patel NC, Crismon ML, Hoagwood K, Johnsrud MT, Rascati KL, Wilson JP, Jensen PS. Trends in the use of typical and atypical antipsychotics in children and adolescents. J Am Acad Child Adolesc Psychiatry. 2005;44:548–556. doi: 10.1097/01.chi.0000157543.74509.c8. [DOI] [PubMed] [Google Scholar]
  • 20.Aman MG, De Smedt G, Derivan A, Lyons B, Findling RL, Risperidone Disruptive Behavior Study Group Double-blind, placebo-controlled study of risperidone for the treatment of disruptive behaviors in children with subaverage intelligence. Am J Psychiatry. 2002;159:1337–1346. doi: 10.1176/appi.ajp.159.8.1337. [DOI] [PubMed] [Google Scholar]
  • 21.Snyder R, Turgay A, Aman M, Binder C, Fisman S, Carroll A, Risperidone Conduct Study Group Effects of risperidone on conduct and disruptive behavior disorders in children with subaverage IQs. J Am Acad Child Adolesc Psychiatry. 2002;41:1026–1036. doi: 10.1097/00004583-200209000-00002. [DOI] [PubMed] [Google Scholar]
  • 22.Findling RL, McNamara NK, Branicky LA, Schluchter MD, Lemon E, Blumer JL. A double-blind pilot study of risperidone in the treatment of conduct disorder. J Am Acad Child Adolesc Psychiatry. 2000;39:509–516. doi: 10.1097/00004583-200004000-00021. [DOI] [PubMed] [Google Scholar]
  • 23.Steiner H, Petersen ML, Saxena K, Ford S, Matthews Z. Divalproex sodium for the treatment of conduct disorder: a randomized controlled clinical trial. J Clin Psychiatry. 2003;64:1183–1191. doi: 10.4088/jcp.v64n1007. [DOI] [PubMed] [Google Scholar]
  • 24.Donovan SJ, Stewart JW, Nunes EV, Quitkin FM, Parides M, William D, Susser E, Klein DF. Divalproex treatment for youth with explosive temper and mood lability: a double-blind, placebo-controlled crossover design. Am J Psychiatry. 2000;157:818–820. doi: 10.1176/appi.ajp.157.5.818. [DOI] [PubMed] [Google Scholar]
  • 25.Campbell M, Adams PB, Small AM, Kafantaris V, Silva RR, Shell J, Perry R, Overall JE. Lithium in hospitalized aggressive children with conduct disorder: a double-blind and placebo-controlled study. J Am Acad Child Adolesc Psychiatry. 1995;34:445–453. [PubMed] [Google Scholar]
  • 26.Campbell M, Small AM, Green WH, Jennings SJ, Perry R, Bennett WG, Anderson L. Behavioral efficacy of haloperidol and lithium carbonate: a comparison in hospitalized aggressive children with conduct disorder. Arch Gen Psychiatry. 1984;41:650–656. doi: 10.1001/archpsyc.1984.01790180020002. [DOI] [PubMed] [Google Scholar]
  • 27.Malone RP, Delaney MA, Leubbert JF, Cater J, Campbell M. A double-blind placebo-controlled study of lithium in hospitalized aggressive children and adolescents with conduct disorder. Arch Gen Psychiatry. 2000;57:649–654. doi: 10.1001/archpsyc.57.7.649. [DOI] [PubMed] [Google Scholar]
  • 28.Coccaro EF, Kavoussi RJ. Fluoxetine and impulsive aggressive behavior in personality-disordered subjects. Arch Gen Psychiatry. 1997;54:1081–1088. doi: 10.1001/archpsyc.1997.01830240035005. [DOI] [PubMed] [Google Scholar]
  • 29.Aman MG, Binder C, Turgay A. Risperidone effects in the presence/absence of psychostimulant medicine in children with ADHD, other disruptive behavior disorders, and subaverage IQ. J Child Adolesc Psychopharmacol. 2004;14:243–254. doi: 10.1089/1044546041649020. [DOI] [PubMed] [Google Scholar]
  • 30.Pliszka SR, Greenhill LL, Crismon ML, Sedillo A, Carlson C, Conners CK, McCracken JT, Swanson JM, Hughes CW, Llana ME, Lopez M, Toprac MG, Texas Consensus Conference Panel on Medication Treatment of Childhood Attention-Deficit/Hyperactivity Disorder The Texas Children's Medication Algorithm Project: report of the Texas Consensus Conference Panel on medication treatment of childhood attention-deficit/hyperactivity disorder, part I. J Am Acad Child Adolesc Psychiatry. 2000;39:908–919. doi: 10.1097/00004583-200007000-00021. [DOI] [PubMed] [Google Scholar]
  • 31.Achenbach TM. The Manual for the ASEBA School-Age Forms and Profiles. University of Vermont; Burlington, Vt: 2001. [Google Scholar]
  • 32.Conners CK. Conners' Rating Scales–Revised Technical Manual. Multi-Health Systems; North Tonawanda, NY: 1997. [Google Scholar]
  • 33.Hellings JA, Weckbaugh M, Nickel EJ, Cain SE, Zarcone JR, Reese RM, Hall S, Ermer DJ, Tsai LY, Schroeder SR, Cook EH. A double-blind, placebo-controlled study of valproate for aggression in youth with pervasive developmental disorders. J Child Adolesc Psychopharmacol. 2005;15:682–692. doi: 10.1089/cap.2005.15.682. [DOI] [PubMed] [Google Scholar]
  • 34.Stigler KA, Desmond LA, Posey DJ, Wiegand RE, McDougle CJ. A naturalistic retrospective analysis of psychostimulants in pervasive developmental disorders. J Child Adolesc Psychopharmacol. 2004;14:49–56. doi: 10.1089/104454604773840481. [DOI] [PubMed] [Google Scholar]
  • 35.Kaufman J, Birmaher B, Brent D, Rao U, Flynn C, Moreci P, Williamson D, Ryan N. Schedule for Affective Disorders and Schizophrenia for School-Age Children–Present and Lifetime Version (K-SADS-PL): initial reliability and validity. J Am Acad Child Adolesc Psychiatry. 1997;36:980–988. doi: 10.1097/00004583-199707000-00021. [DOI] [PubMed] [Google Scholar]
  • 36.Coccaro EF, Berman ME, Kavoussi RJ. Assessment of life-history of aggression: development and psychometric characteristics. Psychiatry Res. 1997;73:147–157. doi: 10.1016/s0165-1781(97)00119-4. [DOI] [PubMed] [Google Scholar]
  • 37.Kavoussi RJ, Coccaro EF. Divalproex sodium for impulsive aggressive behavior in patients with personality disorder. J Clin Psychiatry. 1998;59:676–680. doi: 10.4088/jcp.v59n1206. [DOI] [PubMed] [Google Scholar]
  • 38.Barkley RA, McMurray MB, Edelbrock CS, Robbins K. Side effects of methylphenidate in children with attention deficit hyperactivity disorder: a systemic, placebo-controlled evaluation. Pediatrics. 1990;86:184–192. [PubMed] [Google Scholar]
  • 39.Pelham WE, Gnagy EM, Burrows-Maclean L, Williams A, Morrisey SM, Chronis AM, Forehand GL, Nguyen CA, Hoffman MT, Lock TM, Fiebelkorn K, Coles EK, Panahon CJ, Steiner RL, Meichenbaum DL, Onyango AN, Morse GD. Once-a-day Concerta methylphenidate versus three-times-daily methylphenidate in laboratory and natural settings. Pediatrics. 2001;107:e105. doi: 10.1542/peds.107.6.e105. [DOI] [PubMed] [Google Scholar]
  • 40.Dunlap WP, Cortina JM, Vaslow JB, Burke MJ. Meta-analysis of experiments with matched groups or repeated measures designs. Psychol Methods. 1996;1:170–177. [Google Scholar]
  • 41.Pappadopulos E, Woolston S, Chait A, Perkins M, Connor DF, Jensen PS. Pharmacotherapy of aggression in children and adolescents: efficacy and effect size. J Can Acad Child Adolesc Psychiatry. 2006;15:27–39. [PMC free article] [PubMed] [Google Scholar]
  • 42.Abikoff H, McGough J, Vitiello B, McCracken J, Davies M, Walk-up J, Riddle M, Oatis M, Greenhill L, Skrobala A, March J, Gammon P, Robinson J, Lazell R, McMahon DJ, Ritz L, Research Unit on Pediatric Psychopharmacology ADHD/Anxiety Study Group Sequential pharmacotherapy for children with comorbid attention-deficit/hyperactivity and anxiety disorders. J Am Acad Child Adolesc Psychiatry. 2005;44:418–427. doi: 10.1097/01.chi.0000155320.52322.37. [DOI] [PubMed] [Google Scholar]
  • 43.Delbello MP, Schwiers ML, Rosenberg HL, Strakowski SM. A double-blind, randomized, placebo-controlled study of quetiapine as adjunctive treatment for adolescent mania. J Am Acad Child Adolesc Psychiatry. 2002;41:1216–1223. doi: 10.1097/00004583-200210000-00011. [DOI] [PubMed] [Google Scholar]
  • 44.Scheffer RE, Kowatch RA, Carmody T, Rush AJ. Randomized, placebo-controlled trial of mixed amphetamine salts for symptoms of comorbid ADHD in pediatric bipolar disorder after mood stabilization with divalproex sodium. Am J Psychiatry. 2005;162:58–64. doi: 10.1176/appi.ajp.162.1.58. [DOI] [PubMed] [Google Scholar]

RESOURCES