Skip to main content
NCBI home page
Journal of Pediatric Psychology logoLink to Journal of Pediatric Psychology
. 2011 Feb 19;36(7):753–765. doi: 10.1093/jpepsy/jsr006

Moderators and Predictors of Clinical Outcome in a Randomized Trial for Behavior Problems in Pediatric Primary Care

David J Kolko 1,✉, Yu Cheng 1, John V Campo 2, Kelly Kelleher 2
PMCID: PMC3146758  PMID: 21335616

Abstract

Objectives To evaluate putative moderator, predictor, and treatment parameter variables in relation to three outcomes in a clinical trial that compared a modular protocol for on-site, nurse-administered intervention (PONI) and enhanced usual care (EUC) for pediatric behavioral problems in primary care. Methods Patients were 163 clinically referred children for behavior problems in six primary care offices. PONI consisted of seven treatment modules adapted from prior treatment trials with this population, whereas EUC involved a facilitated referral to a community provider. Outcome measures were based on standardized scales reflecting one parent-rated aggregate (child dysfunction) and one child-rated aggregate (child health), and diagnostic interviews with both informants (remission in oppositional defiant disorder). Results Moderator analyses revealed that PONI was more effective than EUC in reducing child dysfunction by 12-month follow-up among Caucasian children, whereas EUC was more effective than PONI among non-Caucasian children. In the full sample, child health improvement was predicted by the severity of the child’s depression and anxiety, and level of family conflict. Duration of child exposure to cognitive-behavioral treatment in PONI was related to greater improvement in overall child health, but other treatment parameters were unrelated to outcome. Conclusions These few significant relationships notwithstanding the findings indicate that the two treatments had robust effects on several outcomes and across selected child, parent, family, and treatment variables. The findings extend efforts to incorporate mental health services in pediatric practice.

Keywords: behavior problems, clinical referral, disruptive behavior disorders, on-site mental health services

Introduction

Interventions for child and adolescent mental health disorders have recently been evaluated in the pediatric primary care setting to enhance its capacity to deliver evidence-based practice (EBP) for common psychosocial problems (i.e., calls to help families respond to challenges and crises, advocacy with local service agencies, and professionals; see Kelleher & Stevens, 2009). Clinical trials have documented changes in behavior problems (BP) following Internet-based psychoeducation (Borowsky, Mozayeny, Stuenkel, & Ireland, 2004), on-site behavioral family intervention (Turner & Sanders, 2006), and a 1-day group and individual parenting skills training program (Hayes, Matthews, Copley, & Welsh, 2008). A recent trial in preschoolers with Oppositional Defiant Disorder (ODD) found parallel improvements in 1-year outcomes for cases that received an EBP (Incredible Years) either delivered on-site by a nurse or off-site by a local pediatric psychologist, or bibliotherapy only (Lavigne, LeBailly, Gouze, Cichetti, Jessup et al., 2008). These results are consistent with a recent study in which a four-session group intervention (Parent–Child Interaction Training or PCIT) and exposure to manual outlining the basic steps of PCIT had comparable effects on child BP (Berkovits, O'Brien, Carter, & Eyberg, 2010). Other primary care interventions have reported modest results for behavioral parent training with Attention Deficit Hyperactivity Disorder (ADHD; Wolraich, Bickman, Lambert, Simmons, & Doffing, 2005), and clinical benefits in adolescent depression for an Internet-based curriculum or on-site collaborative care (Asarnow et al., 2005; Voorhees et al., 2009).

The transportability of evidence-based treatments to new pediatric settings, populations, and providers is likely to be enhanced by understanding how the characteristics related to each of these elements differentially affect treatment response and outcome. For example, efforts to examine variables that moderate, predict, and/or mediate treatment outcome in response to alternative intervention conditions, and the role of key treatment parameters, have been encouraged to optimize treatment selection for a given patient (La Greca, Silverman, & Lochman, 2009). Briefly, moderators reflect those pretreatment variables that may affect the relationship between treatment and outcome, and, thus, may specify on whom and under what conditions treatments exert different effects, whereas mediators reflect those processes occurring during treatment that explain or specify how and why the intervention exerts its effect (Hinshaw, 2007; Kraemer, Wilson, Fairburn, & Agras, 2002). Some variables may be found to predict overall outcome across alternative treatments, rather than being associated with differential treatment effects.

An important next step for understanding how to improve the mental health treatment of BP in health care settings involves determining which child, caregiver, and family characteristics are associated with positive outcomes, especially given the varied rates of improvement reported in trials conducted in primary care. For example, in the aforementioned ODD treatment study (Lavigne, LeBailly, Gouze, Cicchetti, Pochyly et al., 2008), girls showed more improvement than boys in the on-site, nurse-led EBP, but boys showed more improvement in the bibliotherapy-only condition. Follow-up analyses from that study also found that caregivers with less education were more likely to attend off-site treatment delivered by a pediatric psychologist, whereas those with higher education responded better to the on-site nurse-administered treatment (Lavigne, LeBailly, Gouze, Binns, Keller et al., 2010). Van den Hoofdakker et al. (2010) found superior treatment effects for behavioral parenting training (vs. routine clinic practice) in children with no or single-type comorbidity and mothers with high parenting efficacy in children with both BP and ADHD. Unfortunately, only two of these eight outcome studies examined moderators, predictors, or mediators of treatment. Although some of the same moderators have been documented with BP children (e.g., age, comorbidities, low socioeconomic status; (Lundahl, Risser, & Lovejoy, 2006), the relationships between these variables and outcomes have varied in both magnitude and direction across studies. For example, child comorbidity has been associated with better (Beauchaine, Webster-Stratton, & Reid, 2005; Gardner et al., 2009) or worse outcome (van den Hoofdakker et al., 2010), or been unrelated to outcome (Kazdin & Whitley, 2006). These mixed results limit their implications for the refinement of treatments delivered in primary care.

A few of the aforementioned variables and several other variables have been found to predict overall outcome, regardless of treatment condition. The identification of general predictors is clinically important to identify potential risk factors for poor outcome in primary care settings (Reyno & McGrath, 2006). For example, Lavigne, LeBailly, Gouze, Cicchetti, Jessup et al., 2008 identified several baseline variables related to improved outcome in ODD children (e.g., child internalizing problems, functional impairment, difficult temperament, parenting distress, negative life events), although those families with lower levels of these variables still had fewer BP at posttreatment and follow-up. Lower socioeconomic status and minority status predicted attendance in the behavioral parent training program reported in that study (Lavigne et al., 2010).

Finally, the mechanism of action for a given treatment can be understood by evaluating potential mediators of treatment, although none of the trials in primary care examined treatment mediation. In the general treatment literature, changes in parenting practices have been found to mediate improvement in BP following PCIT (Chaffin et al., 2004), whereas child-reported loneliness has mediated changes in social anxiety following cognitive-behavioral treatment (Alfano et al., 2009), though these types of analyses are not reported in primary care studies. Likewise, few studies have examined dose–response (Bickman, Andrade, & Lambert, 2002) or other key parameters of treatment (e.g., primary content, fidelity; Kelley, de Andrade, Sheffer, & Bickman, 2010) in primary care research, with one exception. Lavigne, LeBailly, Gouze, Cicchetti, Jessup et al., 2008; found that cases receiving at least seven sessions of a behavioral parent training program showed a significantly better outcome than those who received bibliotherapy. Such developments are needed to promote the efficiency and clinical yield of novel treatment designed to address children’s BPs and their comorbidities (e.g., ADHD) in the pediatric primary care setting (Kolko, 2009).

We recently completed a randomized clinical trial for 163 children with BP in primary care offices that compared a brief, modular protocol representing an on-site, nurse-administered intervention (PONI) with a facilitated referral to a local community provider that reflected enhanced usual care (EUC; Kolko, Campo, Kelleher, & Cheng, 2010). PONI cases were significantly more likely to receive a complete mental health services, had a higher dose of care, and reported fewer service barriers and more consumer satisfaction. PONI cases showed significantly greater improvements on two of the six primary child outcomes (i.e., parent-reported individualized problem behavior ratings, child-reported overall child-health status) and in the proportion of cases showing remission in ADHD at 1-year follow-up. But, both conditions reported comparable but significant improvements on four primary clinical outcomes (i.e., parent-reported externalizing problems, overall child problems, and interpersonal strength; teacher-reported overall child problems), and in remission rates for behavior disorders at follow-up.

The documentation in our initial study of several parallel improvements following on-site care (PONI and outside specialty referral (EUC) highlights the need to identify any characteristics that help to individualize assignment to either of these treatments (La Greca et al., 2009), and may promote a more personalized approach to intervention in the pediatric primary care setting. Based on the few outcome studies reported in pediatric primary care, we identified those variables that were found to serve as either or both treatment moderators and overall predictors of outcome, and those parameters of treatment that were associated with improved outcome. Accordingly, we selected many of the same variables for examination in this study in order to determine if their specific relationship to similar outcomes could be replicated. It is noteworthy that all but two of these variables were tested as moderators or predictors. Accordingly, this study evaluates several child, parent, and family variables as moderators or predictors of treatment outcome, the prerequisite conditions for treatment mediation, and key treatment parameters in relation to outcome, based on their status as reported in the pediatric treatment outcome literature. This diverse set of variables was examined in relation to two key child outcomes based on the standardized rating scales (child dysfunction, child health) and one outcome that reflects remission from ODD, the most prevalent psychiatric disorder found in this sample upon diagnostic interview.

Methods

Patients

The sample consists of 163 children who met inclusion criteria for age (6–11 years), primarily referral for BP, modest severity of BP (score of 6; 75th percentile) on the externalizing BP scale of the Pediatric Symptom Checklist (PSC-17), and parental willingness to participate in an approved clinical trial during a screening interview (Kolko et al., 2010). These children did not meet any of three exclusion criteria: presence of emergent psychiatric or medical symptoms (e.g., psychosis, suicidality with a plan, unstable medical illness), alternative treatment needs (e.g., eating disorders/anorexia, obsessive compulsive disorder, pervasive developmental disorder), or participation in a parallel mental health intervention (e.g., parenting classes).

On average, the sample was young (M = 8.1 years), male (65.0%), and Caucasian (79.8%), and lived with primary caregivers who were married (75.4%). The average level of externalizing problems on the PSC-17 was high (M = 8.6; SD = 2.2) and most of the sample had scores above the clinical cutoff (90th percentile) on its externalizing (81.6%), attentional (70.6%), and internalizing (56.2%) problems scales. The children had an average of two Axis I-psychiatric disorders; 153 children (94%) met criteria for at least one disorder, which reflected primarily ODD or ADHD. As described below, the children were randomized into two treatment conditions which did not differ significantly on any of these background or any other pretreatment clinical rating scales or diagnostic interview measures (p = .98–.08).

Pediatric Primary Care Settings

Children were recruited from six pediatric practices affiliated with Children’s Community Pediatrics (CCP), a community-based primary care network affiliated with the Children’s Hospital of Pittsburgh. Study sites covered 40,598 pediatric lives from four geographic regions in the Pittsburgh area and varied by urban/suburban location. These large, group practices included 30 pediatricians, 19 nurses or nurse practitioners, 5 medical assistants, and 37 office/administrative staff, all of whom agreed to participate in this study.

Staff

Nurses

Two registered nurses with at least 1 year of medical-surgical experience with children in a pediatric setting, but without prior mental health training, were hired and then trained to administer both PONI and EUC in three practices each. Nurses received 4 months of intensive ‘on-the-job’ training and ongoing supervision in administering all screening procedures and both intervention conditions using several methods (e.g., didactic seminar, observation of taped and live applications, role playing, in vivo practice, feedback on videotaped clinical interactions specific to PONI and EUC). Nurses were also trained to complete a study database, interact with all office staff and Primary Care Physicians (PCPs), and participate in study case review and office practice meetings. Supervision by a Masters’-level clinician with experience administering the protocol was supplemented by access to a part-time child and adolescent psychiatrist for diagnostic/medical and medication questions. Each nurse was assigned to three practices, where they assisted with screening and initial assessments for all study participants prior to randomization and then completed treatment condition-specific tasks. The nurse’s daily, on-site schedule was based on practice demands, PCP preferences, patient flow, and office availability.

Pediatricians

All 30 of the available pediatricians agreed to participate in this study by completing background and case status forms, and by responding to the needs or suggestions of the nurse. Each pediatrician confirmed a willingness to follow the parameters of the two intervention conditions.

Research Assessors

Two BA-level, research-assessment staff, naive to the child’s assigned intervention, administered all research assessments. One staff member administered the rating scales while the other conducted the diagnostic interview. The assessors received extensive training to administer all measures and identify safety risks, and participated in weekly supervision and reviewed diagnostic interviews with the supervisor and study psychiatrist.

Screening Procedures for Sample Ascertainment and Randomization

Potential subjects with BPs were identified in three ways. Two options included direct referral by the child’s PCP to the study nurse or direct parental requests for a referral for treatment of BP which were also forwarded to the nurse. Parents of these children either completed (on-site) or were administered (by phone) a 6-item BP screening (Externalizing scale on the Pediatric Symptom Checklist-17; Gardner et al., 1999) which, if positive, followed a brief screening interview to confirm eligibility. The third option involved asking parents to complete the screening form attached to a study flyer that was distributed in the practice waiting room. The waiting room form was distributed by the study nurse or a research assistant (when the nurse was unavailable); however, only the study nurse completed a follow-up screening interview for those cases with positive screen. The follow-up screening interview was conducted to establish the four inclusion criteria (age, referral concern for BP, severity of externalizing behavior, study interest). Per practice preference, we also sought and received an endorsement for study participation from each child’s PCP.

About 1 week after the screening interview, eligible children and their parents who were amenable were scheduled for a pretreatment research assessment conducted by the two RAs who administered several standardized rating scales and a diagnostic interview. Upon completion of the research assessment, the nurse identified those children who did not meet any of the three exclusions (e.g., emergent psychiatric symptoms, need for an alternative treatment, parallel treatment). All fully eligible subjects were then enrolled (informed consent/asset forms completed) and then randomized to PONI or EUC. Of the 2,301 children referred for screening, 578 met the behavior problems screening criterion, 178 completed the full diagnostic/research assessment, and 163 were randomized to PONI or EUC.

Assessment Procedures

Four research assessments were conducted with each family every 6 months (pretreatment, posttreatment, 6- and 12-month follow-up), each lasting about 2 hr for children and 2.75 hr for caregivers. Each informant was paid in increasing amounts for these assessments ($20, $30, $40, $50). All rating scales and diagnostic interviews were collected at the four research assessments. The pretreatment assessment was completed in the office practice, whereas the other assessments could be completed in a setting selected by the caregiver (e.g., pediatric practice, family residence, study office).

Child Outcomes: Standardized Rating Scales of Child Dysfunction and Child Health

Although several outcome variables from different informants were reported in the initial outcome study, two-key outcome variables were selected for evaluation based on the primary care literature, each from a different informant, to minimize the number of tests. The first outcome variable was based on the parent-reported Strength and Difficulties Questionnaire (SDQ; Bourdon, Goodman, Rae, Simpson, & Koretz, 2005) which includes four child problems scales (e.g., conduct, emotional, hyperactive/inattentive, peer problems) and a prosocial behavior scale. All 35 items are rated on 0–2 point Likert scales. As recommended, we reported the aggregate score for the four child difficulties subscales with higher scores reflecting greater child dysfunction. The individual subscales in the SDQ have excellent psychometric properties, including criterion and predictive validity (Goodman & Goodman, 2009).

The second outcome variable was based on the child-reported Child Health and Illness Profile (CHIP; Rebok et al., 2001), which provides an assessment of health-related behaviors. The CHIP includes 45 items tapping different behavioral, emotional, and physical aspects of health that are rated on 5-point scales which yield scores for five factors, namely, satisfaction (i.e., general well-being in terms of one’s health and self-esteem), comfort (i.e., absence of physical problems or activity limitations, positive emotions), resilience (i.e., positive interpersonal and intrapersonal activities that promote health, such as family involvement, social problem solving, and physical activity), risk avoidance (i.e., absence of behaviors that may be risks to future health or development), and achievement (i.e., successful performance both academically in school and socially with peers). We report an aggregate score based on all five scales, with higher scores reflecting better overall health. The measure yields t-scores for interpretation and has good psychometric properties, including criterion validity (Riley et al., 2004) and predictive validity to future health care use (Forrest, Riley, Vivier, Gordon, & Starfield, 2004).

Child Outcome Variables: Diagnostic Interviews for Psychiatric Disorders

Diagnostic interviews were completed by a research assessor to further describe the clinical severity of the sample and evaluate outcome. The Schedule for Affective Disorders and Schizophrenia for school-aged children for DSM-IV-Present and Lifetime (Kiddie-SADS-PL; Kaufman, Birmaher, Brent, Rao, & Ryan, 1996) was administered by trained diagnostic interviewers to detect current Axis I-child psychiatric disorders. Parents and children were interviewed separately and then jointly to address any discrepancies. Inter-rater reliabilities for a subset of 36 cases were high (k’s = 0.75–1.00), which parallel the results for a similar clinical trial with ODD or CD children (Kolko et al., 2009). We examine the rates for ODD here.

Moderator Variables

Demographic Background

Based on the pediatric literature, five background factors were examined: child age as a proxy for general developmental level, child gender, child ethnicity (Caucasian vs. minority); caregiver education (no college vs. any college), and household income.

Child Dysfunction and Impairment

One child variable was examined, namely, having two or more Axis I-psychiatric disorders to reflect the presence of comorbidity.

Parent Dysfunction

Two parent self-report variables were selected: reports on the three scales that aggregate to form a negative parenting practices scores on the Alabama Parenting Questionnaire (APQ; Shelton, Frick, & Wooten, 1996), given its role in the emergence and treatment of child BP, and reports of the level of importance to learning-parent management skills (parenting) as a proxy for parental efficacy based on the Credibility of Treatment Scale (COTS; Kolko et al., 2009).

Predictor Variables

Demographic Background

Based on the reviewed pediatric literature, two background factors were examined: child ethnicity (Caucasian vs. minority), and household income.

Child Dysfunction and Impairment

We selected three child variables. One was the severity of child depression on the 11-item Short Mood and Feelings Questionnaire (SMFQ; Angold et al., 1995). The second was the severity of anxiety on the 5-item brief screen from the Scale for Anxiety and Related Emotional Disorders which assesses the severity DSM-IV anxiety disorders in children and adolescents. The last score was the level of child functional impairment in the family, peer, work, and school domains evaluated on the 13-item, parent-completed Columbia Impairment Scale (CIS; Bird, Shaffer, Fisher, & Gould, 1993).

Parent and Family Dysfunction

Three variables were selected based on prior research. One reflected the negative parenting practices aggregate sore (e.g., corporal punishment, inconsistent discipline) from the parent-completed APQ (Shelton et al., 1996). Each practice is rated on a 1–5 point Likert scale. We also included the total score for the 21-item Caregiver Strain Questionnaire (CGSQ) which reflects perceived caregiver burden due to having a child with BP (Brannan, Heflinger, & Bickman, 1997). The 46-item Life Events Checklist (LECL) was used to capture the child’s exposure to common events that occur within the family (Tiet et al., 2001), most of which are considered as general family adversities (e.g., mother or father lost job, failing a grade). We report the net total positive events reported on this measure. Finally, we included the 9-item conflict-scale score from the Family Environment Scale (FES; Moos, Insel, & Humphrey, 1974) to reflect on the level of exposure to family conflict in the home. Descriptive information for all of these moderator and predictor variables at study intake is shown in Table I.

Table I.

Descriptive Statistics for All Moderators, Predictors, and Treatment Parameters (n = 163)

Variable M (SD) n (%)
Moderators
    Child age (years) 8.6 (1.7)
    Child gender (male) 106 (65)
    Child ethnicity (Caucasian)a 130 (79.8)
    Caregiver education (any college) 129 (83)
    Household income (in thousands)a 58.1 (24.9)
    Child diagnostic comorbidity (≥2 Axis I disorders) 107 (65.6)
    Alabama Parenting Questionnaire: Negative Parenting Practices 11.5 (2.1)
    Credibility of Treatment: Importance of Parenting 13.5 (3.9)
Predictors
    Short Mood and Feelings Questionnaire 16.2 (10.3)
    Scale for child and anxiety-related disorders—5-item screen 1.9 (1.8)
    Columbia Impairment Scale score 18.7 (7.2)
    Caregiver Strain Questionnaire score 52.7 (13.4)
    Life events checklist: net total positive events −2.4 (4.9)
    Family Environment Scale: Conflict scale score 2.8 (2)
Treatment Parameters
    Treatment Summary Report: Total Hours (≥6) 57 (47.5)
    Services Provided Log: CBT (>3) 43 (54.4)
    Services Provided Log: PMT (≥6) 42 (53.2)
    Average Treatment Fidelity Rating 91.4 (9.1)
Open in a new tab

aTested as predictor as well.

Treatment Parameters

Dose, Content, and Fidelity

We included four variables in this domain. For both conditions, the overall number of hours of treatment (dose) delivered was taken from a Treatment Summary Report (TSR) completed either by the nurse (PONI) or an outside provider that could include the child’s PCP (EUC). For PONI only, the nurses recorded the number of hours spent delivering each treatment module or service (in 15-min blocks) to each case on a Services Provided Log (SPL) developed for this study. The two variables reported from this log reflected the number of hours they delivered parent management training (PMT) and child cognitive-behavioral treatment, which reflected the two primary treatment modules used in PONI. Finally, we included a summary of the supervisor’s treatment fidelity ratings reflecting the mean percentage of correctly administered content or procedures in the two treatment protocols to examine the role of treatment adherence on outcome.

Treatment Procedures and Conditions

General Procedures

In both conditions, nurses reviewed the results of the pretreatment research assessment rating scales and diagnostic interviews, discussed specific treatment recommendations, and provided a written summary to the parent and PCP that was placed in the chart. They also attended practice group meetings and applied study procedures in accord with practice or family input to enhance study viability (e.g., local referral source lists, personalized use of ADHD medication guidelines). None of these adaptations changed the substantive parameters of assessment or intervention.

PONI

In PONI, nurses delivered the protocol in the practice and collaborated with the PCP. The protocol was designed to be administered on an individualized basis in 6, 1.5 hr visits generally over the course of 3 months, but in no more than 6 months. Phase one of treatment consisted primarily of cognitive-behavioral skills training for children and parents (M = 10.8 hr, SD = 4.1). Phase two was initiated for families needing follow-up or maintenance to ensure clinical progress on key targets (M = 2.1 hr, SD = 2.3). They also referred cases to outside services for aftercare only after the posttreatment assessment.

The protocol consisted of seven brief treatment modules (Kolko et al., 2010) found to be efficacious in prior outcome studies for behavior disorders (Kolko, 1995; Kolko et al., 2009), child firesetters (Kolko, Watson, & Faust, 1991), and aggressive/abusive families (Kolko, 1996), many of which have been integrated in Alternative for Families: A Cognitive-Behavioral Therapy (AF-CBT, see afcbt.org). The following modules each included several core components or skill areas directed to children, parents, or both participants together: Child cognitive-behavioral treatment (CBT; training in anger control, self-instructions, or social skills to promote self-management and prosocial behavior), an optional ADHD medication trial (e.g., review of ADHD symptom ratings, medication recommendations, PCP medication prescription, follow-up session to review ratings), enhanced PMT [training in positive reinforcement, noncorporal discipline involving contingency management (e.g., withdrawal of privileges), and anger management skills], developmental expectations and peer enrichment (i.e., information on child development and age-appropriate behavioral expectations, training to promote child involvement in peer and recreational activities), family psychoeducation and skills training (PAST; discussions of family rules and decision making, communication and problem-solving skills training as alternatives to coercion or physical force), school/teacher consultation (e.g., individualized behavioral suggestions based on teacher ratings to facilitate classroom management or educational remediation), case and crisis management (i.e., calls to help families respond to challenges and crises, advocacy with local service agencies and professionals; see Kolko et al., 2010 for a description of this content).

The PMT, CBT, and PAST modules were administered first to teach basic skills, followed next by the remaining modules, as needed, to address problems related to ADHD, school misbehavior, problematic peer/community relations, and/or family crises/conflicts. The extent to which these latter modules were administered was based on the severity of the child’s identified BP and the family’s perceived skill repertoire. Services were terminated when the family sought to end care, met its treatment goals, or reached the 6-month service limit. A total of 80 (94%) PONI cases received services. Medication was prescribed for 17 children with ADHD, all by the child’s PCP. In a few cases, families who requested or were thought to benefit from other services were referred to an appropriate community provider prior to posttreatment (6-month) assessment.

EUC

In EUC, the nurses offered treatment recommendations following the diagnostic assessment and facilitated the child’s outside referral by directly contacting a local community provider who accepted the family’s insurance and had expertise with BP using a list of approved providers and based on parental approval. This information was conveyed to the PCP and documented in the medical record. EUC services were diverse across subjects and participating practices. A total of 34 (42.5%) cases received services from 24 outside mental health specialists and 8 of the children’s PCPs.

Procedures for Maintaining and Documenting Treatment Integrity

Several procedures were administered to uphold the integrity of the PONI and EUC conditions (e.g., use of manuals, didactic training, supervisor review of audiotapes/progress notes, weekly team meetings, individual supervision). For PONI, the supervisor reviewed a set of randomly selected session tapes (30% of 704 or 211) and rated the number of correctly administered treatment task components per topic covered to document treatment integrity. The supervisor documented a high level of adherence to the treatment protocol (M = 91%; range = 68–100%). For EUC, the nurse also completed routine progress notes and an abbreviated Services Provided Log to document any family contacts. Where consent was available, all records and audiotapes of selected EUC feedback sessions (15% of 230 or 35) were reviewed by the supervisor for integrity monitoring of key tasks (e.g., gave recommendations, made facilitated referral). Integrity ratings were also high in EUC (M = 94%; range = 87–100%).

Data Analysis

We included all enrolled subjects in our analyses, per the intent-to-treat principle. Descriptive analyses were conducted for all pretreatment moderator and predictor variables, the treatment parameters, and the two rating scale outcomes at all time points. We ran a series of analyses investigating whether the specific variables moderated, mediated, or predicted the treatment outcomes. To facilitate interpretation, continuous putative moderators, mediators, or predictors were standardized by taking the difference between each observation and the mean of all observations for each variable and dividing by their standard deviations. We refer to them as “high” or “low” which corresponded to 1 SD above or below their average levels. All of the analyses were also conducted using presence/absence of ODD at each time point as the third outcome.

For moderator analysis, we applied mixed effects models to the two aggregate scale outcomes at all time points. The treatment assignment, period as a factor (baseline, posttreatment, 6-month and 12-month follow-up), a potential moderator, the treatment by period, treatment by moderator, and moderator by period interactions, and the treatment by period by moderator interaction were treated as fixed terms and subject was treated as a random term to account for individual subject variability. If the coefficients associated with the treatment by moderator interaction and treatment by period by moderator interaction were significant, contrasts were set up to test how the effects of PONI over EUC across time (changes in composite outcomes from baseline) varied across the subgroups formed by the moderator.

For predictor analyses, we used mixed effects models with terms including a potential predictor, period, and the predictor by period interaction. A significant predictor by period interaction suggests the predictor is associated with the changes in composite outcomes over time. Next, mediator analyses were explored following the guidelines outlined by (Kraemer et al., 2002) who indicated that a risk factor mediates another factor if it follows and is correlated with the other factor, and the effect of this risk factor and/or interaction with the other factor is significantly nonzero. Lastly, we investigated how four treatment parameters (dose, two content-related variables in PONI, treatment fidelity) influenced posttreatment outcome using regression models. Because we tested eight moderators and predictors each, we opted to use a more stringent significance level of .01 as suggested by Lavigne, LeBailly, Gouze, Cicchetti, Jessup et al., 2008, but maintained a conventional significance level to evaluate the four PONI treatment parameters.

Results

Moderator Analyses

We examined the role of eight pretreatment variables as potential moderators. Only one of the models involving the two child-rating scale outcomes had a significant treatment by moderator interaction or three-way interaction. We further examined this model using contrasts to see whether there was a significant treatment by moderator interaction in the changes that occurred from baseline to posttreatment, 6-month follow-up, and 12-month follow up. One significant interaction was found involving treatment condition × child ethnicity on the change in overall child dysfunction from baseline to 12-month follow-up (est = 8.90, t = −4.05, df = 559, p < .001). The means and SDs of the changes found in child dysfunction from baseline to 12 months were: (a) PONI, Caucasian children (M = −7.00, SD = 1.31), (b) EUC, Caucasian children (M = −0.04, SD = 1.46), (c) PONI, non-Caucasian (M = −2.95, SD = 0.65), and (d) EUC, non-Caucasian (M = −4.89, SD = 0.72). As depicted in Figure 1, the results suggested that PONI was more effective than EUC in reducing child dysfunction by 12-month follow-up among Caucasian children, whereas EUC was more effective than PONI among non-Caucasian children.

Figure 1.

Figure 1.

Open in a new tab

Moderator effects of child ethnicity on reductions in SDQ total scores from pretreatment to 12-month follow-up in the PONI and in EUC.

We also examined treatment moderators for remission from ODD (n = 100) in the subset of cases that were so diagnosed at intake. Of these 100 children, 86 had remitted from the disorder. Mixed model analyses found no significant treatment moderators of remission in ODD.

Predictor Analyses

Next, we examined whether any of the eight proposed variables predicted overall outcomes when patients in the two conditions were combined. Figure 2 presents graphs representing the means at each time point for high and low level of each of four continuous predictor variables that were significantly related to overall outcome. Three variables predicted level of improvement in child health by 12-month follow-up, namely, severity of self-reported depression, F (3,421) = 10.4, p < .001, severity of self-reported anxiety, F (3,422) = 8.78, p < .001, and level of parent-reported family conflict, F (3,417) = 5.90, p < .001. That is, children who were high in level of depression, anxiety, and exposure to family conflict showed a greater increase in overall health by 12-month follow-up than those children who were classified as low in the levels of these three variables. Children who were high in level of functional impairment showed a greater reduction in severity of child dysfunction by 12-month follow-up than those with a low level of impairment, F (3,445) = 8.04, p < .001. However, it bears mentioning that the children with higher (vs. lower) levels of all four of these background characteristics still had worse postreatment outcomes. There were no significant predictors of remission in ODD.

Figure 2.

Figure 2.

Open in a new tab

Means for the high and low levels of each of four continuous-predictor variables that were significantly related to the primary child outcomes.

Mediational Analyses

Tests of potential mediators of outcome which were constructed using all observations pooled across the four time points (pre- and posttreatment, 6- and 12-month follow-up). First, the three conditions for a mediational model were explored. The relationships between treatment condition and all three outcomes were not significantly different through follow-up. Therefore, no further tests of mediation were conducted.

Treatment Parameter Analyses

The four treatment parameters (e.g., total hours, amount of exposure to CBT and PMT content, fidelity ratings) were examined as potential contributors to clinical outcome at posttreatment in PONI only: the two continuous rating scale aggregates were divided into “low” versus “high” groups using a median split. The distributions of the categorized variables across the two treatment groups had modest variability. Next, linear regression models were fitted to determine whether the four categorized variables and treatment condition were associated with posttreatment outcomes in PONI or EUC. For PONI, those who received more hours of child CBT showed a greater improvement in overall child health (−2.58) than those who received fewer hours of child CBT (0.01), t = 4.34, p < .05. Regarding remission from ODD at intake, we found no significant relationships among PONI cases and only one significant relationship among EUC cases. In EUC, those who completed treatment had a higher rate of remission from ODD than those who did not complete treatment (62% vs. 18%, p = .003).

Discussion

This study evaluated several variables proposed as treatment moderators or predictors, and treatment-specific parameters, in relation to two key outcomes collected in a randomized clinical trial for young children with BPs that was conducted in the pediatric primary care setting (Kolko et al., 2010). The initial trial compared a PONI with an EUC condition consisted of a facilitated referral to a local mental health provider. Given the relative comparability of the two treatments, the current analyses sought to identify child, parent, or family variables related to long-term clinical outcomes on two sets of measures. Aggregate outcomes reflecting reports of child dysfunction or health were related to one moderator, four predictors, and one treatment parameter, whereas remission in ODD was related to only one treatment parameter. The nature and implications of these findings for extending the application of mental health treatments to the pediatric primary care are discussed.

Of eight child, parent, and family treatment moderators reported in the pediatric literature, only one significant moderator was identified in this study. Specifically, PONI was more effective in improving self-reports of the children’s health at 12-month follow-up in Caucasian children, whereas EUC was more effective among non-Caucasian children. Although this finding was not anticipated given the lack of ethnic differences in outcome in our prior studies (Kolko, 1996; Kolko et al., 2009, 2010), it is possible that the facilitated referral options available only in the EUC condition-permitted families to have more choice in their selection of an outside community provider who may have been more attentive to the background or needs of ethnic minority children. As was found in our initial outcome study, more than 80% of the cases enrolled in the study were Caucasian, which may have limited the nurse’s experience (e.g., ability to engage, address clinical problems) in serving ethnic minority children. At the same time, treatment outcome did not vary by severity of child BP, parenting practices, parental dysfunction, and family and peer problems, or child demographics or family disadvantage, which is consistent with several outcome studies or reviews (Baydar, Reid, & Webster-Stratton, 2003; Beauchaine et al., 2005; Lavigne, LeBailly, Gouze, Cicchetti, Jessup et al., 2008; Rickards, Walstab, Wright-Rossi, Simpson, & Reddihough, 2007; van den Hoofdakker et al., 2010). The overall absence of moderators for each treatment highlights the robust and comparable effects found for both the primary care-based nurse administered treatment and facilitated referral to a local mental health provider for BP children. Accordingly, both treatments would be expected to yield similar improvements for cases varying in the severity of various child, parent, and family problems (Kazdin & Whitley, 2006; Kolko, 2009).

Three child functioning variables and one family variable were found to predict the two aggregate outcomes in the full sample. Consistent with some recent outcome studies (Beauchaine et al., 2005; Lavigne, LeBailly, Gouze, Cicchetti, Jessup et al., 2008), children with a higher level of functional impairment showed greater improvement in child dysfunction scores than those with lower levels of impairment. This was the only predictor of parent-reported child difficulties or problems. Children with higher initial severity of internalizing problems (i.e., depression and anxiety) or level of exposure to family conflict also showed a greater increase in child health than those with lower levels of these three characteristics. The findings regarding internalizing problems are also consistent with those reported in a treatment study with ODD children (Lavigne, LeBailly, Gouze, Cicchetti, Pochyly et al., 2008). This may suggest that comorbid-internalizing symptoms, which reflected a heightened level of personal distress may somehow facilitate treatment participation or response in children presenting with BP, although the specific explanation for this finding requires more definitive evaluation. Likewise, children from families with higher levels of family conflict or chaos may benefit more from family-based or -centered services that target family support and interactional patterns, often using parent–child sessions to enhance communication and problem-solving repertoires. Of course, no evidence was found that child or parent demographics, parental distress, or parenting practices dysfunction were related to overall outcome, suggesting that PONI and EUC are equally helpful for those who differ in background and caregiver functioning, a conclusion that differs from some of the results of other studies in this area (Lavigne, LeBailly, Gouze, Cicchetti, Pochyly et al., 2008) or in other areas (Kendall, Hudson, Gosch, Flannery-Schroeder, & Suveg, 2008).

None of the four treatment parameters was related to any of the child outcomes, except one. The duration of time, the child spent receiving training in CBT skills during PONI was related to the child’s health at posttreatment, raising the question as whether improvement in children’s health-promoting behaviors is influenced by explicit training in some of the skills designed to encourage this behavior (e.g., self-control, assertion, social skills) which were targeted during the child CBT. Indeed, prosocial skills training in BP children has been associated with improved social adjustment (Kolko, Loar, & Sturnick, 1990). At the same time, treatment dose and fidelity were unrelated to either child dysfunction or child health. The absence of dose–response relationships is common in child outcome studies (Bickman et al., 2002).

Because the initial requirements for testing mediation were not met, tests of meditation were not conducted. Consequently, this study cannot determine the psychosocial or clinical processes whose targeting may underlie or cause subsequent clinical improvement. Other studies have found mixed evidence in terms of treatment mediators with some studies reporting that changes in negative-parenting mediate outcome in PCIT (Chaffin et al., 2004), or that loneliness mediates changes in social anxiety following behavioral treatment (Alfano et al., 2009). Unfortunately, few studies in primary care settings have reported the results of any tests for possible mediation.

The limitations of this study deserve mention. The EUC comparison condition was clearly enriched in terms of its content and duration relative to what was standard care in these practices, in part, because study PCPs preferred to compose treatments that parents would find highly acceptable and beneficial. The lack of group differences in the initial outcome study might also suggest some limited effectiveness to both treatments as their levels of observed improvement may be due to regression toward the mean. Perhaps the use of existing referral options in each practice as a comparison condition might have yielded more group differences. Second, it is possible that we did not include some potential moderators and predictors of outcome (e.g., medical status) from the general literature (Lundahl et al., 2006), though we included several variables across domains as reported in primary care studies.

In summary, this study examined several potential moderators and predictors of treatment outcome, and a few treatment parameters, in a clinical trial that compared a PONI with EUC consisting of a facilitated referral to an outside specialty mental health provider. Out of several diverse background variables, only child ethnicity was associated with differential response to the two treatments in terms of the severity ratings of child dysfunction, three variables predicted level of improvement in child health (greater severity of depression and anxiety, greater exposure to family conflict), and duration of child CBT was the only treatment parameter related to improved child health. In the context of these few significant relationships, the general findings of this study highlight the robustness of both treatment conditions and provide further support for the delivery of diverse clinical services to children presenting with BP in the pediatric primary setting (Borowsky et al., 2004; Hayes et al., 2008; Kolko, 2009; Lavigne, LeBailly, Gouze, Cicchetti, Jessup et al., 2008; Lavigne, LeBailly, Gouze, Cicchetti, Pochyly et al., 2008; Lavigne et al., 2010; Turner & Sanders, 2006), as well as to children with other clinical problems (Kolko, 2009). Further studies are needed to enhance the overall clinical impact, efficiency, and cost-effectiveness of behavioral health interventions in pediatric settings.

Funding

National Institute of Mental Health (Grant MH63272, partial).

Conflicts of interest: None declared.

Acknowledgments

We acknowledge the research and clinical staff of the Services for Kids in Primary Care Program (SKIP), the clinical and administrative staff affiliated with the participating pediatric practices from Children’s Community Pediatrics of Children’s Hospital of Pittsburgh, and the Advanced Center for Intervention Services Research (David Brent, MD, PI).

References

  1. Alfano C A, Pina A A, Villalta I K, Beidel D C, Ammerman R T, Crosby L E. Mediators and moderators of outcome in the behavioral treatment of childhood social phobia. Journal of the American Academy of Child & Adolescent Psychiatry September. 2009;48(9):945–953. doi: 10.1097/CHI.0b013e3181af8216. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Angold A, Costello E J, Messer S C, Pickles A, Winder F, Silver D. Development of a short questionnaire for use in epidemiological studies of depression in children and adolescents. International Journal of Methods in Psychiatric Research. 1995;5:237–249. [Google Scholar]
  3. Asarnow J R, Jaycox L H, Duan N, LaBorde A P, Rea M M, Murray P, Anderson M, Landon C, Tang L, Wells KB. Effectiveness of a qualtiy improvement intervention for adolescent depression in primary care clinics. Journal of the American Medical Association. 2005;293(3):311–319. doi: 10.1001/jama.293.3.311. [DOI] [PubMed] [Google Scholar]
  4. Baydar N, Reid M J, Webster-Stratton C. The role of mental health factors and program engagement in the effectiveness of a preventive parenting program for Head Start mothers. Child Development. 2003;74:1433–1453. doi: 10.1111/1467-8624.00616. [DOI] [PubMed] [Google Scholar]
  5. Beauchaine T P, Webster-Stratton C, Reid M J. Mediators, moderators, and predictors of 1-year outcomes among children treated for early-onset conduct problems: A latent growth curve analysis. Journal of Consulting & Clinical Psychology. 2005;73(3):371–388. doi: 10.1037/0022-006X.73.3.371. [DOI] [PubMed] [Google Scholar]
  6. Berkovits M D, O'Brien K A, Carter C G, Eyberg S M. Early identification and intervention for behavior problems in primary care: A comparison of two abbreviated versions of parent-child interaction therapy. Behavior Therapy. 2010;41(3):375–387. doi: 10.1016/j.beth.2009.11.002. [DOI] [PubMed] [Google Scholar]
  7. Bickman L, Andrade A R, Lambert E W. Dose response in child and adolescent mental health services. Mental Health Services Research. 2002;4(2):57–70. doi: 10.1023/a:1015210332175. [DOI] [PubMed] [Google Scholar]
  8. Bird H, Shaffer D, Fisher P, Gould M S. The Columbia impairment scale (CIS): Pilot findings on a measure of global impairment for children and adolescents. International Journal of Methods in Psychiatric Research. 1993;3:167–176. [Google Scholar]
  9. Borowsky I W, Mozayeny S, Stuenkel K, Ireland M. Effects of a primary care-based intervention on violent behavior and injury in children. Pediatrics. 2004;114(4):e392–399. doi: 10.1542/peds.2004-0693. [DOI] [PubMed] [Google Scholar]
  10. Bourdon K, Goodman R, Rae D, Simpson G, Koretz D S. The strengths and difficulties questionnaire: U.S. normative data and psychometric properties. Journal of the American Academy of Child & Adolescent Psychiatry. 2005;44(6):557–564. doi: 10.1097/01.chi.0000159157.57075.c8. [DOI] [PubMed] [Google Scholar]
  11. Brannan A M, Heflinger C A, Bickman L. The Caregiver Strain Questionnaire: Measuring the impact on the family of living with a child with serious emotional disturbance. Journal of Emotional & Behavioral Disorders. 1997;5(4):212–222. [Google Scholar]
  12. Chaffin M, Silovsky J F, Funderburk B, Valle L A, Brestan E V, Balachova T, Jackson S, Lensgraf J, Bonner BL. Parent-child interaction therapy with physically abusive parents: Efficacy for reducing future abuse reports. Journal of Consulting & Clinical Psychology. 2004;72(3):500–510. doi: 10.1037/0022-006X.72.3.500. [DOI] [PubMed] [Google Scholar]
  13. Forrest C B, Riley A W, Vivier P M, Gordon N P, Starfield B. Predictors of children's healthcare use: The value of child versus parental perspectives on healthcare needs. Medical Care. 2004;42(3):232–238. doi: 10.1097/01.mlr.0000115196.02425.05. [DOI] [PubMed] [Google Scholar]
  14. Gardner F, Connell A, Trentacosta C J, Shaw D S, Dishion T J, Wilson M N. Moderators of outcome in a brief family-centered intervention for preventing early problem behavior. Journal of Consulting and Clinical Psychology. 2009;77(3):543–553. doi: 10.1037/a0015622. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Gardner W, Murphy M, Childs G, Kelleher K, Pagano M, Jellinek M S, McInerny T, Wasserman M, Nutting P, Chiapetta L. The PSC-17: A brief pediatric symptom checklist with psychosocial problem subscales. A report from PROS and ASPN. Ambulatory Child Health. 1999;5:225–236. [Google Scholar]
  16. Goodman A M, Goodman R. Strengths and difficulties questionnaire as a dimensional measure of child mental health. Journal of the American Academy of Child & Adolescent Psychiatry. 2009;48(4):400–403. doi: 10.1097/CHI.0b013e3181985068. [DOI] [PubMed] [Google Scholar]
  17. Hayes L, Matthews J, Copley A, Welsh D. A randomized controlled trial of a mother-infanct or toddler parenting program: Demonstrating effectiveness in practice. Journal of Pediatric Psychology. 2008;33(5):473–486. doi: 10.1093/jpepsy/jsm085. [DOI] [PubMed] [Google Scholar]
  18. Hinshaw S P. Moderators and mediators of treatment outcome for youth with ADHD: Understanding for whom and how interventions work. Ambulatory Pediatrics. 2007;7(1S):91–100. doi: 10.1016/j.ambp.2006.04.012. [DOI] [PubMed] [Google Scholar]
  19. Kaufman, Birmaher B, Brent D A, Rao U, Ryan N D. KIDDIE-SADS-present and lifetime version (K-SADS-PL) Pittsburgh, PA: Instrument developed at Western Psychiatric Institute and Clinic; 1996. [Google Scholar]
  20. Kazdin A E, Whitley M K. Comorbidity, case complexity, and effects of evidence-based treatment for children referred for disruptive behavior. Journal of Consulting and Clinical Psychology. 2006;74(3):455–467. doi: 10.1037/0022-006X.74.3.455. [DOI] [PubMed] [Google Scholar]
  21. Kelleher K, Stevens J. Evolution of child mental health services in primary care. Academic Pediatrics. 2009;9(1):7. doi: 10.1016/j.acap.2008.11.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Kelley S, de Andrade A, Sheffer E, Bickman L. Exploring the black box: Measuring youth treatment process and progress in usual care. Administration and Policy in Mental Health and Mental Health Services Research. 2010;37(3):287–300. doi: 10.1007/s10488-010-0298-8. [DOI] [PubMed] [Google Scholar]
  23. Kendall P C, Hudson J L, Gosch E, Flannery-Schroeder E, Suveg C. Cognitive-behavioral therapy for anxiety disordered youth: A randomized clinical trial evaluating child and family modalities. Journal of Consulting and Clinical Psychology. 2008;76(2):282–297. doi: 10.1037/0022-006X.76.2.282. [DOI] [PubMed] [Google Scholar]
  24. Kolko D J. Multimodal partial/day treatment of child antisocial behavior: Service description and multilevel program evaluation. Continuum. 1995;2:3–24. [Google Scholar]
  25. Kolko D J. Individual cognitive-behavioral treatment and family therapy for physically abused children and their offending parents: A comparison of clinical outcomes. Child Maltreatment. 1996;1:322–342. [Google Scholar]
  26. Kolko D J. Options for the delivery of mental health services. In: McInerny T K, editor. American academy of pediatrics textbook of pediatric care. Elk Grove Village, IL: American Academy of Pediatrics; 2009. [Google Scholar]
  27. Kolko D J, Campo J V, Kelleher K, Cheng Y. Improving access to care and clinical outcome for pediatric behavioral problems: A randomized trial of a nurse-administered intervention in primary care. Journal of Developmental & Behavioral Pediatrics. 2010;31(5):393–404. doi: 10.1097/DBP.0b013e3181dff307. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Kolko D J, Dorn L D, Bukstein O G, Pardini D A, Holden E A, Hart J D. Community vs. clinic-based modular treatment of children with early-onset ODD or CD: A clinical trial with three-year follow-up. Journal of Abnormal Child Psychology. 2009;37:591–609. doi: 10.1007/s10802-009-9303-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Kolko D J, Loar L L, Sturnick D. Inpatient social-cognitive skills training groups with conduct disordered and attention deficit disordered children. Journal of Child Psychology and Psychiatry. 1990;31:737–748. doi: 10.1111/j.1469-7610.1990.tb00814.x. [DOI] [PubMed] [Google Scholar]
  30. Kolko D J, Watson S, Faust J. Fire safety/prevention skills training to reduce involvement with fire in young psychiatric inpatients: Preliminary findings. Behavior Therapy. 1991;22(2):269–284. [Google Scholar]
  31. Kraemer H C, Wilson G T, Fairburn C G, Agras W S. Mediators and moderators of treatment effects in randomized clinical trials. Archives of General Psychiatry. 2002;59(10):877–884. doi: 10.1001/archpsyc.59.10.877. [DOI] [PubMed] [Google Scholar]
  32. La Greca A M, Silverman W K, Lochman J E. Moving beyond efficacy and effectiveness in child and adolescent intervention research. Journal of Consulting and Clinical Psychology. 2009;77(3):373–382. doi: 10.1037/a0015954. [DOI] [PubMed] [Google Scholar]
  33. Lavigne J V, LeBailly S A, Gouze K R, Binns H J, Keller J, Pate L. Predictors and correlates of completing behavioral parent training for the treatment of oppositional defiant disorder in pediatric primary care. Behavior Therapy. 2010;41(2):198–211. doi: 10.1016/j.beth.2009.02.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Lavigne J V, LeBailly S A, Gouze K R, Cicchetti C, Jessup B W, Arend R, Pochyly J, Binns H J. Predictor and moderator effects in the treatment of oppositional defiant disorder in pediatric primary care. Journal of Pediatric Psychology. 2008;33(5):462–472. doi: 10.1093/jpepsy/jsm075. [DOI] [PubMed] [Google Scholar]
  35. Lavigne J V, LeBailly S A, Gouze K R, Cicchetti C, Pochyly J, Arend R, Jessup B W, Binns H J. Treating oppositional defiant disorder in primary care: A comparison of three models. Journal of Pediatric Psychology. 2008;33(5):449–461. doi: 10.1093/jpepsy/jsm074. [DOI] [PubMed] [Google Scholar]
  36. Lundahl B, Risser H J, Lovejoy M C. A meta-analysis of parent training: Moderators and follow-up effects. Clinical Psychology Review. 2006;26(1):86–104. doi: 10.1016/j.cpr.2005.07.004. [DOI] [PubMed] [Google Scholar]
  37. Moos R H, Insel P M, Humphrey B. Family work and group environment scales. Palo Alto, CA: Consulting Psychologists Press; 1974. [Google Scholar]
  38. Rebok G, Riley A, Forrest C, Starfield B, Green B, Robertson J, Tambor E. Elementary school-aged children's reports of their health: A cognitive interviewing study. Quality of Life Research: An International Journal of Quality of Life Aspects of Treatment, Care & Rehabilitation. 2001;10:59–70. doi: 10.1023/a:1016693417166. [DOI] [PubMed] [Google Scholar]
  39. Reyno S M, McGrath P J. Predictors of parent training efficacy for child externalizing behavior problem: A meta-analytic review. Journal of Child Psychology and Psychiatry. 2006;47(1):99–111. doi: 10.1111/j.1469-7610.2005.01544.x. [DOI] [PubMed] [Google Scholar]
  40. Rickards A L, Walstab J E, Wright-Rossi R A, Simpson J, Reddihough D S. A randomized controlled trial of a home-based intervention program for children with autism and developmental delay. Journal of Developmental and Behavioral Pediatrics. 2007;28(4):308–316. doi: 10.1097/DBP.0b013e318032792e. [DOI] [PubMed] [Google Scholar]
  41. Riley A W, Forrest C B, Rebok G W, Starfield B, Green B F, Robertson J A, Friello P. The child report form of the CHIP-child edition: Reliability and validity. Medical Care. 2004;42(3):221–231. doi: 10.1097/01.mlr.0000114910.46921.73. [DOI] [PubMed] [Google Scholar]
  42. Shelton K K, Frick P J, Wooten J. Assessment of parenting practices in families of elementary school-age children. Journal of Clinical Child Psychology. 1996;25:317–329. [Google Scholar]
  43. Tiet Q Q, Bird H R, Hoven C W, Moore R, Wu P, Wicks J, Jensen P, Goodman S, Cohen P. Relationship between specific adverse life events and psychiatric disorders. Journal of Abnormal Child Psychology. 2001;29(2):153–164. doi: 10.1023/a:1005288130494. [DOI] [PubMed] [Google Scholar]
  44. Turner K M T, Sanders M R. Help when it's needed first: A controlled evaluation of brief, preventive behavioral family intervention in a primary care setting. Behavior Therapy. 2006;37(2):131–142. doi: 10.1016/j.beth.2005.05.004. [DOI] [PubMed] [Google Scholar]
  45. van den Hoofdakker B J, Nauta M H, van der Veen-Mulders L, Sytema S, Emmelkamp P M G, Minderaa R B, Hoekstra P J. Behavioral parent training as an adjunt to routine care in children with attention-deficit/hyperactivity disorder: Moderators of treatment response. Journal of Pediatric Psychology. 2010;35(3):317–326. doi: 10.1093/jpepsy/jsp060. [DOI] [PubMed] [Google Scholar]
  46. Voorhees B W, Fogel J, Reinecke M, Gladstone T, Stuart S, Gollan J, Bradford N, Domanico R, Fagan B, Ross R, Larson J, Watson N, Paunesku D, Melkonian S, Kuwabara S, Holper T, Shank N, Saner D, Butler A, Chandler A, Louie T, Weinstein C, Collins S, Baldwin M, Wassel A, Vanderplough-Booth K, Humensky J, Bell C. Randomized clinical trial of an Internet-based depression prevention program for adolescents (Project CATCH-IT) in Primary Care: 12-Week outcomes. Journal of Developmental and Behavioral Pediatrics. 2009;30(1):14. doi: 10.1097/DBP.0b013e3181966c2a. [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Wolraich M L, Bickman L, Lambert E W, Simmons T, Doffing M A. Intervening to improve communication between parents, teachers, and primary care providers of children with ADHD or at high risk for ADHD. Journal of Attention Disorders. 2005;9(1):354–368. doi: 10.1177/1087054705278834. [DOI] [PubMed] [Google Scholar]

Articles from Journal of Pediatric Psychology are provided here courtesy of Oxford University Press

RESOURCES