Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2019 Aug 5.
Published in final edited form as: Prev Sci. 2017 Apr;18(3):305–311. doi: 10.1007/s11121-016-0689-5

Treatment Effects of a Primary-Care Intervention on Parenting Behaviors: Sometimes it’s Relative

Anne Shaffer 1, Oliver Lindhiem 2, David Kolko 2
PMCID: PMC6681444  NIHMSID: NIHMS1043268  PMID: 27469458

Abstract

The goal of this brief report is to demonstrate the utility of quantifying parental discipline practices as relative frequencies in measuring changes in parenting behavior and relations to child behavior following intervention. We explored comparisons across methodological approaches of assessing parenting behavior via absolute and relative frequencies in measuring improvements in parent-reported disciplinary practices (increases in positive parenting practices in response to child behavior; decreases in inconsistent discipline and use of corporal punishment) and child behavior problems. The current study was conducted as part of a larger clinical trial to evaluate the efficacy of a collaborative care intervention for behavior problems, ADHD, and anxiety in pediatric primary care practices (Doctor Office Collaborative Care; DOCC). Participants were 321 parent-child dyads (M child age = 8.00, 65% male children) from 8 pediatric practices that were cluster randomized to DOCC or Enhanced Usual Care (EUC). Parents reported on their own discipline behaviors and child behavior problems. While treatment-related decreases in negative parenting were found using both the absolute and relative frequencies of parenting behaviors, results were different for positive parenting behaviors, which showed decreases when measured as absolute frequencies but increases when measured as relative frequencies. In addition, positive parenting was negatively correlated with child behavior problems when using relative frequencies, but not absolute frequencies, and relative frequencies of positive parenting mediated relations between treatment condition and outcomes. Our findings indicate that the methods used to measure treatment-related change warrant careful consideration.

Keywords: parenting, discipline, measurement, externalizing behavior

Currently, the need for mental health services among children outstrips the availability of specialty mental health, and mental health service delivery to youth is not always optimal (Kieling et al., 2011). Increasing access to mental health services via pediatric primary care is one potential solution to this public health need, and addresses the need to prevent later serious problems such as juvenile delinquency. The Services for Kids in Primary-care (SKIP; Kolko, Campo, Kilbourne, Hart, Sakolsky, & Wisniewski, 2014) treatment research program (www.skipprogram.org) integrates personalized behavioral health services in practice settings serving pediatric patients. The provision of these services in a primary care setting is well-suited to addressing early-developing behavior problems, with the ultimate goal of preventing later severe conduct problems and criminality. The current study, known as SKIP2, was designed to test a collaborative care model to be implemented in primary-care doctors’ offices, where mental health clinicians are trained as care managers to administer behavioral treatment modules designed to address child behavior problems.

The goal of this brief report is to provide a methodological replication and extension of previous findings (e.g., Lindhiem, Shaffer, & Kolko, 2014) in demonstrating the utility of quantifying parental discipline practices as relative frequencies (i.e., rates of specific discipline practices as a proportion of all discipline practices reported) in measuring changes in parenting behavior and relations to child behavior following intervention. The majority of parenting intervention research has employed an absolute frequency measurement approach, and there are many strong treatment effects documented in the extant literature regarding reductions in aggressive or ineffective discipline practices (Eyberg, Nelson, & Boggs, 2008; Kaminski, Valle, Filene, & Boyle, 2008). However, the literature also includes other counter-intuitive findings that must be reconciled. For example, aggressive and nonaggressive discipline are sometimes found to be positively correlated (e.g., Carr, Taylor, & Robinson, 1991), most likely due to multiple strategies being used with noncompliant children. For most parents, the selection of discipline strategies is not a binary choice. Finally, potential problems with ceiling or floor effects may plague parenting research, especially in the context of prevention trials, where clinical goals may be to prevent the occurrence or increase in aggressive parenting behaviors, but where statistically significant reductions in the absolute frequency of aggressive parenting behaviors may not be measureable. With the current study, we endeavor to show that proportion scores allow an interpretation of any given parenting behavior as the likelihood that it was used under circumstances that required a parental response (i.e., if a child were noncompliant).

This relative frequency type of measurement approach can be used when the assessment focuses on parental responses to child behaviors. This approach has been used to quantify positive parenting behaviors, such as praise and reinforcement (e.g., Chamberlain et al., 2008; Schuhmann et al., 1998), with findings from multiple intervention studies showing that positive parenting practices increase in proportion to all parenting behaviors. Other studies have addressed the use of proportion scores in the measurement of harsh behavioral discipline, noting that examining parents’ disciplinary behavior as a proportion score accounting for child and adolescent aggressive behavior avoids potential confounding of severity between harsh discipline and aggressive behavior (e.g., Simons, Wu, Lin, Gordon, & Conger, 2000) and assesses parents’ behavior as dependent on child/adolescent behavior (e.g., Snyder & Patterson, 1995). As reported by Lindhiem and colleagues (2014), treatment outcome data regarding aggressive and nonaggressive parenting behaviors, as measured using the CTS-PC, have appeared notably different when absolute and relative frequency methods are directly compared. In data spanning three years, relative frequencies of aggressive discipline (measured as a proportion of all possible reported disciplinary behaviors) decreased, whereas nonaggressive strategies increased – in direct contrast to decreases in nonaggressive discipline found with the same dataset when absolute frequencies were measured, as noted above (Lindhiem et al., 2014).

In this paper, we directly compare the relative frequency method for quantifying positive parenting, inconsistent discipline, and corporal punishment, with the more commonly reported methods of counting absolute frequency discipline scores. This paper extends previous research using this statistical method by including both positive and negative parenting behaviors, using a parenting measure that has not previously been testing using relative frequencies. These methods are compared using data obtained from an 8-site clinical trial of a collaborative care model for the treatment of child behavior problems. Parenting behaviors were measured over 18 months of follow-up via parent-report comprising two measures of ineffective discipline in response to noncompliance (i.e., inconsistent discipline and corporal punishment) and one measure of positive parenting behaviors in response to desirable child behaviors (e.g., praise). We first hypothesize improvements in parent-reported disciplinary practices will result from participation in a collaborative care treatment model targeting child behavior problems, relative to an enhanced usual care treatment group (Kolko et al., 2014), in terms of increases in positive parenting practices in response to child behavior, and decreases in inconsistent discipline and use of corporal punishment, and attendant improvements in child behavior problems. We then explore comparisons across methodological approaches of assessing parenting behavior via absolute and relative frequencies.

Method

Participants

Participants were parent-child dyads (N = 321) presenting with child behavior problems in primary-care settings. Children ranged in age from 5 to12 (M = 8.00, SD = 1.97). Approximately two-thirds were male (65% boys, 35% girls). Child diagnoses included ADHD (64%), disruptive behavior disorders (41%), and comorbid anxiety disorders (16%). The sample was 2.5% Hispanic, 21% Black/African American, 81% White. Parents were biological mothers (91%), biological fathers (5%), adopted mothers (2.5%), adopted fathers (0.3%) and grandmothers (1.3%). The sample included parents who were married/remarried (64%), single (22%), divorced (9%), separated from spouse (4.4%), and widows/widowers (0.3%). Parental education levels included junior high (0.3%), some high school (1.9%), high school diploma or GED (20%), some college (19%), associate degree (16%), 4-year college degree (30%), and graduate or professional degree (13%). Median household income was $50,000 to $74,999, and most parents were employed (55% full-time, 15% part-time).

Procedures

Study design.

The current study featured cluster randomization in which eight pediatric practices in the greater Pittsburgh area were assigned to provide either the experimental intervention, Doctor Office Collaborative Care (DOCC; Kolko, 2006), or the control intervention, Enhanced Usual Care (EUC). Four masters-level social workers were trained for four months to provide case management for both treatment conditions. To monitor treatment fidelity, the senior clinician supervised assessments, progress notes, and audio files, with weekly feedback. Each case manager worked in one DOCC and one EUC practice, completing approximately five clinical tasks common to both conditions; otherwise, no sessions were scheduled in EUC, so there was no intervention content used with those cases, and no contamination across conditions. Participants completed assessments at baseline, 6-months, 12-months, and 18-months.

Recruitment procedures.

Participants were referred to the case managers by their primary care provider. Social workers screened patients using the Pediatric Symptom Checklist 17. Eligibility criteria included a score above the 75th %ile on the externalizing behavior subscale. Families who passed initial screening then completed an assessment to screen for exclusion criteria. Exclusion criteria included homicidal/suicidal ideation or intent and concurrent behavioral services. Of the 787 families who were referred to the study, 576 were screened, and 321 met eligibility criteria and agreed to participate in the study. The participation rate was 95% for those who met the eligibility criteria.

Treatment conditions.

DOCC.

DOCC (Kolko, 2006) is a four-phase (engagement, self-management, behavioral change, and maintenance) treatment that addresses behavioral issues related to disruptive behavior disorders, anxiety disorders, and ADHD in a primary-care setting. Each phase consists of several sessions, depending on individual needs. DOCC was designed to integrate behavioral health services with primary care. As part of the collaborative care model, the case manager coordinates closely with the primary care provider to address behavioral problems. This coordination includes the prescription of medication by the primary care provider. In addition to the four core phases, DOCC features three supplemental phases that can be implemented based on unique needs of each individual family.

EUC.

Participants assigned to the Enhanced Usual Care condition first received a brief session that provided psychoeducation. After the psychoeducation session, participants were referred to a local mental health clinic. In the EUC condition, mental health services and primary care were not integrated, though children could receive medication through the primary care provider.

Measures

Parenting strategies.

Study measures were administered as self-report on a tablet computer. Researchers administering the assessments were blind to experimental conditions. Discipline and reinforcement strategies were assessed using the Alabama Parenting Questionnaire (APQ; Shelton, Frick, & Wootton, 1996). The APQ consists of 5 subscales, three of which were used in the current study because they are specific responses to child behavior. The two remaining subscales, Involvement and Poor Monitoring, describe more general parent behaviors. For the purposes of relative frequency scores, it is necessary to employ measures that are responses to/dependent on child behavior. The Inconsistent Discipline subscale comprises 6 items, including threatening punishment but not following through, or letting a child talk his/her way out of punishment. Corporal Punishment includes three items: spanking, slapping, or hitting a child with a belt, switch, or other object. Positive Parenting included 6 items, such as praising cooperative behavior using rewards. For each item, the respondent is asked about the use of each strategy within the past six months. Response options were provided on a 5-point scale from “never” to “always.” The APQ was completed independently by parents at each of the four assessments.

Absolute frequency scores.

Absolute frequency scores for inconsistent discipline, corporal punishment, and positive parenting were calculated by summing the responses for the items within each category. Scores at pre-treatment ranged from 6 to 26 for inconsistent discipline (M = 15.63; SD = 3.48), from 3 to 13 for corporal punishment (M = 5.03; SD = 1.65), and from 15 to 30 for positive parenting (M = 26.31; SD = 2.88).

Relative frequency scores.

Relative frequency scores for inconsistent discipline, corporal punishment, and positive parenting were calculated by dividing the sum of the scores for the items within a category by the sum of the scores for all 15 items. Resulting scores can range from 0.00 to 1.00. Scores at pre-treatment ranged from .15 to .48 for inconsistent discipline (M = .33; SD = .05), from .05 to .21 for corporal punishment (M = .11; SD = .03), and from .36 to .77 for positive parenting (M = .56; SD = .06). It must be noted that these relative frequency scores cannot be interpreted as true proportions or ratios. Given that the APQ items are rated on an ordinal scale (not ratio) the resulting scores cannot be interpreted as absolute values but rather as relative trends that are either increasing or decreasing over time.

Child Behavior Problems

Child behavior problems were assessed using the disruptive behavior disorder subscale of the Vanderbilt Assessment Scale-Parent Version (VAS-Parent; Wolraich, Hannah, Baumgaertel, & Feurer, 1998). Each of the 22 disruptive behavior items are rated on 4-point Likert scale (0 = Never; 1 = Sometimes; 2 = Often; 3 = Very Often). The VAS-P has well-established reliability and validity. Detailed psychometric properties of the VAS-Parent are described in detail in the literature (Wolraich, Lambert, Doffing, Bickman, Simmons, & Worley, 2003).

Data Analyses

Data analyses were conducted using SPSS 21 and hierarchical linear modeling (HLM; Raudenbush & Bryk, 2002). Tests for normality and other diagnostics indicated that the variables did not significantly violate the statistical assumptions behind the planned analyses. Therefore, we proceeded with the analyses without additional transformations, allowing for a more straightforward interpretation of the results. Details of these diagnostics and guidelines for analyzing proportion data are discussed by Chen and colleagues (Chen, Cheng, Berkout, & Lindhiem, this issue). Intraclass correlation coefficients (ICCs) indicated trivial clustering effects of the child and parent variables within clinics (ICCs < .02), supporting two-level (time within participants) HLM models.

Piecewise hierarchical linear models were tested with time nested within participants. Full maximum likelihood estimation was used for all models. The level 1 equations for the unconditional models were Yti = π0i + π1i(Response) + π2i(Maintenance) + eti, where Yti is the observed outcome score at time t for participant i. The “Response” variable was coded 0, 1, 1, and 1 for each of the four time points, allowing for the estimated treatment slope to be interpreted as the baseline to six-month post change. The “Maintenance” variable was coded 0, 0, 1, and 2, corresponding to each of the four time points, allowing for the estimated maintenance slope to be interpreted as the average amount of change during the 12-month follow-up period (between the 6-month and 18-month assessments). Finally, to test the benefit of DOCC compared to EUC, we estimated conditional models with treatment condition as a level-2 variable (DOCC = 1; EUC = 0).

Results

Overview

Table 1 summarizes the conditional models in which treatment condition (DOCC vs. EUC) was modeled at level 2. Figure 1 depicts the changes over time in each of the three parenting variables as absolute and relative frequencies. Differences were pronounced for positive parenting. Using the absolute frequency method, there were no measurable changes in positive parenting. In contrast, positive parenting increased when quantified as a proportion of parenting behaviors. Decreases were evident for inconsistent discipline and corporal punishment, both as absolute and relative frequencies.

Table 1.

Summary of HLM Model Slopes with Treatment Condition (DOCC vs. EUC) Modeled at Level 2

Outcome Variable Response slope Maintenance slope
Treatment Effect B SE T p B SE t p
APQ ID raw −0.457 0.282 −1.622 0.106 −0.441 0.139 −3.173 0.002
 DOCC vs. EUC −0.970 0.374 −2.594 0.010 0.303 0.184 1.645 0.101
APQ ID proportion −0.005 0.004 −1.079 0.281 −0.005 0.002 −2.012 0.045
 DOCC vs. EUC −0.013 0.006 −2.364 0.019 0.004 0.003 1.417 0.157
APQ CP raw −0.481 0.101 −4.745 <0.001 −0.061 0.051 −1.199 0.232
 DOCC vs. EUC −0.184 0.147 −1.251 0.212 0.004 0.071 0.061 0.952
APQ CP proportion −0.008 0.002 −4.133 <0.001 −0.000 0.000 −0.249 0.803
 DOCC vs. EUC −0.002 0.003 −0.539 0.591 −0.000 0.001 −0.040 0.968
APQ PP raw −0.036 0.186 −0.196 0.845 −0.072 0.102 −0.712 0.477
 DOCC vs. EUC 0.077 0.264 0.291 0.771 −0.156 0.145 −1.079 0.281
APQ PP proportion 0.013 0.004 3.153 0.002 0.005 0.002 2.146 0.033
 DOCC vs. EUC 0.015 0.006 2.502 0.013 −0.004 0.003 −1.390 0.166
Open in a new tab

APQ = Alabama Parenting Questionnaire; CP = Corporal Punishment; DBD = Disruptive Behavior Disorder; ID = Inconsistent Discipline; PP = Positive Parenting

Figure 1.

Figure 1.

Open in a new tab

Absolute frequency (1a) vs. relative frequency (1b) of inconsistent discipline. Absolute frequency (2a) vs. relative frequency (2b) of corporal punishment. Absolute frequency (3a) vs. relative frequency (3b) of positive parenting. APQ = Alabama Parenting Questionnaire; DOCC = Doctor Office Collaborative Care; EUC = Enhanced Usual Care. Error bars represent standard errors.

Absolute Frequency Method

Overall change.

Using the absolute frequency method, inconsistent discipline and corporal punishment both decreased from baseline to the 6-month assessment for the DOCC group (both slopes were negative, p < .001). Inconsistent discipline decreased further between the 6-month and 18-month assessments (negative slope, p = .002) while corporal punishment remained stable (slope not significantly different from zero, p = .101) during this maintenance phase. Positive parenting remained unchanged from the baseline to the 6-month assessment (slope not significantly different from zero, p = .978) but declined between the 6-month and 18-month assessments (negative slope, p = .035).

DOCC vs. EUC.

Using the absolute frequency method, parents in the DOCC condition showed a greater reduction in inconsistent discipline compared to parents in the EUC condition from baseline to 6-months after baseline, p = .010. Both conditions continued to decline in the use of inconsistent discipline from 6 to 18 months after baseline, p = .002, but with no additional benefit for the DOCC condition, p = .101. There were no differences between conditions for corporal punishment or positive parenting, either during the acute or maintenance phases.

Relative Frequency Method

Overall change.

Using the relative frequency method, results were similar for inconsistent discipline and corporal punishment but strikingly different for positive parenting. Inconsistent discipline and corporal punishment both decreased for the DOCC group from baseline to the 6-month assessment (both slopes were negative, p < .001) and then remained stable between the 6-month and 18-month assessments (both slopes not significantly different from zero, p > .05) Positive parenting increased from baseline to the 6-month assessment (positive slope, p < .001) and then remained stable between the 6-month and 18-month assessments (slope not significantly different from zero, p > .05)

DOCC vs. EUC.

Using the relative frequency method, parents in the DOCC condition showed a greater reduction in inconsistent discipline compared to parents in the EUC condition from baseline to 6-months after baseline, p = .019. Both conditions continued to decline in the use of inconsistent discipline from 6 to 18 months after baseline, p = .045, but with no additional benefit for the DOCC condition, p = .157. These results were similar to the results obtained using the absolute frequency method. Also similar to the absolute frequency method, there were no differences between conditions for corporal punishment, either during the acute or maintenance phases. However, the results were significantly different using the two methods for positive parenting. Using the relative frequency method, parents in both conditions increased in the use of positive parenting strategies from baseline to the 6-month assessment, p = .002, but parents in the DOCC condition evidenced a greater increase, p = .013. Both conditions continued to increase in the use of positive parenting from 6 to 18 months after baseline, p = .033, but with no additional benefit for the DOCC condition, p = .166.

Correlations between Positive Parenting and Child Behavior Problems

Notably, correlations between positive parenting and child behavior problems using the absolute frequency method and the relative frequency method differed, across all time points. Using the absolute frequency method, correlations ranged from −.04 to .05 (M = .03), all p-values = ns. Using the relative frequency method, a very different pattern emerged. Specifically, correlations ranged from −.11 to −.36 (M = −.27), all p-values < .05. This is a moderate negative correlation and can be interpreted to mean that parents who reported using more positive parenting strategies (relative to other parenting strategies) also reported fewer child behavior problems.

Furthermore, positive parenting behaviors mediated the link between treatment condition (DOCC = 1; EUC = 0) and reduction in behavior problems using proportion scores but not absolute scores. Mediation was tested using the SPSS PROCESS macro with 1000 bootstrap samples for bias corrected confidence intervals (Hayes, 2013). Using absolute scores, the indirect effect of treatment condition on child behavior problems via positive parenting was not significant (indirect effect B = −.02, SE = .16, 95% CI = −8.97 to 0.82). For proportion scores, there was a significant indirect effect indicating mediation by positive parenting (indirect effect B = −1.79, SE = .74, 95% CI = −3.39 to −0.44).

Discussion

The primary goal of the current study was to investigate treatment-related changes in parenting practices following a collaborative care treatment model implemented in pediatric primary-care settings (vs. enhanced usual care) for school-age children with disruptive or anxious behavior problems that were calculated using two alternative methods. Based on self-reported parenting behaviors, parents who completed the intervention reported reduced use of inconsistent discipline and corporal punishment following the intervention. However, our results also highlight that the methods used to quantify discipline practices warrant special consideration in intervention and prevention contexts: when the goal is to both reduce ineffective parenting practices and bolster positive parenting efforts, measuring changes in the absolute frequencies of those behaviors can lead to spurious or counterintuitive findings. To illustrate the different conclusions that these different analytic approaches could yield, it is worth considering the typical hoped-for treatment outcome for behavioral parenting interventions: not just a reduction in ineffective parenting practices, but also a reduction in child problem behavior. If absolute frequencies are used to measure parent discipline behaviors, one might expect an overall reduction in ineffective parenting behaviors over the course of successful treatment, as found in the current study. However, although it is expected that successful treatment results in decreased engagement in harsh or ineffective parenting behaviors, it is also plausible that effective discipline strategies (i.e., time out, redirection) would become less frequent if child problem behavior also decreases over the course of treatment. This scenario outlines justification for the continued use of relative frequencies as a measure of treatment outcome in parenting intervention and prevention studies: the measurement of parents’ discipline strategies may be best captured as dependent on the occurrence of children’s behavior that requires disciplinary response.

In comparison to the absolute frequency method, decreases in corporal punishment and inconsistent discipline were still observed from pre-treatment to 6-month follow-up. Interestingly, the pattern of change over time was different across methods for positive parenting: whereas the absolute frequency method resulted in no significant changes in positive parenting behaviors from pre- to post-treatment, and declines in positive parenting behaviors at the 18-month assessment, results using the relative frequency method showed increases in positive parenting from baseline to 6 months, and stable positive parenting at the 18-month follow-up. Relative frequencies of positive parenting also mediated relations between treatment condition and child behavior outcomes. Our findings using the relative frequency method of assessing parental discipline highlight the utility of this approach and corroborate previously reported findings. These findings replicate and extend previous research using this methodological approach with different measures of parent behavior (e.g., Chamberlain et al., 2008; Lindhiem et al., 2014; Schuhmann et al., 1998) and include both negative and positive parenting behaviors to observe different patterns across them. We also found that significant correlations between positive parenting behavior and child problem behavior were only detected when using relative frequency measures of parenting behavior, despite theoretical reasons to expect moderate and negative correlations between positive parenting and child disobedience/noncompliance.

The current findings should be interpreted in the context of the study’s limitations and scope. As noted previously (Kolko et al., 2014) the broad range of clinical content modules delivered as part of the intervention study design prevents more fine-grained analyses of specific treatment process questions, although all treatment modules were built from evidence-based intervention practices. The treatment outcome measures for the current study are limited to parental reports using a single measure in order to serve as a straightforward demonstration of statistical techniques. However, the treatment-related improvements documented in this study via parent-report data are corroborated by previously reported improvements rated by clinicians, research staff, and caregivers (Kolko et al., 2014). Finally, while sole reliance on parent report limits the generalizability of our findings, we expect that similar measurement methods would be effective with observer or clinician reports, and offer this as a suggestion for future research.

Our findings indicate not only that changes in parenting behavior are a proximal outcome of treatments ultimately directed at child behavior change, but also that the methods used to measure treatment-related change warrant careful consideration. To date, studies that utilize relative frequency scores as measures of parental discipline are considerably rarer than studies of absolute frequencies. Our suggestion is that relative frequency or proportion scores are a straightforward alternative to typical assessment procedures that offer methodological and conceptual advantages to the interpretation of treatment and prevention outcome data, particularly for studies with practical limitations on the availability of observational data or alternative assessment techniques.

a. Funding:

This research was supported by grants from the National Institute of Mental Health (MH093508; MH062272).

This research was supported by grants from the National Institute of Mental Health (NIMH) to the second author for the preparation of this article (MH093508) and to the third author for the research data collected in this study (MH063272). The authors would also like to thank Charles Bennett for assistance with formatting and editing of the manuscript.

b.

Disclosure of potential conflicts of interest: The authors declare no conflicts of interest.

c.

Ethical approval: All data collection procedures were carried out with approval from, and in compliance with, the IRB at the University of Pittsburgh.

d.

Informed consent: Informed consent was obtained from all individual study participants.

References

  1. Burke JD, Pardini DA, & Loeber R (2008). Reciprocal relationships between parenting behavior and disruptive psychopathology from childhood through adolescence. Journal of Abnormal Child Psychology, 36, 679–692. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Carr EG, Taylor JC, & Robinson S (1991). The effects of severe behavior problems in children on the teaching behavior of adults. Journal of Applied Behavior Analysis, 24, 523–535. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Chamberlain P, Price J, Leve LD, Laurent H, Landsverk JA, & Reid JB (2008). Prevention of behavior problems for children in foster care: outcomes and mediation effects. Prevention Science, 9, 17–27. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Chen K, Cheng Y, Berkout O, & Lindhiem O (this issue). Analyzing proportion scores as outcomes for prevention trials: A statistical primer. Prevention Science. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. DeGarmo DS, Patterson GR, & Forgatch MS (2004). How do outcomes in a specified parent training intervention maintain or wane over time? Prevention Science, 5, 79–89. [DOI] [PubMed] [Google Scholar]
  6. Eyberg SM, Nelson MM, & Boggs SR (2008). Evidence-based psychosocial treatments for children and adolescents with disruptive behavior. Journal of Clinical Child and Adolescent Psychology, 37, 215–237. [DOI] [PubMed] [Google Scholar]
  7. Hayes AF (2013). Introduction to mediation, moderation, and conditional process analysis. New York: Guilford Press. [Google Scholar]
  8. Kaminski JW, Valle LA, Filene JH, & Boyle CL (2008). A meta-analytic review of components associated with parent training program effectiveness. Journal of Abnormal Child Psychology, 36, 567–589. [DOI] [PubMed] [Google Scholar]
  9. Kazdin AE (2010). Problem-solving skills training and parent management training for Oppositional-Defiant Disorder and Conduct Disorder In Weisz JR & Kazdin AE (), Evidence-based psychotherapies for children and adolescents, 2nd edition (pp. 211–226). New York: Guilford Press. [Google Scholar]
  10. Kazdin AE, & Nock MK (2003). Delineating mechanisms of change in child and adolescent therapy: Methodological issues and treatment recommendations. Journal of Child Psychology and Psychiatry, 44, 1116–1129. [DOI] [PubMed] [Google Scholar]
  11. Kieling C, Baker-Henningham H, Belfer M, Conti G, Ertem I, Omigbodun O, Rohde LA, Smith S, Ulkuer N, & Rahman A (2011). Child and adolescent mental health worldwide: evidence for action. The Lancet, 378 (9801), 1515–1525. [DOI] [PubMed] [Google Scholar]
  12. Kolko DJ (2006). Doctor-office collaborative care management: Clinician treatment manual Unpublished manuscript, Services for Kids In Primary-care (SKIP), Department of Psychiatry, University of Pittsburgh School of Medicine, Pittsburgh, PA. [Google Scholar]
  13. Kolko DJ, Campo J, Kilbourne AM, Hart J, Sakolsky D, & Wisniewski S (2014). Collaborative care outcomes for pediatric behavioral health problems: A cluster randomized trial. Pediatrics, 133, e981–e992. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Kolko DJ, Dorn LD, Bukstein O, & Burke JD (2008). Clinically referred ODD children with or without CD and healthy controls: Comparisons across contextual domains. Journal of Child and Family Studies, 17, 714–734. [Google Scholar]
  15. Laird RD, Pettit GS, Bates JE, & Dodge KA (2003). Parents’ monitoring relevant knowledge and adolescents’ delinquent behavior: Evidence of correlated developmental changes and reciprocal influences. Child Development, 74, 752–768. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Patterson GR, DeGarmo D, & Forgatch MS (2004). Systematic changes in families following prevention trials. Journal of Abnormal Child Psychology, 32, 621–633. [DOI] [PubMed] [Google Scholar]
  17. Patterson GR, & Reid JB (1970). Reciprocity and coercion: Two facets of social systems In Neuringer C, & Michael J () Behavior modification in clinical psychology. New York: Appleton-Century-Crofts. [Google Scholar]
  18. Raudenbush SW, & Bryk AS (2002). Hierarchical linear models: Applications and data analysis methods. (2nd Ed.). Thousand Oaks, CA: Sage. [Google Scholar]
  19. Sandler IN, Schoenfelder EN, Wolchik S, & MacKinnon D (2011). Long-term impact of prevention programs to promote effective parenting: Lasting effects but uncertain processes. Annual Review of Psychology, 62, 299–329. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Shaffer A, Lindhiem O, Trentacosta C, & Kolko DJ (2013). Examination of bidirectional effects between parenting practices and child externalizing behavior: A cross-lagged panel analysis in the context of a psychosocial treatment and 3-year follow-up. Journal of Abnormal Child Psychology, 41, 199–210. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Shaw DS, Connell A, Dishion TJ, Wilson MN, & Gardner F (2009). Improvements in maternal depression as a mediator of intervention effects on early childhood problem behavior. Development and Psychopathology, 21, 417–439. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Shelton KK, Frick PJ, & Wootton JM (1996). Assessment of parenting practices in families of elementary school-age children. Journal of Clinical Child Psychology, 25, 317–329. [Google Scholar]
  23. Stormshak EA, Bierman KL, McMahon RJ, & Lengua LJ (2000). Parenting practices and child disruptive behavior problems in early elementary school. Journal of Clinical Child Psychology, 29, 17–29. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Wolraich ML, Hannah JN, Baumgaertel A, & Feurer ID (1998). Examination of DSM IV critieria for attention deficit /hyperactivity disorder in a county-wide sample. Journal of Developmental & Behavioral Pediatrics, 19, 162–168. [DOI] [PubMed] [Google Scholar]

RESOURCES