Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2016 Jan 31.
Published in final edited form as: J Child Fam Stud. 2013 Dec 7;24(2):499–504. doi: 10.1007/s10826-013-9862-7

Helping the Noncompliant Child: An Assessment of Program Costs and Cost-Effectiveness

Amanda A Honeycutt 1, Olga A Khavjou 2,✉, Deborah J Jones 3, Jessica Cuellar 4, Rex L Forehand 5
PMCID: PMC4308978  NIHMSID: NIHMS547306  PMID: 25642124

Abstract

Disruptive behavior disorders (DBD) in children can lead to delinquency in adolescence and antisocial behavior in adulthood. Several evidence-based behavioral parent training (BPT) programs have been created to treat early onset DBD. This paper focuses on one such program, Helping the Noncompliant Child (HNC), and provides detailed cost estimates from a recently completed pilot study for the HNC program. The study also assesses the average cost-effectiveness of the HNC program by combining program cost estimates with data on improvements in child participants’ disruptive behavior. The cost and effectiveness estimates are based on implementation of HNC with low-income families. Investigators developed a Microsoft Excel-based costing instrument to collect data from therapists on their time spent delivering the HNC program. The instrument was designed using an activity-based costing approach, where each therapist reported program time by family, by date, and for each skill that the family was working to master. Combining labor and non-labor costs, it is estimated that delivering the HNC program costs an average of $501 per family from a payer perspective. It also costs an average of $13 to improve the Eyberg Child Behavior Inventory intensity score by 1 point for children whose families participated in the HNC pilot program. The cost of delivering the HNC program appears to compare favorably with the costs of similar BPT programs. These cost estimates are the first to be collected systematically and prospectively for HNC. Program managers may use these estimates to plan for the resources needed to fully implement HNC.

Keywords: cost analysis, cost-effectiveness, behavioral parent training program, disruptive behavior disorders

Introduction

Disruptive behavior disorders (DBD) in children can lead to delinquency in adolescence and antisocial behavior in adulthood if left untreated (Piquero, Farrington, Welsh, Tremblay, & Jennings, 2009). DBD may also have considerable economic implications in terms of excess health services, education, and criminal justice costs (Cohen, 1998; Foster & Jones, 2005; Scott, Knapp, Henderson, & Maughan, 2001). By the time they reach adulthood, children with DBD incur costs that are up to 10 times higher than children without DBD (Scott et al., 2001). Several evidence-based behavioral parent training (BPT) programs have been created to treat early onset DBD (Eyberg, Boggs, & Algina, 1995, 2001; Hutchings et al., 2007; Jones, Daley, Hutchings, Bywater, & Eames, 2008; McMahon & Forehand, 2003; Sanders, 2008; Sanders, Markie-Dadds, Tully, & Bor, 2000). These interventions instruct parents through one-on-one or group training sessions to reward their child’s positive behaviors and address negative behaviors with clear and consistent consequences (e.g., using time-outs). This paper focuses on one evidence-based BPT program developed in the early 1970s to treat DBD: Helping the Noncompliant Child (HNC) (McMahon & Forehand, 2003). Despite the long history of HNC as a clinic-based treatment option for youth with DBD, less is known about the cost-effectiveness of this approach. Given that analysis of cost data informs decision making by policy makers and health care and insurance providers, this paper aims to advance the literature on the cost-effectiveness of BPT by describing estimates of HNC costs and cost-effectiveness from a recent pilot study.

To provide a brief overview of the program as context, HNC is a criterion-based BPT program that evolved from the Hanf Model (Reitman & McMahon, 2013). HNC is provided through weekly sessions with individual families of children between 3 and 8 years of age. Each HNC session lasts 60 to 90 minutes and centers on parents’ mastery of a sequence of five parenting skills designed to enhance effective management of child disruptive behaviors. Parents move through the program at their own pace based on their mastery of each skill before advancing to the next skill. Families typically complete the HNC program in 8 to 12 sessions (McMahon & Forehand, 2003). In the first phase of HNC, parents learn methods for establishing a positive and mutually reinforcing parent-child relationship. The skills addressed during Phase I are (1) attending, (2) rewarding, and (3) ignoring. Once parents master Phase I skills, they are taught the following Phase II HNC parenting skills: (4) clear instructions and (5) time-out. For each of the five skills, parents progress to a new skill when specific behavioral criteria for the previous skill have been met as assessed by the HNC therapist (McMahon & Forehand, 2003).

The HNC program has been shown to be efficacious in comparison with a control group of families who were placed on a waiting list for HNC and in comparison with an alternative intervention, family systems therapy (McMahon & Forehand, 2003). When HNC completers were compared with matched community controls who were never identified as needing DBD treatment, problem behavior outcomes were found to be similar between groups, with follow-up assessments conducted at 2 months to 14 years post-treatment (McMahon & Forehand, 2003). These findings suggest that HNC is an efficacious treatment that may prevent children from progressing to more severe deviant behaviors. Nevertheless, because of limited resources available to address children’s mental health needs, especially for children from low-income families, the value of implementing parent training programs in community settings depends on their costs as well as their effectiveness (Kazak et al., 2010). Therefore, it is important to provide data not only on behavior change resulting from evidence-based interventions like HNC but also on costs to implement these programs.

In this paper, we provide detailed cost estimates from a recently completed pilot study for the HNC program overall and for each skill taught in the program. These cost estimates are the first to be collected systematically and prospectively for HNC. Program managers may use these estimates to plan for the resources needed to fully implement HNC. We also assess the average cost-effectiveness of the HNC program by combining program cost estimates with data on improvements in child participants’ disruptive behavior. Of importance, our cost and effectiveness estimates are based on implementation of HNC with low-income families. These families are more difficult to engage and retain in treatment than higher-income families (Ingolsby, 2010; Jones et al., 2013, Jones et al., in press), potentially leading to higher program costs. However, as low-income youth are at risk for DBD and underserved, it is important to examine the costs of providing services to this population. In addition to examining the cost and cost-effectiveness of HNC implementation, we use information from the published literature to compare the economic impacts of HNC and two other BPT interventions for young children with DBD.

Method

Overview

Our assessment of costs and cost-effectiveness of the HNC program was conducted as part of a pilot research study where participants were randomly assigned to receive the HNC intervention versus a technology-enhanced version of HNC. The current cost and cost-effectiveness analysis included only participants receiving the HNC intervention (McMahon & Forehand, 2003). Details on recruitment and assessment procedures, intervention features, therapist training, and supervision of the pilot study are given elsewhere (Jones et al, in press).

Participants

Families were included in the project if they met criteria for “low-income” (i.e., adjusted gross income did not exceed 150% of the federal poverty limit, which takes into account both income and number of residents in the home); they had a child in the 3- to 8-year-old age range (age range for which HNC was developed and tested); and the child exhibited disruptive behaviors in the clinical range as evidenced by meeting or exceeding clinical cutoffs on the caregiver report of the Eyberg Child Behavior Inventory (ECBI) Severity or Intensity Subscales (Eyberg & Pincus, 1999).

Exclusion criteria were (1) child developmental or physical disability that precluded use of HNC skills; (2) caregiver current diagnosis of substance abuse/dependence, mood, or psychotic disorder; and/or (3) family involvement with Department of Social Services related to abuse/neglect.

Procedure

Low-income families were recruited via (1) advertisements targeting areas, work places, and retail outlets with an overrepresentation of low-income parents (48% of pilot families); (2) health care, social service, and other agencies that serve low-income families (28% of pilot families); (3) local schools (19% of pilot families); and (4) word-of-mouth (5% of pilot families). A brief (20-minute) phone screen was conducted to determine interest and initial eligibility, which was then confirmed by a clinic-based interview, which also included consent and a more extensive caregiver assessment battery. During treatment, therapists collected data on engagement and skill generalization, as well as costs (e.g., number of sessions). Within 2 weeks of treatment termination, the caregiver assessment battery was re-administered (post-assessment). All assessments and sessions were completed at a community-based university training clinic. Families were typically seen in the late afternoon to evening (school-aged children) or morning to early afternoon (preschool children), with appointments chosen by families to enhance convenience. Caregivers were compensated $50 per assessment.

Measures

For this analysis, we collected information about the costs to deliver the HNC intervention to the families who were recruited for participation in the pilot research study and randomly assigned to receive the HNC treatment. We did not include costs associated with research-specific activities, such as recruitment of participants or administration of the grant. We separately analyzed and reported costs for program start-up and program delivery. Although start-up costs would typically include the cost to obtain training for therapists, the therapists in this study were already trained in HNC. Consequently, the HNC training and supervision costs were not included here. As a note, the cost of a 2-day training session for HNC therapists is $5,000 plus travel expenses for two trainers. Additional booster sessions and conference calls are recommended over the next 2.5 years at an additional cost of approximately $15,000. Although these post-training activities are recommended, many therapists receive the initial training only.

Program delivery costs were estimated based on the resources required to deliver HNC to pilot study participants. We quantified program delivery costs using a payer approach to value the time and other resources associated with program delivery. First, we calculated labor costs as the value of time spent on the program by therapists and other program staff. Second, we valued non-labor costs based on the payments made for program materials and equipment and on the value of any donations.

Investigators developed a Microsoft Excel-based costing instrument to collect data from therapists on their time spent delivering the HNC program. The instrument was designed using an activity-based costing approach, where each therapist reported program time by family, by date, and for each skill that the family was working to master, including an initial orientation session. Therapists reported time spent on each family both during and outside of face-to-face sessions, including time spent on phone calls, review of session notes, meetings with the program supervisor, and other activities. Therapist time was valued using the 2010 median national hourly wage for masters-level mental health counselors from the Bureau of Labor Statistics. Non-labor program delivery costs included purchases of materials and supplies (e.g., toys, handouts) used to support the program. We calculated the average labor cost for the orientation session and to achieve mastery of each of the five HNC skills. We also estimated the total cost of the HNC program per family, which we calculated as average labor plus average non-labor cost per family.

Program cost-effectiveness was assessed by combining average per capita program delivery costs with changes observed in child disruptive behavior measured using the pre- and post-intervention ECBI intensity scores (Eyberg & Pincus, 1999). The ECBI, a commonly used outcome measure for BPT programs for young children with DBD, has substantial validity and reliability data (Eyberg & Pincus, 1999). We calculated a cost-effectiveness ratio as the average program delivery cost of HNC across participating families divided by the average change from baseline in the ECBI intensity score. A statistically significant reduction in the ECBI intensity score reflects improvement in children’s behavioral outcomes. Start-up costs were excluded from the cost-effectiveness ratio because these are “sunk” costs that are not relevant when comparing the benefits of program delivery to the costs. The effectiveness measure used in our analysis was collected as part of pre- and post-assessments for the pilot research study (Jones et al, in press). The cost-effectiveness ratio represents the cost to achieve a 1-point decrease in the ECBI intensity score for a child whose family participated in the HNC program. Because costs and outcomes varied across the participating families, we also assessed the impact of uncertainty in the cost-effectiveness estimate by calculating a 95% confidence interval using a non-parametric bootstrapping approach and 1,000 replications from the pilot study cost and effectiveness data (Glick, Doshi, Sonnad, & Polsky, 2007). Bootstrapped HNC costs and changes on the ECBI intensity scale were also used to construct a cost-effectiveness acceptability curve, which shows the probability that HNC is cost-effective for different maximum amounts that decision makers might be willing to pay for a 1-point decrease in the ECBI intensity scale (Fenwick & Byford, 2005).

Results

Of the low-income families who completed a clinic-based eligibility interview (n = 48), 24 were ineligible for the pilot RCT (e.g., pending Department of Social Services investigation) and 2 eligible families chose not to enroll. Using restricted random assignment to force equal sample sizes, the remaining 22 eligible low-income families were randomly assigned to HNC or the experimental treatment. The initial three families randomized served as practice cases for each of the three project therapists. Of the remaining 19 families, 10 were randomized to HNC.

Two families dropped out of the HNC treatment, each notifying project staff prior to dropout and citing a major family stressor (e.g., parent illness and surgery). Given the pilot nature of the project, we considered data only from those families that completed the HNC program (n = 8), where program completion required mastering all five HNC program skills. Half of the participating children were boys, and the average age of children was 5.75 years. The participating caretakers were all women, with an average age of 38 years. Eighty-eight percent of the caretakers worked part- or full-time, and 62% were unmarried.

Table 1 reports mean program delivery labor costs for the orientation session and for each of the five program skills among program completers. Skills 5 (Time Out) and 2 (Rewarding) required the most therapist time (an average of 2 sessions per skill) and thus were the most expensive skills, costing an average of $127 and $123, respectively, per client to master. Besides the orientation, Skill 1 (Attending) and Skill 3 (Ignoring) were mastered most quickly by program completers (with an average of 1 session each, costing $64 and $57 per family, respectively). The average therapist cost for a family to master a skill was $82. It took program completers an average of 10 therapist sessions (with a range of 7 to 12) to master all five skills with a total labor cost per family of $491.

Table 1.

Labor costs per family that completed the Helping the Noncompliant Child (HNC) intervention (n = 8)

Skill Achieved Average Cost per Family
Orientationa $50
Skill 1: Attending $64
Skill 2: Rewarding $123
Skill 3: Ignoring $57
Skill 4: Clear Instructions $70
Skill 5: Time Out $127
Mean cost per skill $82
Total cost per completer $491
Open in a new tab
a

One orientation session was added to the HNC intervention in the pilot trial to provide introductory information and materials.

The non-labor costs associated with HNC program delivery were as follows: cost of therapist manuals was $2.70 per enrolled family; cost of toys was $4.60 per enrolled family; and cost of handouts was $2.70 per enrolled family. The total non-labor cost per enrolled family was $10. Combining labor and non-labor costs, we estimate an average cost of $501 per family to deliver the HNC program from a payer perspective.

The average reduction per child in the ECBI intensity score was 39.88. Combining cost and effectiveness outcomes, the cost-effectiveness ratio indicates that it cost an average of $13 to improve the ECBI intensity score by 1 point for children whose families participated in the HNC pilot program (95% confidence interval of $4 to $21). The cost-effectiveness acceptability curve (Figure 1) indicates that the probability of the HNC program being cost-effective would exceed 90% provided that decision makers were willing to pay at least $20 per 1-point improvement in the ECBI intensity score. The probability of obtaining cost-effective results increases to over 95% if decision makers are willing to pay an average of $25 per point of improvement on the ECBI intensity score.

Figure 1.

Figure 1

Open in a new tab

Cost-effectiveness acceptability curve of the HNC program: probability that the program is cost-effective at various willingness-to-pay levels

Discussion

The findings indicate that it costs an average of $501 per family to deliver the HNC program and a cost of $13 for each point improvement in the ECBI intensity score. To place the costs of implementing HNC in perspective, we explored published results on cost and cost-effectiveness estimates of two other BPTs with many of the same program components: Parent-Child Interaction Therapy (PCIT) (Brinkmeyer & Eyberg, 2003) and Incredible Years (IY) (Webster-Stratton & Hancock, 1998). The former program, like HNC, is administered individually to families, whereas the latter program is administered in a group format. Both of these programs have been shown to produce improved parent and child behavioral outcomes (Eyberg, Nelson, & Boggs, 2008). We caution against direct comparisons of our results with other studies as different approaches were used to calculate costs. We point out the differences and where possible report costs of PCIT and IY that are relevant for our analysis. To make results more comparable, we converted published costs of these other programs to 2010 U.S. dollars.

For PCIT, the average cost of therapist time and materials was estimated at approximately $500 per family with a cost-effectiveness ratio of $11 per 1-point improvement in the ECBI intensity score (Goldfine, Wagner, Branstetter, & McNeil, 2008). For IY, the costs of materials and staff time to deliver the parenting skills component of the program was estimated to cost approximately $550 per family in the United States (Foster, Olchowski, & Webster- Stratton, 2007). Cost-effectiveness, however, was not assessed as part of this study. In the United Kingdom, IY was estimated to cost approximately $800 per family with a cost-effectiveness ratio of $30 per 1 point improvement in the ECBI score using assumptions similar to those in the current HNC analysis (Edwards, Ceilleachair, Bywater, Hughes, & Hutchings, 2007). In Ireland, IY group sessions and home visits were estimated to cost approximately $2,000 per family with a cost-effectiveness ratio of $100 per 1 point improvement in the ECBI score (O’Neill, McGilloway, Donnelly, Bywater, & Kelly, 2013). In addition to the cost directly associated with conducting sessions (which is what we assessed for HNC), this IY estimate also includes the cost of recruiting participants, conducting home visits, and the cost of facilities. It appears that when relevant costs are taken into account, the costs across the three programs are similar.

Although not relevant for cost-effectiveness analysis, estimates of start-up costs are useful for program planners who are considering whether to implement HNC. Costs associated with training therapists to provide the HNC intervention represent a sizeable initial investment; however, these costs are total rather than per capita costs for a training session. The training costs will be much lower when spread across multiple therapists and families. Per capita estimates can be derived from our total start-up cost estimates using various assumptions about the number of therapists participating in a training session and the caseload of each therapist (Goldfine et al., 2008).

Our analysis had a number of limitations. First, only 10 families were enrolled in the HNC arm of the pilot study, and 8 completed the intervention. Second, the sample is not representative of a “typical” clinic-referred sample as it was selected based on low family income, and low-income families have been found to be more difficult to engage and retain in therapy, including in BPT (Ingoldsby, 2010; Jones et al., 2013, Jones et al, in press). Financial strain and associated difficulties, such as under-employment and lack of adequate health insurance, are associated with lower levels of engagement and, in turn, retention in BPT (Eyberg et al., 2008; McMahon & Forehand, 2003). As a result, the costs presented here may be higher than costs for middle- or higher-income families.

Another limitation is that we did not estimate the costs incurred by program participants (e.g., time spent practicing and mastering each skill) because our analysis was conducted from a payer perspective, rather than from a societal perspective. Although we considered approaches for collecting information about participants’ time spent on HNC activities, we ultimately decided against collecting these data, given that we were unable to validate participants’ self-reports.

An additional limitation is that we did not have a usual care control group in the pilot study, and, as a result, changes from baseline in outcomes and costs among HNC participants were compared with a no-intervention scenario, which is assumed to have zero costs and no change from baseline in outcomes. In fact, HNC families were the control group for the pilot study (Jones et al, in press), as the main goal of that study was to assess outcomes among participants of a technology-enhanced HNC program compared with the standard HNC program. Our future work will use a larger sample to assess the incremental cost-effectiveness of the technology-enhanced version of HNC versus the standard HNC program.

Conclusions

Cost and cost-effectiveness analyses can provide useful information to help with planning for the implementation and dissemination of parenting programs and to understand whether investments in such programs are worthwhile. Although economic evaluations have assessed the costs and cost-effectiveness of several other individual and group BPT interventions, including IY and PCIT, the results described in this paper are the first to systematically assess program costs and cost-effectiveness for HNC. We found costs of $501 per family to deliver the HNC program and a cost of $13 for each point of improvement in the ECBI intensity score among low-income HNC participants. HNC would be viewed as a good investment if, in the long-term, it resulted in averted costs of at least $500 per family from reductions in criminal activity, special education, and other negative social impacts—findings supported by theLong et al. (1994) long-term follow-up of HNC participants.

Footnotes

Disclosure Statement: There are no conflicts of interest.

Contributor Information

Amanda A. Honeycutt, RTI International, 3040 Cornwallis Road, Research Triangle Park, NC 27709, USA

Olga A. Khavjou, Email: okhavjou@rti.org, RTI International, 3040 Cornwallis Road, Research Triangle Park, NC 27709, USA.

Deborah J. Jones, University of North Carolina at Chapel Hill, Campus Box 3270, Chapel Hill, NC 27599, USA

Jessica Cuellar, University of North Carolina at Chapel Hill, Campus Box 3270, Chapel Hill, NC 27599, USA.

Rex L. Forehand, University of Vermont, Burlington, VT 05405, USA

References

  1. Aos S, Lieb R, Mayfield J, Miller M, Pennucci A. Benefits and costs of prevention and early intervention programs for youth. Olympia, WA: Washington State Institute for Public Policy; 2004. [Google Scholar]
  2. Brinkmeyer M, Eyberg SM. Parent-child interaction therapy for oppositional children. In: Kadzin AE, Weisz JR, editors. Evidence-based psychotherapies for children and adolescents. New York, NY: Guilford; 2003. pp. 20–223. [Google Scholar]
  3. Cohen MA. The monetary value of saving a high risk youth. Journal of Quantitative Criminology. 1998;4:5–33. [Google Scholar]
  4. Edwards RT, Ceilleachair A, Bywater T, Hughes DA, Hutchings J. Parenting programme for parents of children at risk of developing conduct disorder: Cost effectiveness analysis. British Medical Journal. 2007;334:682–687. doi: 10.1136/bmj.39126.699421.55. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Eyberg S, Funderburk B, Hembree-Kigin T, McNeil C, Querido J, Hood K. Parent-child interaction therapy with behavior problem children: One and two year maintenance of treatment effects in the family. Child and Family Behavior Therapy. 2001;23:1–20. [Google Scholar]
  6. Eyberg SM, Boggs SR, Algina J. Parent-child interaction therapy: A psychosocial model for the treatment of young children with conduct problem behavior and their families. Psychopharmacology Bulletin. 1995;31:83–91. [PubMed] [Google Scholar]
  7. Eyberg SM, Nelson MM, Boggs SR. Evidence-based psychosocial treatments for children and adolescents with disruptive behavior. Journal of Clinical Child and Adolescent Psychology. 2008;37:215–237. doi: 10.1080/15374410701820117. [DOI] [PubMed] [Google Scholar]
  8. Eyberg SM, Pincus D. Eyberg Child Behavior Inventory & Sutter-Eyberg Student Behavior Inventory-Revised: Professional manual. Odessa, FL: Psychological Assessment Resources; 1999. [Google Scholar]
  9. Fenwich E, Byford S. A guide to cost-effectiveness acceptability curves. British Journal of Psychiatry. 2005;18:106–108. doi: 10.1192/bjp.187.2.106. [DOI] [PubMed] [Google Scholar]
  10. Foster EM, Jones DE. The high costs of aggression: Public expenditures resulting from conduct disorder. American Journal of Public Health. 2005;95:1767–1772. doi: 10.2105/AJPH.2004.061424. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Foster EM, Olchowski AE, Webster-Stratton CH. Is stacking intervention components cost-effective? An analysis of the Incredible Years program. Journal of the American Academy of Child and Adolescent Psychiatry. 2007;46:1414–1424. doi: 10.1097/chi.0b013e3181514c8a. [DOI] [PubMed] [Google Scholar]
  12. Glick HA, Doshi JA, Sonnad SS, Polsky D. Economic evaluation in clinical trials. New York, NY: Oxford University Press; 2007. [Google Scholar]
  13. Goldfine ME, Wagner SM, Branstetter SA, McNeil CB. Parent-child interaction therapy: An examination of cost-effectiveness. Journal of Early and Intensive Behavior Intervention. 2008;5:119–141. [Google Scholar]
  14. Hutchings J, Gardner F, Bywater T, Daley D, Whitaker C, Jones K, et al. Parenting intervention in Sure Start services for children at risk of developing conduct disorder: Pragmatic randomised controlled trial. British Medical Journal. 2007;334:678–693. doi: 10.1136/bmj.39126.620799.55. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Ingoldsby EM. Review of interventions to improve family engagement and retention in parent and child mental health programs. Journal of Child and Family Studies. 2010;19:629–645. doi: 10.1007/s10826-009-9350-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Jones DJ, Forehand R, Cuellar J, Kincaid C, Parent J, Fenton N, Goodrum N. Harnessing innovative technologies to advance children’s mental health behavioral parent training as an example. Clinical Psychology Review. 2013;33:241–252. doi: 10.1016/j.cpr.2012.11.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Jones DJ, Forehand R, Cuellar J, Parent J, Honeycutt A, Khavjou O, Gonzalez M, Anton M, Newey G. Development and pilot evaluation of a technology-enhanced behavioral parent training program for children’s disruptive behavior disorders (in press) [Google Scholar]
  18. Jones K, Daley D, Hutchings J, Bywater T, Eames C. Efficacy of the Incredible Years Programme as an early intervention for children with conduct problems and ADHD: Long-term follow-up. Child: Care, Health and Development. 2008;34:380–390. doi: 10.1111/j.1365-2214.2008.00817.x. [DOI] [PubMed] [Google Scholar]
  19. Kazak AE, Hoagwood K, Weisz JR, Hood K, Kratochwill TR, Vargas LA, Banez GA. A meta-systems approach to evidence-based practice for children and adolescents. American Psychologist. 2010;65:85–97. doi: 10.1037/a0017784. [DOI] [PubMed] [Google Scholar]
  20. Long P, Forehand R, Wierson M, Morgan A. Does parent training with young noncompliant children have long-term effects? Behaviour Research and Therapy. 1994;32:101–107. doi: 10.1016/0005-7967(94)90088-4. [DOI] [PubMed] [Google Scholar]
  21. McMahon RJ, Forehand R. Helping the noncompliant child: Family-based treatment for oppositional behavior. 2nd ed. New York, NY: Guilford Press; 2003. [Google Scholar]
  22. O'Neill D, McGilloway S, Donnelly M, Bywater T, Kelly P. A cost-effectiveness analysis of the Incredible Years parenting programme in reducing childhood health inequalities. European Journal of Health Economics. 2013;14:85–94. doi: 10.1007/s10198-011-0342-y. [DOI] [PubMed] [Google Scholar]
  23. Piquero AR, Farrington DP, Welsh BC, Tremblay R, Jennings WG. Effects of early family/parent training programs on antisocial behavior and delinquency. Journal of Experimental Criminology. 2009;5:83–120. [Google Scholar]
  24. Reitman D, McMahon RJ. Constance “Connie” Hanf (1917–2002): The mentor and the model. Cognitive and Behavioral Practice. 2013;20:106–116. [Google Scholar]
  25. Sanders MR. Triple P-Positive Parenting Program as a public health approach to strengthening parenting. Journal of Family Psychology. 2008;22:506–517. doi: 10.1037/0893-3200.22.3.506. [DOI] [PubMed] [Google Scholar]
  26. Sanders MR, Markie-Dadds C, Tully LA, Bor W. The triple P-positive parenting program: A comparison of enhanced, standard, and self-directed behavioral family intervention for parents of children with early onset conduct problems. Journal of Consulting and Clinical Psychology. 2000;68:624–640. [PubMed] [Google Scholar]
  27. Scott S, Knapp M, Henderson J, Maughan B. Financial cost of social exclusion: Follow up study of antisocial children into adulthood. British Medical Journal. 2001;323:191–195. doi: 10.1136/bmj.323.7306.191. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Webster-Stratton C, Hancock L. Training for parents of young children with conduct problems: Content, methods, and therapeutic processes. In: Schaefer CE, Briesmeister JM, editors. Handbook of parent training. New York, NY: John Wiley; 1998. [Google Scholar]

RESOURCES