The Cost-Effectiveness of Parent-Child Interaction Therapy: Examining Standard, Intensive, and Group Adaptations

Abstract

Purpose:

This study examined the cost-effectiveness of standard parent-child interaction therapy (PCIT) and three adaptations: intensive-PCIT (I-PCIT), small group PCIT, and large group PCIT.

Methods:

This study used cost-effectiveness analyses to calculate average cost-effectiveness ratios, which represents the average cost for one family to change one standard deviation on each outcome measure: externalizing behavior problems, positive parenting skills, negative parenting skills, child compliance, and parenting stress.

Results:

While it had the lowest initial set up cost, results indicated that standard PCIT was the least cost-effective option in reducing child disruptive behaviors and in increasing child compliance. Large group PCIT was the most cost-effective in increasing positive parenting skills and child compliance and in reducing negative parenting skills and parenting stress. I-PCIT was the most cost-effective in reducing child disruptive behaviors and the second most cost-effective option in increasing positive parenting skills and child compliance and in decreasing negative parenting.

Conclusions:

As large group and I-PCIT were the most cost-effective in different domains, both could be recommended to parents as treatment options. Future research should confirm our cost-effective results within community settings.

Disruptive behavior disorders (DBDs), including oppositional defiant disorder, conduct disorder, and attention-deficit/hyperactivity disorder, are among the most prevalent mental health disorders in childhood (Danielson et al., 2018; Polanczyk, Salum, Sugaya, Caye, & Rohde, 2015). DBDs are diagnosed in 50% of children across health care sectors such as alcohol and drug services, child welfare, juvenile justice, and public schools (Garland et al., 2001). Characterized by aggression and defiance, these disorders show great persistence over time (Frick & Loney, 1999). Left untreated, the pervasive constellation of symptoms during early childhood can persist into later stages of development and are predictive of poor academic performance, substance abuse, and criminal activity (Kent et al., 2011; Loe & Feldman, 2007; Moffitt, Caspi, Dickson, Silva, & Stanton, 1996; Wilens et al., 2011). Once adults, these individuals incur a cost that is 10 times greater than their healthy counterparts (Scott, Knapp, Henderson, & Maughan, 2001), while another study estimated up to $2.3 million dollars could be saved via successful treatment of high-risk youth with disruptive behavior (Cohen & Piquero, 2009). The high public health cost of DBDs occurring across multiple sectors of care, highlight the need for evidence-based and cost-effective treatments to be implemented within an early intervention framework.

One model for examining if early intervention programs are worth sustained investment is RE-AIM (Glasgow, Vogt, & Boles, 1999). The five key components of RE-AIM are: reach: representativeness of individuals who participated in the program and potential barriers to participation (e.g., cost); efficacy/effectiveness: the impact of an intervention on specified outcome criteria and assessing positive and negative outcomes, including economic outcomes; adoption: representativeness of settings and staff that are willing to start a program; factors associated with adoption can include cost, level of resources, and expertise required; implementation: intervention integrity; maintenance: long term-maintenance of gains at the individual level (do participants maintain behavior change) and the organizational level (extent to which a treatment is sustained over time in an organization) (Gaglio, Shoup, & Glasgow, 2013; Glasgow et al., 1999). As seen in the RE-AIM model cost, including the cost to implement, sustain a treatment, and time, impacts multiple components. One efficacious treatment to address DBDs in young children is parent-child interaction therapy (PCIT) (Eyberg, Boggs, & Algina, 1995; Zisser & Eyberg, 2010). Given its success and widespread implementation, PCIT represents a viable treatment option for community dissemination; however, the cost-effectiveness is still largely unknown. Given the negative trajectory of DBDs and their high cost to society, this paper aims to examine the cost-effectiveness of PCIT and its different adaptations.

To provide a brief overview of PCIT for context, therapists coach parents in-vivo to use specific skills to increase the occurrence of positive and prosocial behaviors in their children. Three recent meta-analyses have shown the overall effectiveness of PCIT, indicating significant decreases in child disruptive behaviors and parental stress, while showing significant improvements in positive parenting and the quality of the parent-child relationship (Cooley, Veldorale-Griffin, Petren, & Mullis, 2014; Thomas, Abell, Webb, Avdagic, & Zimmer-Gembeck, 2017; Ward, Theule, & Cheung, 2016). Furthermore, longitudinal studies have shown persistence of treatment gains up to three years later (Boggs et al., 2005; Hood & Eyberg, 2003).

While there is growing literature examining the cost-effectiveness of other behavioral parent training interventions in treating DBDs (Frey et al., 2019; O’Neill, McGilloway, Donnelly, Bywater, & Kelly, 2013; Sampaio et al., 2018) surprisingly only two studies have examined the cost-effectiveness of PCIT (French, Yates, & Fowles, 2018; Goldfine, Wagner, Branstetter, & Mcneil, 2008). Goldfine and colleagues (2008) examined the overall cost-effectiveness of PCIT, reporting that it cost roughly $1,000 today to produce a one standard deviation reduction in externalizing behaviors on the Eyberg Child Behavior Inventory (ECBI; Eyberg, 1999) from pre- to post-treatment. French, Yates, and Fowles (2018) compared home-based delivery to standard clinic delivery and found that home delivery of PCIT was initially more expensive and significantly less cost-effective than clinic based PCIT. While these previous papers are both strides in the right direction, no adaptation of PCIT delivered in a clinic setting has been examined through a cost perspective and implementation framework.

Although standard PCIT has been shown to be comparable in cost to other effective treatments for DBDs (Goldfine et al., 2008), such as Incredible Years (Foster, Olchowski, & Webster-Stratton, 2007), the overall cost to implement can still be substantial, limiting its potential dissemination and adoption. One adaptation that may combat some of these limitations is group PCIT. While previous studies have been conducted in small groups with few families (Nieter, Thornberry, & Brestan-Knight, 2013), one recent paper examined a large group adaptation, embedded within a larger summer program (Graziano, Ros, Hart, & Slavec, 2018). Alternatively, an intensive adaptation (I-PCIT) (Graziano et al., 2015) was created, which increases the time and frequency of sessions and is completed within two weeks. While both adaptations show initial promise, no cost analyses have been conducted.

The Current Study

Given parenting interventions are the recommended first line of treatment for children with DBDs (Wolraich et al., 2019), assessing the cost-effectiveness of these treatments is critical to better inform decision making and increase chances of adoption as suggested by the RE-AIM framework. Parents may also be hesitant to engage in treatment due to its high cost and time commitment (Spoth & Redmond, 2000). As PCIT has become a gold standard for treating DBDs in young children and many adaptions are being created and implemented, it is critical to evaluate both their effectiveness and cost. Additionally, it is important to understand not only their effectiveness as it relates to impacting child behavior, but addressing related treatment targets, such as improving overall parenting skills and decreasing parental stress (Kazdin & Whitley, 2003). Therefore, this paper examined the cost-effectiveness of standard PCIT and three of its adaptations: I-PCIT, small group PCIT, and large group PCIT. Given previous work showing a large group format could be the most cost-effective (McRoberts, Burlingame, & Hoag, 1998), we predicted that large group PCIT would be the most overall cost-effective option across outcomes.

Method

Design

All costs were calculated based on Yates’ (1999) method, which includes the basic cost categories to the emphasis on multiple outcomes (Yates, 1999). Calculations were based on a new clinician starting and implementing treatment in a clinical setting. As used in previous PCIT cost-effectiveness papers (French et al., 2018; Goldfine et al., 2008) it was assumed that a full-time clinician could see about 25 cases per week, at one hour per case (i.e., 25 hours of treatment per week). Due to the fact that most of the studies were randomized controlled trials conducted in a research setting, there was a limited number of clients that could be seen. For the purposes of this paper, data were extrapolated, and costs were calculated based on a full-time clinician setting up a practice. Costs incurred due to research (e.g., participant payment) were not included in the total cost, as they would not occur in a clinical or community setting. While it is understood that clinicians may be involved in other treatments, this method was chosen to standardize the total hours across different treatments in order to directly compare their costs. Second, as therapy happens in clinic and community settings it is important to understand how much it would cost if a clinician wanted to start implementing PCIT.

Treatment

All treatments were based on standard PCIT principles.

Standard PCIT.

Standard PCIT is broken down into two distinct phases, conducted in one-hour weekly sessions: Child-Directed Interaction (CDI) and Parent-Directed Interaction (PDI; Eyberg et al., 1995; Zisser & Eyberg, 2010). During CDI, parents follow their child’s lead in play by using the PRIDE skills (i.e., do skills): Praising the child, Reflecting the child’s statements, Imitating the child’s play, Describing the child's behavior, and using Enjoyment. Parents learn to apply PRIDE skills to the child’s appropriate play and ignore undesirable behaviors. They are also taught to avoid verbalizations that take the lead away from the child during the play (i.e., don’t skills), including questions, commands, and negative statements (e.g., criticism). During PDI, parents set limits to reduce child noncompliance/negative behavior and learn to use effective commands and consistently follow through with timeout for noncompliance. Parents also learn how to deal with aggressive behavior and public misbehavior. The first session for each phase involves a “teach” session where the skills are presented to the parent. The teach session for each phase is followed by a series of coaching sessions where the therapist coaches the parent in-vivo through a one-way mirror (using a wireless headset) on their use of the CDI and PDI skills with their child. In traditional PCIT, parents must meet “mastery” criteria after each phase to progress and complete treatment. Mastery of CDI is met when parents are able to demonstrate a high level of positive parenting skills during a five-minute observation period. Parents complete PDI and graduate from PCIT after they demonstrate mastery in implementing appropriate consequences during a five-minute interaction with their child.

Throughout treatment, parents are asked to complete a daily homework assignment, which is a 5-minute at home, special play time with their child practicing the PRIDE skills. After the family moves into phase two, parents are also told to practice issuing commands and following through with the time-out sequence at home. To monitor progress each week, parent-child interactions during session are coded using the Dyadic Parent-Child Interaction Coding System (DPICS) (Eyberg, 2013; Eyberg & Robinson, 1981) to assess parenting skills. To monitor the child’s progress each week, the parent fills out the ECBI, which assesses behavior problems. The number of sessions required to complete PCIT varies as progression through the program is data-driven and dependent on parents mastering the necessary skills in both phases.

While previous research has shown it takes families an average of 13 weeks to complete PCIT (Chaffin, Taylor, Wilson, & Igelman, 2007; Eyberg et al., 1995; Gallagher, 2003), the largest PCIT study ever conducted (n = 1,318) reported an average of 20.5 sessions for families to meet mastery criteria and complete treatment (Lieneman, Quetsch, Theodorou, Newton, & McNeil, 2019). For the purposes of this study, we averaged the two treatment lengths ((13+20.5)/2=16.75) and used 16 sessions in our standard PCIT analyses to be conservative. Data were pulled from three meta-analyses that examined the effectiveness of PCIT (Cooley et al., 2014; Thomas et al., 2017; Ward et al., 2016) and the largest PCIT study (Lieneman et al., 2019).

Multiple meta-analyses allow for an increase in representativeness and reach of participants. However, meta-analyses can also lead to limits in reporting, as seen in Table 1, especially around implementation and adoption which are critical in advancing the literature and dissemination of PCIT. For adoption, there was no information about how different settings or the degree to which PCIT was delivered by various staff impacted outcomes. No paper discussed if staff were willing to initiate a program or if implementation continued after the studies concluded.

Table 1.

Utilizing the RE-AIM framework across adaptations

		Standard PCIT				I-PCIT		Small Group			Large Group
Categories within RE-AIM		Cooley et al., (2014)	Ward et al., (2016)	Thomas et al., (2017)	Lineman et al., (2019)	Graziano et al., (2015)	Graziano et al., (in press)	Niec et al., (2016)	Neiter et al., (2013)	Foley et al., (2016)	Graziano et al., (2018)
R	N	11 studies (n=640)	12 studies (n=372)	23 studies (n=1,144)	n=1,318	n=11	n=30	n=39	n=27	n=20	n=154
	Age in years	Range: 1.5 - 12	Range: 2 - 5	Range: 1.5 - 15	Range: 2-7	Range: 3 – 8	Range: 2 – 6.29	Range: 3 - 6	Range: 2 - 7.67	Average=6.45	Average=5.06
	Ethnicity	N/A	69% Caucasian	N/A	53.5% Caucasian	73% Latinx	85% Latinx	54.6% Caucasian	63% Caucasian	95% Caucasian	82% Latinx
	Diagnosis/Comorbidity	Allowed for comorbidities; diagnosis not required	Required diagnosis of ADHD, ODD, and/or CD	Allowed for comorbidities; diagnosis not required	Allowed for comorbidities; diagnosis not required	Allowed for comorbidities; diagnosis not required	Allowed for comorbidities; diagnosis not required	Required diagnosis of ODD or CD	Allowed for comorbidities; diagnosis not required	Allowed for comorbidities; diagnosis not required	Allowed for comorbidities; diagnosis not required
	Inclusion/Exclusion	Studies used traditional formats of PCIT; clear inclusion & exclusion criteria	Studies used traditional formats of PCIT; clear inclusion & exclusion criteria	Included adaptations of PCIT; clear inclusion & exclusion criteria	Inclusion criteria was any family that came into the clinic for PCIT services. No exclusion criteria were presented	Clear inclusion & exclusion criteria	Clear inclusion & exclusion criteria	Clear inclusion & exclusion criteria	Clear inclusion & exclusion criteria	Families with a documented history of or perceived risk for child abuse/neglect; clear inclusion & exclusion criteria	Clear inclusion & exclusion criteria
E	Attrittion	N/A *	N/A *	N/A *	914 families attended at least 4 sessions; of these remaining families, 681 terminated treatment	Only one session was missed by a single family and rescheduled	Attrition rate of 3%	Attrition rate of 26%	Attrition rate of 37%	Attrition rate of 5%	Attrition rate of 2.6%
	Primary Outcomes	ECBI; PSI	ECBI	ECBI; PSI; DPICS	ECBI	ECBI; PSI; DPICS; TAI	ECBI; PSI; DPICS; TAI	ECBI; PSI; DPICS; TAI	ECBI; PSI; DPICS; TAI	ECBI; PSI; DPICS; TAI	PSI; DPICS; BASC; TAI
	Additional Analyses	N/A	Examined if gender or diagnosis moderated treatment outcome	Compared results from studies with modifications to PCIT and studies with active versus nonactive controls	Compared ECBI changes for those who completed treatment and those who terminated after at least 4 sessions	N/A	Moderation analyses on how parental stress impacted response to treatment	N/A	Families who didn’t complete treatment didn’t statistically differ than those who did	N/A	Examined if demographic variables differed for families who completed the follow-up to those that didn’t
A	Who delivered the treatment	N/A *	N/A *	N/A *	Community-based clinicians	Clinical psychology graduate students, who were trained in PCIT	Clinical psychology or mental health counseling graduate students, who received at least 40 hours of training in PCIT	Graduate-level therapists	Graduate-level therapists	Graduate-level therapists; treatment as usual (TAU) delivered by employees at the community mental health center	Clinical psychology graduate students who were trained in PCIT
	Supervision	N/A *	N/A *	N/A *	N/A	Supervised by a licensed clinical psychologist, who is a certified PCIT trainer	Supervised by a licensed clinical psychologist, who is a certified PCIT trainer	Supervised by a licensed psychologist with expertise in PCIT	Supervised by a licensed psychologist with expertise in PCIT	Supervised by a licensed psychologist with expertise in PCIT	Weekly supervision by a licensed psychologist who was also a certified PCIT trainer
	Setting	Reported that studies that occurred in multiple settings	N/A *	Reported where each study was conducted	Community mental health agencies	University clinic	University clinic	University clinic	Conducted in Child Advocacy Centers (CAC)	Community setting	A local school
	Is program still ongoing?	N/A *	N/A *	N/A *	While not reported in the paper, does seem like PCIT is a service still offered in these clinics that participated	To our knowledge, still an option given to parents; however, was not presented in the	To our knowledge, still an option given to parents; however, was not presented in the paper	N/A	N/A	N/A	The summer camp including this adaptation is still occurring; this information is not presented in the paper but rather on the corresponding author’s website
I	Comparison Group	Study was primarily interested in looking at studies that implemented a treatment versus control group to	9 studies had a control group comparison	All studies had a control comparison	No comparison group	No comparison group; did compare effect sizes standard PCIT	Comparison group received standard, time-limited PCIT	Compared to individual PCIT with the same number of sessions	No comparison group	Compared TAU group	No comparison group; did compare effect sizes to standard PCIT
	Fidelity	N/A *	N/A *	N/A *	N/A	Detailed descriptions on how fidelity was assessed	Detailed descriptions on how fidelity was assessed	Detailed descriptions on how fidelity was assessed	Detailed descriptions on how fidelity was assessed	Detailed descriptions on how fidelity was assessed	Detailed descriptions on how fidelity was assessed
	Cost to family	N/A *	N/A *	N/A *	N/A	N/A	As study was part of research, intervention was provided at no cost to the families	N/A	As study was part of research, intervention was provided at no cost to the families	Families received treatment at no cost and were compensated for participating	Families received the larger intervention at a subsidized cost via a local or federal grant; no additional cost for families to attend large group PCIT
	Incentives	N/A *	N/A *	N/A (did not examine individual studies reviewed the meta-analysis)	N/A	N/A	Gift cards ($25) were provided to families at the post and follow-up assessments	N/A	Provided childcare, transportation, dinner, rewards to parents who completed homework (e.g., coupons to local restaurants)	Offered individual make-up session and provided meals and childcare for siblings during sessions.	Dinner and childcare were provided during all sessions
	Cost of Implementation	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A
M	Maintenance of gains	N/A *	N/A *	Addressed maintenance of gains; 4 studies had follow-up of the ECBI, indicating maintenance of treatment effects	No follow up	Treatment gains were maintained 4 months post treatment	Almost all treatment gains were maintained 6-9 months post treatment	Treatment gains remained 6 months post-treatment	N/A	N/A	Subsample of 90 families completed a follow-up 6-9 post treatment; all effects were maintained

Note. PCIT = parent child interaction therapy; I-PCIT = intensive PCIT; R = reach, E = efficacy/effectiveness, A = adoption, I = implementation, M = maintenance, ECBI = Eyberg child behavior inventory, PSI = parenting stress index TAI= The Therapy Attitude Inventory, which measures treatment satisfaction, ADHD = attention-deficit/hyperactivity disorder; CD = conduct disorder, ODD = oppositional defiant disorder, N/A = data was not available/presented.

^*we did not examine individual studies reviewed in the meta-analysis.

I-PCIT.

The first adaptation examined was I-PCIT (Graziano et al., 2015). This adaptation was modeled after a successful, intensive OCD treatment (Storch et al., 2007). I-PCIT deviates from standard PCIT in three main ways. First, I-PCIT is delivered in 90-minute sessions compared to the standard 60-minutes. Second, cases are seen five days a week for two weeks at the clinic (i.e., everyday Monday through Friday for two weeks), as compared to once a week in standard PCIT. Lastly, the length of treatment does not require families to meet mastery in either CDI or PDI, as entire the treatment is always 10 sessions. The first phase consists of 1 CDI teach and 4 CDI coach sessions, while the second phase consists of 1 PDI teach and 4 PDI sessions. Parents are still coached in-vivo using an earpiece, through a one-way mirror. There are two studies that have examined I-PCIT: an open trial feasibility study (Graziano et al., 2015) and a randomized control trial (RCT) comparing I-PCIT to standard, time-limited PCIT (Graziano, Ros, & Hare, in press). In the open trial feasibility study there were 11 families, while the RCT had 30 families in each treatment group.

Large group PCIT.

The third adaptation examined was large group PCIT, which was delivered in 90-minute sessions, once a week, over the course of 8 weeks (Graziano et al., 2018). Each group consisted of 15-20 families led by two therapists, for a total of 154 families. For the purposes of this study, calculations were based on 15 parents to be more conservative in analyses. The entire treatment is 8 sessions: 1 CDI teach and 3 CDI coach sessions, with 1 PDI teach and 3 PDI sessions. For each session, the majority of time is focused on standard PCIT, while the last 30-45 minutes incorporated school readiness topics. Following didactic discussions, parents rotate practicing their newly acquired skills with their own children for 10–15 minutes. During this practice time, two therapists rotate among the subgroups of parents to provide direct live coaching, while the other parents observe and practice coding CDI skills. After completing the first practice period, parents in each subgroup provide constructive feedback to the parent who was practicing. Afterward, two more rotations are competed allowing for more parents to practice their skills with their own children. Upon completion of the entire practice period, everyone reconvenes together in order to discuss their progress and problem solve any issues that arose. It is important to note that every child has at least one parent coached by a therapist during CDI practice and once during PDI practice. Therefore, large group PCIT differs from standard or small group PCIT in a few ways: 1) its ability to serve a large group of parents at once, 2) it does not require parents to achieve mastery criteria, making it time-limited with only 8 sessions in total, 3) parents are only individually briefly coached twice during the entire treatment compared to being extensively coached every session in standard PCIT, and 4) it allows parents to interact and observe other parents practicing their skills, creating a more social environment. As this occurs in a group setting in a large room, parents are not coached via a one-way mirror or an earpiece; clinicians sit next to the parent and live coach them on the skills.

As seen in Table 1, the large group PCIT study did not use the ECBI, making it hard to directly compare to other PCIT studies on child externalizing behavior problems. It is also important to note that this treatment was initially developed for and implemented within the context of a larger treatment (i.e., the summer treatment program for kindergartners; STP-PreK) (Graziano, Slavec, Hart, Garcia, & Pelham, 2014). However, this treatment could easily be implemented in an identical way within a clinic setting, not including the other portions of the STP-PreK. While research has shown that large group PCIT is effective (Graziano et al., 2018), it does limit the findings. To combat these limitations and obtain a different perspective on the cost-effectiveness of group PCIT, we also examined the cost-effectiveness of small group PCIT.

Small group PCIT.

After a comprehensive literature search, only three studies examining the effectiveness of small group PCIT using standard PCIT principles were found (Foley, McNeil, Norman, & Wallace, 2016; Niec, Barnett, Prewett, & Shanley Chatham, 2016; Nieter et al., 2013). Each study was assessed for number of children in each group, length of each session, frequency each session occurred, total number of sessions, number of therapists, supplies needed, and treatment effect sizes. Across the 3 studies, each of these categories were averaged to create one estimate for implementing small group PCIT and then included in the cost-effectiveness analyses. Two studies used traditional methods to coach parents (i.e., one-way mirror, earpiece) (Niec et al., 2016; Nieter et al., 2013), while one study used a similar format as large PCIT by coaching parent in the same room (Foley et al., 2016).

Nieter et al. (2013) was a pilot study with 27 families, with a mixed clinical sample of primarily low SES community families. Foley et al. (2016) compared small group PCIT with 20 families to treatment as usual (TAU), only in families that had a documented history of or a perceived risk for abuse/neglect. Niec et al. (2016) was an RCT examining small group PCIT, which included 39 families, compared to individual PCIT with the same number of sessions.

Clinical Outcomes

For all outcomes, when multiple studies were combined to create one effect size, a weighted average effect size was computed.

Externalizing behaviors.

In line with standard PCIT recommendations, the ECBI was used across studies. The ECBI is a 36-item parent report measure of disruptive behavior that assesses behavior on two scales: intensity scale and problem scale. The intensity scale measures the severity of the behavior, and the problem scale measures how problematic the behavior is for the parent. For the purpose of this study, the intensity scale was used. To assess treatment response for standard PCIT, data was used from three meta-analyses (Cooley et al., 2014; Thomas et al., 2017; Ward et al., 2016) and the largest PCIT study (Lieneman et al., 2019). Additionally, as large group PCIT was embedded in a larger treatment (Graziano et al., 2018) and did not utilize the ECBI, it was not included when examining externalizing behavior problems.

Parenting stress.

The Parenting Stress Index-Short Form (PSI-SF) (Abidin, 1995) is a parent self-report scale that measures stress in the parent-child relationship due to parent distress, difficult child behavior, and dysfunctional parent-child interaction. The PSI-SF contains 36 items rated on a 1 (strongly disagree) to 5 (strongly agree) scale. For the purposes of this study, only studies including a total stress score were included. For standard PCIT only two of the meta-analytic papers (Cooley et al., 2014; Thomas et al., 2017) reported on the PSI. To obtain the correct effect sizes, we individually examined every study reported within the meta-analytic papers. Studies were excluded in our analyses if they were unpublished dissertations, did not report the total stress score, or were assessed in a study utilizing a group format. Effect sizes were calculated from the remaining papers and a weighted averaged effect size was computed to be used in analyses for standard PCIT.

Parenting skills.

As recommend by standard PCIT, DPICS was used across studies. DPICS is an observational, behavioral coding system developed to assess the quality of parent-child interactions in a standardized format (Eyberg, Nelson, Ginn, Bhuiyan, & Boggs, 2013). Two composite categories are created to measure changes in parent’s skills: do skills (i.e., behavior descriptions, reflections, & praises) and don’t skills (i.e., questions, commands, & criticisms). For standard PCIT, only one meta-analysis reported parenting skills (Thomas et al., 2007). Again, to obtain the correct effect sizes for the “do” and “don’t” skills, we referred to the original papers cited in the meta-analysis, excluding studies that utilized a group format. Effect sizes were calculated from each paper listed and then a weighted averaged effect size for “do” skills and “don’t” skills were used in analyses for standard PCIT.

Observed compliance.

Observed compliance was also measured using codes obtained from the DPICS. Observed child compliance was calculated by taking the number of times the child complied to parent commands divided by the total number of parent commands that gave the child an opportunity to comply. None of the small group PCIT studies included this measure. For standard PCIT, only one meta-analysis presented data on compliance (Thomas et al., 2007). We again referred to the original papers cited to obtain the correct effect sizes and then computed a weighted averaged effect size, excluding studies examining a group format.

Measuring Provider Time

As mentioned above, all treatment costs were based on 25 hours of treatment per week. Therapist time was valued using the most recent (2017) median national hourly wage for Clinical, Counseling, and School Psychologists from the Bureau of Labor Statistics (U.S. Department of Labor, May 2017) at the time the analyses were conducted. To calculate the number of families that could be seen in one year, 48 weeks was used as the available time frame. Due to the fact that groups can take longer to put together and to be more conservative in the cost analyses, 44 weeks was used.

Administrative time.

To account for administrative time (e.g., client notes) 15 minutes of clinician time was added on to each session. For group, 30 minutes of clinician time was added on to each session.

Clients.

To calculate the cost for a clinician(s) to serve one family, the total number of session hours for one child to complete therapy plus total administrative time was multiplied by clinician wage, $36.47 (U.S. Department of Labor, May 2017). For example, for one family to complete standard PCIT, it is a total of 16 hours of treatment. Additionally, there is a total of 4.0 hours of administration time (i.e., 15 minutes for each of the 16 sessions) making the total 20 hours for one clinician to provide standard PCIT. To calculate the cost, 20 x $36.47 = $729.40. For large and small group PCIT, the cost was multiplied by 2 as there were 2 therapists. Further in large group PCIT, and some small group PCIT, it is recommended to have assistants, due to the large number of parents and children. Their time was calculated using the current minimum wage, $8.25 (U.S. Department of Labor, May 2017), and added to the total cost for clinician. The total cost for all clinicians was then divided by the number of children per group to get the cost to serve one family.

Training.

The cost of PCIT training was standard across all treatments. It was calculated based on the international PCIT website (http://www.pcit.org/). For one therapist to be trained, with subsequent supervision, it is $5,500, which based on Goldfine et al., (2008) will last about 10 years. This cost includes a full year of consultation, ongoing video session review, and protocol manuals. Therefore, $5,500 was divided by the number of children seen over a 10-year period to calculate the cost to each child. Further as this training, and consequent supervision, is approximately 60 hours that the therapist cannot see clients, lost billable hours were factored in. This was calculated using the hourly reimbursement rate for Medicare ($84.74), multiplied by 60, and also then divided by the number of children seen over a 10-year period to calculate the cost to each family.

Measuring Space & Equipment

Rent.

Rent was estimated based on a recent study (Graziano & Hart, 2016) that rents space in Miami, FL where 3 of the studies were conducted. The rent in that study was $2,500 per month for one large room (i.e., $2.44 per square foot). For large group PCIT, 2 large rooms are needed, making rent $5,000 per month or $60,000 per year. Given that there are no standardized measurements, for standard or I-PCIT, we measured our PCIT therapy and control rooms, which were about 208.25 ft². For standard and intensive PCIT, rent will be approximately $510 per month or $6,097.56 per year. To calculate the rent cost per family, the number of families predicted to be seen each year was divided by the cost of rent per year. For small group PCIT, the rent per family was calculated for each study and then averaged together.

Toys.

The toys chosen were standard across all treatments and based on the international PCIT website (http://www.pcit.org/). For large group PCIT, 3 sets of all the toys are needed. For small group PCIT the cost of toys were calculated for each version of small group PCIT then averaged to get one final cost. Costs were calculated based on the toys being replaced every year. Therefore, to calculate the cost to each child, the total cost of the toys was divided by the number of children that could be seen in one year.

Equipment.

All technical equipment (i.e., earpiece, speakers) were also standard across all treatments and based on the international PCIT website (http://www.pcit.org/). Costs of table and chairs (including time out chair) were based on Amazon prices, which were found to be inexpensive and easily accessible. For large group PCIT, 5 tables and 16 chairs were calculated. For small group PCIT all equipment and supplies were calculated for each version of small group PCIT then averaged to get one final cost. Costs were calculated based on the equipment being replaced every five years. Therefore, to calculate the cost to each child, the total cost of the equipment was divided by the number of children that could be seen in five years.

Construction.

As analyses are based on a new clinician starting, it is most likely that the building they rent space in will not have the one-way mirror needed for treatment. Therefore, we included a one-time construction cost for materials and to remodel the room. Based on estimates for the cost of buying, shipping, and installing the one-way mirror, the total for materials and construction cost is $5,000. These costs were only applied to the treatments where a construction and a one-way mirror would be needed. Based on Goldfine et al. (2008), we estimated that this one-time cost would last at least 20 years before any more work would have to be done. To calculate the cost to each child, $5,000 was divided by the number of children that could be seen in 20 years.

Economic Analysis

A commonly used method to examine the relationship between time, cost to implement treatment, and outcome is a cost-effectiveness analysis (Drummond, Sculpher, Claxton, Stoddart, & Torrance, 2015). These analyses can be calculated using an average cost-effectiveness ratio (ACER), which is the average total cost for one family to complete treatment divided by the treatment’s average effectiveness. For the purposes of this paper, the ACER represents the average cost to move one family one standard deviation on each outcome measures. For all effect sizes Cohen’s d was used, indicating the effect from pre to post treatment. This method can be used to assess treatments from different perspectives (e.g., client perceptive). For the purposes of this paper, a provider perspective was chosen based on the available data.

Results

Initial Cost to Implement Treatment

The initial cost to implement a treatment included initial PCIT training, lost billable hours while at training, all supplies, toys, and construction costs (if needed). The initial cost to set up a practice for both standard PCIT and I-PCIT was $18,352.69. Next, small group PCIT cost $23,197.67, followed by large group PCIT costing $25,840.96. The initial difference in cost is mostly due to the fact that group PCIT require more therapists to be hired and trained.

Cost and Effect per Family

The cost for one family to complete each PCIT treatment is seen in Table 2. Large group PCIT was the least expensive per family, followed by small group PCIT and I-PCIT, while standard PCIT was the most expensive per family.

Table 2:

Breakdown of Costs Per Family

	Clinician Cost	Rent	Training	Lost Billable Hours	Supplies/Equipment	Toys	Construction	Total Cost to Serve 1 Family
Standard PCIT	729.40	81.30	7.33	6.78	5.58	9.02	3.33	842.74
I-PCIT	638.23	84.69	7.64	7.06	5.81	9.39	3.47	756.29
Small Group PCIT	438.29	71.76	4.74	4.38	1.76	8.73	1.21	530.87
Large Group PCIT	97.60	45.46	0.83	0.77	0.30	1.54	0.00	146.50

Note. Each category represents the cost to serve one family. PCIT = parent-child interaction therapy. I-PCIT = intensive parent-child interaction therapy

Individual ACERs

Externalizing behaviors.

In reducing children’s externalizing behaviors, I-PCIT was the most cost-effective, costing 2.5 times less than standard PCIT and 1.4 times less than small group to decrease one standard deviation on the ECBI-I. Small group PCIT cost 1.7 times less than standard PCIT, making standard PCIT the least cost-effective option, as seen in Table 3.

Table 3:

The Average Cost-Effectiveness Ratios for Each Adaption of PCIT

	Total Cost Per Family	Externalizing Behaviors (ECBI)b	Positive Parenting (do skills) b	Negative Parenting (don't skills) b	Parenting Stress (PSI)b	Observed Compliance b	ACER for ECBI-I c	ACER for Do Skills c	ACER for Don’t Skills c	ACER for PSIc	ACER for compliance c
Standard PCIT	842.74	1.03	2.98	1.97	1.21	0.91	818.19	282.79	427.78	696.48	926.09
I-PCIT	765.29	2.36	3.39	2.19	0.89	0.84	320.46	223.09	345.34	849.76	900.34
Small Group PCIT	530.87	1.16	0.74	1.19	0.98	N/Aa	457.65	717.39	446.10	541.7	N/Aa
Large Group PCIT	146.50	N/Aa	1.40	1.37	0.78	0.35	N/Aa	104.65	106.93	187.82	418.57

Note. PCIT = parent-child interaction therapy. I-PCIT = intensive parent-child interaction therapy, ECBI = Eyberg child behavior inventory, PSI = parenting stress index, ACER = average cost-effectiveness ratio.

^aData was not available.

^bCohen’s d, pre to post treatment.

^cFormula for the ACER is the cost to serve one family divided by the effect of each outcome.

Positive parenting skills.

In improving positive parenting skills, as measured by the do skills in DPICS, large group PCIT was the most cost-effective option, costing 2.7 times less than standard PCIT to increase one standard deviation in positive parenting skills. Further, large group PCIT cost about 2.1 times less than I-PCIT and 6.9 times less than small group PCIT. I-PCIT was also 1.3 times less than standard PCIT and 3.2 times less than small group PCIT, making small group PCIT the least cost-effective option in improving positive parenting skills.

Negative parenting skills.

In decreasing the occurrence of negative parenting, as measured by the don’t skills in DPICS, large group PCIT was the most cost-effective option, costing 4 times less than standard PCIT to decrease one standard deviation in negative parenting skills. Large group PCIT also cost about 3.2 times less than I-PCIT and 4.2 times less than small group PCIT. Again, I-PCIT cost 1.2 times less than standard PCIT and 1.3 times less than small group PCIT, making small group PCIT the least cost-effective option.

Parenting stress.

In decreasing total parent stress, large group PCIT was the most cost-effective adaptation, costing 3.7 times less than standard PCIT for one family to decrease one standard deviation on the PSI. Large group PCIT also cost about 4.5 times less than I-PCIT and 2.9 times less than small group PCIT. Additionally, small group PCIT cost 1.3 times less than standard PCIT and 1.5 times less than I-PCIT, making I-PCIT the least cost-effective option.

Observed compliance.

In increasing children’s frequency of compliance, large group PCIT was the most cost-effective option, costing 2.2 times less than standard PCIT and I-PCIT to increase one standard deviation in observed child compliance. I-PCIT and standard PCIT were close in cost, but standard PCIT was the least cost-effective option.

Discussion

PCIT is regarded as one of the gold standards in behavior parent training for young children with DBDs. Recently, the field has moved towards adapting treatments in order to increase their cost-effectiveness and facilitate their implementation and adoption in the community. While the field has produced empirically supported, effective therapies, they are expensive to implement. Results show that initial set up costs are expensive, ranging from about $18,300 to almost $25,000, indicating that while PCIT is effective, it is expensive to implement. Given that early intervention of DBDs can prevent future behavior problems and reduce long-term costs (Cohen & Piquero, 2009) it is important to examine PCITs cost-effectiveness to make the most informed decisions for clinicians and families. This is the first paper to examine the cost-effectiveness of three adaptations of PCIT delivered in a clinic setting. The current paper also goes a step further in examining not only child outcomes, but the cost-effectiveness of other treatment targets of PCIT. Our results indicate that large group PCIT appears to be the best overall cost-effective option, followed by I-PCIT.

Standard PCIT

With regard to standard PCIT, our results indicated that it was the least cost-effective option in reducing child disruptive behaviors and increasing observed child compliance. Further, it was the second least cost-effective option in improving positive parenting and decreasing negative parenting skills and parenting stress. While it had the lowest initial set up cost, other adaptations of PCIT are less expensive to implement when compared to their effectiveness. As standard PCIT is implemented more than its adaptations, as evident by the large quantity of studies conducted (Cooley et al., 2014; Thomas et al., 2017; Ward et al., 2016), it is important to understand that it may not be the most cost-effective option. One factor influencing the cost-effectiveness is the number of sessions for families to complete treatment, which can vary significantly. For example, Lineman et al., (2019) reported that 26% of families required 25 or more sessions to meet mastery and complete treatment. While waiting for families to meet mastery has shown to have its benefits (Thomas et al., 2017), it can create low caregiver motivation and become too costly (Lieneman et al., 2019). If 20.5 sessions was used in the current analyses, standard PCIT would have been the least cost-effective option across all outcome domains, indicating the importance of examining cost-effectiveness of time-limited PCIT.

Small Group PCIT

Next, small group PCIT was the least cost-effective adaptation in improving positive parenting skills and decreasing negative parenting skills, while also being the second least cost-effective option in reducing child disruptive behaviors. While groups have been shown to be more cost-effective (Duncan, MacGillivray, & Renfrew, 2017), it is also important to consider initial cost of treatment, treatment length, and the number of families able to be treated. As evident in our ACER for small group PCIT, group therapy may not always be the most cost-effective option.

I-PCIT

The intensive adaptation of PCIT was the most cost-effective in reducing child disruptive behaviors. It was also the second most cost-effective option in increasing positive parenting skills and child compliance and in decreasing negative parenting. However, I-PCIT was the least cost-effective adaptation in decreasing parental stress. These results indicate I-PCIT was less effective in reducing parenting stress relative to its cost to implement. Given that parents may have been experiencing high levels of stress related to their child for some time, two weeks may not be sufficient enough time to significantly reduce parenting stress compared to longer treatments. Additionally, while families assigned to I-PCIT in the RCT were less likely to initiate treatment, potentially due to the intense commitment, once engaged these families were more likely to complete treatment when compared to standard, time-limited PCIT (Graziano et al., in press). As treatments struggle with high attrition, including PCIT (Danko, Garbacz, & Budd, 2016), I-PCIT may be a helpful adaption in minimizing attrition and improving cost-effectiveness.

Large Group PCIT

Finally, large group PCIT was the most cost-effective in increasing positive parenting skills, child compliance, and in reducing negative parenting skills and parenting stress. It is important to note that there is no cut-off in the field for what consists of large versus small group. In terms of our study, “large” group PCIT had about three times more families compared to the “small” groups. However, large group PCIT cost about three to seven times less per family than small group, indicating the cost-effectiveness of large group may not be purely a function of treating more families at once.

Overall

As hypothesized, our results indicate that large group PCIT was the most overall cost-effective option, followed by I-PCIT. Giving parents two cost-effective options might aid in initial buy-in to treatment and increase retention, which many treatments struggle in (Werba, Eyberg, Boggs, & Algina, 2006). The group aspect may appeal to parents as it can create a supportive social environment and allow parents to develop relationships with others experiencing similar difficulties. Additionally, a group format allows parents to observe and learn from each other. Future work is needed in understanding how social support plays a role in treatment outcomes across adaptations.

Furthermore, as large group PCIT was the most cost effective in reducing parental stress, followed by small group, it is also possible the social dynamics of a group could be key to reducing parental stress. In one small group PCIT study, the authors examined if a self-reported measure of social support differed in small group PCIT compared to individual PCIT (Niec et al., 2016). They found no statistical difference but cited anecdotal evidence that the small group families routinely interacted outside of treatment. However, the authors did not examine if social supported impacted parental stress. While studies have found parenting stress reduces in a group format compared to individual (Danino & Shechtman, 2012), almost no studies have examined this difference in children with DBDs. Given that high parenting stress is a reason parents seek services (Jones, Putt, Rabinovitch, Hubbard, & Snipes, 2017) it is also an important mechanism of treatment to examine. Parents struggling with difficult children experiencing high levels of stress (Eyberg, Boggs & Rodriguez, 1993) may be motivated to see quicker results in their children, making an intensive, two-week treatment more appealing. While I-PCIT was the least cost-effective option in reducing parental stress, moderation analyses in the RCT indicated that parents in I-PCIT experiencing high levels of stress had significantly greater decreases in child externalizing behaviors compared to standard, time-limited PCIT (Graziano et al., in press). As it is particularly important to understand who can benefit most from what treatment (Glasgow, Lichtenstein, & Marcus, 2003), these findings highlight the need for future research on I-PCIT, its long-term outcomes, and how it directly compares to other adaptations.

As highlighted in the RE-AIM framework, to increase implementation and adoption in community settings it is also important to consider the training time and commitment required, which can present significant barriers (Jones et al., 2017). PCIT provides a great foundation as it is evidence-based and its training is comparable in price to other evidence-based treatments. However, in standard PCIT the need for parents to meet mastery criteria can make treatment considerably longer, even lasting 72 weeks, which may also contribute to high attrition (Lieneman et al., 2019). Given that large group PCIT, which had the lowest number of treatment hours, and I-PCIT, the shortest length of treatment, were overall the most cost-effective, it appears that implementations in a community setting should focus on more time-limited adaptations of PCIT. Additionally, PCIT can be implemented in children with varying levels of disruptive behaviors and allows for comorbidities, which increases its external validity and chances of adoption. Further, as many community clinics treat a broad range of symptoms, PCIT has been slightly adapted and shown to be effective in treating children with a range of disorders including anxiety (Pincus, Eyberg, & Choate, 2005) and autism spectrum disorder (Ros & Graziano, 2019).

While this paper is a step in the right direction, it is not without limitations. One limitation of our cost-effectiveness analyses is that we pulled data from different papers, including meta-analyses, which combine costs and outcomes from multiple studies over different years. Although th1is does allow for better reach in examining PCIT across many different studies. In addition, because we pulled data from previously completed studies, we were unable to account for the effects of attrition on cost. Future research should randomize children to each of the four treatment conditions and concurrently track expenses, for both client and provider. Assessing the cost and effectiveness of each treatment in real time would allow for statistical comparisons and for the cost of each family per session. Given that there is only one study of large group PCIT, future research should examine large group PCIT as a stand-alone treatment in order to examine the individual effects on the child. Additionally, as I-PCIT was based on two studies, it is important to be cautious when interpreting the results, as future research is needed in further examining these adaptations.

Another limitation is most studies were conducted in a university-based clinic setting, as seen in Table 1, and all studies were conducted in the setting of research trials. It is important to note that PCIT studies have been implemented within a community setting (Danko et al., 2016; Self-Brown et al., 2012) and some were included in our cost analyses (Foley et al., 2016; Nieter et al., 2013). A critical next step is to further our understanding of the RE-AIM framework within each adaption and confirm our cost-effective results within community settings. Additionally, we did not examine every adaption of PCIT. Future research should also examine PCIT as a prevention effort (Berkovits, O'Brien, Carter, & Eyberg, 2010) in order to assess the cost-effectiveness and potential long-term implications and potential savings. As the field is also moving toward more internet delivered approaches (Jent, Brown, & Weinstein, 2018), future research should also examine the cost-effectiveness of internet-based delivery of PCIT. Finally, one of the most common outcomes in cost-effectiveness analyses is the quality adjusted life years (Drummond et al., 2015), which we were unable to use as none of the studies incorporated it. While this measure is typically used in the medical literature, it may be important for more clinical studies to incorporate it allowing for a broader interpretation of results. As stated in the RE-AIM model, is it important to include broader outcomes and those that allow for comparison to public health goals (Glasgow et al., 2003; Glasgow et al., 1999). Despite its limitations this is the first study to report on the cost-effectiveness of different clinic-based adaptations of PCIT and to include associated treatment targets (i.e., stress, parenting skills), serving as a steppingstone for future research.

Conclusion

Given the well-established nature of behavior parent training programs like PCIT, recent calls to actions (Aarons, Hurlburt, & Horwitz, 2011; Glasgow et al., 2003) have focused on the transportability and real-world applications treatment, with emphasis on cost. The current paper contributes to this emerging literature by demonstrating that large group and I-PCIT are the more cost-effective options across multiple domains, combating limitations of standard PCIT and increasing chances of implementation and adoption in the community. On one hand, large group PCIT can serve many families at once and embeds a social support system, which may play a key role in reducing parental stress. Alternatively, parents looking for a quicker remedy or are resistant to the long commitment of many treatments have the option of I-PCIT, which is also cost-effective and yields similar treatment outcomes. In line with the goals of RE-AIM, future research should also consider how family variables (e.g., high versus low parental stress) influences the cost-effectiveness of PCIT. Disentangling the variables that may moderate the cost-effectiveness of each adaptation is critical, given an individual's unique characteristics play a role in treatment response. Combining a more individualized approach to treatment, while simultaneously taking into account the cost-effectives, may yield for the most beneficial treatment outcomes to society and families.