A Community Mental Health Implementation of Parent-Child Interaction Therapy (PCIT)

Abstract

Parent-Child Interaction Therapy (PCIT) has been identified as an evidence-based practice in the treatment of externalizing behavior among preschool-aged youth. Although considerable research has established its efficacy, little is known about the effectiveness of PCIT when delivered in a community mental health setting with underserved youth. The current pilot study investigated an implementation of PCIT with primarily low-socioeconomic status, urban, ethnic minority youth and families. The families of 14 clinically referred children aged 2–7 years and demonstrating externalizing behavior completed PCIT initial assessment, and 12 began treatment. Using standard PCIT completion criteria, 4 families completed treatment; and these families demonstrated clinically significant change on observational and self-report measures of parent behavior, parenting stress, and child functioning. Although treatment dropouts demonstrated more attenuated changes, observational data and parent-reported problems across sessions indicated some improvements with lower doses of intervention. Attendance and adherence data, referral source, barriers to treatment participation, and treatment satisfaction across completers and dropouts are discussed to highlight differences between the current sample and prior PCIT research. The findings suggest that PCIT can be delivered successfully in an underserved community sample when families remain in treatment, but that premature dropout limits treatment effectiveness. The findings suggest potential directions for research to improve uptake of PCIT in a community service setting.

Introduction

Disruptive behavior disorders (e.g. attention-deficit hyperactivity disorder, oppositional defiant disorder, conduct disorder) are among the most common reasons children are referred for mental health services (Kazdin, Mazurick, & Siegel, 1994; Reid, 1993). This fact is unsurprising in light of the reported prevalence rates for these disorders, which are as high as 7% for ADHD, 10% for conduct disorder, and 16% for ODD (American Psychiatric Association, 2000). The DSM-IV criteria for ODD and CD may even exclude some “sub-clinical” externalizing youth who are nevertheless experiencing functional impairment (Rowe, Maughan, Costello, & Angold, 2005), which suggests that prevalence rates of children in need of treatment for disruptive behavior might be higher still. These findings are sobering considering the likelihood for untreated disruptive behavior problems to persist and the association between childhood disruptive behavior and negative outcomes, such as delinquency and criminality (Farrington, 1995; Loeber, Green, Keenan, & Lahey, 1995; Vitelli, 1997).

Following the pattern displayed by research on a range of clinical issues (Bernal & Scharrón-Del-Río, 2001), controlled clinical trials of treatment programs for disruptive behavior disorders have typically included low numbers of ethnic minorities, resulting in multiple calls for increased representation (e.g., Kazdin, 2003; Miranda et al., 2005). In a recent meta-analytic review of controlled studies of evidence-based treatments for ethnic minority youth, Huey and Polo (2008) were unable to identify any interventions that could be categorized as well-established for those populations using the efficacy criteria delineated by the Task Force of Division 12 of the American Psychological Association (Chambless et al., 1996). Nevertheless, the full extent to which ethnic minority youth have been excluded from evidence-based practice research remains unclear due to inconsistent reporting of participant demographics within the literature (Weisz, Doss, & Hawley, 2005).

Part of the clinical research disparity between ethnic minority and white populations appears to stem from the settings in which the groups are most likely to seek services. Ethnic minority youth are less likely to initiate or complete psychosocial treatment (Rawal, Romansky, Jenuwine, & Lyons, 2004; Snowden, 1999; Sue, Fujino, Hu, Takeuchi, & Zane, 1991) and, when they do, treatment is most likely to occur in a community setting (Takeuchi, Bui, & Kim, 1993; Weersing & Weisz, 2002). Indeed, the majority of efficacy trials with youth have been conducted in highly controlled settings that do not closely resemble service clinics (Southam-Gerow, Weisz, & Kendall, 2003; Weisz et al., 2005). Simultaneously, treatment received in community mental health clinics is less likely to include evidence-based “best practices” (Weisz, Han, & Valeri, 1997). It is essential that clinical studies of evidence-based practices move beyond controlled research settings and into the settings where ethnic minority youth routinely receive treatment, such as community mental health centers (CMHCs).

Provision of treatment in CMHCs presents a new set of challenges for the use of evidence-based practices, in that attrition from treatment is higher in CMHCs (Garcia & Weisz, 2002). At present, the reasons for these attrition levels are unclear; however, high rates of client dropout and other aspects of engagement, such as attendance and adherence, represent a significant concern when assessing treatment effects. Engagement is a particularly complex issue in child and family treatment due to the influence of multiple individuals (e.g., child, parents, teachers) who may not share a common conceptualization of the problem or equal motivation for change. Among low-SES, ethnic minority children and families, a range of physical (e.g., service accessibility, scheduling) and psychosocial barriers (e.g., stigma, social support) have been identified in the literature (Gross, Julion, & Fogg, 2001; Harrison, McKay, & Bannon, 2004; McKay, Stoewe, McCadam, & Gonzales, 1998). Nevertheless, findings have often been inconsistent across studies with regard to the presence of any particular set of client characteristics or barriers for clients who discontinue services (McKay & Bannon, 2004). Investigation of barriers to parental treatment engagement is of particular concern for the families of children who display externalizing behavior since parent training has emerged as a efficacious “best practice” for those problems (Eyberg, Nelson, & Boggs, 2008; Serketich & Dumas, 1996). In behavioral parent training models, the skills and behaviors of caregivers are the primary focus of treatment, placing considerable importance on their engagement in therapy.

Parent-Child Interaction Therapy (PCIT; Eyberg & Robinson, 1982; Zisser & Eyberg, 2008) is a manualized parent training intervention that has received substantial empirical support in the treatment of disruptive behavior across multiple reviews (e.g., Brestan & Eyberg, 1998; Gallagher, 2003; Thomas & Zimmer-Gembeck, 2007). PCIT draws heavily from established parenting, social learning, and attachment theories and proceeds along two successive phases: Child-Directed Interaction (CDI) and Parent-Directed Interaction (PDI). In CDI, goals include strengthening the parent-child relationship by coaching parents in their use of nondirective play therapy skills designed to support social interaction and increase positive parent behaviors, such as praise. The PDI phase builds on CDI skills with the introduction of unambiguous parental commands, limit-setting, and consistent follow-through with consequences (e.g., timeout) to increase compliance and decrease disruptive behavior.

A wealth of research has established empirical support for the efficacy of PCIT in children with disruptive behavior disorders immediately following treatment and at follow-up (e.g., Hood & Eyberg, 2003; Nixon, Sweeney, Erickson, & Touyz, 2004; Schuhmann, Foote, Eyberg, Boggs, & Algina, 1998). Among other changes, PCIT has been documented to result in substantial increases in positive parental attention and child compliance and decreases in children’s externalizing behavior. In PCIT efficacy studies, attrition rates have typically ranged from 27 to 47% (Bagner & Eyberg, 2007; Boggs et al., 2004; Fernandez & Eyberg, 2009; Schuhmann et al., 1998; Werba, Eyberg, Boggs, & Algina, 2006). Data from other community-based applications of PCIT (Chaffin et al., 2009; McNeil, 2007; Phillips, Morgan, Cawthorn, & Barnett, 2008; Timmer, Urquiza, Zebell, & McGrath, 2005) suggest wide variability in rates of attrition, from 12% to 77%.

Research has also provided some support for the use of PCIT with ethnic minority populations (e.g., Capage, Bennett, & McNeil, 2001; Chaffin et al., 2004; McCabe & Yeh, 2009). Despite this progress, a majority of PCIT research with ethnic minority clients has been conducted within controlled clinical trials rather than in service clinics, and one study was retrospective. Due to insufficient evidence, Eyberg (2005) has concluded that PCIT cannot yet be identified as an empirically-supported treatment for African American children. For the current study, we used the standard PCIT protocol, which allows for individualized tailoring to the needs of each client. Further investigation remains necessary before PCIT can be recognized as an empirically-supported treatment across ethnic minority groups.

In a review of youth psychotherapy outcome research, Weisz et al. (2005) identified striking differences between the existing research base and characteristics of typical clinical practice with regard to (1) enrollment of samples, (2) service providers, and (3) settings in which treatment was provided. Although intervention studies for conduct problems fared slightly better than those for some other categories (e.g., anxiety, ADHD), only 2% of the conduct problems studies reported representative characteristics across all three dimensions, suggesting distressingly low representativeness. Most often, studies of interventions for conduct problems included youth who were recruited rather than independently treatment-seeking, were treated by researchers or graduate students, and occurred in controlled research settings. Although, as cited above, PCIT has received strong empirical support regarding its efficacy, the majority of the outcome research on PCIT has been conducted with predominantly Caucasian families seen at university or hospital-based clinics (Gallagher, 2003).

The current research strives to bridge the gap between the existing PCIT research base and the settings in which ethnic minority youth are likely to receive services by assessing a PCIT program established in a community mental health center (CMHC). Our program represents a useful transition point between the highly controlled research settings that are typical of PCIT research and real world service settings. The program satisfies Weisz et al.’s (2005) criteria for clinical representativeness with regard to participant enrollment and treatment setting because it provides intervention to treatment-seeking, clinic-referred youth in a clinical service setting. However, unlike typical clinical treatment providers, who usually consist of practicing clinicians (or even paraprofessionals), our service providers included the study’s principal investigator and graduate students in a clinical psychology doctoral program, a characteristic more common to the existing evidence base.

In addition to the dimensions identified by Weisz and his colleagues, the clinical characteristics of children included in PCIT research also warrant attention. Eligibility criteria in the majority of PCIT studies have included a diagnosed disruptive behavior disorder and/or clinical elevations on standardized behavioral measures (e.g., Bagner & Eyberg, 2007; Boggs et al., 2004; Eisenstadt, Eyberg, McNeil, Newcomb, & Funderburk, 1993; Funderburk et al., 1998). In contrast, community treatment settings commonly provide services to individuals who do not necessarily meet diagnostic criteria for major DSM diagnoses but are nevertheless presenting with problems substantial enough to result in clinical referral (Snowden, Storey, & Clancey, 1989). The effectiveness of and barriers to evidence-based treatments such as PCIT within community-based populations become important questions as the field moves increasingly toward dissemination of empirically-derived “best practices” (Weisz, Jensen, & McLeod, 2005).

The aim of the current study was to conduct a detailed case analysis of treatment effectiveness, engagement, treatment acceptability, and barriers to successful completion in a pilot community-based application of PCIT serving a predominantly low-SES, urban, ethnic minority population. The article reports on families seen in the first 30 months (July 2005 to February 2008) of the program’s operation. Due to the small size of the pilot sample, we used descriptive rather than statistical analysis to examine variables that may relate to success of PCIT in a CMHC. Client engagement in treatment is conceptualized using Nock and Ferriter’s (2005) descriptions of attendance and adherence in parent training studies. Especially in populations who are at high risk for low treatment engagement, research investigating the role of treatment attrition, attendance, and adherence is essential.

Method

Participants

Study participants included 14 children and their caregivers, who sought treatment at an urban CMHC, were eligible for PCIT based on their clinical presentations, completed initial assessment, and provided research consent. (Three additional potential study participants did not complete initial assessment, and a fourth family did not provide research consent.) The 14 families were referred for treatment from multiple sources including schools and external agencies, CMHC outreach workers, and, in 4 cases, self-referral. Participating caregivers consisted of 12 single mothers (or, in one case, a grandmother), 1 single father, and 1 married couple. In addition, one father who lived separately from the mother and child began treatment but left soon after the initial assessment session. Due to lack of research consent, his data were not included in our analyses. Two of the single-parent participants were foster parents, who each had been caregivers for their child for over a year and were in the process of adopting them.

Study children were 64% male with a mean age of 3.7 years (SD = 1.4; range: 2–7). Child ethnicity was 50% African American, 29% Multiracial (1 Latino and Caucasian, 3 Latino and African American), and 21% Latino. In the case of the two foster children, ethnicity of parent (Caucasian) differed from ethnicity of child (1 African American and 1 Latino and African American). Primary child diagnoses were Oppositional Defiant Disorder (ODD; 36%); Attention-Deficit Hyperactivity Disorder (ADHD), Combined Type (29%); Disruptive Behavior Disorder Not Otherwise Specified (DBD NOS; 21%); ADHD, Primarily Inattentive Type (7%); and combined DBD NOS and Autistic Disorder (7%). Although specific data on parental income were not available, 79% of clients received public assistance (e.g., Medicaid) and 14% were charged a reduced fee for treatment, based on their financial status. On the basis of their ethnic and socioeconomic characteristics, participating youth were from demographic groups less likely to receive high-quality mental health services. Of the 14 families, 2 only attended assessment sessions before discontinuing). Therefore, the data described below include only the 12 families who received at least one treatment session.

Setting

The CMHC in which the study was conducted has been offering a range of preventive and mental health services for high-risk youth and their families for over 30 years. Services focus on families with limited means (i.e., on public aid, non-insured, or under-insured). In 2007, 85% of clientele were non-white and 84% were on public assistance. Unlike most CMHCs, which are free-standing service centers, the CMHC in this research is located on a university campus and serves as a primary training clinic for doctoral clinical psychology students. In addition to 27 part- and full-time staff, 15–20 student therapists provide services.

For the majority of families (9 of 12), sessions were held in two CMHC therapy rooms. The rooms were equipped with living room furniture, a round table, and chairs. Both were connected by an observation room with one-way mirrors for monitoring the rooms. One room served as the primary therapy room for meetings with families and observations of parent-child interactions. Following the beginning of PDI, the second room functioned as the timeout room with slight modifications to remove potentially dangerous or easily damaged items (e.g., lamps, low-hanging pictures, easily movable chairs). For the other three families, sessions were held with a similar configuration (including a room with a one-way mirror but no living room furniture) at a local daycare center (their referring agency).

Measures

Child and Parent Functioning

Child Behavior Checklist (CBCL; Achenbach & Rescorla, 2000; 2001)

The CBCL is a caregiver-report form designed to measure the severity of behavioral and emotional symptoms in children. Two versions were used, the CBCL 1½–5 and the CBCL 6–18, to cover the age range of clients (2–7 years). The measures contain 99 and 112 items, respectively, and yield Internalizing, Externalizing, and Total Problems scale scores. Inter-item consistency for these three subscales is strong for both the CBCL 1½–5 (α=.89, .92, .95; Achenbach & Rescorla, 2000) and the CBCL 6–18 (α=.91, .92, .94; Achenbach & Rescorla, 2001). In addition, recent research has supported the construct validity and use of the CBCL 1½–5 scales with low-income African American and Latino parents, especially for the externalizing scale (Gross et al., 2006).

Eyberg Child Behavior Inventory (ECBI; Eyberg & Pincus, 1999)

The ECBI is a 36-item disruptive behavior rating instrument completed by children’s caregivers. The ECBI Intensity Scale measures the frequency of disruptive behaviors on a 7-point scale and the Problem Scale measures whether or not parents find those behaviors problematic. Intensity Scale scores of ≥132 are above the clinical cutoff (Eyberg & Pincus, 1999). In their recent assessment of the ECBI’s utility in a sample of low- and middle-income African American and Latino parents, Gross and colleagues (2007) found inter-item correlations ranging from α=.92 to .95 for the Intensity Scale and from α=.90 to .94 for the Problem Scale.

Parenting Stress Index-Short Form (PSI-SF; Abidin, 1995)

The PSI-SF is a 36-item parent-report measure of strain experienced in parenting responsibilities and interactions with children. The measure contains three subscales: Parental Distress, Parent-Child Dysfunctional Interaction, and Difficult Child, as well as a Total Stress score. In an investigation of the psychometrics and utility of the PSI-SF in a low-SES, African American sample, Reitman, Currier, and Stickle (2002) found good inter-item consistency within each subscale and evidence for the construct validity of the three factor solution described above.

Dyadic Parent-Child Interaction Coding System III (DPICS III; Eyberg, Nelson, Duke, & Boggs, 2005)

The DPICS is an observational method of measuring the content and quality of parent-child interactions during a series of standardized situations. Initial assessment and post-treatment assessment situations consist of child-led play, parent-led play, and clean-up, which are presented in increasing order of parental direction and control. The initial and post-treatment DPICS assessments last 25 minutes, including a 5-minute warm-up period for child-led play and parent-led play before 5 minutes of coding and ending with 5 minutes of clean-up. In addition to assessing parent-child interactions in the initial and post-treatment assessments, we conducted 5-minute assessments of the parent and child in play at the beginning of most CDI and PDI treatment sessions in order to assess parental mastery of PCIT skills. All assessment observations were videotaped and audiotaped to allow for later DPICS coding.

The DPICS system provides standard definitions for coding parent verbalizations and child responses to commands in real time. Parent verbalizations consist of the following categories: Praise, including both specific (Labeled) and general (Unlabeled) positive evaluations of child behavior; Reflections, involving the repetition of a child’s speech or paraphrasing its meaning; Behavior Descriptions, where parents describe a child’s current activities; Criticisms, which include correcting, disapproving, or sarcastic speech directed toward the child; Direct Commands, consisting of specific statements for the child to do something; Indirect Commands, consisting of suggestions or requests for the child to do something; Questions, which ask for information or add a “question” tag to a comment; and Neutral Talk, which includes any other verbalization that does not fit one of the previous categories. The child’s response to parent commands is coded as compliance or noncompliance to each command for which there was an opportunity to comply.

Graduate and undergraduate psychology students coded DPICS assessments. They received extensive training in the system, including reading the DPICS manual, reviewing definitions in didactic sessions, completing homework exercises, practicing coding with experienced coders, and thereafter meeting regularly to review discrepancies and reduce observer drift. An undergraduate student first transcribed video segments into an Excel spreadsheet by watching the videotape and listening to the audiotape. A second student then rechecked the transcripts for accuracy. Next, one observer coded parent and child behaviors using the DPICS system. A second observer independently coded 75% of segments to assess interobserver agreement. Kappa reliabilities for individual behaviors were .77 (negative talk), .88 (direct commands), .77 (indirect commands), .86 (labeled praise), .84 (unlabeled praise), .92 (questions), .80 (reflections), .83 (behavior descriptions), .81 (neutral talk), .68 (child compliance), .56 (child noncompliance), and .63 (no opportunity for compliance). According to Landis and Koch (1977), Kappa values between .41 and .60 are considered moderate, between .61 and .80 substantial, and above .81 “almost perfect.”

Attendance and Adherence

Session attendance

Parents’ attendance at scheduled treatment sessions served as one measure of client engagement and treatment dose. Nock and Ferriter (2005) defined treatment attendance as “delivery of the agreed upon treatment participants … to the treatment setting for scheduled appointment” (p. 151). In our study, treatment sessions were considered scheduled if the meeting was arranged during the previous session or if they were scheduled between sessions by phone. Non-treatment sessions (e.g., those devoted primarily to assessments or scheduling issues) were not included in our counts. Typically, client attendance at the next session was confirmed at the conclusion of each week’s meeting. Client cancellations (by telephone) and no-shows were both considered nonattendance but were tracked separately.

Homework completion

Homework completion provided a measure of parental treatment adherence outside of the therapy sessions. As a standard part of PCIT, parents are requested to complete daily 5-minute practice sessions during CDI and an additional 10-minute practice session during PDI (15 minutes total) in order to solidify PCIT skills and promote generalization of the skills. Parent completed homework sheets from the treatment manual (Eyberg & Child Study Lab, 1999), on which they recorded the date, activity, and any problems or questions that arose. We calculated homework completion by tallying the number of days of homework completed per week of treatment. CDI homework occurred during both the CDI and PDI phases, whereas PDI homework occurred only during PDI. Therefore, we calculated separate tallies for CDI and PDI homework.

Dropout

We defined dropout as occurring when a caregiver explicitly told the therapist that s/he wished to end treatment or quit coming to all further scheduled parent training sessions and failed to return calls despite 6 or more weeks of repeated, weekly documented staff efforts to re-contact and re-engage the parent by phone or mail.

Satisfaction and Barriers

Therapy Attitude Inventory (TAI; Eyberg, 1993)

The TAI is a 10-item parent-report measure that assesses satisfaction with the therapy process and its outcomes. Parents rate items on a 1 to 5 scale, with higher ratings indicating greater satisfaction. Items assess constructs including parents’ confidence in carrying out discipline and satisfaction with the parent training intervention. Previous research with the TAI has yielded high internal consistency and 4-month test-retest reliability (Brestan, Jacobs, Rayfield, & Eyberg, 1999). TAI data were available for 10 of the 12 families who attended at least one treatment session (3 completers and 7 noncompleters).

Barriers to Treatment Participation Scale (BTPS; Kazdin, Holland, Crowley, & Breton, 1997)

The BTPS is a 58-item measure parents completed at the end of treatment regarding psychological and practical barriers to client engagement. The first 44 items list potential barriers to treatment (e.g., “Treatment did not seem necessary,” “I felt that treatment cost too much”) and are rated on a 5-point scale (1 = never a problem, 5 = very often a problem). Consistent with the practices of the measure authors (Kazdin et al., 1997), we used a total barriers score in the current study. Kazdin and colleagues (1997) reported acceptable levels of inter-item consistency for total barriers. Items 45–58 list 14 critical events (e.g., “I lost my job or had a change in income”) that might affect treatment participation. Respondents indicate which of the events (if any) they experienced while receiving treatment. The total number of critical events is tallied separately from the total barriers score. BTPS administration did not begin until the PCIT program was already underway, and one treatment dropout family could not be reached for a premature termination session. Consequently, only 10 of the 12 families (4 completers and 6 noncompleters) provided BTPS data.

Procedure

Initial and Post-Treatment Assessment

Following intake, families who were identified as potential candidates for PCIT attended two initial assessment sessions consisting of a clinical interview, DPICS observation, and administration of the child and parent functioning measures. In addition, the therapists provided an overview of PCIT and the research study and reviewed the informed consent form.

For families who completed treatment, post-treatment assessment (DPICS and the parent and child functioning measures) typically occurred 1–2 weeks following the session in which families met mastery criteria. Additionally, families completed the BTPS and TAI. For families who discontinued treatment prematurely, every effort was made to conduct an “early termination” session in which the therapists solicited qualitative feedback about treatment and administered a subset of the measures (ECBI, TAI, BTPS). Termination sessions were offered free of charge at either the PCIT clinic or community locations (e.g., daycare center).

Parent-Child Interaction Therapy (PCIT)

Intervention followed the protocol detailed in the PCIT treatment manual (Eyberg & Child Study Lab, 1999) and PCIT treatment literature and is therefore only described briefly below. PCIT therapists (four female, three male) included six doctoral clinical psychology students and one faculty supervisor. Five therapists were Caucasian and two were Asian-American. All therapists had specialized training in PCIT, and three had completed a one-week training workshop with the founder of the PCIT model. Treatment sessions were audiotaped, typically lasted 90 minutes, involved initial check-in periods during which homework and other pressing issues were discussed, and concluded with feedback, planning, and a homework assignment for the following week.

PCIT treatment proceeded along two phases, Child-Directed Interaction (CDI) and Parent-Directed Interaction (PDI), the combination of which is designed to strengthen parent-child relationships and increase children’s compliance and prosocial behavior. The goals of CDI and PDI are described in the introduction. Early sessions focused on teaching skills and later sessions focused on use of the skills in everyday situations and problem-solving behavior management issues outside the clinic. Following an initial check-in and review of homework, CDI and PDI coaching sessions typically began with 5-minute DPICS observations in order to assess parental mastery of PCIT skills. We used the standard mastery criteria for CDI and PDI (Eyberg & Child Study Lab, 1999) to determine parents’ readiness to move on to PDI and graduation, respectively. In addition, the protocol indicates that treatment continues until parents’ ECBI scores have dropped to within one-half of a standard deviation of the normative mean (≤114) and the client and therapist agree that the parent can effectively manage the child’s behavior (Eyberg & Child Study Lab, 1999). For treatment completers, the average number of sessions to graduation was 14.0 (SD = 1.8). For noncompleters, the average number of sessions to termination was 6.4 (SD = 4.9).

Treatment Integrity

We operationalized treatment integrity as adherence to the PCIT manual, using the PCIT fidelity checklists (Eyberg & Child Study Lab., 1999). A trained observer reviewed a random sample of 25% of each family’s audiotaped sessions for integrity. The observer coded each checklist item as present, absent, or not applicable. We randomly selected 10% of these coded sessions for a reliability check by an independent coder. Integrity checks yielded 91% adherence to the PCIT protocol with 89% interrater reliability. Although our level of adherence was slightly below that of some PCIT efficacy studies, which often report levels near 97% (e.g., Bagner & Eyberg, 2007; Werba et al., 2006), it is comparable to applications with more complex cases, such as clients with internalizing and externalizing comorbidities (90%; Chase & Eyberg, 2008).

Reduction of Treatment Barriers

Additional efforts were made by PCIT therapists in order to reduce barriers to treatment. CMHC-wide practices included access to free transit cards for public transportation to sessions and scheduling sessions in the evening if requested. PCIT therapists maintained a liberal absence policy (e.g., continuing to see cases even when families had frequent cancellations or no shows, as long as they did not meet dropout criteria listed above, and allowing families to take a break of several weeks from treatment if needed to deal with personal issues), and they responded individually on an ad hoc basis to treatment barriers (e.g., writing a caregiver’s employer to request a regular evening off so the family could attend therapy sessions, meeting with a family after repeated absences to discuss treatment barriers and problem-solve solutions). For five families referred from the same local daycare center, PCIT therapists offered to conduct sessions at the daycare site rather than the CMHC (three families accepted this arrangement). Except for scheduling sessions in the evening (used by two treatment completers) and allowing a break of several sessions from treatment (used by one treatment completer), the other practices were implemented only with dropout families, because attendance was not an issue.

Results

Of the 12 families who completed at least one treatment session, 4 completed treatment successfully and 8 left treatment prior to meeting mastery criteria (treatment dropouts). Three of the eight noncompleters attended at least one PDI treatment session (PDI Teach or beyond) before discontinuing. Completers reached PDI Teach in an average of 6 sessions. Dropouts who reached PDI did so in an average of 8 sessions. Furthermore, three of the four treatment completers were self-referred, compared to one of the eight dropouts.

Table 1 presents data for treatment completers and treatment dropouts on all self-report measures at initial assessment and, when applicable, post-treatment assessment. For one family in which two parents were consistently involved in treatment, mean scores were used for each variable. Between assessments, treatment completers demonstrated decreases of varying magnitudes on all self-report scales and subscales. For dropouts, ECBI Intensity and Problem scales showed a milder decreasing trend from initial assessment to final assessment. Post-treatment PSI-SF and CBCL scores were unavailable for treatment dropouts.

Table 1.

Mean Level (and SD) on Self-Report Measures at Initial and Final Available Assessment for Treatment Completers and Dropouts

	Completers (n = 4)		Dropouts (n = 8)
	Initial Assessment	Final Assessment	Initial Assessment	Final Assessment
CBCL Internalizing	56.75 (11.79)	46.00 (3.37)	53.00 (11.34)	--
CBCL Externalizing	62.38 (4.27)	52.63 (4.27)	66.50 (15.91)	--
CBCL Total	60.75 (8.42)	49.13 (6.59)	62.13 (10.80)	--
PSI-SF Parental Distress	29.13 (5.75)	25.50 (10.47)	30.38 (9.88)	--
PSI-SF Parent-Child Dysfunctional Interaction	20.75 (6.45)	16.13 (2.53)	22.38 (4.93)	--
PSI-SF Difficult Child	36.75 (4.03)	25.38 (4.64)	32.50 (10.50)	--
PSI-SF Total Stress	86.63 (15.49)	66.88 (17.63)	85.25 (23.83)	--
ECBI Intensity	135.75 (4.57)	90.00 (23.34)	138.44 (40.28)	116.88 (41.55)
ECBI Problem	10.63 (3.82)	3.50 (3.11)	17.50 (7.21)	6.25 (7.50)

Note. Dashed line indicates data are not available.

Figure 1 displays ECBI Intensity scores over time for both treatment completers and treatment dropouts. On average, completers demonstrated a relatively steady downward trend, finishing well below the completion criterion of 114. In contrast, dropouts as a group showed less overall change, with four of eight families ending treatment below the 114 criterion.

Figure 1.

Mean ECBI Intensity Scores for Treatment Completers and Dropouts across Sessions. When a family completed treatment or dropped out, the last available score was carried forward for use in calculating mean scores for all subsequent sessions. The final session mean score represents the post-treatment assessment for treatment completers.

In order to determine whether or not decreases on the ECBI Intensity scale were clinically significant and meaningful in the extent to which they represented the client’s return to a “normal” level of functioning, we utilized the method described by Jacobson, Roberts, Berns, and McGlinchey (1999). In their method, clinical significance on any given measure is established based on two criteria. First, the change must be statistically reliable, as determined by calculation of the reliable change index (RCI; Jacobson, Follette, & Revenstorf, 1984). The RCI is calculated by dividing the difference between pre-treatment and post-treatment scores by the standard error of the difference between the scores (Jacobson & Traux, 1991). RCI scores greater than 1.96 are determined to be sufficiently large to be statistically reliable (Jacobson et al., 1999). The second criterion states that, at the conclusion of treatment, previously clinically-identified clients should be indistinguishable from a normal population. Only clients whose initial ECBI scores were in the clinical range and post-treatment scores were within one standard deviation of the normative mean were considered to have met this criterion. Three treatment completers and five treatment dropouts had pre-treatment ECBI Intensity scores in the clinical range (≥132). Based on the Jacobson et al. (1999) criteria, two of the three eligible treatment completers demonstrated reliable, clinically significant gains over the course of the PCIT intervention. In addition, another treatment completer, whose pre-treatment score was one point below the clinical cutoff, also demonstrated statistically reliable change. Using the final available treatment score, two of the five eligible treatment dropouts demonstrated similar reliable and significant change.

Consistent with some prior PCIT research (e.g., Bagner & Eyberg, 2007; Chaffin et al., 2004; Pearl, 2008), the DPICS categories in the current study were condensed into the positive “Do” Skills (Praise, Reflections, and Behavior Descriptions) and negative “Don’t” Skills (Criticisms, Questions; and Commands) in order to assess parental behavior change during the child-led play. Figures 2 and 3 display the mean number of Do Skills and Don’t Skills, respectively, per 5-minute observations for treatment completers and treatment dropouts in child-led play during initial assessment and, for completers, post-treatment assessment. The figures also display the mean number of skills for each group across 5-minute CDI coding observations at the beginning of most treatment sessions. Only dropouts for whom at least one CDI treatment observation was available were included in the graphs. Data were combined based on the number of times families were observed in CDI rather than by session number. This was done because families proceeded through treatment at different rates and because some sessions (e.g., Teach sessions and PDI Coach 1) did not include observations. Graphs were created with the last available score carried forward for families who completed or dropped out of treatment before all clients ended PCIT. Completers had an average of 7.25 CDI coding observations (range = 5 – 10), and dropouts had an average of 9.00 (range = 2 – 10).

Figure 2.

Mean Number of Do Skills (Praise, Reflections, Behavioral Descriptions) during Five-Minute Play Observations. When a family completed treatment or dropped out, the last available score was carried forward for use in calculating mean scores for all subsequent sessions. The bars represent skills in child-led play during initial and (for treatment completers) post-treatment assessments, and the boxes and diamonds represent skills during CDI coding in treatment sessions. Only dropouts for whom at least one CDI treatment observation was available were included in the graphs. Data were combined based on the number of times families were observed in CDI rather than by session number.

Figure 3.

Mean Number of Don’t Skills (Commands, Questions, Criticisms) during Five-Minute Play Observations. When a family completed treatment or dropped out, the last available score was carried forward for use in calculating mean scores for all subsequent sessions. The bars represent skills in child-led play during initial and (for treatment completers) post-treatment assessments, and the boxes and diamonds represent skills during CDI coding in treatment sessions. Only dropouts for whom at least one CDI treatment observation was available were included in the graphs. Data were combined based on the number of times families were observed in CDI rather than by session number.

Figure 2 shows that, at initial assessment, both groups displayed low mean levels of Do Skills in child-led play. By post-treatment, completers evidenced a threefold increase in Do Skills. Likewise, Figure 3 indicates that mean numbers of Don’t Skills were elevated for both groups at the initial assessment, although completers as a group appeared to exhibit higher initial levels of Don’t Skills relative to dropouts. Over the course of treatment, completers showed a large reduction in Don’t Skills. The session-by-session data from treatment observations indicate that both completers and dropouts increased their use of Do Skills and decreased their use of Don’t Skills substantially across sessions. Nevertheless, completers appeared to demonstrate more immediate and larger improvements than dropouts on both variables.

Table 2 displays treatment attendance and adherence data. Total treatment session attendance ranged from 12 to 15 for completers (M = 13.5) and from 1 to 14 for noncompleters (M = 6.4). In addition, trends were observed in which dropouts cancelled more sessions, had higher “no show” rates, and completed less homework during CDI than treatment completers. During PDI, the two dropout families who remained in treatment demonstrated higher homework completion than the treatment completers.

Table 2.

Attendance and Homework Adherence for Treatment Completers and Dropouts

	Completers (n = 4)		Dropouts (n = 8)
	Mean Total (SD)	Percentage (SD) of Total Scheduled/Assigned	Mean Total (SD)	Percentage (SD) of Total Scheduled/Assigned
Treatment Sessions Attended	13.5 (1.3)	90.1% (7.9)	6.4 (4.9)	47.7% (17.3)
Treatment Sessions Cancelled	1.5 (1.7)	8.6% (9.9)	4.8 (3.5)	39.6% (19.4)
Treatment Session No-Shows	0.25 (0.5)	1.8% (3.6)	1.75 (2.2)	12.7% (15.4)
CDI Homework Completed		62.7% (9.9)		47.4%a (21.9)
PDI Homework Completed		39.1% (17.2)		59.5%b (16.8)

^an = 7

^bn = 2

Table 3 displays findings on treatment satisfaction and barriers at the end of treatment. The TAI generally indicated a high degree of satisfaction with treatment. Among the available reports, treatment completers indicated a somewhat higher mean level of satisfaction than treatment dropouts. On the BTPS, the total barriers scores ranged from 53.0 to 68.0 for treatment completers and from 57.0 to 100.0 for treatment dropouts. The mean level of reported barriers to treatment was higher for treatment dropouts than treatment completers; conversely, treatment completers endorsed a higher number of critical events than treatment dropouts. As can be seen in Table 4, completers and dropouts reported very similar types of critical events, but completers endorsed them at higher rates than dropouts. For both groups, the most commonly endorsed critical event related to experiencing the illness or death of a close friend or relative.

Table 3.

Treatment Satisfaction and Barriers at the End of Treatment

	Completers	Dropouts
Treatment Satisfaction	47.00 a (3.00)	39.62 c (5.79)
Total Barriers to Treatment	62.75 b (6.65)	74.17 d (15.45)
Critical Events	1.25 b (1.50)	.67 d (.82)

^an = 3

^bn = 4

^cn = 7

^dn = 6

Table 4.

Barriers to Treatment Participation Scale – Critical Events Endorsed

Item	Percentage of participants endorsing item
	Completers (n = 4)	Dropouts (n = 6)
My medical insurance did not cover this treatment.	25%	17%
I lost my job or had a change in income.	25%	0%
I got a job or changed jobs.	25%	17%
A close friend or relative got very sick or died during treatment.	50%	33%

Discussion

The current pilot study yielded mixed findings surrounding effectiveness and engagement in PCIT services delivered to urban, low SES, ethnic minority children and their parents. Compared to the bulk of the existing PCIT research, clients sought treatment in a community mental health center that was more representative of clinical practice in terms of sample enrollment and treatment setting (Weisz et al., 2005). Nevertheless, despite individualized efforts to reduce treatment barriers and enhance engagement, dropout was high, a finding consistent with prior research in community settings. Results suggest that completers demonstrated clinically significant change on measures of parent and child behavior and that treatment dropouts demonstrated more attenuated improvements. Below, the findings are discussed in the context of client attrition and treatment engagement data (e.g., attendance, adherence, satisfaction, and barriers).

Consistent with prior PCIT research, families who completed treatment in the current study reported improvements in their children’s externalizing behavior on multiple measures. In addition, we assessed the clinical significance of client change on the ECBI Intensity scale and found that 50% of the completers both met the initial criteria for clinical elevations and demonstrated clinically significant and reliable change on the measure. A third family began one point below the clinical cut-off but otherwise met these change criteria. The lack of clinically significant and reliable change for all treatment completers was likely influenced by the fact that our sample was not screened for study inclusion on the basis of diagnosis or ECBI scores. Completers scored an average of 135.75 at pre-treatment, whereas samples from efficacy studies of PCIT for externalizing behavior have typically ranged from 156.40 to 178.38 (Bagner & Eyberg, 2007; Eyberg et al., 2001; Schumann et al., 1998).

Treatment completers also demonstrated noteworthy changes in use of PCIT skills during observations of parent-child interactions. Compared to initial assessment, at post-treatment assessment, parents evidenced increased positive attention skills (Do Skills) and decreased commands, criticisms, and questions (Don’t Skills) during child-led play. For completers, the mean levels of Do Skills and Don’t Skills at the two assessment points are similar to those reported for treatment samples in other PCIT research (e.g., Bagner & Eyberg, 2007). A unique aspect of this study was our examination of changes in parents’ skills during the course of treatment, as reflected in their performance of Do Skills and Don’t Skills in 5-minute CDI coding observations at the beginning of coaching sessions. Three patterns are worth noting. First, relative to initial assessment, both treatment completers and dropouts demonstrated an immediate jump in the mean number of Do Skills and a drop in the mean number of Don’t Skills in the first CDI coding session (i.e., after CDI Teach), and these mean levels continued to increase across sessions. This pattern suggests that both groups of parents improved their positive attention skills during PCIT and indicates that some treatment changes manifested themselves very quickly. As expected, changes in ECBI scores progressed at a much slower rate. Unlike observations of parents’ behavior, ECBI scores in PCIT are theorized to change as a direct result of the impact of parent behavior change on child behavior which, in turn, influences parental perceptions of externalizing problems. Second, as expected, parents who completed treatment showed, on average, better performance on both Do Skills and Don’t Skills during treatment sessions than dropouts. Third, parents who completed treatment used more Do Skills during CDI coding sessions in the latter half of treatment than they demonstrated at post-treatment assessment. This pattern also was expected, in that the mastery criteria for PCIT are designed to help parents “overlearn” the skills by boosting them to a level beyond what is typical in everyday interactions. Further, the structure and parental instructions used at initial and post-treatment assessment are intentionally different than for the 5-minute CDI coding observations in order to assess parents’ generalization of skills. Finally, at post-treatment assessment, parents were aware that they had met mastery criteria and would soon be completing treatment. This knowledge might have impacted parental motivation to demonstrate a high level of skill mastery.

Despite the gains described above for treatment completers, the relatively high rate of dropout underscores the difficulties inherent in providing services to this population. Using standard PCIT completion criteria, our 67% treatment dropout rate is above the rates reported for outpatient child psychotherapy, which often range from 40 to 60% (Kazdin, 1996), and it exceeds rates commonly reported in controlled PCIT research. Nevertheless, our dropout is consistent with some other studies of youth psychotherapy. For example, in a prospective study of client dropout from behavioral management training for adolescents with ADHD, Friars and Mellor (2007) reported a 67% dropout rate. In their study, dropout was related to parental perceptions of problem severity and intervention strategy utility. In addition, recent work by Fernandez and Eyberg (2009) identified client socioeconomic status as the single best predictor of PCIT attrition. The low-SES nature of our sample likely placed the clients in the current study at heightened risk for dropout.

Another factor that may have played a role in predicting dropout relates to referral context. Pre-treatment differences were observed regarding client referral source, with a higher percentage of completers (75%) than dropouts (13%) self-referring to treatment. Although previous findings from research investigating the link between referral source and dropout from child therapy have been mixed (Kazdin, 1996), our results suggest a possible relationship in our sample. Perhaps clients from underserved populations who self-refer are likely to be motivated and invested in the treatment process and consequently complete treatment at higher rates. By contrast, three of the eight clinically referred families were mandated to participate in PCIT treatment as a condition for allowing their children to remain in a daycare program, and all three of these families dropped out of treatment. Given our small sample, no definitive statements about the role of referral context can be made, but the findings point to areas to be investigated in further research.

Regarding direct measures of client treatment engagement, completers appeared to demonstrate higher mean levels of attendance and fewer cancellations relative to dropouts. Further, trends emerged in which homework completion was higher for completers than dropouts during CDI but not during PDI. However, the small number of dropouts who continued into the PDI phase (n = 2) makes interpretation difficult. Treatment completers also showed a trend toward being more satisfied with therapy than dropouts, a finding consistent with prior PCIT research (Boggs et al., 2004, Brestan et al., 1999). Thehigh level of satisfaction endorsed by completers was generally equivalent to that reported in other PCIT studies (e.g., Bagner & Eyberg, 2007; Brestan et al., 1999; Matos, Torres, Santiago, Jurado, & Rodríguez, 2006).

Treatment satisfaction may be influential among ethnic and cultural minority families, whose backgrounds and values often do not match those of the treatment originators or service providers, and who sometimes report more negative expectations for treatment (Richardson, 2001). Research on cognitive match (i.e., assumptions, expectations, and attitudes about therapy) between therapists and clients has suggested that this variable might help to explain the differential effects observed in other studies (Zane et al., 2005). Additionally, premature termination from treatment for child conduct problems has been linked to incongruities between parents’ conceptualizations of the problem and the treatment provided (Miller & Prinz, 2003). Anecdotal reports from PCIT therapists in the current study identified some of these barriers. For example, one participant, who was African American, explained that she believed PCIT represented a “white” parenting method. In addition, the mother of a Latina parent, with whom the parent and her son lived, appeared to disapprove of the therapy and was unwilling to become involved. Although therapists worked to validate parents’ feelings about the cultural relevance of PCIT and to fit PCIT skills with their existing parenting styles, both of the parents referenced above eventually dropped out of treatment.

The current sample of urban, ethnic minority children and their families were drawn from a population that commonly experiences low mental health service utilization, often as a result of their overrepresentation within lower socioeconomic status communities (Harrison et al., 2004; Kataoka, Zhang, & Wells, 2002; Owens et al., 2002; Proctor & Kalaker, 2002). In prior research, Kazdin and colleagues (1997) discovered that families dropping out of treatment for child externalizing behavior demonstrated higher total barriers scores than treatment completers. In our sample, a similar trend was discovered in which dropouts scored approximately 11 points higher on the BTPS.

Previous PCIT research with more traditional clinical samples has documented a range of common treatment barriers and reasons for client dropout. The most frequent reasons included clients leaving due to insufficient time for treatment, difficulties with child care, transportation issues, and disaffection with the treatment approach (Boggs et al., 2004; Harwood & Eyberg, 2006). In the current study, both completers and dropouts reported very similar critical events during treatment, the most frequent being the serious illness or death of a close friend or relative, endorsed by two families in each group. One family dropped out of treatment shortly after the shooting death of the identified client’s father. The father had recently completed CDI Teach in order to participate in PCIT with his son and former girlfriend, who were already engaged in treatment. Another client’s caregiver missed multiple sessions during the time her sister entered hospice care and, shortly afterward, discontinued treatment. For treatment completers, the serious illness of a caregiver’s sister and, in another case, a father-in-law, were experienced during PCIT but did not derail treatment. In previous research, Kazdin and colleagues (1997) found that therapist and client perceptions of treatment barriers were only moderately correlated, suggesting that therapists might not be aware of all relevant impediments to engagement. Furthermore, it is possible that, even within the same type of critical event, dropouts in the current study either experienced a more severe or impactful variation of a similar problem or were less able to successfully cope with problems that arose.

Although it is difficult to determine the precise impact of each critical event on client continuation in treatment, our anecdotal reports mirror national data that have established the greater likelihood of community violence and serious health problems among ethnic minority families (Aday, 1994; Bureau of Justice Statistics, 1997). Nevertheless, considering the trends toward a higher rate of critical events, but lower rate of treatment barriers among completers, our data appear to be consistent with Nock and Kazdin’s (2005) prediction that treatment barriers may be more important in predicting dropout than critical events. It may be that critical events result in a temporary disruption of engagement while chronic barriers continue to interfere throughout the treatment process.

In sum, our findings extend previous PCIT outcome research to a more representative community mental health center serving predominantly low-SES, urban, ethnic minority youth. Although treatment dropout levels were high in our sample, those who did complete treatment demonstrated gains across a variety of self-report and observational measures of child functioning and parent-child interactions. Clients who dropped out of treatment prematurely exhibited different trends, relative to completers, on measures of attendance, adherence, and treatment barriers. Nevertheless, some dropouts demonstrated noteworthy treatment gains as well (as measured using established criteria for clinical significance). The high level of dropout in the current study raises the issue of how much treatment is sufficient for families who are difficult to engage. Although this is, in essence, an empirical question, the current results suggest that meaningful gains are possible even for PCIT clients with “incomplete” participation. Whether relaxation of CDI or PDI completion criteria would result in a higher “completion rate” with similar functional outcomes is unknown, but it is a worthwhile direction for further inquiry with community samples, who are frequently less likely to remain engaged in treatment. Simultaneously, our findings exemplify the multiple service barriers common in high-risk populations and underscore the need for continued diligence in the identification of treatment barriers and methods of engagement enhancement for chronically underserved youth. Further, given the finding that all three families who were required to attend PCIT as a condition for allowing their child to remain in daycare dropped out of treatment, the results call into question the wisdom of initiating PCIT with families who do not voluntarily enter treatment, at least in the absence of concerted efforts at motivational enhancement prior to beginning PCIT (Chaffin et al., 2009). Indeed, researchers studying difficult-to-engage clients and families are increasingly turning to targeted interventions and techniques designed to bring about greater participation in treatment (e.g., Chaffin et al., 2009; McKay et al., 2004; Nock & Ferriter, 2005; Watt & Dadds, 2007). Although the integration of engagement enhancement interventions into CMHCs has proceeded slowly, increased efforts to directly address treatment barriers, such as those identified above, are clearly warranted in the context of EBP service delivery. For example, indicated pre-treatment engagement strategies might include phone-based identification and planning surrounding barriers to participation (McKay et al., 2004).

Limitations of the current pilot study include a small sample size and the lack of experimental control. Further, our comparisons of treatment completers and dropouts across sessions by carrying forward final scores (in the figures presented) imposed an assumption of stability that may not reflect what would have occurred had families continued to be available for assessment. In addition, because our study did not include a waitlist control or random assignment to treatment conditions, we cannot rule out alternative reasons (i.e., maturation) for the positive changes exhibited by completers. Nevertheless, based on findings that have documented higher levels of dropout from waitlist than intervention groups (e.g., Werba et al., 2006), the inclusion of a waitlist control might have resulted in even higher attrition from our study.

In conclusion, this pilot study provided some evidence for the effectiveness of unmodified PCIT with an urban, ethnic minority population, but it also identified significant engagement issues and high rates of client dropout. A particular strength of this study was the use of the CMHC setting for mental health service delivery, but this setting likely contributed to increased difficulties with attendance and adherence. Engagement enhancement interventions, such as standardized telephone contact focused on active problem identification and problem solving combined with initial in-person engagement interviews, have been found to increase initial and subsequent attendance at treatment sessions in a sample of urban, ethnic minority children and adolescents (McKay et al., 1998). Nock and Kazdin (2005) also documented significant increases in treatment motivation, attendance, and adherence for parents who received a brief participation enhancement intervention during behavioral training. Similarly, Chaffin and colleagues (2009) recently found that PCIT treatment retention was improved among low- and moderately-motivated parents in the child welfare system who received a group-based motivational intervention. Such findings have clear implications for PCIT in light of data that show significantly better long-term functioning for treatment completers than dropouts (Boggs et al., 2004), and they represent an essential area of continued research.