Factors Influencing Implementation of Evidence-Based Mental Health Interventions for Infants and Young Children

Abstract

Large scale efforts have been made to adopt evidence-based practices (EBPs), such as Parent-Child Interaction Therapy (PCIT) and Child-Parent Psychotherapy (CPP), for infants and young children within community mental health settings. The current study investigated the implementation of PCIT and CPP across agencies serving an ethnically diverse Medicaid population throughout a large urban county. Surveys were completed online by 20 program managers representing 16 birth to five mental health agencies implementing both CPP and PCIT; questions addressed intake and referral processes, training and supervision in EBPs, treatment fidelity, and patient outcomes/satisfaction. Results indicated a wide variety of approaches used to select treatment approaches and cited the intensive EBP training processes, staff turnover, and patient attrition as barriers to sustainability. Implications regarding implementation of EBPs for infants and young children are discussed, including issues related to patient care, training and supervision, treatment fidelity, program sustainability, and barriers to system change.

Serious social-emotional and behavioral disorders in early childhood are estimated to occur in 9% to 14% of the general population and as many as 24% of low-income families.^1–2 Adverse experiences and traumatic stress are among the factors having the largest negative impacts on the mental health of young children.³ Infants and young children can experience a range of mental health problems including anxiety, mood disturbance, traumatic-stress symptoms, disruptive behaviors, parent-child relationship issues, and disturbances in feeding or sleep.⁴

The field of infant and early childhood mental health is a specialized area of practice that requires training in multiple foundational competencies. Mental health interventions for infants and young children are relationship-focused, emphasizing parent-child (or other primary caregiving) relationships within the treatment through dyadic (parent-child), family, and parenting approaches.⁵ Two of the mental health interventions for young children with strong empirical support and broad applications are Child-Parent Psychotherapy (CPP) and Parent-Child Interaction Therapy (PCIT).^6–7

Parent-Child Interaction Therapy and Child-Parent Psychotherapy have several commonalities. They both focus on the parent-child relationship, are primarily dyadic (including both parent and child in sessions), involve child-directed play, and incorporate developmental guidance for parents. Both treatments have an evidence base for treating some of the same mental health issues in early childhood (e.g. children exposed to child abuse; children with disruptive behavior); however, the two treatments are distinct in their theoretical frameworks and their therapeutic approaches.

Child-Parent Psychotherapy targets children ages birth to five years and their caregivers. Originally developed for those exposed to domestic violence, traumatic bereavement, and other types of traumatic experiences, the model has been applied more broadly, including as a preventative intervention for infants and young children who are at risk due to factors such as parental mental health disorders or parent-child relationship issues.^8–9 A relationship-based treatment model, CPP is influenced by psychodynamic and attachment theories, as well as other theoretical orientations that target the child, the parent, and the parent-child relationship.^6,10

The assessment and engagement phase of CPP includes gathering detailed information about both the child’s and the parent’s history of traumatic events and providing psychoeducation to parents (and children depending on age) about the link between traumatic experiences and current symptoms. During the core intervention phase, sessions focus on parent-child interactions, with the therapist making interpretations that: support the parent-child relationship; help them to understand each other’s perspectives; point out the link between traumatic experiences and the symptoms, play themes, and relationship challenges that emerge during sessions; and helping the parent to be a protective presence for the child. For toddlers and preschool age children, toys are specifically selected to elicit play related to the child’s trauma.

Randomized controlled trials of CPP have demonstrated reduction in child behavioral problems and maternal stress and improved parent-child attachment as well as other positive outcomes related to the parent-child relationship.^11–14 Training involves an 18 hour initial didactic training, 18 months of supervised practice including weekly reflective practice with a CPP supervisor, participation in monthly case conferences, 12 hours of follow up competency workshops, and completion of four CPP cases. Therapists complete required reading, present in case conferences, and complete self-reflection fidelity measures^15–16 The fidelity measures are a set of checklists completed by the therapist in collaboration with their supervisor or CPP trainer at various stages of treatment. For example, the Foundational Phase fidelity measures include a 19-item checklist regarding procedural fidelity related to specific therapeutic interventions to be implemented during assessment and engagement; a 12-item checklist regarding procedural fidelity related to the feedback session with the caregiver; four fidelity checklists that address potential sources of challenge and the therapist’s self-assessment of their capacity to meet those challenges in the domains of Reflective Practice, Emotional Process, Dyadic-Relational, and Trauma Framework; a Procedural Fidelity checklist addressing procedural challenges such as scheduling sessions; and a CPP Case Conceptualization and Content Fidelity checklist addressing how the therapist has targeted each of 14 CPP objectives during the course of the foundational stage of treatment. Similar checklists are completed after every 12 sessions of the Core Intervention phase, and after the Termination phase.

Parent-Child Interaction Therapy is designed for children ages two to seven years who frequently engage in disruptive behaviors.¹⁷ Caregivers in PCIT learn positive parenting and behavioral management strategies through parent-only didactic sessions and parent-child in-vivo coaching sessions; the therapist typically observes the parent-child dyad from behind a two-way mirror and coaches the parent via bug-in-the-ear technology.¹⁸ Therapists monitor progress weekly via observational assessments of parent acquisition of parenting strategies (e.g. use of labeled praise and enthusiasm; avoiding questions; ignoring minor misbehavior) and parent-report outcome measures of child behavior; immediate feedback is provided to parents in each session about their progress in utilizing specific parenting strategies.⁷ A meta-analysis of 23 randomized controlled or quasi-experimental trials of PCIT found that PCIT significantly reduced parental stress and child externalizing problems when compared to the control groups; PCIT is considered a best practice for treating trauma-related symptoms such as defiance, aggression, hyperactivity, and anxiety.^18,19 PCIT training involves (1) a 40-hour training, (2) skill review in real time or through video recordings of PCIT competencies, (3) achievement of a minimum of 80% agreement with PCIT trainer on the Dyadic Parent-Child Interaction Coding System, Fourth Edition (DPICS-IV), and (4) completion of two PCIT cases through the full course of treatment while participating in supervision and consultation.

Based upon the promising outcomes for families treated using CPP or PCIT, large scale efforts have been made to adopt trauma-informed systems of care and treatment to address the complex needs of at-risk populations.²⁰ Various models for dissemination and implementation (D-I) for EBPs have been proposed.^21–24 One such model developed by Fixsen and colleagues outlines six Stages of the Implementation Process.²⁰ In the Exploration and Adoption stage, an organization assesses their consumers’ needs, explores treatment options, and selects an EBP to adopt. The Program Installation phase includes tasks such as setting up structural supports for the EBP (e.g., funding, staffing, establishing referral sources) and beginning to treat clients. In the Initial Implementation phase, changes occur at both the individual practitioner (e.g., skill level with the EBP) and organizational (e.g., culture around and capacity to implement the EBP) levels. Organizations move into the Full Operation phase once the program is fully staffed, clinicians carry full client loads, and practitioners are demonstrating proficiency and skill in the model. With time, the Innovation phase includes clinically-indicated adaptations to the EBP to meet the needs of the client become accepted practice while maintaining fidelity to the core components of the EBP. Lastly, in the Sustainability phase, the practice is maintained even when faced with organizational challenges such as changes in staff, leadership, and funding sources.

Little research has been conducted evaluating the translation of D-I models into clinical practice.^25–26 Implications of effective D-I of trauma-focused interventions for young children within a public health framework include issues related to patient care (e.g., client satisfaction, engagement, attrition, match between intervention choice and family needs, culturally-responsive provision of services, and treatment efficacy), funding streams, training and supervision, treatment fidelity, and program sustainability, to name a few.²⁵

In Los Angeles County, the Department of Mental Health (LACDMH) implemented the broad expansion of evidence-based practices (EBPs) utilizing funding derived from the California Mental Health Services Act.^27–29 The implementation of new EBPs presented many challenges to mental health agencies including efficiently training many clinical staff in new models, adapting internship and fellowship training program curricula, addressing logistical considerations, such as tracking of outcome measures, and getting the “buy in” of both clinicians and clients to the new treatment models.²⁸ PCIT and CPP were among the evidenced-based practices implemented in this initiative. A recent report on the implementation of the Mental Health Services Act documented the demographics of children served in Los Angeles County in the past year. ³⁰ For PCIT, 1,410 children were served, and demographic data indicated 66% were male, and ethnic backgrounds included 52% Hispanic, 27% “Other,” 11% African American, 8% non-Hispanic White, 2% Asian, and less than 1% Pacific Islander. For CPP, 1,572 children were served, and demographic data indicated 52% were male, with ethnic backgrounds of 53% Hispanic, 24% “Other,” 13% African American, 7% non-Hispanic White, 2% Asian, and 1% Pacific Islander.

The current study investigated the dissemination and implementation of CPP and PCIT across community mental health agencies funded by LADMH throughout Los Angeles County. The study was conducted nine years after DMH shifted the system of care to a focus on EBPs, including CPP and PCIT; therefore, it provides an opportunity to learn from agencies who have been implementing the models for an extended period of time. The study aim is to document and analyze the procedures used by a range of mental health agencies who are implementing and sustaining CPP and PCIT and the rationale for those approaches. Implications regarding optimal dissemination and implementation of evidence-based treatments for young children and barriers to system change and sustainability will be explored.

Methods

Sample selection and participants

DMH provided a list of the 32 community mental health agencies that had contracts allowing them to bill for both CPP and PCIT. Each agency was contacted to confirm whether or not they were currently providing both treatment modalities and to obtain contact information for the individual(s) who manage implementation of those interventions. Out of 32 agencies, 20 confirmed that they were continuing to provide CPP and PCIT and therefore were eligible for the study.

Survey respondents were 20 supervisors and managers representing 16 community mental health agencies that provide both CPP and PCIT to children ages birth to five years through contracts with the county DMH. Four agencies had two different people managing the two treatment models, so two people at those agencies completed the survey. The other 12 agencies (75%) had one person managing both CPP and PCIT. Participants answered questions about the practice(s) that they managed, resulting in 16 people responding to questions about CPP, and 15 about PCIT (for one of the agencies, the manager for CPP participated in the survey, but the manager for PCIT did not). Participants were mental health professionals with either a master’s degree (n = 17; 85%) or a doctoral degree in psychology (n = 3; 15%), and all but one were licensed (n = 19; 95%).

Data collection and data analysis

The study was approved by the IRB of the host institution and consisted of distribution of an on-line survey. The survey was developed for the present study and consisted of multiple choice and short answer questions (survey available upon request from the corresponding author). The survey took approximately 20 minutes to complete and had 28 questions. Areas covered by the survey included (1) background information about the survey respondent (e.g. educational background; whether the respondent provided therapy and supervision in CPP and/or PCIT); (2) structure of the agency and service provision (e.g. “Do you have staff therapists that focus on children 5 and under?”; “Does your agency have training programs?” with list of options for type of trainees; “How often does your agency provide PCIT/CPP outside of the clinic setting?” with options including “never,” “less than half the time,” “more than half the time,” “always”) (3) referral, intake, and treatment assignment process (e.g. “What are the top three referral concerns for children birth to five?” with open-ended text box for response; “Please describe the typical decision-making process regarding what type of treatment/treatment model is offered to each family” with open-ended text box for response; “By whom is the decision made?” with choices: “person who conducted phone screen; intake therapist; assigned therapist for ongoing treatment; program manager; other”) (4) adoption and initial implementation of CPP and PCIT (e.g. “How long has PCIT/CPP been implemented at your agency”; “When PCIT/CPP was first implemented at your agency, were clinicians internally trained by an in-house trainer?” (yes/no); was an outside trainer brought to your agency to train clinicians? (yes/no); were clinicians sent to trainings outside of your agency? (yes/no); (5) strategies for sustaining CPP and PCIT (e.g. “Are new staff members provided opportunities for training in PCIT/CPP? (yes/no); “what type of supervision and/or consultation is provided for PCIT/CPP?” with choices: “regular individual supervision; regular group supervision; regular consultation calls with outside trainer; as-needed in-person consultation”) and (6) barriers to implementation and sustainability (e.g. “What, if any, barriers has your agency experienced with implementation and sustainability of PCIT/CPP” with open-ended text box for response; “What are the three most commonly reported reasons for treatment dropout reported by PCIT/CPP trained clinicians?”).

The survey was distributed electronically using Qualtrics, delivered via email. The initial email with the link to the survey included an IRB-approved Informed Consent form; by clicking on the survey participants were providing their consent to participate, and the IRB granted a waiver of signed consent. Individuals were contacted up to three times by email to remind them to complete the survey. Surveys were completed by 16 (80%) of the 20 eligible agencies.

Most of the questions had restricted answer choices (yes/no or multiple choice). Descriptive statistics were used to summarize closed-ended survey responses. For the open-ended short-answer questions the researchers developed a coding manual to summarize the responses for analysis, using an inductive approach.³¹ Two researchers (the first and second authors) coded all responses independently and then met to discuss changes needed to the coding manual to address differences in interpretation of responses. Final decisions about codes were added to the revised coding manual, and then both coders re-coded all responses that had discrepant codes in the first round.

Results

Characteristics of program managers’ roles, agencies, and service delivery to young children

In addition to their roles managing the implementation of these evidence-based practices, most participants (n = 14; 70%) provided direct services using CPP (n = 12; 60%) and/or PCIT (n = 9; 45%), and all but one (n = 19; 95%) provided clinical supervision to therapists using CPP (n = 17; 85%) and/or PCIT (n = 11; 55%).

All 16 agencies were private, nonprofit mental health centers that provided mental health services to children with Medicaid insurance. Most of the agencies (n = 13; 81%) had staff who focused on working with children aged birth to five. In addition to CPP and PCIT, the agencies provided the following EBPs to children ages birth to five years: Managing and Adapting Practice (n = 12; 75%); Triple P Positive Parenting Program (n =10; 63%); Incredible Years (n = 7; 44%); Trauma-Focused Cognitive Behavior Therapy (n = 3; 19%), and Reflective Parenting (n = 2; 13%).^32–36 The majority of agencies (n = 14; 88%) had one or more training programs, including social work internship (n = 13; 81%), social work fieldwork (n = 9; 56%), marriage and family therapy training (n = 10; 63%), psychology practica (n = 8; 50%), psychology internship (n =7; 44%), psychology postdoctoral fellowship (n =7; 44%), and occupational therapy training (n = 2; 13%). Most agencies (n = 12; 86%) that had training programs included opportunities for trainees to work with children aged birth to five years.

Respondents were asked the primary referral concerns or presenting problems for children age birth to five coming to the agency, and responses were coded for key content. The most common referral reason was exposure to trauma, endorsed by all 16 agencies (100%). Behavior problems such as disruptive behavior, aggression, or preschool expulsion were cited by 88% (n =14). Four agencies (25%) noted developmental delays or autism spectrum disorder (often co-occurring with other problems). Regarding referral sources, all respondents reported referrals from the child welfare system (100%), and most (n = 14; 88%) reported parents self-referring or word of mouth. Other common referral sources included teachers/school personnel (n = 11; 69%), primary care physicians (n = 8; 50%), and developmental disability/early intervention programs (n = 6; 38%). Respondents were asked about collaboration with other service systems, and all (100%) endorsed working with the child welfare system, schools, and developmental disability/early intervention system. About half the agencies (n = 9; 56%) reported collaborating with primary care physicians.

Implementation of CPP and PCIT

The majority of participants reported that their agency had been implementing CPP for more than 5 years (n = 14; 88%) and PCIT for more than 5 years (n = 10; 67%). The number of therapists currently trained to provide CPP ranged from 2 to 24 (M = 9.69; SD = 6.04), and for PCIT the number of therapists ranged from 1 to 20 (M = 6.06; SD = 5.05).

All but one agency provides CPP and PCIT in both English and Spanish. Although most services are offered in the clinic setting, all agencies reported that CPP is provided outside of the clinic for at least some families, with 40% of the agencies providing CPP in the field more than half the time. CPP is provided in the field in the home setting (n = 16; 100%), school setting (n = 7; 44%), or another community setting such as a park (n = 6; 38%). Similarly, most agencies (n = 12; 80%) reported that PCIT is sometimes provided in the field, although most agencies provide PCIT services in the field less than half the time. Field-based PCIT services are offered in the home (n = 12; 80%), school (n = 2; 13%), and in a specially designed mobile van (n = 2; 13%). Some agencies use a co-treatment model including another discipline in the provision of services in CPP (n = 7; 44%) or PCIT (n = 5; 33%), including occupational therapists, speech-language pathologists, parent partners, case managers, and behaviorists.

Supporting and sustaining evidence-based practice

When they began implementing CPP and PCIT, few agencies had clinicians already trained in the models (n = 4 agencies for each model). Before launching their CPP program, most agencies (n = 14; 88%) sent clinicians to trainings outside the agency, one agency had an in-house trainer, and one brought in outside trainers. The initial training for PCIT was more varied across agencies, with 8 (53%) bringing in an outside trainer, 6 (40%) using an in-house trainer, and 3 (20%) sending clinicians to trainings outside the agency. When new staff are hired, most agencies provide them training in CPP (n = 14; 88%) or PCIT (n = 12; 80%). For CPP, training for new staff is usually done by sending them to outside training (n = 13; 81%), whereas for PCIT, it is usually provided by an in-house trainer (n = 11; 73%). Funding for training is provided by the DMH (CPP: n = 15; 94%; PCIT: n = 12; 80%). Agencies also use grants to train staff (CPP: n = 2; 12%; PCIT: n = 8; 53%).

Table 1 provides information about the supervision, consultation, and quality improvement protocols that agencies reported using to support clinicians in using the models with fidelity. For both models, most agencies report providing regular individual and group supervision as well as as-needed consultation, suggesting that the agencies have built the capacity in-house to support their clinicians in implementing the model. CPP is more likely to also offer regular consultation calls with an outside trainer. All but one agency uses fidelity measures to support treatment fidelity in both models, and chart reviews are another common approach to assess fidelity. PCIT agencies are more likely to offer advanced trainings or booster sessions for trained staff, and review of video recording.

Table 1.

Agency Supports for Therapists: Supervision, Consultation, and Fidelity Monitoring

Domain	Activity	Child-Parent Psychotherapy (n = 16) (n, %)	Parent Child Interaction Therapy (n = 15) (n, %)
Supervision/consultation available to staff providing therapy	Regular individual supervision	15 (94)	12 (80)
	Regular group supervision	11 (69)	8 (53)
	Regular consultation calls with outside trainer	11 (69)	4 (27)
	As-needed in-house consultation	12 (75)	13 (87)
Quality improvement protocols to support treatment fidelity	Use of fidelity measures	15 (94)	15 (100)
	Supervision/consultation	16 (100)	13 (87)
	Chart review	13 (81)	12 (80)
	Advanced training sessions/boosters	10 (63)	12 (80)
	Video recording/review	4 (25)	11 (73)

Decision-making process in selecting evidence-based practices for families

The survey included questions about the process for making decisions about what type of treatment to offer to each family. Regarding the timing of the decision, 13 respondents (65%) said the decision is made prior to the intake assessment (e.g. based on screening information), and 14 (70%) said the decision is made following the intake assessment; seven respondents indicated both options, depending on the situation. When asked who usually makes the decision regarding what type of treatment to offer to a family, responses included a team (n = 8; 40%), program manager (n = 5; 25%), intake therapist (n = 4; 20%), or the assigned therapist who will be providing ongoing treatment (n = 5; 25%). Many respondents explained in the comments that decisions can be changed depending on ongoing needs or information gathered during intake or treatment.

The survey included a list of variables that might be considered in making the decision about what treatment is offered to a young child and their family, and respondents rated each variable in terms of its importance in influencing the decision-making process (Likert scale from 1, “not at all important” to 4, “very important”). Respondents rated child’s symptoms as highly important (M = 3.9), and experiences of the child/family (e.g. trauma history) as highly important (M = 3.8). Also viewed as important, but less highly, were availability of a therapist with training in the model (M = 3.3) and family preferences (M = 3.2). Least important was funding available for the treatment modality (M = 2.5).

More than half (n = 11; 55%) of respondents indicated that their agency uses a decision-tree, algorithm, or other procedure to determine which treatment modality the family will be referred to. They explained more about their process in an open-ended question, which was coded. In their responses, many participants described the nuanced way in which the child’s symptoms and family’s needs are considered. For example, one participant wrote,

Dependent on the issues that the child is presenting with in the intake, we will assess the needs, not just of the child, but of the family. If the issue is based on the trauma of the child vs. the shared trauma of the family, if the child’s behavior is becoming disruptive and impairing her functionally, if attachment has been compromised, and/or if parental confidence is low, we will determine the best model for addressing the child and family symptoms.

A common approach considers the relative importance of trauma versus disruptive behavior: “Decisions are based on the client’s symptoms and the family choice. Generally, if the client’s behaviors are disruptive in nature and there is no trauma, then the treatment options are MAP, Triple P, or PCIT. If there is trauma, then the treatment is CPP.” Two participants described the use of a more formal decision-tree. One explained, “With CPP and PCIT, we do use a decision-tree. We see if the presenting symptoms and behaviors are more trauma-related or disruptive. We utilize the TSCYC and the ECBI [parent report outcome measures] during the assessment to also help with making that decision.” Others referenced decision trees but indicated that the use of such algorithms was not primary to the decision-making process. For example, one participant reported, “We have a decision tree to guide selection of a treatment model; however, most clinicians are used to making these decisions, and they are able to determine treatment recommendations without the use of the decision tree.”

Barriers to implementation and accommodations

The survey included an open-ended question asking “what barriers, if any, has your agency experienced with implementation and sustainability of [CPP/PCIT]?” Responses were coded and the themes are presented in Table 2. The most common barrier for both EBPs was logistics of the training process. For example, one respondent described training for CPP: “Implementation was somewhat difficult because of the stringent requirements (18 months, including 3 trainings, frequent phone consultation, and increased group and individual supervision). However, we found the process helpful and enjoyable.” A common concern expressed with CPP, in addition to the length of training, was the lack of opportunities for train-the-trainer so that agencies could develop in-house training. Several respondents described barriers related to the length of time to train in PCIT, the need for clients to successfully complete treatment in order to “count” toward training, and lack of regular opportunities to obtain PCIT training. For both modalities, the length and cost of training combined with staff turnover create challenges for sustainability.

Table 2.

Barriers to Implementing and Sustaining Evidence-Based Practices (EBP)^*

Domain	Barrier Identified by Survey Participants	Child-Parent Psychotherapy (n = 16) (n, %)	Parent Child Interaction Therapy (n = 15) (n, %)
Training	Logistics of the training process	8 (50)	5 (30)
	Loss/lack of in-house trainer	3 (19)	4 (27)
Staff	Staff turnover	4 (25)	5 (30)
	Low number of therapists trained in the modality	0	4 (27)
	Low number of Spanish-speaking clinicians trained	1 (6)	2 (13)
	Lack of foundational early childhood training	2 (12)	1 (7)
Clients/Families	Difficulty with retention of cases/attrition	0	3 (20)
Systems	Funding/financial difficulties	3 (19)	2 (13)
	Low referrals streams for EBP	1 (6)	3 (20)

^*Open-ended responses to the survey question: “What barriers, if any, has your agency experienced with implementation and sustainability of CPP/PCIT” were coded and the themes identified are listed.

Several respondents wrote about the logistics of the treatment modalities that were challenging, but the specifics varied by EBP. One respondent noted the challenge of parents having to come to the clinic for PCIT in order to have access to the one-way mirror and audio equipment. Another noted, regarding PCIT, “concern about introducing a behavioral treatment modality that doesn’t address the attachment or trauma,” or “caregiver mental health issues.” For CPP, a participant noted “longer period of assessment at the outset of treatment, which can sometimes be difficult for the family to engage with.” Another felt that the amount of fidelity measures and documentation of fidelity for CPP was excessive. Finally, another participant expressed the challenge of “implementing CPP practice in the home with multiple children and limited space.”

Another way to assess barriers was a survey question asking about the most common reasons for dropout from each type of therapy. Responses were open-ended and coded for themes. Table 3 summarizes the number of responses by therapy model, in the domains of child symptoms, caregiver engagement, caregiver challenges, and logistics. For CPP, the three most common reasons for dropout were caregiver mental health issues, lack of parent engagement/buy-in regarding the approach to treatment, and DCFS issues such as change in placement or a child abuse report. One respondent noted, regarding CPP, “parents/caregivers have difficulty acknowledging their child’s trauma or how their own trauma impacts their caregiving.” Another reported about “DCFS-involved families where the case is closed…[the] family no longer wants to participate in mental health services.” For PCIT, the most common reasons for dropout were caregiver/family system stressors, attendance issues, and lack of parent engagement/buy-in regarding the treatment model. One participant stated that “parents hesitate to engage in PDI [Parent Directed Interaction] or feel they do not have enough time to ‘process’ what is going on outside of session content (e.g., other stressors).” Another respondent reported that “parents do not want to do the coaching format of PCIT.”

Table 3.

Reasons for Treatment Dropout^*

Domain	Reason for Dropout	Child-Parent Psychotherapy (n = 16) (n, %)	Parent Child Interaction Therapy (n = 15) (n, %)
Symptoms	Symptom improvement	1 (6)	4 (27)
	Lack of treatment progress	3 (19)	3 (20)
Engagement	Lack of parent engagement/buy-in re the model	9 (56)	7 (47)
	Attendance issues	6 (38)	8 (53)
Caregiver issues	Caregiver mental health issues	9 (56)	5 (33)
	Caregiver/family system stressors	5 (31)	8 (53)
	DCFS issues (e.g. change in placement; child abuse report)	7 (44)	4 (27)
Logistics	Practical issues (lack of child care, transportation barriers, insurance issues)	2 (13)	4 (27)

^*Survey participants were asked, “What are the three most commonly reported reasons for treatment dropout reported by CPP/PCIT-trained clinicians?”; responses were coded and themes are listed

Participants were asked an open-ended question about commonly reported treatment adaptations or modifications that clinicians make to meet the unique needs of families with young children at their agency. Some adaptations addressed logistical issues, such as providing services outside of the clinic/in the field (n = 6; 30%) and accommodating parents/caregivers schedules (n = 5; 25%). Others addressed the needs of other family members, including involving siblings in treatment (n = 4; 20%); providing additional support to parents/caregivers such as addressing parental mental health needs, disabilities, and parenting stress (n = 6; 30%); and incorporating multiple caregivers/family members (e.g. foster parents, biological parents, other family supports) (n = 6; 30%). Some adaptations addressed individual needs of children, including modifications to address a child’s developmental delay (n = 5; 25%) or incorporation of additional services (e.g. interdisciplinary consultation, assessment referrals, therapeutic behavioral services) (n = 4; 20%). Three participants reported modifying the treatment approach to include individual rather than dyadic work with the child. One participant noted accommodating parent preferences, giving the example of a parent not wanting to use time-out which is part of the standard PCIT approach. Finally, two participants reported modifications to the PCIT protocol to address child trauma (e.g. providing space during the PCIT session to process trauma; omitting the “swoop and go” or time-out strategies).

Discussion

Despite the various challenges with regard to implementation and sustainability identified by the community mental health agencies, survey responses suggested that each agency was committed to implementing evidence-based practices for infants and/or young children and their families, and had successfully sustained the practices for years. As a whole, managers surveyed highlighted the need for early childhood mental health services and their interest in identifying strategies to improve the field’s ability to implement and sustain EBPs for the birth to five population, especially for at-risk families with trauma and numerous barriers to treatment.

A high level of variability was evident with regard to the processes through which agencies assigned treatment modalities for particular families. Responses suggest that a high level of clinical judgement is required to adequately assess the nuances in child and family needs to best direct them to the most appropriate treatment. While many factors were considered in the decision-making process, the most important factors appeared to be the child’s symptoms and the child/family characteristics, with additional factors including therapist availability and family preference. Most often, survey respondents reported weighing whether a child exhibits disruptive behaviors versus a trauma history; however, behavioral problems are not uncommon in traumatized youth, which highlights the complexities when deliberating on the best treatment for each family.^37–39

The complexity and clinical judgement involved in the decision-making processes suggests that providing early childhood EBPs requires foundational knowledge of early childhood mental health principles, understanding of the differences between the CPP and PCIT models, and the ability to balance the needs of the child and family with the resources available within the agency. Given the multitude of factors clinicians must consider when determining the best fit of treatment modality, decision-making guidelines appear to be needed.

A tool such as a decision-tree may provide some guidance; however, although half of the agency staff members mentioned having a decision-tree, most reported they do not actually use it. It may be difficult for a decision algorithm to address the complexity of the clinical, logistical, and individual diversity factors that may be important for consideration in treatment selection. Osofsky and colleagues described considerations for treatment selection for young children impacted by trauma including a child’s age, type and sequelae of trauma exposure, logistical constraints, and determination of who the client is (focus on child, parent, or parent-child relationship).¹⁰ These considerations are embedded within the broader context of the therapist’s training background and theoretical orientation and the family’s background. Future research is needed to examine which factors are associated with positive treatment outcomes for PCIT and CPP to inform the development of decision-making guidelines when identifying the best fit of modality for each individual family. To date, no randomized controlled trials have tested the relative efficacy of the two models for children with overlapping indications (e.g. preschoolers with both trauma history and disruptive behavior). Further increasing complexity, PCIT and CPP are not the only EBPs for young children. Interventions such as Video Interaction Guidance, Attachment and Biobehavioral Catch-up, and Watch, Wait and Wonder, among other treatment approaches, have an evidence base for infants and young children.⁵

Although each agency staff member described their commitment to early childhood EBPs and thoughtful approach to selecting a treatment modality for each family, they also described challenges with regard to the training required for using the CPP and PCIT models that impacted the sustainability of the programs. For both modalities, the length, cost, and highly-involved nature of the training process posed barriers to implementation. Moreover, the high staff turnover common in community mental health agencies leads to challenges with sustainability, particularly after agencies have already invested time and resources in training these individuals. For PCIT, this challenge is somewhat mitigated by the ability to train an in-house trainer, rather than primarily relying on outside training opportunities, as is the case with CPP. However, regardless of the model, agency supervisors and managers highlighted the lack of prior foundational training in early childhood to ground clinicians as they learn these modalities. While the developers of PCIT and CPP are responsible for balancing the need for ease of dissemination and implementation with maintenance of fidelity to their treatment model, recommendations for the mental health field as a whole can be gleaned from the results. Increased inclusion of foundational early childhood mental health principles and exposure to work with young children within graduate program curricula would help prepare clinicians to enter the workforce.^40–41 Further, attention to staff burnout within community mental health agencies serving at-risk populations of children and families is important to reduce staff turnover and ultimately enhance not only EBP sustainability but overall service delivery.⁴² Finally, development of train-the-trainer models supports sustainability of EBPs over time.

Challenges with transportation were a common logistical barrier to families’ participation in treatment. Participants reported accommodations of providing home-based or community-based treatment. This is consistent with recommendations in the literature to consider the potential benefits of home-based PCIT in reducing attrition.⁴³

Implementation and sustainability of both PCIT and CPP were also impacted by difficulties with parent engagement. Although the specific reasons for lack of engagement may differ between models, the challenge in this area suggests the need for additional reflection on the family’s match with the treatment approach or other contextual factors that may be impacting engagement. Ghosh Ippen and Lewis describe a diversity-informed approach to understanding engagement in CPP that involved four core concepts: 1) each individual’s experiences and history shapes their perceptions of intervention, 2) differences in history and experience between the family and the provider may affect treatment engagement, 3) one’s emotional state impacts how effective they can be at holding in mind the experiences of another, and 4) reflective practice is crucial to upholding a diversity-informed approach, particularly when providing infant mental health treatment such as CPP.⁴⁴ Similarly, while the standard PCIT approach has been found effective across many groups, population-specific adaptations have been created and researched to further tailor PCIT to be more congruent with the cultural beliefs and values of various groups, which may decrease the risk for attrition in these groups.^43,45,46 In sum, cultural sensitivity and reflective practice are essential when selecting the treatment approach as well as throughout the course of treatment for every family.

Families may also benefit from additional psychoeducation regarding each treatment modality at the onset of treatment to inform treatment selection and ensure that treatment approaches match well with parents’ preferences and understanding of their child’s needs. Reflection on indirect communication about barriers (e.g. frequent no-shows) and exploration of a family’s initial experiences in treatment may provide information to clinicians to guide treatment planning. Offering options for shifts in treatment approach at the first signs of challenges with engagement may increase the parents’ experience of feeling like an equal agent within their child’s treatment and improve their overall buy-in to mental health treatment.

Differences between the clinical challenges described for PCIT versus CPP were described and seem to be related to the structure and approach of each modality. For PCIT, the highly structured treatment approach may leave less room for families to process the impact of family stressors commonly experienced by the at-risk population served by community mental health programs. All survey respondents cited trauma exposure and/or involvement with child-protective services as the most common reason for referral to mental health treatment. Some participants described implementing adaptations to PCIT by including additional parent sessions to provide time and space to address the multitude of stressors faced by families. McNeil and Hembree-Kigin developed recommendations for PCIT providers working with families experiencing major life stressors, including developing attendance contracts with caregivers at the onset of treatment, scheduling weekly appointments at the same time and day of the week, keeping the assessment process brief and moving to intervention quickly, developing realistic treatment goals, and referring the family for treatment targeting their additional stressors.¹⁶

Within CPP, the most commonly cited reason for treatment dropout was challenges related to caregiver mental health issues. Lieberman and Van Horn outlined possible variations in the typical joint child-parent format during CPP sessions.⁸ These variations included extensive collateral sessions when the caregivers require additional support regulating their own feelings related to overwhelming trauma reminders; child-only sessions when the caregiver is significantly triggered by their child’s behavior; and co-parenting treatment sessions when the child is caught in the middle of the parental conflict. While reflection on caregivers’ childhood experiences and the ways in which they impact parenting are embedded within CPP, clinicians must also be well versed in determining when adult mental health issues are beyond the scope of CPP treatment and require referrals for additional individual treatment for parents. Further, collaboration with outside service systems, including parents’ individual therapists when applicable, is highly recommended for both CPP and PCIT.

In addition to the strong empirical support for PCIT and CPP in the academic literature, other researchers have documented efforts to adopt these models within community mental health systems.^47,48 Pearl and colleagues examined the effectiveness of the dissemination of hospital-based PCIT to community clinicians in Cincinnati.⁴⁸ They reported challenges with high attrition rates among high-risk families, as well as staff turnover. Problems with attrition were also identified in a study examining the state-wide dissemination and implementation of PCIT in Delaware.⁴⁹ Additional time and effort spent in the Exploration and Adoption phase through information sharing within and across organizations was found to be beneficial to the long-term adoption of the treatment, since the stakeholders across each level within the system bought in to sustaining the practice. Both Cincinatti and Delaware found that having on-site or local trainers decreased practitioner dropout.^48,49 In a study of the implementation and sustainability of CPP in Boston, reflective consultation was found to be an essential component to providing clinicians a space to attend to their own emotional reactions and thereby sustain engagement in trauma work.⁵⁰ Reflective consultation also contributed to sustaining the organizational infrastructure to support the use of the EBP. Further, ongoing program evaluation and/or self-assessment was recommended to facilitate continuous improvements in the organization’s dissemination efforts.^49,50

Despite the challenges identified, the agencies surveyed appear to have reached the Full Operation stage of implementation, given that they have sustained the models over a period of years and are able to train new staff.²² Some of the agencies have strived to meet the complex needs of the community they serve by entering the Innovation stage of implementation, describing clinical modifications to the standard treatments. However, a limitation of the study is that only agencies currently providing CPP and PCIT were included; there were an additional 12 agencies (38% of the total) that had contracts to provide the models but had stopped providing one or both of them. Therefore, information about the challenges with sustainability that influenced those agencies was not available. Future studies investigating factors that led to the discontinuation of various EBPs would provide valuable information regarding the significant barriers to implementation and the strategies utilized to combat such barriers that were found to be unsuccessful. Lessons learned from these agencies may be beneficial to the field to guide recommendations to enhance the dissemination and implementation of early childhood EBPs.

Additional limitations of this study include the modest sample size and the limited generalizability of the results given the homogeneous sample including only community clinics funded by the Department of Mental Health in one county. While the sample of agencies in it of itself was homogenous, the families served by the agencies are predominantly of low socioeconomic status, consisting of a myriad of ethnic and cultural backgrounds, and are considered an accurate representation of populations that seek the services of publically-funded community clinics, representing a strength of the study. One limitation of the study is that the survey did not include specific questions about the cultural composition of the clients served in each clinic, nor specific questions about the use of culturally-response practices. Another limitation is that clinics that provided either PCIT or CPP, instead of both EBPs, were excluded from the study; therefore, additional information was not obtained related to how agencies treat young children when they are limited to one EBP. Moreover, the information gathered only specifically relates to PCIT and CPP and did not directly assess the implementation of other EBPs used for infants and young children, such as Managing and Adapting Practices (MAP) and Incredible Years - Preschool.^51,52

Implications for Behavioral Health

Recommendations regarding implementation of EBPs for infants and young children can be gleaned from the study findings. Increased attention to organizational and systems changes during the earliest stages of dissemination and implementation are recommended to support EBP sustainability. For example, agencies interested in adopting infant and early childhood EBPs may benefit from incorporating training in early childhood principles to provide clinicians delivering CPP and PCIT the foundational skills necessary to work with the birth to five population. In addition, Joyce and Showers found that learning of a complex repertoire of skills, such as training in various EBPs, is most effective when the individual is in the role of a new learner or trainee.⁵³ As such, graduate programs in psychology and social work could incorporate into their curricula more opportunities for education in fundamental early childhood mental health principles and/or training in EBPs.

Moreover, increased time spent identifying stakeholders and champions of EBP implementation across the various system levels will increase the likelihood that the EBPs will be sustained. Participants in the study identified various barriers to sustainability, specifically related to the intensive training processes, high staff turnover, and high levels of family attrition in treatment. Increased communication across agencies during the Exploration and Adaptation phase will likely lead to deepened understanding of the unique needs of the community served as well as increasing commitment to efforts to implement the EBP to address those needs. In addition, regular program evaluation during an agency’s implementation of an EBP is also recommended to thoroughly understand the families’ responses to the treatment intervention and the clinicians’ skills and areas of need to support family engagement, with special attention on cultural factors that may influence treatment engagement. Further, developers of EBPs may consider modifications to their training processes and fidelity measures in order to reduce the time highly taxed clinicians within community mental health agencies need to devote to training in these practices.

Lastly, future research is needed to inform the development of decision-making guidelines to offer the most appropriate treatment given each family’s individual needs. A randomized controlled trial comparing PCIT and CPP could identify various child, caregiver, or situational factors that may suggest one treatment is better suited to meet a family’s needs than the other. Future studies may also build upon the current study and include qualitative interviews with birth to five program managers to assess the nuanced clinical judgement used when deciding between the models and further explore cultural considerations in the application of these treatment models.