Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2016 Jul 6.
Published in final edited form as: J Public Child Welf. 2015 Jul 6;9(4):362–381. doi: 10.1080/15548732.2015.1060918

Evidence-Based Parenting Programs for Maltreating Parents: Views of Child Protective Services Caseworkers

Keri LM Pinna 1, Lynn K Lewis 2, Canan Karatekin 3, Angela Lamb-Onyiga 2, Ashley Hirilall 4, Sabrina D Jones 2
PMCID: PMC4864603  NIHMSID: NIHMS752631  PMID: 27182218

There has been a growing movement in recent years toward using evidence-based programs (EBPs) for psychosocial interventions in general, and with maltreating parents in particular (Rubin, 2011; Thyer & Myers, 2010). There is an especially strong need for effective and evidence-based interventions for maltreating parents, as rates of repeat maltreatment tend to be very high (Connell et al., 2009; Hindley, Ramchandani, & Jones, 2006). Although there were almost no EBPs for maltreating parents in the 1990s, a number of such programs have been developed for, or adapted to, this population in recent years (e.g., Chaffin, Bard, Bigfoot, & Maher, 2012). However, effect sizes of interventions drop considerably from trials conducted under ideal conditions to trials conducted in more realistic community settings (Durlak & DuPre, 2008; Weisz, Donenberg, Han, & Weiss, 1995), highlighting the limitations inherent in this first of many steps along a long road to full-scale implementation (Westfall, Mold, & Fagnan, 2007). Thus, in addition to moving towards the development of EBPs, the need to better understand challenges to implementing these EBPs in community settings has become increasingly recognized (Maynard, 2009). Rigorous research has been conducted on the dissemination and implementation of interventions in youth mental health, prevention, and health promotion (Durlak & DuPre, 2008; Novins, Green, Legha, & Aarons, 2013). However, viewpoints of critical stakeholders such as Child Protective Services (CPS) caseworkers have been excluded. The present report seeks to fill this gap in the literature by exploring CPS case worker views on barriers to implementing an empirically based parenting program – the Positive Parenting Program (Triple P; Sanders, 2012) – with maltreating parents.

Referral sources play a critical gateway role in routing target clientele into interventions, and thus their perceptions of the intervention can greatly affect the provision and success of the interventions (Fixsen, Blase, Naoom, & Wallace, 2009). For example, one implementation study found that a lack of appropriate referrals was a significant factor in the failure of SafeCare to reach large numbers of potential clients (Whitaker et al., 2012). Despite such evidence, knowledge and perceptions of EBPs by those who refer clients to interventions are rarely included in evaluation studies of EBPs.

The little research that is available suggests that caseworkers are often not exposed extensively to the interventions to which they refer their clients (Kazak et al., 2010). Other research suggests that caseworkers may not have a clear understanding of the difference between EBPs and other services available in the community that are not evidence based (Katz, 2011). One report suggests that caseworkers may not see the need for EBPs in real-world practice (Luongo, 2007). Some have suggested that targeted training in EBPs may enhance the success of implementation, though this is currently a much debated issue (Luongo, 2007; Rubin, 2011). Even when referrals are made, attrition rates are high (Chaffin et al., 2012; Karatekin, 2013). Such high rates of attrition are highly concerning given that interventions cannot be effective if families do not engage or complete a sufficient dosage of the intervention to experience its potential benefits.

In addition to concerns about referrals to and retention in EBPs, concern is often raised that the evidence upon which the interventions are based may not generalize to the complexities inherent in child maltreatment cases (Aarons & Palinkas, 2007; Allen, Gharagozloo, & Johnson, 2011). This concern is often valid, based on the methodological rigor that is applied to the study of EBPs. Service providers legitimately consider it critical that EBPs adequately fit needs of the families with whom they work (Aarons & Palinkas, 2007). Similar concerns are likely also true of caseworkers who would refer families to an EBP. However, many EBPs have not been studied to determine how well they fit the need of specific racial/ethnic or other cultural groups (Durlak & DuPre, 2008; Novins et al., 2013). Ethnic minorities are under-represented in most intervention studies (Rubin, 2011; Sue, Zane, Nagayama Hall, & Berger, 2009), complicating any efforts to examine outcomes as a function of race and ethnicity (Petrosino, 2000), and few EBPs are designed specifically to meet the needs of various ethnic/racial groups (Chaffin et al., 2012). The dearth of research on this issue of generalizability may contribute to suspicion among caseworkers regarding the extent to which EBPs will meet the needs of families on their case loads (Hoagwood & Olin, 2002; Karatekin, 2013).

Triple P stands out as one exception to the general rule of insufficient consideration of generalizability. Triple P is a five-level intervention that is designed to be flexibly delivered to meet the particular needs of a given family or population of families (Prinz, Sanders, Shapiro, Whitaker, & Lutzker, 2009). Level one is a universal intervention aimed at raising awareness of and disseminating information about the importance of supporting parenting through brochures, posters, newspaper columns, billboards, etc. Level two provides brief seminars and primary care support for parents who are generally functioning well, but have one or two minor concerns. Level three provides targeted counseling (average of four 15 to 30 minute consultations) to parents of children with mild to moderate behavioral difficulties. Level four (considered the standard level) delivers more extensive intervention to parents who need intensive support, and level five provides intensive support for families with more extensive family systems problems (e.g., partner conflict, mental health concerns, stress, or anger management problems).

The Triple P system was developed with specific attention to representing diverse families in both print and video materials that supplement the intervention, and is among the few empirically based parenting programs that have shown promise of being effectively applied to the problem of child maltreatment (Prinz et al., 2009). In one study, focus groups were conducted with culturally diverse parents to explore the extent to which elements of the program were considered to be acceptable and relevant across groups (Morawska et al., 2010). Results suggested that Triple P elements were equally acceptable and culturally relevant across groups. However, this research is not without its limitations. African Americans (who are over-represented in the American CPS system; Putnam-Hornstein, Needell, King, & Johnson-Motoyama, 2013) were not included in the sample. Further, the potential impact of racial/ethnic matching between clients and service providers has not been considered, despite evidence that such matching may enhance intervention outcomes in minorities (Cabral & Smith, 2011; Campbell & Alexander, 2002), and clear evidence that clients tend to prefer racially matched clinicians, particularly among African Americans (Cabral & Smith, 2011).

The present study sought to address the gap in the literature, which fails to adequately account for the role of CPS caseworkers in the implementation of such EBPs as Triple P. Two focus groups were conducted with CPS caseworkers who were exposed to Triple P (level four) to varying degrees. These focus groups explored caseworkers’ views of Triple P. Specifically, we explored the extent to which caseworkers’ views would vary as a function of training. Because minority families are over-represented on CPS-caseloads, we also explored the extent to which views would be particularly salient with respect to issues of cultural sensitivity.

Methods

University-Community Collaboration

CPS in a large metropolitan area served as the community collaborator, and was represented by two CPS administrative staff, a staff psychologist, and two case worker supervisors. Partners with the University of Minnesota included a psychologist/faculty member from the Department of Psychiatry's Center for Personalized Prevention Research; the Center's Field Director; and a research technician. A postdoctoral trainee and undergraduate research assistant in Child Psychology later joined the university team, as well. University partners were all European-American except for one Asian-American (undergraduate research assistant). The lead university partner was male. The remaining university partners were female. Community collaborators included three African-American women (administrative lead and two case worker supervisors), two European-American administrators, and one European-American male (staff psychologist). Three of the university partners and two of the community partners were trained in Triple P either prior to (first author) or in the course of the study (UMN lead, Field Director, and both CPS supervisors).

The collaboration between CPS and UMN was initiated by the university, which sought to pilot Triple P with parents who had been cited for concerns of possible child maltreatment. Triple P was recommended by the university partners as an EBP that has shown promise with diverse (Cabral & Smith, 2011) and maltreating families (Prinz et al., 2009). Although evidence for benefit with maltreating families was based on delivery of the full Triple P system (levels one through five), the cost of such implementation was prohibitive. Thus, level four (standard) Triple P was piloted as a first step in determining the feasibility of integrating this EBP into the CPS system.

Monthly large group meetings were hosted by CPS to discuss implementation. CPS identified young mothers of young children as the population of greatest concern for child maltreatment and high recidivism. Thus, inclusion criteria for the feasibility pilot were established as follows: active CPS case; at least one target child between the ages of 3 and 8 years; no history of sexual abuse perpetrated by the parent under investigation; and no concerns of developmental delay in the parent. Initial inclusion criteria also limited referrals to mothers under investigation and specified that mothers must be between the ages of 18 and 28. To increase the sample size, criteria were adjusted approximately four months after initiation of the feasibility pilot to include fathers and to increase the upper age limit of parents to 32.

Family & Practitioner Characteristics

In addition to the above noted inclusion and exclusion criteria, families referred to Triple P resided primarily in the inner city. They were disproportionately African American or of another minority status. Some, but not all, target children were placed in foster care at the time of Triple P services. Triple P services were delivered through a local Community Mental Health Center (CMHC) that was contracted to serve CPS-referred families. The CMHC supervisor was a European American woman. The four Triple P service providers were all women. One was European American, one was Hispanic, and two were Asian American. The implementation of Triple P services was supported by monthly meetings between the university lead/postdoctoral fellow and CMHC supervisor/service providers. These meetings were held at the CMHC.

Case Worker Training

Case worker schedules precluded their attendance at the above-described monthly large group meetings. However, they were responsible for referring families to Triple P, based on the above described inclusion and exclusion criteria. Caseworkers then maintained contact with both the Triple P practitioner and family, serving as a common point of contact throughout the engagement and retention process.

In order to ensure that caseworkers were familiar with the service to which they were referring families, they were trained in Triple P one of two ways. Caseworkers either completed the same 3-day training course as that which service providers completed (Full Training Group; N = 17) or were provided with a condensed 3-hour overview of Triple P (Condensed Training Group; N = 38). Both trainings were delivered at CPS. The first training was delivered by a certified Triple P trainer. The second was delivered by the Center Field Director, who had also completed the full 3-day training. The first/full training was delivered though didactic lectures, review of video vignettes, provision of reference materials, and active role plays. The second/condensed training was delivered primarily through didactic lecture and provision of reference materials.

The second training was delivered in response to concerns that insufficient referrals were being received by the CMHC to properly evaluate the feasibility of delivering Triple P to CPS-involved families. The goal of the second training was to increase the number of caseworkers who could make referrals for the feasibility trial. The addition of new referring case workers, increasing parental age restrictions to 32, and allowing fathers to also be referred proved insufficient. University-community dialogue suggested that caseworkers may be referring families to alternative/pre-existing services in lieu of Triple P out of concern that Triple P may be a poor fit to the needs of CPS-involved families. Focus groups were thus designed to explore these potential explanations to low rates of referral.

Focus Groups

Two focus groups were conducted with CPS caseworkers to assess their perceptions of how well Triple P fit the needs of CPS-involved families. The focus groups were designed collaboratively between university and community partners. Specifically, focus group design was discussed in monthly large group meetings between university and community partners. Focus group questions were drafted by university partners and reviewed by community partners for feedback and modification. Initial questions were designed to assess caseworkers’ understanding of Triple P. These questions were followed with a semi-structured review of the program's primary components. Caseworkers were then asked about what family characteristics they believed were important to consider, when evaluating the fit of Triple P to family needs. Questions subsequently elicited views on which components of Triple P are/would be most/least helpful to CPS-involved families, and why. If views of cultural fit had not already been discussed, this was specifically assessed. Caseworkers were also asked what, if any, specific changes they would like to see made to the program. An interview guide was developed, based on these mutually agreed-upon set of focus-group questions. CPS caseworker supervisors participated in both focus groups. However, because they did not carry active caseloads, their participation served a supportive role, helping to ensure rapport and clarity of communication between facilitators and caseworkers, as well as serving as the official community presence.

Given variation in case workers’ level of training in Triple P, the first focus group was completed with those caseworkers who had completed the full 3-day training, while the second focus group was completed with those caseworkers who had completed only the condensed 3-hour overview. Recruitment was coordinated through CPS supervisors, who emailed flyers to all caseworkers who had received training in Triple P for the purpose of the feasibility pilot (N = 55). Both focus groups were conducted by the first author and the field coordinator.

Each focus group began with a review of informed consent. Caseworkers then complete a brief self-report questionnaire assessing their own demographics, demographics of their total current caseloads (i.e., not limited to families who would be eligible for the feasibility pilot), and information characterizing their caseloads, including (1) frequency of maltreatment types, (2) frequency with which families on caseloads were engaged in parenting education, and (3) case complexity (i.e., ages of children served, racial/ethnic background, non-English speaking, types of maltreatment, need for financial assistance, homeless/highly mobile status, average number of services to which they typically referred families).

After completing the self-report questionnaire, focus group questions began with a probe of case workers’ understanding of Triple P. Additional information was then provided to help refresh case worker memories regarding the primary components of Triple P. Handouts were provided to supplement this verbal refresher. Subsequently, interview questions sought to explore family characteristics deemed to be important when working with CPS-involved families, including those described on the questionnaire (e.g., number and ages of children, racial/ethnic background, case complexity), and those not described on the questionnaire. This established a context for discussion about the pros and cons of delivering Triple P with CPS-involved families. Participants were then asked which components of Triple P they believed were helpful (or potentially helpful) for CPS-involved families, and which components of Triple P they believed were unhelpful (or would likely be unhelpful). Similar questions were prepared regarding intervention characteristics that were or were not good fits for the cultural backgrounds of CPS-involved families. Participants were then asked what they might change about the intervention in order to make it more helpful for this population. Both focus groups were audiotaped and lasted approximately one hour.

Participants

The overall participation rate for the 55 participants who were invited was 38.2% (N = 21). Focus group one (full training) was comprised of 12 participants, while focus group two (condensed training) was comprised of nine participants. Five participants (two from the full training group; three from the condensed training group) did not provide their age. Of the remaining 16 participants, mean age was 44.12 (SD = 11.31). Most (76%) were female. Two participants from the group two did not report their race/ethnicity. Of the 19 who reported race/ethnicity, 11% were Asian, Asian American, or Hmong; 37% were Black or African American; 5% were Hispanic or Latino; 37% were White, Caucasian, or European (not Hispanic); and 11% were multiracial. Age, gender, and minority status did not differ between groups ps ≥ .24. However, participation rate for the full training group was significantly greater (70.59%) than the condensed training group (23.68%) χ2(1) = 10.95, p < .01.

Estimated Caseload Characteristics

Participants estimated an average caseload of 9.05 families (SD = 4.43), each family having an average of 3.56 children (SD = 3.36). Average age of target children was evenly distributed across the age range, with slightly fewer children in the 13 years and older category, compared to younger categories (21.22% infant to two-year-olds, 28.48% three- to eight-year-olds, 22.30% nine- to 12-year-olds, and 16.89% 13-year-olds and older). The average caseload was comprised primarily of African American families (65.37%), with an average of 16.49% of families being of another minority race or ethnicity. The average caseload was further comprised primarily of neglect concerns (60%), followed by physical abuse (24.74%), emotional abuse (11.53%), and sexual abuse (3.21%). All characteristics were similar between the full and condensed training groups, ps ≥ .11.

Approach to Data Analysis

Data analysis began with extraction of focus group themes through an iterative process, as guided by Morgan, Krueger, and King (1998). In phase one, university and community partners who were present at the focus groups discussed the themes that they subjectively heard during the focus groups. This helped ensure that community partners were able to provide feedback about themes that would inform subsequent iterations of theme identification. Time commitments precluded community participation in subsequent iterations. Thus, university partners (first and fifth authors) led the remaining iterations of theme identification. Transcripts were reviewed without note-taking, followed immediately by another read through, this time assigning broad descriptive codes with each change of speaker, such as “benefit,” “concern,” “experience,” or “request.” During the second phase of coding, specifiers were added to the existing codes. For example, “concern” was specified as “cultural concern” versus “concern regarding case complexity,” etc. Next, the first and fifth authors independently coded each line of transcription using the established codes. The need for additional refinement of codes was noted by both authors during this process. Thus, remaining iterations modified codes as necessary. Both focus groups were coded by consensus during the final phase.

The final phase identified 501 codes (note: lines sometimes reflected multiple themes; thus, the number of codes is not equivalent to the number of lines present in the transcription). Among these codes, six primary themes were identified. For example, the statement, “When I looked at your criteria and I didn't refer anybody because it was too limited,” was coded as a concern about criteria. Two primary themes were highly pervasive and so were broken down into nine secondary themes. For example, both of the following statements were coded as implementation concerns: “When I made the initial referral for both, this was one of my questions to them - if they were going to be willing to do the homework...” “I referred a couple of families, but they were very resistant...” While both statements were coded as implementation concerns, the secondary theme in the first statement was coded as homework concern, while the secondary theme in the second statement was coded as resistance. Three secondary themes were further broken down into five tertiary themes. For example, the statement, “...one of the issues I had is I think that people who are delivering the message should sound like the people who are going to receive the message...,” was coded as an implementation concern as its primary theme. Its secondary theme was coded as a practitioner concern (e.g., versus parental resistance). Finally, the tertiary theme was coded as issues of race/culture. Primary, secondary, and tertiary themes are described in detail in the Results section.

Themes were dummy coded at each level. For example, all implementation concerns were coded as a “1” for this variable, while all other statements were coded as a “0”; only those expressing concerns about persistence were coded as a “1” on that level two variable, while all other statements were coded as a “0”; etc. Theme prevalence is represented as percentage of codes within each theme at each level. The number of speakers that represented each theme is noted for descriptive purposes.

The final phases of analysis involved a series of quantitative analyses designed to determine which themes at the primary, secondary, and tertiary level were most pervasive, and whether pervasiveness of themes differed between or within focus groups. First, categorical themes (i.e.; six primary themes) were compared between groups by entering the categorical theme variable and focus group variable into χ2 tests. This step served to determine whether the two focus groups would most appropriately be analyzed as a single sample, versus two separate groups differing in level of training.

Significant χ2 tests were decomposed in two ways: (1) to determine which themes differed significantly between groups, and (2) to determine which themes were most prominent within the full and condensed training groups. Follow-up z-tests for proportions were conducted for all significant χ2 tests to determine whether the proportion of each theme differed significantly between and within groups (Cohen, 2001; Joose, n.d.). Bonferroni corrections were applied to control for Type I error.

Results

Six primary themes were noted, within which secondary and tertiary themes could be specified. Following is a description of each primary theme and the secondary and tertiary themes that could be specified, within higher order themes.

Primary Theme #1: Restrictive Criteria

Twenty-nine codes were identified that reflected concerns about criteria for inclusion in the feasibility study. Criteria of concern included restricting the age of the parent, exclusion of non-parental caregivers, and exclusion of fathers during early phases of the study. In addition, participants expressed concern that child age restrictions resulted in exclusion of children who were either too young or too old to qualify for these services.

Primary Theme #2: Implementation Concerns

Nearly three-hundred codes (N = 289) about implementing Triple P with CPS-involved families were identified. These codes could be categorized into five secondary themes, including (1) concerns about implementing homework (N = 19), (2) concerns that various practitioner characteristics may not fit well with CPS-involved families (N = 76), (3) concerns that Triple P services did not adequately address issues of parent resistance (N = 19), (4) concerns that Triple P was ill-equipped to account for the complexities inherent in CPS cases (N = 114), and (5) concerns that Triple P may not be sensitive to the cultural diversity represented amongst CPS-involved families (e.g., being disproportionately African American and living in the inner city; N = 61).

Three of these secondary themes were further broken down into tertiary themes. First, practitioner concerns were broken down into three tertiary themes: (1) issues of race/culture (e.g., practitioners may not look or act like CPS-involved families or may not be sensitive to families’ cultural needs; N = 27), (2) concerns that practitioners may not be aware of the complexities involved in CPS cases (see below for further description of case complexity issues) and may thus be ill-prepared to meet these complex needs (N = 18), and (3) concerns that practitioners may not pursue families with enough persistence to ensure their continued engagement in and successful completion of services (N = 18).

Second, case complexity was broken down into six tertiary themes: (1) concerns that Triple P does not adequately account for the impact of poverty on parenting (N = 7), (2) concerns that parent and/or child mental health problems may reduce the effectiveness of Triple P (N = 18), (3) concerns that parental chemical dependency may also interfere with the effectiveness of Triple P (N = 11), (4) concerns that cognitive limitations, including reading deficits and undisclosed/undetected developmental delay, may impair parents’ understanding of service content, particularly that which is related to workbook content and activities (N = 13), (5) concerns that Triple P does not account for complex family dynamics such as teenage parents, grandparents raising grandchildren, co-parenting in the presence of domestic violence, fluidity of some father figures being in and out of the home, and extended family member involvement in the family system (N = 35), and (6) concern that Triple P does not account for the impact of families’ trauma histories, including intergenerational sexual abuse, previous domestic violence, and children being exposed to the effects of chemical dependency (N = 22).

The fifth secondary theme within implementation was concern that insufficient cultural sensitivity may prevent families from benefiting from Triple P (N = 61). Participants described that CPS-involved families are disproportionately of minority background, and often live in the inner city. In addition to concerns about language barriers with immigrants, focus group participants expressed concern that workbook illustrations and video clips may not adequately represent minority families living in the inner city.

Primary Theme #3: Positive Experiences

Despite extensive implementation concerns, focus group participants were able to describe subjective experiences of success with Triple P (N = 26). Descriptions of such positive experiences were based on families reporting to caseworkers that they felt as though they had “gotten something out of” Triple P. Similarly, some caseworkers described seeing positive changes in families that they attributed to their participation in the intervention.

Primary Theme #4: Positive Expectations

In addition to reporting positive experiences with families benefiting from Triple P, some caseworkers described a belief that various characteristics of Triple P may be beneficial to CPS-involved families (N = 16). For example, some participants believed that engaging families in role plays as a “hands on” way of teaching parenting skills would likely be helpful. Some participants also believed that general clinical skills displayed by Triple P providers would likely be a good match to the parents’ needs (e.g., accepting that a given parent believes that harsh discipline is the best approach to managing misbehavior, while gently introducing the prospect that another approach may be considered).

Primary Theme #5: Service Modifications

When given the opportunity to make suggestions about the kinds of changes that could potentially be made to Triple P to enhance how well it meets families’ needs, participants had many ideas (N = 104). Discussion of service modifications was categorized into two secondary themes. The first category was direct statements of service modifications that participants believed would enhance the fit of Triple P to families’ needs (N = 31). Such direct requests for service modifications included more directly addressing the parent-child relationship, more directly addressing what the parent needs to do to avoid repeat referral to CPS, and ensuring that service providers are available to take on new families referred for services. This latter request reflected participants’ frustration that Triple P practitioners could not guarantee that they would always have space on their caseload to take on a new family referred by a caseworker.

The second category within the service modifications theme was contemplations of modifications that may be helpful. These contemplations were characterized by less confidence that such modifications would be helpful (N = 67). Two tertiary themes were identified within this category. First, focus group participants contemplated whether expanding intervention materials to include greater coverage of normal versus abnormal child development might be helpful (N = 22). Examples included teaching parents about how often a diaper might need to be changed and exploring the difference between “collar popping” (i.e., turning up the collar on one's shirt, rather than folding it down) and other behaviors of greater concern. Participants were especially interested in how helpful it might be to teach parents about the impact of trauma on child development and what might be done to remediate educational neglect. In addition to contemplations of addressing a wider spectrum of parenting needs, participants complained that Triple P focuses too much on the child and not enough on what the parent did that led to the CPS case (N = 10). Although these complaints were clear, participants did not elaborate, so no direct requests for modification could be coded.

Primary Theme #6: Knowledge Errors

On occasion, CPS workers requested service modifications that reflected errors in their knowledge of Triple P services (N = 37). For example, racial diversity is represented in workbook illustrations (though they correctly noted that inner city depictions are not well-represented). Participants also requested that families’ real life situations be used to illustrate how parenting skills might be applied, which is an existing component of both Triple P training and delivery of the intervention.

Between- and Within-group Comparisons

Preliminary χ2 tests were conducted to determine whether themes varied in pervasiveness. Significant variation was noted amongst primary themes χ2 (5) = 29.20, p < .01 (Table 1), secondary themes within the implementation theme χ2 (1) = 30.67, p < .01 (Table 2), and within both tertiary implementation themes (i.e. tertiary themes within practitioner characteristics χ2 (1) = 34.02, p < .01; and case complexityχ2 (1) = 51.59, p < .01; Table 3). Secondary themes within the service modifications theme did not vary p ≥ .70.

Table 1.

Pervasiveness of Primary Themes

Primary Theme Full Training (N = 279) Condensed Training (N = 229) z-test p
Concerns about Criteria* 9% a,b.c 3%a,b,c 2.76 .01
Concerns about Triple P Implementation 54% a,d,e,f,g 62% a,d,e,f,g 1.81 .07
Positive Experiences 7% d,h 4% d,h 1.45 .15
Positive Expectations* 5% e,i 1% e,j 2.54 .01
Knowledge Errors* 4% b,f,j 12% b,f,h,j,k 3.35 < .01
Service Modifications 22% c,g,h,I,j 20% c,g,I,k .55 .58
Open in a new tab

Notes: Between group comparisons:

*

indicates significant differences between groups at p ≤ .05 after Bonferonni correction. Within group comparisons: Values with the same superscript letters differ significantly within group, after Bonferonni correction ps ≤ .05.

Table 2.

Pervasiveness of Secondary Implementation Concerns

Secondary Themes within Implementation Full Training (N = 148) Condensed Training (N = 141) z-test p
Homework 5%a,b,c,d,e 9%a,b,c,d,e 1.34 .18
Practitioners -- -- -- --
    Including Culturally-Relevant Characteristics* 36%a,f,g 16%a,f,g,h 3.86 < .01
    Excluding Culturally-Relevant Characteristics* 30%h,i,j 2%bi,j,k 6.44 < .01
Parent Resistance 7%f,h,k,l 6%f,I,l,m,n .34 .73
Case Complexity 43%c,I,k,m,n 36%c,g,J,l,o 1.22 .22
Cultural Sensitivity -- -- -- --
    Excluding Culturally-Relevant Practitioner Characteristics* 10%d,g,m 33%d,h,m 4.78 < .01
    Including Culturally-Relevant Practitioner Characteristics* 16%e,j,l,n 48%e,k,n,o 5.85 < .01
Open in a new tab

Notes: Between group comparisons:

*

indicates significant differences between groups at p ≤ .05 after Bonferonni correction. Within group comparisons: Values with different superscript letters differ significantly within group, after Bonferonni correction ps ≤ .05.

Table 3.

Pervasiveness of Tertiary Implementation Concerns

Tertiary Themes within Implementation Full Training Condensed Training z-test p
Practitioner Characteristics N = 53 N = 23 -- --
    Issues of Race/Culture* 20%a,b 83%a,b 4.83 < .01
    Awareness of Case Complexity* 42%a 4%a 3.18 < .01
    Practitioner Persistence* 39%b 9%b 2.51 .01
Case Complexity N = 63 N = 51 -- --
    Poverty 13%a,b 0% -- --
    Mental Health 24%a,c 10%a 1.91 .06
    Chemical Dependency 20%d 0% -- --
    Cognitive Issues 16% 8%b 1.26 .21
    Family Dynamics* 4%c,d 65%a,b,c 6.66 < .01
    Trauma Histories 24%b 18%c .76 .45
Open in a new tab

Notes: The z-test could not be calculated for poverty or chemical dependency due to the absence of the tertiary specification within the condensed training group. Between group comparisons:

*

indicates significant differences between groups at p ≤ .05 after Bonferonni correction. Within group comparisons: Values with the different superscript letters differ significantly within group, after Bonferonni correction ps ≤ .05.

Between Group Comparisons

Follow-up z-tests were conducted to determine which themes differed significantly between groups (Tables 1-3). Concerns about criteria and statements of positive expectations were more pervasive in the full training group, while knowledge errors were more pervasive in the condensed training group. Implementation concerns and discussions of potential service modifications did not differ between groups.

Although implementation concerns did not differ significantly at the primary level, between group differences were noted for both practitioner characteristics and cultural sensitivity, at the secondary level (remaining secondary themes for implementation did not differ between groups; Table 2). Concerns about practitioner characteristics were more pervasive in the full training group, while the reverse was true for cultural sensitivity. This was true for both ways of analyzing practitioner characteristics (i.e., when retaining racial/cultural aspects of practitioner characteristics in the practitioner characteristics theme or considering them as part of the cultural sensitivity theme).

At the tertiary level within the implementation theme (Table 3), practitioner characteristics theme, the full training group expressed more pervasive concerns about practitioner's awareness of case complexity and practitioner persistence, while the condensed training group expressed more pervasive concerns about race/culture. Within the case complexity theme, family dynamics concerns were more pervasive in the condensed training group, while poverty and chemical dependency were noted only by the full training group.

Within Group Comparisons

In addition to between-group differences in themes, variations in the pervasiveness of themes within each group was also considered. At the primary level, implementation concerns were the most pervasive within both groups (Table 1). Specifically, concerns about how well Triple P addressed case complexities was amongst the most pervasive implementation concerns for both groups (Table 2). At the tertiary level, case complexity themes that were specific to family dynamics were the least pervasive theme in the full training group, but the most pervasive within the condensed training group.

Within the full training group, concerns about practitioner characteristics were similarly as pervasive as case complexity concerns, before recoding culturally-relevant characteristics to the cultural sensitivity theme (Table 2). In contrast, concerns about practitioner characteristics were less than half as pervasive as case complexity concerns in the condensed training group before recoding. These relative differences in the pervasiveness of practitioner characteristics versus case complexity maintained when recoding practitioner characteristics of race/culture to the cultural sensitivity theme.

At the tertiary level within practitioner characteristics (Table 3), concerns about racial/cultural characteristics was about half as pervasive as other practitioner characteristics for the full training group, while concerns about racial/cultural characteristics dominated practitioner characteristic concerns for the condensed training group. In addition to tertiary concerns about practitioner characteristics involving race/cultural characteristics, the condensed training group expressed high levels of concern about the extent to which Triple P, itself, was culturally sensitive (Table 2).

While implementation concerns were the most pervasive in both groups (with the above noted variations in which types of implementation themes were most pervasive within each group), service modifications were the second most pervasive primary theme for both focus groups. Given that secondary themes within the service modifications theme did not vary significantly according to the above-reported χ2-test, these results are considered for the total sample. Specific requests for service modifications were less than half as pervasive (32%) as were contemplations of potentially helpful service modifications (68%), z = 7.64, p < .01.

Discussion

Triple P is an EBP that has shown promise with maltreating parents. However, this program is not yet being implemented widely in community settings. CPS caseworkers are an integral part of the community system within which EBPs such as Triple P are being implemented. Their viewpoints on the barriers inherent in implementing Triple P with maltreating parents in the community were explored through semi-structured focus groups. A unique opportunity allowed for the examination of possible differences in caseworker views based on their level of familiarity with Triple P. Focus groups explored the extent to which caseworkers’ views of Triple P would vary as a function of training in the EBP. Given the over-representation of minority families involved in CPS for child maltreatment, the extent to which views would be particularly salient with respect to issues of cultural sensitivity was also examined. Focus group results revealed that concern about cultural sensitivity was the second most pervasive implementation theme expressed across both focus groups, following concerns about case complexity.

Concerns about cultural sensitivity were clearly more prevalent within the condensed training group. These concerns included practitioner characteristics, such as racial/ethnic matching to maltreating families. Although this concern is not specific to Triple P, evidence suggests that racial matching significantly enhances outcomes of EBPs, particularly for African Americans (Cabral & Smith, 2011; Campbell & Alexander, 2002), who represent the largest proportion of minority families in the CPS system. None of the Triple P practitioners were African American, which was of great concern to caseworkers. These concerns, however, were again limited primarily to the condensed training group.

While concerns about cultural sensitivity were greater in the condensed training group, both groups expressed similar levels of concern that Triple P does not account for families’ trauma histories (e.g., domestic violence, multi-generational sexual abuse). Such concerns may be valid, given evidence that trauma-related symptoms adversely impact both parenting practices (Gewirtz, Polusny, DeGarmo, Khaylis, & Erbes, 2010) and child development (Dinshtein, Dekel, & Polliack, 2011). It is unclear the extent to which parent training programs such as Triple P may overcome these adverse impacts of trauma.

Strengths & Limitations

This report represents the first known data to consider the role of caseworker perceptions of Triple P in its community-based implementation. CPS staff were actively engaged in the collaborative process through which these focus groups were planned, implemented, and analyzed. This collaboration with community entities is believed to enhance the likelihood that research will better meet the needs of the community (Allen et al., 2013). Further, focus group methodology took advantage of a unique opportunity to consider the extent to which level of caseworker training in Triple P may impact perceptions of the extent to which Triple P may meet the needs of maltreating families. Finally, the mixed methods reflected in this study are increasingly valued as necessary to best inform our understanding of how to best meet community needs (Novins et al., 2013).

The small sample size is a significant limitation. Differential attendance at focus groups by the full training versus condensed training group, and/or a self-selection bias may also have impacted results. The greater participation by the full training group may have resulted in a more accurate representation of that group's views than the condensed training group. Results are further limited to the extent that data were not available to determine whether perceptions of Triple P may have been associated with rates of referral to Triple P services versus services otherwise available in the community. Data were also not available to determine the extent to which community referrals may have been appropriate. Finally, results of the present research speak only to implementation of Triple P. It is unclear the extent to which these results may generalize to other empirically based parenting programs.

Conclusions & Future Directions

Much research on the effectiveness of interventions examines characteristics of the clients, therapists, and the interventions themselves, but there has been little focus on referral sources such as CPS caseworkers (Fixsen et al., 2009; Glisson & Green, 2011). Caseworkers’ collaboration with service providers is essential for maximal outcomes, so caseworkers’ perceptions of EBPs is a likely key to success. Thus, their training, expectations, and values should be taken into account when designing and evaluating interventions (Thomlison, 2003).

Results of the present research support and extend developing theory regarding the role of caseworkers when designing and evaluating interventions. Specifically, our results suggest that caseworker training may impact expectations of program success in real world practice. Such expectations may translate into referral practices that enhance or detract from successful implementation of EBPs.

Future research should seek to better understand caseworkers’ perceptions, and the manners in which these perceptions may impact EBP implementation. Such research should specifically seek to further explore the extent to which caseworker training may impact successful implementation of EBPs via their perceptions of intervention fit. Results of the present research suggest that, to the extent that caseworkers are minimally exposed to EBPs (a likely reality associated with typical resource constraints), caseworker trainers may need to highlight manners in which EBPs are sensitive to maltreating families’ diversity characteristics and trauma histories. Future research, as suggested herein, will inform recommendations for policies regarding the extent to which referral sources should be trained in the EBPs that they do not directly implement.

Acknowledgments

This research was supported by NIMH grant P20 MH085987 and pilot funds from the Center for Personalized Prevention Research.

References

  1. Aarons GA, Palinkas LA. Implementation of Evidence-based Practice in Child Welfare: Service Provider Perspectives. Administration and Policy in Mental Health and Mental Health Services Research. 2007;34(4):411–419. doi: 10.1007/s10488-007-0121-3. doi:10.1007/s10488-007-0121-3. [DOI] [PubMed] [Google Scholar]
  2. Allen B, Gharagozloo L, Johnson JC. Clinician Knowledge and Utilization of Empirically-Supported Treatments for Maltreated Children. Child Maltreatment. 2011;17(1):11–21. doi: 10.1177/1077559511426333. doi:10.1177/1077559511426333. [DOI] [PubMed] [Google Scholar]
  3. Allen ML, Hurtado GA, Yon KJ, Okuyemi KS, Davey CS, Marczak MS, Svetaz VM. Feasibility of a parenting program to prevent substance use among Latino youth: a community-based participatory research study. American Journal of Health Promotion: AJHP. 2013;27(4):240–244. doi: 10.4278/ajhp.110204-ARB-52. doi:10.4278/ajhp.110204-ARB-52. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Cabral RR, Smith TB. Racial/ethnic matching of clients and therapists in mental health services: a meta-analytic review of preferences, perceptions, and outcomes. Journal of Counseling Psychology. 2011;58(4):537–554. doi: 10.1037/a0025266. doi:10.1037/a0025266. [DOI] [PubMed] [Google Scholar]
  5. Campbell CI, Alexander JA. Culturally competent treatment practices and ancillary service use in outpatient substance abuse treatment. Journal of Substance Abuse Treatment. 2002;22(3):109–119. doi: 10.1016/s0740-5472(02)00221-0. [DOI] [PubMed] [Google Scholar]
  6. Chaffin M, Bard D, Bigfoot DS, Maher EJ. Is a structured, manualized, evidence-based treatment protocol culturally competent and equivalently effective among American Indian parents in child welfare? Child Maltreatment. 2012;17(3):242–252. doi: 10.1177/1077559512457239. doi:10.1177/1077559512457239. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Chaffin M, Friedrich B. Evidence-based treatments in child abuse and neglect. Children and Youth Services Review. 2004;26(11):1097–1113. doi:10.1016/j.childyouth.2004.08.008. [Google Scholar]
  8. Cohen BH. Explaining psychological statistics. 2nd ed. Wiley; New York: 2001. [Google Scholar]
  9. Connell CM, Vanderploeg JJ, Katz KH, Caron C, Saunders L, Tebes JK. Maltreatment following reunification: predictors of subsequent Child Protective Services contact after children return home. Child Abuse & Neglect. 2009;33(4):218–228. doi: 10.1016/j.chiabu.2008.07.005. doi:10.1016/j.chiabu.2008.07.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Crea TM, Crampton DS. The context of program implementation and evaluation: A pilot study of interorganizational differences to improve child welfare reform efforts. Children and Youth Services Review. 2011;33(11):2273–2281. doi:10.1016/j.childyouth.2011.07.012. [Google Scholar]
  11. Dingfelder HE, Mandell DS. Bridging the research-to-practice gap in autism intervention: an application of diffusion of innovation theory. Journal of Autism and Developmental Disorders. 2011;41(5):597–609. doi: 10.1007/s10803-010-1081-0. doi:10.1007/s10803-010-1081-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Dinshtein Y, Dekel R, Polliack M. Secondary Traumatization Among Adult Children of PTSD Veterans: The Role of Mother-Child Relationships. Journal of Family Social Work. 2011;14(2):109–124. doi:10.1080/10522158.2011.544021. [Google Scholar]
  13. Durlak JA, DuPre EP. Implementation matters: a review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology. 2008;41(3-4):327–350. doi: 10.1007/s10464-008-9165-0. doi:10.1007/s10464-008-9165-0. [DOI] [PubMed] [Google Scholar]
  14. Fixsen DL, Blase KA, Naoom SF, Wallace F. Core Implementation Components. Research on Social Work Practice. 2009;19(5):531–540. doi:10.1177/1049731509335549. [Google Scholar]
  15. Gewirtz AH, Polusny MA, DeGarmo DS, Khaylis A, Erbes CR. Posttraumatic stress symptoms among National Guard soldiers deployed to Iraq: Associations with parenting behaviors and couple adjustment. Journal of Consulting and Clinical Psychology. 2010;78(5):599–610. doi: 10.1037/a0020571. doi:10.1037/a0020571. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Glisson C, Green P. Organizational climate, services, and outcomes in child welfare systems. Child Abuse & Neglect. 2011;35(8):582–591. doi: 10.1016/j.chiabu.2011.04.009. doi:10.1016/j.chiabu.2011.04.009. [DOI] [PubMed] [Google Scholar]
  17. Hindley N, Ramchandani PG, Jones DPH. Risk factors for recurrence of maltreatment: a systematic review. Archives of Disease in Childhood. 2006;91(9):744–752. doi: 10.1136/adc.2005.085639. doi:10.1136/adc.2005.085639. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Hoagwood K, Olin SS. The NIMH blueprint for change report: research priorities in child and adolescent mental health. Journal of the American Academy of Child and Adolescent Psychiatry. 2002;41(7):760–767. doi: 10.1097/00004583-200207000-00006. doi:10.1097/00004583-200207000-00006. [DOI] [PubMed] [Google Scholar]
  19. Joose SA. [March 6, 2014];Two-proportion z-test calculator. n.d. Retrieved from http://in-silico.net/statistic/ztest.
  20. Karatekin C. A Survey of Organizations Serving Child Welfare-Involved Families and Children in Hennepin County, MN. Child and Adolescent Social Work Journal. 2013 doi:10.1007/s10560-013-0317-1.
  21. Katz LF. Child-centered practices for the courtroom and community: a guide to working effectively with young children and their families in the child welfare system. Paul H. Brookes Pub; Baltimore: 2011. [Google Scholar]
  22. Kazak AE, Hoagwood K, Weisz JR, Hood K, Kratochwill TR, Vargas LA, Banez GA. A meta-systems approach to evidence-based practice for children and adolescents. American Psychologist. 2010;65(2):85–97. doi: 10.1037/a0017784. doi:10.1037/a0017784. [DOI] [PubMed] [Google Scholar]
  23. Luongo G. Re-Thinking Child Welfare Training Models to Achieve Evidence-Based Practices. Administration in Social Work. 2007;31(2):87–96. doi:10.1300/J147v31n02_06. [Google Scholar]
  24. Maynard BR. Social Service Organizations in the Era of Evidence-Based Practice: The Learning Organization as a Guiding Framework for Bridging Science to Service. Journal of Social Work. 2009;10(3):301–316. doi:10.1177/1468017309342520. [Google Scholar]
  25. Morawska A, Sanders M, Goadby E, Headley C, Hodge L, McAuliffe C, Anderson E. Is the Triple P-Positive Parenting Program Acceptable to Parents from Culturally Diverse Backgrounds? Journal of Child and Family Studies. 2010;20(5):614–622. [Google Scholar]
  26. Morgan DL, Krueger RA, King JA, editors. Focus group kit. SAGE Publications; Thousand Oaks, Calif: 1998. [Google Scholar]
  27. Novins DK, Green AE, Legha RK, Aarons GA. Dissemination and implementation of evidence-based practices for child and adolescent mental health: a systematic review. Journal of the American Academy of Child and Adolescent Psychiatry. 2013;52(10):1009–1025.e18. doi: 10.1016/j.jaac.2013.07.012. doi:10.1016/j.jaac.2013.07.012. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Petrosino A. Mediators and moderators in the evaluation of programs for children. Current practice and agenda for improvement. Evaluation Review. 2000;24(1):47–72. doi: 10.1177/0193841X0002400102. [DOI] [PubMed] [Google Scholar]
  29. Prinz RJ, Sanders MR, Shapiro CJ, Whitaker DJ, Lutzker JR. Population-Based Prevention of Child Maltreatment: The U.S. Triple P System Population Trial. Prevention Science. 2009;10(1):1–12. doi: 10.1007/s11121-009-0123-3. doi:10.1007/s11121-009-0123-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Putnam-Hornstein E, Needell B, King B, Johnson-Motoyama M. Racial and ethnic disparities: A population-based examination of risk factors for involvement with child protective services. Child Abuse & Neglect. 2013;37(1):33–46. doi: 10.1016/j.chiabu.2012.08.005. doi:10.1016/j.chiabu.2012.08.005. [DOI] [PubMed] [Google Scholar]
  31. Rubin A. Programs and Interventions for Maltreated Children and Families at Risk: Clinician's Guide to Evidence-Based Practice. Vol. 9. John Wiley & Sons; 2011. [Google Scholar]
  32. Sanders MR. Development, Evaluation, and Multinational Dissemination of the Triple P-Positive Parenting Program. Annual Review of Clinical Psychology. 2012;8(1):345–379. doi: 10.1146/annurev-clinpsy-032511-143104. doi:10.1146/annurev-clinpsy-032511-143104. [DOI] [PubMed] [Google Scholar]
  33. Sue S, Zane N, Nagayama Hall GC, Berger LK. The Case for Cultural Competency in Psychotherapeutic Interventions. Annual Review of Psychology. 2009;60(1):525–548. doi: 10.1146/annurev.psych.60.110707.163651. doi:10.1146/annurev.psych.60.110707.163651. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Thomlison B. Characteristics of evidence-based child maltreatment interventions. Child Welfare. 2003;82(5):541–569. [PubMed] [Google Scholar]
  35. Thyer BA, Myers LL. The quest for evidence-based practice: A view from the United States. Journal of Social Work. 2010;11(1):8–25. doi:10.1177/1468017310381812. [Google Scholar]
  36. Weisz JR, Donenberg GR, Han SS, Weiss B. Bridging the gap between laboratory and clinic in child and adolescent psychotherapy. Journal of Consulting and Clinical Psychology. 1995;63(5):688–701. doi: 10.1037//0022-006x.63.5.688. [DOI] [PubMed] [Google Scholar]
  37. Westfall JM, Mold J, Fagnan L. Practice-based research--“Blue Highways” on the NIH roadmap. JAMA: The Journal of the American Medical Association. 2007;297(4):403–406. doi: 10.1001/jama.297.4.403. doi:10.1001/jama.297.4.403. [DOI] [PubMed] [Google Scholar]
  38. Whitaker DJ, Ryan KA, Wild RC, Self-Brown S, Lutzker JR, Shanley JR, Hodges AE. Initial implementation indicators from a statewide rollout of SafeCare within a child welfare system. Child Maltreatment. 2012;17(1):96–101. doi: 10.1177/1077559511430722. doi:10.1177/1077559511430722. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES