Caregiver-relevant perspectives from a multi-stakeholder collaborative advisory board on adapting a child mental health intervention to be delivered in child-welfare settings.

Abstract

Adapting evidence based mental health interventions (EBI) to be provided in child welfare (CW) settings by CW workers could reduce barriers to families receiving mental health care. In order to promote implementation success, the adaptation of EBIs should include the perspectives of those who deliver and those who receive the EBI. The following study uses qualitative methods to elicit and analyze caregiver-relevant perspectives and adaption recommendations from CW stakeholders about the 4Rs and 2Ss Strengthening Families Program, an EBI for youth disruptive behavior disorders, to be implemented in CW settings. Recommendations included adjusting curriculum to better fit the culture of recipients and conveying the importance of openness and respect to providers.

Background

Children involved in the child welfare (CW) system experience disproportionately higher rates of disruptive behavior disorders (DBDs), including hyperactive, oppositional, and/or aggressive behavior, compared to the general population (Administration for Children and Families (ACF), 2005; Burns, Phillips, Wagner, Barth, Kolko, Campbell, & Landsverk, 2004; Costello, Angold, Burns, Stangl, Tweed, & Erkanli, 1996; Hinshaw & Lee, 2003; Merikangas, Nakamura, & Kessler, 2009). However, many families involved in the CW system face significant barriers to accessing, engaging in, and being retained in child mental health services (Lau & Weisz, 2003;Warner, Malinosky-Rummell, Ellis, & Hanson, 1990, as cited in Hansen & Warner, 1994). A limited supply of qualified mental health providers, coupled with an elevated need for such providers (Gorman-Smith, Tolan, Henry, & Florsheim, 2000), is a frequent barrier to accessing services, especially in lower-income, urban communities (ACF, 2005; Asen, 2002; Burns et al., 2004). Other common barriers include transportation issues and lack of money, childcare difficulties, and conflicts between work/family and other services (Kemp, Marcenko, Hoagwood, & Vesneski, 2009). Stressors common to families involved in the CW system, such as caregiver substance abuse or domestic violence, have also been linked to reduced child mental health treatment compliance and success (Hurlburt, Nguyen, Reid, Webster-Stratton, & Zhang, 2013; Kerkorian, McKay, & Bannon, 2006; Leslie, Aarons, Haine, & Hough, 2007; Reyno & McGrath, 2006; Rishel, Greeno, Marcus, Sales, Shear, Swartz, et al., 2006). Further, families who are mandated to receive services, in particular, may not self-identify as having service needs, which can lead to difficulties in participation and retention (Dawson & Rooney; Berry, 2002; Rooney, 1992). Prior unsatisfactory experiences with the CW system or service providers themselves may discourage parents from engaging in other traditional service delivery systems (Kekorian et al., 2006; Palmer, Maiter, & Manji, 2006), as can stigma and negative perceptions about seeking care (Alvidrez, 1999; Snowden, 2001).

The child welfare system’s need for EBIs that strengthen families has only intensified with the passage of the federal Family First Prevention Services Act, which expands child welfare-involved families’ access to prevention services (e.g., mental health treatment, parent skill-based programs)—as long as those prevention services have evidence that they work (National Conference of State Legislatures, 2019). Behavioral Parent Training (BPT) EBIs (e.g., Incredible Years, Parent-Child Interaction Training), have been successful in reducing youth behavioral difficulties (Eyberg, Nelson, & Boggs, 2008; Kaminski & Claussen, 2017), and can also help avert future child maltreatment (e.g., Chaffin et al., 2004).

Few evidence-based interventions (EBIs) have been successfully implemented in CW contexts (Aarons, Fettes, Flores, & Sommerfeld, 2009; Chaffin, Silovsky, Funderburk, Valle, Brestan, Balachova, et al., 2004). The existing literature on EBIs that have been implemented in CW settings suggests significant implementation barriers, including caseworkers’ large caseloads, multiple responsibilities, and limited opportunities for sharing knowledge (Aarons, Hurlburt, & Horwitz, 2011; Collins-Camargo & Millar, 2012; Michalopoulos, Ahn, Shaw, & O’ Connor, 2012); limited funding for additional training, supervision, and other supports required for EBIs (Michalopoulos et al., 2012); and the potential need for modifications to adapt EBIs developed for behavioral health settings to their new settings (McKleroy, Galbraith, Cummings, Jones, Harshbarger, Collins, et al., 2006). Additionally, parenting programs have frequently demonstrated low participation rates, underscoring the need for strong consumer engagement and collaboration in their design and implementation (Sanders & Kirby, 2012). To increase access to child mental health EBIs for those involved in the CW system, innovative approaches are needed to address the limited supply of qualified mental health providers available to serve youth in many socioeconomically disadvantaged communities and as well as the difficulties associated with engaging families into parenting services.

Our study draws upon task-shifting strategies emerging from the developing world as well as implementation science to guide the process of implementing a child mental health EBI in CW services. Task-shifting provides a practical and cost-efficient overarching framework for facilitating EBI implementation where there are shortages of trained specialized professionals (WHO, 2008). Prior task-shifting efforts in low-, middle, and high-income countries have been found to successfully increase EBI access as well as reducing behavioral health symptoms in children and adults (Barnett et al., 2018; Patel et al., 2011). Task-shifting includes providing EBI’s in new settings already accessed by the population of interest (Patel, Chowdhary, Rahman, & Verdeli, 2011; Verdeli, Clougherty, Bolton, Speelman, Lincoln, Bass, et al., 2003; Barnett, Gonzalez, Miranda, Chavira, & Lau, 2018; Bhutta, Lassi, Pariyo, & Huicho, 2010). In CW, community-based organizations (CBOs) already providing services to biological/permanent caregivers and their children to reduce maltreatment risk and prevent out-of-home placement (referred hereafter as placement prevention services) are well-situated to house EBIs that address child mental health, thereby improving service access and utilization among their consumers. In this case, behavioral health EBIs would be delivered by existing CBO staff, namely CW caseworkers, who typically lack advanced specialized behavioral health training (Aarons, Fettes, Sommerfeld, & Palinkas, 2012; Gopalan, Hooley, Winters, & Stevens, 2019).

Secondly, task-shifting involves modifying EBIs for lay providers who do not have specialized behavioral health training, but who are members of the communities they serve or are already employed in existing low-resource settings (Patel et al., 2011; Verdeli et al., 2003; Barnett et al., 2018). Modifications to the EBI may also be guided by the exigencies of the new implementation context, as well as provider and consumer needs. In our study, we further drew upon implementation science, which highlights the importance of partnerships between researchers and knowledge users (e.g., providers, consumers) to promote successful implementation and outcomes. Specifically, our study uses the Practical, Robust, Implementation, and Sustainability Model (PRISM; Feldstein & Glasgow, 2008) to guide EBI modification efforts. PRISM is an implementation determinants framework that outlines multiple key domains that support the successful uptake of new practices, including the perspectives of both consumers and providers regarding their views on the intervention to be implemented (e.g., usability, perceived burden) as well as identifying important characteristics (e.g., existing staff capacities; child/parent mental health needs) which could impact implementation. Implementation success relies on the intervention being perceived by consumers and providers as acceptable, feasible, and sustainable in their new environments (Feldstein & Glasgow, 2008). The input of caregivers with their own lived experience of being involved in CW services is, therefore, critical to ensure that the EBI meets the needs of caregivers involved in CW services (Sanders & Kirby, 2012). Such information would also inform the third step in the task-shifting framework, where behavioral health specialists or EBI consultants provide training, and regular supervision and monitoring, to non-specialized providers on the modified EBI (Patel, et al., 2011; Verdeli, et al., 2003; Barnett, et al., 2018; Bhutta, et al., 2010).

A key gap in the literature, however, lies in understanding caregivers’ perspectives on how EBIs should be adapted to meet their needs, circumstances, and characteristics. Proactively soliciting and considering their perspectives may increase the likelihood of implementation success, and subsequent consumer engagement, by identifying potential barriers and modifying the EBI to address those barriers prior to initial installation.

Current Study

This paper focuses on the first phase of a larger study, which examined the feasibility and acceptability of implementing a child behavioral health EBI in a CW context designed to avert out-of-home placement. Following the task-shifting framework, this study focuses on the first stage of modifying the EBI so that it could be implemented by CW providers. Given the salience of consumer and provider perspectives as identified in implementation science, we utilized an Integrated Knowledge Translation (IKT; Jull et al., 2017) approach. IKT strategies primarily focus on enhancing the applicability of research findings through the integration of both researcher and knowledge user (e.g., consumer, provider) expertise (Jull et al., 2017). To elicit feedback from both CW staff and caregivers, we formed a Collaborative Advisory Board (CAB). The CAB comprised members from various roles within CW placement prevention services: administrators, supervisors, case planners, and biological/permanent caregivers. Previous work reported on CW staff perspectives and recommendations on the types of modifications that should be made to the EBI to help ensure successful implementation in a CW setting (Gopalan, Hooley, Winders, & Stephens, 2019). The current paper builds on this prior work by focusing on CW stakeholder perspectives relevant to caregiver concerns.

The 4Rs and 2Ss for Strengthening Families Program (4Rs and 2Ss; see Chacko et al., 2015 for more complete descriptions) is the intervention we used in this study. It is a multiple family group-based behavioral parent training program involving caregivers and their children. Held over the course of 16 weekly sessions, this EBI focuses on four empirically supported family-level influences on, and processes for treating, child behavioral issues: rules, responsibilities, relationships, and respectful communication (Chorpita & Daleiden, 2009; Garland, Hawley, Brookman-Frazee, & Hurlburt, 2008). The 4Rs and 2Ss session content also addresses two key factors that impact behavioral health service and outcomes: stress and social support (Kazdin, 1995; Kazdin & Whitley, 2003; Wahler & Dumas, 1989). Also, by virtue of having multiple families present, group members can efficiently enhance their support networks and reduce social isolation (Harris & Fallot, 2001; McKay, Gonzales, Stone, Ryland, & Kohner, 1995).

The 4Rs and 2Ss program was also designed to address the more common logistical barriers to utilizing child mental health services, including those that frequently impact CW-involved families. Families are provided childcare for children 5 and under, transportation expenses, a meal at each session, as well as extensive phone outreach between sessions to address concrete treatment barriers (Dawson & Berry, 2002; McKay & Bannon, 2004; McKay et al., 2011). 4Rs and 2Ss also addresses attitudinal barriers to participation and/or retention, such as stigma around mental health and prior negative service experiences (McKay & Bannon, 2004; McKay et al., 2011). Specifically, group processes normalize family difficulties, validate families’ existing expertise in problem solving, and encourage families to help one another in the change process (Gopalan & Franco, 2009). Each 4Rs and 2Ss session involves psychoeducation, didactic discussion, hands-on activities, in vivo skill development and rehearsal, as well as weekly homework exercises which help families increase their capacity to develop new resources. Such content and processes also align with many trauma-informed service recommendations (Freeman, 2001; Harris & Fallot, 2001).

The current paper provides a complementary perspective to previous work examining CW staff perspectives (Gopalan et al., 2019). This study elicits caregiver relevant concerns from CAB members which included caregivers to help ensure the successful implementation of the 4Rs and 2Ss. This study has two main aims which seek to address a gap in the literature around incorporating caregiver perspectives in treatment adaptations:

1. Identify caregiver-relevant perspectives, informed by caregiver individual characteristics as well as their understanding of the 4Rs and 2Ss intervention, on the potential facilitators and barriers to implementing the 4Rs and 2Ss in CW prevention services.

2. Draw upon caregiver-relevant perspectives to identify potential modifications to the 4Rs and 2Ss and/or strategies that could help overcome implementation barriers.

Method

Research Design

The methodology for the current study follows an Integrated Knowledge Translation (IKT; Jull et al., 2017) approach. IKT motivated our development of a Collaborative Advisory Board (CAB) which included researchers, providers, and parent consumers of CW services. We formed the CAB to elicit stakeholder perspectives on the intervention and receive guidance on necessary intervention and service-delivery modifications per the PRISM framework (Feldstein & Glasgow, 2008). This study employed a qualitative design using ethnographic observation (Emerson, Fretz, & Shaw, 2011) and document analysis (Bowen, 2009) to ascertain perspectives relevant to caregivers on a modified version of the 4Rs and 2Ss. This study is based on data gleaned from CAB meetings during the first phase of multiphase study examining the adaptation and implementation of the 4Rs and 2Ss in CW settings (Gopalan, 2016; Gopalan et al., 2019). The University of Maryland IRB approved this study and monitored its execution.

Participants

The CAB comprised research staff (n=4) with expertise in the 4Rs and 2Ss and who had multiple years of experience in the CW sector, caseworkers (n=2), supervisors (n=2), and an administrator (n=1) from a CW CBO, and caregivers who had CW involvement histories (n=3). In total, there were four researchers, five CW staff, and three caregivers. Caregivers were recruited from a local child welfare advocacy organization, the Child Welfare Organizing Project (CWOP; https://cwop.org/) in order to elicit the perspectives of caregivers with prior CW experience. Providers were recruited from a one community-based organization (CBO) providing placement prevention services and which had agreed to serve a site for examining the feasibility and acceptability of the modified EBI. The caregivers in the CAB were female, with ages ranging from 35–44 years old, one caregiver identified as African American and the other two as Latina. CW staff CAB members (n = 5) were also female, ages ranged between 25–65 years old. CW staff CAB members identified as White/Caucasian (n=1, 20%), Black/African American (n=2, 40%), and Latina/Hispanic (n = 2, 40%). Most CW staff had at least five years of experience, and the balance (n = 2, 40%) had more than 10 years. CW staff CAB members all had a bachelor’s degree, and two had an advanced degree (n=2, 40%).

CAB meeting procedures

The full CAB (i.e. researchers, CW staff, and caregivers) held seven meetings over a six-month period beginning in the summer of 2014. They met on Saturday mornings in a university research center in New York City, each session lasting approximately 4 hours. The CAB members received the following incentives at each meeting: public transportation reimbursement, breakfast and lunch, childcare, and a $20 gift card. Research staff prepared and facilitated each session.

The meetings’ overarching focus was to adapt the 4Rs and 2Ss based on CAB member feedback so that it could be delivered in CW settings by frontline CW workers in placement prevention services, as well as be responsive to clients’ needs. Researchers identified those facets of the 4Rs and 2Ss intervention which could not be modified to ensure fidelity to key model components as determined by the treatment developer, (e.g., content focused on 4Rs and 2Ss, provision of homework, practicing skills in vivo, discussion/activities within and between families). The PRISM framework guided our meetings and organized CAB member feedback related to those components of the intervention which could be modified. CAB members reviewed each session of the 4Rs and 2Ss treatment manual and completed PRISM-guided, feedback worksheets. CAB members subsequently provided recommendations on training, supervision, and implementation. Additional details on these procedures are reported elsewhere (Gopalan, 2016; Gopalan et al., 2019).

Data collection procedures

The research staff trained three, first-year graduate social work students to be non-participating notetakers during the CAB meetings. Each notetaker received training on field note writing and ethnographic observational methods (Emerson et al., 2011; Wolfinger, 2002). During the CAB meetings, notetakers captured member feedback and suggestions. Notetakers also completed a post-session worksheet that provided PRISM-informed headings to organize CAB member feedback. Research staff reviewed notetakers’ raw field notes and the worksheets after each session. The universe of artifacts we used for the analysis of this paper included notetaker field notes and worksheets, CAB members feedback worksheets, pictures of other written products generated during the meetings, and relevant email communications (n=106 artifacts).

Data coding procedures

Our analysis involved both deductive and inductive approaches (Barbour, 2001; Hsieh & Shannon, 2005; Onwuegbuzie, Dickinson; Leech, & Zoran, 2009) and proceeded in three phases. We began the first phase of our analysis using a check-coding procedure (Miles & Huberman, 1994), two research staff coded a segment of artifacts using a priori PRISM constructs (deductive approach). We assessed inter-rater agreement using Microsoft Excel. The coders, along with a third arbiter, discussed discrepancies, refined constructs and definitions, and added emergent themes (inductive approach). We incorporated an inductive approach to capture salient themes not included in the PRISM constructs. The coders then applied the updated codebook to a new segment of artifacts. This process continued until the coders reached theme saturation and inter-rater agreement reached 90%. At which point, the coders uploaded all artifacts and the final codebook into Dedoose Version 7.0.23.

The second analysis phase included coding all the artifacts based on the finalized codebook. Two coders divided the artifacts and independently coded them. To strengthen the credibility of the findings, coders randomly selected 20% of the artifacts to co-code, which yielded an 89.66% inter-rater agreement. Given that the universe of artifacts included email communications and other relevant documents to the project, we wanted to ensure that we co-coded 20% of CAB derived artifacts. As such, we engaged in a second round of co-coding (with a new second coder). The resultant inter-rater agreement was 89.1%.

The third analysis phase involved meaning making across artifacts (Strauss & Corbin, 1990). We selected artifacts which had been assigned at least one of three code combinations: “client and characteristics”, “client and intervention perspectives”, and “client barrier”. The first author reread all the artifacts with these codes and extracted themes from them which addressed caregiver-relevant perspectives of the intervention and any recommendations for modifications based on those perspectives, co-authors corroborated the themes.

To support the trustworthiness of our findings we engaged in prolonged engagement with the CAB members. We held seven meetings over six months. This span of time permitted us to build a trusting relationship with the CAB members and allowed us to understand the CAB member context with greater specificity. Secondly, three non-participating note takers independently captured the process and content of our CAB meetings. Their notes were reviewed at the conclusion of each session by other research team members who attended the meeting to ensure accuracy. The combination of CAB-member-generated data and note-taker-data allowed us to triangulate our findings. Third, during our analysis we independently co-coded 20% of the artifacts to promote credibility. As a result, we have confidence that these processes strengthened the trustworthiness of our subsequent findings.

Findings

The meaning making phase generated nine themes which we organized into three main categories: Caregiver characteristics, caregiver perspectives of CW workers providing a modified intervention, and suggested intervention modifications. Supporting data included CAB members’ words verbatim as well as quotes from note-takers’ ethnographic observations. A strength of this study was the ability to capture data from multiple sources and viewpoints, which we utilized to modify the 4Rs and 2Ss intervention.

Caregiver characteristics

This category includes literacy concerns, competing external demands, and internal resources as they relate to parents who are involved in the CW system and participate in an evidenced-based intervention.

Literacy concerns.

CAB members expressed concerns about caregivers’ ability to access the intervention materials. More specifically, CAB members were concerned about caregivers who cannot read, and for those caregivers whose primary language is not English.

Caregiver: “Language accessibility is a major concern. There should be accessibility for Spanish speakers and for those with literacy issues.”

Competing external demands.

CAB members expressed that families may not participate in the intervention because CW-involved parents would need to prioritize activities that meet the family’s basic needs over participating in the intervention. A note taker summarized this sentiment from a caregiver who conveyed “when she was struggling as a parent and involved in the child welfare system, she did not feel motivated to invest time in activities whose payoff was in the future. Dealing with the immediate present always trumped that. She believes many parents will feel this way, and this is a barrier to achieving buy-in.”

CAB members suggested providing gift cards and transportation assistance to allay caregivers’ financial concerns and to support buy-in from those participating in the intervention. A note taker summarized a caregiver’s recommendations when they “suggested [providing] gift cards… since so many parents are struggling to afford enough food and other essentials.” Another note taker summarized a conversation between CAB members, “Parents agreed that transportation and food represent potential financial burdens for participants, and so snacks and metro cards should be provided.”

Internal resources.

Caregivers stressed the importance of acknowledging the mental health of participants and considering it an important factor to caregiver/family participation.

Caregiver: “…parent mental health issues can be a barrier… [a] parent has to be in a decent enough place mental health wise to participate in the intervention”.

CAB members also discussed the importance of acknowledging the strengths of families.

Caregiver: “Parents sometimes can’t do it [identify the strengths of their family] due to lack of confidence, and case planners need to validate the families.”

Caregiver perspectives of CW workers providing a modified intervention

This category reflected CAB members’ perspectives relevant to caregivers on implementing the 4Rs and 2Ss program by CW workers. Sub-themes included stigma, prior negative experiences with CW services, CW staff as mandated reporters, and the CW staff approach to the intervention.

Stigma.

Caregivers identified two types of stigma. The first was stigma associated with being a part of the CW system.

Caregiver: “…some people may not want their friends in their business, may not want people to know what is going on; so they try to handle things on their own, so I think it might be difficult for some families to integrate [into the child welfare system]”.

The second type of stigma involved child mental health issues. A note taker captured a CAB members’ feedback “… [there is] stigma surrounding child mental health issues and the societal propensity for parent-blaming.”

Prior negative experiences with CW services.

Caregivers expressed concerns about prior negative experiences with CW services and how this may impact parents’ ability to participate in an intervention provided by CW staff.

Caregiver: “…many clients arrive at agencies with prior trauma from past interactions with service providers.”

A note taker captured a similar sentiment from a caregiver who conveyed many agencies do not reflect the people that they serve, and fail to meet the needs of people with less education or with language barriers, and offered examples of agency workers being unaware of major issues that clients are facing, and of families being passed along from agency to agency. She noted that many people do not want to ask for help because they are afraid of the system.”

CW staff as mandated reporters.

A universal concern from caregivers was the 4Rs and 2Ss facilitators’ dual role. From their perspective, the 4Rs and 2Ss group facilitators were both the providers of the intervention and mandated reporters connected to a CW agency. More specifically, caregivers expressed concerns about their ability to share experiences candidly in a group setting where the person delivering the intervention is a mandated reporter and linked to CW services. One caregiver expressed their concerns stating,

Caregiver: “I just want to say [the 4Rs and 2Ss] is about expressing feelings and you can’t because of the mandated reporters.”

A note taker also captured a CAB caregiver’s perspective on the matter who suggested, “…having [the 4Rs and 2Ss] facilitated by mandated reporters could be experienced as a barrier for many participants. [Caregivers] collectively expressed the need for families to have realistic expectations about the group. A safe space needs to be created, including emphasizing that the facilitator is a mandated report[er]. There needs to be a safe space to talk to someone who is not a mandated reporter. This is important because sometimes in group, a lot of hurt comes out.”

CW staff approach to the intervention.

Caregivers expressed concerns about how CW staff would approach the intervention and how the staff would be perceived by parents. First, caregivers were concerned about parents not wanting a CW staff member without children delivering a parenting intervention.

Caregiver: “You can’t save what you don’t know.”

A note taker captured this theme from a caregiver CAB member “…the [caregiver] disliked taking a parenting class from a twenty-something childless woman from outside their community with no experience in child welfare… people enter the social service field wanting to save others they have little knowledge of.” The theme was further expounded upon by a note taker who captured the overall tenor of the discussion, “People laughed at the idea of people without kids telling parents how to parent. Group members appeared to identify with the sentiment, ‘Go have kids and then tell me about parenting skills.‘”

The second concern centered on the need for trust and respectful communication from CW staff.

Caregiver: “… [Child welfare staff] underestimate how educated their clients are. They can act judgmental and superior. This is alienating for clients.”

Caregiver: “Workers should talk with people not at them. Respectful communication at all levels is important. It’s important that parents have a voice and are heard when expressing concerns.“

Intervention modifications

This category includes suggested modifications to the intervention, and has two subthemes: materials and activities of the intervention and the importance of openness and respect

Intervention materials and activities.

Caregivers expressed a desire to acknowledge the different cultural backgrounds of the group. One CAB member suggested having exercises that focused on culture, emphasizing the group is a safe space. Caregivers also strongly supported modifying the 4Rs and 2Ss curriculum to address culturally defined parenting values. Caregivers described the importance of activities which create a sense of connection with their children,

Caregiver: “[I liked] the activities where I had to be the role of my son and my son had to act like me. It was funny!”

A note taker captured this sentiment from a caregiver, “[The client] shared that when she and her son participated in [4Rs and 2Ss], they’d played a [game] in which they had to demonstrate their knowledge of each other. It was fun for them and got them talking. [The client] appeared happy when reflecting upon this activity. She had a big smile on her face. I sensed that she felt good remembering the experience. It seemed to me that the activity had made her and her son feel closer.” CAB members also suggested to post words of encouragement on the walls. One caregiver provided examples such as the phrases “ ‘triumph over trauma’ and ‘it’s not how you fall, but how you get up’.”

The importance of openness and respect.

Caregivers expressed that group composition is an important consideration.

Caregiver: “[Group members] really need to get to know [each other], with an emphasis on openness and respect. [The] group needs to be consistent.''

CAB members also felt the intervention should focus on shared experiences. One caregiver provided an example,

Caregiver: “Oh, she’s been traumatized too, she’s alright.”

Caregivers unanimously felt that transparency was imperative. A note taker captured a caregiver’s comment, “Everyone (I took this to mean everyone in the agency or involved in the intervention) needs to know what the process is… [and] an explanation of roles within the intervention would be helpful”. Caregiver CAB members shared that it was also important for facilitators to get to know the families who will be participating in the intervention. A caregiver elaborated,

Caregiver: “The facilitator should know the history of each parent/family and that parents and families with similar histories and concerns should be grouped together. We need to make sure their [meaning the families who are participating in the intervention] issues are addressed.”

Caregiver CAB members unanimously felt it was important to have facilitators with shared experiences. A notetaker captured this sentiment, “They want assistance from a peer level... Real-world experience and relatability on the part of the provider are helpful in creating meaningful connections with clients. Clients want to work with people they perceive as peers. Using providers with firsthand experience is helpful. The parent advocate role is really important. Clients/parents find it hard to be ‘told’ what to do from a worker without kids.”

Discussion

The current study explored CW stakeholder perspectives relevant to caregiver views on the 4Rs and 2Ss intervention being delivered in the context of CW placement prevention services. This study contributes to the current literature by exploring the unique perspectives of caregivers as it relates to the adaptation and implementation of an intervention within CW services. Task-shifting and the PRISM implementation determinants framework guided this study and posited that inclusion of caregiver perspectives is a critical component to the success of EBI implementation.

The findings from this study are largely consistent with the extant literature on caregiver experiences with child welfare services and programs, particularly with respect to perceived barriers. Caregivers’ competing external demands and literacy levels have been raised as potential barriers to engagement and participation in child welfare services in other studies (Marcenko, Newby, Lee, Courtney, & Brennan, 2009; Lewis, Feely, Seay, Fedoravicis, & Kohl, 2016; Pinna, Lewis, Karatekin, Lamb-Onyinga, Hirillal, & Jones, 2015). Caregivers’ prior negative experiences with traditional service systems (Kemp et al., 2009; Kerkorian et al., 2006; Anderson, 2006), and feelings of fear, shame, and stigma around their circumstances (Scholte, Colton, Casas, Drakeford, Roberts, & Williams, 1999), may also deter them from participating in services and can disrupt the formation of a positive client-worker relationship. Given these challenges, the value of peer support in child welfare--specifically, having caregivers who have successfully navigated the child welfare system mentor or support caregivers currently in the system--has been widely acknowledged in the literature (Berrick, Young, Cohen, & Anthony, 2011; Nilsen, Affronti, & Coombes, 2009). Not only are peer supports likely to share similar life circumstances and characteristics with system-involved caregivers, but they are also unburdened by the power differential that CW providers face and they may also provide a greater level of motivation and support simply by virtue of having been through the system themselves (Berrick et al., 2011; Nilsen et al., 2009). Salient connections with the extant literature include: caregiver competing demands, their wariness of CW providers, the value of shared experiences with peer-providers and addressing literacy barriers.

Competing Demands

CAB caregivers felt strongly that competing demands to meet basic needs such as food and shelter often supersede parents’ ability to engage in a new intervention. This finding is consistent with the existing research on child welfare service engagement, which suggests that the survival needs of parents who are economically disadvantaged often take precedence over their service needs (Marcenko et al., 2009; Lewis, Feely, Seay, Fedoravicis, & Kohl, 2016). When parenting programs actively help families address these stressors, this can free up the personal resources necessary for caregivers to focus on their parenting (Ingoldsby, 2010) and facilitate their continued participation in the program (Wong, Roubinov, Gonzales, Dumka, & Millsap, 2013).

Caregiver wariness of CW providers

Caregivers reported feeling wary of CW providers as group facilitators. This was in connection to prior negative experiences with the CW system, fears over workers’ role as mandated reporters, and concerns about workers being mismatched to parents or devaluing their experiences and knowledge. Parents involved in CW often have long and sometimes contentious histories with service systems (Kemp et al., 2009). Those prior negative experiences shape future service interactions, which can result in mounting alienation from treatment systems if negative encounters persist (Kerkorian et al., 2006; Anderson, 2006). This is especially true for families of color who may already distrust CW organizations due to the historical overrepresentation of children of color and traumatic generational experiences with the CW system (Evans-Campbell, 2006; Jimenez, 2006).

High-quality parent training programs are characterized by collaborative, equal relationships between parents and program staff (Powell, 1988). Yet, CW workers may already be perceived by parents as authority figures as opposed to collaborators. Parents involved in CW often struggle with issues of power and powerlessness when interacting with CW workers (Kemp et al., 2009; Stephens, Gopalan, Acri, Bowman, & McKay, 2018). In addition to being required to comply with certain court- or agency-mandates, parents may also have had experiences with CW workers who use authoritarian or coercive interactional styles to obtain parental compliance (De Boer & Coady, 2007). This can lead to anger and mistrust among parents as well as intense feelings of powerlessness around the power that CW workers and the system wield against them (Lalayants, 2013; Mirick, 2013).

The value of shared experiences

CAB members explained that parents want to work with someone who has the lived experience of parenting such as facilitators who themselves are parents. This finding fits within the broader literature of parent advocates (aka peers, veteran parents, etc.) (Berrick et al., 2011; Lalayants, Baier, Benedict, & Mera 2015). Caregivers involved in CW who have participated in peer-led support groups have shared that they were able to be their authentic self and could speak candidly (Lalayants et al., 2015). They felt the peer-led group was a judgment free zone (Lalayants et al., 2015). Participants in peer-led interventions have found them to be sources of emotional and material support, encouragement, comfort and hope (Berrick et al., 2011; Lalayants et al., 2015). The qualities of these peer-led groups mesh with the CAB members hopes and desires for the 4Rs and 2Ss. The CAB members clearly articulated that the presence of a CW worker introduces a degree of guardedness and suspicion that are counter to the impact a peer would have. Notwithstanding the positive endorsement for peer-involvement from the CAB and the literature, others have noted that peer involvement in CW services is different than other social service settings and requires particular care and attention (Nilsen, Affronti, Coombes, 2009). For example, the CW context introduces the notion of mandated involvement, and peers may be experienced by parents as an extension of that mandate.

Addressing low literacy levels

Participants also identified concerns over the accessibility of the intervention materials, specifically for parents with low literacy and those with limited English proficiency. Other studies examining the implementation of parent training EBIs in CW settings have raised concerns about how low literacy may impair parents’ understanding of workbook content and activities (e.g., Pinna et al., 2015).

Low caregiver health literacy, which is defined as “the degree to which individuals have the capacity to obtain, process, and understand basic health information and services needed to make appropriate health decisions” (Ratzan & Parker, 2000; U.S. Department of Health and Human Services [U.S. DHHS] (n.d.), is one possible explanation for the documented under-enrollment in programs known to improve child health (Pati, Mohamad, Cnaan, Kavanagh, & Shea, 2010) and is associated with poor preventive care behaviors and poor child health outcomes (Sanders, Federico, Klass, Abrams, & Dreyer, 2009). Strategies to improve health literacy include providing “plain language” written and oral communications which break down complex information into understandable chunks, using simple language and defining technical terms, and prioritizing important points so that they come first. The “audience” for those materials should then test the materials before, during, and after they are developed (U.S. DHHS, n.d.).

Comparison of CW staff and parent advocate perspectives

A companion publication highlights the perspectives of CW staff as it relates to implementing the 4Rs and 2Ss intervention in a CW setting (Gopalan et al., 2019). Both the provider and caregiver perspectives included concerns about the CW dual role of engaging families in the 4Rs and 2Ss intervention while at the same time being a mandated reporter. In order to address this concern, CAB members recommended transparency about CW staff’s dual role, and providing an opportunity for family members to share their concerns. CAB members also agreed that families have life demands that may negatively influence their ability to engage in a new intervention. To address this concern, CW staff and caregivers agreed that the length of the intervention should be truncated to 8 sessions over the course of 2 months, and that a supplementary home visit guide should accompany the intervention as well as concrete support like food, transportation, and childcare. While a major theme from the CW staff perspective was the feasibility of implementing a new intervention due to workload and a lack of direct staff support, a major theme from the caregiver perspective was the importance of openness and respect. Caregivers emphasized the value of understanding the life experiences of clients and how a strengths-based approach is needed when addressing life-long patterns of parenting practices. For resource allocation, a CW staff perspective focused on ensuring external resources were available (e.g. funding, training, consultation), whereas a caregiver perspective focused on client’s having the appropriate internal resources (e.g. mental health stability, feeling respected). To better support CW workers’ knowledge and support of clients’ internal needs, two training modules were added (trauma informed care and child development). These modules aimed to further help providers deliver the intervention in an empathetic and sensitive way.

Limitations and implications

The findings of this study should be interpreted within the context of its limitations. Our study is exploratory and is focused on one specific EBI and its characteristics differ in certain ways from other disruptive behavior disorder interventions. Additionally, our sample viewed the intervention from their service system context which may differ from others. These differences could be linked to geography-based variation. CW system policies and practices vary between municipalities and perspectives vary depending on the position of the person within the CW service continuum (e.g. child protection, foster care, placement prevention). CAB members in this study spoke from the perspective of placement prevention services. The transferability of our findings should be viewed within the context of our sample’s reality. Notwithstanding the study’s limitations, the findings generated several practice implications.

To our knowledge, this is the first study of its kind to integrate task-shifting and implementation science approaches to facilitate implementation of a child mental health EBI in a CW context. As indicated earlier, we acknowledged the need to balance maintaining fidelity to the core components of the EBI as defined by the treatment developer, while also tailoring it to promote initial installation and consumer engagement. Our approach entailed identifying from the outset those key components required to maintain treatment fidelity (i.e., content related to 4Rs and 2Ss, homework, practicing skills in vivo, discussion/activities during session). Such an approach may facilitate future task-shifting efforts of other EBI’s into novel settings.

Our study also underscores the importance of consumer perspectives when designing and adapting interventions (Sanders & Kirby, 2012). Parenting programs tend to have low participation rates, which result in limited population-level improvements in children’s social, emotional and behavioral difficulties. Adopting a broader public health view to enhance the reach of parenting programs requires strong consumer engagement and collaboration (Sanders & Kirby, 2012). Consumer engagement involves the inclusion of parents’ knowledge and experiences in the development, delivery, evaluation, and funding of parenting programs. Parent perspectives contribute to the ecological fit of interventions, thereby better meeting parent’s needs. More collaborative approaches with consumers are needed because, to date, parents have had little involvement in parenting program design, evaluation, implementation, and scale-up (Sanders & Kirby, 2012). Our study provides examples of how including and considering a caregiver perspective improved the fit of the 4Rs and 2Ss for a CW setting.

Our next steps include exploring the feasibility, acceptability, and fidelity of the adapted 4Rs and 2Ss intervention in a CW setting. After we completed phase 1 of this project, we modified the 4Rs and 2Ss intervention and implemented it in a placement prevention program (phase 2 of the study). We are in the process of analyzing those results to determine the feasibility and acceptability of the adapted intervention from the perspectives of participating CW staff and parent consumers.

Conclusion

CAB members provided us with important insights on how clients would perceive the 4R2S and provided guidance on how the intervention could be adapted to better fit clients’ needs and context. These suggestions led to modifications of the intervention and a pilot implementation. Our next step includes analyzing the success of the modified intervention’s implementation. Children involved in CW experience higher prevalence rates of disruptive behavior disorders (Administration for Children and Families (ACF), 2005; Burns, Phillips, Wagner, Barth, Kolko, Campbell, & Landsverk, 2004; Costello, Angold, Burns, Stangl, Tweed, & Erkanli, 1996; Hinshaw & Lee, 2003; Merikangas, Nakamura, & Kessler, 2009), and they encounter substantial barriers receiving adequate mental health services (Lau & Weisz, 2003; Warner, Malinosky-Rummell, Ellis, & Hanson, 1990, as cited in Hansen & Warner, 1994). Implementing EBIs in CW settings could reduce these barriers. However, few EBIs have been successfully implemented in CW settings (Aarons, Fettes, Flores, & Sommerfeld, 2009; Chaffin, Silovsky, Funderburk, Valle, Brestan, Balachova, et al., 2004). Including the caregiver perspectives about the EBIs is key to their successful subsequent implementation (Feldstein & Glasgow, 2008).