Feasibility of improving child behavioral health using task-shifting to implement the 4Rs and 2Ss program for strengthening families in child welfare

Abstract

Background

Children whose families are involved with child welfare services manifest disproportionately high levels of behavioral difficulties, which could be addressed in community-based organizations providing services to prevent out-of-home placement. Unfortunately, few evidence based practices have been successfully implemented in child welfare settings, especially those originally delivered by mental health providers. Given that such settings typically employ caseworkers who lack prior mental health training, this is a significant barrier to implementation. Consequently, the overall aim of the current study is test the feasibility of shifting a mental health intervention from specialized services to community-based organizations. It uses task-shifting and the Practical, Robust, Implementation, and Sustainability Model (PRISM) to implement an evidence based intervention to reduce child behavior difficulties, originally provided by mental health practitioners, so that it can be delivered by caseworkers providing placement prevention services to child welfare-involved families. Task-shifting involves (1) modifying the intervention for provision by non-mental health providers; (2) training non-mental health providers in the modified intervention; and (3) establishing regular supervision and monitoring by mental health specialists.

Methods

This study uses the 4Rs and 2Ss Program for Strengthening Families, a multiple family group service delivery model to reduce child behavior difficulties, as the example intervention. This intervention has had prior success with child welfare-involved families. The proposed study objectives are: (1) to tailor the content, training, and supervision of the intervention for delivery by caseworkers serving child welfare-involved families; and (2) to assess the feasibility and acceptability of the modified intervention. Mixed quantitative and qualitative methods will assess feasibility and acceptability from key stakeholders (caseworkers, supervisors, administrators, caregivers). In Phase I, a Collaborative Advisory Board will be convened (1) to modify the intervention to be delivered by caseworkers in placement prevention service settings; and (2) to develop training and supervision protocols for caseworkers. In Phase 2, the modified intervention will be pilot tested for delivery by n=4 caseworkers to n=20 families receiving placement preventive services (where children manifest behavior problems). Mixed quantitative/qualitative methods will be used to assess feasibility and acceptability.

Discussion

This protocol will be of particular interest to agency administrators, program managers, and researchers interested in developing and testing cross-setting implementation guidelines for similar evidence based practices.

BACKGROUND

Children who remain at home with their permanent caregivers following a child welfare investigation manifest disproportionately higher rates of behavior problems (e.g., hyperactive, oppositional, disruptive, and/or aggressive behavior¹) compared to national rates^2,3. The current study focuses on this subgroup as it represents the majority of youth involved in child welfare services within the United States⁴, but who are less likely to utilize child mental health services compared to youth placed into foster care⁵. Untreated, such behavioral problems are a costly (up to 10 fold increase) ⁶ public health concern as they also create additional service needs due to increased risk for future maltreatment^7,8, out-of-home placement⁹, high school dropout¹⁰, and delinquency¹¹. Effective evidence-based practices (EBPs), such as behavioral parent training interventions^12,13 known to successfully reduce child behavior difficulties, are needed particularly for those families who are identified as high risk, but where there remains the possibility of averting a traumatizing out-of-home placement, as well as reducing the risk of other future maladaptive outcomes.

Many such behavioral parent training programs are offered in child mental health settings. However, reduction of maltreatment risk and improving family stability through parent training fall well within the purview of community-based organizations (CBOs) which provide a comprehensive array of placement prevention services (also known as ‘In Home Family Preservation’, ‘Preventive Services’) for families mandated or referred by child welfare authorities following maltreatment investigations^14,15, as well as a small proportion of families with similar difficulties voluntarily seeking services¹⁶. Rather than refer families to separately housed (and often overburdened¹⁷) child mental health providers, CBOs are logical platforms for effective interventions to reach child welfare involved children with behavior problems. At the same time, many such EBPs were originally designed to be delivered by advanced mental health providers (e.g., Parent-Child Interaction therapy¹³), a substantial implementation barrier, given that the typical child welfare workforce includes a sizable proportion of caseworkers lacking advanced specialized mental health training^18,19.

Despite recent efforts to implement EBPs developed for small clinics in child welfare settings ^14,20, there is limited guidance available about successful strategies for implementing EBPs from one service setting to another^21,22. In addition to lacking ancillary psychiatric support or providers with extensive mental health backgrounds, caseworkers have sizeable workloads and multiple responsibilities, with few available avenues to support knowledge sharing^23,24. CBOs also differ from child mental health settings with regards to perceived roles and responsibilities among staff, consumer characteristics, fiscal and regulatory requirements, the capacity of organizations to integrate innovation into practice, as well as state-specific licensure and certification laws which may prohibit formal mental health treatment by non-licensed individuals. As a result, modifying intervention processes to align with caseworkers’ scope of practice may be necessary as well as subsequent evaluation of the adapted model’s feasibility, acceptability, and effectiveness^22,25. In addition, partnering with community stakeholders (e.g., consumers, providers) and EBP experts around adapting intervention processes and materials, increasing agency capacity, and developing collaborative partnerships, can leverage differing stakeholder competencies in the service of promoting successful implementation and outcomes^21,22,26.

The current study attempts to increase guidance regarding cross-setting adaptations and implementation processes. Specifically, task-shifting strategies²⁷ and the Practical, Robust, Implementation and Sustainability Model (PRISM²⁸) are utilized to modify the 4Rs and 2Ss for Strengthening Families Program (4R2S), an evidence-based multiple family group service delivery model to reduce child behavioral difficulties, for implementation in CBOs serving families involved in placement prevention services. The modified intervention will subsequently be pilot-tested for feasibility and acceptability.

Task-Shifting

Endorsed by the World Health Organization (WHO) for use by developing countries with shortages in mental health professionals, task-shifting involves redistributing tasks from professionally trained workers to those with less training and fewer qualifications²⁷. As illustrated in Figure 1, successful task-shifting efforts^29–32 in the developing world have involved (1) modifying the intervention to be delivered by non-mental health workers in their existing settings (e.g., simplifying language, length, service delivery modality), (2) training non-mental health workers, and (3) providing consistent supervision for task-shifted non-mental health workers by competent mental health providers. As a result, task-shifting strategies have successfully reduced adult depression using Cognitive Behavior Therapy and Interpersonal Psychotherapy in Pakistan, Uganda, India, and South Africa by a host of lay-level providers^29,33–35. Most importantly, the beneficial effects were achieved by continuously supervised³⁵ workers with no mental health background and relatively short training (as little as 2 days^31,32,34). Within the United States, task-shifting from highly trained professionals to providers with less training and experience has been used in efforts to improve dental care in rural settings (NIH Grant R36HS019117-01A1), promote healthy body image among college undergraduates³⁶, provide behavioral parent training for low-income families in Head Start³⁷, increase access to services for aging individuals with dementia³⁸, as well as numerous efforts shifting tasks from physicians to lesser skilled health professionals (e.g., nurses) to increase access to medical services^39–42. To the author’s knowledge, task shifting has yet to be used to implement EBPs from child mental health to the child welfare service sector, where mental health specialists are not typically employed. In the current study, task-shifting will provide an overarching conceptual framework to translate mental health clinicians’ skills and strategies so that they can be used by caseworkers while maintaining compliance with licensure-related restrictions on scope of practice.

Figure 1.

Task-Shifting Strategies

Successful task-shifting efforts to increase access to mental health treatment which in the developing world involve: (1) Modifying the original intervention so that it can be delivered to non-mental health specialists and in the contexts within which they work; (2) Training non-mental health specialists in the modified intervention; (3) Providing regular supervision with a mental health specialist^29–32

PRISM

Specific modifications to the intervention, development of training and supervision protocols, and measurement will be further guided by PRISM, a practical, robust, and prescriptive model built upon multi-disciplinary implementation models. Table 1 documents the PRISM domains and elements of interest to maximize implementation success, and identifies potential areas of modification for the current study. PRISM emphasizes multiple domains that should be considered in order to promote implementation success. Specifically, PRISM domains will specify aspects within the task-shifting approach (i.e., intervention modification, training, and supervision) that will be addressed to support implementation success across CBOs. Moreover, PRISM domains guide measurement of feasibility and acceptability in the proposed study. Although the PRISM framework has been frequently cited within the implementation literature as informing studies, the author is unaware of any studies which utilize PRISM as a conceptual framework informing implementation in the child welfare service sector. Moreover, PRISM is utilized in the current study as a framework for intervention adaptation into a new setting such that the resulting modified 4R2S intervention is maximized for subsequent implementation success. Such use of an implementation framework may be increasingly necessary to guide basic intervention development in order to increase the likelihood that newly developed interventions are integrated into everyday practice. This study focuses on PRISM domains emphasizing intervention perspectives, recipient characteristics, and the external environment. Once feasibility and acceptability are established, the next stage in this line of research will be to conduct a larger scale study incorporating multiple CBOs, addressing the PRISM domain focused on implementation and sustainability infrastructure, and examining implementation and intervention effectiveness outcomes.

Table 1.

PRISM Domains, Elements, and anticipated modification/development focus

PRISM Domains	PRISM Elements	Anticipated Modification/Development Focus
		Intervention Modifications	Training	Supervision	Othera
Intervention perspectives - Organization	Perceived ease of use and usability	x	x	x
	Perceived adaptability	x		x
	Ability to observe results	x		x
	Burden on frontline staff workload and cost	x	x	x
	Barriers of frontline staff for delivery and access	x	x	x
	Relevance to frontline staff scope of practice	x	x	x
	Flexibility with frontline staff tasks	x
	Perceived familiarity of intervention by organization staff		x	x
	Awareness of the intervention within the organization		x		x
	Organization staff agreement with intervention focus, processes, and outcomes	x
	Perception of frontline staff to achieve anticipated outcomes through intervention delivery		x	x
	Inertia of previous practice		x	x
	Frontline staff self-efficacy and motivation to deliver intervention		x	x
	Perceived ability of intervention to engage staff and clients	x	x	x	x
	Organization readiness for the intervention				x
	Alignment with organization mission and change capacity	x			x
	Need for inter-department coordination	x			x
	Strength of evidence base for clinical target area		x
Intervention Perspectives - Client	Perception of intervention’s ability to benefit client regardless of clients’ stage of change	x
	Perception of intervention’s ability to provide client choices	x
	Addresses barriers to client’s ability to follow advice and set collaborative goals/action plans	x
	Seamless transition between program elements	x			x
	Intervention addresses barriers to service use and access	x			x
	Minimizes client burden	x
	Intervention ability to set collaborative goals and action plans with clients	x
	Ability to provide client feedback on successes and failures	x
	Concerns about confidentiality	x			x
External Environment	Community resources for referrals	x	x	x	x
	Fiscal and regulatory requirements	x			x
	Payor satisfaction	x			x
	Competition amongst other organizations	x			x
Recipient characteristics - Organization	Use of existing staffing and capacities	x	x	x	x
	Clinical risk management policies	x	x	x	x
	Supervision structure			x
	Culture of “risk-taking”				x
	Organizational health	x	x	x	x
	Management support and communication				x
	Shared goals and cooperation				x
	Data and decision support tools				x
Recipient characteristics - Client	Mental health needs	x	x	x
	Knowledge and beliefs	x	x	x
	Demographic characteristics	x	x	x
	Competing Demands	x	x	x
Implementation Infrastructure	Performance Data				x
	Dedicated Implementation Team				x
	Adopter training and support		x	x	x
	Relationship and communication between adopters and bridge researchers				x
	Adaptable protocols and procedures	x
	Facilitation of sharing of best practices				x
	Plan for sustainability				x

Note:

^aMarketing with organization, workload restructuring, organizational infrastructure

The 4Rs and 2Ss for Strengthening Families Program

Finally, this study utilizes the 4Rs and 2Ss for Strengthening Families Program (4R2S) to reduce child behavior difficulties as an example EBP (also known as “Multiple Family Groups for children with disruptive behavior disorders”). This program is currently listed on the Substance Abuse and Mental Health Services Administration’s (SAMHSA) National Registry of Evidence-based Programs and Practices (NREPP; http://nrepp.samhsa.gov/ProgramProfile.aspx?id=41#hide1). 4R2S consist of weekly sessions of multiple family groups (6–8 families) convened over 4 months, where engagement is promoted via extensive phone outreach, childcare, transportation expenses, and a meal provided at each session. Core treatment components integrate 2 decades of research regarding (1) caregiver engagement in child mental health treatment, as well as (2) behavioral parent training and family therapy strategies that address empirically supported family-level influences on disruptive behavior disorders (See Table 2). This intervention is an ideal example of an EBP to use in this study as 4R2S (1) has already been successfully delivered using facilitator teams which include at least one mental health clinician and a non-professional parent advocate (parent consumers of child mental health services) ^43–48; (2) prioritizes engagement and retention of low-income, minority families by targeting logistical (i.e., lack of transportation & childcare, conflicting demands) and perceptual barriers (i.e., stigma) to service use^49–51, (3) is manualized for straight-forward delivery by community-based providers, (4) exemplifies similar EBPs well-known to reduce child behavior difficulties, and (5) does not require specialized placement (i.e., Multidimensional Treatment Foster Care⁵²), or costly space/equipment requirements (i.e., Parent-Child Interaction Therapy¹³).

Table 2.

Summary of empirically supported family-level influences on ODD and CD

4R2S Target	Empirically supported family/parent skill	4R2S Goals
Rules79–87	Family Organization Consistent Discipline	Clarify Rules, consequences, rewards
Responsibility81,87,88	Family interconnectedness Positive behavioral expectancies	Clarify responsibilities, expectations, supports needed, contributions acknowledged
Relationships88–93	Family Warmth Within family support, time spent together	Schedule for positive family interaction
Respectful communication85,94,95	Family communication, family conflict	Listening and talking skills for parents and children
Stress50,51	Parenting hassles and stress, life events	Identification of stressors undermining family change, promotion of positive exchanges
Social support49,50	Social isolation	Within family and external support plan

Moreover, 4R2S has demonstrated benefits for child behavioral difficulties, functional capacities, and family processes^{43,45–48,53,54} in a recently completed effectiveness trial involving a sample of 320 low-income, predominantly minority (Black/African American, Latino) adult caregivers and children (ages 7–11) evidencing Oppositional Defiant Disorder or Conduct Disorder. Forty percent of participant caregivers indicated having prior or current child welfare involvement at baseline. Compared to participants receiving services as usual (e.g., individual therapy, family therapy), at post-test participants reported significantly reduced child disruptive behavior difficulties (Cohen’s d = 0.35) and increased child social skills (Cohen’s d = 0.32). At six-months’ follow-up, participants who received 4R2S continued to report significantly reduced child behavior difficulties (Cohen’s d = 0.34), as well as decreased functional impairment on peer relations (Cohen’s d = 0.27), when compared to participants receiving services as usual^43,53. Families involved in child welfare services within this sample manifested similar rates of attendance compared to families not involved in child welfare services⁵⁵. Moreover, families involved in child welfare services who received 4R2S reported significantly reduced child behavior difficulties (Cohen’s d = 0.61), and functional impairment on peer relations (Cohen’s d = 0.44) at six-months follow-up, when compared to child-welfare involved families receiving services as usual⁵⁶.

Primary objectives

The overall aim of the current study is test the feasibility of shifting a mental health intervention from specialized child mental health services to community-based child welfare organizations.

Objective 1

The first objective of this study uses a task-shifting framework and PRISM to modify the content, training, and supervision of 4R2S for delivery by child welfare caseworkers in a CBO providing placement prevention services to families and their children with behavior problems. Additionally, task-shifting and PRISM will guide any changes to existing CBO caseload structure and workload in order to promote successful implementation of the modified 4R2S.

Objective 2

The second objective of this study is to assess the feasibility and acceptability of the modified 4R2S in a CBO providing placement prevention services.

METHODS

Phase I (Objective 1)

Participants and settings

To accomplish Objective 1, relevant stakeholders invited to consult on the Collaborative Advisory Board (CAB) will include n=2 caseworkers, n=2 supervisors, n=2 agency administrators from CBOs providing placement prevention services; n=3 parent consumers of placement prevention services. Additional CAB members will include research staff who have had extensive clinical experience conducting the 4R2S intervention. Specifically, the CAB will meet regularly over a period of 4–6 months to modify the 4R2S intervention to create guidelines for ensuring intervention quality and feasibility based on existing task-shifting literature^{27,29–32,34,35,57}.

Modification of the 4R2S intervention

The CAB will be charged with (1) modifying the existing 4R2S intervention for delivery by CBO caseworkers and identifying any potential changes needed to be made within the CBO to support implementation, as well as (2) developing a protocol for training and (3) a mental health supervision framework of CBO caseworkers. This process will be informed by the task-shifting framework (see Figure 1) and the following PRISM domains: Intervention perspectives (e.g., perceived burden, usability/adaptability, barriers), Recipient characteristics (e.g., existing staffing capacities, child/family mental health needs, competing demands and barriers), and External environment (e.g., child welfare authority performance indicators, community resources). Research staff will take detailed written field notes of the CAB process (e.g., decision-making, final products), and summarize raw written notes into a structured observation guide^58,59 with pre-specified headings based on task-shifting strategies as well as PRISM domains and elements (e.g., Domain I: Considering the Intervention from the organization’s perspective) to assist with making subsequent modifications to the intervention, training, and supervision.

CAB sessions are structured to first provide orientation to the overall study. Strategies include team-building exercises, didactic review of study background and procedures, as well as didactic and experiential training in the initial 4R2S intervention curriculum sessions. CAB members are subsequently instructed to provide verbal and written feedback that addresses task-shifting strategies and PRISM domains and elements for each session. Secondly, research staff will engage CAB members in a series of discussions focused on addressing the overall 4R2S intervention structure and processes, implementation into the CBO context, as well as caseworker training and supervision. Discussion points are informed by task-shifting strategies and PRISM domains focused on intervention perspectives, recipient characteristics, and external environment. Finally, CAB members will review and provide feedback on all research materials (e.g., recruitment materials, surveys, focus group guides) to ensure readability and acceptability for potential participants.

Subsequent modifications to the 4R2S intervention, training, and supervision will incorporate CAB feedback, methods and findings from prior studies^{43,44,53–55,60} as well as collaboration with 4R2S developers to ensure core concepts and processes are maintained during the modification process. Recommendations from the CAB to alter existing caseload and/or workload structures will be negotiated between research staff and CBO administration in order to maximize implementation success.

Training for case workers

Training for caseworkers will integrate recommendations from the extant literature on task shifting^{27,29–31,34,35}, implementation science⁶¹, and training for EBPs^62,63. As a result, training will include a mixture of (1) didactic instruction on child mental health issues and related family factors, engagement and group facilitation skills, and safety protocols for potential medical and psychiatric emergencies; (2) active and participatory learning strategies (e.g., practice, modeling, role-play, and vicarious learning); (3) provision of an easy-to-follow manual; and (4) ongoing supervision by 4R2S experts. As the original 4R2S intervention were co-led with mental health clinicians, many of whom had substantial expertise in group and family work, supervision with caseworkers will initially focus on adherence to the 4R2S intervention model. However, as caseworkers in the current study will not have had prior mental health training, supervision protocols will further incorporate Stoltenberg’s Integrated Developmental Model⁶⁴, which will allow supervisors to monitor and adjust the quality of supervision according to supervisee development (i.e., changes in self and other awareness, motivation, autonomy). For example, supervision in the early part of the task-shifted 4R2S intervention may focus on increasing caseworkers’ confidence and motivation, managing anxieties and frustrations, and providing a high level of structure and direction regarding the clinical skills needed to deliver 4R2S (e.g., conducting group-level interventions, managing child and adult behavior within group sessions) as well as adherence to the intervention model. As caseworkers become more proficient and confident, supervision is likely to shift so that caseworkers develop greater understanding of their clients’ world from cognitive and affective perspectives.

Following the completion of all modifications and development of training/supervision, all revised research materials and procedures will be re-submitted for approval at all participating Institutional Review Boards (IRBs).

Phase II (Objective 2)

In Phase II, n=4 caseworkers at a participating CBO will be trained to deliver 4R2S to n=20 caregivers/n=20 youth receiving placement preventive services at CBOs and whose children manifest disruptive behavior difficulties. Quantitative and qualitative approaches will be used to gather information from caseworkers, supervisors, agency administrators, and caregivers to assess feasibility and acceptability.

Population

This study will consist of 4 purposively selected samples: n=4 caseworkers, n=4 supervisors, n = 2 administrators, n = 20 youth with behavior difficulties, and their caregivers (n=20). Sample sizes for caregivers and youth account for an estimated 20% attrition rate (overestimating the 10% attrition from the 4R2S effectiveness study^43,53), so that each 4R2S group will contain at least 8 families. Although larger sample sizes were considered, smaller samples are sufficient in order to obtain detailed data to establish initial feasibility and acceptability. Such information will be subsequently utilized to revise the modified 4R2S prior to testing in a larger scale study. A purposive sampling strategy was chosen rather than random selection due to the small sample sizes and desire to enroll engaged participants who can provide important information on feasibility and acceptability.

Caseworkers Inclusion Criteria

Eligible caseworkers (age 21 and older) include n=4 caseworkers who are (1) employed in a CBO contracted to provide placement prevention services; (2) English-speaking; (3) have completed a bachelor’s-level degree; and (4) have had no prior mental health training (e.g., post-graduate training and/or employment as a mental health clinician, clinical licensure).

CBO Supervisors Inclusion Criteria

Eligible CBO supervisors (age 21 and older) include n=4 supervisors who (1) supervise the caseworkers enrolled in the current study; (2) are English-speaking; and (3) employed at the CBO participating in this study. CBO supervisors are not required to have prior mental health training.

CBO Administrators Inclusion Criteria

Eligible CBO Administrators (age 21 and older) include n=2 administrators who (1) hold an executive leadership role within the CBO participating in the study; (2) are English-speaking; and (3) employed at the CBO participating in this study.

Identified Child Inclusion Criteria

Eligible youth include n=20 children (hereafter referred to as “Identified Child”) whose permanent caregiver receives placement prevention services at a CBO. Multiple methods will be utilized for screening children who manifest a range of disruptive behavior difficulties who might not be selected if relying on a single screening strategy. This study will enroll English-speaking children ages 7–11 (based on original 4R2S effectiveness study criteria) who meet one or more of the following inclusion criteria: (1) Caregiver report of the presence of serious disruptive behavior difficulties using the Disruptive Behavior Disorders Rating Scale (DBD) ⁶⁵ (e.g., meeting symptom criteria for Oppositional Defiant or Conduct Disorders); (2) History of out-of-home placement in the past year due to the child’s disruptive behavior difficulties. As a result, these youth may include those temporarily placed in foster care but returned to their permanent caregivers at the time of initial consenting.

Caregivers Inclusion Criteria

Eligible caregivers (hereafter referred to as “Legal Guardian/Primary Caregivers”) include up to n = 20 adult caregivers (age 21 and older) (1) who are receiving placement prevention services, (2) have a child meeting above criteria, and (3) are English speaking. If more than 1 caregiver or child is present and willing to participate in the study, they will both be consented, but only 1 caregiver will be the data reporter for the family. Research Staff will record the number of families screened and subsequently enrolled.

Exclusion criteria

Potential family (e.g., caregiver, identified child) participants will be excluded from the study if they present with significant cognitive impairment (as reported by CBO staff) that interferes with understanding the informed consent process. Moreover, participants with emergency psychiatric needs that require services beyond those within an outpatient setting (e.g. hospitalization, specialized placement outside the home), needed care will be secured, rather than study participation. However, if participants are able to remain in the permanent home following an emergency psychiatric evaluation, they may be eligible to participate.

Recruitment and screening procedures

To recruit caseworkers, supervisors, and agency administrators for Phase II, research staff will make multiple presentations at staff and management meetings. Visually appealing fliers will be provided to CBO staff to give to potential participant families (Identified Child, Legal Guardian/Primary Caregiver). Additionally, research staff will have a strong on-site presence at the CBO to provide as-needed informational sessions on the study. All interested individuals may contact research staff directly or allow their contact information to be released to research staff. Finally, research staff will provide in-depth information about the study as well as obtain informed written consent (adult participants) and assent (child participants). Once potential family participants (Identified Child, Legal Guardian/Primary Caregiver) have provided written consent, research staff will administer a screening questionnaire to determine if caregivers and youth are eligible for the study. Of those families deemed eligible to participate in the study, research staff will inform them that additional primary caregivers and siblings may attend 4R2S groups, and will need to be consented/assented as well. Legal Guardian/Primary Caregivers can establish individual meetings with research staff in order to consent/assent additional primary caregivers (aged 21 and older, additional caregiver to Identified child) and siblings (6–18 years old, sibling to Identified child).

Setting

The proposed setting is a publicly funded CBO offering placement prevention services to families. This is a different CBO than those employing CAB members from Phase I, allowing for consideration of a greater diversity of contextual factors then using the same CBO that employed CAB members. Convenience sampling was utilized, such that research staff approached the state child welfare authority regarding implementing Phase II, who subsequently disseminated study information to five major jurisdictional authorities in the state. After meeting with several representatives from each jurisdiction, the current CBO was chosen as the sole research site due their ability to implement the study within the time constraints of the current grant funding. The placement prevention program, called “In Home Family Preservation”, focuses on stabilizing families and reducing the likelihood of foster care entry. This is accomplished through the direct provision, coordination, or referral to services such as counseling, medical and psychological evaluation, parenting classes, and entitlement assistance to families referred as a result of maltreatment allegations, as well as families voluntarily seeking services.

Measures

Measures and related study constructs are presented in Table 3. In addition to project-developed measures, the proposed study also uses measures that were previously incorporated in the 4R2S effectiveness study^43,53. The DBD rating scale assesses if children meet DSM 5⁶⁶ diagnostic criteria for Oppositional Defiant Disorder, or Conduct Disorder. Psychometrics of the DBD indicate acceptable test-retest reliability before and after treatment (r = 0.49 – 0.61) and good internal consistency (Cronbach’s alpha=0.82–0.92). As a measure of concurrent validity, the correlation between the DBD and measures of impairment range from 0.79 – 0.93^65,67. The Kazdin Barriers to Treatment (KBT) scale is a useful measure for understanding the specific barriers to accessing treatment. In addition to looking at the overall average score and comparing to the project-defined cutoff levels, individual items which may be most endorsed by caregiver participants (e.g., “My child refused to come to session”) can be examined in order to inform subsequent intervention modification. Internal consistency for the overall score for experimental participants from the 4R2S effectiveness study was high (Cronbach alpha = 0.94)⁵⁵. The Metropolitan Area Child Study (MACS) Treatment Program Satisfaction scale further provides a quantitative measure of participant acceptability for the task-shifted 4R2S intervention. Among experimental participants from the 4R2S effectiveness study, good internal consistency was noted at the mid-point and immediately following the 4R2S intervention (Cronbach alpha = 0.88 – 0.95) ⁵⁵.

Table 3.

Measurement and Analysis of Objective 2

HF: High Feasibility; HA: High Acceptability
Study Construct	PRISM Domain	Quantitative measure	Sample Qualitative Questions
Demographics	Recipient	Project-developed surveya,c,d,e; i	Not applicable
Organizational Readiness	Recipient	Organizational Readiness for Change (ORC)a,d,e; i	How did characteristics of your agency affect implementation of the modified intervention? a,d,e, iii
Feasibility	External environment	CW performance indicators: % of enrolled families whom CW performance indicators for casework contact goals are met within most recent 6 month period b,iii	How were you able to meet requirements by external agencies by using the modified intervention?a,d,e, iii
Feasibility	Recipient	Client characteristics: Participant Flow: % of children/caregivers meeting inclusion criteria among those screenedb,iii Organizational capacities (ability of caseworkers to deliver 4R2S with fidelity): Caseworker Fidelity Ratingsb,ii HF: % of components scored as “Partially met” or “Fully Met”	What helped or got in the way of delivering the modified intervention as it was designed? a,d,e, iii
Feasibility	Intervention perspectives	Client perspectives Attendance logsb,iii HF: % of children/caregivers who attend ≥80–100% of sessions Kazdin Barriers to Treatment (KBT), c,iii HF: % of participants with average score ≤ 2 Organization perspective Lyons Acceptability, feasibility, appropriateness scale (LAFAS)–feasibility subscalea,d,e; iii HF: % of participants with average score ≥ 4	How feasible was it to implement the modified intervention? What were the challenges? What helped? a,d,e, iii Please describe things that influenced your decision to participate or not participate in the modified intervention? c,iii
Acceptability	Intervention perspectives	Client perspectives Metropolitan Area Child Study (MACS) Treatment Program Satisfaction Scalec,iii HA: % of participants with average score ≥ 3 Organization perspective on intervention LAFAS questionnaire – acceptability and appropriateness subscales a,d,e, iii HA: % of participants with average score ≥ 4 Evidence Based Practice Attitude Scalea,d,e,i, iii (EBPAS) HA: % of participants with average score ≥ 3 at post-intervention	What facilitated/hindered your satisfaction with the modified intervention? a,c,d,e,iii What are the benefits/challenges of using task-shifting to implement EBPs in CW setting? a,d,e, iii

Informant:

^aCaseworkers,

^bResearch Assistants,

^cCaregivers,

^dSupervisors;

^eAdministrators

Timing:

ⁱPre intervention;

ⁱⁱOngoing,

ⁱⁱⁱPost-intervention

Additional measures are also incorporated in the current study. Subscales of the Organizational Readiness for Change (ORC) ⁶⁸ will measure training needs, adaptability, orientation to change, stress, and efficacy among CBO staff. The Evidence Based Practice Attitudes Scale (EBPAS) assesses how CBO staff would respond in the future to implementing 4R2S and other EBPs. As reported in a series of articles^69–71, the EBPAS demonstrates moderate to good internal consistency for the total score (Cronbach alpha: 0.74–0.79). Finally, the Lyons Acceptability, Feasibility, and Appropriateness Scale (LAFAS) provides a quantitative measure of feasibility and acceptability. No psychometric information is currently available for the LAFAS.

Research Procedures

Baseline Surveys

All consented and eligible participants will complete baseline surveys which assess for demographic information, attitudes towards evidence-based practice, organizational readiness for change and training needs (CBO staff only) as well as child welfare involvement history (Legal Guardian/Primary Caregivers only).

Training Caseworkers and Supervisors

Existing training methods from prior effectiveness trial and state-wide dissemination of 4R2S^43,45,53,60 will be tailored for use with CBO caseworkers. Such methods involve training service providers and at least one supervisor per site. Training for caseworkers and supervisors will be based on procedures and content developed during Phase I of this study. Training will cover the 4R2S model’s core competencies (see Table 2), group facilitation and engagement skills, as well as any additional content identified during Phase I CAB recommendations, via a mixture of didactic lecture, group discussion, and role-play. Caseworkers and supervisors will complete a 4R2S Knowledge and Skills test. Caseworkers who do not pass at the 80% level will be provided as-needed booster training and be allowed to re-take the test. Only those who pass the test will be allowed to be involved in the study. Caseworker training for the task-shifted 4R2S intervention will be tracked via total number of hours, number of attendees, content covered, and written feedback from caseworkers and supervisors for additional modifications.

Conducting 4R2S groups

Following training, CBO caseworkers will deliver the refined and adapted 4R2S intervention to families at the CBOs or possibly in the participants’ residence for missed sessions. 4R2S will be delivered such that two groups will be conducted in the participating CBO. However, each group will be facilitated by 2 different pairs of caseworkers (2 groups total, 4 caseworkers total). Consequently, 2 new supervisors and 1 new administrator will recruited for data collection involving the second group. The length of the resulting task-shifted 4R2S intervention and mode of delivery (multiple family groups only held at CBOs or groups plus home-based service delivery) will depend on guidance from the CAB. At this time, the original 4R2S intervention is held in weekly group sessions involving approximately 6–8 families (at least 2 generations per family present) over the course of 4 months. At each session, families are provided with child care for younger children (under 6 years old), refreshments, and transportation expenses. Research staff will ensure that all caseworkers have sufficient materials and resources to conduct 4R2S groups. Meals, childcare and transportation costs will be covered for all participants at each 4R2S session.

During the course of delivering the task-shifted 4R2S intervention (up to 4 months), all group sessions will be observed via telephone or web-based video streaming by research staff in order to complete treatment fidelity rating forms and record session attendance. Additionally, caseworkers will participate in weekly 1-hour long conference calls led by research staff (including mental health clinicians with advanced training in child mental health treatment and expertise in the 4R2S intervention) for supervision. Following each 4R2S supervision call, research staff will document the quantity and quality of the supervision call as well as any recommendations for intervention, training, or supervision modification. Research staff will also take written notes during these supervision calls in order to provide real-time insights to research staff on implementation progress. Any challenges noted during each call will allow research staff to provide ongoing problem-solving support in order to facilitate implementation success.

Post-test Assessment

Upon completion of the task-shifted 4R2S intervention, CBO caseworkers, supervisors, administrators, and Legal Guardian/Primary Caregivers will complete quantitative measures (feasibility, acceptability) as well as participate in separate, homogeneous, audio-recorded focus groups (e.g., caregivers only) or individual interviews (see Table 3 for sample questions).

Following completion of post-test assessments (surveys and focus groups/interviews) Research staff will record the information extracted from the CBO’s administrative records for each family regarding referral status, associated maltreatment allegations, casework contacts, service goals and services received, prior out-of-home placement and associated allegations.

Analytic Plan

Quantitative and qualitative results will be integrated during analysis to complement each other in a QUANT→QUAL (given fidelity and attendance will be recorded first) design⁷².

Quantitative measures and analysis

Table 3 presents information on constructs, corresponding PRISM domains, measures, informants, timing, and methods of analyses. Univariate statistics (means, SD, frequencies) as well as percentages of participants reporting project-defined benchmarks for high feasibility (HF) and acceptability (HA) will be computed. The modified 4R2S will be considered feasible and acceptable if the majority of responses (>50%) exceed the HF and HA benchmarks.

Qualitative methods and analysis

All audio-recorded focus group/interviews will be transcribed verbatim. Written transcripts will be reviewed for accuracy. All written transcripts, as well as all other study documentation (e.g., field interviewer notes, supervisor call notes) will be coded and analyzed using methods by Morgan and Kitzinger^73–76. Specifically, an initial codebook will be developed by drawing together and comparing a priori (e.g., feasibility, acceptability, external environment, perceptions of the intervention, recipient characteristics) and emergent themes. Rapid qualitative assessment strategies will also be incorporated, which involve documenting initial case summaries categorized by a priori codes⁷⁷, as well as remaining open to the emergence of new and unexpected themes. Codes across respondents will be compiled based upon the topics of feasibility and acceptability. For example, a content code “barriers” will be included to identify any content reflecting obstacles to utilizing task shifting or implementing 4R2S. Codes may be modified as new data are analyzed (i.e., dividing “barriers” into multiple codes such as “system barriers” regarding any organizational obstacles to delivery and “caregiver barriers” for any caregiver-related impediments). A report detailing overall experiences and differences will be compiled by respondent type.

Mixed-Methods Integration

Quantitative and qualitative data will be collected sequentially (quantitative followed by qualitative) and analyzed separately, but focused on answering the overarching and specific research questions being addressed. Research staff will integrate both data types at the interpretation stage of analysis. Quantitative data will be visually juxtaposed (See Table 3) next to relevant qualitative themes in order to aid in interpretation of the combined findings. For example, a research question is whether the modified 4R2S intervention is feasible for CBOs providing child welfare services. Both quantitative and qualitative results will be placed side-by-side in order to compare and contrast, thus determining whether they support convergence (i.e., results confirm each other) or expansion (i.e., results generate additional information about what factors promote or limit feasibility and acceptability).

ETHICAL APPROVAL

Institutional Review Board (IRB) protocol approvals for this study have been obtained from the University of Maryland, Baltimore and the Maryland Department of Human Resources. Both protocols will be kept current.

DISCUSSION

Information gathered from Phase I written notes will be compiled into preliminary guidelines documenting the process, successes, and “lessons learned” when utilizing task-shifting strategies. To date, main modifications to the 4R2S intervention for implementation in child welfare include: reducing the length of the model to 9–13 sessions, providing optional instructions if sessions are conducted individually with families during home visiting, addressing literacy concerns by enhancing text size and utilizing more visual information in the intervention manual used by families, adding additional intervention and training content requested by the CAB (e.g., confidentiality concerns, access to local parent advocate information, trauma-informed principles, mindfulness exercises, group and child management). Additionally, organizational-level recommended changes included temporary caseload reduction for group facilitators. Findings from Phase I will be disseminated in a future manuscript currently under development.

Phase II processes will provide data on initial feasibility and acceptability, as well as solidify methods (e.g., manual, training, supervision, enrollment) to support a larger-scale study testing the effectiveness and implementation success of task-shifting in CBOs. Overall practical and operational issues involved in performing the study (e.g., obtaining multiple IRB approvals, engaging hard-to-reach participants) will also be discussed as part of barriers and challenges to performing this type of research. In the larger scale study, implementation and sustainability infrastructure will be addressed, as well as average cost estimates across several different CBOs.

It is acknowledged that a single CBO site is a limitation of the current study, as the contextual factors on implementation and outcomes may vary across different organizations. However, the current study is a first step towards understanding initial feasibility and acceptability of implementing the 4R2S intervention in a child welfare context. The planned larger scale study will build on these findings by using multiple CBO’s, which will ideally be chosen to reflect a range of organizational contexts (e.g., implementation readiness, culture, climate, size). This study innovatively utilizes task-shifting strategies drawn from international public health efforts to scale-up EBPs in low-resource countries. To the authors’ knowledge, task shifting has yet to be used to implement child mental health EBPs in child welfare settings. An additional innovation involves the use of PRISM to guide the study’s design and evaluation. Refinement of task-shifting and PRISM strategies can add to the growing arsenal of implementation strategies⁷⁸ that could be used as components within comprehensive, multilevel approaches to promote successful EBP implementation. The proposed study will provide generalizable knowledge about using methods to facilitate cross-setting implementation for similar EBPs. Such strategies may also provide an innovative way to increase EBP access and reduce costs within transforming child-serving systems.

LIST OF ABBREVIATIONS

EBP

Evidence Based Practices

4R2S

The 4Rs and 2Ss for Strengthening Families Program

CBO

Community-Based Organization

CW

Child Welfare

PRISM

Practical, Robust, Implementation and Sustainability Model

CAB

Collaborative Advisory Board

DBD

Disruptive Behavior Disorder (Rating Scale)

HF

High Feasibility

HA

High Acceptability

Biography

Geetha Gopalan is an Assistant Professor at the University of Maryland School of Social Work, Adjunct Assistant Professor at the New York University School of Medicine Department of Child and Adolescent Psychiatry, Faculty Fellow at the Advanced Center for State Scale-Up for EBPS for Children (IDEAS), and Faculty Fellow at the McSilver Institute for Poverty, Policy, and Research.