Pilot Randomized Trial of a Family Management Efficacy Intervention for Caregivers of African American Adolescents with Disruptive Behaviors

Introduction

Family caregivers of adolescents with Oppositional Defiant Disorder and/or Conduct Disorder (ODD/CD) face significant challenges related to their child’s care. In addition to managing their adolescent’s disruptive behaviors, caregivers must also attend frequent, mandatory, and at times urgent meetings with professionals in mental health, education, child welfare, and juvenile justice child service systems (Evans, Sibley, & Serpell, 2009; Podolski & Nigg, 2001; Tervo, 2012). Caregivers report that these interactions can leave them feeling stressed, disrespected and blamed for their adolescents’ behaviors, and often excluded from decision-making about their adolescent’s care (Baker-Ericzén, Jenkins, & Brookman-Frazee, 2010; Oruche et al., 2015; Valenzuela & Smith, 2016). These experiences can lead to caregiver disengagement from and resistance to recommended evidence-based regimens for their adolescent’s care, which in turn, contributes to poor adolescent outcomes (Acri, Bornheimer, Jessell, Flaherty, & McKay, 2016; Acri, Gopalan, Lalayants, & McKay, 2015; Gopalan, Dean-Assael, Klingenstein, Chacko, & McKay, 2011; Schattner, 2014). Moreover, caregivers have high rates of physical and mental health problems, caregiver stress, and low quality of life that can be exacerbated by stressful interactions with child service professionals (Gerkensmeyer, Perkins, Scott, & Wu, 2008; Gopalan et al., 2011).

Absent from the clinical literature are interventions aimed at empowering caregivers of adolescents with ODD/CD with the requisite communication and problem-solving skills needed to engage in satisfying interactions with professionals across child service systems (Baker-Ericzén et al., 2010; Oruche et al., 2015). If deemed efficacious, such interventions could serve as an adjunct to boost caregiver engagement in existing child service programs, which in turn may result in better child behavioural health outcomes (Alegría et al., 2008; Kutash, Duchnowski, Green, & Ferron, 2011). Guided by findings from our foundational descriptive study and a community advisory board of caregivers of adolescents with ODD/CD, we developed the Family Management Efficacy (FAME) intervention to improve caregiver communication and problem-solving skills and self-efficacy needed to successfully navigate interactions within the family and with child service system professionals, reduce caregiver stress, and improve their quality of life. As an important first step, we conducted a pilot study to examine feasibility and acceptability of FAME intervention content, delivery, appropriateness of our selected measures, and a preliminary examination of outcomes in response to FAME.

Caregiving for Adolescents with ODD/CD in Community Care

Adolescents with disruptive behavior disorders, especially those in low-income minority families, present with some of the nation’s most pressing mental health challenges. Over 10% of adolescents 13-18 years of age are diagnosed with ODD/CDs. ODD is characterised by angry or irritable mood and defiant, vindictive behaviors, while CD is characterised by aggressive, destructive behaviors and serious violations of rules (American Psychiatric Association [APA], 2013). Adolescents with ODD/CD have a chronic illness trajectory and are at risk for poor educational attainment, substance abuse, and involvement with the criminal justice system (Pardini & Fite, 2010). The negative effects of ODD/CD persist well into young adulthood and beyond (Burke, Rowe, & Boylan, 2014; Hinshaw & Lee, 2000).

The effective treatment of adolescents with ODD/CD depends on their caregivers’ active engagement with multiple child service professionals (Brampton et al., 2017; Stagman & Cooper, 2010). Active caregiver involvement with the mental health system is required due to the complex treatment regimens needed for adolescents with ODD/CD and the high risk nature of their behaviors (Green, 2007; McKay & Bannon Jr, 2004; Pennarola et al., 2015; Sayal, Washbrook, & Propper, 2015). The adolescents’ problem behaviors also necessitate frequent parental involvement with the school system. School suspension and expulsion rates for students with emotional and behavioral disorders are estimated at 64%, with estimated dropout rates at 40%^, compared to 7% in the general student population (Bradley, Doolittle, & Bartolotta, 2008; Wagner et al., 2005). In addition, some caregivers of adolescents with ODD/CD have extensive involvement with the criminal justice system as these adolescents have high rates of criminal behavior, arrests, and incarcerations (Aalsma, Brown, Holloway, & Ott, 2014). Furthermore, some caregivers are involved with the child welfare system; 23% of child welfare placements are due to behavioral problems, and 31% of those in child welfare placements are at risk of criminal arrest (Marrast, Himmelstein, & Woolhandler, 2016). Caregivers report that encounters with these child service systems are often time-consuming, stressful, stigmatizing, exhausting, and unhelpful (Ooi, Ong, Jacob, & Khan, 2016; Oruche, Draucker, Alkhattab, Knopf, & Mazurcyk, 2014).

Unabated caregiver stress can lead to poor caregiver physical and emotional health, lowered caregiver quality of life, and problematic family functioning (Cohen & Janicki-Deverts, 2012; Saunders, 2003). In addition, persistent stress can impair the capacity of caregivers to effectively contribute to their adolescent’s mental health treatment (Gopalan et al., 2011). The stress of caring for adolescents with ODD/CD can be particularly high for low-income African American caregivers because of disadvantaged social position and associated adversities such as low educational attainment, low literacy, and low accumulation of relevant knowledge, communication and problem solving skills critical for navigating child service systems and improving family functioning (Alegria, Green, McLaughlin, & Loder, 2015; Alegría et al., 2008; Gengler, 2014; Simons et al., 2016).

Interventions are needed to bolster caregivers’ perceived self-efficacy to manage interactions with child service system professionals, reduce caregiver stress, and improve quality of life and family functioning. Several caregiver support and skills training interventions to reduce stress associated with their child’s care related to disruptive behaviors have been developed (Acri & Hoagwood, 2015; Barlow, Smailagic, Huband, Roloff, & Bennett, 2014; Gerkensmeyer et al., 2013; McKay et al., 2011; Williford & Shelton, 2008; Wittkowski, Dowling, & Smith, 2016). These interventions have demonstrated benefit in the areas of caregiver psychosocial functioning including depression, anxiety, stress (Barlow et al., 2014; Kuravackel et al., 2017; Da Paz & Wallander, 2017); caregiver competence and confidence (Barlow et al., 2014; Kuravackel et al., 2017); caregiver knowledge and management of their child’s behavior problems (Butler & Titus, 2017; Kuravackel et al., 2017); and caregiver efficacy in navigating and accessing mental health and social services for their child (Jamison et al., 2017; Rodriguez et al., 2010). For example, Kutash, Duchnowski, Green, and Ferron (2011) found positive benefits in a parent-to-parent support group in caregiver perceived efficacy in obtaining needed mental health services for their child in special education. Thomas et al. (2017) found positive effects in a group psychosocial intervention including improved ability to work with mental health providers and school system personnel among Latino caregivers of children with mental health and other health needs. However, no interventions have been developed to help low-income African American parents of adolescents with ODD/CD develop strategies and skills to manage stressful interactions with child service system professionals in order to improve the quality of those encounters and enhance shared decision-making regarding their child’s care (Baker-Ericzén et al., 2010; Gengler, 2014; Sales, 2003).

Several interventions have been developed to empower adult ethnic minorities with schizophrenia, depression, or other psychiatric diagnoses prepare for more effective provider interactions (Alegria, 2008; Lara-Cabrera et al., 2016). For example, the Right Questions Project is a three-session health education intervention to teach adult patients to identify important issues related to their illness or treatment, formulate questions, and develop plans to communicate and act in effective ways aimed to elicit provider responses that address their identified mental health care needs (Alegria, 2008). The DECIDE intervention, which includes a three training sessions delivered by a case manager (Alegria et al., 2014; 2018), demonstrated positive effects in the areas of communication, therapeutic alliance, and shared decision making between patient and provider, and patient-perceived quality of care leading to improved engagement and retention in care. Aspects of this interventions can be transferred to interventions focusing on parents of adolescents with ODD/CD.

In response to the need for an effective intervention for this population, our research team developed the FAME intervention. The main purpose of FAME is to increase caregivers’ self-efficacy in managing interactions with child service systems by improving communication and problem-solving skills and leveraging their social networks to support their efforts. The FAME intervention is innovative in two ways. First, it is the first caregiver-centered intervention that targets interactions with a variety of child service systems professionals rather than focusing on healthcare professional interactions exclusively. Second, it targets caregivers of adolescents with ODD/CD, a group not well-represented in prior research. In the long term, we expect that improved caregiver interactions with child service professionals across systems will result in greater engagement with care, increased family follow through with recommended regimens (i.e. service outcomes), and ultimately decreases in ODD/CD symptoms and behavior problems among the adolescents.

FAME Intervention Development

The process by which FAME was developed is described in-depth elsewhere (Oruche et al., 2017). Briefly, we conducted a descriptive study with 15 families of adolescents with ODD/CD to ascertain their main challenges and mental health needs (Oruche et al., 2015; Oruche et al., 2014). The results indicated the caregivers’ interactions with child service professionals were often aversive and a major stressor which suggests the need for an intervention (Oruche et al., 2014). The caregivers felt burdened by frequent, often unscheduled meetings with child service systems (e.g., being “called to” school or having the police “show up” at their house). Furthermore, they were left out of treatment decisions and frustrated by the lack of available, effective programs and services (Oruche et al., 2015; Oruche et al., 2014). The caregivers said they require mental health support, particularly, a desire to meet with others who experienced similar problems and professional interventions that focused on family communication, conflict resolution, education about the adolescents’ disorder, and strategies to improve interactions with child service systems (Oruche et al., 2015). To receive stakeholder input on the intervention, we also convened a community advisory board of five African American caregivers of adolescents with ODD/CD from low-income families and six clinicians who provided mental health services for caregivers of children with serious mental disorders.

We determined that the intervention would be based on several principles drawn from three well-established theoretical and practice models: The Family-Based Network Episode Model of Access to Care [F-NEM] (Costello, Pescosolido, Angold, & Burns, 1998), Social Learning Theory (Bandura, 1986), and Problem-Solving Therapy (Nezu, Nezu, & Colosimo, 2015). The F-NEM (Costello, Pescosolido, Angold, & Burns, 1998) depicts how responses to mental health problems are embedded in family and community social networks and posits that these networks can be galvanized to enhance mental health treatment (Costello et al., 1998). Social Cognitive Theory, with a focus on self-efficacy, posits that behaviors are learned through observation, imitation, and modeling (Bandura, 1986). Problem-Solving Therapy is based on a diathesis-stress model of psychopathology and includes a variety of strategies to enhance the development of adaptive skills (Nezu, Nezu, & D’Zurilla, 2012; Nezu, Nezu, & Colosimo, 2015).

This foundational work informed three major decisions about the structure and content of the intervention. First, the intervention would be delivered in a multiple caregiver group format so that caregivers could learn and receive support from others with similar experiences. Second, consistent with views of families in the African American community, kin (e.g., siblings, aunts, grandmothers) or fictive kin (e.g., intimate family friends) would accompany primary caregivers to group sessions to provide emotional support (Bussing et al., 2003). Third, session content would focus on managing interactions with multiple child service systems to reduce stress associated with these interactions.

In this article, we report pilot study findings of the FAME intervention. We conducted the pilot study in a large publicly funded mental health center with low income African American caregivers of adolescents diagnosed with ODD/CD. The primary aim was to examine the feasibility (i.e., enrollment/attrition, measurement completion, session attendance, and homework completion) and acceptability (i.e., caregiver satisfaction scores and interviews) of the FAME intervention. We hypothesized that a majority of eligible caregivers would consent to study participation and complete the study protocol (a priori thresholds set at ≥50% and ≥60% respectively), ≥75% of participants would complete measures across all time points, caregivers and kin would have ≥50% session attendance, and caregivers would complete ≥50% of assigned homework. For acceptability, we hypothesized that caregivers would report a high level of satisfaction (≥3.0 on a 4-point scale) with the FAME intervention. Our secondary aim was to explore changes in FAME caregiver outcomes when compared to caregivers receiving treatment as usual (TAU) at one week post-intervention (T2) and two months post-intervention (T3). We hypothesized that the FAME group would show larger gains in self-efficacy, problem solving skills, stress, quality of life, family functioning, and social network size/density compared to the TAU group.

Methods

Study Design

We used a pilot randomized trial design, with participants randomized to the FAME intervention or treatment as usual (TAU) group.

Participants

This study received Institutional Review Board approval from the investigators’ university, and we obtained informed consent from all study participants. Study participants were primary caregivers (hereafter referred to as caregivers) of African American adolescents (ages 12 to 18 years) who had been diagnosed with ODD or CD by a mental health provider. Caregivers were eligible for the study if they could identify one kin or fictive kin (hereafter referred to as kin) who could accompany them to the group sessions. Other inclusion criteria were as follows: (a) aged 21 or older (caregivers and kin), (b) English-speaking (caregivers and kin), and (c) feeling stressed by the adolescents’ behaviors (score > 5 on a 0 to 10 scale [none to extremely]) (caregivers). Exclusion criteria for the caregivers and kin were as follows: (a) incarceration, (b) diagnosis of a serious mental health disorder (e.g., schizophrenia, bipolar disorder), and (3) homelessness or residing in a shelter, which could limit full participation in the study. Caregivers were also excluded if the state had custody of the adolescent.

We recruited participants from the Child and Adolescent Program of a large publicly-funded community mental health center in the Midwestern United States. We recruited participants in three cohorts in order to convene three multiple family groups with 4 to 6 caregivers in each group to maximize group dynamics. We enrolled Cohort 1 from July to August 2015; Cohort 2 from October to November 2015; and Cohort 3 from January to February 2016. Although the intended sample would not support inferential tests of efficacy, it was determined to be sufficient to conduct a preliminary, descriptive examination of scores in response to study conditions (Julious, 2005).

Study Procedures

The Child and Adolescent Program provided the Principal Investigator (PI) with secure access to an encrypted case list of all children and adolescents served in the clinic. First, the PI reviewed the case list to identify African American adolescents between ages 12 and 18 who were diagnosed with ODD/CD. Trained research staff then mailed a study flyer and recruitment letter, co-signed by the PI and Child and Adolescent Program director, to eligible caregivers and made a follow-up telephone call 7 to 10 days after the recruitment mailings were postmarked. If the caregivers expressed interest in participation and had supportive kin who would accompany them to the sessions, the research staff conducted a scripted telephone screening to determine if the caregivers and kin met study criteria.

All caregiver/kin dyads who met inclusion criteria were invited to attend a 2-hour group enrollment session where they completed the informed consent process and baseline study measures. Upon completion of the measures, the caregivers and kin each received a $50 and $30 gift card, respectively. The time to complete baseline measures was about 60 to 90 minutes. Following consent, dyads were randomly assigned to the FAME or TAU group. A biostatistician created a computer generated randomization list, and the project manager used sealed numbered opaque envelopes to determine group assignment.

Treatment as Usual

Treatment As Usual (TAU) consisted of standard outpatient mental health care including individual treatment for the adolescent (i.e., therapy, case management, and/or medication) delivered in a variety of settings (e.g., clinics, schools, homes). While TAU could include some family involvement, multiple family group interventions were not part of standard care. The FAME study materials were made available to TAU participants after the study was completed.

FAME Intervention

In addition to standard care, participants randomized to the treatment group received FAME which is a manualized intervention consisting of weekly two-hour sessions delivered over six consecutive weeks. Two trained mental health professionals facilitated the sessions. The lead facilitator was a master’s prepared social worker, and the co-facilitator a master’s prepared Registered Nurse. Session one was an orientation session, session two focused on strengthening interactions within the family, sessions three through five focused on effective communication and problem solving strategies to strengthen interactions with child service systems (i.e., mental health, education, child welfare, juvenile justice), and session six focused on self-care. Informational tip sheets, role-play activities, and weekly action plans for practice at home were used to enhance and reinforce session content. Each participant received a certificate of completion at the end of the intervention. The intervention activities are described in detail elsewhere (Oruche et al., 2017).

Intervention Fidelity

To enhance intervention fidelity, recruiters, data collectors, and session facilitators received formal training by the PI. Data collectors were blind to group assignments and received two four-hour training sessions and on-going supervision by the PI. FAME facilitator training consisted of 10 hours of didactic and role-play experiences, facilitated by the PI, to learn and practice delivery of the manualized intervention protocol. All FAME intervention sessions were audio recorded and reviewed by the PI using a treatment fidelity checklist for each session. The PI provided retraining as needed based on checklist scores. In addition, facilitators completed a brief reflection on successes, challenges, and areas for improvement at the end of each session and attended a meeting with the PI, discussed their reflections and received feedback.

Assessments

The caregivers in both groups completed self-report measures three times over the study period: baseline (T1), immediately post-intervention (T2), and 2-months post-intervention (T3). The kin in both groups completed only the baseline demographic form at T1. We provide a brief description of the measures below.

Demographics and Baseline Measures

Family Information Form.

This form was developed by the research team to gather demographic information for caregivers and kin and included items about age, gender, race/ethnicity, educational attainment, household income, and marital status. Caregivers also provided information on their adolescents’ age, gender, education, and race/ethnicity.

Child Behavior Checklist (CBCL) – Caregiver Report.

The CBCL – Caregiver Report was used to measure the severity of the adolescents’ behaviors (Achenbach & Rescorla, 2001). The CBCL is widely used for children and adolescents between ages 4 and 18 years. The CBCL has 113-items and yields a total, internalizing and externalizing problem score; the latter was used for this study. Externalizing T scores of 60 to 63 are in the borderline clinical range, and scores above 63 are considered to be clinically significant problems.

Feasibility and Acceptability

Feasibility.

To examine feasibility, we used tracking logs and field notes to record the following information: (1) enrollment rates of caregiver/kin dyads, reasons for ineligibility, reasons given by potential participants for declining participation; (2) caregiver attrition rates and reasons given by caregivers for not attending sessions or completing the intervention; (3) measurement completion rates, (4) caregiver/kin attendance across the six sessions; and (5) rates of completion of caregiver homework. Based on a review study by Chacko and colleagues (2016), we set the following a priori thresholds to determine adequacy of our feasibility and acceptability outcomes: (1) enrollment rate ≥ 50%; (2) attrition rate ≤ 40% (i.e., retention rate ≥60%); (3) measure completion rate across all time points ≥ 75%; (4) caregiver/kin attendance ≥50%; and (5) homework completion ≥ 50%.

Client Satisfaction Questionnaire (CSQ-8).

The CSQ-8 was used to measure caregiver acceptability and satisfaction with the FAME intervention (Larsen, Attkisson, Hargreaves, & Nguyen, 1979). The measure includes eight items, scored on a 4-point ordinal scale, with total scores ranging from 8 to 32 and higher scores indicating greater satisfaction. Cronbach’s alpha for this study was 0.94 immediately post-intervention (T2).

Caregiver Interviews.

Two study team members, trained in qualitative interviewing, conducted semi-structured interviews with caregivers in the intervention group two months post-intervention. The interviewers inquired about the caregivers’ experiences with FAME, barriers to attendance and participation, and suggestions for improving the intervention structure and delivery. All interviews were audio-recorded and transcribed by a trained transcriptionist.

Caregiver Outcomes

General Self-Efficacy Scale (GSE).

The GSE was used to measure caregiver self-efficacy (Schwarzer & Jerusalem, 1995). The measure includes 10 items, scored on a 4-point ordinal scale (1 = not true at all; 4 = exactly true). Item responses are summed for a total score, with higher scores indicating greater self-efficacy. In this study, the Cronbach alpha for the total scale items was 0.81.

Social Problem-Solving Inventory–Revised Short Version (SPSI-R:S)

The SPSI-R:S was used to measure caregivers’ problem-solving strengths (D’Zurilla, Nezu, & Maydeu-Olivares, 2002). The measure includes 25 items, scored on a 5-point ordinal scale (0 = not at all true of me and 4 = extremely true of me). Higher scores indicate better problem solving. In this study, the Cronbach alpha for the total scale items was 0.80.

The Perceived Stress Scale (PSS).

The PSS was used to measure the degree to which caregivers perceived their lives as stressful (Cohen, Kamarck, & Mermelstein, 1983). The questionnaire includes 10 items, scored on a 5-point ordinal scale (0 = never and 4 = very often). Total scores could range from 0 to 40, with higher scores indicating greater stress levels. In this study, the Cronbach alpha for the total scale items was 0.77.

Pediatric Quality of Life Inventory—Family Impact Module (PedsQL).

The Family Impact Module of the PedsQL was used to measure the caregivers’ quality of life, including physical functioning, emotional functioning, social functioning, cognitive functioning, communication, worry, daily activities, and family relationships (Varni, Sherman, Burwinkle, Dickinson, & Dixon, 2004). The measure includes 36 items scored on a 5-point ordinal scale (0 = never and 4 = almost always). Higher scores indicate better quality of life. In this study, the Cronbach alpha for items on the PedsQL Family Impact Module was 0.94.

Family Adaptability and Cohesion Scale IV (FACES IV).

The FACES IV was used to measure family cohesion, flexibility, and communication (Olson, 2011). The measure includes 24 items scored on a 5-point ordinal scale (1 = almost never and 5 = almost always). Higher scores indicate greater family cohesion, flexibility, and communication. In this study, the Cronbach alphas for the items on the subscales were 0.80 for family cohesion, 0.71 for family flexibility, and 0.76 for family communication.

Family Life Difficulty Scale of the Family Management Measure (FaMM).

The Family Life Difficulty Scale of FaMM was used to measure the caregiver’s perceived life difficulty in managing the care of their adolescent (Grey, Knafl, & McCorkle, 2006). The measure includes 14 items, scored on 5-point ordinal scale (1 = strongly disagree and 5 = strongly agree). Higher scores indicate greater perceived difficulty. In this study, the Cronbach alpha for the total scale items was 0.84.

The PhenX Social Network Battery.

The PhenX Social Network Battery was used to assess the size and density of the caregivers’ social network (PhenXToolkit: Social Network Battery). Social network includes persons who provide information or help regarding “Important Matters” and “Health.” The battery contains both open-ended (e.g., Who are the most important people in your life right now?) and close-ended (e.g., How close are you to this person? [1/very close, 2/sort of close, 3/not very close]) questions. The respondent may identify as many as 10 persons in their social network. For each person identified, the respondent then answers 11 questions about him or her. This information is used to calculate social network density, which reflects the strength of the ties within the network. Although this instrument was designed for digital administration, due to time, connectivity, and resource constraints we developed a user-friendly paper format.

Data Analyses

Quantitative feasibility and acceptability data were analyzed using descriptive statistics. Qualitative data were analyzed using content analysis. A case-by-variable two-dimensional matrix was used; each row of the matrix represented a participant and each column represented a satisfaction factor (e.g., intervention elements, barriers to attendance) (Miles, Huberman, & Saldana, 2014). Relevant text units from the transcripts were extracted, coded, and placed in the appropriate cells by four team members. The research team clustered the codes into meaningful categories to describe participant reactions to the intervention.

Demographics of the FAME and TAU groups were compared using the Wilcoxon rank sum test for continuous variables and the Fisher’s exact and chi-square tests for categorical variables. Mean CBCL scores were compared between groups by using a linear model with terms for group, adolescent’s age and gender.

The six caregiver outcomes (i.e., self-efficacy, problem solving, stress, quality of life, family functioning, and social network size/density) were summarized at T1, T2, and T3. Outcomes post-baseline were compared between groups by means of a linear mixed-effects model with terms for group, visit, group-by-visit interaction, baseline outcome value and random subject intercept which incorporated into the model the correlation of each subject’s repeated measurements. From the models, we estimated group differences and effect sizes at each visit. Effect sizes were calculated as the estimated group difference divided by the model-based estimate of the standard deviation. We generated 95% confidence intervals for the effect sizes by means of resampling caregiver data with replacement (Field & Welsh, 2007) to generate 2000 bootstrap samples and refitting the models to each bootstrap sample. The 2.5^th and 97.5^th percentile of the effect size distribution were the limits for confidence intervals. Using Cohen’s d guide, effect sizes are considered small if 0.2, moderate if 0.5 and large if 0.8 (Cohen, 1988). To assess internal consistency of scales, we computed Cronbach’s alpha using baseline data.

We used the social network battery to evaluate the caregivers’ social network size and density. For each outcome, we calculated a Cliff’s delta effect size for comparison between the FAME intervention and TAU groups. We selected this effect size because distributions of network characteristics in our sample deviated from the normal distribution. It has to be noted that the Cliff’s delta ranges from -1 (all the values in the intervention group are smaller than the corresponding values of the TAU group) to 1 (all the values in the intervention group are larger than the corresponding values of the TAU group). We based the corresponding 95% confidence intervals on the nonparametric bootstrap methodology based on 2000 replication as described earlier.

Results

Sample

Twenty caregiver/kin dyads participated in the study. Eleven dyads were randomized to the FAME group and nine to the TAU group. Table 1 summarizes the baseline characteristics of the sample and the demographic information of the adolescents. We included all participants who completed any baseline measures with an intent-to-treat analysis. The mean age of the caregivers was 46.7 (SD=14.1), and 18 (90%) were female. Seventeen caregivers (85%) had a high school diploma or greater, 10 (50%) were employed, and 11 (55%) had an income at or below poverty level. The mean age of the kin was 43.9 (SD=14.9) years, and 16 (80%) were female. One kin member identified by the participant did not attend any group or data collection sessions. The mean age of adolescents was 15.3 (SD=1.5) years, and 12 (60%) were female. Adolescents in the TAU group had significantly greater mean total CBCL T scores, although there were no significant differences in mean externalizing CBCL T scores between groups.

Table 1.

Participant Characteristics

Participant Characteristics	FAME	TAU	p value
Child
N	11	9
Age at enrollment , years, M(SD); range	15.9(1.3); 13.8-17.7	14.6(1.5); 12.7-17.7	0.0799
Gender, n(%)
Male	6(54.5)	2(22.2)	0.1968
Female	5(45.5)	7(77.8)
Race, n(%)
African American	11(100)	8(88.9)	0.4500
Mixed	0(0)	1(11.1)
Ethnicity, n(%)
Not Hispanic or Latino	9(81.8)	6(66.7)	0.6169
Not Reported	2(18.2)	3(33.3)
ODD/CDa Severity
CBCLb Externalizing Score	72.1(12.4)	72.9(13.2)	0.0647
Caregiver
N	11	9
Age at enrollment , years, M(SD); range	47.4(14.9);30.3-70.4	45.8(13.8);32.7-74.1	0.9999
Gender, n(%)
Male	0(0)	2(22.2)	0.1895
Female	11(100.0)	7(77.8)
Race, n(%)
African American	11(100)	8(89)	0.4500
Mixed	0(0)	1(11)
Ethnicity, n(%)
Not Hispanic or Latino	9(81.2)	6(66.7)	0.6169
Not Reported	2(18.2)	3(33.3)
Education
9th-12th grade	1(9.1)	2(22.2)	0.4374
High school graduate	3(27.3)	2(22.2)
Some college or certification	6(54.5)	3(33.3)
College graduate	1(9.1)	2(22.2)
Employment
Full-time	3(27.3)	1(11.1)	0.7214
Part-time	4(36.4)	2(22.2)
Homemaker	2(18.2)	2(22.2)
Not employed	1(9.1)	2(22.2)
Retired	1(9.1)	2(22.2)
Income
Less than $19,000	7(63.6)	4(44.4)	0.3618
$20,000-$39,000	4(36.4)	3(33.3)
$40,000-$69,000	0(0)	2(22.2)
Kin
N	12c	8d
Age at enrollment , years, M(SD); range	43.8(14.7); 23.6-69.8	44.2(16.2);22.1-63.8	0.9079
Gender, n(%)
Male	2(16.7)	2(25.0)	0.9999
Female	10(83.3)	6(75.0)
Race, n(%)
African American	11(91.7)	6(75.0)	0.5368
White or Caucasian	1(8.3)	2(25.0)
Kin Relationship to Caregiver
Husband	0(0.0)	1(12.5)
Partner	1(8.3)	0(0.0)
Mother, Step Mother	2(16.7)	3(37.5)
Friend	7(58.3)	0(0.0)	0.0249
Sibling	1(8.3)	2(25.0)
Other Relative	1(8.3)	1(12.5)
Other	0(0.0)	1(12.5)

Note:

^aODD=Oppositional Defiant Disorder; CD = Conduct Disorder.

^bChild Behavior Checklist

^cOne caregiver in the intervention group had 2 kin.

^dOne caregiver in the control group did not have a kin.

Feasibility

Figure 1 displays caregiver enrollment and attrition rates, reasons for declining study participation, and reasons for withdrawal. Forty-four caregivers were screened for eligibility, and 36 met eligibility criteria. Five caregivers were ineligible because they could not identify a kin who could accompany them to group sessions. Of those who were eligible, twenty were enrolled and randomized to the FAME (n = 11) or TAU (n = 9) for an enrollment rate of 56% which met our a priori threshold of ≥ 50%. Attrition rate at T3 was 35% (or retention rate of 65%), which met our a priori threshold of ≤40% attrition or ≥ 60% retention. Measurement completion rates for the FAME group were as follows: 11 caregivers completed measures at T1 (100%), seven at T2 (63%), and six at T3 (55%). Caregivers in FAME who did not complete the measures at T2 or T3 were unresponsive to reminder calls, had work conflicts, or had relocated out of town. For the TAU group, nine caregivers completed at T1 (100%), seven at T2 (78%), and seven at T3 (78%). One caregiver in the TAU group who did not complete T2 or T3 measures was lost to follow-up and one had given birth and was therefore unavailable to complete assessments. Overall measurement completion rates (both groups combined) were 100% at T1, 71% at T2, and 67% at T3, which met did not meet our a priori threshold of ≥75% across time points.

Figure 1.

Consort diagram

Of the 11 caregivers assigned to the FAME group, three (27%) never attended a session. They indicated this was because of the stress of the holidays, the identified kin declined to attend the sessions, and/or the sessions were scheduled at an inconvenient time of day. Caregivers/kin attendance rates by session are summarized in Table 2. In all, six caregivers (55%) and five kin (42%) attended at least four of six sessions, which met our a priori threshold of ≥ 50% for caregivers but not for kin. The reasons given by the caregivers for not attending sessions included work conflicts, lack of transportation, and travel.

Table 2.

Number of FAME Sessions Attended by Caregivers and Kin

Sessions	Caregivers	Kin
0	3	4
1	1	1
2	1	2
3	0	0
4	2	2
5	2	2
6	2	1
Totals	11	12

Caregivers completed homework assignments 50% of the time, which met our a priori threshold of ≥ 50%. The reasons for not completing the homework included forgetting to do it, not having enough time, or not experiencing the problem identified in that homework assignment.

Acceptability

Out of a possible total score of 32 on the CSQ-8 (Larsen et al., 1979), the mean caregiver satisfaction scores were 30.57 (SD = 1.40) at T2 and 28.50 (SD = 3.02) at T3. In other words, caregivers reported average satisfaction score of 3.8 at T2 and 3.7 at T3, which exceeded our a priori threshold of ≥ 3. Interview data also indicated that caregivers found the FAME intervention to be highly acceptable. The participants indicated that they liked the following aspects of the intervention: having opportunities to share stressful experiences, having an open platform for discussion, experiencing a safe and supportive environment, expanding and strengthening their social network, experiencing peer-to-peer learning, and gaining knowledge about their own responses to managing care their adolescents’ behaviors. The participants had the following suggestions for modifying the groups: including more fathers, holding groups at a variety of locations (e.g., in their communities), offering the group meetings at a variety of times (e.g., daytime, evenings, or weekends), offering transportation assistance (e.g., bus passes), and recruiting more broadly (e.g., at other mental health centers). The participants also suggested that they would prefer more “hands-on time” and less lecture time during FAME sessions.

Preliminary Outcomes

The effect sizes with 95% confidence intervals at T2 and T3 for the caregiver outcomes of self-efficacy, problem solving, stress, quality of life, and family functioning are summarized in Table 3. Caregiver social network size/density are summarized in Table 4. The effect size is the standardized estimate of the average FAME versus TAU difference of within subject change from baseline to follow-up. At T2, we found small positive effect sizes for the FAME intervention for stress, family cohesion, and communication, with a moderate effect size for quality of life. At T2, we found a large negative effect size for self-efficacy, and moderately large negative effect sizes for problem solving, flexibility and family life difficulty. At T3, we found large positive effect sizes for the FAME intervention for family cohesion and quality of life, and moderately large positive effect sizes for family communication and social network density. At T3, self-efficacy and problem solving had large negative effect sizes and flexibility had a small negative effect size. The 95% confidence intervals show the precision of the estimate of the effect sizes. In this study, the confidence intervals were all fairly wide due primarily to the small sample size.

Table 3.

Summary of Caregiver and Family Outcomes with Effect Sizes

Caregiver Outcomes	Measures	Group	Time Points			Effect Sizes*
			T1(n=20)	T2(n=14)	T3(n=12)	T2	T3
Self-efficacy	GSE	FAME	30.18(3.92)	29.86(2.41)	29.00(2.76)	−0.61(−2.20,0.61)	−1.43(−2.60,0.12)
		TAU	30.78(6.24)	31.86(5.24)	33.17(5.91)
Problem-Solving	SPSI-R:S	FAME	98.30(6.90)	97.17(6.24)	95.20(5.93)	−0.26(−2.08,1.20)	−1.11(−2.23,1.07)
		TAU	94.22(13.16)	98.00(10.36)	103.00(7.07)
Stress	PSSa	FAME	20.18(4.00)	20.57(5.41)	21.50(4.59)	0.07(−1.35,1.29)	0.43(−1.34,1.55)
		TAU	20.67(7.92)	19.00(9.40)	18.83(8.23)
Quality of Life	PedsQL	FAME	29.17(11.18)	30.23(8.01)	34.84(20.55)	0.46(−0.59,1.92)	0.97(−0.63,2.70)
		TAU	31.14(12.61)	33.68(12.33)	31.52(15.12)
Family Functioning	FACES IV
Cohesion		FAME	65.18(11.84)	65.43(17.21)	69.17(12.42)	0.16(−1.06,1.50)	0.93(−0.42,2.80)
		TAU	72.44(12.51)	69.86(9.30)	67.00(18.22)
Flexibility		FAME	54.45(15.20)	48.00(14.64)	47.17(14.97)	−0.37(−1.98,0.95)	−0.29(−1.90,1.10)
		TAU	63.89(14.72)	57.86(8.09)	57.00(18.42)
Communication		FAME	45.00(22.30)	38.86(24.13)	45.83(29.52)	0.31(−1.03,2.28)	0.57(−0.62,3.33)
		TAU	58.67(19.54)	48.71(27.11)	51.33(29.38)
Family Life Difficulty	FaMMb	FAME	44.09(11.63)	44.00(10.95)	39.83(12.95)	−0.34(−1.87, 0.95)	0.42(−2.35,0.43)
		TAU	50.33(9.27)	47.71(12.46)	38.33(17.27)

Note: CI = Confidence interval; TI = baseline; T2 = immediately post intervention; T3= 2 months post intervention; GSE = General Self-Efficacy; SPSI-R:S = Social Problem Solving Inventory-Revised Short Version; PSSI = Perceived Stress Scale; PedsQL = Pediatric Quality of Life Inventory – Family Impact Module Parent Report; FACES IV = Family Adaptability and Cohesion Scale IV; FaMM = Family Life Difficulty Scale of the Family Management Measure.

^*Standardized mean difference between groups based on the model-based estimates of difference and standard deviation of the dependent variables.

^aFor this measure, higher scores indicate greater stress levels;

^bFor this measure, higher scores indicate greater family life difficulty.

Table 4.

Summary of Caregiver Social Network Size/Density with Effect Size

		T1	T2	T3	T2 vs T1	T3 vs T1
	Group	Median (IQR)	Median (IQR)	Median (IQR)	Cliff’s delta (95% CI)	Cliff’s delta (95% CI)
Important Matters	FAME	4.0 (2.0,6.0)	2.0 (2.0,4.0)	4.5 (2.0,6.0)	−0.27 (−0.80, 0.33)	0.29 (−0.24, 0.76)
	TAU	5.0 (3.0,6.0)	5.0 (3.0,6.0)	3.5 (3.0,6.0)
Health	FAME	3.0 (1.0,6.0)	1.0 (1.0,3.0)	4.0 (2.0,5.0)	−0.18 (−0.75, 0.46)	0.12 (−0.60, 0.67)
	TAU	3.0 (2.0,6.0)	3.0 (2.0,4.0)	3.0 (2.0,6.0)
Density	FAME	0.8 (0.7,1.0)	1.0 (0.7,1.0)	0.9 (0.8,1.0)	0.06 (−0.58, 0.69)	−0.62 (−1.00, −0.04)
	TAU	0.9 (0.6,1.0)	1.0 (0.7,1.0)	1.0 (1.0,1.0)

Note. TI = baseline; T2 = immediately post intervention; T3= 2 months post intervention; IQR= Interquartile range; CI = Confidence interval.

Discussion

FAME is an innovative six-week intervention, delivered as a multiple caregiver group, for caregivers of low income African Americans adolescents diagnosed with ODD/CD. The aim of FAME was to improve caregivers’ self-efficacy in managing interactions with child service systems thereby resulting in decreased caregiver stress, enhanced caregiver quality of life, and improved family functioning, and long-term adolescents’ behavior health outcomes. Findings from our pilot study reveal that FAME was highly acceptable and feasible in the areas of study enrollment/retention, caregiver attendance, and homework completion; however, measurement completion and kin attendance were lower than anticipated warranting further attention to the inclusion of kin and structure of our measurement sessions. Findings suggest that FAME benefited caregivers in the areas of family cohesion, communication, and quality of life; however, FAME caregivers had lower observed self-efficacy and problem solving than those receiving TAU suggesting the need for intervention and measurement refinement.

Findings from the CSQ-8 and our qualitative interviews suggest that FAME was well received by the participants. As found in other studies of caregivers of children with mental health problems in multiple caregiver groups (McKay et al., 2011; McKay, Harrison, Gonzales, Kim, & Quintana, 2002) our participants were particularly satisfied with the opportunity to engage, support, and learn from one another. We believe access to other families who share similar experiences in caring for adolescents with ODD/CD normalize the challenges that caregivers experience and convey the message that “one is not alone” (Chacko, Wymbs, Chimiklis, Wymbs, & Pelham Jr, 2012; McKay et al., 2002). We conclude that delivering FAME in a group format provides a sense of camaraderie among caregivers, even in the face of the multiple stressors and life adversities they experience.

Our feasibility findings suggest that FAME had adequate enrollment and retention rates, and good rates of caregiver attendance and homework completion. Areas lacking included measurement completion rates for FAME participants and low rates of session attendance by kin. Our enrollment rates were above 50% which is consistent with that found in other intervention studies with similar populations (Breitenstein et al., 2012; Chacko, Isham, Cleek, & McKay, 2016; Gross et al., 2009). Attrition rates for FAME participants was also consistent with average rates of 34% found in similar studies with low income caregivers of children and adolescents with disruptive behavior disorders (Chacko, Jensen, et al., 2016). For example, a review of studies on engagement in behavioral parent training interventions for caregivers of children and adolescents with disruptive behaviors found an average enrollment rate of 51%, an attrition rate of 34%, and a rate of failure to attend a single session of 39% (Chacko, Jensen, et al., 2016).

As is often the case with families of adolescents with complex mental health needs, caregivers in our study had multiple competing demands for their time (Chacko, Wymbs, Flammer-Rivera, Pelham, & Walker, 2008; Chacko et al., 2009); however, they did not identify session frequency or duration as challenging. Attendance and homework completion rates for FAME participants were also comparable to that found in previous studies. Chacko and colleagues (2016) reported an average rates of 51% study protocol completion, 50% attendance, and 48% homework completion rates from their review of 226 studies of engagement in behavioral parent training for youths with disruptive behavior disorders. However, attendance rates for kin was low and inconsistent, perhaps because they were not the primary target of intervention, and suggest their optional inclusion in future studies.

Combined measurement completion rates for both groups were less than anticipated post intervention; however, TAU caregivers had higher completion rates. This may have occurred because FAME caregivers completed measures immediately following their last session, and this may have resulted in fatigue. The addition of breaks or shortening the last session may help to increase completion and diminish fatigue.

Our exploratory examination of preliminary outcomes suggests that the intervention, as currently designed, provided benefit in the areas of family communication, cohesion, quality of life, and social network density, but did not benefit caregivers in the areas of self-efficacy and problem-solving. Other studies of group-based interventions for caregivers of youth with disruptive behaviors have shown similar improvements in caregiver psychosocial wellbeing (Barlow et al., 2014) and family cohesion (Hagen, Ogden, & Bjornebekk, 2011). The positive impact of the intervention on quality of life was particularly encouraging because improvements in this domain have been shown to be associated with better emotional and physical health in caregivers and improvements in child behaviors (Palamaro Munsell, Kilmer, Cook, & Reeve, 2012).

The potential positive effects of the intervention on family cohesion and communication is also promising because family functioning tends to be highly problematic in families of adolescents with externalizing behaviors (Keenan-Miller, Peris, Axelson, Kowatch, & Miklowitz, 2012). In addition, an increase in social network density may reflect the intervention goal of leveraging social networks as a way of managing caregiver challenges. Previous descriptive studies found large social network density relative to social network size among African American caregivers with children diagnosed with attention deficit hyperactivity disorders (Bussing et al., 2003). While it may seem paradoxical that the effects of the intervention on social networks appear to be both positive (increasing density) and negative (decreasing size), this is not the case. Previous research tracing the social networks of adults facing their first contact with the mental health treatment system reported similar findings. In the early stage, friends and family rally round the person with mental health problems. However, over time, the network size decreases to become, in essence, the community care team. This more stable team tends to include a smaller number of individuals (i.e., reduced network size) who fulfill specific functions in caregiving and who tend to coordinate their efforts (i.e., greater density) (Perry, 2012).

In contrast, FAME participants did not appear to derive benefit in three important outcomes: self-efficacy, problem-solving, and stress. Although some studies of group-based interventions for caregivers of children with behavior problems found positive effects for self-efficacy and stress (Barlow et al., 2014; McKay et al., 2011; Wittkowski et al., 2016), other studies did not find such benefits (Gerkensmeyer et al., 2013; Williford & Shelton, 2008). For self-efficacy, we were puzzled that the TAU participants reported greater improvement compared to FAME participants. Given large standard deviations, we visually inspected TAU participant self-efficacy scores to see if there were any outliers that might explain why TAU participants had greater improvement, but found none. One possible explanation for this finding is that we used a general self-efficacy measure that might not have captured changes in the knowledge and skills that were the focus of the intervention. There is a need to develop or adapt a self-efficacy measure that is better aligned with the behaviors FAME targets (e.g., asking questions, voicing their concerns, sharing ideas, and partnering with providers in decision making about services) (Wittkowski et al., 2016) and that will thus be more sensitive to intervention effect. Moreover, because we hypothesize that self-efficacy and problem-solving are the primary mechanisms of change in FAME, there is a need strengthen components of the intervention that target these mechanisms. Most importantly, there is a need to increase the time spent in the sessions devoted to role-playing “real life” situations that reflect challenging interactions with child service systems (Mirza, Krischer, Stolley, Magana, & Martin2018). This modification would be consistent with feedback from participants who requested more role-plays and fewer didactic presentations.

Caregivers in the intervention rated their stress scores higher post interventions compared to control. Although reasons for this unanticipated finding are difficult to isolate, we surmise that discussions about problems with child service systems may have brought these issues more sharply into the participants’ awareness and inadvertently contributed to higher levels of perceived stress. Similarly, listening to the challenges experienced by other participants may have stirred up negative emotion and increased stress in the short term. We anticipate that more practice regarding ways to respond to problematic interactions with child-serving systems will improve participants’ self-efficacy that will in return reduce their stress.

Limitations

Here we describe two main study limitations. First, preliminary outcomes should be interpreted cautiously due to our small sample. We used preliminary findings to examine trends in response to the study conditions, but acknowledge and assert that we did not have the statistical power needed to evaluate efficacy. Second, we did not measure improvements in caregiver interactions with child service system professionals directly; instead we used proxy measures of stress, self-efficacy, and problem solving. However, our study findings inform future modification of intervention.

Future Research

Results of this study indicate that FAME intervention need further development. First, because our enrollment was constrained by the requirement to include kin, future research should make this an option rather than a requirement in our next iteration of the intervention. This will allow the inclusion of caregivers who do not have kin support and might benefit most from the social support offered by FAME. Second, future research should incorporate participants’ suggestions for improving enrollment and attendance by modifying our procedures to include publicizing the groups more broadly, addressing transportation difficulties, and offering the group at a variety times of days or days of the week. Third, future research should address participants’ suggestions for improving homework completion by minimizing written work required and focusing more on discussions of participants’ experiences relative to the homework assignments. Fourth, as mentioned above, future research should decrease didactic presentations and include more role-play and discussions of “real-life challenges.” Finally, future research should employ additional retention strategies, such as such as reminder calls and post cards, to boost attendance at group and data collection sessions.

The FAME intervention, which aims to increase caregivers’ knowledge, confidence, and ability to manage their child’s health or chronic illness, is consistent with the concept of caregiver activation (Hibbard, Stockard, Mahoney, & Tusler, 2004; Ruble, Murray, McGrew, Brevoort, & Wong, 2018), an emerging area of research related to caregiving in children’s mental health (Mirza, 2018). We did not address or measure caregiver activation in the current trial but will do so as we further develop the intervention. Specifically, future iterations of the intervention should address the phenomenon of activation more overtly in the intervention content and activities and include an activation measure to assess caregiver (a) knowledge about their adolescents’ service systems, (b) communication skills needed to interact effectively with child service professionals, and (c) caregiver confidence in encounters with child service professionals.

Summary

This study contributes to the emerging literature on caregiver empowerment interventions in adolescent mental health. The FAME intervention holistically aims to increase caregiver self-efficacy in managing interactions with child service systems professionals across mental health, education, child welfare, and juvenile justice systems in low income African American caregivers of adolescents with ODD/CD. Our findings suggest that it was feasible to enroll and retain low income and African American caregivers of adolescents with ODD/CD in a randomized control trial study of the FAME intervention, and that caregivers found the intervention to be highly acceptable. While we have some indication that FAME may benefit caregivers in the areas of family communication, cohesion, and quality of life, several indices of feasibility (kin attendance and measure completion) and lack of observed benefit for self-efficacy and problem solving indicate the need for intervention and measurement refinement. Because caregiver strategies to manage disruptive behaviors are often addressed in routine treatment protocols, FAME, with its focus on managing interactions with child service systems, may prove to be a valuable adjunctive intervention to enhance a variety of outcomes in this highly stressed population.