Meeting the complex needs of urban youth and their families through the 4Rs 2Ss Family Strengthening Program: The “real world” meets evidence-informed care

Abstract

Youth living in poverty face compounding familial and environmental challenges in utilizing effective community mental health services. They have ongoing stressors that increase their dropout rate in mental health service use. Difficulties also exist in staying engaged in services when they are involved with the child welfare system. This study examines the 4Rs 2Ss Family Strengthening Program, developed across four broad conceptual categories related to parenting skills and family processes that form a multiple family group service delivery approach. A total of 321 families were enrolled in this randomized intervention study, assigned to either the 4Rs 2Ss Family Strengthening Program or standard care services. Caregivers and their children randomly assigned to the experimental condition received a 16 week multiple family group intervention through their respective outpatient community mental health clinic. Data was collected at baseline, midtest (8 weeks), posttest (16 weeks), and 6 month follow-up. Major findings include high engagement in the 4Rs 2Ss Family Strengthening Program, compared to standard services. Although child welfare status is not related to attendance, family stress and parental depression are also related to participant engagement in this multiple family group intervention. Involvement in the 4Rs 2Ss Family Strengthening Program resulted in improved effects for child behaviors. Lastly, no evidence of moderation effects on family stress, child welfare involvement, or parental needs were found. The 4Rs 2Ss Family Strengthening Program appeared able to engage families with more complex “real world” needs.

Poverty-impacted youth of color too often face enormous challenges and a wide range of threats, including community-level violence and increased exposure to serious health conditions, such as substance abuse or HIV infection (Brooks-Gunn & Duncan, 1997; Horowitz, McKay, Marshall, 2005). In response, urban families attempt to counteract the negative impact on their children by employing specific strategies, in particular, drawing on positive racial/ethnic socialization parenting practices, as well as trying to buffer their children’s exposure to neighborhood dangers by limiting outside community relationships and time spent away from the home/family (Jarrett, 1995; Rodriguez et al. 2008; Bannon et al., 2008). Despite their best efforts, low-income minority youth evidence the effects of “toxic” stressors and present overlapping vulnerabilities in disproportionate numbers (Brown et al., 1997; Capaldi et al., 2002; Houck, et al., 2006; Tubman et al., 2003). For example, African-American and Latino youth residing in low-income urban communities have been found to be at 4 to 6 times greater risk for serious conduct difficulties in comparison to same-age peers (Angold & Costello, 2001; Tolan & Henry, 1996).

These high rates of conduct difficulties may be explained by both the direct impact of exposure to social ills and the robust body of evidence suggesting that attempting to navigate multiple and persistent stressors can undermine parenting and family protective factors over time. Such stressors can (1) disrupt effective parenting practices (Hausman & Hammen, 1993; Rossi, 1994); (2) undercut protective aspects of family life, such as regular and consistent family interaction and communication opportunities (Fiese et al., 2002); and (3) create a sense of danger and uncertainty that can heighten family conflict and undermine social ties (Baum et al., 1981; Gabarino & Kostelny, 1992; Zill, 1996). These disruptions are often compounded by the adult caregivers’ attempts to cope with their own high levels of stress (Burt & Cohen, 1989). Increased parental stress may be transmitted via the parent-child relationship which directly affects the child’s development and the strength of the adult protective shield (Lieberman & Van Horn, 2004).

Delivering engaging and high impact family-strengthening service options capable of helping families address youth behavioral difficulties as they continue to experience stressful circumstances is both critically necessary and a daunting challenge for public child mental health systems, service delivery organizations and providers (Hoagwood et al., 2010; McKay et al., 2010; 2011). Currently, even when linkages are made to the public child mental health system, lack of service capacity as evidenced by long waiting times for appointments and engagement challenges are prevalent (Gopalan et al., 2010; McKay et al., 2010). For example, numerous research studies, now summarized in several reviews of the engagement literature, have identified a range of concrete (e.g. lack of time, transportation, competing appointments and priorities), perceptual (e.g. concerns related to stigma and negative expectations based on prior interactions with providers) and systemic (e.g. dearth of trained providers, availability of appointments) obstacles that interfere with receipt of appropriate services (Gopalan et al., 2010). Thus, there is a serious need to create evidence-informed service models capable of: 1) addressing the serious needs of urban youth; 2) engaging high risk families; 3) retaining them long enough to address prevalent conduct difficulties; 4) being embedded in urban service systems with scarce service resources and; 4) meeting the often multiple, complex needs of urban families.

Family-strengthening approaches to address youth conduct difficulties in the “real world”

The use of family-based interventions for children exhibiting serious behavioral difficulties and their families has gained substantial empirical support (Bank et al., 1991; Webster-Stratton, 1985; 1990; Sexton and Alexander, 2002). Many of these evidence-based family interventions focus on parent management training or parenting skills (Farmer, Compton, Burns, and Robertson, 2002; Keiley, 2002; Cottrell and Boston, 2002) via behavioral rehearsal, modeling, feedback, coaching, as well as goal setting, family communication, action plans, and building on family strengths. However, it is important to note that “real world” workforce preparation in the areas of systematic, evidence-informed work with parents, families and groups has been identified as an area of weakness, thus creating implementation challenges for many evidence-informed services targeting childhood conduct difficulties (Salerno et al., 2011).

Further, providers in urban practice settings frequently note that child conduct difficulties are not the only presenting issue for families (Hammen et al., 1999). In fact, the clinical picture is often complex, consisting of compelling youth need, high levels of family challenge, particularly high levels of family stress, family safety issues, including child welfare involvement and parental mental health need or substance use. Thus, too often, existing evidence-informed family-strengthening models are perceived as unresponsive to addressing the complex presenting needs of youth and families by urban service providers.

This paper focuses on an evidence-informed service meant to address multiple child and family needs simultaneously via a family-focused, group-delivered model. This model, referred to as the 4Rs and 2Ss Family Strengthening Program, is a protocol driven multiple family group (MFG) approach to enhancing aspects of family life empirically linked with youth conduct difficulties. The 4Rs and 2Ss Family Strengthening Program was most recently examined within an experimental effectiveness study embedded within urban child mental health clinics, all set in high-poverty communities (see McKay et al., 2010; 2011 for details). At baseline, families presented with youth evidencing clinically meaningful, serious conduct difficulties. At the same time, significant proportions of families identified numerous challenges, including those referred to above (high levels of family stress, safety issues, including child welfare involvement and parental mental health need). Thus, the ability of the 4Rs and 2Ss Family Strengthening service model to engage, retain and potentially impact the full range of youth and their families is the focus of this paper.

4Rs and 2Ss Family Strengthening Program

A robust existing literature suggests that family factors have been consistently implicated in the onset and maintenance of childhood behavioral difficulties (Dishion, et al., 1995; Kilgor et al., 2000; Loeber et al., 1998). Kazdin and Whitley (2003) also emphasize specific family factors tied to socioeconomic disadvantage, social isolation, high stress and lack of social support, may undermine parenting and contribute to childhood conduct problems (Keiley, 2002; Kumpfer & Alvarado, 2003; Wahler & Dumas, 1989). This body of research can be summarized under four broad conceptual categories related to parenting skills and family processes that form the targets for the 4Rs 2Ss multiple family group service delivery approach. Rules, Responsibility, Relationships and Respectful communication (named via collaboration with adult caregivers of youth evidencing behavioral difficulties to summarize the evidence base in a manner that: 1) increases the understanding of parents and providers about the importance of specific aspects of family life in the remediation of childhood behavioral difficulties; and 2) enhances the relevance of addressing parenting and family processes, while simultaneously reducing parental or family blame for youth conduct difficulties. In addition, Stress and Social support were added to targets of the 4Rs 2Ss Family Strengthening Program as these have been found to impact child service engagement and outcome (see McKay et al., 2010; 2011 for a fuller description of the 4Rs 2Ss development process). See Table 1 for a summary of evidence-informed targets of the service model.

Table 1.

Summary of 4Rs 2Ss Family Strengthening Program Evidence-informed Targets

MFG target	Family process or parenting skill linked with youth conduct problems or associated with positive youth bsehavioral outcomes
Rules	1) family organization; 2) consistent non harsh discipline practices, including clear behavioral limits, appropriate consequences and reinforcement; 3) parental monitoring and supervision skills
Responsibility	1) family interconnectedness; 2) positive behavioral expectancies for Youth
Relationships	1) family warmth and attachment; 2) within family support and; 3) time spent together
Respectful Communication	1) family communication; 2) family conflict; 3) parent/child interaction
Stress	1) parenting hassles;2) parenting stress; 3) life stressors, 4) socio-economic disadvantage
Social support	Social isolation

MFGs are defined as: 1) a mental health service that involves 6 to 8 families; 2) an intervention that is facilitated by trained clinicians or a clinician and parent advocate; 3) a treatment where at least two generations of a family are present in each session and; 4) psychoeducation and practice activities that foster both within family and between family learning and interaction (O’Shea & Phelps, 1985).The 4Rs 2Ss Family Strengthening Program involves school-age, inner-city children (7 to 11 years) meeting diagnostic criteria for Oppositional Defiant or Conduct Disorders and their families (including adult caregivers, siblings over 6 years) in a 16-week series of groups.

The development of the guiding intervention protocol was informed by recommendations offered by Weisz (2001) and Hoagwood, Burns & Weisz (2002), whereby any new clinic or community-based service model in development begin with piloting in “real world” settings and attend to “nuisance” characteristics of the service delivery process (e.g. characteristics of clinical populations, preferences of consumers, high demand for services and scarcity of service providers) in order to address the significant challenges related to adopting, integrating and sustaining new practices in “real world” child systems (Bickman 1996; Burns & Hoagwood, 2004; Hoagwood & Burns, 2005; Weissman et al., 2006; Weisz et al 2004).

Three of these “nuisance” characteristics (high levels of family stress, safety issues, including child welfare involvement and parental need) and their impact on engagement, retention and outcomes associated with the 4Rs and 2Ss Family Strengthening Program are the focus here.

Complex Clinical Presentations of Urban Families: Family Stress, Child Welfare Involvement and Parental Mental Health

Not only are inner-city youth with challenging behavioral problems negatively affected by extremely stressful circumstances related to poverty, under-resourced schools, substance abuse, lack of access to child mental health clinics, exposure to health epidemics, community violence, and other negative life events (Harrison, McKay & Bannon, 2004; McKay & Bannon, 2004; Monuteaux, 2007), but prior studies cite that nearly every caregiver living in urban neighborhoods experiences parenting stress on some level (Crnic & Greenberg, 1990). Prior studies have also established links between caregivers and their children who live in poverty and increased likelihood of caregivers developing mental health concerns as compared to caregivers not living in poverty (Petterson & Albers, 2001; Radloff, 1975). Liaw and Brooks-Gunn (1994) concluded that 28% of mothers living in poverty also reported high levels of caregiver depression, and highly stressed mothers of families living in poverty often exhibit impairments in not only perceived parenting capacity (Russell, Harris & Gockel, 2008), but they also report decreases in positive parenting practices over time (Kotchick, Dorsey & Heller, 2005; Wahler & Duhmas, 1989).

Kazdin & Whitley (2003) describe the combination of childhood behavioral problems along with parent-level contextual factors such as caregiver stress as existing as a “package” that collectively affect treatment outcomes. In other words, youth disruptive behavioral disorders and caregiver stress often co-exist, and the “package” of both child behavioral difficulties, as well as caregiver stress, must both be addressed to optimize treatment outcomes. Abidin (1975) also described a combination of key caregiver characteristics (e.g. caregiver stress, caregiver depression) in combination with a set of youth characteristics (e.g. behavior) that collectively define the multi-faceted experience of parenting stress, and these child and caregiver factors in conjunction with negative life events and community-level stressors contribute to an increased risk of decreased parenting practices over time (Abidin, 1975; Morgan, Robinson, & Aldridge, 2002).

Furthermore, children whose mothers experience mental health problems are at four to seven times increased odds of also having an emotional or behavioral diagnosis (Ghandour et al., 2012) as compared to children whose mothers are without mental health diagnosis. In turn, higher levels of childhood behavioral problems are predictive of higher levels of caregiver stress (Anastopoulos et al., 1992; Gillberg et al., 1983; Mash & Johnston, 1983; Webster-Stratton, 1988; Morgan, Robinson, & Adlridge, 2002). Additionally, prior studies (Murray, 1992) have confirmed the potentially deleterious effects of maternal depression and child behavioral problems throughout development. Therefore, it may be necessary to impact both child and caregiver mental health needs within families seeking help within child mental health clinics in order to promote overall mental wellness and stronger family life.

Baseline caregiver stress and depression are often not the only factors contributing to the wide range of outcomes in response to child mental health treatment. More specifically, prior research studies have determined that higher rates of behavioral difficulties are exhibited by children who remain with their primary caregivers after the family has experienced a child welfare investigation (Gopalan, 2012; Lau & Weisz, 2003; Leslie, Hurlburt, James, Landsverk, Slymen, & Zhang, 2005). Families involved in the child welfare system are exposed to multiple co-occurring stressors, which hinder their mental health treatment engagement. For example, these families report a greater number of barriers to attending child mental health service appointments compared to families not involved in the child welfare system (Gopalan 2012). Negative experiences with the child welfare system or the absence of a therapeutic alliance may also result in early withdrawal from services. (Kerkorian, McKay, & Bannon, 2006; Palmer, Maiter, & Manji, 2006). In sum, creating evidence-informed service delivery models that can embed within complex, often under resourced care systems, be delivered by the current workforce, and can engage and retain youth and families presenting complex needs is critical and challenging. The 4Rs and 2Ss Family Strengthening Program was an attempt to address these serious urban child mental health service delivery challenges.

Methods

The current study examines the influence of common complex presenting problems (family level stress, child welfare involvement and parental need) on involvement in youth with serious conduct difficulties and their families in the 4Rs 2Ss multiple family group intervention over a 16 week service delivery period. First overall rates of attendance of families in the 4Rs 2Ss program relative to receipt of standard care of are reviewed (see McKay et al., 2010; 2011 for additional details). Then, the effects of baseline family stress, parental depression, and child welfare involvement at baseline are explored relative to overall attendance are explored. Finally, emerging evidence that despite complex baseline needs, families involved in the 4Rs 2Ss Family Strengthening Program are reporting improvement in child conduct difficulties and social competencies relative to those receiving typically offered child mental health care is summarized.

Sample

Caregivers were recruited from October 2006 to October 2010 and from 13 different community-based outpatient child mental health clinics. Each clinic serves urban families in the greater New York City area. Inclusion criteria for the previous trial included: 1) youth between the ages of 7 to 11 years old; and 2) meeting criteria for a DSM-IV diagnosis of Oppositional Defiant Disorder or Conduct Disorder as assessed by a trained research assistant at intake. A total of 321 families were enrolled in the study with 224 randomly assigned to the experimental condition (Multifamily Group Intervention) and 97 randomly assigned to standard care services. Caregivers and their children randomly assigned to the experimental condition received a 16 week multiple family group intervention through their respective outpatient community mental health clinic.

The disparity between the sample sizes is attributed to the use of a 2:1 allocation ratio. Randomized blocking, weighted towards MFG, was incorporated to ensure enough participants were available to run experimental groups. This would ensure participants were receiving services as soon as possible rather than languishing on a waitlist. Ongoing time on the waitlist could reduce participant motivation to return for services and lead to insufficient statistical power overall if recruitment goals were not met. Moreover, the current study was not blinded to study condition. Unequal group ratios only significantly reduce the power of a study when the ratio is 3:1 or more (Dumville et al., 2006). Therefore, this study’s allocation method augments program efficiency with minimal impact on statistical power.

See Table 2 for detailed demographic information pertaining to the entire sample in this study by condition. Participants were primarily of Latino or African American descent and low-income, with just slightly over half of the participants reporting working full-time.

Table 2.

Demographic Characteristics of Participants by Study Condition

		Total		Experimental (n = 224)		Control (n = 97)
Characteristic	N		%a	n	%a	n	%a
Caregiver Ethnicity:
White/Caucasian		30	9.3	21	9.7	9	9.4
Black/African American		96	29.9	62	28.7	34	35.4
Hispanic/Latino		168	52.3	118	54.6	50	52.1
Native American		3	0.9	2	0.9	1	1
Asian/Pacific Islander		2	0.6	1	0.5	1	1
Other		13	4.1	12	5.6	1	1
Child Ethnicity:
White/Caucasian		25	7.8	16	7.6	9	9.4
Black/African American		95	29.6	65	31	30	31.3
Hispanic/Latino		156	48.6	111	52.9	45	46.9
Native American		8	2.5	3	1.4	5	5.2
Asian/Pacific Islander		1	0.3	0	0	1	1
Other		21	6.5	15	7.1	6	6.3
Primary Caregiver:
Mother		252	78.5	174	80.9	78	82.1
Father		6	1.9	5	2.3	1	1.1
Mother and Father		24	7.5	21	9.8	3	3.2
Grandparent		14	4.4	6	2.8	8	8.4
Other		14	4.4	9	4.2	5	5.3
Caregiver Marital Status:
Single		137	42.7	86	40.2	51	52.6
Married or Cohabiting		105	32.7	79	36.9	26	26.8
Divorced		14	4.4	7	3.3	7	7.2
Separated		44	13.7	34	15.9	10	10.3
Widowed		7	2.2	4	1.9	3	3.1
Other		4	1.2	4	1.9	0	0
Family Income:
Less than $9,999		125	38.9	91	44	34	37.8
$10,000 to $19,999		80	24.9	54	26.1	26	28.9
$20,000 to $29,999		47	14.6	32	15.5	15	16.7
$30,000 to $39,999		23	7.2	14	6.8	9	10
$40,000 to $49,999		5	1.6	3	1.4	2	2.2
Over $50,000		17	5.3	13	6.3	4	4.4
Caregiver Education Status:
8th Grade or Less		32	10	27	12.6	5	5.2
Some High School		91	28.3	60	27.9	31	32.3
Completed H.S./G.E.D.		77	24	51	23.7	26	27.1
Some College		70	21.8	47	21.9	23	24
Completed College		21	6.5	16	7.4	5	5.2
Some Grad/Prof. School		6	1.9	5	2.3	1	1
Competed Grad/Prof. School		14	4.3	9	4.2	1	5.2
Caregiver Employment Status:
Employed Full-Time	75	23.4	54	25	21	22.1
Employed Part-Time	53	16.5	39	18.1	14	14.7
Student	18	5.6	13	6	5	5.3
Retired	8	2.5	3	1.4	5	5.3
Disabled	37	11.5	26	12	11	11.6
Unemployed	104	32.4	71	32.9	33	34.7
Other	16	5	10	4.6	6	6.3
Experimental					Control
	n	mean	Standard deviation		N	mean	Standard deviation
Parent age at baseline:	212	36	8.44		95	38	9.3
Child’s age at baseline:	207	9	1.5		91	9	1.3

^a% is out of Total sample size (n = 321)

For purposes of this study, specifically the interest in examining the impact of complex presenting problems at baseline among those in the experimental intervention, 4Rs 2Ss participants were categorized based upon baseline parental stress and depressive symptoms and clustered into three groups (low stress/low depression; moderate stress/depression and high stress/depression) for analysis. These two variables - stress and depression - were grouped together based upon almost perfect correlation between measures (r > .90) (See Table 3 for demographic characteristics of these three groups). Table 4 presents demographic characteristics of the 83 families with a history of child welfare involvement included in the experimental condition relative to those non-child welfare involved families. There were few notable significant differences between all subgroups.

Table 3.

Demographic Characteristics of Interview Participants by Comorbid Stress/Depression Group

	Low (n=57)		Moderate (n=90)		High (n=44)
Demographic Characteristic	n*	M; SD	n*	M; SD	n*	M; SD
Caregiver Age	57	35.5; 8.8	90	34.7; 6.5	44	35.7; 8.2
Primary Caregiver	n	%	n	%	n	%
Mother	43	75	78	87	34	77
Father	2	4	0	0	2	5
Grandparent	1	2	2	2	1	2
Mother and father	8	14	7	8	6	14
Other	3	5	3	3	0	0
Caregiver marital status
Single	20	35	44	49	13	30
Married or cohabitating	25	44	29	32	18	41
Divorced	0	0	3	3	3	7
Separated	8	14	12	13	7	16
Widowed	2	4	0	0	2	5
Other	1	2	1	1	1	2
Caregiver education status
t Eighth grade or less	8	14	9	10	6	14
Some high school	22	39	21	23	11	25
Completed H.S./G.E.D.	9	16	23	26	9	21
Some college	12	21	21	23	11	25
Completed college	3	5	8	9	3	7
Some grad/Prof. school	0	0	2	2	2	5
Completed grad/Prof. school	3	5	5	6	1	2
Caregiver ethnicity
White/Caucasian	4	7	9	10	4	9
Black/African American	18	32	25	28	10	23
Hispanic/Latino	31	54	51	57	25	57
Native American	2	4	0	0	0	0
Asian/Pacific Islander	0	0	0	0	1	2
Other	2	4	5	5	3	7
Caregiver employment status
Employed full time	9	16	25	28	10	23
Employed part time	11	19	21	23	5	11
Student	4	7	7	8	2	5
Retired	2	4	0	0	1	2
Disabled	9	16	7	8	7	16
Unemployed	19	33	25	28	18	41
Other	3	5	4	4	1	2
Family income
Less than $9,999	29	51	36	40	18	41
$10,000 – $19,999	8	14	26	29	11	25
$20,000 – $29,999	8	14	13	14	5	11
$30,000 – $39,999	6	11	5	6	2	5
$40,000 – $49.999	0	0	2	2	1	2
Over $50,000	4	7	5	6	4	9

No between-group differences are significant at the p=0.05 level.

^*Not all totals sum to n = 191 due to missing data

^**Percentages based on total sample (n = 191), representing low and high stress caregivers

Table 4.

Demographic characteristics of child welfare vs. non-child welfare involved families

	Child Welfare Involved (n = 83)		Non-Child Welfare Involved (n = 141)
Characteristic	N	%a	n	%a
Caregiver Ethnicity:
White/Caucasian	7	3.13	14	6.25
Black/African American	29	12.95	33	14.73
Hispanic/Latino	40	17.86	79	35.27
Native American	0	0.00	2	0.89
Asian/Pacific Islander	1	0.45	0	0.00
Other	6	2.68	6	2.68
Child Ethnicity:
White/Caucasian	8	3.57	8	3.57
Black/African American	28	12.50	37	16.52
Hispanic/Latino	36	16.07	76	33.93
Native American	1	0.45	2	0.89
Other	6	2.68	9	4.02
Primary Caregiver:
Mother	66	29.46	109	48.66
Father	4	1.79	1	0.45
Mother and Father	8	3.57	13	5.80
Grandparent	2	0.89	4	1.79
Other	3	1.34	6	2.68
Caregiver Marital Status:
Single	33	14.73	53	23.66
Married or Cohabiting	26	11.61	54	24.11
Divorced	4	1.79	3	1.34
Separated	15	6.70	19	8.48
Widowed	0	0.00	4	1.79
Other	3	1.34	1	0.45
Family Income:
Less than $9,999	40	17.86	51	22.77
$10,000 to $19,999	18	8.04	37	16.52
$20,000 to $29,999	14	6.25	18	8.04
$30,000 to $39,999	5	2.23	9	4.02
$40,000 to $49,999	0	0.00	3	1.34
Over $50,000	2	0.89	11	4.91

Description of participating outpatient child mental health clinic research sites

The 13 outpatient child mental health clinics involved shared common characteristics. All the sites included in the study provide a range of mental health services to youth and their families living in the New York City metropolitan area. Further, the vast majority of youth served at each site are low-income. Generally, youth and families at each of the sites were members of minority groups with the largest proportion of youth being Latino (with ties to Puerto Rico, Dominican Republic or Mexico) or African American.

Recruitment, and informed consent procedures

Institutional Review Board approval was obtained for this study. Providers at each site received information about the study and were given printed materials to provide to their clients about participation in the study. Recruitment strategies included: 1) a strong on-site presence at each of the clinics; 2) on-going reminder telephone contact with the clinic supervisor to encourage the plan of introducing the study to families scheduled for intake appointments; 3) presentations at staff meetings to problem solve any obstacles to recruitment; 4) meetings with families took place during after school and evening hours and concerted efforts to follow-up with the family immediately upon their expression of interest were made.

Potentially eligible youth and their families (based on an intake diagnosis of ODD and CD made by clinical service providers) were informed of the study by their providers. Then, if the family was interested in learning more about the study, they were contacted by a member of the research staff. Informed consent materials were provided to the family by the research staff. If the adult caregiver provided consent and the youth provided assented, then the research staff administered two screening instruments to determine the presence of clinically significant disruptive behavior difficulties to determine study eligibility. If the youth and family were screened as eligible, then the family was immediately randomly assigned to one of the two study conditions.

Description of Multiple Family Group Intervention Protocol

The MFG service delivery strategy is a 16-week series of meetings guided by a protocol (McKay, Gonzales et al., 1995). Groups are held weekly and are facilitated by mental health providers. Groups consist of six to eight families involving adult caregivers and all children over six years of age in the family. Each session follows the same procedures and proceeds through five stages: 1) creating social networks; 2) information exchange; 3) group discussions; 4) individual family practice and; 5) homework assignments. Each group begins with an opportunity for families to interact, and snacks are provided. The informational portion of the group, which lasts approximately 30 minutes, is facilitated by the providers. The remainder of the time is divided equally between group discussions, family practice exercises, and an explanation of the homework assignment (Tolan & McKay, 1996).

Content of the MFG focuses on helping families strengthen four aspects of family life that have been empirically linked to childhood behavioral disorders. Specifically, the intervention focuses on the 4 “Rs” of family life: 1) Rules; 2) Responsibilities; 3) Relationships and; 4) Respectful communication. In addition, four specific weeks are devoted to expanding support for parenting and reducing stressors.

Multiple Family Group Delivery

At each of the clinics involved in the study, agency administrators and clinical supervisors were enlisted to identify potential MFG facilitators. A group meeting with providers was organized by agency leadership. Training of MFG service providers consisted of fourteen modules and was completed in 2 half-days. Training consisted of information related to childhood conduct difficulties, family level factors that have been linked to child mental health outcomes, strategies to enhance engagement and motivation, group facilitation skills and processes specific to MFGs. In addition, participating providers reviewed training videotapes and engaged in practice activities and role plays. Providers also received ongoing supervision for at least one hour per week. This included on-site supervision and group supervisory conferences across research sites.

Data collection procedures and instruments

Data was collected at baseline and attendance data was collected at each MFG session by group facilitators. Outcome data was also collected at midtest (8 weeks), posttest (16 weeks) and 6 month follow-up.

Instruments

Extent of involvement in child mental health services was be measured by the REACH attendance tracking log which recorded the number of sessions that each child/family attended in both conditions. Providers with the assistance of research staff completed these logs weekly. Involvement with child welfare services was measured by a single item asked of adult caregivers at baseline with a set of follow-up probes regarding common types of child welfare involvement (e.g. investigation, enrollment in preventative services, placement). Parent Stress Index (PSI; Abidin, 1995): The short form of the PSI (38 items) was used to assess stress in the parent-child relationship system, with the total child domain score (alpha=.89) used for parenting stress. Depression. Caregiver depression was assessed using the Center for Epidemiological Studies Depression Scale (CES-D; Radloff, 1977). The CES-D is a 20-item assessment that examines depression across four subscales, which can be added together to obtain a total depression score: Depressive Affect, Somatic Symptoms, Positive Affect, and Interpersonal Problems (Radloff, 1977). IOWA Connors Rating Scale. The IOWA CRS (Waschbusch & Willoughby, 2008) is a widely used brief measure of inattentive-impulsive-overactive (IO) and oppositional-defiant (OD) behavior in children completed by parents and teachers. The IOWA Conners Rating Scale consists of ten items each of which is evaluated using a four point likert scale with the following anchors: not at all (0); just a little (1); pretty much (2); and very much (3). The first five items on the IOWA are designed to measure inattentive-impulsive-overactive (IO) behaviors and the second five items are designed to measure oppositional defiant (OD) behaviors. For the current study, we utilized the OD subscale reported at baseline, mid-test, and post-test. Cronbach’s αs for the baseline, mid-test, posttest, and 6-month follow-up assessments were 0.80, 0.83, 0.86, and 0.86, respectively. Social Skills Rating System (SSRS). The SSRS (Gresham & Elliot, 1990) assesses behaviors of youth across three subscales: problematic behaviors, social skills, and academic skills. Caregivers rate the frequency and importance of specific behavior within each scale along a 3-point likert rating, from 0 (“never”) to 2 (“often”). The SSRS Social Skills Subscale (SSRS-SSS), which focuses on the frequency of occurrence of the social skills of cooperation, assertion, responsibility, empathy, and self-control, was utilized in the current study. The SSRS is a well-validated and reliable measure, and it is one of the most widely utilized instruments for measuring social skills. The internal consistencies of the subscales range from .51 to .91 (r = .75). Higher frequency scores indicate more frequent use of prosocial social skills. Cronbach’s αs for the baseline, posttest, and 6 month follow-up assessments for SSRS-SSS were 0.88, 0.91, and 0.92, respectively.

Data Analysis

Descriptive statistics were computed for all measures. Multivariate tests were used to examine significant differences and associations initially. These analyses utilized SPSS statistical software (Version 19). Next, both multivariate analyses including random coefficient modeling (RRM) were performed on attendance rates and outcomes over time. For RRM analyses, the SuperMix program for mixed effects regression models was used (Hedeker, Gibbons, du Toit, & Cheng, 2008). SuperMix uses maximum likelihood estimation to model measurements over time within cases. Within the final model, study participants were nested by the hierarchies of their individual ID, with time treated as a random effect. This form of modeling, also known as hierarchical linear modeling or multilevel linear modeling, allows parameters (intercepts and slopes) for measurements over time within cases to vary between cases, while accurately accounting for correlation between measurements within cases. It also allows for different times and numbers of measurements within people, an appropriate method to model longitudinal change involving data where there is attrition over time with the assumption that the missing data is ignorable (i.e., at least missing at random), which is a reasonable assumption with this data.

Results

First overall rates of attendance of families associated with the 4Rs 2Ss program relative to receipt of standard care of are presented in Figure 1 (see McKay et al., 2010; 2011 for additional details). More specifically, approximately 80% of families were retained in the 4Rs 2Ss over the 16 week intervention protocol. High rates of drop out approaching 90% were associated with typical care condition. Next, at the bivariate level, the association between baseline family stress/ parental depression relative to overall attendance at 4Rs 2Ss group meetings is presented in Table 5. No significant differences in a 3-way comparison of attendance for both caregiver stress and comorbid stress/depression groups were found. Table 6 presents the univariate findings for child welfare involved families relative to those families reporting no child welfare involvement. Again, no significant differences were found between these two groups (Gopalan et al., in press). Finally, Tables 7 and 8 summarize multivariate analyses. These findings reveal that although child welfare status was not significantly related to attendance, family stress and parental depression were significantly associated with overall involvement in the program (Gopalan et al., in press). Finally, overall main effects of involvement in the 4Rs 2Ss Family Strengthening Program relative to typical child mental health care services present statistically significant improvements in child conduct difficulties and social competencies for the experimental group. Results from the multivariate analyses, indicate lower rates of oppositional defiant behaviors on the Iowa Connors OD scale at post-test (β= − 0.03, SE = 0.49, Z = −2.47, p =.01) and 6 month follow-up (β= −1.17, SE = 0.51, Z = −2.29, p =.02). Social skills are also significantly higher at post-test (β= 3.44, SE = 1.52, Z = 2.26, p =.02) in comparison to standard child mental health services (Chacko et al., under review; Gopalan et al., in press). No evidence of moderation effects on family stress, child welfare involvement or parental needs were uncovered.

Figure 1.

MFG Attendance (in comparison to rates on retention in outpatient urban individualized mental health services)

Table 5.

Mean & Median Attendance between Stress & Comorbid Stress/Depression Groups

Low Stress Caregivers	Mid Stress Caregivers	High Stress Caregivers	Low Comorbid Stress/Depression Caregivers	Mid Comorbid Caregivers	High Comorbid Caregivers
N=47	88	56	57	90	44
M=10.40	9.62	8.34	8.72	10.17	8.84
SD=4.68	SD=5.09	SD=5.51	SD=5.3	SD=4.87	SD=5.43

Table 6.

Univariate analyses of average number of sessions attended for child welfare involved families relative to non-child welfare involved families (by number of sessions offered and percentage offered per quarter – 4 weeks – of intervention)

	Total(n = 224)			Child Welfare Involved (n = 83)			Non-Child Welfare Involved(n = 141)
Variable	N	Mean	SD	n	Mean	SD	n	Mean	SD
Mean Attendance	224			80	8.46	4.89	140	9.17	4.97
12 Sessions	42	8.02	4.11	13	7.23	4.38	29	8.38	4.01
16 Sessions	178	9.14	5.11	67	8.70	4.98	111	9.38	5.18

Table 7.

Multivariate analyses comparing MFG Attendance over time by Child Welfare Status

Variable	Estimate (β)	Std. Err.	Z-value	p-value
intercept	0.69	0.03	20.03	0.00	**
Child Welfarea	−0.03	0.06	−0.49	0.62
Attendance (quarter)	−0.04	0.01	−3.65	0.00	**
Child Welfare X Quarter	−0.01	0.02	−0.49	0.63

^**p < .005

^a

Child welfare status indicator: 0 = non-child welfare involved, 1 = child welfare involved

Log Likelihood	−142.28
−2 Log Likelihood (Deviance)	284.57
Akaike’s Information Criterion	302.57
Schwarz’s Bayesian Criterion	332.21
Number of free parameters	9

Table 8.

Mixed Effects - Risk for Dropout (non-attendance)

	Model 11				Model 22
Effect	b	SE	HR	95% C.I.	p	b	SE	HR	95% C.I.	p
Fixed Effects
High Caregiver Stress1	0.29	0.13	0.75	0.58–0.97	0.03*	--	--	--	--	--
(Reference: Low Stress)
High Comorbid Stress/Depression2	--	--	--	--	--	0.2	0.14	0.74	0.60–0.91	0.02 *
(Reference: Low Comorbid)
Childhood Behavioral Difficulties3	0.01	0.01	0.99	0.96–1.01	0.37	0.01	0.01	1.01	0.98–1.04	0.57

^*p≤ 0.05

^**p≤ 0.01

^***p≤ 0.001

¹Parent Stress Index (PSI-SF; Abidin, 1995). Categorical scale.

²Comorbid Stress & Depression Scale (α=0.82). Categorical scale.

³IOWA Conners Rating Scale (Loney & Milich, 1982). Continuous scale.

Discussion and Implications

The multiple family group is a family-centered, group delivered, evidence-informed service delivery approach that has been designed based upon research related to factors that impact the engagement and retention of children and families, specifically: 1) seeking services is often associated with stigma (Alvidrez, 1999); 2) parents of children with mental health difficulties have reported fears of being blamed for their child’s problems and these fears may in turn influence decisions to continue in services over time (McKay et al.,1996), and; 3) mutual support and normalization of family struggles with child mental health needs could create more receptivity to treatment and potentially offer encouragement for family-level change needed to reduce child disruptive behavioral difficulties (McKay et al., 2010).

Further, the MFG capitalizes on: 1) empirically supported, family-focused approaches consistently associated with reductions in child disruptive behavior (Bank et al., 1991; Kumpfer & Alvarado, 2003; Webster-Stratton, 1985; 1990; Sexton & Alexander, 2002); 2) a protocol driven approach that has been developed with maximum input from youth, adult caregivers and providers and successfully implemented by “real world” service providers across a diverse array of outpatient clinic settings (McKay et al., in press; McKay et al., 1998; McKay, Harrison et al., 2002) and; 3) accumulated data supporting an association between MFG service involvement and improvements in engagement and child/family-level outcomes (Fristad et al., 2003; 2002; McKay, Harrison et al., 2002; McKay, Gonzales, Quintana et al., 1998; Stone & McKay, 1996).

Finally, the MFG service delivery model has been specifically designed to target a set of weaknesses, namely insufficient capacity and high inefficiency within the current delivery system (McKay et al., 2004). Even the current number of children and their families approaching the public mental health service system far outstrips the availability of services and the number of service providers. Thus, MFGs are meant to specifically expand opportunities to receive care within provider organizations that struggle with service capacity and adequate levels of funding and which also have no reasonable expectation for additional resources to expand service slots in the near term.

Results indicate that although there were subgroups of families who experienced high levels of stress, parental mental health need and/or child welfare involvement, the 4Rs 2Ss program appeared to be able to engage families with more complex needs overall with some evidence that stress/depression was related to attendance. However, in relation to outcomes, improvements were noted overall over time for experimental group participants relative to comparison services and there was no evidence that baseline needs related to stress/depression or child welfare involvement undermined improvement in child outcomes overall.

These results need to be interpreted cautiously as this is the first large scale study that has assessed the 4Rs 2Ss Family Strengthening Program. These findings are in need of replication with a larger sample of families given this sample size may have hampered exploration of these moderators. However, given the high rates of overall attendance modeled in Figure 1, the 4Rs 2Ss shows promise in overcoming the serious engagement challenges too often seen within urban service delivery.

Figure 2.

Figure 3.

Table 9.

Means and Standard Deviations for Outcomes Measures by Time and Treatment Group

	MFG								SAU
Variable	Baseline M (SD)		Mid-test M (SD)		Post-test M (SD)		6 month Follow- up M (SD)		Baseline M (SD)		Mid-test M (SD)		Post-test M (SD)		6 month Follow- up M (SD)
Iowa Connors OD	9.28	(3.4)	7.7	(3.6)	7.8	(3.8)	7.4	(3.9)	9.1	(3.7)	8.3	(3.5)	9.0	(3.8)	8.83	(4.0
SSRS-SSS	38.27	(10.6)			43.6	(11.5)	43.2	(12.0)	39.7	(11.1)			39.7	(10.8)	41.51	(12.

a						34
Tests of Group Differences
Outcome Assessment Period Variable	Contrast Estimate (b)	SE	Z- statistic	p-value	Effect Size (Cohen’s d)
Iowa Mid-test	−0.46	0.52	−0.88	0.38	0.13
Connors OD Post-test	−1.22	0.49	−2.47	0.01	* 0.35
6 month Follow-up	−1.17	0.51	−2.29	0.02	* 0.34
SSRS-SSS Post-test	3.44	1.52	2.26	0.02	* 0.32
6 month Follow-up	1.33	1.59	0.84	0.4	0.12

MFG= Multiple Family Group; SAU= Services As Usual; IOWA Connors ODD= Iowa Connors Oppositional/Defiant Subscale; SSRS-SSS= Social Skills Rating Scale Social Skills Subscale

Note. IOWA Connors OD= Iowa Connors Oppositional/Defiant Subscale; SSRS-SSS= Social Skills Rating Scale Social Skills Subscale