Evaluating the Efficacy of the Family Check-Up Online: A School-Based, eHealth Model for the Prevention of Problem Behavior During the Middle School Years

Abstract

This study evaluated the efficacy of a family-centered preventive intervention, the Family Check-Up (FCU), delivered as an online, eHealth model to middle school families. To increase accessibility of family-centered prevention in schools, we adapted the evidence-based FCU to an online format, with the goal of providing a model of service delivery that is feasible, given limited staffing and resources in many schools. Building on prior research, we randomly assigned participants to waitlist control (n = 105), FCU Online as a web-based intervention (n = 109), and FCU Online with coaching support (n = 108). We tested the effects of the intervention on multiple outcomes, including parental self-efficacy, child self-regulation, and child behavior, in this registered clinical trial (NCT03060291). Families engaged in the intervention at a high rate (72% completed the FCU assessment) and completed 3-month post-test assessments with good retention (94% retained). Random assignment to the FCU Online with coaching support was associated with reduced emotional problems for children (p = .003, d = −.32) and improved parental confidence and self-efficacy (p = .018, d = .25) when compared with waitlist controls. Risk moderated effects: at-risk youth showed stronger effects than did those with minimal risk. The results have implications for online delivery of family-centered interventions in schools.

More than 40 years of prevention research have demonstrated the efficacy of family-centered approaches for improving a range of child problem behaviors and negative health outcomes, including aggression, delinquency, depression, achievement problems, and substance abuse. Parenting skills training and support for families have consistently been shown to be the most effective interventions for improving long-term child behavioral outcomes across development, with effects lasting from middle school to the young adult years (Dishion, Chamberlain, Forgatch, & Pelham, 2016; Kazdin, 2010, Prinz & Dumas, 2004; Stormshak, DeGarmo, Chronister, & Caruthers, 2018). The efficacy of these models has been well established; however, successful dissemination of family-centered prevention into community systems, such as schools, has been more difficult to accomplish. Much of this work was conducted by Dr. Thomas Dishion and his colleagues, and as such, this article is a tribute to his work and legacy. Dr. Dishion’s focus on dissemination and implementation toward the end of his career, particularly his work on disseminating the Family Check-Up (FCU) model in schools, has inspired this research and our path forward.

Despite overwhelming evidence suggesting the efficacy of family-centered interventions, attempts to scale up systemwide family-centered, behavioral health treatments in schools have been met with a variety of challenges. One problem is that a high proportion of children and adolescents in schools never receive treatments they need, and a very small percentage of parents participate in parenting or family interventions to address behavior problems (Prinz & Sanders, 2007; Zubrick et al., 1995). The result is an overwhelming level of need complicated by limited staff resources or support infrastructures in schools to manage the demand for services. For example, of students in need of behavioral support, two-thirds of their parents never have any contact, help, or guidance from public school professionals (Noel, Stark, Redford, & Zukerberg, 2015).

One approach to address the limited access to mental health treatments for children has been an increase in school-based mental health programs. In these programs, schools partner with mental health agencies to provide services to students and families or provide mental health services from within the school. While the number of school-based mental health programs continues to increase in the United States, the many barriers to implementing them effectively have not been overcome, despite more than 20 years of focused attempts (Weist, 2005; Weist et al., 2014). School-based mental health is most effective when the interventions delivered are targeted and focused on children with externalizing problems (Sanchez et al., 2018), because school staff are more likely to observe externalizing behavior and apply interventions to those students. The majority of school-based interventions that target at-risk or high-risk children focus exclusively on the child or on the school context (e.g., Positive Behavioral Interventions & Supports [Horner, Sugai, Todd, & Lewis-Palmer, 2005]; Good Behavior Game [Kellam et al., 2008]). Typically, school-based interventions use a response-to-intervention (RTI) approach that frames services within a tiered (universal, selected, and indicated interventions) model delivered in the school, but they often lack functional coordination between home and school. Few of these interventions target known risk factors for mental health and substance use problems, such as family management skills, contextual risk factors, and long-term family support. As a result, these interventions are often associated with short-term gains that are context specific and lack generalizability.

Our scientific knowledge base demonstrates that for nearly all effective child and adolescent mental health and substance use interventions, working with families is a primary underpinning. Unfortunately, school-based intervention staff continue to struggle with successfully engaging and collaborating with families (Hoagwood, 2005; Jensen & Hoagwood, 2008; Smolkowski et al., 2017; Weist et al., 2014). School-based interventionists face an almost insurmountable array of parent engagement challenges, including transportation issues; school operational hours; lack of child care options for younger siblings; school-based staff comfort with, and training for, partnering with families; and the ongoing training issues and expense caused by administrative and staff turnover and other competing educational priorities (Forman, Olin, Hoagwood, Crowe, & Saka, 2009; Stormshak et al., 2016). As a result, most interventions that target parenting practices are unrealistic for schools (Christensen, 2003). Yet, data strongly suggest that helping parents engage in parent management through motivational strategies creates long-term change (Dishion, Forgatch, Chamberlain, & Pelham, 2016; Dishion & Kavanagh, 2003; Kazdin, 2002; Stormshak, Fosco, & Dishion, 2010) and that interventions for high-risk students in public school environments that target parenting are efficacious (e.g., Atkins, Hoagwood, Kutash, & Seidman, 2010). Thus, it seems imperative that we find a cost-effective, efficient, and pragmatic means of improving student education and behavioral health success rates using parenting interventions, in spite of the dire economic situations and parent engagement challenges facing most school behavioral health systems today. It is also critical that any pragmatic intervention solution be designed such that it can be implemented with fidelity without requiring extensive school staff time.

The FCU is a brief, cost-effective preventive model that potentially fits these criteria. The FCU was originally developed to be delivered in community-based settings, including schools. This ecological developmental model of adaptation and risk behavior (see Figure 1) was iteratively developed in a series of randomized trials to prevent the escalation of problem behaviors in young adolescents. The FCU was designed to target early-adolescent risk factors known to predict later problem behavior and substance use, such as poor parental monitoring, deviant peer affiliation, and lack of positive parenting (Dishion, McCord, & Poulin, 1999; Patterson, Dishion, & Yoerger, 2000). It was developed as a potential alternative to interventions for antisocial behavior of the late 1980s and early 1990s that focused on group-based treatments. Groundbreaking work by Dr. Dishion (Dishion & Andrews, 1995) showed these interventions to be iatrogenic and harmful and to lead to increases in behavior problems rather than reductions. This work led to the development of the FCU, an individualized, tailored approach to intervention that did not involve group treatment for problem behavior.

Figure 1.

Ecological model predicting adaptation from middle to late adolescence.

The FCU evolved from a series of efficacy trials (Dishion & Kavanagh, 2003; Dishion et al., 2008; Dishion & Stormshak, 2007; Stormshak, Dishion, Light, & Yasui, 2005; Stormshak et al., 2011), with the goal of implementation within a multi-tiered school-based family engagement model (Dishion, 2011; Fosco et al., 2014). The original FCU, developed by Dishion and Kavanagh, was applied to middle school youth in a randomized controlled trial (RCT) called Project Alliance, initiated in 1994. Since then, more than 30 peer-reviewed articles, published by multiple authors about these data, have demonstrated the impact of the FCU on both prevention of problem behavior and promotion of healthy child development. Project Alliance 2, beginning in 2005, replicated the original research and expanded the model to include a briefer assessment, culturally relevant observational tasks, and universal delivery in the middle school environment, and families were selected through an adapted, tailored design for additional support and feedback (Dishion & Stormshak, 2007; Stormshak et al., 2010). The efficacy of the FCU has been shown in terms of both proximal and distal developmental outcomes, some more than 10 years after intervention exposure. These effects include decreases in substance use through high school and into early adulthood; reduction in antisocial behavior, including fewer arrests; decreases in deviant peer association; lower levels of adolescent depression and suicide ideation; and stabilization of attendance and academic achievement through high school (Fosco, Frank, Stormshak, & Dishion, 2013; Stormshak et al., 2011; Stormshak et al., 2010; Van Ryzin, Stormshak, & Dishion, 2012). Similar FCU findings have been demonstrated among samples of early childhood and elementary students, such as increases in school readiness mediated though increases in inhibitory control and language development (Lunkenheimer et al., 2008) and improvements in teacher ratings of problem behavior after delivery of the FCU at kindergarten entry (Garbacz, McIntyre, & Stormshak, 2018). Analyses have strongly suggested that changes in the parenting constructs of positive parenting, limit setting, and monitoring, and decreased family conflict mediate the positive developmental outcomes and reductions in risk behavior over time (for a more thorough review, see Dishion et al., 2016).

To build on this foundation of efficacy trials, we conducted an effectiveness trial of the FCU in 41 middle schools where the FCU was further adapted to be embedded within a tiered family engagement model (Positive Family Support) synergistic with ongoing school-wide positive behavior interventions and support programs (PBIS; Dishion et al., 2019; Fosco et al., 2014; Smolkowski et al., 2017). In this effectiveness trial, we encountered many of the same economic and systemic barriers to effective parent engagement. Although we found that at-risk students responded more favorably to the intervention, with parents reporting fewer negative contacts with schools in the intervention group, our findings overall were quite limited. Perhaps even more frustrating was how difficult it was to get school staff to engage caretakers in the FCU intervention component, which likely contributed to our limited findings. In prior RCTs, engagement in the FCU was the most important predictor of improved outcomes (Connell, Dishion, Yasui, & Kavanagh, 2007; Stormshak, Connell, & Dishion, 2009; Van Ryzin et al., 2012). Multiple factors led to our implementation problems and limited findings, including poor school funding and staffing resources; high teaching staff and administrative staff turnover; changes in staffing for support specialists, including counselors and behavioral support specialists who were delivering the intervention; staff time allocation; and the challenge of motivating school counselors and mental health staff to adapt their work to focus on family and parent systems rather than exclusively on the child (Dishion et al., 2019; Stormshak et al., 2016).

The intensity, frequency, and duration of the challenges we faced engaging parents in the FCU within a school setting suggested that an electronic health (eHealth) intervention developed around the FCU and delivered via the Internet might help solve many of these implementation challenges. Advantages of an eHealth approach include ability to develop an FCU intervention that could be stand-alone (fully automated), thereby reducing delivery and training costs and concurrently, increasing public health reach. The eHealth version may also include live contact with counseling or coaching staff, either via face-to-face sessions or via phone or video calls. The coach option, when feasible, is hypothesized to help increase adherence through a supportive accountability model (Mohr, Cuijpers, Lehman, 2011) and has been associated with enhanced outcomes for eHealth interventions when compared with those delivered without a coach or support (Day & Sanders, 2018). A coach model for the FCU would also enable titration of support and intensity of delivery, consistent with the adaptive, tailored model of FCU intervention delivery.

Increases in multimedia technology and Internet usage make it easier for mental health interventions to reach much of the population. In a recent report by the Pew Research Center in 2018, roughly 90% of adults currently use the Internet and 95% use a mobile phone (77% smartphones). Recent advances in web-based mental health and substance use intervention technology (Danaher, McKay, & Seeley, 2005, Danaher et al., 2013; Danaher & Seeley, 2009) suggest that schools and their families could benefit from a brief, online family management intervention adapted and targeted to their needs.

Improvements and increases in technology and broadband Internet enable interactive Internet access and have been linked with higher levels of engagement in interventions with families (Jones, 2014). Several evidence-based parenting curricula have been adapted to web-based programs that are self-administered or assisted by trained human support (Cardamone-Breen et al., 2018; Enebrink, Högström, Forster, & Ghaderi, 2012; Gelatt, Adler-Baeder, & Seeley, 2010; Kling, Forster, Sundell, & Melin, 2010; Sanders, Baker, & Turner, 2012; Sanders, Kirby, Tellegen, & Day, 2014; Sourander et al., 2016; Yap et al., 2018). A number of web-based parenting interventions have been effective in reducing risk outcomes in children. For instance, the Triple-P Online Brief (TPOL-Brief; Turner & Sanders, 2013) is a brief web-based self-directed application of the Triple-P (Positive Parenting Program), a multilevel system of both universal and targeted interventions for parents of children with early-onset conduct problems (Baker, Sanders, Turner, & Morawksa, 2017; Sanders et al., 2012). Results from an RCT evaluating the efficacy of the TPOL-Brief showed decreases in the use of ineffective parenting strategies and higher levels of parent confidence in self-efficacy, with outcomes sustained at 8-week and 9-month follow-up (Baker et al., 2017). Results from other RCTs that examined the efficacy of the TPOL-Brief have demonstrated significant improvements in parenting and child behavior outcomes, including lower levels of dysfunctional parenting, lower levels of disruptive child behavioral and emotional difficulties, less reported parental anger and use of negative parenting styles, reductions in partner conflict, and lower levels of maternal adjustment difficulties (Sanders et al., 2012; Sanders, Kirby, Tellegen, & Day, 2014). In a meta-analysis of digital-based parent training for parents of children with disruptive behaviors, researchers found positive improvements in outcomes of child behavior, parent behavior, and parental confidence in self-efficacy (Baumel, Pawar, Kane & Correll, 2016). An RCT by Cardamone-Breen et al. (2018) implemented a single-session, individually tailored, web-based parenting intervention that targeted parenting factors known to affect the development of depression and anxiety in adolescents. The intervention resulted in significant improvements in parenting factors but did not support improvement in adolescent depression or anxiety symptoms. In addition, results from a study that examined the effects of Partner in Parenting, a tailored web-based parenting intervention designed to prevent depression and anxiety symptoms in adolescents, indicated positive short-term improvements in parenting behaviors associated with adolescents’ risk for depression and anxiety (Yap et al., 2018).

Adherence to and effectiveness of online behavioral health treatments can be enhanced by the inclusion of guided human support (e.g., technician-level coach, skilled therapist; Mohr et al., 2011). For instance, an online application of the COMET parent management training program, a web-based program with email support and an online discussion forum, demonstrated lower levels of child behavior problems, with outcomes maintained at 6-month and 18-month follow-up assessments (Enebrink et al., 2012; Högström, Enebrink, Melin, & Ghaderi, 2015). Also, in a study comparing Triple-P Online with and without telephone support, results suggested that although self-directed online approaches to parenting interventions are effective, telephone support enhances effective engagement and outcomes for families (Day & Sanders, 2018). However, in another evaluation study of the Triple-P Online, results demonstrated no significant differences between the types of delivery format (Sanders, Dittman, Farruggia, & Keown, 2014). Clearly, there is support for delivery of parenting skills interventions via the Internet, and a variety of programs have adapted content for online delivery with some success. No programs, however, had been adapted for school-based delivery with the goal of supporting schools in their intervention delivery to families of at-risk youth.

In this study, we tested the efficacy of FCU Online, a web-based, eHealth implementation of the FCU intervention, and compared delivery of the model as a stand-alone, web-based delivery versus a web-based, coach-supported delivery model. Building on the adolescent version of the FCU originally tested in Project Alliance, we randomly assigned 322 families of middle school youth to receive either the intervention delivered with a coach or as a stand-alone web program, or to a waitlist control. We predicted that delivery with the coach would be associated with improved outcomes when compared with web-only or waitlist control, including reductions in child behavior problems, emotional problems, and parent self-efficacy. In prior research using the FCU, we found that high-risk families benefit more than do low-risk families across a number of outcomes, including parenting skills (Dishion et al., 2008; Leijten et al., 2015; Stormshak, Connell, & Dishion, 2009). We have found few moderating effects of gender and race in prior studies, although a preventative effect for depression was found for females who started with a high baseline of antisocial behavior and low levels of depression in middle school, suggesting the FCU prevented the growth of depression over time into the high school years (Connell, Stormshak, Dishion, Fosco, & Van Ryzin, 2018). On the basis of this prior research, we predicted that at-risk youth and their parents would show greater improvements than would those who were not at risk (Smolkowski et al., 2017). We also examined the potential moderating effects of child gender and race/ethnicity, and family socio-economic status, but we had no predictions regarding their impact on treatment outcomes.

Method

Participants

Participants were primary caregivers of middle school children across eight schools in Oregon, including two rural, two urban, and four suburban schools ranging in size from 294 to 839 students. Schools were chosen for participation in the project based on the percentage of students who qualified for free or reduced-cost lunch; this population ranged from 52% to 100%, with an average of 68%. During two consecutive school years, all families with a sixth or seventh grade student enrolled in one of the participating schools were contacted and invited to participate via an email message sent to families from the principal. From this population, 432 families initially responded. The CONSORT diagram (Moher et al., 2010) presented in Figure 2 displays participant flow and allocation to condition. Some of the initial respondents were excluded from the study because they did not return the initial survey (n = 66), declined to participate (n = 8), or did not meet inclusion criteria (n = 11; i.e., not in sixth or seventh grade, not a student at a participating school). Siblings, those who were primarily Spanish speakers, and families who had participated in other FCU studies (n = 25) were assigned to a nonexperimental condition during which they received support but were not included in the analytic sample. Families who consented and returned the baseline assessment (n = 322) were randomly assigned to one of three conditions, including FCU Online with coach (web+coach; n = 108), FCU Online delivered as a web-only intervention (n = 109), and a waitlist control group (n = 105). Our goal was to recruit 300 families for the study, and once we had reached our goal we discontinued recruitment in the schools.

Figure 2.

Participant flow and allocation to intervention condition.

Table 1 reports demographic characteristics of participants across the three conditions. The parents in the study were predominately female and were average age 40.0 years (SD = 6.1); 35% had a 4-year college degree or graduate training. The children in the study were approximately gender balanced, mostly White, and most were living in suburban environments.

Table 1.

Study Demographic Characteristics

	Waitlist control (n = 105)		Web-only intervention (n = 109)		Web+coach intervention (n = 108)
	n	%	n	%	n	%
Parent’s gender (% female)	96	91.4	93	85.3	102	94.4
Parent’s education level
Less than high school degree	1	1.0	2	0.9	3	2.8
High school degree or GED	18	17.1	19	17.4	11	10.2
Partial college	36	34.3	34	31.2	37	34.3
2-year associates degree	12	11.4	13	11.9	19	17.6
4-year degree	24	22.9	25	22.9	21	19.4
Graduate training.	14	13.3	16	14.7	17	15.7
Child’s gender (% female)	49	46.7	54	49.5	51	47.2
Child’s race
Asian	1	1.0	3	2.8	0	0.0
Black	1	1.0	4	3.7	1	0.9
Hispanic	4	3.8	8	7.4	9	8.4
Native American	1	1.0	1	0.9	2	1.9
Multiethnic	17	16.3	24	22.2	25	23.4
White	80	76.9	68	63.0	70	65.4
Location
Rural	20	19.0	22	20.2	22	20.4
Suburban	58	55.2	61	56.0	61	56.5
Urban	27	25.7	26	23.9	25	23.1

Procedure

Families were invited to participate in the study via an email sent to them from their school. This email explained the study and included a link to a Qualtrics form where interested parents could provide their contact information. A research staff member then contacted all interested parents to explain the goals of the study and provide details about participation. Parents who indicated interest in participating in the study were then mailed a packet that included a parent consent form, a youth assent form, and the parent and youth surveys. Once families returned their surveys and consent/assent forms, they were randomly assigned to one of three experimental conditions. Parents were then sent an email with information about their next steps, relative to their experimental condition.

Condition.

Parents in the two treatment conditions received login information and support to begin the online program. Waitlist control participants were told that they would receive the intervention after the assessment period was completed, and that they would next hear from project staff in 3 months. For families assigned to a treatment condition, one parent per family was chosen as the primary target for the study, although multiple caregivers could participate in the intervention if desired. Participants in the web-only condition were invited to log in to the FCU Online website, where they were asked to complete an intervention assessment, received feedback about their data, and were provided with online tools to support their parenting in areas that were identified as challenges by the assessment. Participants in the web+coach condition logged in to the same FCU Online website, where they had access to the same assessment, materials, and tools. In addition, a family coach contacted parents to help them set goals, talk through their results, offer support via the phone, and help motivate them to improve parenting practices. Families in all three conditions were mailed post-test questionnaires 3 months after they completed the pre-test, regardless of their condition. Parents were paid $100 for each assessment wave.

Training of therapists.

Family coaches in this study were doctoral-level or master’s-level psychologists who had been previously trained in the FCU through a variety of means, such as attending a training workshop and working on prior projects that used this model. Specific training content included training on the protocol, developmental norms, motivational interviewing techniques, and specific support for parents of middle school youth (Stormshak & Dishion, 2009). After training, therapists were observed and scored on the COACH rating scale for fidelity over five coached sessions with a parent (Smith, Dishion, Shaw, & Wilson, 2013). All therapists’ COACH ratings were within the satisfactory range (ranging from 4.8 to 6.8; minimum score of 4 out of 9). Once therapists were providing the FCU Online independently, weekly group supervision meetings that emphasized case conceptualization and delivery of feedback were held to maximize treatment fidelity. Supervision was provided by licensed, clinical psychologists with experience delivering the FCU.

Family Check-Up (FCU)

The original FCU model involves three steps and a menu of intervention services that are adapted and tailored to families’ needs. It follows a tiered service delivery model (Dishion & Kavanagh, 2003; Dishion & Stormshak, 2007). Consistent with an RTI approach to service delivery (Fuchs & Fuchs, 2006), targeted interventions are modified to fit the families’ needs based on assessment and risk status.

Key features.

The FCU’s key features make it an ideal intervention for online delivery: (a) interventions are tailored and adapted to family needs on the basis of assessments and collaborative agreements with family caregivers, (b) parents’ motivation to change is explicitly targeted in the feedback session, and (c) a health maintenance model of intervention is followed, with interventions repeated as needed. These features make the FCU cost effective and viable for the prevention of long-term patterns of problem behavior and substance use.

The FCU is a selected intervention in that it engenders training in skilled family management to encourage positive family interactions and improve family management skills. For some families, the FCU will be the primary intervention and will motivate caretakers to engage in systematic changes that will reinforce their youth’s well-being. For other families who are at higher risk, follow-up intervention services and support will be indicated. In this study, four types of follow-up intervention services—positive behavior support, limit setting, monitoring, and effective problem solving and communication—were available to caregivers who were randomly assigned to the FCU Online, with services tailored and adapted to the child’s and family’s specific needs. The intervention actively promoted self-selection into the most appropriate services.

FCU intervention protocol.

The FCU Online version motivates caregivers to promote and use positive parenting skills and effective communication with their adolescent to reduce family conflict and subsequent problem behavior. The original FCU involves at least three sessions, including (a) an initial rapport-building interview, (b) an assessment with the caregiver(s), and (c) a feedback session during which the results of the assessment and initial interview are discussed with the caregivers, attention is focused on the caregivers’ readiness to change, and specific change options are delineated.

The FCU is a motivational intervention designed to elicit improved parenting and enhance contextual supports for children. Motivational interviewing (MI), first developed as a treatment for substance abuse (Miller & Rollnick, 2002), elicits change by activating motivation from within a collaborative relationship. MI has been successfully applied to many school mental health programs and is natural to the school setting in that the interventions associated with MI are brief and targeted (Frey et al., 2011). MI has also been successfully adapted to a variety of Internet interventions (Friederichs, Bolm, Oenema, Guyaux, & Lechner, 2014; Ingersoll et al., 2015; Mohr et al., 2013).

Basic Process of FCU Online

To begin using the FCU Online program, parents in both treatment conditions first logged in to the website on their computer or mobile device using their family identification number. This step allowed families to create a page that was exclusively viewed by the parent or parents who were participating. Parents first identified strengths and challenges to parenting their middle school child as part of the initial assessment. Then they completed the FCU assessment, which included content similar to prior protocols but abbreviated for online delivery. Measures targeted key domains relevant to middle school youth, such as achievement at school, problem behavior, emotional problems, parenting skills, family conflict, and motivation to change. Parents created goals and learned about the FCU Online process. After the assessment was completed, a computer-generated feedback form was available to the family. In the web+coach condition, completion of the assessment prompted the program to notify the coach to call the family to schedule a time to review the feedback and results. In the web-only condition, feedback was delivered in an interactive format, with parents able to explore areas of strengths and challenges and learn more about their own family and assessment results.

The family feedback session occurred either on the phone or via video conferencing (e.g., Skype) for the web+coach version, or automated online for the web-only version. The feedback session was both strengths-based and specific. The goals of feedback sessions were to (a) share assessment findings with caregivers regarding areas of strengths and challenges, (b) engage in a motivationally enhanced discussion about promoting positive changes, and (c) provide a menu of resources to facilitate the family change process. Emphasizing areas of strengths and challenges reduces parents’ defensiveness and enables them to feel empowered to pursue strategies for making changes relevant to their family’s needs.

After the feedback, a menu of options tailored for the family and based on the assessment was presented. These options included parenting modules guided by our Everyday Parenting curriculum (Dishion, Stormshak, & Kavanagh, 2011). The choice to continue with the curriculum and pick options based on the assessment or parent interest is part of the model of tailoring and adapting the intervention to the family and integrates the principles of MI by allowing for self-directed change. The parenting modules consist of content and structured activities that are designed to help parents learn each skill. Parents were also given the opportunity to practice parenting skills and track their progress. Parents who chose to engage in this practice could sign up to receive text messages that periodically prompted them to try out new skills learned from the website. A coach checked in with families in the web+coach condition to assess progress and support gains in positive behavior. Parents could log in to the website as often as they would like, to learn more or participate in any of the other modules. Families in the coach condition could choose to participate in at least one therapy/coaching contact; 73% chose to do so, and 72% chose to review the FCU with their coach during the coaching sessions. Of those who received any coaching contact, the average number of contacts was 1.8 (SD = 1.2).

Content of Follow-Up Sessions and Web Activities

On the basis of assessment results, parents were offered a variety of follow-up parenting skill activities. Interventions delivered online must be brief and effective in order to be viable, and they must capture that attention of participants quickly in order to increase engagement (Kelders, Bohlmeijer, & Van Gemert-Pijnen, 2013). The tailored and adaptive nature of the FCU intervention gives it an online delivery potential that is brief, meaningful, and interactive. Each of the following online modules derived from our Everyday Parenting curriculum (Dishion et al., 2011) included content that is animated, video based, and interactive, with activities for tracking behavior and providing ongoing feedback to parents about their success. Each module also highlighted behaviors that target school success at this age, including homework completion, attendance, and home-to-school planning.

Positive behavior support module.

Three teaching content areas focus on positive behavior support: making effective requests, praise, and incentives for behavior. The “making requests” area helps parents understand and learn basic skills for promoting behavior change in their child. The “praise” content area shows parents how to develop skills for using positive language with their adolescent. The “incentive” content area helps parents develop skills for using incentives and planning how to manage their child’s behavior.

Limit-setting module.

This module includes tracking behavior, providing structure and clear instructions, and identifying appropriate consequences. In the “tracking behavior” area, parents are asked to track both desired and undesired behaviors over time, such as homework completion. The “providing structure” area shows parents how to establish clear commands and expectations, and “identifying appropriate consequences” includes working with the adolescent to develop incentives and consequences that are meaningful for reducing negative behavior and increasing positive behavior.

Monitoring.

This content area provides guidance in listening skills and clear expectations while tracking the child’s activities with friends and at home. Monitoring includes supervising cell phone usage, Internet activities, and peer interactions at home, at school, in the neighborhood, and online. A number of activities to help parents develop these skills are available.

Problem solving and communication.

This content area teaches effective problem solving and negotiation skills. The online module presents relevant skills, such as accepting responsibility, recognizing the youth’s positive contributions, and building on strengths. Activities and interactive content are delivered to improve problem solving and communication skills, both at home and with school personnel, including activities online and via mobile texting. Additional details regarding the formative development of the FCU Online program are reported in Danaher et al (2018).

Measures

Demographic characteristics.

Family background variables were obtained from primary caregivers through the use of standard demographic questions, including queries about parent gender, age, and education level, as well as child gender and race/ethnicity. A family’s geographic location was determined by the school the child attended. Schools were categorized as urban, suburban, or rural, relative to their communities.

Program use.

Participants’ usage of the online program was assessed unobtrusively by the website to allow us to observe how often and for how long each participant interacted with the online program overall, as well as with each module of the program (e.g., the “monitoring” module) and with various engagement activities (e.g., videos and animations, practice activities). For the present analysis, we investigated number of unique logins, total time spent in the program, time spent on assessment and in the library of available parenting support materials, and time spent on each of the four modules (positive behavior support, limit setting, monitoring, problem solving and communication).

Outcomes

Effortful control.

Parents reported on their teen’s effortful attention control via questionnaire using an eight-item subscale of the Early Adolescent Temperament Questionnaire (Ellis & Rothbart, 2005). In this questionnaire, parents are asked to use a five-point scale with endpoints of almost always untrue and almost always true to indicate how well each of eight statements describes their child, such as “has a hard time finishing things on time” and “pays close attention when someone tells her/him how to do something” (effortful control M = 3.33, SD = 0.92; α = .91).

Problem behavior.

Parents reported on their teen’s problem behavior using the Strengths and Difficulties Questionnaire (SDQ; Goodman, 1997), a well-validated measure for assessing problem behavior at this age. The SDQ is a 25-item brief behavioral screening questionnaire that consists of five subscales with five questions each. For the purposes of the present analysis, we focused on three subscales: Emotional Problems, Conduct Problems, and Hyperactivity. In this questionnaire, parents were asked to indicate the extent to which statements were true of their child’s behavior in the past month using a three-point scale (not true, somewhat true, and certainly true). The Emotional Problems subscale included items such as “many worries or often seems worried” and “often unhappy, depressed, or tearful” (M = 2.51, SD = 2.24; α = .73). The Conduct Problems subscale included items such as “often loses temper” and “often fights with other youth or bullies them” (M = 1.29, SD = 1.67; α = .71). The Hyperactivity subscale includes items such as “restless, overactive, cannot sit still for long” and “easily distracted, concentration wanders” (M = 3.71, SD = 2.69; α = .80).

Parenting skills and self-efficacy.

Using a measure developed for the purposes of this study, parents were provided with a list of 14 parenting skills that closely corresponded to skills that were supported in the FCU Online parenting curriculum, such as “providing praise and encouragement for good behavior” and “setting clear rules and expectations for behavior.” For each parenting skill, parents were asked to indicate the importance of this skill during the middle school years using a five-point scale with end points of not at all important and very important (sense of parenting importance M = 4.52, SD = 0.29; α = .72). They were also asked to indicate their level of confidence in using each skill with their child via a five-point scale with endpoints of not at all confident and very confident (sense of parenting confidence M = 4.14, SD = 0.57; α = .87).

Parent self-efficacy was also assessed with eight items adapted from the Parenting Task Checklist (Sanders & Woolley, 2005). Using a five-point scale with endpoints of not confident at all and very confident, parents were asked to indicate their level of confidence in handling eight negative child behaviors, such as “your child arguing with you about rules” or “your child’s mood swings” (confidence in teen behaviors M = 2.76, SD = 0.79; α = .92).

Moderators

In addition to child’s gender and racial/ethnic minority status, moderators included family socioeconomic status (SES) risk and child’s pre-test behavior problems. Family SES risk was assessed as a cumulative risk index (range 0–6, M = 1.03, SD = 1.23) measuring multiple risk factors. The index is based on six dichotomous demographic indicators: the parent does not have a high school diploma or GED, the parent is single, the parent’s income is below the poverty threshold (relative to family size), the parent is a recipient of financial assistance (e.g., food stamps), the parent is currently unemployed, or the family’s home is overcrowded (in the bottom one-third of the sample for room-to-occupant ratio).

Child’s pre-test behavior problems is a baseline risk behavior measure based on the SDQ Total Difficulties score, which includes the Hyperactivity, Emotional Problems, and Conduct Problems subscales, as well as the Peer Problems subscale. The Peer Problems subscale is similarly scored to the other subscales and is based on five items, such as “has at least one good friend” and “generally liked by other children.” Scores for each of the four subscales were categorized according to normal, borderline, and abnormal score thresholds, per Goodman (1997). The pre-test behavior risk measure thus identifies the highest threshold (0 to 2) of risk a child scored on any of the four subscales.

Data Analysis

Preliminary analyses included a descriptive summary of participant characteristics and use of the website. Descriptive statistics and plots were then used to screen outcomes for normality. Next, chi-square tests and one-way ANOVAs were used to compare conditions on demographic characteristics and pre-test values of the outcomes to assess whether randomization produced initially equivalent groups. A multivariate logistic regression model was used to compare study condition, demographic characteristics, and pre-test values of the outcomes between participants who dropped out of the study at post-test versus those who completed the post-test assessment.

Intent-to-treat analyses of condition effects were evaluated with fixed effects growth models using SAS 9.2 PROC MIXED and estimated using full-information maximum likelihood. Variability in outcomes from pre-test to post-test was modeled as a function of condition, time (coded in months since pre-test), and a Condition × Time interaction term. The Condition × Time interaction term is a test of the efficacy of the intervention. Three a priori dummy coded comparisons were examined: waitlist control (reference group) versus web-only, waitlist control (reference group) versus web+coach, and web-only (reference group) versus web+coach. Effect sizes for the Condition × Time interaction are based on condition differences in change in the outcomes over time and are equivalent to Cohen’s d (Feingold, 2009).

Child’s gender, minority status, family SES risk, and pre-test behavior problems were examined as potential moderators of intervention effects. Moderation models included main effects of the condition, time, moderator, and all two- and three-way higher order interaction terms. SES risk and behavior problems were mean centered. The critical p value for tests of the Condition × Time effect and tests of moderation was .017, the traditional .05 p value divided by three a priori comparisons, to adjust for the increased likelihood of Type I errors when making multiple comparisons.

Results

Preliminary Analyses

A summary of program use metrics is provided in Table 2. Overall, use of the site was robust (an average of 72% of users completed the FCU assessment), but the web+coach group used the program significantly more than did the web-only group. Compared with the web-only group, the web+coach group had approximately twice as many unique logins on average (4.4 vs. 2.1) and spent 47% more minutes on the site (65.0 minutes vs. 44.3 minutes). The web+coach group completed significantly more online assessments than did the web-only group (82% vs. 62%).

Table 2.

Program Use Summary

	Web-only		Web+coach
Number of unique logins [M, (SD)]	2.10	(2.30)	4.39	(4.26)
Completed the online assessment (% yes)		61.5		82.4
Time in minutes [M, (SD)]
Site navigation	9.35	(11.67)	15.78	(20.93)
Assessment	20.67	(11.84)	23.83	(14.40)
Positive behavior support section	8.79	(12.10)	11.14	(14.89)
Limit setting section	3.26	(7.52)	5.43	(10.82)
Monitoring section	1.40	(4.49)	3.47	(9.95)
Problem solving and communication section	1.05	(3.14)	4.41	(12.41)
Library	0.35	(1.12)	0.92	(2.95)
Total time	44.85	(32.89)	64.98	(63.38)

Note. Time in minutes using the site is reported for only those parents who completed the FCU online assessment (67 web-only parents and 89 web+coach parents). Group differences between web-only and web+coach for unique logins, completion of online assessments, site navigation, open communication, and total time are statistically significant at p < .05.

The outcomes approximated normal distributions except conduct problems and prosocial behaviors, which were normalized with a logarithmic transformation. Participants in the three conditions did not differ significantly at the p < .05 level on gender, area (rural, suburban, urban), minority status, SES risk score, or pre-test measures of the outcomes, indicating that randomization produced initially equivalent groups. Attrition was 7% at post-test, and results of the multivariate logistic regression model showed that failure to provide post-test data was significantly associated only with pre-test measures of hyperactivity (OR = 1.44, p = .005). Parents who reported higher pre-test values of hyperactivity for their child were more likely to drop out of the study.

Intent-to-treat analyses.

Table 3 provides means and standard deviations for the outcomes at each assessment point for each of the three conditions. Table 4 provides the correlations of the pre-test measures (correlations range from r = −.67 to r = .60, mean | r | = .30). Table 5 provides a summary of the Condition × Time effects from the growth models for the three a priori comparisons. A comparison of the waitlist control group and web+coach groups showed a significant differential change in emotional problems from pre-test to post-test. Relative to the waitlist control group, the web+coach group showed significantly greater decreases in parent report of their child’s emotional problems (p = .003, d = −.32), a small to medium effect (Cohen, 1988).

Table 3.

Descriptive Statistics for Study Outcomes

	Pre-test		3-month post-test
	M	SD	M	SD
Effortful control
Waitlist control	3.34	0.95	3.29	0.95
Web-only intervention	3.37	0.92	3.39	0.88
Web+coach intervention	3.19	0.91	3.37	0.84
Sense of parenting importance
Waitlist control	4.49	0.30	4.47	0.31
Web-only intervention	4.51	0.31	4.48	0.35
Web+coach intervention	4.55	0.24	4.59	0.27
Sense of parenting confidence
Waitlist control	4.13	0.54	4.08	0.62
Web-only intervention	4.16	0.64	4.23	0.56
Web+coach intervention	4.11	0.51	4.19	0.53
Confidence in teen behaviors
Waitlist control	2.75	0.78	2.75	0.75
Web-only intervention	2.85	0.79	2.80	0.76
Web+coach intervention	2.66	0.74	2.87	0.68
Conduct problems
Waitlist control	1.05	1.38	1.24	1.54
Web-only intervention	1.37	1.81	1.32	1.63
Web+coach intervention	1.44	1.65	1.26	1.51
Hyperactivity
Waitlist control	3.45	2.69	3.31	3.43
Web-only intervention	3.72	2.46	3.20	2.02
Web+coach intervention	4.19	2.87	3.47	2.40
Emotional problems
Waitlist control	2.34	2.27	2.56	2.42
Web-only intervention	2.56	2.28	2.36	2.23
Web+coach intervention	2.78	2.12	2.20	1.98

Note. M = mean; SD = standard deviation.

Table 4.

Pearson Correlations Among Outcomes at Pre-test

		1	2	3	4	5	6	7
1	Effortful control	1.0
2	Sense of parenting importance	.13	1.0
3	Sense of parenting confidence	.33	.38	1.0
4	Confidence in teen behaviors	.25	.12	.60	1.0
5	Conduct problems	−.47	−.03	−.31	−.35	1.0
6	Hyperactivity	−.67	−.07	−.26	−.24	.49	1.0
7	Emotional problems	−.36	−.06	−.26	−.34	.35	.29	1.0

Note. Correlations greater than |.11| are statistically significant at p < .05.

Table 5.

Condition × Time Parameter From Growth Models

Contrast and outcomes	Estimate	SE	t	p	d
Waitlist control vs. web-only
Effortful control	.018	.026	0.680	.499	.056
Sense of parenting importance	−.003	.014	−0.200	.845	−.027
Sense of parenting confidence	.035	.024	1.460	.147	.178
Confidence in teen behaviors	−.012	.029	−0.400	.690	−.046
Conduct problems	−.011	.009	−1.210	.226	−.130
Hyperactivity	−.119	.076	−1.560	.120	−.115
Emotional problems	−.133	.079	−1.690	.092	−.170
Waitlist control vs. web+coach
Effortful control	.051	.025	2.010	.046	.164
Sense of parenting importance	.016	.011	1.410	.161	.179
Sense of parenting confidence	.027	.021	1.300	.196	.152
Confidence in teen behaviors	.053	.028	1.910	.057	.209
Conduct problems	−.009	.010	−0.930	.356	−.102
Hyperactivity	−.105	.075	−1.390	.167	−.113
Emotional problems	−.237	.078	−3.050	.003	−.324
Web-only vs. web+coach
Effortful control	.033	.025	1.310	.192	.108
Sense of parenting importance	.019	.014	1.420	.156	.211
Sense of parenting confidence	−.007	.022	−0.340	.731	−.040
Confidence in teen behaviors	.063	.027	2.380	.018	.254
Conduct problems	.002	.011	0.180	.855	.020
Hyperactivity	.011	.076	0.140	.885	.010
Emotional problems	−.104	.078	−1.330	.184	−.137

Note. Bolded p value and d statistic are significant at p < .017.

No other statistically significant (at p < .017) Condition × Time effects were found, although some trend-level effects (p < .10) are worth noting. Most notable is that emotional problems showed trend-level differences that favored the web-only group relative to the waitlist control group (p = .092, d = −.17). The impact of the program with regard to decreasing youth emotional problems is thus a consistent finding for both active intervention groups relative to the waitlist-only group. The inclusion of a coach in the online training was associated with greater increases in effortful control (p = .046, d = .16) and parents’ confidence in dealing with teen behaviors (p = .057, d = .21), relative to waitlist controls. However, these findings were not consistent for the web-only group. The web+coach group further showed greater increases in parents’ confidence in dealing with teen behaviors, relative to the web-only group (p = .018, d = .25).

Moderation effects.

Parent report of child behavior problems moderated the Condition × Time effect for effortful control (estimate = 0.03, SE = 0.01, t = 2.41, p = .015) and sense of parenting confidence (estimate = 0.03, SE = 0.01, t = 2.42, p = .017) for the web-only versus waitlist control comparison. The Condition × Time effects for effortful control and sense of parenting confidence were examined separately for youth identified as at risk because they had received a borderline or abnormal pre-test behavior problem score (n = 60, 28%) and youth not at risk who had scored in the normal range for the four subscales (n = 154, 72%). The study was not powered to significantly detect differences between youth at risk and not at risk, but the effect sizes associated with the Condition × Time effect for each group elucidated the potential moderating effects. The Condition × Time effect size for effortful control for the at-risk group (estimate = 0.22, SE = 0.18, t = 1.27, p = .208, d = .26) was approximately six-fold the effect for the not-at-risk group (estimate = −0.03, SE = 0.09, t = −0.40, p = .693, d = .04). Similarly, the Condition × Time effect size for sense of parenting confidence (estimate = 0.32, SE = 0.17, t = 1.87, p = .068, d = .57) was approximately 10-fold for the not-at-risk group (estimate = 0.03, SE = 0.08, t = 0.42, p = .677, d = .06). These results are consistent with other published effects of the FCU and suggest that at-risk families benefit more from the intervention than do those not at risk.

Discussion

Building on the work of Dishion and colleagues, we set out to develop and test a feasible tool for delivering the FCU to families in the school context. In this efficacy trial, we randomly assigned middle school parents to receive either our eHealth FCU Online with coaching support, the FCU Online as a stand-alone web-based model, or school as usual. We found that the FCU Online with coaching support was the most effective approach to improving several key outcomes at post-test, including youth emotional problems and parent self-efficacy, with small to moderate effect sizes. We also found that families who reported higher levels of youth behavior problems at baseline responded to the FCU Online with improvements in youth behavior and in parenting that were significantly better than those of parents who had reported low levels of behavior problems, building on prior work suggesting that the FCU is more effective with high-risk than with low-risk youth and families (Dishion et al., 2008; Pelham, Dishion, Tein, Shaw, & Wilson, 2017; Smolkowski et al., 2017). These findings underscore the importance of examining risk at baseline, especially in FCU treatment studies, because there is now strong support for the moderating effect of risk on FCU outcomes across a range of studies. Our results show promise for the FCU Online as a potential program for wide-scale dissemination in schools.

The FCU’s brief, tailored and adaptive intervention approach for families has consistently shown effects on outcomes across development based on approximately 3–7 hours of intervention contact, depending on age and study population (Dishion, Nelson, & Kavanagh, 2003; Dishion et al., 2008; Stormshak et al., 2010). The early development of the FCU was focused on reducing treatment time by tailoring the intervention to families on the basis of the initial assessment, therefore enabling a higher likelihood of uptake in community settings (Dishion & Stormshak, 2007). The actual “dosage” needed to obtain significant effects of the FCU may be a function of the population age, demographics, and delivery context. When interventions are adapted to online programs, the dosage necessarily decreases because online programs tend to have lower rates of usage in “time” than do programs supported by a clinician and delivered in person. However, time may not be the only important factor when examining dosage. Content delivery may be more expedient online, and parents may be more likely to complete activities associated with the intervention when they are delivered on their phones or computers. In this study, participants with a coach received an average of 65 minutes of intervention, which may be comparable to the support a school could provide in person, given limited resources and low staffing in many schools. Additional research is imperative to further our understanding of how this program can be better used to examine and identify the proper “dose” needed to have an impact on behavior.

Related to dosage is the concept of delivering “kernels” of intervention treatment to individuals (Embry & Biglan, 2008). A kernel is a behavioral influence shown through research to effect specific behavior change. When interventions are delivered online, the brevity of the approach necessitates that the intervention be reduced to include kernels of behavioral change. One such kernel is the notion of “praise,” or positive parenting. An online intervention that can increase praise and positive parenting could have far-reaching effects on child behavior through this specific focus, even if the program had little focus on other behaviors. Expanding our understanding of these high-impact behaviors will help refine online delivery systems and target the intervention content.

When the FCU was first developed by Dishion and colleagues, the intent was to develop a brief intervention that could be delivered in schools. But in reality, several attempts to embed the FCU in schools and communities proved difficult and were associated with limited outcomes, and ultimately long-term sustainability of the program was compromised (Smolkowski et al., 2017; Stormshak et al., 2005; Smith et al., 2013). In schools and in community health agencies, lack of staff, high staff turnover, and poor training led to difficulties sustaining the model. Furthermore, schools have difficulty sustaining programs and delivering them with fidelity (Ringwalt et al., 2003). Our work in schools, in particular, suggested that an online version that requires less staff time and more convenient access would be easier for schools to implement and ultimately maintain in the school setting. An implementation trial is needed that would examine uptake of the FCU Online in regular school settings, regardless of limited staff support, and identify how best to disseminate this evidence-based intervention. With an online training site, administrative site, and support for users via consultation, the FCU Online could potentially be implemented with little to no “in person” training, greatly reducing the cost of the model and expanding the program’s reach.

The FCU Online program is grounded in a public health framework and the RE-AIM (reach, effectiveness, adoption, implementation, maintenance) model of health promotion, which emphasizes the advantage of reaching large populations. Even if the percentage of families and children impacted is relatively low, the intervention may be significant to the health and well-being of the general population (Glasgow, Vogt, & Boles, 1999), especially if it is efficacious and implemented effectively in communities. The least-effective public health interventions are those at the top of the health impact pyramid, which focus on behavioral change (Frieden, 2010); to be effective, those interventions must reach a large portion of the population. The FCU Online has the potential to exert a sizable public health impact if it can reach a large percentage of the population, is accessible to families through schools, and is shown to be associated with behavioral change. Furthermore, our research suggests that the FCU benefits high-risk families more than low-risk families, consistent with prior research (Pelham et al., 2017; Shelleby, Shaw, Dishion, Wilson, & Gardner, 2018). Screening families for risk and implementing the intervention as a selective or indicated approach to prevention may reduce the cost of implementation and increase outcomes across various community settings (e.g., Moore et al., 2016).

Our study is not without limitations. First, the sample was fairly homogenous with respect to race/ethnicity, although representative of the region from which it was drawn. Future research is needed to test the efficacy of the FCU Online intervention with more-diverse samples. Second, the outcomes were based on parent self-report measures that could be obtained via online surveys. As such, measurement biases could have resulted from the mono-method assessment modality, including socially desirable responding. Third, although we corrected for the number of contacts that were performed, numerous outcomes were examined that could affect the potential for inflation of Type I error. Fourth, this article reports about the acute phase results of the trial and does not include the long-term trajectories through the 12-month follow-up. Future reports about the trial will describe the maintenance of gains and long-term effects between the study conditions, as well as an examination of dose-response effects.

Our work, beginning with Dr. Dishion’s scientific vision, has built on more than 20 years of research on the FCU intervention to prevent behavioral problems in early childhood, middle school, and adolescence. The FCU began as a series of efficacy trials originally developed by Dishion and colleagues. The intervention has been applied to a variety of populations (infancy to young adults) across a range of settings (pediatric care, schools, community health). Researchers and practitioners across multiple disciplines, including nursing, social work, psychology, education, and pediatric medicine, are now trained in the FCU. Among Dr. Dishion’s many strengths were his original ideas, sensitivity to the needs of families and the greater public, and his success as a widely published researcher. Because results of the FCU have been so far reaching, future research associated with the FCU will surely be innovative, extensive, and impactful, which will be Dr. Dishion’s true legacy.