Testing the Feasibility of a Briefer School-based Preventive Intervention with Aggressive Children: A Hybrid Intervention with Face-to-Face and Internet Components

Abstract

This study describes the results from a feasibility study of an innovative indicated prevention intervention with hybrid face-to-face and web-based components for preadolescent youth. This intervention includes a considerably briefer set of face-to-face sessions from the evidence-based Coping Power program and a carefully integrated internet component with practice and teaching activities and cartoon videos for children and for parents. The Coping Power — Internet Enhanced (CP-IE) program introduces a set of cognitive-behavioral skills in 12 small group sessions for children delivered during the school day and 7 group sessions for parents. Eight elementary schools were randomly assigned to CP-IE or to Control, and six children at each school were identified each year based on 4^th grade teacher ratings of aggressive behavior. Path analyses of teacher-rated disruptive behavior outcomes for 91 fifth grade children, across two annual cohorts, indicated Control children had significantly greater increases in conduct problem behaviors across the 5^th grade year than did CP-IE children. This much briefer version of Coping Power provided beneficial preventive effects on children’s behavior in the school setting similar to the effects of the longer version of Coping Power. The website materials appeared to successfully engage children, and parents’ use of the website predicted children’s changes in conduct problems across the year.

Aggressive children can create challenges in school settings, and can ultimately cause large costs to society and their families through the expensive substance use, court, mental health, and school services they require (Foster, Jones & Conduct Problems Prevention Research Group, 2005). Youth who engage in the most persistent, severe, and violent antisocial behavior in the adolescent years are most likely to initiate aggression as children rather than as adolescents (Lahey, Waldman & McBurnett, 2001). Children’s aggressive behavior during the preadolescent years is associated with concurrent emerging conduct problems, and importantly predicts the likelihood of these children emitting more serious conduct problems involving criminal behavior and substance abuse during adolescence (Coie, Lochman, Terry, & Hyman, 1992; Lochman, & Wayland, 1994). The primary period of escalation of youths’ police arrests and substance use occurs in the period from early to late adolescence (Patterson, Dishion, & Yoerger, 2000). Thus, a key time to provide preventive intervention to aggressive children is in the preadolescent years, just before their transition to middle school (e.g., Greenberg et al., 2003).

It has been recognized for many years that schools represent optimal environments to provide interventions to prevent delinquency and substance abuse outcomes because of the amount of time that children spend within the school setting (Allen et al., 1976; Holt & Grills, 2016). School-based programs that prevent serious adolescent violence and conduct problems even for just a portion of the at-risk children receiving the intervention (and with overall small effect sizes for the full samples) can be cost-effective because of the reduction they can produce in costly correctional and mental health services as youth develop through adolescence and young adulthood (Foster, Jones, & Conduct Problems Prevention Research Group, 2006). However, existing prevention programs for at-risk children with aggressive behavior are often lengthy, which can create barriers for the implementation and sustained use of the programs in schools (e.g., Lochman, Powell, Boxmeyer, Qu, Sallee, Wells, & Windle, 2015).

Length of Intervention: Barriers and Opportunities for Briefer Hybrid Interventions

Owens and colleagues (2002) have reported that structural barriers exist that are associated with low utilization of mental health services. One central structural barrier is that intervention can be perceived by participants and practitioners to be too demanding and too lengthy (Kazdin, Holland, & Crowley, 1997). The latter is a particularly important concern for interventions dealing with children’s aggressive behavior, as it has been argued that longer intervention periods may be necessary to make sustained changes in children’s otherwise stable aggressive behavior (Kazdin, 2005). As a result of this perceived need to have an intervention sufficiently long to address the many risk factors associated with children’s aggressive behavior, the Coping Power program was developed originally to have 34 child sessions and 16 parent sessions (nearly 60 actual intervention hours). Despite successful outcomes, parent and youth participants, practitioners, and school settings have expressed concerns about the length of the intervention, and these concerns can decrease the practical public health significance of evidence-based interventions.

There are encouraging indications that briefer interventions can be effective. A meta-analysis has reported that stronger effect sizes for outcomes from behavioral parent training programs have been found for briefer 5 session programs in comparison to programs that last 16 or more sessions (Maughan, Christiansen, Jensen, Olympia, & Clark., 2005). Similarly, a briefer version of Coping Power (24 child sessions, 10 parent sessions) has produced significant reductions in teacher ratings of children’s externalizing behaviors at longer-term follow-ups (Lochman et al., 2014). Such results are encouraging, and suggest that innovative methods, such as use of multimedia interventions, could be used to create an even briefer, more efficient version of the program that could be embraced more readily by parents, counselors, and other school personnel, and thus more successfully engaging a wider range of at-risk children and their parents.

An innovative way of offering briefer interventions in a preventive intervention is to include internet-based content (Lightfoot, Comulada, & Stover, 2006). Children are easily engaged by multi-media and computer-based interventions (Paperny, 1997) and internet delivery of program content allows the intervention to be more accessible and efficient (Bishop, Bryant, Giles, Hansen, & Dusenbury, 2006; Comer et al., in press; Taylor, Webster-Stratton, Feil, Broadbent, Widdop, & Severson, 2008). Internet applications have produced positive effects on children’s internalizing symptoms and management of chronic health conditions, but have been rarely examined rigorously as an intervention for aggressive and conduct problem behavior in children (Boydell et al., 2014; Stinson et al., 2009). In this “flipped learning” approach to intervention delivery, transmission of new information takes place primarily during out-of-meeting time (with traditional face-to-face provision of information being replaced with out-of-class activities such as video-based content), with face-to-face time used primarily for active, collaborative learning, and application of the information (Abeysekera & Dawson, 2015). Although cognitive behavioral programs, including the traditional form of Coping Power, do include out-of-session homework activities, this approach expands on that.

Although internet delivery of interventions may be sufficient in providing simple information that can help to alleviate internalizing problems and improve coping with health conditions (Boydell et al., 2014; Stinson et al., 2009), the personal assistance of a clinician in a hybrid intervention delivery model may be important in modifying more significant, stable behavioral habits such as aggression (Lightfoot et al., 2006). In one of the few reports on this form of intervention delivery with children with behavior problems, Taylor, Webster-Stratton, and colleagues (2008) used a hybrid model of the Incredible Years program which combined computer and web-based delivery of information and video modeling vignettes along with direct professional intervention through home visits and phone calls. This hybrid model of the Incredible Years program with Head Start families with preschool children led to a high level of parental satisfaction with the program and to an increased completion rate of the program. However, no evidence-based hybrid models are available for indicated prevention or for preadolescent aggressive children during the critical developmental period when prevention may reduce the expected escalation in serious antisocial behaviors in early adolescence. The hybrid Coping Power program examined in this paper, which was based on the evidence-based full-length Coping Power program, is an effort to respond to this need.

Coping Power Program

The Coping Power Program (Lochman & Wells, 2002a) was developed as an indicated prevention program (National Research Council and Institute of Medicine, 2009) for preadolescents who are at-risk because their problem behaviors can be precursors for later severe conduct problems, substance abuse, and juvenile delinquency. Coping Power was derived from earlier research on the Anger Coping Program (Larson & Lochman, 2011; Lochman, Nelson, & Sims, 1981), which in turn derived from early efforts to introduce prevention into school-based settings (Allen, Chinsky, Larcen, Lochman, & Selinger, 1976). The Anger Coping Program had been refined, tested, and disseminated with a multidisciplinary set of clinical child psychologists and school psychologists (Larson & Lochman, 2011; Lochman, Burch, Curry, & Lampron, 1984; Lochman, Lampron, Gemmer, Harris, & Wyckoff, 1989).

The Coping Power Program is a comprehensive, cognitive-behavioral, multi-component intervention program that is based on a contextual social-cognitive model of children’s aggressive behavior which places children at risk for later serious conduct problems (criminal behavior; substance abuse; Lochman & Wells, 2002a), and has been designed for delivery in school settings (Powell, Boxmeyer, Baden, Stromeyer, Minney, Mushtaq, & Lochman, 2011). Evidence-based cognitive behavioral indicated prevention programs have been found to be effective for preschool and early elementary school children displaying disruptive behaviors (Matthys & Lochman, 2010), including the Incredible Years (Webster-Stratton, 2005) and Parent-Child Interaction Therapy (Brinkmeyer & Eyberg, 2003) programs. However, although there are behavioral parent training programs, such as McMahon and Forehand’s (2003) family-based treatment for oppositional behavior and Kazdin’s (2005) Parent Management Training, that can be used for older preadolescent aggressive children, there are few evidence-based programs that provide integrated preventive intervention for both parents and children. Kazdin et al. (1992) have developed an effective program for preadolescents with disruptive behavior disorders which includes both problem-solving skills training for the children and parent management training for the parents. However, this program was developed to be delivered individually in outpatient clinics. The Coping Power program was developed as an indicated prevention program to be delivered in school settings, thus permitting it to address critical issues in school-based mental health (Holt & Grills, 2016), and addresses a wider array of children’s social-cognitive difficulties, including an emphasis on emotional regulation training.

The contextual social-cognitive model is based on research on the social-information processing distortions and deficiencies of aggressive children (e.g., Dodge, Lochman, Harnish, Bates, & Pettit, 1997; Lochman & Dodge, 1994; Dodge, Laird, Lochman, Zelli, & Conduct Problems Prevention Research Group, 2002), and on parenting behaviors that have been found to be associated with children’s aggressive behavior (e.g., Stormshak, Bierman, McMahon, Lengua, & Conduct Problems Prevention Research Group, 2000). Preadolescent children with aggressive behavior problems have been found to have distortions in their appraisals of others, including incorrect encoding of others’ emotions and forming hostile attributional biases when the children are faced with ambiguous social situations (e.g., Lochman & Dodge, 1998; Yaros, Lochman, Rosenbaum, & Jiminez-Camargo, 2014), deficiencies in their problem-solving solving skills, including reliance on dominance- and revenge-oriented social goals, inadequate generation of competent solutions to social problems, and expectations that aggressive solutions will lead to positive outcomes (e.g., Dunn, Lochman, & Colder, 1997; McDonald & Lochman, 2012; Pardini, Lochman, & Frick, 2003).

Children develop these social-cognitive processes in part through their interactions within their family and peer environments. Children’s aggressive behavior has been found to be partially maintained by parenting practices consisting of low levels of parental involvement and monitoring and high levels of harsh and inconsistent discipline (Matthys & Lochman, 2010). The Coping Power’s child and parent components are designed to directly target these social-cognitive and familial risk factors (Lochman, Wells, & Lenhart, 2008; Wells, Lochman, & Lenhart, 2008). For example, the child components which address emotion awareness, anger management, and perspective-taking are directed at children’s appraisal abilities, and the components which address problem solving skills and dealing with deviant peer pressure, are focused on problem-solving deficits.

The Coping Power parent program has sessions that focus on developing more positive parent-child interactions, with greater use of social reinforcement, sessions that target monitoring and supervision skills, and sessions that address alternative discipline strategies that are less harsh and inconsistent discipline. Coping Power also explicitly focuses on ways in which parents can support children’s academic achievement and behavior change at school, including by reinforcing collaborative parent-teacher interactions, through specific activities in sessions and by including school counselors and teachers in some parent group sessions. Parents are given strategies for supporting their child’s academic work at home. Parents are taught to use a “homework completion system” (e.g., a school-to-home assignment notebook) that allows for increased parent-teacher communication about homework assignments and parental monitoring of homework completion. Parents are also encouraged to proactively schedule a parent-teacher conference, rather than waiting until problems arise. Parents are provided with potential questions and topics for the parent-teacher conference, and role-play this interaction in the therapeutic setting. It has been proposed that parent engagement in school-based mental interventions has important effects on outcomes (Castine, Flanagan, & Howard, 2016; Greenberg, Weissberg, O’Brien, Zins, Fredericks, Resnick, & Elias, 2003), and improvements in parent-teacher relationships in prior research have been found to lead to positive changes in children’s behaviors (Henderson & Mapp, 2002; Sheridan et al., 2012).

Evidence-base for the full Coping Power Program

Research trials have shown that randomization to the full Coping Power intervention results in improvements, relative to control groups, through one-year follow-ups in substance use, delinquent behavior, and teacher-rated behavioral problems (Lochman & Wells, 2003, 2004), indicating the successful prevention effects of this indicated prevention program. Research has also shown that these intervention effects are mediated by changes in children’s social cognitions and parents’ use of consistent discipline, and that real-world school counselors could be trained to provide the intervention that affected children’s externalizing problems according to teacher, parent and self-ratings at the end of intervention (Lochman et al., 2009; Lochman & Wells, 2002a). Coping Power is typically delivered in small groups, but individual delivery can produce stronger effects in the school setting for some children (Lochman et al., 2015). The intervention has helped to prevent the declines seen in control children in language arts grades in the two years following intervention (Lochman et al., 2012). Long-term follow-up effects, three to four years after the intervention, have been found on youths’ externalizing behavior in school settings in two separate studies (Lochman et al., 2013, 2014) and on substance use in a clinic setting (Zonnevylle-Bender et al., 2007). Coping Power has been found to be effective in reducing children’s externalizing behavior problems and enhancing academic outcomes in the school setting, although the effect sizes through longer-term follow-up periods are in the small to medium range. Based on reviews of the evidence-base in these efficacy studies, the Coping Power program has been rated as having positive effects on externalizing behavior in the What Works Clearinghouse (Institute of Education Sciences), as a “promising program” by Blueprints for Healthy Youth Development and by CrimeSolutions.gov (National Institute of Justice), and as a “well-supported program” by the California Evidence-Based Clearinghouse for Child Welfare.

A central concern in delivery of a school-based program: Parent attendance

A particular challenge in the delivery of school-based preventive interventions, affecting the magnitude of their effects, is the ability to involve parents in adequate levels of attendance in parent sessions (Garvey, Julion, Fogg, Kratovil, & Gross, 2006; Lochman, Boxmeyer, Powell, Roth, & Windle, 2006). If parents do not attend sessions then they are not receiving the full dose of evidence-based strategies to address the children’s problem behavior. When they do attend sessions, parents become aware of what their children are learning in the children’s groups (and thus have an opportunity to reinforce the children’s new social behaviors and emotional regulation), they learn how to monitor and reinforce the “positive opposites” to their children’s problem behaviors, they acquire methods for coping with their own stress, and they gain social support from other parents in their efforts to improve children’s behavior. Improvements in parents’ behavioral management skills and their social support network have both been found to mediate and predict children’s behavioral improvement during intervention (Lochman & Wells, 2002a; Conduct Problems Prevention Research Group, 2002).

Parent (and child) attendance in intervention is conceptualized as one aspect of participant engagement in interventions. Terms such as participant engagement, attendance, and involvement have often been used interchangeably, but intervention engagement in recent years (Becker et al., 2015; Ellis, Lindsey, Barker, Boxmeyer, & Lochman, 2013; Ingoldsby, 2010; Minney, Lochman, & Guadagno, 2015 Staudt, 2007) has been identified as having at least two key elements: behavioral engagement (e.g., attendance at sessions; use of websites) and motivational or attitudinal engagement (e.g., emotional involvement in the intervention process; commitment to treatment). Parent attendance in parent groups has been found to be related to the strength of children’s outcomes in dosage analyses (Lochman et al., 2006) in some studies, but not related to outcomes in other studies (Nix, Bierman, McMahon, & Conduct Problems Prevention Research Group, 2009). A set of recent studies have begun to examine predictors of parent attendance in parent behavioral training groups, and, for example, have found poorer attendance by parents who had experienced more stressful life events and who had more limited adult social supports (Minney, Lochman, & Guadagno, 2014). Thus some parents, including those most in need, may be at particular risk for low attendance in parent groups.

In prior Coping Power outcome studies, the research focus has been on behavioral engagement rather than motivational engagement, and children have been found to receive high levels of doses of the child intervention, with session attendance rates typically at 90% or higher. However, parent attendance at parent sessions has varied widely. As a prevention program, parents are approached by intervention staff to be invited into the Coping Power program in schools, rather than parents initiating the contact with intervention staff as is the case in outpatient treatment settings. As a result, some parents may be less motivated to regularly attend parenting sessions since they had not sought them out. Parent overall attendance at sessions often has been in the 30–40% range in prior Coping Power studies, limiting the strength of outcome effects (Lochman et al., 2006).

One possible avenue for stimulating parental attendance in intervention can be through their children’s positive engagement in early phases of the intervention (Ellis, Lindsey, Barker, Boxmeyer, & Lochman, 2013). Ellis et al. (2013) found that the more that children were positively engaged in early sessions (high attendance rates, high completion of behavioral goal forms, high compliance with group rules) then parents increased their attendance at parent meetings in mid-intervention sessions (controlling for the rates of attendance in early parent sessions). Other adaptations that could enhance parent attendance include making the intervention as brief as possible, and by including interesting internet-based material to the intervention, all aims of the Coping Power — Internet Enhanced Program.

Hybrid Coping Power — Internet Enhanced (CP-IE) Program

The innovative CP-IE program developed to include both face-to-face and internet components can be delivered more efficiently, with substantially fewer face-to-face sessions with clinicians than the full Coping Power program, and thus may lead to easier implementation of the intervention in schools. There are two major ways in which the delivery of the CP-IE program differs from the traditional Coping Power program. First, the CP-IE program includes internet-based content to present the psychoeducational aspects of the program (e.g., What are anger management skills?; What is meant by perspective-taking and How does poor perspective-taking relate to angry aggressive behavior?; How can children solve social problems in competent ways?) in engaging ways that can stimulate children’s retention of the concepts and motivate the children and parents to engage in the application of the skills in their real-world social situations. The internet-delivered programming includes some elements that are regular practice and homework features of the Coping Power program (e.g., completion of weekly behavioral goal sheets and skill practice forms), along with an award-winning animated cartoon series, the “Adventures of Captain Judgment,” that had been developed specifically to demonstrate skills being presented to children at different points in the Coping Power program (Lochman et al., 2009; Lochman, Nelson, & Boxmeyer, 2010)¹. Second, because much of the basic content about cognitive-behavioral skills is presented to children and parents on their website the CP-IE program has 60% fewer face-to-face sessions than traditional Coping Power, thus considerably streamlining intervention delivery. CP-IE, as developed, includes only 12 child sessions delivered every other week and 7 parent group sessions, along with the internet accessible media components which participants access between sessions. In contrast, the traditional Coping Power program has 34 child sessions and 16 parent sessions. The way in which session topics from the full Coping Power program is condensed into the briefer set of CP-IE face-to-face sessions is illustrated in Table 1. Because of the internet component, all of the concepts and skills in the full Coping Power program are still included in CP-IE, but the delivery format is dramatically different. The CP-IE intervention is delivered over a six month period of time that is anticipated to be long enough to provide repeated reinforcement for emerging prosocial behaviors and skills to reduce children’s aggressive behaviors.

Table 1.

Characteristics of Intervention and Control Schools

School Characteristics	Coping Power — Internet Enhanced (CP-IE) Schools (N=4)		Control Schools (N=4)
	Mean	Range	Mean	Range
Number of 4th Grade Students	62	50–84	57	37–105
Percentage of Students with Free/Reduced Lunch	78	66–88	79	28–98
Percentage of African American Studentsa	82	64–95	80	31–100

^athe remaining students were primarily Caucasian

Coping Power as a Tier 2 intervention in schools

Cognitive-behavioral interventions have been noted to have considerable promise as evidence-based interventions for children’s emotional and behavioral disorders within the school setting (Mayer, Van Acker, Lochman, & Gresham, 2009; Powell et al., 2011). From the perspective of tiered support systems in school-based programs, the briefer form of Coping Power (CP-IE: Coping Power — Internet Enhanced) examined in this paper can be conceptualized as a comprehensive Tier 2 intervention for elementary school children with specific deficits related to their problem behaviors (McDaniel, Bruhn, & Mitchell, 2015). Tier 2 interventions often involve the provision of explicit strategies to address the problem area evident in small groups of children (Saeki, Jimerson, Earhart, Hart, Renshaw, Singh, & Stewart, 2011), and CP-IE has the potential of providing a brief but comprehensive and intensive intervention to address the specific behavioral deficit areas associated with children’s aggression (Stormont & Reinke, 2013).

The Current Study

The current study tested the initial feasibility and effects of CP-IE, a more efficient hybrid version of the Coping Power program, with 97 aggressive 5^th grade children through random assignment of eight elementary schools to Coping Power — Internet Enhanced (CP-IE) versus a Control condition. Children were screened for participation in this study in 4^th grade, and they participated in the CP-IE and Control conditions throughout 5^th grade. It was expected that the outcomes of this study can have implications for CP-IE as a potential Tier 2 intervention in schools.

A first aim of the study was to test condition differences in children’s behavior across the 5^th grade year. Two linked hypotheses are that:

• 1a)Control children will have higher rates of teacher-rated aggressive than CP-IE children (Hypothesis 1a) and
• 1b)Control children will have higher rates of conduct problem than CP-IE children (Hypothesis 1b).

These two paired hypotheses test whether a much shortened version of Coping Power can still have effects on children’s problem behaviors follows from the earlier study that found that a less dramatic reduction in Coping Power length had produced relative preventive effects on externalizing behavior problems (Lochman et al., 2014). This study will thus determine whether outcome effects are apparent in the at-risk aggressive behaviors and in the conduct problems which are the focus of this indicated prevention program, with this markedly more efficient version of the program.

The study also explored a second aim, examining whether the changes in outcome behaviors that were significantly affected by CP-IE are related to child and parent attendance at CP-IE face-to-face sessions and to their use of the internet websites. Although prior research has been equivocal about the predictive role of parent attendance in sessions, parent and child use of the internet component and their attendance at face-to-face sessions, they may be important process predictors of children’s behavioral outcomes in the CP-IE program.

Method

Participants

Eight elementary schools from one school system in Alabama were recruited to participate in this study. No schools declined to participate. All of the schools in this public school system, including the eight involved in this study, were using Schoolwide Positive Behavioral Intervention and Supports (SWPBIS; Bradshaw, 2013; Dunlap, Kincaid, Horner, Knoster, & Bradshaw, 2014) at all three tiers, within a three-tier system. The participating schools were located in urban and suburban areas, and ranged from predominantly Caucasian and middle class to impoverished and 100% racial/ethnic minority. The eight schools were paired based on size, racial/ethnic distribution, and eligibility for free and reduced meals. One school from each pair was randomly assigned to each condition, yielding four schools per condition. The average rates of fourth grade students, of students with free/reduced lunch, and of African American students within the intervention and control conditions are contained in Table 1, and the rates indicate that the two sets of schools were relatively comparable in these student characteristics. Schools were randomized rather than children to avoid contamination between conditions. Counselors and teachers of children in the Control condition may have begun to use program elements (e.g., goal sheets) with Control children, thus potentially confounding condition effects.

To identify children likely to benefit from the Coping Power Program, fourth grade teachers completed 6-item screening ratings of overt aggressive behaviors in the late spring of the academic year on all children in their classrooms for whom parental consent existed, following IRB guidelines. Ratings were compiled across schools, and a cut-off score corresponding to the 30^th percentile was determined. This cut-off score has been used to examine social and contextual risk factors which significantly differentiate aggressive from nonaggressive children (e.g., Fite, Colder, Lochman, & Wells, 2006; Pardini, Lochman, & Wells, 2004). Children whose scores fell at or above the cut-off score were considered eligible for participation. Schools varied in the number of children who fell within the eligible range, but six children were recruited from each school. For each school, a randomized list of eligible children was created, and families were contacted according to their placement on the list until six children were enrolled (in one school seven rather than six children were enrolled due to two parents responding to the last recruitment call at the same time). In most cases, the cohort was filled before all eligible families were contacted. This procedure was completed for each of the two annual cohorts, resulting in a total sample size of 97 children (49 in CP-IE and 48 in Control condition). However, because of missing Time 1 or Time 2 data for six children, the current analyses were conducted with a sample of 91 children (47 CP-IE, 44 Control). At the time of recruitment, children ranged in age from 9 to 12 years (Mean = 10.2). The descriptive characteristics of the children, their caretakers, and their family incomes can be found in Table 2.

Table 2.

Participant, Caretaker and Family Characteristics

Characteristics	Overall	CP-IE	Control
Caretaker
Mean Age (range)	37.16 (25–72)	37.36 (25–64)	36.95 (25–72)
Relation to Child
- Biological Parent	77	38	39
- Grandparent	9	6	3
- Other Relative	5	3	2
Child
Mean Age (range)	10.20 (9–12)	10.13 (9–11)	10.27 (10–12)
Gender (% boys)	63%	60%	67%
% Repeated Grade	15	7	8
Race
- African American	85	42	43
- Other	6	5	1
Family Income
< 15,000	41%	34%	48%
15,000 – 24,999	23%	26%	20%
25,000 – 49,999	23%	23%	23%
> 50,000	13%	17%	9%

Procedure

At school, baseline pre-intervention data (Time 1) were gathered from teachers six weeks after the start of fifth grade and post-intervention assessments were collected in the late spring of fifth grade (Time 2). Teachers received $10 for each child assessed. All study procedures were approved by the University of Alabama Institutional Review Board.

Coping Power – Internet Enhanced (CP-IE)

CP-IE face-to-face sessions

This study tested a novel hybrid adaptation of Coping Power. CP-IE included 12 in-person manualized child group sessions and 7 in-person manualized parent group sessions, with the remainder of the content from the full-length Coping Power child and parent programs (Lochman, Wells, & Lenhart, 2008; Wells, Lochman, Lenhart, 2008) being delivered via an intervention website (thus applying a flipped learning approach). Table 3 indicates how topics from the full Coping Program were presented within the briefer CP-IE face-to-face sessions. The intervention utilizes cognitive behavioral strategies to teach children specific social problem-solving, goal-setting and emotional regulation skills, and to teach parents parenting skills. The CP-IE in-person child sessions took place during the school day. Children attended group sessions at times that were negotiated with classroom teachers. Four of the groups occurred during the lunch period, two occurred early in the morning (starting by 8:10am), and two in the afternoon (after 1:15pm). The in-person parent sessions were held in the evenings at a meeting space at The University of Alabama. Parent groups were held on Thursday evenings from 6:00–7:30pm. To maximize parent engagement, dinner, a supervised child waiting room, and an attendance stipend ($5 per session) were provided for each parent meeting.

Table 3.

Full-Length Coping Power (CP) versus Internet-Enhanced (CP-IE) Coping Power

Child Component			Parent Component
CP Session # Full CP	CP-IE Session # CP-IE*	Description of topics and skills	CP Session # Full CP	CP-IE Session # CP-IE*	Description of Topics and Skills
1	1	Group Structure & Goal Setting	1	1	Introductions & Academic Support at Home
2–3	1	Setting Personal Behavioral Goals	2	1	Academic Support at Home
4	2	Organizational & Study skills	3–4	2	Stress Management for Parents
5–6	3	Awareness of Feelings & Arousal	5	3	Improving the Parent-Child Relationship and
7	3	Anger and Self-Control	6	3	IgnoringMinor Disruptive
8–11	4–5	Using Self-Statements for Anger Coping	7	4	Giving Effective Instructions to Children
12–14	6	Perspective-Taking	8	4	Establishing Household Rules and Expectations
15–19	7	Social Problem-solving	9–10	5	Discipline and Punishment
20–22	8	Group creates videotape	11	7	Making the Most of the Summer
23	8	Review	12	1	Academic Support at Home
24	2	Organizational & Study Skills	13	6	Building Family Cohesion
25	9	Teacher Expectations & Conflict	14	6	Family Problem Solving
26	9	Friendship/Group Entry	15	6	Family Communication
27	9	Group Entry/Peer Negotiation	16	7	Long-Term Planning and Termination
28	9	Sibling Conflict
29	10	Peer Pressure
30	10	Peer Pressure Refusal Skills
31	10	Neighborhood Problems
32	11	Resisting Peer Pressure
33	11	Reinforcing Strengths & Peer Relationships
34	12	Review & Termination Session

Intervention manuals were developed for delivery of the CP-IE child and parent groups. A Coping Power staff member and the school counselor co-led each of the 12 planned in-person child group sessions. Each group was led by a Master’s level clinician, and the co-leaders had Bachelors or Masters degrees. A total of eight interventionists led or co-led the 8 groups across the two cohorts. Half of the interventionists were males and half were females; half of the interventionists were Caucasian and half were of other races. Two Coping Power staff members co-lead the in-person parent group sessions. In-person child group sessions lasted approximately 50 minutes each (range 40–55 minutes). In-person parent group sessions lasted approximately 90 minutes each (range 60–95 minutes). Face-to-face group time focused on discussing recent web-based content, active skills-training and practice opportunities, and tailored group discussion centered on personal application of the intervention content.

CP-IE internet component

A new website was developed for the CP-IE internet-delivered content, and it had separate webpages (with separate log-ins) for children, for parents, for teachers, and for the counselors. Materials on the website included: 1) a place for children and teachers to report on children’s daily attainment of personal behavior goals; 2) brief animated video clips on the children’s webpages from the Adventures in Captain Judgment series, in which a superhero (Captain Judgment) and group of children (Team Judgment) encounter a series of problems and learn to use specific social-emotional skills to solve the problems (paralleling the skills the children learn in Coping Power in-person sessions); 3) corollary video clips for parents on the parents’ webpages highlighting key parenting skills and allowing parents to preview the videos for the child Coping Power lessons; 4) printable handouts on both the children’s and parents’ webpages, such as worksheets for identifying long-term and short-term goals, for tracking incidents that trigger angry feelings, for brainstorming solutions to social problems, for conveying parenting tips, such as how to give your child labeled praise and how to support your child’s academic success; and 5) quizzes on both the children’s and parents’ webpages for assessing children’s and parents’ comprehension of didactic and video content.

The Adventures of Captain Judgment includes 3 to 6 minute episodes that were developed to accompany each of the original 34 child sessions and 16 parent sessions of Coping Power. Each episode was designed to teach skills in a format that is engaging and interesting to the children and parents. A theme was developed for the child video series in which a superhero (“Captain Judgment”) brings a team of five children together (“Team Judgment”) and teaches the children to use specific social-emotional skills to address problems they face. The children then go on to help other characters utilize the skills they have learned, thus becoming the experts themselves as the video series progresses. In the parent video series, an animated character named Dr. Insight summarizes the key teaching points from each of the Coping Power parent sessions, and introduces clips from recent Coping Power child videos so parents can not only understand the skills their children are learning, but also support the learning process in which their children are engaged.

Web-based content (videos, handouts, quizzes) was made accessible on the website by the Coping Power leader, as it became relevant to a particular intervention unit, and was removed from the website once the intervention unit had been completed. New web-based content remained available for a minimum of 2–3 weeks, to ensure that children and parents had adequate time to access it.

Children earned points for utilizing the CP-IE the website and were able to apply the points earned to purchase prizes at in-person sessions. For example, children could earn 5 points for each quiz they submitted through the website, 2 points for each video they watched and responded to questions about, and 1 point for each day they indicated whether or not they met their behavior goal and their teacher agreed. Children also received bonus points if their parent utilized the CP-IE website. Parents received their own incentives for utilizing the CP-IE website (e.g., school supplies given out at parent meetings).

Several aspects were specifically included in the CP-IE website to maximize user interest. Children were able to personalize the website by selecting from a range of avatars/characters and by creating their own usernames. A large, visually-appealing graphic appeared on the home page that allowed participants to track their point attainment and completion of CP-IE program activities. Pictures of the prizes that children could earn (and their point values) were readily available and children were able to “purchase” prizes through the website, as they earned enough points. Prizes purchased via the website were distributed at the next in-person session. The CP-IE website also included a messaging system, in which children, parents, and teachers could message their Coping Power leader with questions or updates. The children particularly made use of the messaging feature. The website did not allow children or parents to communicate with each other directly. However, the Coping Power leader (website administrator) could send messages and reminders to the entire group.

In general, implementing preventive interventions in schools can be perceived by school personnel as being intrusive as potentially interfering with the academic work that teachers are doing in the classroom (Allen et al., 1976). Proponents of school-based cognitive-behavioral programs, and of the controlled research to test these programs, have emphasized the need to attend to implementation barriers in schools and to approach delivery of cognitive-behavioral services in constructive and planful ways that involve teachers and other school personnel in the decision-making process (Van Acker & Mayer, 2009). The general approach in implementing Coping Power, based on prior positive working relationships with area school systems, is to first seek approval for the project from the school system’s superintendent’s office, followed by meetings with building principals to seek their permission, and meetings (if permitted) with school counselors and classroom teachers to seek their approval. In these meetings handouts are provided and staff discuss what the benefits to the children and the schools will be, and what the teachers and school staff would be expected to do. One task that teachers are expected to do as part of the program, if they participate, is to monitor and report on children’s behavioral goal completion on a daily basis. One feature of CP-IE that teachers reported that they particularly liked, in an informal way during the meetings with teachers, was the web-based reporting of children’s goal attainment. They gave positive feedback about being able to report on children’s goals at their own discretion on the website rather than relying on paper-and-pencil goal sheets carried by the children.

Also, it was known in advance that schools would have to give the children access to computer time each day in order for the children to report on goals (and complete activities if they didn’t have a home computer). The project’s in-home interviews indicated which children did and did not have access to a computer outside of school before meetings were scheduled with teachers and counselors. In these meetings, staff problem-solved with the teachers about the best time of the day for children to use the classroom computers to report on goals and complete Coping Power activities. By having a set time of day that children could use the classroom computers, the program didn’t disrupt the normal routine.

Intervention training and integrity

The primary co-leader for each of the CP-IE groups was a Master’s level interventionist who had been trained to criterion to use the full Coping Power program in several prior school-based randomized trials of Coping Power, receiving multi-day workshop training, and performance feedback about implementation of the program, using the intensive training model that has been found to be effective for Coping Power (Lochman et al., 2009). The approach by Lochman, and colleagues (2009) is consistent with the conclusion that a combination of highly structured intervention materials (through manuals) and of monitoring and performance feedback (through review of session tapes) has been linked to better intervention effects with children (Weisz, Donenberg, Han, & Weiss, 1995; Stirman, Crits-Cristoph, & DeRubeis, 2004). During the implementation of CP-IE, the lead interventionist received weekly supervision meetings from a doctoral-level clinical psychologist with more than 9 years of experience implementing Coping Power. These supervision meetings (a) reviewed completed session’s objectives and activities, focusing on children’s and parents’ absorption of content, process issues in the groups and problem-solving about handling intervention delivery problems, and (b) previewing and planning for the upcoming session. This lead interventionist in turn supervised the seven co-leaders weekly, providing feedback about their intervention delivery and preparing them for the upcoming session. The developer of the Coping Power program, the clinical supervisor and the lead interventionist also had weekly meetings to discuss the delivery of the internet component of the CP-IE program.

The integrity of intervention implementation can affect intervention outcomes (Lochman, Boxmeyer et al., 2009; Schoenwald & Hoagwood, 2001). Intervention integrity refers to the degree to which an intervention is implemented as planned, and is concerned with the accuracy with which treatments are implemented based on an established intervention protocol (Gresham, 1989; Gresham & Lochman, 2009). Several steps were taken in the current study to have adequate fidelity of implementation (Gresham & Lochman, 2009). First, the objectives and specific activities for each child and parent session were detailed in intervention manuals. Second, interventionists completed a measure of fidelity after completing each intervention session, rating whether they had covered each session objective “completely,” “partially,” or “not at all.” CP-IE leaders indicated that they “completely” (91% child group; 87% parent group) or “partially” (7% child group; 13% parent group) completed group intervention objectives, indicating a high rate of self-reported intervention fidelity. All child and parent intervention sessions were video-recorded so the clinical supervisor could give performance feedback. However, because of limited resources, the videos were not rated by an independent observer for completion of intervention objectives.

Measures

The types of measures employed in this study include: (a) screening measures to identify children with elevated levels of aggressive behavior; (b) measures of intervention effects on youth aggressive behavior and conduct problems (c) measures of attendance at in-person intervention sessions; (d) measures of website use; and (e) measures of participant satisfaction with the CP-IE intervention.

Child aggression screening measures

In order to identify at-risk aggressive children, fourth grade teachers were asked to complete the Teacher Report of Reactive and Proactive Aggression (Dodge, Lochman, Harnish, Bates, & Pettit, 1997) for all of the children in their class. Teachers rated the frequency of six aggressive behaviors on a 5-point Likert scale (total scores range from 6 to 30). The scale includes three items reflecting instrumental or proactive aggression (e.g., “this child threatens or bullies others in order to get his or her own way”) and three items reflecting emotionally reactive aggression (e.g., “when this child has been teased or threatened, he or she gets angry easily and strikes back”). The total Aggression Screen score displayed adequate internal consistency, with an alpha value of .97 for all of the children in the classrooms who were screened, and .85 for the 97 children screened into this study. This form of a teacher screening measure has been shown to have predictive validity in prior research (Hill, Lochman, Coie, Greensberg, & Conduct Problems Prevention Research Group, 2004; Lochman & Conduct Problems Prevention Research Group, 1995), and the cut-offs used in this study have produced expected differences between aggressive children who have met the cut-off and nonaggressive children (Deming & Lochman, 2008; Fite, Colder, Lochman, & Wells, 2006; Pardini, Lochman, & Wells, 2004; Williams, Lochman, Phillips, & Barry, 2003), indicating adequate construct validity.

Outcome measure

The Behavior Assessment System for Children (BASC-1; Reynolds & Kamphaus, 1992) is a behavior problem checklist that was completed for this study by children’s primary teachers. The BASC-1 was used rather than more recent editions of the BASC to make the results most comparable to teacher-rated behavioral outcomes from prior Coping Power studies (Lochman et al., 2006, 2009, 2014) which had used the BASC-1. The BASC-1 has demonstrated strong reliability (Chronbach’s alpha of .80–.89) and construct validity (Doyle, Ostrander, Skare, Crosby, & August, 1997; Reynolds & Kamphaus, 1992). The construct, discriminant and convergent validity of the teacher-report for the BASC-1 has been demonstrated with factor analyses (Kamphaus, Huberty, DiStefano, & Petoskey, 1997), with significant discriminant analyses for diagnosed disorders such as ADHD (Vaughn, Riccio, Hynd, & Hall, 1997) and with high correlations with other teacher-rated scales assessing externalizing behavior problems (Jarratt, Riccio, & Siekierski, 2005; Vaughn, Riccio, Hynd, & Hall, 1997). Correlations between the BASC-1 and Achenbach’s Teacher Report Form have been .90 for the Aggression subscale and .75 for the Conduct Problems subscale (Vaughn et al., 1997).

For this study, the two subscale scores from the BASC were used: Aggression (14 items) and Conduct Problems (10 items). The Time 2 (T2) and Time 1 (T1) Aggression and Time 2 Conduct Problem scores have normal distributions with acceptable levels of skewness and kurtosis (respectively, for the two scores: T2 Aggression: 0.51, −0.21, T1 Aggression: 0.60, 0.07, T2 Conduct Problems: 0.78, 0.03). Time 1 Conduct Problems scores have somewhat higher skewness and kurtosis levels (1.41, 2.48, respectively), but the normality of these distributions overall is adequate (Byrne, 2010; George & Mallery, 2010). The internal consistency, assessed with Chronbach alphas, was adequate in this sample for the Aggression (T1: .92; T2: .94) and Conduct Problems subscales (T1: .74; T2: .73). The Aggression and Conduct Problem subscales were relatively highly intercorrelated at T1, r (91)=.72, p<.001, and at T2, r (91)=.75, p<.001. There also was relatively high stability across the two timepoints for Aggression, r (91)=.65, p<.001, and Conduct Problems, r (91)=.55, p<.001.

Measure of attendance at in-person intervention sessions

Coping Power leaders documented child and parent attendance at each in-person session. Leaders maintained a written documentation folder for each child and parent group, in which they documented each participant’s attendance and engagement immediately following each session. This included documenting the total length of each session. If a participant was not present for a meaningful portion of the session (e.g., was only present for 5 minutes of a 45-minute session), the leader made an effort to make-up the missed content with that participant prior to the next session, or noted when this did not occur. This written data was converted to electronic data which quantified each participant’s level of exposure to the content from each session held.

Measure of website use

The CP-IE website automatically captured a range of usage data, including number of child and parent log-ins, time spent on the website during each log-in, and percentage of web-based activities completed. The administrator for the website was the lead interventionist, and he could log-in and receive information about which children and parents had logged-in to the child or parent websites. Excel files with this data could be downloaded directly from the website, and then used in analyses. For the present study we used data about the number of times that children and parents logged into their respective websites.

Measure of participant satisfaction with the CP-IE intervention

At the end of the CP-IE intervention, children and parents provided quantitative feedback on their satisfaction with CP-IE, using a brief rating form created for this study and specifically addressing the elements of the CP-IE program. Children and parents responded individually to four questions, about their experience with the CP-IE intervention: (a) “How did you like Coping Power overall?;” (b) “How helpful were the in-person Coping Power parent meetings?” for the parents and “How helpful were the in-person Coping Power child meetings?” for the children; (c) “How helpful was the material on the Coping Power parent website?” for parents, and a similar item asked children about the child website; and (d) “How easily were you able to access the Coping Power parent website?,” with a similar asking children about the child website (on a 0–7 Likert scale, with higher responses reflecting more positive perceptions and greater ease of use). There were separate questionnaires for children and parents.

Planned Analyses

The study used a randomized trial design, with matched pairs of schools randomized to intervention or control. Intervention effects investigated in Aim 1 (Hypotheses 1a and 1b) were tested in a PATH analysis model using the SAS 9.4 CALIS procedure’s Maximum Likelihood Method, with a covariance structure analysis. The 91 cases with full T1 and T2 data were included; the six cases with missing data (four missing Time 1 scores, two from each condition, and two Control cases missing Time 2 scores) were excluded. The PATH analysis model conducted in SAS is a multivariate linear model based on a diagram that specifies the relationships between variables, as indicated in Figure 1. The model tested in the SAS PATH procedure included the two T2 outcome variables (Aggression; Conduct Problems), and these T2 outcomes covaried with each other. The T2 outcomes were predicted by the independent variable, which was Condition (CP-IE; Control), and by the relevant T1 variable (e.g. T1 Aggression for the T2 Aggression variable). Main effects of Condition on the Time 2 outcome scores for aggression and for conduct problems test the study’s two hypotheses. The PATH approach in this model is a precise statistical approach for randomized experiments that focus on whether the intervention has produced differences in posttest scores, and that have pretest (Baseline) and posttest measurements. In comparison to other approaches, such as repeated measures designs or use of difference scores, this regression approach within CALIS increases statistical power by reducing within-group error variance, and also can address chance imbalances between conditions at baseline (Tabachnick & Fidell, 2007; Vickers & Altman, 2001). Including both outcome variables in the same PATH model avoids multiple analyses and permits the covariance between the two outcomes to be modeled. Cohen d effect sizes were computed for the condition effects on the two outcome scores with the T2-T1 mean score differences for CP-IE minus Control, being divided by the pooled T1 standard deviations for the two Conditions. Paired t-tests were used to determine whether there were significant changes in the behavioral outcomes from pre to post within each of the conditions. Finally, for the second Aim, partial correlations were computed to indicate how intervention process variables (attendance; website use) related to posttest outcome scores that had been significantly affected by the intervention, controlling for baseline outcome scores. Only statistically significant (p<05) results are discussed.

Figure 1. PATH analytic model with standardized path results.

The model depicts the effect of the Condition variable (Coping Power — Internet Enhanced vs Control) on the T2 (post-intervention) teacher-rated BASC subscales for Aggression and Conduct Problems, controlling for the T1 scores for those BASC subscales. The two T2 outcome variables were modelled to covary with each other. E is the error associated with each of the measured outcome variables.

*p <.05

Results

The means and standard deviations of the outcome variables for the CP-IE intervention and control children are contained in Table 4. The two conditions have comparable participants, as there are similar baseline T scores from the BASC Aggression subscale, t (89) = −0.32, ns, and from the BASC Conduct Problem BASC subscale, t (89) = 0.55, ns, with no significant differences. The two conditions are also comparable in children’s descriptive characteristics (see Table 2 for rates by Condition), including: children’s age, t (89) = 1.54, ns; children’s race, χ² (2) = 2.72; gender, χ² (1) = 0.39, ns; probability of having repeated a grade before baseline, χ² (1) = 0.14; and family income level, t (89) = −1.43, ns.

Table 4.

Means and Standard Deviations of Teacher—rated Behavior Assessment System for Children (BASC) scales

Variable	Time 1			Time 2
	Mean	SD	N	Mean	SD	N
Intervention Condition
Aggression	60.74	10.71	47	60.64	10.82	47
Conduct Problems	56.79	12.53	47	59.43	11.95	47
Control Condition
Aggression	59.95	12.65	44	62.36	14.04	44
Conduct Problems	58.18	11.72	44	64.36	13.99	44

Note. t scores are used for the BASC Aggression and Conduct problems subscales

Aim 1: Path Analysis Testing Hypothesis 1a and Hypothesis 1b

As seen in Table 5, the Condition effect for the BASC Conduct Problem Behavior score was significant, while controlling for T1 Conduct Problems, indicating that the change in Conduct Problems of children in the Control group across the 5^th grade year was significantly different than for children in the CP-IE condition. The Cohen’s d effect size for this path was .29. Cohen’s d was computed as the CP-IE difference score (Time 2-Time 1) minus the Control difference score (Time 2-Time 1), divided by the pooled T1 standard deviations for the two Conditions. There were also significant paths from T1 to T2 Conduct Problems, from T1 to T2 Aggression, and between T2 Aggression and T2 Conduct Problems. Inspecting the mean scores for Conduct Problems, the significant Condition results indicate that Hypothesis 1b was supported, as the Control group had worse Conduct Problem scores than did CP-IE children². There was no significant Condition effect on the Aggression score (Cohen’s d effect size: .21), and Hypothesis 1a was not supported.

Table 5.

Standardized PATH Analysis Results, Maximum Likelihood Estimation

Path	Estimate	Standard Error	t Value	p value
T1 Aggression to T2 Aggression	0.61	0.06	10.16	.001
Condition to T2 Aggression	−0.09	0.08	−1.14	.258
T1 Conduct Problems to T2 Conduct Problems	0.56	0.06	8.78	.001
Condition to T2 Conduct Problems	−0.17	0.08	−1.97	.049
T2 Aggression covarying with T2 Conduct Problems	0.412	0.06	6.61	.001

Note. SAS PROC CALIS t tests are equivalent to large-sample Z tests; N=91

Paired t-tests

Paired t-tests were conducted to examine whether there were significant changes in aggression and in conduct problems across time, separately within the two conditions. There were non-significant changes in the intervention children’s aggressive behaviors, t (46) = −0.08, ns, and conduct problems, t (46) = 1.72, ns. The Control children had nonsignificant changes in aggressive behavior, t (43) = 1.50, ns, but significantly increased their conduct problem behaviors from pre to post testing, t (43) = 3.67, p<.001. To further describe this effect on conduct problems, 34% of Control children were in the clinical range (T score greater than or equal to 70) at post-intervention on conduct problems, in comparison to 17% of CP-IE children.

Aim 2: CP-IE Website Use and Attendance as Predictors of Outcomes Significantly Affected by Condition

Children attended their face-to-face sessions at a very high rate, at nearly 100% (mean of 99.83%, SD = 0.01), while parent attendance was markedly lower, at 34.00% (SD = 30.00). Because child attendance was so high, it was not considered in the correlations below. Children’s completion of activities on the website (66.60%, SD = 30.24) was also three times higher than their parents’ completion of website activities (20.67%, SD = 30.04). When the child and parent attendance and website use variables were correlated, parent website use was highly correlated with children’s website use, r (47) = .54, p < .001, and with parent attendance at sessions, r (47) = .64, p < .001. Children’s website use was not significantly correlated with parent attendance at sessions, r (47) = .21, ns.

Pearson correlation coefficients between the three process variables (parent attendance; parent and child website use) and the Conduct Problem outcome variable (T1 and T2 levels) significantly affected by Condition are presented in Table 6. Three partial correlations were then computed, correlating each of the three process variables with Time 2 Conduct Problems, while partialing out Time 1 Conduct Problems. These partial correlations indicate whether a process variable was related to changes in the Conduct Problem outcome for children in the CP-IE condition. Children’s website use, r (46) = −.36, p = .014, and parents’ website use, r (46) = −.36, p = .015, were significantly inversely related to CP-IE children’s Time 2 Conduct Problems, controlling for Time 1 Conduct Problems. Higher website use was related to greater reductions in teacher-rated Conduct Problems. Parents’ rate of attendance at parent sessions was not significantly related to Time 2 Conduct Problems, controlling for Time 1, r (46) = −.23, p = .13.

Table 6.

Pearson Correlation Coefficients Indicating the Relation Between Process Variables and Time 1 and Time 2 Levels of Conduct Problems for Children within the CP-IE Condition

Process Variables	Conduct Problemsa
	Time 1	Time 2
Percent of Activities Children Accessed on Website	−0.13	−0.31*
Percent of Parent Sessions Attended	−0.20	−0.35*
Percent of Activities Parents Accessed on Website	−0.23	−0.38**

Note. CP-IE is Coping Power — Internet Enhanced.

^aN = 47.

^*p<.05,

^**p<.01

Satisfaction Ratings by CP-IE Children and Parents

Children and parents who attended sessions rated their satisfaction with the CP-IE program, using 0–7 Likert scales. Children and parents reported liking the CP-IE program very highly overall (means of 6.63 and 6.83, with standard deviation of 1.33 and 0.38, respectively), and in a fairly equivalent way with no significant differences between children and parents in their liking of the program, t (58) = −.79, ns. Children found the website to be slightly easier to access than parents (means of 5.70 and 5.29, with standard deviations of 1.97 and 2.26, respectively), but again the differences were not statistically significant, t (56) = 0.75, ns. Although both children and parents found the in-person Coping Power meetings to be very helpful (means of 6.40 and 6.72, with standard deviations of 1.45 and 0.72, respectively, in a very similar manner, t (58) = −1.07, ns, children reported that they found the website to be much more helpful than did parents (means of 6.08 and 4.61, with standard deviations of 1.33 and 2.35, respectively), t (56) = 2.97, p=0.004. While the children rated the website (6.08) to be nearly as helpful as their face-to-face sessions (6.40), parents rated the face-to-face sessions (6.72, which was nearly at the maximum of being helpful) as being markedly more helpful than the parent website (4.61, which was in the midpoint range of being helpful).

Discussion

The results from this randomized study of an innovative intervention, with hybrid face-to-face and internet Coping Power components, provide partial support that this much briefer, more efficient delivery model of this evidence-based program is feasible, and that it can reduce aggressive children’s risk for escalation in conduct problem behaviors in the school setting across the fifth grade year. The CP-IE has immediate prevention effects on reducing the rate of growth of children’s planful rule breaking behaviors (conduct problems) across the 5^th grade year, but does not have significant post-intervention effects on hostile aggressive behaviors. In prior research with the full Coping Power program, the intervention had significant effects at a three-year follow-up after the intervention on both planful, deliberate proactive aggression and on impulsive reactive aggression (Lochman et al., 2013), but the intervention produced significant effects at post-intervention only on planful proactive aggression (Lochman et al., 2002b). The immediate intervention effects appear to occur more on outcomes which require children to deliberately assess the likely consequences of their disruptive behaviors.

The findings for CP-IE effects on conduct problems underscore the importance of conducting rigorous intervention research on programs that include new technologies such as internet-based applications, because of their promise for wide-spread use nationally and internationally (Boydell, Hodgins, Pignatiello, Teshima, Edwards, & Willis, 2014; Stinson, Wilson, Gill, Yamada, & Holt, 2009). The CP-IE significantly reduced at-risk children’s risk for increases in conduct problems (Hypothesis 1b), but did not have a statistically significant effect on aggressive behavior (Hypothesis 1a). The intervention effects on conduct problems can have significant policy implications because it addresses a costly outcome in youth (Foster, Jones, et al. 2005), and the results suggest that comparable intervention effects can be obtained by a much shorter cognitive-behavioral intervention, and can thus be more efficiently implemented and have important implications for use as a Tier 2 support strategy in schools (McDaniel, Bruhn, & Mitchell, 2015; Stormont & Reinke, 2013). This innovative version of CP-IE can be palatable to school personnel and families, as long as (at least in this study) the shortened intervention is accompanied by an internet component.

Similar to what has been observed in prior studies of Coping Power in the past when the program effects have been assessed within one school year (e.g., Lochman et al., 2014), an untreated control group of at-risk aggressive children significantly escalated their rates of conduct problems behavior problems across the 5^th grade year (from fall to spring of 5^th grade). This increase in the current study was evident even though all of the schools were using SWPBIS (Bradshaw, 2013; Sugai & Horner, 2006) at all three Tiers. The Coping Power — Internet Enhanced (CP-IE) program had a preventive effect by preventing the sharp increase in teacher-rated conduct problems evident among at-risk control children. In the current study, the mean for the control group’s level of conduct problems moved into an at-risk clinical range (t scores at or above 60) by the end of the 5^th grade year, while the CP-IE children’s mean scores remained below that cut-off. Illustrating a prevention effect at post-intervention, the Control group had group had twice the rate of children in the BASC clinical range (T scores at 70 or above) than did children who had received CP-IE (34% versus 17%, respectively).

The size of the CP-IE program’s post-intervention effects on teacher ratings of problem behaviors are in the range of the effect sizes produced by the full CP program at post-intervention (e.g., Lochman et al., 2014). The effect size for CP-IE’s effect in preventing the sharp increase in conduct problems at post-intervention displayed by control children (Cohen’s d = .29) is in the same range as has found on behavioral outcomes for delivery of a full Coping Power program (Cohen’s d of .32 for Externalizing Behaviors in the Lochman et al., 2014 study; .30 for proactive aggression in the Lochman & Wells 2002b study). The obtained effect size for CP-IE is also in the same range as the mean effect sizes found in meta-analyses for other evidence-based programs such as the Incredible Years program effect on disruptive behavior (Cohen’s d =.27; Menting, de Castro, & Matthys, 2013), and the Multisystemic Therapy program’s effects on delinquency (Cohen’s d =.20; Van der Stouwe, Asscher, Stams, Dekovic, & van der Laan, 2014). The outcome effects for CP-IE are achieved with 60% fewer face-to-face sessions than are used in the full-length Coping Power program. Thus, the CP-IE outcomes are produced more efficiently.

We estimate that each set of traditional Coping Power parent and child groups (for 6 children and their caretakers), with a group leader and co-leader, requires a total of 150 hours of clinical time (includes 1 hour of preparation time and travel to the school, 1 hour of group delivery, and 1 hour for group-related paperwork, for 34 child group sessions and 16 parent group sessions). In contrast, we estimate that each set of CP-IE parent and child groups, with the same number of children, parents and leaders per group, requires a total of 76 hours of clinical time (includes 1 hour of internet/website work in addition to the typical time for preparation for group sessions, running sessions, and for group-related paperwork, for 12 child sessions and 7 parent sessions). Including the cost of program and curriculum materials and typical staffing costs for program delivery (with a master’s level group leader and a bachelor’s level co-leader), we estimate that total cost for a traditional Coping Power group is $5,844 and the total cost of a CP-IE group is $3,254, yielding a savings of $2,590 for the CP-IE group. Even when the extra time required for the clinician to manage the websites and respond to children’s website messages, the CP-IE program appears to be markedly more cost-effective than the traditional CP program.

The CP-IE Web-Based Component

The effectiveness of this brief version of CP, with its reduced number of face-to-face sessions, in addressing conduct problems may occur for several reasons, including (a) the face-to-face sessions are tightly integrated with internet-provided material, (b) the animated cartoon series appeared to activate children’s attention to the Coping Power skills, and (c), like the full version of Coping Power, CP-IE included parent and child components that address specific risk factors for children’s conduct problems as indicated in the contextual social-cognitive model serving as a basis for the program. Despite the sharp reduction in face-to-face sessions, all of the CP intervention objectives in the child and parent Coping Power components are retained. The website material introduces the social-emotional and behavioral skills to children and parents before their face-to-face sessions. Thus, the children and parents seem to acquire knowledge of the new skills through the internet component, and the face-to-face sessions may permit the children and parents to perform and practice skills they have learned, and to apply them to their particular circumstances. This integration of the two formats in the intervention is consistent with the cross-channel additivity hypothesis (Okdie et al., 2014) which presumes that the overall intervention could be strengthened by taking advantage of the unique properties of each media channel (in the current case, internet and face-to-face interaction). The use of multiple channels in intervention delivery uses the internet and the narrative stories in the cartoon series on the webpages to increase children’s and parents’ participation in the program and their retention of material (Kennedy, 2004), while retaining the face-to-face time with the clinician.

The increased participation may have occurred because the children identified with the characters in the Captain Judgment animated series. Previous research has consistently found that people who identify with characters in a story are more motivated to watch and process the information in the story (Cohen, 2009; Moyer-Guse, 2008). The website material was well-received by the children, as evident in their satisfaction ratings. Group leaders’ also reported anecdotally in supervision sessions that children often discussed details from the website in group sessions. Children came into the sessions discussing the details of the cartoon video episodes that illustrated the social-emotional skills they would address in session that week, indicating they had cognitively incorporated the relevant concepts. Through children’s use of the “message” function on the website (some children chatted with the therapist daily through the message function, and some even multiple times a day), children may have deepened their therapeutic alliance with the therapist. If so, enhanced alliance may have increased children’s motivation to work on program tasks, and helped them find the clinicians’ social reinforcement to be, in fact, reinforcing. The likelihood that program skills could be generalized to children’s interactions with peers and adults may have been enhanced because the children recalled the skills frequently throughout the week because of their chats with the clinician. These anecdotal reports from group leaders are speculative, and future research can determine whether these factors are indeed mechanisms operating in the CP-IE program.

Finally, prior research with the full Coping Power program has found that program effects on children’s outcomes are mediated by their change in specific social-cognitive skills (Lochman & Wells, 2002a). Although unable to test mediation in the current study because of sample size, changes in children’s attributional processes and problem-solving skills may assist their inhibitory control, and may permit them to better avoid behaviors that can lead to negative consequences for them.

Parents Reactions to, and Use of, the Internet Component

Some parents were interested in the website, but in general parents were less enthralled with their website than their children were with theirs. Based on their satisfaction ratings, it was evident that parents saw the overall CP-IE program as being very helpful, but they preferred the face-to-face sessions to the website. This suggests that a hybrid version of the program, which retains face-to-face time with the clinician, as developed here, may be particularly important for parents of at-risk children. Future research can explore if parents of at-risk children may be less open to a purely internet intervention.

An important finding indicated that parents’ website use did significantly predict the change in children’s conduct problems across the 5^th grade year, with the greatest effect on conduct problems, relative to the Control group, occurring for parents who were most active on the website. Although we lack data to confirm the mechanisms accounting for the predictive ability of parent’ website use on outcomes, it may be that those parents who were most active on the website learned the parenting skills more deeply, and may also indicate that they became more attentive to their children and their children’s actions (enhancing monitoring, consistency of discipline, and likelihood of giving more reinforcement for children’s positive behaviors). In a parallel way, children’s own website use also predicted their reduced rate of increase in conduct problem behavior. Although website use predicted conduct problem outcomes, other “third variables” may have been the cause of these associations. Thus, parents who were more ready to prevent deterioration in their children’s behavior may have been more motivated to access the website, or parents who experienced fewer stressful life events or who received more social support (both predictors of parent attendance in parent meetings in the past; Minney et al., 2015) may have been more likely to access the website more. Future research could explore the active mechanisms which account for this effect of website use.

Lessons Learned about Using Technology-enhanced Intervention

A series of challenges were encountered in implementing the internet portion of the intervention, requiring the program developers to refine the webpage platform and content several times to optimize children’s and parents’ perceptions of the website component. Valuable anecdotal feedback was provided by parents and children during group sessions throughout the program, including open discussion at the end of the intervention. Consistent with prior reports of barriers to use of internet applications (Boydell et al., 2014; Christensen, Reynolds, & Griffiths, 2011), some participants had low use of the website. Staff observations during the early piloting, informal discussions after each cohort completed intervention, and in CP-IE parent groups that some parents did not appear comfortable using computers to locate intervention websites, and many homes did not have computers with wireless access. However, as indicted in the Satisfaction Survey reports in the Results section, most parents reported they could access the internet (mean rating of 5.29 on a 0–7 scale). Thus, the website was re-made to make it “mobile-friendly” so the webpages could be readily accessed and used from smart phones and tablet devices. Access to smart phones was very common in the sample, evident in parent discussions in CP-IE intervention sessions, and was often parents preferred method for accessing the internet. An app is being developed which will make it even easier for parents and children in the future to access the CP-IE websites.

Parents’ anecdotally indicated during CP-IE parent groups in the first cohort that it was important to have some elements and information on the parents’ webpage that was in addition to what was covered in the parent group sessions, to increase parents’ interest in the website. Parents had widely varying reactions to the Dr. Insight cartoon on the webpage, with many finding it informative and humorous, but several parents reported during sessions they didn’t like cartoons of this sort. Thus, consistent with reviews that have suggested that the breadth and flexibility of online applications can affect successful delivery of mental health services (Boydell et al., 2014), it was important to have useful, novel and interesting information in other formats on the parents’ webpage dashboard in addition to the cartoon series.

Children reported in their groups that most elements of the webpage, including the cartoon series on the children’s webpage, were useful. Consistent with research reviews that found that internet features that enhance interactivity may influence outcomes (Stinson et al., 2009; Wantland et al., 2004), children visited their CP-IE website more often once the site was altered to: a) allow children to select their own screen name and avatar, thus personalizing the site for themselves; b) permit children to browse a colorful section showing the “prizes” they could earn with their points, and to see a current update of the number of points they have earned and are available for them to use to buy prizes; c) have content that was simple enough, and graphically interesting enough, to be attractive to children who had lower cognitive functioning, so they could read and respond to the content; d) permit children to self-report on their goal attainment, in addition to the typical procedure of having teachers/parents monitor and report on children’s goal attainment; and e) permit easy “chats” with the clinician through the personal messaging system. Because of the concern about possible deviancy training among children in the group if they were permitted to have unstructured conversations with each other, children were not permitted to message each other. In addition, permitting children to earn points when their parents accessed the parents’ website appeared to increase children’s encouragement of their parent to become more engaged with the parent website.

Strategies for Addressing General Barriers in Implementing School-Based Prevention

Certain challenges were experienced in the delivery of the CP-IE program in the schools. The strategies that evolved for dealing with some of the most frequent challenges are summarized below.

• Coping Power leaders sometimes were not readily given time by school staff to meet with children at regular points during the school day.

  Strategy: Leaders were as flexible as possible to come at any day or time offered, including meeting during lunch if necessary.

• Guidance counselors were not consistently available to serve as co-leaders.

  Strategy: Group leaders tried to anticipate counselor absences and retain counselor as co-leader. When counselors still consistently missed, a Coping Power staff member served as co-leader instead.

• Teachers sometimes did not provide daily feedback on children’s personal goal attainment.

  Strategy: Provided reminders to teachers in-person, via CP website, and via their official school email account. Leader also brought a phone or tablet when the leader came to gather a child for group and the teacher could report on children’s goal attainment then. Also allowed teachers to provide feedback on website or on paper, depending on their preference.

• Teachers needed to allow or remind children to use a school computer for daily goal reporting and/or to complete website activities.

  Strategy: set up a specific schedule, such as first thing every morning, or during scheduled computer lab time.

• Teachers or administrators expressed frustration when intervention children exhibited periodic behavior problems.

  Strategy: Staff are responsive to specific concerns and describe the behavior ‘shaping’ process. Sometimes staff describe that intervention skills may prevent worsening of behavior problems during the year (as seen in the results for this study), and into the following middle school years, rather than leading to immediate, observable behavioral improvements.

Limitations

There are a set of limitations to this study, and the limitations can lead to future directions for research. First, this was a small-scale feasibility study, so future research with larger samples in multiple geographic sites will be important to determine the generalizability of effects.

Second, follow-up assessments are not available for the full sample, and thus it is unclear how durable the effects are over time. Future research should include longer-term follow-up to determine whether the program is successful in preventing more serious conduct problems in adolescence.

Third, because the teachers knew that children were receiving intervention, their ratings may have been biased; Teachers in future follow-up studies could be blind to intervention, which may provide less biased data. In addition, future studies can include direct behavioral observation of children’s school behavior.

Fourth, because there was no direct comparison between CP-IE and a randomly assigned condition in which children would receive typical Coping Power, it cannot conclusively be determined the comparative effects of the two versions of CP. Future research can compare CP-IE to the full Coping Power program as well as service-as-usual in the schools.

Fifth, because there was no randomly assigned “pure” internet condition, there is no direct evidence that the hybrid version of CP would have been better than an internet-alone version. An interesting next research study would involve comparing CP-IE to an internet-only Coping Power condition which would include all of the psychoeducational content of Coping Power, but no face-to-face intervention sessions. If an internet-only condition produced outcomes similar to the hybrid CP-IE program examined in the current study, then a version of Coping Power that would be even more efficient and with broader reach, would be available.

Sixth, although we were able to provide estimates of staff time required for the project, there was only limited anecdotal data available to make cost-effectiveness analyses. It would be important in future research to gather information about court involvement, and mental health and social services used by the families, costs related to antisocial behavior (Foster, Jones & Conduct Problems Prevention Research Group, 2006), along with a detailed precise recording of staff time required to maintain the website to determine whether CP-IE is indeed more cost-effective than the traditional version of CP.

Seventh, because of limited resources, the videos of sessions were not rated by an independent observer for completion of intervention objectives. Optimally, conclusions about intervention integrity can be best made when there are multisource ratings (Perepletchikova, Treat, & Kazdin, 2007) that include a direct measure that monitors therapist adherence (Sheridan, Swanger-Gagné, Welch, Kwon, & Garbacz, 2009), in addition to the procedures used in this study (i.e., manuals with detailed step-by-step procedures and activities, and therapist report of completion of session objectives; Gresham & Lochman, 2009).

Eighth, this study only used behavioral indicators of intervention engagement (attendance, website use) and did not include motivational measures of engagement. Behavioral and motivational indicators of engagement have been predicted by different factors (Lochman, Powell et al., 2009) and have differentially predicted outcomes (Nix et al., 2009). Future research should include indicators of motivational engagement as well as behavioral engagement.

Summary

This study with a randomized control group is the first to demonstrate the feasibility of a flipped intervention, with hybrid face-to-face and internet Coping Power components, for preadolescent aggressive children and their parents. The CP-IE program reduces children’s risk for escalating conduct problem behaviors at school during the 5^th grade year. Of most importance, the hybrid intervention is briefer and can more efficiently produce similar post-intervention behavioral effects as has been evident with the full program. More efficient delivery of a structured cognitive-behavioral intervention like Coping Power can substantially add to its ability to be disseminated more widely and assist more children. Results also indicated that parents who are more engaged with the website have children display the prevention effect of not having the significant increases in conduct problem behaviors evident among Control children.