Intervention to Strengthen Emotional Self-Regulation in Children with Emerging Mental Health Problems: Proximal Impact on School Behavior

Abstract

A model for teaching children skills to strengthen emotional self-regulation is introduced, informed by the developmental concept of scaffolding. Adult modeling/instruction, role-play and in vivo coaching are tailored to children’s level of understanding and skill to promote use of skills in reallife contexts. Two-hundred twenty-six kindergarten—3rd grade children identified with elevated behavioral and social classroom problems from a population-based screening participated in a waitlisted randomized trial of the Rochester Resilience Project derived from this model. In 14 lessons with school-based mentors, children were taught a hierarchical set of skills: monitoring of emotions; selfcontrol/reducing escalation of emotions; and maintaining control and regaining equilibrium. Mentors provided classroom reinforcement of skill use. Multi-level modeling accounting for the nesting of children in schools and classrooms showed the following effects at post-intervention: reduced problems rated by teachers in behavior control, peer social skills, shy-withdrawn and off-task behaviors (ES 0.31–0.47). Peer social skills improved for girls but not for boys. Children receiving the intervention had a 46% mean decrease in disciplinary referrals and a 43% decrease in suspensions during the 4-month intervention period. Limitations and future directions to promote skill transfer are discussed.

Problems with self-regulation of emotions are linked to the etiology and maintenance of diverse problems including depression (e.g., Forbes and Dahl 2005), antisocial behavior (Hinshaw 2002), addictions (Kuntsche et al. 2007) and suicidal ideation (Wyman et al. 2009). To reduce or prevent the occurrence of these problems, interventions to strengthen emotion self-regulation processes are widely used. For example, anger control skills are taught to reduce emerging aggressive-disruptive problems (e.g., Lochman et al. 1993), strategies to reduce ‘downward emotional spirals’ are taught in cognitive-behavioral interventions for depression (e.g., Asarnow et al. 2002), and increasing tolerance of distressing emotions is a focus of interventions to reduce children’s self-harm behaviors (e.g., Katz et al. 2004). Emotion competencies are also taught in classroom prevention curricula (Greenberg et al. 1995). Evidence regarding the effectiveness of these skill-focused interventions varies according to the targeted problem, age of child, and existence of co-occurring problems. Many such interventions reduce problems including depression (Asarnow et al. 2002) and disruptive behavior problems (Lochman et al. 1993; Webster-Stratton and Hammond 1997). However, few have demonstrated long-term generalized benefits. Minimal transfer of new skills by children to settings outside the intervention context is a limitation (Webster-Stratton and Hammond 1997), suggesting that strengthening children’s adoption and generalization of skills may increase their effectiveness.

Few conceptual models have addressed the specific processes involved in teaching children cognitive and behavioral skills in interventions. Questions receiving little attention include: What types of new knowledge are involved? How do children acquire and combine different types of knowledge to create skills they apply beyond the intervention context? Drawing on theories of learning and research on development of self-regulation processes, we describe a multi-stage approach for teaching young school-age children skills to strengthen their self-regulation of emotions. Our approach combines adult modeling and verbal instruction, role-play and ‘in-vivo’ coaching, and the pace and focus of adult-child interactions are tailored to meet a child’s level of understanding, informed by the developmental concept of scaffolding (Wood et al. 1976). We applied this approach in a school-based intervention for children with emerging behavioral and social-emotional problems (Rochester Resilience Project) and report results from a wait-listed randomized trial evaluating its impact over a 4-month intervention period.

Emotion Systems and Self-Regulation Processes—Developmental Considerations

A large literature on emotion systems and self-regulation has been advanced from fields ranging from neuroscience to developmental psychology. Although it is beyond the scope of this paper to review that literature, we outline several themes that have implications for interventions that teach children emotional self-regulation skills.

Most current models emphasize the functional role of emotions and the reciprocal balancing between emotion systems and higher brain centers in self-regulation processes (Derryberry and Tucker 1992; Gross 1998). Emotions initiate a series of predictable physiological and experiential changes (Lang 1995) and coordinate brain activity through a series of systems developed to address different adaptive challenges (LeDoux 2002). In tandem with emotion systems, a mode of functioning characterized by reappraisal and regulation of emotional responses serves to organize human experience and behavior according to diverse literatures (Carver et al. 2008; Metcalf and Mischel 1999) including neuro-imaging studies (Ochsner et al. 2002). Within this two-mode perspective on self-regulation, simultaneous processing and reciprocal balancing is ongoing between (a) emotion systems characterized by automatic, reflexive and associational processing, and (b) a control mode characterized by top-down processing, including symbolic functioning required for planning and strategic behavior (Carver et al. 2008).

Many childhood psychosocial problems, reflecting diverse etiological processes, involve maladaptive integration of cognitive controls and emotion systems that affect how children experience, express and regulate emotional responses. For example, high levels of negative affect and escalation of anger in parent-child interactions are among the factors associated with emerging childhood externalizing behavior patterns (Hinshaw 2002). The role of emotion regulatory problems in aggressive-disruptive behavior is underscored by evidence that dysregulated emotion responses are a specific factor differentiating children with shared risk factors such as ADHD who have aggressive behavior problems from those who do not (Melnick and Hinshaw 2000). Mood disorders are also associated with emotion regulatory problems at several different levels. For example, depression is associated with decreased ability to shift attention and reappraise negative emotions (Levin et al. 2007), a deficit that may increase depressed youths’ vulnerability to ‘downward emotional spirals’ in response to stress (Asarnow et al. 2002). Due to these associations, many skill-based interventions focus on strengthening emotion self-regulation processes, which include skills to modulate the intensity and duration of emotional states (Forbes and Dahl 2005; Thompson 1994). For preadolescent children, cognitive inhibitory control and expanded language for emotions are developmentally-salient skills associated with adaptive emotion self-regulation (Riggs et al. 2006). In the intervention model we present below, explicit attention is given to matching the skills, and the manner in which they are taught, to children’s abilities and level of understanding.

Although frequently considered an individual characteristic, emotion regulation is conceptualized within developmental models as a transactional process first arising out of an infant’s experience of patterns of arousal and modulation of emotion in relationships (NICHD Early Child Care Research Network 2004). Infants’ and caregivers’ mutual averting of their gaze to modulate arousal illustrates this dyadic process. This process evolves but remains transactional as children assume more independence in self-regulatory processes, with an increasing number of other adults and peers influencing the emotional demands that children experience as well as their response to those demands (Dodge 1989). Developmental models also underscore how patterns of emotional exchange contribute to the formation of children’s sense of self and to mutual expectancies within relationships (e.g., Cicchetti and Toth 1998). For example, the resistance to change of negative, coercive exchanges between parents and children (Bates et al. 1995) and maladaptive exchanges linked to maternal depression (e.g., Jameson et al. 1997) are posited to stem from maladaptive expectancies and attributions within parent-child relationship systems. This dyadic, transactional perspective suggests that children’s emotion self-regulatory patterns occur within a relational field, and the dyadic nature of that system can present obstacles to, and opportunities for, modification. To incorporate a dyadic perspective in our intervention approach, children develop a relationship with a supportive adult from whom they learn new strategies, receive reinforcement for use of skills and assistance in developing narratives about their own use of skills. The adult conducts reviews that include preparing the child to anticipate opportunities and challenges in using newly acquired skills.

The developmental literature also underscores the diverse ways that children learn about emotions and emotion regulation in relationships. Children learn about emotions in themselves and others, and about language for emotions through observation and non-verbal behavior exchanges as well as through verbal instruction (e.g., Dunn and Munn 1987). Non-verbal and verbally-mediated teaching approaches engage both visual and auditory processing channels that may provide children with complementary knowledge about emotions and suit different learning styles. Recognizing these diverse learning vehicles, the model we outline incorporates several strategies including adult modeling of skills, verbal instruction, and adult-child role-plays.

Types of Knowledge and Developing Skill

The distinction between declarative and procedural knowledge (Anderson 1993) is useful when considering the preparation children need to acquire new skills. Corresponding to two primary memory systems (Squire et al. 1993), declarative knowledge refers to information that can be verbally stated and is available for conscious recollection, whereas procedural knowledge is represented implicitly. Procedural knowledge is embodied and reflects learning within motor, perceptual and other systems. Simply put, declarative knowledge reflects what a person knows and procedural knowledge reflects what a person does (LeDoux 2002). For many types of learning, the acquisition of skill follows a predictable sequence (Sun et al. 2001), beginning with the acquisition of declarative knowledge (i.e., how to perform the task). In this initial stage, the individual must access declarative knowledge in order for a task to be performed, which requires focused attention. Repeated practice is a primary means by which individuals learn aspects of how a task is performed, and when a skill has become ‘proceduralized’ it can be performed more automatically with less reliance on recall of declarative knowledge (Anderson 1993).

The processes of learning and use of new knowledge are also influenced by experiential state and context. The concepts of ‘cold’ and ‘hot’ cognition (Damasio 1994) illustrate changes in attributions, self-perceptions and other processes that accompany emotional activation and are related to state-dependent effects of learning and recall (LeDoux 2002). One implication is that learning and practice opportunities that span across both emotionally neutral contexts that typify ‘cold’ cognition and contexts evoking challenging ‘hot’ cognition are likely to promote transfer of new skills to settings that pose challenges for children. Second, emotional activation associated with stress weakens use of declarative knowledge and strengthens procedural knowledge (LeDoux 1998). This relationship between emotional responses and the two primary memory systems involved in learning suggests that under stressful situations that evoke challenging emotions children will be more likely to respond in ways that are highly familiar or proceduralized. In such situations they also will be less capable of accessing declarative knowledge needed to make purposeful choices for behavior, based on what they know they ‘should do’ or what they believe that others expect them to do. How skill-focused interventions can assist children in developing more adaptive, proceduralized strategies for emotion self-regulation is a principal challenge.

Present Study

We developed the Rochester Resilience Project to address the needs of young children with emerging behavioral and social-emotional problems by providing an accessible school-based intervention (Wyman et al. 2004). In the context of a relationship with an intervention mentor over four months, children learn and practice behavioral and cognitive skills designed to strengthen their self-regulation of emotions and address specific goals to improve school adaptation. The present study reports results from a randomized trial of this intervention with kindergarten—3rd grade, urban children. The primary questions addressed were as follows: (1) Determine the intervention impact on classroom behavioral and social-emotional functioning at the end of the 4-month intervention period, as well as on formal disciplinary incidents from school records during the intervention period. (2) Identify differences in intervention response as a function of children’s grade level, sex, and classroom-level differences. Older children were expected to show greater benefit due to more developed cognitive skills such as planning and self-initiated behaviors required to utilize their new skills (David-Ferdon and Kaslow 2008). We also expected that children would benefit more from classrooms with lower overall levels of problems due to more positive classroom behavior norms that may reinforce children’s adoption of more adaptive self-regulatory strategies. A final exploratory aim was to determine if children presenting with different patterns of classroom problems responded differently to the intervention. In prior skill-based interventions to address behavior problems, for example, young children with conduct problems and coexisting anxiety/depression problems have fared better in some short-term intervention studies (Beauchaine et al. 2005).

Method

Participants and Design

Participants were 226 children in two elementary schools in a mid-sized city (total population about 230,000). Guided by a developmental epidemiology prevention approach (Kellam et al. 1999), our evaluation focused on a representative subgroup of children, which minimizes selection bias and strengthens inferences about intervention effects (Brown et al. 2007). The defined population in this case consisted of kindergarten—3rd grade children manifesting problems in two or more of the following domains: behavioral, social-emotional, and/or on-task learning behaviors at school. This population was selected due to evidence that those problems predict future mental health disorders, substance abuse problems and need for services (Jones et al. 2002) and evidence that problems in multiple domains further elevate that risk (Kellam et al. 1982).

For the purpose of identifying children with elevated problems from the population, all kindergarten—3rd grade teachers completed behavior ratings for all children four—six weeks after school started. Special education classrooms were excluded. Screening was conducted over two years. Teachers received standardized instructions from a member of the research team to complete the screening measure. A different measure was used for screening in year 1 and in year 2. Teachers completed the 32-item Teacher-Child Rating Scale (T-CRS; Hightower et al. 1986) in the first year. To reduce time burden on teachers, the 13-item AML-Revised screening measure (Lorion and Cowen 1984) was used in year 2. AML refers to the following behavior domains: aggression (A), shy-anxious or moody (M), and behaviors that interfere with learning (L). Both the T-CRS and AML-R have subscales assessing aggressive-disruptive, withdrawn-anxious, and on-task classroom behaviors. The T-CRS also has a subscale assessing peer social skills, whereas peer items are included in the AML-R aggressive behavior subscale. Subscales of the AML-R and T-CRS covering related constructs are moderately correlated, and both measures have been extensively used in evaluating school-based programs in the U.S.; age and gender norms for both measures have been established for urban, suburban, and rural settings (Cowen et al. 1996). Children scoring ≤30th percentile on two or more subscales on the screening measure (i.e., lowest 1/3rdof adjustment), using norms appropriate to the child’s grade and gender, were deemed ‘screen positive’ and recruited for the intervention trial. If two or more siblings screened positive, one was randomly selected.

Seven-hundred seventy-four children were screened by teachers in year 1, and 17.7% (137) met the positive screen criteria. Excluding children who enrolled in the first year, 693 children were screened in year 2 and 21.4% (148) met the positive criteria. Individual identifiers were not retained for children from year 1 who had not enrolled, per agreement with the schools; consequently, the total number of children screened over the two years, or the number who screened ‘negative’ in year 1 and ‘positive’ in year 2, could not be determined. Of 285 children who screened positive over two years, 261 were enrolled (92% recruitment). Rate of enrollment was comparable in both years. No differences were found between enrolled and non-enrolled children. Once enrolled, children were stratified by gender and randomized within classrooms using computer generated pseudo-random numbers to either immediate intervention or to a wait-list for later intervention. This type of classroom blocked design allowed us to hold fixed teacher and classroom context effects in examining impact and is efficient in statistical power (Brown and Liao 1999). Comparison of intervention and control groups on baseline measures after the first year of enrollment showed a significant difference between children in the two groups on baseline aggressive-disruptive behavior. To address this imbalance in the second year, children in the same classroom were stratified by baseline behavior control and gender before being randomly assigned to intervention or control condition.

Parents of eligible children were sent a letter inviting their child to participate followed by phone contacts. Written consent from parents and verbal assent from children were obtained. The University of Rochester Institutional Review Board approved the study protocol. After enrollment but before the intervention began, 13.4% of children were lost to attrition: 23 (8.8%) relocated to other schools and 12 (4.6%) were assigned to special education classrooms, which precluded their continued participation. None withdrew for other reasons, and none of the remaining 226 children were lost to attrition after the intervention started. Participants were from 59 classrooms, with an average of 3.5 students per classroom participating (range 1–10). Males were 54.4% of the sample. Across the two schools 60% of children in the school population were African American and 90% were eligible for free or reduced school lunch fee. The race/ethnicity proportions of our sample were as follows: 61.5% Black, 25.7% Hispanic, 8.4% White, and 4.4% other.

There were no differences on any baseline measure (see below) between children who enrolled in the first or second year, suggesting that the change in teacher screening measure did not introduce a difference in children enrolling in year 1 versus year 2. Characteristics of the longitudinal sample by condition are summarized in Table 1. The intervention group (n=111) and control group (n=115) were balanced on gender, race/ethnicity, and grade level. Children in the wait-list group received the same intervention in the spring of the year they enrolled.

Table 1.

Sample Characteristics by Condition

	Intervention (n=111)		Control (n=115)
	N	%	N	%
Gender
Male	62	55.9	61	53.0
Female	49	44.1	54	47.0
Ethnicity
Black	71	64.0	68	59.1
Hispanic	26	23.4	32	27.8
White	10	9.0	9	7.8
Other	4	3.6	6	5.2
Grade
Kindergarten	27	24.3	27	23.5
1	35	31.5	32	27.8
2	24	21.6	24	20.9
3	25	22.5	32	27.8

Rochester Resilience Project Intervention

Interventionists (Resilience Mentors) teach children a hierarchically-ordered set of skills from emotion self-monitoring to cognitive-behavioral strategies in 14 weekly lessons (Cross and Wyman 2004) and assist children in applying those skills to meet goals established with teachers. The intervention model conceptualizes emotion self-regulation skills as tools to assist children in reducing problems but not necessarily sufficient. Aggressive behavior, for example, is more likely to occur if a child escalates his/her anger, but may also occur under calm emotional conditions. Consequently, children require adult guidance and reinforcement in choosing adaptive behaviors. The intervention causative model (Wyman et al. 2004) derives from research linking problems in emotion self-regulation with behavior, mood, and social relationship problems as those deficits interact with risk factors in family, classroom and peer group settings (Cole et al. 1996; Shields and Cicchetti 1998).

In the first phase, activities establish the Resilience Mentor as an empathic adult informed about each child’s life context, strengths and challenges. Through adult-led interactive learning and ‘in-vivo’ practice (i.e., use in natural settings), children are taught three hierarchically-ordered skills: (1) Monitoring of one’s own and others’ emotions through emotional vocabulary, cues to identifying feelings in self and others and intensities of feelings. Mentors introduce ‘feelings check-in’ as a standard practice that serves as a teaching tool about feelings and a transition to skills focused on managing feelings. In conceptualizing self-monitoring, we followed LeDoux’s (1998) definition of feelings as those experiential changes brought into working memory via attention, from the broader domain of physiological and experiential changes associated with emotional reactions. This definition suggests that self-monitoring of feelings is an active process requiring attention, which can be learned. (2) Self-control and reducing escalation of emotions is taught through the concept of a ‘feelings thermometer’ to depict intensity along the dimension of an emotion; the concept of using ‘mental muscles’ is taught as a tool for children to monitor feelings and gain control to stop feelings from entering a ‘hot zone.’ (3) Skills for maintaining control and regaining equilibrium are taught through strategies including taking a deep-breath, ‘stepping back’ (behaviorally and/or cognitively) from emotionally intense situations, and using an ‘imaginary umbrella’ to protect oneself from hurtful words; and developing realistic appraisals of control over aspects of their life. The cognitive and behavioral skills taught to children are labeled in simple terms suited to developmental level. In the context of developing skills for managing emotional reactions, the interventionist uses ‘in vivo’ opportunities to address the feelings and reactions that pose problems for each child and to encourage and reinforce alternative, more adaptive behaviors (e.g., self-calming substitution for aggressive behavior; assertive response for withdrawn children).

Figure 1 illustrates the sequence of adult-child interactions designed to promote children’s acquisition and use of new skills. Influenced by the concepts of ‘scaffolding’ (Wood et al. 1976) and zone of proximal development (Vygotsky 1978), learning opportunities are focused on the dynamic and continually evolving range of knowledge and skills a child is capable of attaining under guidance from a competent adult. Verbal and nonverbal methods (e.g., adult modeling) are used to provide bridges between prior knowledge and new learning and to introduce structure into the interaction, with the goal of transferring to the child increased independent use of new skills. In introducing each skill component, Mentors begin with modeling and verbal teaching in unchallenging, neutral situations within semi-private intervention rooms. The child’s role in learning new skill components is increasingly active and personalized, from guided imitation of the adult model to purposeful practice with adult prompts, to adult ‘coaching’ of the child to use new skills in emotionally challenging contexts. ‘In-vivo’ practice is aimed at increasing a child’s proceduralized skill (Sun 1997) in contexts in which ‘hot cognition’ is activated (Damasio 1994) and experience in applying those skills in diverse interpersonal contexts. Mentors implement each lesson using a standardized structure that parallels the overall approach, with verbal and non-verbal teaching of the topic/skill, practice, and review and anticipatory guidance to transfer skills. To assist children in learning and using skills, training for mentors includes attention to calibrating the level of support each child requires over time, which is expected to be dynamic and variable. Reinforcement and feedback from Mentors in settings in which children use new skills is also critical to successful skill transfer (Cross et al. 2009). Mentors collaborated with teachers to identify classroom situations in which the mentors could provide reminders to children to use new skills, and cues were provided (e.g., a sticker or button for the child to take as a reminder).

Fig. 1.

Scaffolding model for teaching emotion self-regulation skills

Four female paraprofessionals employed by the School District were trained as Resilience Mentors to deliver the intervention (two in each school). For each of the 14 weekly lessons, children met individually with their Mentors for approximately 25 minutes in a private setting during the school day. Teachers received an orientation to the intervention during a faculty meeting at the beginning of the school year and met with Resilience Mentors within two weeks after children were assigned to condition to establish behavioral goals for children receiving the intervention. Thereafter, teachers received bi-monthly updates on the intervention skills during regularly scheduled grade-level meetings that lasted approximately 15 minutes. Attendance of teachers was nearly 100%. The purpose was to inform teachers of the skills children were learning and their role in collaborating with mentors to identify situations for mentors to reinforce children’s use of skills. Teachers were not trained to teach or ‘coach’ children in their use of skills. Parents were invited to receive a 30 minute orientation to the intervention through a meeting with their child’s Resilience Mentor at school or in their home (54.1% participated).

Measures

The same measures were used in both years of the study to evaluate intervention impact.

Classroom Behavior and Social-Emotional Functioning

At least one month after teachers completed screening measures and before children were randomly assigned to intervention or control condition, teachers rated each child’s functioning using the Teacher-Child Rating Scale—Version 2.1 (Hightower et al. 1986). The 32-item T-CRS assesses problems and competencies in four subscale-domains: (1) Behavior Control (“Accepts imposed limits”), (2) Task Orientation (“Functions well even with distraction”), (3) Assertiveness vs. withdrawn, anxious behavior (“Nervous, frightened”), (4) Peer Social Skills (“Classmates like to sit near this child”). Teachers rate how well each descriptor fits the child’s behavior using a 5-point Likert scale (Strongly Disagree—Strongly Agree). One-month test-retest coefficients for T-CRS subscales range from 0.80–0.90 (Hightower et al. 1986), and moderate stability of over a 2–3 year period based on ratings of the same child by different independent raters (Wyman et al. 1993). In our sample, internal reliability of subscales ranged from 0.80–0.94. Studies correlating the T-CRS with the longer Achenbach CBCL suggest adequate convergent and divergent concurrent validity (Perkins and Hightower 2000). Teachers completed the C-TRS a second time four months later for all children enrolled in their classroom, without reference to their initial ratings for each child.

Office Disciplinary Referrals and Out-of-School Suspension

Office Disciplinary Referrals (ODRs) were collected from school records for three months before and for four months after the intervention began during the same school year. ODRs were recorded on a form used school district-wide requiring documentation of a specific behavioral infraction (e.g., hitting another child), a practice that increases validity of those records as an index of behavioral status (Irvin et al. 2004). The validity of ODRs as indicators of behavioral status and school adaptation has been supported by their association with lower academic achievement and school bonding over time (e.g., Bryant et al. 2000). Out-of-school suspensions were also collected for the same intervals. The decision to initiate suspension was initiated by school principals and to receive a suspension a child had to receive a formal hearing after a specific infraction (e.g., carrying a weapon). Rates of suspensions increase in proportion to the seriousness of behavior problems measured through other methods (e.g., Huisinga and Jakob-Chien 1998), suggesting their utility as an indicator of conduct problems. Rates of both disciplinary referrals and suspensions vary systematically across schools (Irvin et al. 2004). Our classroom blocked design (i.e., random assignment of children to intervention or control groups from the same classroom) allowed us to test intervention impact on ODRs and suspensions by holding fixed those school contextual effects (Brown and Liao 1999).

Statistical Analyses

We tested baseline equivalence of intervention and control groups using ordinary t tests for continuous measures, chi-square tests for categorical variables, and logistic regressions for dichotomized count data. To determine whether attrition might have differentially affected intervention and control groups, we conducted analyses comparing children who completed follow-up with those who were missing.

Our analyses of intervention impact took into account the nesting of children within classrooms that are further nested within schools. To examine intervention impact on continuous data we used a three-level linear mixed-effects model (LMM), with level 1 including individual covariates (sex, grade, race/ethnicity), level 2 accounting for classroom variability and level 3 adjusting for variability across schools. Schools were included in each model as a fixed factor and classrooms were included as random factors. All models include the dependent variable baseline scores as a covariate plus all other level 1 covariates. All analyses used an intent-to-treat approach. We tested changes due to the intervention using models involving baseline and baseline X intervention status interactions to determine if the intervention had differential impact as a function of a child’s baseline level of problems. We also tested for interactions of intervention status by grade level and sex. For teachers’ ratings, we compared the intervention changes from baseline using an adjusted version of Cohen’s d effect sizes (ESs; Rosenthal 1994) in models that included other level-1 covariates. We also tested for Level 2 (classroom) by intervention interactions in two ways, first using the classroom mean levels on each baseline variable and second using the number of screened-positive children in each class as indicators of level of classroom problems. We hypothesized that the intervention impact may be greater for children in classrooms with lower levels of problems.

A common approach for analyzing frequencies of count data, such as disciplinary referrals, is the Poisson log-linear regression model. However, examination of the data distribution revealed that there was an excessive number of zeros that was above and beyond the expected frequency under the Poisson law, a phenomenon known as structural zeros (Cheung 2002; Kowalski and Tu 2007). Within our context, the percent of structural zeros represent the subgroup that was not at risk for the count outcome of interest such as disciplinary referrals. Because forcing a Poisson model to fit such data would bias the expected number of counts and mask the non-risk subgroup, we employed an alternative approach to account for the presence of structural zeros in the analysis.

An appropriate model for count responses in the presence of structural zeros is the zero-inflated Poisson (ZIP) regression (Lambert 1992; Kowalski and Tu 2007). A ZIP model is a special mixture model having two components: one component fits the regular Poisson regression; the other models the occurrence of the structural zeros using logistic regression. Under the ZIP model two sets of estimates are provided for assessing the contribution of a predictor. Estimates under the Poisson model assess its role in predicting the mean frequency of the response and those from the logistic part evaluate the effect of the predictor on the occurrence of structural zeros, or the non-risk subgroup. For example, in ZIP modeling if the relationship between a predictor and disciplinary referrals shows the predictor is significant in the logistic part of the model and non-significant in the Poisson regression, this variable only predicts the likelihood of being in the non-risk subgroup, not on the frequency of such events; if a predictor is significant only in the Poisson model, it only predicts the mean frequency of the disciplinary referrals not the propensity to be in the non-risk subgroup.

To determine if the intervention impact varied systematically for children presenting with different patterns of problems, we performed cluster analysis on baseline teacher ratings by using the agglomerative hierarchical clustering algorithm. This non-parametric approach produces partitions of the sample by a series of successive fusions of the individuals into groups, with the data ultimately reduced to a single cluster containing all individuals. The ‘optimal’ number of clusters is chosen by a close inspection of the two-dimensional diagram known as a dendogram, which illustrates the fusions and local maximum R-square criteria. We further took into account the distribution of sample size across the clusters to ensure that all clusters had reasonably large sample sizes. These considerations led to the selection of a three cluster solution. We repeated our primary analyses using three-level linear mixed-effects models (LMM), adding cluster group as a main effect and a condition by cluster group interaction, the latter used to test if the intervention impact differed by a child’s cluster membership.

All analyses were conducted using SAS version 9.1 and MPlus5.

Results

Rates of Attrition and Baseline Comparisons by Condition

Rates of attrition were equivalent in the two groups of randomly assigned children. Follow-up participation was 84.7% and 88.5%, respectively, for children in intervention and control groups. Attrition was not associated with any baseline measure nor differentially predicted by condition as a function of any baseline variable, i.e., no condition X baseline measure interactions. Children in the intervention condition received an average of 12.2 lessons with interventionists (range 7–14).

Children in the intervention and control groups were comparable on baseline measures with the exception of Behavior Control rated by teachers. Specifically, teachers rated children in the intervention group as lower on Behavior Control (M=20.02, SD=7.11) compared to controls (M=22.25, SD=7.77) (t=2.26, p<0.03). In the 3-month pre-intervention interval, 30.6% of children in the intervention condition (34/111) had one or more office disciplinary referrals (range: 1–10) and 25.2% of controls (29/115) had one or more referrals (range: 1–9). The likelihood of having any disciplinary referral was comparable in the two groups (OR: 1.29; 95% CI: 0.72, 2.32, p=0.40). The mean number of referrals made by teachers was 1.88; 61% of teachers (36/59) made a referral, with 83% of those referring making 1–3 referrals. In the pre-intervention interval, 4.5% of children in the intervention group (5/111) were suspended versus 2.6% of controls (3/115), which was also a comparable suspension rate by intervention condition (OR: 1.87; 95% CI: 0.33, 10.48, p=0.48). The eight children receiving suspensions were in 7 different classrooms.

Intervention Impact

Children who received the intervention showed improved functioning in all domains of behavior rated by teachers. Table 2 summarizes adjusted means and results of intervention impact from the multi-level models. The average intervention effects (see Table 2, column 3) were medium-sized and positive for Task Orientation (ES=0.33), Behavior Control (ES=0.31), Assertive vs. Withdrawn Behaviors (ES=0.37), and Peer Social Skills (ES=0.47).

Table 2.

Classroom Behavior Rated by Teachers at 4-month Follow-up by Condition: Intention to Treat Analyses of Intervention Impact

Variable	Control (n=115)		Intervention (n=111)		Intervention impact			ICC classroom
	M	SD	M	SD	Effect size (ES)
					ES	95% CI	p
Task orientation	20.69	8.39	22.68	8.09	0.33	0.03, 0.67	0.01	0.07
Behavior control	21.64	7.73	23.41	8.04	0.31	0.05, 0.67	0.02	0.13
Assertive-withdrawn	24.68	7.43	26.61	6.07	0.37	0.03, 0.71	0.01	<0.001
Peer social skills	23.61	7.49	26.35	7.99	0.47	0.12, 0.83	<0.001	<0.01

Effect size is the difference in means for intervention and control groups after adjusting for a linear effect of baseline, divided by the total standard deviations; ICC intraclass correlation

Children receiving intervention also had reduced office disciplinary referrals and suspensions during the 4-month intervention period (see Table 3). For referrals, there was no significant predictor in the logistic part of the model, indicating that the intervention did not have an effect on the propensity to have any referrals. However, the Poisson part of the model indicated that the mean frequency of this count outcome was predicted by intervention condition and baseline disciplinary incidents. Using the parameter estimates, we can calculate the mean frequency of referrals in the intervention relative to the control condition by exponentiating the estimate. In this case, the ratio of the mean office disciplinary referrals of the intervention to that of the control condition is exp(−0.619)=0.54. Thus, children receiving the intervention had a 46% decrease in mean office disciplinary referrals as compared to controls. In the intervention interval, 25% of children receiving intervention had one or more new office referrals (range 1–5) compared to 32% of controls who had one or more new incidents (range 1–24). Sixty-six percent of teachers (39/59) made one or more referrals, with 86% of those referring making 1–3 referrals. For suspensions, the logistic part of the model indicates the intervention predicted the likelihood of no suspension at the follow-up.

Table 3.

Predictors of Disciplinary and Suspension Events from Zero Inflated Poisson Regression Models

Parameters	Coefficient	Standard error	z value	p value
Disciplinary Referrals
Logistic Regression Part
Disciplinary Incidents to baseline	−0.93	0.94	−0.99	0.32
Condition (Intervention=1)	0.37	0.36	1.04	0.30
Gender (Male=1)	0.28	0.35	0.80	0.42
Poisson Regression Part
Disciplinary Incidents to baseline	0.29	0.06	4.65	<0.001
Condition (Intervention=1)	−0.62	0.30	−2.04	0.04
Gender (Male=1)	0.09	0.25	0.35	0.73
Suspensions
Logistic Regression Part
Suspension Events to baseline	0.17	0.33	0.50	0.62
Condition (Intervention=1)	−2.79	1.45	−1.93	0.05
Poisson Regression Part
Suspension Events to baseline	0.75	0.25	3.02	0.002
Condition (Intervention=1)	−0.57	0.23	−2.48	0.013

The Poisson part of the model indicates that the mean frequency of suspension events was also predicted by treatment condition and baseline suspension events. Using the parameter estimates, since exp(−0.565)=0.57, children in the intervention condition had 43% decrease in mean suspension events as compared to controls. In the 4-month interval after the intervention began, 1.8% of children in the intervention condition were suspended (2/111; range=1) compared to 6.1% of controls (7/115; range=1–5). All nine children receiving suspensions had different teachers.

Intervention impact was not different on any outcome by the year a child participated, which we tested by re-running each model including year as an independent variable.

We extended our analyses of intervention impact on disciplinary incidents using logistic regression to estimate the number of children who would need intervention to reduce the occurrence of a new office disciplinary referral (i.e., number needed to treat, NNT). The logistic regression showed that children receiving intervention were about one-half as likely to have any discipline incident (OR=0.48; 95 CI: 0.23, 0.98), p=0.04); overall, 14 children would need to receive intervention (NNT) in order to reduce one child receiving an office disciplinary referral during the 4-month intervention interval. The logistic regression model for suspensions was directionally similar but non-significant (OR=0.27; 95 CI: 0.05, 1.54), p=0.14); overall, 22 children would need to receive intervention to reduce one child receiving a suspension.

Moderators of Intervention Impact

Neither a child’s baseline level of problems or grade was associated with differential intervention impact. Sex was associated with differential intervention impact for Peer Social Skills, as indicated by a significant Sex X Condition interaction for that variable using multilevel modeling (β= −4.23, SE=1.59, p=0.0087). We estimated our multi-level model testing intervention impact on Peer Social Skills separately for girls and boys; the intervention improved peer social skills for girls but not for boys. For girls, the intervention had a large positive effect on peer social skills (ES=0.90; 95% CI: 0.30, 1.49, p<0.001). For girls, post-intervention peer social skills means were as follows for intervention (M=27.27, SD=8.24) and controls (M=22.61, SD=6.90). In contrast, for boys post-intervention peer social skill means were as follows for the intervention group (M= 25.72, SD=7.77) and controls (M=24.76, SD=7.84).

Classroom-level variation on children’s Behavior Control and Task Orientation was moderate (see intraclass correlations in column 4 of Table 2). Those ICCs were larger for Behavior Control (0.13) than Task Orientation (0.07). In models that tested for Level 2 (classroom-level) by intervention condition interactions, we found no evidence of a differential intervention impact associated with either mean classroom levels of child problems or with the number of children in a classroom who screened positive for the intervention.

We extended our testing of intervention moderators by examining patterns of baseline classroom problems. A 3-cluster solution was optimal to describe the pattern of classroom problems at baseline. Cluster group 1 was largest, comprised of 146 children (64.6% of the sample); groups 2 and 3 had 51 (22.6%) and 28 (12.4%) children, respectively. Overall, children in group 2 had fewest problems. The three groups had similar levels of behavior control problems and more substantial differences in other domains. Mean baseline Behavior Control scores were as follows: group 1 (M=20.58, SD=7.33), group 3 (M=20.25, SD=6.34), and group 2 (M=23.20, SD=8.07). Children in group 3 were characterized by high levels of problems related to academic functioning. Mean baseline Task Orientation scores were: group 3 (M=16.68, SD=4.21), group 1 (M=18.72, SD=7.23), and group 2 (M=24.47, SD=8.23). Children in groups 3 and 1 had highest peer problems. Mean Peer Social Skills scores were: group 3 (M=17.75, SD=5.71), group 1 (M=22.07, SD=4.92), and group 2 (M=34.33, SD=3.34). Group 1 had highest withdrawn behaviors. Mean Assertive vs. Withdrawn scores were: group 1 (M=21.92, SD=5.91), group 2 (M=31.14, SD=5.21), and group 3 (M=30.25, SD=4.76). We estimated multi-models entering cluster group and cluster group by intervention condition interactions and found no evidence of differential impact of the intervention as a function of a child’s cluster group membership. Nor was there any moderating effect of parent participation in the single intervention visit.

Discussion

We tested the impact of the school-based Rochester Resilience Project designed to strengthen emotional self-regulation skills in the context of assisting children in addressing individualized goals. The intervention targeted children with elevated aggressive-disruptive and social-emotional problems identified through a population-based screening of all kindergarten – 3rd graders in two urban schools. In 14 lessons, children were taught a hierarchical set of skills: monitoring emotions; self-control and reducing escalation of emotions; and maintaining control and regaining equilibrium. The implementation model and sequence of adult-child interactions was designed to assist children in developing proceduralized skills (Sun 1997) that can be used with increasing flexibility in situations of emotional activation.

The intervention had a positive impact on children’s classroom behaviors and rates of disciplinary incidents, including improved behavior control (i.e., fewer aggressive-disruptive problems), on-task learning behaviors and peer social skills, and less shy-withdrawn and more assertive behaviors. The reductions in classroom problems reported by teachers were medium-sized, with average standardized effect sizes ranging from 0.31–0.47. On peer social skills, the intervention impact varied by sex. For girls, the intervention had a large effect on improved peer social skills (standardized ES=0.90), whereas social skills did not change for boys. The intervention also reduced rates of office disciplinary referrals and suspensions. Relative to controls, children receiving the intervention had a 46% mean decrease in office disciplinary referrals and a 43% decrease in mean suspensions during the 4-month intervention period. On disciplinary referrals, the intervention impact was primarily to reduce the frequency of those events rather than the propensity to have any referrals. For out-of-school suspensions, the intervention reduced both the propensity to have a suspension and the frequency. Specifically, whereas 1.8% (2/111) of children in the intervention condition received a suspension up to a maximum of one time, 6.1% of controls (7/115) were suspended, with five suspensions being the maximum number.

Rather than targeting a single problem area, our intervention targeted children with emerging problems characteristic of externalizing behavior patterns, shy-withdrawn behaviors characteristic of internalizing problems, as well as problems such as ‘off-task’ behaviors related to classroom academic functioning. Conceptually, this focus stems from our intervention model that teaches a common set of skills to enhance emotional self-control and reduce reactivity that are then individualized to address specific goals. Pragmatically, this addresses observations that aggressive-disruptive, internalizing and learning problems frequently co-occur in young children (e.g., Wyman et al. 2009) and evidence that problems across multiple domains have an additive risk effect (Kellam et al. 1982). Overall, our findings suggest that the intervention reduced problems across the targeted domains, irrespective of different patterns of classroom problems prior to the intervention. The intervention also reduced the escalation of formal disciplinary incidents and suspensions, which are associated over time with lower academic achievement and school bonding and with increased conduct problems (Bryant et al. 2000; Huisinga and Jakob-Chien 1998). Evidence that the intervention reduced serious behavior problems was strengthened by the reduction in suspensions, in addition to reducing more frequently occurring office referrals. Suspensions were initiated by principals and required a formal hearing, and the school district’s policy required that suspensions only occur in response to a few specific infractions, including hitting a teacher or carrying a weapon. Such specificity increases the validity of those records (Irvin et al. 2004).

Older child age was not associated with enhanced benefit from this intervention, contrary to our hypothesis. We expected that older children would be better equipped to learn and apply new emotion self-regulation skills due to their more advanced language development and capacity for planning and self-directed behavior. However, the present study leaves unanswered several questions regarding the relationship between age, cognitive maturation, and children’s adoption and maintenance of skills. First, this study did not ascertain the extent to which children employed skills they learned in the intervention or maintained gains beyond the 4-month intervention. During the intervention period, children received regular support from a school-based interventionist. Congruent with the developmental concept of scaffolding, we expect that children’s capacity to use new strategies for emotion self-regulation in an increasingly independent fashion beyond the intervention period will be more dependent on cognitive maturity and executive function skills such as shifting attention, response inhibition, and planning (Pennington and Ozonoff 1996). Thus, cognitive maturity may be more important for ongoing maintenance and transfer of skills.

A number of questions about the mechanisms whereby this intervention enhanced children’s adaptation can be addressed in future studies. Children learned cognitive and behavioral skills to assist them in monitoring emotions, decreasing emotional reactivity and increasing self-calming, which, according to our intervention model, were posited to reduce individualized, targeted problems. However, cognitive and behavioral skill training may also directly strengthen executive-functioning skills related to academic performance and functioning (Riggs et al. 2006). Teachers were given information about the skills children learned. Although not specifically trained to teach or ‘coach’ children to use those skills, teachers may have learned new strategies that improved their classroom management and interaction with children. Clarifying the extent to which the gains in children’s adaptation observed in this study were due to children’s adoption of new emotional self-regulation strategies, to enhanced teacher competence with children, or both due to transactional changes in teacher-child interactions, are questions for future research. In addition, few studies have evaluated different components of children’s knowledge and attitudes about skills they learn through interventions, including declarative knowledge, efficacy to use skills, or the extent to which new skills become proceduralized. Developing measures to assess those domains of skill knowledge and testing their impact on mediating improved functioning associated with interventions should be an important priority to clarify and enhance conceptual models underlying skill-based interventions. Further investigation of differences in intervention impact for girls and boys is also warranted. We found that girls benefited more than boys in terms of improved peer social skills, and the reasons are unknown. We note that all Mentors were female. It is possible that congruence of child-Mentor pairs on sex and other characteristics may influence the extent to which children perceive mentors as valid models for assisting them with social skills. How the fit between children and their interventionists on sex and other characteristics influences learning and transfer of cognitive and behavioral skills is an intriguing issue that warrants future study.

The high rate of children’s participation in this intervention trial underscores a strength of school-based programs, which is the ability to provide an accessible intervention to many children who otherwise might not receive services. Currently, approximately one in eight children with an emotional or behavioral disorder ever receive treatment in the mental health system (O’Connell et al. 2009); in family-based intervention trials, often a large proportion of those eligible never participate in that intervention (Braver and Smith 1996). In comparison, in the present school-based program that used interventionists employed by and based in schools, most eligible children participated. Over two years, 285 children out of the population of kindergarten—3rd grade children ‘screened positive’ for elevated classroom problems, 261 enrolled in the intervention, and 226 (or 79% of all eligible) completed the four-month follow-up. Among 111 children in the intervention arm, the average number of intervention lessons completed was 12.2 out of 14. In addition to effectiveness, the reach or proportion of the population that participates in or receives an intervention has a major effect on the potential population-level impact of that intervention (Glasgow et al. 2004). This study demonstrates the potential for the Rochester Resilience Project model to reach large numbers of low income minority children who have low access to mental health services. Determining the efficacy and effectiveness of this intervention under different implementation conditions are also important questions for future research.

The implementation model for this intervention was designed to assist children in adopting skills that they can apply flexibly and appropriately in contexts that present challenges to them. An important aspect of this model is providing opportunities for children to practice skills in ‘hot cognition’ contexts (Damasio 1994) to promote transfer of new learning, which also requires reinforcement by adults. To further extend and strengthen the transfer of new learning, we suggest several areas for further investigation. First, testing the impact of additional training for teachers in using the intervention skills and addressing specific challenges in supporting children’s use of skills can clarify the role of teachers and other adults in promoting children’s maintenance of skills. Second, additional active cueing or ‘coaching’ of children by school-based interventionists to use skills in different settings and contexts of the school (e.g., hallway transitions) can determine the amounts of in-vivo practice required to assist children in adopting new strategies. The role of parents in reinforcing children’s use of skills should also be investigated; parent involvement and use of skills with their child may be essential to promote skill transfer and maintenance over time.

Several limitations of this study should be noted. Teachers, who were the primary source of information about children’s functioning, were aware of the time-period in which children were receiving intervention or served as controls. Although school records for disciplinary referrals were another source of information, teachers’ knowledge of children’s status could have influenced their propensity to refer children to school administrators to address behavior concerns. Thus, teacher expectations cannot be ruled out as a source of bias. This study’s reliance on teacher ratings also limited the potential to capture children’s internalizing problems; in general, concordance is low between children’s self-report of distress and reports from adults, including teachers (Wyman et al. 2009). In addition, this wait-listed study did not provide evidence about intervention impact after the intervention period. Assessing the longer-term impact of children’s participation on their functioning in school and in other settings, including peer groups, is needed to determine the potential for the school-based Rochester Resilience Project model to strengthen adaptation among young urban children in adversity.