Preventing Internalizing Problems in 6–8 Year Old Children: A Universal School-Based Program

Abstract

The Aussie Optimism Program: Feelings and Friends (AOP-FF) is a 10 week, universal mental health promotion program based on social/emotional and cognitive and behavioral strategies. The aim of the current study was to evaluate the efficacy of a universal Cognitive Behavioral Therapy based program in preventing and reducing internalizing problems in 6–8 year olds (Years 1–3 in Australia). Year 1–3 students from a low SES primary school (N = 206) were randomly assigned in classes to either an intervention or a control group and assessed at baseline and post-test. Results showed a significant (p = 0.009) and small to moderate (partial eta-squared = 0.034) pre-post decrease in parent-reported anxiety symptoms for the intervention group, in conjunction with a non-significant (p = 0.708) and negligible (partial eta-squared = 0.001) pre-post increase for the control group. A larger randomized controlled trial assessing longer term effects is needed. In addition the program needs to be simplified for year 1–2 students with a separate more developmentally appropriate program for year 3 students.

Introduction

Prevention of internalizing disorders in early childhood

The two internalizing disorders, anxiety and depression, are recognized as the most common and increasingly prevalent childhood disorders (Tonmyr et al., 2011; Essau et al., 2012a). Anxiety and depression have been posited to be temporally linked (Hansell et al., 2012), and some researchers have argued that the high comorbidity and symptoms overlap indicate that the two internalizing disorders are part of a single entity or personality construct (Watson et al., 2005). Anxiety and depression are evident in early childhood and are often associated with significant daily impairment and interference with a child's interpersonal and academic functioning (Roberts and Bishop, 2003; Hirshfeld-Becker et al., 2010; Trudeau et al., 2012). While clinical trials of anxiety and depression intervention programs indicate that internalizing disorders in late childhood and early adolescence may be effectively treated (Monshouwer et al., 2012), a significant proportion of children continue to experience difficulties post-intervention (Pössel et al., 2011). As a consequence, it is critical that the emphasis is on prevention in dealing with internalizing disorders in children, and for prevention to begin sooner rather than later.

The transition to primary school has been identified as a vulnerable stage for many children (Tomb and Hunter, 2004; Goodwin et al., 2012) due to several factors such as spending a significant period away from their families for the first time; entering a new social environment where the need for social approval and to fit in with peers becomes significant; and being evaluated in terms of how well they perform compared to their peers (McLoone et al., 2006; Goodwin et al., 2012). As such, manifestations of social, separation and performance anxiety may emerge when children enter primary school.

Risk and protective factors for internalizing disorders in childhood

Prevention programs can work by either reducing the influence of risk factors, or by enhancing and developing protective factors in order to promote resiliency in children (Dadds et al., 2000; Romeo et al., 2011). As with risk factors, protective factors have generally pertained to individual child characteristics (e.g., temperament, cognitive style, social skills, self-efficacy, and esteem) or environmental and learning experiences such as effective parenting, high family cohesion, positive parent-child relationships (Davis et al., 2000; Dick-Niederhauser and Silverman, 2004; Jakobsen et al., 2012). The presence of positive social support networks have been found to be an important protective factor against the development of emotional disorders, particularly in the face of stressful life events (Heaney et al., 2010; Grav et al., 2012; Tsai et al., 2012). Another important protective factor is a child's coping skills repertoire (Wright et al., 2010), with problem focused coping strategies being associated with greater mastery and positive psychological adjustment than avoidant or emotion focused coping skills (Dick-Niederhauser and Silverman, 2004; Klein et al., 2011). The role of an avoidant coping style in the maintenance of internalizing disorders, particularly anxiety, is widely recognized (Barlow, 2002; Eisenberg et al., 2012). For instance, in the avoidance learning model, it is hypothesized that fears or phobias will fail to extinguish if one learns to avoid the feared stimulus (Dymond et al., 2011). McLoone et al. (2006) argued that to effectively treat children with comorbid anxiety and depression, components of treatment should include enactive programming (e.g., proactive engagement in therapy via exposure and pleasant activity scheduling).

It has been recommended that building up social and emotional competencies should begin in preschool (Greenberg et al., 2003; Conner et al., 2011). Emotional competence includes a child's capacity to recognize, monitor and regulate their emotions, appreciate the perspectives of others, establish prosocial relationships and goals, as well as effective interpersonal skills and problem solving abilities (Hessler and Katz, 2010). Emotional awareness and knowledge in young children have been found to be predictive of social competence and emotional well-being in later childhood (Fine et al., 2003; Oades-Sese et al., 2011). Moreover, emotional competence has been linked with positive school and classroom adjustment, enhanced academic learning, and found to be predictive of psychological health in later childhood (Linaries et al., 2005; Miller et al., 2006).

Social competence is a significant prerequisite in the formation of positive, stable, and enduring peer relationships (Gagnon and Nagle, 2004). Furthermore, peer relationships have been implicated as mediating and protective factors in psychological adjustment, as well as encouraging more positive school transitions, better school adjustment, increased achievement and school liking, decreased school avoidance, protection against victimization in school, and more healthy adjustments to negative life experiences (Bukowski and Adams, 2005; Masten, 2005; Kochel et al., 2012).

Preventive interventions in the field

Prevention programs can be either universal, where the program is implemented for a designated population regardless of risk, selected, where the program is targeted to a population at risk, or indicated, where the program is aimed at individuals displaying early symptoms of the disorder (Dick-Niederhauser and Silverman, 2004). Advantages of universal programs include their ability to avoid the stigma of labeling that is inherent in selective and indicated interventions, avoiding the potential to overlook “at risk” children due to inadequate screening measures, as well as the ability to provide the opportunity for all children to reap the benefits of the program and develop resiliency (Donovan and Spence, 2000; Kösters et al., 2012).

The majority of prevention programs for internalizing disorders have targeted older children (e.g., middle or later primary years; Barrett and Turner, 2001; Lowry-Webster et al., 2001; Quayle et al., 2001; Roberts et al., 2003, 2004; Rooney et al., 2006, 2013a,b; Fox et al., 2012; Johnstone et al., 2014; Myles-Pallister et al., 2014). There are comparatively few prevention programs specifically targeting internalizing disorders in younger children. One such program is the Queensland Early Intervention and Prevention of Anxiety Project (QEIPAP; Dadds et al., 1997) which is adapted from the Coping Koala prevention program (Barrett et al., 1994) and employs primarily, cognitive-behavioral approaches. Dadds and his colleagues evaluated the efficacy of the program in a sample of 7–14 year olds, who were randomly assigned to a control or 10-week school based child and parent focused psychosocial intervention group. While both groups demonstrated improvements in anxiety symptomology post-intervention, these gains were maintained at 6 month and 2 year follow-up only in the intervention group (Dadds et al., 1997, 1999).

The PATHS (Promoting Alternative Thinking Strategies) curriculum is a widely known program designed to promote social and emotional competence through cognitive skills building in Grades 1–3 children (Kam et al., 2004; Saltali and Deniz, 2010). In evaluating PATHS, it was found that compared to controls, intervention children showed improvements on social problem-solving, emotional understanding, and teacher ratings of internalizing and externalizing problems (Kam et al., 2004; Saltali and Deniz, 2010). However, PATHS doesn't target internalizing problems and consequently the effect sizes found for internalizing problems have been 0.22 in the low range. PATHS is also designed as a multiyear program, having the additional drawback of being potentially cost ineffective due to the resources needed to run the program over a lengthy period of time.

Supporting the findings of PATHS, evaluations of a similar program, the I Can Problem Solve program (Anliak and Sahin, 2010) was found to significantly improve cognitive problem solving abilities, and reduce impulsivity and inhibition, with enduring effects at 1 year follow up (Shure, 1997; Anliak and Sahin, 2010). Support was also found for the Incredible Years Classroom Social Skills and Problem Solving Curriculum (Child Dinosaur Program; Webster-Stratton and Reid, 2004), which also looks at strengthening social and emotional competence in young children. However, all effects for internalizing problems have been in the low to moderate effect size range (Durlak et al., 2011).

While existing programs designed for younger children have targeted social and emotional competence, a program specifically targeting the internalizing disorders anxiety and depression does not presently exist and effect sizes of existing established programs on internalizing problems have been low to moderate (Durlak et al., 2011). A comprehensive and cost effective program, targeting internalizing problems, to be implemented widely throughout the school system, needs to be developed and evaluated. Designed to meet the developmental needs of 6–8 year old children, this program should combine evidence-based cognitive behavioral strategies, while simultaneously having a strong focus on building social and emotional competence.

Rationale for the current study

The aim of the current study was to evaluate the efficacy of a universal Cognitive Behavioral Therapy based program in preventing and reducing internalizing problems in 6–8 year olds (Years 1–3 in Australia).

Areas to be targeted

Based on the evidence reviewed, a focus of the program was to promote children's emotional knowledge, awareness, and emotional regulation. A further component included was improving prosocial friendship skills and effective social problem-solving skills, as well as to encourage perspective-taking. These skills are designed to minimize interpersonal conflict, decrease rejection and negative appraisals from peers, and enhance status among peers, thereby increasing children's positive support network (Spence, 2003; Glick and Rose, 2011). In addition, the role of effective coping skills is important in promoting resiliency, self-efficacy and a sense of mastery. Rodebaugh et al. (2004) have argued that to address depressive and anxiety disorders from a cognitive-behavioral theoretical perspective, behavioral strategies such as enactive programming (e.g., assisted graded exposure, providing strategies to face the feared stimulus) should be included. The inclusion of enactive programming maximizes chances of addressing both types of internalizing disorders. As such, the emphasis of the program is also to build upon existing programs by providing children with a repertoire of effective coping skills that are directly linked to anxiety and depression. These include relaxation training, pleasant activity scheduling, facing one's fear instead of avoidance, and seeking social support when needed. No study to date has included behavioral strategies in combination with building social and emotional competence. In this way, the current program targeted internalizing disorders more specifically than any other program to date.

Universal school-based approach

The current program employed a universal approach to prevention, and was available to all children in the school rather than limited to those “at risk.” As universal programs are offered to all children, it is logical for the program to be part of the school curriculum. Previous researchers have found that preventive programs that are incorporated into the existing school health curriculum increase their cost-effectiveness, utility, and accessibility (Calear and Christensen, 2010; Kösters et al., 2012).

In sum, the current program adhered to several important factors salient to preventive programs. It:

1. Encompassed a developmental framework,

2. Integrated a variety of successful, evidence-based approaches,

3. Incorporated existing risk and resiliency literature, with a dual focus on social and emotional competence, and on developing an effective repertoire of coping skills salient to internalizing difficulties,

4. Employed a universal approach, utilizing the school environment as a place where change can occur.

Hypotheses

It is hypothesized that:

• H1a Universal Prevention Effects: At post-test, children in the intervention group will have lower depression and anxiety scores on self and parent reports, improved social skills and more well-developed emotion knowledge than children in the control group.

• H1b Universal Prevention Effects: At post-test, children in the intervention group who score in the normal range at pre-test will maintain a healthy status (i.e., parental and self-report measures of internalizing symptoms within the normal range).

• H2 Selected or Indicated Prevention Effects: Children in the intervention group who were initially identified as being “at risk” (i.e., who exhibit symptoms of internalizing difficulties at pre-test) will exhibit a significant decline in internalizing difficulties at post-test.

Methods

Ethics statement

Informed consent was obtained from both next of kin and the children in written form. The child consent form contains information that the child can understand and has their signature of consent on it and the case is the same for the parents. This study has been approved by the Curtin University Human Research Ethics Committee. If needed, verification of approval can be obtained either by writing to the Curtin University Human Research Ethics Committee, c/- Office of Research and Development, Curtin University of Technology, GPO Box U1987, Perth, 6845 or by telephoning 9266 2784.

Participants

Participants were 206 Years 1–3 students in one large Primary school in the Perth Metropolitan area. Figure 1 illustrates progress through the phases of randomized control.

Figure 1.

Flow diagram of the progress through the phases of the randomized control trail.

The double stream Primary School came from a very low socioeconomic (SES) status area in the Perth Metropolitan area and was placed in the 10th lowest decile of the H-Index which is used to rank SES in schools in Western Australia. The sample consisted of 85 boys and 107 girls (14 students did not give their gender), with approximately equal numbers of males and females across the conditions [intervention: boys (46.1%), girls (53.9 %); control: boys (42.2%), girls (57.8%)]. Participants were aged between 6 years 5 months, and 9 years 6 months.

To evaluate the pilot program for each year group, the 10 Year 1–3 classes were blocked into year groups. From within these three blocks, the classes were then randomly allocated to the intervention and usual care control conditions. The numbers of students in each year group in both intervention and control conditions are presented in Table 1. Students in the intervention group received the preventive program, whilst the control students received their regular health education lessons.

Table 1.

Number of students in each year and group.

	Year 1	Year 2	Year 3
Intervention group	28	27	51
Control group	42	39	19

Measures

Self-report measures

The Children's Depression Inventory (CDI; Kovacs, 1992) is a 27-item self-report measure that assesses cognitive, affective and behavioral symptoms of depression in children aged 7–17. Two-week test-retest reliability and internal consistency coefficients have been reported to be above 0.80 (Osman et al., 2010) with concurrent validity coefficients ranging from 0.66 to 0.81 (Kovacs, 1992). Item 9 (relating to suicidal ideation) was removed in accordance with the WA Department of Education's standards, resulting in a 26-item questionnaire. Other researchers who have similarly removed this item found internal consistency remained high (Cronbach's alpha = 0.89; Cole and Jordan, 1995; Cole et al., 2011).

The Spence Children's Anxiety Scale (SCAS; Spence, 1998; Essau et al., 2012b; Zhao et al., 2012) is a child self-report measure of symptoms relating to six subsets of anxiety disorders: social phobia, separation anxiety, obsessive-compulsive disorder, panic-agoraphobia, generalized anxiety and fears of physical injury in children aged 8 and above. The SCAS shows high internal consistency for the total scale as well as for each subscale, with coefficient alphas ranging from 0.60 to 0.92 (Spence, 1998). Test-retest reliability is moderate (0.63 after 3 months, and 0.60 after 6 months; (Spence, 1998)). Convergent validity with the Revised Children's Manifest Anxiety Scale (RCMAS) is acceptable (r = 0.71).

The Assessment of Children's Emotional Skills (ACES, Schultz et al., 2004) was used to assess children's emotional knowledge. The ACES is designed to measure emotion attribution accuracy in children aged 6 and older, and consists of three subscales concerning social behaviors, social situations, and facial expressions. Only the social behaviors and situations subscales were used in this study. Each subscale contains 15 items, in which children are asked to label the protagonist's feelings as “happy, sad, mad, scared, or no feeling.” Cronbach's alpha for the all the items across the three sections—social behaviors, situations, and facial expressions—is moderate (α = 0.68).

Parent measures

The Parent version of Spence Children's Anxiety Scale (SCAS-P; Spence, 1998) is designed to evaluate parents' perceptions of their child's level of anxiety symptoms. Satisfactory to excellent reliability and validity coefficients have been reported for all subscales, for both child and parent versions (Nauta et al., 2003). The SCAS-P was found to discriminate well between anxious and community children, and between the different anxiety disorders. Satisfactory to high internal consistency was found for the total scale (Cronbach's alpha = 0.89), and the six subscales (ranging from 0.55 to 0.82). Child reports were found to correlate moderately well with parent reports for internalizing symptoms (r = 0.66). The correlation between SCAS-P and CBCL-internalizing (r = 0.27) is significantly higher than the correlation between SCAS-P and CBCL-externalizing (r = 0.12; z = 3.15, p < 0.001), providing evidence for both divergent and convergent validity.

The parent form of the Strengths and Difficulties Questionnaire (SDQ-P; McCrory and Layte, 2012) consists of 25 items assessing conduct problems, hyperactivity, emotional symptoms, peer problems, and prosocial behavior. Goodman (2001) has reported sound Cronbach alpha coefficients for the sub-scales (mean = 0.73). Numerous studies have found the SDQ-P to have good concurrent and discriminant validity in non-clinical samples (Goodman, 2001; Muris et al., 2003; McCrory and Layte, 2012). The present study used the SDQ-P total difficulties (the sum of the conduct problems, hyperactivity, emotional symptoms, and peer problems scales) and SDQ-P prosocial scale.

Teacher measures

A 3-item scale teacher-rated Social Competence (SC-T) scale was constructed based on three pertinent domains of social competence, which were also the three aspects targeted by the program: social skills (e.g., verbal and nonverbal skills such as listening and turn taking), empathy, and social problem-solving. Teachers rated each child on a 5-point Likert-type scale (significantly below average to significantly above average). Test-retest reliability for the control group was high (r = 0.82). High internal consistency was also noted (Cronbach's alpha = 0.92 at pre-test and 0.95 at post-test). The SC-T was found to be significantly correlated with the prosocial scale of the SDQ-P (r = 0.25, p < 0.05).

Feeling and friends program: manual production and program content

A literature review and Delphi analysis isolated the key components that should be targeted in the manual. The manual was written by the researcher and an educational specialist, who was an experienced Year 1–3 teacher. Contributions and editing were also provided by three experienced clinical psychologists/lecturers, two of whom developed the existing Aussie Optimism Programs. The Feelings and Friends program comprised 10 modules and was designed to support the Western Australian Health and Physical Education Curriculum, and to be implemented by class teachers to whole classes. The Feelings and Friends program was postulated to have its effect by targeting salient risk and protective factors for internalizing difficulties. Specifically, the program was designed to enhance social and emotional competence, in combination with equipping children with behavioral coping skills. Topics and brief content information of each module are detailed in Table 2.

Table 2.

Summary of feelings and friends modules.

Module	Topic	Aim of Module
1	My feelings	For students to increase their emotional knowledge via normalizing emotions, and identifying a range of feelings
2	Body clues & first aid for feelings	For students to appreciate physiological manifestations of various emotions
		For students learn emotion regulation skills, be introduced to the idea that there are strategies that can be used to cope with unpleasant feelings, and learn to self soothe
3	First aid for being scared and worried	For students to learn and practice strategies to deal with anxiety (e.g., relaxation, confronting fear, pleasant activity scheduling)
4	First aid for anger	For students to learn and practice strategies to deal with fear (e.g., self soothe, STOP!)
5	Other people's feelings	For students to be aware of other people's feelings (e.g., by listening and looking for external body clues)
6	Caring about people's feelings	To foster empathy, and encourage respect and concern for other people's emotions
		For students to learn that different people may have different feelings in the same situation
7	Friendly habits	To reinforce prosocial behavior as well as identify and discourage anti-social behavior, particularly by fostering perspective taking and exercising empathy
8	Solving problems	For students to learn the process of problem-solving
9	Mad, sad & glad solutions	For student to be able to think about consequences of solutions in terms of their costs and benefits to the self and others
10	Quiz time!	Review knowledge, understandings, and skills developed throughout the program

Procedure

This pilot trial of the Aussie Optimism Feeling and Friends program was carried out in conjunction with the larger scale Aussie Optimism whole school intervention. The volunteer school was in the 10th lowest decile of the H-Index, a measure of socio-economic status according to the Western Australian Government Department of Education and Training (McMillan and Western, 2000). A 2-stage consent process was employed, whereby active consent was sought from all parents of Years 1–3 children, followed by a passive consent procedure. Ninety-five percent of parents actively or passively consented to their child's participation. Initial data were collected at the start of Term 3 (Time 1), approximately a week prior to the start of the intervention.

The CDI and SCAS (given their recommended age range) were used with the Year 3 children only because of the poor reliability and validity noted on self-report measures with very young children (e.g., Myers and Winters, 2002; Chrisman et al., 2006). A list of questionnaires administered to children of the different age groups is presented in Table 3.

Table 3.

Questionnaires administered to each year group.

SELF REPORT MEASURES
CDI, SCAS	Year 3 only
ACES	Year 1–3
PARENT REPORT MEASURES
SCAS-P, SDQ-P total difficulties, SDQ-P prosocial	Year 1–3
TEACHER REPORT MEASURE
SC-T	Year 1–3

Students were provided with brief information about the project, and verbal assent to participate was sought. Standardized instruction and individual questions were read aloud to small groups or a class, by trained personnel who facilitated the administration and collection of questionnaires. Students were screened for clinical levels of depressive and anxious symptomology and parents of children who had elevated scores were confidentially contacted via the school. Relevant information and referral options were discussed with parents of children at risk. This process was repeated upon completion of the program in Term 4 (Time 2).

Training was provided to the teachers who agreed to be part of the study, and included detailing the rationale for promoting mental health within the school curriculum, teaching the various components in the program via direct instruction and role-playing, as well as supervised practice with the materials. Training was provided by the researcher and two registered clinical psychologists.

Process evaluation and social validity

To ensure the integrity of the program, facilitators of the program were asked to self-monitor their implementation of the program by completing a checklist and rating of the content covered in each module. Independent integrity measures were obtained by random observation of two of the teachers administering two different modules. This allowed for the comparison of teacher and expert ratings of program implementation, which provided a measure of implementation fidelity. Social validity aspects of the Years 1–3 program were examined by asking the students to rate their enjoyment of the program and parents to rate the effectiveness and utility of the program. Small focus groups were also held with students and teachers regarding usefulness, enjoyment of the program, suggested improvements and implementation issues.

Data analysis

The psychometric data (ACES, CDI, SCAS, SCAS-P, SDQ-P total difficulties, SDQ-P pro-social, SC-T) were analyzed with a multi-level mixed effects linear regression model (Bryk and Raudenbush, 1987) as implemented through SPSS's Generalized Linear Mixed Models (GLMM: SPSS Version 19) procedure. The regression model was “mixed” in the sense that it included both random effects (child, school year) and fixed effects (group: intervention, control; time: pre-test, post-test). The regression model was “multi-level” in the sense that time was nested within child, and child was nested within school year. In order to optimize the likelihood of convergence, a separate GLMM analysis was run for each outcome. In each analysis, GLMM assumed a normal probability distribution for the outcome and linked it to the fixed effects (group, time, Group × Time) with an identity function. If the outcome did not have a normal distribution, then the parameter estimates of the covariance matrix were computed with robust statistics.

A clinical cut-off of 17 on the CDI has been used to indicate students who are at-risk of depression (Kovacs, 1992); and a clinical cut-off of 42 on the SCAS has been used to indicate students who are at-risk of anxiety (Kovacs, 1992). These cut-offs were used to classify children into healthy and at-risk groups at both pre-test and post-test. This generated four binary outcomes at post-test (post-test incidence of depression risk [yes/no]; post-test recovery from depression risk [yes/no]; post-test incidence of anxiety risk [yes/no]; post-test recovery from anxiety risk [yes/no]). The post-test binary outcomes were, once again, analyzed with a multi-level mixed effects regression model as implemented through SPSS's GLMM procedure. This time, however, GLMM used a binomial probability distribution for the outcomes and linked them to a single fixed effect (group) with a logit function. Once again, in order to optimize the likelihood of convergence, a separate GLMM analysis was run for each of the four binary outcomes.

Unlike repeated measures ANOVA (or ANCOVA), the multi-level mixed effects regression model does not rely on participants providing data at every assessment point; GLMM uses all the data present at each assessment point thereby reducing the impact of subject attrition on statistical power. Moreover, GLMM is relatively robust to unequal group sizes.

Results

There was no child attrition at post-test for either group; however, 18 intervention parents and 11 control parents dropped out at post-test yielding attrition rates of 29 and 21%, respectively. There were no differences between the intervention and control drop-outs in terms of their pre-test outcome measures. Descriptive statistics for intervention and control groups on all outcomes are reported in Table 4.

Table 4.

Pre-test and Post-test means (sds) for intervention and control groups.

Group and Measure	N	Pre-test	Post-test
INTERVENTION
ACES	101	15.44 (2.99)	17.34 (3.33)
CDI (Year 3 only)	51	11.41 (8.05)	10.15 (9.56)
SCAS (Year 3 only)	51	32.75 (19.21)	30.89 (19.60)
SCAS-P	62	15.94 (8.80)	13.20 (7.02)
SDQ-P total difficulties	60	10.71 (7.10)	10.08 (6.08)
SDQ-P prosocial	61	7.82 (1.71)	7.84 (1.90)
SC-T	65	9.15 (2.10)	9.14 (2.37)
CONTROL
ACES	84	14.39 (3.24)	15.45 (3.34)
CDI (Year 3 only)	15	12.62 (11.17)	11.06 (9.34)
SCAS (Year 3 only)	15	35.66 (18.61)	29.05 (17.40)
SCAS-P	53	14.48 (7.39)	14.61 (8.01)
SDQ-P total difficulties	53	9.88 (6.51)	9.80 (6.22)
SDQ-P prosocial	54	7.93 (1.73)	8.09 (1.72)
SC-T	61	8.87 (1.73)	8.44 (1.78)

Primary analyses: student-reported outcomes

ACES

The Group × Time interaction was non-significant [F_{(1, 375)} = 3.39, p = 0.067, partial eta-squared = 0.009]. The group main effect was also non-significant [F_{(1, 375)} = 3.04, p = 0.082, partial eta-squared = 0.008]. There was, however, a significant main effect of time [F_{(1, 375)} = 66.68, p < 0.001, partial eta-squared = 0.151] indicating a significant pre-post improvement in emotional skills for both groups.

CDI

The Group × Time interaction was non-significant [F_{(1, 127)} = 0.02, p = 0.886, partial eta-squared = 0.000], as were the main effects of group [F_{(1, 127)} = 0.52, p = 0.474, partial eta-squared = 0.004], and time [F_{(1, 127)} = 1.12, p = 0.292, partial eta-squared = 0.009].

SCAS

The Group × Time interaction was non-significant [F_{(1, 129)} = 0.41, p = 0.525, partial eta-squared = 0.003], as were the main effects of group [F_{(1, 129)} = 0.02, p = 0.898, partial eta-squared = 0.000], and time [F_{(1, 129)} = 1.22, p = 0.272, partial eta-squared = 0.009].

At-risk analysis

There was no significant difference between groups in the proportion of students who moved from pre-test healthy to post-test at-risk (depression: p = 0.719; anxiety: p = 0.425), and no significant difference between groups in the proportion of students who moved from pre-test at-risk to post-test healthy (depression: p = 0.997; anxiety: p = 0.530).

Secondary analyses: parent-reported and teacher-reported outcomes

SCAS-P

The Group × Time interaction was significant [F_{(1, 197)} = 4.43, p = 0.037, partial eta-squared = 0.022]. LSD (least significant difference) post-hoc contrasts indicated a significant pre-post decrease in anxiety symptoms for the intervention group (p = 0.009, partial eta-squared = 0.034), but no pre-post change for the control group (p = 0.708, partial eta-squared = 0.001).

SDQ-P total difficulties

The Group × Time interaction was non-significant [F_{(1, 194)} = 0.07, p = 0.793, partial eta-squared = 0.000], as were the main effects of group [F_{(1, 194)} = 0.85, p = 0.359, partial eta-squared = 0.005], and time [F_{(1, 194)} = 0.04, p = 0.837, partial eta-squared = 0.000].

SDQ-P social skills

The Group × Time interaction was non-significant [F_{(1, 197)} = 0.02, p = 0.893, partial eta-squared = 0.009], as were the main effects of group [F_{(1, 197)} = 0.12, p = 0.744, partial eta-squared = 0.001], and time [F_{(1, 197)} = 0.28, p = 0.595, partial eta-squared = 0.001].

SC-T

The Group × Time interaction was non-significant [F_{(1, 215)} = 0.12, p = 0.729, partial eta-squared = 0.001], as were the main effects of group [F_{(1, 215)} = 0.31, p = 0.577, partial eta-squared = 0.001], and time [F_{(1, 215)} = 0.00, p = 0.949, partial eta-squared = 0.000].

Gender effects

The Group × Time interaction reflects the intervention effect. A significant Gender × Group × Time interaction, therefore, would indicate that gender moderates the intervention effect. The three-way interaction was significant for SCAS-P [F_{(1, 190)} = 7.75, p = 0.006, partial eta-squared = 0.039]. Post-hoc LSD contrasts indicated a significant pre-post decrease in parent-reported anxiety for the intervention males (p = 0.004), in conjunction with a significant increase for the control males (p = 0.025). There was no pre-post change for the intervention females (p = 0.372) or the control females (p = 0.222). It appears that the previously reported pre-post decrease in parent-reported anxiety for the intervention group was carried by the males.

The Gender × Group × Time interaction was also significant for SC-T [F_{(1, 193)} = 8.20, p = 0.005, partial eta-squared = 0.041]. Post-hoc LSD contrasts indicated a significant pre-post increase in teacher-reported social competence for the intervention females (p = 0.041), in conjunction with a non-significant pre-post decrease for the control females (p = 0.112). There was a non-significant pre-post decrease for the intervention males (p = 0.079) and no change for the control males (p = 0.578).

Program fidelity and integrity

In total, 100% of the modules were completed by two of the five intervention classes, 90% with one class, 80% with another, and 20% by the fifth class. As one of the classes completed only 20% of the modules, a multiple regression, controlling for pre-test levels, was conducted to determine whether there was a dosage effect that would warrant excluding this class from analysis. There was no evidence of a dosage effect for any of the measures (p > 0.05).

In some cases, not all of the components of a particular module were completed or the component was not completed as specified in the manual. Unfortunately, log books were not completed by two of the five facilitators. Therefore, it was unclear to what extent these classes completed all components. Of the three who completed the log books, at least 95% of each group session was completed either in full as presented or with modifications to suit student needs.

Complete independent integrity check data were only obtained for one of the facilitators. In this case, there was good agreement between the entries provided by the facilitator and observer.

Process evaluation

Student enjoyment of the program was high, with 92% of children reporting that they enjoyed the program. Parents rated their children's enjoyment of the program highly (M = 3.9, SD = 1.1) on a scale of 1–5, with 1 indicating “not at all” and 5 indicating “very much.” Parents also rated a moderately high overall satisfaction of the program (M = 3.3, SD = 1.1) Parents reported moderate ratings of the effectiveness of the program (M = 2.9, SD = 1.2), and noticing positive changes in their children (M = 2.4, SD = 1.2). Few parents reported noticing negative changes in their children since participating in the program (M = 1.3, SD = 0.82). It appears though, that children did not talk to their parents about the program to a great extent (M = 2.0, SD = 1.0). As this may have impacted on parent's evaluation of the program, secondary analyses were conducted on students who were rated as having spoken to their parents to a fair degree (i.e., item rating ≥ 3). Parents from this sample rated their children's enjoyment more highly (M = 4.5, SD = 0.68) as well as overall usefulness of the program (M = 3.9, SD = 1.0).

Qualitative feedback was generally positive, with facilitators reporting implementation success across the Year 2 and 3 groups, and teachers of these year groups reporting that the experience was “worthwhile” and “has a lot of potential to help the children.” However, two teachers (of Year 1 classes) commented that the program was too difficult for their students and needed to be adapted and simplified.

Discussion

The aim of the present study was to evaluate the short term efficacy of a universal school based prevention program that was piloted for the prevention of anxiety and depression in Years 1–3 children in an Australian primary school.

Internalizing symptoms

Hypothesis 1a: universal prevention effects

With regards to internalizing symptoms, students in the prevention group did not show a greater decrease in depressive or anxiety symptoms compared with students in the control group. As such, the hypothesis that children in the prevention group would have significantly lower depression and anxiety scores as compared to children in the control group was not supported. This finding is in contrast with the findings of researchers targeting the prevention of internalizing disorders in older children, such as Barrett and Turner's (2001) universal prevention of anxiety program (FRIENDS for children), and Jaycox et al.'s (1994) selected study of depression program. In both studies, the intervention groups reported fewer internalizing symptoms at post-test than participants in the control condition. However, these results are compatible with Dadds et al.'s (1997) prevention of anxiety program (the QEIPAP which targeted younger children aged 7 years and above). In this study, Dadds and his colleagues did not find a significant difference between intervention and control groups in terms of anxiety symptoms immediately post intervention, with both groups showing improvements. However, this improvement in anxiety symptoms was only maintained in the intervention group at follow-up.

While no significant effects were found for child self-reports, parents of children in the prevention group reported a significantly greater decrease in their children's level of anxiety compared to parents of children in the control group. The issue of congruence between parent and child report has been problematic, with numerous researchers noting poor agreement between parent and child reports (e.g., Petot et al., 2011). Moreover, parent-child agreement was found to be lower with regards to internalizing symptoms (e.g., Rodriguez, 2011) as compared to observable behavior and externalizing symptoms (e.g., March et al., 1997 as cited in Nauta et al., 2003). Furthermore, with regards to concordance between parent and child interviews, poorer agreement was noted for younger as compared to older children (Klein, 1991; Petot et al., 2011). In determining which informant is more reliable in terms of diagnostic interviews, Klein (1991) concluded that “it is generally believed that the informant who provides the greater prevalence for a disorder is more likely to be accurate” (p. 196). Parents of children in the intervention condition, in contrast, reported significantly less anxiety issues in their children than those parents of children in the control condition and the effect size for this was in the small to moderate range. This finding partially supports Hypothesis 1a. It is unlikely that the significant findings observed in the parental reports were due to reactivity effects (i.e., parents in the prevention group responded in a biased manner because they were aware that their child was receiving the prevention program) as a similar improvement should have been evident on the other parent measure (SDQ) as well. Taken together, this may provide some preliminary support for the hypothesis that children in the prevention group exhibited less anxiety levels than children in the control group.

Hypothesis 1b: universal prevention effects

A second hypothesis was that children in the intervention group who scored in the normal range at pre-test would maintain a healthy status at post-test (Universal prevention effects). With regards to self-reported levels of anxiety or depression, the study provided no evidence that a greater proportion of children in the intervention group who score in the normal range at pre-test maintained their pre level scores.

Hypothesis 2: selected or indicated prevention effects

A third aim was to examine the prevention effects of the intervention on “at risk” children (Selected or Indicated prevention effects). It was predicted that a greater proportion of intervention children who exhibited symptoms of internalizing difficulties at pre-test would exhibit a decline in internalizing difficulties at post-test. There was no evidence of this found to support this hypothesis. This is not consistent with results found with the PENN program (e.g., Gillham et al., 2007) as well as the Aussie Optimism Program (e.g., Roberts et al., 2004). However, it is important to note that no follow up of these results have been carried out and effects may have been found at later time periods.

Risk and protective factors: social and emotional competence

Children in the prevention group did not show a greater improvement in their emotional competence than children in the control group, which did not support the hypothesis that children in the prevention group would have more well-developed emotional knowledge at post-test than children in the control group, although again no follow-up was carried out where effects may have been found.

Teachers of students in the prevention group did not rate any greater improvement in their students' social competence than teachers of students in the control group. This is in contrast to findings by Kam et al. (2004). In this study, students who received PATHS displayed improvements in emotional competence (e.g., greater emotional understanding and recognition) but there was no significant differences in the intervention group in terms of teacher reported social competence. The authors postulated that this may have due to the program's stronger and early emphasis on emotional competence, and postulated that the program did not sufficiently focus on the acquisition of social competencies (Kam et al., 2004). Likewise, the current program initially focused on emotional competence (which is a natural prerequisite for social competence), and areas of social skills and social problem-solving were emphasized later on in the program. It is plausible that social competency skills and emotional skills may take time to be integrated, and may be more readily measurable at follow up testing. Another possible explanation for this lack of significant difference between groups in terms of gains in social competence may in part be attributed to the suspect validity of the social competence instrument employed in this study. This teacher measure of social competence may not be sufficiently sensitive to change.

Taken together, these results provide some support for the programs' potential preventive effect. The program appears to either directly or indirectly to ameliorate anxiety symptoms. It is hypothesized that the reduction of anxiety symptoms in the prevention group could come through direct teaching and application of behavioral strategies (e.g., using relaxation). Whether there are indirect pathways through improving emotional competence (e.g., general improvement in emotional regulation) or social competence remains to be evaluated via longer term follow-up and improved measures of social competence.

Explanations for lack of significant findings

Perhaps one of the most obvious explanations for the lack of significant findings is the lack of program fidelity achieved by two of five teachers in the intervention group. The fact that the program was not completed or adhered to may have reduced the full potential of effects that might have been found. The lack of effect could be due to a latency effect, whereby a period of time is required for elements of the program (such as the strategies and concepts) to be adopted and integrated for effective use (Quayle et al., 2001). This latency effect has been found in several preventive studies such as Dadds et al.'s (1997) selective study, as well as in Quayle et al.'s (2001) and Roberts et al.'s (2003) universal studies. In these studies the critical effects were only apparent at 6 month follow-up. As such, it is plausible that the program's preventive effects could be more readily observable at a later time, when use of behavioral strategies and concepts become more practiced and generalized.

Another potential explanation for the lack of significant effects could be the relative brevity of the intervention and dilution of effects for anxiety and depression. It is feasible that the universal program spanning 10 sessions in total (the last being a review) is insufficient to bring about significant and lasting acquisition of protective skills.

Finally, the absence of significant effects for universal programs are not wholly unexpected, and are consistent with several research findings such as those from the meta-analysis conducted by Merry et al. (2003) and from Sheffield et al. (2006) evaluation of universal, indicated, and combined approaches.

Universal prevention programs generally require large sample sizes in order to capture clinically significant population effects. Cujipers (2003) estimated that 30, 211 participants across experimental and control conditions are required to capture a 15% decrease in the incidence of major depression, one of the common mental health disorders.

Global social validity of the program

A further aim of the study was to ascertain the level of social validity and consumer satisfaction of the program so as to ensure the long-term viability of program. The vast majority of students reported that they enjoyed the program, with older students showing particularly good retention and being able to identify how the program was beneficial in their lives. Parents commented on the merits of the program, with some parents noting a number of positive changes in their children. Teacher feedback regarding the efficacy and usefulness of the program varied. Overall, the Years 2 and 3 teachers positively endorsed the program. However, the 2 Year One teachers who did not complete the program rated the program less favorably, stating the developmental inappropriateness of the program to their younger children being a major limitation. On the whole, child, parent and teacher feedback converge on the overall merits of such a prevention program, with the majority of teachers and students describing the experience as worthwhile and enjoyable.

Limitations of the study and directions for future research

Several methodological limitations pertaining to the actual program and its implementation must be noted. One of the most important being the lack of program implementation fidelity for two of the five intervention classes. The program was run concurrently with at least one other universal program which meant that the demand on the teachers' resources was high. Thus, it is recommended that future programs be conducted in schools who are not simultaneously running other programs.

Another potential limitation was that the program appeared too complex for Year 1 and 2 students, many of whom are only just beginning to learn to read and write. The complexity of the program may have reduced the likelihood of gaining the full benefit for year 1 and 2 students so it is recommended that a separate developmentally appropriate with less reading and writing be developed for this age groups and the current version of the program maintained for Year 3 students.

There are several limitations pertaining to assessment issues. Firstly, self-report measures of internalizing symptoms were only available for the Year 3 children. This may have diluted the intervention effects that may have been found if self-report measures of internalizing difficulties were available for all year levels. In addition, as a result of random assignment, the reliance on only Year 3's resulted in a small control group, and the disproportionate sizes of both groups, may well have resulted in a corresponding loss of power. The decision not to use the self-report questionnaires also resulted in a reliance on parent measures of internalizing symptoms, where the response rate was not ideal. Secondly, the lack of program effects in terms of gains in social competence may in part be attributed to the validity of teacher ratings on the social competence measure.

Based on feedback regarding the developmental appropriateness and length of each session, future revisions of the Feelings and Friends program should target the length of the program by using more, but briefer modules. The program also needs to be simplified for the younger age group (incorporating Years 1 and 2). In addition, in light of the importance of soliciting family support and active familial engagement in the prevention process, it would be helpful to incorporate a parent component.

Conclusions

There was some evidence suggesting that the program was able to prevent anxiety in children. However, longer term follow up is required in further trials to investigate whether the program has longer term effects on anxiety, depression, or social and emotional competence. Although this study suffered from some implementation and assessment limitations, these results, when taken in the context of previous research, provide grounds for optimism in the field of early prevention of internalizing disorders. This study highlights the importance and usefulness of targeting emotional wellbeing in young children. It also highlighted a number of program implementation issues that can guide future researchers on how to overcome some difficulties encountered in real-world applications of programs in schools. The findings of the present study provide evidence that with further modification, the Feelings and Friends program may be of benefit to the mental health of 6–8 year old children and it is clear that a larger trial of a developmentally modified program with follow up periods needs to be carried out.

Author contributions

EP—Conducting the study and contribution in writing the manuscript draft. RR—Principal Investigator of the Healthway grant that funded the current project, contribution to introduction and discussion of the manuscript. RK—Methodology and data analysis. MN—Chief Investigator of the current project, revising the draft and contribution to the discussion section of the manuscript. MD—Revising the drafts and conceptualization of the project. NB—Contribution to introduction and discussion sections of the manuscript. SH—Revising drafts and contribution to the discussion section of the manuscript.

Funding

This study was supported by the Western Australian Health Promotion Foundation (Healthway), through research grant (15148).

Conflict of interest statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.