Preliminary Investigation of a Novel Cognitive Behavioural Therapy Curriculum on the Wellbeing of Middle Schoolers

Abstract

Objectives

A number of school-based interventions for preventing and attenuating symptoms of anxiety and depression in youth have been developed worldwide but evidence of their effectiveness is mixed. None of these curricula stem from existing children’s literature, however, the Harry Potter (HP) series has been identified as potentially imparting Cognitive Behavioural Therapy concepts.

Methods

This study aimed to broadly capture, at an interim stage, the feasibility of a pilot HP curriculum aimed at imparting CBT skills to middle-schoolers in order to inform full development of the curriculum. The study design further included a non-randomized, Group (HP; Control) by Time (Baseline, Post-Intervention, Follow-up) mixed factorial approach retrospectively examining change in scores on a “Well-Being and Resiliency Survey” (WBRS) which assessed multiple symptomatic and social domains.

Results

In total, 232 grade seven and eight students participated in the curriculum over two years with no dropouts. Compared to 362 controls, there were no significant differences in WBRS scores between groups at post or one-year follow-up.

Conclusions

This pilot study demonstrated the feasibility of rapid and broad implementation of the intervention. Preliminary analyses showed no indication of effectiveness which may be the product of the intervention being in an interim stage during the study period and/or the retrospective design and limitations in data quality. Specifically, these results suggest that the WBRS may be a suboptimal instrument for measuring the effectiveness of this intervention. A prospective trial of the complete, revised curriculum with validated measures is required to provide an adequately assessment of its impact.

Introduction

Mental disorders affect nearly 20% of school-aged youth (Kieling et al., 2011) and more than half of these disorders will arise before the age of 14 (Merikangas et al., 2010). Universal interventions are considered a quintessential element of efforts to lower the burden of youth mental disorders (Offord et al., 1998) and the school environment has been identified as a key venue for mental health promotion (Joint Consortium for School Health, 2018; Wei & Kutcher, 2012). Two inter-related areas of study in this domain are mental health literacy (MHL) interventions and psychotherapy-informed interventions to prevent the onset of anxiety and depressive disorders.

The concept of MHL was first proposed by Jorm in 1997 to denote “knowledge and beliefs about mental disorders which aid their recognition, management or prevention” (Jorm et al., 1997) and has since been expanded to include additional aspects such as decreasing stigma of mental disorders and improving help-seeking (Kutcher et al., 2016). Much of the early literature regarding MHL was of poor quality (Wei et al., 2013). However, recent rigorous research examining school-based MHL curricula developed in Canada suggests that they can lead to improved knowledge and attitudes about mental disorders across the country (Kutcher et al., 2015) and can be adapted to improve mental health knowledge and reduce stigma in youth in developing countries (Ravindran et al., 2018). The “InSciEd Out” curriculum in the United States has yielded modest benefits in improving knowledge and encouraging help-seeking (Yang et al., 2018), while a two lecture curriculum delivered in Japan yielded significant benefits including a nearly two-fold increase in students who reported they would seek help if they were depressed (Ojio et al., 2015). Most intriguingly, the multi-national Saving and Empowering Young Lives in Europe (SEYLE) study demonstrated that a five hour intervention (two hours of interactive lectures and three hours of role-play) in addition to a 32-page booklet delivered in school to 15 year olds decreased the risk of suicide attempt by more than 50% over 12 months compared to a control condition (Wasserman et al., 2015). Of note, the MHL arm was the only one to result in a significant difference with neither gatekeeper training nor questionnaire-based arms of the trial separating from the control arm (Wasserman et al., 2015).

Numerous school-based psychotherapy-informed curricula, most focused on imparting principles of cognitive behavioural therapy (CBT), have been widely implemented and tested for their capacity to prevent anxiety and depressive disorders with some studies showing positive results on some measures (Calear et al., 2009; Challen et al., 2014; Lock & Barrett, 2003; Merry et al., 2004; Pössel et al., 2004) and some showing no effect (Araya et al., 2013; Buttigieg et al., 2015; Dray et al., 2017; Horowitz et al., 2007; Roberts et al., 2010; Sawyer et al., 2010). A 2016 Cochrane review focusing predominantly on school-based, CBT interventions for preventing depression found small positive effects (Hetrick et al., 2016). However, methodological limitations as well as findings that universal interventions were often not superior to attention placebo controls (i.e. controls who had received the same degree of time and attention as those in the intervention group) led the authors to judge the evidence insufficient to support the implementation of such programs (Hetrick et al., 2016). The CBT-based Penn Resiliency Program (PRP) is one of the most rigorously studied school interventions. While initial meta-analytic results suggested a small but meaningful impact on depressive symptoms (Brunwasser et al., 2009), more recent research was unable to demonstrate any positive impact of the program (Bastounis et al., 2016). Furthermore, the Op Volle Kracht (OVK) intervention developed in the Netherlands based on the PRP was similarly ineffective (Kindt et al., 2014; Tak et al., 2016). In contrast, a systematic review of school-based interventions for anxiety found that 11 of 16 interventions universally delivered to all students resulted in significant differences compared to controls often with medium to large effect sizes (Neil & Christensen, 2011).

While some school-based mental health interventions show promise in promoting mental wellbeing, evidence is mixed and there is no established consensus for which of the myriad of programs ought to be implemented (Fazel et al., 2014; Weare & Nind, 2011). Some evidence does suggest that interventions targeting younger children may have the largest effect (Lock & Barrett, 2003). The sustainability of time-consuming, universal health curricula has also been questioned given the many demands on young students (Han & Weiss, 2005; Ojio et al., 2015; Santor & Bagnell, 2013).

Our group has attempted to identify an innovative solution to some of these concerns by developing a curriculum which includes elements of MHL and CBT and is based on the popular and widely available Harry Potter (HP) series (Sinyor et al., 2017). Previous school-based interventions have historically been subsumed under the umbrella of more general health literacy⁷ however, to our knowledge, the HP curriculum is the only intervention to be embedded within a reading literacy curriculum delivered as part of regular English studies. The curriculum is also focused on resilience (of both the protagonist and the author) which has been identified as an important avenue of emphasis for mental health promotion (Brooks, 2006; Davydov et al., 2010; Greenberg et al., 2001). Indeed, it has been argued that the focus of school-based MHL programs must shift from a more narrow emphasis on preventing mental disorders to a broader focus on improving overall mental health (O’Mara & Lind, 2013). Furthermore, a small literature in the developing world has also demonstrated poorer youth MHL than in more developed countries (Aluh et al., 2018; Ganasen et al., 2008) supporting the urgent need for interventions that could be widely disseminated across nations and a curriculum based on the Harry Potter novels could be ideally suited for that purpose given that the novels have reportedly sold more than 500 million copies worldwide and have been translated into more than 80 languages (Pottermore, 2018).

An early framework of the curriculum was piloted in grade seven and eight classes at a school board in Ontario. Anecdotal feedback from both students and teachers was uniformly positive (Sinyor M. personal communication, October 7, 2016). Informal focus groups conducted by the school board itself to evaluate the curriculum included comments from students emphasizing a) the relatability of the character, b) how helpful it was to see that both Harry Potter and classmates also deal with anxiety, sadness and distress in general and a sense of not being alone, c) the positive impact of information on how to handle difficult situations, as well as d) the utility of a list of personalized “stress busters” that each student developed as part of the intervention. In particular, the students noted that it helped them to appreciate the multiple ways that one can cope with distress, to develop strategies to manage their anxiety, to be more willing to share their feelings with others, and to feel more connected to their classmates. Although this pilot implementation occurred while the curriculum was at an interim stage and was not originally designed as a research study, the enthusiastic feedback from students suggested that testing its impact even at this early time-point would be worthwhile and could help to inform the curriculum’s further development. The students had not been given any validated research measures, however the school board’s own yearly “Well-Being and Resiliency Survey” was available for analysis and that formed the basis of the analysis.

Methods

Study Population

This research was approved by the Research Ethics Board at Sunnybrook Health Sciences Centre (#409 2017). Subjects in this study were middle school students from a single, small, rural publicly funded school board in Eastern Ontario. The population of interest included all students who received an early draft version of the HP curriculum in the first two years in which it was being piloted. This included five classes taught by four teachers in the Spring 2016 semester and 13 classes taught by 11 teachers in the Spring 2017 semester. As this was intended as a public health intervention aimed at potentially improving the mental health of all youth, it was delivered to entire classes within the board. For comparison, all remaining grade seven and eight students in the school board acted as controls divided into two groups. The first comprised all students from schools where the HP curriculum was implemented who were in classes that did not receive it (same-school controls). The second comprised a randomly selected group of students from schools from the same board where no classes received the HP curriculum (different-school controls). Note that neither group received a CBT-based curriculum or intervention during the study period.

Of note, since this intervention was not initially expected to become a research study, selection of participating teachers and classes was non-random. Each year, the school superintendent sent a message to English teachers asking whether they had space and interest to add the curriculum to their agenda. Those teachers who replied in the affirmative were given training in delivery of the curriculum as outlined below.

Intervention

A framework for the Harry Potter-based CBT curriculum has been published previously (Sinyor et al., 2017). In brief, the series’ author J.K. Rowling has stated publicly that she pursued CBT treatment and embedded her experience of depression within the 3^rd Harry Potter book, The Prisoner of Azkaban (Sinyor et al., 2017). Our work suggests that she also embedded CBT skills into the novel; indeed, through that lens, the narrative can be understood as Harry Potter becoming depressed and learning CBT skills to find a path to resilience (Sinyor et al., 2017). The intervention, therefore, is a literature study in which students read the novel and learn CBT skills along with the main character. It involves nine modules that follow the book’s narrative arc: 1) Psychoeducation A (risk factors for emotional distress), 2) Psychoeducation B (understanding depression and its treatment), 3) Introduction to Cognitive Distortions, 4) Introduction to CBT Treatment, 5) Key CBT skills (fear hierarchies, behavioural activation, managing cognitive distortions, identifying core beliefs), 6) Psychoeducation C (handling setbacks), 7) Putting learned CBT skills into practice, 8) Advanced Management of Cognitive Distortions/Core Belief Work, 9) Relapse Prevention/Consolidation of Learning.

Teachers were provided with two to three hours of lectures on principles of CBT as well as a discussion of how they relate to the novel. Each was then provided with a detailed description of the key CBT learning requirements for each set of chapters and how they are depicted in the book along with a list of suggested reflection questions for the students (Supplement A, available from the author by request). As this was a curriculum pilot, teachers had wide latitude in terms of how to impart the CBT concepts and used a variety of techniques including written assignments, group discussion and role-play. Most teachers completed the curriculum in 8–12 weeks.

Measures

Given that the HP intervention began as a curriculum framework and a pilot intervention not originally intended for formal research, the students involved did not complete standard research measures. However, all students at the school board complete a 127 item self-report, 5-point Likert “Well-Being and Resiliency Survey” (WBRS) in the late fall of each year (Supplement B, available from the author by request). That survey includes questions relating to numerous health and wellbeing outcomes of interest which formed the basis for the analysis including questions related to cognitive (and psychological) flexibility, depressed mood, anxiety, neurovegetative symptoms including sleep and appetite, social support (peer and adult), self-esteem, beliefs about personal effectiveness, attitudes about help seeking, empathy, substance use, exercise, bullying/aggression, social involvement, community belongingness, family involvement, emotional awareness, openness to communicate at home, and love of learning. A preliminary set of composite measures for each outcome of interest derived from the WBRS questions were proposed by the primary investigator. These measures and the complete list of WBRS questions were circulated among study investigators who include experts in mood and anxiety disorders, child and adolescent psychiatry and CBT. The study team then revised the composite measures until consensus was reached and the final agreed upon versions comprised the variables included in the analysis. (Supplement C, available from the author by request, has a complete list of questions used for each variable). Table 1 presents a logic model describing the rationale for inclusion of all variables.

Table 1.

A Logic Model Describing the Harry Potter Curriculum, Activities and Expected Benefits.^*

Target Symptoms	Program Objectives	Activities	Outcomes
Deficits in cognitive flexibility	For students to understand the concept of cognitive distortions, to recognize them and to learn strategies for cognitive restructuring (along with the book’s protagonist).	Psychoeducation about cognitive distortions (particularly through the narrative related to the novel’s fortune-teller character – Professor Trelawney) with student assignments asking them to spot these in the book’s characters and themselves as well as discussions and assignments on how to identify alternative, more rational thoughts.	Improved cognitive flexibility
Deficits in general mental health literacy	To impart an understanding of risk factors and protective factors for distress and mental illness.	Class discussion and assignments highlighting how bullying (specifically via the characters Dudley, Malfoy and Snape) put the main character and people in general at risk for poor mental health outcomes. Assignments imparting the notions that having positive relationships and communicating feelings can be protective against the evolution of distress and mental illness as well as class discussions in which effective communication is modeled and encouraged by teachers.	Decreased social aggression/bullying and improved peer social support/empathy Improved social/community/family involvement, communication at home and confidence that emotions can be expressed in healthy ways
Distress (including anxiety and depression) and inadequate skills for coping with it	To normalize distress as an expected part of life, to teach students about the symptoms of anxiety and depression and to provide individualized skills for managing distress.	Class discussion and assignments about distress and mental illness and how they manifest (with particular emphasis on the dementor characters). Through these activities students develop an individual distress management plan including strategies for spotting and addressing cognitive distortions as well as a toolbox of “stress busters” to cope with distress including activities such as listening to music, exercising, avoiding substances, and being with friends. Students are also taught that when these fail, they should seek help as the main character of the book does with positive outcomes.	Improvement in depressive, anxiety and neurovegetative symptoms as well as self-esteem/self-image, beliefs about personal effectiveness, help seeking attitudes. Also, decreased substance use and increased exercise
Deficits in general literacy**	To instill a love of reading books and greater understanding of allegory and hidden meanings in texts.	Textual analysis in which connections between the narrative and mental health are discussed for each chapter.	Improved love of learning

^*Note that the curriculum has no explicit content about the idea that teachers care about the mental wellbeing of their students. Nevertheless, it was hypothesized that group discussions about mental health as well as the overall effort undertaken would improve students’ beliefs that their teachers listen to them and care about their wellbeing.

^**Note that although the curriculum was primarily devised to improve mental health literacy and mental wellbeing, it also functions as a general literacy curriculum.

Note that some items on the WBRS, particularly those concerning depression, anxiety and neurovegetative symptoms are similar to those in validated Likert scales such as the Revised Child Anxiety and Depression Scale (RCADS) (Chorpita et al., 2005). For example, WBRS items such as “I get no pleasure from usual activities and am often sad, unhappy, or depressed,” “I worry a lot and am anxious or fearful,” and “My family would say that I don’t get enough sleep” are similar to the RCADS items “I feel sad or empty”, “Nothing is much fun anymore “, “I worry about things”, and “I have trouble sleeping”. Other items have similarities to scales examining coping skills such as the Life Problems Inventory (LPI) (Rathus et al., 2015). For example, WBRS items such as “In the last month, I have regularly used marijuana or other drugs” and “I can become aggressive or violent when I’m angry” are similar to LPI items “I often get high on street drugs like marijuana or other drugs” and “I’ve lost my temper and really yelled or screamed at someone”/“I’ve physically hurt or attacked someone”. Many items, however, have no analogue in validated scales.

Each student’s grade, sex and age on January 1^st of the year in which they received the intervention (or were studied as a control) were also provided by the school board.

Statistical Analysis

For students who received the HP pilot curriculum in Spring 2016 and related controls, Fall 2015 survey data were used as a baseline (T1) with Fall 2016 (T2) and Fall 2017 (T3) survey data as one and two year post-intervention data, respectively. For students who received the HP pilot curriculum in Spring 2017 and related controls, Fall 2016 survey data were used as a baseline (T1) with Fall 2017 survey data as one year post-intervention data (T2) (i.e. two year post-intervention data were only available for the subset of students who received the curriculum in its first year).

The study design was a non-randomized, Group (HP Curriculum; Controls) by Time (Baseline/T1, Post-Treatment/T2, Follow-up/T3) mixed factorial design with the WBRS scale scores as the primary dependent variable. Because of expected multicollinearity between the multiple domains of the dependent measure, a multivariate analysis of variance (MANOVA) was conducted with the 20 composite variables derived from the WBRS as the dependent variables with Group (HP, controls) as the between-group variable. To maximize power by retaining the full sample involved during the acute phase of the study (Baseline to Post) and not limiting the power based on significant attrition from post- to follow-up, analyses were conducted separately at post-treatment and follow-up with baseline WBRS scores as the co-variate),

Differences between students who received the HP curriculum and controls were tested using a t-test for age and Chi-squared tests for sex and grade.

Results

All teachers, students and classes who began the HP pilot curriculum ultimately completed it. That is, there were no “dropouts”. This encompassed 107 students who received the intervention the first year it was offered (Spring 2016) and 125 who received it in the second year (Spring 2017). Although there was no formal plan or mechanism for implementation monitoring in this study, all teachers involved self-reported completing the entire nine-module curriculum with their classes. Furthermore, the study team reviewed sample written exercises from students as well as videos of classes taking part in group reading, class discussions, and team-based exercises at various points in the curriculum. All were in line with expectations for appropriate implementation with no examples of improper or inadequate implementation identified.

In total, 594 students were included in this study including these 232 students who received the intervention and 362 controls (116 in 2016 and 246 in 2017; 133 same-school, 229 different-school). There was no difference in mean age between students who received the HP pilot curriculum (12.64±0.52) and controls (12.62±0.51) (t=0.602, df=592, p=0.55) or student grade (HP curriculum: 88 grade 7 (37.9%), 144 grade 8 (62.1%) vs. Controls: 147 grade 7 (40.6%), 215 grade 8 (59.4%); χ² 0.43, p=0.52). There were a greater proportion of girls in the group who received the HP curriculum (133/232, 57.3%) than in the control group (175/362, 48.3%) (χ² 4.57, p=0.03) and multivariate analyses found that overall the HP curriculum group had poorer survey scores reflecting greater difficulties at baseline (Wilks’ λ =0.929, F(2.325), p=0.01).

Results of the MANOVA and values for all variables at each time point are shown in Table 2. The multivariate effects revealed that there was no difference between groups in change scores at year 1 (Wilks’ λ =0.977, F (0.692) p=0.829) or year 2 (Wilks’ λ =0.852, (1.591), p=0.063). In the absence of multivariate effects at the broad omnibus scale level, no univariate effects for individual domains were permissible. Further, there were no effects of age, sex or grade and results did not differ whether controls were taken from the same or a different school.

Table 2.

Mean Differences between the Harry Potter Cognitive Behavioural Therapy (CBT) Intervention and Control Groups in Variables Derived from a Well-Being and Resiliency Survey from Baseline to Year 1 and Year 2 Follow-up

	CBT			Controls
	Baseline (T1)	Year 1 follow-up (T2)	Year 2 follow-up† (T3)	Baseline (T1)	Year 1 follow-up (T2)	Year 2 follow-up† (T3)
	n=236	n=232	n=103	n=366	n=362	n=110
	Mean (SD)	Mean (SD)	Mean (SD)	Mean (SD)	Mean (SD)	Mean (SD)
Cognitive Flexibility	22.76 (3.85)	23.78 (3.58)	24.50 (2.83)	23.74 (3.73)	24.43 (3.27)	24.54 (3.25)
Depressive Symptoms	12.39 (2.30)	11.81 (2.44)	11.47 (2.55)	12.50 (2.25)	12.26 (2.24)	12.20 (2.26)
Neurovegetative symptoms	6.93 (1.79)	6.39 (1.95)	5.82 (1.86)	7.18 (1.85)	6.79 (1.77)	6.59 (1.89)
Anxiety symptoms	6.69 (2.28)	6.19 (2.30)	5.61 (2.22)	6.68 (2.31)	6.39 (2.32)	6.25 (2.53)
Peer Social Support	36.29 (5.94)	36.58 (5.75)	36.16 (5.61)	37.65 (5.59)	37.52 (5.58)	37.31 (6.07)
Teacher Social Support	7.79 (1.58)	7.45 (1.59)	7.35 (1.68)	7.87 (1.68)	7.77 (1.63)	7.74 (1.71)
Help Seeking Attitudes	11.46 (2.03)	11.13 (2.38)	10.87 (2.28)	11.81 (1.94)	11.62 (2.20)	11.73 (2.13)
Empathy and Caring	21.16 (3.23)	20.66 (3.18)	21.01 (2.79)	21.04 (3.16)	20.96 (2.99)	21.39 (2.61)
Personal Effectiveness	16.31 (2.72)	16.12 (2.69)	15.98 (2.46)	16.74 (2.69)	16.60 (2.46)	16.65 (2.33)
Self-esteem/positive self-image	16.22 (3.14)	15.53 (3.17)	14.83 (3.23)	16.62 (2.92)	16.09 (3.16)	15.62 (3.11)
Substance use	9.62 (1.19)	9.41 (1.19)	9.01 (1.76)	9.76 (.87)	9.56 (1.10)	9.28 (1.48)
Social Aggressive/Bullying	12.54 (2.52)	11.57 (2.39)	11.47 (2.17)	12.14 (2.51)	11.50 (2.45)	11.37 (2.47)
Openness/Communication at home	4.18 (0.86)	3.89 (0.99)	3.80 (1.05)	4.16 (0.93)	4.03 (1.02)	3.99 (0.99)
Love of learning	3.84 (0.93)	3.91 (0.94)	3.96 (0.90)	3.97 (0.92)	4.03 (0.87)	4.02 (0.79)
Engagement in exercise	3.81 (1.14)	4.18 (0.96)	3.97 (1.04)	3.87 (1.08)	4.24 (0.88)	4.07 (0.93)
Social involvement	15.61 (3.08)	15.34 (3.14)	14.74 (3.39)	15.67 (3.19)	15.46 (3.42)	14.61 (3.87)
Community belonging	7.61 (1.61)	7.67 (1.58)	7.46 (1.61)	8.00 (1.54)	7.94 (1.63)	7.89 (1.67)
Emotional expression	3.75 (0.90)	3.74 (0.91)	3.72 (0.96)	3.85 (0.86)	3.81 (0.91)	3.77 (0.90)
Family Involvement	17.20 (2.96)	16.48 (3.04)	16.13 (3.19)	17.49 (2.91)	17.01 (3.31)	16.97 (3.22)

^†For those CBT and control students who had received a 2-year follow-up.

^§Multivariate test: All values are statistically non-significant.

Discussion

This retrospective pilot study had divergent outcomes related to preliminary feasibility and impact of the HP pilot curriculum. As described above, this was not planned as a formal research study but rather the intervention was developed and piloted in the context of community outreach with the intervention only formally presented to the school board in March of 2016. The results that more than 100 students were recruited and received the intervention by the end of that school semester, that more were recruited the following year, that all completed the intervention, that informal implementation monitoring suggested high fidelity to intervention model, and that at least anecdotally, reports from both school staff and the students who participated in the curriculum themselves suggested that they had benefited greatly collectively make a strong case for feasibility. However, this study also showed no demonstrable impact on wellbeing in middle school students who received it compared to controls. The focus on middle-schoolers is a major strength of this study as previous MHL research has been criticized for an over-emphasis on university students who are beyond the most common age of disorder onset (Kutcher et al., 2016). While the preliminary impact findings here are contrary to expectations and somewhat disappointing, there is a question of how to interpret them given significant limitations of the study design relating to its retrospective nature, data quality and focus on a curriculum that was still under development as it was being implemented. This last point is of particular note given evidence that low intensity CBT-based interventions may be less likely to yield the same benefits seen with high-intensity CBT (Lovell et al., 2017).

Taking the findings at face value, the null hypothesis that the curriculum is ineffective at improving any measure of student wellbeing is confirmed. Although unexpected, this conclusion is supported by other studies of school-based CBT interventions in the literature which failed to show benefit (Araya et al., 2013; Bastounis et al., 2016; Buttigieg et al., 2015; Dray et al., 2017; Hetrick et al., 2016; Horowitz et al., 2007; Kindt et al., 2014; Roberts et al., 2010; Sawyer et al., 2010; Tak et al., 2016). Furthermore, some research suggests that teacher-delivered mental health-related interventions are less effective, although at least one systematic review of interventions for youth anxiety found that teacher-delivery did not appear to affect outcomes (Neil & Christensen, 2011). One possible interpretation of the available evidence is that CBT skills are often too challenging to impart in a classroom setting, however such a conclusion may go beyond what can reasonably be inferred from this study. An important unresolved question is whether the same results would be observed if teachers were to provide a revised and augmented version of the HP curriculum. Regardless, the fact that there was absolutely no indication of benefit on the WBRS instrument, suggests that it may not be an optimal measure for a future prospective RCT and that more targeted measures examining students’ understanding of and use of CBT skills (i.e. what the curriculum is intended to teach) may be more appropriate primary metrics rather than non-specific measures of well-being such as the WBRS.

The results of this study must therefore be interpreted with a significant note of caution. As mentioned, this research had substantial limitations which are likely to have influenced its outcome. First, it was designed retrospectively without standard research measures including established feasibility metrics or measurements of the integrity of the intervention across classes. It would be reasonable to question whether a single, self-report measure delivered nearly a full year after the intervention was likely to capture adequately the intervention’s impact or even whether it had adequate sensitivity to capture change in the outcomes of interest. Second, while all teachers were provided with the same training and framework for the curriculum that they ultimately delivered, they were still developing and refining it during the first two years of its implementation. This fact cannot be sufficiently underscored. The original version of the curriculum included nine modules and a five page overview of the key learning points to be covered. The current iteration of the curriculum has twelve modules and is well over 300 pages long with much of that space devoted to a far more detailed explanation of the CBT concepts to be imparted for teachers who are likely to have little to no previous familiarity with them. Therefore, while it is reasonable to examine pilot data during the initial stages of a project like this one for any evidence of early impact, it would drastically overstate the quality of the evidence to draw firm conclusions about the effectiveness of the final, manualized version of the curriculum. It may instead be more appropriate to conclude narrowly that an HP curriculum with limited training, offering low intensity-CBT, may not yield benefit while the potential benefit of enhanced education and training for teachers paired with a high-intensity HP-based CBT model is unknown and untested. Note also that the revised curriculum was used throughout the school board in winter 2019 and teacher feedback, albeit anecdotal, reflects excellent uptake. Third, selection of schools and classes that received the HP curriculum, and therefore the students who eventually became subjects and controls for this study, was non-random and could have biased results of this study in either direction. In particular, the fact that students who received the intervention had poorer baseline scores suggests non-random group allocation and, indeed, it may be that teachers with students who had greater mental health challenges were more apt to be early adopters of the curriculum. Fourth, this study relied exclusively on self-report measures and it is unable to identify whether changes occurred that were only observable by teachers and/or parents. In short, for a study of this design, a negative result regarding impact on mental wellbeing is much more challenging to interpret than a positive one and a prospective trial accounting for these shortcomings and focusing on the updated, higher-intensity version of the HP curriculum would be necessary to form more reliable conclusions. Fifth, a potential drawback of the intervention itself would be that some youth do not relate to this specific character or book. Acceptability of the book was not formally measured, however informal focus groups conducted by the schoolboard identified some youth who had received the curriculum and reported not liking the book/topic in advance. Nevertheless, they still reported finding it of interest once they learned of the hidden mental health message embedded within the narrative. Sixth, this study only assessed change in middle schoolers on a one-year timescale. While that duration is preferable to a study of only weeks or months, it may be that a universal intervention such as the HP curriculum could have effects that would only become apparent years or even decades later. Finally, it should be noted that the HP curriculum is not intended as a substitute for mental health treatment, only that it may be worthy of further development and investigation as an ancillary public health intervention.

This study was a preliminary investigation of a pilot curriculum for improving mental wellbeing in middle schoolers. It was conducted during a period while the curriculum was in development under the assumption that quantitative metrics from the school would match positive anecdotal evidence from the same span. However, this was not observed and there was no change on any identified measure following the HP curriculum. These results are challenging to interpret given methodological limitations and point to the need for a more rigorous, prospective study of the finished intervention. This retrospective study did demonstrate a clear signal of feasibility of the intervention which has been adopted by more teachers and classrooms in successive years. Ultimately this should provide ample opportunity for further studies and suggests that if such studies show benefit, the HP curriculum may potentially be a highly generalizable and scalable intervention. Finally, even if the HP curriculum does not improve metrics of youth wellbeing in the short or medium term, that does not necessarily imply that it is of no use. Having an opportunity to discover the deeper meaning of a beloved book and to discuss mental wellbeing, distress management and resilience in an open, fun and engaging way with peers may yield long-term benefits that prove elusive to measurement in the moment.