Implementation of initiatives to prevent student stress: process evaluation findings from the Healthy High School study

Abstract

Process evaluation of public health interventions is important for understanding intervention results and can help explain why interventions succeed or fail. This study evaluated implementation of a school-based intervention combining educational and environmental strategies to prevent stress among Danish high school students. We investigated dose delivered, dose received, fidelity, appreciation, barriers and facilitators at the 15 intervention schools using mixed methods and multiple data sources: questionnaires among students, teachers and school coordinators; semi-structured interviews with school coordinators; telephone interviews with student counsellors; and focus group interviews with students and teachers. Implementation varied by schools and classes. Half of the intervention schools delivered the environmental strategies. For the educational strategies, dose delivered differed according to intervention provider. Students reported a lower dose received compared with dose delivered reported by school staff. Overall, student counsellors, school coordinators and students—especially those with low perceived stress—were satisfied with the stress preventive initiatives while teacher satisfaction varied. Five main barriers and three facilitators for implementation were identified. The use of multiple data sources and data methods created new knowledge of the implementation process which is important for the interpretation of effect evaluation and development of future interventions.

Introduction

Many adolescents report high levels of stress [1–3]. It is disturbing that their stress levels follow a similar pattern to those of adults [2, 4]. In the Danish National Youth Study 2014 [1], 15% of girls and 7% of boys in high school felt stressed on a daily basis. The acute fight-or-flight stress response is an essential survival mechanism that is generally protective and enhances performances under challenging conditions. However, frequent and/or prolonged activation of the body’s stress–response system can have seriously negative consequences for the individual [5, 6]. Adolescent stress has been linked with poor academic performance [7, 8], suicidal behaviour [9, 10], disturbed sleep [11, 12], negative mental health outcomes such as anxiety and depression [13–18] and a wide range of unhealthy behaviours including physical inactivity, unhealthy eating [19, 20] and alcohol consumption [19, 21].

The transition to high school is a stumbling time for many adolescents. It includes a radical shift in the school context with lower levels of teacher support and higher demands for independent academic performance compared with primary school [22, 23]. The transition is characterized by social changes as students make new friends and create new peer groups [24]. Stressors reported to be experienced most frequently by adolescents are those relating to the school environment [2, 25, 26]. Given the prevalence rates and impact of adolescent stress, it is essential that effective stress prevention is available. The school is an important setting for public health interventions as most children and adolescents across social and ethnic strata spend considerable time in school [27]. Universal school-based programmes have the potential to reduce the social stigma often associated with stress as students are not singled out.

Reviews have concluded that combined environmental (changing organizational practices or altering the physical or psychosocial environments) and individual (e.g. mindfulness training or relaxation techniques) interventions have the greatest potential to alleviate stress in the workplace [28–30]. Research on multi-level school-based stress preventive interventions is limited. We are not aware of any such studies in a high school setting. Therefore, the Healthy High School (HHS) study was developed to promote well-being among high school students in Denmark using combined educational and environmental initiatives. The HHS study included prevention of stress as one of five pathways to higher levels of well-being. The other four pathways were physical activity, meal habits, sleep and sense of community in the school and classroom.

Multi-component interventions are complex to implement [31, 32] and are rarely implemented as intended [33, 34]. High levels of implementation are associated with better intervention outcomes [33, 35]. Process evaluation is needed to understand how and why an intervention succeeded or failed [33, 36–39]. We conducted a thorough process evaluation of educational and environmental initiatives designed to prevent stress in the HHS study. This study aimed to (i) assess dose delivered and dose received, (ii) explore fidelity and intervention satisfaction, (iii) investigate whether the stress preventive initiatives appealed differently to students in different subgroups and (iv) identify barriers and facilitators to implementation.

Materials and methods

The HHS study

The HHS study builds on a sociological framework [40]. We used the Intervention Mapping protocol [41] to plan the intervention, implementation and evaluation of the HHS study in a systematic fashion. The study is based on a thorough needs assessment among the target group, literature reviews, theory and best practice from Danish high schools. The study is registered in Current Controlled Trials (ID: ISRCTN43284296, 28 April 2017) and has been described thoroughly elsewhere [42]. The two-armed cluster-randomized controlled trial included 15 intervention schools and 15 control schools. The intervention was implemented among all first-year high school students during the school year 2016–17. Stress was addressed through three educational activities (curriculum, time management and a smartphone app) and three environmental initiatives (stress policy, half-yearly counselling sessions and annual coursework plans) (Table I). The delivery of the app was delayed due to re-organizations in the mobile app development company. Consequently, only a few students used it and, therefore, the app will not be included in this study. Participating high schools were asked to select a school coordinator for the study (e.g. a teacher or principal). Their main task was to work as local HHS ambassadors including redistributing information about the intervention and the evaluation of the study to school staff and students.

Table I.

Description of the stress preventive initiatives in the HHS study

Initiatives	Description	Delivered by	Timing
Educational initiatives
Curriculum	The curriculum consisted of teacher manuals and curricular activities for first-year students including assignments and reading material. Stress was included in 5 out of 15 lessons: 4 mandatory lessons and 1 optional lesson (in total, 405 min). The stress lessons were designed to change social norms and cognitive factors such as knowledge, awareness and outcome expectancies and planned for two subjects (Social Studies and Introduction to Natural Science).	Teachers	August 2016 to May 2017
Time management initiative	The aim was to introduce students to time management tools. A week before the course, students were asked to record how they spent their time on an hourly basis in a standardized time management worksheet. The course also provided students with information about how to maintain high energy levels throughout the school day. The project group suggested that the course was conducted either in each class separately or for all first-year students collectively in an auditorium.	Student counsellors	September/October 2016
Environmental initiatives
Stress policy	The school management received a policy template and was encouraged to involve the student council, teachers and other relevant participants in developing the policy and to adopt a clear action plan with tasks, responsible persons and a timeline.	School management, student council, teachers and other relevant school staff	August 2016 to May 2017
Half-yearly counselling sessions	The aim was to support student well-being, to identify or prevent potential academic, social and emotional problems among students and to ensure that students got the proper support if needed.	Student counsellors	September/October 2016 and February/March 2017
Annual coursework plan	An overview of the annual workload (placement of e.g. homework and assignments) giving students the opportunity to plan and manage their time realistically. Coursework plans should include dates for handout of assignments, assignment due dates and time and expected amount of time needed to complete the assignments.	Teachers	August/September 2016

Data collection

The design of the process evaluation of the stress preventive initiatives including selection of relevant sources, methods and timing of data collection was guided by a process evaluation protocol [43] and well-recognized conceptual frameworks [33, 38, 39] (Fig. 1). Theoretically, the process evaluation focused on dose delivered, dose received, fidelity and appreciation and explored barriers and facilitators to implementation of the stress preventive initiatives at the 15 intervention schools [33, 38]. We used multiple data sources and mixed methods to provide a multi-faceted process evaluation as described below.

Fig. 1.

Timeline and data collection methods used for the process evaluation of the stress preventive initiatives in the Healthy High School study.

Questionnaires

We used follow-up questionnaire data (May 2017) from students, teachers and school coordinators. The school coordinators answered questions about organizational, physical and social characteristics of the high schools, ongoing health promotion initiatives and organizational capacity to implement such initiatives. The teachers reported how they had implemented the HHS curriculum component, their perceived barriers to the implementation and their appreciation of the curriculum. The students reported their participation in and appreciation of the stress preventive initiatives.

Interviews with students, teachers, school coordinators and student counsellors

We asked school coordinators at all intervention high schools to invite teachers and students to participate in focus group interviews. An urban and a rural high school agreed to participate within the time frame. We conducted two mixed-gender focus group interviews with students (50 min) and teachers (55 min), respectively, about their experiences with and appreciation of the HHS teaching material. Teachers selected students randomly from their classes. The focus group interviews took place in classrooms during school hours. We interviewed 11 of the 15 school coordinators about their reasons for participating in the HHS study, barriers and facilitators for implementation and their appreciation of the stress preventive initiatives. Seven interviews were conducted at the high schools and four by telephone (33–75 min). Five out of nine invited student counsellors agreed to participate in a telephone interview (25 min) about their experiences with the time management course and half-yearly counselling sessions.

Operationalization of process evaluation concepts

Dose delivered (quantitative data): the amount of the stress preventive initiatives delivered to students by teachers, student counsellors and school coordinators at the class and school levels. Information about dose delivered of the environmental initiatives and the time management initiative was not obtained at the class level. Furthermore, it was not possible to estimate the number of stress lessons delivered by teachers at each school and in each school class due to low teacher response rates on these items. Therefore, this information was retrieved from student data: aggregated to the class or/and school levels, an initiative was regarded as delivered if more than half of the students reported having attended the initiative. Dose received (quantitative data): the extent to which students received and engaged in the stress preventive initiatives, e.g. number of attended lessons reported by students at the student, class and school levels. Fidelity (qualitative data): the extent to which the stress preventive initiatives were delivered by teachers’, student counsellors’ and school coordinators’ according to implementation manuals, such as curriculum guidelines. Appreciation (quantitative and qualitative data): satisfaction with the stress preventive initiatives among students, teachers, student counsellors and school coordinators. We examined whether the intervention appealed differently to students according to gender, occupational social class (OSC) and stress level. Table II summarizes operationalization of the included process evaluation concepts in the questionnaires and variables used for characterization of students and high schools.

Table II.

Operationalization of the quantitative process evaluation measures and variables used for characterization of students and high schools

Stress preventive initiatives	Respondents	Measure	Response categories	Explanation for variables used in the analyses and results
Process evaluation measure: dose delivered
Curriculum	Teachers	Teachers were presented with a short description of the HHS curriculum and asked: ‘Did you use the HHS curriculum during the school year 2016–17 among your first-year students?’	‘Yes’, ‘No’, ‘Don’t know’	Dose delivered at teacher level: ‘Yes’
Stress lessons	Students	Students were presented with a short description of each stress lesson and asked if they attended each lesson	‘Yes, definitely’, ‘Yes, I think so’, ‘No, I don’t think so’, ‘No, definitely not’	Dose delivered at class and school level: more than half of students reported ‘Yes, definitely’/‘Yes, I think so’. Dose delivered of stress lessons were categorized as high (4 lessons), medium (2–3 lessons) and low (0–1 lessons) dose
Time management initiative
	School coordinators	During this school year (2016–17): did the high school conduct a course about study skills for first-year students e.g. note taking methods, reading techniques or time management techniques?	‘No’, ‘Yes, among selected first-year students’, ‘Yes, among all first-year students’, ‘Don’t know’	Dose delivered at school level: ‘Yes, among all first-year students’ AND ‘Time management techniques’ selected
		What topics were covered in the study skills course (select all that apply)?	‘Note taking methods’, ‘Reading techniques’, ‘Time management techniques’, ‘Other. Write:’, ‘Don’t know’
	Students	Did you talk about time management techniques in class with a student counsellor?	‘Yes’, ‘Yes, we talked about it, but not with a student counsellor’, ‘No’, ‘Don’t know’	Dose delivered at class level: more than half of students reported ‘Yes’/‘Yes, we talked about it, but not with a student counsellor’ AND ‘Yes’ i.e. the student completed the time management worksheet
		Student were presented with a picture of a time management worksheet and asked: ‘Did you track how you spent your time for a week during this school year?’	‘Yes’, ‘No’, ‘Don’t know’
Stress policy	School coordinators	During this school year (2016–17): did your high school get a new or changed student health and well-being policy?	‘No, we do not have a health and well-being policy’, ‘No, we did not change our health and well-being policy during the school year’, ‘Yes, we made significant changes to our health and well-being policy during the school year’, ‘Yes, we have developed a new health and well-being policy during the school year’	Dose delivered at school level: ‘No, we did not change our health and well-being policy during the school year’/ ‘Yes, we have developed a new health and well-being policy during the school year’/ ‘Yes, we made significant changes to our health and well-being policy during the school year’ AND ‘Stress’ was included in the policy
		What type of issues does the policy cover?	‘Well-being’, ‘Stress’, ‘Alcohol’, ‘Sleep’, ‘Smoking’, ‘Physical activity’, ‘Food and meals’, ‘Loneliness’, ‘Drugs’, ‘Bullying’, ‘Other. Write:’
Half-yearly counselling sessions
	School coordinators	During this school year (2016–17): are the high school offering student counselling for all first-year students?	‘No’, ‘Yes, among selected first-year students’, ‘Yes, once among all first-year students during the school year’, ‘Yes, twice among all first-year students during the school year’, ‘Don’t know’	Dose delivered at school level: ‘Yes, twice among all first-year students during the school year’
	Students	Students were presented with a short description of the aim of the half-yearly student counselling and asked: ‘How many counselling sessions did you attend during this school year’?	‘None’, ‘1’, ‘2’, ‘3 or more’, ‘Don’t’ know’	Dose delivered at class level: more than half of students reported ‘2’
Annual coursework plan
	School coordinators	During this school year (2016–17): does the high school follow any of these principles/rules: Students know dates for when assignments are handed out well in advance (e.g. via an annual cycle of work) Students know assignment due dates well in advance (e.g. via an annual cycle of work) We have a limit on number of assignments per week	‘To a great extent’, ‘To some extent’, ‘To a lesser extent’, ‘Not at all’, ‘Don’t know’	Dose delivered at school level: all three questions were marked as: ‘To a great extent’
	Students	Here are some statements about your assignments:		Dose delivered at class level: more than half of students answered ‘Yes, in all courses’ in both statements about assignments AND ‘Always’ in the question about limit on number of assignments per week
		‘In my class we know when our assignments will be handed out well in advance’	‘Yes, in all courses’, ‘Yes, in most courses’, ‘Yes, but only in some courses’, ‘No’, ‘Don’t know’
		‘In my class we know assignment due dates well in advance’	See above
		‘How often does your teachers respect the limit on assignments per week?’	‘Always’, ‘Often’, ‘Sometimes’, ‘Rarely’, ‘Never’, ‘We don’t have a limit on assignments per week’
Process evaluation measure: dose received
Curriculum	Students	Students were presented with a picture of the cover of the HHS curriculum and asked: ‘Have you been taught the HHS curriculum during this school year?’	‘Don’t know’, ‘No, never’, ‘Yes, but only a few times’, ‘Yes, many times’	Dose received of the HHS curriculum: ‘Yes, but only a few times’/‘Yes, many times’
Stress lessons	Students	Students were presented with a short description of each stress lesson and asked if they attended each lesson	‘Yes, definitely’, ‘Yes, I think so’, ‘No, I don’t think so’, ‘No, definitely not’	Dose received of each stress lesson: ‘Yes, definitely’/‘Yes, I think so’. Dose delivered of stress lessons were categorized as high (4 lessons), medium (2–3 lessons) and low (0-1 lessons)
Time management initiative	Students	Did you talk about time management techniques in class with a student counsellor?	‘Yes’, ‘Yes, we talked about it, but not with a student counsellor’, ‘No’, ‘Don’t know’	Dose received: ‘Yes’/‘Yes, we talked about it, but not with a student counsellor’ AND ‘Yes’ i.e. the student completed the time management worksheet
		Students were presented with a picture of a time management worksheet and asked: ‘Did you track how you spent your time for a week during this school year?’	‘Yes’, ‘No’, ‘Don’t know’
Half-yearly counselling sessions	Students	Students were presented with a short description of the aim of the half-yearly counselling sessions and asked: ‘How many counselling sessions did you attend during this school year’?	‘None’, ‘1’, ‘2’, ‘3 or more’, ‘Don’t’ know’	Dose received: ‘2’/‘3 or more’
Annual coursework plan	Students	Here are some statements about your assignments:		Dose received: students answered ‘Yes, in all courses’ in both statements about assignments AND ‘Always’ in the question about limit on assignments per week
		‘In my class we know when our assignments are handed out well in advance’	‘Yes, in all courses’, ‘Yes, in most courses’, ‘Yes, but only in some courses’, ‘No’, ‘Don’t know’
		‘In my class we know assignment due dates well in advance’	See above
		‘How often does your teachers respect the limit on assignments per week?’	‘Always’, ‘Often’, ‘Sometimes’, ‘Rarely’, ‘Never’, ‘We don’t have a limit on assignments per week’
Process evaluation measure: appreciation
Curriculum	Students	Students were asked to rate how much they liked the HHS curriculum	Rating scale from 0 (worst) to 10 (best)	Average score
	Teachers	Teachers were asked to rate their level of agreement with five statements about the curriculum: ‘I liked that the curriculum was available online’, ‘The curriculum covered official learning goals defined by the Danish Ministry of Education’, ‘The curriculum was difficult to use’, ‘I will definitely use all or some of the curriculum again’, ‘The curriculum was too difficult for students in the first year of high school’	‘Strongly agree’, ‘Agree’, ‘Neither agree nor disagree’, ‘Disagree’, ‘Strongly disagree’	‘Strongly agree/Agree’, ‘Neither agree nor disagree’, ‘Disagree/Strongly disagree’
Time management initiative	Students	Students were asked to rate how much they liked the time management course and exercise, respectively	Rating scale from 0 (worst) to 10 (best)	Average score
Half-yearly counselling sessions	Students	Students were asked to rate how much they liked the half-yearly counselling sessions	Rating scale from 0 (worst) to 10 (best)	Average score
Variables used for characterization of students and high schools
Characterization of students
Gender	Students	Are you a boy or girl?	‘Male’/‘Female’	Male/female
Age	Students	When were you born?	‘Date’/‘Month’/‘Year’	Continuous
OSC	Students	Does your father/mother have a job? What is his/her job title?	Text field	Responses were coded from I (highest) to V. We added a Category VI to include economically inactive parents who receive unemployment benefits, disability pension or other kinds of transfer income. Each student was categorized by the highest-ranking parent into high (I–II), middle (III–IV) and low (V–VI) OSC
Immigrant background	Students	Where was your mother/father/you born?	‘Denmark’, ‘Poland’, ‘Turkey’, ‘Germany’, ‘Iraqis’, ‘Bosnia Herzegovina’, ‘Other. Write’, ‘Don’t know’	Based on the definitions of Statistics Denmark, each student was categorized as being Danish (having at least one parent born in Denmark regardless of own country of birth), a descendant (born in Denmark to both parents born outside Denmark) or an immigrant (born abroad to both parents born outside Denmark)
Perceived stress	Students	The Perceived Stress Scale 10-item version (PSS-10). The PSS-10 assesses the extent to which people find their life unpredictable, uncontrollable and overloaded	5-Point Likert scale ranging from ‘Never’ to ‘Very often’	Scores on the PSS-10 range from 0 to 40 with higher scores indicating higher perceived stress. The PSS-10 was categorized into low (0–13), moderate (14–26) or high (27–40) perceived stress
Contextual factors
School size	School coordinators	Number of students in the school year 2016–17	Text field	Mean number of students per high school
Student/teacher ratio	School coordinators	Number of students in the school year 2016–17	Text field	Number of students in the high school divided by number of teachers in the high school
		Number of teachers in the school year 2016–17	Text field
Student/student counsellor ratio	School coordinators	Number of students in the school year 2016–17	Text field	Number of students in the high school divided by number of school counsellors in the high school
		Number of student counsellors in the school year 2016–17	Text filed
School popularity	School coordinators	Our high school is a popular school which many young people want to attend	‘Strongly agree’, ‘Agree’, ‘Neither agree nor disagree’, ‘Disagree’, ‘Strongly disagree’	‘Strongly agree’/‘Agree’
Implementation capacity
Well-functioning student council	School coordinators	The student council at our high school is well functioning	‘Strongly agree’, ‘Agree’, ‘Neither agree nor disagree’, ‘Disagree’, ‘Strongly disagree’, ‘We don’t have a student council’	‘Strongly agree’/‘Agree’
Team working with health promotion and well-being	School coordinators	In this school year (2016–17): does your high school have a team working with health promotion and well-being?	‘Yes’, ‘No’, ‘Don’t know’	‘Yes’

Data analysis

Descriptive statistics were used to assess dose delivered and dose received of the stress preventive initiatives. We used chi-square tests and/or the one-way analysis of variance to assess whether the initiatives appealed differently to students in different subgroups and to explore differences between (i) students with and without data at follow-up and (ii) high schools with and without teacher data. We used SAS 9.4 (SAS Institute, Inc., Cary, NC, USA) for the statistical analyses selecting a 0.05 significance level a priori.

All interviews were digitally recorded and transcribed verbatim. The qualitative data were imported into NVivo11 (NVivo qualitative data analysis software; QSR International Pty Ltd Version 11, 2015). C.T.B. analysed each interview and generated codes both from the topics in the interview guides (the process evaluation concepts) and iteratively from the data. The codes were discussed and refined by the research group.

Ethics approval and consent to participate

The HHS study adheres to all Danish ethical standards and has been approved by the Danish Data Protection Agency (J. No. 2015-57-0008). The Regional Scientific Ethical Committee, the Capital Region of Denmark, reviewed the HHS study and concluded that formal ethical approval was not required (J. No. 16018722). Written information was sent to principals, teachers and student councils at all invited high schools explaining the implications of participating in the study. For all data collection methods, participants were informed that participation was voluntary that their information would be used for research purpose only and treated confidentially.

Results

Study population

This study included 2047 students at follow-up of whom 1561 answered the questionnaire (response rate, 76.3%). The average age was 16.3 years, most students were females (63.1%) and of Danish origin (88.4%). Half of students were categorized as high OSC (Table III). All 15 school coordinators completed the follow-up questionnaire. They were mostly part of the school management (e.g. principles and pedagogical administrators) and had been employed at the high school for 5 or more years (data not shown). We invited all 463 teachers from the 15 intervention high schools to answer the questionnaire about the HHS curriculum, 108 of whom agreed to participate (response rate, 23.4%). Ten of 15 high schools reported teacher data; range, 4–29 responses per high school. The teachers taught various subjects, e.g. Danish, biology and history (data not shown). The average high school size was 563 students. The mean number of students per student counsellor and teachers was 165 and 8.8, respectively (Table III).

Table III.

Baseline characteristics of students and high schools in the process evaluation study

		Students included in the process evaluation study (n = 1561)	Students not included in the process evaluation studya (n = 486)	P-value
Student characteristics (individual level)
Females, % (n)		62.6 (951)	59.1 (276)	0.17
Missing, n		42	19
Age, mean (SD)		16.2 (0.9)	16.2 (0.8)	0.25
Missing, n		42	19
OSC, % (n)				<0.01
High social class (I + II)		49.7 (776)	42.4 (198)
Middle social class (III + IV)		35.0 (547)	30.6 (146)
Low social class (V + VI)		10.8 (169)	18.8 (88)
Unclassifiable, % (n)		4.4 (69)	8.1 (38)
Immigrant background, % (n)				<0.01
Danish origin		88.4 (1342)	69.6 (325)
Descendant		9.3 (141)	26.1 (122)
Immigrant		2.2 (33)	3.9 (18)
Missing, n		43	19
School characteristics (school level)	All high schools (n = 15)	High schools with teacher datab (n = 10)	High schools with no teacher datab (n = 5)	P-value
Perceived stress, % (n)				0.00
Low perceived stress (0–13)		54.0 (794)	52.6 (241)
Moderate perceived stress (14–26)		43.2 (635)	40.8 (187)
High perceived stress (27–40)		2.8 (41)	6.6 (30)
Missing, n		91	28
Number of students per school, mean (SD)	563 (274.4)	486.1 (294.2)	716.8 (156.6)	0.13
Number of students per counsellor, mean (SD)	165.0 (60.8)	159.5 (64.4)	176 (58.1)	0.64
Number of students per teacher, mean (SD)	8.8 (1.9)	8.8 (2.3)	8.8 (0.6)	0.99
The school is a popular school, n (%)	6 (40.0)	4 (40.0)	2 (40.0)	0.76
The school has a well-functioning student council, n (%)	10 (66.7)	10 (70.0)	3 (60.0)	0.62
The school has a team working with health promotion and well-being, n (%)	9 (60.0)	6 (60.0)	3 (60.0)	1.00

^aStudents who were invited to participate in follow-up but did not answer the questionnaire.

^bComparison of high schools with and without questionnaire data from teachers.

Dose delivered

Most high schools (13/15) delivered the time management course for all first-year students. Around half of the high schools delivered half-yearly counselling sessions for all first-year students (7/15), organized an annual coursework plan for all first-year classes (8/15) and had a stress policy at follow-up (7/15). Of the 108 responding teachers, 39.7 delivered the HHS curriculum (school range, 0.0–66.7%). Based on student responses, a high dose (4 of the HHS stress lessons) was delivered at 3 high schools and in 11 classes. The average dose delivered reported by students was 1.4 stress lessons at the school level and class level (range, 0–4), respectively (Table IV).

Table IV.

Dose delivered and dose received of the stress prevention initiatives in the HHS study

	Dose delivered (reported by students, teachers or school coordinators)		Dose received (reported by students)
Stress preventive initiatives	School level	Class level	Student levela
Curriculum	39.7%a	34/79	41% (SR: 10–62%, CR: 0–100%)
Stress lessons
High dose (4 lessons)	3/14	11/79	13% (SR: 0–58%, CR: 0–71%0
Medium dose (2–3 lessons)	3/14	19/79	34% (SR: 12–71%, CR: 0–100%)
Low dose (0–1 lessons)	8/14	49/79	53% (SR: 2–88%, CR: 0–100%)
Number of lessons, mean (SD)	1.4 (1.8) (0–4)	1.4 (1.5) (0–4)	1.5 (1.0) (SR: 0.5–3.4, CR: 0.2–3.5)
Time management initiativeb	—	26/79	27% (SR: 0–68%, CR: 0–83%)
Time management course	13/15	31/79	44% (SR: 19–72%, CR: 0–87%)
Time management exercise	—	46/79	49% (SR: 0–95%, CR: 0–100%)
Stress policy
Had a stress policy at first follow-up	7/15	—	—
Developed a stress policy during the school year 2016–17	2/15	—	—
Half-yearly counselling sessions	7/15	31/79	43% (SR: 16–80%, CR: 0–100%)
Annual coursework planc	8/15	1/79	2% (SR: 0–7%, CR: 0–50%)
Dates for hand out	12/15	2/79	10% (SR: 3–27%, CR: 0–60%)
Assignment due dates and time	13/15	12/79	35% (SR: 19–51%, CR: 0.0–100%)
Time use for assignments	9/15	1/79	9% (SR: 0–24%, CR: 0–50%)

^aExpressed as mean percentage; calculated as the sum of the percentage of students/teachers at each school divided by the number of schools. SR, school range; CR, class range.

^bHave implemented the time management course and the exercise.

^cHave implemented all elements of the annual coursework plan.

Dose received

Around one-fourth of students (27%) participated in the time management initiative (the course and the time management exercise). Students received the time management exercise to a higher degree than the actual course (49% versus 44%). Four out of 10 students received the half-yearly counselling sessions. The lowest dose received was found for the annual coursework plan, which only 2% of students received. Forty-one percent of students were taught the HHS curriculum during the school year, and 13% of students received a high dose of the HHS stress lessons. On average students received 1.5 stress lessons; school range, 0.5–3.4; class range, 0.2–3.5. The level of dose received for all initiatives varied by schools and school classes (Table IV).

Fidelity

The qualitative data illustrated differences in fidelity of implementation across high schools. Student and teacher focus group interviews indicated that teachers at the two high schools adhered to curriculum guidelines.

Teacher: In the Introduction to Natural Science course, I guess we just used it as it was. We did exactly what it said.

Some teachers used the HHS website for teaching the curriculum while others handed out copies of the material and used their usual platforms e.g. Google Docs or Lectio (school intranet). However, most of the school coordinators stated that the HHS curriculum was implemented in very different ways, e.g. at some high schools, the curriculum was taught in Biology and not in the Introduction to Natural Science course as prescribed. At other high schools, the curriculum was implemented by school coordinators in class meetings (time devoted to classroom discussion) to ease reduce teachers’ workload (low fidelity).

Interviews with student counsellors revealed that the half-yearly counselling sessions and the time management course were implemented at most high schools but in different ways. The time management course was primarily conducted by student counsellors in each class separately consistent to the implementation manual (high fidelity). They used the HHS material but added extra material or exercises such as Kahoot! (a game-based platform). It was novel for the student counsellors to teach about time management:

We usually talk more generally about study habits and things like that, but not quite as much about planning their time.

The interviewed student counsellors invited all first-year students to a counselling session in the beginning of the school year. This was the usual practice at all high schools and done prior to the HHS study. However, the HHS sessions focused more on well-being than standard practice. The interviews also indicated that only few high schools organized a follow-up meeting for all first-year students in a systematic fashion as prescribed (low fidelity). At most high schools, student counsellors only met students again if the student had high absenteeism, a teacher referred the student or the student self-referred (standard practice). Fidelity of implementation of the stress policy and the annual coursework plans were not explicitly discussed during interviews.

Appreciation

Differential appreciation

The mean appreciation score for all stress preventive initiatives was highest among students reporting low stress levels (Table V).

Table V.

Students’ appreciation of the stress preventive initiatives in the HHS study by subgroups on a scale from 0 to 10

	Curriculum, mean (SD)	P-value	Time management course, mean (SD)	P-value	Time management exercise, mean (SD)	P-value	Half-yearly counselling sessions, mean (SD)	P-value
All students	5.59 (2.26)		5.59 (2.28)		4.98 (2.57)		5.93 (2.26)
Gender		0.20		0.15		0.23		0.21
Females	5.53 (2.16)		5.50 (2.21)		5.10 (2.53)		5.88 (2.22)
Males	5.72 (2.38)		5.77 (2.40)		4.84 (2.66)		6.10 (2.37)
OSC		0.47		0.42		0.29		0.64
High (I + II)	5.64 (2.22)		5.49 (2.39)		5.13 (2.51)		5.94 (2.19)
Medium (III + IV)	5.57 (2.22)		5.70 (2.07)		4.88 (2.65)		5.81 (2.35)
Low (V + VI)	5.37 (2.59)		5.82 (2.27)		4.71 (2.64)		6.02 (2.26)
Stress level at baseline		P < 0.01		P < 0.01		0.02		0.00
High	4.72 (2.68)		5.14 (2.19)		4.13 (2.17)		5.11 (2.14)
Moderate	5.29 (2.13)		5.11 (2.29)		4.74 (2.56)		5.72 (2.31)
Low	5.86 (2.28)		5.97 (2.21)		5.25 (2.61)		6.19 (2.22)

Time management initiative

Among students, the mean appreciation score for the time management course and the related time management exercise was 5.59 and 4.98 (Table V). Overall, the interviewed student counsellors liked the stress preventive initiatives and the HHS material. They especially expressed enthusiasm for the time management exercise as it gave them valuable information about students’ daily tasks and prioritizations:

Well, I think it worked quite well. And I think it gave us some interesting information about how they actually use their time, what they prioritise. For example, it was surprising how many don’t get much sleep at night, how much time many of them spend on part-time jobs outside of school and things like that […].

Several counsellors had previously used a similar exercise as a stress management tool in their counselling but not systematically among all students. Counsellors expressed that it was time consuming to complete but an eye-opener for students. It initiated good discussions about e.g. time-consumers, delaying tactics and sleep:

I could hear that some of them were surprised by how much time they spent on nothing in particular. And some were surprised by how little they slept, or how much they slept.

Some counsellors stated that it would be better to schedule the course later (either in spring or in the second year of high school) as students did not realize the relevance of the course at the beginning of high school:

But then again, at that time they hadn’t experienced a lot of pressure yet (…)

Half-yearly counselling sessions

Among all stress preventive initiatives, the students especially appreciated the half-yearly counselling sessions (mean appreciation score = 5.93) (Table V). All interviewed student counsellors also appreciated the half-yearly counselling sessions. They acknowledged the first session as an occasion to create a safe space for students to discuss sensitive events, while they had contrary views on the relevance of a second counselling session for all students. Some counsellors stated that it would be too time consuming, and that they would reach the students in need anyway. Others indicated that a second session would put them in touch with students who despite doing well in school and a happy appearance are facing stress, loneliness or other challenges. Counsellors emphasized that it was important that they were responsible for the counselling sessions as it was their main competence. They explained that teachers often forget to follow-up with student counsellors or psychologists if they are the ones conducting the counselling sessions.

HHS curriculum

Of the 50 teachers who used the HHS curriculum, 38.1% indicated that they would use all or some of the material again, and 39.5% liked that the material was available online (Fig. 2). More than half of the teachers (58.5%) reported that the curriculum covered official ministerial learning goals. Eight out of 10 teachers disagreed that the material was too difficult for first-year students and that the material was difficult to use. Among students, the mean appreciation score for the HHS stress lessons was 5.59 (Table V). Interviews with students and teachers indicated that it was particularly experiments on sleep and stress that made a big impression on students.

Fig. 2.

Teachers’ appreciation of the HHS curriculum (percentages).

Boy: Well, it was a new way to learn, because it wasn’t […]. You don’t just sit and read about what others have experienced. You are to help figure it out yourself, so you are involved in a more active way.

The stress experiments aimed to measure pulse, blood pressure and pupil size before and after exposure to a stressor such as music from a horror movie or shouts/screams. However, students and teachers suggested that the curriculum was updated with new and less predictable stressors including concentration tests and virtual reality games.

Teacher: (…) So they [students] had the idea of using a game instead. They got one of those virtual reality games, and the person they had to do the measurements on, they had them play this game, which has some crazy, startling effects. That worked really well. They thought about it and took it a step further in a cool way. So it’s just about finetuning it a bit to make it possible.

Students expressed enthusiasm for the novelty of the HHS material. It made studying more fun and interesting:

Boy: Well, it wasn’t just about looking at a book from ten years ago. It was something new, which was created within the last couple of years. And it was a bit more fun than looking at a statistic from 2008.

The interviewed teachers at the two high schools disagreed about the academic level of the HHS curriculum. In agreement with teacher questionnaire data on appreciation, teachers at one high school appreciated the material overall. They liked the extensive focus on methods, methodology critique and experiments in the curriculum. They found it important that students learn to be critical readers and know about e.g. bias and source of error. However, they found the material a bit too ambitious timewise. They suggested we highlighted the three most important learning goals of each lesson, so teachers know what to prioritize:

But the idea behind it, I would say, it’s like one of those ideal lessons that you’re asked to do in the teacher training programme. And the learning objectives and things like that have been written down, so it was all, you might say, perfectly set out in writing and all that. But as I said, the scope might not have been quite in line with what you can realistically achieve in an hour.

At the other high school, especially the Natural Science teachers found the level too low for high school-level reading:

Yes, it was a bit humiliating to have to stand there and pretend to be an authority who teaches science. And turn up with SUCH a flimsy foundation for talking about stress (…). It’s totally … I mean, it was so flimsy that it was… humiliating!

At both high schools, the Natural Science teachers stated that it does not make sense to talk about stress in the beginning of the first high school year. Students need to know about the nervous and endocrine systems but are not introduced to these complicated topics before the second year of high school.

First of all, it’s incredibly difficult to talk about stress from a biological point of view in the first year. I mean, you can’t, really… You’d have to get to B-level to do that, where we have some material about the nervous system and hormones. Then it would make sense, then it might even be an eye-opener for them: ‘Okay, so this is what we’re talking about’ and ‘That’s what explains all the symptoms you get when you suffer from stress’.

All teachers agreed on the relevance of talking about time management techniques with the first-year students e.g. how to plan and prioritize daily tasks:

You can talk about being busy, and you can talk about planning, and you can talk about focus: ‘How can I learn how to structure my day better and prioritise the right things?’ That’s all fine, and it makes good sense for the first-year students to think about these things.

On the contrary, students found it highly relevant to learn about stress in high school. They stated that they gained new knowledge about stress, and that the HHS material raised their awareness of own and classmates’ behaviour.

Boy: But it was also pretty fun, I mean it was something we haven’t talked much about before. It’s not like I ask my friends ‘How much do you sleep in the evening?’ Or at night. And it was kind of interesting to see what the others did. For Marie and Otto for example, they sleep maybe four hours a night, and then there were some who slept 10 hours, so that’s a big difference.

Barriers and facilitators

Based on the qualitative interviews, we identified five main barriers for implementation: (i) Timing: high schools received the invitation to participate in the HHS study (April/May 2016) and the intervention material (August 2016) too late. One school coordinator explained, that they decided the budget for the upcoming school year in January/February making it difficult to allocate resources for the project. School coordinators found it challenging to involve teachers in decision-making processes during exam periods (May–June) as they are busy and unavailable. Also, teachers had already planned their courses. Ideally, they should have known about the curriculum in April at the latest. (ii) External ministerial demands: since 2016, high schools have faced spending cuts of 2% annually resulting in e.g. dismissal of teachers. At the same time, high schools were preparing implementation of a new comprehensive education reform. School coordinators were cautious about giving teachers too many extra tasks as they were under a huge time pressure. (iii) Additional tasks: some teachers stated that the HHS material did not cover official learning goals, and the material was perceived as an extra task rather than as a substitute. The timetable is very tight for first-year students, and teachers do not have time to teach lessons in addition to the compulsory curriculum. Furthermore, Natural Science teachers conduct prolonged courses about a specific topic e.g. earthquakes, and, therefore, it was difficult to fit in a few lessons about stress. (iv) Competing interests: the project fell into oblivion during the intervention period as it drowned in other mandatory tasks such as teaching and preparation of implementation of the upcoming education reform. (v) Project fatigue: high schools are a popular setting for both health promotion and other projects, and there was a general project fatigue among interviewed participants.

We identified three main facilitators for implementation: (i) Comprehensive guidelines and room for adaptation: the detailed curriculum guidelines including suggestions for reading material and exercises and reference to official learning goals facilitated teachers’ implementation of the HHS curriculum. Especially teachers who normally do not teach about stress and health (e.g. physics teachers or chemistry teachers) found the guidelines useful. Interviewed teachers and school coordinators also mentioned that the possibility for adaptation facilitated implementation e.g. that the HHS curriculum could be used in combination with teachers own material. (ii) Shared values: implementation was eased when the high schools’ set of values were compatible with the HHS study, and/or the intervention material could fit in with existing routines, practices or policies. (iii) Scientific research: school coordinators valued that the HHS study was based on theory and evidence.

Attrition analysis

Students who did not answer the follow-up questionnaire (n = 486) were more likely to be descendants (26.1% versus 9.3%, P < 0.01), categorized as low OSC (18.8% versus 10.8%, P < 0.01) and experience high levels of stress (6.6% versus 2.8%, P = 0.00) compared with responding students (n = 1561). There were no significant differences between high schools with and without teacher responses (Table III).

Discussion

Intervention dose and fidelity

This is one of the first thorough process evaluations of a school-based stress preventive intervention combining educational and environmental initiatives. Dose delivered of the HHS stress preventive initiatives varied: half of the intervention schools delivered the environmental initiatives (stress policy, half-yearly counselling sessions and annual coursework plan), while the dosage of the educational initiatives differed according to intervention provider. Student counsellors delivered a high dose of the time management course while teachers delivered a low dose of the HHS stress lessons. In line with other studies, students reported lower levels of implementation compared with school staff [44, 45]. The level of implementation and adaptation varied by schools and school classes, e.g. contrary to the implementation manuals, school coordinators at some high schools delivered the HHS curriculum instead of teachers.

Appreciation

Overall, interviewed student counsellors, school coordinators and students appreciated the stress preventive initiatives. The students particularly appreciated the half-yearly counselling sessions (mean appreciation score = 5.93). For the curricular activities, especially the stress experiments made an impression on the students. Students also preferred hands-on activities to e.g. discussion topics in previous school-based mental health interventions [46, 47]. The focus group interviews indicated that the teacher appreciation varied, with highly engaged and enthusiastic teachers at one high school and teachers disliking the curriculum and focus on stress at the other high school. Displeased teachers had not been involved in the decision to participate in the HHS study, even though we encouraged school managements to include school staff and students in this process. A feeling of ownership among implementers including shared decision-making is essential for successful implementation [33, 35]. School personnel who are pressured by the school management to deliver new programmes seldom implement them very well [33].

The HHS intervention was designed as a universal school-based stress preventive intervention. Thus, the initiatives aimed to reach and appeal to all students. In agreement with this, we found no gender and OSC differences in appreciation scores. However, the highest appreciation score across all initiatives was observed among those reporting low baseline stress levels. Highly stressed students probably need more extensive psychological treatment for stress which was not included in the HHS study.

Barriers and facilitators

Barriers of implementation included late delivery of intervention material, time issues especially due to external ministerial demands, declining engagement of school staff over time, perception of the curriculum as an extra task and a general project fatigue among school staff. Facilitators included flexibility in use of the intervention material, compatibility between research and educational objectives and participation in a research project. Consistent to our findings, previous studies emphasized the importance of accounting for time aspects other than preparation time e.g. time related to contextual factors such as external ministerial demands [48, 49]. One way to facilitate implementation is to develop intervention material that is regarded as helpful to an existing task [50, 51]. We involved high school teachers in the development of the HHS curricular activities to ensure that they adhered to official learning goals and could be integrated into teachers’ existing obligations. Teachers confirmed that the curriculum covered official learning goals in the questionnaires, while interviewed teachers expressed that they viewed the curricular activities as an extra obligation. The curricular activities were divided among several subjects to reduce teacher workload but spreading the curricular activities across several subjects impeded implementation at some high schools. Teachers and school coordinators suggested developing complete course programmes to be used for exams. The detailed curriculum guidelines were helpful for some teachers while others felt they made less room for adaptation and creativity. In line with this study, ‘the Boost study’ found that some teachers preferred specification of overall learning objectives rather than receiving a detailed guideline [48]. This is also supported by Rogers’ Diffusion of Innovation Theory which describe how teachers’ perceptions of the relative advantages or complexity of an intervention in relation to the existing curriculum influence implementation [52]. The HHS material allowed for adaptation, however, most teachers thought they should follow the HHS guidelines strictly. This should be communicated more explicitly in future material.

Strengths and limitations

Study strengths included use of a systematic process evaluation protocol [43], multiple data sources and data collection methods, a large student sample, high response rates among students and school coordinators, assessment of several process measures covering different aspects of the implementation process and assessment of implementation at the individual, class, and school levels. Finally, the analyses were conducted with no prior knowledge of intervention effectiveness.

The low response rates among teachers challenged the generalizability of questionnaire findings as it might be a certain group of teachers who responded e.g. those most engaged in the intervention. We, however, used the best available data to inform dose delivered, namely aggregated student data. A selected group of students completed the follow-up questionnaire, and the study sample may not be fully representative. The measures used to assess implementation of the stress preventive initiatives were developed specifically for the HHS intervention. We conducted a brief pilot testing of the new items resulting in minor revisions, but we did not have time to do a comprehensive validation of the questionnaire. To increase the chances of successful implementation, we developed initiatives that could be integrated into the high schools’ standard practices. Therefore, it was not required that students knew that the teaching and counselling originated from the HHS study. This approach challenged our process evaluation as it might have been difficult for students to distinguish between HHS initiatives and standard practices when responding to questionnaires and interviews. The HHS research group was involved in both the design and evaluation of the intervention. We perceive it as an advantage as our thorough knowledge of the intervention made it easier to ask more detailed questions related to implementation during interviews [53]. To minimize social desirability bias and to encourage participants to share both positive and negative experiences, we emphasized that the questionnaire and interviews were not a test, and that there were no right or wrong answers. We started each interview with open-ended questions and did not perceive that interviewed participants withheld any information during the interviews as reflected by the displeased teachers at one high school. Finally, we would have liked to conduct focus group interviews at more high schools to get a more nuanced picture of contextual differences in implementation. However, the two high schools represented both the positive and negative end of the appreciation score.

Implications for research and practice

In this study, we have reported results from the process evaluation of the stress preventive initiatives. An important next step for the HHS study is to evaluate the intervention effect on the primary (well-being) and secondary outcomes (stress, sleep, sense of community, PA and meal habits). Moreover, analyses of the process evaluation data on the other intervention components will help clarify the implementation process of the entire HHS intervention. These results will be reported in future publications.

Future studies should develop validated measures of student reported dose as it seems easier to collect data among students compared with teachers. Strategies should be explored to increase teachers’ response rates. We tried to increase teacher response rates by emphasising the importance of all teachers answering the questionnaire regardless of their involvement in implementation. We also offered a pair of movie tickets to two randomly selected responding teachers.

Teachers expressed that first-year students need to know more about basic biology before being taught about the complex nature of stress. Teachers and student counsellors, however, agreed that it was relevant to teach students time management skills, and that these sessions could easily be included in student counsellors’ yearly introduction to study techniques. The implementation process of the educational initiatives may have been easier for student counsellors compared with teachers as the time management course was a one-time event and consistent with their standard practices. Reinke et al. [54] found that teachers viewed school psychologists as having the primary role in most aspects of mental health service delivery in the school. A study by Frydenberg et al. [55] indicated that delivery of a coping skills programme to secondary school students was most successful when teachers and psychologists worked together. This indicates that future studies could benefit from giving counsellors a more prominent role in school-based stress preventive interventions as their role and time at school is most often dedicated to these tasks.

Future school-based interventions should consider including a component targeting students with high stress levels.

It seems to be more challenging to do intervention research in high schools compared with primary schools due to higher academic demands and teachers having a stronger professional identity. In recent years, Danish high schools have been expected to implement an ever-increasing number of new initiatives resulting in project fatigue. Health promotion in high schools appears to be a secondary priority over academic attainment. It is crucial that teachers perceive the interventions as highly relevant to educational and learning objectives and consistent with school priorities.