A translational approach to characterization and measurement of health-promoting school ethos

Summary

A health promoting schools (HPS) approach is hypothesized to influence student health and wellbeing by promoting a ‘school ethos’ that reflects the physical environment, social relations, organisational structure, policies and practices within schools. This complex set of factors makes health promoting school ethos (HPSE) challenging to define and measure. This work sought to theorise, develop and pilot a measure of HPSE as the context for implementation of HPS initiatives. We used a multi-method, iterative process to identify relevant HPSE concepts through triangulation of conceptual literature, existing tools and the tacit knowledge of school stakeholders. The HPSE measurement tool was administered to 18 elementary schools through a principal and teacher survey and an environmental assessment, followed by the development of HPSE scores for each school. Testing for internal consistency of items was used to examine theorized concepts, and scores for each school are summarised. HPSE included eight conceptual dimensions with internal consistency ranging from α = 0.60 to α = 0.87. Total HPSE scores across schools (N = 18) ranged from 1 to 8 (mean = 3.94, SD = 2.1), with 28–65% of schools reporting ‘high’ on respective HPSE dimensions. Schools included a heterogeneous mixture of HPSE scores, particularly across different dimensions. Our novel approach to tool development allowed us to conceptualize HPSE using a flexible process comprising different types and sources of evidence. The HPSE tool holds potential for identification and measurement of critical components of different school context as it relates to HPS.

INTRODUCTION

Schools are an important intervention setting to improve the future health and wellbeing of children through enhancing learning, providing social support and establishing lifelong healthy habits (Baranowski et al., 2000; Bonell et al., 2014). This capacity has been recognized internationally through recommendations that encourage a health promoting schools (HPS) approach (International Union of Health Promotion and Education, 2009; World Health Organization, 2012). In Canada, HPS is often referred to as ‘Comprehensive School Health’ and focuses on four interrelated principles of action: Social and Physical Environment, Teaching and Learning, Partnerships and Services, and Healthy School Policy (Pan-Canadian Joint Consortium for School Health, 2013). Regardless of the term, HPS is hypothesized to influence student health and well-being by facilitating improvements in the physical environment, social relations, organisational structure, policies and practices within the school that in turn support the health and wellbeing of children (International Union of Health Promotion and Education, 2009; World Health Organization, 2012). An HPS approach uses a broad ecological model, with a focus on creating school and community environments to support health behaviours of students, thereby influencing health and learning outcomes (World Health Organization, 2012). While HPS interventions hold a great deal of promise for supporting the lifelong development of healthy behaviours in children (Lister-Sharp et al., 1999; Veugelers and Schwartz, 2010; McIsaac et al., 2015), challenges persist in their implementation and evaluation (Bierman, 2002; Butler et al., 2008; Keshavarz et al., 2010; Kremser, 2011).

Complexity and implementation of the health promoting schools approach: the role of school ethos

The implementation of an HPS approach is complex as it targets a range of factors that vary according to the needs or capacity of each school (Butler et al., 2008; Keshavarz et al., 2010). Additionally, implementation occurs gradually over time and school practices might develop, innovate or adapt in response to specific policies or contextual factors (Lee, 2009; McIsaac et al., 2017). As a result, schools adopting an HPS approach may not completely fulfil all core principles of the approach. Health promoting practices may also exist in schools that do not formally adhere to an HPS approach due to the traditional focus on health and physical education curricula in schools (Keshavarz Mohammadi et al., 2010). Consequently, the evaluation of HPS interventions and measurement of school context is a current challenge (Deschesnes et al., 2003).

A school’s ethos represents one possible way to characterize the combination of factors that make up the context of a school; and although understood as being essential for HPS (Rowling and Samdal, 2011; Samdal and Rowling, 2011), there is a lack of published research on the role of school ethos in HPS approaches. School ethos has been described as the various physical and social structures that shape a school’s context including, administrative support and leadership, staff support, school connectedness, morale and stability, and financial and human resources (Parcel et al., 2003). However, evidence defining and operationalizing a specific school ethos important for HPS initiatives, or a ‘health promoting school ethos’ (HPSE) is not yet clear (Hoy et al., 1991, 1992). For this project, and in collaboration with knowledge users, school ethos was conceptualized to represent an entire spectrum of structures and processes that collectively represent the context of a given school. HPSE was best conceptualized for this collaborative work as an aspect of school context that could help us to better understand how HPS initiatives might translate into improved student wellbeing.

Health promoting school ethos: the need for theorising and measurement

The measurement of contextual aspects and connection to health outcomes in existing school ethos tools and literature is sparse, and the link between measurement items and theoretical components is often unclear (Hoy et al., 1991; Hoy and Tarter, 1992; Higgins-D’Alessandro and Sadh, 1998; Hart et al., 2000; Thapa et al., 2013; Reid, et al., 2015). Measurement of an HPSE in the context of HPS is more complex as its manifestation is based on core principles of action unique to an HPS approach (Social and Physical Environment, Teaching and Learning, Partnerships and Services and Healthy School Policy). This inherent complexity may not be well suited to traditional approaches to measurement development. Current processes for developing theory and measures for intervention evaluation are often procedurally rigid and inductive, typically starting with literature review to identify important concepts or theory then operationalizing and measuring predictive factors and outcomes (Armstrong et al., 2006; LaRocca et al., 2012). However, these methods may have limited success in accurately characterizing the role of varying contexts in which the intervention is implemented (i.e. the health promoting ethos within a school), suggesting that a more nuanced process driven approach may be warranted. Alternative methods of theory development and evaluation allow for flexibility in the development of theory and measurement tools to accommodate contextual considerations for programs or initiatives, while maintaining scientific rigour (Pawson et al., 2005; Graham et al., 2006; Kothari et al., 2012; Pawson, 2013). This includes prioritizing a sound theoretical foundation, documentation of processes, and greater transparency and details in reporting results (Pawson et al., 2005; Pawson, 2013).

PURPOSE OF RESEARCH

To address the gaps in research related to how we develop measurement tools to accommodate for context in HPSs, we used a translational (i.e. process driven) method to develop and explore a measure of HPSE with the potential to explain differences in student health and wellbeing across heterogeneous school contexts.

METHODS

While tool development procedures typically begin with a conceptual framework from the literature or by adapting an existing tool, the relevance of such tools to the actual context of schools is a critically important consideration for research (Graham et al., 2006), particularly in relation to population health interventions. We, therefore, applied a translational model of knowledge to action (multi-source model with knowledge funnel) (Rockett et al., 1995) to the characterization and tool development by triangulating theoretical components, existing HPS tools and the tacit knowledge of project knowledge users. We used a three-phase, multi-method, iterative process to develop and assess an HPSE (Figure 1). Based on the conceptual framework and measurement items derived, a measurement tool was piloted and delivered to all participating schools.

Fig. 1:

Procedure for development of Health Promoting School Ethos Measurement Tool.

Sample and setting

Our study included a teacher, principal and observational audit across 18 schools that represent a rural school board in Nova Scotia (Canada) with a population of approximately 60 000 people. This school board was involved in a provincial HPS initiative, with 10 schools having adopted the HPS approach at the time of data collection. However, reflective of the real-world nature of this study, health-promoting activities were present in all 18 schools, as a result of mandatory nutrition policy and school health curricula and an emphasis on afterschool physical activities and mental wellbeing across the province. Throughout the project we used principles of integrated knowledge translation to ensure the research was conducted according to stakeholder knowledge needs so that the results would help to inform policy and practice (Bowen and Graham, 2013). Establishing a research advisory committee and communicating regularly with knowledge users to guide the development of the research methods, instrument design, data collection and dissemination strategies addressed these potential needs.

Procedure for HPSE characterization and measurement

The three phases applied in this study were (1) characterization and development of the measurement tool, (2) administration to schools and (3) calculation of a score (see Figure 1).

Phase 1: Development of HPSE measurement tool

Through literature review, tool identification and consultations with schools and subject expert tacit knowledge, we identified potential theoretical components relevant to a supportive school ethos or a health promoting school. Most relevant evidence was prioritised to ensure that theoretical concepts were based on current conceptualisations of HPS and we applied an iterative approach to refine our approach to tool development. A collection of HPS measurement tools were also identified by academic and non-academic partners, and used as a source of measurement items. Tools in active use that held conceptual coherence with HPS used in this project were also prioritised. Finally, engagement of project knowledge users (i.e. schools) was a key element of this project. As we reviewed literature and developed the tool, we regularly consulted with stakeholders involved with supporting the implementation of HPS at a board-level to ensure meaningfulness of the data being collected. We also consulted with school principals to inquire about their perspectives on what contributes to school ethos to enable their school to support health and wellbeing.

The evidence generated from these three evidence sources were triangulated to co-create the conceptual framework. Theory-based literature was reviewed for relevant constructs, which were screened and discussed by the lead authors (T.L.P. and J.L.M.). These concepts were presented to the advisory committee and feedback was sought to identify potential missing concepts relevant to them. The items from existing measurement tools were then examined and cross-referenced with theoretical concepts and grouped by the lead authors (T.L.P. and J.L.M.) with advice from the scientific team (K.S., S.K. and S.F.L.K.) and disagreements were discussed (Creswell and Clark, 2007). Items were reviewed again by team and redundant items were removed. Concepts provided from knowledge users not included in existing measurement items were added to ensure relevance of tool. The tool was pilot tested in one school by one of the lead authors and refined following discussion by the team.

Phase 2: Administer HPSE measurement tool

Data collection across 18 school occurred in the spring of 2014 following ethics approval from the Dalhousie Health Research Ethics Board and permission from the participating school board. We collected information on HPSE through surveys from school leaders and teachers and an audit of the school environment by a trained observer allowing for a rich source of data for each concept. The audit was completed through a ‘walk-around’ of the school following guided plan that related to specific aspects of the school related to our measure of HPSE. This audit was completed with support of a school staff member and included photographs to capture physical features and responses to questions related to various aspects of HPSE (e.g. resources and promotion for healthy eating, access and availability to physical activity, safety and accessibility of the school). Photos were used to provide an objective assessment of relevant concepts using an assessment tool (Supplementary Appendix S1) to rate the presence and degree of a particular concept (scale from 1 to 5) from each photo (J.K. and T.L.P.).

Phase 3: Development of HPSE score

Relative scores were created for each dimension of the conceptual framework of HPSE, and were tested for internal consistency. To allow for flexibility in the conceptual development of HPSE without excluding constructs at this early stage, a liberal Cronbach’s Alpha was chosen. Therefore, when internal consistency was < 0.6 or poor to unacceptable, similar indicators were combined until the alpha was  > = 0.6. Indicators for constructs were calculated as the means of items for each school, which were then recoded as being above (=1) or below (=0) the median for the indicator score. Finally, indicator scores were summed to create an overall HPSE score.

RESULTS

HPSE theoretical constructs and measurement

The final HPSE tool included measurement items developed from the triangulation of conceptual literature regarding school ethos and HPS (Supplementary Appendix S2), existing measurement tools for comprehensive school health (Supplementary Appendix S3) and consultation with project knowledge users (Supplementary Appendix S4) resulted in ten constructs. These were operationalized for measurement to collectively represent a set of theoretical constructs for an HPSE (Table 1). Data sources for items included surveys, auditor observations and photographs. Survey items were taken from a range of tools included in full in Supplementary Appendix S3, for example ‘Sense of Belonging’ included items in our principals’ survey from the Queensland Health and Education toolbox such as an agreement on a scale of 1–5 ‘Girls are encouraged to be as involved as boys in physical activity?’. ‘Reinforcement of Health’ included items in our teachers’ survey from the Healthy School Planner such as on a scale of 1–5 rating how often the teacher had ‘eaten healthy meals and snacks’ in front of their students and included reliable health education components in their classroom lessons; and ‘Aesthetics’ included prompts to school auditors to take photos of environmental features of interest, and then assessment of photos on a scale of 1–5 for aspects of the captured environment that indicate it is pleasing to look at using the assessment tool (full items from all sources available upon request from authors). Following data collection, these original 10 conceptual dimensions were reduced to eight (connectedness/sense of belonging and healthy curriculum/reinforcement of health were combined) after tests for internal consistency.

Table 1:

Resulting constructs for health promoting school ethos (HPSE) measurement

Original HPSE construct dimensions	Definition
Aesthetics	School features for different areas (i.e. grounds, classrooms, school entrance etc.) that are pleasant to look at, or improve the overall feeling of a space (e.g. the play area is spacious and pleasant to look at, it is clean, well-maintained, colourful, etc.)
Safe surrounding	School features that support awareness of and action to address threats to safety (e.g. there is a guarded cross walk near the school) in different areas of the school (i.e. crosswalks, play areas, hallways, etc.)
Connectedness	Fostering relationships between students, and between students and staff (e.g. safe places to interact, student engagement through visual cues)
Sense of belonging	Engagement of students and staff and their feeling part of the wider school community
Consciousness of health	Awareness, value and prioritization of health and wellbeing in the school and school curriculum
Reinforcement of health	Reinforcing health-promoting behaviours through modelling, incentives/disincentives and feedback from staff to students
Healthy curriculum	Curriculum and specific classroom activities that equip students with knowledge, skills and attitudes to lead healthy lives outside of the school setting
Resources	School features that indicate the socioeconomic status of a school that could support student health (e.g. large open areas to play with quality play equipment or many different kinds of sport equipment to use)
Availability	School features present for different parts of the school (i.e. cafeteria, grounds, hallways, classrooms, etc.) that supports healthy eating, physical activity and mental wellbeing for students (e.g. there is a bowl of fruit on a desk or a soccer ball in a room).
Accessibility	The ease or difficulty in using school features that exist to support healthy eating, physical activity and mental wellbeing for students (e.g. the bowl of fruit is behind the teachers desk and difficult to reach or the soccer ball is in a locked room) across different areas of the school (i.e. grounds, classrooms, cafeteria, etc.)

HPSE score

The final HPSE included eight conceptual dimensions with internal consistency (Cronbach’s alpha) ranging from 0.60 to 0.87. Total HPSE score across schools (N = 18) ranged from 1 to 8 (mean = 3.94, SD = 2.1) (possible range 0–8), with 28–65% of schools reporting high levels of respective HPSE dimensions (Table 2). The most common dimensions rating high were: Consciousness of health, Safe surroundings and Availability. The least common dimension with a high rating was Reinforcement of health.

Table 2:

Summary of health promoting school ethos (HPSE) score results for eight conceptual dimensions (n = 18 schools)

Final HPSE construct dimensions	No. items	Alpha	No. rated high	% Rated high	Data sources
Consciousness of health	13	0.71	10	56	Principal and teacher survey
Safe surrounding	11	0.63	10	56	Principal survey and photo assessment
Reinforcement of health	15	0.74	5	28	Teacher survey
Sense of belonging	26	0.80	9	50	Principal and teacher survey
Availability	24	0.67	10	56	School environment audit
Accessibility	9	0.60	9	50	School environment audit, photo assessment
Aesthetics	10	0.74	9	50	School environment audit, photo assessment
Resources	12	0.87	9	50	School environment audit, photo assessment

‘No. rated high’ = score greater than the median score for all schools.

The number and nature of highly ranked dimensions varied greatly between schools. One school rated high on all eight dimensions, two schools rated high on seven dimensions, two schools rated high on six dimensions, two schools rated high on five dimensions, seven schools rated high on three dimensions, two schools rated high on two dimensions and two schools rated high on only one dimension (Table 3).

Table 3:

Health promoting school ethos (HPSE) score distribution of dimensions by schools (from 0 to 8)

Final HPSE construct dimensions	High scores by school
	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18
Consciousness of health	X	X		X			X	X	X	X					X	X	X
Safe surroundings	X	X	X		X		X	X	X		X	X			X
Reinforcement of health	X	X					X		X			X
Sense of belonging	X		X	X			X	X				X		X	X			X
Availability	X	X				X	X		X	X	X			X	X		X
Accessibility	X	X	X	X	X	X							X				X	X
Aesthetics	X	X			X		X	X	X			X			X			X
Resources	X	X				X	X	X	X		X	X			X
Total	8	7	3	3	3	3	7	5	6	2	3	5	1	2	6	1	3	3

X: ‘high score’ = score greater than the median score for all schools.

DISCUSSION

The HPS approach is hypothesized to positively influence the health and wellbeing of students. However, existing school culture or ethos may be an important consideration in understanding school context for implementation or evaluation of HPS. Evaluation of school-based population health interventions, like HPS, contain additional challenges given that researchers seek to evaluate interventions that are led and maintained outside of the academic sphere, often within short timeframes and under circumstances with restricted resources (Hawe and Potvin, 2009). Therefore, to better characterize school context for the evaluation of an HPS approach, we employed a novel method to measure HPSE and develop a score to capture this multidimensional construct for each school.

Reflections on characterizing and measuring an HPSE

Characterization of HPSE resulted in eight final dimensions. Previous research has explored many of the conceptual dimensions represented in this work that have been shown to be important for various aspects of school, teacher and student outcomes. In particular, the importance of aesthetics in relation to the satisfaction of students and teachers; safe surroundings in relation to emotional and physical wellbeing of students (Taylor and Hansen, 2005) and perceptions of organizational satisfaction (Goodwin, 2013); sense of belonging and the emotional health of students in terms of their acceptance and experience of membership in a community (Osterman, 2000); consciousness of health and the dissemination of health messages through school curricula and other practices to influence the beliefs and behaviours of students (Kilgour et al., 2015); reinforcement of health to support teacher–student relationships (Jennings and Greenberg, 2009) and modelling of health behaviours (Kibbe et al., 2011); resources and the capacity of schools to provide support to teachers and students (Stolp et al., 2014); and the availability and accessibility of how the school environment (e.g. infrastructure, equipment or food) influences healthy behaviours in students (Bonell et al., 2013). The breadth of the dimensions that emerged using our process demonstrates the importance of using multiple sources of evidence for this work, particularly the contextualized knowledge provided by the school stakeholders (Nonaka and Von Krogh, 2009). Similar to other population health interventions, researchers were not involved in the development or implementation of the intervention (Hawe and Potvin, 2009) but maintained engagement with knowledge users from the very early stages of the project. This partnership was key to ensuring that the tool was not only methodologically and theoretically sound, but that it could capture the context of schools in order to support further implementation of HPS in this region. In addition, this work sought to capture ‘ethos’ in a quantitative way to provide schools with a quick and replicable method to capture indicators of their school context. However, an even more holistic definition of school ethos has been used in previous research defined as school culture or climate with a greater reliance of qualitative methods to exposure the functions and processes of a school (Parcel et al., 2003).

Developing an HPSE score

The HPSE scores varied greatly across a number of health supportive school ethos concepts. This may demonstrate the complexity of school context, and, therefore, the importance of understanding this context in more detail before HPS implementation takes place (Keshavarz et al., 2010; Kremser, 2011; Squires et al., 2015). For example, some schools may not require all facets of the HPS approach; perhaps some schools are well resourced, but lack a consciousness of health that is needed to support HPS aims and goals. As previous research has suggested, each school is unique and reflects the leadership, students, parents and community it serves, creating a large variation in school context that requires careful consideration prior to HPS implementation (McIsaac et al., 2017).

Flexibility of a translational approach

A potential strength of a translational approach to characterization and measurement is the potential to adapt to other school contexts. For instance, the HPSE tool was created to include the process of engaging a range of stakeholders. This allowed for the inclusion of concepts that were important to knowledge users, but may not have been reflected in the literature or existing measurement tools (Kothari et al., 2012). In addition to engagement, the observational school audit allowed for the collection of photographs of each individual school. The analysis of these photos provided the means to introduce complex contextual variation within and between schools that may not have been as robustly captured through surveys or auditor checklists (Iwamoto et al., 2004), contributing to enriching qualitative concepts including aesthetics, resources and safe surroundings. This can support detecting subtle differences between HPS and non-HPS school environments or practices that may otherwise go undetected (Kontak et al., 2017). The use of these methods may also create challenges for the on-going validation of measurement tools using traditional methods, and more work to determine the impact of measuring and assessing context is needed.

Strengths and limitations

This work focused on the process and development of a tool to quantify a relative score to reflect health prompting school ethos. An important next step would be to refine this process and test for reliability and validity across different school contexts, and explore the range of scores and variability in different regions of Nova Scotia, the country and beyond. The HPSE score was developed to provide a ranking of schools, rather than an objective measure of health-promoting school ethos at a school, therefore future work would be needed to examine the predictive ability of the scores developed here with school or student outcomes. Also, to allow for flexibility in the conceptual development of HPSE a liberal Cronbach’s Alpha was chosen (≥ 0.6), this requires further analysis in relation to school or student outcomes to determine their validity. In addition, assessment of photos was used to contribute to several of the HPSE dimensions. However, these photos may have missed or not captured other important aspects of the school environment despite training and providing the research assistants with a guided audit tool to direct photo capture.

Given the range of theoretical dimensions collected, developing appropriate measures that were sound and had minimal burden to school staff was a challenge (Rick and Goodman, 1991). This required the use of short surveys for principals and teachers, in order to better understand staff consciousness of health, school sense of belonging, reinforcement of health in the curriculum and the policy context. Comparatively, aspects of the physical environment including accessibility, availability and safe surroundings required trained researcher assistants to conduct an observational audit and take photos to assess these dimensions. The complex nature of the concepts and data also makes traditional validation challenging, as the translational approach to theory development and measurement employed here can create different constructs for measurement when used in a new context. In addition, it relies heavily on well-trained auditors and careful photography. Even though data collectors had been trained, school audit data was sometimes incomplete, or completed with supplementary details that required interpretation. In these instances, consensus was reached and new variables were coded as needed. Also, collecting some of the social dimensions of school ethos was challenging from both tools used: the survey (where social desirability bias can dominate) and the audit (where it is difficult to capture social interaction). An enhanced approach could include observational methods, where auditors observe social interactions between students and staff—however, this would add additional burden to the data collectors, school and additional resources from typically low resourced evaluations in population health intervention research.

CONCLUSIONS

Our novel approach to tool development allowed us to conceptualize HPSE using a flexible process comprising different types and sources of evidence. The HPSE tool holds potential for identification and measurement of critical components of different school context as it relates to HPS. This approach requires further refining and testing to ensure a reliable, valid measurement of HPSE that is adaptable to different contexts. This will help move the knowledge base forward and support population health intervention researchers to better capture important contextual influences.

FUNDING

This work was supported by a Population Health Intervention Research grant from the Canadian Institutes of Health Research (CIHR) (FRN 127082). Sara Kirk held a CIHR Canada Research Chair at the time the research was conducted. Jessie-Lee McIsaac was supported by a postdoctoral fellowship from KT Canada (CIHR) as well as a Bright Red Graduate Research Award from the Heart and Stroke Foundation of Canada and the Canadian Cancer Society (grant #703878). Tarra Penney was supported by the Cambridge International Scholarship from the Cambridge Commonwealth, European & International Trust.