Studies evaluating of health interventions at schools: an integrative literature review

ABSTRACT

Objective:

to identify and analyze the available evidence on the strategies used in the studies evaluating health interventions at school.

Method:

this is an integrative review searching in LILACS, CINAHL, CUIDEN, ScienceDirect, and PubMed. From the pre-defined inclusion and exclusion criteria, there were 121 articles chosen to compose the sample.

Results:

english studies (97.5%), with a quantitative approach (80.2%), related to the interventions carried out in the Region of the Americas (54.6%) and the European Region (23.1%) predominated. For the most part, they are interventions as programs (70.2%), interested in evaluating results (73.5%) from the value judgment (83.4%). Prevalence of interventions focused on efficacy, effects or impact, and activities carried out on interventions were focused on physical activity, healthy eating, sexual and reproductive health, mental health, and use of tobacco, alcohol, and other drugs. They are worked through activities of clinical monitoring, health promotion and disease prevention.

Conclusion:

the evidence indicates that the evaluations of health interventions in the school focus the results produced in programs through the judgment of value. The topics most addressed were healthy eating, physical activity, prevention of alcohol and other drugs, among others.

Introduction

The offering of equal and equitable education across the globe has been one of the flags raised by international organizations in encouraging children and adolescents to be enrolled in schools. Also, it is also necessary to develop health interventions so these individuals do not have the educational process interrupted under the influence of diseases and other health problems ¹ .

In these spaces, it is possible to contribute to the development of learners through interventions that subsidize educational success through the provision of care that may not be experienced at school. The lack of school health interventions together with the compromising situations (diseases and/or aggravations) prevalent in schools make the learning process difficult, where it is necessary to provide continuous care to the health of the children and adolescents in the school environment ² .

In 1995, the World Health Organization (WHO) encouraged the development of the global strategy of Health Promoting Schools, characterized by the constant search to strengthen the capacity to promote a healthy life, with an incentive to learn and working conditions, to respond health needs of the school community ³ .

Under this influence and based on the Ottawa Charter and as a result of the First International Conference on Health Promotion held in 1986, discussion groups were set up to encourage the development of health promotion strategies in school settings, such as the European Network of Health Promoting Schools and the Latin American Network of Health Promoting Schools ⁴ .

This fact contributed to creating several experiences all over the planet. However, because they are countries with diverse economic, political, cultural, and other contextual characteristics, it is necessary to consider that these interventions can be executed based on different objectives. Also, difficulties and challenges can be found in their implementations, requiring the need to develop evaluative processes that seek to improve them.

This is possible since the evaluation has been considered as an important tool to aid the management of health interventions in the search for better answers to services resulting from improvement, resolution and better quality ⁵ .

The evaluations of the health interventions at school have been carried out and involve specific topics such as prevention of depression and anxiety ⁶ , the offering of physical activities to students ⁷ and drug prevention among schoolchildren ⁸ . No studies have been found that synthesize comprehensively the evaluation of health interventions in school in the world and its different themes.

The purpose of this study is to synthesize the studies that carried out evaluations of health interventions at school, so their strategies and the diversity of interventions directed to this space can be identified. This will contribute to other research being carried out from the findings presented here.

The study advances knowledge as it presents the evidence on strategies used in evaluations of health interventions at school, enabling other health interventions at school to be evaluated as well.

Evaluations should be carried out by management and by health and education professionals to contribute to the promotion of health at school. It is observed by the activities developed in the daily life that the nurse performs interventions in the school environment in a continuous way.

The actions or health activities carried out in the school, such as policies, programs, projects, services, and systems, were considered an intervention. Evaluations are understood as the decision-making processes that aid in the improvement of the health interventions in the school. Therefore, there is an intimate relationship between the two, since it is not possible to evaluate interventions without knowing their organization.

In view of this information, the objective is to identify and analyze the available evidence on the strategies used in the evaluation studies of health interventions in the school.

Method

The integrative review of the literature was the methodological strategy chosen to respond to the proposed objective. It consists of a broad methodology of research analysis to synthesize knowledge about a given topic. The elaboration of a study of this nature runs through five stages: identification of the problem and elaboration of the guiding question; search for studies in the literature; evaluation of data found in the studies; data analysis with synthesis and their conclusions and the presentation of the integrative review ⁹ .

A protocol was built by the researchers to guide the construction of the study. The guiding question was elaborated with the help of the PICO strategy ¹⁰ (P: health interventions at school, I: evaluation studies, C: not applicable, O strategies used). What evidence is available about the strategies used in the evaluation studies of health interventions at school?

The search for the studies was carried out in May 2017 through the Portal of Journals of Capes with access through the Federated Academic Community (CAFe) in which the researchers are linked. The studies were selected in the electronic databases of Literatura Latino-Americana e do Caribe em Ciências da Saúde (LILACS), Cumulative Index to Nursing and Allied Health Literature (CINAHL), Base de Datos Bibliográfica de la Fundación Index (CUIDEN), ScienceDirect and US National Library of Medicine (PubMed).

To proceed with the search, descriptors in Health Sciences (DeCS) were used for the databases in Portuguese and Spanish and the corresponding Medical Subject Headings (MeSH), for the search in the electronic bases in English. The Boolean operators AND and OR were used to cross the descriptors as follows: “Serviços de Saúde Escolar AND Avaliação de Programas e Projetos de Saúde OR Avaliação de Serviços de Saúde”, “Servicios de Salud Escolar AND Evaluación de Programas y Proyectos de Salud OR Evaluación de Servicios de Salud” e “School Health Services AND Program Evaluation OR Health Services Research”.

The inclusion criteria consisted of articles published in the last five years, available for free access in full in English, Spanish and Portuguese, which addressed the proposed theme. Editorials, letters to the editor, review studies, theses, dissertations, articles and studies that did not correspond to the relevant theme within the scope of the review were excluded.

The research and selection of the studies were carried out by two researchers, simultaneously. When a situation of divergence happened, a consensus was sought with the participation of an auxiliary researcher. The process of searching and selecting the studies followed the PRISMA recommendations ¹¹ and is represented in Figure 1.

Figure 1. Flowchart of identification of the selection process of the selected studies to compose the integrative review. Natal, RN, Brazil, 2017.

* LILACS: Literatura Latino-Americana e do Caribe em Ciências da Saúde † CINAHL: Cumulative Index to Nursing and Allied Health Literature ‡ CUIDEN: Base de Datos Bibliográfica de la Fundación Index § PubMed: US National Library of Medicine

The initial search in each database using the search term crossings was preceded by its refinement (free online access and publication in the last five years) in each of the electronic bases from the use of the available tools in the electronic portals. In the sequence, the titles and abstracts were read, making a quantitative of 124 studies in all the electronic bases. Also, repeated studies were excluded, where the final sample resulted in 121 articles.

The analysis of the selected studies was performed based on the pre-selected variables in the protocol construction. To organize them, a spreadsheet created in Microsoft Excel Software was used, containing the following items: database, journal, language, year of publication, methodological approach used, geographical location where the intervention was performed according to WHO classification, type (structure, process and outcome) according to Avedis Donabedian’s theoretical reference ¹² , level of evaluation (description, measurement, judgment, negotiation) according to the classification identified in Furtado’s study ¹³ , type of evaluation, besides description of the themes and characteristics of the intervention.

The analysis and discussion of the results were based on the national and international literature on school health and health evaluation.

Results

The characterization of the selected studies ^{14 - 134} included in the integrative review is presented below. In Table 1, information regarding the year and language of publication, the approach used in these surveys and the place of execution of the interventions are observed.

Table 1. Characterization of the studies regarding the year, language, approach and place of intervention. Natal, RN, Brazil, 2017.

Variable	N*	%†
Year of publication
2013	32	26.5
2014	32	26.5
2015	34	28.0
2016	19	15.7
2017	4	3.3
Language
English	118	97.5
Spanish	3	2.5
Type of research approach
Quantitative	97	80.2
Qualitative	10	8.2
Quantitative and Qualitative	14	11.6
Regions of interventions
African Region	5	4.1
Region of the Americas	66	54.6
Southeast Asia Region	7	5.8
European Region	28	23.1
Eastern Mediterranean Region	4	3.3
Western Pacific Region	11	9.1
Total	121	100.0

*N: number; †%: percentage

It should be noted that the lower percentage of studies published in 2017 is explained by the fact that the data collection was carried out with the year still in progress.

Information regarding the classification of the type of intervention, proposed evaluative dimension and level of evaluation performed are shown in Table 2.

Table 2. Characterization of the studies regarding the type of intervention, dimension evaluated and level of evaluation performed. Natal, RN, Brazil, 2017.

Variable	N*	%†
Type of Intervention
Policy	6	5.0
Program	85	70.2
Project	19	15.7
Service	9	7.4
System	2	1.7
Dimension evaluated
Structure	2	1.7
Process	24	19.8
Result	89	73.5
Structure and Result	2	1.7
Process and Result	3	2.5
Structure, Process, and Result	1	0.8
Level of Evaluation
Description	11	9.1
Measure	3	2.5
Judgment	101	83.4
Negotiation	6	5.0
Total	121	100.0

*N: number; †%: percentage

It was also possible to describe the interventions regarding the themes, characteristics, and typologies of evaluations proposed in their methodologies, as presented in Figure 2.

Figure 2. Characterization of studies evaluating health interventions in school. Natal, RN, Brazil, 2017.

Thus, the data show that the evaluation of effects, efficacy, and impact was the most prevalent among the types of evaluation found.

It is observed that interventions of various themes have been carried out in the school environment, whether they are in a broader theme, such as activities that aim to encourage healthier lifestyles or more specific themes such as those that prevent accidents from occurring among learners.

These themes are worked with the school community through strategies that address the continuous clinical follow-up of learners ^{25 - 26 , 30 - 31 , 46 - 47 , 63 , 74 , 98 -95, 108 , 132} , through activities to prevent health problems ^{52 , 68 , 90 , 92 , 97 , 126} , availability of resources in school spaces as a way of encouraging changes in habits ^{15 , 19 , 27 , 54 , 71 , 100 , 121 - 122 , 129} , educational activities with students and other members of the school community^{(14-24,27-29,32-45,49-51,53-62,64-67,69-73,75-77, 79-89,91,93,98-107,109- 125,127-131,133-134)} and group activities ⁹⁶ .

Discussion

The high number of studies that integrated the integrative review shows the possible concern of the researchers in offering quality interventions in the school environment, which can be done when submitting them to the evaluation processes. However, a smaller number of publications were observed in the last two years of the analysis, which can be explained by the inclusion of a year still in progress during the review period.

The predominance of studies published in the English language reflects the fact that this is a widely disseminated language in the world. Therefore, it was considered by the scientific community as a universal language to contribute to the dissemination of publications in the world and to promote better access ¹³⁵ .

When analyzing the types of approaches used in evaluative surveys of health interventions at school, it was identified that most of them used exclusively the quantitative approach. In contrast, there were studies that proposed qualitative analysis. However, this type of research requires the appropriation of several means to understand the problems, which is possible with the integration of these two types of approaches ¹³⁶ .

The observation that there was a greater predominance of studies published in the Region of the Americas, mainly in the United States of America, and in the European Region, shows the understanding that the largest world economic powers are located there, and can consequently have greater investments in the execution of searches. In contrast, the regions in which the nations with the lowest human development indicators are found are also those with a reduced number of health interventions in school.

According to the classification of the interventions proposed for analysis, it was identified that most of them refer to programs to improve students´ health. On the other hand, there is a discrete presence of interventions classified as systems that offer the search for health care to learners.

Prevalently, it was identified that most evaluative research sought to carry out evaluations focusing on the results of the interventions. It is important to emphasize the need to consider the development of these evaluations so they do not only focus on the presence or absence of the results brought about by the intervention, but also try to understand the factors that influenced the process ¹³⁷ . It is also considered that, although studies are usually found that deal with outcome evaluations, the structure and process can influence the presentation of what is expected of these interventions ¹² . Therefore, it is important to emphasize the equal importance of these dimensions in proposals for the execution of activities directed at school spaces.

Considering the level of evaluation of the research carried out, a large number of studies with an evaluation at the judgment level were found. Thus, evaluations of health interventions at school exceeded descriptive and measurement evaluations when judging value. However, there is still a small number of evaluations that propose a process of negotiation between those involved in the school environment ¹³⁸ .

Thus, although the nomenclatures attributed to the evaluations are diversified, they can be performed identically. Nevertheless, it is noted that these types of proposed evaluations aim for interventions to perform the search for better results ¹³⁹ .

The evaluations found may also influence the evaluation levels identified from the historical generations. Thus, in its first generation, the evaluations were descriptive. In the second generation of evaluations, there was research that sought to measure performance. The third generation has evaluations that try to judge the merits of the interventions. In the fourth generation, there is an evaluation that advocates the participatory negotiation process among the different participants in the intervention ¹³ .

The fourth-generation evaluation can be carried out in school health interventions, since they have activities that allow the participation not only of the students but also of teachers, other school workers, health professionals, parents or responsible for the students in their proposals, as well as representatives of the community in which the schools are inserted ¹⁴⁰ .

Regarding the themes in which these interventions were proposed, similar results were observed in a study carried out in the United Kingdom on the structures of the World Health Promoting Schools ¹⁴¹ . The following themes of interventions prevailed: physical activities, healthy eating, sexual and reproductive health, mental health and tobacco use, alcohol and other drugs.

As for the characteristics of these interventions, it was found that besides to continuous follow-up of students, preventive activities, health promotion with the school community and use of talk circles, there is the availability of resources to encourage a change of habits. This last characteristic corroborates with a research carried out in Canada, which refers to the presence of objects and food to encouraging the promotion of healthy eating and physical activities ¹⁴² .

The diversity of the themes and the characteristics related to health interventions at the school evaluated is essential in changing habits for the life of the students and the community where they live. This is identified in a study that shows that, these activities are of great importance to the health of the population although they seem small actions in face of the diversity of needs found ¹⁴¹ .

It should be emphasized that the limitations of this research were related to the high number of studies eligible for an integrative review, although the criteria chosen were delimited. Also, it is observed that the use of more than one researcher for data analysis may influence the presented results, although they were able to perform this activity.

Conclusions

The evidence found in the studies show that school health programs are the most commonly evaluated interventions, especially at the value judgment level. These studies are mainly focused on evaluating the results produced by the interventions, corroborated in the typologies of impact, effects, efficacy, and results.

The themes of the studies are related to healthy eating, physical activity, mental, sexual and reproductive health, as well as prevention of the consumption of alcohol, crack and other drugs. They are implemented through strategies of continuous clinical monitoring, health promotion, disease prevention and health problems, and group activities involving students and other members of the school community.