Consulting with young people to inform systematic reviews: an example from a review on the effects of schools on health

Abstract

Background

There has been increasing interest in involving the public in systematic reviews as they provide a shortcut to the evidence and arguably have greater influence over policy decisions and ultimately people's lives. Case examples of this involvement are rare, especially for reviews focused on children and young people. This study describes the process and impact of consulting with a young people's advisory group to inform decision making in a systematic review on the effects of schools and school environment interventions on children and young people's health.

Methods

Consultations were conducted with a pre‐existing group of young people brought together to advise on public health research. Their views were sought at two key stages: (i) at the beginning when general views relating to the policy problem under study were elicited; and (ii) half‐way through to advise on how to focus the review on key priorities.

Results

Young people's involvement in our review ensured that the scope of our review was appropriate and that issues which were important to young people were considered. The group was especially valuable in terms of prioritizing in a relevant and meaningful way. A crucial additional impact of involvement was young people providing ‘early signals’ of key themes for the synthesis.

Introduction

There has been increasing recognition internationally that research should involve patients and the public on topics relevant to their lives.1, 2 A key driver in the health sector has been the World Health Organization's Declaration of Alma Ata which states that people ‘have the right and duty to participate individually and collectively in the planning and implementation of their health care’.3 This extends to children and young people who have the right to express their views freely in all areas that they are involved in, as enshrined in Article 12 of the convention on the Rights of the Child.4 Children and young people are recognized as competent social actors capable of making sense of their lives and are considered stakeholders in decisions about their health needs and behaviour, alongside professionals and other adult stakeholders.

Involving patients and the public, including young people, both in the generation of research and its use enhances the opportunity for optimal health outcomes.5 It is also supported by ethical and political arguments that highlight the rights of service users to have their views heard, and the accountability and governance of publicly funded activities.6, 7 Indeed, the cornerstone of evidence‐based medicine is now the integration of clinical expertise, best evidence and patient values into the decision‐making process for patient care.8 The move towards more participatory approaches in health research thus reflects an ontological shift which highlights the value of a more phenomenological approach grounded in the lived experiences of those being researched.9

Public involvement in health care has become important internationally in both service development and research,2, 10, 11 particularly in regard to systematic reviews. Systematic reviews aim to identity, appraise, select and synthesize research evidence on a particular topic, providing a balanced overview of evidence and thus have greater influence over policy decisions, and in turn people's lives than individual studies.12 The range of public involvement can been simplified into three key approaches: publically led (the public designs and undertakes the research and researchers participate at the invitation of the public), collaboration (an on‐going partnership between researchers and the public); and consultation (researchers seek the views of the public on key aspects of the research).1, 13 The continuum of participation, however, should not be interpreted as suggesting that more participation or control is necessarily better. The choice of approach to participation should be closely linked to the purpose and desired outcome of the research.

Consultation methods are most commonly used for systematic reviews.1, 13 Consultation involves the researcher asking patients and the public about their views and using these to influence decision‐making, interpretations of evidence and/or the language used in research.9 The approach allows the researcher to obtain views which may then contribute to the research process but is not necessarily committed to act on them. More collaborative approaches to consultation require reviewers to work on an on‐going basis with patients and the public throughout the review. Typically, this is done via an ‘advisory’ group that meets with researchers at several points to discuss progress and help determine the direction of the review at key decision‐making stages.12

As patient and public involvement in systematic reviews becomes increasingly common, there is a need to share experiences and critical thinking about the processes, outcomes, benefits and challenges of involvement, which is currently absent in the published research. While some case studies exist,14, 15, 16, 17, 18 examples of how to involve children and young people in systematic reviews are particularly limited.15, 16 There is also a general tendency to emphasize processes of involvement with few examples on how processes impact review outputs.16, 17

This study adds to the small body of case studies by describing the process of consulting with a young people's advisory group to inform decision making in a systematic review on the effects of schools on children and young people's health.19 It also extends the existing body of work by tracing how the process of consulting with the young people's advisory group led to changes in our review.

Methods

Description of the review

The systematic review discussed in this paper, to which young people contributed, concerned the effects of schools and school environment interventions on young people's health.

In the context of numerous systematic reviews and evaluations highlighting the disappointing results of health education programmes delivered via the curriculum, we assessed the potential of other approaches to improving health in schools, such as through modifying the ‘school environment’. Rather than treating schools merely as sites for health education, the ‘school environment’ approach treats schools as physical and social settings which can influence health. School environments can influence health directly by addressing, for example, school food provision or bullying policies and can shape health indirectly via addressing, for example, student disengagement or staff–student relationships which may impact on health. While this is a growing field of public health research, few reviews had been conducted on school environments and health, and these either did not report the accumulation of recent research20 or focused only on certain outcomes.21, 22, 23, 24 Therefore, a research team based in the United Kingdom (UK) led by academics in the fields of public health and sociology with expertise in systematic reviews sought to search comprehensively, map and synthesize the available international research on the effects of the school environment on health.

The review was conducted in two stages. In stage 1, we produced an ‘evidence map’ where we identified a broad array of potentially relevant literature and descriptively coded characteristics of the studies. In stage 2, we conducted five in‐depth reviews corresponding to our review questions which examined: (a) theories of schools' influence on student health,25 (b) outcome evaluations of interventions modifying the school environment to promote student health,26 (c) process evaluations of these interventions, (4) multilevel studies of the effects on student health of school‐level factors,27 and (5) qualitative research on the processes by which the school environment influences student health.28 We identified two separate stakeholder groups to provide expert advice on the review: (i) adult professionals working in policy, practice and research; and (ii) young people. The focus of this paper is the young people's advisory group. Details on the methods of our review including the flow of literature can be found elsewhere.19

Involving young people in the review process

While this review of international studies was led and conducted by researchers, we wanted its outputs to be sensitized to the perspectives of young people from a UK setting to help ensure its relevance. Therefore, we chose to consult with an advisory group of young people. The consultation approach we used had the advantage of providing structured and relatively quick (compared to other more collaborative approaches) opportunities to seek views at key decision stages, while researchers retained control of the overall conduct of the review. We felt a more collaborative approach would have been extremely time consuming and costly (e.g. training young people as reviewers), with limited evidence of added value to outputs. Two face‐to‐face advisory group consultations, supplemented with an online discussion forum were conducted with DECIPHer's young people's public involvement group called ALPHA (Advice Leading to Public Health Advancement). DECIPHer is a UK Clinical Research Collaboration public health research centre of excellence focusing on the health of children and young people. With a commitment to involving children and young people in the research process, a panel of young people (the ALPHA group) was formed by DECIPHer. At the time of our review, 25 young people (aged 14–19) from Bristol and South Wales were registered in the ALPHA group. The group was recruited via existing youth provisions and advertisements in the local press and online. Participants were predominantly white and middle class, with a small number of ethnic minority participants. There were no specific healthcare needs that required special arrangements for their participation. The group met monthly and sessions attracted on average 12 members. Young people participating in the programme were trained in research including sessions on public health, the research cycle, ethics procedures and reviewing materials. The sessions were supported by qualified youth workers (individuals who facilitate voluntary, associational and informal learning with people aged 14–25) with extensive experience of facilitating youth groups. This group was chosen as an advisory body for our review because it allowed access to an established group of young people already familiar with health research who met frequently enough (once per month) for us to seek their views at the decision stages of the review. Members were not rewarded directly from researchers. However, they received £15 vouchers for their monthly participation (not specific to this research project), had food and transport provided and were eligible for an annual residential teambuilding activity. Consultations with the ALPHA group were conducted at two key points: at project inception, for advice on setting the scope of the review; and at the review mapping stage, for advice on focusing the review on key priorities (Fig. 1).

Figure 1.

Summary of consultation process and impact.

Project inception – setting the scope of the review

Consultation at this stage consisted of face‐to‐face and on‐line consultations. Thirteen young people participated in a face‐to‐face consultation in meeting rooms at the University of Cardiff in September 2010. The session lasted just over one hour and was facilitated by two researchers with oversight from a young people research officer, who is a qualified youth worker. The purpose of the consultation was to find out what the terms ‘health’ and ‘well‐being’ meant to advisory group members and to elicit their perspectives on how schools can impact on their health and well‐being with a view to informing the development of our review questions. The session was designed to be as interactive as possible. A five‐minute presentation was given by researchers to explain the aim of the session. Small group discussions were used to explore: their understanding of ‘health’ and ‘well‐being’; how schools could affect health and well‐being; and what a good/bad school for health and well‐being would be like. Key discussion points were then fed back to the group and participants identified priority areas for school health and well‐being. Researchers took extensive notes during the consultation which were then scrutinized for key themes relevant to the review.

Online consultations provided an opportunity for additional participation. A social networking site was launched hosted by the online company Grou.PS (http://grou.ps/home). Advisory group members were invited to join the online group and provide any further views on the topics discussed at the face‐to‐face consultations via the online discussion room which they could access at their convenience. To register on the site, members of the advisory group emailed a member of the research team for log‐in details. Advisory group members were able to create an avatar profile if they wished to remain anonymous.

Evidence mapping stage – focusing the review on key priorities

Having consulted with the advisory group regarding the scope of the review, we produced a descriptive map of the available research evidence. Our searches identified 1017 studies that met the inclusion criteria set out for the review. Studies were descriptively coded based on title and abstract to identify key characteristics [e.g. study design; the feature of school examined in the study (e.g. policies, catering); health topic].

The evidence map showed that the research landscape was extensive and focused on diverse health topics across a range of school environment features (such as physical infrastructure, teaching, policies, catering, management, pastoral care). To produce a manageable synthesis of evidence, we needed to focus the review on a smaller number of health topics and features of the school environment, which had credibility with young people.

A second consultation was therefore conducted to establish which health topics young people viewed as a priority for us to focus on. This consultation was conducted with 13 young people (with considerable overlap from the first consultation, with the exception of four new members) from the ALPHA group in May 2011. Two researchers and the same young people's research officer facilitated a group discussion and a consensus development exercise.

The advisory group was presented with a list of health topics identified in the evidence map including: anxiety; injury; pregnancy; obesity; young offending; smoking; drug use; violence; sexual risk behaviour; school disengagement; and sexual health. Researchers probed the group to discuss which health topics were most relevant to them, why and in what context. Views were orally summarized by researchers back to the group throughout the consultation to help ensure accurate representation. Discussions continued until all health topics on the list were discussed. One researcher collected unstructured written notes of young people's views (which were later summarized in bullet points thematically by health topic after the consultation event). The final list of health topics was agreed by the group. This list was then written on to individual sheets of paper and distributed to each group member. Group members were then asked to vote individually (using stickers) on the sheets provided for the topics they thought were important to consider in the review. They could allocate their stickers as they liked, either all against one health topic, or more evenly spaced. When all had placed their votes, the sheets were collected and researchers recorded the most popular topics, which were then shared and discussed as a group. During the group discussion, researchers confirmed whether the final tally of votes was acceptable to the group and gave members an opportunity to explain their choice of voting or provide any comments. The process we used was similar to another review related to sexual health promotion.14 The consensus method allowed young people who may have been uncomfortable or disliked participating in group discussions, to contribute views via the anonymous individual voting activity. In this way, consensus methods addressed the inequality of participation and the tendency of open discussion to be dominated by a subset of voices.

Results: using the consultations to inform our systematic review

Understanding the problem and developing the research aim

At the outset of our review, we developed broad research questions geared towards developing a map of evidence and theories related to the review. The advisory group was consulted at this stage to get a sense of the health topics important to them and what features of schools they perceived were important to health. We drew on their views (presented below), alongside the views of the adult professional advisory group, and our own interests as researchers to finalize the review questions to inform stage 1 ‘evidence mapping’ which would set the parameters of our searches and inclusion criteria.

The consultations with the young people's advisory group highlighted the importance of considering mental and social health outcomes alongside physical health. Physical well‐being was only briefly discussed, often referencing well‐known public health messages to eat healthily (e.g. 5 A DAY – a national campaign to encourage the consumption of at least five portions of fruit and vegetables each day). Discussions focused on social relationships between students, mental well‐being and student–staff relationships. The advisory group also confirmed the importance of examining the effects of learning and teaching and student participation on health in schools. For example, having a good relationship with teachers at school was considered crucial to happiness and academic success (e.g. doing well in exams); the opportunity for students to have a say in the running of schools was seen as a way of promoting well‐being, and student councils were seen as a positive example of this. This was consistent with our own interest as researchers on the influence of schools' social environments on health. While the review would have focused on the school social environment without any input from the advisory group, it was reassuring to researchers that young people also cited this as an important feature of the school environment.

By contrast, the initial consultation also identified an important area we had not previously considered. The advisory group highlighted the importance of the physical environment, making connections between poor toilets, canteens and classroom facilities with poor health and well‐being. For example, one participant explained that their school had refused to provide toilet paper after students used it to block the sinks. Other group members explained that large class sizes meant classrooms were often cramped. This resulted in discomfort for students and less attention and control from teachers, which group members felt negatively affected their ability to learn. The consultation thus proved invaluable in identifying the school physical environment as an important feature of the school environment, which might have been overlooked by researchers.

The advisory group discussed the importance of a wide range of health topics and features of the school environment, which they stressed were irrevocably interlinked. For example, group members suggested that some students might cope with exam stress by eating unhealthy foods during their revisions or using drugs. They also raised the important issue of physical space in schools and its impact on health. This called for an exploratory review that necessarily involved diverse groups of interventions, school‐level influences and which considered both student and staff health. We therefore developed a broad approach to searching electronic databases to take account of the multidisciplinary nature of the review and developed our inclusion criteria to address themes emerging in consultations (see protocol29). We were aware that this would yield a large number of studies, but further consultations would allow us to narrow our scope at the mapping stage once we had a sense of the characteristics of studies across the research landscape.

Setting key priorities and focusing the review

The evidence map was presented to the young people's advisory group whom we consulted for a second time to help focus the review for the in‐depth synthesis.

The result of the consensus exercise identified relationships as the most important health related issue for advisory group members, as they suggested that most other health issues tended to stem from this. The second was anxiety, which was identified as a common outcome of poor relationships and was expressed in a variety of risk behaviours such as drug use and self‐harm. There was complete consensus on the importance of these two health topics. In their discussions, the advisory group reiterated messages from the first consultation: they were concerned with social aspects of schools, such as relationships between students and between staff and students; the importance of mental well‐being; student voice in developing policies in schools; and the importance of the physical environment, such as outdoor space and school facilities (e.g. canteen).

Impact of consultation on review

In sum, four key issues related to school health emerged consistently in group consultations: physical environment, learning and teaching, student participation and social relationships (between students and between staff and students). Informed by these, our review focused on how schools are organized and managed; how they deliver teaching, pastoral care and discipline; and schools' physical environments. These reflect more ‘upstream’ determinants of health than proximal determinants such as what food schools provide or how they deliver physical education.

We also decided to focus on studies related to student rather than staff health. This was not informed by our consultations but was a pragmatic decision to help ensure the review was manageable and coherent in terms of scope (of the topic) and scale (in terms of how many studies would be included).

The final decision of how to focus the review was informed by various factors including: what features of schools (e.g. social relationships or discipline and pastoral care) are most pertinent to testing the hypotheses to be derived from our review of theoretical literature; which have been focused on the least in existing reviews; which were of most interest to our advisory groups, both young people and adult professional groups; and which we, as researchers, were most interested in. We found that young people's views of what to prioritize or how to conceptualize the school environment were consistent with the adult advisory group, the theoretical literature and our own research interests. This congruence provided us with the confidence and rationale to make decisions in our review.

In addition to helping steer decisions, young people's views were also valuable in flagging potential gaps in the evidence. For example, in the first consultation group members identified ‘physical appearance’ or ‘self‐image’ as a health concern. Positive self‐image was important in establishing self‐esteem and developing friendships. Young people who were considered overweight did not ‘fit in’ and were often left out of friendship groups. The group also highlighted the pressures involved in having the ‘appropriate’ image, for example being ‘cool’ or ‘pretty’, which could result in feelings of anxiety and stress. We did not identify studies in our review that explored the social pressures of physical appearance on young people, thus suggesting a new avenue for investigation. This finding highlights the value of involving patients and the public in setting research agendas.

Group members at both consultations also pointed out that different aspects of health and well‐being are interlinked. For example, in the second consultation, group members suggested that anxiety arising from exam stress often leads to drug use, which in turn could lead to poor student–staff relationships. This highlighted that the boundaries of the review which separated the ‘core business’ of schools from health and well‐being activities are more fuzzy in practice and that in conducting our review, we should be careful not to falsely dichotomize studies on this basis. This was consistent with the established literature which suggests that seemingly separate health issues are irrevocably connected.30 This helped inform how we approached our synthesis. For example, in the qualitative synthesis, we used a meta‐ethnography approach31 whereby we initially grouped included studies based on health topics, identifying key themes and concepts within each of these. However, recognising the overlap and interconnectedness of health issues and features of schools (e.g. canteen environment, outdoor space, bullying policies), we then synthesized themes across health topics to identify a set of ‘meta‐themes’ which cut across all health topics and features of the school environment.

The consultations also provided the team with ‘early signals’ of themes of salience in our qualitative and multilevel synthesis. For example, young people in our advisory group stressed in the first consultation that the pressures of academic attainment in schools were connected to health risk behaviours such as drug use, disengagement and self‐harm. They also felt that teachers were more committed to achieving good student attainment on exams than to well‐being. These themes also emerged prominently in studies we included in our qualitative review. We discuss this at some length in our full review and suggest that the increased marketization of education may have serious health implications for students.19

Another ‘early signal’ highlighted by the advisory group was that student participation in decisions at school was important for health and well‐being. This featured prominently in the studies we included and we discuss this as a main recommendation in our report.19 Therefore, in addition to informing decisions in the review, the consultations helped ensure that the professional research perspective was not the only one brought to bear on interpreting the literature.

The first consultation with the young people's advisory group identified how wider societal structures such as social class impact on school health. According to the group, schools from areas of higher deprivation may have fewer resources and larger class sizes which could negatively affect well‐being. They also mentioned that an awareness of fellow students' social class could affect the atmosphere in a school. Amongst students in some schools there was an emphasis placed on how much your parents earn, how you speak, where you live and students could be made to feel out of place or ashamed if they were from a poorer background. This was consistent with our own concern that the review should actively search and identify data related to inequalities. Unfortunately, few data on this were available to include in our review. However, we were able to make links between data emerging from individual studies to the wider context of education policy in the review (e.g. the marketization of education and its impact on health inequalities).

Online and face‐to‐face consultations

Online consultations were not a successful engagement medium. Only two group members registered on the website and only one posted their views (consistent with that reported in face‐to‐face consultations). When asked why there was such little activity at the second group consultation, there was a feeling that a proprietary online social network was not favoured as it would require logging in for a single purpose: to comment on our research. The advisory group felt that integrating their involvement in research into a medium which they (and crucially their wider social network) were already engaged, such as Facebook, would be more desirable, in contrast to a standalone platform.

The group consultations on the other hand were successful in engaging young people and yielded rich data that helped inform key decisions in our review. All young people participated in providing views either via open discussion or smaller group or individual activities (i.e. individual voting exercise). While we did not formally evaluate this aspect of our work, young people appeared generally happy to work within the structure we provided and understood many of the terms and approaches we used, likely as a result of their training as part of the ALPHA group.

Discussion

This paper has described the process of consulting with a young people's advisory group and the impact this had on our systematic review of school health. In sum, there were a number of benefits of involving young people via face‐to‐face consultations: their participation helped ensure that issues which were important to young people were considered; they flagged ‘early signals’ of key issues for the synthesis; and provided researchers with confidence in their decision making at key stages of the review.

While face‐to‐face consultations yielded important insights for our review, the online consultations were not as successful as young people reported they were averse to signing up to social networking platforms they were not already familiar with. Nevertheless, there are a number of benefits of web‐based consultations, such as providing researchers with greater flexibility to elicit views when unanticipated issues arise and potentially being more convenient for advisory group members.32 Future research should test this approach using popular social networking sites and mobile applications and work with young people to develop the approach.

While consultation is a relatively less intensive method of involvement on the continuum of participation, we found it to be appropriate for this research. We feel that to have meaningful involvement of the public in a review, the context in which participation is sought should determine the nature of the participatory approach. Our systematic review was intentionally researcher‐led, meaning that decisions ultimately lay with researchers, and the review was carried out and disseminated by researchers. However, we wanted our focus to have relevance in the real world and be sensitized to the views of young people. Consultations were an ideal mechanism to achieve this. Thus, while young people played a role in informing the review in various ways, as outlined above, their contribution was a component of wider factors of influence, rather than a driver of decision making.

This paper has provided a description of not only the process of involving young people in systematic reviews, but also the impact of doing so on the review output, an area which has been neglected in previous accounts of participatory research.33 However, our study is not without limitations. In conducting the consultations with young people we did not have an explicit aim to critically examine the processes, challenges and opportunities of overtly participative systematic reviews. This paper was developed at the conclusion of our review and arose from reflections by members of the research team with an interest in the potential of participatory research in the context of systematic reviews. Future research should explicitly aim to investigate the process and challenges at the outset, as well as consider what young people think and feel about being involved in research, as the evidence base on this is weak.34 Generally, we felt the young people involved were able to openly express their views during discussions. This might have been because they knew one another through membership on the group, but attended different schools and/or they received training in the value and purpose of research. The presence of a trusted youth worker appeared to facilitate conversation. The ALPHA group predominantly consisted of white, middle‐class young people and thus views may not be applicable to the experiences of young people from other ‘social locations’. It would have been valuable to also illicit views from children to reflect a broader age range, but due to limited resources this was not possible. Finally, we hoped to involve young people at the end of our review to share our results but were unable to do this due to limited resources and time. Future work should plan to conduct consultations at the end of the review because stakeholders have a right to know where researchers have/have not included their views and their input could help researchers to explain and disseminate research to a wider audience, bridging the gap between the public and researchers. We feel the ALPHA group is an invaluable resource for those wishing to involve young people in research. There should be more work establishing young people's panels in different places, and the ALPHA group is a good case example of how this can work to improve research, as well as outcomes (e.g. self‐esteem) for the young people involved.35

Conclusion

This paper highlights both the process and impact of consulting with young people in a systematic review on the effects of schools on student health. We conclude that consultations via an advisory group of young people are a valuable way to carry out systematic reviews because they are based on an ethical and political framework of participation. Consultations support the decision‐making process while ensuring that the professional research perspective is not the only one bringing to bear on the literature; and may also give reviewers an early sense of what key themes are likely to emerge in the synthesis.

Conflict of interest

No conflict of interests have been declared.

Sources of funding

This project was funded by the National Institute for Health Research Public Health Research (NIHR PHR) Programme (project number 09/3002/0). Visit the PHR Programme website for more information. The views and opinions expressed therein are those of the authors and do not necessarily reflect those of the PHR Programme, NIHR, NHS or the Department of Health.