Broadening public participation in systematic reviews: a case example involving young people in two configurative reviews

Abstract

Background

Arguments supporting the involvement of users in research have even more weight when involving the public in systematic reviews of research. We aimed to explore the potential for public involvement in systematic reviews of observational and qualitative studies.

Methods

Two consultative workshops were carried out with a group of young people (YP) aged 12–17 years to examine two ongoing reviews about obesity: one about children's views and one on the link between obesity and educational attainment. YP were invited to comment on the credibility of themes, to propose elements of interventions, to suggest links between educational attainment and obesity and to comment on their plausibility.

Results

Researchers had more confidence in review findings, after checking that themes identified as important by YP were emphasised appropriately. Researchers were able to use factors linking obesity and attainment identified as important by YP to identify limitations in the scope of extant research.

Conclusion

Consultative workshops helped researchers draw on the perspectives of YP when interpreting and reflecting upon two systematic reviews. Involving users in judging synthesis credibility and identifying concepts was easier than involving them in interpreting findings. Involvement activities for reviews should be designed with review stage, purpose and group in mind. © 2015 The Authors. Research Synthesis Methods published by John Wiley & Sons, Ltd.

1. Introduction

Policy‐relevant research can inform and enlighten policy makers, practitioners and the public about the world in which we live. Because research may impact on all these groups, it is important to broaden the range of people who actively participate in decisions about research questions and processes.

There are both ethical and utilitarian arguments for involving service users and the wider public in research (Entwistle et al., 1998). Ethically, public involvement is seen as a right when a stakeholder group might be affected by the topic under consideration, or as part of good governance in ensuring transparency of decisions and accountability in the use of public funds. Instrumentally or for utilitarian reasons, involving service users and the public in the research process may lead to research that better reflect their needs and priorities. By making use of their knowledge, expertise and networks, researchers can provide more relevant, higher quality research that is more widely communicated (Barber et al., 2011; Stewart and Liabo, 2012).

Whilst the evidence‐base for public involvement in research is composed of mainly case study reflections, insights into the methods and impact of this involvement are accumulating (Brett et al. 2012; Staley, 2009). Service users and members of the public have contributed through consultations, but they have also led and co‐designed research studies (Boote et al., 2015; Turner and Beresford 2005). They have participated in analysis (e.g. Gillard et al., 2010; Tuffrey‐Wijne and Butler 2010) and in activities to interpret and communicate research findings (Brett et al., 2012). Within research programmes, as well as individual projects, they have identified and prioritised research questions and topics (Kreis and Schmidt, 2013; Oliver et al., 2015) and contributed ideas about health that have led to new research hypotheses and questions (Caron‐Flinterman et al., 2005). As Oliver et al. describe, the ultimate aim of involvement exercises is “to have outsiders influence research” (Oliver et al., 2014). They describe three types of impact possible through involvement. Firstly, that researchers and service users' perspectives may be changed (including through identification of new areas of potential study or contextualising personal experience). Secondly, research quality may be impacted. Oliver et al. cite focus, design, ethics and conduct as aspects of research, which may be affected. Finally, involvement may increase the relevance of research to policy, practice and personal concerns (Oliver et al. 2014).

Involving service users and the wider public in systematic reviews of research is even more important, because systematic reviews provide short cuts to critically appraised evidence for policy decisions that affect people's lives (Rees and Oliver, 2007). Case examples where users have been involved in systematic reviews have helped identify challenges and facilitate strategies to overcome them (Boote et al., 2015; Vale et al., 2012). Boote et al. described reviewers working with users on advisory groups and in panels, as well as through a virtual forum or using Delphi processes via email (Boote et al., 2015). We are aware of only two other studies of involvement in reviews whose questions relied on observational or qualitative studies (Rees et al., 2004; Rose et al., 2003). Both used advisory groups established at the start of the review as their mechanism for seeking input, with the second of these also being led by service‐user researchers who brought personal experience of the topic under review. Guidance for public involvement in systematic reviews (e.g. Staley 2012) tends to assume that the purpose of reviews is to identify intervention impacts, whereas many kinds of review question can be addressed using systematic methods. There is also little guidance for reviewers on how to support involvement in various review tasks.

Review tasks are affected, in particular, by the extent to which a review aims to aggregate and/or configure research findings (Gough et al., 2012). Reviews can be seen to fall on a continuum. At one end, the more ‘aggregative’ reviews primarily aim to sum the findings of similar studies so as get a more precise estimate of the direction and/or magnitude of effect and estimate the degree of confidence we have that these findings are accurate. These kinds of reviews tend to work with concepts that are already well established at the outset of the review and are often structured using the PICOT format (Population, Intervention, Comparison, Outcome, Timeframe). At the other end are the more ‘configurative’ reviews that aim to develop theory to describe the picture built up by a body of research. These reviews tend to deal with concepts that are initially less clear or emerge during the review process and to develop new conceptualisations and theories in an iterative way throughout a review. In reality, many reviews do both, for example, aggregating effect sizes with meta‐analysis, but then also developing hypotheses (configuring) through more exploratory analyses. The stages in these reviews, and where they are most appropriately open to influence from public participation, differ according to where each review lies on this spectrum. All reviews have stages where judgments must be made, around focus, inclusion, synthesis or interpretation. Any involvement of users in reviews may be affected by where user perspectives may be best utilised to inform these judgments.

An opportunity to explore these areas further arose with two systematic reviews about children and obesity. These reviews presented two challenges: how to support involvement of non‐researchers in reviews at the more configurative end of the spectrum and how to support the involvement of young people. Involving children and young people in systematic reviews is also a relatively new phenomenon, with only two previous cases known to the authors; both involved young people in reviews of intervention effects (Garcia J and the PSHE Review Group, 2006, Liabo et al., 2012). Readers interested in participatory work with children and young people may find the work of several bodies helpful (e.g. Kirby, 2004; Shaw, 2011; Powell and Smith, 2009) both as participants and through their active involvement in the planning and process of research (Brady et al., 2011 and in press). Although there is less of an evidence base in relation to children and young people's involvement in research practice compared with adults (Brownlie, 2009), the case for their involvement has been explored in a number of publications (e.g. Alderson, 2001; Kirby et al., 2003; Kellett, 2005; Shaw et al., 2011).

This paper, however, focuses in the main on our learning about involvement in configurative reviews.

Here, we describe public involvement in different stages of two highly configurative systematic reviews. We describe and reflect upon the methods of involvement used and their possible impact on the review processes and products. Reports of the two systematic reviews have been published elsewhere (Caird et al., 2011, Rees et al., 2009). A parallel consultation was also run as part of the review on obesity and attainment so as to gather teacher perspectives, but this is not reported here (for further detail, see Caird et al., 2011).

The aims of the paper are to

• describe the approaches used to elicit young people's views in relation to topics and draft findings in two configurative systematic reviews;

• describe the perspectives that were elicited and the extent to which these influenced the reviews; and

• reflect on how well these approaches worked and what this might mean for others planning similar involvement activities.

2. Methods

2.1. An opportunity for public involvement in configurative reviews

The Evidence for Policy and Practice Information and Co‐ordinating (EPPI)‐Centre, at the Institute of Education in London, produces systematic reviews in a range of public policy areas. During the period in which this work was conducted, the centre was conducting a series of policy‐relevant reviews commissioned by the Department of Health, who had oversight of the foci and scope of each review.

The EPPI‐Centre has a commitment to researching and carrying out user involvement wherever feasible and useful. In this case, an opportunity to work with a group of young people was identified. The National Children's Bureau (NCB) Research Centre, a charity working with children and young people, was running ‘PEAR (Public health, Education, Awareness, Research: our voices, our health)’, a Wellcome Trust‐funded project aiming to help young people learn about, inform and influence public health research and policy, develop links between young people and public health researchers, research bodies and policy makers, and produce and distribute information about public health issues and research to other young people (Brady et al., 2012). For more information about PEAR, please see their website www.ncb.org.uk/PEAR. This two‐year project worked in London and Leeds with two groups of around a dozen young people in each group, aged 12–17 years.

2.2. Deciding how and when to involve young people in reviews

The EPPI‐Centre and the NCB teams realised they had an opportunity to help each other meet their agendas. The EPPI‐Centre team agreed to co‐facilitate two workshops at meetings of the PEAR group to contribute to the educational aims of the PEAR programme and elicit young people's views about our ongoing systematic reviews on childhood obesity. It should be noted that the workshops were framed as public involvement exercises, rather than as research. As such, the sampling, process and ideas generated by the workshops should not be considered as equivalent to research processes. But we were also able to draw on the expertise of the group's facilitators in the National Children's Bureau Research Centre, who had considerable expertise in supporting young people's involvement in research. Moreover, in the absence of best practice guidance about involving young people in systematic reviews, we decided to try a range of approaches. These approaches are described in the succeeding texts.

A researcher from the review team [KO] was invited to two meetings, one held for each of the London and Leeds groups, to provide an introduction to systematic reviewing and involve them in ongoing reviews. The focus of the meetings was systematic reviewing, obesity and obesity research. Prior to each group's workshop, scripts and materials were developed by the research team, in collaboration with the NCB senior researcher leading the PEAR project [L‐MB].

The researchers identified stages in two ongoing systematic reviews at which involvement might be useful (Figure 1). One of the reviews sought understanding about children's (aged 4–11 years) perspectives of body size, by synthesising findings from interview‐based and questionnaire‐based research (a ‘views review’); the other aimed to bring together observational studies to explore the quantitative relationship between childhood obesity and educational attainment (a ‘correlational’ review). For the views review, the synthesis of primary research into thematic areas was almost complete. Here, it was felt that involving users could be useful in providing a check on the credibility of the synthesis and in developing review implications. For the correlational review, the review was at an earlier stage: relevant literature had been screened, but not yet synthesised. This timing meant that young people would be able to look at the variables and pathways that had been examined in the literature; this might aid the identification of new factors and so contribute to a theoretical framework to help to synthesise study findings (Figure 1).

Figure 1.

Review stages at which involvement occured.

It should be noted that these stages of involvement were decided on the basis of opportunity rather than planned a priori before the start of the review process. As such, we were consulting the young people in their capacity as an advisory group, with research knowledge as well as their own lived experience as young people. This model of standing young people's advisory groups, which researchers draw on at different stages of the research process, is the standard way in which young people, and many adults, are involved in health research (INVOLVE, n.d., Clinical Research Networks Children, n.d.). Because two reviews were ongoing and had reached different points, there was an opportunity to involve the group in different review stages.

2.3. Characteristics of young people who were involved

The PEAR group members were arguably not a ‘representative’ group of young people and were of a different age group from the children in the views studies. However, as outlined previously, the intention was not to create a stratified sample but to engage in public involvement activity in their capacity as a self‐selected advisory group, with research knowledge and their own lived experience as young people. It is also likely that they were more aware of health consequences and issues than most young people, given their research‐literacy and understanding of public health from the training and experience they had had in the group prior to attending their meetings.

2.4. Role of researchers in the consultations

Guidelines for actively involving young people in research are similar to those for public engagement in research more generally, in that they emphasise that facilitation is central and that care needs to be taken to address autonomy, ability and openness (Shaw et al., 2011). Both workshops were facilitated by two or more NCB staff known to the young people. The researcher who presented the review data was in her mid‐twenties, which we thought might help to create an atmosphere of equality. The facilitators reminded the young people of the group on ground rules at the beginning of the workshop; these included tolerance to others, manners and taking turns. Our aim was to provide an atmosphere where young people felt their views were accepted and valued and that fostered open and easy discussion. The workshops were scripted to ensure that no leading questions or prior suggestions were introduced by the researcher, and young people were assured that asking questions at any time was acceptable. A glossary was provided, and technical terms were explained at the beginning of the sessions. We sought to use terminology of an appropriate level and language, which would have been familiar to their age group, from school and outside.

The workshops lasted approximately two and a half hour each and were split into two parts. The first focused upon systematic review methods and the topic of obesity. The young people were introduced to the concept of systematic reviewing through a series of presentations and exercises, such as the application of inclusion criteria to selected studies and the assessment of relevance and methods. The use of systematic reviews for decision‐making was discussed and illustrated with reference to a consultation on obesity prevention and treatment run by the UK's National Institute for Health and Clinical Excellence (National Institute for Health and Clinical Excellence, 2006). Young people were asked what they might want to know from reviews if they were making decisions in the area of obesity and why they thought obesity was important.

The second part of each workshop aimed to elicit ideas from young people about two ongoing reviews. Methods to achieve this are described in the succeeding texts. About 65 min were allocated to the reviews. The aims for involvement exercises and the approaches used are summarised in (Figure 2 ).

Figure 2.

What approaches were used for consultation?

After each stage of the workshop, the researchers explained to the group how their contributions might be used and thanked them for their time. At the end of the workshops, the views and opinions expressed by the young people were then reported back to other members of the research team by the researcher [KO], and later, the written materials generated in the consultations were collated and written up by the NCB facilitators. At a later point, the research team also provided the PEAR group's members with a short description of how their contributions to the workshops had been used by the review team.

2.5. Involvement exercises for a synthesis of children's views about obesity

Young people were presented with a description of the synthesis of research of children's views of obesity. They have shown the review title, methods including number and type of included studies and a diagram that represented the themes (Figure 3) that had emerged from the synthesis. They were asked: “Do the findings seem credible and believable to you? Do you agree with them? Why? Or why not?” They were also asked if they thought there were any important themes missing. They were given sheets with these questions on. Young people made notes on their sheets, discussed the findings in pairs and then offered suggestions in open conversation, which were collated by the facilitator. To identify themes of importance, the young people were given stickers to put next to the five themes they thought most important. This stage was conducted to try and initiate group discussions about obesity and to help the young people to engage with the topic.

Figure 3.

Views review synthesis.

The researcher then discussed how reviews try to derive implications from the evidence base and went through an example (using Thomas et al., 2003) that illustrated how researchers developed conclusions from a research synthesis and collated findings to try and come up with learning points for policy makers. They were provided with sticky notes and asked to think about the implications of the themes. Suggestions for what to think about included “What would you like to see done [about these problems described in the review]? Who might be affected? What do you think schools, parents, teachers, etc should do?” The young people worked on this individually and then in pairs. They were asked to stick their ideas about interventions near the relevant theme on a large A3 poster of the themes. They were asked to group their ideas into clusters of similar implications. There was a group discussion following the activity. Facilitators took notes and later fed these back.

2.6. Involvement exercises for a correlational review of the relationship between obesity and educational attainment

Young people were consulted to gain their perspectives on the putative relationship between obesity and educational attainment. We wanted to see whether research had evaluated moderating variables considered to be important by young people and to illustrate gaps where this had yet to be done.

Young people were introduced to the preliminary findings of the review. First, the context, purpose and funding of the review were discussed. Young people were then asked if they thought there was a relationship between obesity and educational attainment and then asked on their own to write down factors that they thought might mediate the relationship. Next, they were given a set of cards with factors derived from the research literature and asked to rank them in order of plausibility and importance. They were also asked to group the factors into potential pathways linking obesity and educational attainment. The facilitators encouraged discussion about causative and coincidental relationships, and how relationships may be mediated by other factors. Young people made notes and offered suggestions in open conversation, discussed and revised suggestions until agreement between them had been reached.

2.7. Actions taken by facilitators

The facilitators provided the review teams with collated ideas from the workshops – specifically, the important themes identified from the views review, the notes made by the young people on the A3 diagram of the themes in the views review, their ideas for elements of interventions and the factors and links between obesity and attainment that had been proposed for the correlational review. Once back in the office, these were shared with the review team and possible next steps were discussed. Below, we describe the actions that we took.

3. Results: eliciting and using young people's perspectives in reviews

This section describes the ideas that were elicited from the young people and the actions taken by the review team. This is followed by a summary section reflecting on the overall lessons from attempts at involvement in both reviews.

3.1. Young people's ideas about the credibility and implications of the views synthesis

Young people commented on the credibility of the findings about children's views. Most young people agreed that the synthesis appeared to have covered the most important issues for children. It described how the findings appeared believable because the themes were ones that were often spoken about, in their experience. Areas that they felt might be missing from the themes were the influence of the media and the effectiveness of strategies for achieving and maintaining a healthy size. The generalisability of some findings was discussed, and it was noticed by the young people that the ethnicity of children who participated in studies included in the systematic review was not clear from the synthesis diagram. Finally, in response to the questions about what they thought should be performed, a list of potential ways to prevent obesity was also produced and a grouping of these ideas provides some insights into young people's perspectives on how obesity might be addressed. The young people identified a range of potential elements of interventions, sometimes focusing solely on outcomes, for example, the need to improve attitudes to body size diversity. They had also suggested parents and teachers were important influences, along with the media and schools. They had sometimes proposed very specific interventions (e.g. ‘provide cheaper, healthier foods in schools – fruit’, ‘co‐curricular activities to help children get physical (maybe compulsory once a week)’). The review team noted that, between them, they had a sophisticated understanding of interventions, for example, that interventions could have multiple, and even undesirable outcomes, or that different strategies might be needed for different age groups.

3.2. Actions taken by the views review team

The review team used the ideas provided by the young people to check the draft synthesis. The review's raw data (quotes and author interpretations in the included studies) were checked for the presence of ideas about media influences and intervention effectiveness (mentioned by the young people, but not present in the diagram of the synthesis themes), to see if these had been overlooked or had not been given sufficient emphasis in the themes and accompanying narrative.

Children in the studies had not talked about the effectiveness of action aimed at healthy body sizes; however, they had referred in several instances to the media. Children had made references to celebrities or to television programmes, and these appeared in the draft report under the themes ‘acceptable and ideal bodies’, ‘body comparisons’ and ‘generalised anxiety and pressure’. However, study authors had not described or quoted children as talking specifically about feeling influenced by the media. It did not seem appropriate, therefore, to turn this idea of media influence into a theme or modify the synthesis structure further.

We did not use the information on themes considered important by the young people. This exercise was useful in promoting group discussion, but we felt it was not appropriate to include any such ranking data in the final report.

The second aim was to use the consultations to help develop implications for the review. The research team read and discussed the elements of interventions that the young people had proposed. Although this was done before the review's implications section was written, the lead reviewer, when she was writing up the implications, had difficulty interpreting these proposed elements. Arguments that the young people may have used to link the synthesis themes and their suggestions were not available to her, especially when these ideas had been written in very specific terms (e.g. provide weekly compulsory physical activity in schools) next to themes that felt more general in nature – being about, for example, how children in the included studies had talked about the consequences of obesity. At other times, the comments seemed to run counter to the ideas presented in the synthesis (e.g. ‘Encourage children to take responsibility for their own health’ – whereas the synthesis themes represented findings that saw children: (i) feeling generalised anxiety and pressure to be thin and (ii) assessing their own sizes and being critical about their perceived size or dissatisfied. These initial responses, elicited in a short amount of time without opportunity for in‐depth discussion, were not sufficiently well‐developed to include as implications explicitly based on a synthesis.

In the end, the views of the children represented in the review's included studies were taken as the starting point for the recommendations. The instrumental implications of the synthesis are needed to be framed mainly in terms of the range of outcomes that interventions might need to target (the need to consider self‐esteem and assertiveness, understanding about multiple influences on size and beliefs about the extent of individual control over size in addition to healthy behaviours and body size).

For these reasons, this part of the involvement exercise was felt to be less successful, in that it did not usefully contribute to developing implications from the review.

3.3. Young people's ideas about the topic of the correlational review

Few young people initially thought that obesity and attainment were associated. However, they all considered obesity and attainment to be important issues for them. They showed an awareness of the issues that influence how they think and feel about obesity. When prompted, they considered a range of causes and effects that might be associated with obesity. The young people's suggestions fell into two groups. Some suggestions made by the young people appeared to be observed links (e.g. deprived areas tended to have few places for exercise), whereas others were more explanatory (Playing computer games more makes people less active; the more time spent playing computer games, the less time is spent studying). Young people considered parental influence and circumstances (including family income and poverty) and bullying and emotional health to be the most important factors that might explain an association between obesity and educational attainment.

3.4. Actions taken by the correlational review team

Following the workshops, the review team thematically grouped and tabulated the new factors and mechanisms suggested by young people, and those examined in the academic literature included in the review.

The reviewers employed a coding framework devised using factors and mechanisms identified at the protocol stage (from the background literature) and those emerging from the studies being reviewed. They checked that this framework captured all of the young people's ideas. They subsequently compared the factors and mechanisms examined in the academic literature with those identified by the young people during the workshop.

Whilst young people identified low academic attainment resulting from poor mental and emotional health among obese children as an important causal pathways, only four of the 29 included studies made adjustment for mental and emotional health variables. Bullying and victimisation of obese children were also considered to be a highly credible moderating variable of the relationship between obesity and educational attainment by young people. The reviewers looked into the literature for any mention of any form or discrimination or stigmatisation, including bullying, as part of a causal pathway linking obesity and attainment. Whilst six included studies suggested that there may be a causal pathway between obesity, discrimination/stigmatisation and low academic performance, no studies adjusted for bullying or victimisation as moderating variable. These findings represent a significant divergence in the perspectives of young people and researchers. However, it is possible that differences in meaning and interpretation of language and concepts lead to this divergence, although the review team believes that reference to bullying or victimisation would have been picked up by the review team's use of discrimination/stigmatisation as an umbrella coding category.

The discrepancy between young people's understandings of how obesity and attainment are linked and the body of research evidence prompted the reviewers to focus part of the review's discussion on gaps in the evidence‐base. The team produced implications, one emphasising how the variables used in statistical analyses failed to capture many of the potential causal factors identified by young people. It was argued that if large observational studies are to help us understand people's lives, they need to engage with the social lives of their participants and amass data that reflect the people's social experiences, as these may be key to understanding health and other behaviours.

3.5. How young people's ideas contributed to the two systematic reviews

In this section, we summarise the results presented previously about the young people's ideas, the impact of these contributions on the reviews and reflections about the process. A summary is provided in Figure 4.

Figure 4.

How did the consultations with young people influence the conduct and interpretation of reviews? YP, young people.

In the views review, the aim was to carry out a credibility check and to develop implications. Although this first aim was relatively successful, resulting in arguably greater credibility for the review's findings, the second part did not work as well, as described previously, and this aspect of the PEAR group's work made no identifiable contribution. Factors limiting this success were a lack of time and a mismatch between the tasks and the nature of the review.

The production of a qualitative synthesis is a complex task requiring immersion in the data, which was just not possible in the time available. Instead of the richer, thicker account of the synthesis that was produced in time for the review's final report, the young people in the consultation were presented with a diagram of the labels for the different themes in the synthesis. The interpretive synthesis contained a very large number of themes (19), which would have been difficult to interpret, and there was little time for interaction between the young people and the researcher who might have been able to further explain the ideas behind the labels – and the review author who drafted the review's implications section was also not present at the workshops. In addition, the method used to elicit young people's ideas focused on identifying interventions that seemed appropriate, given the review's findings. The review, following the original scope agreed with the funder, set out to explore the nature and boundaries of a ‘problem’ (included studies could look at children's views on being too thin, for example, as well as ideas about being overweight; there was no focus on any particular group of outcomes). The synthesis was broad, including the values and consequences that children attached to body size, and descriptions of experiences that children related to their own or others' body size. Often, the views of children in the included studies had not been presented in much detail, and they had not themselves been asked what they thought should be done. Overall, whilst the young people in the consultation identified creative ways in which the problem of obesity might be addressed, it was felt these were too far from the original aims for the review. For these reasons, tying the two sets of information together (the young people's ideas about interventions from the workshops and the review's synthesis of study findings) seemed inappropriate.

In the correlational review, an initial potential aim had been to help develop a conceptual framework from the links and mechanisms suggested by young people. In practice, this was not possible. However, once the reviewers had synthesised the included studies, it was possible to use the ideas elicited from the young people (by comparing them with the ideas in the included studies). One of the review's three implications pertained to gaps in the evidence‐base that otherwise might not have been identified by the review team. This comparison was a simpler task and perhaps more suited to how the ideas had been recorded.

4. Discussion

A brief consultation workshop with young people about two systematic reviews in progress increased researchers' trust in the synthesis of view's research and increased their sensitivity to issues important to young people and thus enabled us to see gaps in the literature about bullying and victimisation that may link obesity and attainment.

This paper has described the use of a consultative workshop to consult with a group of young people on two systematic reviews that asked exploratory questions about children and obesity. As outlined in Oliver et al., (2014), involvement may impact on research through changing perspectives of researchers or users, many influence research quality or the research may become more relevant to policy, practice and personal concerns (Oliver et al., 2014). We argue that these consultative workshops changed the review teams' perspectives, influenced the quality of a synthesis and made the analysis of the correlational review more relevant to young people's priorities. We also suggest that this process had wider influences on the young people and on the review teams through a future work.

As argued previously, the review processes and products were different as a result of involving young people. The qualitative synthesis was given credibility, and insight was gained into the types of interventions that young people consider useful – often, despite the topic being obesity, these were not health‐related. The young people provided a set of additional perspectives that could challenge any pre‐conceived conceptual framework held by the reviewers when drawing implications from the review. Involvement of young people in the review of observational studies enabled reviewers to critique research addressing the link between obesity and attainment according to the perspectives of people who have the closest experience of both. Both systematic reviews became more likely to be relevant to young people and young people's priorities. The reviewers gained confidence in the credibility of the views synthesis. Thus, the consultation changed the perspectives of the reviewers and improved the quality of the synthesis. For the correlational review, the young people raised questions about the exhaustiveness of the evidence base, enabling the reviewers to identify potential gaps in the literature that, although outside the scope of the systematic review in question, could inform future work. Thus, the analysis was able to become more relevant to young people's priorities.

The consultations may have influenced the researcher's perspectives. On reflection, such a change would seem more likely when there is more opportunity for discussion and direct interaction. The consultations could be seen as the start of a longer term discussion, because researchers from the EPPI‐Centre and the PEAR group met further time, 18 months after the workshops, described here to discuss the draft findings of a subsequently commissioned review of the views of young people (aged 12–18 years). This second consultation was able to spend more time discussing the substance of the subsequent review, partly because the PEAR group's members had already familiarised themselves with systematic review methods in the workshop described in this paper.

It should also be noted that the PEAR group appreciated the detailed feedback they were provided with after the workshop and gained from the workshops themselves. For more details about the PEAR project and its evaluation, please see the PEAR evaluation report (Davey 2011), and for further reflections on the process and lessons learned, please see Brady et al. (2011), which was co‐written with some of the young people (Brady et al., 2011).

We have identified some limitations to our approach and some potential lessons learned. It was a challenging though valuable experience, increasing our understanding of the methodological problems in involving users in systematic reviews. Challenging, because, due to time constraints, only one reviewer was able to attend both workshops so that young people's ideas were filtered through this individual to the rest of the team who were thus removed from the process. Because of the nature of the group, the reviews had limited opportunity to question the young people about their ideas, which, like any creative activity, required time to develop. In addition, because the researchers attended the PEAR group as guests, they did not have direct access to the raw data (posters, post‐it notes etc.) collected by the facilitators. The methods of involvement inevitably influenced the ideas we elicited. For example, in the views review, the young people had been asked about potential interventions, as well as asked to consider meanings and experiences, rather than to engage with a wider discussion about experiences of body size. Additionally, in the correlational review, although every chance was given for young people to develop their own ideas before reflecting on the research‐derived links, it is possible that the format of the methods and the fact of raising the question of whether there is a link itself created bias. The facilitators contributed to the discussions and were asked, for example, to give their own opinions, which may have influenced their responses. More success may have come from greater direct interaction and more systematic data collection between those to be involved and the review team, with each seeking clarification and further explanation where necessary and with the review findings acting as a reference point for two‐way discussion.

The workshops built on the existing involvement expertise of the NCB and the EPPI‐Centre. The time taken up by the consultations was short, but an NCB facilitator and the researcher who attended the workshops spent up to half a day each planning and developing materials for the activities beforehand. Only one researcher attended the workshops, and she was not trained extensively in participation methods. The preparation for the workshops was undertaken jointly, but again was not time‐consuming or requiring of expertise – although with greater time allowed, a more considered approach could have been taken. Both groups (researchers and PEAR) learned from the process. However, we were not able to pilot the methods, and given the lack of empirical data about involving young people in research, we had little to draw on from others' experiences.

There were advantages to having such an engaged, research‐literate and public health‐literate group of young people to work with on the reviews – logistically and pragmatically, it was easier to slot into an ongoing programme of work, which PEAR was carrying out rather than trying to organise' focus groups or other data collection with a new group of ‘users’, which would have been research data collection or consultation rather than active public involvement (INVOLVE, n.d.). The involvement exercises described previously were facilitated by the existing commitment of these young people to the PEAR group, involving outsiders in these reviews made our thinking more explicit throughout the review process, and the match of that group's agenda with the wishes of the review team to work with young people meant that the young people were able to engage with these two reviews in an efficient way.

Active involvement is not based around the need to produce generalizable findings and so does not seek representative samples of people. It is worth noting, however, that the young people were older than the children in the studies, nor did they – unlike many other user groups – have a specialist interest in childhood obesity. Whilst any difficulties prompted by the stigma of obesity were hopefully pre‐empted by the early discussion of workshop ground rules (respect and tolerance), obesity is unavoidably a sensitive issue, and this may have made sharing ideas in the workshops more challenging. The young people were mainly not overweight or obese, which may have influenced their perceptions of the review findings or limited their ability to empathise with the obese children under discussion. Elsewhere, it has been noted that not all children and young people have equal opportunities for participation (Brady et al., 2011). It therefore could have been valuable to consult with additional young people.

Despite these limitations, we have shown that rapid consultation methods are helpful if the aim is to generate hypotheses, to identify priorities for review topics, or to check validity of a research synthesis.

5. Conclusion

Consultative workshops helped researchers draw on the perspectives of young people when interpreting and reflecting upon the findings of two configurative systematic reviews. Involving this group in generating hypotheses and testing synthesis credibility was easier than involving them in interpreting findings. Involvement activities for systematic reviews need to be designed with the review stage, purpose and group being involved in mind.

Competing interests

The authors declare no competing interests.

Contributions

K. O. had the idea for the paper and prepared the first draft of the manuscript. She presented the systematic reviews to the young people and invited their involvement at the workshops and analysed the workshop data. She was also a reviewer on both systematic reviews.

K. O. and S. O. designed the workshop activities; K. O. and L. M. B. delivered them with assistance from local facilitators.

R. R. and J. K. led a review each, about children's views of obesity and about obesity and attainment, respectively, including reflecting on the influence that young people had on their review.

S. O. devised the methods for the workshop with K. O. and contributed to the review about obesity and attainment. She set the young people's influence of the two reviews in the context of systematic review methodology.

J. T. is PI and is responsible for the programme of work as a whole. He contributed to planning and devising the reviews' methodology and the timings of the participation activities.

All authors critically reflected on user involvement in systematic reviews and contributed to the manuscript.

Oliver, K. , Rees, R. , Brady, L.‐M. , Kavanagh, J. , Oliver, S. , and Thomas, J. (2015) Broadening public participation in systematic reviews: a case example involving young people in two configurative reviews. Res. Syn. Meth., 6: 206–217. doi: 10.1002/jrsm.1145.