Involving children and young people in paediatric research priority setting: a narrative review

Abstract

Objective

The objective of this study is twofold: first, to describe the methods used when involving children and young people (CYP) in developing a paediatric research agenda and, second, to evaluate how the existing literature describes the impact of involving CYP. We distinguish three forms of impact: impact on the research agenda (focused impact), impact on researchers and CYP (diffuse impact) and impact on future research (research impact).

Design

A narrative review of MEDLINE, PsycINFO, Web of Science and Google Scholar was conducted from October 2016 to January 2022. The included studies involved at least one CYP in developing a research agenda and were published in English.

Results

22 studies were included; the CYP involved were aged between 6 years and 25 years. Little variation was found in the methods used to involve them. The methods used were James Lind Alliance (JLA) approach (n=16), focus groups (n=2), workshop (n=2), research prioritisation by affected communities (n=1) and combined methods (n=1). Impact was rarely described: focused impact in nine studies, diffuse impact in zero studies and research impact in three studies.

Conclusion

This study concludes that the JLA approach is most frequently used to involve CYP and that all methods used to involve them are rarely evaluated. It also concludes that the reported impact of involving CYPs is incomplete. This study implies that to convince sceptical researchers of the benefits of involving CYPs and to justify the costs, more attention should be paid to reporting these impacts.

Key messages.

• This study provides an overview of different methods used when involving children and young people (CYP) in research priority setting.

• The James Lind Alliance method is most frequently used to involve CYP in priority setting even though the method is rarely evaluated.

• This study shows that little is reported about the impact of research priority setting with children or young people.

• Implementation plans of research agendas are rarely described, while it is considered a waste of resources should the project end with publishing the research agenda.

Introduction

The idea that children should be treated as passive subjects in research is changing. They are more and more involved as active agents.¹ The involvement of children is now recognised as a best practice and is an essential requirement for paediatric research funding allocation by funders in the UK, Australia, the USA and the Netherlands.^{1 2}

Children should be involved in every phase of the research, starting with what research should be about, in so-called research agendas. Paediatric research agendas used to be predominantly developed by professionals and researchers.³ Increasing evidence illustrates that research questions prioritised by professionals may not be aligned to those experiencing the disease.⁴ At worst, this results in limited research money being spent on research that is not important to patients, and money is wasted.³ This raised a call for collaboration with children and young people (CYP) as equal partners to develop research agendas.

Thus far, the involvement of CYP in developing research agendas appears to be limited. Few studies purely include CYP in developing those agendas. More often, adults act as a proxy for CYP’s views.⁵ A systematic review by Odgers and colleagues published in 2017 showed that 25% of studies reported some parental or caregiver involvement. Only in 5% of the studies were children involved directly.⁶ This is partly explained because there is no agreement on what might constitute best practice for involving CYP in developing a research agenda.⁷ Moreover, the involvement of CYP may bring age-specific barriers and challenges such as increased workload, unknown impact on the research agenda and power imbalances.⁷

Efforts to develop engaging and developmentally appropriate strategies that involve CYP in developing a research agenda are lacking. The most well-known example is the James Lind Alliance (JLA) method. The JLA unites patients, carers and clinicians to identify and prioritise the top 10 unanswered research questions in so-called priority setting partnerships (PSPs). Odgers and colleagues question the extent to which the JLA method may be well suited to involve CYP, although they do not clarify this claim.⁶ Previous studies have not dealt with identifying what methods are well suited to involve CYP in PSPs.⁸

One of the most significant discussions about involving CYP is that the impact of their involvement is not clear.⁹ Reasons for assessing this are numerous: to improve the involvement of CYP, to convince sceptical researchers of its benefits, to reduce tokenistic involvement, to justify the cost of the involvement of CYP, and to increase funding for their involvement.¹⁰ Therefore, it is strongly recommended to conduct more research that critically examines this impact.^{11 12} We distinguish three forms of impact, of which the first two were described before.¹³ (1) The effect of the involvement of CYP on the research agenda (focused impact), (2) The effect of the involvement of CYP on researchers and CYP themselves (diffuse impact) and (3) What is reported on action plans for assessing the effect of the research agenda on future research (research impact). Assessing these forms of impact may be challenging but documenting the contributions and incorporations of these contributions into the research priority setting may be feasible and would be welcomed by many contributors.¹⁰ This paper has two key aims. First, we will identify the methods used to involve CYP in formulating a research agenda and perform a first exploration on the evaluation of these methods. Second, the study aims to assess what is reported about the impact of involving CYP in research priority setting.

Methods

We conducted a narrative review to gain a qualitative perspective on the methods used to involve CYP in developing a research agenda and the reported impact of this involvement.

Search strategy

The research team cocreated the literature search strategy in collaboration with an information librarian. We used the Medical Subject Headings and text words for ‘children’, ‘priority setting partnerships’ and ‘research agenda’. Online supplemental file 1 provides more details about the search strategy. Each search term within the three categories were combined with the Boolean operator “OR” and the three different categories were combined with the Boolean operator “AND”. Databases searched were MEDLINE, EBSCOhost, Web of Science, Google Scholar and the JLA website. The included articles were uploaded in the programme Rayyan QCRI (Qatar Computing Research Institute (Data Analytics), Doha, Qatar) and duplicates were removed.

Study selection

The research team specified the inclusion criteria after a thorough consultation. Articles were included in this review if developing a paediatric research agenda with the involvement of at least one CYP aged below 18 years was reported and if the articles were written in English and were published between October 2016 and March 2022 (to add more research agendas that have been developed with CYP to the four already identified by Odgers and colleagues).⁶ For the inclusion, we have chosen a three-step approach: (1) the first author screened the title and abstracts of 557 articles; (2) all articles for which it was unclear whether they should be included were intensively discussed with the last author; moreover, the articles that were already included were discussed in detail; (3) in the final step, the inclusion was discussed with the research team. The same three-step approach was chosen for the inclusion of the 89 full-text articles.

Data analysis

A narrative synthesis was performed. To systematically describe data from the included studies, two data extraction forms in Microsoft Excel were developed. Descriptive information of the studies (eg, title, authors and method used to involve CYP) were reported on the first data extraction form. The second form was developed to chart data on the age and the number of the CYP involved, the phase of the involvement and the impact of the involvement. To assess the impact of the research priority setting, we divided impact into three forms: focused impact, diffuse impact and research impact. The data were extracted by LP and discussed with the research team.

Checklist

We used the 32-item checklist developed by Odgers and colleagues to assess the transparency of reporting of research priority setting. They extracted items from good practice principles to develop the checklist. Another frequently used checklist, the Guidance for Reporting Involvement of Patients and Public checklist (GRIPP2),¹⁴ was developed to help improve the quality, consistency and transparency of reporting patient and public involvement in research. The checklist of Odgers differed from the GRIPP2 checklist in that it was developed to assess the reporting of research priority setting specific. Therefore, we decided to use the checklist of Odgers instead of the GRIPP2 checklist.

The original checklist of Odgers was not developed to specifically assess the reporting on developing a research agenda together with CYP. Therefore, we added three items to make sure the checklist covers important aspects of involving CYP. Next, the items will be further explained. The first item, ‘describes the method used to involve CYP in developing a research agenda’, was added to the list because we agree with Flynn and colleagues that appropriate strategies that involve CYP are lacking.¹⁵ The second and third items were added to the list to assess different forms of impact: ‘describe the impact of the involvement of CYP on the research agenda’ (focused impact) and ‘describe the impact of the research priority setting on the participants (diffuse impact). We rephrased the original item 29: ‘describe how impact will be measured’ as ‘describe how the impact of the research agenda on future research will be measured’ (online supplemental file 5).

Results

Twenty-two studies were included in this review (figure 1). Most of the studies were conducted in the UK (n=13) (online supplemental file 2 and figure 1). The CYP involved were aged between 6 and 25 years. Seventeen studies involved children below the age of 18, and 2 studies did not report the age of the CYP involved. The number of the CYP involved in the included studies ranged from 1 to 108. Four studies did not report the number of CYP involved (see tables 1 and 2).

Figure 1.

Preferred Reporting Items for Systematic Reviews and Meta-Analyses flowchart.

Table 1.

Included studies that used the JLA approach

Authors (year)	Title	Topic	Children/young people	Method	Country
Obeid et al16 (2020)	Cocreating research priorities for anorexia nervosa: the Canadian Eating Disorder Priority Setting Partnership	Anorexia nervosa	15–25 years: steering committee (n=1), first survey (n=33), workshop (n=3)	James Lind Alliance	Canada
Knight et al27 (2016)	Defining priorities for future research: results of the UK Kidney Transplant Priority Setting Partnership	Kidney transplantation	<18 years: (n=1) and 18–24 years (n=2) in prioritisation	James Lind Alliance	UK
Verwoerd et al17 (2021)	Dutch patients, caregivers and healthcare professionals generate first nationwide research agenda for juvenile idiopathic arthritis	JIA	10–15 years: focus group meetings with children with JIA; focus groups are implemented especially for children	James Lind Alliance	The Netherlands
Grant et al18 (2019)	Engaging patients and caregivers in research for paediatric inflammatory bowel disease: top 10 research priorities	Paediatric Inflammatory bowel disease	111 patients with IBD, ages between 10 and 85 years included in solicitation survey and 25 patients with IBD, ages between 11 and 35	James Lind Alliance	Canada
Fackrell et al19 (2019)	Identifying and prioritising unanswered research questions for people with hyperacusis: James Lind Alliance hyperacusis priority setting partnership	Hyperacusis	0–4 years: prioritisation (n=4), 10–20: identification (n=7), prioritisation (n=11)	James Lind Alliance	UK
Morris et al20 (2018)	Identifying primary care patient safety research priorities in the UK: a James Lind Alliance priority setting partnership	Primary care patient safety	16–24 years: first survey (n=4) and second survey (n=5)	JLA	UK
Rankin et al21 (2020)	Identifying priorities for physiotherapy research in the UK: the James Lind Alliance physiotherapy priority setting partnership	Physiotherapy	Identification 9–88 years, prioritisation 17–89 years	JLA	UK
Hollis et al31 (2018)	Identifying research priorities for digital technology in mental healthcare: results of the James Lind Alliance priority setting partnership	Digital technology in mental healthcare	Identification <15 (n=6) and 16–24 years (n=63), prioritisation <15 years (n=3) and 16–24 years (n=62)	JLA	UK
Lim et al26 (2019)	Joint production of research priorities to improve the lives of those with childhood onset conditions that impair learning: the James Lind Alliance priority setting partnership for ‘learning difficulties’	Childhood conditions that impair learning	<25 years: (n=41) in prioritisation and (n=5) in the final workshop	JLA	UK
Birnie et al22 (2019)	Partnering for pain: a priority setting partnership to identify patient-oriented research priorities for paediatric chronic pain in Canada	Paediatric chronic pain	<18 years: national survey (n=33), prioritisation (n=6) priority setting workshop (n=3)	JLA	Canada
Ismail et al30 (2021)	Research priorities and identification of a health-service delivery model for psoriasis form the UK psoriasis priority setting partnership	Psoriasis	Identification <16 years (n=7) and 17–24 years (n=33), prioritisation<16 (n=7) and 17–24 years (n=67)	JLA	UK
Peeks et al29 (2019)	Research priorities for liver glycogen storage disease: An international priority setting partnership with the James Lind Alliance	Liver glycogen storage disease	Median age 12 (n=unclear)	JLA	The Netherlands
Lam et al25 (2019)	Research priorities for the future health of multiples and their families: the Global Twins and Multiples Priority Setting Partnership	Health priorities for multiples and families	<20 years: (n=4) survey 1 and (n=1) survey 2	JLA	UK
Aldiss et al23 (2019)	Research priorities for young people with cancer: a UK priority setting partnership with the James Lind Alliance	Young people with cancer	13–24 years: first survey (n=108), second survey (n=58), workshop (n=7), steering group (n=5)	JLA	UK
Vella-Baldacchino et al24 (2019)	Research priorities in children requiring elective surgery for conditions affecting the lower limbs: a James Lind Alliance priority setting partnership	Children requiring elective surgery for the lower limbs	Workshop (n=4), no age specified	JLA	UK
Finer et al28 (2018)	Setting the top 10 research priorities to improve the health of people with type 2 diabetes: a diabetes UK James Lind Alliance Priority Setting Partnership	Diabetes type 2	First survey <20 years (n=5)	JLA	UK

IBD, inflammatory bowel disease; JIA, juvenile idiopathic arthritis; JLA, James Lind Alliance.

Table 2.

Included studies that used other methods than the James Lind Alliance approach

Authors (year)	Title	Topic	Children/young people	Method	Country
Schilstra et al34 (2021)	‘We have all this knowledge to give, so use us as a resource’: partnering with adolescent and young adult cancer survivors to determine consumer-led research priorities	Cancer	19–22 (n=4) workshop	Workshop and survey	Australia
Shattuck et al36 (2018)	A national research agenda for the transition of youth with autism	Youth with autism	Young adults, no age specified (n=2), involved in national research agenda meeting	Scoping review, stakeholders interview, 2-day national research agenda meeting, Delphi survey and two reviews	USA
von Scheven et al35 (2020)	Research questions that matter to us: priorities of young people with chronic illnesses and their caregivers	Young people with chronic illnesses	15–18 years (n=6) and 21–22 years (n=5)	Research prrioritisation by affected communities method	USA
Lopez-Vargas et al33 (2018)	Research priorities for childhood chronic conditions: a workshop report	Childhood chronic conditions	8–14 years (n=3)	Workshop	Australia
Manikam et al32 (2016)	Using a co-production prioritisation exercise involving South Asian children, young people and their families to identify health priorities requiring further research and public awareness	South Asian children and health priorities	16–24 years: number not specified	Focus groups	UK
Parsons et al9 (2017)	What do young people with rheumatic disease believe to be important to research about their condition? A UK-wide study	Young people with rheumatic disease	11–15 years (n=30) and 16–24 years (n=33), all involved in different focus groups	16 focus groups	UK

Checklist

The transparency of reporting score was averaged across the studies. The scores of those included ranged from 11 to 27 items out of 36 items (online supplemental file 3, figure 2). Strikingly, few studies reported the impact of the CYP on the agenda (n=9), the action plans for implementing priorities (n=8), the evaluation of the PSP (n=6), methods used to involve CYP (n=5) and how impact of the research agenda will be measured (n=3). No studies reported how the feedback was integrated and whether the research priority setting impacted the participants (online supplemental file 3, figure 3). The completed checklist can be found in table 3.

Table 3.

Checklist of Odgers (adjusted)

	Yes	No
Context and scope
1. Define geographical scope.	9 16–23 25–36	24
2. Define health area or focus.	9 16–36	None
3. Define end users of research.	9 16–36	None
4. Define the target audience.	17 20 26 35	9 16 18 19 21–25 27–34 36
5. Identify the research focus.	9 16–36	None
6. Identify the type of research question.	9 16 17 19–23 25–27 29–34 36	18 24 28 35
7. Define the time frame.	None	9 16–36
Governance and team
8. Describe selection of the project leader/s and team.	16 17 20 24 27–31	9 18 19 21–23 25 26 32–36
9. Describe the characteristics of the project leader/team.	16 17 22 23 25 26 28–31	9 18–21 24 27 32–36
10. Training or experience in research priority setting.	16–21 23–31	9 22 32–36
Inclusion of stakeholders
11. Define the inclusion criteria for stakeholder groups involved in the priority setting partnership.	9 18 20 22 23 25 27 32–35	16 17 19 21 24 26 28–31 36
12. State the strategy or method for identifying and engaging.	9 16–36	None
13. Indicate the number of participants and/or organisations involved.	9 16–36	None
14. Describe the characteristics of stakeholders.	9 16–23 25–36	24
15. Time investment of the stakeholders	16 17 19 22–27 34–36	9 18 20 21 28–33
16. Reimbursement for participation	9 22 33–35	16–21 23–32 36
Identification and collection of research topics
17. Describe methods for collecting all research topics or questions.	9 16–36	None
18. Describe methods for collating and/or categorising topics.	9 18 19 21 23 27–29 31 33 35	16 17 20 22 24–26 30 32 34 36
19. Describe methods or reason for initial removal of topics or questions.	16–23 25–30	9 24 31–36
20. Describe methods for refining research questions/topics.	16 18–23 26 27	9 17 24 25 28–36
21. Cross-check to identify if research questions have been answered.	16–21 23–27 29–31 36	9 22 28 32–35
22. Describe number of research questions/topics.	16–31 33 35 36	9 32 34
Prioritisation of research topics
23. Describe specific methods to involve children	9 17 18 25 26	16 19–24 27–36
24. Describe methods for prioritising or achieving consensus.	9 16–31 33–36	32
25. Provide reasons for excluding research topics/questions.	21 26–31	9 16–20 22–25 32–36
Output
26. Define specificity of research priorities.	9 16 17 20–24 26 28–36	18 19 25 27
Evaluation and feedback
27. Describe how the research priorities exercise was evaluated.	9 16 17 22 26 34	18–21 23–25 27–33 35 36
28. Describe how priorities were made accessible by stakeholders	20 24 26–29	9 16–19 21–23 25 30–36
29. State how feedback was integrated.	None	9 16–36
Dissemination, translation and implementation
30. Outline the strategy or action plans for implementing priorities.	17 18 20 24 26 28 29 31	9 16 19 21–23 25 27 30 32–36
31. Describe how the participant impacted the research agenda.	17–19 22 24 27–29 33	9 16 20 21 23 25 26 30–32 34–36
32. Describe how the research the research priority setting process impacted the stakeholders.	None	9 16–36
33. Describe how impact will be measured.	26 28 29	9 16–25 27 30–36
Funding and conflict of interest
34. State sources of funding.	9 17–25 28 30 31 33 34 36	16 26 27 29 32 35
35. Outline the budget and/or cost.	None	9 16–36
36. Provide declaration of conflict of interest.	9 16 17 19–21 23 24 26 28–31 33–36	18 22 25 27 32

Methods used in paediatric priority setting

Little variation was found in the methods used to involve CYP in paediatric research priority setting. The JLA approach was the most frequently used method (n=16).^16–31 This was followed by focus groups (n=2),^{9 32} a workshop approach (n=2),^{33 34} the research prioritisation by affected communities (RPAC) method (n=1). The RPAC method directly involves individuals from under-represented groups in identifying, ranking and prioritising their unanswered questions about their health conditions.³⁵ In one study, different methods were combined³⁶ (online supplemental file 4, figure 4).

The JLA method divided the involvement of children into four phases. A total of 358 children were involved in the identification of research questions16 17 19 20 22–25 27; 287 children were involved in the prioritisation of research questions16 17 19 20 22 23 25–27; 38 children were involved in the prioritisation workshop17 22–24 26 33 34 36 and seven children were involved in the steering group^{16 18 23} (online supplemental file 4, figure 5). To ensure the involvement of paediatric patients of all age categories, Verwoerd and colleagues added focus groups with children in all phases of the JLA method.¹⁷ Similarly, Grant and colleagues organised additional focus groups for younger participants but only at the identification phase.¹⁸ Nonetheless, Lim and colleagues found that focus groups were problematic for the younger participants; therefore, they were contacted individually.²⁶ The advantages of the JLA were as follows: it is a rigorous method for the establishment of priorities¹⁶; CYP reported their involvement as positive and powerful^{16 23}; and it fulfils many of the criteria for good practice in priority setting.²⁷ Examples of the criteria that have been used were using a comprehensive approach and inclusiveness of stakeholders.³⁷ Disadvantages of the JLA were as follows: prioritisation in this manner is highly subjective^{16 18}; CYP are less represented in almost all phases of the priority setting process20 23 26 27; and researchers themselves need to refine the research questions.²⁵

Two studies used focus groups to involve CYP.^{9 32} Manikam and colleagues organised two focus groups involving 7–10 CYP.³² They were asked to prioritise research topics that were submitted by healthcare professionals. Parsons and colleagues organised thirteen focus groups, in which a total of sixty CYP were involved.⁹ In these focus groups, CYP were asked to identify the research questions themselves. No advantages or disadvantages were reported using focus groups to involve CYP.

A workshop was used to involve CYP by two research teams.^{33 34} Both teams used the JLA method as a basis for their workshop. Lopez-Vargas and colleagues organised a workshop in which CYP first had to present their prepared research questions and then had to vote for their top three priority questions.³³ Schilstra and colleagues used the workshop to clarify why each priority mattered to the CYP and how they would address the priorities. This approach extended the impact of survey-based approaches by enabling CYP to compare their experiences and actionable research questions were developed.³⁴ In contrast, survey-based approaches may require less of the CYP’s time than workshops. Furthermore, Schilstra and colleagues found that recruitment to an in-person workshop can be challenging and time-consuming.³⁴

Another method used to involve CYP was the RPAC.³⁵ Following the RPAC method, two focus groups were organised. In the first focus group, individuals shared their experiences and generated a list of research questions. In the second focus group, individuals prioritised the topics they want researchers to focus on. In both focus groups, 11 CYPs were involved. An advantage of the RPAC is that it was developed to directly involve patients using their personal experiences rather than beginning with survey data.³⁵ No disadvantages were reported.

Reported impact of paediatric priority setting

This study focused on three forms of impact: focused impact, diffuse impact and research impact. Diffuse impact was not described at all.

In nine studies, the focused impact was described.17–19 22 24 27–29 33 Focused impact of the included studies can be divided into two categories: different research questions and different research priorities. In the first category, CYP have different research questions than researchers have. In the second category, CYP have the same research questions, but they prioritised the questions differently than the researchers did (table 4).

Table 4.

Description of focused impact

Study	Focused impact
Knight et al27	‘A number of questions considered during the process were submitted by non-professionals and would not have been considered without their involvement’.	Different questions
Verwoerd et al17	‘For both patients and carers, 60% of the questions were selected, for clinicians it was 80%. For the focus groups 2 out of 5 were parts of the final top 10.”
Lopez-Vargas et al33	"For children, there was an emphasis for research to help them maintain a sense of normality and to be empowered for self-management and partnership in care.”
Vella-Baldachchino et al24	“While the surgeon’s questions focused on the management of specific conditions, the JLA PSP top priorities also included other questions.”
Grant et al18	‘Many of the questions were similarly ranked across patient/caregiver and clinicians, whereas some had differences in ranks’.	Different priorities
Fackrell et al19	‘There were notable differences in the interim prioritization between patients and professionals (professionals: effective treatments, patients: causes)’. “Using weighted ranking, top 10 reflected the mixed priorities from all stakeholders’.
Birnie et al22	‘Our involvement of youth and family members led to different identified priorities compared to prior priority setting efforts with no public or youth involvement’.
Peeks et al29	‘It is important to note that these priorities did not match those deemed by professionals alone. Professionals prioritized metabolic control, and the role of diet. Patients emphasized the importance of natural progression of disease and complications’.
Finer et al28	‘It is notable that the final top 10 research priorities identified in the final workshop differed considerably form those ranked at the interim priority setting’.

Action plans for assessing the research impact were described in three studies^{26 28 29} (table 5). Noteworthy is that assessing the research impact of research priority setting is as challenging as assessing focused impact. Assessing the research impact takes a long time, and this requires the research team to be involved for a longer time span.

Table 5.

Description of research impact

Study	Research impact
Lim et al26	‘Assessing the long-term impact of the PSP is important, however measuring and evaluating the impact is challenging and can take a long time’.
Peeks et al29	‘To both monitor and share information on future research projects that result from these top priorities’.
Finer et al28	‘The impact of the priority setting partnership on future research investment will be monitored and reported on by Diabetes UK’.

Discussion

In this study, we identified that the JLA method is most frequently used to involve CYP in developing a research agenda and that the impact is insufficiently described at best. The results add to the rapidly expanding field of involvement of CYP. Our study showed that the involvement of CYP in developing research agendas has grown since 2016. Previously, only four research agendas were formulated together with CYP.⁶ Five years later, this involvement has increased fivefold, resulting in 22 research agendas. This growth indicates the change in the position of CYP in research.

JLA method most frequently used method

The JLA method was most frequently used to involve CYP in developing a research agenda. van Seventer and colleagues argue that although the outcomes of involving CYP in developing a research agenda have been described, reflecting on the method used to involve CYP is hardly performed.⁸ Yet, Verwoerd and colleagues did evaluate the JLA method, and they were one of the first who integrated additional focus groups to involve the younger children in developing a research agenda.¹⁷ They found it to be of added value because otherwise, the views of adolescents and young adults would have been over-represented.³⁸ Our results indicate that only six studies evaluated the method used. Therefore, more information is needed to justify the statement about that JLA method not being well suited to CYP.⁶

Impact is insufficiently described at best

There is widespread acknowledgement that analysing the focused impact is challenging because it is difficult to know which contribution of the CYP made the difference in developing the research agenda. Yet, this study shows that nine of the included studies attempted to describe the contribution of CYP. It is noteworthy that no studies reported the diffuse impact. The main goal of developing a research agenda together with CYP is to provide the most important research questions. Yet, we should keep in mind that researchers with a positive experience in partnering with CYP in research are most likely to implement a similar collaboration in the future.³⁹ CYP with a positive involvement experience gain knowledge and confidence which can affect their own lives and work and can provide motivation to be involved in later studies.³⁹ Therefore, diffuse impact could also be an important argument for involving CYP.

The JLA recognises that the partnerships between patients, clinicians and professionals may have an impact on the people who participate in them and on the research agenda itself. Interestingly, the JLA guidebook does not elaborate on how to evaluate this impact. The guidebook does provide valuable recommendations on how to maximise the research impact of the agreed priorities.⁴⁰ The guidebook might have been more all-encompassing if it encouraged researchers to evaluate the focused and diffuse impact as well.

Publishing a research agenda should be a tool, not a stand-alone goal

Only eight of 22 studies reported the action plans to implement the research agenda, and only 3 of these reported keeping track of the research impact. This marginal reporting on the postprioritisation phase is seen in JLA PSPs in general.³⁹ As a result, little information is available about whether the research agenda is implemented. Jongsma and colleagues interviewed the participants involved in their PSP. Participants considered the PSP a waste of money and time, should the project end with the publication of the top 10 priorities.⁸ This is a striking outcome because our study showed that only a few studies described continuing the project after publishing the research agenda. Staley and colleagues suggested extending the partnership to cover impact-oriented activity beyond publishing the agenda.³⁹ Taking the results of our study into account, we agree with this proposal so plans can be implemented, and the impact of the research agenda can be measured. Awareness about the fact that publishing the research agenda is not a stand-alone goal is important. Influencing research practice and thereby changing paediatric care should be the goal striving for. Publishing a research agenda is an important tool for achieving that.

Limitations

A limitation of this study is the inability to retrieve how many CYP of a specific age group were included. In the included studies, the age of the CYP was divided into broad categories. Although the agendas developed together with children have increased from 4 to 22 in 5 years, we did not compare the number of the research agendas that have been developed together with children to the total of research agendas. Therefore, we cannot state anything about the relative growth compared with the total.

Future research and conclusion

This study aimed to identify the methods used to involve CYP in developing a research agenda and to assess what is reported about the impact of involving CYP in research priority setting. We found that the JLA method is most frequently used even though it is rarely evaluated as to whether it is appropriate for involving CYP. This study suggests that an evaluation on the methods should be performed to understand if these are appropriate for the involvement of CYP. Furthermore, this study concludes that reporting the impact remains rare. To be able to measure the impact, researchers should perform a qualitative study focusing on what researchers and CYP believe are important characteristics when measuring the impact of developing a research agenda together. This could lead to an operationalised definition of impact. In our follow-up study, we will start with this. Furthermore, we recommend expanding the guidelines on involving children in developing a research agenda with information on how to evaluate the impact.

Ethics statements

Patient consent for publication

Not applicable.

Ethics approval

Not applicable.