Improving Collaborative Engagement in Health State Valuation: A Scoping Review of Current Practices and Emerging Recommendations

Abstract

Background and Objective

Collaborative engagement with individuals invested in or affected by health research, beyond researchers themselves, is advantageous and encouraged by major funding bodies. However, the degree of collaborative engagement in health state valuation is unclear. A scoping review was conducted to (i) identify recommendations on best practice in collaborative engagement in health economics and related literature; (ii) identify examples of collaborative engagement in valuation studies; and (iii) map (ii) onto (i) to identify current practice and future recommendations.

Methods

Eight databases were searched in March-May 2024, with grey literature searches in August-September 2024. For objective (i), reports or manuscripts in health economics or patient-reported outcome measure development/evaluation of any date providing recommendations for collaborative engagement were included. For objective (ii), articles published since 2019 featuring health state valuation and collaborative engagement were included. Best practice recommendations were extracted and thematically synthesised. Examples of collaborative engagement were extracted and mapped against recommendations.

Results

Twenty-two records featuring recommendations and 15 valuation studies were included. A 15-item framework of emerging best practice recommendations for collaborative engagement was synthesised. Most examples of collaborative engagement involved patients and/or experts helping inform health states for valuation. There was no evidence for 9 out of 15 synthesised recommendations having been applied in any of the valuation studies and only minimal evidence was extracted for the remaining six.

Conclusions

Collaborative engagement in health state valuation is underdeveloped and unaligned with literature recommendations. A 15-point framework has been developed as a strategic starting point for developing guidance to improve practice in the field.

Supplementary Information

The online version contains supplementary material available at 10.1007/s40273-025-01550-8.

Key Points for Decision Makers

Engagement with collaborators beyond the researchers themselves is seen as advantageous and good practice in health research. However, the extent of collaborative engagement in health state valuation is unclear.

This scoping review is the first to systematically identify the degree of collaborative engagement reported in health state valuation studies and map it to synthesised recommendations extracted from the health economics and related patient-reported outcome measure literature.

Collaborative engagement in health state valuation is underdeveloped and unaligned with synthesised recommendations from the health economics and patient-reported outcome measure literature. The synthesised recommendations framework from this review presents a strategic starting point for further work to develop consensus-based guidelines for best practice in collaborative engagement in health state valuation.

Introduction

In health research and practice, collaborating with a range of individuals and groups is seen as increasingly important [1, 2]. Such collaborations extend beyond the researchers themselves, to involve and engage other “individuals, organisations or communities that have a direct interest in the process and outcomes of a project, research or policy endeavour [or who are affected by its outcomes]” [3]. Importantly, collaboration is more than research participation and involves engagement on a peer-to-peer level to help inform, interpret, implement, disseminate and/or evaluate research [4]. In health economics, non-researcher collaborators may include, but may not be limited to, members of the public, patients, clinicians, subject experts, payers and purchasers, and policy representatives or decision makers [3]. In this article, we use the term ‘collaborators’ in preference to ‘stakeholders’, owing to the latter’s problematic connotations reflecting historical power differentials between groups [5].

Engagement with non-researcher collaborators is theorised to actively benefit research projects. For example, by increasing the relevance of the research and fostering reciprocal endorsement with a broader group of individuals involved in or affected by the project, improving the impact and uptake of research outcomes [6]. Further, involving and engaging potential collaborators, such as patients and/or the public in research they are invested in, or affected by, complies with modern ethical and moral standards exemplified in the ‘nothing about us, without us’ philosophy [7, 8]. Accordingly, major funding bodies, such as the National Institute for Health and Care Research in the UK and National Health and Medical Research Council in Australia, require evidence of collaborative engagement as standard in their applications for funding [9, 10].

In certain related fields, such as outcomes research (including the development/evaluation of patient-reported outcome measures [PROMs]), collaboration with patients, members of the public and/or other individuals is an established practice [11, 12]. However, in health economics research concerned with health state valuation (i.e. eliciting quantitative preferences for differing health states), collaborative engagement is less apparent [13]. Nevertheless, clear opportunities for collaboration exist in health state valuation research, including with patients whose health states are being valued, target participants of valuation studies, public funders of healthcare systems, end users of value sets and decision makers, amongst others. Many methods choices in health state valuation involve normative or value judgements by the researcher (e.g. whose preferences should be elicited), which could benefit from collaborative engagement and consultations with users of and those involved in or affected by the research [13–15]. While contemporary valuation studies are beginning to emerge with elements of collaborative engagement incorporated [16], there is currently an absence of guidelines or recommendations for best practice, potentially compromising the standard and quality of collaborative engagement in health valuation. While guidance exists in complementary fields (e.g. in PROM development) [17], as well as generic guidance [18, 19], health state valuation studies have a number of unique aspects that could benefit from bespoke best practice guidelines to help standardise and optimise the quality of collaborative engagement in the field. Establishing a tradition for high-quality collaborative engagement in valuation research is recognised as valuable and necessary to ‘future-proof’ the discipline [16].

Developing best practice guidance is iterative, involving multiple sequential stages. An initial necessary first step is to map what is already known. This includes establishing current practice(s); drawing on existing guidance from both within the field and in allied discipline(s); and identifying gaps in process and knowledge. A scoping review is particularly suited to achieve this as a methodology designed to identify and map available evidence; establish gaps in knowledge or practice; and generate recommendations for future research and evidence synthesis [20]. When scoping the literature, rather than general guidelines/recommendations, which are nonetheless useful, we are interested in specific recommendations emerging from within and contextualised to the field of interest (i.e. health economics). To draw on knowledge from a discipline where collaborative engagement is more well established, we chose to broaden the scope of the review to include recommendations that exist within the allied discipline of PROM development/evaluation. The two disciplines have a clear cross-over, as health state classification systems are often derived from PROMs to produce preference-weighted scoring algorithms for generating utilities [21].

Accordingly, in this paper, we present a scoping review that systematically identifies: (i) existing best practice recommendations for collaborative engagement in health economics and the complementary discipline of PROM development/evaluation; (ii) contemporary health state valuation studies reporting collaborative engagement; and (iii) gaps and future recommendations, by mapping the results of (ii) onto (i).

The following three research questions were addressed:

1. What are the characteristics and key components of existing best practice guidelines on collaborative engagement in health state valuation and complementary fields (including health economics and PROM development/evaluation)?

2. What examples and characteristics of collaborative engagement activities exist in recent health state valuation studies?

3. How do examples of recent collaborative engagement activities in health state valuation (question 2) map against existing guidelines on collaborative engagement in health state valuation and complementary fields (question 1)?

Methods

Protocol and Registration

The protocol for this review was made publicly available online prior to study commencement (10.15131/shef.data.25442377.v1). Best practice guidance from the Joanna Briggs Institute was followed [22]. The article has been prepared using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews checklist, available in Appendix 1 in the Electronic Supplementary Material (ESM) [23].

Search Strategy

Two database searches were conducted on Ovid MEDLINE, Embase via Ovid, CINAHL via EBSCO, PsycINFO via Ovid, Science Citation Index, Social Sciences Citation Index and Conference Proceedings Indexes for Science and Social Sciences (Web of Science Core Collection), Econlit via Ovid. In addition, Tufts Cost-Effectiveness Analysis Registry and MathSciNet (American Mathematical Society) were searched for search 1 only. The first search (search 1) was designed to identify existing recommendations on best practice in collaborative engagement in the field of health economics and/or PROM development/evaluation. The second search (search 2) was designed to identify any recent examples of collaborative engagement in actual health state valuation studies. Search 1 was conducted between 25 and 26 March, 2024 from inception (i.e. with no date limit). Search 2 was conducted on 1 May, 2024 and limited to the last 5 years (i.e. from 2019) to identify recent examples in the field. Given contemporary recognition of the importance of collaborating with key partners (e.g. patients, decision makers), we considered it likely that most relevant examples would be uncovered within the previous 5 years. No language restrictions were implemented in the searches.

Search 1 combined keyword terms and synonyms for guidance (e.g. guidelines, standards, position, recommendation); collaborators (e.g. patient, public, expert, decision maker); methodologies/terms indicative of collaboration (e.g. focus group, workshop, peer led, involve*, engag*, consult*, collaborat*); and field (e.g. health economics, PROMS). Search 2 combined the keyword terms and synonyms for collaborators; methodologies/terms indicative of collaboration; and health state valuations (e.g. health state, utility*, preference based, valu*). The full database search strategies are provided in Appendix 2 in the ESM.

For search 1, additional complementary grey literature searches were conducted on relevant professional organisations including The International Society for Quality of Life Research, The Professional Society for Health Economics and Outcomes Research (ISPOR), relevant health technology assessment bodies (e.g. National Institute for Health and Care Excellence, US Food and Drug Administration), World Health Organization and Google Scholar (see Appendix 2 in the ESM for a full list of organisations searched). These were last searched on 19 August, 2024. For search 2, additional searches of value set publications on the EuroQol website (https://euroqol.org/information-and-support/resources/value-sets/) and Google Scholar were conducted and records screened. These were last searched on 15 September, 2024. It was not possible to access unpublished valuation reports directly from EuroQol because of author permissions. In response to peer review, additional grey literature searches of the websites of other commonly used generic preference-weighted measures (including the 15D, Assessment of Quality of Life [AQoL], Health Utilities Index [HUI], PROMIS-Preference Scoring System [PROPr], Quality of Well-Being Scale [QWB], Short Form 6 Dimensions [SF-6D]) were searched in October 2025 (see Appendix 2 of the ESM for websites searched). The reference lists of all included articles were screened for potential additional inclusions.

Evidence Selection

Records retrieved from the searches were stored in EndNote (v21, 2023; Clarivate Analytics, Philadelphia, PA, USA) and de-duplicated, before being exported to Microsoft Excel for screening. Titles and abstracts of retrieved records were screened against the eligibility criteria (Table 1). This process was conducted independently by two reviewers (VG, GS), with discrepancies reconciled by a third reviewer (PP). Any records that were deemed potentially eligible were reviewed at full text independently by two reviewers (VG, GS). Any disagreement at the full-text stage was reconciled by a third reviewer (PP). Reasons for exclusion were recorded at the full text stage.

Table 1.

Eligibility criteria for the review

(i) Existing guidance

Reports or published manuscripts (i.e. not abstracts or other non-full-text sources) written in English

In the fields of health economics or PROM development/evaluation

Provides recommendations for collaborative engagementa in research

(ii) Primary research

Published manuscripts or EuroQol valuation reports (i.e. not abstracts or other non-full-text sources) written in English published from 2019 onwards

Primary research (of any research design) [i.e. not secondary research, systematic reviews, meta-analyses or letters]

Includes a health state valuation methodology (i.e. a recognised method that derives people’s quantitative preferences for living in different health states)

Features/describes collaborative engagement

PROM patient-reported outcome measure

^aFor operational definitions of scoping review terms, please see the study protocol (https://doi.org/10.15131/shef.data.25442377.v1)

Data Extraction

Data were extracted from each included record by two independent reviewers (VG, GS). A data extraction spreadsheet was prepared on Excel (one for each search) adapted from the Joanna Briggs Institute standardised data extraction form [24]. The spreadsheet was piloted on the first four records from each search prior to full extraction. No modifications were required. Any discrepancies in data extraction were reconciled by a third reviewer (PP). Data extracted were adapted for each article type (i.e. guidelines vs valuation studies) and included: article details (e.g. authorship, context, country, field); methodological details (e.g. study design, participant characteristics, collaborators involved); details of collaborative engagement (e.g. definition, activities, purpose); engagement outcomes (if reported); and/or best practice recommendations (if reported). Finalised data extraction forms are available in Appendix 3 of the ESM.

Data Synthesis

Extracted data were presented and synthesised narratively. Best practice recommendations extracted were thematically synthesised by two researchers (PP, JC), by applying descriptive codes to extracted quotes in Excel. This process was iterative, and descriptive codes were refined over time until a final set was agreed (see Appendix 3 of the ESM for codes applied to the extracted data). Synthesised recommendations were then tabulated into a thematic framework and a summary description was written for each based on the extracted quotes. The proposed thematic framework of recommendations was agreed with the rest of the research team. Agreed data extracted from health state valuation studies was tabulated and mapped against the synthesised framework of recommendations by two researchers (PP, JC) to identify coverage and gaps in practice. During mapping, for each synthesised recommendation, extracted evidence was graded as sufficient (✔), insufficient (x) or uncertain (?) evidence that the recommendation had been applied in the valuation study (with referral back to the source article if in any doubt). Specifically, the raters looked for descriptive evidence demonstrating awareness of the recommendation and an attempt to implement it, with a grading approach that emphasised leniency. The grading criteria used are in Appendix 4 in the ESM. Sub-grouping of the results by age (i.e. paediatric vs adults) and type of collaborator (e.g. patient, public, decision maker) was considered a priori but not pursued because of insufficient observations to draw conclusions by sub-grouping the data.

Results

Evidence Selection

The evidence selection process is summarised in Fig. 1. In search 1 (recommendations), 3449 records were identified in scholarly databases (2558 after duplicates removed). One hundred and two records were reviewed at full text, with 14 included in the review. One hundred and seventy-nine additional records were identified from grey literature searching and reference screening, with seven included in the review. One additional record was added following peer review, for a total of 22 records.

Fig. 1.

Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow diagram. CE collaborative engagement, HE health economics, PROMs patient-reported outcome measures.

Source: Page MJ, et al. BMJ 2021;372:n71. https://doi.org/10.1136/bmj.n71.38 [38].

In search 2 (health state valuation studies), 3082 records were identified in scholarly databases (1743 after duplicates removed). Eighty-two records were reviewed at full text, with 15 included in the review. Eighty additional records were identified from the website searching, Google Scholar and reference screening, but none was deemed eligible for inclusion in the review because there was no evidence of collaborative engagement, giving a total of 15 records.

Characteristics of Included Evidence

Table 2 summarises the characteristics of included evidence from search 1 that provides at least one extracted recommendation for collaborative engagement. Half of the records were classified in the field of health economics (n = 11) and half in PROM development/evaluation (n = 11), with corresponding authors based across four countries (with concentrations in the UK, n = 9, USA, n = 5 and Canada, n = 5). The majority of records (n = 19) addressed collaborative engagement within the context of patient and/or public involvement, including caregivers, with only a few discussing wider engagement with other invested individuals, such as clinicians, payers and policy makers [25–27]. On average, six best practice recommendations were extracted from each record (range = 1–11), with the content of these recommendations informed in a variety of ways. One article was focused on patient and public involvement in child health research [28], and one featured a recommendation for involving children [29]. All other articles focused on adult collaborators.

Table 2.

Characteristics of records included in search 1 (containing recommendations for collaborative engagement)

References	Country	Context/purpose	Field	Definition of collaborative engagement provided	# recommendations extracted	How recommendations were formed
1. Addario et al. (2020) [39]	CAN	Guidance to support the involvement of patients and patient advocates in PRO development, implementation and dissemination in cancer clinical trials	PROM development/evaluation	Yes	10	Based on examples from the literature, existing guideline recommendations and authors’ own experience
2. Aiyegbusi et al. (2024) [26]	UK	Develop consensus-based recommendations to reduce respondent burden	PROM development/evaluation	No	2	International Delphi and consensus process
3. Al-Janabi et al. (2021) [40]	UK	Reflections and recommendations on PPI in health economics methods research	Health economics	Yes	3	Reflections on PPI input as part of a health economics research project
4. Carlton et al. (2020) [29]	UK	Emerging framework for incorporating public involvement into PROM development	PROM development/evaluation	Yes	11	Based on authors’ experience of public engagement when developing PROMs
5. Clearfield et al. (2020) [41]	USA	Guidance for developing patient engagement for core outcome set development	PROM development/evaluation	Yes	7	Lessons learned from authors’ organisations’ patient engagement work and patient-centred COS development
6. D’Agincourt‐Canning et al. (2024) [42]	CAN	Development of a framework for creating PROMs and PREMs with indigenous peoples	PROM development/evaluation	No	7	Literature review and qualitative interviews with researchers and the indigenous community
7. Grundy et al. (2019) [43]	UK	Provide examples and lessons learned from PI in the development of a mental health PROM	PROM development/evaluation	Yes	7	Reflections from a PROM development project involving expert service users
8. Harrington et al. (2020) [27]	USA	Comprehensive review of existing definitions with the goal of generating a definition of patient engagement in the context of health research	Health economics	Yes	4	Based on examples from the literature, incorporating a multi-collaborator approach (including researchers and patients)
9. Harvard et al. (2023) [44]	CAN	Identify key questions concerning PPI in health economic modelling, provide preliminary answers, and encourage future debate and study	Health economics	No	4	Reflections from the authors’ perspective, informed by engagement experience and work in health economics
10. Hawton et al. (2021) [45]	UK	Designed to contribute towards future guidance for patient involvement in health economics	Health economics	Yes	8	Author reflections on their experiences and those of collaborators involving patients in health economics research
11. Hudon et al. (2022) [46]	CAN	Guidance for including patient partners in research aiming to support the implementation of PROMs	PROM development/evaluation	Yes	1	Through a participatory action research project in which academic and non-academic collaborators developed guidelines
12. Kandiyali et al. (2019) [28]	UK	Provide examples of how PPI has been used in health economics research in child health and promote future use of PPI	Health economics	Yes	7	Case studies and reflections from researchers from their experience of PPI in health economics
13. Morris et al. (2014) [47]	UK	Guidance intended for the UK NHS to improve neurodisability-related care for children (including outcomes)	PROM development/evaluation	Yes	5	Reflections on PPI input in a research project
14. Oortwijn et al. (2022) [25]	NL	Guidance intended for designing and using deliberative processes for HTA, intended to facilitate participatory decision making	Health economics	No	10	Recommendations were formed by a joint task force by HTA International and ISPOR
15. Perfetto et al. (2017) [48]	USA	Provide an overview of the NHC Patient-Centred Value Model Rubric designed to assist patient centeredness and engagement in research	Health economics	No	2	Review of existing rubrics, roundtable discussion and multi-collaborator peer review
16. Perfetto et al. (2018) [49]	USA	Identifying and highlighting emerging good practices in patient engagement in value assessment	Health economics	Yes	10	NHC Value Workshop survey and focus group of its members
17. PFMD (2021) [50]	NR	Clarification of regulatory and HTA expectations of patient engagement and patient experience data across authorities	Health economics	No	5	A collation of current efforts (e.g. from published reports and documents)
18. Rivera et al. (2021) [51]	UK	Adapt SPIRIT-PRO guidance to a user-friendly format for patient partners and to co-design a web-based tool to support uptake	PROM development/evaluation	Yes	1	Authors’ reflections based on experience with PPI/co-design in the generation of guidance
19. Staniszewska et al. (2021) [52]	UK	To develop a framework that identifies the nature and type of PI in modelling and supports its implementation	Health economics	Yes	7	Drew on learning from a prior study. Iteratively co-produced guidelines with public contributors
20. Wiering et al. (2017) [53]	NL	Investigate PROM developers’ perspectives on involving patients, including benefits and negative aspects	PROM development/evaluation	No	4	Based on the results of a qualitative study exploring PROM developers’ perspectives on involving patients in the development of PROMs
21. Wilson et al. (2018) [54]	USA	Propose a framework for engagement of patients/patient patterns in selecting and/or designing COAs as endpoints	PROM development/evaluation	Yes	10	Based on authors’ own experiences engaging patients and patient partners in COA development
22. Wilson et al. (2022) [55]	CAN	Reports on ways in which patients and caregivers were engaged across early economic evaluation processes in a project evaluating a new cancer treatment	Health economics	Yes	1	Based on research experience, engaging patients and a caregiver

CAN Canada, COA clinical outcome assessment, COS core outcome set, HTA health technology assessment, ISPOR The Professional Society for Health Economics and Outcomes Research, NHC National Health Council, NHS National Health Service, NL The Netherlands, NR not reported, PI public involvement, PPI patient and public involvement, PREM patient-reported experience measure, PROM patient-reported outcome measure, SPIRIT-PRO Standard Protocol Items: Recommendations for Interventional Trials-Patient Reported Outcome Extension

Table 3 summarises the characteristics of included evidence from search 2 (i.e. valuation studies). The majority were vignette studies in a range of health conditions (n = 12) and were conducted in the UK (n = 8) or Australia (n = 3). With the exception of two studies that also involved collaboration in recruitment and training and the feedback of study findings [30, 31], all studies used collaborative engagement to inform/review health states (and related content) for valuation. The type and number of collaborators varied, but typically included clinical experts and/or people with lived experience.

Table 3.

Characteristics of records included in search 2 (health state valuation studies including collaborative engagement)

Reference	Country	Context/design	Collaborators involved	Purpose of engagement	Engagement activities
1. Chou et al. 2020 [56]	TW	Vignette study to estimate health state utilities in patients with breast cancer using VAS and TTO	Expert panel (medical oncologist, two clinical pharmacists); several “non-medical people”	Inform/review health states for valuation	Expert panel and “non-medical people” reviewed and provided feedback to inform health states
2. Cooper et al. 2023 [57]	UK	Vignette study to estimate health state utilities for high-grade non-muscle invasive bladder cancer with a general public sample using TTO	Cancer patient advisor	Inform/review health states for valuation	Advisor reviewed and provided feedback to inform health states.
3. de Freitas et al. 2023 [58]	UK	Vignette study to estimate health state utilities of cardiovascular health states in general public participants using TTO	Two expert nephrologists	Inform/review health states for valuation	Clinicians reviewed and provided feedback to inform health states
4. Gallop et al. 2022 [59]	UK	Vignette study to estimate health state utilities for patients/caregivers related to oral immunotherapy for peanut allergy using VAS and EQ-5D-5L/EQ-5D-Y	Two expert clinicians; a patient advocacy group; and an individual with peanut allergy	Inform/review survey and health states for valuation	All collaborators reviewed and provided feedback to inform survey and health states
5. Hughes et al. 2023 [60]	UK	Vignette study to estimate health state utilities for Fabry disease in general public participants using TTO	One clinical expert	Inform/review health states for valuation	Clinical expert reviewed and provided feedback to inform health states
6. Ju et al. 2021a [30]	AU	Vignette study to estimate health state utilities for cervical cancer among indigenous adult women using SG	IRG, which “comprised several respected Indigenous adults with diverse backgrounds from across South Australia”	Inform/review health states for valuation; aid recruitment and training; oversee feedback of study findings	IRG assisted in the development of health states; recruitment and training of research officers; and oversaw delivery and feedback of study findings
7. Ju et al. 2021b [31]	AU	Vignette study to estimate health state utilities for HPV infection and early stage OPSCC among indigenous Australians using SG	Same as above	Inform/review health states for valuation; aid recruitment and training; oversee feedback of study findings	IRG assisted in the development of health states; recruitment and training of research officers; and oversaw delivery and feedback of study findings
8. Kim et al. 2022 [61]	KR	Vignette study to estimate utilities for prostate-related health states with general public participants using VAS and SG	One clinical expert; two laypersons	Inform/review health states for valuation	All collaborators reviewed and provided feedback to inform health states
9. Lo et al. 2022 [62]	UK	Vignette study to estimate utilities for patient and caregiver health states in TSC by the UK general public using TTO	One clinical expert	Inform/review health states for valuation	Clinical expert reviewed and provided feedback to inform health states
10. Matza et al. 2019 [63]	UK	Valuation of health states derived from the PASI to derive preference-based scoring algorithm in general public participants using TTO	Four clinical experts	Inform/review health states for valuation	Clinical experts reviewed and provided feedback to inform health states
11. Matza et al. 2020 [64]	UK	Valuation of health states derived from the Neuro-QoL to derive a preference-based scoring algorithm in general public participants and people living with MS using SG	“Project advisers” (number not reported)	Inform/review health states for valuation	Advisors helped in the selection of health states
12. McLeod et al. 2022 [65]	AU	DCE study to evaluate treatment preferences and trade-offs in outcomes related to pulmonary exacerbations of CF in children with CF and their carers	People affected by CF (number not reported)	Inform/review content of DCE for valuation	Advisors informed wording, levels of attributes and plausibility of content used in DCE
13. Takumoto et al. 2022 [66]	JP	Vignette study to estimate health state utilities for metastatic pancreatic cancer in the general public using composite TTO and VAS	Oncology physicians (unclear how many in total); four oncology pharmacists; one quality-of-life expert	Inform/review survey and health states for valuation	All collaborators reviewed and provided feedback to inform survey and health states
14. Tolley et al. 2022 [67]	UK	Vignette study to estimate health state utilities for excessive daytime sleepiness caused by obstructive sleep apnoea in general public participants using TTO and VAS	One clinical expert; patients and patients’ partners (number not reported)	Inform/review health states for valuation	All collaborators reviewed and provided feedback to inform health states
15. Vermeulen et al. 2024 [68]	NL	Vignette study to quantify the impact of receiving a dementia risk presentation on health state utilities in the Dutch general public using DCE and TTO	Two clinical experts; three experts in stated preference studies; people comparable to the target population (number not reported); one expert in the Dutch language	Inform/review survey, health states and DCE for valuation	All collaborators reviewed and provided feedback to inform survey and health states, including content for DCE

AU Australia, DCE discrete choice experiment, HPV human papillomavirus, IRG Indigenous Reference Group, JP Japan, KR Korea (the Republic of), MS multiple sclerosis, Neuro-QoL Quality of Life in Neurological Disorders, NL The Netherlands, OPSCC oropharyngeal squamous cell carcinoma, PASI Psoriasis Area Severity Index, SG Standard Gamble, TSC tuberous sclerosis complex, TTO time trade-off, TW Taiwan, VAS visual analogue scale

Synthesised Recommendations

Fifteen recommendations for best practice in collaborative engagement were synthesised from the extracted data (Table 4). All were cited in at least five of the 22 records included in the review (in search 1). Recommendations concerned the planning/design, conduct, evaluation and reporting of collaborative engagement. The most common recommendations included the need to involve collaborators at key stages throughout the research (n = 15) and to foster sustained and meaningful (vs tokenistic) engagement (n = 12). This was followed by identifying suitable representatives from target collaborative groups (n = 11), adapting resources for collaborators (n = 10) and developing a detailed collaboration plan or protocol (n = 10).

Table 4.

Synthesised recommendations from the included literature

	Label	Description	Example extracts	N Papers
1	Develop a collaboration plan/protocol	Developing a detailed collaboration plan (or protocol) as part of the wider research protocol is essential. This should include the methods/means of collaboration throughout key stages in the research process (#3) as well as the goals, desired outcomes and scope. Collaborative involvement from representatives of target groups in developing the collaboration plan is encouraged so that it can be mutually agreed upon, and the plan (and any associated resources) should be understandable to collaborators (#11). At the planning stage, an appropriate degree of collaboration should be identified that adds perceived value to the project (and its outcomes) relative to its costs (#5). Target groups for collaboration (e.g. patients, decision-makers) should be outlined as well as how many collaborators from each group will be identified (ensuring appropriate diversity in key characteristics, #2). Responsibilities and roles should be clearly defined (#4), as well as the practical elements of collaboration (#6), including logistics and resources/compensation (#9). Any necessary training for researchers and collaborators to enable good collaboration should be detailed in the protocol (#8). When planning the collaboration, a proposed strategy to facilitate sustained and meaningful collaboration throughout the project should be outlined (#10), as well as detailing how good ethical principles will be adhered to (#12). The way in which the collaboration will be evaluated (e.g. assessing its impact) should be pre-specified (#13), and the dissemination/reporting strategy as part of the study protocol should include a plan for reporting on collaboration (#15), including the ways study findings will be directly shared with target collaborator groups (#14). Finally, the collaboration plan should be revisited periodically and updated as necessary, in accordance with the need to adopt a flexible and responsive approach to collaboration throughout the project (#7)	“The importance of developing a clear PI [public involvement] plan is essential. The research team needs to consider the remit of their project, and which stages of PROM development they will be undertaking… In addition, there should be clarity on both sides around the expectations in terms of financial compensation and expected input. The plan should also include details of how the reporting of PI activities will occur and the extent to which the impact of PI will be evaluated.” [29] “Establish a protocol for partnering with stakeholders. The protocol should include but is not limited to: how representativeness will be achieved; the targeted patient population(s); objectives for engagement; expectations/ roles; and logistics such as number of hours, meetings, compensation, and anticipated final deliverables.” [49]	10
2	Involve suitable representatives from target groups	Care needs to be taken in identifying suitable representatives from target collaborator groups, recognising that different collaborators may be more suitable at particular stages of the research. Where possible, diverse representatives from target groups should be identified. It is important to ensure breadth in collaborators’ characteristics appropriate to the research question or clinical area. Example characteristics where diversity may be sought include, but are not limited to, sex/gender; age; race/ethnicity; socioeconomic background; disease characteristics; prior knowledge/experience; and study-specific expertise. Desired areas of breadth and diversity are likely to differ by collaborator group (e.g. patients vs regulators) and it is important to acknowledge that collaborators cannot be fully representative of the target groups from which they are identified. For some target groups (e.g. children), additional considerations involve whose engagement should be sought (e.g. children or primary caregivers). Researchers should outline and justify their process and rationale for including particular collaborators in the collaboration plan (#1)	“It should also be noted that PI may involve input from more than one individual. Involving more than one person increases the breadth of experience to the project, allows those involved to support and encourage each other, and allows for multiple perspectives. PI members cannot be representative of everyone who has a specific condition [23, 24]. Having a PI team is beneficial to allow for wider diversity and experiences... There may be established PI groups for the given health condition, or there may be a need to establish a new PI group, which should be as diverse as possible.” [29] “These values and perspectives are typically provided by a variety of stakeholders (patient[s], public/citizens, providers of care, payers, producers and innovators of health technology, principal investigators in research, and policy makers) and experts (e.g. experts in medicine, law, ethics, economics, epidemiology, bioengineering, and patient-based evidence).” [25]	12
3	Involve at key stages throughout the research process	Involvement of collaborators should be considered throughout the entire research process, from inception and at key stages identified in the collaboration plan (#1). Through engaging collaborators early and continuously throughout the research process, the goal is to avoid token inclusion or sporadic consultation. Engagement should begin as early as possible (i.e. at the question or problem-development stage) and may incorporate, but is not limited to, formulation of objectives and hypotheses; decisions on study design and methods (including data collection and recruitment strategies); design of participant-facing materials and resources; interpretation of findings; and dissemination of results. Ideally collaboration should occur throughout a project, recognising that it may not be feasible or appropriate to involve collaboration in every stage of the research (#5)	“Rather than sporadic touchpoints or token inclusion, it is important to include patients, carers, family members, and advocates in all aspects of the COS development process... This method of incorporating patient values “upstream”, i.e. before pivotal trials begin, increases the patient centeredness of evidence available for further HEOR and across decision-making contexts (regulatory, coverage and reimbursement, patient/clinician).” [41] “By incorporating “all stages of the research process,” we intend to cover the full spectrum of research activities, including planning, conduct, and dissemination. The phrase “all stages” is used to indicate that patient engagement may be possible, and should be considered, at any point in a study. Nevertheless, it is not meant to function as a mandate that engagement must happen at all stages of a single study, which may be neither feasible nor appropriate.” [27]	15
4	Ensure responsibilities and roles are clearly defined	The responsibilities and roles of people involved in the collaboration should be clearly defined in the collaboration plan (#1). Collaborators should have a meaningful role in the decision-making process and can help agree on the proposed project governance structure. Written task descriptions and/or terms of reference should be produced to define the role of any collaborator(s). This will help to ensure collaborators understand their roles and responsibilities before making a decision about their involvement in the project (#12)	“Stakeholders and experts can have financial, professional, or reputational interests in the outcomes of deliberative processes, which points to the need for their roles to be clearly defined... Participants may represent the views of others or only express their individual views” [25] “Define the patient role on the project. Prior to selecting a patient/patient partner, it is important to establish the capacity within which the patient/patient partner will be engaged and communicate these roles clearly to all parties… Clearly communicate responsibilities for specific project tasks and the impact that their participation is bringing to the project… Patients/patient partners should be involved in the decision-making process to determine how the working relationship will be structured; this will ensure that their unique needs can be met throughout the partnership.” [54]	8
5	Ensure collaboration adds value to the research project	Collaboration should not be undertaken for the sake of it or to ‘tick boxes’ and should be designed in a way that adds demonstrable value to the research process. While potential collaboration should be considered at each stage of the research process (#3), key stages for collaborative engagement should be identified as part of the collaboration plan (#1) that contribute to the goals of the project and where target collaborators can bring additional insights and benefit beyond that of the research team. The benefits of collaboration should be balanced against the costs of carrying it out, which may differ across projects and project stages. Each point of collaboration should have a rationale and expected outcome/impact, which should be amendable to evaluation (#13)	“Reflecting on the process of selecting tasks, we suggest three elements that are needed for PPI [Patient and public involvement] to add real value to a research task. First, the task must be necessary in meeting the research goals of the project. This may sound obvious, but it means that tasks should not be invented for the purposes of PPI. This creates unnecessary work for everyone and undermines the legitimacy of PPI. Second, the task should be one where the academic research team alone have a knowledge or skills gap. It would not really make sense, for example, for PPI participants to execute analysis that the researchers are trained and experienced in, simply for the sake of it. Third, the lay participants should have some additional knowledge or skills, in relation to the task, above that which the professional researchers possess. This means a task will not necessarily be suitable for PPI just because the academic team are struggling with it—there should be a reasonable expectation that PPI participants can bring additional insights to the task. [40] “As with any study where public involvement is planned, researchers undertaking health economic research need to consider who to involve and why, when and how to involve them (...) The ‘why’ needs to consider the rationale for involvement in the context of the particular study question.” [29]	6
6	Consider practical arrangements	As with all research activities, the practical arrangements of facilitating collaboration need to be considered and carefully scoped out as part of the collaboration plan (#1). Collaborative activities should be designed to meet the needs of collaborators, which will likely differ depending on collaborator type (e.g. patient partner vs. decision maker). Practical factors need to be considered alongside the facilitation of relationship building and meaningful engagement (#10). Examples of practical considerations include format of meetings (e.g. accessibility issues, appropriate use of technology, comfort breaks, and scheduling); who to involve together in collaborative discussions and in what form (e.g. number of collaborators, the mix of collaborators who may or may not work well together, whether to allow for independent meetings as well as discussion groups); and managing power differences and/or imbalances to facilitate collaborator input	“Disease areas may have more than one patient organization or foundation, with differing missions or views of the condition; it is important to understand their roles in the community and how they do, or do not, work together.” [41] “... the following are practical considerations for accommodating the varying needs of patient partners... Scheduling of meetings can be particularly challenging for teams with patient stakeholders, and may require extra flexibility and creative solutions. Meetings of long duration may be a considerable burden so researchers should consider breaking up meetings or tasks into discrete tasks with shorter duration... Travel may be difficult for patients with active chronic disease and their caregivers so the study team should consider adopting teleconferences or web conference meetings in lieu of in-person meetings. Note that the ability of patient partners to participate in remote meetings may limit the participation of those patient partners who lack access to technology; this limitation should be considered during study design, and accommodations should be made to provide them with necessary equipment where possible...In addition, face-to-face meetings in the early stages of the project can help with relationship building... Alternate patients and patient partners. Any study design should thus include plans to have alternate individuals to fill in or include more than one patient partner on a patient task force.” [54]	8
7	Adopt a flexible approach	Researchers need to adopt a flexible approach to collaboration, accommodating the preferences and needs of collaborators. In doing so, researchers must acknowledge the potential burden that may be experienced by certain collaborators, such as people living with health conditions, and reflect this in their planning (#1). Collaborators should have a say in which areas of the research process they want to be engaged and in what manner, recognising that preferences may change over time and that plans should be responsive to any changes in circumstances	“Stakeholders’ input into both the ways and levels at which they want to be engaged should be central in establishing methodologies of engagement, and frameworks should be reflexive, as stakeholders’ preferences may change over time and with additional experience.” [55] “In some cases, the patients and caregivers who have the most relevant expertise to contribute to the development of a PRO are those with a significant burden of illness. The project timeline should be structured to reflect this, allowing for flexibility and plenty of time for responses from patient partners. Project leaders may need to adjust the numbers involved to ensure the required level of patient contribution throughout the process.” [39]	6
8	Provide necessary training	To facilitate effective collaboration any necessary training should be provided. This includes training for researchers in collaborative engagement (e.g. on working with and communicating with lay partners, facilitating and managing group discussions). This should also include project-specific training and/or briefing sessions for collaborators, where needed. Researchers should demonstrate that they have considered whether the research team and collaborators have the necessary skills and expertise to be involved in the proposed collaboration activities, and provide training if necessary	“Making PPI work also requires efforts from academic researchers. Two of the academic team trained in qualitative research methods. However, in common with most health economists, the academic team had not been exposed to PPI through their training and career development. As with lay participants, a combination of written materials and interactive training would undoubtedly help. Specific areas of training could include the following: the practicalities of recruiting and communicating with lay participants, running tasks, facilitating group discussion, safeguarding welfare, handling different perspectives, and ending a research project.” [40] “The need for teams (particularly researchers) to undergo training in PI was seen as vital to developing appropriate skills that support the process. Facilitation of PI is key and a PI lead role is considered essential... Similarly, the need for the public contributors to undergo training in the basic methods of health economic evaluation was also seen as crucial.” [52]	9
9	Ensure collaboration is appropriately resourced	When planning research projects, collaboration needs to be appropriately resourced, both in terms of monetary budgets and in terms of researcher and collaborator time. Where appropriate, for example when involving lay patient and public members, collaborators should be fairly reimbursed for their time and any expenses incurred relative to the research project	“Extensive service user involvement needs to be adequately planned and budgeted for... In line with best practice, service users in the expert service user group and the scientific group had their travel expenses reimbursed and they were paid for their time for attending the meetings and preparing for them.” [43] “To ensure effective patient engagement and participation, appropriately resourced capacity building to support the patient community and other stakeholders will also need to be considered and integrated.” [50]	9
10	Foster sustained and meaningful engagement	Engagement with collaborators should be meaningful and collaborative rather than superficial and tokenistic. The goal of collaboration should be to develop reciprocal partnerships, where at appropriate times collaborators’ inputs are given equal weight to members of the research team. This relies on the development and demonstration of meaningful peer-to-peer relationships with collaborators (and collaboration groups) facilitated by the setting of expectations and adopting principles of inclusion, trust, respect and equality. Regular and direct communication and good interpersonal skills can help to maintain a positive group dynamic; researchers should be open and responsive to feedback from collaborators. Collaborative activities should be ‘safe spaces’ where all opinions are valued and collaborators are not criticized (#12). Clear guidance on dispute/conflict resolution and dealing with disagreements should be developed at the beginning of the collaborative process (#1). As research projects may last several years and may extend further into dissemination and/or follow-on work, the collaboration plan, should consider how engagement will be maintained throughout and beyond the research process. This may include provisions for handling ‘drop-out’ and the replacement of collaborators throughout the project	“This more comprehensive and complex approach brings with it the need for communication and facilitation skills in order to achieve meaningful and productive engagement. An open dialogue between all stakeholders that allows discussion of both the patient experience and formal care systems is crucial for mutual understanding. Also important is the support required for all team members, including regular and direct communication, use of a partnering system between clinical researchers and patients and patient advocates where appropriate and opportunities for reflection on the collaborative process.” [39] “As other researchers have found, taking the time to establish strong working relationships, typified by shared understanding and trust, can generate greater positive impacts from involvement… Maintaining, extending, and strengthening these relationships over time has required us all to be flexible in our approach as we are challenged by each other’s perspectives… Crucially, we have aimed to make our involvement sessions enjoyable, to be clear about what impact the HEMS group’s input will have on the research, and to ensure that they know how much we value this input… providing informal opportunities for researchers and PPI contributors to socialise together can help foster good working relationships, and our experience supports this.” [45]	12
11	Adapt communication and resources for collaboration	Communication and any associated resources used throughout collaboration need to be suitable for target collaborators, including adapting any technical terminology to be lay-friendly. Definitions of technical terms may need to be provided as well as plain language summaries of key material to be discussed and any outcomes. As well as being understandable to collaborators, communication should be culturally and ethically appropriate. Methods of communication can also be adapted for the preferences of target groups (e.g. digital vs face-to-face interactions) and additional innovative adaptations may be needed when working with specific groups, such as children to ensure good engagement (e.g. interactive play, learning technologies)	“Researchers need to explain health economic research in lay language to enable PPI members to effectively participate. This requires commitment by both researchers and patients (including members of the public, parents and caregivers) to a dialogue, a belief that involving people in the research will improve it, and a belief that it is possible to facilitate the process to enable people to improve the research. Such considerations about language apply to PPI with advisory groups of all ages but may be especially true when working with children. Related to the issue of jargon is the selection of age-appropriate approaches and tasks where children are to be involved.” [28] “We have found the key is to go back to the basics of what we are trying to achieve and to clearly define any core concepts, e.g. “plausibility”. A particularly useful piece of advice from our PPI practitioner is that patient involvement materials do not need to be technically correct in the way that would be expected by an academic audience. Rather, they should convey sufficient information to enable patients to understand the purpose of the research and to engage with it meaningfully.” [45]	10
12	Maintain good ethical standards	While collaboration to inform research is conducted on the level of a peer-to-peer partnership and typically does not necessitate formal ethical approval, all parties are still expected to adhere to good ethical principles throughout the collaboration. As part of establishing roles and expectations (#4) collaborators should give informed consent to be involved in the project’s collaboration and collaborative activities, with an awareness that they are doing so voluntarily and can withdraw if they wish. The well-being of all parties must be considered and maintained, especially if discussing sensitive issues. Principles of confidentiality and anonymity must also be considered and respected, with collaborators made aware of what information will be disclosed when, including via dissemination of the research. The cultural practices of any collaborating communities should be respected and upheld	“There are no official approval requirements to initiate PI activities, such as ethics or governance approvals; however, there are some ethical considerations to consider such as mutual respect, reciprocity, shared commitment, and personal integrity... The emotional wellbeing of PI participants must be maintained, and the research team should be mindful of this. There may be cases where different PI groups could inform different aspects of the PROM development to avoid overburdening a few individuals.. .PI for very rare conditions raises issues relating to anonymity. Within a small patient community, PI members may recognize the identity of anonymised respondents via their characteristics e.g. age/gender/health or condition-stage.” [29] “An ‘ethical space’ is formed when Indigenous and Western societies, with their different cultures and worldviews, are poised to engage each other in a distinct but shared space. Understanding and observing the cultural protocols of an Indigenous community is essential when entering an ethical space with that community.” [42]	5
13	Evaluate collaborative engagement	To facilitate best practice in collaborative engagement, monitoring and evaluation is essential. This involves assessing whether (and the extent to which) collaboration has achieved its intended aims (i.e. assessing impact). Researchers should also evaluate the experience and satisfaction with collaboration for all parties involved (e.g. did collaborators feel engaged, valued, listened to). Research teams should be able to identify what has changed (if anything) as a result of collaborative engagement and the extent to which it has been successful. Pre-existing methods exist to evidence this, such as producing an ‘impact log’, common in patient and public involvement, and evaluation may occur throughout the project (i.e. evaluating the outcomes of collaborative meetings) rather than just reflectively at the end. Evaluation methods should be systematic and should be pre-specified as part of the collaboration plan (#1). Evaluation enables shared learnings to inform future best practice	“Monitoring and evaluation are necessary to determine if a deliberative process is achieving its intended aims, to identify aspects that are being done well, and ways in which the process might be improved... HTA bodies should start with a comprehensive description of how a desired change is expected to happen from a deliberative process in their context, including the resources (inputs) needed along with the rationale for such change...” [25] “Evaluating process and outcome is crucial. One method proposed by our Reference Group was an impact log to capture key contributions after each meeting or key interaction. The log then forms the basis of a narrative model, which can provide a qualitative ‘story of model development’ in a way that replicates the quantitative elements with narrative, capturing key decisions, key assumptions, values and other aspects of discussion that public contributors feel are important.” [52]	7
14	Feedback to collaborators/the community	Decisions made and outcomes of the research process need to be shared with collaborators and the community that they represent, ideally prior to any wider dissemination activities. Study information and outputs should be tracked and shared with collaborators in a transparent and accessible manner. This may require the bespoke production of forms of dissemination suitable and accessible for target groups (e.g. lay summaries). Collaborators can advise on the best ways of disseminating information to the groups they represent, with consideration of alternative dissemination formats	“Accountability—Ensure results are shared and used to inform changes and communicate how those changes were implemented. Measures to ensure accountability should be co-created with Indigenous partners in discussions before starting the study. After the study is completed, survey developers should make findings available to communities so that communities can implement changes to existing services or respond to new opportunities based on the findings... Knowledge dissemination, for instance, requires presenting and sharing findings with the community first. Study outcomes should be crafted to be presented in a way that is accessible, informative, culturally appropriate to the community context and protective of participant confidentiality.” [42] “the use of feedback loops to further inform the patient community on the use and impact of the data, will help to strengthen and empower the patient community. This will enable the patient community to further contribute and participate in the design, collection, generation and analysis of PXD.” [50]	5
15	Ensure full and transparent reporting	Any collaboration undertaken should be fully and transparently reported in core outputs emerging from the research project (e.g. journal papers, reports). This includes who was involved, what was done, how it was done, and why, as well as the impacts and outcomes of collaboration and their influence on the research project. Any evaluation of the collaboration should also be reported (#13). Collaborators should be clearly distinguished from research participants and any perceived or real conflicts of interest should be disclosed. Where restrictions (e.g. journal word limits) make the full and transparent reporting of collaboration difficult, then supplementary online material should be used	“In the reporting of PROM development researchers should clearly distinguish between patients as PI where they are co-producing the instrument versus patients as research participants. In PROM development the line may be blurred. For example, a focus group could be held to discuss instrument layout with a sample of patients as research participants or with a PI group.” [29] “Characterize the engagement accurately when citing or acknowledging patient–community participation; the type and level of engagement should be described; and whether or not those who were engaged endorse the final product… Acknowledge and disclose real, potential, or perceived conflicts of interest by all participants.” [49]	5

COS core outcome set, HEMS The Health Economics and Multiple Sclerosis [Patient Involvement Group], HEOR health economics and outcomes research, HTA health technology assessment, PI public involvement, PPI patient and public involvement, PRO patient-reported outcome, PROM patient-reported outcome measure, PXD patient experience data

^#XX is a cross-reference to the number of another synthesised recommendation in the table

Mapping of Evidence

Table 5 shows the results of mapping the available evidence from included valuation studies featuring collaborative engagement against the synthesised best practice recommendations. There was no evidence for 9 out of 15 synthesised recommendations having been applied in any of the valuation studies reviewed. For recommendation #2 ‘Involve suitable representatives from target groups’, three studies provided sufficient evidence [30, 31, 68] and nine provided uncertain evidence, as multiple collaborators were involved but no evidence was provided that these were suitable or had sufficient diversity or breadth relative to the target group(s). For recommendation #3 ‘Involve at key stages throughout the research process’, three studies provided uncertain evidence [30, 31, 68], as collaborators were involved in more than one stage of the work, but it was unclear if these were key stages in the project (and why collaborators were not involved in further stages). For recommendation #5 ‘Ensure collaboration adds value to the research project’, one study provided sufficient evidence [31], and ten provided uncertain evidence, as the collaborative activity was described as “helpful” or adding value, but no stated rationale or expected outcome was provided nor any indication the value of the desired outcome was evaluated or evaluable. Finally, two studies provided sufficient evidence for recommendation #14 ‘Feedback to collaborators/the community’ and uncertain evidence for recommendation #10 ‘Foster sustained and meaningful engagement’ and recommendation #15 ‘Ensure full and transparent reporting’ [30, 31]. This is because evidence suggested a meaningful reciprocal collaboration may have occurred but there was insufficient detail to confirm this. Further, the authors included more details on engagement than comparative articles, but they were not fully transparent, and some additional details were included as acknowledgements only.

Table 5.

Mapping included studies against synthesised recommendations

Synthesised recommendation		Included studies [reference]
		1 [56]	2 [57]	3 [58]	4 [59]	5 [60]	6 [30]	7 [31]	8 [61]	9 [62]	10 [63]	11 [64]	12 [65]	13 [66]	14 [67]	15 [68]
1	Develop a collaboration plan/protocol	x	x	x	x	x	x	x	x	x	x	x	x	x	x	x
2	Involve suitable representatives from target groups	?	x	?	?	x	✔	✔	?	x	?	?	?	?	?	✔
3	Involve at key stages throughout the research process	x	x	x	x	x	?	?	x	x	x	x	x	x	x	?
4	Ensure responsibilities and roles are clearly defined	x	x	x	x	x	x	x	x	x	x	x	x	x	x	x
5	Ensure collaboration adds value to the research project	?	?	?	x	?	?	✔	?	x	x	x	?	?	?	?
6	Consider practical arrangements	x	x	x	x	x	x	x	x	x	x	x	x	x	x	x
7	Adopt a flexible approach	x	x	x	x	x	x	x	x	x	x	x	x	x	x	x
8	Provide necessary training	x	x	x	x	x	x	x	x	x	x	x	x	x	x	x
9	Ensure collaboration is appropriately resourced	x	x	x	x	x	x	x	x	x	x	x	x	x	x	x
10	Foster sustained and meaningful engagement	x	x	x	x	x	?	?	x	x	x	x	x	x	x	x
11	Adapt communication and resources for collaboration	x	x	x	x	x	x	x	x	x	x	x	x	x	x	x
12	Maintain good ethical standards	x	x	x	x	x	x	x	x	x	x	x	x	x	x	x
13	Evaluate collaborative engagement	x	x	x	x	x	x	x	x	x	x	x	x	x	x	x
14	Feedback to collaborators/the community	x	x	x	x	x	✔	✔	x	x	x	x	x	x	x	x
15	Ensure full and transparent reporting	x	x	x	x	x	?	?	x	x	x	x	x	x	x	x

✔ sufficient evidence provided in a study for recommendation, x insufficient evidence provided in a study for recommendation, ? uncertain level of evidence provided in a study for recommendation (see Appendix 4 of the ESM for criteria)

Discussion

This scoping review represents a first of its kind exploration into the degree of collaborative engagement in contemporary health state valuation studies and how this evidence maps onto existing recommendations that have been provided in the health economics and PROM literature. The methodological approach applied was rigorous and followed best practice guidance for scoping reviews [22]. A synthesised framework of ‘best practice’ recommendations was produced, designed to guide future research into collaborative engagement in health state valuation internationally, and which may have broader applicability (e.g. within PROM research and affiliated disciplines).

While examples of recent collaborative engagement activities were identified in the health state valuation literature, the reporting of these was generally superficial in nature. The majority of examples centred on the development and validation of health states for valuation, particularly in vignette studies, with minimal evidence of engagement throughout the research project or at key stages (e.g. choice of stated preference methods, selection of the relevant sample, study planning, design, modelling methods, selection of the ‘best’ model and dissemination). Indeed, the mapping exercise demonstrated a lack of alignment between what is being recommended for collaborative engagement in the fields of health economics and PROM development/evaluation and what is being reported in health state valuation studies. It is important to note that the mapping is indicative only of current practice and reflects what is reported; it is possible that additional collaborative engagement activities are being conducted but not being reported (or not being reported in full). However, without publicly available evidence there is no way to verify this. Further, the review is not intended to be critical of included articles, but to highlight that collaborative engagement in the field of health state valuation appears to lag behind other fields in health research [32].

It is our intention that the synthesised framework of recommendations (Table 4) be viewed as a starting point for future work in developing best practice guidance for collaborative engagement in health state valuation. This will require further research and development to ensure the recommendations extracted here are ‘fit-for-purpose’ for use guiding best practice in collaborative engagement in health state valuation. It will also require the application of consensus-based methods to ensure that any resultant guidance is acceptable to the community of researchers who work in health state valuation [33–35]. To this end, we recognise that the synthesised recommendations are neither exhaustive nor prescriptive and flexibility is necessary in their application. We also recognise that modifications are likely necessary to account for different research settings. For example, very few recommendations focused on collaborative engagement in children, which is an area requiring special consideration [36]. Further, different recommendations may be more or less applicable to collaboration with different groups (e.g. the public, patients, decision makers) and may need to be further specified to be appropriate.

This scoping review has clear strengths, including a broad search strategy, which incorporated additional evidence beyond scholarly databases and an inclusive definition of collaboration in the search terms (i.e. that extended beyond patients and the public). However, limitations should be acknowledged. First, in order to make the review manageable and maximally relevant, the search for recommendations was restricted to the health economics discipline and the complementary field of PROM development/evaluation, as a parallel where engagement is more established. While commonalities are expected, additional recommendations may have emerged from a wider search/inclusion criteria (e.g. including generic guidance or that of other disciplines) and this emerging framework may need to be adapted over time, to take additional evidence into account. Equally, it could be argued that the synthesis should be restricted to recommendations from the health economics literature only. However, examples of all 15 recommendations were extracted from papers in the field of health economics, suggesting such a restriction is unlikely to change the study outcomes. A further limitation is that each included article was given equal weight during the synthesis of recommendations and future research should address their relative importance in the context of health state valuation.

As alternative existing guidelines exist for forms of collaborative engagement, such as patient and public involvement, any further use of the synthesised recommendations in this study (e.g. as a starting point for localised best practice guidance for collaborative engagement in health state valuation) should only be undertaken with consideration of other pre-existing guidelines, such as the Guidance for Reporting Involvement of Patients and the Public (GRIPP) checklists [18, 19].

Following our definition of collaborative engagement (which involves a peer-to-peer relationship between researchers and non-researcher collaborators), we aimed to include and extract evidence that was suggestive of collaboration and not participation in the review. Some articles also contained evidence explicitly suggestive of qualitative research activities to inform health states for valuation, but this evidence was not included as this represents research and not collaboration. Finally, we restricted included content to English only and focused only on the contemporary literature for health state valuation studies. While collaborative engagement has increased in health research over time [37], it is possible that extending the time frame for the searches may have revealed additional relevant information.

Conclusions

Current reported collaborative engagement in health state valuation studies is underdeveloped and does not appear to align with recommendations proposed within health economics and the related PROM literature. An emerging framework of recommendations for best practice in collaborative engagement represents a strategic starting point for moving towards guidance for improving collaborative engagement in health state valuation and related fields.

Supplementary Information

Below is the link to the electronic supplementary material.

Funding

This review has received funding from the EuroQol Research Foundation (1711-RA).

Declarations

Conflicts of interest/competing interests

Nancy Devlin, Michael Herdman, Simone Schieskow and Janine Verstraete are members of the EuroQol Group. The funder influenced our decision to search for any additional evidence specifically relevant to the EuroQol Research Foundation as part of our search strategy, but none was found. The funder at large had no further role in the review process. Philip A. Powell, Victoria Gale, Gurdas Singh, Anthea Sutton and Jill Carlton have no conflicts of interest that are directly relevant to the content of this article.

Ethics approval

Not applicable.

Consent to participate

Not applicable.

Consent for publication

Not applicable.

Availability of data and material

Not applicable.

Code availability

Not applicable.

Authors’ contributions

Concept and design: JC, ND, MH, SS, JV. Acquisition of data: VG, GS, AS. Analysis and interpretation of data: JC, PAP. Drafting of the manuscript: JC, PAP, AS. Critical revision of the paper for important intellectual content: JC, ND, VG, MH, PAP, SS, AS, JV. Obtaining funding: JC, ND, MH, SS, AS, JV. All authors certify that they meet the ICMJE criteria for authorship. All authors approved the final version of the manuscript and agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.