Table 3.

Identified gaps in patient engagement practises and process with example qualitative quotes reported at the theme level to support the identified gaps

Category (theme)	Criteria with gap identified (was there attention to…..?)	Illustrative open text responses supporting identified gap
Selection of participants and adequate representation	Clear description of the criteria followed to identify patient representatives needed	“Unlikely diversity [is] reflected as only patient advocates involved. Detail about who and how they were selected [is] not available”
Empowerment of stakeholders	Training for their roles and responsibilities with training material accessible to all participants*	“ ….assumed patient organisations train their patient representatives”
Transparency of roles, scope of involvement, and decision-making structure	Communicating any changes that could occur during the PE initiative up-front	“owner of the initiative informed that the information is described in confidential material that cannot be shared”**
Communication and feedback	Legal agreements written in a clear and accessible way and adapted to the target population	“legal agreements existed and included details but I can’t assess if they were suited to the needs of participants.”
	Include a dissemination and communication plan sharing the process and outcomes	“Process and outcomes are shared but detail about plans for this are not available “
Sustainability	Ensure the formation and maintenance of a partnership between all stakeholders	“It was sponsored by a company with a specific goal and for a limited time period” and “limited to the funding from the EU program” “This is part of the company sponsorship with organisation[x], so although this particular activity was a one off, it is part of a broader relationship between organisation[x] and the company”
Legal and ethical considerations	Code of conduct, which clearly states the (ethical) principles, governance requirement, rules and procedure of participation for all stakeholders involved	“To my knowledge specific forms, guidelines and templates were not created. However, given the organisations had official collaborative relationships as Working Party members, it is understood that legal and privacy contracts would have been managed as appropriate for any such arrangement”
	Privacy policy that describes policy to maintain data privacy of engaged patients in the engagement	“Contracts were signed between the patient organisation and company. However, legal agreements are not accessible to patients/patient organisations”
	Identification and addressing potential discriminatory, coercive, intimidating, and unethical behaviours, towards all stakeholders, before, during and after their participation?
	Management of potential conflicts of interest (up to avoidance)? disclosure, transparency and accountability
	Presentation of the terms and conditions of all policies and confidentiality agreements, in a clear and accessible way to the stakeholders involved
Supportive resources	Clear, transparent and equitable (fair) financial compensation framework to be in place and made available for patient representatives who participate	“Three patients in the steering committee received financial compensation, but compensation framework is not available in the information collected.”
Impact for R&D	Propose metrics to measure impact of PE	“Company [x] has created an evaluation survey to collect information on learnings and outcomes from a sponsor and member perspective, self-reported results are available. The survey doesn't measure specific metrics, but does include some questions that are proxies for impact.”
Learning and reflection [on the PE practices]	Propose methods, tools and monitoring systems to evaluate the PE practice systematically and at appropriate phases of the process	“They state a survey was used to get feedback from stakeholders about how to improve the design of the initiative and so on, but a framework for evaluation is not discussed” “Evaluation was informally processed by feedback and sharing among the leads and learnings are being incorporated into the 5 year plan” and, “They evaluated through live discussion as they progressed; no official criteria or assessment”
	Propose an evaluation framework for evaluation of the PE initiative
	Link between the evaluation criteria and the aims and practices of the PE practice

Gaps are identified and arranged under their respective overarching theme. Example qualitative responses from open text reported at the theme level are included and in some cases are relevant to more than one gap within a theme

*Taking into consideration languages, impairments, literacy levels, cultural background and the circumstances of (vulnerable) patients involved

**Open text responses reported at the theme level only. Semi-quantitative analysis revealed the gap at all levels