Perspectives on how to build bridges between regulation, health technology assessment and clinical guideline development: a qualitative focus group study with European experts

Abstract

Objective

Improving synergy among regulation, health technology assessment (HTA) and clinical guideline development is relevant as these independent processes are building on shared evidence-based grounds. The two objectives were first to assess how convergence of evidentiary needs among stakeholders may be achieved, and second, to determine to what extent convergence can be achieved.

Design

Qualitative study using eight online dual-moderator focus groups.

Setting

Discussions had a European focus and were contextualised in four case studies on head and neck cancer, diabetes mellitus, multiple sclerosis and myelodysplastic syndromes.

Participants

Forty-two experienced (over 10 years) European regulators, HTA representatives and clinicians participated in the discussion.

Interventions

Participants received information on the case study and research topic in advance. An introductory background presentation and interview guide for the moderators were used to steer the discussion.

Results

Convergence may be achieved through improved communication institutionalised in multistakeholder early dialogues, shared definitions and shared methods. Required data sets should be inclusive rather than aligned. Deliberation and decision-making should remain independent. Alignment could be sought for pragmatic clinical trial designs and patient registries. Smaller and lower-income countries should be included in these efforts.

Conclusion

Actors in the field expressed that improving synergy among stakeholders always involves trade-offs. A balance needs to be found between the convergence of processes and the institutional remits or geographical independence. A similar tension exists between the involvement of more actors, for example, patients or additional countries, and the level of collaboration that may be achieved. Communication is key to establishing this balance.

Strengths and limitations of this study.

• Although qualitative research remains subject to interpretation, the results were systematically generated from highly experienced participants, using a qualitative focus group approach.

• Through the design of this study, the focus groups generated an overview of tangible approaches and nuanced considerations for improving the alignment among stakeholders in the European treatment access pathway.

• The set-up of the focus groups breaks with the in-silo (or bi-silo) approaches from previous endeavours, combining the input of major healthcare decision-makers, that is, regulators, health technology assessment organisations and clinicians.

• The results should be interpreted in the European context; within this context, the transferability of the approaches to all European countries was carefully taken into account.

• Due to the focus on the processes for collaboration, the patient perspective was not included here; future research might add to these results by explicitly focusing on patient (or developer) involvement in regulatory, health technology assessment and clinical guideline processes.

Introduction

Decision-making for regulatory approval, health technology assessment (HTA) and clinical guideline (CG) development are independent processes, founded on shared evidence-based grounds.¹ The high-paced introduction of innovative and often personalised treatments increases complexity in decision-making in these healthcare settings.^2–4 Small populations cause imprecise efficacy estimates, and single-arm and other non-randomised or controlled trials aggravate the uncertainties in relative effectiveness assessments.^5–8 Positioning medicines in a long treatment pathway is challenging.^9–12 Despite having a common foundation, this complexity creates fragmentation among actors’ decision-making.

The fragmentation contributes to duplication in efforts by decision-makers that may be prevented but are hampered by poorly coordinated processes.¹ The divergencies in healthcare decision-making complicate the understanding of the information underlying individual treatment decisions by physicians and patients.¹³ Manoeuvring through the European framework to obtain market and patient access is a costly and resource intensive process for (small) pharmaceutical developers. Fragmentation may eventually unnecessarily hamper or delay patient access and provokes the risk of undesirable economic consequences in the case of expensive treatments.

Interest in the dynamics occurring at the intersection of regulators and HTA organisations is slowly growing, focusing on the consequences of conditional approvals, stakeholder interactions and post-approval data.^{6 14–16} Other bi-silo initiatives such as GINAHTA and EUnetHTA also target the bridge between HTA and CG development, exploring common methods and collaboration.^{17 18} Only one previous holistic endeavour described commonalities and differences in the criteria used by various health decision-making disciplines while highlighting the implications for practice and research.¹ Several solutions to overcome fragmentation were presented, focusing on the identification of data requirements among the silos that overlap. However, an elaborate discussion on how to achieve these agreements between decision-makers to overcome fragmentation is still missing.

In this study, we congregated authoritative actors from the regulatory field, HTA organisations and CGs across Europe to collectively discuss tangible ways to improve synergies among their processes for clinical decision-making for health technologies. The overarching aim of this study was to identify ways to reduce system fragmentation across healthcare decision-makers and break in-silo thinking. Two more specific objectives were, first, to assess how convergence of evidentiary needs among stakeholders may be achieved. Second, it intended to identify to what extent convergence can be achieved.

Methods

A qualitative design was used, consisting of eight focus groups contextualised in the HTx case studies on diabetes mellitus (DM), head and neck cancer (HN), multiple sclerosis (MS) and myelodysplastic syndromes (MDS). HTx is a Horizon 2020 project supported by the European Union lasting for 5 years from January 2019, with the aim to create a framework for the next generation HTA to support patient-centred, societally oriented, real-time decision-making on access to and reimbursement for health technologies throughout Europe.¹⁹ Case studies were used to make the discussion more specific and the high diversity across the four cases ensured generalisability of the results to other disease areas. Results are reported in accordance with the Consolidated criteria for Reporting Qualitative research checklist.^{20 21}

Research team and reflexivity

Authors MLDB and MM (DM), HGML and RAV (HN), WG and MH (MS), AM-T and JW (MDS) moderated focus groups in pairs. All moderators except one (MH) were at least at PhD level. They were involved for their relevant experience in regulatory or HTA science and practice (range 3–36 years). A training was organised prior to the focus groups to emphasise the research objectives and align moderation techniques among the pairs. The moderators were acquainted with some of the participants through previous endeavours, though not to all.

Two preparatory information sheets on the objectives and one of the case studies were disseminated to the participants 2 weeks prior to study commencement (see online supplemental S1 for an example). A presentation at the start of the focus groups introduced the participants to the research topic in more detail and familiarised them with the moderators and their professional backgrounds. The presentation also put the focus groups in the context of the umbrella project HTx. Consent for recording the focus groups was asked at the start of the meeting, and consent for listing the participant’s name was requested over email after dissemination of the manuscript.

Study design

The purposive selection of European expert participants was based on seniority in either an executive or strategic role in the regulatory, HTA or CG field, preferably related to one of the case studies. Participants were identified based on authoring relevant literature (eg, on aligning evidentiary requirements) or CGs (eg, for the case studies), through the author’s networks and subsequent snowballing. Invitations and two reminders were sent through email. At least one participant from each stakeholder group was present in each online dual-moderator (mini) focus group in Zoom (see figure 1 for details).²² The average experience of 42 accepting participants was 29 years (range 11–49; see online supplemental S2). A total experience of 1213 person-years inputted the results. Each focus group was centred around one of the case studies to maximise the variation in therapeutic areas exhibiting critical and diverse decision-making challenges.

Figure 1.

Participant selection and distribution over case studies and focus group days. HTA, health technology assessment.

To facilitate discussion, an interview guide was developed with input from all moderators during a meeting prior to the focus groups (see online supplemental S3). It indicated that conflicting opinions should not necessarily be resolved, rather, follow-up questions were to be asked for clarification. The 2-hour focus groups were audio and visual recorded by a technical assistant present in each session and turned into transcripts using Microsoft’s web version of Word (Microsoft, Redmond, Washington, USA).²³ Ambiguity was resolved through repetitive manual assessment of the recordings, making return of transcripts for comments or correction unnecessary. Repeated focus groups and field notes were not required for this research question.

Patient and public involvement

Patient organisations are part of the HTx project and have been involved in formulating the research topic and approving the final manuscript. Patient participation in the focus groups was not appropriate in the research scope.

Analysis and findings

Underpinning the design was a three-step thematic content and ethnographic analysis, allowing for both systematic analysis and detailed interpretative accounts.^{22 24} Themes and relationships were identified through the formation of numerous coding categories using descriptive coding techniques on all transcript sections, performed independently by two authors (MH and MM) in NVivo V.12 Pro (QRS International, Burlington, Massachusetts, USA) (step 1).²⁵ In line with the interview guide, the code formation was built on a few commonly discussed alignment strategies in literature (eg, sharing methods, early dialogue, joint scientific advice) and the eminent PICO (population, intervention, comparator, outcomes) framework.^26–30 The codes and their content were formed independently by the two authors, and were further combined, subdivided or relocated through author triangulation until consensus (MH, MM and RAV) (step 2).^{31 32} Results were complemented by illustrative quotations (step 3), for selection we considered the distribution of quotes across stakeholder groups, case studies and countries.

During the interpretation of code content, apparent agreement (not consensus) or disagreement among the perspectives was scrutinised. The results were assessed for data saturation in the first step of the analysis. Saturation on all themes was reached after coding six focus groups, but not for subthemes. The final coding tree as well as codes not used in this study are shown in online supplemental S4. A summary of the findings was shared with participants to be checked for errors.

Results

The themes and subthemes that have been identified in the focus groups are visualised in figure 2. Summarising recommendations based on these themes are presented in boxes 1 and 2. Quotes from the ethnographic analysis are shown in online supplemental S5.

Figure 2.

Final node tree with (sub)themes for each of the two research questions.

Box 1. Summary of actions that may improve convergence between regulators, health technology assessment (HTA) organisations and clinical guideline developers.

Communication

1. Institutionalising continuous communication between decision-makers on recommendations and underlying uncertainties throughout the treatment lifecycle in a transparent (including for patients) manner may improve understanding and alignment.

2. Increased mutual understanding between stakeholders may improve decision-making by enlarging the context considered in decisions, decisions and may therefore reduce duplication of efforts by downstream decision-makers. This may also facilitate expectation management, for example, on the availability and quality of data, the collaboration with stakeholders or on new treatments, allowing for anticipation.

3. Using similar methods, definitions and procedures by decision-makers could ensure a common language and thus prevent miscommunication.

Formalised interaction

1. Multistakeholder early dialogue (including patient, clinician, regulatory and HTA representatives) may be instituted to develop a core data set, and the scientific consultation moment may be employed to share these requirements in the (parallel/joint) scientific advice for the pharmaceutical company. There was disagreement on the inclusion of pharmaceutical companies in these dialogues.

Internal factors

1. Proper governance structures should assign responsibilities to specific stakeholders and individual leaders could promote motivation to improve alignment. The governance structure should create an open culture, receptive to different ways of working.

2. Transparency on decision criteria and working procedures could manage expectations and guidance documents, for example, on data generation, which could improve transparency.

External factors

1. Centrally organised legislative measures, mandates or other incentives could provide institutions, carrying a specific responsibility, with the tools to live up to their task. Stakeholders should be incentivised towards the common goal, that is, facilitating care for the patient.

Box 2. Summary of the extent to which we may achieve convergence between regulators, health technology assessment organisations and clinical guideline developers.

Data generation

1. Evidence should not necessarily be aligned, rather it should be ensured that the required elements for all decision-makers are included.

2. There is a need for guidance on alternative (pragmatic, innovative, dynamic) trial designs based on the perspectives of all three stakeholders, in particular clinicians. A centrally coordinated patient registry system may facilitate alignment and foster re-evaluations by all stakeholders in a lifecycle approach.

Independence and remits

1. Data sets, methods, policies and timelines may be better aligned, and the assessment and decision-making (deliberation) processes should remain independent.

2. For some countries or institutions with shared interests, regional collaboration or reliance in decision-making may make sense if decisions cannot be made individually.

Timing the alignment

1. A lifecycle approach with re-evaluations on all stakeholder levels may be achieved with more collaboration but added benefit to stakeholders should first be proven and may only be worthwhile if there is a mandate to act on these re-evaluations.

Prioritising alignment efforts

1. A lifecycle approach and early dialogue increase pressure on resources, however, collaboration may save time and other resources in the long run. Products with high unmet medical needs and high uncertainty should be prioritised.

2. A clear definition and continuous reassessment of unmet medical needs could be the basis for prioritisation. Horizon scanning, in case it is centrally coordinated, might be a tool that could facilitate prioritisation in alignment, collaboration and a lifecycle approach.

Transferability

1. Transferability is very important to ensure all stakeholders and countries can participate in the new methods and policies, and this may be achieved by the active involvement of smaller, less-resourced countries.

How may we achieve convergence of evidentiary needs among stakeholders?

Communication

Improving communication across institutions was the most dominant theme, including among decision-makers and between countries. An institutional level for communication was repetitively preferred over (bilateral) convenience contact between individuals. Communication should encompass transparency on considerations and uncertainties underlying the decisions. Communication to the general public as well as between decision-makers should be clear and understandable, always keeping in mind the patient as a potential audience.

Participants felt that all stakeholders would benefit from having an improved mutual understanding of the tasks and processes of others, enabling stakeholders to put their decisions into a larger context. Participants highlighted the importance of understanding health-economic principles when developing treatment guidelines, and reversely the importance of understanding clinical treatment practice to accurately advise on cost-effectiveness and reimbursement. As another example, for a regulator giving scientific advice to pharmaceutical companies, it is important to realise the added value of quality of life and other patient-reported outcomes, whereas the HTA organisation benefits from a clear understanding of the defined patient population in the market authorisation label. Education or awareness creation could be achieved through conferences or improved by regular institutionalised communication. A better understanding of the needs downstream and the recommendations upstream could also reduce duplication of efforts.

Managing expectations was discussed in the context of being realistic on available amounts and quality of data that may be generated up until market authorisation or reimbursement. It may also relate to expectations about other stakeholder’s tasks and processes as discussed above. Finally, it may include expectations on new treatments, that is, what does society expect from a new treatment? Education and creating awareness about other stakeholders (quite tangible) are required to be able to manage expectations (less tangible). Managing expectations may, however, result in others expecting things to happen in a certain way, therefore it commits actors to doing things in this particular way (eg, a standard layout of reports or a fixed moment for communication). It may therefore reduce the flexibility or pragmatism in dealing with new unexpected situations.

Communication between stakeholders and mutual understanding would be aided by a shared or common language. With a common language, participants referred to the use of similar methods, adoption of likewise definitions and employing standardised outcome sets.

Most importantly, there seemed to be agreement on the benefits of collaborating to define patient populations consistently. The wording in regulatory product information has a significant impact on how HTA assessors and clinicians can include the patient population in their reports and guidelines. Collaboration on patient population definitions might mitigate to some extent the discrepancies and contradictions between regulatory product information, reimbursed (sub)populations and CG recommendations. Additionally, collaboratively defining patient populations may aid the development of data generation plans for the identification of subpopulations. For instance through the distinction between a population that requires the collection of histological or genetic information, whereas post hoc stratification in phase III may suffice for a diabetic subpopulation.

Moreover, employing shared methods would act like a shared language. It could increase mutual understanding and ensure independence for downstream stakeholders as it may serve as a guarantee that information is assessed in a certain way. Decisions would remain independent while building on the same foundation. A participant highlighted that there have been attempts to align the European Public Assessment Report (EPAR) discussion produced in the regulatory context with the EUnetHTA relative-effectiveness assessment (REA) discussion used for HTA decision-making. It seemed that there was little additive information in the REA (mostly differences in interpretation) compared with the EPAR unless it concluded by saying whether ‘drug A is better than drug B’. The right to draw this conclusion makes the value of these REA reports higher and increases uptake by other HTA organisations and guideline developers.

Disease-specific outcome sets, such as the ICHOM (International Consortium for Health Outcome Measurement) outcome sets, that are developed with all relevant stakeholders could aid data generation and improve the data quality for all three decision-makers. Participants noted that it is important that they would represent a balance between patient-relevant outcomes (what matters to them) and workable outcomes for regulators, HTA and clinicians (what is measurable).

Formalised interaction

The discussion seemed to distinguish between early dialogue and scientific advice. Early multistakeholder dialogue in the pre-competitive space could aim to reach some level of agreement on a core data set, that is, required for all decision-making processes downstream (eg, based on PICO, in line with the aim of ICHOM sets). Scientific advice referred to the process of advising pharmaceutical companies on trial design, ideally after the early dialogue and based on the established collective data set.

Early dialogues were discussed by some as a public event (eg, all could listen in) to secure transparency. It could facilitate expectation management on the timing, quality and amounts of data generated as well as foster implementation of evidence-generation plans if all users would be involved in drafting the plans. It may reduce surprises or confusion later if the in-silo thinking makes place for integrated conversation. Participants agreed that patients should be represented in the early dialogue as a bottom-up shared denominator to ensure the inclusion of patient-relevant outcomes and discuss their risk tolerance in clinical trials. Some indicated that the early dialogue should exclude the pharmaceutical industry to prevent any conflict of interest, others argued that all stakeholders should participate to create a level playing field. Early dialogue should cover the full scope of evidence generation (ie, what should be collected now, and what can be collected later) and could prioritise the design of pragmatic trials, ensuring that it serves all decision-makers.

According to participants, there is a clear desire from regulators, HTA organisations and the pharmaceutical industry to have scientific consultation. Building the advice on a collaboratively established required core data set through early dialogue might benefit alignment by creating a shared foundation on which downstream stakeholders can build their individual decisions. Involving downstream users of the core data set at an early stage in the development of indication-specific elements may save time later in the process. For implementation purposes, it is important to show the added value of scientific advice and subsequent data generation to companies and other stakeholders (eg, do the odds of authorisation improve with advice and additional data?).

Internal factors

Alignment requires a change in the embedded way that all stakeholders are working. Clear governance and leadership could encourage a culture that is open to doing things differently. So, it is important to think about a common interest, the ‘why’ would we change certain things, to motivate for change. At an institutional level, stakeholders may be given responsibilities for specific tasks, such as initiating the previously described early dialogue or managing data generation, which could ensure that tasks will be fulfilled. At an individual level, cultural change may require the motivation of individual leaders and the confidence among employees that bottom-up signalling may have an impact. Having one leading stakeholder for initiating early dialogues, preferably at the European level, would be helpful.

Throughout the discussion, there was a clear call for increased transparency. This is related first to stakeholder processes, for example, in the existing dialogues and data generation. Consistency in data generation procedures through guidance could increase transparency. Participants highlighted the importance of the open-access movement. They felt that there are no sufficient mechanisms for decision-makers to access individual patient data from trials or registries and that the current information governance often allows only for aggregated or summarised data.

External factors

Participants discussed the importance of a legal basis and the appropriate incentives that support alignment, for example, incentivising participation in early dialogues to generate a core data set (as described under ‘data generation’). Participants also indicated that institutions need the mandates as a kind of toolbox to enforce their remits, for example, to properly execute special pathways (expedited, or conditional approval pathways). They feared that without this mandate, there would be no sufficient leverage to change things, for example, to follow-up on requested additional data. What such a legal basis should look like remained undecided, although discussed examples included legal consequences if post-licencing data was not generated or a demand for access to data, dossiers or treatments in smaller regions, which is in line with the new European pharmaceutical legislation. It was also suggested that the European Commission should take a larger role in the development of these legal consequences.

To what extent can we converge evidentiary needs among stakeholders?

Data generation

Participants indicated that there is not necessarily a need for alignment of evidence, but rather for developing core data sets that contain all the information needed throughout the early treatment lifecycle. In particular, more attention is needed for the selection of the comparator treatment and patient-relevant outcomes. Participants felt the need for a stronger clinical and patient voice in data generation to ask the most relevant questions and pinpoint unmet medical needs. Existing and frequently updated European Medicines Agency (EMA) guidance on data generation would be a starting point and would increase in value by systematically consulting the data requirements of downstream decision-makers. The EMA likely has the strongest mandate to request data, so there seemed to be some agreement that this task would best belong to the regulator. However, the regulatory participants indicated that they experienced it as difficult to get clinicians and HTA organisations to be involved in previous early dialogue efforts.

Participants highlighted the importance of alignment on the development and implementation of alternative trial designs, including prospective registry-based trials as well as iterative, adaptive or pragmatic trials and modelling approaches for follow-up. Alternative designs were not regarded as the silver bullet for all data generation problems, as long-term effects were viewed as the most stringent data generation issue that will always require time. New designs would make the line between trial and clinical treatment setting thinner, creating legislative challenges. Thinning of this line may (further) disincentivise payers to reimburse treatments. It would increase the uncertainty and risks for payers, as these ought to be explicitly described in trial settings but are less emphasised in real-world settings. This may create a vacuum where no data is generated due to uncertainty while existing uncertainty remains unresolved as no new information is generated.

Additionally, participants saw the need for alignment on real-world data (RWD) use and preferred registry data in parallel to or as a follow-up to randomised controlled trials (RCT), when RCTs are not feasible or desired, and for off-label treatments. The main obstacles preventing alignment were lacking centralised coordination among a wide variety of initiatives, privacy legislation and difficulties to maintain registries that generate independent outcomes while being dependent on financial support from pharmaceutical companies.

Independence and remits

While participants recognised the benefit of further streamlining, preserving the different remits of the respective stakeholders is an important condition for collaboration. The agreement seemed that it is justified to have separate decisions for market approval (risk-benefit), treatment access (relative effects) and clinical usage (treatment decisions). Clearly communicating the perspective taken (eg, the remit or individual vs population perspective) is important for interpreting recommendations or decisions as different perspectives may emphasise different criteria (eg, more on safety or more on economic criteria). Further centralisation of HTA than currently proposed with the new HTA regulation³³ was not perceived as feasible due to different national reimbursement pathways, national economic contexts as well as different disease and population characteristics. Several stakeholders expressed a wish for shared requirements for the assessment of clinical studies. Also, independence between HTA and CG developers was emphasised. Using the same methods and criteria in HTA and CGs can make guidelines more restrictive than intended due to considering cost-effective options rather than all clinical options. Moreover, differences in mandates and funding of HTA assessments and guideline development may complicate the implementation of treatments into a guideline in a holistic manner, for example, considering treatment sequences.

Levels of collaboration range from sharing information to making collaborative decisions. The latter set a high bar and can prove to be very difficult. Examples in the last category are the cross-border reliance structure at the EMA, the new HTA regulation, international treatment guideline development and regionally shared reimbursement decisions. The start of shared decision-making requires a time investment to get acquainted and manage perspectives, while later it may save time. Participants felt that collaboration may also be inspiring as you learn from each other and incorporate the best ideas from different parties. In shared decision-making, it is important to stick to the topics that are shared (eg, clinical assessments rather than finances, or cost-effectiveness in treatment line rather than individual treatment decisions). Participants indicated the importance of aligning shared information, however, that different perspectives should not influence individual decision-making. If European-wide collaboration proves to be difficult, regional initiatives may be an initial way forward.

Timing the alignment

Aligning procedural timelines may reduce lags and simultaneously facilitate collaboration as institutions or countries work on similar dossiers at the same time. Proactive communication on interim steps in processes towards downstream stakeholders as well as early dialogue could prevent the necessity to assess inadequate information which is time-consuming and nourishes discussion. Clinicians indicated that an important time gap to fill is between HTA decisions and CG recommendations. Guideline development is often initiated after the completion of preceding assessments. Clinicians felt constantly like they were playing catch-up since guideline development processes are lengthy. As soon as guidelines are completed, new treatments have entered the market and are reimbursed. Additionally, payers may sometimes only reimburse if treatments are positioned in a guideline. Therefore, clinicians expressed the desire for ‘living’ guidelines, where substantial new information or new products are directly incorporated into existing dynamic documents, and by these means communicated to HTA organisations, payers, regulators and patients.

Participants viewed data generation as an ongoing process of reviewing treatments over time (from development to clinical use), known as a lifecycle approach. Considering the wider narrative of the full treatment pathway was perceived to be the strength of CGs. As treatment practice is constantly evolving, the pathway should be reassessed with every new treatment in living guidelines. However, the added value of additional data generation to the parties responsible for generation has not yet been demonstrated. Participants felt that the mandate to act on these re-evaluations is currently not very strong but would demonstrate the added value of reassessments (in or decrease in prices, change treatment positions, etc). The value of a lifecycle approach may be different according to institutions, as was believed to be demonstrated by the fact that no(t many) treatments have so far been retracted by the Committee for Medicinal Products for Human Use (CHMPbased on additionally generated efficacy evidence, whereas prices or treatment positions have changed.

Repetitively, participants highlighted the importance of not making the system overly complex. A complex system would jeopardise maintaining transparency and thus comprehension for all stakeholders. It would create an inordinately slow and expensive system, and it would make the system less pragmatic or dynamic when change is needed. As indicated by clinicians, mostly in real-world treatment settings, without the luxury of financial and personnel resources, data collection is sometimes too resource intensive. The collection of quality-of-life data sometimes does not happen for this reason, due to the inability of compliance by patients and clinicians. Registry-based data generation and CG development were suggested to benefit from simplicity.

Prioritising alignment efforts

Participants believed that much of the current duplication in efforts could be prevented. Early dialogues could facilitate prioritisation of treatments that require more attention, but also require planning, preparation, attendance and the risk that treatments do not make it ‘to the next round’. Also, alignment and culture change require additional (initiation) resources. Registries may increase the administrative burden for clinicians and require development, implementation, maintenance and skilled people. There is a need for piloting, prioritising and increased efficiency, for example, through digital meetings. Participants agreed that unmet medical needs should in principle be the most important factor in prioritising. There is no harmonised definition for unmet medical needs and no coordinating institute to guide what is needed. A centralised approach for horizon scanning, such as the International Horizon Scanning Initiative, was mentioned as a tool to facilitate prioritisation by anticipation on, for example, products with expected high uncertainty.

Transferability

The transferability of methods, policies and data is of utmost importance in achieving alignment. Discussions related to transferability touched on differences in healthcare systems, financial resources and country size, of which the latter two were most profound. Smaller countries would not be the most logical candidate to initiate or lead collaborative dialogues as they may have fewer resources at their disposal. Rather, let them participate and fine-tune outcomes to fit their situation. Participants indicated that sharing specific advice (eg, treatment A is better than treatment B) or documents (eg, draft EPARs) might save resources in smaller countries because they can rely on these outputs. More alignment across jurisdictions and involvement of less-resourced countries in early dialogues may create a learning opportunity and will increase comprehension. Transparency on, sharing of and open access to data may reduce additional waiting times for smaller countries. Currently, (international) CGs do not often consider economic discrepancies between countries, introducing problems when an expensive treatment is recommended as a first-line option. Participants stressed the involvement of smaller countries in patient registry development as they often have more difficulties maintaining registries after clinical trials have ended.

Discussion

This paper builds on the recognition that (evidentiary) criteria for decision-making in market authorisation, HTA and CGs are largely overlapping.¹ The similarity between the present findings and those described by Schünemann et al stresses the importance of measures to improve alignment. Schünemann et al aimed to reconcile how evidence is used across stakeholders and where the overlap lies in decision-making criteria to identify bridging opportunities. This current paper takes the effort to build bridges one step further, aiming to provide tangible system recommendations and nuanced considerations to further improve alignment between decision-makers in a European-wide setting.

Both the Schünemann et al and present paper independently highlight the need for improved communication, including deliberative considerations that informed judgement.¹ They discussed the agreement on essential criteria for decision-making, a core data set that includes prioritisation of the PICO elements and the essential outcomes and measurement techniques. Vreman et al recently demonstrated that regulator-imposed post-approval studies rarely resolve HTA’s concerns, for several reasons including a lack of aligned timing.³⁴ Existing parallel scientific advice procedures, however, demonstrated the difficulties with reaching full alignment on PICO elements, predominantly regarding the comparator.^{29 35–37} Communication is particularly important if stakeholders do (initially) disagree. The present study suggests that a multistakeholder early dialogue feeding subsequent scientific advice may be a useful tool to achieve increased agreement between decision-makers. Disagreement existed on the inclusion of pharmaceutical companies in these dialogues. If done so, trusted frameworks should be used to manage conflicting (financial) interests, such as those developed by the Guidelines International Network.³⁸ The use of similar methods and shared definitions was also a common theme in both papers. Schünemann et al highlighted three methods for (1) primary research evaluation and systematic reviews, (2) uncertainty of evidence rating and (3) model trustworthiness. Focus group participants in the present study added methods for quality of life measurement, common disease models, evidence generalisability, using and interpreting RWD and methods for guideline development.

Contextual factors, such as decision criteria from other stakeholders, perspectives and countries, were discussed as factors to consider during one’s decision-making. New elements that emerged in the present study were the social aspects of alignment, such as mutual awareness creation, expectation management and trust building. Also, the various levels of collaboration that may be achieved, the practical aspects such as aligning timelines, the required prioritisation to increase feasibility and the transferability of the recommendations were not discussed previously. However, the latter is critical in achieving alignment and calls for carefully considering any additional aspects when developing or implementing new policies.³⁹

Throughout the focus group discussions, no agreement was observed on where the exact line between collaboration and alignment on the one hand, and independence in respect of remits on the other should be drawn. It was suggested to consider decision criteria from other stakeholders in the larger context of one’s own institution’s decisions to improve alignment and mutual awareness. Contrarily, many participants indicated that the remits, the decision-making processes or the deliberation should stay independent and may not be influenced by others (whether other decision-makers or pharmaceutical developers), as this may negatively affect the objectivity of the decision. This thin line has been discussed previously in light of the regulatory and HTA interface.³⁰ A similar balance was discussed regarding the inclusion of multiple stakeholders (patients, payers, sponsors) and multiple countries (including low-income and middle-income) as opposed to the level of collaboration that may be achieved. Efforts were said to be preferably inclusive, however, having many different views and needs may complicate and slow the process. Future research should aim to better clarify these balances.

Strengths and limitations

Including clinicians in this discussion highlighted some important benefits of alignment that would have not been recognised in a single interaction between regulators and HTA representatives. Clinicians elucidated the challenges arising downstream, for example, by the definition setting for patient populations and the over-alignment of methods resulting in restrictive guidelines. Future studies should also involve patients, as well as pharmaceutical companies and payers. Another strength of this endeavour is the scientific robustness of the focus group design, execution and reporting. Given the qualitative nature of this study, the results are subject to the selection of participants and interpretation of the researchers. The authors aimed to the best of their ability to describe the viewpoints of the stakeholders in this manuscript, to which none of the participants had objections. A list of consenting participants is shown in the supplements. The implementation of the recommendations in this study will be subject to contextual factors, for example, depending on the countries or stakeholders involved, the momentum, or economic or healthcare factors. How to successfully do this may be studied per individual context. Similarly, further research may be needed to find solutions when persistent disagreement among stakeholders exists. Provided the differences across healthcare systems, these results should be interpreted in a European context and may not be transferable to other geographical regions.

Conclusion

Alignment among decision-making processes of regulators, HTA organisations and CG developers may be achieved through institutionalised communication. Multistakeholder early dialogues as well as shared definitions and methods may enhance mutual understanding, manage expectations and build trust. Core collaborative data set requirements as requested from sponsors or generated post-launch should be inclusive rather than better aligned. On top of this shared foundation, participants considered it critical that the deliberation and decision-making remain independent, respecting objectivity and the individual stakeholder’s remits as well as geographical independence. Smaller and lower-income countries should actively be included in alignment efforts. A balance needs to be found between alignment and independence as well as between including all stakeholders or countries and the level of collaboration that may be achieved. Alignment efforts should be prioritised towards the treatments most critical to patients with high decision-making uncertainty.

Data availability statement

Data are available upon reasonable request. Full transcripts of all the focus groups with anonymised participant codes can be made available upon request to the corresponding author.

Ethics statements

Patient consent for publication

Not applicable.

Ethics approval

The research protocol and data management plan were approved by the institutional review board of the division of pharmacoepidemiology and clinical pharmacology on 7 April 2021 under number 2021.03.16 UPF2104. Participants gave informed consent to participate in the study before taking part.