Bridging the gap: Can International Consortium of Health Outcomes Measurement standard sets align outcomes accepted for regulatory and health technology assessment decision‐making of oncology medicines

Abstract

Standard outcome sets developed by the International Consortium for Health Outcomes Measurement (ICHOM) facilitate value‐based health care in healthcare practice and have gained traction from regulators and Health Technology Assessment (HTA) agencies that regularly assess the value of new medicines. We aimed to assess the extent to which the outcomes used by regulators and HTA agencies are patient‐relevant, by comparing these to ICHOM standard sets. We conducted a cross‐sectional comparative analysis of ICHOM standard sets, and publicly available regulatory and HTA assessment guidelines. We focused on oncology due to many new medicines being developed, which are accompanied by substantial uncertainty regarding the relevance of these treatments for patients. A comparison of regulatory and HTA assessment guidelines, and ICHOM standard sets showed that both ICHOM and regulators stress the importance of disease‐specific outcomes. On the other hand, HTA agencies have a stronger focus on generic outcomes in order to allow comparisons across disease areas. Overall, similar outcomes are relevant for market access, reimbursement, and in ICHOM standard sets. However, some differences are apparent, such as the acceptability of intermediate outcomes. These are recommended in ICHOM standard sets, but regulators are more likely to accept intermediate outcomes than HTA agencies. A greater level of alignment in outcomes accepted may enhance the efficiency of regulatory and HTA processes, and increase timely access to new medicines. ICHOM standard sets may help align these outcomes. However, some differences in outcomes used may remain due to the different purposes of regulatory and HTA decision‐making.

The extent to which the outcomes used for market access and reimbursement decision making are patient relevant were assessed, by comparing these to standardized outcome sets developed by the International Consortium for Health Outcomes Measurement (ICHOM).

Abbreviations

ADRs

adverse drug reactions

CEA

cost‐effectiveness assessment

CR

complete response

DFS

disease‐free survival

EFS

event‐free survival

EMA

European Medicines Agency

EUnetHTA

European Network for Health Technology Assessment

FDA

Food and Drug Administration

HRQoL

health related quality of life

HTA

Health Technology Assessment

ICHOM

International Consortium for Health Outcomes Measurement

IQWiG

Institute for Quality and Efficiency in Health Care

N/A

not applicable

NICE

National Institute for Health and Care Excellence

ORR

objective response rate

OS

overall survival

PFS

progression‐free survival

PR

partial response

PROMs

patient reported outcome measures

QALY

quality‐adjusted life‐year

REA

relative effectiveness assessment

RFS

regression‐free survival

TTP

time to progression

VBHC

value‐based health care

ZIN

National Health Care Institute

1. INTRODUCTION

Value of health care is becoming more important than only assessing the volume provided, since healthcare costs are rising faster than available healthcare budgets. ¹ One of the concepts that tries to address this is value‐based health care (VBHC), which aims to achieve the best possible health outcomes to patients for the lowest possible cost. ¹ , ² A vital element of VBHC is the collection of health outcomes through a standardized approach. ¹ The International Consortium for Health Outcomes Measurement (ICHOM) has developed so‐called standard sets that focus on outcomes that are relevant for patients and that may facilitate the evaluation of VBHC in healthcare practice. ³

VBHC has been embraced in the assessment of innovative medicines, since it focuses on improving the value for money in health care and may support regulatory and Health Technology Assessment processes. ⁴ Regulatory bodies authorize innovative medicines for market access based on the scientific assessment of the efficacy, safety, and pharmaceutical quality. Subsequently, HTA agencies conduct an assessment of these innovative medicines for pricing and reimbursement decisions, which focus on a relative effectiveness and/or cost‐utility analysis. ⁵ Regulatory bodies and HTA agencies mostly use similar clinical data for their assessments, preferably from randomized clinical trials.

The use of outcomes relevant to patients is important in VBHC, as well as in regulatory and HTA assessments. However, based on an application containing similar evidence regulatory bodies may authorize an innovative medicine for market access, while HTA agencies may not approve it for reimbursement. ⁶ , ⁷ , ⁸ , ⁹ , ¹⁰ Although different perspectives to the relevance of outcomes may be due to the different remits of regulatory bodies and HTA agencies, some alignment in the use of those outcomes may promote more consistent and timely access to valuable innovative medicines. ¹⁰ , ¹¹ , ¹² , ¹³ Since ICHOM claims to include health outcomes that matter most to patients and has involved patient representatives to develop standard sets, ¹⁴ , ¹⁵ , ¹⁶ it may be an initiative, which could support this further alignment. Therefore, we aimed to assess the extent to which the outcomes used by regulatory bodies and HTA agencies are patient‐relevant by comparing these outcomes to those defined by ICHOM.

2. MATERIALS AND METHODS

We conducted a cross‐sectional comparative analysis of the content of ICHOM standard sets, and publicly available regulatory and HTA assessment guidelines. These assessment guidelines provide instructions to pharmaceutical companies who intend to submit an application for the assessment of an innovative drug regarding marketing authorization or pricing and reimbursement decision‐making. We especially focused on oncological indications, because currently many new oncology medicines are developed, which are accompanied with substantial uncertainty on the relevance of these treatments for patients. Additionally, different ICHOM standard sets are available for several types of cancer. We extracted regulatory assessment guidelines with a focus on oncology and additionally identified general HTA assessment guidelines.

In particular, we identified in November 2018 five ICHOM standard sets that focus on oncological conditions. ³ These standard sets included colorectal cancer, ¹⁷ breast cancer, ¹⁸ lung cancer, ¹⁹ localized prostate cancer, ²⁰ and advanced prostate cancer. ²¹ ICHOM is a not‐for‐profit organization, which aims to develop a minimum set of standardized outcomes that really matter to patients. ³ Each standard set provides a recommendation on the outcomes, including patient‐reported outcome measures (PROMs), that are relevant to patients with a specific medical condition. ICHOM standard sets are developed over a period of nine months in International Working Groups that consist of 15 to 20 members, and include leading clinicians, outcomes researchers, registry leaders, and patient advocates. The outcomes that are included in the standard sets are selected based on several criteria, such as psychometric quality and burden of assessment. ³ Before completing a standard set, key stakeholders are invited for an open review.

For the assessment of the regulatory guidelines, two regulatory bodies were included, which represent the two regions (the United States and Europe) with the highest spending on pharmaceuticals worldwide, ²² , ²³ namely the Food and Drug Administration (FDA) and the European Medicines Agency (EMA). For the HTA guidelines, we selected three HTA agencies representing three European jurisdictions for inclusion: the Dutch National Health Care Institute (ZIN), the National Institute for Health and Care Excellence (NICE), and the Institute for Quality and Efficiency in Health Care (IQWiG). NICE and IQWiG represent two of the four largest European jurisdictions, and ZIN and NICE are recognized as pioneers within HTA by actively collaborating within different European projects. One platform that facilitates the collaboration between European HTA agencies to conduct relative effectiveness assessments on a European level was also included: the European Network for Health Technology Assessment (EUnetHTA). ²⁴ EUnetHTA is funded by the European Union to facilitate HTA collaboration in Europe. HTA agencies and institutes from 30 European countries have become involved as partners. In order to support efficient production and use of HTA in European countries, EUnetHTA facilitates joint assessments. These assessments are produced by at least four EUnetHTA partners in different European countries and can be used for HTA decision‐making by all EUnetHTA partners. In addition, EUnetHTA developed the “HTA core model,” which is a methodological framework for the production and sharing of HTA information.

To identify regulatory and HTA assessment guidelines, author RK first searched the websites of the regulatory bodies and HTA agencies between mid‐October and mid‐November 2018. During this search, weblinks on the homepage were used, as well as the following search terms: endpoint, outcome measure, oncology, cancer, assessment, colon cancer, lung cancer, prostate cancer, breast cancer, patient‐reported outcome, and endpoint oncology. Second, assessment guidelines were included if they were published in English or Dutch, were final documents, focused on market authorization or pricing and reimbursement decision‐making, and focused on the acceptability of outcomes.

In order to extract data from the ICHOM standard sets, and regulatory and HTA assessment guidelines, we developed a standardized coding scheme by deductive content analysis. ²⁵ Authors RK and RV independently assigned codes to the ICHOM standard set about colorectal cancer, the EMA guideline “Guideline on the evaluation of anticancer medicinal products in man”, and the NICE guideline “Guide to the methods of technology appraisal 2013.” Any disagreements were discussed and resolved by consensus. Based on these discussions, the standardized coding scheme was assessed and adjusted where needed. Subsequently, authors RK and RV independently assessed the ICHOM standard set about breast cancer, the FDA guideline “Guidance for industry clinical trial endpoints for the approval of cancer drugs and biologics,” and the IQWiG guideline “General methods version 5.0.” The remaining standard sets and assessment guidelines were coded by author RK, since a second reviewer was deemed unnecessary based on the degree of consensus after two rounds of validation on 6 documents in total. All data were stored and analyzed using NVIVO 12. ²⁶

3. RESULTS

Based on the website search five ICHOM standard sets and 50 assessment guidelines were identified (Figure 1). All five ICHOM standard sets were included. Of the 50 assessment guidelines identified, a total of 15 were excluded due to the lack of focus on discussing the acceptability of outcomes in assessments, 5 were excluded because these were draft documents, 5 were excluded because they were duplicates, and 3 were excluded due to lacking focus on market access or reimbursement assessments. In total, 22 assessment guidelines were included since these focused on providing guidance on the acceptability of outcomes, of these ten were published by a regulatory body and twelve were published by an HTA agency or EUnetHTA. Both regulatory bodies published general assessment guidelines for oncological products; in addition, both provided guidelines for specific oncological conditions including breast cancer (FDA and EMA), lung cancer (FDA and EMA), and prostate cancer (EMA). All HTA agencies and EUnetHTA published assessment guidelines for medical conditions in general. Separate information was provided in these guidelines regarding relative effectiveness assessments and cost‐effectiveness assessments. In addition, IQWiG published a guideline focusing on the use of intermediate outcomes in oncology.

FIGURE 1.

Flowchart of guideline identification, and in‐ and exclusion

Mortality estimates were mentioned in all the guidelines and standard sets. The specific term “mortality” was used by EUnetHTA, NICE, IQWiG, and ZIN, and other terms used include “survival” (ZIN, EUnetHTA, NICE, FDA), “overall survival” (EUnetHTA, NICE, FDA, EMA, ICHOM), and “increase in life expectancy” (ZIN, IQWiG; Table 1 and Appendix S1). EUnetHTA, FDA, and EMA accepted overall survival (OS) as the most persuasive outcome to estimate clinical benefit (data not shown); likewise, ICHOM recommended OS as an outcome in all included standard sets (Table 2). Definitive outcomes, such as survival, were accepted by NICE, ZIN, IQWiG, FDA, and EMA as primary outcomes in their assessments (Table 3, and Appendix S2).

TABLE 1.

Acceptability of outcomes in regulatory and HTA decision‐making of innovative medicines as compared to ICHOM standard sets

Outcomes	HTA		Regulatory	ICHOM
	Reimbursement		Market approval	Value‐based health care
	REA	CEA
Mortality	X	X	X	X
Morbidity	X	X	X	X
Safety	X	X	X	X
Intermediate outcomes	X	X	X †	X
Progression
PFS	X	X	X †	X
DFS	X	‐	X †	‐
EFS	‐	X	X	‐
TTP	‐	‐	X †	‐
RFS	‐	‐	‐	X
Tumor response	‐	‐	X †	X
PROMs	X	X	X	X
Symptom reduction	X	‐	X	X
HRQoL	X	X	X	X
QALY	‐	X	‐	‐
Composite outcomes	X	X	X	‐
Biomarkers	X	‐	X	‐

Abbreviations: ‐, this outcome was not discussed or mentioned in the guideline or standard set; CEA, cost‐effectiveness assessment; DFS, disease‐free survival; EFS, event‐free survival; HRQoL, health‐related quality of life; HTA, Health Technology Assessment; ICHOM, International Consortium for Health Outcomes Measurement; PFS, progression‐free survival; PROMs, patient‐reported outcome measures; QALY, quality‐adjusted life‐year; REA, relative effectiveness assessment; RFS, regression‐free survival; TTP, time to progression; X, this outcome was mentioned in the guideline or standard set.

^†The FDA allows this outcome to be included in the assessment for accelerated approval and regular approval.

TABLE 2.

Acceptability of outcomes specific for lung cancer, breast cancer, and prostate cancer as published by FDA and EMA, in addition to their general guidelines, and ICHOM

Outcome	FDA	EMA	ICHOM
	Market approval	Market approval	Value‐based health care
Lung cancer
Overall survival	X	X	X
Progression
PFS	X †	X	‐
DFS	X	‐	‐
TTP	X †	‐	‐
Tumor response	X †	X ‡
PROMs	X	X	X
HRQoL	‐	X	X
Reduction symptoms	X	‐	‐
Safety	‐	‐	X
Breast cancer
Overall survival	X	X	X
Progression
PFS	‐	X	‐
DFS	X	X	‐
EFS	X	X	‐
RFS	‐	‐	X
Tumor response	X †	X	‐
PROMs	‐	‐	X
HRQoL	‐	‐	X
Safety	‐	X	X
Prostate cancer
Overall survival	N/A	X	X
Progression			X §
PFS		X	‐
DFS		X	‐
Distant metastasis‐free survival		X	‐
PROMs	N/A	X	X
HRQoL		‐	X
Safety	N/A	‐	X
Other
Use of pain medication		‐	X
Symptomatic skeletal event		‐	X

Abbreviations: DFS, disease‐free survival; EFS, event‐free survival; EMA, European Medicines Agency; FDA, Food and Drug Administration; HRQoL, health‐related quality of life; ICHOM, International Consortium for Health Outcomes Measurement; N/A, not applicable; PFS, progression‐free survival; PROMs, patient‐reported outcome measures; RFS, regression‐free survival; TTP, time to progression.

^†FDA may use PFS, TTP, and tumor response rates for lung cancer to support both regular and accelerated approvals, and specifically allows tumor response rates for breast cancer to support accelerated approval.

^‡EMA may accept tumor response rates as outcome in exploratory studies for early evaluation approvals.

^§ICHOM recommends the collection of the following outcomes for prostate cancer regarding progression: development of metastasis (advanced and localized prostate cancer), development of castration‐resistant disease (advanced prostate cancer), biochemical recurrence (localized prostate cancer), procedures needed for local progression (advanced prostate cancer).

TABLE 3.

Hierarchy of outcomes accepted by regulatory bodies, HTA agencies, and ICHOM

Institute	Primary outcomes	Secondary outcomes
EUnetHTA		Non‐definitive outcomes (eg, morbidity, function, HRQoL), ADRs
Life‐threatening disease	Long‐term and definitive outcomes (eg, mortality or survival), ADRs, PFS d , HRQoL g	Morbidity, HRQoL
Non–life‐threatening disease	Mortality or survival	Not mentioned
First assessment	Morbidity, PROMs, HRQoL	Not mentioned
Re‐assessment	Definitive clinical outcomes (eg, mortality and survival)	Not mentioned
Economic evaluation	Definitive clinical outcomes on morbidity and mortality (eg, stroke, fracture) Life‐years gained, QALYs	Not mentioned
FDA
Regular approval	Survival improvement, OS, PROMs, intermediate outcomes, PFS, improvement in physical functioning or tumor‐related symptoms, time to progression of cancer symptoms, toxicity, improvement in DFS d , durable complete response d , substantiated ORR d , TTP f	Tumor measurement and response, PROMs, HRQoL, biomarkers
Accelerated approval	Intermediate outcomes, DFS, PFS, TTP, ORR, CR	Not mentioned
EMA	Efficacy (eg, survival), safety (eg, tolerability and severe or life‐threatening ADRs), TTP d , PFS d , time to symptomatic tumour progression d	HRQoL, symptom deterioration, PROMs
Single agents and combination therapies	Cure rate, OS, PFS, DFS, event rate d , symptom control d , time to symptomatic progression d	ORR, rate of tumor stability, symptomatic tumor progression, HRQoL, PROMs
Treatment with curative intent	PFS b , d , DFS e , EFS b , ORR b , increased cure rate c , OS c , EFS d , CR a , d , CR+PR a , d , major increase ORR b , d , major increase in EFS c , d or PFS c , d	Not mentioned
Treatment intended to achieve long‐term disease control	PFS a , b , improved survival c , major benefit in PFS c , d	Not mentioned
Palliative therapy	Prolonged OS, improved symptomatic control, HRQoL	Not mentioned
Adjuvant therapy	Increased cure rate, OS, DFS d , safety d	CR
Neoadjuvant therapy	OS, PFS, DFS, enabling surgery, and organ preservation	Not mentioned
ICHOM	OS, PROMs, complications i , cause specific survival j , cause of death j , treatment‐related mortality j , place of death i , j , preference for place of death i , j , RFS i , j , PFS i , j , PCR i , j , CR i , j , margin status i , j , biochemical recurrence j , reoperation due to positive margins j , procedures for local progression j , symptomatic skeletal event j , development of metastasis j , development of castration‐resistant disease j , stoma status j , use of pain medicine j , time from diagnosis to treatment j , hospital admission at the end of life i , j	Not mentioned

Abbreviations: ADRs, adverse drug reactions; CR, complete response; DFS, disease‐free survival; EFS, event‐free survival; EMA, European Medicines Agency; EUnetHTA, European network for Health Technology Assessment; FDA, Food and Drug Administration; HRQoL, health‐related quality of life; HTA, Health Technology Assessment; IQWiG, the Institute for Quality and Efficiency in Health Care; NICE, the National Institute for Health and Care Excellence; ORR, objective response rate; OS, overall survival; PFS, progression‐free survival; PR, partial response; PROMs, patient‐reported outcome measures; QALY, quality‐adjusted life‐year; TTP, time to progression; ZIN, the Dutch National Health Care Institute.

^aWhen reduced or similar toxicity is expected.

^bWhen increased toxicity is expected.

^cWhen a major increase in toxicity is expected.

^dBy exception, this outcome may be used as primary outcome. This may be related to a specific patient population or treatment, for example, for patients with solid tumors, in small populations, in the adjuvant setting, or in late line therapy.

^eWhen improved cure rate is the objective.

^fWhen the majority of deaths is unrelated to cancer.

^gHRQoL may be used as a primary outcome when the questionnaire was developed with the objective to capture the specific impact of a given pathology.

^hThis outcome may by exception be used as primary outcome.

ⁱICHOM recommends to assess these for a specific group of patients, for example, patients with advanced disease or patients with curative intent.

^jICHOM recommends this outcome for a selection of indications: reoperation due to positive margins for breast cancer; time from diagnosis to treatment, treatment‐related mortality for lung cancer; stoma status, PFS, PCR, or CR, margin status, preference for place of death, hospital admission at the end of life for colorectal cancer; use of pain medicine, procedures for local progression, symptomatic skeletal event, development of metastasis, development of castration‐resistant disease for advanced prostate cancer; biochemical recurrence and development of metastasis for localized prostate cancer; cause‐specific survival for advanced and localized prostate cancer; RFS for breast cancer and colorectal cancer; place of death for colorectal cancer and lung cancer; cause of death for breast cancer, colorectal cancer, and lung cancer.

Collection of morbidity estimates and safety estimates (eg, adverse events, complications) was discussed by all HTA agencies, regulatory bodies, and in all ICHOM standard sets (Table 1 and Appendix S1).

Intermediate outcomes were accepted, sometimes under certain conditions, by all regulatory bodies, ICHOM standard sets, and HTA agencies (Table 1, Table 3, and Appendix S2). For example, the FDA specified “surrogate endpoints for accelerated approval must be reasonably likely to predict clinical benefit,” which suggests validity does not always have to be fully established. All HTA agencies mentioned the importance of including valid intermediate outcomes, meaning an established relationship between the intermediate (eg, progression‐free survival) and definitive outcome (eg, survival). However, the level of validity, which was acceptable, differs between HTA agencies (eg, IQWiG required a higher level of validity than ZIN, NICE, or EUnetHTA). IQWiG published a guideline regarding the use of intermediate outcomes in oncology, which highlighted the importance of assessing the validity of an intermediate outcome. More specifically, based on validation studies for colon and breast cancer regarding the use of intermediate outcomes for survival, IQWiG found the validity insufficient to allow any final conclusions based on these intermediate outcomes. ²⁷

Disease progression estimates were accepted by all HTA agencies, except for IQWiG, both regulatory bodies and in ICHOM standard sets (Table 1 and Appendix S1). In disease‐specific guidelines for lung, breast, and prostate cancer, progression estimates were acceptable for both the FDA and EMA, whereas ICHOM suggested the collection of progression estimates for breast cancer and colorectal cancer (Table 2 and Appendix S1).

PROMs were discussed in all guidelines and standard sets (Table 1 and Appendix S1). The reduction in symptoms was an acceptable outcome for EUnetHTA, IQWiG, FDA, and EMA. In the ICHOM standard sets, the reduction in symptoms was included in disease‐specific Health Related Quality of Life (HRQoL) questionnaires, such as questions regarding arm and breast symptoms in the case of breast cancer (data not shown). The term “health‐related quality of life” was mentioned in all the guidelines and standard sets. All HTA agencies recommended the use of a generic HRQoL instrument, and both NICE and ZIN specifically recommended the use of the EQ‐5D. In addition to a generic HRQoL instrument, EUnetHTA, ZIN, and IQWiG mentioned the acceptability of disease‐specific HRQoL instruments to complement generic instruments. The EUnetHTA guideline specified “Disease‐specific HRQoL instruments may be useful for more in‐depth assessment of the generic HRQoL dimensions affected by an intervention.” ²⁸ Both regulatory bodies indicated that the use of a validated or a generally accepted HRQoL instrument was important; additionally, the EMA specifically mentioned that a HRQoL questionnaire may be generic or disease‐specific. In all ICHOM standard sets, disease‐specific HRQoL instruments were recommended, such as the EORTC QLQ‐C30 and EORTC QLQ‐LC13 for lung cancer, and EORTC‐QLQ‐C30 and EORTC‐QLQ‐CR29 for colorectal cancer (data not shown).

The term “biomarker” was mentioned in the guidelines of EUnetHTA, NICE, IQWiG, FDA, and EMA (Table 1 and Appendix S1). NICE and IQWiG specifically indicated biomarkers may be used to support treatment decisions; therefore, biomarkers seem to be mainly used to identify specific patient groups to target for treatment. However, the FDA mentioned in their guidelines that biomarkers have not served as primary outcomes for cancer drug approval; however, “the FDA has accepted tumour markers as elements of a composite endpoint.” ²⁹

Estimates for tumor response (eg, partial complete response, objective response rate) were accepted by both regulatory bodies and ICHOM (Table 1 and Appendix S1). Tumor response was mentioned as acceptable outcome in the FDA and EMA guidelines for lung cancer and breast cancer (Table 2), while ICHOM recommended tumor response for colorectal cancer (Appendix S1). The acceptability of tumor response as outcome measure for prostate cancer was not mentioned by the EMA or FDA, nor for breast cancer, lung cancer, or (localized or advanced) prostate cancer by ICHOM (Table 2 and Appendix S1).

Finally, ICHOM suggested some outcomes, which were not mentioned by the regulatory bodies or HTA agencies, including place of death, stoma status, and reoperation due to positive margins (Table 3).

4. DISCUSSION

This study confirms that outcomes that matter to patients are mostly also relevant for market access and reimbursement. However, some differences remain, which is especially apparent regarding the acceptability of intermediate outcomes. These are recommended in ICHOM standard sets, but regulatory bodies are more likely to accept these than HTA agencies. ICHOM standard sets emphasize the importance of collecting all recommended outcomes, while regulatory and HTA guidelines only indicate that all relevant outcomes should be collected. When considering disease‐specific guidelines, both regulatory bodies and ICHOM standard sets recommend collection of OS. However, differences appear regarding the collection of other outcomes. For example, tumor response is accepted by both regulatory bodies for lung and breast cancer, while ICHOM only recommends this outcome for colorectal cancer.

We showed that OS is viewed by the EMA and FDA as the most persuasive evidence, which confirms previous findings. ³⁰ In HTA assessments regarding the relative effectiveness of oncological medicines, it has been shown that data on OS are most crucial for decision‐making on the value of these products. ⁸ However, OS data are not always mature when submitted for regulatory or HTA assessment. ⁸ , ³⁰ , ³¹ Therefore, intermediate outcomes, such as progression‐free survival, may be accepted by regulatory bodies ³⁰ , ³¹ and HTA agencies. ⁴ , ⁵ However, our study suggests that regulatory bodies are often less stringent regarding the acceptability of intermediate outcomes than HTA agencies, which is confirmed by previous studies. ³² , ³³ The FDA, for example, accepts intermediate outcomes that will reasonably likely predict clinical benefit for accelerated approval, whereas HTA agencies only accept validated intermediate outcomes. Additionally, between HTA agencies the required level of validity also varies, which was also demonstrated in the study of Kleijnen et al. ⁸

By comparing outcomes accepted by regulatory bodies and HTA agencies to ICHOM standard sets, we have added another dimension to this discussion. This study showed a difference in the use of generic and disease‐specific guidelines, where HTA agencies provide generic guidelines, regulatory bodies’ oncology‐specific guidelines, and ICHOM disease‐specific guidance. Although HTA agencies generally require generic outcomes to allow comparability between indications for their reimbursement decision‐making, additional disease‐specific outcomes could help to identify to which extent new oncological medicines will affect the quality of life of patients. ICHOM could assist HTA agencies in choosing outcomes most relevant to patients. Additionally, when both regulatory bodies and HTA agencies make use of ICHOM standard sets to define acceptable outcomes these may become better aligned.

To improve the timely access of new medicines that provide a real benefit to patients, and enhance the efficiency of regulatory and HTA processes alignment between these processes is becoming increasingly important. ⁶ , ⁷ , ¹¹ , ¹² Synergy may be created by sharing information, choosing similar outcomes, aligning the timing of procedures, parallel scientific advice, and collaboration around real‐world evidence generation. ⁶ , ⁸ , ⁹ , ¹⁰ , ¹¹ , ¹³ , ³² , ³⁴ Although regulatory and HTA processes have different purposes, which partly may explain their different perspectives on outcomes and subsequent conclusions, increasing alignment is important to support more equal access to medicines for European patients and may also be feasible as previous studies have outlined several options to increase alignment. ⁹ , ³³ , ³⁵ , ³⁶ , ³⁷ A possible further alignment of the regulatory and HTA processes needs further collaboration and additional discussion between all stakeholders involved. ¹⁰

HRQoL is a PROM where more alignment between regulatory and HTA assessments may be possible. ⁹ Regulatory bodies accept both disease‐specific and generic HRQoL questionnaires, while HTA agencies mostly rely on generic HRQoL questionnaires because it also needs to fulfill the requirements for their economic evaluations. On the other hand, ICHOM standard sets indicate the importance of using disease‐specific HRQoL questionnaires. Methods, such as mapping, may be used to extrapolate results from disease‐specific questionnaires to calculate generic quality of life, which could be used in HTA economic evaluations. However, HTA agencies are generally not prone to use this specific method, because of the possible biases involved. Nevertheless, other methods may be explored, which could be acceptable for HTA agencies to use.

This study has some limitations. First, we selected regulatory bodies and HTA agencies situated in Europe, except for the FDA, which is based in the United States. Therefore, we do not provide a global perspective on regulatory and HTA assessment guidelines. Second, we assessed regulatory and HTA assessment guidelines and not the actual assessment reports. We decided to first assess which outcomes would be preferred before looking into the difference between the ideal and actual situation. However, in practice it may not always be feasible to collect these outcomes. Therefore, regulatory bodies and HTA agencies may accept different outcomes than discussed in assessment guidelines.

A strength of this study is the inclusion of ICHOM standard sets to assess outcomes, which are believed to be relevant to patients. Some publications suggest that ICHOM standard sets use PROMS that are satisfying to patients. ¹⁴ However, the extent to which these standard sets are patient‐relevant may also be questioned. ³⁸ Some of the PROMS recommended by ICHOM standard sets seem to have been developed with limited patient involvement. For example, the HOOS‐Physical Function Short Form was included by the ICHOM standard set for Hip and Knee Osteoarthritis, while a study showed that some questions were unimportant to Dutch patients. ³⁸

To increase early access to medicines with an added value, a greater level of alignment is of importance to all stakeholders involved. Further collaboration and additional discussions are needed between these stakeholders to progress further possible alignment between regulatory bodies, HTA agencies, patients, and clinicians on the most relevant outcomes for decision‐making. However, we still need to realize that regulatory and HTA processes have different contexts and distinct purposes, where regulatory bodies determine whether a medicine is effective and has acceptable side effects, while HTA agencies assess the effectiveness of a medicine to what is used in clinical practice and whether its added value is reasonable compared with the additional costs. This may necessitate some differences in the outcomes used. Additionally, some outcomes are more likely to be accepted by regulators than HTA agencies; therefore, medicines that gain market access may not become available to patients due to a negative reimbursement decision. To ensure pharmaceutical companies are aware of the outcomes necessary for market access and reimbursement assessments conducting early parallel scientific advice with regulatory bodies and HTA agencies is relevant. To conclude, it is envisioned that in future concepts of VBHC in which market authorization, reimbursement decision‐making, and quality control of health care come more closely together, the use of outcomes will be much more aligned.

CONFLICT OF INTEREST

No conflict of interest.

AUTHOR CONTRIBUTIONS

RK, DD, MB and WG contributed to the study conception and design. RV and RK performed data collection. All authors reviewed the results and approved the final version of the manuscript.

Supporting information

Supplementary Material

DATA AVAILABILITY STATEMENT

Data were derived from public domain resources. The data that support the findings of this study are available from the corresponding author upon reasonable request.