Evolving Recommendations for Patient Populations Among Oncology Medicines: A Quantitative and Qualitative Analysis

Abstract

After a medicine has been tested in pivotal trials, regulators, health technology assessment (HTA) organizations, and professional societies make decisions about the patients best served by the medicine. This study assesses how the patient populations for oncology medicines (2010–2023) are defined (1) at trial, (2) regulatory submission, (3) upon approval for marketing authorization, (4) at submission, and (5) recommendation by the HTA, and (6) in clinical guidelines in Australia, Canada, the Netherlands, the United Kingdom, and the United States. Based on 25 populations for oncology medicines, we developed a framework for describing oncology populations consisting of 20 elements in four domains: disease specifications, patient characteristics, treatment position, and exclusion criteria. In exploratory analyses, we tabulated any observed variation in these framework elements throughout the six steps in the lifecycle of a medicine. On average, 10 (95% confidence interval [CI]: 9.2–10.9) potential adjustments were made, 2.3 (95% CI: 2.0–2.5) by each decision‐maker. The adjustments by pharmaceutical developers focused mostly on the disease specifications (0.5 of the average 0.8 adjustments, 63%), while adjustments by regulators, HTA organizations, and guideline developers predominantly targeted the treatment's position (range: 0.5/1.3 [36%] in guidelines to 0.6/1.0 [58%] in regulatory approvals). Each decision‐maker on average modifies 1.0 element (out of 2.3 [43%]) that was previously adjusted by another decision‐maker. The multiple differences observed in the description of patient populations reflect inconsistency in reporting between decision‐makers, complicating communication to patients and potentially affecting access to medicines. The developed framework can support consistent reporting across stakeholders and countries.

Study Highlights.

• WHAT IS THE CURRENT KNOWLEDGE ON THE TOPIC?

Due to the different stakeholder remits and the varying availability of data throughout a medicine's lifecycle, recommendations on eligible patient populations for oncology medicines vary across decision‐makers. Defining the right population, however, is of critical importance, as it is highly influential in the (cost‐) effectiveness that will be found.

• WHAT QUESTION DID THIS STUDY ADDRESS?

This study developed a population description framework with which was explored how recommended patient populations for novel oncology medicines evolve after clinical trial inclusion based on developer submissions and reviews by regulators, HTA organizations, and clinical guideline committees.

• WHAT DOES THIS STUDY ADD TO OUR KNOWLEDGE?

The defined patient population for an oncology medicine was, on average, adjusted twice at every step from trial inclusion to uptake in clinical guidelines. Ten adjustments were made for each medicine's decision sequence. Our results, furthermore, suggest that decision‐makers often accept the developers translation from strictly defined trial populations to a general description of eligible patient populations in regulatory and HTA submissions. However, the way the recommended populations are described is not consistent.

• HOW MIGHT THIS CHANGE CLINICAL PHARMACOLOGY OR TRANSLATIONAL SCIENCE?

Understanding how recommendations on the patient population shift across decision‐makers—from trial inclusion to market authorization, reimbursement, and clinical guideline development—can further inform evidence generation, reimbursement, and prescribing decisions, herewith improving equal access to medicines and communication to patients and their clinicians.

Precisely defining the correct population for a medicine is a critical part of patient care. Clinical trials often study highly demarcated patient populations, while marketing authorization is sought for more generalized populations. Health technology assessment (HTA) organizations may recommend medicines for reimbursement in (sub)populations showing a favorable relative effectiveness or cost‐effectiveness. Medical professional societies develop clinical guidelines to further specify the patient population to help guide clinical decision making and communication. The difference between the populations tested vs. treated in practice vastly contributes to the infamous efficacy‐effectiveness gap, which can result in disappointment in the achieved results in clinical practice. ¹ , ² , ³

In recent years, there have been numerous cases in which pharmaceutical developers, regulators, HTA organizations, and professional societies differed in the populations in which they recommended the use of a particular medicine. ⁴ , ⁵ In multiple sclerosis, the distinction between various subtypes of the disease has been subject to debate among decision‐makers, as these differed across trials and treatment practice. ⁶ , ⁷ , ⁸ For regulators and HTA organizations, the most often reported uncertainties during assessments related to what would constitute the most appropriate patient population. ⁹ Variations in recommendations across decision‐makers or countries may complicate decisions on the population covered in reimbursement, leading to variation in access to these medicines, or introducing uncertainty in individual treatment decisions for patients. ⁷ Studies in this area have focused primarily on regulatory‐HTA alignment or were limited to a small set of medicines for a single disease. ⁴ , ⁵ , ⁹ , ¹⁰ , ¹¹ , ¹²

To reduce disparate recommendations within reimbursement decision making, new European HTA regulation will obligate assessments to be performed jointly by European member states, starting with oncology medicines in 2025. ¹³ , ¹⁴ Thus, member states need to agree on certain elements in these joint assessments, including the population or populations for which these oncology medicines will be assessed. ⁵ , ¹⁰ At the same time, defining patient populations has become more challenging, particularly for oncology, since cancer medicines target increasingly small populations, have become more complex, and are used in diverse treatment pathways. ¹⁵ Hence, the EMA has recognized the need for a consistent reporting approach for patient populations and took an initial step by developing a guide for regulatory assessors. ¹⁶

In this study, we sought to explore how recommended patient populations are described for a subset of oncology medicines across clinical trials, indication labels, reimbursement recommendations, and clinical practice guidelines, and to propose a reporting framework to enhance consistency across all respective stakeholders.

MATERIALS AND METHODS

We conducted a systematic content analysis of clinical trial publications, regulatory and HTA assessment reports, and clinical practice guidelines to identify differences in the reporting of the population across the stakeholders.

Cohort selection

Australia, Canada, the Netherlands, the United Kingdom (UK), and the United States (US) were included for their large markets or leading regulatory, HTA, and professional organizations. Using the HTA information in the NAVLIN database (Eversana, Milwaukee, WI), all oncology medicines were selected that had been assessed for the same indication (e.g., non‐small‐cell lung cancer or diffuse large B‐cell lymphoma) by the HTA organizations in Australia, Canada, the Netherlands, and the UK between January 2010 and February 2023. ¹⁷ Medicines that had not been assessed by all four HTA institutions were not included, as we aimed for equal cohorts across countries to ensure the internal validity of the results. The medicines from this cohort that had been assessed by the Institute for Clinical and Economic Review (ICER), an independent HTA organization, were included for the US.

Document selection

First, the NAVLIN database was used to extract HTA reports from the Pharmaceutical Benefits Scheme (PBS) in Australia, the Canadian Agency for Drugs and Technologies in Health (CADTH), the Dutch National Health Care Institute (ZIN) in the Netherlands, the National Institute for Health and Care Excellence (NICE) in the UK, and ICER in the United States. ¹⁷

Next, the marketing authorisation reports that matched the diseases assessed in the HTA reports were manually extracted from the websites of the Therapeutic Goods Administration (TGA) in Australia, Health Canada (HC), the European Medicines Agency (EMA) for the Netherlands and the UK (none of the medicines were assessed by the Medicines and Healthcare products Regulatory Agency [MHRA]), and the Food and Drug Administration (FDA) in the United States. Reassessments by regulators and HTA organizations were excluded, and indication extensions were included as separate medicine‐indication combinations.

The pivotal trials described in the regulatory reports were searched in Pubmed and used to extract the patient populations defined in clinical trials. If two pivotal trials assessed the same indication, populations were combined and included.

National clinical practice guidelines that aimed to inform clinicians on individual treatment decisions were included. These were extracted from EviQ in Australia, ¹⁸ Cancer Care Ontario in Canada, ¹⁹ the guideline database of the Federation Medical Specialists for solid tumors, and the HOVON guidelines for hematological tumors in the Netherlands, ²⁰ , ²¹ the NICE guidelines database in the UK, ²² and the National Comprehensive Cancer Network in the United States. ²³ Screening for documents was performed in February–March 2023.

Data extraction

Data were systematically extracted from the documents by MH and gathered in Microsoft Excel (Microsoft, Redmond, WA). A subset of the data (one medicine in each country) was independently extracted and analyzed by TO for validation. Any discrepancies were resolved through discussion, and final interpretations were used for the dataset. For all documents, the extracted information included the country, the decision‐maker, the year and month of publication, the medicine's non‐proprietary name, the brand name, and the disease area.

For clinical trial publications, we established the patient population based on the inclusion criteria. For regulatory reports, we established the population based on the language in the proposed label at submission (proposed or submitted population in final assessment reports) and the label that was ultimately approved. For HTA reports, we established the population based on the proposed population in the submission (proposed, submitted, or regulatory‐approved population in final reports) and the population that was recommended for reimbursement. Since clinical guidelines contain broader disease recommendations than the specific indications required in this study, we established the population for clinical guidelines by extracting comments on populations that would reasonably fall within the same indication as was assessed in the trials, by regulators, and by HTA organizations. For example, if trials, regulatory, and HTA reports assessed a medicine as a second‐line treatment in patients with a certain mutation, we collected the information for patients with that mutation receiving a second‐line treatment, and not the information on patients receiving first‐line treatment or without the mutation. This was to minimize differences introduced by the time lags (e.g., indication extensions) between regulatory and HTA report publication and guideline publication. Data collection was limited to the final recommendation sections of regulatory and HTA reports and did not include information on eligible populations listed elsewhere in the documents (such as within a summary of medicinal product characteristics document [SmPC]).

The final file contained data on the defined patient population at six timepoints (when available): the pivotal trial, regulatory submission, regulatory approval, HTA submission, HTA recommendation, and clinical guideline (Figure 1 ).

Figure 1.

Six decision steps in the medicine's lifecycle that were assessed in this study.

Population framework (outcome measures)

A framework was developed, consisting of a set of elements used to describe an oncological population, such as the tumor type, disease severity, treatment aim, position of medicine or combination usage, which is in line with the guide developed by the EMA. ¹⁶ This framework was developed in a deductive way during the data collection, adding elements until no new elements were encountered in the population descriptions (data saturation). We used the framework to explore the differences and adjustments made to patient population descriptions in each of the six decision steps. The description of each included patient population was first restructured according to this framework and then we quantified the adjustments in the description by the decision‐makers, see ‘data analysis and reporting’.

Data analyses and reporting

Four exploratory analyses were performed to assess (1) how oncology populations are described, (2) which descriptive elements are most frequently adjusted, (3) the impact of these adjustments, and (4) any contradictions between the adjustments by the different decision‐makers.

First, to assess how patient populations are described by each decision‐maker, we determined the frequency of each element in the framework being reported by each decision‐maker. This was reported as the average percentage that describes, for example, how often a decision‐maker reports a specific element for a patient population.

Second, to assess which elements of the population description are most and least adjusted by decision‐makers, we determined the frequency of adjustments to each of the population elements. In each decision sequence, we listed per element if it had been adjusted as compared to the population defined in the step before. We distinguished between major and minor adjustments (see explanation under “missing data and definitions” and Table S1 ). When data on the population were missing in a certain step, we considered this as no adjustment. The adjustments were reported as an average percentage that describes for which share of the decision sequences a particular element of the patient population was adjusted at some point across the six decision steps.

Third, to assess whether the adjustments would be restrictive or expanding the popution, the frequency of expansions and restrictions was determined for the major adjustments to the population elements by each decision‐maker. As such, adjustments were labeled either as expansive or restrictive (see explanation under “missing data and definitions” and Table S1 ). This was reported as the mean number of adjustments, including confidence intervals, that were made to the patient population by each decision‐maker. It was reported separately for expansive (positive) and restrictive (negative) adjustments, hence making the population larger or smaller, respectively. As a validation, the analysis was performed for each country and for each medicine individually (Figure S2 ).

Fourth, to assess potentially misaligned adjustments by decision‐makers, we determined the frequency of adjustments that modify an adjustment made in a previous step, using the major adjustments from Analysis 3 (Table S2 ). This was reported as the mean number of modifications of previous adjustments to the patient population by an individual decision‐maker. Additionally, these modifications were presented as a share of the mean number of all the adjustments by that decision‐maker (Analysis 3).

Missing data and definitions

Throughout the analyses, we assumed that the adjustments were made intentionally. For example, if an element was listed in Step 1 but not in Step 2, this was considered a deliberate adjustment. To be more specific, if the trial inclusion criteria restricted patient inclusion to 18 years or older and the submitted regulatory population did not specify an age or did not mention “adult” patients, this was considered as letting go of the age restriction.

The categorization of adjustments as major or minor was prediscussed with ASK, applied by MH, and validated by a second author, TO (Table S1 ). Major adjustments were the changes that could potentially impact the number or type of patients that are eligible for the treatment, such as age restrictions or prior treatment requirements. Minor adjustments were considered unlikely to have an impact on the eligible number of patients, such as differences in terminology or alternative diagnostic tests. For Analyses 3 and 4, minor adjustments were excluded, and for Analysis 2, minor adjustments were included.

The labeling of specific adjustments as expanding or restricting the population was discussed and agreed upon by two authors (MH and ASK) and validated by another author, TO (Table S1 ). Expansive adjustments were defined as the changes that potentially increase the number of patients that are eligible to receive a medicine. This, for example, included adjustments that moved the medicine to an earlier treatment line or broadened the prior treatment criteria. Restrictive adjustments were defined as changes that potentially decrease the number of patients that are eligible to receive a medicine. This, for example, included adjustments that moved the medicine to a later treatment line or restricted the prior treatment criteria. Changes that likely have no impact on the population size or for which it is not feasible to determine the direction of the population size change were not included in this analysis.

RESULTS

Out of 453 cancer drug‐indication combinations, 25 had an HTA assessment report in Australia, Canada, the Netherlands, and the UK (Table 1 ). The US data were also available for six of the 25 combinations, leading to a total of 106 decision sequences for comparison. The 25 medicine‐indication combinations covered 24 medicines for 11 different diseases (venetoclax was included twice, for acute myeloid leukemia and chronic lymphocytic leukemia). All medicines received marketing authorisation between 2012 and 2021. We identified 30 associated pivotal trials (Table S3 ).

Table 1.

Medicines and documents included (green) and missing (white) to determine reported population definitions

Brand name	Generic name	Disease	First global approval	No. trials included	Regulatory submission					Regulatory approval					HTA submission					HTA approval					Guideline uptake
					AU	CA	NL	UK	US	AU	CA	NL	UK	US	AU	CA	NL	UK	US	AU	CA	NL	UK	US	AU	CA	NL	UK	US
1 Verzenio	Abemaciclib	Breast cancer	2018	2
2 Zytiga	Abiraterone	Prostate cancer	2012	2
3 Calquence	Acalabrutinib	Chronic lymphocytic leukemia	2020	1
4 Tecentriq	Atezolizumab	Non‐small‐cell lung cancer	2017	1
5 Cabometyx	Cabozantinib	Renal cell carcinoma	2016	1
6 Libtayo	Cemiplimab	Non‐small‐cell lung cancer	2021	1
7 Tafinlar	Dabrafenib	Melanoma	2013	1
8 Darzalex	Daratumumab	Multiple myeloma	2016	1
9 Imfinzi	Durvalumab	Non‐small‐cell lung cancer	2018	1
10 Rozlytrek	Entrectinib	Non‐small‐cell lung cancer	2020	1
11 Halaven	Eribulin	Breast cancer	2011	1
12 Imbruvica	Ibrutinib	Chronic lymphocytic leukemia	2014	1
13 Zejula	Niraparib	Ovarian cancer	2017	1
14 Opdivo	Nivolumab	Non‐small‐cell lung cancer	2015	1
15 Lynparza	Olaparib	Ovarian cancer	2014	2
16 Tagrisso	Osimertinib	Non‐small‐cell lung cancer	2016	1
17 Ibrance	Palbociclib	Breast cancer	2016	2
18 Keytruda	Pembrolizumab	Non‐small‐cell lung cancer	2016	1
19 Perjeta	Pertuzumab	Breast cancer	2013	1
20 Polivy	Polatuzumab vedotin	Diffuse large B‐cell lymphoma	2020	1
21 Kisqali	Ribociclib	Breast cancer	2017	1
22 Kadcyla	Trastuzumab emtansine	Breast cancer	2013	1
23 Venclexta	Venetoclax	Acute myeloid leukemia	2021	2
24 Venclexta	Venetoclax	Chronic lymphocytic leukemia	2016	1
25 Brukinsa	Zanubrutinib	Waldenstrom's macroglobulinemia	2021	1

In green are the patient populations that were available for each medicine in each decision step and in each country. AU, Australia; CA, Canada; NL, the Netherlands; UK, United Kingdom; US, United States. Information was not available in every document. For example, the submitted HTA indications were not available in ICER reports (n = 6), and in one case not for NICE due to a terminated assessment, which also excludes the final HTA population for that case. The submitted regulatory indications were not available in Health Canada reports for the older medicines (n = 16). PBS from Australia had the most negative HTA recommendations (n = 11) being excluded from the analysis.

All 106 decision sequences had patient populations specified in trials; 88 had data available on the populations submitted to regulators (63 without EMA duplicates in the NL and UK); 105 had data available on the populations defined at the time of regulatory approval (80 without EMA duplicates); 99 had data available on the populations submitted for HTA; 90 had data available on the recommended HTA populations; and 92 had data available on the populations specified in clinical guidelines. This resulted in 530 included patient populations that allowed for 436 comparisons to populations defined in previous decision steps.

Population framework and descriptions

The population framework included 20 elements in four domains (Figure 2 ): disease specifics (8 elements), patient characteristics (4), positioning in a treatment pathway (6), and exclusion criteria (2). First, the framework includes the targeted disease specifics, which cover the tumor location, tumor histology, protein expression or hormone sensitivity, genetic alterations, criteria for diagnostic testing, the severity of disease (including spread or size and/or the staging), and the occurrence of the tumor (relapsed, refractory, recurrent disease or an irresectable or operable tumor). Second, the framework includes patient characteristics that cover the patient's age, sex, fitness or performance score, and other subgroups, such as the pre‐ or post‐menopausal status for women. Third, the framework includes the treatment positioning, which covers prior treatments that are required, the decision context of these prior treatments (progression on or failure of a previous treatment, or just ‘having received’), the treatment line (e.g., number of prior treatments), treatment goal or intent, a required combination of treatment, and the duration requirements for prior treatments or wash‐out periods. Lastly, the framework allows for the description of explicitly excluded patients. Table S4 provides examples for each framework element.

Figure 2.

Patient population elements reporting among trials, regulatory and HTA submissions, and reports, and clinical guidelines. Using the developed framework, this chart shows the reporting frequency (x‐axis) for each population element (y‐axis). Blue (top cluster) shows the elements related to the disease specifics, in red (second cluster) the patient characteristics, in yellow (third cluster) the positioning of the medicine in a treatment pathway, and in purple (bottom cluster) the exclusion and other criteria. The x‐axis should be interpreted as the reporting frequency of population elements across the 25 medication‐indication combinations and 5 countries, e.g., the reporting frequency of genetic predispositions (e.g., NTRK gene fusions) was 32% in the trials. In other words, genetic information was provided in trial inclusion criteria for 8 out of the 25 medicine‐indication combinations (8/25 = 0.32). The reporting frequency for age was 67% in the regulatory approvals (e.g., “for adult patients with …”). In other words, an age specification was provided for 54 out of 80 regulatory‐approved patient populations, excluding 25 EMA duplicates.

Trials had the most elaborate descriptions of eligible populations, consisting of 10.6 (out of 20) population framework elements on average. Patient populations were described in a similar fashion in the regulatory and HTA steps (as compared to trial and clinical guideline descriptions), using 6.8 elements to describe a population for regulatory submission, 8.2 in regulatory approval, 7.4 for HTA submission, and 8.6 in HTA recommendations. On average, 1.4 and 1.2 elements were added to the population descriptions between developer‐led submissions and the regulatory and HTA approvals, respectively. Guidelines used 9.0 elements on average.

The disease element was reported in all 530 population descriptions (Figure 2 ). Trials had the highest reporting frequency for 10 of the 20 elements. Notably, regulators never reported on the fitness of patients (0/80, in 0% of the populations) such as the Eastern Cooperative Oncology Group (ECOG) performance score, “fit patients only,” or “high‐risk patients.” Time‐related elements such as “treat for a maximum of 24 months” (2/80, 2%) and tumor staging (4/80, 6%) were also rarely reported by regulators. HTA approvals and clinical guidelines more commonly reported fitness elements (22/90, 24% in HTA and 27/92, 29% in guidelines), as well as time elements (13/90, 14% in HTA and 9/92, 10% in guidelines). Finally, guidelines regularly included treatment goals (41/92, 44%) such as the intention of palliative or adjuvant treatment.

Adjustments in the population elements

Across 106 decision sequences, the fitness or risk element was adjusted most frequently (92/106, 87% of medicine‐indication combinations) (Figure 3 ). This included, for example, when listing the ECOG performance status or simply referring to “fit patients.” Additionally, many adjustments occurred in the prior treatment criteria: which prior treatment (73/106, 69%), the context such as required progression or response (62/106, 58%), and how many prior treatments would be needed (48/106, 45%). The descriptions for tumor occurrence (68/106, 64%) and spread or size (53/106, 50%) were also adjusted in a majority of the decision sequences. The tumor spread and size covered the inclusion or exclusion of (locally) advanced or metastatic disease. The occurrence covered adjustments in descriptions of relapsed, refractory, or recurrent disease.

Figure 3.

Proportion of drug‐indication combinations adjusted at least once throughout the six steps of the decision sequence, per patient population element. The x‐axis shows the proportion of medicines per indication for which an element was changed; the y‐axis shows the population elements. The percentages next to the y‐axis indicate the proportion of major changes, while the percentages to the right of the bar indicate the proportion of total changes. Blue (top cluster) shows the elements related to the disease specifics; in red (second cluster) the patient characteristics; in yellow (third cluster) the positioning of the medicine in a treatment pathway; and in purple (bottom cluster) the exclusion and other criteria.

The treatment goal (2/106, 2%) and sex (9/106, 8%) were not often reported; hence, they were seldom adjusted. Sex was only adjusted for some breast cancer indications through the explicit inclusion or exclusion of men. Out of 1,394 total adjustments in these 106 medicine‐indication combinations, 68% (n = 952) were classified as major vs. 32% (n = 442) as minor.

Restrictive and expansive major adjustments per step

For each medicine assessed, an average of 2.3 (95% confidence interval [CI]: 2.0–2.5) major population adjustments were made at every step. Throughout the full decision sequence, 10.0 (95% CI: 9.2–10.9) major adjustments as compared to the previous step were made to each population (i.e., in any of the elements, the same elements may be adjusted multiple times at different steps). In total, 88% (834/952 major adjustments) of the adjustments could be classified as restrictive or expansive.

Most adjustments were made by pharmaceutical developers (5.0; 95% CI: 4.6–5.4) in the generalization of the population for regulatory submission compared to the strictly defined trial population (4.3 expansive and 0.8 restrictive) (Figure 4 ). The fewest adjustments were made by developers (1.4; 95% CI: 1.7–1.1) in their submissions to HTA organizations, compared to the regulatory‐approved population (0.6 expansive [95% CI: 0.5–0.8] and 0.7 restrictive [95% CI: 0.5–0.9]).

Figure 4.

Major adjustments in the restrictive (negative) or expansive (positive) directions per decision maker, compared to the previous decision. The y‐axis is the mean number of adjustments made to each medicine as shown per step, that is, regulators apply 1.0 restrictive adjustments to the population compared to the submitted indication. In other words, for every medicine being assessed for a specific indication, one element is adjusted in a restrictive way by each regulator during regulatory review. The positive values represent expansive adjustments whereas the negative values represent restrictive adjustments. The difference between the negative and the positive value represents the total mean number of adjustments made in that decision step. The numerator and denominator are the number of adjustments divided by the total number of documents included per step (medicine‐indication combination * number of countries). HTA, health technology assessment; MA, marketing authorisation.

Although most of the population‐expanding adjustments (n = 4.3) from the demarcated trial population to the generalized label in regulatory submission were accepted, regulators and HTA organizations also imposed some restrictive adjustments. Compared to the submissions, on average 1.0 (95% CI: 0.8–1.3) restrictive adjustment per assessed medicine was made by regulators (vs. 0.4 expansive, 95% CI: 0.2–0.6) and 1.3 (95% CI: 1.0–1.6) by HTA organizations (vs. 0.3 expansive, 95% CI: 0.2–0.4). A comparison between regulatory and HTA major adjustments can be found in Figure S1 . For clinical guidelines, we observed a similar amount of expansive (1.4, 95% CI: 1.1–1.8) vs. restrictive adjustments (1.3, 95% CI: 1.0–1.6).

The restrictions applied by pharmaceutical developers in the regulatory submission tended to focus on disease‐specific elements (0.5 out of 0.8 adjustments, 63%), while their expansions focused on patient characteristics (1.8/4.3, 42%). Both the expansions and restrictions in later steps tended to focus on treatment positioning (ranging from 0.5/1.3 [36%] restrictions in guidelines to 0.6/1.0 [58%] restrictions in regulatory approvals). HTA organizations applied the most restrictions related to patient characteristics (0.4/1.3, 28%). Validating sensitivity analyses on expansive and restrictive adjustments for individual countries and medicine‐indication combinations can be found in Figure S2A–O .

Modifications to previous adjustments

The major adjustments to the population were regularly modified by stakeholders in later steps of the decision sequence. On average, 1.0 (95% CI: 0.9–1.1, 43% of total adjustments per step [1.0/2.3]) adjustments made in each step modified an adjustment that was made in one of the previous steps. An example of such modification included a trial that considered patients eligible “after two courses of chemotherapy.” The population defined by the developer in the regulatory submission indicated the need for response to “prior chemotherapy.” This was reverted to trial criteria by the regulator into “at least two courses.” In line with the direction of change by the regulator, this was further restricted by HTA organizations to “at least four prior courses.” As a second example, docetaxel was required as previous therapy in a trial, which was adjusted to ‘a previous taxane’ in the regulatory submission. This was reverted back to docetaxel by the regulator in the authorized indication. A third example was visible for a “3rd‐line treatment” in the clinical trial that was expanded by the regulator to “2nd line” in the approved indication, but was in contrast to the regulatory adjustment restricted by the HTA organization to a “4th line” in the indication approved for reimbursement.

The major adjustments applied by developers in the regulatory submission were most frequently modified in subsequent steps, ranging from 0.6 modifications out of 1.5 total adjustments (37%, 71 total modifications) by regulators to 1.1/1.6 (78%, 106) modifications by developers themselves in HTA submissions (Figure 5 ). Adjustments made by regulators were least frequently modified in subsequent steps, ranging from 0.4/1.4 (27%, 37 modifications) modifications by developers to 0.5/1.6 (30%, 41) modifications in HTA and 1.5/2.7 (56%, 116) modifications in guidelines (see “vs. regulator” in Figure 5 ).

Figure 5.

Modifications to the major adjustments by previous decision‐makers. This graph shows the average number of modifications in each decision step to a previous adjustment in an earlier step. The x‐axis can be interpreted as the number of elements that are modified for each medicine (on average) in each decision step, that is, during HTA review, 0.42 elements (1 element in 42% of the medicines or 2 in 21% of the medicines) were modified, which had previously been adjusted by the developer in the HTA submission. This can be either a modification that opposes a previous adjustment, reverts a previous adjustment to what it was before (for example to trial inclusion criteria), or follows up on a previous adjustment in the same direction (expansive or restrictive) but even more extreme. The percentage shows the proportion of such modifications of the total number of adjustments made in that step (1.6 in the case of HTA, 0.42/1.6, [26%]). The numerator and denominator are the number of modifications of previous adjustments divided by the total number of adjustments made (Figure 4).

Regulators and HTA organizations never made an expanding adjustment to a population element that was restricted in an earlier step by developers or regulators. However, previously expanded elements were sometimes restricted in later steps, mostly by HTA organizations (14 elements across 84 HTA assessments) (Sankey diagram in Figure S3 ).

DISCUSSION

Based on 25 populations for oncology medicines, we were able to develop a population framework that describes eligible patients for a medicine in detail, allowing for consistency between decision‐makers and countries. The framework consists of 20 reporting elements in four domains: disease specifics, patient characteristics, positioning in a treatment pathway, and exclusion criteria. Our framework aligns with the approach taken in the framework for regulatory assessors published by the EMA. ¹⁶ Both frameworks have overlapping elements, such as the severity of disease, aim of treatment, place of product, and use in combination. Our new framework is more specific on each of these elements, including diagnostic or genetic criteria, separating tumor staging from a general description of the tumor “advancedness” and detailing prior treatment requirements. This allows for communication of the nuances necessary for decision making.

The patient population descriptions for the included oncology cohort were on average adjusted twice at every step from trial inclusion, to the submission and approval for marketing authorization, to the submission and approval for HTA, and finally to inclusion in clinical guidelines. One of the two adjustments in each step represents an element that had been adjusted by another decision‐maker in an earlier step. Discrepancies in reporting the patient populations as well as the variety in the remits of decision‐makers play a critical role in these findings. ²⁴ , ²⁵ , ²⁶ However, the many and varying adjustments in later decision steps decision‐makers seem to not agree on when and to what extent this is appropriate. This is supported by the provided examples in our last presented exploratory analysis.

On average, 10 adjustments were made throughout a single decision sequence for one medicine. Five out of 10 adjustments are made in the initial step, translating the demarcated clinical trial population into a regulatory label. Regulators, HTA organizations, and guideline developers imposed fewer adjustments in each step (the other 5 adjustments distributed over four decision steps). These results suggest that regulators, HTA organizations, and clinical guideline developers accept most of the developers adjustments from trial to a general description of eligible patient populations.

Developers made the most population‐expanding adjustments, in particular from their pivotal trials to the regulatory submission and to a lesser extent in HTA submissions. This may be for a large part due to the difference in reporting in pivotal trials vs. regulatory labels. Additionally, some inclusion or exclusion criteria from the trial may have been reported elsewhere in regulatory documents rather than in the indication statement, although such less‐prominently featured information is less likely to be understood by prescribers and communicated well to patients. Regulators accepted the majority of these developer‐initiated adjustments, although not all expansions were accepted. Hence, most population restrictions were implemented by regulators and HTA assessors. Such restrictions often targeted the treatment positioning elements. In particular, the restrictions by HTA organizations focused on patient characteristics and treatment positioning, reflecting the different interpretation of clinical trial results in the context of existing treatment options and national healthcare contexts. ⁵ , ¹⁰ , ²⁷ The restrictions applied by developers, on the other hand, tended to focus on disease‐specific elements and are more likely to be informed by the context of clinical trial results. Clinical guideline adjustments did not show a clear direction or dominant elements that were altered. The disease‐broad scope (e.g., all of NCSLC), different structure (based on the patient rather than the treatment) and often long timeframes between reimbursement recommendations and guideline publication (e.g., possibly new data was available) make it difficult to properly interpret the number and nature of adjustments in the last decision step. This may partly explain the relatively high number of adjustments (two per medicine decision sequence) in guideline inclusion.

Although no other studies have compared patient population descriptions in each step of the regulatory sequence, some publications suggested that agreement on the patient population was reached for 77% of the medicines in parallel regulatory and HTA scientific advice. This suggests that some of the adjustments—if not caused by describing differences—we observed may be prevented by early stakeholder dialogue. ² The large number of adjustments we found indicates a diverse interpretation of the clinical evidence on patient populations in different contexts. In the final analysis, we have observed that the adjustments made by developers, as compared to the adjustments in the other steps, were more frequently modified in subsequent decision steps. Regulatory adjustments were less frequently modified in other population recommendations. A difference in remits between institutions (e.g., strong regulatory mandate) and disagreement on evidence interpretation (e.g., between developers and other stakeholders) may, to some extent, explain these findings. ²⁸ Furthermore, the frequency of adjustments across all 20 framework elements in the clinical guideline uptake suggests that indications as formulated by developers in trials, and then by regulators and HTA assessors, may not fully align with the needs in clinical practice. ⁴ , ²⁹

The classification of major and minor adjustments, as well as expansive and restrictive adjustments, was subject to author interpretation. For example, not reporting age restrictions (while the previous step did) theoretically implies all ages are eligible and were therefore considered a “major” adjustment. If such implicit adjustments are considered ‘minor,’ the current results may provide an overestimation of the major adjustments. Similarly, moving a medicine to a later treatment line was considered a restrictive adjustment because one can assume that in every treatment line a few patients decide not to continue treatment. The actual impact on the patient population is, however, unclear. The impact would be most prominent for adjustments between trial and regulatory submission because trial inclusion criteria may specify many elements that are not necessarily included in the label but may still be part of the regulatory decision (e.g., exclusion criteria in the trial might be listed as a contra‐indication). This method using author classification is frequently employed and accepted in this type of research as it is often—despite its limitations—the only method suitable available that allows for the quantification of public text documents. ⁹

The developed framework can improve consistent patient reporting and support discussing and communicating the rationale behind patient populations eligibility decisions. This seems critical as previous research indicated that uncertainty on the recommended population is reported in 60–95% of the medicines by both regulators and HTA organizations. ⁹ The population framework presented in this study can support the discussions on the populations for oncology products going through the joint EU HTA assessments. For a successful joint assessment, it is critical to find a clearly defined patient population among all EU member states. To further strengthen discussions and decisions on patient populations, research could focus on gaining insight into the motivation and the rationales behind the adjustments to populations by each decision‐maker. Additionally, it is useful to consider the impact of the adjustments on the actual population size as well.

Limitations

Although our data illustrate that changes to defined patient populations are very common, our results do not indicate whether expansions or restrictions are justified or appropriate. Other studies have focused in great detail on product‐specific decision sequences throughout the six steps, highlighting the justification and impact of eacht decision. ³⁰ However, this is only feasible for a very limited number of medicines. Second, our results do not reflect the actual impact on the size of the described/recommended population. Third, we averaged over institutions and jurisdictions, meaning that our results may apply to a greater or lesser extent to individual jurisdictions. Our stratified analyses in Figure S2A–O detail the results by jurisdiction and medicine‐indication combination, suggesting no major differences between countries except for the submissions to HTA that are not practiced in the US. The results may not all be transferable to other jurisdictions, as different institutions employ different assessment frameworks, and collaborations exist between the included institutions. ³¹ , ³² One of our earlier studies looked into regulatory, HTA, and guideline recommendations for multiple sclerosis medicines in European countries and found similar discrepancies in treatment line and sub‐indication recommendations. ¹ Fourth, we should stress that some element adjustments are interdependent; for example, a menopausal status change may affect the recommended endocrine therapy combination. Lastly, our final selection of medicines included 25 oncology medicines. Although we see no apparent reasons why the results would differ for other oncology medicines, we cannot be certain that the results apply to all oncology medicines.

CONCLUSION

The multiple adjustments to recommended patient populations for oncology medicines reflect inconsistency in reporting populations between decision‐makers and indicate that their diverse remits can result in different recommended populations. A consistent reporting framework across stakeholders and countries can further benefit the discussion on evidence generation, inform decisions on reimbursement and prescribing, and improve patient communication and potentially equal access to medicines.

AUTHOR CONTRIBUTIONS

M.A.H. wrote the manuscript. M.A.H., R.A.V., W.G.G., A.K.M.‐T., and A.S.K. designed the research. M.A.H. and T.A.O. All authors approved the final version of the manuscript.

FUNDING

This work was performed under the umbrella of the HTx project. The project has received funding from the European Union's Horizon 2020 research and innovation program under grant agreement No 825162. This dissemination reflects only the author's view and the Commission is not responsible for any use that may be made of the information it contains. Furthermore, a grant from the Catharine van Tussenbroek fund was received for the conduct of this study.

CONFLICT OF INTEREST

This study was conducted and finalized while MH was a PhD candidate at Utrecht University. After her PhD, she started working for Roche Nederland B.V. RV joined Roche Nederland B.V. after initiation of this study. All other authors declared no competing interests for this work.

Supporting information

Data S1