Reimbursement recommendations for cancer drugs supported by phase II evidence in Canada

Abstract

Background

Phase ii data are increasingly being used as primary evidence for public reimbursement for oncologic drugs. We compared the frequency of reimbursement recommendations for phase ii and phase iii submissions and assessed for variables associated with a positive or conditional recommendation.

Methods

We identified submissions made to the pan-Canadian Oncology Drug Review’s Expert Review Committee (perc), of the Canadian Agency for Drugs and Technologies in Health, July 2011 to July 2019, that were supported only by phase ii data. We identified variables within the perc’s deliberative framework, including clinical and economic factors, associated with the final reimbursement recommendation. We conducted a multivariable analysis with logistic regression for these variables: feasibility of phase iii study, hematologic indication, and unmet need.

Results

We identified 139 submissions with a perc final recommendation. In 27 instances (19%), the submission had only phase ii evidence, and a positive recommendation was issued for 63% of them (the positive recommendation rate was 82% for submissions with phase iii evidence). Clinical benefit (p < 0.001), unmet need (p = 0.047), and patient alignment (p = 0.015) were associated with a positive recommendation. If a future phase iii study was deemed feasible for submissions with only phase ii evidence, then in univariable (p = 0.040) and multivariable analysis (p = 0.024), the perc was less likely to recommend reimbursement (odds ratio: 0.132).

Conclusions

Although more than half the oncologic submissions with phase ii data were recommended for public reimbursement, compared with submissions having phase iii data, they were less likely to be recommended. A positive or conditional recommendation was more likely if clinical benefit and alignment with patient values was demonstrated. The perc was less likely to recommend reimbursement for submissions with phase ii evidence if a phase iii trial was deemed possible.

INTRODUCTION

Clinical trials have emerged as the ideal method in the evaluation of new medical interventions. Phase ii trials are typically used to determine an intervention’s biologic effect and response rate in the target patient population, helping researchers to determine whether clinical merit is sufficient to proceed to a phase iii trial. Phase iii randomized trials compare an intervention with the current standard of care with respect to long-term outcomes, including survival¹. Classically, phase iii trials were designed to facilitate market authorization for a drug and, in some jurisdictions, public reimbursement; however, phase ii trials are increasingly positioned to meet those purposes, for reasons that are multifactorial. Considerations include the avoidance by pharmaceutical companies of a more costly and time-consuming phase iii trial by demonstrating sufficient effect in phase ii and a desire for expedited drug reimbursement by consumers or patients.

In Canada, the Canadian Agency for Drugs and Technologies in Health [cadth (https://www.cadth.ca/)], an independent not-for-profit organization, is responsible for providing health care decision-makers with objective evidence to help them make informed decisions about the optimal use of health technologies, including providing evidence-based recommendations to participating federal, provincial, and territorial governments for public reimbursement of novel therapies. Specifically, those tasks are allocated to the Common Drug Review for non-oncologic drugs and to the pan-Canadian Oncology Drug Review (pcodr) for oncologic drugs. Established in 2010, pcodr reviews all oncologic drugs based on 4 dimensions: clinical benefit, economic evaluation, adoption feasibility, and alignment with patient-based values. Those dimensions are collectively reviewed by a committee of experts, the pcodr Expert Review Committee [perc (https://www.cadth.ca/collaboration-and-outreach/advisory-bodies/pcodr-expert-review-committee-perc)], and a decision is made to recommend reimbursement or reimbursement conditional on other factors being mitigated, or not to recommend reimbursement. In particular, clinic benefit and an economic evaluation are assessed by further subcommittees independent of perc: namely, the Clinical Guidance Panel (cgp) and the Economic Guidance Panel respectively. Those two groups give their recommendations in separate reports to the perc.

The landscape of cancer care has changed rapidly since about 2010, especially with the emergence of immunologic therapies. An increasing number of biologics have been approved for specific conditions, and patient groups, clinicians, and pharmaceutical companies all have an interest in whether those agents are approved for public reimbursement. There is interest among stakeholders for more transparency in the decision-making process. In the past, patient groups have reported on perc recommendations for their subset of malignancies^2,3. An independent study also considered whether certain factors appeared to carry more weight in perc recommendations⁴. Across all submissions analyzed (n = 91), aspects of clinical and patient value carried the greatest weight in the final decision to either recommend or reject reimbursement. Economic impact did not play a role in leading to a rejection, but it was the key factor in leading to recommendations conditional upon improving cost-effectiveness. The study did not, however, evaluate the differential results associated with submissions based on phase ii compared with phase iii trials.

Interest has also been evident among stakeholders and within the perc to further analyze how reimbursement recommendations are made for therapies submitted with nonrandomized data rather than with phase iii data. That interest is in part driven by a growing perception that drugs with evidence below the level of a randomized controlled trial are categorically not recommended for reimbursement. We therefore set out to compare the frequency of reimbursement recommendations in submissions with phase ii and phase iii data. We also aimed to determine which factors, if any, play a key role in positive recommendations for reimbursement for drugs with nonrandomized data.

METHODS

We extracted data from publicly available recommendation reports made by the perc between July 2011 and July 2019. We identified submissions based on phase ii trials (defined as nonrandomized data), and we extracted disease, treatment, and submission characteristics found within the final public recommendation report from the perc. The unit of analysis was the number of submissions received rather than the drug count, because a single drug might be submitted multiple times for various indications. Resubmissions were treated as individual submissions, separate from the previous ones, because they would have been re-evaluated based on new data (for example, a new clinical trial, a new price from the manufacturer). Each submission was assessed by two reviewers independently; conflicts were resolved by discussion and a third review if consensus could not be reached.

Variables

We identified 14 variables within the recommendation summary statements; those variables were categorized and coded according to recommendations made by the perc in the 4 quadrants of their deliberative framework: clinical benefit, patient-based values, economic evaluation, and adoption feasibility. Notably, because phase ii evidence is generally less robust than a phase iii study, the perc also, for all submissions supported by phase ii evidence only, makes a conclusion about whether, in their view, a phase iii trial is feasible or possible to conduct. Because that factor was unique to our subset of submissions, we included the perc conclusion concerning the feasibility of a phase iii trial as a variable. Variables are listed and fully explained in Table I.

TABLE I.

Strength of association of a positive or conditional recommendation for reimbursement with selected variables

Variable	Description	Frequency [n (%)]	Association
			Approved (n)	Rejected (n)	Fisher p value
Clinical benefit
Hematologic drug	Submission indication is to treat hematologic malignancy
	• Not a hematologic indication	11 (41)	5	6	0.224
	• Hematologic indication	16 (59)	12	4
Phase III	Identifies whether a phase III study is feasible, according to the Clinical Guidance Panel
	• Not possible	16 (59)	13	3	0.040
	• Possible	11 (41)	4	7
Overall clinical benefit	Conclusion of pERC with respect to overall clinical benefit
	• No benefit	10 (37)	0	10	<0.001
	• Uncertain or net benefit	17 (63)	17	0
Clinical Guidance Panel clinical benefit	Conclusion of the Clinical Guidance Panel with respect to overall clinical benefit
	• Possible benefit	9 (33)	2	7	0.004
	• Net benefit	18 (67)	15	3
Toxicity	Manageability of adverse events
	• Not manageable	4 (15)	1	3	0.128
	• Manageable	23 (85)	16	7
Quality of life	Treatment impact on quality of life
	• Inconclusive	17 (63)	11	6	1.000
	• Not worse or better	10 (37)	6	4
Patient-based values
Unmet need	Identifies a gap in current standard of care for indicated malignancy
	• No unmet need	5 (19)	1	4	0.047
	• Unmet need	22 (81)	16	6
No other treatment	Availability of acceptable alternatives
	• Other treatments available	21 (78)	12	9	0.363
	• No other treatments	6 (22)	5	1
Patient alignment	Alignment of drug attributes with patient-centred values identified by patient advocacy groups
	• Does not align	6 (22)	1	5	0.015
	• Aligns	21 (78)	16	5
Economic evaluation
ICER	The size of incremental cost effectiveness ratio in relation to a threshold valuea
	• <$140,000	8 (30)	7	1	0.190
	• ≥$140,000	18 (67)	10	8
Cost effective	Conclusion of the Economic Guidance Panel on treatment cost efficacy in relation to current standard therapy
	• Not cost-effective	25 (93)	15	10	0.516
	• Cost-effective	2 (7)	2	0
Adoption feasibility
Additional costs	Fees associated with infrastructure or testing
	• No fees	6 (22)	3	3	0.638
	• Additional fees	21 (78)	14	7
Administration	Type of drug
	• Oral	13 (48)	6	7	0.120
	• Non-oral	14 (52)	11	3
Budget impact	Impact estimated on the basis of patient population size and available alternatives
	• Small	2 (7)	2	0	0.516
	• Uncertain or underestimated	25 (93)	15	10

^aThe deemed implicit maximum acceptable cost-effectiveness threshold across all submissions in previous analysis⁴.

Statistical Analysis

We use descriptive statistics to assess differences in the approval rates and drug indications for all submissions with phase ii or phase iii data. We extracted variable data from all submissions and used the Fisher exact test to characterize associations between individual binary variables and the final recommendation. The outcome of interest was defined as recommendation for reimbursement compared with rejection: “yes” if the result from the deliberation was a full or conditional recommendation; “no” if the decision was to not recommend. A conditional recommendation is given when a drug has demonstrated clinical benefit, but approval is contingent on improving cost-effectiveness. Although cadth does not have an explicit threshold, we established the incremental cost effectiveness ratio (icer) as a binary variable—less than $140,000 or $140,000 and greater per quality-adjusted life–year (qaly)—because an external study had previously calculated that threshold to be the implicit maximum acceptable cost-effectiveness threshold across all submissions⁴.

We addressed whether certain factors were associated with a positive recommendation for submissions supported by phase ii trials. A priori, we chose 3 factors that were hypothesized to be associated with a positive or conditional recommendation for inclusion in a multivariable logistic regression model: feasibility of completing a future phase iii study, lack of other available or alternative treatments (unmet need), and hematologic malignancy as the indication (compared with solid tumours). We hypothesized that filling an unmet need in treatment might be a positive predictor for a reimbursement recommendation. In addition, the feasibility of completing a phase iii study might negatively predict for reimbursement because higher-quality evidence about clinical benefit could become available. Lastly, because of the propensity for submissions concerning hematologic indications to be focused on rare disease subtypes and indications, we hypothesized that hematologic indications might be associated with a recommendation for reimbursement. We used an alpha of 0.05 as the threshold for statistical significance. We did not adjust for multiple comparisons.

All analyses were performed using the IBM SPSS Statistics software application (build 1.0.0.1275: IBM, Armonk, NY, U.S.A.).

RESULTS

Descriptive Statistics

Of 169 unique submissions filed through the pcodr process, 139 had final recommendations issued by the perc (Figure 1). Most submissions (81%) were supported with phase iii evidence. Of the submissions supported with phase ii evidence, 23 unique drugs were represented (associated with 27 submissions), indicating a pattern of multiple submissions for different indications. Among the submissions supported with phase ii evidence, 16 were for a hematologic indication; the remaining submissions were for solid tumour indications. The rate of acceptance was lower for submissions supported with phase ii evidence (63%) than with phase iii evidence (82%, Table II). In 1 submission, the icer was inestimable, and that submission was therefore excluded from the univariate icer category.

FIGURE 1.

Drug submissions to the pan-Canadian Oncology Drug Review (pCODR) for reimbursement status between July 2011 and July 2019. pERC = pCODR Expert Review Committee.

TABLE II.

Recommendations for submissions supported by phase II and phase III data

Variable	Phase II (nonrandomized)	Phase III (randomized controlled trial)
Submissions [n (%)]	27 (19)	112 (81)
Recommendation [n (%)]
Approved	2 (7)	8 (7)
Approved with conditions	15 (56)	84 (75)
Rejected	10 (37)	20 (18)

Univariable Analysis

Submission attributes were tested in univariable analyses for their association with the final recommendation (Table I). Of the tested attributes, feasibility of a phase iii study (p = 0.040), overall clinical benefit (p < 0.001), unmet need (p = 0.047), and alignment with patient values (p = 0.015) were all associated with a positive or conditional reimbursement recommendation. In all submissions in which the perc recommended approval for reimbursement (with or without conditions), the perc members agreed that there was evidence for net clinical benefit.

Continuous Variables

The mean icers for the positive and negative recommendations were $299,535 per qaly and $243,282 per qaly respectively; no significant association of icer and recommendation for approval was observed (p = 0.19). The mean icers for positive recommendations and conditional recommendations were $40,712 per qaly and $334,045 per qaly respectively.

The median sample size for submissions was 127 patients (range: 15–546 patients). On univariate analysis, no significant association between sample size and recommendation for approval was observed (p = 0.89).

Multivariable Analysis

A priori, we chose, for a multivariable logistic regression model, 3 factors that were hypothesized to be associated with a positive recommendation. The feasibility of a phase iii study was found to be significantly associated with a negative recommendation (p = 0.024; odds ratio: 0.132); unmet need and hematologic drug were not found to be significant.

DISCUSSION

Our study first establishes that drugs supported by phase ii data are considered by the perc, with more than half being recommended for public reimbursement (with or without conditions). Given that clinical benefit and alignment with patient values are key areas that lead to a positive or conditional recommendation, those actions further suggest that, in reimbursement decisions, phase ii evidence is increasingly being accepted in addition to traditional phase iii studies. Other countries with similar funding models—such as the United Kingdom and New Zealand—do not explicitly state the type of evidence that they receive in a submission^5,6. Australia’s drug funding agency cites their preference for randomized phase iii trials, but does not exclude nonrandomized studies from a submission⁷. Increasing transparency in funding processes around the world might help in trial planning for pharmaceutical companies, because it has also been previously shown that reimbursement decisions vary globally⁸.

We found that, in submissions with nonrandomized data, the presence of overall net clinical benefit, unmet need, patient alignment, and lack of phase iii trial feasibility (as deemed by the perc) were significantly associated with a reimbursement decision. “Overall clinical benefit” had a direct correlation with a positive recommendation for reimbursement and was a decision reached in consensus by the perc. In contrast, “cgp conclusion of clinical benefit” was still significant, but did not have 100% predictability: discrepancy between the overall perc recommendation and the cgp conclusion of clinical benefit was observed for 5 submissions. The cgp reported that the relevant studies “maybe” had clinical benefit; later review by the perc concluded that benefit was uncertain. Only studies that were reviewed and deemed to have definitive clinical benefit by the perc were then recommended for reimbursement.

The feasibility of conducting a phase iii trial or the presence of an ongoing phase iii trial was associated with a negative recommendation. Generally, those decisions were reached because the perc reasoned that the nonrandomized evidence provided did not offer sufficient support for greater clinical benefit over current standard therapy, as was the case, for example, with a submission for the use of crizotinib in patients with ALK-positive advanced non-small-cell lung cancer⁹. However, because a phase iii trial was already underway, the perc suggested that the manufacturer make a resubmission once trial results were published. The perc has no set criteria that it uses to determine feasibility; the determinations are deliberative and evolve after extensive discussion. Thereafter, for transparency, they are documented within the recommendation. We do note that previous perc decisions about phase iii feasibility have touched on the available sample size studied within the phase ii trial and whether other drugs have been studied in the phase iii setting for the same indication. In contrast, phase iii trials were deemed not feasible by the cgp if the medical condition or indication was felt to be rare.

Furthermore, clinical benefit is difficult to assess in cancer drugs because studies do not all report on the same outcomes. A study looking into cancer drugs approved by the European Medicines Agency found that most drugs entered the market without clear evidence concerning overall survival or quality of life¹⁰, further underlining the importance of having good support from clinical trials for reimbursement decisions. An assessment of clinical benefit and comparable reimbursement decisions globally was not within the scope of the present study.

In comparison, Skedgel et al.⁴ used available perc recommendations between 2011 and 2017 to assess factors leading to reimbursement and also found clinical benefit to be a significant predictor of reimbursement. Unmet need and the feasibility of a phase iii trial were not studied. In contrast, they found the severity of side effects, which we have labelled “toxicity,” also to be significant in considering all submissions. That significance could possibly be attributed to a larger sample size and, perhaps, the adverse events being more emphasized in phase iii studies with longer follow-up periods. With respect to economic analyses, it is important to note that cadth does not have an explicit icer threshold for submissions. We used an icer cut-off of $140,000 per qaly to dichotomize that variable into high and low values, because $140,000 per qaly was previously found to be the implicit maximum cost-effectiveness threshold across all submissions⁴. In contrast, Skedgel et al. looked at the icer model certainty, which was also not a significant predictor. Previous work by Rocchi and Mills¹¹ found that the median icer was $168,000 per qaly, supporting the fact that most submissions would not be considered cost-effective by the Economic Guidance Panel. In general, across those studies, a negative economic evaluation did not lead to a negative recommendation, but instead was mostly contributory to a recommendation being conditional upon improving cost-effectiveness. Skedgel et al. reported a similar result, finding that an icer of $150,000 per qaly or more predicted a conditional positive recommendation for reimbursement⁴. However, a lack of clear guidelines brings challenges to patients and pharmaceutical companies alike. At the time of the present analysis, the Patented Medicine Prices Review Board has put together a committee to update pricing guidelines and is in the process of seeking public input¹².

The main limitation of our study is the small sample size of 27 submissions supported by phase ii data, which might yield an imprecise estimate of the magnitude of any associations. Given the small sample size and the alpha cut-off of 0.05, the possibility of a phase iii trial being conducted was also the only significant predictor in our multivariable analysis. The multivariable analysis could be repeated in future once more submissions supported by phase ii evidence are reviewed. Nevertheless, we have included all submissions supported by phase ii evidence ever submitted to the pcodr process since that body’s inception. Our findings therefore represent the totality of the available evidence without any sampling involved (and hence no sampling bias). We also acknowledge that, although we looked thoroughly at the 4 quadrants in the perc’s deliberative framework to select our variables, there might also be other variables that are appropriate for analysis.

CONCLUSIONS

In summary, the perc accepts cancer drug submissions with phase ii evidence for review, and positive or conditional recommendations for reimbursement are associated with substantial clinical benefit, presence of unmet need, patient alignment, and lack of feasibility of a phase iii trial. High economic impact as determined by the Economic Guidance Panel usually leads to a recommendation conditional upon improving cost-effectiveness. Although more than half the submissions with phase ii evidence are recommended (with or without conditions), the perc has still recommended a higher percentage of submissions with phase iii evidence—an observation showing that, although phase ii evidence is becoming more commonly used, randomized trials are still preferred for inferring benefit and safety, based on our findings of a higher approval percentage rate for reimbursement recommendations. Future studies might consider a re-evaluation of the predictive factors when more phase ii submissions are available, a more in-depth comparison of phase ii predictors with those in other countries, and extension of the present analysis to the setting of non-cancer drugs.