Real-world evidence for option value in metastatic melanoma

Abstract

BACKGROUND:

The concept of real option value (ROV) suggests there is added value in treatments that extend life because they enable a patient to live long enough to benefit from future innovative treatments. Real-world evidence of this novel value element is scant, limiting its consideration in formal value assessments.

OBJECTIVE:

To calculate the ROV in clinical practice of ipilimumab for treatment of advanced melanoma, with evaluation of survival until availability of cancer immunotherapy (CIT).

METHODS:

This was a retrospective analysis of electronic health records from a US nationwide deidentified database including data from approximately 280 cancer clinics. Participants were patients with advanced or metastatic melanoma diagnosed after January 1, 2011, who initiated treatment before April 19, 2015, and were treated with first-line and second-line ipilimumab or chemotherapy, up to availability of CIT. The proportions of patients surviving and receiving CIT and overall survival by line of therapy were calculated. Baseline demographics were used to weight Kaplan-Meier curves using stabilized inverse probability of treatment weighting. ROV was estimated for patients receiving first-line or second-line ipilimumab with or without subsequent CIT and first-line or second-line chemotherapy with or without subsequent CIT.

RESULTS:

Overall, 721 patients were included in the study, with a total sample size of 733 (12 patients in both groups). For first-line ipilimumab, 50% of patients survived to the availability of CIT, while only 18% of first-line chemotherapy users survived to the same date. For second-line ipilimumab, 37% of patients survived to availability of CIT vs 21% of patients using second-line chemotherapy. 45% of first-line ipilimumab and 52% of second-line ipilimumab patients who survived to the availability date received CIT. ROV for first-line ipilimumab averaged 3.7 months of additional survival, while those who initiated second-line ipilimumab averaged 4.8 months. The combined estimated ROV was 3.9 months.

CONCLUSIONS:

This study provides real-world evidence of ROV and adds to the growing literature that may support inclusion of this novel value concept for innovative therapies alongside more traditional measures of value. Further evaluation of ROV in clinical areas with varying survival and innovation is warranted.

What is already known about this subject

• The concept of real option value (ROV) suggests that value may be generated when a medical technology (eg, drug) extends life for a patient long enough so that they may potentially benefit from the next treatment innovation.

• There is a lack of real-world evidence from clinical practice demonstrating ROV.

• The objective of this analysis was to calculate the ROV observed in real-world clinical outcomes data from a US electronic health record database using advanced melanoma as an example.

What this study adds

• Of 721 patients with advanced or metastatic melanoma, 50% receiving first-line ipilimumab and 37% on second-line ipilimumab survived to approval of pembrolizumab (a new cancer immunotherapy [CIT]) vs 18% and 21% who received first-line and second-line chemotherapy, respectively.

• Estimated ROV averaged 3.7 months (35% of conventional survival gain) and 4.8 months (57% of conventional survival gain) for first-line and second-line ipilimumab-treated patients, respectively.

• This study provides evidence from clinical practice of ROV and supports its inclusion when assessing or projecting the potential value of novel therapies.

Rapid developments in medical innovation and their associated costs have generated a need to define and assess the value of medical technologies. Estimating value via cost-utility analyses (CUAs) has become more commonplace, including among value framework developers such as the Institute for Clinical and Economic Review and the American College of Cardiology.¹ While costs and quality-adjusted life-years (QALYs) often form the basis of value assessments in CUAs, they may neglect other relevant aspects of value. Recently, additional novel elements of value have been proposed to augment the existing cost-utility framework, with a number of these new elements relating to the concept of uncertainty.^1,2

One novel value concept related to uncertainty is real option value (ROV), which suggests that value may be generated when a medical technology (eg, drug) extends life for a patient long enough so that they may potentially benefit from the next treatment innovation.³ Becker et al. first formally recognized the concept in 2007 and suggested that ROV should be considered as additive to conventional outcome measures such as survival and QALYs.⁴ The ROV calculation fits into existing CUA frameworks by modifying life-years or QALYs gained through accounting for the additional survival that some patients may gain from future innovations. For example, the introduction of zidovudine in 1987 for the treatment of HIV/AIDS not only improved patients’ life expectancy but also increased the likelihood of patients living to the availability of highly active antiretroviral therapy in 1996, thus further increasing the survival gains for zidovudine.^5,6 Of note, ROV is distinct from the value of hope, which reflects the value that patients place on the chance of a cure.^1,2

To date, there have been limited empirical data generated on ROV; thus, despite its merits, the high degree of uncertainty on its validity as a value element has restricted the incorporation of ROV into cost-effectiveness analyses. ROV can be calculated either before (based on forecasts) or after (based on actual results) innovation occurs. Several studies have historically estimated ROV (based on actual results) using modeling techniques and randomized controlled trials^5,7,8 and have largely focused on oncology indications, including metastatic renal cell carcinoma and advanced melanoma.^8-10 Although modeling studies enable estimation of ROV, they typically evaluate a specific set of treatments and scenarios (ie, first-line therapy followed by a future second-line innovator treatment). In reality, ROV may occur across a wide variety of scenarios when a new drug is approved, and uptake of innovative treatments, even if the patient survives, is unlikely to be 100%. Estimating ROV using real-world data may address some of these limitations; however, there is a lack of studies leveraging real-world evidence from clinical practice demonstrating ROV.

Advanced melanoma has been of particular interest in ROV studies, owing to rapid treatment innovations.⁸ The first immunotherapy, ipilimumab, demonstrated survival benefit over the standard of care (interleukin-2) and was approved in 2011.¹¹ Novel cancer immunotherapies (CITs), including pembrolizumab and nivolumab, were subsequently approved in advanced melanoma in 2014 and demonstrated additional survival benefits, with some patients exhibiting long-term remission. Metastatic melanoma therefore provides an ideal setting to explore the use of an ROV approach to assess how innovations that prolong survival can enable patients to benefit from the next novel treatment.

The objective of this analysis was to calculate the ROV for a cohort of patients with advanced melanoma treated with ipilimumab prior to the availability of CITs using observed real-world clinical outcomes data from a US electronic health record database as an example.

Methods

STUDY DESIGN

This retrospective cohort study used the nationwide Flatiron Health electronic health record-derived database, comprising longitudinal, deidentified, patient-level structured and unstructured data, curated via technology-enabled abstraction.^12,13 Data originated from approximately 280 cancer clinics (~800 sites of care) across the United States.

TREATMENT GROUPS AND INCLUSION CRITERIA

The study included patients diagnosed with melanoma (International Classification of Diseases, Ninth Revision, Clinical Modification [ICD-9-CM] 172.x or Tenth Revision [ICD-10-CM] C43x or D03x), with at least 2 distinct clinic encounters in the database on or after January 1, 2011, who had advanced (stage III) unresectable or metastatic (stage IV) melanoma diagnosed after January 1, 2011, and who initiated treatment before April 19, 2015. Patients were treated with ipilimumab or chemotherapy (paclitaxel, temozolomide, carboplatin, cisplatin, dacarbazine). Additional inclusion criteria included at least 1 clinic visit within 120 days of advanced or metastatic diagnosis and receipt of treatment within 120 days of metastatic diagnosis or advanced diagnosis/local recurrence date (if unresectable). Patients were assigned to 4 treatment groups: (1) first-line ipilimumab, (2) first-line chemotherapy, (3) second-line ipilimumab, (4) first-line and second-line chemotherapy (did not receive first-line ipilimumab; Supplementary Figure 1 ^{(139KB, pdf)} , available in online article). Patients could receive ipilimumab as monotherapy or in combination; combinations included ipilimumab plus chemotherapy and ipilimumab plus targeted therapy. Patients who received first-line ipilimumab followed by second-line chemotherapy were grouped with the first-line ipilimumab treatment group. Patients treated with ipilimumab who had their last clinic visit before September 4, 2014, and did not have a date of death were excluded, as they were assumed to be lost to follow-up. Patients were followed up from the initiation of first-line/second-line therapy until death or last known activity, including whether they received CIT (pembrolizumab or nivolumab) as second-line or third-line therapy. The study cutoff date for patients to be treated with ipilimumab was April 19, 2015.

OUTCOMES AND CALCULATION OF ROV

The calculation of ROV has 2 main components: (1) the size of the population receiving the next innovation and (2) the benefit of the next innovation. Estimating ROV from these components can be summarized as follows:

$$ROV=\%Treatedwiththenextinnovation\times \Delta \text{OS}$$

The eligible cohort to potentially benefit from ROV was patients receiving ipilimumab prior to the next innovation availability, defined as September 4, 2014—the US Food and Drug Administration (FDA) approval date for second-line pembrolizumab (see Figure 1 for key dates of approval and data release for CITs in this study). The proportion treated with the next innovation was calculated using the number who received a CIT divided by the total eligible cohort. Differences in overall survival (OS) reflect the benefit of the next innovation and were calculated between those patients who were treated with the next innovation (CIT) and those who were not, as described in the Statistical Analysis section.

FIGURE 1.

Key Dates for Cancer Immunotherapy Approval and Data Becoming Publicly Available

To estimate ROV, in each line of therapy, we first multiplied the proportion of patients treated with ipilimumab who received subsequent CIT by the difference in mean survival between patients treated with ipilimumab who received and did not receive subsequent CIT. This estimate by line of therapy was then weighted by the proportion of patients receiving ipilimumab in first-line vs second-line therapy to calculate the overall ROV for ipilimumab. We also report ROV as a percentage of the conventional survival gain, as has been reported in studies previously¹⁰:

$$ROVas\%ofconventionaloutcome=\frac{ROV}{O{S}_{Ipi}-O{S}_{Chemo}}\times 100$$

In the formula above, OS_Ipi is the OS for those treated with ipilimumab without any subsequent CIT and OS_Chemo is the OS for those treated with chemotherapy without any subsequent CIT.

While the proportion treated with the next innovation accounts for variations in the timing of patient treatment relative to the next innovation date, to understand the drivers of this percentage, we further examined the percentage treated by the proportion surviving to the next innovation and the proportion receiving CIT treatment among those surviving to the next innovation date. The proportion of patients surviving to the approval date of immunotherapy (referred to as “next innovation date”) was estimated by dividing the number surviving to this date by the total number of patients who initiated therapy in each treatment cohort before the innovation date. The proportion of patients who received the next innovative treatment was estimated by dividing those who received CIT by the total number of patients surviving to the next innovation date.

To ensure robust estimates of survival outcomes for ipilimumab as an input to the ROV calculation, we expanded the patient pool for survival analyses to include patients treated up until April 19, 2015—the date that first-line pembrolizumab data were presented at the 2015 American Association for Cancer Research (AACR) meeting. Prior to this, no evidence was available to support CIT use as first-line treatment; hence, ipilimumab was the standard of care. Although first-line pembrolizumab did not receive FDA approval until December 18, 2015, it was assumed that some patients would receive off-label pembrolizumab after the presentation of related data at AACR given the strength of results; thus, the standard of care between April 19, 2015, and December 18, 2015, was more ambiguous, and outcomes for patients in this period were more at risk of selection bias. The appropriateness of including patients after the next innovation date (September 4, 2014) and the data cutoff (April 19, 2015) was determined by comparing real-world progression-free survival (rwPFS) and OS outcomes for those treated in this period and those treated prior to the next innovation date. As rwPFS and OS were similar in this comparison, it was considered appropriate to include these patients for estimates of survival only.

Because the number of patients available who initiated chemotherapy prior to the ipilimumab approval date was limited, patients were selected if they initiated chemotherapy at any time during 2011. When patients who initiated chemotherapy prior to ipilimumab approval were compared with chemotherapy at any time during 2011, OS and rwPFS were similar; hence, it was considered appropriate to make this assumption.

STATISTICAL ANALYSIS

Descriptive statistics were used for continuous and categorical variables. Wilcoxon test or chi-square test statistics were used to compare differences in baseline patient characteristics by treatment group at first-line and second-line treatment. Kaplan-Meier estimates for OS were estimated by treatment group and line of therapy. For OS among patients who received first-line ipilimumab vs first-line chemotherapy, Kaplan-Meier curves were weighted using stabilized inverse probability of treatment weighting using age, region, sex, advanced stage at diagnosis, year of first-line initiation, and BRAF gene status. Year of first-line initiation was included to account for variations in timing of treatment initiation. For OS among first-line ipilimumab patients who received second-line chemotherapy vs those who received second-line CIT, the Kaplan-Meier curves were weighted with a similar approach to construct the weights. Statistical testing comparing the 2 survival curves was not performed because this study did not set out to measure the relative effectiveness of second-line therapy.

Survival outcomes for the proportion of patients surviving and receiving the next innovative treatment were stratified for the ipilimumab first-line and second-line treatment groups by those who received CIT (at any time point) and those who did not. Among those who received first-line ipilimumab, OS was estimated for those receiving second-line chemotherapy vs second-line CIT, with the date of initiation of second-line therapy as the index date. Mean and median survival were calculated for each cohort.

Results

STUDY DISPOSITION AND PATIENT CHARACTERISTICS

In total, 721 patients were included in this study, 12 of whom were included in both the first-line and second-line chemotherapy group, making the total sample size 733. Patient characteristics, including demographics and the proportion of patients receiving each treatment, are shown in Supplementary Table 1 ^{(139KB, pdf)} (available in online article). Baseline characteristics, including stage at advanced diagnosis, were similar between the first-line and second-line treatment groups. The proportion of patients receiving ipilimumab as first-line treatment changed significantly over time, from 3% in 2011 to 44% in 2014 (P < 0.001). The populations in the first-line treatment groups were primarily BRAF negative (71%).

PROPORTION OF PATIENTS SURVIVING TO THE NEXT INNOVATION DATE AND RECEIVING CIT

Among patients treated before the next innovation date, half of patients (50%) treated with first-line ipilimumab survived to the next innovation date (September 4, 2014; pembrolizumab FDA approval as second-line therapy), while only 18% of first-line chemotherapy users survived to the same date. Similarly, 37% of patients using second-line ipilimumab survived to the next innovation date (pembrolizumab approval) compared with 21% of patients using second-line chemotherapy (Figure 2). The median time from ipilimumab initiation to CIT initiation was 5.2 months. The proportion of patients surviving to the next innovation date varied by year of treatment initiation across all treatment groups (Figure 2); fewer patients receiving first-line ipilimumab in 2011 survived versus those treated closer to the next innovation date (before September 4, 2014). No patients who received second-line treatment in 2011 survived to the next treatment innovation. The proportions of patients surviving increased in 2014 to 64% and 38% for second-line ipilimumab and second-line chemotherapy, respectively (Figure 2).

FIGURE 2.

Proportions of Patients Surviving to the Next Innovation Date (A) and Receiving Cancer Immunotherapies (B)

Among patients who received first-line ipilimumab and survived to the next innovation date, 45% (n = 68) received subsequent treatment with CIT; the majority (71%) received a second-line CIT, while the remainder received CIT in third line or later (Figure 2). For patients receiving second-line ipilimumab, 52% received CIT as third-line or later.

OVERALL SURVIVAL FOR FIRST-LINE AND SECOND-LINE IPILIMUMAB

OS for patients treated with first-line ipilimumab without subsequent CIT treatment was longer than for patients treated with first-line chemotherapy without subsequent CIT treatment (median OS = 6.9 months vs 5.3 months; mean OS = 21.7 months vs 11.1 months). A longer OS was observed among patients treated with first-line ipilimumab who received subsequent CIT at any time compared with those who did not (median OS = 38.7 months vs 6.9 months; mean OS = 38.3 months vs 21.7 months; Figure 3).

FIGURE 3.

Overall Survival for First-Line Ipilimumab^a

Patients who received second-line ipilimumab and then subsequently received CIT (at any time) (median OS = 24.7 months; mean OS = 38.9 months) had a longer OS compared with patients who did not receive a subsequent CIT (median OS = 3.2 months; mean OS = 13.9 months; Figure 4).

FIGURE 4.

Overall Survival for Second-Line Ipilimumab

REAL OPTION VALUE

ROVs for patients who initiated first-line ipilimumab, second-line ipilimumab, and overall (first-line plus second-line) are shown in Table 1. Patients on first-line ipilimumab averaged an additional 3.7-month survival (34.9% of conventional survival gain for ipilimumab vs chemotherapy) versus 4.8 months for those who initiated second-line ipilimumab (56.5% of conventional survival gain for ipilimumab vs chemotherapy). The combined ROV was 3.9 months. There were insufficient patients in the study who received chemotherapy followed by CIT to enable estimation of the ROV for chemotherapy.

TABLE 1.

Real Option Value

	First-line ipilimumab		Second-line ipilimumab
	No subsequent CIT	Subsequent CIT	No subsequent CIT	Subsequent CIT
Proportion of patients in each cohort, %a	77.6 (236/304)	22.4 (68/304)	80.9 (72/89)	19.1 (17/89)
Mean overall survival, months	21.7	38.3	13.9	38.9
ROV (survival, months)	22.4% × (38.3 − 21.7) = 3.7		19.1% × (38.9 − 13.9) = 4.8
ROV (% of conventional survival gain)	3.7 ÷ (21.7 − 11.1) × 100 = 34.9		4.8 ÷ (13.9 − 5.4) × 100 = 56.5
Proportion of patients receiving first-line/second-line ipilimumab, %	77.4		22.6
Overall ROV	3.9 months

^aUsing the number in each cohort divided by total number receiving first-line or second-line ipilimumab.

CIT = cancer immunotherapy; ROV = real option value.

Discussion

To the best of our knowledge, this is the first study to measure ROV using real-world data in oncology. We observed a larger proportion of patients on ipilimumab who survived to the next treatment innovation date and greater uptake of CIT among those who survived on ipilimumab relative to chemotherapy. Among those who survived to the availability of the next innovation, a substantial survival benefit was observed for those receiving a subsequent CIT compared with those who did not. To our knowledge, this has not been measured previously using real-world data and is an often-overlooked aspect in value assessment.

The additional survival observed with CIT differed from the clinical trials where nonsignificant improvements in OS were observed (median OS of 14.7-16 months with CIT compared to 10.9-14 months with chemotherapy), although significant crossover in the studies (> 40%) hinders interpretation.^14-16 However, our OS and ROV estimates are not directly comparable to clinical trial results due to differences in settings, study designs (none of the pivotal trials measured survival of ipilimumab followed by CIT), and calculations of ROV (that account for the proportion receiving treatment). Despite this, the additional survival benefit for CIT is not unexpected given other outcomes of the clinical trials that demonstrated a significant progression-free survival (PFS) and duration of response (DOR) vs chemotherapy in the refractory setting (PFS hazard ratio = 0.5; 95% CI = 0.39-0.64; P < 0.0001; DOR = 32 months vs 13 months).^14-16

While this study provides an empirical estimate of ROV for the time period that ipilimumab was available prior to availability of CIT, it should be acknowledged that the ROV estimate will vary depending on when within this time period a patient initiated ipilimumab. Potentially larger ROV estimates would be expected for time periods closer to the next innovation date due to the larger proportion of ipilimumab patients and the greater proportion of patients who survived and received CIT. The increase in ipilimumab use near the innovation date is consistent with previous research demonstrating the influence of ROV on treatment decisions made by cancer clinicians, with greater use of potentially curative treatments as new breakthrough innovations are on the horizon.³ Furthermore, the relationship between time and ROV also has implications for forecasting estimates of ROV, as therapeutic areas with rapid innovation, thus shortening the time between new treatments, would be expected to result in larger ROV estimates.

These findings enhance our understanding of the value that innovation can bring to melanoma therapy. There were an estimated 13,500 incident cases of metastatic melanoma in the United States from 2011 to 2014.¹⁷ Assuming 60% of these cases were ipilimumab-treated, this would represent an ROV gain of 2,500 life-years or ~35% of the direct incremental gain from ipilimumab (7,200 life-years). This may be an underestimate because patients with stage III unresectable or recurrent melanoma who may also benefit from improved survival due to future treatments were not included.

Previous modeling studies have estimated the realized ROV of ipilimumab using data from randomized controlled trials and modeling techniques. Thornton-Snider et al estimated ROV in metastatic melanoma and found that the conventionally calculated survival gain from ipilimumab ranged from 15% to 49% depending on the year of ipilimumab initiation.⁸ The authors assumed that all those who survived to CIT availability received subsequent CIT. Another limitation to modeling approaches is that they generally make simplified assumptions concerning treatment pathways (eg, first-line ipilimumab followed by CIT) and may lack clinical trial data to support specific treatment pathways—not including multiple treatment pathways may potentially underestimate ROVs.

Our study was uniquely able to address some of the limitations of previous approaches. First, we leveraged real-world evidence to provide actual estimates for proportions surviving to the next innovation date and receiving CIT. Additionally, we demonstrated that multiple scenarios may occur in the real world when a new product is launched, and there may be multiple pathways in which ROV occurs. Although the majority of patients received first-line ipilimumab followed by second-line CIT, some received CIT in later lines or received second-line ipilimumab followed by CIT. Lastly, trial data demonstrating the effectiveness of CIT post-ipilimumab are lacking; however, we identified a specific cohort of real-world patients who initiated CIT post-ipilimumab and measured their survival.

This evidence of real-world ROV has potential implications for future health technology assessments and policy making. Arguments against inclusion of ROV have focused on the high degree of uncertainty around estimates and the limited empirical evidence on methods to incorporate ROV based on forecasting approaches.^18,19 Previous studies have generated forecasting approaches to estimate ROV in oncology; however, there is significant uncertainty around the uptake of new innovations.^10,20 Utilizing real-world evidence may help to refine forecasting modeling approaches by informing input values, such as the uptake of new innovations, thereby helping to reduce uncertainty and build confidence in the methodology.

In addition to further research being needed on forecasts of innovation uptake, additional research is warranted to facilitate inclusion of ROV consistently in value assessments and augment CUAs.¹ While some recent studies have examined probabilities of new innovations,²¹ additional research is needed to understand how these may vary across specific therapeutic areas and drug classes. Additionally, further studies examining the magnitude of future benefits would also allow more robust ROV analyses.

LIMITATIONS

As is common to all real-world analyses, data are subject to potential misclassification. Patients may have received care outside the Flatiron network and therefore their data were not fully captured. The inclusion criteria employed sought to mitigate this risk. It should be noted that the measurement of ROV and immortal time bias are interconnected. Immortal time bias refers to a period of time for which a treatment group cannot have the outcome (ie, death) during follow-up because the treatment group definition requires that they survive to a certain time point. This bias can lead to an overestimate of a treatment effect; however, we did not measure the relative effectiveness of alternative treatment strategies (ie, ipilimumab followed by CIT vs no CIT/chemotherapy)—which would be interpreted as an immortal time bias—but the additional survival due to the ability of surviving patients to receive a novel second-line therapy.

Observational studies are subject to confounding with the potential to bias results. We sought to mitigate this risk by using an inverse probability of treatment weighting approach; despite limitations in the variables available to construct the weights, unknown or unmeasured confounders might still be present. Ideally, future analyses would leverage additional confounding variables to construct weights to reduce bias. Further, there were challenges in identifying and designing the appropriate counterfactual cohort. Patients who did not receive the standard of care may be systematically different, but as we were unable to discern reasons why (using this dataset), additional research would be needed to understand this. Additionally, we were not able to account for the ROV of chemotherapy, owing to limited sample size.

Finally, we did not estimate the ROV potentially gained via the introduction of BRAF inhibitors and combinations with MEK inhibitors, although these could be considered in assessing ROV because they provide alternate pathways by which ROV may occur. Only a proportion of patients (BRAF positive) may have been eligible to benefit from these innovations. In our study, there were 7 patients who received ipilimumab prior to the FDA approval date of vemurafenib (August 17, 2011). All patients survived to the FDA approval date; 5 patients (71%) were tested for BRAF status and 1 patient (14%) received a targeted therapy following ipilimumab. Exclusion of these patients from the ROV calculation may underestimate the ROV gained from ipilimumab, although larger sample sizes than observed in this study would be required to estimate the ROV.

Conclusions

This study provides real-world evidence of measurable ROV, corresponding to 35%-57% of the conventional survival attributable to ipilimumab, and adds to the growing literature that may support inclusion of this novel value concept for innovative therapies alongside more traditional measures of value. While this study may inform the uptake of future innovations for ex ante approaches to ROV in melanoma or oncology, further research is warranted to understand the occurrence of ROV in different diseases and to continue to refine forecasting approaches.