Precision Oncology: The Intention-to-Treat Analysis Fallacy

Abstract

It has recently been suggested that precision oncology studies should be re-analyzed using the intention-to-treat (ITT) methodology developed for randomized controlled clinical trials. This re-analysis dramatically decreases response rates in precision medicine studies. We contend that the ITT analysis of precision oncology trials is invalid. The ITT methodology was developed three decades ago to mitigate the problems of randomized trials, which try to ensure that both arms have an unselected patient population free from confounders. In contrast, precision oncology trials specifically select patients for confounders (that is biomarkers) that predict response. To demonstrate the issues inherent in an ITT re-analysis for precision cancer medicine studies, we take as an example the drug larotrectinib (TRK inhibitor) approved because of remarkable responses in malignancies harboring NTRK fusions. According to large-scale studies, NTRK fusions are found in ~0.31% of tumors. In a non-randomized pivotal study of larotrectinib, 75% of the 55 treated patients responded. Based upon the prevalence of NTRK fusions, ~18,000 patients would need to be screened in order to enroll the 55 treated patients. Utilizing the ITT methodology, the revised response rate to larotrectinib would be 0.23%. This is, of course, a dramatic underestimation of the efficacy of this now FDA-approved drug. Similar issues can be shown for virtually any biomarker-based precision clinical trial. Therefore, retrofitting the ITT analysis developed for unselected patient populations in randomized trials yields misleading conclusions in precision medicine studies.

Perspective

Precision cancer medicine is defined by a strategy that requires an understanding of the molecular and biologic characteristics of a patient’s tumor(s), followed by constructing a therapeutic approach that precisely targets those abnormalities.¹ Even with shared histopathologies or organ of origin, genomics has unveiled a remarkably complicated biologic landscape, with metastatic malignancies manifesting molecular complexity that differs from tumor to tumor.^2–4 In order to impact heterogeneous cancers in a precise manner, one must customize (or personalize) the therapeutic regimen. This new precision-personalized model differs from conventional strategies for cancer management in that it reflects a patient-centric, rather than drug-centric approach. This shift in oncology treatment paradigms requires that outcome analyses are adapted to accurately assess this emerging, innovative strategy for patient care.

Recently, some investigators have suggested⁵ that the outcomes of precision oncology trials should be re-analyzed using the intention-to-treat (ITT) methodology, which was first developed three decades ago, before the advent of molecularly matched therapy⁶ and four decades after randomized controlled trials (RCTs), whose problems it was designed to remedy.⁷ RCTs compare two treatment arms in patients with shared histologies, but not necessarily shared underlying tumor biologies. These trials frequently suffer from non-compliance and missing outcomes, making it difficult to compare the two arms. The ITT analysis resolves issues that plagued traditional RCTs by including every patient who is randomized to both arms, regardless of protocol deviations, non-compliance, withdrawal, and any other reason(s) that supervenes after randomization.⁸

It is not surprising that when ITT analyses are performed in the context of precision oncology trials, the response rates are dramatically lowered. Combining all patients, whether unmatched, matched, untreated or treated, is akin to combining both arms of randomized clinical trials in order to assess whether either investigational regimen is effective, and institutes anachronistic arbitrary statistical assumptions. The first time we administered individualized combination therapies to patients with lethal malignancies in the I-PREDICT study,⁹ for example, patients had better outcomes if they had high percentages of their identified genomic alterations matched. This is not surprising since metastatic tumors have remarkably complex molecular portfolios and each tumor tends to be unique.^2–4 Patients with high matching scores (>50%; usually reflecting more than half of their genomic alterations being matched) had significantly longer progression-free (PFS) and overall survival, and objective response rates (ORR) of 45% versus 16% for low matching scores (≤50%).⁹ A re-analysis that includes all patients consented, however, reduces the ORR to 11%,⁵ and does not differentiate between patients that were well matched versus poorly matched. The reduction in ORR is because only 49% of patients received matched treatment; the most common reason patients were not treated was because their disease was advanced at consent (a not uncommon problem in precision medicine trials) and they deteriorated or died before therapy could be started.⁹ ITT re-analysis is misleading in the non-randomized trial setting where treatment assignment is not by chance alone.

To further demonstrate the issues inherent in an ITT analysis for precision cancer medicine studies, we take as examples three randomized trials critical for U.S. Food and Drug Administration (FDA) approval (Table 1): crizotinib (ALK inhibitor, lung cancer);¹⁰ trastuzumab (Her2 antibody, gastro-oesophageal cancers);¹¹ and vemurafenib (BRAF inhibitor, melanoma).¹² An average of 3,626 patients (range: 2,107 to 4,967) were screened in order to find an average of 539 patients (15% of screened patients) with the cognate alteration. Hence, the reported crizotinib ORR was 65% in the treated ALK-rearranged patients. Had half of the screened patients (i.e., ITT for that arm of the randomized trial) been included, the ORR would be only 4.6%. We already know that <5% of lung cancers harbor ALK rearrangements. The actual ORR versus recalculated ITT ORR were similarly 47.3% versus 7.3% (trastuzumab) and 48.4% versus 10.1% (vemurafenib). These drugs are now approved by multiple regulatory agencies, with widely demonstrated high levels of efficacy. The original studies clearly demonstrated the clinical utility of molecular profiling and precision targeting of tumors. An ITT re-analysis is misleading and invalid since the very purpose of the studies was to identify the genetic confounder(s) that predict response, rather than to eliminate confounders in the unselected patient populations that are the subject of RCTs. Indeed, as shown in Table 1, one must find the right patients for these drugs, even if it means screening thousands of individuals and treating only a small subset.

Table 1.

Randomized clinical trials with molecularly matched targeted agents.

Author	Study	Disease	Investigational agent	Control	Screened	Randomized	Investigational arm	Evaluable for CR + PR in investigational arm	Reported CR + PR in investigational arm	Reported ORR	Intention-to-treat ORR*
Shaw et al,10 2013	PROFILE 1007	Locally advanced or metastatic ALK-positive lung cancer who had received one prior platinum-based regimen	Crizotinib	Chemotherapy with pemetrexed or docetaxel	4967	347	173	173	113	65.3%	4.6%
Bang et al,11 2010	ToGA	Gastric or gastrooesophageal junction cancer were eligible for inclusion if their tumours showed overexpression of HER2 protein by immunohistochemistry or gene amplification by fluorescence in-situ hybridisation	Trastuzumab plus chemotherapy	Chemotherapy alone	3803	594	298	294	139	47.3%	7.3%
Chapman et al,12 2011	BRIM 3	Previously untreated, metastatic melanoma with the BRAF V600E mutation	Vemurafenib	Dacarbazine	2107	675	337	219	106	48.4%	10.1%

^*Calculated based upon half of the screened patients.

Abbreviations: CR = complete response; PR = partial response; ORR = objective response rate

Other examples such as those of NTRK fusion matched therapy are also pertinent. Based upon a recently reported analysis of 13,467 adult tumor samples from The Cancer Genome Atlas (TCGA), we showed that NTRK fusions were observed in 0.31% of cancers.¹³ Moreover, in a non-randomized New England Journal of Medicine study of larotrectinib, a highly selective TRK inhibitor, 75% of the 55 treated patients had an objective response.¹⁴ Based upon the incidence of these mutations, we calculated that 17,742 patients would need to be screened in order to accrue the 55 patients that were initially consented. Thus, based on the ITT methodology, we would calculate a 0.23% objective response rate for larotectinib. This is, of course, not a true reflection of the efficacy of this now FDA-approved drug.

Precision-personalized medicine unlocks a new paradigm for addressing the cancer problem. Some of the results to date have been stunning, including the near-normalization of life expectancy for chronic myelogenous leukemia—a previously lethal disease—when the molecular BCR-ABL aberration is targeted by cognate inhibitors,¹⁵ and the high rate of durable responses in solid tumors with ALK rearrangements or NTRK fusions after ALK or NTRK targeting, respectively.^10,14 Evaluating current outcomes requires analytic methodologies modernized for the patient-centered model.

It took 40 years to develop a statistical analysis (ITT analysis) that addressed the problems of RCTs. Though precision medicine may be in its nascence, it has become increasingly apparent that we need to now focus on therapeutic approaches that lead to more dramatic improvements in ORRs and survival endpoints, and those approaches should be based on a deep scientific understanding of individual tumor biology. As we hone our appreciation of cancer biology and drug targeting, retrofitting decades old statistical methods, such as the ITT analysis developed for unselected patient populations yields misleading conclusions. Properly assessing outcomes requires analytic methodologies modernized for the patient-centered model that specifically seeks out the biomarkers of responsiveness and, hence, represents the tenets of precision-personalized medicine.

HIGHLIGHTS.

• Precision cancer medicine uses biomarkers to customize cognate targeted therapies.

• Intention-to-treat (ITT) reduces bias in randomized trials of unselected patients.

• Precision medicine trials pre-screen patients to find those with the biomarker.

• Prescreening may require hundreds of patients to find one with a rare biomarker.

• ITT analyses that include prescreened patients are unsuitable for precision trials