Early Readout on Overall Survival of Patients With Melanoma Treated With Immunotherapy Using a Novel Imaging Analysis

Key Points

Question

Can machine learning identify a combination of quantitative imaging features that can predict survival with immunotherapy better than conventional size-based assessment?

Findings

In a prognostic analysis of prospectively collected clinical trial data from 575 patients with a diagnosis of advanced melanoma, a random forest algorithm found that 4 computed tomography imaging features, 2 related to tumor size and 2 reflecting changes in tumor imaging phenotype, best estimated overall survival with immunotherapy. The combination of these features (signature) outperformed Response Evaluation Criteria in Solid Tumors 1.1, the standard method based on tumor diameter.

Meaning

These findings suggest that radiomics and machine learning can analyze information captured by routine computed tomographic scans to improve clinical decision making for patients with melanoma treated with immunotherapy.

This prognostic study assesses whether a signature on computed tomographic imaging derived from tumor size, density, and shape and its change as treatment is administered can estimate overall survival in patients with advanced melanoma.

Abstract

Importance

Existing criteria to estimate the benefit of a therapy in patients with cancer rely almost exclusively on tumor size, an approach that was not designed to estimate survival benefit and is challenged by the unique properties of immunotherapy. More accurate prediction of survival by treatment could enhance treatment decisions.

Objective

To validate, using radiomics and machine learning, the performance of a signature of quantitative computed tomography (CT) imaging features for estimating overall survival (OS) in patients with advanced melanoma treated with immunotherapy.

Design, Setting, and Participants

This prognostic study used radiomics and machine learning to retrospectively analyze CT images obtained at baseline and first follow-up and their associated clinical metadata. Data were prospectively collected in the KEYNOTE-002 (Study of Pembrolizumab [MK-3475] Versus Chemotherapy in Participants With Advanced Melanoma; 2017 analysis) and KEYNOTE-006 (Study to Evaluate the Safety and Efficacy of Two Different Dosing Schedules of Pembrolizumab [MK-3475] Compared to Ipilimumab in Participants With Advanced Melanoma; 2016 analysis) multicenter clinical trials. Participants included 575 patients with a diagnosis of advanced melanoma who were randomly assigned to training and validation sets. Data for the present study were collected from November 20, 2012, to June 3, 2019, and analyzed from July 1, 2019, to September 15, 2021.

Interventions

KEYNOTE-002 featured trial groups testing intravenous pembrolizumab, 2 mg/kg or 10 mg/kg every 2 or every 3 weeks based on randomization, or investigator-choice chemotherapy; KEYNOTE-006 featured trial groups testing intravenous ipilimumab, 3 mg/kg every 3 weeks and intravenous pembrolizumab, 10 mg/kg every 2 or 3 weeks based on randomization.

Main Outcomes and Measures

The performance of the signature CT imaging features for estimating OS at the month 6 posttreatment landmark in patients who received pembrolizumab was measured using an area under the time-dependent receiver operating characteristics curve (AUC).

Results

A random forest model combined 25 imaging features extracted from tumors segmented on CT images to identify the combination (signature) that best estimated OS with pembrolizumab in 575 patients. The signature combined 4 imaging features, 2 related to tumor size and 2 reflecting changes in tumor imaging phenotype. In the validation set (287 patients treated with pembrolizumab), the signature reached an AUC for estimation of OS status of 0.92 (95% CI, 0.89-0.95). The standard method, Response Evaluation Criteria in Solid Tumors 1.1, achieved an AUC of 0.80 (95% CI, 0.75-0.84) and classified tumor outcomes as partial or complete response (93 of 287 [32.4%]), stable disease (90 of 287 [31.3%]), or progressive disease (104 of 287 [36.2%]).

Conclusions and Relevance

The findings of this prognostic study suggest that the radiomic signature discerned from conventional CT images at baseline and on first follow-up may be used in clinical settings to provide an accurate early readout of future OS probability in patients with melanoma treated with single-agent programmed cell death 1 blockade.

Introduction

The success of immune checkpoint blockade (ICB) in enhancing a patient’s immune response against cancer has led to many clinical trials examining ICB alone or in combination with other investigational agents. Early radiographic markers of treatment efficacy reliably associated with overall survival (OS) could help identify which patients are most likely to benefit from ICB.

Change in tumor size is currently the basis for early assessment of efficacy in clinical trials evaluating cancer therapeutics, using standard computed tomographic (CT) scans and classified according to Response Evaluation Criteria in Solid Tumors 1.1 (RECIST 1.1). The ability of RECIST 1.1 to assess ICB efficacy is challenged by atypical patterns of response, including pseudoprogression, an initial increase in tumor burden followed by tumor shrinkage in a small fraction of patients.^1,2,3 To address this challenge, modified RECIST in cancer immunotherapy (iRECIST) requires confirmation of progressive disease on subsequent scans. However, neither standard RECIST 1.1 nor iRECIST reliably estimate OS from ICB,⁴ and both rely solely on anatomical information regarding tumor size.

Computed tomographic scans used in standard practice provide additional information that radiomics and machine learning can extract as quantitative features, including tumor volume (which affords improved estimates of tumor size), heterogeneity of tumor density (a surrogate of tumor vascularity and necrosis), and tumor shape (irregular margins being associated with more aggressive growth patterns). We applied radiomics and machine learning to data from 2 large clinical trials of ICB in advanced melanoma, KEYNOTE-002 (Study of Pembrolizumab [MK-3475] Versus Chemotherapy in Participants With Advanced Melanoma)⁵ and KEYNOTE-006 (Study to Evaluate the Safety and Efficacy of Two Different Dosing Schedules of Pembrolizumab [MK-3475] Compared to Ipilimumab in Participants With Advanced Melanoma),⁶ to identify CT changes objectively and reproducibly at 3 months after starting ICB treatment. With a principal goal of improving clinical management, we tested the hypothesis that a radiomic signature derived from tumor size, density, and shape and its change as treatment is administered can estimate OS in patients with melanoma.

Methods

Patients

Using data provided by the Vol-PACT⁷ (Advanced Metrics and Modeling With Volumetric CT for Precision Analysis of Clinical Trial Results) public-private partnership, we retrospectively analyzed CT scans and clinical data from 575 adults with unresectable stage III/IV melanoma from 2 landmark randomized clinical trials^5,6 (Figure 1). Data were obtained from November 20, 2012, to June 3, 2019. KEYNOTE-002⁵ enrolled 540 patients to test whether pembrolizumab improved progression-free survival and OS compared with the investigator’s choice of chemotherapy; KEYNOTE-006⁶ enrolled 834 patients to test whether pembrolizumab improves OS compared with ipilimumab. Tumor assessments were performed using standard CT scans of the chest, abdomen, and pelvis with contrast and standard RECIST 1.1 criteria before starting treatment (baseline), at month 3, every 6 weeks through week 48, and every 12 weeks thereafter. Patients deemed clinically stable could continue immunotherapy and imaging follow-up, even if tumors were classified as progressive disease as per RECIST 1.1. The research protocol was approved by the institutional review board of Columbia University with a waiver of informed consent for the use of retrospective deidentified data. This study followed the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) reporting guideline.

Figure 1. Study Participants.

IO indicates immunotherapy.

Imaging Features

Radiologists used a picture archiving and communications system–type image viewing platform to perform semiautomatic volumetric segmentation of all measurable lesions at baseline and month 3 (eMethods 2 in the Supplement) on every CT scan section where detected (eFigure 1 in the Supplement). Principal component analysis created 20 linearly uncorrelated imaging features.^8,9 Four additional features were created to characterize tumor burden in each of the 4 most frequent metastatic sites, plus a fifth feature representing overall tumor burden (eMethods 3 in the Supplement). All lesions were aggregated to compute a total tumor burden imaging signature for each patient at baseline and month 3 using these 25 features (eMethods 2 and 3 in the Supplement). The difference or change in radiomics features between baseline and month 3 after treatment initiation was computed.

Training and Validation Sets

A total of 668 patients had both baseline and month 3 scans. Patients received intravenous therapy every 2 or every 3 weeks in KEYNOTE-002 and KEYNOTE-006. The 3 treatment groups consisted of pembrolizumab (n = 465), investigator’s choice of chemotherapy (n = 93), and ipilimumab (n = 110). Merck & Co, Inc, provided 80% of patients from the clinical trials.

The Vol-PACT consortium randomly assigned patients to training (model development) and validation (performance evaluation) sets (Figure 1).⁷ Random assignment was stratified by treatment with differing randomization ratios for each trial to address the overall objectives of the project.⁷

The assigned training sets for all immunotherapy groups (pembrolizumab [n = 178] and ipilimumab [n = 74]) were combined and used to train our signature (n = 252). We evaluated the performance of this signature in the validation set of the patients in each treatment group independently to assess its generalizability. This report presents results in the pembrolizumab validation set (n = 287) because it is the largest immunotherapy cohort, best suited to deriving statistical comparisons with standard metrics of assessing response.

Model Building

A machine-learning algorithm was developed using random forests (eFigure 1 in the Supplement) to select among 25 candidate variables measured at baseline and month 3 (eMethods 3 in the Supplement). The resulting model, termed the radiomics signature, was trained to estimate OS using a random forest method suitable for censored survival data in patients treated with IO therapy randomized to the training set (n = 252). Analysis of OS was landmarked at month 3 to avoid immortal time bias.¹⁰ The signature combined individual imaging features selected in the training set by the analysis of variable importance with Harrell concordance index as an error rate for discrimination. The radiomics signature estimated the individual probability of OS for each patient as a scale of 0 to 1 calculated at month 3. These values represented the extremum estimated prognoses (from the most to the least favorable OS). The signature was the most parsimonious model achieving an error rate less than 1% lower than the best training set model.

Model Validation

The signature was trained to estimate OS as a continuous variable and was able to estimate OS probability at all points beyond the landmarks from month 3 to year 5 after treatment initiation (eMethods 5, eTables 12 and 13, and eFigure 8 in the Supplement). Once the final signature in the training set was completed, the clinical data in this validation set were unlocked by Vol-PACT, and only 1 signature was tested in the validation set. The radiomics signature performance was evaluated by the error rate in its association with OS. This evaluation was performed using the validation set, which had not been used in any previous analysis. Patients were divided into 3 groups (trichotomized) based on the continuous output of the signature by applying the same distribution as RECIST 1.1; that is, the proportions of patients exhibiting a partial or complete response, stable disease, and progressive disease were calculated and applied to form 3 groups of that same proportion who had the best, intermediate, and worst prognosis according to the signature.

We sought to validate the performance of this signature using the area under the time-dependent receiver operating characteristics curve (AUC) for estimation of OS status at 6 months posttreatment landmark (9 months after randomization) in the validation set. The analysis at 6 months posttreatment landmark was justified by statistical considerations to provide well-balanced data sets with a significant number of events and limited censoring with longer follow-up. Our secondary objective compared the trichotomized signature with the trichotomized RECIST 1.1 results.

RECIST 1.1

The KEYNOTE images were reviewed at the Computational Image Analysis Laboratory at Columbia University Medical Center by 2 radiologists experienced in cancer imaging and blinded to other clinical outcomes, who performed tumor response assessments at each point according to RECIST 1.1 and iRECIST. Pseudoprogression was defined as an initial increase in tumor burden meeting RECIST 1.1-defined progressive disease, followed by stabilization or decreased tumor burden meeting the iRECIST definition of stable disease, partial response, or complete response. As per iRECIST guidelines, imaging beyond progression was needed to establish pseudoprogression.

Statistical Analysis

Data were analyzed from July 1, 2019, to September 15, 2021. Analyses were conducted using MatLab, version 9.5 (MathWorks), Excel, version 2019 (Microsoft Corporation), and R, version 3.6.2 (R Program for Statistical Computing). Statistical significance was defined as 2-sided P < .05 (α = .05), with Bonferroni correction for multiple comparisons. Groups were defined by categories having occurred before the month 3 landmark, and outcome events (OS) were only considered if occurring after the landmark.

Results

Imaging Features

Semiautomated tumor segmentation was successfully performed on CT scans from 575 patients (Figure 1). The 4 most frequently measured metastatic sites were lymph node (354 [53.0%]), lung (257 [38.5%]), liver (127 [19.0%]), and adrenal gland (75 [11.2%]). Details on participants and image features are shown in eTables 1 to 9 and eFigures 2 to 6 in the Supplement.

Model Development: Combining Radiomics Features Associated With OS

In the training set, a random-forest algorithm ranked the variable importance of the 50 imaging values (25 features assessed at baseline and their delta values at month 3) for the estimation of OS. Nine of the 50 features had above-average importance (eFigure 7 in the Supplement). The estimation error of models combining imaging values was evaluated (eTables 10 and 11 in the Supplement).

The final signature was a random forest for survival model with 5 variables (4 imaging variables and treatment type). The imaging variables included (1) volumetric growth (absolute tumor volume difference); (2) tumor volume; (3) absolute change in component 4, a quantitative representation of tumor spatial heterogeneity; and (4) absolute change in component 2, a quantitative representation of tumor edge phenotype. The random forest included 100 000 trees, a forest terminal node size of 15, and 3 variables tried at each split. The resampling strategy used to grow trees was sampling without replacement and log-rank split rule. The error rate of the signature was 27.7%.

The following results indicated that the 4-feature radiomics signature was best suited to estimate a patient’s OS (eTables 10 and 11 and eFigures 7-9 in the Supplement). First, adding additional features did not decrease the error rate of the radiomics signature for OS classification (signature, 27.7%; top 9 features, 26.8%; all features, 27.1%) (eTable 10 in the Supplement). Second, the 4 features were generalizable to all treatment groups. Third, the 4 features were not correlated with each other and independently evaluated distinctive imaging patterns. Fourth, component 4 was composed of strongly correlated imaging features appraising uniformity and entropy, indicators of tumor spatial heterogeneity previously associated with OS (eFigure 5 in the Supplement).^11,12,13 The association of each imaging biomarker with OS probability was computed (eFigure 8 in the Supplement).

Model Validation: Comparison of Novel Radiomics Signature With RECIST 1.1 in Patients Treated With Pembrolizumab

Our primary goal was to identify a radiomic signature that could correlate with, and possibly emerge as a surrogate for, OS. Having established a correlation with OS, we then compared the performance of the radiomics signature and that of RECIST 1.1 at month 3 in estimating OS 6 months posttreatment landmark (Figure 2 and Figure 3 and eTables 12 and 13 in the Supplement). Trichotomized RECIST 1.1 response reached an AUC of 0.80 (95% CI, 0.75-0.84), whereas the trichotomized radiomics signature significantly (P < .001) outperformed RECIST 1.1 with an AUC of 0.89 (95% CI, 0.86-0.92) and without overlap in 95% CI. The continuous signature (nontrichotomized continuous value ranging from 0 to 1) estimates outcome with even greater precision: AUC of 0.92 (95% CI, 0.89-0.95).

Figure 2. Prediction of Overall Survival (OS) Using Radiomics Signature vs Response Evaluation Criteria in Solid Tumors 1.1 (RECIST 1.1) in the Validation Set.

The regulatory end point defined progression-free survival and overall response rate. In the pembrolizumab validation set (n = 287), trichotomized RECIST 1.1 response classified patients at 3 months as having a complete or partial response (93 of 287 [32.4%]), stable disease (90 of 287, [31.3%]), and progressive disease (104 of 287 [36.2%]). The signature was trichotomized from best to worse predicted prognosis using the same distribution as RECIST 1.1 (classified patients into 3 groups, including 32.4%, 31.3%, and 36.2% of patients as cutoffs to divide the signature results into top, middle, and bottom groups). The area under the time-dependent receiver operating characteristics curve for prediction of survival status at 6 months posttreatment landmark was 0.92 (95% CI, 0.89-0.95) for the continuous signature, 0.89 (95% CI, 0.86-0.92) for the trichotomized signature, and 0.80 (95% CI, 0.74-0.84) the trichotomized RECIST 1.1.

Figure 3. Signature vs Response Evaluation Criteria in Solid Tumors 1.1 (RECIST 1.1) in the Validation (Pembrolizumab Only) Cohort.

In the pembrolizumab validation set (n = 287), we further observed that the radiomics signature provides a better prognostic separation than trichotomized response per RECIST 1.1 at month 3. We used Kaplan-Meier curves. The signature as well as the 4 features were trichotomized using the same distribution as RECIST 1.1, ranging from best to worst predicted prognosis: complete or partial response (CR/PR; 93 of 287 [32.4%]) to stable disease (SD; 90 of 287 [31.3%]) and progressive disease (PD; 104 of 287 [36.2%]). This comparison was intended to show that the radiomics signature is unambiguous and parsimonious and provides a better separability of classification. Shaded areas indicate 95% CIs.

Kaplan-Meier curves further demonstrate that the radiomics signature provides better prognostic separation than the trichotomized RECIST 1.1 response at the same point. The radiomics signature as well as the 4 features were trichotomized (to facilitate comparison to distribution of trichotomized RECIST 1.1) ranging from best to worst estimated prognosis: complete or partial response (93 of 287 [32.4%]), stable disease (90 of 287 [31.3%]), and progressive disease (104 of 287 [36.2%]) (Figure 3 and eFigure 10 in the Supplement). Overall, our results suggest that the radiomics signature is unambiguous and parsimonious and provides a better separability of classification.¹⁴

eFigure 11 in the Supplement shows that although partial and complete responses constitute a homogeneous category, the RECIST 1.1 stable and progressive disease categories are extremely heterogeneous in terms of the signature-estimated OS. Hence, the signature redefines stable and progressive disease as a prognostic spectrum. Finally, 9 of 11 patients (81.8%) whose tumors were categorized as pseudoprogression at month 3 by iRECIST had a month 3 signature favorable for OS (eFigures 11-13 and eTable 4 in the Supplement).

Discussion

Using CT scans acquired at multiple institutions in 2 multicenter clinical trials, we have identified a radiomics signature that can estimate OS among patients with a diagnosis of unresectable melanoma. The signature can estimate OS outcomes early and with further validation could be deployed for clinical use.

The signature quantifies an imaging phenotype in all measurable tumors. Among all candidate features, 4 were selected for the signature. Two of these, tumor volumetric growth at month 3 and tumor volume at month 3, relate to tumor burden. Although tumor volume affords a more accurate estimate of burden than diameter, selection of these features by machine learning reaffirms that changes in tumor burden are an underlying biological factor in the response to therapy and in turn in overall survival. Two other features, component 4 (eFigure 5 in the Supplement) and component 2 (eFigure 4 in the Supplement), characterize changes in tumor spatial heterogeneity and texture. Heterogeneity has been linked with poor outcome,^11,13,15,16 and CT scans noninvasively assess tumor spatial heterogeneity as density. Prior work^11,13 suggests that heterogeneity is similar to tumor size in providing a sensitive indicator of response to therapy across cancer types. Additional work is needed to determine whether these features might actually be surrogates to characterize changes in the lesion’s edge, which appear to be indicators of melanoma response to immune checkpoint inhibitors and could reflect changes in the tumor microenvironment, a putative mechanism of immunotherapy action. Of note, previous clinical trials^11,13 and studies in a research setting^12,17 have demonstrated that these features are relatively robust.

Patients with an objective RECIST 1.1 response unsurprisingly have favorable predicted OS using the radiomic signature. By identifying new reproducible features combined with tumor burden, our signature substantially improves estimation of OS. Nine of 11 patients with initial RECIST 1.1 progressive disease were ultimately classified as having pseudoprogression and were correctly classified by their month 3 signature as likely to have a favorable estimated outcome (eFigure 13 in the Supplement). This ability to discriminate among patients with progressive disease at month 3 could allow physicians to discuss alternative treatments earlier, appropriately set goals of care, or look for clinical trials. In clinical trials, earlier efficacy markers could help to identify promising drugs or combinations long before OS is reliably estimated. RECIST 1.1 provides an assessment of efficacy for many therapies, but is an imperfect, often poor surrogate of OS.⁴ With regard to immunotherapy, iRECIST and related response criteria stress the need for subsequent scans to confirm progression. Although possibly important for some patients, data from other studies have suggested that pseudoprogression is rare at 2.8%.¹⁸

Strengths and Limitations

Our work offers several advances compared with prior literature.^19,20,21 Although previous studies have relied on small data sets collected at single institutions, our results using data from diverse medical centers participating in international clinical trials suggest that our model can successfully be translated into clinical practice across similar heterogeneities in patient population and image acquisition parameters. Previous work^19,20,21 has shown that radiomics could identify melanoma tumors sensitive to immunotherapy based on pretreatment imaging signature (eTable 1 in the Supplement). By selecting 1 pretreatment and 3 treatment features, our signature suggests that a model that takes a single feature into account cannot perform as well as a model that combines features and that the value of pretreatment features can be augmented by treatment features. We followed the TRIPOD reporting guideline, including benchmarking our results against the current standard of care for response assessment in melanoma, RECIST 1.1. Our signature performed better than RECIST 1.1 for outcome estimation (Figure 2). Additional advancements include analyzing all measurable lesions to take into account total tumor burden, reducing the dimensionality of imaging features to a limited number of components to avoid overfitting, and estimating patients’ OS as a continuous censored variable.

This study is limited by its retrospective approach, although data were collected prospectively. Both radiomics and RECIST 1.1 rely on the existence of measurable tumor lesions on a CT scan. Dercle et al⁷ reported previously that clinically annotated imaging data can be extracted in 8 of every 10 patients with solid tumors enrolled in clinical trials, and outcomes in this subgroup of patients remained representative of the entire cohort of patients. Another limitation that may limit the generalizability of our results to other cancers is the focus exclusively on patients with unresectable melanoma. Melanoma was an excellent model because follow-up in these trials is long, and concepts of immune-related response criteria are most extensively described in patients with melanoma. Nonetheless, 3 of the 4 features identified in this analysis were also identified in an ancillary study in patients with colorectal cancer from the Vol-PACT consortium.⁷ Last, our analysis is landmarked at month 3, meaning that the OS of patients can be estimated by early changes between baseline and month 3. We chose to compare the performance of the signature to RECIST 1.1 for the estimation of OS at month 6 posttreatment landmark (month 9 after randomization). Different points could have been selected, but it is important to stress that the signature was trained and validated to estimate OS as a continuous variable. Hence, the signature is able to estimate OS probability at all points (Figure 3) by combining pretreatment and treatment information.

Conclusions

The radiomics and machine learning prognostic model described herein could be widely translated into clinical practice because it only requires the identification and segmentation of target lesions on routine regularly performed CT scans applying publicly available software and a clinical decision support tool. The time required for manual lesion segmentation (approximately 1 minute per lesion per scan) limits the clinical application of our signature, but deep-learning techniques for the automatic detection and segmentation of lesions are being developed²² that will allow widespread performance of this task automatically after acquisition of CT scans. Once in clinical use, our signature could give clinicians in practice an early readout of the likelihood of success when treating patients with advanced melanoma with pembrolizumab. In clinical trials, this new approach could complement other response assessment metrics and increase the efficiency of drug discovery for cancer.

Supplement.

eMethods 1. Collection of Annotated Images in the Vol-PACT Consortium

eMethods 2. Computation of Imaging Features and Clinically Annotated Images

eMethods 3. Twenty-five Uncorrelated Imaging Features at Baseline and Month 3

eMethods 4. Development of the Signature in the Training Set: Ranking the Importance of Variables to Forecast Overall Survival (OS)

eMethods 5. Clinical Validation of the Signature in the Validation Set

eTable 1. State-of-the-Art on the Use of Radiomics in Patients With Cancer Treated With IO

eTable 2. Clinical Trials Characteristics: Hazard Ratio in Vol-PACT vs Original Phase III Clinical Trial Results

eTable 3. Comparison of Reported Patients, Received Source Imaging Data and Measured Tumor Burden

eTable 4. Pseudoprogression in Patients Treated With Pembrolizumab

eTable 5. Kaplan Meier Estimate of OS in the Overall Population for Each Treatment Group

eTable 6. Distribution of Tumor Imaging Phenotype in Patients on Baseline CT Scans for 10th to 90th Percentiles in the Overall Population (n = 575)

eTable 7. Distribution of Change in Tumor Imaging Phenotype in Patients From Baseline to Month 3 CT Scans for 10th to 90th Percentiles in the Overall Population (n = 575)

eTable 8. Distribution of Tumor Imaging Phenotype in Patients on baseline CT Scans for Each Treatment Type

eTable 9. Distribution of Change in Tumor Imaging Phenotype in Patients From Baseline to Month 3 CT Scans for Each Treatment Type

eTable 10. Error Rate as a Function of Node Size in the Training Set

eTable 11. Importance of Variables

eTable 12. Performance of the Signature in the Validation Set Including Ipilimumab, Pembrolizumab, and Chemotherapy (n = 416)

eTable 13. Kaplan Meier Estimate of median OS and OS Probability Landmarked at 3 Months in the Pembrolizumab Validation Set

eFigure 1. Machine-Learning Workflow

eFigure 2. Sites of Segmented Lesions: Analysis per Lesion (12 077 Lesions)

eFigure 3. Principal Component Analysis

eFigure 4. Contribution of Radiomic Features to Component 2

eFigure 5. Contribution of Radiomic Features to Component 4

eFigure 6. Contribution of Radiomic Features to Component 11

eFigure 7. Importance of Variables

eFigure 8. Standardized Mortality as a Function of Features Included in the Signature

eFigure 9. Correlation Between the Signature and Delta Tumor Burden

eFigure 10. The Signature Outperforms RECIST in the Validation (Pembrolizumab Only)

eFigure 11. Waterfall Plot Displaying the Signature Value in 4 Response Categories in Patients Treated With Pembrolizumab

eFigure 12. Box Plot Displaying Tumor Imaging Phenotype at Baseline and Month 3 According to Patients’ Best Overall Response

eFigure 13. On a Patient Level, the Radiomics Signature Predicts Where the Patient Is in the Prognostic Spectrum Based Upon Change in Tumor Phenotype During Treatment

eReferences