Abstract
Tumor vasculature is a key component of the tumor microenvironment that can influence tumor behavior and therapeutic resistance. We present a new imaging biomarker, quantitative vessel tortuosity (QVT), and evaluate its association with response and survival in patients with non–small cell lung cancer (NSCLC) treated with immune checkpoint inhibitor (ICI) therapies. A total of 507 cases were used to evaluate different aspects of the QVT biomarkers. QVT features were extracted from computed tomography imaging of patients before and after ICI therapy to capture the tortuosity, curvature, density, and branching statistics of the nodule vasculature. Our results showed that QVT features were prognostic of OS (HR = 3.14, 0.95% CI = 1.2 to 9.68, P = 0.0006, C-index = 0.61) and could predict ICI response with AUCs of 0.66, 0.61, and 0.67 on three validation sets. Our study shows that QVT imaging biomarker could potentially aid in predicting and monitoring response to ICI in patients with NSCLC.
Tumor vasculature imaging biomarker can potentially aid in predicting and monitoring response to ICI in patients with NSCLC.
INTRODUCTION
Immune checkpoint inhibitors (ICIs) have revolutionized the treatment paradigm in non–small cell lung cancer (NSCLC) (1, 2) and are now the standard of care either alone or in combination with chemotherapy as the first-line therapy in treatment-naïve patients and as the second-line therapy in chemotherapy-refractive patients (3). As of 2018, almost every patient with advanced NSCLC without targetable mutations is treated with ICI either in the first-line setting or as subsequent lines of therapy. First-line platinum-based cytotoxic chemotherapy for patients with advanced NSCLC produces unstable responses at best (4, 5).
Landmark clinical trials leading to the approval of ICIs in NSCLC have demonstrated an association between Programmed cell death ligand 1 (PD-L1) expression and response to ICI; hence, PD-L1 tumor expression levels are used as routine clinical predictive biomarker for deciding the regimen of anti–PD-L1 immunotherapy (IO) (6). However, these trials demonstrated that some patients with low PD-L1 expression can still have clinical responses with these agents. This has now led to adding chemotherapy to ICIs in patients with PD-L1 tumor expression less than 50% and ICI monotherapy for PD-L1 greater than 50% as the standard of care. However, response rates to ICI monotherapy remain modest (27% in PD-L1–positive NSCLC in the first-line setting, 45% in PD-L1–high subgroup, and 19% in the second-line setting) (7). There is thus an unmet clinical need for biomarkers to identify patients who are most likely to benefit from ICIs and determine the potential nonresponders who can be spared both the financial cost and side effects of IO (8, 9). While exploratory markers such as tumor mutational burden (10) are now being evaluated, there continues to be a need for identifying noninvasive biomarkers, both for predicting and monitoring response to ICIs.
Radiomics refers to the process of image analysis that results in high-throughput extraction of subvisual and quantitative features from radiologic scans including x-rays, computed tomography (CT), ultrasound, and magnetic resonance imaging. Recently, radiomic approaches have been applied in the context of prognosticating outcome and predicting response to IO (11–14). Most of these studies have analyzed the tumoral shape and textural radiomic features for predicting response and outcome in patients with NSCLC. For example, the authors in (15) presented a method involving changes in the textural radiomic features of CT images to predict overall survival (OS) and response to IO in patients with NSCLC. Trebeschi et al. (11, 16) showed the utility of radiomic features in predicting response to IO and patient outcomes in metastatic NSCLC.
Tumor vasculature is a key component of the tumor microenvironment that can influence invasiveness, metastatic potential, and therapeutic refractoriness. Cancer cells encourage the growth of blood vessels to feed the tumor by producing vascular endothelial growth factor, thus creating an immune-excluded phenotype of tumors (17, 18). In the immune-excluded phenotype, there are underlying mechanical or chemical barriers between infiltrating lymphocytes and the tumor site in which antiangiogenesis or particular antistromal therapy might be of benefit in enhancing the efficacy of IO (19). A promising strategy in anticancer therapy is tumor blood vessel normalization. Many studies showed how less twisted vessels are able to counteract metastasis formation and favor chemotherapeutic drug delivery to tumors (20). In addition, it has been demonstrated that aberrant vessel morphology potentiates treatment resistance and lack of durable therapeutic response by reducing drug transfer to the tumor bed (21).
In this work, we present and validate quantitative vessel tortuosity (QVT), a new imaging biomarker for predicting the response and outcome prognosis of patients with NSCLC treated with ICI therapies. We hypothesize that the tumor vasculature is more twisted in nonresponders compared to responders to ICI. In addition, we hypothesize that the vasculature twistedness on nonresponders to ICI causes antitumor T cells to accumulate at the tumor site but fail to efficiently infiltrate the tumor accounting for therapeutic refractoriness. In this study, we sought to evaluate our QVT imaging biomarker on a total number of 507 NSCLC cases in terms of predicting response, monitoring response, and prognosticating outcome. Toward this end, we used contrast-enhanced CT scans from 162 patients with advanced NSCLC before and after two to three cycles of PD-1/PD-L1 ICI therapy from three sites. The association of QVT and PD-L1 expression (PD-L1low and PD-L1high) was evaluated using a set of 204 patients with early-stage NSCLC. We analyzed the QVT association with gene set enrichment analysis (GSEA) pathways on a set of 92 patients with early-stage NSCLC with available RNA sequencing data. In addition, we evaluated the prognostic and predictive potential of QVT on a set of 45 patients with NSCLC who underwent a combination of ICI and chemotherapy.
RESULTS
Experiment 1: Association of baseline QVT features with response to ICI and OS
A linear discriminant analysis (LDA) machine learning classifier was used to determine the ability of the selected QVT features in discrimination of patients into responders and nonresponders to ICI. The response prediction classifier yielded an area under the curve (AUC) of 0.74 [95% confidence interval (CI) = 0.73 to 0.75] on the training (D1) and corresponding AUCs of 0.66 (95% CI = 0.58 to 0.81), 0.61 (95% CI = 0.56 to 0.78), and 0.67 (95% CI = 0.59 to 0.88) on D2, D3, and D4 validation sets, respectively, for patients treated with ICI monotherapy. The classifier’s performance on predicting response for patients treated with different ICI agents is illustrated in Table 1. In addition, using a model trained with QVT features on D1 (ICI monotherapy), the model had an AUC = 0.64 (95% CI = 0.57 to 0.82) in predicting response to chemoimmunotherapy on D7. Figure 1A shows a feature expression cluster heatmap of the most discriminating QVT features for responder and nonresponder patients in the training set (D1). As may be observed, a number of baseline QVT features showed statistically significant differential expression between responders and nonresponders in D1.
Table 1. Multiagent response prediction analysis (with metric AUC) of QVT features on validation sets.
AUC is reported for two models including the baseline QVT model trained with pretreatment scans and the delta QVT model trained with both pre- and posttreatment scans.
| Dataset | AUC on D2 | AUC on D3 | AUC on D4 | |||
| Baseline | Delta | Baseline | Delta | Baseline | Delta | |
| All agents | 0.66 | 0.92 | 0.61 | 0.85 | 0.67 | – |
| Nivolumab | 0.67 | 0.90 | 0.77 | 0.85 | 0.62 | – |
| Pembrolizumab | 0.90 | 0.73 | 0.63 | 0.72 | 0.95 | – |
| Atezolizumab | – | – | 0.65 | – | – | |
Fig. 1. Association between delta QVT features, response to ICI, and OS.
(A) Unsupervised clustering of QVT features and patients revealed two dominant patient groups with high- and low-risk groups. High-risk group is associated and aligned with nonresponders to ICI. (B to E) Kaplan-Meier survival curves represent a significant difference in OS between patients with low and high QRS on D1, D2, D3, and D7 sets. (F) ROC curves of response prediction model trained with QVT (AUCs of 0.66 and 0.61 on D2 and D3, respectively) and delta QVT (AUCs of 0.92 and 0.85 on D2 and D3, respectively).
Seven stable baseline QVT features were found to be prognostic of OS. These features corresponded to statistics of vasculature curvature, tortuosity, branching, and distribution of acute and obtuse angles measured from each of the three consecutive points on the vessels’ centerline. The QVT risk score (QRS) stratified patients into high- and low-risk groups in D1 with P = 0.0006, hazard ratio (HR) = 3.14 (95% CI = 1.2 to 9.68), and C-index = 0.61. The QRS was found to be prognostic in three validation sets (D2, D3, and D4) with HR = 2.49 (0.95% CI = 1.17 to 5.32), P = 0.002, and C-index = 0.62 (difference of median OS = 10 months) in D2; HR = 2.12 (0.95% CI = 1.04 to 4.29), P = 0.014, and C-index = 0.61 (difference of median OS = 14 months) in D3; and HR = 2.98 (0.95% CI = 1.12 to 7.93), and P = 0.04 (difference of median OS = 6.3 months) in D4. The multivariable OS analysis results for the QVT features are shown in Table 2. In addition, a Cox proportional hazards analysis yielded HR = 2.22 (95% CI = 1.38 to 3.6), P = 0.001, and C-index = 0.7 in predicting OS for patients in D7 who underwent a combination of ICI and chemotherapy. A multivariable analysis with combination of PD-L1 expression and QRS revealed that QRS is the only variable that is significantly associated with OS in D7 [QRS: HR = 2.17 (95% CI = 1.33 to 3.55), P = 0.001; PD-L1: HR = 1.47 (95% CI = 0.46 to 4.7), P = 0.51, and C-index = 0.71]. Figure 1 (B to E) illustrates a Kaplan-Meier estimation of OS via QRS in low- and high-risk patients in each of D1, D2, D3, and D7.
Table 2. Multiagent OS analysis of QVT features on validation sets.
| Dataset | D2 (N = 50) | D3 (N = 27) | D4 (N = 23) |
| Nivolumab | HR = 2.64 (1.37–5.1) |
HR = 2.08 (1.12–4.45) |
HR = 3.11 (1.17–8.28) |
| P = 0.00867 | P = 0.021 | P = 0.0368 | |
| Pembrolizumab | HR = 2.08 (1.912–4.75) |
HR = 2.27 (1.969–5.3) |
– |
| P = 0.026 | P = 0.00987 | ||
| Atezolizumab | – | – | – |
Experiment 2: Association between delta QVT features, response to ICI, and OS
The ICI response prediction model trained with delta QVT features yielded an AUC of 0.92 (95% CI = 0.84 to 0.97) and 0.85 (95% CI = 0.71 to 0.99) on D2 and D3, respectively. Figure 1F shows the receiver operating characteristic (ROC) curve of the response prediction models trained with baseline and delta QVT features. The multivariable response prediction results of delta QVT features are illustrated in Table 1.
Looking at the distribution of angles measured from any three consecutive points of the vasculature, we observed that in the pretreatment scans of the responders to therapy, the distribution of obtuse angles was almost doubled compared with nonresponders, which means that responders primarily consisted of less tortuous vessel branches (Fig. 2, A and B). Section SA gives more details on the QVT features. Analysis of the changes in vessel tortuosity between pre- and posttreatment scans revealed that the number of acute angles associated with the vessels was significantly reduced after treatment in responders, while these acute angles remained nearly the same or increased after treatment in nonresponders (Fig. 2, C and D). Figure 3 illustrates three-dimensional (3D) tortuosity and curvature maps for responders and nonresponders between pre- and posttreatment scans. The mean curvature and torsion values for the responder case decreased after treatment with respect to pretreatment scan, while the same feature increased after treatment for the nonresponder case. In addition, QVT features f7, f8, f11, and f12 corresponding to statistics of vessel curvature were found to be significantly different between responders and nonresponders in terms of both baseline and delta QVT features (fig. S5, A and B). Moreover, the distribution of acute angles (f68) for baseline and curvature statistics (f9 and 10) and distribution of obtuse angles (f52) of delta QVT features were also significantly different between responders and nonresponders to ICI. The complete description of features can be found in the Supplementary Materials. Delta QVT features were found to be prognostic of OS on D2 and D3 with HR = 2.64 (95% CI = 1.37 to 5.1) and P = 0.008 and HR = 0.245 (0.0647 to 0.925) and P = 0.0006, respectively.
Fig. 2. Responder tumor to ICI has less tortuous vessel branches compared to nonresponder.
(A and B) CT scans of a responder and nonresponder to IO with corresponding 3D rendering of the tumors and the corresponding vasculature. (C and D) The distribution of angles between any three consecutive points of the vasculature for all responder (C) and nonresponder patients (D), which was measured for both pretreatment (blue bars) and posttreatment (red bars) CT scans. Angles were binned from 1 to 15 with 1 referring to the most obtuse (wide) angles and 15 referring to the most acute angles.
Fig. 3. 3D tortuosity and curvature maps for responder and nonresponder before and after treatment.
Baseline and posttreatment scans of a responder and nonresponder to ICI are illustrated in the left. Second column from the left represents the 3D renderings of the tumors and associated vasculature. Third and fourth columns show the curvature and tortuosity maps of the vasculature that reflects the regional curvature of vessels in 3D and the extent of convolutedness of each vessel, respectively. The mean curvature and torsion values for the responder case decreased after treatment with respect to the pretreatment scan, while the same feature increased after treatment for the nonresponder case.
Experiment 3: Molecular, histological, and radiogenomic underpinning of QVT
Three QVT features were found to be strongly associated with PD-L1 expression. As shown in Fig. 4 (A to C) , statistically significant differences were found between QVT features of the PD-L1low and PD-L1high groups (P < 0.002). Moreover, significant difference was found between QRS features of the low– and high–PD-L1 groups (P = 0.0023). In addition, the QVT features were found to be strongly associated with tumor-infiltrating lymphocyte (TIL) density on hematoxylin and eosin (H&E) images of baseline biopsy scans.
Fig. 4. Molecular, histological, and radiogenomic underpinning of QVT features.
(A to C) Statistically significant differences were found between QVT features of the low–PD-L1 and high–PD-L1 groups. The TIL grouping factor histomorphometric feature (F3) was found to be statistically significantly (r = −0.56 and P = 0.001) correlated with QVT1 and QVT6 both referring to the distribution of the most acute angles on the vasculature. (D) The correlation matrix between TIL and QVT features. TIL and QVT features arranged in rows and columns, respectively.
The mean of the TIL grouping factor was found to be statistically significantly correlated with a QVT_1 feature (r = −0.56 and P = 0.001). The same feature also showed moderately high correlation with QVT (QVT_6) feature. Both QVT features refer to the distribution of the acute angles of the vasculature in which QVT_1 and QVT_6 refer to the distribution of the acute angles within the range of 1° to 12° and 61° to 72°. Figure 4D shows the corresponding correlation matrix. The matrix represents TIL and QVT features arranged in rows and columns, respectively. The correlation value between each pair of TILs and QVT features is presented as circles in the corresponding matrix element.
We found that wingless-integrated (WNT) signaling pathway was up-regulated in the high-QVT phenotype group. In addition, blood vessel morphogenesis and the fibroblast growth factor (FGF), an angiogenic growth factor (22), pathway were found to be up-regulated in the high-QVT phenotype group.
DISCUSSION
Recent evidence indicates that angiogenesis, lymphangiogenesis, and the tumor environment have important immunomodulatory roles that contribute to the immune evasion of tumors (17). It has been also observed that angiogenesis, invasion, and vessel proliferation might be important regulators of PD-L1 expression, given the association of these processes with malignant progression (23). It is well acknowledged that most tumors trigger an immune response modulated by TILs. Previous studies have reported an association between higher density of TILs in patients and favorable responses to IO (24, 25). It has been shown that the tumor vasculature can actively suppress antitumor immune responses (26), and expression of vascular adhesion molecules in blood vessels correlates with the TIL density in the tumor microenvironment. The structurally and functionally aberrant tumor vasculature contributes to the protumorigenic and immunosuppressive tumor microenvironment by maintaining a cancer cell’s permissive environment characterized by hypoxia, acidosis, and high interstitial pressure while simultaneously generating a physical barrier to T cell infiltration. Recent research has also shown that blood endothelial cells forming the tumor vessels can actively suppress the recruitment, adhesion, and activity of T cells. In this work, we present a noninvasive quantitative measurement of vessel twistedness or tortuosity as a novel imaging analytic technique to evaluate the association between vessel convolutedness, response to ICI therapy, and OS of patients with advanced NSCLC.
A few studies have investigated the role of radiomic texture features of the nodule on CT scans in predicting response to different therapies. Authors in (13) found that the skewness of the intensity histogram measured in Hounsfield units and relative volume of air in the segmented tumor were associated with treatment response. They also found that response to IO was correlated negatively with the tumor convexity and positively with the edge–to–core size ratio. Trebeschi et al. (11) used radiomic features on contrast-enhanced CT scans to predict responses to IO in patients with metastatic NSCLC treated in a second-line setting. Their results showed that more heterogeneous tumors with irregular patterns of intensities have a better OS. In another study, Tang et al. (12) presented an approach for developing a predictor of OS in patients with NSCLC based on a pathology-informed radiomic model. Sun et al. (14) showed a radiomic approach to assess tumor-infiltrating CD8 cells and response to anti–PD-1 or anti–PD-L1 IO.
Our work differs from these previous approaches in that it involves using mathematical measurements from nodule vasculature (QVT) to predict OS and distinguish responders from nonresponders in patients with NSCLC treated with ICI, as opposed to radiomic texture–based measurements. Because ICIs work by modulating the PD-L1 axis, we also investigated the association of QVT features with PD-L1 expression in early-stage ICI-naïve patients and found that QVT was strongly associated with PD-L1 expression. Our results show that most acute angles of the blood vessel vasculature are also inversely associated with TIL expression. This is in line with favorable responders to ICI having higher TIL expression and corresponding lower QVT. Moreover, radiogenomic analysis of the QVT features showed that tumors with a highly tortuous vasculature are associated with WNT signaling pathway, blood vessel morphogenesis, and the FGF pathway. Activation of WNT signaling pathway has been shown to correlate with immune exclusion (27) across human cancers, which are usually more aggressive cancers often with refractory to therapy.
This study is the first to demonstrate that tumor vessel tortuosity measurements extracted from routine contrast CT images are associated with response to ICIs and prognostic of OS in patients with metastatic NSCLC treated with ICI. Our group has also previously shown the utility of an initial version of QVT features in distinguishing benign and malignant nodules in patients with NSCLC (28). The QVT biomarker was validated in two independent test sets, accrued from three different sites and across three different IO agents. In addition to ICI monotherapy, we showed the potential of QVT to be predictive of response and prognostic of survival in patients treated with first-line combination chemotherapy + IO. This study uniquely provides a cellular, molecular, and genomic underpinning for the imaging-derived QVT features that were identified to be associated with response to ICI therapy. We also assessed the stability of QVT features in test-retest scans and then measured their stability against segmentation errors. As illustrated in section SB, 22 of 74 QVT features were found to be moderately stable with an intraclass correlation coefficient of >0.4. The number of stable features dropped from 22 to 19 features when we added segmentation-associated noise. We additionally measured the performance of our models as a function of CT slice thickness parameter. As may be observed in the Supplementary Materials, the AUC of QVT classifier decreased slightly with increasing slice thickness.
We do acknowledge, however, that our study did have its limitations. The size of ICI-treated cohorts both for training and validation was relatively small. Second, the study was completely retrospective in nature. Independent prospective validation is needed to show the prognostic and predictive utility of QVT on patients with NSCLC treated with IO. In addition, the association of QVT and TIL features could only be done on a small subset of cases due to nonavailability of tissue for pathomic analysis. The radiogenomic analysis too could only be carried out on a different cohort of early-stage patients due to lack of tissue on the ICI cohort for mRNA sequencing due to it being retrospective in nature.
Despite the limitations, QVT could potentially serve as a tool for monitoring and predicting response to ICI and help identify patients with NSCLC who are likely to benefit from IO. It enjoys several advantages over currently deployed biomarkers in not only being noninvasive but also being inexpensive, derived from routine radiographic imaging, and nondisruptive of normal clinical workflow. The tumor vasculature not only was able to predict and monitor tumor behavior and response to ICI but also could predict response in patients treated with the combination of chemotherapy and IO. In future work, we will seek to validate QVT in the context of prospective biomarker-driven clinical trials.
MATERIALS AND METHODS
Datasets
This Health Insurance Portability and Accountability Act of 1996 (“HIPAA”) regulations–compliant study was approved by the institutional review board (IRB 02-13-42C) at the University Hospitals Case Medical Center, and the need for informed consent was waived. A total of 507 NSCLC cases were included in this multisite validation study to explore the various aspects of our imaging biomarker. We studied the association of baseline and delta QVT features (defined as the absolute change in QVT features between baseline and posttreatment scans) with response to therapy and OS of the patients with NSCLC treated with ICI and a combination of ICI and chemotherapy. We also studied the molecular, cellular, and radiogenomic underpinning of QVT features by exploring the association of QVT with biological pathways, PD-L1 expression, and TIL density in H&E images. To predict and monitor treatment response and prognosticating outcome in ICI-treated patients, N = 162 contrast CT scans of patients with advanced NSCLC before and after two to three cycles of PD-1/PD-L1 ICI therapy (nivolumab/pembrolizumab/atezolizumab) were included. Samples in which the board-certified radiologist could not isolate a measurable pulmonary nodule on CT scans or the CTs with poor image quality were excluded. The ICI-treated cohort from three institutions was divided into four subsets including D1 = 62 for training and D2 = 50, D3 = 27, and D4 = 23 for validation. A set of 112 patients from January 2012 to August 2017 at the Cleveland Clinic Foundation (CCF) was included consecutively, and patients were divided to D1 = 62 for training and D2 = 50 for internal validation set. Moreover, D3 = 27 patients continuously admitted from 2014 to 2017 at the University of Pennsylvania Health System were identified and used as the first independent test set. In addition, D4 = 23 patients from 2018 to 2020 at the University Hospitals Cleveland Medical Center were included in this study as the second independent test set. All patients underwent a baseline contrast CT scan before starting treatment with ICIs. Posttreatment scans were available only for D1, D2, and D3.
An overwhelming majority of the cohorts of patients were treated in an era when ICIs were approved only in the second-line setting at which point PD-L1 expression was not routinely performed for all patients with NSCLC. Because the prescription of PD-1/PD-L1 inhibitors in this setting does not mandate PD-L1 quantification (except pembrolizumab), many of the patients in these cohorts received nivolumab or atezolizumab without prior PD-L1 testing. To assess whether QVT features are associated with PD-L1 expression, a separate cohort of D5 = 204 patients with early-stage NSCLC between April 2004 and April 2015 with available diagnostic CT scans from CCF was included in this study. For QVT and TIL density associative analysis, we used a subset of D1 (N = 31) cases (digitized histology scans of baseline biopsies were only available for 31 cases from CCF). A dataset of D6 = 92 patients with early-stage NSCLC from The Cancer Imaging Archive (TCIA) with available RNA sequencing data was included for the radiogenomic analysis. The prognostic and predictive potential of QVT biomarkers for predicting response in a combination of ICI and chemotherapy was evaluated on a set of D7 = 45 patients with nonsquamous NSCLC who underwent ICI (pembrolizumab) and chemotherapy (pemetrexed) between October 2015 and August 2018 at CCF. Figure 5 illustrates the data inclusion strategy for the various experiments that comprised this study.
Fig. 5. Data inclusion and experimental workflow.
Demographics and clinical variables
Eastern Cooperative Oncology Group performance status and tumor node metastasis stage and clinical staging per the American Joint Committee on Cancer staging system were used in this study alongside clinical variables including age, sex, and tumor histology. All patients (except D5 and D6) included in this study had metastasis and so were classified into stage IV. Demographics and clinical characteristics for patients were available for D1, D2, D3, and D4 datasets and are summarized in table S5. None of the following clinical features including gender, race, smoking status, histology subtype, and epidermal growth factor receptor mutation status were found to be prognostic of OS.
Image acquisition, nodule detection, and vasculature segmentation
CT scans were acquired from all ICI-treated patients at baseline and immediately after two to three cycles (6 to 8 weeks) of ICI treatment for D1, D2, and D3. Scans were acquired using a multislice (Philips Healthcare, General Electric Health Care, Siemens Healthcare) CT system with a tube voltage of 100 to 120 kilovolt peak, slice thickness (spacing) of 1 to 5 mm (mean = 2.82 mm and SD = 0.71 mm), and in-plane resolution of 0.75 × 0.75 mm. All CT images were captured with patients in inspiration breath-hold phase after contrast injection. All scans were acquired using the facilities’ CT chest protocol and standard image reconstruction (15). In the case of multifocal nodules, the primary nodule was selected according to the radiology report at baseline and tracked and delineated in the posttreatment with 3D SLICER software by a board-certified cardiothoracic radiologist (with 8 years of experience).
Vascular feature extraction
The manually segmented target nodules were used to compute the volume of interest (VOI) and subsequently segment the nodule-associated vasculature and extract vascular features. Lung regions were automatically isolated from the surrounding anatomy using a multithreshold-based algorithm (29). The vasculature within the lung regions was segmented from lung parenchyma by applying a vessel enhancement filter followed by a multithreshold algorithm. The VOI is defined as a rectangular prism region that has the nodule in the center. The size of the VOI is defined relatively with respect to the size of the nodule. A region growing algorithm was used for the segmentation of the nodule vasculature (30) within the VOI. A fast-marching algorithm (31) was then used to identify the centerlines of the 3D segmented vasculature. Figure 6 illustrates the process of vasculature segmentation. A set of 74 QVT features were measured from points, branches, and the entire vasculature centerlines. These features pertain to the tortuosity, curvature, and branching statistics as well as the volume of the vasculature. Curvature at a point on the vascular centerline segment is measured by fitting a circle that approximates the shape of the segment the best. The tortuosity of a vascular segment is measured as the ratio of its centerline length with respect to the length of a straight line that connects the starting and ending points of its centerline. In addition to the initial set of QVT features that the authors previously presented (28), the angles of any three consecutive points of the vasculature were measured, and the distribution of these angles was dichotomized into 15 bins. We also assessed the stability of QVT features in test-retest scans and then measured their stability against segmentation errors. Additional details of stability analysis and sensitivity of QVT features to CT parameters are provided in sections SB and SC.
Fig. 6. The main workflow of QVT feature extraction.
(A) Identifying tumor position by a radiologist. (B) Segmentation of nodule and lung regions. (C) Vasculature segmentation. (D) Identifying nodule-associated vasculature. (E) Extraction of the vessel’s centerlines. (F) Extraction of QVT features from centerlines.
Statistical analysis
Classification
The primary endpoint of this study was primary clinical response defined by response evaluation in solid tumors (RECIST) v1.1. Patients who did not receive ICI after two cycles due to lack of response or progression as per RECIST were classified as “nonresponders,” and patients who had radiological response or stable disease as per RECIST and clinical improvement were classified as “responders.” An LDA classifier was trained on D1 with the stable and discriminating vascular features to predict the RECIST-based response. Within the discovery set D1, the classifier was trained in a threefold cross-validation setting. The procedure was iterated over 200 runs. The performance of the response prediction classifier was assessed with the ROC as AUC. In addition, an unsupervised hierarchal clustering analysis (using the clustergram function in MATLAB) was conducted on QVT features (15).
Survival analysis
The secondary endpoint of this study was OS, which was defined as the time from the date of the disease diagnosis until the date of death (or until the date that the patient was last known to be alive if censored). The median follow-up of OS posttreatment was 16 months (range, 1 to 45 months). The Kaplan-Meier survival analysis and log-rank statistical tests were performed to assess the univariable discriminative ability of the features on OS (32). The prognostic value of vascular features on OS was estimated by using the QRS. To build the multivariate signature for OS, the least absolute shrinkage and selection operator (LASSO) Cox regression model (33) was used to identify the prognostic features from the subset of 74 stable QVT features in the training set (D1). A QRS was computed for each patient according to a linear combination of selected features with corresponding nonzero coefficients from the LASSO Cox model in the training set (D1). On the basis of the cutoff value of QRS on D1, the patients on D2, D3, and D4 were stratified into high- and low-risk groups. A multivariable Cox proportional hazards model was used to evaluate the ability of the QRS in predicting OS. In addition, relative HRs with 95% CI were calculated. The median follow-up was also estimated with the reverse Kaplan-Meier method (15, 34).
Association of QVT with PD-L1 expression
Correlation analysis of QVT features with PD-L1 expression was also performed. In this regard, PD-L1 > 50% was used as cutoff value to divide patients in D5 into PD-L1low and PD-L1high groups. The Wilcoxon rank sum significance test was then performed on QVT features between PD-L1low and PD-L1high groups to evaluate whether there were significant differences between QVT feature and PD-L1 level expression. All tests were two-sided, and P values less than 0.05 were considered statistically significant.
Association of QVT with TIL density on digital pathology images
For QVT and TIL density associative analysis on subset of 31 cases from D1, we used an automated detection of TILs in H&E images (35) followed by computational spatial clustering metrics. A watershed-based algorithm (36) was first applied to segment nuclei on the image. Considering that lymphocyte nuclei are generally distinguished from other cell nuclei by their smaller size, more rounded shape, and a darker homogeneous staining, we classified the segmented nuclei into either lymphocytes or nonlymphocytes (mainly, tumor cells) using nuclei texture, shape, and color features (37). Twelve features quantifying the density or compactness of TILs were extracted from the surgical specimens. Each lymphocyte is characterized by its own local morphological feature and by a set of contextual features to describe the lymphocyte and its neighborhood. Lymphocytes are grouped under a Dirichlet process Gaussian mixture model, which involves clustering the data via a nonparametric Bayesian framework that describes distributions over mixture models with an infinite number of mixture components. The advantage of such grouping is that one does not need to make any assumptions about the number of TIL clusters. Each image is then characterized by the histogram of occurrences of the identified TILs within the particular partition defined by the groups (15). Details regarding the extracted features are provided in (35). To investigate the QVT-TIL associations, a pairwise Spearman correlation was performed between each of the top QVT and TIL compactness measures followed by Benjamini-Hochberg method (38) to adjust the P values and control for the false discovery rate (FDR; <0.01).
Association with GSEA pathways
A dataset of D6 = 92 patients with early-stage NSCLC from TCIA with available mRNA sequencing data was included for radiogenomic analysis. Radiogenomic analysis was performed using mRNA sequencing data obtained with Illumina Genome Analyzer Sequencing version 2 (Illumina, San Diego, CA, USA). The Cancer Genome Atlas gene expression data are publicly available for download (39). An empirical analysis using the Wilcoxon rank sum test of the 22,126 genes across the high and low QVT yielded a set of differentially expressed genes. The Benjamini and Hochberg method was used to adjust the P values and control for the FDR (<0.01). Gene Ontology analysis was performed to identify distinct biological processes (40, 41), which structures and classifies genes on the basis of the known molecular and cellular biological processes and provides the relationship between those processes. These pathways were chosen on the basis of their biological significance in regulating immune response, cell adhesion, and carcinogenesis. GSEA was applied on major identified biological processes/pathways to determine separate enrichment scores for each pairing of a sample and gene set (42). The lists of genes involved in each pathway were obtained from the Molecular Signatures Database. Last, a pairwise Wilcoxon rank sum test on enrichment scores was performed across high– and low–QVT feature groups to obtain the strength of association between the pathway enrichment score and the feature values.
Acknowledgments
Funding: Research reported in this publication was supported by the National Cancer Institute under award numbers 1U24CA199374-01, R01CA249992-01A1, R01CA202752-01A1, R01CA208236-01A1, R01CA216579-01A1, R01CA220581-01A1, R01CA257612-01A1, 1U01CA239055-01, 1U01CA248226-01, and 1U54CA254566-01; National Heart, Lung, and Blood Institute, 1R01HL15127701A1; National Institute of Biomedical Imaging and Bioengineering, 1R43EB028736-01; National Center for Research Resources under award number 1 C06 RR12463-01, VA Merit Review Award IBX004121A from the U.S. Department of Veterans Affairs Biomedical Laboratory Research and Development Service the Office of the Assistant Secretary of Defense for Health Affairs, through the Breast Cancer Research Program (W81XWH-19-1-0668) and the Prostate Cancer Research Program (W81XWH-15-1-0558 and W81XWH-20-1-0851); the Lung Cancer Research Program (W81XWH-18-1-0440 and W81XWH-20-1-0595); the Peer Reviewed Cancer Research Program (W81XWH-18-1-0404); the Kidney Precision Medicine Project (KPMP) Glue Grant; the Ohio Third Frontier Technology Validation Fund; the Clinical and Translational Science Collaborative of Cleveland (UL1TR0002548) from the National Center for Advancing Translational Sciences (NCATS) component of the National Institutes of Health (NIH) and NIH roadmap for Medical Research; and the Wallace H. Coulter Foundation Program in the Department of Biomedical Engineering at Case Western Reserve University. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH, the U.S. Department of Veterans Affairs, the Department of Defense, or the U.S. government. The funders had no role in data collection, data analysis, data interpretation, or preparing the results.
Author contributions: Conceptualization: M.A., M.K., P.Pr., and A.M. Methodology: M.A., M.K., P.Pr., and A.M. Software: M.A. and P.Pr. Validation: M.A., M.K., P.Pr., and A.M. Formal analysis: M.A., M.K., P.Pr., and P.F. Investigation: M.A. and P.Pr. Resources: M.A. Data curation: K.B., A.G., V.S.V., P.Pa., P.D.V., and V.V. Writing—original draft preparation: M.A., M.K., P.Pr., and P.Pa. Writing—review and editing: M.K., M.A., A.G., V.V., and A.M. Visualization: M.A. Supervision: V.V. and A.M. All authors have read and agreed to the published version of the manuscript. All authors confirm that they had full access to all the data in the study and accept responsibility to submit for publication. A.M. acts as the guarantors of the study.
Competing interests: A.M. is an equity holder in Elucid Bioimaging and Inspirata Inc. In addition, he has served as a scientific advisory board member for Inspirata Inc., Astrazeneca, Bristol Meyers-Squibb, and Merck. Now, he serves on the advisory board of Aiforia Inc. He also has sponsored research agreements with Philips, AstraZeneca, and Bristol Meyers-Squibb. His technology has been licensed to Elucid Bioimaging. He is also involved in an NIH U24 grant with PathCore Inc. and three different R01 grants with Inspirata Inc. P.Pa. has a financial and advisory relationship with Astrazeneca and Jazz Pharmaceuticals. A.M. and P.Pr. are inventors on a patent related to this work filed by Case Western Reserve University (no. US9483822B2, filed 28 January 2015, published 1 November 2016). A.M. is an inventor on four patents related to this work filed by Case Western Reserve University (no. US9767555B2, filed 10 December 2015, published 19 September 2017; no. US9984462B2, filed 15 September 2017, published 29 May 2018; no. US10004471B2, filed 2 August 2016, published 26 June 2018; and no. US10398399B2, filed 27 March 2018). A.M. and M.A. are inventors on a patent related to this work filed by Case Western Reserve University (no. US10064594B2, filed 2 August 2016), published 4 September 2018). A.M., M.A., and V.V. are inventors on a patent related to this work filed by Case Western Reserve University (no. US10441215B2, filed 9 February 2018, published 15 October 2019). The authors declare that they have no other competing interests.
Data and materials availability: All data needed to evaluate the conclusions in the paper are present in the paper and/or the Supplementary Materials. The dataset D6 used in this study is publicly available open source and can be accessed through the corresponding sources: https://wiki.cancerimagingarchive.net/display/Public/NSCLC+Radiogenomics#28672347a99a795ff4454409862a398ffc076b98. Code, model files, and extra software used in this manuscript to reproduce the results are available at https://zenodo.org/record/7301761.
Supplementary Materials
This PDF file includes:
Sections SA to SF
Figs. S1 to S5
Tables S1 to S5
REFERENCES AND NOTES
- 1.Borghaei H., Paz-Ares L., Horn L., Spigel D. R., Steins M., Ready N. E., Chow L. Q., Vokes E. E., Felip E., Holgado E., Barlesi F., Kohlhäufl M., Arrieta O., Burgio M. A., Fayette J., Lena H., Poddubskaya E., Gerber D. E., Gettinger S. N., Rudin C. M., Rizvi N., Crinò L., Blumenschein G. R. Jr., Antonia S. J., Dorange C., Harbison C. T., Graf Finckenstein F., Brahmer J. R., Nivolumab versus docetaxel in advanced nonsquamous non–small-cell lung cancer. N. Engl. J. Med. 373, 1627–1639 (2015). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Reck M., Rodríguez-Abreu D., Robinson A. G., Hui R., Csőszi T., Fülöp A., Gottfried M., Peled N., Tafreshi A., Cuffe S., O’Brien M., Rao S., Hotta K., Leiby M. A., Lubiniecki G. M., Shentu Y., Rangwala R., Brahmer J. R.; KEYNOTE-024 Investigators , Pembrolizumab versus chemotherapy for PD-L1–positive non–small-cell lung cancer. N. Engl. J. Med. 375, 1823–1833 (2016). [DOI] [PubMed] [Google Scholar]
- 3.Pardoll D. M., The blockade of immune checkpoints in cancer immunotherapy. Nat. Rev. Cancer 12, 252–264 (2012). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Linardou H., Gogas H., Toxicity management of immunotherapy for patients with metastatic melanoma. Ann. Transl. Med. 4, 272 (2016). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Spain L., Walls G., Julve M., O’Meara K., Schmid T., Kalaitzaki E., Turajlic S., Gore M., Rees J., Larkin J., Neurotoxicity from immune-checkpoint inhibition in the treatment of melanoma: A single centre experience and review of the literature. Ann. Oncol. 28, 377–385 (2017). [DOI] [PubMed] [Google Scholar]
- 6.Patel S. P., Kurzrock R., PD-L1 expression as a predictive biomarker in cancer immunotherapy. Mol. Cancer Ther. 14, 847–856 (2015). [DOI] [PubMed] [Google Scholar]
- 7.Lesterhuis W. J., Haanen J. B. A. G., Punt C. J. A., Cancer immunotherapy—Revisited. Nat. Rev. Drug Discov. 10, 591–600 (2011). [DOI] [PubMed] [Google Scholar]
- 8.Thom B., Mamoor M., Lavery J. A., Baxi S. S., Khan N., Rogak L. J., Sidlow R., Korenstein D., The experience of financial toxicity among advanced melanoma patients treated with immunotherapy. J. Psychosoc. Oncol. 39, 285–293 (2021). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Kennedy L. B., Salama A. K. S., A review of cancer immunotherapy toxicity. CA Cancer J. Clin. 70, 86–104 (2020). [DOI] [PubMed] [Google Scholar]
- 10.Li R., Han D., Shi J., Han Y. X., Tan P., Zhang R., Li J., Choosing tumor mutational burden wisely for immunotherapy: A hard road to explore. Biochim. Biophys. Acta Rev. Cancer 1874, 188420 (2020). [DOI] [PubMed] [Google Scholar]
- 11.Trebeschi S., Kurilova I., Călin A. M., Lambregts D. M. J., Smit E. F., Aerts H., Beets-Tan R. G. H., Radiomic biomarkers for the prediction of immunotherapy outcome in patients with metastatic non-small cell lung cancer. J. Clin. Oncol. 35, e14520 (2017). [Google Scholar]
- 12.Tang C., Hobbs B., Amer A., Li X., Behrens C., Canales J. R., Cuentas E. P., Villalobos P., Fried D., Chang J. Y., Hong D. S., Welsh J. W., Sepesi B., Court L., Wistuba I. I., Koay E. J., Development of an immune-pathology informed radiomics model for non-small cell lung cancer. Sci. Rep. 8, 1922 (2018). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.D. Saeed-Vafaa, R. Bravob, J. A. Dean, A. El-Kenawic, Nathaniel, M. Père, M. Strobla, C. Daniels, O. Stringfieldd, M. Damaghid, Ilke, Tunalid, L. V. Browna, L. Curtin, D. Nicholb, H. Peck, R. J. Gilliesd, J. A. Gallaherb, Combining radiomics and mathematical modeling to elucidate mechanisms of resistance to immune checkpoint blockade in non-small cell lung cancer. bioRxiv 10.1101/190561 (2017). [DOI]
- 14.Sun R., Limkin E., Vakalopoulou M., Dercle L., Champiat S., Han S., Verlingue L., Brandao D., Lancia A., Ammari S., Hollebecque A., Scoazec J., Marabelle A., Massard C., Soria J., Robert C., Paragios N., Deutsch E., Ferté C., A radiomics approach to assess tumour-infiltrating CD8 cells and response to anti-PD-1 or anti-PD-L1 immunotherapy: An imaging biomarker, retrospective multicohort study. Lancet Oncol. 19, 1180–1191 (2018). [DOI] [PubMed] [Google Scholar]
- 15.Khorrami M., Prasanna P., Gupta A., Patil P., Velu P. D., Thawani R., Corredor G., Alilou M., Bera K., Fu P., Feldman M., Velcheti V., Madabhushi A., Changes in CT radiomic features associated with lymphocyte distribution predict overall survival and response to immunotherapy in non-small cell lung cancer. Cancer Immunol. Res. 8, 108–119 (2019). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Trebeschi S., Drago S. G., Birkbak N. J., Kurilova I., Cǎlin A. M., Delli Pizzi A., Lalezari F., Lambregts D. M. J., Rohaan M. W., Parmar C., Rozeman E. A., Hartemink K. J., Swanton C., Haanen J. B. A. G., Blank C. U., Smit E. F., Beets-Tan R. G. H., Aerts H. J. W. L., Predicting response to cancer immunotherapy using noninvasive radiomic biomarkers. Ann. Oncol. 30, 998–1004 (2019). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Hendry S. A., Farnsworth R. H., Solomon B., Achen M. G., Stacker S. A., Fox S. B., The role of the tumor vasculature in the host immune response: Implications for therapeutic strategies targeting the tumor microenvironment. Front. Immunol. 7, 621 (2016). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Li M. O., Wan Y. Y., Sanjabi S., Robertson A. K. L., Flavell R. A., Transforming growth factor-β regulation of immune responses. Annu. Rev. Immunol. 24, 99–146 (2006). [DOI] [PubMed] [Google Scholar]
- 19.Duan J., Wang Y., Jiao S., Checkpoint blockade-based immunotherapy in the context of tumor microenvironment: Opportunities and challenges. Cancer Med. 7, 4517–4529 (2018). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Jain R. K., Normalizing tumor vasculature with anti-angiogenic therapy: A new paradigm for combination therapy. Nat. Med. 7, 987–989 (2001). [DOI] [PubMed] [Google Scholar]
- 21.Stylianopoulos T., Munn L. L., Jain R. K., Reengineering the physical microenvironment of tumors to improve drug delivery and efficacy: From mathematical modeling to bench to bedside. Trends Cancer 4, 292–319 (2018). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Cross M. J., Claesson-Welsh L., FGF and VEGF function in angiogenesis: Signalling pathways, biological responses and therapeutic inhibition. Trends Pharmacol. Sci. 22, 201–207 (2001). [DOI] [PubMed] [Google Scholar]
- 23.Xue S., Hu M., Li P., Ma J., Xie L., Teng F., Zhu Y., Fan B., Mu D., Yu J., Relationship between expression of PD-L1 and tumor angiogenesis, proliferation, and invasion in glioma. Oncotarget 8, 49702–49712 (2017). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Zhang Y. L., Li J., Mo H. Y., Qiu F., Zheng L. M., Qian C. N., Zeng Y. X., Different subsets of tumor infiltrating lymphocytes correlate with NPC progression in different ways. Mol. Cancer 9, 4 (2010). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Goc J., Germain C., Vo-Bourgais T. K. D., Lupo A., Klein C., Knockaert S., de Chaisemartin L., Ouakrim H., Becht E., Alifano M., Validire P., Remark R., Hammond S. A., Cremer I., Damotte D., Fridman W. H., Sautès-Fridman C., Dieu-Nosjean M. C., Dendritic cells in tumor-associated tertiary lymphoid structures signal a Th1 cytotoxic immune contexture and license the positive prognostic value of infiltrating CD8+ T cells. Cancer Res. 74, 705–715 (2014). [DOI] [PubMed] [Google Scholar]
- 26.Ganss R., Hanahan D., Tumor microenvironment can restrict the effectiveness of activated antitumor lymphocytes. Cancer Res. 58, 4673–4681 (1998). [PubMed] [Google Scholar]
- 27.Luke J. J., Bao R., Sweis R. F., Spranger S., Gajewski T. F., WNT/β-catenin pathway activation correlates with immune exclusion across human cancers. Clin. Cancer Res. 25, 3074–3083 (2019). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Alilou M., Orooji M., Beig N., Prasanna P., Rajiah P., Donatelli C., Velcheti V., Rakshit S., Yang M., Jacono F., Gilkeson R., Linden P., Madabhushi A., Quantitative vessel tortuosity: A potential CT imaging biomarker for distinguishing lung granulomas from adenocarcinomas. Sci. Rep. 8, 15290 (2018). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Hu S., Hoffman E. A., Reinhardt J. M., Automatic lung segmentation for accurate quantitation of volumetric x-ray CT images. IEEE Trans. Med. Imaging 20, 490–498 (2001). [DOI] [PubMed] [Google Scholar]
- 30.Rudyanto R. D., Kerkstra S., van Rikxoort E. M., Fetita C., Brillet P. Y., Lefevre C., Xue W., Zhu X., Liang J., Öksüz İ., Ünay D., Kadipaşaogˇlu K., Estépar R. S. J., Ross J. C., Washko G. R., Prieto J. C., Hoyos M. H., Orkisz M., Meine H., Hüllebrand M., Stöcker C., Mir F. L., Naranjo V., Villanueva E., Staring M., Xiao C., Stoel B. C., Fabijanska A., Smistad E., Elster A. C., Lindseth F., Foruzan A. H., Kiros R., Popuri K., Cobzas D., Jimenez-Carretero D., Santos A., Ledesma-Carbayo M. J., Helmberger M., Urschler M., Pienn M., Bosboom D. G. H., Campo A., Prokop M., de Jong P. A., Ortiz-de-Solorzano C., Muñoz-Barrutia A., van Ginneken B., Comparing algorithms for automated vessel segmentation in computed tomography scans of the lung: The VESSEL12 study. Med. Image Anal. 18, 1217–1232 (2014). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Sethian J. A., Fast marching methods. SIAM Rev. 41, 199–235 (1999). [Google Scholar]
- 32.Jain P., Khorrami M., Gupta A., Rajiah P., Bera K., Viswanathan V. S., Fu P., Dowlati A., Madabhushi A., Novel non-invasive radiomic signature on CT scans predicts response to platinum-based chemotherapy and is prognostic of overall survival in small cell lung cancer. Front. Oncol. 11, 744724 (2021). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Fan J., Li R., Variable selection for Cox’s proportional hazards model and frailty model. Ann. Statist. 30, 74–99 (2002). [Google Scholar]
- 34.Jazieh K., Khorrami M., Saad A., Gad M., Gupta A., Patil P., Viswanathan V. S., Rajiah P., Nock C. J., Gilkey M., Fu P., Pennell N. A., Madabhushi A., Novel imaging biomarkers predict outcomes in stage III unresectable non-small cell lung cancer treated with chemoradiation and durvalumab. J. Immunother. Cancer 10, e003778 (2022). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Corredor G., Wang X., Zhou Y., Lu C., Fu P., Syrigos K., Rimm D. L., Yang M., Romero E., Schalper K. A., Velcheti V., Madabhushi A., Spatial architecture and arrangement of tumor-infiltrating lymphocytes for predicting likelihood of recurrence in early-stage non-small cell lung cancer. Clin. Cancer Res. 25, 1526–1534 (2019). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Vincent L., Soille P., Watersheds in digital spaces: An efficient algorithm based on immersion simulations. IEEE Trans. Pattern Anal. Mach. Intell. 13, 583–598 (1991). [Google Scholar]
- 37.Xing F., Yang L., Robust nucleus/cell detection and segmentation in digital pathology and microscopy images: A comprehensive review. IEEE Rev. Biomed. Eng. 9, 234–263 (2016). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Ferreira J. A., Zwinderman A. H., On the Benjamini–Hochberg method. Ann. Statist. 34, 1827–1849 (2006). [Google Scholar]
- 39.Bakr S., Gevaert O., Echegaray S., Ayers K., Zhou M., Shafiq M., Zheng H., Benson J. A., Zhang W., Leung A. N. C., Kadoch M., Hoang C. D., Shrager J., Quon A., Rubin D. L., Plevritis S. K., Napel S., A radiogenomic dataset of non-small cell lung cancer. Sci. Data 5, 180202 (2018). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Ashburner M., Ball C. A., Blake J. A., Botstein D., Butler H., Cherry J. M., Davis A. P., Dolinski K., Dwight S. S., Eppig J. T., Harris M. A., Hill D. P., Issel-Tarver L., Kasarskis A., Lewis S., Matese J. C., Richardson J. E., Ringwald M., Rubin G. M., Sherlock G., Gene ontology: Tool for the unification of biology. The Gene Ontology Consortium. Nat. Genet. 25, 25–29 (2000). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.The Gene Ontology Consortium , Expansion of the gene ontology knowledgebase and resources. Nucleic Acids Res. 45, D331–D338 (2017). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Subramanian A., Tamayo P., Mootha V. K., Mukherjee S., Ebert B. L., Gillette M. A., Paulovich A., Pomeroy S. L., Golub T. R., Lander E. S., Mesirov J. P., Gene set enrichment analysis: A knowledge-based approach for interpreting genome-wide expression profiles. Proc. Natl. Acad. Sci. U.S.A. 102, 15545–15550 (2005). [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Sections SA to SF
Figs. S1 to S5
Tables S1 to S5