Emerging Applications of Artificial Intelligence in Neuro-Oncology

Abstract

Due to the exponential growth of computational algorithms, artificial intelligence (AI) methods are poised to improve the precision of diagnostic and therapeutic methods in medicine. The field of radiomics in neuro-oncology has been and will likely continue to be at the forefront of this revolution. A variety of AI methods applied to conventional and advanced neuro-oncology MRI data can already delineate infiltrating margins of diffuse gliomas, differentiate pseudoprogression from true progression, and predict recurrence and survival better than methods used in daily clinical practice. Radiogenomics will also advance our understanding of cancer biology, allowing noninvasive sampling of the molecular environment with high spatial resolution and providing a systems-level understanding of underlying heterogeneous cellular and molecular processes. By providing in vivo markers of spatial and molecular heterogeneity, these AI-based radiomic and radiogenomic tools have the potential to stratify patients into more precise initial diagnostic and therapeutic pathways and enable better dynamic treatment monitoring in this era of personalized medicine. Although substantial challenges remain, radiologic practice is set to change considerably as AI technology is further developed and validated for clinical use.

Summary

Advances in artificial intelligence applied to radiomics and radiogenomics in neuro-oncologic imaging will improve our diagnostic, prognostic, and therapeutic methods, helping propel the field into an era of precision medicine.

Artificial intelligence (AI) is a broad term that describes any task performed by a computer that normally requires human intelligence, including explicit rule-based systems, as well as computer algorithms that do not require hard-coded rules (Fig 1). Machine learning, which falls under the umbrella of AI, is a branch of data science that enables computers to learn from existing “training” data without explicit programming to make predictions about new data points. Deep learning is a subclass of machine learning based on neural networks, containing a large number of layers, made possible due to recent computational advances. Machine learning and deep learning methods are being increasingly adopted for radiomics research, which relies on medical imaging data as quantitative imaging biomarkers. The overarching goal of AI-based research in neuro-oncologic imaging is to better understand the complex manifestations of heterogeneous central nervous system (CNS) neoplasms in hopes of improving patient outcomes.

Figure 1:

Diagram shows overview of terms encompassed by artificial intelligence and their nested relationships with each other.

Machine Learning Methods

Most applications of machine learning in medical imaging have relied on supervised forms of machine learning, which consist of algorithms that are trained on “ground truth” labels. Labels can include different classes of diagnoses (eg, high- vs lower-grade glioma), different prognoses (eg, long vs short survival), or different classes that exist within a single set of image volumes (eg, enhancing tumor vs necrotic tissue vs edema vs normal brain tissue). When provided with sufficient examples of the different classes, algorithms “learn” how to classify novel data. Supervised machine learning methods include logistic regression, support vector machines, and random forests, as well as tools useful for clinical decision support (eg, decision trees and Bayesian networks) (1,2). In general, these traditional supervised approaches are applied to explicitly engineered intermediate features, often after a step of feature reduction, which is necessary to reduce model complexity and avoid overfitting (ie, memorizing the training sample cases rather than learning the relevant pattern)—a prevalent problem that creeps into many machine learning studies without proper “held out” validation samples. These approaches, while powerful, oftentimes require extensive, domain-specific, expert knowledge about the underlying biologic basis of the process being studied. Another frequently used category of machine learning is unsupervised algorithms, such as k-means clustering, which can generate novel groupings or categories from complex data sets and have important roles in discovery science and big data.

Deep Learning Methods

Deep learning grew out of a desire to model the tiered organization of the mammalian brain’s visual cortex, where hierarchically organized layers process increasingly complex intermediate visual features such as lines, edges, shapes, and entire visual objects (3–6). The recent growth of computing power through parallel graphical processing units and improved mathematical optimization methods enabled upscaling the architectures of these neural network models to contain many intermediate layers, differentiating deep learning from traditional neural networks, which were first conceived in the 1940s (3–6). Through an iterative process of up-dating model weights (“backpropagation”), these algorithms learn to appropriately identify lower and intermediate level image information in order to maximize classification performance. Typically, convolutional neural networks (CNNs), a class of feed-forward neural networks, have been used for image-based problems. In recent years, deep learning approaches have achieved super-human performance on visual processing tasks, as benchmarked by the ImageNet challenge, a yearly competition to test algorithms on visual recognition tasks (7,8). Interestingly, the weights derived from these networks trained on ImageNet can be adapted for new tasks, including medical images, in a process termed “transfer learning.” Although some deep learning architectures for medical image processing use transfer learning, others use custom image-naive architectures, which may perform better depending on the specific task. Although deep learning models can run quickly and do not require as much manual intervention as traditional machine learning approaches, they tend to require large amounts of labeled training data in order to be robust to data variability.

Radiomics in Neuro-Oncology

Although clinical radiology generally relies on visual assessment of images in subjective and qualitative terms, the field of radiomics extracts information from clinical images for use as quantitative imaging biomarkers (9,10). The first essential step of radiomics generally involves lesion segmentation (Fig 2, A), which is generally preceded by image preprocessing steps including skull stripping, intensity normalization, and alignment of image volumes from different modalities. A variety of methods have been used for segmentation, ranging from manual labeling and/or annotation and semiautomated methods (11) to more recent deep learning methods (12–14).

Figure 2:

Workflow of radiomics in neuro-oncology. A, After preprocessing steps, multimodal MR images are segmented by using automated or manual methods. B, This is followed by feature extraction with use of a variety of different techniques. C, Machine learning methods are then trained on the features to generate models of underlying molecular markers and predict survival. Deep learning models can be used for performing each of the described steps individually or in a more comprehensive fashion (bottom pathway of figure). EGFRvIII = epidermal growth factor receptor variable III, IDH = isocitrate dehydrogenase, MGMT = O⁶-methylguanine-DNA-methyltransferase.

The next step of radiomics with traditional machine learning involves the extraction of quantitative features, including basic shape, size, and intensity metrics, as well as more complex features derived from a variety of statistical approaches applied to the images, for example, histogram-based features, texture-based features, fitted biophysical models, spatial patterns, and deep learning features (Fig 2, B). A variety of different machine learning models can then be applied to the intermediate quantitative features in order to “mine” them for significant associations, allowing them to predict crucial information about a tumor, such as infiltrating tumor margins, molecular markers, and prognosis (Fig 2, C), which are relevant for therapeutic decision making. Alternatively, deep learning approaches to radiomics in neurooncology generally necessitate less domain-specific knowledge compared with the explicitly engineered features for traditional machine learning, allowing them to make predictions without explicit feature selection or reduction steps.

Public Neuro-Oncology Radiomic Data and Competitions

Advances in the field of segmentation and radiomics within neuro-oncology have been supported by data made available through the Cancer Imaging Archive (TCIA) (15), which is part of a larger effort from the Cancer Genome Atlas (TCGA) (http://cancergenome.nih.gov/). Since 2012, TCIA data have been further curated through the annual Multimodal Brain Tumor Image Segmentation Benchmark (BraTS) challenge (16,17), which seeks to improve the accuracy of automated glioma segmentation and survival prediction with preoperative MR images. This public data set contains multimodal images of high- and lowergrade gliomas with expert manual segmentations with five labels (healthy brain tissue, necrosis, edema, and nonenhancing and enhancing tumor). Deep learning approaches have surpassed more traditional segmentation methods to win the BraTS challenges in 2016 (12) and 2017 (14).

Current Methods in Neuro-Oncologic Imaging

Most research in neuro-oncology has focused on diffuse gliomas, World Health Organization (WHO) grade II–IV tumors, which are typically divided into lower-grade gliomas (WHO grade II and III) and glioblastoma (WHO grade IV) (18). Much work has focused on glioblastoma, given that it represents more than half of malignant primary brain tumors and has an aggressive course and grim prognosis (19). Lowergrade gliomas can sometimes progress into glioblastoma, and these are known as secondary glioblastomas. A variety of other brain tumors, including WHO grade I tumors, pediatric CNS tumors, primary CNS lymphoma, and brain metastases, encompass important areas of neuro-oncology but represent lessactive areas of research given smaller sample sizes, more disease heterogeneity, and relatively lower morbidity.

Given its excellent soft-tissue contrast, MRI is the central tool for tumor detection and characterization. Conventional MRI sequences, which include pre- and postcontrast T1-weighted imaging, T2-weighted imaging, and T2-weighted fluid-attenuated inversion recovery (FLAIR) sequences, are good at helping delineate tumor volume and morphologic characteristics. Unfortunately, contrast enhancement is nonspecific and the detection of foci of tumor infiltration within the T2-weighted FLAIR signal intensity abnormality is nearly impossible with conventional sequences (20). Most institutions rely on other advanced MRI methods that are more sensitive to this crucial aspect of tumor physiology. Diffusion-weighted imaging is a useful method for evaluating areas of high cellularity and can be extended to diffusion tensor imaging, which can help identify tissue microstructure and depict neoplastic infiltration in areas of brain that appear normal on conventional MR images (20–22). Perfusion MRI with dynamic susceptibility-weighted contrast material enhancement, dynamic contrast enhancement, or arterial spin labeling exploits the neoangiogenic properties of gliomas. MR spectroscopy, which depicts the distribution of chemical metabolites such as choline, creatine, and N-acetylaspartate, is used clinically for grading gliomas and identifying regions of tumor infiltration (23,24). Despite the potential viability of these advanced modalities, they are usually interpreted in a qualitative manner. Their widespread adoption is further complicated by variability across sites, imaging units, and postprocessing methods.

Genomics and Radiogenomics in Neuro-Oncology

Vast mutational, molecular, and microenvironment heterogeneity in CNS neoplasms substantially complicates diagnostic and treatment approaches. Diffuse gliomas typically harbor more than 60 genetic alterations, encompassing several major cellular pathways (25). Developing a better understanding of these cellular pathways is crucial for improving diagnostic methods and delivering targeted therapies. In fact, given the prognostic significance of different mutations, molecular markers have recently taken a larger role in establishing an “integrated diagnosis,” as seen in the most recent 2016 WHO classification of CNS tumors (26) (Fig 3). For example, glioblastoma is now fundamentally grouped according to the presence of mutations in isocitrate dehydrogenase (IDH), with worse survival seen in IDH wild-type gliomas (27). Seminal work analyzing gene expression data of subjects from TCGA (https://cancergenome.nih.gov/) identified prognostically influential gene expression–based subgroups, namely proneural, neural, classic, and mesenchymal, representing distinct cellular pathways with prognostic and therapeutic implications (28).

Figure 3:

Genomic and radiogenomic landscape of diffuse gliomas. Diffuse gliomas are fundamentally differentiated according to presence of IDH mutation. IDH mutant gliomas are typically lower grade (World Health Organization [WHO] grades II–III) but can sometimes be glioblastoma (GBM) (WHO grade IV), in which case they usually arise from a lower-grade astrocytoma. IDH mutant gliomas are subdivided according to presence of 1p19q codeletion, which defines an oligodendroglioma, and are associated with “poorly circumscribed” margins. The 1p19q non-codeleted tumors are characterized by “circumscribed” margins and exhibit the “T2–fluid-attenuated inversion recovery (FLAIR) mismatch” pattern. IDH wild-type gliomas are typically glioblastomas but may sometimes be a lower-grade astrocytoma or oligodendroglioma not otherwise specified (NOS). Epidermal growth factor receptor (EGFR) mutations and O⁶-methylguanine-DNA-methyltransferase (MGMT ) methylation status are important molecular and prognostic markers. EGFR mutant gliomas are associated with increased cerebral blood volume (CBV ). Methylated MGMT gliomas are associated with “masslike” T2-FLAIR signal intensity abnormality and heterogeneous and/or nodular enhancement pattern, whereas unmethylated MGMT gliomas are associated with “infiltrative” T2-FLAIR signal intensity abnormality and thick enhancement pattern with central necrosis. Background shading represents overall prognosis (green = best, yellow = intermediate, red = worst). wt = wild-type.

Further advancing our understanding of neuro-oncology is the nascent field of radiogenomics (imaging genomics), which correlates imaging characteristics with genetic, mutational, and expression patterns (29,30). Radiogenomics also has the power to dynamically monitor the microenvironment over the course of treatment, potentially allowing for a reduced number of repeat biopsies or resections. Glioma radiogenomics (Fig 3) has begun to characterize radiomic phenotypes of several candidate genetic alterations, including IDH mutation, O⁶-methylguanine-DNA-methyltransferase (MGMT) methylation, epidermal growth factor receptor (EGFR) splice variant, and the 1p/19q codeletion. Additional work in radiogenomics has attempted to develop a systems-level understanding between gene expression patterns and radiomics.

Candidate Genetic Alterations

Mutations in IDH1 and IDH2 are found in 70%–80% of lower-grade gliomas but only 5%–10% of glioblastomas, which are usually secondary glioblastomas, arising from lower-grade gliomas (27,31–32). IDH mutant gliomas result in the accumulation of D-2-hydroxyglutarate, an oncometabolite absent in IDH wild-type tumors. D-2-hydroxyglutarate can be detected with high specificity with MR spectroscopy (33–35). Although advanced MR spectroscopy sequences (eg, two-dimensional localized correlation spectroscopy at 7.0 T [36]) are highly reliable in the detection of 2-hydroxyglutarate, they are not readily available in most centers, which limits their utility. Interestingly, visually apparent imaging biomarkers, including indistinct margins and T2-FLAIR mismatch (regions within the tumor that are hyperintense on T2-weighted images but hypointense on FLAIR images), have been shown to be useful in the differentiation of IDH mutant from IDH wild-type gliomas (37). CNNs applied to conventional MRI modalities have been used to differentiate IDH mutant gliomas from IDH wild-type tumors with 92% accuracy, consistent with prior visual assessment and underlying pathophysiology that IDH wild-type tumors demonstrate more infiltrative, ill-defined margins (38).

Approximately 33%–57% of diffuse gliomas exhibit hypermethylation of the promoter of the MGMT gene, encoding for a DNA repair protein (39,40). MGMT promoter hypermethylation has been associated with better prognosis owing to improved sensitivity to alkylating agents (eg, temozolomide) (39,40). Radiomic studies have identified distinct imaging signatures for this molecular marker. Several groups have been able to predict MGMT methylation status with up to 88% accuracy by combining texture features with traditional supervised machine learning methods (41–43). In addition, several deep learning architectures have been shown to predict MGMT methylation status (44,45). Korfiatis et al (44) obtained up to 94.9% accuracy with only T2-weighted images and without the need for previous tumor segmentation. By performing principal component analysis on their final CNN layer, Chang et al (38) found that nodular and heterogeneous enhancement and “masslike FLAIR edema” were helpful for predicting MGMT methylation status, with up to 83% accuracy.

EGFR is a receptor tyrosine kinase that regulates normal cellular growth in epithelial cell lines (46). EGFR mutations are present in approximately 40% of glioblastomas but are rarely found in lower-grade gliomas (31). The most common extracellular EGFR mutation in glioblastoma is the splice variant III (EGFRvIII), which is found in 31% of patients (47). Radiogenomic studies performed with perfusion imaging have demonstrated a moderate relationship between EGFR amplification and tumor blood flow and volume (48,49). Support vector machine–based approaches have shown that EGFRvIII mutant gliomas exhibit deep peritumoral infiltration, which is consistent with a more aggressive and/or infiltrative phenotype (50). Further multivariate approaches incorporating a larger set of multiparametric features have shown that EGFRvIII tumors have increased neovascularization and cell density, as well as a spatial preference for frontal and parietal regions (51,52).

The codeletion of chromosome arms 1p/19q is present in approximately 30% of lower-grade gliomas (but is not present in glioblastoma) and, in combination with an IDH mutation, now defines an oligodendroglioma (26). The 1p/19q codeletion has been shown to have a protective effect on prognosis (53). Comparison of lower-grade gliomas with and without the 1p/19q codeletion showed that noncircumscribed borders, heterogeneous signal on T1- and/or T2-weighted images, and lower apparent diffusion coefficients are strongly associated with 1p/19q codeletions (37,54). More recent work with use of CNNs found that the 1p/19q codeletion is associated with increased enhancement, left frontal predominance, and ill-defined margins on FLAIR images with mass effect (33), with up to 93% accuracy (55).

Systems-Level Radiogenomic Approaches

Complementary to these candidate gene approaches, a systemslevel radiogenomic approach has been used by some groups to better understand how radiomic markers relate to gene expression patterns more globally, which is important considering that each tumor contains a combination of different mutations. An early study (56) looked at the relationship between gene expression modules with several neuroradiologist-defined MRI patterns, including contrast-to-necrosis ratio, subventricular zone involvement, contrast-to-T2 ratio, and T2 heterogeneity patterns. The investigators found that activation of specific gene expression programs can be inferred from imaging traits; for example, the hypoxia module was associated with contrast enhancement and the proliferation module was associated with mass effect. Grossmann et al (57) compared genomic pathway enrichment analyses with volumetric tumor phenotypes, finding that immune response and apoptosis cellular pathways were associated with necrosis pathways and that tumor bulk and edema were associated with homeostasis and cell cycling pathways. Zinn et al (58) evaluated the relationship between radiomic features in gliomas and nonoverlapping mutations in TP53, PTEN, and EGFR genes in 29 patients from TCGA. Although there were minimal differences in conventional MRI volumetric parameters, texture analyses yielded distinct and partially nonoverlapping radiomic feature sets and a variety of gene expression signatures uniquely associated with these three mutations (TP53 with angiogenesis, PTEN with invasion, and EGFR with immune response).

Prognostication in Neuro-Oncology

In current clinical practice, prognostication is based on histologic tumor grade and clinical models incorporating the patient’s age, sex, and functional status (eg, the Karnofsky performance status scale) (59). In addition, some of the previously discussed specific molecular markers are now central in diagnosis and prognosis. However, imaging features and radiomic metrics are not used in any widely adopted clinical prognostic model, despite their potential to capture underlying tumor biology and outcomes.

Radiomic Prediction of Prognosis

Previous radiomic studies have shown that basic imaging metrics, including maximal dimension and enhancing volume, are predictive beyond clinical models (60,61). A rule-based model combining clinical, imaging, and genetic variables resulted in the best predictive accuracy in patients in TCIA (62). Diffusion, perfusion, and MR spectroscopy measures have also been found to be prognostic (63,64), and more recent studies have used machine learning methods to predict patient survival on the basis of multiparametric MR images (65–67). Kickingereder et al (65) identified 11 crossvalidated features portending a poor prognosis, including volume, shape, texture, and wavelet features. These features were exclusively derived from FLAIR images within the contrastenhancing portion of the tumor. Macyszyn et al (66), using features explicitly extracted from traditional and advanced MRI sequences, developed a support vector machine model to predict survival group (low, medium, and high), with up to 80% accuracy in training and prospective replication cohorts. The most predictive features in this model included tumor volumes, angiogenesis (enhancing tumor volume), peritumoral infiltration (peak perfusion height), cell density (trace diffusion values), and distance to the ventricles.

Systems-Level Radiomic Approaches for Prognostication

With use of unsupervised machine learning methods, radiomic features can also be used to generate novel subgroups that may more closely align with the underlying biology of gliomas. Itakura et al (68) performed clustering of features capturing the shape, texture, and edge sharpness of 121 solitary glioblastomas with postcontrast T1-weighted MRI. They identified three clusters—premultifocal, spherical, and rim enhancing. These clusters were subsequently validated in 144 multi-institution subjects from TCIA and had significant differences in survival. The best prognosis was seen in the rim-enhancing subgroup, and the worst prognosis was seen in the premultifocal group. Rathore et al (43) also applied an unsupervised high-dimensional clustering algorithm to a comprehensive feature set derived from conventional and advanced MRI in 208 patients with glioblastoma. The resulting subtypes were remarkably similar to those found in the study by Itakura et al (68), clustering into rim-enhancing, irregular, and solid subgroups. These subgroups were associated with distinct survival estimates, with the best survival seen in the rim-enhancing subgroup. The clusters were also associated with particular anatomic locations, molecular subtypes, and genetic variations, including IDH, MGMT, and EGFRvIII. Although these subgroups are still preliminary, requiring larger sample sizes and additional validation to ensure their robustness, they show the promise of unsupervised methods in providing more precise subgrouping and prognostication than clinical models and molecular markers currently used in WHO classification.

Treatment Response Assessment in Neuro-Oncology

Current standard-of-care treatment for glioblastoma consists of maximal safe resection followed by radiation and chemotherapy with temozolomide (69), whereas lower-grade gliomas may be treated with surgery and/or chemoradiation. The addition of tumor-treating fields has more recently been shown to have an additional survival benefit in glioblastoma (19). Although not yet proven, an array of clinical trials using immunotherapy are being applied to treat patients with glioblastoma (70), including ones that target specific molecular pathways such as EGFR (71).

Clinical Response Assessment

Increasing size of T2/FLAIR signal abnormality with new or increasing areas of enhancement after combined radiation and chemotherapy, known as pseudoprogression (72), makes the evaluation of treatment response particularly challenging and is more common in MGMT methylated and IDH mutant tumors. Conversely, antiangiogenic agents (eg, bevacizumab) can result in pseudoresponse, which consists of a dramatic reduction of enhancement by altering the blood-brain barrier, but do little to alter progression of the infiltrating component—with no improvement in overall survival (73).

Initial guidelines for response assessment followed the Macdonald criteria (74), which incorporated only the size of enhancing components. More recently, the evaluation of treatment response has been described with the Response Assessment for Neuro-Oncology, or RANO, criteria (75), which incorporates changes in enhancing tissue and T2/FLAIR nonenhancing signal intensity abnormalities in addition to clinical status. A recent modification to the RANO criteria (76) changed the baseline assessment to be the first postradiation treatment image rather than the postresection image and provides some response assessment rubrics to identify pseudoprogression. However, RANO is still a limited tool for assessing response, especially considering that it uses two-dimensional measurements, which are subjective, and does not incorporate advanced imaging modalities such as MRI perfusion, diffusion tensor imaging, and MR spectroscopy, despite their clinical utility and use at most institutions. Newer immunotherapy agents have been shown to elicit complex inflammatory responses (77), adding additional difficulties in response assessment. The immunotherapy RANO criteria (78,79) extends the timeline for determining progression in an attempt to help account for immune inflammatory-related pseudoprogression.

Radiomic Prediction of Pseudoprogression and Progression

The differentiation of pseudoprogression from true progression remains a crucial diagnostic dilemma, for which AI methods are well suited (Fig 4). Several radiomic studies have had moderate success by evaluating diffusion-weighted imaging (80,81) and dynamic susceptibility-weighted contrast enhancement measures (82,83). Machine learning approaches that incorporate multiple measures from both diffusionweighted imaging and dynamic susceptibility-weighted contrast enhancement have also had success in predicting pseudoprogression (84,85). Notably, although most previous studies have used longitudinal clinical and radiologic follow-up to determine pseudoprogression, histologic examination of repeat resections often shows a combination of treatment-related changes and recurrent and/or residual tumor. Wang et al (86) used a multivariate logistic regression model, incorporating measures derived from dynamic susceptibility-weighted contrast-enhanced MRI and diffusion tensor imaging within the enhancing tissue to predict pseudoprogression on the basis of histologically classified cases of true progression, pseudoprogression, and mixed response, with an area under the curve of 0.90. Akbari et al (87) conducted a support vector machine–based analysis of conventional and advanced MRI features by training the model on pathologists’ scores of true progression versus pseudoprogression, demonstrating a high correlation (r = 0.86) between pathologic and radiomic scores of pseudoprogression.

Figure 4:

Treatment response in neuro-oncologic imaging. After standard-of-care treatment with combined radiation therapy and chemotherapy, increasing T2–fluid-attenuated inversion recovery (FLAIR) signal intensity abnormality and new and/or increasing size of enhancing lesions are often seen. Artificial intelligence (AI )–based “virtual biopsy” could assist in distinguishing underlying biology and segregating treatment response into three possible categories: true progression (>75% recurrent and/or residual glioma at pathologic examination), mixed response (25%–75% recurrent and/or residual glioma at pathologic examination), and pseudoprogression (>75% treatment-related changes). Categories dictate distinct therapeutic approaches. In this example, the new enhancing lesion was found to represent 100% treatment-related changes at pathologic examination, with few atypical cells.

Radiomic Predictions of Infiltration and Recurrence

Despite the difficulty in differentiating infiltrating neoplasm from edema by using conventional qualitative approaches, there is substantial promise for machine learning methods to identify margins of infiltrative tissue on preoperative MR images. These delineations may be used to guide extended surgical resections, localized biopsies, and radiation treatment planning. With use of a voxel-wise logistic regression model, FLAIR and apparent diffusion coefficient maps were shown to be sufficient to enable prediction of areas of future tumor recurrence (88). Akbari et al (89) developed and Rathore et al (90) refined a multivariate support vector machine approach, incorporating features from conventional and advanced MRI modalities after registering areas of glioblastoma recurrences to preoperative MR images. This approach generates predictive spatial maps of infiltrated peritumoral tissue (Fig 5), with approximately 90% cross-validated accuracy.

Figure 5:

Predictive maps of tumor infiltration. Multimodal pre-operative conventional and advanced MRI data were analyzed with support vector machines to generate an estimated infiltration map overlaid on postcontrast T1-weighted image (red areas signify higher risk). Postcontrast T1-weighted MR image obtained at 6-month follow-up (right) demonstrates area of recurrence near site of highest predicted infiltration (arrow).

For another approach, which uses pathologic-radiologic relationships, Chang (91) developed a fully automated system to register biopsy sites from neuronavigational cross-hairs to the preoperative MR images by using a CNN in 36 patients. Multimodal imaging measures at the biopsy sites were then used to train a network on a cell density counting method applied to pathology images (Fig 6). They found an inverse relationship between cellularity and apparent diffusion coefficient and FLAIR values and a direct relationship between degree of enhancement and cellularity. This approach effectively generates noninvasive maps of cell density, which are useful for identifying the infiltrative margins of gliomas. Given that current surgical resection is guided largely by means of the enhancing tumor alone, these methods have substantial promise in developing noninvasive means to stratify patients into clinical trials and guiding more aggressive treatments. Indeed, the method by Akbari et al (89) has already spurred a clinical trial with intensified radiation to areas of infiltrative tumor for patients who have recently undergone an initial resection.

Figure 6:

Predicted tumor cellularity map. Left, voxelwise linear regression model was applied to multimodal preoperative MRI trained on automated cell counts of biopsies localized to different regions on MR image and used to generate a map of predicted cellularity (red areas signify more cells). Right, photomicrographs of biopsy specimens from regions of tumor with high and low cellularity (hematoxylin-eosin stain; original magnification, ×400). (Reprinted, with permission, from reference 91.)

AI Applications in Neuro-Oncologic Imaging: Non-Glioma Evaluation

Machine learning approaches have also been applied to other CNS tumors, particularly brain metastases and CNS lymphoma, with the potential to clarify diagnostically ambiguous situations and/or improve workflow efficiency and accuracy. Initially, a variety of image-processing methods, such as threedimensional template matching, were used to detect and localize brain metastases (92,93). More recent work with use of three-dimensional CNNs has shown increasing promise (94,95), with potential to aid in stereotactic radiation therapy planning. In some situations, the differentiation among brain metastases, primary CNS lymphoma, and glioblastoma is not possible with existing clinical and imaging paradigms. To address this problem, Wang et al (96) developed a decision tree and multivariate logistic regression model incorporating diffusion tensor imaging and dynamic susceptibility-weighted contrast-enhanced MRI metrics from the enhancing region to differentiate among these three entities. In another study (97), random forest analysis applied to extracted radiomic texture and wavelet features enabled the differentiation of nonnecrotic glioblastoma from CNS lymphoma, with a performance superior to that of three human readers. Sometimes patients are found to have brain metastases without a known primary site. Machine learning approaches have been applied to this clinical scenario on the assumption that underlying molecular differences and their downstream effects on local environments are likely to exhibit different radiomic features. For example, Ortiz-Ramón et al (98) were able to differentiate brain metastases due to lung cancer, melanoma, and breast cancer by using a random forest model applied to features derived from two- and three-dimensional texture analyses of T1-weighted postcontrast sequences.

Promises and Challenges of AI in Neuro-Oncologic Imaging

AI methods, given their ability to discern patterns and combine information in a way that humans cannot, show substantial promise for the future of radiology and precision medicine. An ideal AI-based diagnostic system for neuro-oncology would incorporate all relevant multimodal imaging data with clinical information and molecular markers to make precise predictions of biologically based and clinically relevant subtypes for a new tumor diagnosis, in line with the precision medicine movement (99). This information could help stratify patients into tailored treatments predicted to be most effective, including determining whether a patient would benefit from standard or supertotal resection and/or targeted intensification of radiation therapy to areas of infiltrative tumor, chemotherapy, or novel therapeutic agents that target specific cellular mechanisms (Fig 7).

Figure 7:

Schematic of future artificial intelligence–based neuro-oncologic imaging and clinical management workflow. A, Initial lesion detection and analysis system would generate a probabilistic differential of lesion(s) seen on patient’s initial brain MR image (precision diagnostics). It would also recommend additional useful imaging examinations, laboratory tests, or tissue sampling. B, Glioma-specific module could make personalized predictions of molecular markers, survival, and treatment responses (precision diagnostics), thereby recommending optimal treatment plan(s), which would be continuously updated on the basis of follow-up imaging (precision therapeutics). CNS = central nervous system, DTI = diffusion tensor imaging, EGFR = epidermal growth factor receptor, EGFRvIII = epidermal growth factor receptor variable III, IDH = isocitrate dehydrogenase, MGMT = O⁶-methylguanine-DNA-methyltransferase, TTFields = tumor-treating fields.

AI also has great potential for monitoring both standard treatments and novel treatments such as immunotherapy. Although complex inflammatory responses seen in immunotherapy (76) would require further validation of AI models that could monitor these new treatments (100), they have the potential to quickly determine treatment efficacy, thus allowing for dynamic adjustment during the course of treatment. In this regard, AI methods applied to advanced imaging could ultimately offer a personalized treatment response prediction superior to that of current methods.

Challenges

Major challenges to the promises of AI in radiology include high-quality ground truth data, generalizable and interpretable methods, and user-centric workflow integration (101). Concerns regarding the “black box” nature of these algorithms have somewhat diminished given the continuing development of methods, such as saliency maps (102) or principal component analysis (38), that can “unbox” the networks by interrogating internal algorithm feature vectors. A better mechanistic under-standing between feature patterns and underlying biology will be helpful both for clinical acceptance and for improving the biologic and treatment relevance of the patterns revealed by these methods.

One of the premier challenges in AI research is the availability of large, well-annotated data sets. Unfortunately, studies with relatively small sample sizes are prone to measurement error (103). TCIA and BraTS have made substantial progress in the creation of centralized, well-labeled data for glioma image analysis, whereas non–glioma-based research has been limited by a lack of public data sets. Nevertheless, the vast majority of available data remain siloed within individual institutions and hospital systems. In order for these algorithms to improve further, larger and more heterogeneous data sets (likely orders of magnitude larger) may be needed to improve the generalizability of an algorithm’s performance across different imaging sites, acquisition parameters, and patient populations (104). An important component of assembling such data sets is the sharing of data among institutions, similar to efforts in Alzheimer disease (105) and autism (106) research. Other ways of improving data sets include statistical techniques to harmonize the data sets and to introduce more uniform data collection by adoption of standardized neuro-oncology imaging protocols across institutions (76). Interestingly, novel deep learning methods, namely generative adversarial networks, have shown promise in improving performance by generating synthetic data (107).

An additional barrier to the development of more robust algorithms in the field of neuro-oncologic imaging, and radiomics more generally, is the lack of clear, targeted “use cases” or specific tasks against which their performance can be benchmarked. Other than the BraTS competition, the measured performance of an individual algorithm is highly taskdependent, data set–dependent, and strongly influenced by the particular scientific question, all of which limit comparison of different algorithms developed by different groups. The newly formed American College of Radiology Data Science Institute (https://acrdsi.org/) is helping define standard use cases, annotation tools, and data sets, which should greatly help with standardization and benchmarking relevant to both academic pursuits and commercial ventures.

In addition, although the U.S. Food and Drug Administration is developing pathways for approval of these emerging tools, there are many unanswered questions, including the generalizability of the methods and when retraining is appropriate.

Pathways to Clinical Implementation

Despite the growing use of AI algorithms in research settings, there are major obstacles to the efficient and consistent deployment of these sophisticated algorithms in a clinical setting. The system must be easily integrated into the radiologist’s workflow (electronic medical record system, picture archiving and communication system, and dictation software) to be adopted. Furthermore, many of the segmentation and radiomic methods require manual intervention and the use of a variety of in-house pipelines and have lengthy processing times. There has been relatively little work done to develop tools for easily translating and sharing these methods. In fact, most publications do not provide enough information to re-create their method independently. A few examples of open source tools that may facilitate sharing of different methods are Modelhub (http://modelhub.ai/), Pyradiomics (http://www.Radiomics.io/) (108), and the Cancer Imaging Phenomics Toolkit (https://med.upenn.edu/cbica/captk/), which was developed to facilitate clinical translation of these tools (109,110). Alternatively, optimal solutions for integration into routine clinical workflow may ultimately be provided through emerging commercial ventures.

The “holy grail” of AI in neuroradiology (111) might consist of a fully automated system, integrated into the radiologic workflow, that analyzes images in real time and provides a quantitative and probabilistic draft report. A more general diagnostic system for brain MR images might first assess the probability of whether a newly identified lesion represents a specific neoplasm or a neoplastic mimic (Fig 7, A). Such a system would recommend additional advanced imaging protocols and/or sequences as needed. A more specific glioma evaluation system (Fig 7, B) would then generate a personalized prediction of relevant molecular markers (precision diagnostics) and prognosis, as well as specific treatment recommendations (precision therapeutics). This automated system could also be used for monitoring treatment in “real time,” with more precise quantitative reporting tools to track changes in both conventional and advanced imaging parameters as well as patterns derived from deep learning.

Until AI methods are completely integrated into daily practice, it will be the role of the “centaur” radiologist (112), formed as a synergy of humans and computers, to integrate information from images, AI tools, and health records to improve the precision of radiology and health care.

Conclusion

The overarching goal of this line of research is to improve the outcomes of patients affected by CNS neoplasms through improvements in diagnostic and treatment methods. AI tools that combine clinical, radiomic, and genomic information into predictive models hold substantial promise for guiding and monitoring personalized therapeutics. However, many challenges exist and much work needs to be done to bring the promise of this field into fruition. Nevertheless, radiologic practice will substantially change as AI technology continues to improve to be able to enhance radiologists’ accuracy and efficiency. It is crucial for the future radiologist to understand and appropriately use these powerful tools as they become more integrated into everyday clinical practice in the coming years.

Essentials.

• Artificial intelligence (AI) algorithms are driving neuro-oncology radiomics research forward by identifying complex patterns in advanced MRI with important diagnostic, prognostic, and therapeutic implications.

• Radiogenomics in neuro-oncology offers insights into underlying cellular and molecular mechanisms of cancer biology.

• Radiomic and radiogenomic tools provide a means of noninvasive sampling of tumor microenvironments, allowing for a dynamic and comprehensive evaluation of regionally heterogeneous central nervous system tumors.

• Despite substantial challenges, targeted clinical implementation of AI methods in neuro-oncology is set to transform the field into an era of precision medicine.

Abbreviations

AI

artificial intelligence

BraTS

Multimodal Brain Tumor Image Segmentation Benchmark

CNN

convolutional neural networks

CNS

central nervous system

EGFR

epidermal growth factor receptor

FLAIR

fluid-attenuated inversion recovery

IDH

isocitrate dehydrogenase

MGMT

O⁶-methylguanine-DNA-methyltransferase

TCGA

the Cancer Genome Atlas

TCIA

the Cancer Imaging Archive

WHO

World Health Organization