Machine learning for semi­automated classification of glioblastoma, brain metastasis and central nervous system lymphoma using magnetic resonance advanced imaging

Abstract

Background

Differentiating glioblastoma, brain metastasis, and central nervous system lymphoma (CNSL) on conventional magnetic resonance imaging (MRI) can present a diagnostic dilemma due to the potential for overlapping imaging features. We investigate whether machine learning evaluation of multimodal MRI can reliably differentiate these entities.

Methods

Preoperative brain MRI including diffusion weighted imaging (DWI), dynamic contrast enhanced (DCE), and dynamic susceptibility contrast (DSC) perfusion in patients with glioblastoma, lymphoma, or metastasis were retrospectively reviewed. Perfusion maps (rCBV, rCBF), permeability maps (K-trans, Kep, Vp, Ve), ADC, T1C+ and T2/FLAIR images were coregistered and two separate volumes of interest (VOIs) were obtained from the enhancing tumor and non-enhancing T2 hyperintense (NET2) regions. The tumor volumes obtained from these VOIs were utilized for supervised training of support vector classifier (SVC) and multilayer perceptron (MLP) models. Validation of the trained models was performed on unlabeled cases using the leave-one-subject-out method. Head-to-head and multiclass models were created. Accuracies of the multiclass models were compared against two human interpreters reviewing conventional and diffusion-weighted MR images.

Results

Twenty-six patients enrolled with histopathologically-proven glioblastoma (n=9), metastasis (n=9), and CNS lymphoma (n=8) were included. The trained multiclass ML models discriminated the three pathologic classes with a maximum accuracy of 69.2% accuracy (18 out of 26; kappa 0.540, P=0.01) using an MLP trained with the VpNET2 tumor volumes. Human readers achieved 65.4% (17 out of 26) and 80.8% (21 out of 26) accuracies, respectively. Using the MLP VpNET2 model as a computer-aided diagnosis (CADx) for cases in which the human reviewers disagreed with each other on the diagnosis resulted in correct diagnoses in 5 (19.2%) additional cases.

Conclusions

Our trained multiclass MLP using VpNET2 can differentiate glioblastoma, brain metastasis, and CNS lymphoma with modest diagnostic accuracy and provides approximately 19% increase in diagnostic yield when added to routine human interpretation.

Introduction

Glioblastoma (GB), central nervous system lymphoma (CNSL), and brain metastasis together represent a large proportion of brain tumors encountered in clinical neuro-oncology. GBs comprise 40% to 50% of primary brain tumors in adults, while brain metastases are found in 10% to 30% of adults with a systemic malignancy, of which nearly half of cases appear on imaging to be solitary metastases (1,2). Primary CNSL comprise up to 4% of primary CNS tumors, with an additional small contribution of secondary CNSL (3,4).

Differentiating these entities may be difficult using conventional magnetic resonance imaging (MRI), as significant potential overlap exists in the degree of post-contrast enhancement and peritumoral FLAIR signal hyperintensity across the 3 tumor classes (5,6). However, establishing the correct diagnosis is important for guiding therapy, as each of these tumor classes carries a different prognosis and requires unique management. GB is an aggressive malignancy generally requiring surgical management and possible adjuvant therapy (7). In the absence of a known malignancy, the diagnosis of a brain metastasis necessitates a metastatic workup to identify the primary disease, which will then guide therapy. Primary CNS lymphoma is generally managed with chemoradiation therapy (8).

The use of advanced MRI including perfusion and diffusion has been investigated for improving the ability to distinguish GB, CNSL, and brain metastasis. For example, CNSL tends to have lower ADC values in comparison to GB and metastasis (9,10), although overlap has been shown (11,12). Perfusion parameters including CBV and permeability measures have also shown promise in differentiating GB, PCL, and metastasis (13-16).

In recent years, machine learning models, including support vector machines (SVMs) and multilayer perceptrons (MLPs), a type of simple neural network, have successfully been used for semi-automated brain tumor classification (17-25). Several of these studies have relied on texture analysis of conventional MR sequences. While other studies have included analysis of perfusion data (17-19,21), the inclusion of permeability parameters has not been commonly reported. The purpose of this study was to investigate whether supervised training of a multiclass SVM or MLP applied to MR perfusion and permeability datasets could reliably differentiate GB, CNSL and brain metastasis using automated feature selection.

Methods

Patients

This retrospective study was conducted between July 2014 and August 2016 according to an approved institutional review board. Inclusion criteria were as follows: (I) histopathologically-proven intracranial GB, CNSL or metastasis and (II) preoperative brain MRI including DSC, dynamic contrast enhanced (DCE) and diffusion weighted imaging (DWI).

Image acquisition

Image acquisition was performed on a 3.0T scanner. DWI was acquired using single-shot spin-echo EPI (TR/TE 4,100/95 ms; FOV: 220 mm × 220 mm; matrix: 128 mm × 128 mm; slices: 30 mm × 5 mm). Diffusion gradients were applied along three orthogonal directions with b=0 and 1,000 s/mm². DCE perfusion was accomplished using a 3D radial volumetric interpolated examination sequence with the following parameters: TR/TE 4.75/2.2 ms; FA 10°; FOV 220 mm × 220 mm; matrix 256 mm × 192 mm; 30 mm × 5 mm slices with temporal resolution of 6 seconds over a 4-minute acquisition time. Varying flip angle (3°, 5°, and 12°) methodology was implemented for the generation of T1 maps (26). Dynamic susceptibility contrast (DSC) perfusion was performed with a single-shot gradient-echo EPI sequence with the following parameters: TR/TE 1650/30 ms, FA = 90°, FOV 220 mm × 220 mm, matrix 128 mm × 128 mm, 25 mm × 5 mm slices, and 60 dynamic frames.

Image preprocessing

MR perfusion studies were processed using commercially available FDA-approved software (Olea Sphere, Olea Medical SAS, La Ciotat, France). The arterial input function was selected automatically and multiparametric perfusion maps were calculated using an extended Toft model (27) for DCE and block-circulant singular value decomposition technique (28) for DSC. The conventional images (FLAIR and post-contrast T1WI); ADC; CBV and CBF normalized to contralateral white matter (relative CBV and CBF; rCBV and rCBF) from DSC perfusion; and volume transfer constant from extravascular extracellular space (EES) to plasma (K-trans), rate constant from EES to plasma (Kep), EES volume per unit tissue volume (Ve), and blood plasma volume per unit tissue volume (Vp) from DCE perfusion datasets were then exported from the software for subsequent analysis.

The exported images were coregistered to standard Montreal Neurological Institute coordinates by the Functional MRI Software Library (FSL; FMRIB Analysis Group, Oxford, UK, version 5.0) using a 12-degree of freedom transformation and a mutual information cost function (29). This was followed by visual inspection to ensure adequate alignment. Additional preprocessing steps performed in FSL included brain extraction and histogram normalization.

Using a consensus approach, 3-dimensional VOIs were drawn manually on enhancing tumor and peritumoral non-enhancing T2 hyperintensity (NET2) on coregistered T1C+ and FLAIR images, respectively. For patients with more than one tumor, the VOI was confined to the largest lesion. NET2 was defined as the T2 hyperintense region on FLAIR images within 2 cm around the enhancing tumor, excluding necrotic tissue and the enhancing component itself.

For each patient, the T1C+ and NET2 VOIs were applied as inclusion masks to the rCBV, rCBF, K-trans, Kep, Vp, Ve, and ADC maps using FSL to remove all image data outside of the respective VOIs (Figure 1). This generated a total of 14 tumor volumes (7 parameters for each of the 2 VOIs) for each patient.

Figure 1.

Example of the preprocessing pipeline utilized to register and segment an MRI brain study prior to the machine learning analysis steps. Sequences (several of which are pictured) are coregistered to standard Montreal Neurological Institute geometry (A). VOIs are then drawn manually to delineate the enhancing tumor component (T1C+; B) and area of NET2 (C). These T1C+ and NET2 VOIs are used to create tumor volumes (to the right of the arrows in B, C, respectively), which are processed in the subsequent machine learning steps. MRI, magnetic resonance imaging; VOI, volume of interest; NET2, non-enhancing T2 hyperintensity.

Machine learning

The 14 total T1C+ and NET2 extracted volumes for each patient were then further processed by custom code built by one of the authors (NCS) using the Scikit-learn library v0.18.1 for Python (30). Supervised training of each ML model was accomplished using tumor volume imaging data labeled with the tumor diagnosis. The MLP design utilized a single hidden layer, rectified linear unit activation and an alpha of 0.0001.

Validation of the trained model was performed using a leave-one-subject-out cross-validation structure, described with the following equation:

where K equals the total number of subjects and E equals the error. Leave-one-subject-out entails running K folds, each fold including K-1 subjects for training and the remaining subject held out for validation. True error is then calculated as the average error rate from all K folds.

Separate SVM and MLP models were trained for each head-to-head and three-class tumor volume comparison. For both the SVM and MLP approaches, 56 separate models were trained and validated (14 tumor volumes for each of the 3 head-to-head comparisons and 14 tumor volumes for the three-class comparison), yielding a total of 112 individual trained ML models. For each of the 112 individual tests, a new, “naïve” model was trained. Feature selection was performed de novo automatically by the Scikit-learn module within the nested cross-validation structure to prevent biasing of the trained model that would result from performing feature selection upon the entire subject set. One-versus-rest validation was employed for the three-class comparison.

Trained model total accuracy and receiver operating characteristic data were collected for each of the 112 training and validation cycles.

Subjective interpretation

All imaging studies were reviewed by two board certified neuroradiologists blinded to histopathological diagnosis in independent sessions. Readers were instructed to review all available MR images in each patient and use their best clinical judgment to assign each case with a diagnosis of lymphoma, GB or metastasis.

Statistical analysis

Statistical analysis was performed using IBM SPSS Statistics 24 for Windows (Released 2016; IBM Corp., Armonk, NY, USA). Cohen kappa scores were calculated to quantify the strength of agreement between machine and human interpretation with respect to the ground truth (histopathological diagnosis). For all statistical analysis a P value <0.05 was considered significant.

Results

Twenty-six patients (16 male, 10 female; age 61.8±9.3 years) with glioblastoma (n=9), CNSL (n=8), and metastasis (n=9; lung carcinoma =4, esophageal carcinoma =1, melanoma =1, neuroendocrine carcinoma =1, rectal carcinoma =1, thyroid carcinoma =1) meeting the inclusion criteria were identified.

Classification accuracy

The best performing ML training-validation cycles are included in Table 1. The trained multiclass ML models were able to differentiate the 3 diagnostic classes with a maximum of 69.2% accuracy (kappa 0.540, P=0.01), which was obtained by training an MLP utilizing the Vp values from the perilesional NET2 VOIs (MLP Vp_NET2). Receiver operating characteristic for this trained MLP Vp_NET2 model is presented in Figure 2.

Table 1. Head-to-head and three-class accuracy results by model and tumor volume types.

Parameter	Diagnosed correctly (%)	AUC	Sensitivity	Specificity	Class-specific accuracy	Kappa score (P value)
GB vs. metastasis
MLP
KtransT1C+	83.3	0.83	GB: 1.0; Met: 0.67	GB: 0.67; Met: 1.0	N/A	0.667 (<0.01*)
VpT1C+	77.8	0.78	GB: 1.0; Met: 0.56	GB: 0.56; Met: 1.0	N/A	0.556 (<0.01*)
VeT1C+	77.8	0.78	GB: 1.0; Met: 0.56	GB: 0.56; Met: 1.0	N/A	0.556 (<0.01*)
VpNET2	77.8	0.78	GB: 0.78; Met: 0.78	GB: 0.78; Met: 0.78	N/A	0.556 (0.02*)
VeNET2	77.8	0.78	GB: 0.89; Met: 0.67	GB: 0.73; Met: 0.86	N/A	0.556 (0.02*)
KepNET2	77.8	0.78	GB: 0.78; Met: 0.78	GB: 0.78; Met: 0.78	N/A	0.556 (0.02*)
SVM
ADCNET2	72.2	0.78	GB: 0.56; Met: 0.89	GB: 0.89; Met: 0.56	N/A	0.444 (0.046*)
VpNET2	72.2	0.72	GB: 0.44; Met: 1.0	GB: 1.0; Met: 0.44	N/A	0.444 (0.02*)
KepNET2	72.2	0.72	GB: 0.44; Met: 1.0	GB: 1.0; Met: 0.44	N/A	0.444 (0.02*)
ADCT1C+	66.7	0.72	GB: 0.33; Met: 1.0	GB: 1.0; Met: 0.33	N/A	0.333 (0.06)
VpT1C+	66.7	0.72	GB: 0.33; Met: 1.0	GB: 1.0; Met: 0.33	N/A	0.333 (0.06)
Metastasis vs. lymphoma
MLP
VpNET2	82.4	0.83	Met: 0.78; CNSL: 0.88	Met: 0.88; CNSL: 0.78	N/A	0.648 (<0.01*)
VeNET2	82.4	0.83	Met: 0.78; CNSL: 0.88	Met: 0.88; CNSL: 0.78	N/A	0.648 (<0.01*)
KepNET2	82.4	0.83	Met: 0.78; CNSL: 0.88	Met: 0.88; CNSL: 0.78	N/A	0.648 (<0.01*)
SVM
rBVT1C+	70.6	0.62	Met: 1.0; CNSL: 0.38	Met: 0.38; CNSL: 1.0	N/A	0.388 (0.04*)
VpT1C+	70.6	0.62	Met: 1.0; CNSL: 0.38	Met: 0.38; CNSL: 1.0	N/A	0.388 (0.04*)
VpNET2	64.7	0.62	Met: 1.0; CNSL: 0.25	Met: 0.25; CNSL: 1.0	N/A	0.261 (0.11)
VET1C+	64.7	0.62	Met: 1.0; CNSL: 0.25	Met: 0.25; CNSL: 1.0	N/A	0.261 (0.11)
GB vs. lymphoma
MLP
KepNET2	64.7	0.65	GB: 0.67; CNSL: 0.63	GB: 0.67; CNSL: 0.63	N/A	0.292 (0.23)
VeNET2	58.8	0.60	GB: 0.44; CNSL: 0.75	GB: 0.67; CNSL: 0.55	N/A	0.190 (0.40)
KepT1C+	58.8	0.59	GB: 0.56; CNSL: 0.63	GB: 0.63; CNSL: 0.56	N/A	0.179 (0.46)
ADCNET2	58.8	0.59	GB: 0.56; CNSL: 0.63	GB: 0.63; CNSL: 0.56	N/A	0.179 (0.46)
rBVT1C+	58.8	0.58	GB: 0.67; CNSL: 0.50	GB: 0.60; CNSL: 0.57	N/A	0.168 (0.49)
rBFT1C+	58.8	0.58	GB: 0.67; CNSL: 0.50	GB: 0.60; CNSL: 0.57	N/A	0.168 (0.49)
SVM
VENET2	58.8	0.60	GB: 0.33; CNSL: 0.88	GB: 0.88; CNSL: 0.33	N/A	0.201 (0.31)
ADCT1C+	58.8	0.63	GB: 1.0; CNSL: 0.14	GB: 0.56; CNSL: 1.0	N/A	0.131 (0.27)
ADCNET2	58.8	0.63	GB: 1.0; CNSL: 0.14	GB: 0.56; CNSL: 1.0	N/A	0.131 (0.27)
rBFT1C+	58.8	0.61	GB: 0.22; CNSL: 1.0	GB: 1.0; CNSL: 0.53	N/A	0.212 (0.16)
KEPNET2	58.8	0.61	GB: 0.22; CNSL: 1.0	GB: 1.0; CNSL: 0.53	N/A	0.212 (0.16)
GB vs. metastasis vs. lymphoma
MLP
VpNET2	69.2	0.77/0.77†	GB: 0.67; Met: 0.67; CNSL: 0.75	GB: 0.60; Met: 1.0; CNSL: 0.60	GB: 0.73; Met: 0.88; CNSL: 0.85	0.540 (<0.01*)
VeNET2	65.4	0.74/0.74†	GB: 0.44; Met: 0.78; CNSL: 0.75	GB: 0.57; Met: 0.58; CNSL: 0.86	GB: 0.69; Met: 0.73; CNSL: 0.88	0.470 (<0.01*)
rBFT1C+	53.9	0.65/0.65†	GB: 0.56; Met: 0.56; CNSL: 0.50	GB: 0.23; Met: 1.0; CNSL: 0.44	GB: 0.58; Met: 0.85; CNSL: 0.65	0.308 (0.02*)
KepNET2	53.9	0.65/0.65†	GB: 0.67; Met: 0.67; CNSL: 0.25	GB: 0.50; Met: 0.55; CNSL: 0.67	GB: 0.65; Met: 0.69; CNSL: 0.73	0.299 (0.03*)
SVM
VpNET2	57.7	0.74/0.74†	GB: 0.44; Met: 1.0; CNSL: 0.25	GB: 0.94; Met: 0.41; CNSL: 1.0	GB: 0.77; Met: 0.62; CNSL: 0.77	0.378 (<0.01*)
VpT1C+	53.8	0.68/0.67†	GB: 0.22; Met: 1.0; CNSL: 0.38	GB: 1.0; Met: 0.41; CNSL: 0.89	GB: 0.73; Met: 0.62; CNSL: 0.73	0.302 (<0.01*)
ADCNET2	50.0	N/A	GB: 0.56; Met: 0.89; CNSL: 0.0	GB: 0.65; Met: 0.59; CNSL: N/A	GB: 0.62; Met: 0.69; CNSL: N/A	0.235 (0.06)

The top 3 performing models are included for each category. *, P values are significant; ^†, macro- and micro-averaged AUC scores, respectively. T1C+, volumes defined by lesional post-contrast enhancement; NET2, volumes defined by peritumoral non-enhancing T2 signal hyperintensity. GB, glioblastoma; Met, metastasis; CNSL, central nervous system lymphoma.

Figure 2.

Receiver operating characteristic for the three-class test using an MLP model trained with Vp_NET2 volumes (one-vs.-rest validation). Class 0, glioblastoma; Class 1, metastasis; Class 2, lymphoma. MLP, multilayer perceptron.

Head-to-head comparisons for each pair of diagnostic groups demonstrated higher maximum accuracies than the three-class comparisons: 83.3% for GB and metastasis (MLP Ktrans_T1C), 82.4% for metastasis and lymphoma (MLP Vp_NET2, MLP Ve_NET2, and MLP Kep_NET2), and 64.7% for GB and lymphoma (MLP Kep_NET2).

Human interpretation

Observers A and B identified 17 and 21 out of 26 cases correctly, respectively. The interobserver agreement was k =0.434 (95% CI, 0.167–0.701).

Cohen kappa scores demonstrated the following degrees of inter-rater agreement with respect to histopathological diagnosis: trained multiclass MLP Vp_NET2 k =0.540 (95% CI, 0.275–0.805), observer A k =0.479 (95% CI, 0.201–0.757), and observer B k =0.712 (95% CI, 0.489–0.934).

Conclusions

The growing interest in machine learning techniques for automated image classification has generated anticipation that this technology may very soon aid radiological diagnosis in clinical practice (31). While many research efforts towards this end have focused on interpretation of conventional CT and MR imaging, the multimodal nature of advanced MRI presents an intriguing target for machine learning experimentation. This study demonstrates that an MLP trained using quantitative perfusion, permeability and diffusion MR imaging can independently differentiate 3 brain tumor classes with a diagnostic accuracy comparable to that of trained neuroradiologists. Additionally, when used in conjunction with human evaluation as a computer-aided diagnosis (CADx) tool the diagnostic yield is increased by approximately 19% over unaided human interpretation.

In this series, the greatest diagnostic accuracy obtained by the ML models in the three-class experiments was achieved by the MLP model trained using Vp_NET2, which yielded a kappa value of 0.540 (P=0.001) indicating a moderate correlation with the correct histopathological diagnosis. Interestingly, the best accuracy for the multiclass SVC models was also achieved by utilizing the Vp_NET2 tumor volumes. The Vp parameter (fractional plasma volume) reflects blood plasma volume per unit tissue volume (32) and has shown utility in previous studies for characterizing tumor grade (33) and enabling differentiation of GB and metastasis (34).

The results of this study suggest that differences among the diagnostic classes in the extent of vascularity within the non-enhancing T2 signal hyperintense region surrounding the enhancing tumor component could be used to differentiate the tumor classes (35,36). This is logical, since it is well-known that NET2 surrounding enhancing glioblastoma is likely to represent infiltrative (microscopic) tumor, which may feature neovascularity reflected in the Vp values (37). In contrast, it has been shown that neovascularization is not a prominent histologic feature in CNS lymphoma, which has lower microvascular density as compared with GB (38,39). Therefore, NET2 in lymphoma more likely correlates with densely packed cells with less vascularity as compared with GB and hence lower expected Vp values. NET2 associated with metastatic tumors, which are typically non-infiltrative, is more likely to represent vasogenic edema, which would not be expected to demonstrate elevated vascularity (40,41).

Accuracy results were greater in the head-to-head comparisons than the three-class comparison. This is expected, since narrowing the diagnostic possibilities from 3 to 2 potential diagnoses improves the odds of making a correct classification. However, in the glioblastoma versus lymphoma tests there was a maximum diagnostic accuracy of just 64.7%. The relatively poor ability of the trained models to differentiate glioblastoma and lymphoma within these patients likely also decreased the overall diagnostic accuracies obtained in the three-class tests.

Another possible factor limiting the accuracy obtained in the three-class tests is the use of isolated tumor volumes. This design decision acted as a feature reduction step, “focusing” the model’s attention on the enhancing or NET2 tumor components. It was also an attempt to control for patients with multiple lesions, an imaging feature that if included would have introduced an undesired bias since the purpose of this study was to generate trained models able to differentiate the tumor classes using perfusion and permeability features. However, removing contextual imaging data potentially correlating with a correct diagnosis, such as lesion location, multiplicity, or degree of mass effect on surrounding structures, potentially lowered the diagnostic accuracy of the trained model, disadvantaging it as compared with the human reviewers.

The best accuracy obtained by the trained multiclass ML models was comparable to that of the human reviewers using a simulated real-world clinical workflow that utilized conventional and diffusion-weighted MRI. Further study is required to investigate whether the addition of image texture parameters from conventional MRI in concert with perfusion and permeability parameters may yield ML accuracy superior to that obtained in the current study.

The utility of the trained model may be greater when used as a CADx clinical support tool than as an independent diagnostic tool. Although the diagnostic accuracies of our neuroradiologists were 65% and 81%, respectively, there was a relatively low interobserver agreement between the two readers (16 of 26 cases; k =0.434). Used as a tie-breaker, our best-performing multiclass model resulted in the correct identification of 5 additional cases (19%).

A strength of this investigation is that all patients underwent histopathologic sampling to confirm the diagnosis prior to inclusion. This is crucial since the very purpose of the study is to investigate techniques for differentiating tumors with potentially overlapping imaging features. An additional strength is that feature selection utilized for model training was performed de novo within each cross-validation fold to minimize the risk of biasing and better approximate the performance of the trained model in clinical practice. Some previously described techniques for training ML models to perform multiclass tumor discrimination have pooled best-performing features from head-to-head classifications for subsequent use in multiclass models (17,42). This approach was avoided in the present study because of the risk that features previously extracted from the test subject at hand may be utilized by the trained model, introducing bias and subverting attempts at blinded validation.

Some earlier studies investigating ML for multiclass tumor diagnosis reported high accuracies utilizing developer-specified features such as tumor location, ring enhancement, or hemorrhage (23). This approach was avoided in the current study in favor of using automated feature extraction for several reasons. As opposed to conventional images, perfusion and permeability imaging data are less easily definable in terms of qualitative features. Additionally, the reliance on hard-coded rules may yield a trained model that excels in diagnosing “classic” representations of a given tumor class but struggles with outlier cases in which it is most likely to be of clinical value. Furthermore, automated feature extraction has the significant advantage of scalability when new data are subsequently added to the training set for model refinement.

A potential limitation of this study is the use of manual as opposed to fully automated tumor segmentation, which despite efforts to standardize an approach among the co-investigators likely introduced an element of user-dependency. An additional important limitation of this study is the sample size. Machine learning experiments in image classification generally gain diagnostic accuracy when trained with very large data sets (e.g., subjects numbering in the tens of thousands), however large-scale advanced imaging MRI data sets are not readily available for such experiments. The need for large data sets is particularly relevant when applying deep learning approaches, such as convolutional neural networks. The decision by the authors to instead implement SVC and MLP models was an attempt to maximize the accuracy of the trained model in the setting of this limited training data set while lowering the likelihood of overfitting that may have occurred with a deep learning approach. Although some studies suggest that SVCs may outperform neural networks for image classification when utilizing relatively small datasets (43), in this study the SVC models achieved lower accuracies than the MLP models in the multiclass and head to head diagnostic challenges.

In summary, our trained multiclass MLP using Vp_NET2 can differentiate glioblastoma, brain metastasis, and CNS lymphoma with diagnostic accuracy approaching that of a neuroradiologist and provide approximately 19% increase in diagnostic yield when used as a CADx tool. Further study with larger data sets is required to improve diagnostic accuracy and demonstrate generalizability. Organized efforts by the radiology machine learning community to facilitate the sharing of anonymized diagnosis-specific, multimodal radiologic imaging in a HIPAA-compliant manner are needed to nurture this field of research.

Ethical Statement: Institutional Review Board approval (ID #IF2169016) was obtained prior to this retrospective study.