Diverse Applications of Artificial Intelligence in Neuroradiology

INTRODUCTION

Globally, neurological and mental disorders impact 1 in 3 people across their lifetime.¹ Uniquely positioned to improve imaging diagnosis and clinical management for patients with brain diseases, artificial intelligence (AI) is a computational model that parallels human performance on tasks, often without explicit programming. Several classes of AI have been studied extensively, including machine learning (ML) and deep learning (DL), which utilizes artificial neural networks (NN) inspired by neuronal architectures.^2,3

Imaging research in AI has grown exponentially. As of 2019, there are ~9,000 PubMed indexed articles that match the search on AI and imaging/radiology and ~5,000 articles on AI and neuroimaging/neuroradiology (Figure 1). Consequently, a thorough discussion on the technology and applications of AI in radiology is beyond the scope of this review and can be found elsewhere.^4–8 Discussion of AI for brain tumor, infarct and hemorrhage imaging can be found elsewhere in this issue. Because of the ability of NNs to learn any input-output function,⁹ whether based on imaging or not, the potential applications of NNs to radiology span not only image-based tasks but also the many non-image-based tasks that radiologists perform.

Figure 1.

Cumulative frequency of PubMed indexed articles on AI and imaging/radiology (grey line) and AI and neuroimaging/neuroradiology (blue dashed line). Boolean search query for radiology was “AI/ML/DL/NN and clinical/medical and imaging/radiology/body” and for neuroradiology was “AI/ML/DL/NN and clinical/medical and neuroimaging/neuroradiology/brain.” PubMed last accessed November 27, 2019.

Here, we overview a diverse array of current neuroimaging applications of AI, including tasks such as worklist prioritization, lesion detection, anatomical segmentation, safety, quality and precision education, in a diverse set of diseases ranging from neurodegeneration to trauma. Then, we introduce emerging applications, including multimodal integration for neuroradiology. Finally, we discuss challenges and potential solutions for widespread adoption of AI in neuroimaging clinics worldwide.

DIVERSE APPLICATIONS FOR AI IN NEURORADIOLOGY

Throughout the review, we survey a variety of current and emerging applications for AI, ML and DL towards a range of neurological disorders, from trauma to multiple sclerosis, epilepsy to neurodegeneration. Compared to previous reviews of AI in neuroimaging organized by technical developments and AI methods, here our review is organized by clinical tasks relevant to neuroradiology.

1. WORKLIST PRIORITIZATION

Due to the unique structure, function and location of the brain, diagnosis of neurological diseases is largely dependent on imaging. Time from imaging to proper diagnosis and management determines patient outcomes for acute and chronic neurological diseases.^10–12

Patients presenting with acute ischemic stroke progress in a sequence from door to imaging to diagnosis and intervention. Shorter door-to-puncture time for intravenous thrombolysis and/or mechanical thrombectomy are associated with better long-term functional outcomes.^13,14 One solution to improve door-to-puncture times and outcomes is to reduce imaging-to-diagnosis time. This may be implemented with standardized, automated worklist prioritization. Natural language processing (NLP) algorithms and automated lesion detection networks (described in the next section) may augment efficiency in triaging imaging studies of patients with acute, time-sensitive presentations. NLP and NNs have already been applied to interpretation of chest computed tomography (CT) reports¹⁵ and abdominal CT and pathology reports^16,17 and such tools may be similarly applied to head CT. Future iterations of such automated detection algorithms may be integrated into radiology workflows to improve efficiency. Indeed, current AI tools on the market can send processed imaging data such as CT perfusion maps via text message to stroke physicians and neurointerventional radiologists in real-time. Further, automated segmentation has been applied to workflow integration for patients presenting with intracranial hemorrhage and such networks could be applied to all patients with head CTs presenting with acute neurologic deficits.¹⁸ Despite the high accuracies attained by such networks for detecting findings, challenges of integrating these algorithms into the clinical workflow remain. For example, some intracranial hemorrhages are expected (such as after craniotomy), while others may be unexpected, and therefore much more urgent. Integrating such clinical context into the prioritization score is essential for ensuring that potentially more urgent studies are not de-prioritized.

Worklist prioritization may also improve radiology education. Since radiology trainee performance increases with optimal case exposure and volume,¹⁹ AI may assign specific cases to trainees based on their training profile, to promote consistency in individual trainee experiences.²⁰

2. LESION DETECTION

Neurological diagnoses are made through clinical assessments of patient function and detection and interpretation of lesions on neuroimaging. Furthermore, lesion burden is often associated with functional impairment. Hence, accurate detection and characterization of lesions on neuroimaging is vital to expedite diagnosis and management. Across a variety of brain diseases, there are many research applications for AI in segmentation and quantification of image features.

Trauma

Trauma is a common indication for neuroimaging. DL models such as convolutional neural networks (CNNs) have been designed to detect spinal cord contusion injuries. Demonstrating clinical utility, the quantified injury volumes correlate well with motor impairment.²¹ Many methods utilize CNNs to detect and classify hemorrhages such as epidural and subdural hematomas and subarachnoid and intraparenchymal hemorrhage on head CT scans.^22,23 Additionally, ML techniques such as random forest models have been applied to assess white matter hyperintensities in patients with traumatic brain injury.²⁴

Metastatic Disease

Brain metastases occur in about 20% of patients with cancer. Metastatic disease encompasses a clinically and radiographically heterogeneous set of diseases that often has distinct imaging features and warrants management unique from primary tumors. For example, mucinous metastases can have T2w signal hypointensities, metastatic melanoma can display T1w signal hyperintensities and metastatic breast cancer can demonstrate necrotic and cystic lesions.²⁵ Using the Brain Tumor Segmentation (BraTS) challenge datasets, recent DL methods have assessed metastases in multisequence magnetic resonance (MR) studies, including fluid‐attenuated inversion recovery (FLAIR) and pre‐ and post-gadolinium enhanced T1w images.^26–28 Such networks are therefore useful for stereotactic surgical planning.

Multiple Sclerosis

Multiple sclerosis (MS) diagnosis and progression depends on neuroimaging, wherein patients present with white matter lesions disseminated over space and time.²⁹ Hence, AI can be extremely beneficial for the quantification and tracking of specific lesions in individual patients. A variety of research AI models (including NNs, k-Nearest Neighbors and support vector machines (SVM)) exist for detecting MS lesions with high specificity and sensitivity.^30–32

Diverse Etiologies

Overall, many pathologies demonstrate hyperintense lesions on FLAIR sequence. Methods exist to segment white matter hyperintensities in diseases such as Alzheimer Disease (AD) and small vessel ischemic disease.^33,34 However, in the clinical workflow, the diagnosis that a patient presents with is not always known, and methods are required to recognize abnormalities despite diagnostic uncertainty. Recently, our group has shown that a CNN with U-net architecture can perform at near radiologist-level accuracy for detecting lesions across a variety of pathologies, at fractions of the time.³⁵ Such general task-specific, rather than disease-specific, approaches have the potential to be applied universally to neuroimaging. Ultimately, such methods may decrease the rate of perceptual errors in neuroradiology, which are the basis of most diagnostic errors.^36,37

3. ANATOMIC SEGMENTATION AND VOLUMETRY

Neuroanatomy is variable across individuals and difficult to segment computationally, particularly when anatomy is distorted by underlying pathology. However, AI has made significant inroads in the segmentation and parcellation of cortical and subcortical structures.^38,39 Such anatomical classification AI could be useful for planning stereotactic radiosurgery and surgical resection for pathologies from neoplasia to epileptogenic foci and heterotopias.

Atrophy is essential in diagnosing certain neurodegenerative diseases and AI is well-positioned to enable quantification of anatomical changes. CNNs and SVMs can compare volumetrics of different brain regions including the hippocampus in dementia^40–43 and basal ganglia in Huntington disease.^44,45 For white matter diseases, current DL networks can detect and measure abnormalities perform volumetrics.^34,35 In the future, these tools can be combined with enhanced anatomic linking to calculate changes over time.

4. PATIENT SAFETY AND QUALITY IMPROVEMENT

Protocoling

In addition to diagnosis, patient safety and image quality measures can be improved with AI. The first step of an imaging exam involving a radiologist is often the study protocoling, which can have a major impact on diagnostic accuracy at the time of image interpretation. Methods are being developed for standardizing and improving the protocoling process.⁴⁶

Contrast Agents

Contrast agents and radiotracers pose non-zero risks to patients. Recent studies suggest as few as two doses of gadolinium-based contrast agents can lead to gadolinium deposition in dentate nuclei, globi pallidi and other deep brain structures.⁴⁷ Accordingly, CNNs may reduce the gadolinium dose required to interpret enhancing lesions on brain MRI^48,49 and even radiation exposure on CT.^50,51

For nuclear medicine studies, DL algorithms are being developed to create “synthetic” amyloid positron emission tomography (PET) to reduce radiotracer volume injected. By training CNNs on PET images with lower signal-to-noise ratios, several studies have predicted how amyloid PET images would appear if the patient were given a full radiotracer dose based on PET images representing much lower (<1%) doses of tracer.^52–55 Additional studies leverage reconstruction PET with generative adversarial networks (GANs) to promote more rigorous training of AI systems for diverse inputs.⁵⁶

The clinical impact of such approaches remains to be rigorously studied. While the technical achievements of these dose-reduction algorithms are considerable, the underlying principles of information theory cannot be understated. That is, an AI algorithm cannot create signal if the information is not present in the first place. While much of MR image information is redundant, it is likely that in some unexpected cases, the administration of contrast adds information that cannot be gleaned from the remainder of the sequences. To what extent denoising algorithms will provide the correct predictions in these minority cases remains to be seen. Thus, before their clinical implementation, such algorithms must be diligently evaluated across a diverse spectrum of cases in the real clinical workflow.

Image Reconstruction

Once raw data are acquired, the information contained in sensor space must be reconstructed into an image. Understanding this process as a function that maps an input (sensor space) to an output (the image) reframes this problem as one that could be solved by an AI algorithm, bypassing traditional reconstruction algorithms including the traditional Fourier transform from K-space. Indeed, the AUTOMAP method was recently developed to perform this mapping.⁵⁷ While promising, such new methods will still need to be evaluated in the context of clinical MRIs to determine that they do not compromise patient safety.

Recall Rates

Proper diagnosis depends on image quality. AI can improve the speed and accuracy of diagnosis by reducing recalls and rescans due to artifacts such as motion. The requisite image quality for different scans depends on the indication and the reading radiologist. Therefore, AI for patient diagnosis should be adapted for each patient and radiologist. Indeed, a recent study demonstrated how DL models can reduce recall and rescan rates for imaging studies of MS and stroke.⁵⁸

5. PRECISION MEDICAL EDUCATION

“Precision medical education” is the use of tools such as AI to standardize trainees’ medical skills and knowledge yet also individualize the delivery and practice of those skills and knowledge. In line with this paradigm, our institution has developed the Adaptive Radiology Interpretation and Education System (ARIES).^59,60 ARIES is a Bayesian network where trainees can select a sub-system network (such as spine), input imaging and clinical features (such as T1 intensity, diffusion restriction and age) and receive a differential diagnosis from expert-derived prior probabilities. Probabilities can be computed from radiographic features alone or with radiographic and clinical data, allowing for a nuanced discussion of diagnosis.

For example, if a patient presents with deficits localized to the cervical spinal cord and imaging reveals a single, homogeneously enhancing and expansile intramedullary lesion that does not have diffusion restriction, the differential diagnosis includes ependymoma and astrocytoma (Figure 2). While imaging features may favor ependymoma, clinical features such as patient age (under 30) and gender (male) may favor astrocytoma.

Figure 2.

The Adaptive Radiology Interpretation and Education System (ARIES) distinguishes between ependymoma and astrocytoma in the spine network. (A) Features based on signal, spatial and clinical information are selected by the trainee in blue. Unselected features are grey and the most differentiating unanswered features are highlighted in orange. Differential diagnoses by (B) imaging features only vs (C) a combination of clinical and imaging features are derived from manually selected features (A). Probabilities are calculated by a naïve Bayes network.

Bayesian networks can be integrated with lesion and anatomical segmentation methods and teaching files. These AI-augmented teaching databases can create learning profiles for individual trainees, keep track of progress, and recommend relevant cases for trainees to achieve learning milestones.^20,61 Studies have shown that AI can assist education through optimally spaced repetition⁶² and the same may be true for practice case-based learning and high-fidelity simulation training. Because there is little data that AI can improve student outcomes, validation studies must be performed. Notably, AI may hold utility even for people who do not initially benefit from AI (Figure 3). For example, AI may enable adaptive learning for people who do worse or not sufficiently well when using AI and assist in adapting itself until the trainee benefits.

Figure 3.

Feedback loop framework for longitudinal AI-augmented precision education.

It is important to emphasize that the usage of AI systems is not to scrutinize trainees or replace educators. Instead, AI should be used as a support to trainees and a supplement to teachers as medicine progresses towards precision education.

6. MULTIMODAL INTREGRATION

Combining multiple imaging modalities improves everyday clinical diagnosis, progression tracking and prognostication. Ongoing studies aim to apply similar models and architectures, trained on multiple forms of data track progress and predict prognosis for many chronic diseases.

Multiple Sclerosis

Diagnosis of MS incorporates detection of lesions from neurologic exams and imaging. Such correlation benefits from conventional to advanced imaging sequences.²⁹ ML tools can integrate magnetization‐prepared rapid gradient‐echo (MPRAGE), magnetization‐prepared two inversion‐contrast rapid gradient‐echo (MP2RAGE), 3D FLAIR and double‐inversion recovery (DIR) for the detection of MS lesions.³¹ Additionally, joining NNs with graph theory approaches can classify distinct clinical subtypes of MS based on MR and diffusion tensor imaging (DTI).⁶³

Integrated data may not only improve MS diagnosis but also management. Treatment response to disease-modifying biologic therapy such as natalizumab can be predicted with high dimensional ML models.⁶⁴ In the future, AI used in longitudinal MRIs⁶⁵ may be integrated with clinical assessment, CSF biomarkers and genetic susceptibility for better diagnosis and management of MS (Figure 4).

Figure 4.

Model of multimodal integration of AI in diagnosis and assessment of disease progression in multiple sclerosis.

Epilepsy

Diagnosis and management in epilepsy may also be augmented with AI. Several SVMs have been shown to detect abnormalities in temporal lobe epilepsy with cortical morphology on T1w MRI and diffusion changes on DTI with kurtosis, mean diffusivity, and fractional anisotropy.^66,67 We envision a future AI system to integrate neuroimaging with DL/ML tools for electroencephalography (EEG) event classification^68–70 and DL-based wearable devices.⁷¹

Since timely anti-convulsant therapy is essential for improved outcomes in epilepsy,⁷² future AI may be applied to both diagnosis and management. DL models can predict treatment response to surgical resection.⁷³ AI could serve as a resource for detecting ictal events on surface and intracranial EEG, localizing abnormal neurological foci on imaging, and even planning anti-epileptic drug therapy and surgical resection and in post-treatment monitoring (Figure 5).⁷⁴

Figure 5.

Model of multimodal integration in diagnosis and management of epilepsy. Bilateral mesial temporal sclerosis demonstrates increased signal and volume loss in hippocampal MRI, temporal lobe hypometabolism on ¹⁸F-FDG-PET and ictal events on EEG.

Neurodegenerative Disease

About 1 in 3 Americans over the age of 85 die with AD or related dementias.⁷⁵ However, radiology is poised to improve the diagnosis and management of patients with dementia. Numerous AI networks have trained from large longitudinal datasets such as the Alzheimer’s Disease Neuroimaging Initiative (ADNI), resulting in many diagnostic DL methods for AD, such as models using ¹⁸F-fluorodeoxyglucose (FDG) PET⁷⁶ and structural MRI of the hippocampus⁷⁷ to predict AD onset from 1 to 6 years in advance. Hence, AI may facilitate diagnosis and progression tracking in dementia.

The diagnosis of dementia types is not always straightforward and AI may assist in differential diagnosis. SVMs can distinguish AD from Lewy body and Parkinson’s dementia.⁷⁸ Similarly, CNNs can differentiate mild cognitive impairment (MCI) and AD.⁷⁹ In addition to diagnosis, progression and prognosis, integrative AI can also probe neurobiology. New ML techniques such as Subtype and Stage Inference (SuStaIn) have provided novel neuroimaging and genotype data-driven classifications of diagnostic subtypes and progressive stages for AD and frontotemporal dementia (FTD). SuStaIn has localized distinct regional hotspots for atrophy in different forms of familial FTD caused by mutations in genes including C9orf72, GRN and MAPT.⁸⁰ Thus, neuroimaging AI tools may encourage novel neurobiological discoveries.

Because of the wide prevalence, long natural history and complex clinical landscape of neurodegenerative disease, the integration of many sources of imaging and clinical information is essential.⁸¹ DL has already integrated MRI, neurocognitive and APOE genotype information to predict conversion from MCI to AD.⁸² Combining several AI systems (including structural MRI and amyloid PET) together may augment diagnosis and management along the complex natural history of AD (Figure 6). We envision that AI tools for imaging will be integrated with AI systems designed to examine serum amyloid markers,^83,84 mortality prediction from clinicians’ progress notes and assessments of cognition⁸⁵ and postmortem immunohistochemistry images.⁸⁶ This comprehensive suite of AI models may improve many facets of care in neurodegenerative disease.

Figure 6.

Model of multimodal integration of AI for diagnosis and management in Alzheimer disease. (A) Natural history of Alzheimer disease (adapted from⁸¹). (B) AI-enabled multimodal integration of input features for longitudinal (re)assessment of risk, diagnosis and progression.

In Huntington Disease, an autosomal dominant movement disorder, diagnosis and management may be enhanced by incorporating CAG repeat length data with NNs developed for caudate volumetry,⁴⁵ objective gait assessment⁸⁷ and EEG.^88,89 Such multi-pronged approaches may improve risk stratification, progression monitoring and clinical management in patients and families.

7. TOWARDS A GENERALIZED NEUROIMAGING AI

Currently in its infancy stages, AI has potential to support neuroradiological diagnosis in drafting reports for radiologists to review. Yet, differential diagnosis in neuroradiology is particularly complex. A “generalized” AI capable of end-to-end neuroimaging diagnosis (where inputs are imaging studies and the output is a draft report) will require abilities to: extract imaging features, process clinical notes, parcellate neuroanatomy, construct differential diagnoses, recognize potential sources of artifact, compare with prior studies and more. Future development of more inclusive AI systems may go beyond imaging quantification and incorporate studies from nuclear medicine, electrophysiology, laboratory findings and clinical data. Such generalized AI tools may put these quantified numbers in perspective and eventually generate a comprehensive radiology report, which can be used to make informed clinical decisions in multi-disciplinary discussions. This will require the integration of many different AI systems trained on a vast wealth of neurological conditions.

Even if achievable, an end-to-end AI diagnostic resource for clinicians and radiologists will still require substantial human oversight and supervision. It is essential that radiologists stay in the loop on all aspects of imaging, from ordering the correct study to supervising its quality and interpretation. The significance of the question asked by an imaging order, and its clinical context, cannot be understated; it is currently difficult to imagine an AI algorithm able to account for the unique context in which an imaging study is ordered for a patient. Additionally, as data specialists, radiologists must provide continual, real-time feedback to update the AI system.⁹⁰ This enables physicians to serve as part of the “checks and balances” between human and machine.

Current research trends (Figure 1) do suggest that early prototypes of such a generalized AI are on the horizon. These proposed integrated and knowledgeable AIs build upon existing multi-organ system AI systems.^91,92 An effective generalized AI should also promote precision medical education and produce information that is “interpretable” and “explainable” for trainees and physicians alike.⁹³ While AI is still far from generalized utility, it holds promise to one day improve neuroimaging at all levels, from processing and diagnosis to education and management.

DISCUSSION

AI has myriad neuroimaging applications. Here, we presented several clinical tasks well-suited for AI (including workflow organization, lesion and anatomical segmentation, quality, safety and multimodal integration), for a variety of neurological diseases (trauma, epilepsy, MS, neurodegeneration, etc.)

However, there are many challenges facing AI for such applications. First, effective AI requires immense data for training and validation. Many current neuroimaging databases are disease-specific (such as ADNI or BraTS), so it will be important to integrate many regional and international imaging networks to sufficiently train AI for a variety of diagnoses and patient populations. Second, multimodal AI tools are computationally costly, and significant investment in infrastructure is required. Third, not only are current AI systems all uniquely trained, but they are also uniquely designed. Rigorous best practices are required in designing and validating AI networks. For multimodal integration, it is important to assess the effect of each individual input modality toward output detection and/or diagnosis. Hence, strategies such as “leave-one-out” and accuracy metrics for segmentation and diagnosis should be standardized.⁹⁴ Fourth, clinical AI must balance accuracy and explainability. While clinical decision support systems rely on known features and algorithms, current DL methods are still “black boxes.” Rigorous methods should be developed to allow for better model introspection, which fall under the recent explainable AI methods; such methods are beyond the scope of this review but will be essential for adoption of AI in healthcare.^93,95 Indeed, our group has demonstrated that an AI system combining deep learning and Bayesian networks for brain MRI can generate accurate and interpretable output such as differential diagnoses and lesion characteristics across a variety of neurological disorders^96,97. Fifth, adoption will depend on demonstrated improvement in patient outcomes and radiologist efficiency and collective agreement on the benefits of AI. This will require studies designed similar to clinical drug trials for trainees and attending radiologists. Akin to phase 4 studies, AI tools must be monitored after adoption to fine-tune algorithms. Sixth, physicians must be trained in using AI to recommend such algorithm adjustments. It is likely that an emerging subset of physicians will work with the informatics community to continually monitor and update AI tools. Overall, AI has many diverse applications in neuroimaging currently being validated across the new landscape of precision medicine.

KEY POINTS.

• Artificial intelligence (AI) has potential to improve efficiency and accuracy in neuroradiology.

• AI tools are starting to be validated for tasks such as workflow optimization, segmentation and precision education for a variety of brain diseases.

• Future AI may integrate imaging data with additional diagnostic data from the medical record.

SYNOPSIS.

Recent advances in artificial intelligence (AI) and deep learning (DL) hold promise to augment neuroimaging diagnosis for patients with brain tumors and stroke. Here, the authors review the diverse landscape of emerging neuroimaging applications of AI, including workflow optimization, lesion segmentation, and precision education. Given the many modalities used in diagnosing neurologic diseases, AI may be deployed to integrate across modalities (MR imaging, computed tomography, PET, electroencephalography, clinical and laboratory findings), facilitate crosstalk among specialists, and potentially improve diagnosis in patients with trauma, multiple sclerosis, epilepsy, and neurodegeneration. Together, there are myriad applications of AI for neuroradiology.