Deep learning based detection of intracranial aneurysms on digital subtraction angiography: A feasibility study

Abstract

Background

Digital subtraction angiography is the gold standard for detecting and characterising aneurysms. Here, we assess the feasibility of commercial-grade deep learning software for the detection of intracranial aneurysms on whole-brain anteroposterior and lateral 2D digital subtraction angiography images.

Material and methods

Seven hundred and six digital subtraction angiography images were included from a cohort of 240 patients (157 female, mean age 59 years, range 20–92; 83 male, mean age 55 years, range 19–83). Three hundred and thirty-five (47%) single frame anteroposterior and lateral images of a digital subtraction angiography series of 187 aneurysms (41 ruptured, 146 unruptured; average size 7±5.3 mm, range 1–5 mm; total 372 depicted aneurysms) and 371 (53%) aneurysm-negative study images were retrospectively analysed regarding the presence of intracranial aneurysms. The 2D data was split into testing and training sets in a ratio of 4:1 with 3D rotational digital subtraction angiography as gold standard. Supervised deep learning was performed using commercial-grade machine learning software (Cognex, ViDi Suite 2.0). Monte Carlo cross validation was performed.

Results

Intracranial aneurysms were detected with a sensitivity of 79%, a specificity of 79%, a precision of 0.75, a F1 score of 0.77, and a mean area-under-the-curve of 0.76 (range 0.68–0.86) after Monte Carlo cross-validation, run 45 times.

Conclusion

The commercial-grade deep learning software allows for detection of intracranial aneurysms on whole-brain, 2D anteroposterior and lateral digital subtraction angiography images, with results being comparable to more specifically engineered deep learning techniques.

Introduction

Intracranial aneurysms are the number one cause of non-traumatic subarachnoid haemorrhage¹ and have an estimated prevalence of 2–3%.^2,3 Most aneurysms are located in the anterior cerebral circulation (87%), frequently involving the anterior communicating artery (Acom), the internal carotid artery (ICA) and the middle cerebral artery (MCA).⁴ Aneurysms are usually detected on CT angiography (CTA) or MR angiography (MRA) as part of a work up of intracranial haemorrhage, for symptoms such as headache and cranial nerve palsies, or are discovered incidentally.^5–8 Digital subtraction angiography (DSA), a minimally invasive radiographic procedure, is the gold standard for detecting and characterising aneurysms,^8–10 with visualised aneurysms reported as small as 0.5 mm.¹¹ The sensitivity of DSA for aneurysm detection is directly influenced by operator experience in, for example, choosing which additional vessel projections to acquire, and interventional neuroradiology is described as having a long learning curve.^11–14

Deep learning is an artificial intelligence-based technique using convolutional neural network (CNN) architecture;¹⁵ studies have already demonstrated sensitivities of 93–100% for deep learning algorithms in the detection of aneurysms on MRA.^16,17 Previous deep learning publications focussing on DSA/aneurysms are specifically engineered for aneurysm segmentation,¹⁸ use a two-stage technique to locate specific regions before aneurysm detection,¹⁹ use a spatial information fusion method on 3D rotational angiography²⁰ or incorporate temporal information from multiple DSA frames.²¹ The goal of our study was to determine if commercial-grade deep learning software with previously described applications in mammography^22,23 and radius fracture detection²⁴ could detect intracranial aneurysms using only standard, whole-brain anteroposterior and lateral projections of DSA.

Material and methods

Dataset

Institutional Review Board (IRB) approval by the local ethics committee was obtained prior to beginning this retrospective study. A radiological information system (RIS)-based search for aneurysms on DSA was performed between June 2014–November 2018.

Two hundred and forty patients (136 patients with aneurysms (57%), 104 patients without aneurysms (43%); 157 female, mean age 59 years, range 20–92; 83 male, mean age 55 years, range 19–83) were included in this retrospective study. The inclusion criteria were defined as: age>18 years, available DSA of sufficient image quality with presence of one or more aneurysms (aneurysm patients) or absence of aneurysms and any other pathological findings (aneurysm negative patients). Patients were excluded if surgical material was present on images – this included external osteosynthesis material, ventricular drains, staples, aneurysm clips and coils. Also, due to timing of filling differences, some aneurysms were not visible in the selected arterial contrast phase – these patients were excluded. Of 338 images, 335 single frame anteroposterior and lateral images of a DSA series of 186 aneurysms were included (41 ruptured and 145 unruptured; average size 7.0 mm, standard deviation ± 5.2 mm, range 1–25 mm; total of 372 depicted aneurysms. 23 patients (10% of cases) had multiple aneurysms of which 16 patients (7% of cases) had two aneurysms/image (including one patient on both sides), and two patients (1% of cases) had three aneurysms/image). Three images were excluded (two lateral, one anteroposterior) as the aneurysm was not opacified due to contrast washout. Of 372 images, 371 aneurysm negative study images were retrospectively analysed regarding the presence of intracranial aneurysms. One lateral image was excluded due to motion artefact at the desired timepoint. All DSA images were from the anterior cerebral circulation only.

Aneurysm totals

Forty-nine anterior cerebral artery (ACA; from the A1 to A3 segment, 37 of which were at the anterior communicating artery (Acom)), 40 MCA (from the M1 to M3 segment, 35 of which were at the media bifurcation), and 97 along the ICA (from the extracranial to the communicating segment).

Aneurysm image totals

Twenty-four ACA (6% average size 4.9 ± 2.0 mm), 74 Acom (20% average size 6.1 ± 4.3 mm), 10 MCA (3% average size 4.2 ± 2.0 mm), 69 MCA bifurcation (19% average size 8.6 ± 7.0 mm), 195 ICA (52% average size 7.2 ± 5.0 mm).

All patients with negative studies (for any pathological finding) in the anterior cerebral circulation underwent DSA upon indication by a physician due to the following: 57 (55%) patients with nontraumatic haemorrhage, 16 (15%) suspected vasculitis, nine (9%) stenoses/occlusions of the posterior cerebral circulation (vertebrobasilar system) and 22 (21%) with aneurysms of the opposite side or posterior circulation.

DSA

All DSAs were performed transfemorally with 5F catheters using an Artis Zee Interventional Angiography System (Siemens Healthcare GmbH, Erlangen, Germany). DSA was performed with bilateral selective ICA and vertebral artery injections. An amount of 8 ml of non-ionic contrast medium (Optiray 300, Guerbet, France) was hand-injected and standard anteroposterior and lateral views were routinely acquired.

Single frames of the arterial contrast phase were extracted from the acquired DSA series upon instruction by two board certified neurointerventionalists (VA and ZK) with 7 and 18 years of experience in interventional neuroradiology, respectively. The presence and size of aneurysms (using electronic callipers) was determined in consensus using the original report based on information from 3D rotational DSA images and the aneurysm was labelled as a region of interest (ROI) on the image.

Deep learning and statistical analysis

After removing protected patient information, examinations were retrieved from the local picture archiving and communication system (PACS). In total, 706 DSA Digital Imaging and Communications in Medicine (DICOM) images were analysed. For image perturbation, rotation, scale and shear were varied. The data was initially split into testing and training sets in a ratio of 4: 1 (80%/20%) . Classification results were based solely on unseen validation data. In order to account for overfitting, we repeated the deep learning analysis by use of repeated random sub-sampling validation. By using this so called Monte Carlo cross-validation method, we randomly selected a fraction of the data set to form the training set, and then assigned the rest of the data points to the test set. This process is then repeated multiple times, generating (at random) new training and test partitions each time. The partitions are done independently for each run and the final results were averaged. Sensitivity, specificity, precision, F1 score and the area-under-the-curve (AUC) from receiver-operating characteristic (ROC) analysis was calculated.

Deep learning was performed by use of commercial-grade machine learning software (Cognex, ViDi Suite 2.0, Cognex Inc., Natick, Massachusetts, USA) in supervised mode. ViDi was originally developed as a quality control tool on assembly lines, is not approved for routine clinical use, but has shown promise in radiological studies with applications in mammography and radius fracture detection.^22–24 ViDi learns by example, sweeping the image with a circular feature window, creating a model of typical imaging features in the training set which is then used to estimate whether the validation set’s features are within a tolerable range.²⁴ Computation was performed on a desktop personal computer with a dedicated graphical processing unit (Nvidia GeForce GTX 1080) (Figure 1).

Figure 1.

Flowchart of image processing from patient cohort to deep learning classification results. The grey rectangle signifies steps performed by the deep learning software beginning with image augmentation to splitting the data into validation and training sets for supervised learning. DSA: digital subtraction angiography; A-P: anteroposterior.

Results

User-defined ROIs placed on the aneurysms were represented as circles containing diagonal red lines by the software. Saliency maps representing increased conspicuity of pixel values to represent an aneurysm were provided by the software. These maps were represented in green and overlaid onto all images for visual inspection of the CNN’s performance. If the CNN’s threshold for aneurysm was surpassed within these areas, a dashed red circle was overlaid onto the green heat map by the software indicating 'aneurysm'. This can be seen in the case of a true positive aneurysm of the ophthalmic segment of the ICA (Figure 2) and a false positive at the same location in a different patient (Figure 3). There is no ROI with diagonal red lines in Figure 3 as no aneurysm was defined by the authors. For false negatives, only the user-defined ROI is seen; in some of these cases, areas of the green heat map can be appreciated, however no dashed red line is seen as the threshold for aneurysm detection was not met (Figure 4).

Figure 2.

Example of a true positive result: (a) the 5 mm saccular aneurysm of the right Internal carotid artery, ophthalmic segment, was correctly detected on an anteroposterior view. The user-defined region of interest (ROI; circle with diagonal red lines) and the ROI drawn by the deep learning software (red dashed circle) show a perfect overlap centred over the aneurysm. The subtle green heat map seen in the periphery of the red ROI indicates the saliency map provided by the software. The 3D rotational digital subtraction angiography (DSA) image (b) serves as the gold standard for aneurysm detection (arrow).

Figure 3.

Example of a false positive result: (a) the deep learning software incorrectly labelled the ophthalmic segment of the internal carotid artery (ICA) as an aneurysm on an anteroposterior view (red dashed circle). The green saliency maps reveal weighting around the ophthalmic segment and at the ICA bifurcation. The result seems plausible given the course of the artery and rounded shape protruding from the vessel which can be appreciated on the enlarged image (b). The additional oblique view (c) at the same level (arrow) confirms the location to be aneurysm-negative.

Figure 4.

Example of a false negative result: the aneurysm of the Internal carotid artery (ICA), ophthalmic segment, was not detected by the deep learning software. On image (a) only the user-defined region of interest (ROI; red) can be found. The saliency map (green) indicates weighting near this aspect of the Internal carotid artery, however the structure did not meet the deep learning network’s threshold. Further weighting is seen at the ICA bifurcation. The image without the ROI (b) makes the result seem plausible as the aneurysm appears hidden by the course of the artery on the anteroposterior view. 3D rotational digital subtraction angiography (DSA) (c) image reveals the aneurysm (arrow).

Intracranial aneurysms were detected with a sensitivity of 79%, a specificity of 79%, a precision of 0.75 and a F1 score of 0.77. After the initial run using an 80% training/20% validation split and Monte Carlo cross-validation (run 45 times) using randomly generated training/validation partitions, a mean AUC of 0.76 with a range from 0.68–0.86 was achieved.

Discussion

In this study, we were able to demonstrate the feasibility of applying commercial-grade deep learning software to DSA images for the detection of intracranial aneurysms with a good level of sensitivity and specificity. The applied deep learning algorithm was able to correctly detect 79% of the validation-set intracranial aneurysms on whole-brain anteroposterior and lateral views of the anterior cerebral circulation with a mean AUC of 0.76 (range 0.68–0.86).

Previous studies investigated the potential of computer-assisted detection and localization of intracranial aneurysms in magnetic resonance imaging (MRI). Sichtermann et al. used a CNN-based approach to assess the ability to detect intracranial aneurysms in 3D time-of-flight (TOF) MRI data, reporting a high sensitivity for aneurysm detection depending on the aneurysm size between 90–100%.¹⁶ Similar high sensitivities were reported by Nakao et al., who also investigated CNN in 3D TOF datasets.¹⁷ Discrepancies in sensitivity compared with our study can be explained by the two single frames and 2D nature of the DSA images with associated vessel overlap, obscuring some aneurysms in our study, compared to the 3D TOF datasets based on MRI used by Sichtermann et al. and Nakao et al.

Previous studies employing 2D DSA images include a two-stage CNN architecture proposed by Duan et al.¹⁹ Here, a first step focuses in on a specific region to reduce interference from other regions with a second step dedicated to aneurysm detection. With this technique, they achieved an AUC of 0.942 and an accuracy of 93.5%. Jin et al. used temporal information from a 2D DSA series, capturing contrast medium flow change, and incorporated this into a U-shaped deep neural network reporting detection of 89.3% of aneurysms on an independent test set.²¹ Zeng et al. utilised a 3D rotational angiography spacial information fusion method to create 2D DSA images and report a sensitivity of 99.38% and a specificity of 98.19% of their CNN. The utilization of 3D-rotational angiography data brings with it the corresponding radiation dose which is higher than the 2D images used in our study.²⁰

Many factors including aneurysm size, location, patient age, presence of hypertension and previous subarachnoid haemorrhage are considered when assessing risk of rupture of an intracranial aneurysm. Generally, aneurysms larger than 7 mm with aneurysm locations along the ACA, posterior communicating artery and posterior circulation are thought to have a higher rupture risk.²⁵ Tiny aneurysms, however, such as blister aneurysms measuring less than 1 mm commonly located along the ICA can also rupture.^26,27 Thus, the correct detection, localization and characterization of aneurysms on DSA is essential regardless of size and other factors. The sensitivity of DSA for aneurysm detection is directly influenced by operator experience¹¹ and the successful performance of DSA is described as having a long learning curve.^12–14 A joint statement from the American Academy of Neurology, the American Association of Neurological Surgeons, the American Society of Interventional and Therapeutic Neuroradiology, the American Society of Neuroradiology, the Congress of Neurological Surgeons, the AANS/CNS Cerebrovascular Section and the Society of Interventional Radiology goes even further to claim that 'knowledge of the brain and the ability to correctly interpret a cervicocerebral angiogram is the prerequisite and foundation for the technical performance of cervicocerebral angiography' (p. 194).²⁸ While training simulators such as models and virtual reality can help develop technical skills,^29–31 we have shown that commercial-grade deep learning software can be applied to interpret an angiogram.

Deep learning is a type of representation learning where the learned features are hierarchical. This subform of machine learning automates feature engineering by taking advantage of large quantities of data and flexible hierarchical models.³² Deep learning has recently achieved performance improvements in diverse fields such as image classification, speech recognition, natural language processing and playing games.³³ Deep learning systems encode features by using an architecture of artificial neural networks, an approach consisting of connected nodes inspired by biological neural networks. The architecture is inspired by biological neurons as it takes an input set of values representing features, each multiplied by a corresponding weight. The weighted features are then summed and passed through a nonlinear activation function. An artificial neuron can be viewed as producing a decision by weighing a set of evidence. Neural network architectures called multilayer perceptrons consist of thousands of artificial neurons that can represent very complex non-linear functions. These multilayer perceptrons are typically constructed by assembling multiple neurons to form a layer and by stacking these layers, connecting the output of one layer to the input of the following layer. The composition of features in deep neural networks is enabled by local characteristics and regularities that allow for building complex models derived from relatively small local features. CNNs exploit the same property to efficiently process larger and more variable inputs than is reasonable with multilayer perceptrons. CNNs are well suited for analysis of medical images in which the object of interest tends to vary in shape, orientation and position. CNNs address complex tasks such as image classification with an efficient model architecture based on the following components: convolutions, activation functions, pooling and softmax function.³²

With this study we were able to demonstrate the feasibility of applying commercial-grade deep learning software to whole-brain anteroposterior and lateral 2D DSA images for the detection of intracranial aneurysms. Aneurysms were correctly detected among the multitude of overlapping and tortuous arteries with the resulting differences in contrast attenuation, a difficult task most pronounced along the distal ICA, proximal ACA/Acom and MCA bifurcation on lateral views. This challenge is amplified using only a single contrast-timepoint from a series of images obtained in the dynamic study without specifically engineered deep learning techniques focussing on DSA and aneurysms. While a mean AUC of 0.76 indicates that the examined 2D approach of reading DSA images is of inferior diagnostic capability compared to human readers assessing the corresponding 3D images, our study demonstrates potential directions for future research including advanced aneurysm assessment – such as investigations into rupture risk – or in stroke imaging.

Limitations

We used anteroposterior and lateral views of the anterior cerebral circulation only (injections of the ICA), as 87% of intracranial aneurysms occur here.⁴ Deep learning networks require large amounts of data; thus, the relatively low case number of aneurysms in the posterior cerebral circulation did not fit this requirement. Second, excluding patients with posterior circulation aneurysms, surgical material, coils and clips on images is a limitation as it does not represent what is seen in clinical practice. We feel, however, that this would have introduced a bias, as patients with hardware such as an external ventricular drain are more likely to have an aneurysm than those without. Third, training and validation datasets were separated based on images, not subjects, which also deviates from clinical practice and may overestimate the performance of the algorithm. Furthermore, the lack of an external dataset limits interpretation of our results. There was no criterion regarding optimal aneurysm contrast opacification on the DSA images. Due to contrast dynamics, aneurysms may wash out partially or completely on the selected arterial phase, only images with complete wash-out were excluded. 3D rotational DSA is known to have a higher sensitivity for aneurysm detection than the 2D images used in this study.^34,35 Furthermore, only one vendor system for DSA was used. While we did not directly analyse the 3D images, they served as a gold standard to help detect and localise aneurysms on the 2D images; thus, their additional information was applied to our study. Finally, while we cannot comment on exact specifications of the commercial software’s CNN, the recently published editorial by the Radiology Editorial Board states that 'Commercially available algorithms are considered publicly available' and can thus be independently validated.³⁶

Conclusion

In conclusion, we demonstrated the feasibility of commercial-grade deep learning software for the detection of intracranial aneurysms on whole-brain, 2D anteroposterior and lateral DSA images, with results being comparable to more specifically engineered deep learning techniques.

Conflict of interest

The authors declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.

Funding

This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.

ORCID iDs

Nicolin Hainc https://orcid.org/0000-0003-0916-7387

Christian Blüthgen https://orcid.org/0000-0001-7321-5676

Andrea Bink https://orcid.org/0000-0002-2163-3400