Outcome prediction of intracranial aneurysm treatment by flow diverters using machine learning

Abstract

Objective

Flow diverters (FDs) aim to occlude intracranial aneurysms (IAs) while preserving flow to essential arteries. Incomplete occlusion exposes patients to risks of thromboembolic complications and rupture. A priori assessment of FD-treatment outcome could enable treatment optimization leading to better outcomes. To that end, we applied image-based computational analysis on FD-treated aneurysm patients to extract morphology, pre- and post-treatment hemodynamics, and FD-device characteristics and trained machine-learning algorithms using these features to predict their 6-month clinical outcome.

Methods

Eighty-four FD-treated sidewall aneurysms in 80 patients were retrospectively collected. Based on 6-month angiographic outcomes, IAs were classified as “occluded” (n=63) or “residual” (incomplete occlusion, n=21). For each case, we modeled FD deployment using a fast virtual stenting algorithm and hemodynamics using image-based CFD. Sixteen morphological, hemodynamic and FD-based parameters were calculated for each aneurysm. Aneurysms were randomly assigned to a training or testing cohort in a 3:1 ratio. Student’s t-test and Mann-Whitney U-test were performed on the training cohort to identify significant parameters distinguishing the occluded from residual groups. Predictive models were trained using 4 types of supervised machine-learning algorithms: logistic regression (LR), support vector machine (SVM, linear and Gaussian kernels), k-nearest neighbor (k-NN) and neural network (NN). On the testing cohort, we compared outcome prediction by each model using all parameters vs only the significant parameters.

Results

Training cohort (n=64) consisted of 48 occluded and 16 residual aneurysms; testing cohort (n=20) consisted of 15 occluded and 5 residual aneurysms. Significance tests yielded 2 morphological (ostium ratio and neck ratio) and 3 hemodynamic (pre-treatment inflow rate, post-treatment inflow rate and aneurysm-averaged velocity) discriminants between the occluded (good-outcome) and the residual (bad-outcome) group. In both training and testing, all the models trained using all 16 parameters performed better than all the models trained using only the 5 significant parameters. Among all-parameter models, NN (AUC=0.967) performed the best during training, followed by LR and linear-SVM (AUC=0.941 and 0.914, respectively). During testing, NN and Gaussian-SVM models had the highest accuracy (90%) in predicting occlusion outcome.

Conclusions

NN and G-SVM models incorporating all 16 morphological, hemodynamic and FD-device parameters predicted 6-month occlusion outcome of FD treatment with 90% accuracy. More robust models using the computational workflow and machine learning could be trained on larger patient databases towards clinical use in patient-specific treatment planning and optimization.

IN endovascular treatment of intracranial aneurysms (IAs), flow diverters (FDs) have emerged as an alternative paradigm to coil embolization, particularly in treating wide-neck and challenging aneurysm morphologies.^6,28 Deployed across an aneurysm ostium, the densely woven mesh of FD induces flow stasis in the aneurysmal sac, promoting thrombotic conditions and eventual occlusion of the IA. The mesh-like structure of FDs also facilitates endoluminal reconstruction of the parent artery.^18,34 Over the past few years, FDs have been one of the mainstays of endovascular intervention. However, despite this, approximately 25% of FD-treated IAs fail to reach complete occlusion even after 6 months.^6,15 These FD-treated patients, who experience persistent residual filling in the aneurysm sac, are at risks of thromboembolic complications and aneurysm rupture.^16,32 A priori assessment of FD-treatment outcome could aid the clinicians in treatment optimization and lead to better outcomes.

For coil embolization, recanalization have been shown to correlate with aneurysm morphometrics such as size, neck-to-dome ratio and neck size, which have been used to gauge coil treatment outcome.¹¹ However, for FD-treated IAs, these metrics have not been found to correlate with occlusion outcome.^3,6 Instead, other morphological and hemodynamic metrics have been proposed specifically for FDs that correlate with occlusion outcome. Gentric et al.¹² showed that a large aneurysm ostium is associated with incomplete occlusion after FD-treatment. Mut et al.²¹ demonstrated pre- and post-treatment inflow rate, post-treatment aneurysm averaged velocity, and post-treatment shear rate, were significantly different between occluded and non-occluded IAs after 6 months of FD-treatment. However, it remains unclear if these parameters can predict the FD treatment outcome.

In order to develop models for predicting clinical outcome of FD-treated IAs, we surveyed potential candidate algorithms. In IA research, multivariate logistic regression on untreated morphological and hemodynamic parameters have been used to classify aneurysm rupture status.³⁹ In other areas of medical research, novel machine learning (ML) algorithms have emerged as alternatives to traditional statistical methods to predict clinical outcomes, e.g., using medical imaging data to classify brain tumors and heart diseases.^2,31,36 Unlike univariate statistical analysis that focus on identifying the independently significant differences between averages of parameters in different populations, ML algorithms use given features on the available data on a case-by-case basis to predict an outcome.^2,19,22,27 Furthermore, studies have also shown that ML algorithms could extract nonintuitive linear and nonlinear combinations of parameters, which themselves may be insignificant in univariate statistical analyses.²²

In this study we explored 4 ML algorithms, including not only the most recent algorithms such as support vector machine (SVM), k-nearest neighbor (k-NN) and neural network (NN), but also the traditional multivariate logistic regression, in training predictive models for FD treatment outcome, and compared their performances. The purpose was to establish a proof of concept for mining bigger data towards building models to potentially aid clinicians in a priori treatment planning and optimization.

Methods

Patient Selection

Data for patients treated using the commercial FD Pipeline embolization device (PED) (Medtronic) at the Gates Vascular Institute between 2009 and 2017 were retrospectively collected for this study. The inclusion criteria were the presence of a sidewall aneurysm located at the internal carotid artery (ICA) and treated using a single PED, availability of 6-month follow-up angiographic image and pre-treatment 3D rotational digital subtraction angiography (DSA) image. Aneurysms with prior treatment and those with insufficient quality of pre-treatment DSA were excluded from this study. Approval for the collection and review of patient data was obtained from the institutional review board at the University at Buffalo. Patient consent was waived by the board due to retrospective use of de-identified data. Pre-treatment 3D-DSA images, 6-month clinical outcome data, and general clinical and demographic data were collected for cases that satisfied the inclusion criteria. Based on their angiographic outcome at 6-month follow-up, aneurysms were dichotomized as occluded (complete occlusion) or residual (contrast filling at the neck/dome). Aneurysms were randomly assigned to either the training or the testing cohort in a 3:1 ratio, keeping the ratio of occluded to residual cases equal in the 2 cohorts.

Virtual FD Deployment and Computational Fluid Dynamics Setup

The 3D-DSA images of FD-treated IAs obtained before treatment were segmented using an open-source software package vascular modeling tool kit (vmtk, www.vmtk.org)¹ to obtain surface representation of the vascular geometry of each aneurysm. We could not directly obtain the deployed FDs in post-treatment images due to lack of adequate imaging resolution and artifacts. Therefore, to deploy the FD device in the computational model for each aneurysm, we used our previously reported virtual stenting workflow.^25,26 Details of the workflow are provided in the Supplementary Material. To accurately represent the patient-specific treatment, the actual specifications of the FD-device (diameter and length) was imported into the workflow. To obtain hemodynamics, we ran image-based computational fluid dynamic (CFD) simulations. For each aneurysm, two simulations were performed: untreated and treated with the modeled FD. Technical details of the CFD simulation setup is provided in the Supplementary Material.

Parameters Calculated: Morphological, Hemodynamic, and FD-Related

From 3D IA models, virtual FD deployment, and untreated and treated CFD simulations, morphology, FD-related, and hemodynamic parameters were calculated for each aneurysm. Previously studied morphology metrics,¹⁰ including aneurysm size, neck diameter (ND), size ratio (SR), aspect ratio (AR), and the novel morphometrics neck ratio (NR) and ostium ratio (OsR), were calculated. NR is defined as the ratio of clinical aneurysm neck diameter and parent vessel diameter, and OsR is defined as the ratio of aneurysmal ostium surface to the remaining circumferential surface area of the parent artery. Size, ND, SR, and AR were calculated on the 3D IA models using the integrated clinical software AView;³⁷ NR was calculated on the pre-treatment 2D-DSA images of patients; and OsR was calculated on the 3D surface IA models using a standalone MATLAB workflow.

FD-related parameters, metal coverage rate (MCR) and pore density (PD) were quantified based on virtual FD-deployment results on patient IA models. MCR and PD were calculated across the aneurysm orifice with the deployed FD visible to capture its deployment at the neck.⁹ MCR quantifies the relative area covered by the FD struts at the neck as opposed to open space (pores), and PD quantifies the number of pores per unit area.¹⁷ Virtual FD deployment images of each patient at the aneurysmal neck were obtained in STAR-CCM+, and a standalone MATLAB code was written to quantify the MCR and PD from the deployment images.

From CFD simulation results, time-averaged flow parameters were quantified to measure the effect of FDs on aneurysm hemodynamics. To quantify the pre- and post-FD intra-aneurysmal flow activity, magnitudes of aneurysm averaged velocity (AV) and shear rate (SHR) were volume-averaged inside the aneurysm sac. Flow stasis was quantified by aneurysm inflow rate (IR) and turnover time (TT), defined as the aneurysm sac volume divided by the inflow rate at the neck plane. Increasing aneurysmal flow turnover time can accelerate blood clotting and thrombotic occlusion of the aneurysms.¹⁷ These hemodynamic parameters have previously shown association with the occlusion outcome of FD-treated IAs.^21,38 The subscripts “pre” and “post” were used to distinguish between the untreated and treated hemodynamic values for each parameter.

A total of 16 parameters were calculated for each aneurysm, including 6 morphology-based parameters—size, ND, SR, AR, NR, and OsR; 2 FD-related parameters—MCR and PD; and 8 hemodynamic-based parameters—AV_pre, AV_post, SHR_pre, SHR_post, IR_pre, IR_post, TT_pre, and TT_post.

Statistical Analysis

Statistical analysis was performed on the patient clinical, demographic, morphological, FD-related, and hemodynamic parameters in the training cohort to identify those that differed significantly between the occluded and residual groups. A Shapiro-Wilk test was performed to check for normality of the continuous variables. Differences in parameters between the 2 groups were tested using the Mann-Whitney U-test (for nonnormally distributed data) or Student t-test (for normally distributed data). For categorical variables, a chi-square test was used to test for significant differences between the groups. Statistical significance was defined as p<0.05. All continuous parameter values were subsequently expressed as mean±standard error (SE). Before training the ML models, values of each parameter in the testing and training cohorts were normalized to have a mean value of zero and a standard deviation of 1.

Machine Learning Algorithms

Supervised ML algorithms with binary classification were used to build predictive models. Four ML algorithms were selected for model building since these have shown good performance in clinical healthcare classification studies.^{20,30,31,35,36} The models include the standard statistical logistic regression (LR), support vector machine (SVM, with linear [L-SVM] and Gaussian [G-SVM] kernels), k-nearest neighbor (k-NN), and neural network (NN). An illustration of the concept of each algorithm is shown in Fig. 1. As shown in Fig. 1, LR uses a linear classification line to separate the two groups, SVM uses either a linear (L-SVM) or non-linear (Gaussian, G-SVM) kernel to identify the hyperplane that maximizes the distance between the two groups, k-NN performs predictions on a new data-point based on its euclidean distance from its ‘k’ neighbors, and NN uses a system of interconnected layers that use back-propagation while training to generate non-intuitive combinations of parameters to optimize classification model. Descriptions of each algorithm is provided in the Supplementary Material. Each algorithm was used to train two sets of predictive models: 1) using all 16 parameters as input, and 2) using only the statistically significant parameters.

Fig. 1.

Illustration of different ML classification algorithms. A: Logistic regression where a classification line is fitted on the data. B: Support vector machine, where the best hyper-plane that separates the data is identified by maximizing the margins on its either sides. C: K-nearest neighbor, where the predictions of a new point are made based on its distance from the points in the exiting database. D: Neural network with 2 hidden layers, where a system of interconnected neurons uses back-propagation to learn from the training data.

In-house codes were written to normalize the parameter data and train the predictive models for LR, L-SVM, G-SVM and k-NN in MATLAB (v9.3, R2017, MathWorks) using the Machine Learning Toolbox. For NN algorithm, python 3.6 code was developed using the open-source Tensorflow (v1.4.1, Google) and Keras (v2.0.8, https://keras.io/) libraries. A four-fold cross-validation was used during model training to avoid overfitting on the training cohort.

Training and Testing Accuracy Estimation

Area under the receiver-operator characteristic (ROC) curve (AUC) and 95% confidence intervals were quantified to assess the performance of the models on the training cohort for each algorithm.¹³ The predictive performance of each model was quantified by its accuracy on the independent testing cohort. The flowchart of the model training and testing for all models is shown in Fig. 2.

Fig. 2.

Flowchart for building and testing the predictive models for occlusion outcome of FD-treated IAs. After extraction of morphological, FD-related, and hemodynamic parameters, patients are randomly divided into the training and testing cohorts. Two sets of models are then trained using: 1) all parameters and 2) significant parameters on the training cohort. The predictive performance of these models is then tested on the testing cohort.

Results

Patient Population and Statistical Analysis

Based on the inclusion criteria, 84 IAs in 80 patients were enrolled in this study. At 6-month follow-up, 63 aneurysms had complete occlusion (occluded), whereas 21 aneurysms had residual contrast-filling (residual). The clinical and demographic information of patients in each group is listed in Table 1. The patients in the occluded group had a mean age (±SE) of 56.7±1.8 years; this group included 51 females, 20 patients with hypertension, and 23 patients who were current smokers. The residual group had an average age of 58.5±2.8 years, included 19 females, 10 patients with hypertension and 14 patients with smoking. There was no significant differences in age, sex, hypertension and smoking status between the 2 groups. Upon randomization, the training cohort included 64 aneurysms (48 occluded and 16 residual), with 20 aneurysms reserved to the testing cohort (15 occluded and 5 residual).

Table 1:

Description of patient population

	Occluded (n=63)	Residual (n=21)	P value
Patient Characteristics
Age (years)	56.7±1.8	58.5±2.8	0.53
Female sex (no. pts)	51	19	0.31
Hypertension (no. pts)	20	10	0.49
Smoking (no. pts)	23	14	0.39

^*no. pts=number of patients; continuous values expressed as (average ± standard error)

Table 2 lists the mean and standard error for morphology, FD-device and hemodynamic-based parameter in the 2 groups in the training cohort, with their respective p values. Mean values of all morphological parameters were higher in the residual group except AR. However, only NR and OsR were statistically different (p=0.01 and p<0.001, respectively) between the groups. Both FD-related parameters, MCR and PD were higher in the occluded group than the residual group, but the differences were not statistically significant. In terms of pre-treatment hemodynamic parameters, aneurysm that occluded within 6-months had lower mean values of AV_pre, SHR_pre and IR_pre and higher TT_pre as compared to those that did not occlude. However, only IR_pre was significantly different between the 2 groups (p=0.04). Similar to pre-treatment hemodynamic parameters, occluded group had lower mean values of AV_post, SHR_post and IR_post and higher value of TT_post than the residual group, with AV_post and IR_post showing statistically significant differences between the two groups (p=0.02 and p=0.02, respectively). NR, OsR, IR_pre, AV_post and IR_post had statistically significant differences between the 2 groups, and were used for training models with significant-parameters only; all 16 parameters were used in training all-parameter models.

Table 2:

Statistical analysis of morphology, FD and hemodynamic-based parameters in the training cohort

	Occluded (n=48)	Residual (n=16)	P value
Aneurysm Morphology
Size (mm)	4.99±0.59	5.04±0.89	0.71
ND (mm)	4.42±0.23	5.45±0.68	0.10
SR	1.32±0.18	1.52±0.36	0.47
AR	0.98±0.09	0.97±0.15	0.93
NR	1.17±0.05	1.51±0.16	0.01*
OsR	0.36±0.01	0.55±0.04	<0.001*
FD Device Characteristics
MCR (%)	27.84±0.60	27.54±1.05	0.80
PD (1/mm2)	29.60±1.11	27.03±1.64	0.20
Pre-treatment Hemodynamic Parameters
AVpre (cm/s)	6.86±0.74	11.67±2.75	0.07
SHRpre (1/s)	177.45±20.68	304.56±84.58	0.22
IRpre (ml/s)	0.51±0.09	0.87±0.25	0.04*
TTpre (s)	0.29±0.09	0.16±0.04	0.64
Post-treatment Hemodynamic Parameters
AVpost (cm/s)	1.97±0.30	5.55±1.73	0.02*
SHRpost (1/s)	52.14±9.08	150.27±54.67	0.08
IRpost (ml/s)	0.20±0.03	0.51±0.17	0.02*
TTpost (s)	0.74±0.21	0.41±0.10	0.40

^*statistically significant difference; all values expressed as (average ± standard error).

AR: aspect ratio, AV: aneurysm-averaged velocity, IR: inflow rate, MCR: metal coverage rate, ND: neck diameter, NR: neck ratio, OsR: ostium ratio, PD: pore density, SHR: shear rate, SR: size ratio, TT: turnover time. Suffixes pre and post represent pre-treatment and post-treatment hemodynamic values.

Final Converged Models

Through training, we obtained the final converged models for all algorithms. The training loss function for the LR, L-SVM, G-SVM, k-NN and NN models trained using all-parameters were 0.10, 0.125, 0.156, 0.171 and 0.10, respectively. For the LR, L-SVM, G-SVM, k-NN and NN models trained with significant parameters, the loss functions were 0.121, 0.156, 0.140, 0.203, 0.214, , respectively.

Model Performance on the Training Cohort

To compare the use of significant parameters versus all parameters on the performance of ML models on the training cohort, ROC analysis was performed for all models. As shown in Fig. 3, all-parameter models had better performance for each algorithm as compared to significant-parameter models. Among all-parameter models, NN had the highest training performance (AUC=0.967) followed by LR (AUC=0.941), whereas G-SVM model had the lowest AUC (0.841). However, among significant-parameter models, k-NN had the highest performance (AUC=0.875) followed by the NN model (AUC=0.854).

Fig. 3.

ROC curves for each ML model on the training cohort: built using all parameters (A) and built using significant parameters (B). AUC = area under the ROC curve; G-SVM = Gaussian support vector machine; K-NN: k-nearest neighbor; LR = logistic regression; L-SVM = linear support vector machine; NN = neural network.

Model Accuracy on the Testing Cohort

The predictive accuracy of each model in the testing cohort is shown in Table 3. Overall, models with significant parameter as input had lower accuracy on the testing cohort as compared to the models trained using all parameters. The all-parameter models’ predictive accuracy ranged from 85% to 90%, with LR, L-SVM, and K-NN having 85% and G-SVM and NN having 90% prediction accuracies. On the other hand, the significant-parameter models’ accuracy ranged from 55%–75%, with LR, L-SVM, and G-SVM having the highest accuracy of 75%.

Table 3:

Accuracy of all-parameter and significant-parameter models in the testing cohort

Machine Learning Algorithm	Accuracy
	All-Parameter Model	Significant- Parameter Model
Logistic Regression	85%	75%
Linear Support Vector Machine	85%	75%
Gaussian Support Vector Machine	90%	75%
K-nearest Neighbor	85%	70%
Neural Network	90%	55%

To further analyze the predictive performance of all-parameter models on the testing cohort, individual predictions on 20 IAs in the testing cohort were plotted in a confusion matrix in Fig. 4. The confusion matrix plots the number of correct and incorrect model-predicted outcome (horizontal axis) against the actual outcome (vertical axis) for each aneurysm. The cells with correct and incorrect predictions were shaded green and red, respectively. The most accurate NN model had 2 incorrect predictions, where 1 residual IA was predicted as occluded and 1 occluded IA predicted as residual. The predicted versus actual outcomes of all other models is shown in Fig. 4.

Fig. 4.

Confusion matrix for predictions of ML models (trained using all parameters) on the testing cohort. The vertical axis represents the actual clinical outcome and the horizontal axis represents the model-predicted outcome. Green indicates correct predictions; red indicates incorrect predictions.

Discussion

In endovascular intervention of IAs by flow diverters, clinicians use pre-treatment and immediate post-treatment DSA images of aneurysms to assess the flow stasis induced by the FD.^3,6 However, these images do not provide enough information to assess the long-term outcome of the IA healing. For illustration purpose, Fig. 5 shows two representative FD-treated ICA aneurysms from our cohort, with pre-treatment DSA on the left panel, and 6-month follow-up DSA on the far right panel. Although the FD placement was successful in both cases, their outcomes at 6 months were quite different: the top aneurysm was completely occluded (red circle), while the bottom aneurysm had significant residual filling. This example highlights the fact that despite the successful FD placement, a treatment might be ultimately unsuccessful due to persistent filling (i.e., IA is not occluded over a long period of time), which exposes the patient at risk of thromboembolic complications and rupture.^16,32 We believe that an ability to predict potential failure of FD treatment prior to the intended intervention will improve treatment planning, and thus minimize complications and optimize outcomes.

Fig. 5.

Concept of the methodology of generating clinically practical predictive models for FD-treated IAs. Left panel shows two representative patient-specific ICA aneurysms, which were treated using a single FD. Right panel shows the 6 month-occlusion outcome of both cases: top aneurysm was complete occluded (red circle) while bottom aneurysm had persistent residual filling in the IA. The middle panel show the computational analysis workflow, which uses pre-treatment 3D DSA images to generate 3D aneurysm model, and then models the FD deployment and hemodynamics using virtual stenting and CFD, respectively. The morphological, hemodynamic and candidate FD characteristics ae used as features to train machine-learning algorithms that can predict the 6-month occlusion outcome of FD-treated IAs. Note that neural networks are shown as a representative machine-learning algorithm for predictive model-building.

To that end, we have developed a computational analysis workflow that extracts information from the pre-treatment 3D DSA to potentially predict the treatment outcome. Conceptually, this computational workflow is an extension of the pre-treatment 3D DSA (middle panel in Fig. 5), which extracts pertinent features that include aneurysm morphology, pre- and post-treatment hemodynamics and characteristics of the candidate FD-device. These features lend themselves to building ML models that could predict long-term outcome when trained on a large number of retrospective FD-treated IA cases.

As a proof of concept of the proposed methodology, we retrospectively collected 84 FD-treated ICA aneurysm from our center. We applied the computational workflow to extract 16 aneurysm morphology, FD characteristics and pre- and post-treatment hemodynamic features for each IA. We then trained ML algorithms on 63 FD-treated IAs (training cohort) based on these features and produced models that were 85–90% accurate in predicting the 6-month occlusion outcome in an independent testing cohort, including correct predictions of the 2 example cases shown in Fig. 5. Application of this methodology to larger databases could generate, as well as validate, more robust predictive models, which could potentially help in assessing the outcome of FD-treatment a priori.

It has been argued in the literature that relevant features must be carefully selected for training ML models, especially for larger dataset.⁵ The reason is that including irrelevant features might incur extra computational cost during training, with little contribution to enhance the performance of the models.⁵ However, since our feature space is not very large (16 features), we asked if the so-called “irrelevant” features should be discarded. Therefore, we first performed univariate statistical analysis to significant parameters (could be considered as relevant) and insignificant parameters (could be slated as “irrelevant”). Then, we trained two sets of models, one using all parameters and one with significant parameters only and compared their performance to evaluate whether the inclusion of insignificant parameters helped to increase the model performance. Comparison results showed that all models trained with all 16 parameters had better accuracy (≥85%) than all models trained with only 5 significant ones (≤75%), and for each ML algorithm, all-parameter models outperformed their significant parameter counterparts. This indicates that the nonsignificant parameters are not truly “irrelevant”; they could be relevant through nonintuitive combinations. However, we cannot determine if that was the case due to small sample-size in the training cohort. Studies on larger patient database are required to elucidate the role of irrelevant parameters in predictive model-building.

Healing mechanism of FD-treated aneurysms is currently poorly understood.^{8,21,23,24,38} Based on our statistical results, we found that pre- and post-treatment inflow rate, post-treatment aneurysm-averaged velocity, ostium ratio and neck ratio were significantly higher in the residual group. This could provide some insight into the reason why some FD-treated IAs do not heal. FD intervention aims at diverting the flow away from the IA sac, inducing flow stasis and eventual thrombotic occlusion of the IA. Larger post-treatment inflow rate in the residual group indicates that the FD implantation did not divert enough flow away from the IAs in this group, resulting in ineffective flow diversion. This flow condition could lead to constant replenishment of fresh blood into the IA, which could be reason for the persistent residual filling into the IA sac even after 6-months for residual group. Interestingly, we also found pre-treatment inflow rate to be significantly higher in the residual group as compared to the occluded aneurysms. For these IAs, FD placement evidently cannot provide sufficient flow diversion away from the IA sac to ensure the long-term thrombotic occlusion of the IAs. Even more interestingly, the residual group also had significantly higher ostium ratio and neck ratio. This means that a larger portion of the parent vessel was the opening into the aneurysms in this group, which allowed more flow into the aneurysmal sac.⁴⁰ In these cases, use of alternative strategies like overlapping FDs, compacted FD or FDs with adjunctive coils could be more beneficial for successful outcome, namely occlusion.^9,38 In the ideal case, our models will identify IAs that may not occlude with a single, uniformly implanted FD, promoting the interventionalist to consider these alternative methods.

To find the best predictive models, we have also compared the performance of 4 different ML algorithms for FD treatment outcome prediction, including LR, SVM, k-NN and NN. Results show that NN and G-SVM (90% accuracy) performed slightly better than LR, L-SVM and k-NN (85% accuracy). Almost similar performance of all the algorithms suggest that a larger dataset is required to identify the best algorithm among these.

A common concern with training predictive models is overfitting to the training cohort when there are a large number of variables considered. However, we took steps to mitigate this risk, through both 4-fold cross-validation as well as holding out a test set, to which the models were naïve.²⁹ The models trained using all parameters performed well in training, as well as on the independent testing cohort, confirming that our models are generalizable.

Limitations

First, the CFD had to make simplifying assumptions, like assumed inlet flow waveform because patient-specific waveforms were not available. Second, our sample-size is small and limited to a single center. Future multicenter collaborations could help generate more robust predictive models. Third, the parameters used in our study were entirely derived based on findings from previous studies, which do not exhaustively represent the relevant morphological, device, and hemodynamic features in FD-healing. Fourth, this study is focused on sidewall aneurysms located at the ICA only, and the derived models may not be applicable for FD treatment of IAs at other locations. Last, we did not include patients’ clinical comorbidities and medications in our study, and these could influence the occlusion of FD-treated IAs. Future studies should include clinical comorbidities and medications as input parameters for training the predictive models.

Conclusions

We used ML algorithms to build predictive models for occlusion outcome of FD-treated sidewall IAs located at the ICA. Results show that models incorporating all 16 investigated morphological, hemodynamic, and FD-related parameters perform best with 90% predictive accuracy in an independent testing cohort. The implications are 2-fold: 1) ML algorithms that use all parameters, not just the statistically significant ones, may be utilizing some underlying morphological factors or hemodynamic factors for better outcome classification. 2) Using the computational analysis on larger patient-specific databases, future studies to build robust predictive ML-based models are necessary to allow clinicians to better plan and triage for appropriate treatment of IAs.

Abbreviations

AR

aspect ratio

AUC

area under the ROC curve

AV

averaged velocity

CFD

computational fluid dynamic

DSA

digital subtraction angiography

FD

flow diverter

IA

intracranial aneurysm

ICA

internal carotid artery

IR

inflow rate

K-NN

k-nearest neighbor

LR

logistic regression

MCR

metal coverage rate

ML

machine learning

ND

neck diameter

NN

neural network

NR

neck ratio

N-S

Navier-Stokes

OsR

ostium ratio

PD

pore density

PED

Pipeline embolization device

ROC

receiver operator characteristic

SE

standard error

SHR

shear rate

SR

size ratio

SVM

support vector machine

TT

turnover time