Computer-Assisted 3D Morphology Evaluation of Intracranial Aneurysms

Abstract

Objective:

Precise morphologic evaluation is important for intracranial aneurysm (IA) management. Currently, clinicians manually measure IA size and neck diameter on two-dimensional digital subtraction angiography (2D-DSA) and categorize IA shape as regular/irregular on three-dimensional (3D)-DSA, which might be subject to inconsistency and bias. We investigated whether a computer-assisted 3D analytical approach could improve IA morphology assessment.

Methods:

Five neurointerventionists evaluated sizes, neck diameters, and shapes of 39 IAs using current and computer-assisted 3D approaches. In the computer-assisted 3D approach, sizes, neck diameters, and undulation index (UI, a shape irregularity metric) were extracted by semi-automated reconstruction of aneurysm geometry using 3D-DSA, followed by IA neck identification and computerized geometry assessment.

Results:

Sizes and neck diameters measured by the manual 2D approach were smaller than computer-assisted 3D measurements by 2.01mm (p<0.001) and 1.85mm (p<0.001), respectively. Applying definitions of small IAs (<7mm) and narrow-necked IAs (<4mm) from the literature, inter-rater variation in manual 2D measurements resulted in inconsistent classifications in the sizes of 14 IAs and the necks of 19 IAs. Visual inspection resulted in inconsistent shape classifications of 23 aneurysms among raters. Greater consistency was achieved by using the computer-assisted 3D approach for size (Intraclass Correlation Coefficient [ICC]:1.00), neck measurements (ICC:0.96), and shape quantification (UI, ICC:0.94).

Conclusions:

Computer-assisted 3D morphology analysis can improve accuracy and consistency in measurements compared to manual 2D measurements. It can also more reliably quantify shape irregularity using the UI. Future application of computer-assisted analysis tools could help clinicians standardize morphology evaluation, leading to more consistent IA evaluation.

INTRODUCTION

Precise evaluation of saccular intracranial aneurysms (IAs) morphology is important to management for assessing rupture risk and guiding treatment decisions.¹ Aneurysm size and shape are the primary morphologic criteria used to evaluate the risk of IA rupture. Two prospective large-population studies of unruptured IAs support using these criteria, finding that large (≥7 mm)² and irregularly shaped³ IAs were more likely to rupture than small (<7 mm) and regular IAs. Aneurysm neck diameter is the primary metric used to guide treatment decisions.^{4, 5} Clinical studies have demonstrated that wide-necked IAs (neck >4mm) are less amenable to primary coiling and may be better managed by clipping or use of coiling adjuvants, such as balloon- or stent-assistance.^{4, 5} Because these morphologic parameters play key roles in IA management decisions, they must be measured accurately and precisely.

In practice, clinicians typically measure IA size and neck diameter manually on two-dimensional digital subtraction angiography (2D-DSA) images. Manual 2D measurements have inherent limitations such as restricted views of the aneurysm that may not show the maximum dimension of the sac or neck.^6–8 Furthermore, manual 2D measurements may be prone to intra-rater and inter-rater variations.^{9, 10} For shape evaluation, clinicians typically inspect three-dimensional (3D)-DSA images and classify an IA as “regular” or “irregular.” However, these definitions are vague and highly subjective.^{11, 12}

Recently, several engineering research groups have developed computerized procedures and software packages to provide a more objective, accurate, and consistent morphology assessment of IAs.^13–15 These tools are not yet routinely adopted in clinical practice, and clinicians still rely on manual 2D measurement and visual inspection of IA shape. One reason may be that improvement in accuracy and consistency in morphology evaluation derived from these tools has not been clearly compared to the current clinical practice. In this study, we conducted such a comparison. We evaluated the differences in aneurysm size and neck diameter measurements and the potential impact of these differences on IA morphology assessment. We also compared the current practice of aneurysm shape evaluation with computerized shape quantification in terms of consistency. The findings of this study suggest that computational 3D tools can help clinicians to standardize their morphology assessment.

METHODS

Data Collection and Raters

This study was approved by the University at Buffalo Institutional Review Board (STUDY00000302). We collected 2D-DSA and 3D-DSA images of 39 IAs obtained from 35 consecutive patients who were diagnosed with IAs between January and December of 2012. Nineteen IAs were located on the internal carotid artery, 6 on the middle cerebral artery, 2 on the anterior communicating artery, 1 on the anterior cerebral artery, and 11 in the posterior circulation.

Five neurointerventionists participated in this study. First, using the current clinical approach, they manually calculated IA size and neck diameter and visually categorized aneurysm shape as “regular” or “irregular.” Then, 2 weeks later, while blinded to their previous assessments, they applied a computer-assisted 3D approach to measure the aneurysm size, neck diameter, and a shape metric (referred to as the undulation index [UI]) of the same IAs.¹⁶ The workflow of the computer-assisted 3D approach is outlined in Figure 1, and details can be found elsewhere.¹⁷ To study intra-rater variation, one of the raters repeated the morphology evaluation using both approaches, again in a blinded fashion.

Figure 1:

Computer-assisted three-dimensional (3D) morphology workflow. The workflow includes: A) importing 3D images, to import 3D Digital Imaging and Communications in Medicine (DICOM) images of the intracranial aneurysm (IA); B) segmentation, to generate a 3D geometry of the intracranial aneurysm (IA); C) pre-processing, to isolate the region of interest and remove small vessel branches; D) morphologic analysis, to analyze the IA geometry and extract morphologic parameters.

Current Clinical Approach

For assessment via the current clinical approach (Figure 2A), the raters inspected lateral, anteroposterior, and oblique views of the IA on 2D-DSA images in a picture archiving and communication system (PACS) system. Using, the optimal view of the IA, they manually drew lines on the aneurysm sac to calculated maximum sac size, the maximal distance between any 2 points in the aneurysm sac, and maximum neck diameter, the largest observed diameter in the neck plane. To evaluate IA shape, they visually inspected 3D-reconstructed DSA images of the IAs and classified them as “regular” or “irregular” on the basis of their knowledge. In this assessment, no specific instructions were prescribed to guide the raters on how to evaluate the IA shape.

Figure 2:

Comparison of current clinical practice and computer-assisted 3D approaches for IA morphology evaluation. A) manual measurements of an internal carotid artery (ICA) aneurysm obtained from two-dimensional digital subtraction angiography (2D-DSA) and visual inspection of IA shape on 3D-DSA. B) Computer-assisted 3D size and neck measurements and quantification of shape of the same aneurysm using the undulation index (UI).

Computer-Assisted 3D Approach

The raters performed computer-assisted 3D morphology assessment of the same IAs while blinded to their previous evaluations (Figure 2B). The 3D images had been previously segmented so the raters could be provided with the 3D files of aneurysm geometries (in stereolithography [STL] format). Raters inspected each IA geometry in 3D and isolated the aneurysm sac by manually identifying a neck plane. The morphologic parameters were then automatically calculated by 3D computerized processing of the aneurysm geometry, as described previously.^{16, 17} In this part of the study, the raters evaluated maximum size and maximum neck diameter, as well as UI.¹⁶ UI quantifies the degree of IA irregularity and is defined as UI=1-(V/V_ch), where V is the volume of the aneurysm and V_ch is the volume of the convex hull as the smallest surface that fully encloses the IA volume.

The accuracy of the computer-assisted 3D approach was validated through an in-vitro experiment using patient-specific 3D-printed models. Validation details are provided in the supplementary text, supplementary Figure A, and supplementary Table A.

Statistical Analyses

A paired Student t-test was used to compare the manual 2D and computer-assisted 3D measurements. The correlation between the measurements of the two approaches was assessed using Kendall’s Tau-b correlation coefficient. A p-value of <0.05 was considered statistically significant. The intra-rater and inter-rater variations were evaluated by calculating the intraclass correlation coefficient (ICC) agreement. The ICC values were interpreted as: ≤0.50, poor agreement; 0.50–0.75, moderate agreement; 0.75–0.90, good agreement; and ≥0.90, excellent agreement.¹⁸ Intra-rater and inter-rater agreements for binary ranking of aneurysm shape were described with a Fleiss’ Kappa coefficient. Fleiss’ Kappa coefficients were interpreted as ≤0.2, slight agreement;0.21–0.40, fair agreement, 0.41–0.60, moderate agreement; 0.61–0.80, substantial agreement, and 0.81–1.00 perfect agreement.¹⁹ Statistics were computed using the R: language and environment for statistical computing (www.R-project.org).

RESULTS

Differences in Measurements between Manual 2D and Computer-Assisted 3D

We compared the raters’ averaged size and neck diameter measurements between manual 2D and computer-assisted 3D approaches, Figure 3. A line connects the measurements of the same IA. Manual 2D size and neck diameter measurements were on average 2.01mm (p<0.001) and 1.85mm (p<0.001) less than computer-assisted 3D measurements, respectively. Using thresholds of7mm and 4mm for classifying “small” versus “large” and “narrow-necked” versus “wide-necked,” respectively^{2, 4}, we found significant differences between manual and computer-assisted measurements. Twenty-nine (74%) IAs were small by the manual 2D approach and 17 IAs (43%) were small by computer-assisted 3D analysis. Twenty-seven IAs (69%) were narrow-necked by the manual 2D approach and 15 IAs (38%) were narrow-necked by computer-assisted 3D analysis.

Figure 3:

Scattergrams of size (A) and neck diameter (B) measured using manual 2D and computer-assisted 3D approaches. Each data point represents the average of measurements taken by five raters. A line connects data points for the same IA. The dashed lines represent the 7mm IA size threshold for small to larger classification (A) and the 4mm IA neck diameter threshold for narrow- and wide-necked classifications (B).

The measurement differences between the two approaches could cause discrepancy in IA classification. Twelve IAs (31%) were classified as small (<7mm) by manual 2D measurements but found to be large using computer-assisted 3D analysis. Likewise, 11 IAs (28%) were classified as narrow-necked by manual 2D measurements but found to be wide-necked using computer-assisted 3D analysis. The necks of 4 IAs (10%) could not be identified on 2D-DSA images but were discernable via the computer-assisted 3D approach. We also observed a moderate positive correlation for IA size, 0.61 (p<0.001) and a weak positive correlation for neck diameter, 0.43 (p<0.001) between manual 2D and computer-assisted 3D measurements.

Intra-rater and Inter-rater Variations in IA Size and Neck Diameter Measurements

To evaluate the variation in measurements, intra-rater and inter-rater agreement coefficients with a 95% confidence interval (CI) were calculated (Table 1). Intra-rater agreements for manual 2D measurements were excellent for size, ICC: 0.95 (0.90–0.97), and moderate for neck diameter, ICC: 0.71 (0.46–0.85). Intra-rater agreements improved to excellent for both size, ICC: 1.00 (0.99–1.00), and neck diameter, ICC: 0.99 (0.98–0.99), when the computer-assisted 3D approach was used. Inter-rater agreements for manual 2D measurements were excellent for size, ICC: 0.92 (0.88–0.96), and fair for neck diameter, ICC: 0.52 (0.31–0.74). Inter-rater agreements improved to excellent for both size, ICC 1.00 (0.99–1.00), and neck diameter, ICC: 0.96 (0.94–0.98), when the computer-assisted 3D approach was used.

Table 1:

Intra- and inter-rater agreement coefficients of aneurysm size and neck diameter measurements and aneurysm shape evaluations

Morphologic Index	Approach	Intra-rater Agreement Coefficient (95% CI)	Inter-rater Agreement Coefficient (95% CI)
Aneurysm Size	Manual 2D	0.95 (0.90–0.97)	0.92 (0.88–0.96)
	Computer-Assisted 3D	1.00 (0.99–1.00)	1.00 (0.99–1.00)
Aneurysm Neck Diameter	Manual 2D	0.71 (0.46–0.85)	0.52 (0.31–0.74)
	Computer-Assisted 3D	0.99 (0.98–0.99)	0.96 (0.94–0.98)
Aneurysm Shape	Visual Inspection 3D	0.63 (0.32–0.95)	0.45 (0.35–0.55)
	Undulation Index 3D	0.99 (0.98–0.99)	0.94 (0.91–0.97)

Abbreviations: CI, confidence interval; 2D, two-dimensional; 3D, three-dimensional

We further investigated the impact of variations in measurements on IA morphology assessment. Figures 4 and 5 show variations in size and neck diameter measurements among the raters for manual 2D and computer-assisted 3D approaches. When 7mm was used as a size threshold, the manual 2D approach led to inter-rater discrepancy in the classification of 14 IAs (36%); this discrepancy only occurred in 3 IAs (8%) using the computer-assisted 3D approach. When 4mm was used as a neck diameter threshold, the manual 2D approach led to inter-rater discrepancy in the classification of 19 IAs (49%); this discrepancy only occurred in 6 IAs (15%) using the computer-assisted 3D approach.

Figure 4:

Scattergrams of manual 2D and computer-assisted 3D measurements of the sizes of 39 IAs. The dashed line represents the generally accepted 7mm IA size threshold for small to large classification.

Figure 5:

Scattergrams of manual 2D and computer-assisted 3D measurements of neck diameter of 39 IAs. The dashed line represents the accepted 4mm threshold for narrow- and wide-necked classification. The cross-signs (×) in A indicate that the aneurysm neck was not detectable in 4 of the 2D images; only 3D computer-assisted measurements are reported for those cases.

Visual Shape Inspection Versus Shape Quantification (UI)

Each rater individually inspected the 3D geometry of the IAs and classified them as regular or irregular. Figure 6 is a scatterplot of the 39 IAs’ shape evaluation (either regular or irregular) along with the raters’ average UI. When we compared the raters’ assessments, 41% had good agreement, with 4 IAs (10%) classified as regular and 12 IAs (31%) as irregular. Inconsistency in shape evaluation among the 5 raters was observed in 23 IAs (59%). For cases with inconsistent shape evaluation, an IA was classified into the regular cohort if it was evaluated as regular shape by the majority of the raters (more than 3 raters) and vice-versa. Thus, 25 IAs (64%) were classified irregular and had a significantly higher average UI than regular aneurysms, 0.10±0.05 versus 0.03±0.02, p<0.001 (Figure 6). Intra- and inter-rater agreement coefficients for IA shape evaluation through visual inspection were substantial, Fleiss’ Kappa: 0.63 (0.32–0.95), and moderate, Fleiss’ Kappa: 0.45 (0.35–0.55), respectively (Table 1). Intra-rater and inter-rater agreements were excellent for aneurysm shape quantification (UI calculations), ICC: 0.99 (0.98–0.99) and 0.94 (0.91–0.97).

Figure 6:

Comparison of the UI between “regular” and “irregular” shaped IAs. Aneurysms classified “regular” and “irregular” by all the raters are identified by white circles and black triangles, respectively. Gray squares indicate shapes that were not agreed upon.

Discussion

Morphology evaluation of IA is an important part of aneurysm risk assessment and treatment decision-making. In this study, we showed that aneurysm size and neck diameter are often smaller using the manual 2D approach compared to the computer-assisted 3D approach. Our results showed that manual 2D measurements and visual inspection of aneurysm shape are subject to inconsistencies that might influence IA morphology assessment. The computer-assisted 3D approach resulted in accurate and consistent IA size and neck diameter measurements and quantification of IA shape irregularity using the UI.

Aneurysm Sizes and Neck Diameters Are Smaller in Manual 2D Measurements

Our data demonstrate that measurements of aneurysm size and neck diameter were larger using the computer-assisted 3D approach. This has also been shown by previous studies comparing 2D-DSA and 3D-DSA imaging; IA size and neck diameters manually measured on 3D-DSA are typically larger than measurements on 2D-DSA.^{6, 8, 20} These findings have led to disagreement regarding which imaging modality should be considered the standard of reference. For instance, Brinjikji et al. reported that 3D-DSA tended to overestimate aneurysm neck diameter and that clinicians should rely on 2D-DSA because of its higher spatial resolution and minimal postprocessing to generate the images.²⁰ In contrast, Wong et al.⁶ reported that 3D-DSA has substantial benefits in term of detection of the aneurysm neck and its relation to the parent artery and should be considered as the standard of reference. Kawashima et al.⁸ noted that underestimation of aneurysm size on 2D images is probably because one part of the IA dome frequently overlaps with the parent vessel.

In addition to the type of imaging, making measurements manually could also contribute to the underestimation of aneurysm size and neck diameter. Larrabide et al.¹⁵ reported that when raters manually measured IA size and neck diameter on 3D reconstructed aneurysm geometries, the measurements were smaller than those obtained by computerized processing of the isolated aneurysm sac. This is expected because a computerized algorithm finds the “true” maximum diameter by 3D processing the entire contour of the lesion, whereas manual measurement, even on the 3D geometry of the IA, is restricted to a selected 2D projected view of the IA that may not include the maximum dimension.

Although we observed that the computer-assisted 3D measurements were larger than the manual 2D measurements, the true aneurysm sizes and neck diameters are unknown. Therefore, we cannot say for certain if the computer-assisted 3D measurements overestimate the true size or if the manual 2D measurements underestimate them. To overcome this problem, we validated the accuracy of the computer-assisted 3D method in an in-vitro experiment using 3D-printed IA models (see supplementary material). The results demonstrated that the computer-assisted method could recapitulate the true size and neck diameter measurements of the imaged models. Thus, we surmise that computer-assisted 3D measurements may be closer to reality.

Potential Influence of Measurement Inconsistency in the Assessment of Aneurysms

Previous studies have reported inherent intra-rater and inter-rater variations in manual morphology measurements.^{9, 10} The potential sources of such inconsistency are probably due to difficulties in finding the site of maximum size and neck diameter, which is potentially more challenging in irregularly shaped aneurysms. In our study, based on the calculated ICC, the raters had excellent agreement statistically in aneurysm size measurement using the manual 2D approach. However, when we defined the 7mm threshold for small/large IA classification, we observed a discrepancy in the classification of 36% of the IAs. The investigators of the

International Study of Unruptured Intracranial Aneurysms (ISUIA) have reported that unruptured aneurysms larger than 7mm have a higher risk of rupture compared to small (<7mm) counterparts.² Thus, the discrepancy in IA size classification observed in our study has the potential to influence IA rupture risk assessment and management.

We observed poor agreement in manual 2D neck diameter measurements among the 5 raters. When we defined the 4mm criterion for narrow-necked and wide-necked classifications of IAs, we found a discrepancy in the classification of 49% of the IAs. Fernandez-Zubillaga et al.⁴ reported that primary coiling of wide-necked IAs (>4mm) resulted in incomplete occlusion in 85% of the IAs after 6 months of follow up. They suggested that wide-necked IAs might be better managed by clipping. Therefore, similar to aneurysm size, the discrepancy observed in neck diameter classification in the current study also has the potential to change treatment strategy.

In contrast to the manual 2D measurements, our computer-assisted 3D assessment provided much less variation and more consistent and reliable measurements (Figures 4 and 5). The variation in the computer-assisted 3D approach might be because it requires raters to isolate the aneurysm sac by manually drawing a line over the aneurysm neck. In the future, fully automated algorithms for IA detection and aneurysm neck detection can avoid such variations.^{13, 15}

Aneurysm Shape Quantification Using the Undulation Index to Avoid Inconsistency and Bias in Shape Evaluation

Aneurysm shape irregularity is a major factor when assessing rupture risk.^{3, 21} However, the definition of irregular shape is vague. In the literature, IAs with a bleb, a daughter or a secondary aneurysm, or a bi- or multilobar fundus are considered irregular.^{2, 3, 11, 21} Suh et al.¹¹ studied inter-rater variations in IA shape evaluation among raters who were instructed with different definitions of aneurysm shape, including criteria from the ISUIA² and the Unruptured Cerebral Aneurysm Study (UCAS).³ They reported that the highest inter-rater agreement, Fleiss’ Kappa: 0.63 (0.52–0.72), was achieved when they used the UCAS definition to identify IAs with a daughter sac (i.e., “an irregular protrusion of the wall of the aneurysm”). In our study, we did not instruct raters how to evaluate the IA shape in order to simulate an actual clinical situation.

This may explain why we observed less inter-rater agreement among raters, Fleiss’ Kappa 0.45 (0.35–0.55). In contrast, aneurysm shape quantification using the UI provided excellent agreement among raters. We also observed that IAs that were categorized as “irregular” had a significantly higher UI than their “regular” counterparts. This might indicate that the UI is a good representation of neurointerventionists’ interpretation of aneurysm shape. Additionally, in a recent study of 542 IAs, UI was found to be the most significant rupture discriminative morphological parameter.²² Therefore, we suggest that quantification of aneurysm shape using the UI can avoid inconsistency and bias and is a reasonable alternative for visual classification.

Study Limitations

The 7mm size and 4mm neck diameter thresholds used here are based on previous studies where aneurysm size and neck were measured manually.^{2, 4} Considering our results and the results of others^{6, 8, 20} that demonstrate that 3D-DSA imaging typically produces larger aneurysm measurements than manual 2D approaches, these thresholds may not be valid for 3D image-derived data. In the future, as more clinical centers adopt 3D morphology assessment, correction of these threshold values using 3D datasets may be necessary.

Another limitation of this study is that we do not have manual 3D measurements of the IAs, thus, cannot distinguish the influence of the utilization of 3D images and computerized measurements.

To strengthen the evidence of accuracy and consistency for computer-assisted 3D, such as the approach outlined in the study, a study with more IAs and raters is required. Furthermore, to demonstrate the potential for 3D morphology assessments to impact treatment decisions, longitudinal and multicenter studies are needed.

CONCLUSION

Computer-assisted 3D morphology assessments can provide more accurate and consistent aneurysm size and neck diameter measurements than the traditional, manual 2D approach. Moreover, visual inspection of aneurysm shape is subject to inconsistency and bias and can be avoided by quantification of aneurysm shape using the UI. In the future, the adoption of computer-assisted 3D tools can help clinicians to avoid inconsistency in measurements, standardize their morphology evaluation, and be more consistent in IA assessments.

Highlights.

This study indicates that:

• Manual 2D measurements of intracranial aneurysm sizes and neck diameters are smaller than computer-assisted 3D measurements

• Manual 2D measurement and visual inspection of aneurysm shape are subject to inconsistency

• Variations in measurements and shape evaluation can influence intracranial aneurysm morphology assessment

• Computer-assisted 3D morphology can help clinicians standardize morphology assessments

Abbreviations

2D

two-dimensional

3D

three-dimensional

CI

confidence interval

CT

computed tomographic

DICOM

Digital Imaging and Communications in Medicine

DSA

digital subtraction angiographic or angiography

IA

intracranial aneurysm

ICC

intraclass correlation coefficient

ISUIA

International Study of Unruptured Intracranial Aneurysms

PACS

picture archiving and communication system

STL

stereolithography

UCAS

Unruptured Cerebral Aneurysm Study

UI

undulation index