Fully automated 3D aortic segmentation of 4D flow MRI for hemodynamic analysis using deep learning

Abstract

Purpose:

To generate fully automated and fast 4D-flow MRI-based 3D segmentations of the aorta using deep learning for reproducible quantification of aortic flow, peak velocity, and dimensions.

Methods:

A total of 1018 subjects with aortic 4D-flow MRI (528 with bicuspid aortic valve, 376 with tricuspid aortic valve and aortic dilation, 114 healthy controls) comprised the data set. A convolutional neural network was trained to generate 3D aortic segmentations from 4D-flow data. Manual segmentations served as the ground truth (N = 499 training, N = 101 validation, N = 418 testing). Dice scores, Hausdorff distance, and average symmetrical surface distance were calculated to assess performance. Aortic flow, peak velocity, and lumen dimensions were quantified at the ascending, arch, and descending aorta and compared using Bland-Altman analysis. Interobserver variability of manual analysis was assessed on a subset of 40.

Results:

Convolutional neural network segmentation required 0.438 ± 0.355 seconds versus 630 ± 254 seconds for manual analysis and demonstrated excellent performance with a median Dice score of 0.951 (0.930–0.966), Hausdorff distance of 2.80 (2.13–4.35), and average symmetrical surface distance of 0.176 (0.119–0.290). Excellent agreement was found for flow, peak velocity, and dimensions with low bias and limits of agreement less than 10% difference versus manual analysis. For aortic volume, limits of agreement were moderate within 16.3%. Interobserver variability (median Dice score: 0.950; Hausdorff distance: 2.45; and average symmetrical surface distance: 0.145) and convolutional neural network–based analysis (median Dice score: 0.953–0.959; Hausdorff distance: 2.24–2.91; and average symmetrical surface distance: 0.145–1.98 to observers) demonstrated similar reproducibility.

Conclusions:

Deep learning enabled fast and automated 3D aortic segmentation from 4D-flow MRI, demonstrating its potential for efficient clinical workflows. Future studies should investigate its utility for other vasculature and multivendor applications.

1 |. INTRODUCTION

Quantification of aortic flow is important for diagnosis and patient management in aortic diseases, including aortic valve abnormalities (eg, bicuspid aortic valve, aortic valve stenosis, regurgitation), aneurysm, or coarctation.^1–3 Several studies have shown the potential of 4D-flow MRI for the comprehensive characterization of hemodynamic changes in different aortic diseases.^4–6 However, evaluation of volumetric aortic flow dynamics from 4D-flow MRI requires manual and cumbersome data analysis, including 3D segmentation of the thoracic aorta, which limits clinical translation and interobserver variability.

Previous attempts to automate 3D aorta segmentations have focused on exploiting anatomical landmarks or shape descriptors of the aorta, such as using atlas registration,⁷ deformable surface, and active surface techniques,^8,9 or nonrigid registration.¹⁰ However, these techniques attempted to derive generalizable landmark features from small and homogenous patient cohorts. Translation of such features to larger cohorts with heterogeneous patient populations is therefore challenging. This is, in part, due to the high variability in aortic shape, size, and geometry among different subjects (eg, pediatric vs adult) and disease progression (eg, aortic dilatation, shape changes).

Recent developments in deep learning and convolutional neural networks (CNNs) have demonstrated excellent results in the segmentation of MRI for the left ventricle, cartilage and musculoskeletal tissue, knee joint tissue, prostate, and various other anatomical structures.^11–14 In this work, we sought to develop and evaluate a deep learning–based method that enables fully automated 3D aortic segmentation from 4D-flow MRI for fast and reproducible evaluation of aortic dimensions and flow. Our goal was to develop a dedicated aorta 3D segmentation CNN by leveraging a large data set of over 1000 in vivo aortic 4D-flow MRI scans with established manually generated ground-truth labels, spanning a wide range of ages (pediatric to adult) as well as spanning a number of common aortic diseases, sizes, and shapes. We hypothesize that deep learning–based 4D flow analysis will allow for a faster workflow with similar accuracy compared with manual analysis.

2 |. METHODS

2.1 |. Study cohort

A total of 1193 subjects across a wide age range (2–91 years, median: 44 years), who underwent aortic 4D-flow MRI from 2011 to 2018 across two hospitals, were retrospectively included in this study. A total of 1064 patients underwent standard-of-care cardiothoracic MRI (278 scans from Lurie Childrenʼs Hospital, 786 from Northwestern Memorial Hospital). Additionally, 129 healthy adult controls underwent research cardiothoracic MRI exams. This Health Insurance Portability and Accountability Act–compliant study was approved by the institutional review boards at Lurie Childrenʼs Hospital and Northwestern Memorial Hospital. Patients were retrospectively enrolled with waiver of consent. Controls provided written informed consent as per the institutional review board requirement.

2.2 |. MRI

All subjects underwent 4D-flow MRI with full volumetric coverage of the thoracic aorta (sagittal-oblique 3D volume) using either 1.5T (N = 1053; Aera, Avanto, or Espree; Siemens Healthineers, Erlangen, Germany) or 3T MRI systems (N = 140; Skyra; Siemens Healthineers). The 4D-flow MRI pulse sequence parameters were as follows: spatial resolution = 1.2–3.1 × 1.2–3.1 × 1.2–5.0 mm³, temporal resolution = 32.8–44.8 ms, velocity sensitivity (venc) = 80–500 cm/s, FOV = 124–406 × 180–500 × 38–176 mm³, TE = 2.1–3.0 ms, TR = 4.1–5.7 ms, and flip angle = 7°−25°. Data for all subjects were acquired during free-breathing with respiratory navigator gating and either prospective (N = 1172) or retrospective (N = 21) electrocardiographic gating. For N = 1064 subjects, a contrast agent (either Gadavist, Magavist, Multihance, dotarem, or Ablavar [Bayer Healthcare, Berlin, Germany] or Feraheme [Amag Pharmaceuticals, Waltham, MA]) was administered before the 4D-flow scan acquisition as standard-of-care protocol.

2.3 |. Data Analysis

2.3.1 |. Standard 4D-flow MRI analysis workflow

All subjects were analyzed using a clinically standard 4D-flow preprocessing workflow programmed in MATLAB (MathWorks, Natick, MA) (Figure 1). The 4D-flow MRI data were corrected for eddy currents, noise, and velocity aliasing as described previously.^15–18 Preprocessed data were used to calculate a 3D phase-contrast MR angiogram (PCMRA)¹⁹ and served as the basis for manual 3D segmentation of the thoracic aorta using commercial software (Mimics; Materialise, Leuven, Belgium). The PCMRA was generated by taking the mean sum squared of the time-averaged velocity magnitude data of the 4D-flow scan as follows:

$$\mathit{\text{PCMRA}}=\sum _{t=1}^T\left(M_t^2\left(V_{x,t}^2+V_{y,t}^2+V_{z,t}^2\right)\right)/T$$ (1)

where V_x, V_y, and V_z are the x-velocity, y-velocity, and z-velocity components of the 4D-flow data; M is the magnitude data; and T is the total number of cardiac time frames of the data set.

FIGURE 1.

Analysis workflow for 4D-flow MRI involving corrections for phase offsets, velocity noise, and velocity antialiasing. After preprocessing, a 3D phase-contrast MR angiogram (PCMRA) was generated and used to perform a manual 3D segmentation of the thoracic aorta, which served as labeled data and ground truth for convolutional neural network (CNN) training, validation, and testing. Analysis planes were manually placed perpendicular to the vessel centerline in the ascending aorta (AAo), arch, and descending aorta (DAo). Mag, magnitude; V_x, V_y, and V_z, velocity in the x, y, and z directions

The manual 3D segmentations were performed over a period of 6 years (2012–2018) by over 20 operators. Each manual 3D segmentation was reviewed by two reviewers with 2 and 3 years of experience. To ensure an accurate ground-truth segmentation of the thoracic aorta, only those data sets that both reviewers approved were included in the final cohort. After review, the manual PCMRA-based 3D segmentation served as ground truth for training, validation, and testing of the CNN.

2.3.2 |. Convolutional neural network for automated aorta 3D segmentation

The CNN used was a 3D U-Net network²⁰ with DenseNet-based dense blocks²¹ replacing the original convolution layers. The preprocessed 4D flow–derived 3D PCMRA was automatically center-cropped to a fixed dimension of 128 × 96 × number of slices and used as the input to the CNN. The dense blocks (Figure 2B) featured a constant convolutional channel size of 12, reducing the overall number of parameters and mitigating overfitting. Additionally, the dense blocks concatenated all prior feature maps for use as inputs for subsequent layers, ensuring efficient feature reuse. As shown in Figure 2A, the encoder section used down-sampling layers (max pooling), whereas the decoder section used up-sampling layers (transposed convolution). The final layer consisted of a 1 × 1 × 1 convolution and softmax function to generate a probability of each voxel in the input for the binary classes (background or foreground). The final segmentation was generated by selecting the class with the highest probability per voxel. A composite loss function (softmax-cross entropy and Dice loss) was used during training.

FIGURE 2.

Convolutional neural network architecture and layer structures. All input images were cropped PCMRAs with dimensions of 128 × 96 × S (S = number of slices). A, The CNN consisted of a symmetrical encoder-decoder design from a traditional U-Net network with the addition of dense blocks in each layer. In the encoder sections, the number of layers of the dense block were increased to extract more intricate and complex feature maps, and a max-pooling layer was applied after every dense block. The max-pooling had a kernel size of 2 × 2 × 1 to down-sample feature maps across the encoder section except in the slice number direction, to allow for scans with differing numbers of slices. The decoder section followed the same layout with transposed convolution layers replacing the max-pooling layer to up-sample the input. B, Dense blocks consisted of a series formed by a batch normalization, a linear rectified unit, a convolution layer, and a dropout layer (rate = 0.1). As in the original implementation of DenseNet, feature maps were concatenated after each layer and served as the input for all subsequent layers

The network was coded in Python 3.6.8 (Python Software Foundation, Beaverton, OR) using Tensorflow 1.12.0 (Google, Mountain View, CA). All training and testing were performed on an Intel i7–8700k processor with a Nvidia GTX 1080-Ti GPU. Additional details of the CNN architecture, hyperparameters, and implementation are provided in the Supporting Information.

2.3.3 |. Convolutional neural network performance metrics

To compare manual versus CNN 3D segmentations, the Dice score (DS), Hausdorff distance (HD), and average symmetrical surface distance (ASSD) were calculated as follows²²:

$$\text{DS}\left(X,Y\right)=\frac{2*\left|X\cap Y\right|}{\left|X\right|+\left|Y\right|}$$ (2)

$$\text{HD}\left(X,Y\right)=\text{max}\left({\text{max}}_{y\in Y}\left({\text{min}}_{x\in X}\left(d\left(x,y\right)\right)\right){\text{, max}}_{x\in X}\left({\text{min}}_{x\in X}\left(d\left(x,y\right)\right)\right)\right)$$ (3)

$$\text{ASSD}\left(X,Y\right)=\left(\frac{1}{\left|X\right|+\left|Y\right|}\right)\left(\sum _{x=1}^X{\text{min}}_{y\in Y}\left(d\left(x,y\right)\right)+\sum _{y=1}^Y{\text{min}}_{x\in X}\left(d\left(x,y\right)\right)\right)$$ (4)

where X and Y are binary segmentations and d is the Euclidian distance.

To determine the most common discrepancies in the automated 3D segmentations, all CNN segmentations with DS < 0.90 were manually assessed for notable differences compared with their manual segmentations.

2.3.4 |. Flow, volume, and diameter calculation

For flow quantification, three 2D-analysis planes were manually placed in the ascending aorta (AAo, proximal to the brachiocephalic artery), the aortic arch (distal to the brachiocephalic artery but proximal to the left subclavian artery), and the descending aorta (DAo, distal to the left subclavian artery). The planes were constrained to be perpendicular to an automatically generated aortic centerline. The 3D aorta segmentation was used to define the lumen boundaries. Maximum velocity (95th percentile of velocity at peak systole) and flow-time curves (used to determine peak, net flow, and net regurgitation) were calculated for each plane. In addition, the maximum lumen diameter was extracted by fitting an ellipse with the same normalized second-area moment as the cross section and taking the major axis for each of the three planes.²³ Finally, the volume of the thoracic aorta 3D segmentation was determined.

2.3.5 |. Interobserver variability

Interobserver comparisons of standard manual analysis workflow were performed by two observers with 2 years of experience (HB and MS) on 40 randomly selected data sets from the testing cohort (18 bicuspid aortic valve [BAV], 12 tricuspid aortic valve [TAV], 10 controls). Each observer independently performed a manual 3D segmentation of the aorta and was blinded to the results from the other. Values of DS, ASSD, and HD were calculated between the two observers’ segmentations, as well as between each observer versus the CNN segmentation.

2.3.6 |. Statistical analysis

Segmentation performance metrics (DS, HD, and ASSD) were assessed for normality using the Shapiro-Wilk test, with normally distributed values reported as mean and SD or nonnormally distributed values reported as median (interquartile range). Flow metrics, lumen diameters, and aorta volume were compared with values from manually generated segmentations using Bland-Altman analysis. Bland-Altman limits of agreement (LOA) percent difference was defined as the LOA divided by the mean of the reference manual segmentation. Linear regression was used to assess the impact of demographic and scan parameters (age, venc, heart rate, body mass index, spatial resolution) on CNN performance (DS). Comparisons were performed to assess the CNN performance on different MRI field strength (1.5 T vs 3 T) as well as for sex and the presence of gadolinium contrast agent. An unpaired t-test or Mann-Whitney U-test was used (depending on normality) to perform the comparisons. A P-value < .05 was considered statistically significant.

3 |. RESULTS

3.1 |. Study cohort

After manual review of the 4D-flow MRI segmentations to generate a ground-truth data set, 175 data sets were excluded due to insufficient aorta 3D segmentation quality (N = 57 included regions outside the aorta, N = 56 included nonaorta vasculature [pulmonary arteries/veins, vena cava], N = 30 with partially missing portions of the AAo or DAo, N = 30 with inadequate FOV, and N = 2 with missing data). The remaining 1018 data sets were included in the study. As summarized in Table 1, the study cohorts consisted of 528 BAV patients (384 males, median age = 42 years [range 2–80 years], 507 with contrast), 376 TAV patients with aortic dilation (241 males, median age = 45 years [range 2–91 years], 345 with contrast), and 114 controls (59 male, median age: 48 years, range = (19–81) years, 38 with contrast). A total of 212 pediatric patients were included in the cohort (median age = 14.3 years [range 2–18 years]), to train and test the CNN on more complex aortic geometry that may not be found in adult patients. A wide variety of diseases were represented across the pediatric cohort, including coarctation (N = 46), BAV with coarctation (N = 32), Marfan syndrome (N = 27), and Turner syndrome (N = 18).

TABLE 1.

Demographics and clinical information for all subjects

All	Total (N = 1018)	BAV (N = 528)	TAV (N = 376)	Controls (N = 114)
Age (years)	44 (2–91)	42 (2–80)	45 (2–91)	48 (19–81)
Sex (M/F)	684/334	384/144	241/135	59/55
BMI (kg/m2)	25.8 [22.3–29.5]	25.6 [22.6–29.0]	25.1 [21.5–29.4]	26.8 [24.0–31.4]
Heart rate (bpm)	66 [59–76]	67 [60–76]	66 [59–78]	63 [58–72]
Training	Total (N = 499)	BAV (N = 260)	TAV (N = 192)	Controls (N = 47)
Age (years)	42 (3–91)	39 (3–80)	46 (3–91)	44 (20–78)
Sex (M/F)	345/154	192/68	132/60	21/26
BMI (kg/m2)	25.2 [21.9–29.5]	25.1 [21.6–29.0]	25.1 [22.0–29.9]	26.2 [23.9–30.7]
Heart rate (bpm)	66 [59–77]	68 [60–79]	66 [59–76]	62 [58–70]
Validation	Total (N = 101)	BAV (N = 53)	TAV (N = 30)	Controls (N = 18)
Age (years)	46 (2–81)	43 (2–71)	46 (2–74)	56 (25–81)
Sex (M/F)	61/40	36/17	16/14	9/9
BMI (kg/m2)	25.2 [22.5–28.7]	25.1 [22.5–28.3]	24.6 [21.8–28.8]	25.6 [23.3–31.0]
Heart rate (bpm)	68 [60–75]	68 [61–74]	72 [57–84]	63 [59–67]
Testing	Total (N = 418)	BAV (N = 215)	TAV (N = 154)	Controls (N = 49)
Age (years)	45 (4–91)	45 (9–74)	45 (4–91)	48 (19–79)
Sex (M/F)	279/141	156/59	93/61	29/20
BMI (kg/m2)	25.8 [22.8–29.7]	25.8 [23.3–29.1]	25.1 [20.8–29.0]	27.4 [25.1–31.8]
Heart rate (bpm)	66 [60–76]	67 [60–75]	66 [59–78]	65 [60–76]

Note: Age reported as the median (range). Body mass index and heart rate are reported as median [interquartile range]. Abbreviations: BMI, body mass index; F, female; M, male.

3.2 |. Convolutional neural network for automated aorta 3D segmentation: training, validation, and testing

As illustrated in Figure 3, the 1018 4D-flow MRI scans were randomly divided into 49% (499 of 1018) for training (260 BAV, 192 TAV, 47 controls; age range = 3–91 years, 110 pediatric patients), 10% (101 of 1018) for validation (53 BAV, 30 TAV, 18 controls; age range = 2–81 years, 19 pediatric patients), and 41% (418 of 1018) for testing (215 BAV, 154 TAV, 49 controls; age range = 4–91 years, 83 pediatric patients). The validation data set was used to adjust the network hyperparameters (eg, learning rate). The total training time was 22 hours. The time needed to perform a CNN segmentation for a single 4D-flow data set was 0.438 ± 0.355 seconds compared with 630 ± 254 seconds for manual segmentation.

FIGURE 3.

Study cohort breakdown showing the number of subjects and subgroups (bicuspid aortic valve [BAV], tricuspid aortic valve [TAV], and controls) for CNN training, validation, and testing

3.3 |. Convolutional neural network performance

As summarized in Table 2, the aorta segmentation CNN demonstrated the following performance for the testing cohort: DS: 0.951 (0.930–0.966), HD: 2.80 (2.13–4.35), and ASSD: 0.176 (0.119–0.290). For individual cohorts, the CNN showed mildly lower agreement between automated and manual segmentation for controls (DS: 0.938 [0.916–0.958], HD: 2.50 [2.24–5.17], and ASSD: 0.186 [0.132–0.420]) and best performance for TAV patients (DS: 0.958 [0.944–0.971], HD: 2.40 [2.13–3.16], and ASSD: 0.143 [0.098–0.209]). Figure 4 provides three examples of the manual (red) and automated (blue) segmentations for subjects with an excellent DS (0.958), an intermediate DS (0.898), and the lowest DS (0.766) from the entire testing data set.

TABLE 2.

Metrics of agreement for manual versus automatic segmentation in testing cohort

	All (N = 418)	BAV (N = 215)	TAV (N = 154)	Controls (N = 49)
DS	0.951 [0.930–0.966]	0.946 [0.926–0.963]	0.958 [0.944–0.971]a,b	0.938 [0.916–0.958]a,c
HD	2.80 [2.13–4.35]	3.00 [2.13–6.37]	2.40 [2.13–3.16]a,b	2.50 [2.24–5.17]c
ASSD	0.176 [0.119–0.290]	0.194 [0.130–0.353]a	0.143 [0.098–0.209]a,b	0.186 [0.132–0.420]c

Note: All metrics are reported as median [interquartile range].

Abbreviations: ASSD, average symmetric surface distance; HD, Hausdorff distance.

^aIndicates a significant difference compared with all cohorts.

^bIndicates a significant difference compared with the BAV cohort.

^cIndicates a significant difference compared with the TAV cohort.

FIGURE 4.

Representative 3D segmentations of the thoracic aorta. In each panel, the manual 3D aorta segmentation is shown in red, and the CNN-based automated 3D aorta segmentation is shown in blue. The difference maps show aortic regions with differences between CNN-based segmentation versus manual 3D aorta segmentation. The top panel shows a segmentation with excellent agreement between manual and automated segmentations, despite complex anatomy. The center panel shows a below-average segmentation with a Dice score (DS) of 0.89, and the bottom panel shows the subject who had the lowest DS (ie, the worst agreement between CNN-based vs manual aorta 3D segmentation)

In the testing cohort, 34 data sets had DS below 0.90. Figure 5 summarizes the most frequent discrepancies and errors found between the manual and CNN segmentations in this subgroup. The most common disagreements were aortic root segmentation differences (N = 26; Figure 5A), with the CNN more likely to include the aortic root and outflow tract. Branch segmentation differences were another common discrepancy (N = 10; Figure 5B) with differences in the superior extents of aortic branches. Additionally, there were differences in aortic size where the manual (N = 10; Figure 5C) or the automated segmentation (N = 9; Figure 5D) was larger. Finally, several data sets were found to have discrepancies between the CNN and manual segmentations in distinguishing the AAo from the pulmonary arteries (N = 6; Figure 5E) or in the distal DAo (N = 5; Figure 5F).

FIGURE 5.

Failure modes in all N = 34 CNN-based automated aorta 3D segmentations with DS < 0.90 with a representative example for each failure type. A, In N = 26 cases, manual and automatic segmentations differed at the aortic root. Most commonly (N = 20), the CNN segmented the left ventricular outflow tract while the manual observer cut the segmentation at the aortic valve. B, Branch segmentation differences. Here the CNN segmented the supra-aortic branches more extensively, of which there are N = 10 cases. Examples of N = 9 cases (C) and N = 8 cases (D) show segmentations where the manual or CNN segmentations, respectively, were larger over the entirety of the segmentation. E, Example of N = 6 cases shows an example in which the segmentation of the distal DAo differs between manual and CNN-based segmentations. F, Example of N = 5 cases shows an example in which there were differences between manual and CNN-based segmentations in the region of the AAo, where the pulmonary artery crosses. One segmentation had a low DS due to a poor manual segmentation that was not caught during manual quality control (not shown). Note that each segmentation can have multiple problems (as seen in [E], where there are differences in both the DAo and branches), so the failures do not add to the number of segmentations with DS < 0.90. Abbreviation: PA, pulmonary artery

3.4 |. Effects of demographic and scan parameters on CNN performance

The aorta segmentation CNN performance was stable with age (<0.001 decrease in DS per year, P = .05), venc (<0.001 decrease in DS per cm/s increase, P = .06), heart rate (<0.001 change in DS per beat per minute, P = .48), and body mass index (<0.001 change in DS per 1-kg/m² increase, P = .38). In addition, the CNN performance saw no significant differences in field strength (median 1.5 T: DS = 0.951 [0.933–0.967]; median 3 T: DS = 0.945 [0.916–0.961], P = .19) or sex (median males: DS = 0.950 [0.933–0.966], median females: DS = 0.952 [0.925–0.967], P = .67). The CNN performance was sensitive to voxel size (0.0012 decrease in DS per 1-mm³ increase, P < .001) and the administration of gadolinium contrast (contrast: N = 890, median DS = 0.954 [0.935–0.968]; no contrast: N = 128, median DS = 0.938 [0.921–0.954], P < .001).

3.5 |. Flow, volume, and diameter comparisons

Aortic diameter comparisons across all regions are summarized in Table 3, with Bland-Altman analysis for the testing cohort showing bias: −0.2 to <0.001 mm, LOA: ±1.9 to ±2.4 mm, LOA percent difference: 6.6%−9.4%. Aortic diameter Bland-Altman plot is available in Supporting Information Figure S1. Controls showed the largest differences between manual and CNN segmentations, with LOA greater than 10% in the arch (bias: 0.1 mm, LOA: ±3.0 mm, LOA difference: 12.2%) and DAo (bias: 0.1 mm, LOA: ±2.5 mm, LOA difference: 10.8%). For total aorta volume, Bland-Altman analysis (Supporting Information Figure S2) showed a bias of < 0.001 mL, LOA: ±33.6 mL, and LOA percent difference: 16.3%.

TABLE 3.

Geometry measurements for manual versus automatic segmentation in testing cohort

	All (N = 418)	BAV (N = 215)	TAV (N = 154)	Controls (N = 49)
AAo diameter (mm)
Mean (SD)
Manual	36.4 (7.16)	39.0 (6.37)	34.1 (7.58)	32.3 (3.95)
Automatic	36.2 (6.92)	38.7 (6.10)	34.0 (7.37)	32.5 (3.99)
Bias ± LOA	−0.2 ± 2.4	−0.3 ± 2.5	−0.2 ± 2.1	0.2 ± 2.6
Arch diameter (mm)
Mean (SD)
Manual	24.8 (4.30)	24.9 (3.99)	24.7 (5.01)	24.6 (2.89)
Automatic	24.8 (4.23)	24.8 (3.94)	24.7 (4.89)	24.7 (3.02)
Bias ± LOA	<0.001 ± 2.3	−0.1 ± 2.4	<0.001 ± 1.9	0.1 ± 3.0
DAo diameter (mm)
Mean (SD)
Manual	23.5 (4.37)	23.8 (4.05)	23.1 (5.15)	23.2 (2.59)
Automatic	23.3 (4.31)	23.5 (4.04)	23.0 (4.99)	23.2 (2.70)
Bias ± LOA	−0.2 ± 1.9	−0.3 ± 1.8	−0.1 ± 1.8	0.1 ± 2.5
Volume (mL)
Mean (SD)
Manual	206.6 (82.2)	219.6 (74.9)	201.7 (96.0)	165.1 (38.7)
Automatic	206.6 (80.5)	219.1 (73.1)	201.5 (94.0)	167.6 (41.3)
Bias ± LOA	<0.001 ± 33.6	−0.5 ± 35.0	−0.2 ± 32.7	2.6 ± 30.7

Note: Limits of agreement (LOA) are ± 1.96

^*SD of the differences.

Regional flow and peak velocity comparisons between manual and CNN segmentations are summarized in Table 4. Four BAV subjects with severe AAo velocity aliasing were excluded, resulting in a total of 414 subjects for this analysis. For peak velocity, CNN segmentations for the entire testing cohort showed a low bias of <0.001 m/s and LOAs of 0.01% difference across all regions (Supporting Information Figure S3). Similarly, net flow was found to have a low bias between −0.2 to 0.1 mL/cycle and strong LOAs with 6.4%−9.2% difference at the AAo, arch, and DAo (Supporting Information Figure S4). For peak flow, the automated segmentation had a bias between −0.9 and −0.2 mL/s and LOAs of 6.7%−9.0% difference across all planes (Supporting Information Figure S5). The largest disparity was found for arch peak flow in controls (bias: 0.3 mL/s, LOA: ±31.9 mL/s, difference: 13.7%). For net regurgitation, there was a low bias and LOAs between 14.3% and 15.8% difference across all planes for the testing cohort (Supporting Information Figure S6). Figure 6 provides three examples of the diameter measurements and flow-time curves at the AAo, arch, and DAo for the 3 subjects seen in Figure 4.

TABLE 4.

Flow measurements for manual versus automatic segmentation in testing cohort

	All (N = 414)*	BAV (N = 211)*	TAV (N = 154)	Controls (N = 49)
Peak velocity (m/s)
AAo
Mean (SD)
Manual	1.00 (0.47)	1.22 (0.49)	0.77 (0.33)	0.74 (0.19)
Automatic	1.00 (0.47)	1.22 (0.49)	0.77 (0.33)	0.74 (0.19)
Bias ± LOA	<0.001 ± 0.01	<0.001 ± 0.02	<0.001 ± 0.01	<0.001 ± 0.01
Arch
Mean (SD)
Manual	0.91 (0.33)	1.02 (0.36)	0.82 (0.25)	0.75 (0.16)
Automatic	0.91 (0.33)	1.02 (0.36)	0.82 (0.25)	0.75 (0.16)
Bias ± LOA	<0.001 ± 0.01	<0.001 ± 0.01	<0.001 ± 0.01	<0.001 ± 0.01
Dao
Mean (SD)
Manual	0.93 (0.38)	0.98 (0.39)	0.88 (0.39)	0.82 (0.26)
Automatic	0.93 (0.38)	0.98 (0.39)	0.88 (0.39)	0.82 (0.26)
Bias ± LOA	<0.001 ± 0.01	<0.001 ± 0.01	<0.001 ± 0.01	<0.001 ± 0.01
Net flow (mL/cycle)
AAo
Mean (SD)
Manual	82.5 (25.7)	86.8 (25.5)	78.4 (26.1)	75.9 (21.4)
Automatic	82.6 (25.4)	87.0 (25.3)	78.4 (25.7)	76.2 (21.2)
Bias ± LOA	0.1 ± 5.3	0.2 ± 5	<0.001 ± 5.6	0.3 ± 5.6
Arch
Mean (SD)
Manual	54.1 (19.0)	56.9 (19.6)	52.3 (19.1)	47.4 (12.7)
Automatic	54.0 (18.9)	56.6 (19.5)	52.5 (19.1)	47.5 (12.5)
Bias ± LOA	−0.1 ± 5.0	−0.3 ± 5.0	0.2 ± 4.5	0.1 ± 6.4
Dao
Mean (SD)
Manual	47.9 (15.4)	49.7 (15.3)	46.3 (16.3)	44.5 (11.6)
Automatic	47.7 (15.2)	49.4 (15.2)	46.3 (16.1)	44.8 (11.4)
Bias ± LOA	−0.2 ± 3.5	−0.4 ± 3.8	<0.001 ± 2.5	0.3 ± 4.5
Peak flow (mL/s)
AAo
Mean (SD)
Manual	397.4 (132.4)	434.3 (142.4)	361.5 (113.3)	348.9 (94.5)
Automatic	397.2 (131.2)	433.6 (141.2)	361.5 (111.9)	350.2 (93.2)
Bias ± LOA	−0.2 ± 26.7	−0.7 ± 30.6	0.1 ± 19.8	1.3 ± 27.8
Arch
Mean (SD)
Manual	265.9 (99.2)	291.0 (107.1)	241.4 (87.6)	233.0 (63.0)
Automatic	265.5 (98.5)	289.5 (106.6)	242.2 (87.2)	233.3 (61.4)
Bias ± LOA	−0.4 ± 24.0	−1.5 ± 24.1	0.8 ± 20.5	0.3 ± 31.9
Dao
Mean (SD)
Manual	244.1 (89.7)	262.5 (96.7)	224.3 (82.3)	225.7 (58.5)
Automatic	243.2 (88.9)	260.6 (96.7)	224 (81.2)	227.2 (55.5)
Bias ± LOA	−0.9 ± 19.4	−1.9 ± 20.4	−0.3 ± 14.7	1.5 ± 26.5
Net regurgitation (mL/cycle)
AAo
Mean (SD)
Manual	−5.19 (11.0)	−8.29 (14.2)	−2.37 (5.02)	−0.43 (0.72)
Automatic	−5.17 (11.1)	−8.27 (14.2)	−2.32 (5.07)	−0.43 (0.75)
Bias ± LOA	0.02 ± 0.82	0.02 ± 0.93	0.06 ± 0.63	<0.001 ± 0.34
Arch
Mean (SD)
Manual	−3.92 (8.03)	−6.06 (10.4)	−1.89 (3.56)	−0.88 (0.96)
Automatic	−3.92 (8.05)	−6.04 (10.5)	−1.90 (3.54)	−0.88 (0.94)
Bias ± LOA	<0.001 ± 0.62	0.01 ± 0.78	−0.01 ± 0.38	<0.001 ± 0.21
DAo
Mean (SD)
Manual	−3.92 (7.25)	−5.63 (9.21)	−2.35 (3.95)	−1.35 (1.78)
Automatic	−3.90 (7.26)	−5.60 (9.24)	−2.35 (3.96)	−1.35 (1.73)
Bias ± LOA	0.02 ± 0.56	0.04 ± 0.65	<0.001 ± 0.45	0.01 ± 0.36

Note: Four BAV subjects displayed severe aliasing in the AAo and were excluded in flow analysis. Limits of agreement are ± 1.96

^*SD of the differences.

FIGURE 6.

Representative planar flow analysis for the same subjects shown in Figure 4 (from top to bottom: excellent DS, below average DS, and lowest DS). Left panel: Aorta 3D segmentation difference map (CNN-based vs manual) and three manually placed 2D analysis planes used for quantification of flow and aortic dimensions. Central panel: Mask comparison on each 2D analysis plane and diameter measurement (blue, automated segmentation was larger; red, manual segmentation was larger; purple, shared pixels). Right panel: Flow waveforms for each of the three planes (solid lines, manual 3D aorta segmentation; dashed lines, CNN-based automated aorta 3D segmentation)

3.6 |. Interobserver comparison

The CNN performed as well as our expert observers. Interobserver comparisons (between observers 1 and 2) showed a DS of 0.950 (0.931–0.960), HD of 2.45 (2.13–3.00), and ASSD of 0.173 (0.118–0.242). Between observer 1 and the CNN, the median DS was 0.958 (0.935–0.966), HD was 2.24 (2.13–2.52), and ASSD was 0.145 (0.110–0.211). Similarly, between observer 2 and the CNN, the median DS was 0.953 (0.927–0.967), HD was 2.91 (2.26–4.05), and ASSD of 0.198 (0.114–0.299). Bland-Altman analysis of the flow and diameter comparisons between the two observers and each observer to the CNN are summarized in Supporting Information Table S1. For diameter measurements, peak velocity, net flow, and peak flow, there was a low bias and LOA < 10% difference for the interobserver comparisons as well as for the comparisons of both observers to the CNN. For net regurgitation, all comparisons showed similar LOAs of between 10% and 20% difference.

4 |. DISCUSSION

Using fewer than 1000 4D-flow data sets, we developed a CNN for fast and automated 3D segmentation of the thoracic aorta from 4D-flow MRI data. The main findings of our study were (1) CNN-based aorta 3D segmentation was feasible in <1 second versus 630 seconds for manual 3D segmentation; (2) the CNN showed excellent performance compared with standard manual analysis (median DS = 0.951 across 418 subjects); (3) the CNN was stable across a wide range of ages (2–91), body mass index, different genders, aortic valve phenotypes (bicuspid and tricuspid valves), and MRI field strength; (4) CNN-based 3D segmentation provided reliable quantification of aortic flow and dimensions (Bland Altman analysis: low bias and limits of agreements generally lower than 10% difference of the mean ground-truth values); and (5) the CNN achieved DSs (median observers 1 and 2: 0.953 and 0.958) at the level of our interobserver comparison (median: 0.950).

Previous methods of automated or semi-automated aortic segmentation of 4D flow were either time-consuming or required manual interactions. Bustamante et al used an atlas-based registration to automatically generate time-resolved segmentations, requiring an initial execution time of 3600 seconds.²⁴ An alternative method by van Pelt et al used an active surface segmentation algorithm, which while rapid (execution time: 5 seconds), required an approximate initial isosurface based on manual interactions for efficient and accurate performance.⁹ Our CNN provided fully automated 3D aortic segmentations, requiring no manual inputs, in less than 1 second.

Performance-wise, our CNN was on par with other automated segmentations of the aorta from 2D phase-contrast MRI. Herment et al used a 2D+time deformable surface technique to achieve DSs of 0.945 ± 0.014 in a study using healthy controls and patients with dilated aorta (N = 52).²⁵ A deep learning–based approach was used by Bratt et al for fully automated aortic 2D segmentation and flow quantification, achieving an average DS of 0.94 and high flow correlations to manual and alternative automated segmentation methods (N = 270).²⁶ We achieved a similar level of performance as 2D phase-contrast MRI segmentations for the 3D segmentation of 4D flow. Compared with 2D, 3D segmentations required the incorporation of more complex spatial features to automate and accurately segment. Additionally, flow quantifications were limited with 2D segmentations, inhibiting the ability to examine complex hemodynamics throughout the entire thoracic aorta. With 3D segmentations, we were able to take full advantage of the 4D-flow MRI information.

By providing rapid and accurate aortic flow and diameter metrics, our CNN was able to improve the repeatability and clinical feasibility of 4D-flow MRI. To obtain accurate hemodynamic information, 4D-flow MRI required extensive and time-consuming preprocessing, with manual segmentation being the most cumbersome. Rapid, automatic, and repeatable segmentation of 4D-flow MRI addressed a key limitation in bringing 4D flow into widespread clinical use.

Our interobserver comparison indicated that our CNN was well within human-level performance and comparable to prior work manually segmenting 4D-flow MRI. In a study of 14 patients, van Ooij et al achieved an interobserver DS of 0.94 ± 0.2.²⁷ In addition, our CNN was shown to be robust to patient characteristics and valve morphology, indicating its capabilities for a clinical setting. Further studies are needed to demonstrate our CNN’s performance on patient-specific aortic pathologies and its application to 4D-flow data of other vessels.

Our study had several limitations. To avoid computational memory constraints from processing large 3D data sets, our CNN used cropped 4D-flow data sets as an input, providing segmentations of a certain size. In the future, we will explore using the developed CNN several times in a tiled fashion to cover the whole data set. In addition, our CNN currently only performs accurately using mean sum-of-squares PCMRAs: Attempting to use an alternative PCMRA calculation as an input would require the CNN to be retrained either from scratch or through transfer learning. Another limitation is that our flow analysis was performed using manual plane placement, which may result in observer variability. An automated, deep learning network to designate and separate different regions of the thoracic aorta is currently being developed to provide faster or more robust measures. Furthermore, our data were preprocessed by more than 20 operators with a wide variety of experience and skill, generating a large data set of varying quality. Although our reviewers assessed the quality of the segmentation to ensure an accurate ground truth, the velocity 4D-flow data were not subjected to review, which may result in unusual and inaccurate flow measurements. All data sets underwent a standard 4D workflow²⁸; however, our workflow requires manual user inputs, which when performed poorly can result in a distortion of the flow data.²⁹ For the flow quantitative measures, the same 4D-flow velocity data were used for both the manual and automated flow assessment; as such, any inaccuracies would be present in both data sets. Although our CNN was able to be integrated into a standard 4D-flow analysis pipeline by incorporating manually preprocessed data, we are currently developing a fully automated preprocessing pipeline for 4D-flow MRI to address this limitation. Additionally, our data set was acquired using MRI systems from a single vendor and a specific 4D-flow MRI protocol. This may result in a selection bias. Further work would be required using a multicenter, multivendor cohort to explore the generalizability of our CNN’s performance. Our CNN had a slight decline in performance based on a decline in image quality (resolution) or lower SNR (contrast). Tellingly, our CNN showed the worst performance on controls, the smallest group in our cohort, and the group with the lowest proportion of contrast-enhanced scans. The incorporation of additional noncontrast or down-sampled data sets during training may improve the CNN performance on noncontrast and/or lower-resolution scans. Additionally, our results showed net regurgitation to have the highest variance between manual and CNN segmentations across all cohorts. Net regurgitation volume was generally small compared with net flow and were very sensitive to slight changes in the segmentation. These results were also reflected in our interobserver comparisons, which similarly showed net regurgitation to have the highest variance between the manual observers as well as the observers to the CNN—all showing similar levels of agreement. Another issue is that our automated segmentations were static and generated from time-averaged data. As such, our analysis did not take into account the motion of the aorta throughout the cardiac cycle. Generating a time-resolved segmentation is very time-consuming, requiring the operator to perform 3D segmentation for each cardiac time frame in the 4D-flow data set (generally between 10 and 30 time frames), and is further complicated by reduced signal during diastole when velocities are lower. In the future, we hope to use the transfer learning approach from our current CNN model to perform automated time-resolved segmentation, allowing us to reduce the number of training data sets needed. Finally, although our cohort contained patients with various aortic pathologies (eg, coartation, BAV, Marfan syndrome), we were not able to perform a systematic study to assess how well our CNN performs on that cohort. A future direction of this study is to assess the CNN performance on disease-specific cohorts.

5 |. CONCLUSIONS

A CNN was developed for rapid, fully automated 3D segmentation of the thoracic aorta from 4D-flow MRI, yielding human-level performance and excellent agreement across flow and diameter measurements when compared with manual segmentations. Future studies should investigate its utility for other vasculature and multicenter, multivendor applications.

Supplementary Material

supp materials

FIGURE S1 Aorta diameter correlation and Bland-Altman plots. Values represent the maximum diameter calculated using a best-fit ellipse on manually placed planes between manual and automatically generated 3D segmentations in the testing set (N = 418). Red boxes indicate the ascending aorta (AAo); blue boxes indicate the arch; and green boxes indicate the descending aorta (DAo). Bland-Altman bias and limits of agreement (LOAs) are for all measurements together

FIGURE S2 Aorta volume correlation and Bland-Altman plots. Values represent the volume of the thoracic aorta compared between manual and automatically generated 3D segmentations in the testing set (N = 414). Bland-Altman bias and LOAs are for all measurements together. Four bicuspid aortic valve (BAV) subjects were excluded from this analysis due to severe velocity aliasing

FIGURE S3 Maximum velocity correlation and Bland-Altman plots. Values represent the maximum 95% maximum velocity on manually placed planes between manual and automatically generated 3D segmentations in the testing set (N = 414). Red boxes indicate the AAo; blue boxes indicate the arch; and green boxes indicate the DAo. Bland-Altman bias and limits of agreement are for all measurements together. Four BAV subjects were excluded from this analysis due to severe velocity aliasing

FIGURE S4 Net flow correlation and Bland-Altman plots. Values represent the net flow over the cardiac cycle through manually placed planes between manual and automatically generated 3D segmentations in the testing set (N = 414). Red boxes indicate the AAo; blue boxes indicate the arch; and green boxes indicate the DAo. Bland-Altman bias and LOAs are for all measurements together. Four BAV subjects were excluded from this analysis due to severe velocity aliasing

FIGURE S5 Peak-flow correlation and Bland-Altman plots. Values represent the peak flow through manually placed planes between manual and automatically generated 3D segmentations in the testing set (N = 414). Red boxes indicate the AAo; blue boxes indicate the arch; and green boxes indicate the DAo. Bland-Altman bias and LOAs are for all measurements together. Four BAV subjects were excluded from this analysis due to severe velocity aliasing

FIGURE S6 Net regurgitation correlation and Bland-Altman plots. Values represent the net regurgitation through manually placed planes between manual and automatically generated 3D segmentations in the testing set (N = 414). Red boxes indicate the AAo; blue boxes indicate the arch; and green boxes indicate the DAo. Bland-Altman bias and limits of agreement are for all measurements together. Four BAV subjects were excluded from this analysis due to severe velocity aliasing

TABLE S1 Complete interobserver comparisons across all diameter and flow metrics. Note: The results are presented for comparisons between observer 1 and the CNN (column 1), between observer 2 and the CNN (column 2), and between observers 1 and 2 (column 3)