A Deep Learning Segmentation Pipeline for Cardiac T1 Mapping Using MRI Relaxation–based Synthetic Contrast Augmentation

Abstract

Purpose

To design and evaluate an automated deep learning method for segmentation and analysis of cardiac MRI T1 maps with use of synthetic T1-weighted images for MRI relaxation–based contrast augmentation.

Materials and Methods

This retrospective study included MRI scans acquired between 2016 and 2019 from 100 patients (mean age ± SD, 55 years ± 13; 72 men) across various clinical abnormalities with use of a modified Look-Locker inversion recovery, or MOLLI, sequence to quantify native T1 (T1_native), postcontrast T1 (T1_post), and extracellular volume (ECV). Data were divided into training (n = 60) and internal (n = 40) test subsets. “Synthetic” T1-weighted images were generated from the T1 exponential inversion-recovery signal model at a range of optimal inversion times, yielding high blood-myocardium contrast, and were used for contrast-based image augmentation during training and testing of a convolutional neural network for myocardial segmentation. Automated segmentation, T1, and ECV were compared with experts with use of Dice similarity coefficients (DSCs), correlation coefficients, and Bland-Altman analysis. An external test dataset (n = 147) was used to assess model generalization.

Results

Internal testing showed high myocardial DSC relative to experts (0.81 ± 0.08), which was similar to interobserver DSC (0.81 ± 0.08). Automated segmental measurements strongly correlated with experts (T1_native, R = 0.87; T1_post, R = 0.91; ECV, R = 0.92), which were similar to interobserver correlation (T1_native, R = 0.86; T1_post, R = 0.94; ECV, R = 0.95). External testing showed strong DSC (0.80 ± 0.09) and T1_native correlation (R = 0.88) between automatic and expert analysis.

Conclusion

This deep learning method leveraging synthetic contrast augmentation may provide accurate automated T1 and ECV analysis for cardiac MRI data acquired across different abnormalities, centers, scanners, and T1 sequences.

Keywords: MRI, Cardiac, Tissue Characterization, Segmentation, Convolutional Neural Network, Deep Learning Algorithms, Machine Learning Algorithms, Supervised Learning

Supplemental material is available for this article.

© RSNA, 2022

Summary

A deep neural network leveraging MRI relaxation–based synthetic data for automated segmentation and analysis of cardiac T1 maps demonstrated accurate myocardial segmentation and T1 and extracellular volume calculation relative to interobserver variability.

Key Points

• ■ Synthetic cardiac MRI T1-weighted images can be generated with high blood-myocardium contrast and used in training and testing a convolutional neural network for robust T1 map segmentation.

• ■ Networks that use synthetic T1-weighted images in training and testing achieved higher segmentation success rates (99.2%) compared with networks that use raw T1-weighted images (96.8%) or T1 maps (94.4%).

• ■ The proposed method demonstrated high myocardial Dice coefficient (0.81 ± 0.08) and accurate T1 calculation with respect to experts for an internal test dataset for native T1 (R = 0.87) and postcontrast T1 (R = 0.91) and an external test dataset for native T1 (R = 0.88) acquired at another center with use of a different scanner, T1 sequence, and slice protocol.

Introduction

Cardiac MRI relaxometry is clinically used to quantitatively characterize various cardiovascular conditions, such as myocardial infarction (1), myocarditis (2), amyloidosis (3), and cardiomyopathy (4). Single-slice and multislice short-axis myocardial T1-mapping protocols have enabled quantification of global and local tissue alterations, including edema and fibrosis, across these pathologic states (5). Furthermore, precontrast (native) T1 (T1_native) and postcontrast T1 (T1_post) mapping can be combined to provide an estimate of extracellular volume (ECV). Clinically, T1 and ECV can be used to differentiate cardiac abnormality and potentially grade disease severity and risk stratification (6,7).

Currently, the measurement of segmental myocardial T1 and ECV requires manual delineation of the left ventricle (LV) myocardium, LV blood pool, and right ventricular (RV) insertion point (RVIP) in both native and postcontrast T1 maps. Manual delineation of myocardial contours in cardiac MRI is laborious and susceptible to interobserver variability (8,9), suggesting the benefit of automated segmentation and analysis. Recent studies investigated deep learning methods, such as convolutional neural networks for automated myocardial segmentation and quantitative assessment of cardiac MRI, including cine imaging and T1 maps (10–14). However, to our knowledge, a fully automated convolutional neural network–based method showing robust global and segmental T1 and ECV quantification across image contrasts and T1 acquisition protocols has not been validated against expert observers in internal and external datasets.

In this study, we investigated a fully automated segmental relaxometry method using synthetic contrast augmentation with neural networks (FASTR-SCANN) for automated native, postcontrast T1 map segmentation and quantification of global and segmental myocardial T1 and ECV. For this purpose, we propose the use of “synthetic” inversion recovery images derived from the fitted T1 relaxation signal model, which are generated with high blood-myocardium contrast, during training and testing of a convolutional neural network (15).

Materials and Methods

Patient Data and MRI

Internal dataset.— The Research Ethics Board of Sunnybrook Health Sciences Center approved the study, and all patients provided written informed consent. In this retrospective study, 100 consecutive patient examinations performed between 2016 and 2019 were included (mean age ± SD, 55 years ± 13; 72 men); these were clinically categorized as the following: no abnormality (normal, n = 23), nonischemic dilated cardiomyopathy (n = 23), ischemic heart disease (n = 30), and hypertrophic cardiomyopathy (n = 24) (see Appendix E1 [supplement]). A modified Look-Locker inversion recovery (MOLLI) sequence (1.5 T; GE Healthcare) was used for T1_native and T1_post mapping of three short-axis slices (see Appendix E2 [supplement]). This dataset was divided into training (60 patients; 358 images) and testing (40 patients; 240 images) subsets. The training dataset was manually annotated by a single expert (X.Q., with 14 years of experience in cardiac MRI [denoted as expert-trainer]), and the testing dataset was annotated by two blinded cardiothoracic radiologists (L.G., with 5 years of experience [denoted as expert 1] and L.J.J., with 11 years of experience [denoted as expert 2]). Experts annotated the LV myocardium (epicardium and endocardium), RV blood pool (endocardium), and upper RVIP in one T1-weighted image per slice (see Appendix E3 [supplement]). Hematocrit level was available for all patients for ECV computation.

External test dataset.— An independent, publicly available dataset was used for external testing of our method (10,16). This dataset of 147 patients was acquired with use of the free-breathing slice interleaved T1 (ie, STONE) sequence (1.5 T; Philips) for T1_native mapping of five short-axis slices (735 images). Manual annotations of LV myocardium by a single expert were available. Postcontrast data were not available, and thus, T1_post and ECV could not be assessed.

Automated Pipeline Design

The overview of our method is shown in Figure 1. T1-weighted images were coregistered between inversion times (TIs) to correct for in-plane motion with use of the deformable modality independent neighborhood descriptor approach (17–19). T1 reconstruction was performed by nonlinear fitting of the MRI exponential inversion-recovery signal model:

where t was the TI, S(t) was the T1-weighted image voxel signal at time t, T1 was the T1 relaxation time, and A and B were fitting parameters (see Appendix E4 [supplement]). With use of the values of T1, A, and B from the T1 fitting, a “synthetic” inversion recovery (synthetic T1-weighted) image was generated with use of Equation 1 at desired TIs. We calculated a range of desirable TIs, TI^opt (native: [289, 655] msec; postcontrast: [306, 810] msec), from our training data such that synthetic images generated across TI^opt would have high blood-myocardium contrast for both native and postcontrast series (see Appendix E5 [supplement]). During network training, contrast augmentation was performed by generating synthetic T1-weighted images at randomly sampled TIs in the TI^opt range to augment the training data; these images were used to train a single U-Net model (U-Net_SCANN) for joint T1_native and T1_post segmentation (20). During network testing, synthetic “test candidate” images with varying blood-myocardium contrast were generated at 19 TIs within TI^opt and provided to U-Net_SCANN to obtain LV myocardium, LV blood pool, and RV blood pool predictions. Results derived from the synthetic “test candidate” images were combined with use of label voting to generate final predictions and remove low-confidence pixel labels (regions of uncertainty) (14) (see Appendix E6 [supplement]). The LV myocardium segmentation was checked with use of shape conditions (11), and refinements with use of morphologic operations were applied as necessary; segmentations that did not meet shape conditions after refinement were flagged as failure cases (see Appendix E7 [supplement]). The RV blood pool was used to automatically detect the upper RVIP, and in turn used to determine American Heart Association segments (see Appendix E8 [supplement]). Myocardial T1 values were measured following compression of the epicardial and endocardial contours toward the mid myocardium by 20% to avoid partial volume effects from blood. The American Heart Association 17-segment model was used for segmental T1 analysis (21). Segmental myocardial T1 was used to compute ECV (6) (see Appendix E9 [supplement]).

Figure 1:

Overview of proposed fully automated segmental relaxometry method using synthetic contrast augmentation with neural networks (FASTR-SCANN) deep learning pipeline. Processing began with T1-weighted (T1w) cardiac MRI scans from native and postcontrast series acquired from a single patient. Standard T1 fitting was performed to reconstruct T1 maps. The T1 relaxation curve was also used to generate a set of synthetic T1-weighted images at optimal inversion times. Synthetic images were then forwarded as segmentation candidates to a U-Net for segmentation of left ventricular (LV) blood pool, LV myocardium, and right ventricular (RV) blood pool. Synthetic candidate segmentations were combined with use of pixelwise label voting to obtain the final segmentation result. Interface of LV myocardium and RV blood pool segmentations were used to detect RV insertion point (RVIP) (see Appendix E7 [supplement]). Finally, native and postcontrast segmentation results were used to compute segmental T1 and extracellular volume (ECV) according to the American Heart Association (AHA) 17-segment model.

Training and Evaluation

For all experiments, a U-Net architecture was used (22) (see Appendix E10 [supplement]). We compared U-Net_SCANN to networks trained with raw T1-weighted images (U-Net_RAW) and T1 maps (U-Net_T1MAP) with use of stratified fivefold cross-validation on the training dataset. U-Net_SCANN was evaluated on internal testing data with respect to two testing experts (experts 1 and 2). Automatic versus expert T1 and ECV were compared on a per-patient, per-slice, and per-segment basis. RVIP accuracy was assessed relative to experts by Euclidean distance. U-Net_SCANN was also evaluated on the external test dataset.

Statistical Analysis

Results are reported as the means ± SDs. Data normality was assessed with use of the Kolmogorov-Smirnov test. Segmentation quality was assessed with use of Dice similarity coefficient (DSC) and the segmentation success rate (percentage of segmentations that pass myocardial shape checking). Differences in T1 measurements across network designs were assessed with use of the Friedman test, with the Dunn post hoc test for pairwise comparison. Differences in automatic and average-expert T1 and ECV were assessed with use of 95% CIs obtained with paired t test or Wilcoxon signed rank test, as appropriate based on data normality. The Pearson correlation coefficient (R) and Bland-Altman analysis were used for the comparison of automatic and expert T1 and ECV. Analyses were performed with use of Prism 9 software (GraphPad), with the significance level set at P < .05.

Results

Cross-Validation and Network Comparisons

Figure 2 shows representative segmentation results with use of each of the three network designs, which were compared by examining cross-validation results. The success rates of myocardial segmentation after shape analysis were 99.2% (355 of 358 images) for U-Net_SCANN, 96.8% (347 of 358 images) for U-Net_RAW, and 94.4% (338 of 358 images) for U-Net_T1MAP. Myocardial DSCs for successful segmentations produced by each network versus expert-trainer were 0.84 ± 0.07 for U-Net_SCANN, 0.85 ± 0.06 for U-Net_RAW, and 0.80 ± 0.08 for U-Net_T1MAP (see prediction confidence analysis in Appendix E11 [supplement]). In patients for whom slices were successfully segmented by all network designs, we found no evidence of differences in per-patient myocardial T1_native between expert-trainer (1036 msec ± 71) versus U-Net_SCANN (1030 msec ± 68; ΔT1 = −6.0 msec; P = .33), U-Net_RAW (1042 msec ± 75; ΔT1 = 5.7 msec; P = .70), and U-Net_T1MAP (1032 msec ± 72; ΔT1 = -4.3 msec; P = .50). Similarly, we found no evidence of differences in per-patient myocardial T1_post between expert-trainer (411 msec ± 42) versus U-Net_SCANN (410 msec ± 42; ΔT1 = -0.5 msec; P = .96) and U-Net_RAW (409 msec ± 45; ΔT1 = -1.7 msec; P = .38); however, T1_post was lower for expert-trainer versus U-Net_T1MAP (407 msec ± 43; ΔT1 = -4.3 msec; P = .04).

Figure 2:

Representative T1-weighted images and T1 maps along with automatic left ventricular (LV) endocardial (red), LV epicardial (green), and right ventricular (RV) endocardial (yellow) contours from two patients comparing U-Net_SCANN, U-Net_RAW, and U-Net_T1MAP network designs during fivefold cross-validation. Segmentation results for the 19 synthetic T1-weighted candidate images across a range of inversion times (TI) produced by U-Net_SCANN are shown for (A) T1_native and (B) T1_post series; for each series, areas of uncertainty (Myo. uncertainty) are shown by displaying the final high-confidence myocardial mask (green), and low-confidence uncertain regions (red), obtained from label voting, are shown overlaid on T1 maps. Final segmentation contours from each network design are shown for (C) T1_native and (D) T1_post series, along with manual expert annotations. T1_native = native T1, T1_post = postcontrast T1.

Internal Testing with FASTR-SCANN

Figure 3 shows automated segmentation and per-segment T1 and ECV derived for representative patients in the test set from healthy and various abnormality groups. Segmentation was accurate for both T1_native and T1_post series irrespective of underlying abnormality. Per-segment T1_native and ECV showed regional distributions that were consistent with the diffuse or focal nature of the underlying abnormality.

Figure 3:

Comparison of automatic and expert left ventricular (LV) endocardial (red), LV epicardial (green), and right ventricular (RV) endocardial (yellow) contours along with representative automatic segmentation results across different abnormality groups. (A) LV and RV contours derived from fully automated segmental relaxometry method using synthetic contrast augmentation with neural networks (FASTR-SCANN), expert 1, and expert 2 analysis are shown overlaid on T1_native and T1_post maps. (B) FASTR-SCANN contours overlaid on T1_native and T1_post maps along with American Heart Association bull’s-eye plots show T1 and extracellular volume (ECV) measurements for normal and nonischemic dilated cardiomyopathy (NICM), ischemic heart disease (IHD), and hypertrophic cardiomyopathy (HCM) groups. T1_native = native T1, T1_post = postcontrast T1.

FASTR-SCANN produced successful myocardial segmentation in 93.8% (225 of 240) of slices before shape analysis and 95.4% (229 of 240) of slices after shape analysis. Automatic versus expert DSCs were similar to interobserver DSCs for all regions of interest in both T1_native and T1_post series (Table 1). Furthermore, automatic versus expert DSCs remained similar to interobserver DSCs across abnormality groups (Table 2). The Euclidean distances between automatic and expert RVIP placements (automatic vs expert 1: 5.22 pixels ± 3.88; automatic vs expert 2: 5.44 pixels ± 3.41) were higher than interobserver distances (3.07 pixels ± 3.03; P < .001).

Table 1:

Dice Similarity Coefficients for Automated Segmentation with Use of FASTR-SCANN with Internal Test Dataset

Table 2:

Myocardial Dice Similarity Coefficients across Clinical Cardiac Abnormality Categories with Use of FASTR-SCANN with Internal Test Dataset

We found no evidence of differences between automatic versus mean expert T1_native for per-patient (1053 msec ± 71 vs 1058 msec ± 75; ΔT1: 5.21 msec [95% CI: -0.95, 11.36]; P = .10) and per-slice (1054 msec ± 75 vs 1058 msec ± 81; ΔT1: 4.33 msec [95% CI: -0.04, 8.70]; P = .052) analysis; however, per-segment analysis was different (1051 msec ± 100 vs 1055 msec ± 111; ΔT1: 2.72 msec [95% CI: 0.50, 4.58]; P = .005). Furthermore, we found no differences between automatic versus mean expert T1_post per patient (407 msec ± 42 vs 407 msec ± 41; ΔT1: 0.65 msec [95% CI: -3.21, 1.90]; P = .61), per slice (407 msec ± 48 vs 405 msec ± 47; ΔT1: -1.25 msec [95% CI: -3.29, 0.79]; P = .23), and per segment (407 msec ± 57 vs 406 msec ± 56; ΔT1: 0.33 msec [95% CI: -0.51, 1.20]; P = .94). Automatic per-segment ECVs were lower than mean expert measurements (24.85% ± 7.85 vs 25.18% ± 8.25; ΔECV: 0.12% [95% CI: 0.03, 0.22]; P = .007). Nonetheless, differences in per-segment T1_native and ECV, relative to mean expert measurements, were less than 1% of expert measured values.

There was strong agreement between automatic and expert segmental measurements, as shown in correlations and Bland-Altman analysis (Fig 4), which was similar to interobserver agreement. Representative failure cases were noted and displayed for three slices with thin myocardial wall, artifacts, and dilated myocardium (Fig 5).

Figure 4:

Correlations and Bland-Altman analysis show excellent association and agreement between automatic (Auto) versus average (Avg.) experts and interobserver variability for per-segment native T1, postcontrast T1, and extracellular volume (ECV). Solid and dashed lines in Bland-Altman plots show bias and 95% limits of agreement, respectively. Exp-1 = Expert-1, Exp-2 = Expert-2.

Figure 5:

Representative images show raw T1-weighted (T1w) images and T1 maps with overlaid final myocardial mask (green) and areas of uncertainty (red) for segmentation failure cases due to improper myocardial shape (black and white arrows) in (A) native T1 slice with thin myocardium wall in the anterolateral wall region, (B) postcontrast T1 slice with posterolateral wall blurring due to partial volume effect, and (C) postcontrast T1 slice with dilated myocardium.

External Testing with FASTR-SCANN

Evaluation on the external dataset showed good generalization of U-Net_SCANN. Successful segmentations were produced in 92.8% (682 of 735) of slices before shape adjustment and 95.6% (703 of 735) of slices after shape adjustment, with mean myocardial DSC of 0.80 ± 0.09. Per-slice myocardial T1 measured with use of our automatic method was lower than expert measurements (1120 msec ± 76 vs 1130 msec ± 85; ΔT1: 9.6 msec [95% CI: 8.0, 11.2]; P < .001). Good t between automatic versus manual contours and T1 values was observed (Fig 6). Segmentation results showed high DSC across slice locations despite spatial slice coverage varying between external and internal datasets (see Appendix E12 [supplement]).

Figure 6:

Fully automated segmental relaxometry method using synthetic contrast augmentation with neural networks (FASTR-SCANN) performance for external test dataset. (A) Correlation and Bland-Altman analyses showed good associations and agreement between automatic and expert T1 values. Solid and dashed lines in Bland-Altman plots show bias and 95% limits of agreement, respectively. (B) Representative segmentation results on T1 maps from two patients show good agreement between automatic (Auto) and expert myocardial left ventricular (LV) endocardial (red) and LV epicardial (green) contours.

Discussion

We developed and validated a deep learning method for fully automated segmentation of myocardial T1 maps and segmental T1 and ECV analysis. Important highlights of our FASTR-SCANN method were the following: (a) segmentation quality, T1, and ECV correlations were excellent and similar to interobserver differences; (b) use of MRI physics–based synthetic data optimized blood-myocardium contrast, enhanced data augmentation, and demonstrated robustness across sequence type, acquisition center, MRI vendor, and presence of a contrast agent; and (c) manual expert labeling of training data was efficient given that the network required labeling only a single TI per slice and leveraged synthetic images by means of data augmentation.

Several recent studies have proposed methods for automated analysis of T1 maps. Fahmy et al (10) proposed the use of a U-Net–based architecture for analysis of T1_native by performing segmentation of raw T1-weighted images. Zhu et al (11) leveraged transfer learning from a previously trained T1_native network to segment T2 maps and determine ECV. Farrag et al (12) used a U-Net model for direct segmentation of T1_native and T1_post maps; however, segmental analysis and ECV computation were not performed.

Relative to previous studies, the main differentiator investigated was the use of synthetic data based on T1 relaxation properties in network training and testing (U-Net_SCANN). Compared with two common methods (U-Net_RAW and U-Net_T1MAP), U-Net_SCANN produced the highest rate of successful myocardial segmentations, with a higher myocardial DSC than U-Net_T1MAP, possibly because of better blood-myocardium contrast (see Appendix E5 [supplement]). Although U-Net_RAW produced similar DSCs, it demonstrates a notable disadvantage arising from variable blood-myocardium contrast in raw T1-weighted images due to varying TIs across different sequence designs (eg, MOLLI, shortened MOLLI [or shMOLLI], and saturation recovery single-shot acquisition [or SASHA]) (6), which may limit its generalizability to data acquired with use of different T1 acquisition sequences. In contrast, U-Net_SCANN can be largely independent of T1 sequence and inversion schedule with use of the complete T1-recovery curve to generate synthetic training and testing images at desired TIs in the TI^opt range.

We did note statistically significant differences between automatic versus expert per-segment T1_native and ECV; however, differences were less than 1% of expert measurements. Previous evidence suggests that changes larger than 1% in T1_native, T1_post, and ECV enable detection of cardiac abnormality (23–25). Therefore, our differences are likely not clinically significant. Our method also showed strong performance on an external dataset acquired with use of a different MRI scanner, T1 sequence, and spatial coverage (five slices vs three slices). To our knowledge, this is the first study to show model generalization to external data across center, scanner, and sequence for automated cardiac T1 quantification.

We acknowledge several limitations. First, because of possible image registration errors during T1 reconstruction, there may be mismatch in transferring expert annotations from preselected T1-weighted images to synthetic images for training. Nonetheless, high network performance, which was similar to interobserver differences, indicated that error from this source was minimal. Second, analysis of T1 differences across the three network designs during cross-validation included only patients for whom all slices were successfully segmented by all networks; nonetheless, testing of U-Net_SCANN showed accurate T1 quantification across all patients in internal and external test datasets, relative to experts. Third, our method was tested only with data acquired at 1.5 T with MOLLI and slice interleaved T1 sequences; thus, findings cannot be extrapolated to other field strengths and sequences. Finally, we did not perform an additional registration step between T1_native and T1_post maps, which could help reduce any spatial mismatch between native and postcontrast segments to compute ECV. Nevertheless, strong ECV agreement with experts indicated that these errors were tolerable.

In conclusion, FASTR-SCANN leveraged synthetic contrast augmentation to produce excellent segmentation performance and robust T1 and ECV analysis for cardiac MRI performed across different abnormalities, centers, scanners, and T1 sequences. With future open-source data availability, our approach can be extended to study additional T1 sequences, MRI field strengths, and clinical utility in high-volume clinical settings, where our automated approach may be potentially appealing.