Deep learning–based automatic segmentation of cardiac substructures for lung cancers

Abstract

Purpose:

Accurate and comprehensive segmentation of cardiac substructures is crucial for minimizing the risk of radiation-induced heart disease in lung cancer radiotherapy. We sought to develop and validate deep learning–based auto-segmentation models for cardiac substructures.

Materials and Methods:

Nineteen cardiac substructures (whole heart, 4 heart chambers, 6 great vessels, 4 valves, and 4 coronary arteries) in 100 patients treated for non-small cell lung cancer were manually delineated by two radiation oncologists. The valves and coronary arteries were delineated as planning risk volumes. An nnU-Net auto-segmentation model was trained, validated, and tested on this dataset with a split ratio of 75:5:20. The auto-segmented contours were evaluated by comparing them with manually drawn contours in terms of Dice similarity coefficient (DSC) and dose metrics extracted from clinical plans. An independent dataset of 42 patients was used for subjective evaluation of the auto-segmentation model by 4 physicians.

Results:

The average DSCs were 0.95 (+/− 0.01) for the whole heart, 0.91 (+/− 0.02) for 4 chambers, 0.86 (+/− 0.09) for 6 great vessels, 0.81 (+/− 0.09) for 4 valves, and 0.60 (+/− 0.14) for 4 coronary arteries. The average absolute errors in mean/max doses to all substructures were 1.04 (+/− 1.99) Gy and 2.20 (+/− 4.37) Gy. The subjective evaluation revealed that 94% of the auto-segmented contours were clinically acceptable.

Conclusion:

We demonstrated the effectiveness of our nnU-Net model for delineating cardiac substructures, including coronary arteries. Our results indicate that this model has promise for studies regarding radiation dose to cardiac substructures.

Introduction

Lung cancer is one of the most prevalent types of cancer worldwide, and radiotherapy (RT) is a commonly used treatment option [1]. However, radiation-induced heart disease (RIHD) poses a significant risk to patients undergoing RT for thoracic cancers [2–6], with up to 33 % of lung cancer patients experiencing cardiac complications after RT [3]. The morbidity of RIHDs correlates with the radiation dose received by the heart during treatment [7–10]; e.g., the risk of valvular disease in irradiated patients is 9.2-fold higher than in nonirradiated patients [9]. In current clinical practice, dose-volume constraints are typically applied to the whole heart, considered as a single organ [11,12]. However, relying solely on whole-heart dose volume metrics may not always reliably predict the risk of specific RIHD [13,14]. Increasing evidence suggests that dose to cardiac substructures provides superior predictive power for RIHD compared with whole-heart dose [15–22]. Notably, dose to the left atrium [16], left ventricle [17,19], left anterior descending coronary artery [15,18,21,22], and heart valves [20] has been implicated in the development of RIHDs after RT. However, understanding of the radiosensitivity of cardiac substructures and the relationship between cardiac substructure dose and RIHDs remains limited because of the lack of efficient and accurate means of cardiac substructure segmentation.

Manual delineation of all cardiac substructures is a labor- and time-intensive process that is particularly challenging when non-contrast simulation CT images are used, as they have low substructure contrast and are prone to cardiac motion artifacts. In addition, manual segmentation is often associated with significant intra- and inter-observer variability [23–25]. To address these limitations, multi-atlas–based methods have been extensively investigated for automatic segmentation of cardiac substructures [26–31]. However, these approaches have shown limited accuracy, particularly for coronary arteries [27,29]. In recent years, deep learning models have shown remarkable performance in medical image segmentation, and several deep learning approaches have been proposed to segment cardiac substructures [32–37]. However, not all models include the segmentation of all coronary arteries [32–35]. Monin et al. proposed a mutual enhancing learning-based architecture to delineate detailed cardiac substructures [36]. However, their model was trained on a mixture of contrast and non-contrast CT images, and their performance on solely non-contrast CT images is unknown. The availability of tools capable of accurately segmenting detailed cardiac substructures is still limited. The purpose of this study was to develop and validate a deep learning–based segmentation model to delineate detailed cardiac substructures for further studies regarding radiation dose to cardiac substructures.

Materials and methods

Dataset

This study was approved by our institutional review board (protocol #2021–0071). The dataset used for model development and evaluation comprised 142 non-contrast CT scans from 142 patients with non-small cell lung cancer (NSCLC) treated at our institution. The deep-learning models were trained, validated, and tested on 100 CT scans from 100 patients treated from 2009 through 2014. Those CT scans were averaged 4DCT scans obtained from RT treatment planning, with an image resolution of 0.98 mm × 0.98 mm × 2.5 mm. During the manual delineation, two radiation oncologists were assigned to independently contour a comprehensive set of 19 cardiac substructures. Subsequently, they conducted a cross-validation of each delineation to ensure mutual approval, with challenging cases reviewed by two cardiologists. The delineated cardiac substructures were the whole heart (WH), left atrium (LA), right atrium (RA), left ventricle (LV), right ventricle (RV), ascending aorta (AA), descending aorta (DA), pulmonary artery (PA), pulmonary vein (PV), superior vena cava (SVC), inferior vena cava (IVC), aortic valve (AVV), mitral valve (MVV), pulmonary valve (PVV), tricuspid valve (TVV), left anterior descending coronary artery (LAD), left main coronary artery (LMCA), left circumflex coronary artery (LCX), and right coronary artery (RCA). Substructures were delineated according to institutional guidelines that were based on both published findings [38] and the expertise of MD Anderson cardiologists; details are given in Appendix A. The valves and coronary arteries were delineated as planning risk volumes (PRVs), that is, with volumes that were larger than the actual anatomy, to account for both the cardiac motion effect and the contouring uncertainty. Notably, contrast CT scans were used during the manual delineation to help accurately delineate the coronary arteries. These contrast CT scans were acquired during diagnosis, utilizing a 64-slice CT scanner, with a slice thickness of 2.5 mm and acquisition at deep inspiration breath-hold. A rough rigid registration was applied to align the paired contrast CT and non-contrast CT images. Although the registrations were not entirely precise, the contrast CT scans offered crucial information regarding the direction and extension of the coronary arteries. It is important to emphasize that the contrast CT scans were not involved in model training. We also collected an independent dataset consisting of 42 averaged 4DCT scans from another 42 NSCLC patients treated from 2022 through 2023 for subjective evaluation. The image resolution in this independent validation dataset was 1.17 mm × 1.17 mm × 2.5 mm.

Network architecture

We trained two neural networks to perform auto-segmentation of all 19 cardiac substructures: a commercial two-stage 3D U-Net model and an in-house 3D nnU-Net model (Fig. 1). The commercial 3D U-Net model was trained as a baseline for the deep learning–based model. The patient cohort was randomly divided into three subsets: 75 for training, 5 for validation, and 20 for testing. The cardiac substructures were considered in 4 groups: (1) whole heart, (2) chambers and great vessels, (3) valves, and (4) coronary arteries. The commercial 3D U-Net model consisted of a coarse localization stage, which determined the locations and receptive fields of all cardiac substructures; and a refined segmentation stage, which focused on local individual substructure segmentation. We trained four 3D U-Net localization models for each substructure group, using a coarse isotropic resolution of 3 mm. In the subsequent refined segmentation stage, a 3D U-Net model was trained independently for each individual substructure, with finer resolution ranging from 1 mm to 1.5 mm. For the in-house 3D nnU-Net model, we trained four multi-class full resolution models for each substructure group and combined them into a unified model. The nnU-Net architecture adapted automatically during the pre-processing, data augmentation, and post-processing stages [39]. Details of the network architectures and training strategies are given in Appendix B.

Fig. 1.

Schematic overview of the two-stage 3D U-Net and 3D nnU-Net models. The 3D U-Net model consists of 4 coarse localization networks that derive coarse contours of individual substructures. Each substructure is then auto-segmented by a subsequent refined segmentation network. The 3D nnU-Net model consists of 4 multi-class models that are combined to segment all 19 substructures.

Quantitative evaluation

As a baseline for comparison, we used a multi-atlas auto-segmentation method previously developed by our team [27] (valve segmentation was not included in that method). The auto-segmented contours were quantitatively assessed by comparing them with the manual contours with both geometric and dosimetric metrics. Auto-segmented contours generated by the 3D U-Net, 3D nnU-Net, and multi-atlas method were compared with the manual contours by using the Dice similarity coefficient (DSC), surface Dice similarity coefficient (SDSC; tolerance 1 mm) [40], 95th Haunsdorff distance (HD95), mean surface distance (MSD), volume difference, and volume ratio. Definitions of these metrics are given in Appendix C. The clinical treatment plan was used to create dose-volume histograms for both the auto-segmented contours and manual contours. We compared the mean dose and maximum dose delivered to each substructure defined by the auto-segmented contours and manual contours. Wilcoxon signed-rank tests were used for each cardiac substructure and for the four substructure groups (chambers, great vessels, valves, coronary arteries) in terms of DSC, SDSC, HD95, MSD, absolute mean dose error, and absolute maximum dose error. P values below 0.05 were considered to indicate statistically significant differences in geometric or dosimetric accuracy.

Subjective evaluation

The clinical implications of auto contours may not necessarily be determined by quantitative metrics. To verify their clinical acceptability, we also used subjective evaluation of the auto-segmentation results. The nnU-Net model was applied to an independent cohort of 42 NSCLC patients to delineate the 19 cardiac substructures. Four physicians (2 radiation oncologists and 2 cardiologists) scored the auto contours on a 5-point Likert scale, with a score of 5 meaning use-as-is; 4 indicating minor edits that are not necessary; 3 as minor edits needed; 2 as major edits needed; and 1 as unusable. Scores of 4 or higher were considered clinically acceptable. Details of the scoring are shown in Appendix D.

Results

Comparisons between the manual segmentation by the physicians and the auto-segmentation by the 3D U-Net, 3D nnU-Net, and multi-atlas methods are illustrated in Fig. 2; the DSC, SDSC, HD95, MSD, volume difference, and volume ratio results for all three methods are shown in Table 1. Appendix E presents further cases of the segmentation results of nnU-Net compared to an open-source package, PlatiPy [41]. Detailed statistical test results between nnU-Net, U-Net, and multi-atlas segmentation methods are provided in Appendix F. Both deep learning models outperformed the multi-atlas method for all substructures based on the four metrics (DSC, SDSC, HD95, MSD), except for PV in terms of HD95 and MSD. The apparent improvement of the nnU-Net model over the U-Net model was not significant in individual statistical tests of most large-volume structures, such as WH, LA, RA, LV, RV, AA, DA, PA, SVC, PVV, and TVV (p > 0.05 for DSC). However, the nnU-Net model slightly outperformed the U-Net model in auto-segmenting the great vessels and valves (p < 0.01 for all the 4 metrics) through its superior performance in segmenting the PV, IVC, AVV, and MVV. Specifically, the nnU-Net model yielded superior results for the 4 coronary arteries (p < 0.005 for all the 4 metrics). Notably, the U-Net model failed to contour the LMCA in 9 of 20 cases, as the LMCA appeared in only a limited number of CT slices. The reported LMCA metrics of the U-Net model were calculated based on the successful 11 cases. In contrast, the nnU-Net model was more robust and identified the LMCA in the correct position in all patients. In general, a consistent pattern is noted wherein the U-Net model underestimates volumes across 17 out of 19 substructures, while the nnU-Net model demonstrates a tendency to overestimate volumes in 16 out of the 19 substructures.

Fig. 2.

Comparison of cardiac substructure contours derived from (a) manual delineation, (b) auto contours generated by 3D U-Net, (c) auto contours generated by nnU-Net, and (d) auto contours generated by the multi-atlas method.

Table 1.

Average Dice similarity coefficient (DSC), surface Dice similarity coefficient (SDSC, tolerance 1 mm), 95th Haunsdorff distance (HD95), mean surface distance (MSD), volume difference, and volume ratio of auto-segmented contours generated by the U-Net, nnU-Net, and multi-atlas methods compared with manual contours for 20 patients in the model-testing group. Values in parentheses are standard deviations; bolding indicates the best results of the three methods for DSC, SDSC, HD95 and MSD.

	DSC			SDSC			HD95, mm			MSD, mm			Volume difference, cm3			Volume ratio
	U-Net	nnU-Net	Multi-atlas	U-Net	nnU-Net	Multi-atlas	U-Net	nnU-Net	Multi-atlas	U-Net	nnU-Net	Multi-atlas	U-Net	nnU-Net	Multi-atlas	U-Net	nnU-Net	Multi-atlas
WH	0.95 (0.01)	0.95 (0.01)	0.89 (0.03)	0.77 (0.13)	0.77 (0.13)	0.46 (0.12)	4.9 (2.4)	4.1 (1.2)	11.1 (4.3)	1.5 (0.5)	1.3 (0.5)	3.7 (1.3)	− 6.96 (42.37)	12.15 (42.87)	71.14 (97.89)	1.00 (0.06)	1.02 (0.06)	1.13 (0.12)
Chamber Average	0.91 (0.04)	0.91 (0.02)	0.81 (0.09)	0.76 (0.14)	0.77 (0.11)	0.54 (0.15)	4.9 (3.4)	4.4 (2.8)	11.3 (6.3)	1.3 (0.8)	1.1 (0.5)	3.0 (1.9)	10.43 (18.14)	9.11 (8.77)	21.04 (23.32)	0.97 (0.09)	1.05 (0.07)	1.10 (0.19)
LA	0.91 (0.03)	0.91 (0.02)	0.82 (0.04)	0.78 (0.16)	0.79 (0.11)	0.59 (0.12)	4.4 (3.2)	3.4 (0.8)	9.6 (4.0)	1.1 (0.6)	1.0 (0.3)	2.3 (0.7)	− 4.24 (9.16)	2.78 (4.95)	− 2.06 (16.19)	0.97 (0.08)	1.06 (0.07)	1.04 (0.19)
RA	0.89 (0.05)	0.90 (0.02)	0.78 (0.12)	0.77 (0.14)	0.75 (0.12)	0.53 (0.16)	5.2 (3.4)	4.4 (1.5)	12.9 (8.0)	1.3 (0.9)	1.2 (0.4)	3.3 (2.7)	− 6.12 (24.58)	3.56 (10.79)	− 4.81 (35.58)	0.96 (0.11)	1.06 (0.09)	1.04 (0.23)
LV	0.93 (0.03)	0.93 (0.01)	0.87 (0.04)	0.79 (0.11)	0.80 (0.10)	0.59 (0.12)	4.6 (4.1)	4.1 (2.3)	10.6 (5.6)	1.2 (0.8)	1.1 (0.5)	2.5 (1.1)	− 2.89 (28.21)	9.60 (15.29)	10.65 (33.48)	0.99 (0.08)	1.05 (0.05)	1.09 (0.13)
RV	0.89 (0.04)	0.91 (0.03)	0.78 (0.09)	0.70 (0.12)	0.74 (0.11)	0.44 (0.14)	5.3 (2.6)	5.6 (4.5)	12.3 (6.5)	1.5 (0.7)	1.3 (0.5)	3.7 (2.2)	− 8.96 (11.11)	4.21 (11.61)	30.24 (15.51)	0.94 (0.08)	1.03 (0.08)	1.23 (0.12)
Great Vessel Average	0.84 (0.12)	0.86 (0.09)	0.74 (0.14)	0.77 (0.17)	0.80 (0.16)	0.56 (0.17)	7.8 (9.2)	6.0 (6.0)	13.4 (8.6)	1.5 (1.7)	1.1 (0.8)	2.9 (2.1)	5.07 (6.28)	3.71 (3.36)	11.24 (12.95)	0.88 (0.16)	1.07 (0.16)	1.16 (0.43)
AA	0.93 (0.02)	0.94 (0.01)	0.81 (0.06)	0.88 (0.07)	0.88 (0.12)	0.53 (0.12)	3.4 (1.0)	2.6 (0.5)	14.6 (8.0)	0.7 (0.2)	0.6 (0.2)	3.0 (1.5)	− 6.98 (5.63)	4.71 (3.90)	− 27.89 (19.01)	0.94 (0.04)	1.04 (0.03)	0.78 (0.10)
DA	0.93 (0.03)	0.93 (0.02)	0.82 (0.05)	0.87 (0.10)	0.84 (0.12)	0.58 (0.11)	7.4 (7.5)	7.2 (5.0)	20.0 (11.3)	1.0 (0.8)	0.9 (0.4)	2.9 (1.3)	− 5.51 (6.71)	4.16 (4.46)	− 8.27 (10.75)	0.95 (0.06)	1.05 (0.06)	0.93 (0.09)
PA	0.87 (0.17)	0.92 (0.02)	0.83 (0.05)	0.87 (0.17)	0.91 (0.07)	0.71 (0.13)	6.5 (12.0)	3.1 (1.4)	9.5 (4.1)	1.6 (3.4)	0.7 (0.3)	1.9 (0.7)	− 3.54 (12.92)	3.80 (4.83)	8.95 (11.73)	0.95 (0.21)	1.06 (0.08)	1.17 (0.21)
PV	0.70 (0.08)	0.76 (0.06)	0.64 (0.08)	0.70 (0.14)	0.78 (0.13)	0.62 (0.12)	15.5 (14.3)	11.2 (10.7)	12.7 (7.5)	2.3 (1.7)	1.7 (1.0)	2.5 (1.0)	− 5.03 (3.43)	1.05 (3.44)	5.77 (7.02)	0.72 (0.18)	1.10 (0.25)	1.42 (0.45)
SVC	0.85 (0.05)	0.85 (0.08)	0.72 (0.14)	0.72 (0.13)	0.74 (0.14)	0.51 (0.16)	5.2 (2.9)	4.3 (2.2)	9.6 (4.8)	1.1 (0.4)	1.1 (0.5)	2.4 (1.2)	−1.84 (1.97)	1.63 (1.63)	5.34 (7.06)	0.91 (0.11)	1.09 (0.15)	1.42 (0.54)
IVC	0.76 (0.07)	0.79 (0.08)	0.60 (0.18)	0.59 (0.15)	0.65 (0.17)	0.40 (0.14)	8.9 (4.1)	8.0 (4.5)	14.3 (8.9)	2.1 (0.8)	1.9 (0.9)	4.5 (3.9)	− 2.83 (3.39)	0.28 (4.40)	3.15 (7.31)	0.85 (0.15)	1.05 (0.24)	1.25 (0.45)
Valve Average	0.77 (0.11)	0.81 (0.09)	—	0.67 (0.20)	0.73 (0.19)	—	5.4 (2.3)	4.4 (1.8)	—	1.9 (0.9)	1.6 (0.8)	—	4.77 (4.86)	3.63 (3.10)	—	1.00 (0.39)	1.11 (0.32)	—
AVV	0.82 (0.06)	0.86 (0.05)	—	0.74 (0.19)	0.83 (0.14)	—	4.3 (1.9)	3.4 (1.1)	—	1.6 (0.7)	1.2 (0.6)	—	− 3.03 (5.02)	− 0.67 (3.43)	—	0.89 (0.18)	1.01 (0.16)	—
MVV	0.79 (0.07)	0.83 (0.05)	—	0.62 (0.16)	0.69 (0.18)	—	5.9 (2.3)	4.7 (1.8)	—	2.0 (0.8)	1.5 (0.8)	—	− 4.42 (7.47)	− 0.10 (5.09)	—	0.89 (0.18)	1.04 (0.16)	—
PVV	0.68 (0.15)	0.74 (0.10)	—	0.69 (0.23)	0.77 (0.16)	—	5.6 (2.5)	4.3 (1.5)	—	2.0 (1.0)	1.5 (0.7)	—	0.22 (3.02)	1.27 (2.40)	—	1.21 (0.62)	1.31 (0.48)	—
TVV	0.78 (0.07)	0.78 (0.08)	—	0.61 (0.19)	0.62 (0.20)	—	5.9 (2.0)	5.4 (2.0)	—	2.0 (0.8)	2.0 (0.8)	—	− 2.22 (7.88)	0.53 (6.74)	—	1.00 (0.28)	1.10 (0.25)	—
Coronary Artery Average	0.51 (0.17)	0.60 (0.14)	0.23 (0.17)	0.58 (0.19)	0.68 (0.17)	0.32 (0.20)	9.3 (5.7)	7.3 (4.8)	20.8 (13.6)	2.3 (1.5)	1.8 (1.2)	6.7 (5.9)	1.82 (1.84)	1.10 (1.16)	2.46 (2.27)	0.75 (0.37)	1.07 (0.44)	0.71 (0.64)
LAD	0.57 (0.15)	0.65 (0.10)	0.23 (0.16)	0.69 (0.15)	0.77 (0.12)	0.32 (0.19)	9.2 (4.7)	8.4 (5.4)	26.3 (13.8)	1.7 (0.7)	1.4 (0.7)	7.3 (4.9)	−1.21 (1.96)	0.27 (2.09)	− 2.63 (2.21)	0.84 (0.50)	1.19 (0.66)	0.51 (0.37)
LMCA	0.57 (0.14)	0.58 (0.15)	0.29 (0.16)	0.66 (0.17)	0.75 (0.13)	0.42 (0.17)	5.9 (2.2)	5.4 (2.2)	11.8 (3.4)	1.5 (0.8)	1.5 (0.8)	4.0 (2.8)	− 0.29 (0.31)	0.01 (0.46)	− 0.05 (0.69)	0.79 (0.23)	1.07 (0.35)	1.07 (0.71)
LCX	0.42 (0.18)	0.56 (0.13)	0.22 (0.19)	0.50 (0.19)	0.59 (0.17)	0.29 (0.20)	11.8 (6.7)	8.9 (6.1)	20.7 (12.6)	3.0 (1.8)	2.3 (1.7)	6.5 (4.7)	− 2.35 (2.77)	− 0.78 (1.94)	− 3.50 (2.44)	0.65 (0.35)	0.95 (0.36)	0.36 (0.28)
RCA	0.52 (0.14)	0.61 (0.17)	0.15 (0.16)	0.52 (0.16)	0.62 (0.18)	0.19 (0.16)	8.7 (5.6)	6.7 (4.0)	27.3 (16.7)	2.6 (1.5)	2.0 (1.3)	10.9 (9.4)	−1.53 (1.76)	0.24 (1.12)	− 0.79 (4.28)	0.72 (0.28)	1.07 (0.23)	0.94 (0.84)
All	0.79 (0.18)	0.81 (0.15)	0.66 (0.26)	0.71 (0.19)	0.75 (0.16)	0.49 (0.19)	6.7 (6.3)	5.5 (4.5)	14.3 (9.9)	1.7 (1.3)	1.4 (0.9)	3.8 (3.6)	7.11 (13.37)	6.10 (10.70)	18.52 (33.29)	0.91 (0.28)	1.07 (0.27)	1.04 (0.46)

Abbreviations: WH, whole heart; LA, left atrium; RA, right atrium; LV, left ventricle; AA, ascending aorta; DA, descending aorta; PA, pulmonary artery; PV, pulmonary vein; SVC, superior vena cava; IVC, inferior vena cava; AVV, aortic valve; MVV, mitral valve; PVV, pulmonary valve; TVV, tricuspid valve; LAD, left anterior descending coronary artery; LMCA, left main coronary artery; LCX, left circumflex coronary artery; and RCA, right coronary artery.

Table 2 shows the average absolute errors in mean dose and maximum dose that were computed on the auto-segmented contours, compared to those that were computed based on manual contours. Detailed statistical test results are shown in Appendix G. The 20 testing patients underwent either intensity-modulated radiation therapy (IMRT) or passive scatter proton therapy (PSPT) treatment, with 2 Gy per fraction and prescriptions of 66 Gy, 70 Gy, or 74 Gy. The mean heart dose ranged from 0.01 Gy to 19.30 Gy with a median value of 6.01 Gy. The tumor volumes ranged from 2.0 cm³ to 534.0 cm³, with a median volume of 140.3 cm³. The internal target volume (ITV) was delineated by the physicians to account for motion, with a 5 mm margin from ITV to the planning target volume (PTV). No specific measures for cardiac sparing were considered during the treatments. The overall average absolute errors in mean dose for all auto-segmented substructures were 1.22 (+/− 2.40) Gy for U-Net, 1.04 (+/− 1.99) Gy for nnU-Net, and 2.70 (+/− 4.55) Gy for the multi-atlas method. The corresponding overall average absolute errors in maximum dose were 3.16 (+/− 6.40) Gy, 2.20 (+/− 4.37) Gy, 5.40 (+/− 9.77) Gy. Both deep learning methods demonstrated significantly more consistent dosimetric estimates than the multi-atlas method for all substructures (p < 0.001). The nnU-Net model exhibited more consistent dose results than the U-Net model, with smaller standard deviations in most of the substructures, except for WH, MVV, LAD in terms of mean dose and RA, RV, AA, DA, TVV in terms of maximum dose. Wilcoxon signed-rank tests comparing the nnU-Net vs U-Net models for all substructures resulted in a p value of 0.0015 for errors in mean dose and 0.0001 for errors in maximum dose.

Table 2.

Dose comparisons between manual and auto contours. The numbers listed are average absolute differences between the mean/maximum doses extracted from auto-segmented contours and manual contours for individual substructure. Standard deviations are shown in brackets. The least dosimetric differences of the three methods are bolded.

	Average absolute errors in mean dose, Gy			Average absolute errors in max dose, Gy
	U-Net	nnU-Net	Multi-atlas	U-Net	nnU-Net	Multi-atlas
WH	0.30 (0.27)	0.32 (0.27)	1.42 (1.38)	2.89 (4.60)	2.08 (3.45)	7.30 (11.69)
Chambers	0.50 (0.95)	0.41 (0.80)	1.08 (1.83)	2.20 (4.58)	1.68 (3.46)	4.82 (8.22)
LA	0.86 (1.04)	0.62 (0.71)	2.23 (2.13)	3.13 (7.33)	1.81 (4.57)	3.40 (7.64)
RA	0.90 (1.35)	0.84 (1.25)	1.65 (2.24)	0.63 (0.84)	0.95 (1.09)	5.66 (11.93)
LV	0.07 (0.13)	0.05 (0.12)	0.20 (0.39)	3.31 (4.43)	2.15 (3.56)	5.05 (6.11)
RV	0.17 (0.34)	0.13 (0.27)	0.24 (0.76)	1.72 (2.21)	1.80 (3.51)	5.17 (5.42)
Great vessels	1.70 (2.73)	1.42 (2.09)	3.10 (3.35)	1.72 (6.11)	0.97 (2.59)	2.94 (7.64)
AA	0.90 (0.77)	0.44 (0.35)	3.80 (2.77)	0.25 (0.33)	0.33 (0.86)	1.43 (4.06)
DA	0.79 (2.03)	0.58 (0.79)	2.74 (3.77)	1.08 (2.01)	1.12 (1.60)	3.78 (4.84)
PA	1.77 (1.29)	1.43 (1.40)	3.59 (2.75)	3.46 (12.41)	0.62 (1.60)	1.41 (4.52)
PV	4.21 (4.11)	4.19 (3.07)	5.28 (3.83)	1.51 (2.84)	1.16 (2.61)	3.98 (10.69)
SVC	1.32 (1.62)	1.22 (1.38)	2.25 (3.15)	0.26 (0.36)	0.19 (0.24)	0.75 (1.08)
IVC	1.18 (3.13)	0.65 (1.51)	0.95 (1.50)	3.87 (7.44)	2.39 (4.91)	6.32 (12.30)
Valves	1.00 (1.73)	0.79 (1.33)	—	3.95 (6.02)	2.81 (5.61)	—
AVV	1.08 (1.28)	0.94 (1.47)	—	3.00 (3.47)	1.32 (1.47)	—
MVV	0.39 (0.54)	0.51 (1.09)	—	5.21 (8.47)	4.18 (7.66)	—
PVV	2.22 (2.57)	1.51 (1.52)	—	6.88 (6.18)	4.73 (7.12)	—
TVV	0.32 (1.02)	0.21 (0.67)	—	0.72 (1.23)	1.00 (1.79)	—
Coronary Arteries	1.79 (3.45)	1.54 (2.96)	4.40 (7.78)	5.93 (8.32)	4.01 (5.30)	10.41 (12.43)
LAD	0.96 (1.97)	1.39 (3.16)	2.45 (6.22)	6.97 (7.83)	6.55 (7.58)	10.20 (9.74)
LMCA	2.46 (3.30)	1.95 (2.33)	5.07 (5.91)	5.35 (6.87)	2.50 (2.88)	10.87 (15.58)
LCX	2.41 (3.98)	1.74 (2.70)	4.78 (7.14)	8.30 (10.93)	5.59 (4.79)	10.43 (7.60)
RCA	1.61 (3.91)	1.07 (3.44)	6.11 (12.23)	2.86 (4.84)	1.46 (2.42)	9.96 (15.73)
In total	1.22 (2.40)	1.04 (1.99)	2.70 (4.55)	3.16 (6.40)	2.20 (4.37)	5.40 (9.77)

Abbreviations: WH, whole heart; LA, left atrium; RA, right atrium; LV, left ventricle; AA, ascending aorta; DA, descending aorta; PA, pulmonary artery; PV, pulmonary vein; SVC, superior vena cava; IVC, inferior vena cava; AVV, aortic valve; MVV, mitral valve; PVV, pulmonary valve; TVV, tricuspid valve; LAD, left anterior descending coronary artery; LMCA, left main coronary artery; LCX, left circumflex coronary artery; and RCA, right coronary artery.

Fig. 3 shows the distribution of subjective evaluation scores of each substructure as assessed by four physicians. Table 3 summarizes the clinical acceptance rates (i.e., scores of ≥ 4) for each substructure. The overall clinical acceptance rates for all substructures were determined by the four physicians as 97 %, 93.4 %, 92.0 %, and 93.7 %, respectively. Among all the substructures, the lowest average acceptance rate was the LAD (85.1 %), followed by the LMCA (86.3 %) and the RCA (86.9 %). Other than the PV, LAD, LMCA, LCX, and RCA, the average acceptance rate for all other structures was above 90 %.

Fig. 3.

The distribution of subjective evaluation scores from 4 physicians for automatically segmented contours generated by nnU-Net. In the upper panel, each bar represents the distribution of scores by one physician. The lower panel shows the average distribution among the 4 physicians. Dark green indicates a score of 5 (usable as is); 4, needs minor but not mandatory edits; 3, needs minor edits; 2, needs major edits; and 1, unusable. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

Table 3.

Summary of clinical acceptance rate, in %, for auto-segmented contours generated by nnU-Net for 42 patients evaluated by 4 physicians.

Physician ID	WH	LA	RA	LV	RV	AA	DA	PA	PV	SVC	IVC	AW	MW	PW	TW	LAD	LMCA	LCX	RCA	Overall
1	100	100.0	100.0	100.0	100.0	100.0	100.0	100.0	100.0	100.0	97.6	100.0	92.9	92.9	100.0	85.7	90.5	97.6	85.7	97.0
2	97.6	97.6	100.0	97.6	95.2	100.0	100.0	100.0	100.0	100.0	97.6	100.0	90.5	100.0	81.0	92.9	76.2	69.0	78.6	93.4
3	71.4	100.0	97.6	95.2	95.2	100.0	100.0	97.6	83.3	97.6	97.6	92.9	90.5	97.6	90.5	73.8	83.3	92.9	90.5	92.0
4	92.9	100.0	100.0	100.0	100.0	100.0	100.0	97.6	71.4	100.0	100.0	76.2	97.6	83.3	95.2	88.1	95.2	90.5	92.9	93.7
Mean	90.5	99.4	99.4	98.2	97.6	100.0	100.0	98.8	88.7	99.4	98.2	92.3	92.9	93.5	91.7	85.1	86.3	87.5	86.9	94.0

“Clinically acceptable” is defined as a score of > 4, where 5 indicates usable as is; 4, needs minor but not mandatory edits; 3, needs minor edits; 2, needs major edits; and 1, unusable.

Abbreviations: WH, whole heart; LA, left atrium; RA, right atrium; LV, left ventricle; AA, ascending aorta; DA, descending aorta; PA, pulmonary artery; PV, pulmonary vein; SVC, superior vena cava; IVC, inferior vena cava; AVV, aortic valve; MVV, mitral valve; PVV, pulmonary valve; TVV, tricuspid valve; LAD, left anterior descending coronary artery; LMCA, left main coronary artery; LCX, left circumflex coronary artery; and RCA, right coronary artery.

Discussion

The purpose of this study was to develop a deep learning–based automatic segmentation method for cardiac substructures and assess its clinical acceptability. Accurate delineation of cardiac substructures from non-contrast planning CT scans is challenging because of low-contrast anatomic boundaries and motion-related blurring. Recent developments in deep learning have made it possible to accurately delineate those substructures from non-contrast CT scans. Our nnU-Net model exhibited superior results over similar previously published studies. For instance, the average DSC of chamber segmentation (LA, RA, LV, RV) in our study was 0.91 (+/− 0.02), whereas Morris et al. reported 0.88 (+/− 0.03) [34], Jin et al. reported 0.79 (+/− 0.07) [32], and Chin et al. reported a median DSC of 0.75–0.87 [37]. The MSD of the LAD in our study was 1.4 (+/− 0.7) mm, in comparison with the corresponding reported values of 1.9 (+/− 0.9) mm, 4.1 (+/− 1.7) mm (valves and LAD), and 8.2 mm (median MSD), respectively. In our manual contouring, all cardiac valves and coronary arteries were delineated as planning risk volumes, that is, with volumes that were larger than the actual anatomy, to account for both the cardiac motion effect and the contouring uncertainty. While the implementation of PRVs may potentially enhance the geometric evaluation metrics of valves and coronary arteries, these results offer evidence that the nnU-Net model has great potential for accurately segmenting cardiac substructures.

In our dosimetric evaluations, the dosimetric parameters were quite consistent between the auto-segmented contours and manual contours when the nnU-Net model was used, with the average absolute error in mean dose for all substructures being 1.04 (+/− 1.99) Gy. Notably, the dosimetric difference between auto-segmented contours and manual contours depends on not only the quality of the auto contours but also on the tumor location and dose distribution. Small structures located within steep dose gradients can exhibit a wide range of dose differences. Specifically, auto-segmented coronary artery contours demonstrated considerable dose errors in cases where the tumor was in close proximity to the coronary arteries. For example, in the case of an auto-segmented LAD (DSC = 0.74, SDSC = 0.82, HD95 = 4.8 mm, and MSD = 1.07 mm), the estimated maximum dose was 1.1 Gy. In contrast, the manual contour–based computation yielded a maximum dose of 19.9 Gy. A small segment at the initiation of this LAD, located in the dose region ranging from 3 Gy to 20 Gy, was inaccurately classified as the LCMA by the auto-segmentation (The dose distribution is displayed in Appendix H). Consequently, the mean dose attributed to the LMCA, as estimated by the auto-segmented contour, was 8.5 Gy lower than the manual contours. Therefore, if the treatment target is close to any coronary artery, the auto-segmented coronary artery contour should be carefully reviewed for accuracy to reduce dose discrepancies and avoid violating dosimetric constraints.

In our study, 4 attending physicians (2 radiation oncologists and 2 cardiologists) assessed the clinical acceptance of the auto-segmented contours created by the nnU-Net model from an independent dataset of 42 patients. Of all 19 cardiac substructures, on average, 94 % of the auto-segmented contours were clinically acceptable (scores of ≥ 4) without requiring editing. Among those contours, 86.5 % of the coronary arteries were considered clinically acceptable. Coronary arteries are the most difficult substructures to delineate because of their low contrast, small size, and irregular shape. Cardiac motion can also blur their location relative to nearby other substructures such as the heart chambers, making them more difficult to delineate. We successfully demonstrated the efficiency of the nnU-Net model for auto-contouring these challenging structures.

The results of this subjective evaluation highlight significant variabilities among observers, particularly in assessments of the whole heart, pulmonary veins, and coronary arteries. Some minor auto-contouring discrepancies were observed at the cranial and caudal borders of the whole heart, and in the proximal segments of the pulmonary veins. Although errors such as these are unlikely to have clinical significance, they introduced variability in the subjective scores due to individual physician preferences. In the case of coronary arteries, several factors contributed to the variation in acceptance rates, including the challenges posed by their low contrast, small and tortuous structure, inter-individual anatomic variations, and considerable changes on different CT slices.

Comprehensive datasets of cardiac substructures are not readily available, as they are not routinely delineated in current clinical practice. The difficulty in delineating coronary arteries in particular makes curating coronary artery contours difficult. In this study, we curated an institutional dataset consisting of 100 NSCLC patients with 19 cardiac substructures that were delineated according to our institutional guidelines. The manual contours were verified by both radiation oncologists and cardiologists, and contrast CT was used to cross-check the manual contours delineated from the non-contrast CT scans. Both the valves and the coronary arteries were contoured as planning risk volumes to account for the effects of cardiac motion and contouring uncertainty from low structure contrast. This dataset is a valuable resource for training deep learning models to automatically delineate cardiac substructures. In the future, we will consider making this dataset available through public dataset platform such as The Cancer Imaging Archive [42].

The favorable outcomes obtained from our comprehensive validation process strongly encourage the inclusion of our nnU-Net model into further research application, as it offers time and resource savings. Moreover, interest is increasing in investigating the role of cardiac substructures in cardiotoxicity after RT. However, studies focusing on small substructures such as coronary arteries or valves have been limited by the laborious process of manual contour delineation. The development of an accurate auto-segmentation model that comprehensively delineates cardiac substructures holds the potential to facilitate prospective cardiotoxicity studies on large-scale datasets to better understand the relationship between cardiac events and substructure dose.

One limitation of the current study is that the subjective evaluation lacks validation from external individuals. The physicians who participated in the subjective evaluation may have a preference for the auto-segmented contours, given that they were also responsible for manually delineating the ground truth contours used in model training. Another limitation is that our model was exclusively trained and validated on patients with lung cancer, without cross-departmental or cross-institutional validation. Physicians in other departments, such as breast radiation oncology, may have distinct preferences regarding cardiac substructure contours. Validation within a different institution will also bolster the robustness and generalizability of the model to a broader clinical context.

In conclusion, we have developed and validated the clinical utility of an nnU-Net model for auto-contouring cardiac substructures, including coronary arteries. The model was comprehensively evaluated by using geometric metrics, dosimetric metrics, and subjective assessment. The results showed that 94 % of the auto-segmented contours were clinically acceptable based on thorough evaluations from 4 attending physicians. This tool is readily available for integration into our treatment planning system to facilitate dosimetry studies regarding cardiac substructures in lung cancer radiotherapy.