Convolutional neural networks for automatic tuber segmentation and quantification of tuber burden in tuberous sclerosis complex

Abstract

Objective:

To develop a fully automated algorithm for tuber segmentation and quantification of tuber volume that performs similarly to the gold-standard human neuroradiologist.

Methods:

We used brain magnetic resonance imaging (MRIs) from patients with tuberous sclerosis complex (TSC)to train and validate a convolutional neural network (CNN) which was evaluated on segmentation with the Dice-Sørensen similarity coefficient (DSSC) and on tuber burden quantification with Spearman correlation coefficient against neuroradiologist’s gold standard in the test set.

Results:

We collected 263 MRIs from 196 patients (57% males) with median (25^th percentile-75^th percentile) age of 4.3 (3.0–10.1) years: 176 MRIs in the train set, 39 in the validation set, and 48 in the test set. The final model achieved in the test set a DSSC of 0.820 (95% confidence interval (CI): 0.799– 0.840) in the whole brain, and in the different lobes: 0.831 (0.804–0.850) in left frontal, 0.827 (0.799– 0.853) in right frontal, 0.817 (0.779– 0.842) in left temporal, 0.834 (0.812–0.849) in right temporal, 0.821 (0.783–0.856) in left parietal, 0.840 (0.810–0.865) in right parietal, 0.832 (0.808–0.851) in left occipital, and 0.856 (0.838–0.871) in right occipital. CNN tuber volume quantification nearly perfectly correlated (Spearman correlation coefficient) with the neuroradiologist’s across the whole brain [0.984 (95% CI: 0.971–0.991)], and in the different lobes: 0.966 (0.940–0.981)] in left frontal, 0.973 (0.952–0.985)] in right frontal, [0.936 (0.888–0.964)] in left temporal, 0.967 (0.942–0.982)] in right temporal, 0.989 (0.980–0.994)] in left parietal, 0.983 (0.970–0.990)] in right parietal, 0.992 (0.985–0.995)] in left occipital, and 0.982 (0.968–0.990)] in right occipital, all p-values <0.00001.

Significance:

We generated, trained, validated, and made publicly available a CNN, which achieves a near-perfect correlation with a neuroradiologist gold-standard quantification of tuber burden, allows for objective tuber segmentation, and increases rigor and reproducibility in TSC research across institutions.

INTRODUCTION

Tuberous sclerosis complex (TSC) affects many organ systems¹, but particularly impacts the brain and is associated with high rates of autism spectrum disorder, intellectual disability, and refractory epilepsy². Although TSC is a relatively rare disease –incidence of 1/6,000 to 1/10,000 live births and population prevalence of 1/12,000 to 1/25,000^3–6—, it serves as a model to better understand how interictal epileptiform activity and seizures impact cognitive outcomes in general. Several small uncontrolled studies and later the EPISTOP trial suggest that early treatment, even preventively before the first seizure occurs, may improve neurocognitive and epilepsy outcomes^7–10. In contrast, the PREVeNT randomized clinical trial found that preventive vigabatrin treatment did not improve cognition or drug-resistant epilepsy, although it decreased the probability of infantile spasms¹¹. These discordant results may be partially explained by an unaccounted for confound: tubers that vary widely in number, size, and morphology between individuals^{12, 13}, which is suggested by the long-standing observation that tuber burden is associated with neurological severity, epileptogenesis, and cognitive outcomes^{14, 15}. However, randomized clinical trials in TSC may not be large enough to fully balance confounders¹¹, and manual segmentation of tubers has been too labor-intensive for studies to appropriately adjust for tuber burden^7–10. More urgently, accurate and objective tuber delineation is a crucial step in planning complete resection during epilepsy surgery –the most effective treatment for TSC patients with refractory epilepsy^{16, 17}—, however, there remains no high-performing automated tuber segmentation algorithms available for research or clinical use.

Current automated tuber detection algorithms achieve limited segmentation results. For instance, a CNN, initially developed for multiple sclerosis lesions¹⁹, was re-trained with manually segmented MRIs of patients with TSC, but the CNN-predicted segmentations in the test set still had to be visually inspected and manually corrected before further use in a research study¹⁴. One recent CNN was able to segment tubers on 2-dimensional T1-weighted and FLAIR MRI slices, but only achieved a Dice-Sørensen similarity coefficient (DSSC, as defined in Figure 1) of 0.59 in the test set²⁰. Another advanced deep learning pipeline included a CNN that segmented tubers and a random forest that quantified prediction uncertainty to iteratively improve the gold-standard segmentation during training²¹. Despite this advanced deep learning architecture, it only achieved a DSSC of 0.60 on cross-validation²¹. The modest results in the literature to date may reflect the limitations of the chosen CNN architectures^{14, 20} and the limited use of hyperparameter optimization^{14, 20, 21}, but are more likely the result of the relatively small sample sizes with 20¹⁴, 21²⁰, or 153²¹ MRIs in the training sets.

Figure 1. The Dice-Sørensen similarity coefficient (DSSC) measures the spatial overlap between the gold-standard segmentation (red, first column) and the predicted segmentation (green, second column).

A. No overlap between gold-standard and predicted segmentation: DSSC = 0. B. Partial overlap between gold-standard and predicted segmentation: 0 < DSSC < 1. C. Complete overlap between gold-standard and predicted segmentation: DSSC = 1.

This study aimed to develop, validate, and to make publicly available a high-performing automated tuber segmentation and tuber volume quantification algorithm using three-dimensional CNN architectures, systematic hyperparameter optimization, data augmentation, and the largest database of high-quality brain MRI in patients with TSC currently available. We hypothesized that our final model will perform similarly to the human neuroradiologist gold-standard.

PATIENTS AND METHODS

To meet the word number and the reference number limits, technical subsections and technical citations are presented in e-Methods.

Study design.

This is a retrospective descriptive study. This study applies CNNs, a deep learning method especially apt for computer vision, to the problem of automatic tuber segmentation and tuber burden quantification in brain MRIs.

Data sources.

We obtained brain MRIs and demographic data of patients with TSC from the Tuberous Sclerosis Complex Autism Center of Excellence Research Network (TACERN)²², the Rare Diseases Clinical Research Network (RDCRN)²³, and Boston Children’s Hospital, which together comprises the largest database of brain MRI in TSC to date.

Ethical approval.

For TACERN and RDCRN patients, the study protocols were approved by the Institutional Review Boards at each participating site under the direction of the leading regulatory core at Cincinnati Children’s Hospital Medical Center²². At Boston Children’s Hospital Institutional Review Board the IRB numbers were P00005074 for TACERN, P00009726 for RDCRN, and P00045155 for Boston Children’s Hospital patients not already enrolled in TACERN or RDCRN.

Patients.

We included all patients from the TACEiRN, RDCRN, and Boston Children’s Hospital datasets with a clinical or genetic diagnosis of TSC who had at least one high-quality T1-weighted sequence, T2-weighted sequence, and FLAIR image sequence in the same MRI session. MRIs were collected for clinical or clinical research purposes and reflect regular clinical practice at leading hospitals in which only one MRI was obtained at 1.5 Tesla and the rest were obtained at 3 Tesla. The resolution of the T1-weighted MPRAGE was typically 1mmx1mmx1mm or better. All MRI scanners had quality checks using the American College of Radiology phantom. The TACERN data also used a healthy volunteer phantom and showed highly reproducible measures across scanners and over time. Data acquisition and quality controls have been previously detailed for TACERN²⁴ and BCH¹⁵. The acquisition and quality controls likely varied between TACERN, RDCRN, and BCH. It is important to emphasize that, in machine learning, the more heterogeneous the images the model is developed on, the more robust and generalizable the model will be. All patients were diagnosed with TSC at a center of excellence credentialed by the TSC Alliance based on standard TSC diagnostic criteria¹. We excluded duplicated patients, patients with missing demographic data, and low-quality MRIs (Figure e-1). We did not exclude MRIs with other abnormalities such as generalized atrophy, surgical resection cavities, or post-laser ablation changes to make our CNN robust to potential tuber mimics such as gliosis or cystic changes and to reflect clinical neuroradiological practice where it may be difficult to differentiate a small tuber from an area of gliosis (Figure e-2).

Outcome evaluation.

The primary outcome measure was the DSSC, which ranges between 0, when there is no overlap between the segmentation predicted by the model and the gold-standard manual segmentation, and 1, when there is complete overlap (Figure 1)²⁵. In practice, the DSSC also has a small epsilon term (10⁻⁶ in our study) added to both the numerator and the denominator to prevent division by zero:

$$\mathit{\text{DSSC}}=\left(2*\mathit{\text{true positives}}+\mathit{\text{epsilon}}\right)/\left(2*\mathit{\text{true positives}}+\mathit{\text{false positives}}+\mathit{\text{false negatives}}+\mathit{\text{epsilon}}\right)$$ (1)

We evaluated the correlation between predicted and gold-standard tuber volume using Spearman correlation coefficient (null hypothesis of no correlation). For tuber volume in individual brain lobes and in the overall cortical gray matter, we used masks created from the MNI probabilistic structural atlas distributed with FMRIB Software Library (FSL). Because we calculated the correlation coefficient in several brain regions, we adjusted for multiple comparisons with the Benjamini and Hochberg false discovery rate²⁶ with a threshold (q value) of 0.05, and we present all p-values already adjusted for multiple comparisons.

Tuber segmentation gold-standard.

We manually segmented all tubers using the ITK-SNAP²⁷ software. Because many tubers have tissue bridges with other tubers or are part of a tuber conglomerate, we did not aim to identify individual tubers, but tuber tissue voxel-wise. A board-certified pediatric neurologist and TSC imaging expert (J.M.P.) and a board-eligible/board-certified neuroradiologist (M.D.S.) trained and closely supervised a medical graduate student (C.F.G.) and a board-eligible/board-certified pediatric neurologist (I.S.F.) on tuber segmentation. C.F.G. performed the initial manual segmentation. Then, I.S.F. refined the manual segmentation iteratively with at least three additional rounds per segmentation of visual review and manual corrections by J.M.P. and M.D.S. The dataset was not divided into train, validation, and test subsets until this segmentation process was finalized. Then, after the dataset was divided into train, validation, and test subsets, all MRIs in the test set underwent an additional round of visual revision and manual correction as needed by M.D.S. Changes at that point were minimal. In patients with several MRI sessions, we segmented each set of MRIs independently. Our criteria to identify tubers were: 1) cortical or juxtacortical location with blurring of the gray-white matter interface; 2) T1-hypointensity, T2 and FLAIR hyperintensity (since this may vary in younger children or when there are cystic or calcification changes, T2-weighted image were considered the most reliable sequences for tuber delineation); 3) inclusion of cyst-like tubers and of partially calcified tubers when the cavity or calcification was surrounded by tuber tissue; 4) exclusion of cysts and fully calcified tubers with no other tuber tissue; 5) exclusion of radial migration lines and of imaging pathology limited to the white matter only; and 6) exclusion of brainstem and cerebellar tubers (Figure e-2). Although tubers may have different characteristics on brain MRI²⁸, our inclusion and exclusion criteria were designed to identify the classically-described cortical and juxtacortical tubers.

Image preprocessing.

We skull-stripped brain MRIs with the SynthStrip tool from FreeSurfer, we combined T2-weighted image sequences with NiftyMIC version 0.9, we resampled all T1-weighted image sequences to an isotropic voxel size of 1 mm and reoriented, bias-field corrected, and cropped images with FSL-distributed anatomical processing tools, and then co-registered T2-weighted and FLAIR image sequences to the preprocessed T1-weighted image sequence and then registered all images to the asymmetric version of the MNI 2009c template with EasyReg²⁹. See e-Methods for details.

Division into train, validation, and test subsets.

To avoid patient overlap between subsets, since several patients had more than one MRI session, we included all multi-session patients in the train subset. See e-Methods for details.

Data input and preparation.

We inputted the data as tensors of 3-dimensional T1-weighted, 3-dimensional T2-weighted, and 3-dimensional FLAIR image sequences. No demographic or other non-MRI data were inputted into the model. See e-Methods for details.

Model architecture.

The UNet architecture consists of a contracting path and an expanding path with long skip connections between the two paths (Figure 2). The contracting path (or encoder) extracts feature maps from input MRIs while the expanding path (or decoder) restores the feature maps to the same size as the inputs. The long skip connections propagate feature maps from encoder layers to the corresponding layers of the decoder, which prevents the loss of contextual information due to downsampling in the contracting path and helps recover fine-grain details³⁰. The final layer is a 1×1×1 convolution with a sigmoid activation function that classifies each voxel to either tuber or no tuber. See e-Methods for details.

Figure 2. Schematic representation of TSCCNN3D_dropout, the CNN architecture with the highest Dice-Sørensen similarity coefficient in the validation set, and, therefore, the architecture that was selected in the final model.

CNN: Convolutional neural network. ReLU: Rectified linear unit. 3D: three-dimensional

Loss function.

A loss function quantifies how far away a predicted segmentation is from the gold-standard segmentation. During training, the CNN iteratively modifies its parameters to minimize the loss function. We used the focal Tversky loss function, defined as:

$$\mathit{\text{FTL}}={\left(1-\left(\left(\mathit{\text{true positives}}+\mathit{\text{epsilon}}\right)/\left(\left(\mathit{\text{true positives}}+\mathit{\text{alpha}}*\mathit{\text{false positives}}+\mathit{\text{beta}}*\mathit{\text{false negatives}}\right)+\mathit{\text{epsilon}}\right)\right)\right)}^{\left(1/\mathit{\text{gamma}}\right)}$$ (2)

Tuber segmentation, as with many other medical segmentation problems, suffers from highly imbalanced data: voxels with tubers represent only a very small proportion of the total voxels in the brain tissue. In (2), when beta is greater than alpha, false negatives are penalized more heavily than false positives which makes the focal Tversky loss function more likely to outperform other loss functions in highly imbalanced data. Additionally, small lesions (small tubers in our case) are often harder to detect because when they are not detected (small spatial false negatives) they do not typically contribute much to overall loss. Here, in (2), when setting gamma higher than 1, misclassified predictions (difficult lesions, such as small lesions) are penalized more heavily than obvious lesions, which facilitates detection of small lesions³¹. Of note, when gamma equals 1, the focal Tversky loss simplifies to a Tversky loss; when alpha equals 0.5, beta equals 0.5, and gamma equals 1, the focal Tversky loss simplifies to a DSCC loss.

Data augmentation.

We evaluated the impact of data augmentation on performance by allowing a proportion of the MRI data in the training set to be flipped and rotated 30 degrees in any of the three spatial dimensions, which makes a CNN more robust to the essential features of the tubers and less sensitive to particularities of individual examples. See e-Methods for details.

Training.

We used the Boston Children’s Hospital Enkefalos-v3 High Performance Computer Cluster (E3). We trained our CNN using 2 NVIDIA A100 GPUs using tensorflow mirrored strategy with total batch size 100 (50 per GPU) and 200 epochs. Each model required between 20 and 26 hours for training. We used an adaptative moment estimation with decoupled weight decay regularization (ADAMW) optimizer with an initial learning rate of 0.0002. We then performed a systematic hyperparameter search by evaluating the following hyperparameters (Table e-1): focal Tversky loss (alpha 0.5 and beta 0.5 versus alpha 0.25 and beta 0.75), focal Tversky loss (gamma 1 versus gamma 1.3), CNN architecture (TSCCNN3D versus TSCCNN3D_dropout versus a 3D UNet with a ResNet152 backbone encoder), and data augmentation in the train set (0% versus 25% of data augmented in the train set). As is the standard in artificial intelligence development, we selected the combination of hyperparameters with the highest DSSC in the validation set as the final model and, to evaluate its performance in future datasets, we tested the final model using the test set.

Demographic data and visualizations.

Continuous data are presented as median [25^th percentile (p₂₅) - 75^th percentile (p₇₅)] and mean (standard deviation). Categorical data are presented as frequency (percentage). We used python 3.11.5 and the packages pandas 2.0.3, NumPy 1.24.3, matplotlib 3.10.1, and nibabel 5.3.1 for data analysis and data visualization.

Data availability.

We have made our models and full code available on Zenodo: https://doi.org/10.5281/zenodo.17081689. We have also made our pipeline available in a user-friendly format using Docker so that readers can apply our model to their own data following the instructions in the files Preprocessing_instructions and TSCCNN3D_dropout_instructions.

RESULTS

Demographics.

The data used for the study (Figure e-1) consisted of 263 MRI sessions from 196 patients (57% males, median (p₂₅ – p₇₅) age 4.3 (3.0–10.1) years) (Table 1). There were 176 (66.9%) brain MRIs from 109 patients in the train set, 39 (14.8%) brain MRIs from 39 patients in the validation set, and 48 (18.3%) brain MRIs from 48 patients in the test set (Table 1).

Table 1. Demographic features.

BCH: Boston Children’s Hospital. MRI: Magnetic resonance imaging. N/A: Not applicable. p₂₅: 25^th percentile. p₇₅: 75^th percentile. RDCRN: Rare Diseases Clinical Research Network. SD: Standard deviation. TACERN: Tuberous Sclerosis Complex (TSC) Autism Center of Excellence Research Network.

DATA ON PATIENTS
		Train set	Validation set	Test set	Comparison of the train, validation, and test set p-value	Total
		N=109 patients	N=39 patients	N= 48 patients	N/A	N=196 patients
Sex	Male	70 (64%)	20 (51%)	26 (54%)	0.27155	112 (57%)
	Female	39 (36%)	19 (49%)	22 (46%)		84 (43%)
Origin	TACERN	54 (50%)	26 (67%)	38 (79%)	0.00344	111 (57%)
	RDCRN	47 (43%)	10 (26%)	5 (10%)		69 (35%)
	BCH	8 (7%)	3 (8%)	5 (10%)		16 (8%)
DATA ON MRIs
		N=176 MRIs	N=39 MRIs	N=48 MRIs		N=263 MRIs
Number of MRIs per patient	1	57 (52%)	39 (100%)	48 (100%)	<0.00001	144 (74%)
	2	37 (34%)	0 (0%)	0 (0%)		37 (19%)
	3	15 (14%)	0 (0%)	0 (0%)		15 (8%)
Age at MRI in years	Median (p25-p75)	7.3 (3.1–11.2)	3.2 (3.0–5.1)	3.0 (2.0–3.2)	<0.00001	4.3 (3.0–10.1)
	Mean (SD)	7.6 (5.1)	5.5 (4.9)	3.6 (3.6)		6.6 (5.0)
Origin	TACERN	115 (65%)	26 (67%)	38 (79%)	0.25719	130 (49%)
	RDCRN	47 (27%)	10 (26%)	5 (10%)		111 (42%)
	BCH	14 (8%)	3 (8%)	5 (10%)		22 (8%)
Tuber burden in the whole brain in mm 3	Median (p25-p75)	15,342 (6,683–37,695)	27,726 (7,920–54,212)	50,211 (14,497–100,092)	0.00084	19,702 (7,264–50,103)
	Mean (SD)	25,289 (24,695)	35,135 (33,553)	64,401 (61,611)		33,887 (38,568)
Tuber burden in the gray matter in mm 3	Median (p25-p75)	14,376 (5,884–35,450)	25,657 (5,391–51,129)	47,947 (13,181–94,487)	0.00102	18,882 (5,856–47,860)
	Mean (SD)	23,730 (23,539)	32,795 (31,490)	60,670 (58,252)		31,816 (36,510)
Tuber burden in the left frontal lobe in mm 3	Median (p25-p75)	3,519 (1,203–8,520)	5,154 (915–11,743)	8,170 (1,893–18,371)	0.00489	4,269 (1,272–10,260)
	Mean (SD)	5,743 (6,237)	7,800 (7,985)	13,198 (14,898)		7,409 (9,165)
Tuber burden in the right frontal lobe in mm 3	Median (p25-p75)	3,244 (1,049–7,944)	3,588 (859–9,476)	9,348 (4,017–19,893)	0.00026	3,887 (1,070–9,589)
	Mean (SD)	5,270 (6,065)	7,466 (8,964)	13,721 (12,587)		7,138 (8,701)
Tuber burden in the left temporal lobe in mm 3	Median (p25-p75)	630 (0–2,168)	662 (0–4,103)	2,301 (310–9,882)	0.00470	803 (0–3,487)
	Mean (SD)	2,009 (3,487)	2,644 (3,730)	6,588 (10,051)		2,939 (5,627)
Tuber burden in the right temporal lobe in mm 3	Median (p25-p75)	1185 (80–3,310)	892 (0–4,476)	2,063 (73–10,632)	0.27155	1,185 (37– 4,040)
	Mean (SD)	2,374 (3,198)	3,633 (5,692)	6,322 (9,439)		3,281 (5,492)
Tuber burden in the left parietal lobe in mm 3	Median (p25-p75)	1,125 (191–3,959)	2,050 (113–5,048)	5,266 (614–8,658)	0.00690	1,414 (200–5,213)
	Mean (SD)	2,572 (3,157)	3,351 (4,195)	6,603 (7,749)		3,423 (4,751)
Tuber burden in the right parietal lobe in mm 3	Median (p25-p75)	1,505 (269–3,759)	2,299 (843–7,350)	6,706 (1,034–12,782)	0.00111	1,866 (363–5,529)
	Mean (SD)	2,899 (3,890)	4,532 (4,984)	8,716 (9,540)		4,203 (5,941)
Tuber burden in the left occipital lobe in mm 3	Median (p25-p75)	1,008 (4–2,501)	930 (17–2,201)	1,772 (114–5,376)	0.14172	1,074 (7–2,777)
	Mean (SD)	1,790 (2,335)	1,490 (1,664)	3,123 (3,563)		1,989 (2,584)
Tuber burden in the right occipital lobe in mm 3	Median (p25-p75)	918 (0–1,927)	1,448 (72–3,730)	1,408 (46–5,273)	0.078097	1,038 (2–2,663)
	Mean (SD)	1,567 (2,191)	2,581(3,092)	3,571 (4,744)		2,083 (3,059)

Note that, by rounding percentages to the nearest integer, because of rounding error, some percentages sum up to less than 100% (for example, 99%) or to more than 100% (for example, 101%).

Hyperparameter optimization.

The hyperparameter combination which achieved the highest DSSC (0.826) in the validation set was: 25% data augmentation, TSCCNN3D_dropout architecture, and focal Tversky loss alpha=0.25, beta=0.75, and gamma=1.0 (Table e-1). This combination of hyperparameters was selected as the final model.

Segmentation results in the test set.

The final model achieved a DSSC of 0.820 (95% confidence interval: 0.799– 0.840) when segmenting tubers in the test set. DSSC results were similar in gray matter 0.830 (0.810–0.849), and in the different lobes: 0.831 (0.804–0.850) in left frontal, 0.827 (0.799– 0.853) in right frontal, 0.817 (0.779– 0.842) in left temporal, 0.834 (0.812–0.849) in right temporal, 0.821 (0.783–0.856) in left parietal, 0.840 (0.810–0.865) in right parietal, 0.832 (0.808–0.851) in left occipital, and 0.856 (0.838–0.871) in right occipital. Examples of tuber segmentations in the 64×64×64×3 blocks are presented in Figure 3 and examples of segmentations in the complete MRI are presented in Figure 4. Further examples are presented at https://ivansanchezfernandez.github.io/CNNs_for_tuber_segmentation/Results_test_set.html#images. Because a 2-dimensional surface can only show 2-dimensional slices of our 3-dimensional results, we also provide examples of full MRI volume files with segmentations (please, read the 3D_visualizations_instructions).

Figure 3. Examples of the TSCNN3D segmentation results in 64×64×64×3 blocks.

On visual inspection, the segmentations predicted by the CNN appear quite similar to the gold-standard manual segmentation and appear to represent the tuber tissue well. Each example (A, B, and C) provides the T1-weighted, T2-weighted, and FLAIR, the gold-standard manual neuroradiologist segmentation, the TSCCNN3D_dropout segmentation, and the comparison of the gold-standard and TSCCNN3D_dropout segmentation (correct segmentation in green: overlap between gold-standard and TSCCNN3D_dropout segmentations, under-segmentation in red: areas of gold-standard segmentation that the TSCCNN3D_dropout did not segment, and over-segmentation in blue: areas not segmented in the gold-standard segmentation that the TSCCNN3D_dropout did segment).

Figure 4. Examples of the TSCNN3D segmentation results in the 256×256×256×3 full MRIs.

On visual inspection, the segmentations predicted by the CNN appear quite similar to the gold-standard manual segmentation and appear to represent the tuber tissue well. Each example (A, B, and C) provides the T1-weighted, T2-weighted, FLAIR, the gold-standard manual neuroradiologist segmentation, the TSCCNN3D_dropout segmentation, and the comparison of the gold-standard and TSCCNN3D_dropout segmentation (correct segmentation in green: overlap between gold-standard and TSCCNN3D_dropout segmentations, undersegmentation in red: areas of gold-standard segmentation that the TSCCNN3D_dropout did not segment, and oversegmentation in blue: areas not segmented in the gold-standard segmentation that the TSCCNN3D_dropout did segment).

Tuber burden in the test set.

The tuber burden based on the TSCCNN3D_dropout segmentations was nearly-perfectly correlated with the tuber burden based on gold-standard manual segmentations by neuroradiologist (Table e-2, Figure 5 and Figure e-3). File e-1 provides analyses in various subgroups.

Figure 5. Correlation between the tuber volume based on the gold-standard manual segmentation and the tuber volume based on the TSCCNN3D_dropout segmentation.

Each green dot represents a patient. The dashed diagonal line represents perfect correlation between predicted and gold-standard tuber volume. A. Whole brain. Spearman correlation coefficient: 0.984 (95% confidence interval: 0.971–0.991), p-value <0.00001. B. Gray matter. Spearman correlation coefficient: 0.988 (95% confidence interval: 0.978–0.993), p-value <0.00001.

DISCUSSION

We have developed, validated, and made publicly available a high-performing automated tuber segmentation CNN model. Our final model achieves a DSSC of 0.820. Although there is no benchmark MRI dataset for comparing tuber segmentation performance across different studies, our results surpass by more than 20 percentage points the prior state-of-the-art in automated tuber segmentation (DSSC of 0.60²¹), a 37% improvement. Further, TSCCNN3D_dropout achieved an almost perfect correlation with the gold-standard volumetric estimation of tuber burden both in the whole brain, the gray matter, and in each lobe.

Comparison with prior models.

The performance of automated tuber segmentation algorithms available in the literature is limited by the sample sizes they were trained on, their architecture, and their lack of reusability by clinicians and the scientific public. For instance, a CNN trained on 21 patients achieved a DSSC of only 0.53 on leave-one-out cross-validation and only 0.59 in the test set²⁰. Although the authors used data augmentation during training, the original sample size was small²⁰. Further, the input was constrained to 2D slices of T1-weighted and FLAIR image, without concurrent T2-weighted image²⁰. A larger study trained a more sophisticated CNN architecture with 165 MRIs from 88 TSC patients, but it only achieved a DSSC of 0.605 in the test set²¹. During training, this CNN generated a segmentation with tuber tissue probabilities for each voxel as well as a map of the uncertainty for each probability²¹. The uncertainty map allowed the algorithm to iteratively improve the noisy “gold-standard” manual segmentations during training²¹. Despite this advanced architecture, the batch size was constrained to a single 64×64×64×3 block, hyperparameter optimization was limited, and there was no regularization or data augmentation²¹. Another small study developed a CNN with 14 MRIs in the train set, 7 MRIs in the validation set, and 10 MRIs in the test set³². While the authors reported a DSCC of 0.973, their CNN architecture used 2 fully-connected layers in the final part of the CNN architecture (which flatten the images allowing for classification, but not for segmentation) rather than convolutional layers (used for segmentation), which makes it challenging to interpret the reported segmentation results³². Our study addressed many of the limitations of prior studies: 1) our CNN was trained on a much larger dataset of 176 MRIs from 109 patients with a 3-dimensional multispectral dataset including T1-weighted sequence, T2-weighted sequence, and FLAIR image sequence, 2) we evaluated different sophisticated 3-dimensional CNN architectures, and 3) we systematically searched optimal hyperparameters with a loss function designed to work with imbalanced data. Currently, there is no common benchmark MRI dataset for tuber segmentation and each study evaluated their model in their own different dataset. Therefore, direct performance comparisons between different studies may be inaccurate. However, our results improve the state-of-the-art in automated tuber segmentation algorithms by more than 20 percentage points, that is, a 37% improvement, which suggests some degree of performance improvement. The performance of our CNN is even more remarkable when compared to segmentations by humans: in one of the previously cited studies, segmentations by neurologists achieved an interrater agreement in DSSC of 0.74²⁰, compared to our model’s 0.82. Further, manual segmentation took us on average 2 hours and a half per MRI, but the CNN on a regular laptop segmented each MRI in seconds or a minute. This may facilitate initial segmentation by the CNN followed by supervision by a human expert.

Clinical relevance and scientific significance.

An automated and easily deployable tuber segmentation method can help both clinical epilepsy surgery planning and future TSC outcome research studies. In TSC, as in other developmental and epileptic encephalopathies, the relationship between neurodevelopmental disorders (including autism spectrum disorder), intellectual disability, and epilepsy is complex and poorly understood³³. In TSC, early-onset intractable epilepsy with infantile spasms is associated with worse neurocognitive outcomes³⁴ and accumulated seizure burden predicts neurodevelopment at 3 years of age³⁵, but these associations may just reflect a more severe underlying etiology. Seizure burden and interictal epileptiform discharges are some of the few modifiable predictors of cognitive outcome in epileptic encephalopathies^{35, 36}, but it remains unclear whether treating seizures and interictal epileptiform discharges improves neurodevelopmental outcomes.

Several single-center, open label, retrospective and/or uncontrolled studies have suggested that early treatment of seizures and epileptiform discharges improved both epilepsy and neurodevelopmental outcomes^{7, 8, 37}. Further, the EPISTOP multicenter clinical trial found that patients on preventive treatment with vigabatrin were less likely to develop clinical seizures, refractory epilepsy, and infantile spasms, but there were no differences in autism spectrum disorder or cognition¹⁰. In contrast, the PREVeNT multicenter double-blind placebo-controlled randomized clinical trial found no differences in cognitive scores at 12 and 24 months and no difference in the probability of epilepsy or refractory epilepsy at 24 months, although the preventive arm had lower probability and later onset of infantile spasms¹¹. Different results from different studies are likely due to unadjusted confounding, such as not considering tuber burden in general or in specific locations. Recent work from our group found that autism spectrum disorder diagnosis was not associated with tuber burden globally; however, the presence of tubers specifically intersecting the right fusiform face area was associated with a 3.7-fold increase in autism spectrum disorder diagnosis—work that would not be possible without accurate tuber segmentation³⁸. In the future, our TSCCNN3D_dropout model will enable us to stratify patients by tuber burden and help reduce bias in future TSC studies. To this end, we have made our pipeline available to the clinical and scientific community in a user-friendly format via a Docker container easily used via Docker Desktop³⁹ (Preprocessing_instructions and TSCCNN3D_dropout_instructions).

The most effective treatment of refractory epilepsy is epilepsy surgery, which achieves seizure control in approximately 60% of patients^{16, 17}. The accurate delineation of tubers is crucial for successful epilepsy surgery^{40, 41}. Voxel-wise delineation of tubers allows for obtaining other image quantifications such as tuber volume or distance from the tuber to another tuber or a certain anatomic landmark. The database used for the current study contained only MRI data and basic demographics, but in the future we intend to use the objective and reproducible segmentation with our CNN to study whether resections should be generous⁴² or can be limited to the T2-hyperintense core⁴³. Further, armed with detailed information on tuber burden, future studies can evaluate its impact on neurological severity, epileptogenesis, and cognitive outcomes^{14, 15}, potentially explaining conflicting findings among prior studies that did not fully adjust for tuber burden^{7, 8, 10, 11, 37}.

Strengths, weaknesses, and future directions.

Our 9-center study included a variety of MRI scanners, MRI quality, and MRI acquisition protocols, which makes our CNN robust to noise and variations in data quality and origin. Because our study focused on MRI data, we only present basic demographic and clinical features. More detailed clinical data for the TACERN patients can be found elsewhere²². Our inclusion and exclusion criteria aimed to identify the classical cortical and juxtacortical tubers. However, there were multiple examples of cortical and juxtacortical tubers which were challenging to identify and delineate (https://ivansanchezfernandez.github.io/CNNs_for_tuber_segmentation/Results_test_set.html#images). Our algorithm does not aim to segment large transmantle dysplasias or large areas of dysplastic cortex with limited tuber characteristics. We aimed for a practical definition that allows for both actionable presurgical segmentation and estimation of total tuber volume. Detecting tubers is a particularly challenging task for computer vision because the shape, size, and intensity of tubers varies among and within patients, and may include areas of cystic degeneration and calcification within otherwise typical tuber tissue. The CNN may overestimate the size of some large tubers, although in cases where the predicted segmentations represent the real tuber borders better than the manual segmentations (as suggested in several of our examples https://ivansanchezfernandez.github.io/CNNs_for_tuber_segmentation/Results_test_set.html#images), performance is unjustly penalized because the manual segmentation is, by definition, the gold-standard. In future work, we plan to further develop a version of the “learning from noisy labels” algorithm²¹. Because almost all MRIs in the dataset were acquired at 3 Tesla, we do not have enough information to predict the performance of our model on 1.5 Tesla MRIs. We did not seek food and drug administration (FDA) approval for clinical use, as that would require validation by 3 independent expert readers, for which we currently lack sufficient resources. Our motivation was to improve on prior methods and make our results publicly available for use and further improvement by other groups.

Conclusion.

In this study, we trained, validated, and made publicly available a high-performing automated tuber segmentation CNN which achieved near-perfect correlation with the gold-standard volumetric estimation of tuber burden.

KEY POINTS.

• We developed a convolutional neural network (CNN) to automatically segment tubers and quantify their volume.

• We used 176 MRIs in the train set, 39 MRIs in the validation set, and 48 MRIs in the test set.

• When compared with the neuroradiologist’s gold standard, the CNN achieved a Dice-Sørensen similarity coefficient of 0.820 in the test set.

• Tuber volume quantification nearly perfectly correlated with gold standard in the whole brain (0.984), gray matter (0.988), and each lobe.

DATA AVAILABILITY STATEMENT

We have made our models and full code available on Zenodo: https://doi.org/10.5281/zenodo.17081689.