Computer-aided diagnosis of neonatal acute bilirubin encephalopathy with multi-modal MRI images and convolutional neural networks

Abstract

Acute bilirubin encephalopathy (ABE) is a serious complication of hyperbilirubinemia (HB). Efficiently detecting ABE in non-ABE neonates with HB by magnetic resonance imaging (MRI) techniques remains a great challenge in clinical practice, as both groups may exhibit similar MRI features of T1 hyperintensity in specific brain regions. To this end, a computer-aided (CAD) diagnosis system based on multi-modal MRI images and convolutional neural networks (CNNs) was proposed in this study. A total of 150 patients were included in the study, half of whom were ABE neonates, and the other half were non-ABE neonates with HB. During the hospitalization, each patient underwent a 3 Tesla whole-brain MRI examination, generating a T1-weighted image (T1WI), a T2-weighted image (T2WI), and an apparent diffusion coefficient map (ADC), respectively. These 3 types of MRI images and their combinations were fed into 2 CNNs, i.e., EfficientNetB0 and InceptionV3. A traditional machine learning method, as a baseline, named support vector machine was also applied and its performance was then compared with CNNs. The classifiers’ performance with multi-modal MRI images was superior to that with any single-modal MRI image. Both 2 CNNs outperformed SVM. The best performance was achieved by InceptionV3 with the fusion images of T1WI, T2WI, and ADC, with an accuracy of 0.9622 AUC of 0.9978, and F1-score of 0.9617. The proposed CAD system for automatically diagnosing neonatal ABE based on multi-modal MRI images and CNNs showed an outstanding performance and is potentially applicable in actual clinical practice.

Introduction

Neonatal jaundice is a common clinical manifestation of elevated total serum bilirubin (TSB) in the early neonatal period, occurring in more than 84% of late preterm and term babies, termed neonatal hyperbilirubinemia(HB)¹. Bilirubin predominantly exists in two biologically distinct forms: conjugated and unconjugated bilirubin. Conjugated bilirubin, a water-soluble metabolite processed by hepatic glucuronidation, can be efficiently excreted via the biliary and renal systems. In contrast, unconjugated bilirubin (also referred to as indirect bilirubin) remains unmetabolized by the immature neonatal liver, lacking glucuronide conjugation and thus exhibiting poor aqueous solubility. This metabolic limitation renders neonates particularly vulnerable to HB, as unconjugated bilirubin tends to accumulate in the circulation due to inadequate hepatic clearance capacity. Pathophysiologically, when TSB levels exceed critical thresholds (≥ 25 mg/dL in term infants and 15–20 mg/dL in preterm infants), unconjugated bilirubin—owing to its lipophilic nature—can cross the immature blood-brain barrier, preferentially depositing in bilirubin-sensitive brain regions such as the globus pallidus, subthalamic nucleus, and hippocampus. Such deposition may directly lead to acute bilirubin encephalopathy(ABE), a serious complication of HB, since unconjugated bilirubin is neurotoxic². If left neonates untreated, ABE may progress to chronic bilirubin encephalopathy (CBE), a permanent neurological sequela characterized by lifelong neurodevelopmental disabilities including cerebral palsy, sensorineural hearing loss, and cognitive impairment³. Notabley, the brain damage in the ABE period is reversible if effective treatments, such as phototherapy and/or exchange transfusion, are applied in time⁴. However, the brain damage in the CBE period is irreversible⁵. Therefore, timely diagnosis and treatment of ABE neonates are crucial to preventing permanent brain damage and death.

In clinical settings, the ABE diagnosis is primarily based on the results of laboratory biochemical tests and patients’ clinical characteristics, which have many limitations and poor diagnostic performance^6–8. As a noninvasive, nonradiative, multi-parametric, and high-resolution imaging method, Magnetic Resonance Imaging (MRI) technology has been increasingly used in diagnosing neonatal ABE in recent years. Many studies have reported that the T1 signal value of bilateral globus pallidus (GP) significantly increased during the early ABE, which is considered an important MRI finding for ABE neonates and is widely used in clinical practice^9–11, shown in Fig. 1. Based on this MRI finding, many studies have attempted to diagnose ABE using only T1-weighted images (T1WI); however, the diagnostic performance was barely satisfactory, as demonstrated in our previous study¹². Specifically, we conducted a study employing three diagnostic approaches for ABE: a visual inspection, a semi-quantitative method based on normalizing T1-weighted intensities of the globus pallidus (GP) and subthalamic nuclei, and a deep learning method using the ResNet18 framework. All three methods were applied exclusively to patients’s T1WI; The visual inspection got a poor diagnostic accuracy of 53.58% indicating the difculty of the challenge in real clinical practice. The semi-quantitative approach and ResNet18 achieved a classifcation accuracyof 62.11% and 72.15%, respectively, however such resluts remains insufficient for practical clinical demands. Zhang et al¹³. proposed a deep learning model integrating an attention model with a central network for neonate ABE diagnosis. And the model achieved accuracies of 0.639 on T1WI, 0.738 on T2WI, and 0.845 on multi-model images (T1WI + T2WI). Inspired by the prvious studies and to further improve the ABE diagnostic accuracy, we aimed to investigate the utility of multi-model MRI images. And through an extensive literature, and we found that other MRI modality images, such as T2-weighted image (T2WI) and apparent diffusion coefficients map (ADC), were potentially useful for ABE diagnoses^13–15. Inspired by Wang et al.¹⁴ Cece et al.¹⁵ Zhang et al.¹³, this study investigated the value of multi-modal MRI images (T1WI, T2WI, and ADC) in the automated diagnosis of ABE, aiming to further improve the diagnostic performance. Figure 2 showed the workflow of the study.

Fig. 1.

Axial T1-weighted fast gradient-echo sequence images of the brain. (a) A T1WI of the ABE neonate that shows significant radiographic features: bilateral and symmetrical T1 hyperintensity at the globus pallidus. (b) A T1WI of the non-ABE neonate that no abnormal T1 signal was detected.

Fig. 2.

The workflow of the study.

Data and methods

This study was approved by the research ethics committee of Xinjiang medical university and performed in accordance with the ethical standards as laid down in the 1964 Declaration of Helsinki and its later amendments or comparable ethical standards.

Study subjects

The comparative data were retrospectively collected from routine clinical examinations at the Children’s Hospital of Zhejiang University School of Medicine between 2016 and 2020, with the research protocols receiving approval from the Clinical Research Ethics Committee. The informed consent to participate was obtained from all of the participants’ guardian. A total of 150 HB newborns (TSB > 5 mg/dL)¹⁶ who had the MRI examination at the postmenstrual age of 37–41 weeks during their hospitalization, were included in this study. Of these 150 HB newborns, 75 were clinically diagnosed as ABE with a bilirubin-induced neurologic dysfunction(BIND) score ≥ 1, and the other 75 newborns were identified as non-ABE with HB that has no clinical manifestations of BIND, regarded as a control group. The BIND score, which ranges from 0 to 9, reflects the severity of ABE: scores of 1–3, 4–6, and 7–9 indicate mild, moderate, and severe ABE, respectively. A BIND score was generated according to the patient’s clinical manifestations associated with bilirubin toxicity, which are characterized by : (1) neuromotor signs; (2) muscle tone abnormalities; (3) hyperexcitable neonatal reflexes; (4) a variety of neurobehavior manifestations; (5) speech and language abnormalities; and (6) evolving array of central processing abnormalities, such as sensorineural audiology and visuomotor dysfunctions¹⁷. The patients’ data included in this study was verified by 2 experienced pediatricians with more than 8 years of clinical practice.

MRI acquisition

The patients’ MRI images were acquired from a 3.0-T whole-body MRI scanner (Achievea, Philips Healthcare, Best, The Netherlands) with a routine clinical brain MRI protocol, adhering to experimental requirements. The protocol included the following imaging sequences: T1-weighted fast gradient-echo sequence in the axial direction, especially, repetition time (TR) of 200 ms, echo time (TE) of 2.14ms, a field of view(FOV) of 330×330mm², flip angle of 80°, in-plane resolution of 0.45×0.45mm², and 18 slices with a slice thickness of 4.5 mm to cover the whole brain; T2-weighted turbo spin-echo sequence in the axial direction, with TR of 3000ms, TE of 80 ms, FOV of 230×230 mm², in-plane resolution of 0.34×0.34 mm², and 18 slices with a slice thickness of 4.5 mm to cover the whole brain; Diffusion-weighted echo-planar imaging in the axial direction, with TR of 2109 ms, TE of 80 ms, FOV of 230×230mm², in-plane resolution of 0.90×0.90 mm², 18 slices with a slice thickness of 4.5 mm to cover the whole brain, one non-DWI(b₀), and a single DWI (diffusion-weighted image) at a b value of 800 s/mm². ADC maps were generated by using the following equation:

1

All MRI images included in this study were verified by an experienced radiologist to ensure the quality of the data meets the experiment’s requirement.

Image preprocessing

Firstly, MRI image registration work was performed. We aligned T2WI, and ADC maps to T1WI with the FMRIB’s Linear Image Registration tool (FSL v6.0, FMRIB, Oxford, UK)¹⁸ by a 2D rigid-body transformation. Then, 3 continuous MRI slices of each MRI sequence that contain the largest area of GP were selected as the study data of the subject, because GP is the most vulnerable region of the brain to the toxicity induced by bilirubin with a symmetric pattern of involvement¹⁹. Notably, for most cases, we select the 7th, 8th, and 9th slices from a patient’s T1-weighted imaging sequence since these 3 continuous slides demonstrate the largest area of GP signal, in other words, these 3 continuous slices contain the most useful information for ABE diagnosis. As for the T2WI and ADC maps, the 3 continuous slices with the same position (7th, 8th, and 9th slices) of T1WI were selected as the study data since our MRI images have the same slice center and slice thickness. Thus, a total of 225 slices from 75 ABE patients and 225 slices from 75 non-ABE patient were obtained for each MRI modality. Then, we cropped each slice to remove the boundary area that had no useful MRI signal, and only the brain region was preserved. To meet the input requirements of pre-trained CNNs, we resized each cropped image uniformly to 224×224 pixels for the pre-trained EfficientNetB0 and 227×227 pixels for pre-trained InceptionV3, respectively. Lastly, a max-min normalization technique²⁰ was applied to all the images to normalize the intensities between 0 and 1.

Classification based on SVM and handcrafted MRI features

We performed a classification task of ABE and non-ABE neonates based on the handcrafted features and the support vector machine (SVM). SVM is a classical supervised machine learning algorithm widely used for binary classification tasks, particularly in medical image analysis. The core principle of SVM lies in finding an optimal hyperplane that maximizes the margin between two classes in a high-dimensional feature space, thereby achieving robust generalization to unseen data²¹. For linearly non-separable data, SVM leverages kernel functions (e.g., radial basis function, RBF) to map the original features into a higher-dimensional space where linear separation is feasible, making it suitable for handling the relatively low-dimensional handcrafted features in this study. To implement SVM-based ABE classification, we chose GP and WM as our ROIs (regions of interest) to construct the hadcrafted MRI features, shown in Fig. 3. Specially, the GP and WM ROIs were manually delineated by two experienced radiologists with over 8 years of clinical experience in neonatal brain imaging. Prior to segmentation, both radiologists underwent standardized training to ensure consistent understanding of anatomical boundaries: GP was identified based on its characteristic location (medial to the internal capsule, lateral to the thalamus) and T1 hyperintensity features in ABE neonates, while WM ROIs were selected from subcortical white matter regions adjacent to the GP (avoiding vascular structures and cerebrospinal fluid spaces) as non-bilirubin-vulnerable control regions.

Fig. 3.

ROIs definition on multi-modal MRI images of the patients. The yellow contour area represents the white matter area and the red contour area represents the globus pallidus area. (a−c) are the T1WI, T2WI, and ADC maps of a representative ABE neonate, while (d−f) is the T1WI, T2WI, and ADC maps of a representative non-ABE neonate.

For rater disagreement adjudication, all segmentations were first independently completed by each radiologist. Discrepancies (defined as overlapping rates < 85% measured by the Dice similarity coefficient) were resolved through a consensus process: the two radiologists jointly reviewed the discordant slices, compared their delineations against anatomical atlases of neonatal brains, and discussed the differences based on imaging characteristics and clinical context. A final ROI mask was agreed upon for each slice to eliminate inter-observer variability. Since the GP is the most vulnerable brain region affected by the neurotoxicity of bilirubin while the subcortical white matter (WM) is not²² and for quantification purposes¹², we use the ratio of average intensity in GP and WM as the feature of the patient calculated by the following equation:

2

where and denote the averaged MRI signal intensities in the patient’s GP and WM ROIs. GP_{norm, T1WI,} GP_{norm, T2WI,} and GP_{norm, ADC,} denote for GP_norm on T1WI, T2WI, and ADC, respectively. In this study, we use the feature fusion strategy to observe the performance difference of the classifier. A total of 7 kinds of fusion features were generated from the different modalities of MRI images. For individual single-modal MRI features, they are (1) GP_{norm, T1WI,} (2) GP_{norm, T2WI,} (3) GP_{norm, ADC}. For fusion of any 2 of single-modal MRI features, they are (4) GP_{norm, T1WI,}+GP_{norm, T2WI,} (5) GP_{norm, T1WI,}+ GP_{norm, ADC} (6) GP_{norm, T2WI,}+ GP_{norm, ADC}. For fusion of all 3 single-modal MRI features: (7) GP_{norm, T1WI,}+ GP_{norm, T2WI,}+ GP_{norm, ADC}. Since the above 7 fusion features’ structure is relatively simple, we chose SVM as the classifier to fulfill the classification task of ABE and non-ABE neonates.

Classification based on the CNNs and MRI images

We also conducted the other classification work by using 2 classical pre-trained Convolutional Neural Networks (CNNs) named InceptionV3²³ and EfficientNetB0²⁴, which have been widely used in medical image classification task and achieved remarkable success. i.e., Shamrat²⁵et al. achieved an accuracy of 96.31% in Alzheimer’s disease stages classification based on InceptionV3 and the MRI images. Rasa et al²⁶. achieved an accuracy of 99.95% by using InceptionV3 model for MRI-based brain tumor classification. Yoon et al.²⁷. utilized the EfficientNetB0 model and achieved an accuracy of 96% in the classification of brain tumors from MRI images.

CNNs are a class of deep learning algorithms specifically designed for image-based tasks, characterized by hierarchical feature learning through convolutional, pooling, and fully connected layers. Unlike traditional machine learning methods that rely on handcrafted features, CNNs automatically extract discriminative features from raw image data: shallow convolutional layers capture low-level visual information (e.g., edges, textures), while deeper layers aggregate these features to learn high-level semantic representations (e.g., anatomical structures in medical images). This end-to-end learning paradigm eliminates the need for manual feature engineering and has demonstrated superior performance in medical image classification tasks,

For this study, we selected two state-of-the-art pre-trained CNN architectures—InceptionV3 and EfficientNetB0. InceptionV3, its architecture introduces the “Inception module” to address the trade-off between model depth, width, and computational efficiency. It uses parallel convolutional filters of varying sizes (1 × 1, 3 × 3, 5 × 5) to capture multi-scale features, with 1 × 1 convolutions reducing dimensionality and minimizing computational cost. The auxiliary classifiers (two intermediate classifiers inserted in the middle layers) were used to alleviate gradient vanishing and enhance feature discrimination for deep networks. InceptionV3 has 42 layers and has been widely applied in medical image classification due to its ability to learn fine-grained features from complex anatomical structures, EfficientNetB0 adopts a compound scaling strategy (simultaneously scaling depth, width, and resolution) to achieve optimal performance with minimal computational resources. Compared to traditional CNNs, it uses a more efficient baseline architecture (MobileNetV2-inspired inverted residuals) and squeeze-and-excitation blocks to enhance feature recalibration. EfficientNetB0 is lightweight (only 7.8 million parameters) while maintaining high accuracy, making it suitable for deployment in resource-constrained clinical settings²⁸. To demonstrate the different performance of the pre-trained CNNs on different modalities of MRI images, we adopted a similar strategy of MRI feature fusion described in Sect. 2.4. It is worth noting that the fusion features described in Sect. 2.4 were handcrafted MRI features, but in this section, we combined different modalities of MRI images to form the RGB images as input to the pre-trained CNNs. Since an RGB image is 3-dimensional (Red, Green, and Blue channel) data but our MRI image is 2-dimensional data (grayscale image), we use the following strategies to create the RGB images by using MRI images pre-processed in Sect. 2.3: For individual single-modal MRI images, we duplicated the pre-processed MRI image 3 times to make 3 identical channels to form the RGB image, which are 1)T1WI, 2)T2WI, and 3)ADC. For 2-modal MRI images, any 2 single-modal MRI images from the same patient were put into the red channel and green channel respectively, and the blue channel image was a binary image with all zero values for all pixels. Thus, 3 kinds of fusion RGB images were made, which were 4)T1WI + T2WI, 5)T1WI + ADC, and 6)T2WI + ADC. For 3-modal MRI images, 3 single-model MRI images from one patient were naturally combined to form the RGB images, namely 7)T1WI + T2WI + ADC. Therefore, a total of 7 kinds of fused RGB images were created for each patient, shown in Fig. 4. Each group of fused RGB images comprises 225 images. These images were allocated at the patient level, with 80% assigned to the training set and the remaining 20% to the testing set. Data augment techniques were also applied during the training process of the CNNs to reduce the overfitting problems. A five-fold cross-validation (5FCV) scheme was applied to evaluate the CNNs’ performance with the following metrics:

Fig. 4.

The presentation for 7 kinds of fusion RGB images generated from an ABE neonate’s multi-modal MRI images. (a) T1WI, (b) T2WI, (c) ADC, (d) T1WI + T2WI, (e) T2WI + ADC, (f) ADC + T1WI, (g) T1WI + T2WI + ADC.

3

4

5

6

7

Where TP, FP, TN, and FN denote the sample number of True Positive, False Positive, True Negative, and False Negative, respectively.

The tuning hyperparameters of the CNNs were heuristically set as follows: the stochastic gradient descent with momentum (SGDM) optimizer with a minimum Batch Size of 10 images was used for training the CNNs, the initial learning rate was set to 0.0003, and the number of the epoch was set to 20. To alleviate overfitting, dropout rate was set to 0.5 and data augmentation techniques such as image rotation, random flipping, scaling, cropping were applied during the CNNs training stage, enhancing thier generalization ability. Both models were initialized with pre-trained weights from ImageNet, a large-scale natural image dataset, and we first fine-tuned these pre-trained models on MRI datasets of brain tumors to facilitate better transfer learning to our target dataset (ABE and non-ABE); Then we adopted transfer learning to leverage their generic feature extraction capabilities and mitigate overfitting, implementing partial fine-tuning where 60% of the initial convolutional layers for both InceptionV3 and EfficientNetB0 were frozen to preserve transferable low-level features (e.g., edges, textures), while middle/top convolutional blocks, fully connected layers, and the final classification layer (modified to 2 classes: ABE vs. non-ABE) were unfrozen for task adaptation. Our experiment was implemented in Matlab R2024a running on the Windows 11 operating system, which was installed on a computer with CPU of 12th Gen Intel^® Core™ i5-12500 3.00 GHz and RAM of 8.00GB.

Statistical analysis

The statistical analysis was performed with SPSS 21.0 software. The normality of the data were analyzed by the Kolmogorov-Smirnov normality test to ensure they were normally distributed before variance analysis, the differences in gender distribution between groups of ABE and non-ABE neonates were evaluated by the Chi-square test, while the differences in other clinical features distribution (Normal Distribution) were evaluated by two-tailed T-test. The differences with P < 0.05 were regarded as statistically significant. The result of descriptive statistics is presented in the form of “mean ± standard deviation”.

Result

Table 1 presents the baseline demographics and clinical characteristics of the participants for each study group. Among these characteristics, gender and TSB level are significantly different between the two groups (P < 0.005).

Table 1.

Baseline demographic and clinical characteristics of the patients.

Characteristic	ABE (n = 75)	Non-ABE (n = 75)	p-value
Gender (male)	47 (62.67%)	50 (66.67%)	0.608
Age (days)	10.08 ± 3.64	12.37 ± 5.62	0.004*
Weight (kg)	3.26 ± 0.43	3.38 ± 0.43	0.083
Gestational age at birth (wks)	38.54 ± 1.49	38.58 ± 1.26	0.848
Postmenstrual age at scan (wks)	39.98 ± 1.52	40.35 ± 1.57	0.144
Total serum bilirubin level (mg/dL)	23.45 ± 8.04	18.42 ± 3.71	0.000*
Albumin level (g/L)	38.47 ± 2.99	37.93 ± 2.94	0.266

The results of Sect. 2.4 are shown in Fig. 5; Table 2, and Fig. 6a. Specifically, Table 2 also compares the results of the current study with those from previous studies¹², demonstrating that the proposed methods in this study achieved substantial improvements over prior findings. Figure 5 shows the distribution maps of each single-modal MRI feature, from which we observed a considerable overlap of each single-modal MRI feature in the 2 groups, indicating the difficulty of differentiating the 2 groups’ patients with handcrafted features. Table 2 indicates that the best accuracy of 0.7932, the best F1-score of 0.8884, and the best sensitivity of 0.9494 are achieved by the SVM with fusion features of GP_{norm, T1WI} and GP_{norm, T2WI}. The best AUC of 0.6547 is achieved by the SVM with fusion features of GP_{norm, T2WI} and GP_{norm, ADC}. The best precision of 0.8851 and specificity of 0.6203 is achieved by SVM with GP_{norm, ADC}. The accuracy of 0.7511, 0.7765, 0.6793, and AUC of 0.6057, 0.6464,0.4876 were found for GP_{norm, T1WI}, GP_{norm, T2WI}, and GP_{norm, ADC}, respectively, with optimal cut-off values of 0.1431,1.4352, and 0.7143 determined by the maximum Youden index^28,29. Figure 6a shows the ROC (receiver operating characteristic) curves with AUC values of the SVM classifier that performed on the classification task of handcrafted fusion features of the patients. The best AUC of 0.6547 is achieved by the SVM classifier based on the fusion features of GP_{norm, T2WI} and GP_{norm, ADC}. These results indicated that though the SVM with a single modality MRI feature has limited predictive ability, their combination features improved the performance.

Fig. 5.

Comparisons of handcrafted MRI features between ABE and non-ABE neonates in terms of (a) GP_{norm, T1WI,} (b) GP_{norm, T2WI,} and (c) GP_{norm, ADC}. Especially, The value of GP_{norm, T1WI} is 1.345 ± 0.062 for ABE neonates and 1.405 ± 0.126 for non-ABE neonates, which are significantly different in the two groups(P < 0.05). The value of GP_{norm, T2WI} is 1.342 ± 0.059 for ABE neonates and 1.426 ± 0.146 for non-ABE neonates, which are significantly different in the two groups (P < 0.05). The value of GP_{norm, ADC} is 0.767 ± 0.050 for ABE neonates and 0.774 ± 0.056 for non-ABE neonates, which are not significantly different in the two groups (P > 0.05).

Table 2.

The performance metrics of SVM, EfficientNetB0, and InceptionV3on classifying ABE and non-ABE MRI images.

Classifier	MRI modality	Accuracy	AUC	F1-score	Precision	Sensitivity	Specificity
SVM	T1WI	0.7511	0.6057	0.8631	0.7941	0.9451	0.4177
	T2WI	0.7764	0.6464	0.8843	0.8551	0.9156	0.5823
	ADC	0.6793	0.4876	0.8437	0.8851	0.8059	0.6203
	T1WI + T2WI	0.7932	0.6230	0.8884	0.8348	0.9494	0.5316
	T1WI + ADC	0.7468	0.6178	0.8622	0.7987	0.9367	0.4304
	T2WI + ADC	0.7806	0.6547	0.8861	0.8548	0.9198	0.5823
	T1WI + T2WI + ADC	0.7848	0.6395	0.8849	0.8351	0.9409	0.5316
EfficientNetB0 (5FCV)	T1WI	0.7844 ± 0.0334	0.8742 ± 0.0364	0.7568 ± 0.0534	0.8622 ± 0.0509	0.6844 ± 0.0957	0.8844 ± 0.0474
	T2WI	0.7422 ± 0.0163	0.8116 ± 0.0181	0.7646 ± 0.0192	0.7047 ± 0.0249	0.8400 ± 0.0586	0.6444 ± 0.0613
	ADC	0.7133 ± 0.0301	0.7864 ± 0.0464	0.7261 ± 0.0335	0.6967 ± 0.0360	0.7644 ± 0.0765	0.6622 ± 0.0775
	T1WI + T2WI	0.8667 ± 0.0329	0.9439 ± 0.0214	0.8602 ± 0.0371	0.9007 ± 0.0377	0.8267 ± 0.0635	0.9067 ± 0.0431
	T1WI + ADC	0.8311 ± 0.0293	0.9009 ± 0.0307	0.8358 ± 0.0304	0.8120 ± 0.0266	0.8622 ± 0.0475	0.8000 ± 0.0314
	T2WI + ADC	0.8889 ± 0.0211	0.9541 ± 0.0138	0.8972 ± 0.0168	0.8407 ± 0.0421	0.9644 ± 0.0267	0.8133 ± 0.0590
	T1WI + T2WI + ADC	0.9022 ± 0.0257	0.9831 ± 0.0069	0.8934 ± 0.0286	0.9795 ± 0.0299	0.8222 ± 0.0398	0.9822 ± 0.0259
InceptionV3 (5FCV)	T1WI	0.7578 ± 0.0761	0.8930 ± 0.0274	0.6980 ± 0.1343	0.8694 ± 0.0470	0.6222 ± 0.2218	0.8933 ± 0.0836
	T2WI	0.7311 ± 0.0590	0.8110 ± 0.0435	0.7302 ± 0.0979	0.7226 ± 0.0494	0.7689 ± 0.1736	0.6933 ± 0.1162
	ADC	0.7378 ± 0.0376	0.8381 ± 0.0137	0.7316 ± 0.0718	0.7402 ± 0.0298	0.7467 ± 0.1581	0.7289 ± 0.0926
	T1WI + T2WI	0.8467 ± 0.0489	0.9854 ± 0.0074	0.8155 ± 0.0704	0.9946 ± 0.0108	0.6978 ± 0.1091	0.9956 ± 0.0089
	T1WI + ADC	0.8711 ± 0.0341	0.9528 ± 0.0194	0.8636 ± 0.0474	0.9023 ± 0.0428	0.8400 ± 0.1101	0.9022 ± 0.0573
	T2WI + ADC	0.8867 ± 0.0368	0.9536 ± 0.0111	0.8855 ± 0.0410	0.8888 ± 0.0430	0.8889 ± 0.0843	0.8844 ± 0.0586
	T1WI + T2WI + ADC	0.9622 ± 0.0286	0.9978 ± 0.0015	0.9617 ± 0.0298	0.9651 ± 0.0280	0.9600 ± 0.0495	0.9644 ± 0.0301

Bold value represents the best value of the metric of the model.

Fig. 6.

ROC curves with values of AUC for different classifiers on multi-modal MRI images in terms of (a) SVM (b) EfficientNetB0 (c) InceptionV3.

The results of Sect. 2.5 are shown in Table 2; Fig. 6b,c. Table 2 shows the detailed performance of EfficientNetB0 and InceptionV3 throughout the 5FCV. The best Accuracy, AUC, and F1-score of EfficientNetB0 are 0.9022 ± 0.0257, 0.9837 ± 0.0069, and 0.8934 ± 0.0286. Meanwhile, the best Accuracy, AUC, and F1-score of InceptionV3 are 0.9622 ± 0.0286, 0.9975 ± 0.0015, 0.9617 ± 0.0298. Notably, these best quantitative metrics were all obtained when the CNNs’ input is fusion images of T1WI + T2WI + ADC. T-test indicated that almost all the metrics of InceptionV3, i.e., accuracy, F1-score, and sensitivity, were significantly greater than those of EfficientNetB0 (P < 0.05). DeLong’s test indicated that AUC are also significantly greater than those of EfficientNetB0 (P < 0.005). We also found that the CNNs’ performance on any 2-modality MRI fusion images is significantly better than that on any single-modality MRI fusion images, and CNNs’ performance on 3-modality MRI fusion images is significantly better than that on any 2-modality MRI fusion images (P < 0.05). Among the single-modal MRI images, the best accuracy of 0.7844, 0.7578, and the best AUC of 0.8742, 0.8930 are achieved by EfficientNetB0 and InceptionV3 on the T1WI, respectively, which again approved that the T1WI plays an important role in the diagnosis of neonate ABE. Among the 2-modality MRI images, the best accuracy of 0.8889, 0.8867, and the best AUC of 0.9541, 0.9536 are achieved by EfficientNetB0 and InceptionV3 on the fusion images consisting of T2WI + ADC, suggesting the T2WI and ADC map has complementary information on the diagnosis of neonate ABE. Correspondingly, Fig. 6b,c presented the ROC curves with AUC values of the CNNs, that performed on the different fusion images for a loop of 5FCV.

Discussion

Efficiently detecting ABE in neonates remains a challenge in clinical practice nowadays. In this study, we explored computer-aided diagnosis(CAD) methods based on machine learning and multi-modal MRI images to overcome the challenge. Our study has demonstrated that the traditional machine learning method (SVM), has a limited performance on handcrafted features, i.e., the best accuracy is 0.7932 and the best AUC is 0.6547. We deemed that these results were compromised because the handcrafted features (GP_{norm, T1WI}, GP_{norm, T2WI}, GP_{norm, ADC}) have limited diagnostic information, and other potentially useful information contained in MRI images, such as T1-signal of subthalamic nuclei, hippocampus, etc^30–32. have been neglected. Therefore, we then used whole MRI image data as the input of CNNs, and CNNs could automatically extract the image features after the training, and then classify the ABE images so that the potentially useful information for diagnosing ABE would be considered by CNNs. From Table 2; Fig. 6, we can easily see that the performance of CNNs with multi-modal MRI images is better than that of SVM with handcrafted features. The class activation map (CAM) is a kind of technique to get discriminative image regions used by CNNs to identify a specific class in the image³³. Fig. 7 shows the CAM of InceptionV3 for identifying 6 ABE fusion images consisting of T1WI, T2WI, and ADC maps. Since the region with the darker red color in CAM, the more contribution it made in the CNNs’ classification work, in Fig. 7 we can see that most regions with dark red color are in the place where globus pallidus, subthalamic nuclei, and hippocampus are located, indicating that the MRI signal intensities of these brain regions directly affect the CNNs’ prediction result, which is consistent with previous studies^30–32.

Fig. 7.

CAM of InceptionV3 for identifying 6 ABE fusion images (T1WI + T2WI + ADC). The colormap showed the contribution degree of the pixels in predicting the result with InceptionV3 and the region covered by dark red color contributes the most.

Another important finding in our study is that the performance of machine learning methods with multi-modal MRI data is significantly better than that with signal-modal MRI data (P < 0.05). In other words, with more modality MRI data involved in the study, the performance of the CNNs is gradually improved, which can easily be seen in Table 2; Fig. 6. The CNNs of InceptionV3 outperformed EfficientNetB0 and achieved the best accuracy of 0.9622, the best AUC of 0.9978, and the best F1-score of 0.9617 on the fusion images of T1WI + T2WI + ADC, which again proved that T1WI, T2WI, and ADC maps are useful for diagnosing ABE, providing complementary information for CNNs to differentiate ABE neonates³⁴.

The limitations of the study are the following: First, though we applied data augmentation, dropout techniques³⁵ and pre-trained CNNs³⁶ to the experiment, the overfitting phenomenon still existed during the training process. As shown in Fig. 8, we can see that the training loss decreased after 260 iterations in the training process of InceptionV3, whereas the validation loss stopped decreasing and even increased, suggesting that overfitting might happen. Overfitting is a phenomenon that usually appears during the training process when a CNNs have a complex structure but its input data is relatively simple. In future studies, we should manage the complexity of the classification model and collect more input data for the model to reduce or even prevent the risk of overfitting. Second, the generalization of the CNNs model is unknown since our data was all collected from one hospital named the Children’s Hospital of Zhejiang University School of Medicine. In future studies, multi-center data is necessary to train the model and improve the model’s generalization. Last but not least, advanced MRI techniques, such as diffusion tensor imaging (DTI), susceptibility-weighted imaging (SWI), etc. are likely to be explored in future studies to further improve the performance of the model.

Fig. 8.

A training progress (the accuracy and loss history) of InceptionV3 in 5FCV with the patients’ fusion images consisting of T1WI, T2WI, and ADC map.

Conclusion

Here, we explored the CAD system for automatically diagnosing neonatal ABE by using multi-modal MRI images and CNNs. Among the classifiers, InceptionV3 achieved the best performance on the fusion images of T1WI + T2WI + ADC with an accuracy of 0.9622, AUC of 0.9978, and F1-score of 0.9617. EfficientNetB0 went second, and SVM got a poor performance on the handcrafted features. The results indicated that the CAD system based on CNNs and multi-modal MRI images has great power in diagnosing neonatal ABE, which is potentially applicable in actual clinical practice to improve the efficiency of doctors’ diagnosis of the disease.

Abbreviations

ABE

Acute bilirubin encephalopathy

HB

Hyperbilirubinemia

CNNs

Convolutional neural networks

T

Tesla

T1WI

T1-weighted image 

T2WI

T2-weighted image

ADC

Apparent diffusion coefficient map

SVM

Support vector machine

TSB

Total serum bilirubin

CBE

Chronic bilirubin encephalopathy

MRI

Magnetic resonance imaging

GP

Globus pallidus

BIND

Bilirubin-induced neurologic dysfunction

TR

Repetition time

TE

Echo time

FOV

Field of view

DWI

Diffusion-weighted image

WM

White matter

ROI

Regions of interest

5FCV

Five-fold cross-validation

SGDM

Stochastic gradient descent with momentum

CAD

Computer-aided diagnosis

DTI

Diffusion tensor imaging

SWI

Susceptibility-weighted imaging

Author contributions

Miao Wu designed the project, performed the experiments and data analysis. Miao Wu and Qian Liu and wrote the manuscript.

This work was sponsored by the Project of Top-notch Talents of Technological Youth of Xinjiang (2023TSYCCX0067), the Natural Science Foundation Xinjiang Uygur Autonomous Region (2022D01C436).

Data availability

The de-identified MRI images and non-image clinical data are available from the corresponding author upon request.

Declarations

Competing interests

The authors declare no competing interests.

Ethics approval and consent to participate

The approval was obtained from the Institutional Review Board (IRB) of Xinjiang Medical University on July 25, 2022. The IRB reference number is XJYKDXR20220725001. The informed consent to participate was obtained from all of the participants’ guardian.

Consent for publication

Written informed consent for publication of their clinical details and clinical images was obtained from the patient’s parents.