Deep Learning-Based Automated Detection and Burden Assessment of Paramagnetic Rim Lesions on Quantitative Susceptibility Mapping in Patients With Multiple Sclerosis

Abstract

Objective

This study aimed to develop a deep learning (DL) framework for automated detection and burden assessment of paramagnetic rim lesions (PRLs) by using quantitative susceptibility mapping (QSM) in multiple sclerosis (MS), compare QSM-only and QSM + fluid-attenuated inversion recovery (FLAIR) configurations, and evaluate the clinical relevance of the automated PRL burden.

Materials and Methods

Brain magnetic resonance imaging data, including QSM and three-dimensional (3D) FLAIR data, obtained between January and December 2025, were retrospectively collected from 106 patients suspected of having MS. The dataset consisted of an exploration set (n = 84)—divided into training (n = 54) and internal test (n = 30) sets—and a temporal test set (n = 22). The PRLs were manually segmented to establish the ground truth. A 3D nnU-Net framework was trained using the QSM-only and QSM + FLAIR configurations for patient-level classification, lesion-level detection, and PRL burden assessment. A separate clinical implication cohort (n = 117) was used to assess the associations between the automated PRL burden and clinical outcomes.

Results

In the exploration set (median age: 40.0 years; 67.9% female), 69 (82.1%) patients were PRL-positive, with a total of 705 PRLs. In the internal test set, the QSM-only model showed higher lesion-level sensitivity (72.4% [147/203] vs. 62.1% [126/203], P = 0.004) and precision (78.2% [147/188] vs. 65.6% [126/192], P = 0.009). In the temporal test set, the lesion-level sensitivity was 45.3% (29/64) for the QSM-only model and 54.7% (35/64) for the QSM + FLAIR model (P = 0.181), whereas both models achieved 100% (8/8) patient-level sensitivity. No significant differences were observed in patient-level classification in either test set. A higher automated PRL burden was associated with poorer cognition (P = 0.002).

Conclusion

QSM-based DL models enabled automated detection and burden assessment of PRLs in MS, with the QSM-only model performing comparably to QSM + FLAIR while offering a simplified single-sequence pipeline. The association between the automated PRL burden and cognitive impairment highlights its potential as a biomarker for MS assessment.

INTRODUCTION

Multiple sclerosis (MS) is a chronic disorder of the central nervous system (CNS) driven by immune-mediated mechanisms that result in demyelination, axonal damage, and progressive neurodegeneration. Among the pathological substrates of MS, chronic active lesions (CALs) are recognized as distinctive indicators of persistent inflammation [1]. These lesions are characterized by iron-rich, chronically activated microglia and macrophages at their margins, a feature that makes them particularly detectable using susceptibility-sensitive magnetic resonance imaging (MRI) [2]. Histopathological studies have demonstrated that the microglial rim in chronic MS lesions is not always continuous; instead, it can appear as focal or linear nodules at lesion borders, where signs of active myelin breakdown are often detectable [3]. These lesions are frequently described as smoldering lesions [1]. Within this spectrum, paramagnetic rim lesions (PRLs) have emerged as an imaging correlate of CALs [1,4].

PRLs are defined by the presence of a hypointense rim on susceptibility-sensitive MRI, corresponding to iron-laden microglia and macrophages at the lesion edges [2]. A growing body of evidence supports the role of PRLs as radiological biomarkers of chronic active inflammation and ongoing tissue injury in MS [2]. The development of advanced MRI-derived metrics has improved the ability to study MS lesion pathology, including the central vein sign, susceptibility-based rims, heterogeneous tissue injury across CNS regions, and iron accumulation [2]. Quantitative susceptibility mapping (QSM), in particular, can provide a quantitative measure of tissue magnetic properties and enhance the detection of PRLs [5]. Nevertheless, the current identification of rim lesions on QSM predominantly depends on expert visual reading, which is time-consuming, varies among raters, and is unsuitable for large-scale or routine clinical use.

These challenges highlight the need for robust automated approaches to identify and measure PRLs. Notably, although several studies have explored automated PRL detection, existing deep learning (DL) approaches rely exclusively on multimodal MRI inputs, combining QSM with T2-fluid-attenuated inversion recovery (FLAIR) or other sequences [6,7,8]. In the present study, we included a QSM + FLAIR configuration based on the rationale that FLAIR, as the standard sequence for visualizing demyelinating lesions, may provide complementary anatomical information that delineates the full extent of MS lesions, thereby constraining the search space for paramagnetic rims. However, this multimodal dependency necessitates complex image registration and introduces potential sources of error into the analysis pipeline. In addition, prior methods employed patch-based or slice-by-slice approaches that required preprocessing steps and may lose spatial context. Therefore, this study aimed to train and validate a DL framework for automated PRL burden assessment using QSM with whole-volume input; compare QSM-only and QSM + FLAIR input configurations; and evaluate the clinical relevance of the DL-derived automated PRL burden.

MATERIALS AND METHODS

Study Participants

This study was approved by the Institutional Review Board of the Asan Medical Center (IRB No. 2025-0073). All participants provided written informed consent in accordance with the Declaration of Helsinki. Participants were identified from a single tertiary academic MS center between January 2025 and December 2025 (Fig. 1). During the recruitment period, 253 individuals with confirmed or suspected MS were screened for eligibility. Of these, 224 met the initial eligibility criteria. The remaining 29 patients were excluded for the following reasons: not fulfilling the pre-defined inclusion criteria (n = 2), declined to participate (n = 3), had incomplete clinical records (n = 1), or had suboptimal image quality that precluded reliable QSM analysis (n = 23).

Fig. 1. Flowchart of the study. Between January and December 2025, 253 patients with multiple sclerosis were assessed for eligibility, and 224 patients were enrolled after excluding 29 patients. The patients enrolled between January and August 2025 (n = 201) included 69 PRL-positive patients identified by expert review. Fifteen PRL-negative patients were selected from the remaining 132 patients as balanced negative controls (matched in number to the PRL-positive subset in the internal test set), together with PRL-positive patients to form the exploration set (n = 84). Patients enrolled between September and December 2025 (n = 23) constituted the temporal test set. After excluding one patient due to MRI artifacts, the final temporal test set consisted of 22 patients. The remaining patients (n = 117) were allocated to the clinical implication cohort. Within the exploration set, the training and internal test sets were determined using a pre-specified two-rule allocation procedure performed before model development. ^*Rule 1 (PRL-positive, n = 69): pre-defined, non-randomized allocation by ranking patients in descending order of total PRL lesion count (ties broken by lesion volume) and systematically assigning regular intervals to the training set (n = 54) and the internal test set (n = 15), ^†Rule 2 (PRL-negative, n = 15): all 15 PRL-negative patients selected as balanced negative controls were assigned to the internal test set. The 1:1 class ratio in the internal test set was a consequence of these rules. PRL = paramagnetic rim lesion, RIS = radiologically isolated syndrome, QSM = quantitative susceptibility mapping.

Expert readers identified 69 PRL-positive patients (705 lesions) among the 224 eligible patients recruited between January and August 2025. Using a consecutive sampling approach, an additional 15 PRL-negative patients were selected from eligible PRL-negative patients within the same enrollment period. The number of PRL-negative patients (n = 15) was pre-specified to match the planned size of the PRL-positive internal test set, thereby enabling a balanced evaluation of model specificity rather than reflecting population-level prevalence. No additional matching or selection criteria were applied, and these patients, together with the PRL-positive group, formed the exploration set (n = 84) for the DL model development (Fig. 1).

Patients recruited between September and December 2025 were assigned to the temporal test set (n = 23). After excluding one patient showing substantial MRI artifacts, the final temporal test set consisted of 22 participants (8 PRL-positive, 14 PRL-negative). The exploration and temporal test sets constituted the development cohort of the DL model. The remaining 117 patients were allocated to the clinical implication cohort (Fig. 1).

Within the exploration set, allocation into the training and internal test sets was performed using a pre-defined, non-randomized, two-rule procedure applied before model development. First, the 69 PRL-positive patients were ranked in descending order of the total PRL lesion count, with ties broken by the total PRL lesion volume. Fifteen patients at regular intervals across the ranked list were then systematically assigned to the internal test set, and the remaining patients were assigned to the training set (n = 54; 502 lesions). This pre-defined, interval-based allocation strategy was designed to achieve comparable distributions of the total PRL burden (lesion count and volume) between the two sets. Second, all 15 PRL-negative patients were assigned to the internal test set to evaluate model specificity, yielding the internal test set (n = 30; 15 PRL-positive and 15 PRL-negative). The resulting 1:1 class ratio in the internal test set was thus a direct consequence of the allocation strategy rather than a post-hoc adjustment. In contrast, the temporal test set was defined independently on the basis of the time of MRI acquisition and was not subjected to the allocation procedures applied to the exploration set.

The inclusion criteria required participants to be adults aged 18–90 years who met the 2017 revised McDonald criteria [9] for the diagnosis of MS and had been receiving disease-modifying therapy continuously for at least six months to ensure a stable immunological and radiological state. Other eligibility criteria were the availability of brain MRI scans acquired using standardized protocols, including three-dimensional (3D) FLAIR and QSM sequences. Individuals with contraindications to MRI, those who were pregnant or lactating, and those with other major neurological disorders, such as dementia or Parkinson disease, were excluded.

MRI Acquisition Parameters

All MRI scans were acquired using a 3T MRI scanner (MAGNETOM VIDA, Siemens Healthineers, Erlangen, Germany) with a 64-channel head coil and 3D-gradient recalled echo (GRE) acquisition. The following sequences were acquired: pre- and post-contrast enhanced 3D T1-weighted images (spatial resolution, 1 × 1 × 1 mm³; field of view, 256 × 240; repetition time, 5,000 ms; echo time, 2.9 ms; time inversions, 700/2,500 ms; flip angle, 4°/5°; scan time, 3 minutes 30 seconds), 2D T2 turbo spin echo (spatial resolution, 0.4 × 0.59 × 4 mm³; field of view, 210 × 210; repetition time, 5,910 ms; echo time, 100 ms; flip angle, 150°; scan time, 2 minutes 17 seconds), 3D FLAIR (spatial resolution, 0.5 × 0.5 × 1 mm³; field of view, 256; repetition time, 5,000 ms; echo time, 328 ms; time inversion, 1,600 ms; flip angle, variable; acceleration factor, 6; scan time, 3 minutes 29 seconds), multi-echo susceptibility-weighted image (spatial resolution, 0.5 × 0.5 × 2 mm³; field of view, 257 × 176; repetition time, 40.7 ms; echo time, 7 echoes between 5.42–36.8 ms; flip angle, 20°; acceleration factor, 2; scan time, 4 minutes 50 seconds).

Processing of Quantitative Susceptibility Mapping

From the multi-echo GRE data, QSM was automatically generated using the Total Generalized Variation (TGV) algorithm. Initially, either a morphological mask or phase-based masking procedure was applied. The TGV algorithm integrates Laplacian phase unwrapping, background field removal, and dipole inversion into a single optimization problem, thereby minimizing the error propagation across sequential steps [10].

Image Analysis

For the training and internal test sets, a radiologist (Y.C., with 13 years of experience) and a neurologist (E.J.L., with 18 years of experience), both blinded to the patients’ clinical information, independently reviewed the FLAIR and QSM images and annotated the PRLs on QSM images using 3D Slicer (v.5.2.2, https://www.slicer.org). Subsequently, they performed a side-by-side consensus review to resolve discrepancies in lesion identification and boundary delineation. The adjudicated segmentation masks from both readers were then combined using a union operation to create the final consensus ground-truth labels. All the segmentations were delineated in accordance with a recently established consensus statement [11]. For the temporal test set, ground-truth annotations were performed by a single neuroradiologist (H.H.M., with 8 years of experience in neuroradiology), reflecting typical clinical practice and enabling the evaluation of model performance under real-world annotation conditions. Lesions with less than 10 voxels in total volume were excluded from the analysis to ensure reliable lesion characterization and minimize partial-volume effects.

MRI Preprocessing

Brain extraction on the FLAIR images was performed using FreeSurfer’s SynthStrip algorithm (mri_synthstrip) [12]. Multimodal registration employs a strategic two-step approach to optimize the registration accuracy. Rather than directly registering FLAIR to QSM data, we utilized co-acquired magnitude images as an intermediate reference because of their superior structural contrast and anatomical boundary definition in comparison with QSM images. FLAIR images were first spatially resampled to match the QSM dimensions using the c3d toolkit [13]. Subsequently, the resampled FLAIR images were registered to the skull-stripped magnitude image using FSL FLIRT [14,15] with a 12-degree-of-freedom affine transformation and mutual information cost function. The derived transformation matrix was then applied to align the FLAIR images into the QSM coordinate space (Fig. 2).

Fig. 2. Schematic overview of MRI preprocessing and deep learning-based segmentation. The workflow illustrates a two-step registration approach using magnitude images as an intermediate reference, followed by 3D nnU-Net-based segmentation with QSM-only or QSM + FLAIR input configurations. QSM = quantitative susceptibility mapping, FLAIR = fluid-attenuated inversion recovery, GRE = gradient recalled echo.

DL Model Architecture and Training

The exploration set (n = 84) consisted of a training set (n = 54; PRL-positive only, 502 lesions) and an internal test set (n = 30; 15 PRL-positive patients with 203 lesions, and 15 PRL-negative patients), as described in the Study participants section. This design ensured a balanced lesion burden between the training and internal test sets while preserving the entire PRL-negative pool for evaluation of model specificity.

To evaluate single-modality versus multimodal approaches, two input configurations were prepared: QSM-only and QSM + FLAIR segmentation. This enabled direct performance comparison while controlling for all other methodological variables.

We employed 3D nnU-Net, which is a self-configuring framework that automatically determines the optimal 3D U-Net specifications [16]. The architecture consisted of six encoding-decoding stages with progressive feature expansion (32 → 64 → 128 → 256 → 320), utilizing 3 × 3 × 3 kernels (1 × 3 × 3 in the first stage), instance normalization, and LeakyReLU activation. A patch size of 40 × 256 × 192 voxels was automatically optimized for graphics processing unit (GPU) memory constraints. Preprocessing included automatic z-score normalization of the foreground voxels to achieve zero mean and unit variance. The models were then trained using 5-fold cross-validation (80% training and 20% validation per fold) with combined Dice loss and cross-entropy loss with deep supervision, an SGD optimizer with Nesterov momentum (0.99), and polynomial learning rate scheduling over 1,000 epochs. The training employed 33% foreground oversampling to address class imbalances and standard medical imaging augmentations.

Performance Evaluation

For inference, predicted segmentation masks were obtained by applying the default softmax probability threshold of 0.5 at the voxel level, as implemented in the nnU-Net framework, without additional threshold optimization. Individual lesions were then identified through 3D connected component analysis of the binarized masks. A patient was classified as PRL-positive if at least one predicted lesion was present. The PRL burden was defined as the total count and cumulative volume of predicted lesions per patient. These criteria were fixed across the internal and temporal test sets.

Model performance was assessed at both the patient and lesion levels. Patient-level metrics included sensitivity, specificity, precision, accuracy, and F1-score to distinguish between PRL-positive and PRL-negative patients. Receiver operating characteristic (ROC) curve analysis was also performed. For evaluation of lesion-level detection, the predicted lesions were spatially matched to the ground-truth lesions using 0.3 ≥ intersection over union (IoU) as the matching criterion. The lesion-level metrics included sensitivity, precision, and F1-score for lesion detection. In addition, the detection performance was characterized using free-response ROC (FROC) analysis by sweeping the model’s softmax probability threshold from 0.0 to 1.0, computing the lesion-level sensitivity and the average number of false positives per patient at each operating point.

The IoU threshold of 0.3 for the analysis of lesion-level detection performance was selected because 3D volumetric detection inherently yields lower IoU values than 2D detection owing to the additional spatial dimension [17] and because small lesions can exhibit disproportionately large reductions in IoU from minor spatial offsets, making the conventional 2D threshold of 0.5 overly stringent for lesions with high size variability [18]. A predicted lesion was considered a true positive if its IoU with any ground-truth lesions met this threshold. The proportion of predicted lesions with IoU < 0.3 was additionally reported to characterize the extent of poorly matched detections. For successfully matched lesions, the segmentation quality was further analyzed using Dice similarity coefficient (DSC) and additional metrics (IoU, 95th percentile Hausdorff distance, and volume differences). The implementation code used in this study is available at https://github.com/olnsnlyy/PRL_segmentation.

The PRL burden assessment accuracy was evaluated by comparing the model-predicted and ground-truth PRL counts per patient, derived from the same binarized segmentation outputs, using the mean absolute error (MAE) and Pearson correlation.

Statistical Analysis

Statistical comparisons between the two models were performed on the basis of the data type. McNemar’s test was used to compare the patient-level sensitivity, specificity, accuracy, and lesion-level sensitivity. The chi-square test was used for precision comparisons. For segmentation quality metrics of paired matched lesions, the normality of paired differences was assessed using the Shapiro–Wilk test, and Wilcoxon signed-rank tests were applied because of the non-normal distribution of differences. To investigate the clinical relevance of the PRL burden, participants were stratified according to model-predicted PRL count, derived from the same binarized segmentation outputs described in the Performance Evaluation section: 0 (Group 1), 1–3 (Group 2), and ≥4 (Group 3). Age-adjusted associations between the PRL burden and clinical outcomes, including cognitive performance assessed using the Symbol Digit Modalities Test (SDMT) and physical disability measured using the Expanded Disability Status Scale (EDSS), were examined using analysis of covariance (ANCOVA), with age included as a covariate. Where significant group effects were observed, post-hoc pairwise comparisons were performed using Tukey’s test. Complementary partial Spearman correlation analyses were conducted to evaluate the continuous associations between the PRL burden and clinical scores (SDMT and EDSS), adjusting for age, sex, and disease duration as covariates. All tests were two-tailed, with statistical significance set at P < 0.05.

RESULTS

Characteristics of the Study Participants

A total of 84 participants were enrolled in the exploration set, which was subsequently divided into a training set (n = 54) and an internal test set (n = 30), with an additional 22 participants constituting the temporal test set (Table 1). The cohort was predominantly female (67.9%), with a median age of 40.0 years, low disability (median EDSS score, 2.0), and relapsing-remitting multiple sclerosis (RRMS) as the predominant disease course (97.6%). Baseline clinical characteristics were comparable between the exploration and temporal test sets, as well as between the training and internal test subsets, except that the patients in the internal test subset were slightly older (median age, 44.0 vs. 38.0 years; P = 0.049). Among the 84 participants, 69 (82.1%) were classified as PRL-positive (≥1 PRL) at baseline. A total of 705 PRLs were identified across these individuals, with a mean lesion volume of 397.9 ± 675.5 mm³. Lesions were predominantly located in the juxtacortical (57.5%) and periventricular (36.6%) regions; detailed lesion characteristics are provided in Table 2.

Table 1. Baseline characteristics of the included patients.

		Exploration (n = 84)			Temporal test (n = 22)	P *	P †
		Overall (n = 84)	Train (n = 54)	Internal test (n = 30)
Age, yrs		40.0 [31.0–47.0]	38.0 [30.0–45.0]	44.0 [36.0–49.0]	42.5 [32.0–56.0]	0.332	0.049
Sex, female		57 (67.9)	41 (75.9)	16 (53.3)	12 (54.5)	0.360	0.060
Disease duration, yrs		8.0 [3.0–15.0]	7.0 [3.0–14.0]	8.5 [3.0–16.0]	9.0 [4.0–15.0]	0.842	0.338
EDSS		2.0 [1.2–2.5]	2.0 [1.0–2.0]	2.0 [1.5–3.0]	2.2 [1.0–3.0]	0.289	0.065
Disease course						0.400	0.241
	Relapsing-remitting MS	82 (97.6)	54 (100)	28 (93.3)	20 (90.9)
	Progressive MS	2 (2.4)	0 (0)	2 (6.7)	2 (9.1)

Data are presented as median [interquartile range] for continuous variables and as number (%) for categorical values. P-value were calculated using Wilcoxon rank-sum test for continuous variables and the chi-square test for categorical variables.

^*Exploration vs. Temporal test, ^†Exploration train vs. Internal test.

EDSS = Expanded Disability Status Scale, MS = multiple sclerosis

Table 2. Descriptive information of paramagnetic rim lesions in the exploration set.

		Overall (n = 84)	Train (n = 54)	Internal test (n = 30)
Lesion count		705	502	203
Lesion location
	Periventricular	258 (36.6)	188 (37.5)	70 (34.5)
	Juxtacortical	405 (57.5)	284 (56.6)	121 (59.6)
	Infratentorial	8 (1.1)	7 (1.4)	1 (0.5)
	Deep white matter	34 (4.8)	23 (4.6)	11 (5.4)
Lesion volume, mm3		397.9 ± 675.5	415.7 ± 690.7	353.8 ± 634.1

Data are presented as number of lesions with percentages in parentheses or mean ± standard deviation. No significant differences were found between the train and internal test datasets

Model Performance in the Internal Test Set

Patient-Level Classification Performance

Table 3 lists the performance metrics of both DL models in the internal test set. Both models achieved perfect patient-level sensitivity (100%). The QSM-only and QSM + FLAIR models demonstrated comparable specificity (86.7% vs. 66.7%, P = 0.371), precision (88.2% vs. 75.0%, P = 0.546), and accuracy (93.3% vs. 83.3%, P = 0.371), with no statistically significant differences. The ROC curve analysis (Fig. 3A) showed consistent results.

Table 3. Performance of the deep learning models in the internal test set.

Metric		QSM-only model	QSM + FLAIR model	P
Patient-level classification (n = 30 patients)
	Sensitivity	100 (15/15)	100 (15/15)
	Specificity	86.7 (13/15)	66.7 (10/15)	0.371
	Precision	88.2 (15/17)	75.0 (15/20)	0.546
	Accuracy	93.3 (28/30)	83.3 (25/30)	0.371
	F1-score	93.8	85.7
Lesion-level detection (n = 15 patients, 203 lesions)
	Sensitivity	72.4 (147/203)	62.1 (126/203)	0.004
	Precision	78.2 (147/188)	65.6 (126/192)	0.009
	F1-score	75.2	63.8
Lesion-level segmentation quality
	DSC	0.780 ± 0.133	0.772 ± 0.130	0.026

Data are percentage values with the raw numbers in parentheses, except for DSC presented as mean ± standard deviation. McNemar’s test was used for sensitivity, specificity, and accuracy comparisons. Chi-square test was used for precision. Wilcoxon signed-rank test was applied for segmentation quality metrics. DSC values are reported for all matched lesions per model (n = 147 for QSM-only, n = 126 for QSM + FLAIR). Statistical comparisons were performed on lesions matched by both models.

DSC = Dice similarity coefficient, QSM = quantitative susceptibility mapping, FLAIR = fluid-attenuated inversion recovery

Fig. 3. Performance evaluation on the internal test set. A: ROC curves demonstrating patient-level classification performance. B: FROC curves illustrating lesion-level detection performance, showing the trade-off between sensitivity and average false positives per patient. ROC = receiver operating characteristic, FROC = free-response receiver operating characteristic, QSM = quantitative susceptibility mapping, AUC = area under the curve, CI = confidence interval, FLAIR = fluid-attenuated inversion recovery.

Lesion-Level Detection Performance

For lesion-level analysis of the 203 lesions in the internal test set, the QSM-only model detected 147 lesions in comparison with 126 lesions detected by the QSM + FLAIR model, showing significantly higher sensitivity (72.4% vs. 62.1%, P = 0.004) and precision (78.2% vs. 65.6%, P = 0.009). The FROC curves are shown in Figure 3B.

PRL Burden Assessment

In the internal test set, where the consensus-based ground truth was available, the QSM-only model achieved a PRL count MAE of 1.03 per patient with a correlation coefficient of 0.986 between the predicted and ground-truth counts, in comparison with an MAE of 1.23 and a correlation coefficient of 0.989 for the QSM + FLAIR model; these findings indicated reliable PRL burden estimation by both models.

Segmentation Quality of the Matched Lesions

Of 203 ground-truth lesions, 147 (72.4%) were matched with IoU ≥ 0.3 by the QSM-only model compared to 126 (62.1%) by the QSM + FLAIR model. The proportion of predicted lesions with an IoU < 0.3 was 21.8% for the QSM-only model and 34.4% for the QSM + FLAIR model. Among the matched lesions, the QSM-only model achieved a DSC of 0.780 ± 0.133 compared to 0.772 ± 0.130 for the QSM + FLAIR model (P = 0.026) (Fig. 4). Additional segmentation metrics are presented in Supplementary Table 1.

Fig. 4. Representative segmentation examples from the internal test set. Example cases showing ground-truth annotations (red), QSM-only model predictions (yellow), and QSM + FLAIR model predictions (blue) overlaid on QSM and FLAIR images, demonstrating comparative segmentation performance. QSM = quantitative susceptibility mapping, FLAIR = fluid-attenuated inversion recovery.

Model Performance in the Temporal Test Set

Model performance was evaluated in the temporal test set (n = 22; eight PRL-positive and 14 PRL-negative patients) (Table 4). Both models maintained perfect patient-level sensitivity (100%). No statistically significant difference in specificity (78.6% vs. 64.3%, P = 0.157) or accuracy (86.4% vs. 77.3%, P = 0.157) was observed between the QSM-only and QSM + FLAIR models.

Table 4. Performance of the deep learning models in the temporal test set (n = 22).

Metric		QSM-only model	QSM + FLAIR model	P
Patient-level classification (n = 22 patients)
	Sensitivity	100 (8/8)	100 (8/8)
	Specificity	78.6 (11/14)	64.3 (9/14)	0.157
	Precision	72.7 (8/11)	61.5 (8/13)	0.480
	Accuracy	86.4 (19/22)	77.3 (17/22)	0.157
	F1-score	84.2	76.2
Lesion-level detection (n = 8 patients, 64 lesions)
	Sensitivity	45.3 (29/64)	54.7 (35/64)	0.181
	Precision	30.2 (29/96)	32.7 (35/107)	0.817
	F1-score	36.2	40.9
Lesion-level segmentation quality
	DSC	0.669 ± 0.123	0.651 ± 0.113	0.401

Data are percentage values with the raw numbers in parentheses, except for DSC presented as mean ± standard deviation. Statistical comparisons followed the same methodology as the internal test set. DSC values are reported for all matched lesions per model (n = 29 for QSM-only, n = 35 for QSM + FLAIR). Statistical comparisons were performed on lesions matched by both models.

DSC = Dice similarity coefficient, QSM = quantitative susceptibility mapping, FLAIR = fluid-attenuated inversion recovery

At the lesion level, the performance was lower than that in the internal test set. Of the 64 ground-truth lesions, the QSM-only model detected 29 with 45.3% sensitivity, whereas the QSM + FLAIR model detected 35 with 54.7% sensitivity (P = 0.181). A visual review of the unmatched model predictions revealed that 21 lesions in 10 patients exhibited clear paramagnetic rim characteristics on QSM, suggesting that these may represent true PRLs missed by the single-reader annotation (Fig. 5). These lesions were counted as apparent false positives owing to an imperfect ground truth. The proportion of predicted lesions with IoU < 0.3 was 69.8% for the QSM-only model and 67.3% for the QSM + FLAIR model. For the matched lesions, DSC showed no significant differences between models (0.669 ± 0.123 vs. 0.651 ± 0.113, P = 0.401). The PRL burden estimation yielded an MAE of 1.64 with a correlation coefficient of 0.939 for the QSM-only model and an MAE of 1.95 with a correlation coefficient of 0.967 for the QSM + FLAIR model.

Fig. 5. Representative examples from the temporal test set showing model predictions versus single-reader annotations. Examples show correct detections, false negatives, and model detections exhibiting clear rim characteristics that were not annotated by a single neuroradiologist. QSM = quantitative susceptibility mapping, FLAIR = fluid-attenuated inversion recovery, DL = deep learning.

Clinical Correlations With PRL Burden

After technical validation, the clinical relevance of the PRL burden was assessed in the clinical implication cohort (n = 117). The PRL burden groups were defined using the same binarized segmentation outputs described in the Performance Evaluation section. Clinical metrics were compared across the three PRL groups [5] in the clinical cohort by using age-adjusted ANCOVA, which revealed no significant differences in physical disability (EDSS scores) across the groups, whereas cognitive function (SDMT scores) differed significantly according to the PRL burden (Fig. 6; Supplementary Table 2). Post-hoc analysis demonstrated that Group 3 (≥4 PRLs) had significantly lower SDMT scores than Group 1 (no PRLs) (P = 0.002), indicating that greater PRL accumulation is independently associated with poorer cognitive outcomes.

Fig. 6. Clinical disability across PRL burden groups in the clinical implication cohort. A, B: Box plots depicting SDMT scores across PRL burden groups defined by total PRL count (no PRL, 1–3 PRLs, and ≥4 PRLs). Individual-level data points are presented. Group differences were assessed using an ANCOVA with age. The diamonds indicate age-adjusted estimated marginal means with 95% confidence intervals. Pairwise comparisons were performed using Tukey-adjusted post-hoc tests. PRL = paramagnetic rim lesion, SDMT = Symbol Digit Modalities Test, ANCOVA = analysis of covariance, EDSS = Expanded Disability Status Scale.

Individual-level partial Spearman correlation analyses adjusted for age, sex, and disease duration corroborated these findings (Supplementary Figs. 1, 2). In the clinical cohort, PRL burden was not significantly associated with EDSS scores but showed a significant negative association with SDMT scores for both PRL count and volume (P = 0.007). In the DL cohort, no significant associations were identified for either outcome in the exploration or temporal test sets, although consistent negative trends were observed for SDMT scores (exploration set: count, P = 0.097; volume, P = 0.073; temporal test set: count, P = 0.147; volume, P = 0.176), likely reflecting the limited statistical power in these subsets.

DISCUSSION

This study trained and validated a DL segmentation model for automated detection and burden assessment of PRLs using QSM in patients with MS. Both the QSM-only and QSM + FLAIR models demonstrated robust patient-level classification performance across the internal and temporal test sets, with no statistically significant differences between the two models. At the lesion level, the QSM-only model achieved significantly higher sensitivity and precision in the internal test set, whereas the results were reversed without reaching statistical significance in the temporal test set. Overall, the QSM-only model showed performance comparable to the QSM + FLAIR model, suggesting that FLAIR may not be essential for automated PRL assessment. Clinical correlation analysis demonstrated that the automated PRL burden assessment captured clinically relevant associations with cognitive performance, supporting its potential value as an imaging-based marker of disease-related cognitive involvement in MS.

Our study presents three key technical advances that distinguish this work from the existing approaches for PRL analysis. First, we developed an end-to-end whole-brain analysis framework that processes entire QSM volumes directly, contrasting previous patch-based methods [6,7,8]. This approach eliminated preprocessing bottlenecks and provided a comprehensive spatial context for lesion detection. Second, this is the first demonstration of QSM-only PRL segmentation with clinically acceptable performance. Previous approaches assumed that susceptibility contrast alone was insufficient for precise boundary delineation and relied exclusively on multimodal inputs [6,7]. Our findings suggest that the PRL signatures in QSM contain sufficient discriminative information for accurate segmentation without additional contrast mechanisms. Third, our unified framework performed simultaneous detection and segmentation, avoiding the inconsistencies inherent in sequential approaches where separate pipelines can produce misaligned outputs. Taken together, the comparable performance of the QSM-only model indicates that susceptibility contrast alone provides sufficient information for PRL segmentation. The practical advantage of this approach lies in its simplified pipeline, which eliminates the need for inter-sequence registration and thereby reduces potential sources of errors.

The results of the temporal test set merit careful interpretation. While patient-level classification remained excellent (100% sensitivity), lesion-level performance was lower than that of the internal test set. A critical methodological difference may explain this discrepancy: the temporal test set utilized single-reader annotations reflecting routine clinical practice, whereas the training and internal test sets employed consensus annotations from two independent experts. This reflects a fundamental challenge in medical imaging: single-reader annotations have known inter-rater variability, and subtle or ambiguous lesions may be missed by individual readers. Our models, trained on consensus labels representing a more comprehensive “gold standard,” may detect PRLs that a single clinical reader may overlook. This hypothesis was supported by a visual review of model predictions in the temporal test set, which revealed that 21 of 67 unmatched predictions (in 10 patients) exhibited clear paramagnetic rim characteristics on QSM, suggesting that they represented true PRLs missed by the single reader rather than false positives. However, because these lesions were not adjudicated through consensus re-annotation, they were conservatively counted as false positives in the primary analysis. Thus, formal consensus re-annotation of the temporal test set represents an important future direction to distinguish true model sensitivity gains from actual false positives and to establish appropriate confidence thresholds for clinical deployment. Importantly, lesion-level sensitivity may vary in real-world settings depending on annotation protocols and reader experience.

Beyond detection, the PRL burden assessment demonstrated high agreement between the model-predicted and ground-truth PRL counts, with the QSM-only model achieving an MAE of 1.03 per patient (r = 0.986) in the internal test set and 1.64 per patient (r = 0.939) in the temporal test set. Given that the PRL burden has been associated with greater disability, increased relapse risk, more rapid disease progression, and non-relapsing biological progression [5,11] and that PRL dynamics reflect treatment response in longitudinal settings [19,20,21], reliable automated quantification of the PRL burden is clinically relevant. These outputs may facilitate more efficient applications in both clinical decision-making and research participant stratification while mitigating the time burden of manual delineation and reducing inter-rater variability. In the context of growing evidence of rim heterogeneity with distinct prognostic implications [4,11,22], segmentation-based outputs may also support more refined phenotypic characterization [23]. Although standardized methodologies for PRL identification, burden assessment, and longitudinal monitoring remain lacking [11], our model may provide a more standardized approach for assessing the PRL burden.

Despite these methodological strengths, the study had several limitations related to the design and scope of the evaluation that should be acknowledged. First, the evaluation was cross-sectional and primarily focused on technical detection performance. We did not determine whether automated PRL indices, at the lesion or patient level, are associated with longitudinal outcomes such as changes in EDSS scores, relapses, or progression independent of relapse activity. Although the clinical implication cohort independently demonstrated significant associations between the PRL burden and clinical measures, attempts to replicate these findings in the DL cohort were limited by insufficient statistical power owing to smaller subset sizes. Second, development and testing were conducted at a single center with a modest sample size and a predominance of cases showing RRMS, which limited the generalizability of the findings across disease stages (including secondary progressive MS and primary progressive MS), scanner types, and acquisition protocols. Third, the specificity for non-MS demyelinating conditions (e.g., tumefactive demyelinating lesions, neuromyelitis optica spectrum disorder, and myelin oligodendrocyte glycoprotein antibody–associated disease) was not examined, leaving the applicability of these findings in differential diagnosis settings uncertain.

In conclusion, we developed and validated a DL-based automated system for PRL classification, detection, and burden assessment using QSM in patients with MS. The QSM-only model performed comparably to the QSM + FLAIR model while offering a simplified analysis pipeline. In an independent clinical cohort, patients with a higher PRL burden had poorer cognitive performance. This automated DL-based approach may improve the efficiency and standardization of PRL assessments, thereby supporting both clinical decision-making and research applications in MS management.

Availability of Data and Material

The datasets generated or analyzed during the study are available from the corresponding author on reasonable request.

Supplement

The Supplement is available with this article at https://doi.org/10.3348/kjr.2026.0103.