. Author manuscript; available in PMC: 2024 May 14.

Published in final edited form as: Mult Scler Relat Disord. 2023 Sep 14;79:104968. doi: 10.1016/j.msard.2023.104968

Table 2.

Reliability of paramagnetic rim lesions reported in the literature.

Study	Field Strength	Image type	Sample used for reliability	Intra- or inter-rater reliability?	Quantity measured	Reliability metric	Reliability value
Hagemeier et al. (2012)²⁵	3T	Phase	5 pwMS and 5 HCs	Intra (scan-rescan)	Per-subject number	ICC	1
Hagemeier et al. (2012)²⁵	3T	Phase	5 pwMS and 5 HCs	Intra (scan-rescan)	Per-subject volume	ICC	0.999
Kuchling et al. (2014)⁶^*	7T	T2*w	10 representative cases (7 pwMS patients and 3 HCs)	Intra	Per-subject number	ICC as a two-way mixed test for average measures	0.854
Kuchling et al. (2014)⁶^*	7T	T2*w	10 representative cases (7 pwMS patients and 3 HCs)	Inter	Per-subject number	ICC as a two-way mixed test for average measures	0.796
Sinnecker et al. (2016)³⁵^*	7T	Phase	10 randomly selected pwMS or pwNMOSD	Inter	Per-subject number	ICC as a 2-way mixed test of average measures	0.96
Dal-Bianco et al. (2017)^{27, 28}	7T	FLAIR-SWI	15 rim or non-rim lesions segmented twice	Intra	Per-lesion volume	ICC	0.998
Absinta et al. (2018)⁴^*	3T	Phase	100 lesions from 20 pwMS (16 RRMS, 4 PMS), 5 lesions per pwMS	Inter	Per-lesion rim presence	Fleiss κ	0.71
	3T			Intra	Per-lesion rim presence	Cohen’s κ	0.77
	7T			Inter	Per-lesion rim presence	Cohen’s κ	0.72
	7T			Intra	Per-lesion rim presence	Cohen’s κ	0.77
Barquero et al. (2020)³⁹^*	3T	Phase	124 pwMS (87 RRMS, 21 SPMS, 16 PPMS)	Inter	Per-lesion rim presence	Cohen’s κ	0.73
Clarke et al. (2020)³⁴^*	3T	SWI	25 randomly-chosen scans (from 112 pwCIS and 35 non-MS)	Inter	Per-subject number	ICC (2-way mix model, single measures, absolute agreement)	0.84 (0.64-0.93) (95% CI)
Maggi et al. (2020)¹¹^*	3T	Phase	83 non-MS cases and 83 pwMS	Inter	Per-lesion rim presence	Cohen’s κ	0.79
Dal-Bianco et al. (2021)²⁷^*	7T	FLAIR-SWI	15 randomly selected PRLs	Intra	Per-lesion volume	Dice	0.92 ± 0.05 (mean ± s.d.)
Dal-Bianco et al. (2021)²⁷^*	7T	MP2RAGE	15 randomly selected PRLs	Intra	Per-lesion volume	Dice	0.90 ± 0.02 (mean ± s.d.)
Kolb et al. (2021)³³^*	7T	T2*w phase	110 randomly selected lesions	Intra	Per-lesion rim presence	Cohen’s κ	0.96
Kolb et al. (2021)³³^*	7T	T2*w phase	110 randomly selected lesions	Inter	Per-lesion rim presence	Kendall W	0.88
Maggi et al. (2021)⁴⁰	3T	Unwrapped phase images	118 pwMS (86 RRMS, 32 PMS)	Inter	Per-subject PRL category (0, 1-3, or 4+ PRLs)	Cohen’s κ	0.83
Treaba et al. (2021)³⁷	7T	Phase	5 pwMS	Intra	Per-subject number	Lin’s concordance correlation coefficients	0.99
Treaba et al. (2021)³⁷	7T	Phase	5 pwMS	Inter	Per-subject number	Lin’s concordance correlation coefficients	0.98
Altokhis et al. (2022)³²	7T	SWI-filtered phase	10 randomly selected pwCIS or pwMS	Intra	Per-subject PRL category (0, 1-3, or 4+ PRLs)	ICC	0.95
Altokhis et al. (2022)³²	7T	SWI-filtered phase	10 randomly selected pwCIS or pwMS	Inter	Per-subject PRL category (0, 1-3, or 4+ PRLs)	ICC	0.81
Hemond et al. (2022) ⁸	1.5T	Filtered phase	5 pwRRMS, 3pwSPMS, and 1 pwPPMS	Inter	Per-subject PRL presence	Cohen’s κ	0.65
	3T			Inter	Per-subject PRL presence	Cohen’s κ	0.72
	3T			Intra	Per-subject PRL presence	Cohen’s κ	0.70
Huang et al. (2022)³⁰^*	3T	High-pass-filtered phase	2062 FLAIR-hyperintense no-enhancing lesions from 80 pwRRMS	Inter	Per-lesion rim presence	Cohen’s κ	0.43
		High-pass-filtered phase		Inter	Per-lesion rim presence	Percent agreement	75.8%
		QSM		Inter	Per-lesion rim presence	Cohen’s κ	0.62
		QSM		Inter	Per-lesion rim presence	Percent agreement	92.7%
Krajnc et al. (2022)⁴¹	3T	SWI	75 pwMS	Inter	Per-lesion rim presence	Percent agreement	98.7%
Marcille et al. (2022)²⁹^*	3T	QSM	159 pwRRMS	Inter	Per-lesion rim presence	Bangdiwala’s B-statistics	95.49%
						Cohen’s κ	0.706
						Cohen’s κ	0.858
						Fleiss κ	0.71
Meaton et al. (2022)³⁶	3T	SWI	100 blocks (1/8 brain volume each)	Intra	Per-subject PRL presence	Cohen’s κ	0.696
Meaton et al. (2022)³⁶	3T	SWI	100 blocks (1/8 brain volume each)	Inter	Per-subject PRL presence	Cohen’s κ	0.827
Micheletti et al. (2022)⁷	1.5T	Phase	29 children (13 pwMS and 16 non-MS). 132 white matter lesions, 63 were in the MS group and 69 were in the non-MS group.	Inter	Per-lesion rim presence	Cohen’s κ	0.75
Zhang et al. (2022)⁴²^*	3T	QSM	4,163 T2-FLAIR lesions from 172 pwMS	Inter	Per-lesion rim presence	Cohen’s κ	0.59
Calvi et al. (2023)⁴³^*	3T	SWI	20 pwMS	Inter	Per-lesion rim presence	Cohen’s κ	0.87
Krajnc et al. (2023)⁴⁴	3T	SWI	107 pwMS (97 RRMS, 10 SPMS)	Inter	Per-lesion rim presence	Percent agreement	97.2%

FLAIR – fluid-attenuated inversion recovery; HC – healthy control; ICC – intraclass correlation coefficient; pwNMOSD – people with neuromyelitis optica spectrum disorder; PRL – paramagnetic rim lesion; pwCIS – people with clinically isolated syndrome; pwMS – people with multiple sclerosis; pwPPMS – people with primary progressive multiple sclerosis; pwRRMS – people with relapsing-remitting multiple sclerosis; pwSPMS – people with secondary progressive multiple sclerosis; QSM – quantitative susceptibility mapping; RRMS – relapsing-remitting multiple sclerosis; SPMS – secondary progressive multiple sclerosis; SWI – susceptibility-weighted imaging.

Included in quantitative synthesis.