Skip to main content
NCBI home page
Journal of Neurology, Neurosurgery, and Psychiatry logoLink to Journal of Neurology, Neurosurgery, and Psychiatry
. 1998 Jul;65(1):42–47. doi: 10.1136/jnnp.65.1.42

Precision and reliability for measurement of change in MRI lesion volume in multiple sclerosis: a comparison of two computer assisted techniques

P Molyneux 1, P Tofts 1, A Fletcher 1, B Gunn 1, P Robinson 1, H Gallagher 1, I Moseley 1, G Barker 1, D Miller 1
PMCID: PMC2170149  PMID: 9667559

Abstract

OBJECTIVE—The serial quantification of MRI lesion load in multiple sclerosis provides an effective tool for monitoring disease progression and this has led to its increasing use as an outcome measure in treatment trials. Segmentation techniques must display a high degree of precision and reliability if they are to be responsive to small changes over time. This study has evaluated the performance of two such techniques, the manual outlining and contour methods, in serial lesion load quantification.
METHODS—Sixteen patients with clinically definite multiple sclerosis were scanned at baseline and after two years. Scan analysis was performed twice, independently by three observers using each technique.
RESULTS—For the absolute lesion volumes the median intrarater coefficient of variation (CV) was 3.2% for the contour technique and 7.6% for the manual outlining method (p<0.005), the interrater CVs were 3.8% and 6.1% respectively (p<0.01) and the reliability of both techniques was very high. For the change in lesion volume the intrarater and interrater repeatability coefficients were respectively 2.6 cm3 and 2.8 cm3 for the contour technique, and 3.3 cm3 and 3.7 cm3 for the manual outlining method (lower values reflect higher precision). The values for intrarater and interrater reliability for measuring change in lesion volume were respectively, 0.945and 0.944 for the contour technique, and 0.939 and 0.921 for the manual outline method (perfect reliability = 1.0).
CONCLUSIONS—With such high values for reliability, the impact of measurement error in lesion segmentation on sample size requirements in multiple sclerosis treatment trials is minor. This study shows that a change in lesion volume can be measured with a higher level of precision and reliability with the contour technique and this supports its further application in serial studies.



Full Text

The Full Text of this article is available as a PDF (115.0 KB).

Selected References

These references are in PubMed. This may not be the complete list of references from this article.

  1. Barkhof F., Filippi M., Miller D. H., Tofts P., Kappos L., Thompson A. J. Strategies for optimizing MRI techniques aimed at monitoring disease activity in multiple sclerosis treatment trials. J Neurol. 1997 Feb;244(2):76–84. doi: 10.1007/s004150050053. [DOI] [PubMed] [Google Scholar]
  2. Bastianello S., Bozzao A., Paolillo A., Giugni E., Gasperini C., Koudriavtseva T., Millefiorini E., Horsfield M. A., Colonnese C., Toni D. Fast spin-echo and fast fluid-attenuated inversion-recovery versus conventional spin-echo sequences for MR quantification of multiple sclerosis lesions. AJNR Am J Neuroradiol. 1997 Apr;18(4):699–704. [PMC free article] [PubMed] [Google Scholar]
  3. Bland J. M., Altman D. G. Measurement error. BMJ. 1996 Jun 29;312(7047):1654–1654. doi: 10.1136/bmj.312.7047.1654. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Bland J. M., Altman D. G. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986 Feb 8;1(8476):307–310. [PubMed] [Google Scholar]
  5. Clarke L. P., Velthuizen R. P., Camacho M. A., Heine J. J., Vaidyanathan M., Hall L. O., Thatcher R. W., Silbiger M. L. MRI segmentation: methods and applications. Magn Reson Imaging. 1995;13(3):343–368. doi: 10.1016/0730-725x(94)00124-l. [DOI] [PubMed] [Google Scholar]
  6. Cline H. E., Lorensen W. E., Kikinis R., Jolesz F. Three-dimensional segmentation of MR images of the head using probability and connectivity. J Comput Assist Tomogr. 1990 Nov-Dec;14(6):1037–1045. doi: 10.1097/00004728-199011000-00041. [DOI] [PubMed] [Google Scholar]
  7. Edan G., Miller D., Clanet M., Confavreux C., Lyon-Caen O., Lubetzki C., Brochet B., Berry I., Rolland Y., Froment J. C. Therapeutic effect of mitoxantrone combined with methylprednisolone in multiple sclerosis: a randomised multicentre study of active disease using MRI and clinical criteria. J Neurol Neurosurg Psychiatry. 1997 Feb;62(2):112–118. doi: 10.1136/jnnp.62.2.112. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Filippi M., Horsfield M. A., Bressi S., Martinelli V., Baratti C., Reganati P., Campi A., Miller D. H., Comi G. Intra- and inter-observer agreement of brain MRI lesion volume measurements in multiple sclerosis. A comparison of techniques. Brain. 1995 Dec;118(Pt 6):1593–1600. doi: 10.1093/brain/118.6.1593. [DOI] [PubMed] [Google Scholar]
  9. Filippi M., Horsfield M. A., Campi A., Mammi S., Pereira C., Comi G. Resolution-dependent estimates of lesion volumes in magnetic resonance imaging studies of the brain in multiple sclerosis. Ann Neurol. 1995 Nov;38(5):749–754. doi: 10.1002/ana.410380509. [DOI] [PubMed] [Google Scholar]
  10. Filippi M., Horsfield M. A., Tofts P. S., Barkhof F., Thompson A. J., Miller D. H. Quantitative assessment of MRI lesion load in monitoring the evolution of multiple sclerosis. Brain. 1995 Dec;118(Pt 6):1601–1612. doi: 10.1093/brain/118.6.1601. [DOI] [PubMed] [Google Scholar]
  11. Filippi M., Yousry T., Baratti C., Horsfield M. A., Mammi S., Becker C., Voltz R., Spuler S., Campi A., Reiser M. F. Quantitative assessment of MRI lesion load in multiple sclerosis. A comparison of conventional spin-echo with fast fluid-attenuated inversion recovery. Brain. 1996 Aug;119(Pt 4):1349–1355. doi: 10.1093/brain/119.4.1349. [DOI] [PubMed] [Google Scholar]
  12. Gawne-Cain M. L., O'Riordan J. I., Coles A., Newell B., Thompson A. J., Miller D. H. MRI lesion volume measurement in multiple sclerosis and its correlation with disability: a comparison of fast fluid attenuated inversion recovery (fFLAIR) and spin echo sequences. J Neurol Neurosurg Psychiatry. 1998 Feb;64(2):197–203. doi: 10.1136/jnnp.64.2.197. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Gawne-Cain M. L., Webb S., Tofts P., Miller D. H. Lesion volume measurement in multiple sclerosis: how important is accurate repositioning? J Magn Reson Imaging. 1996 Sep-Oct;6(5):705–713. doi: 10.1002/jmri.1880060502. [DOI] [PubMed] [Google Scholar]
  14. Goodkin D. E., Ross J. S., Medendorp S. V., Konecsni J., Rudick R. A. Magnetic resonance imaging lesion enlargement in multiple sclerosis. Disease-related activity, chance occurrence, or measurement artifact? Arch Neurol. 1992 Mar;49(3):261–263. doi: 10.1001/archneur.1992.00530270075021. [DOI] [PubMed] [Google Scholar]
  15. Grimaud J., Lai M., Thorpe J., Adeleine P., Wang L., Barker G. J., Plummer D. L., Tofts P. S., McDonald W. I., Miller D. H. Quantification of MRI lesion load in multiple sclerosis: a comparison of three computer-assisted techniques. Magn Reson Imaging. 1996;14(5):495–505. doi: 10.1016/0730-725x(96)00018-5. [DOI] [PubMed] [Google Scholar]
  16. Kurtzke J. F. Rating neurologic impairment in multiple sclerosis: an expanded disability status scale (EDSS). Neurology. 1983 Nov;33(11):1444–1452. doi: 10.1212/wnl.33.11.1444. [DOI] [PubMed] [Google Scholar]
  17. Miller D. H., Albert P. S., Barkhof F., Francis G., Frank J. A., Hodgkinson S., Lublin F. D., Paty D. W., Reingold S. C., Simon J. Guidelines for the use of magnetic resonance techniques in monitoring the treatment of multiple sclerosis. US National MS Society Task Force. Ann Neurol. 1996 Jan;39(1):6–16. doi: 10.1002/ana.410390104. [DOI] [PubMed] [Google Scholar]
  18. Miller D. H., Barkhof F., Berry I., Kappos L., Scotti G., Thompson A. J. Magnetic resonance imaging in monitoring the treatment of multiple sclerosis: concerted action guidelines. J Neurol Neurosurg Psychiatry. 1991 Aug;54(8):683–688. doi: 10.1136/jnnp.54.8.683. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Mitchell J. R., Karlik S. J., Lee D. H., Fenster A. Computer-assisted identification and quantification of multiple sclerosis lesions in MR imaging volumes in the brain. J Magn Reson Imaging. 1994 Mar-Apr;4(2):197–208. doi: 10.1002/jmri.1880040218. [DOI] [PubMed] [Google Scholar]
  20. Noseworthy J. H., Vandervoort M. K., Wong C. J., Ebers G. C. Interrater variability with the Expanded Disability Status Scale (EDSS) and Functional Systems (FS) in a multiple sclerosis clinical trial. The Canadian Cooperation MS Study Group. Neurology. 1990 Jun;40(6):971–975. doi: 10.1212/wnl.40.6.971. [DOI] [PubMed] [Google Scholar]
  21. Paty D. W., Li D. K. Interferon beta-1b is effective in relapsing-remitting multiple sclerosis. II. MRI analysis results of a multicenter, randomized, double-blind, placebo-controlled trial. UBC MS/MRI Study Group and the IFNB Multiple Sclerosis Study Group. Neurology. 1993 Apr;43(4):662–667. doi: 10.1212/wnl.43.4.662. [DOI] [PubMed] [Google Scholar]
  22. Plante E., Turkstra L. Sources of error in the quantitative analysis of MRI scans. Magn Reson Imaging. 1991;9(4):589–595. doi: 10.1016/0730-725x(91)90047-p. [DOI] [PubMed] [Google Scholar]
  23. Poser C. M., Paty D. W., Scheinberg L., McDonald W. I., Davis F. A., Ebers G. C., Johnson K. P., Sibley W. A., Silberberg D. H., Tourtellotte W. W. New diagnostic criteria for multiple sclerosis: guidelines for research protocols. Ann Neurol. 1983 Mar;13(3):227–231. doi: 10.1002/ana.410130302. [DOI] [PubMed] [Google Scholar]
  24. Wicks D. A., Tofts P. S., Miller D. H., du Boulay G. H., Feinstein A., Sacares R. P., Harvey I., Brenner R., McDonald W. I. Volume measurement of multiple sclerosis lesions with magnetic resonance images. A preliminary study. Neuroradiology. 1992;34(6):475–479. doi: 10.1007/BF00598953. [DOI] [PubMed] [Google Scholar]
  25. Zhao G. J., Li D. K., Wolinsky J. S., Koopmans R. A., Mietlowski W., Redekop W. K., Riddehough A., Cover K., Paty D. W. Clinical and magnetic resonance imaging changes correlate in a clinical trial monitoring cyclosporine therapy for multiple sclerosis. The MS Study Group. J Neuroimaging. 1997 Jan;7(1):1–7. doi: 10.1111/jon1997711. [DOI] [PubMed] [Google Scholar]
  26. van Walderveen M. A., Barkhof F., Hommes O. R., Polman C. H., Tobi H., Frequin S. T., Valk J. Correlating MRI and clinical disease activity in multiple sclerosis: relevance of hypointense lesions on short-TR/short-TE (T1-weighted) spin-echo images. Neurology. 1995 Sep;45(9):1684–1690. doi: 10.1212/wnl.45.9.1684. [DOI] [PubMed] [Google Scholar]

Articles from Journal of Neurology, Neurosurgery, and Psychiatry are provided here courtesy of BMJ Publishing Group

RESOURCES