The NAIMS Cooperative Pilot Project: Design, Implementation and Future Directions

Summary

The North American Imaging in Multiple Sclerosis (NAIMS) Cooperative represents a network of 27 academic centers focused on accelerating the pace of MRI research in multiple sclerosis (MS) through idea exchange and collaboration. Recently, NAIMS completed its first project evaluating the feasibility of implementation and reproducibility of quantitative MRI measures derived from scanning a single MS patient using a high-resolution 3T protocol at seven sites. The results showed the feasibility of utilizing advanced, quantitative MRI measures in multicenter studies, and demonstrated the importance of careful standardization of scanning protocols, central image processing, and strategies to account for inter-site variability.

Scientific discovery in the field of multiple sclerosis (MS) over the past two decades has resulted in significant progress in our understanding of this complex disease. Novel MRI techniques have contributed substantially by enabling in vivo assessments of MS tissue changes in the central nervous system providing needed insight into disease evolution. Over time, MRI has become an indispensable tool not only in the clinical management of MS patients, but also in investigational study settings.

In recent years, the importance of multi-center collaborative efforts is being increasingly recognized across many scientific fields. In the field of MS imaging, collaborative efforts have the potential to improve efficiency, minimize redundancy, increase generalizability of results to the full disease spectrum, and strengthen the statistical power of studies by providing larger sample sizes. The MAGNIMS (Magnetic Resonance Imaging in MS) network in Europe is an example of a collaborative effort comprised of academic institutions across the continent with an interest in imaging in MS, that is fully independent of any other organization (www.magnims.eu). Since its establishment, the MAGNIMS group has been productive, contributing both original research and consensus statements that have been of importance to the field.^1–4

In North America, despite a large number of academic centers with expertise in imaging in MS, there had yet to be a formal collaborative network. Consequently, a group of MS imaging centers affiliated with the Race to Erase MS Foundation (http://www.erasems.org/) established a collaborative group in 2012 with the goal of accelerating the pace of MRI research in MS though idea exchange, collaboration, and the evaluation of standardized, state-of-the-art MRI protocols in multicenter studies.

The North American Imaging in Multiple Sclerosis (NAIMS) Cooperative was thus established with seven inaugural centers, which included: Johns Hopkins University, the National Institutes of Health, Cedars-Sinai Medical Center, Brigham and Women’s Hospital/Harvard, Oregon Health & Science University, University of California at San Francisco, and Yale University. Since its inception, the NAIMS group has expanded to include 27 centers across North America.

The inaugural project of the seven original NAIMS sites was supported by the Race to Erase MS Foundation and had two primary specific aims: developing and distributing a standardized, high-resolution brain 3T protocol relevant to multi-site quantification of both conventional and advanced MRI measures; and evaluating intra- and inter-site reproducibility of quantitative MRI metrics derived from the protocol in a subject with MS who volunteered to travel to the seven imaging sites across North America. In addition to these goals, the over-arching objective was to establish a productive collaborative relationship within the network of inaugural NAIMS centers.

Participating NAIMS sites obtained local institutional review board (IRB) approval to scan the study subject, and a central IRB at Cedars-Sinai Medical Center approved the pilot study. Prior to the subject visit, each site scanned a healthy control using the study protocol, and these images were assessed by the NAIMS Steering Committee to ensure adequate quality. Modifications were made to study sequences if deemed necessary.

A 45 year old man with stable, relapsing-remitting MS was scanned over a period of four months between September, 2015 and February, 2016. At each site, the subject completed two separate MRI scans. A central study coordinator interviewed the patient the day of each site visit to ensure symptom stability. Each site acquired images on a 3T Siemens MRI scanner, but there were differences in model, software versions, and coils (Table 1) Standardization was optimized through a shared manual of operations. To minimize extraneous factors contributing to variability, the patient was scheduled to undergo scanning at a similar time of day and instructed to self-administer medications required for symptomatic control of lower limb spasms prior to each scan, if needed. Acquired images were uploaded to a central repository, and one NAIMS site with established expertise performed post-acquisition processing for individual sequences. The NAIMS Steering Committee designed the protocol to enable assessments of: lesions, regional atrophy, gray matter pathology, myelination, white matter tract integrity, and network connectivity. Sequences acquired included: 3D T1- magnetization prepared rapid acquisition gradient echo (MPRAGE), 3D T2, 3D fluid-attenuated inversion recovery (FLAIR, Figure 1), 2D dual-echo fast spin echo (DE-FSE), variable flip-angle longitudinal relaxation rate constant (VFA R1), phase-sensitive inversion recovery (PSIR), quantitative magnetization transfer (MT), resting-state functional MRI (fMRI), and diffusion tensor imaging (DTI) (full MRI protocols by Siemens MRI model presented in e-file 1).

Table 1.

3T MRI Platforms Across Inaugural NAIMS Sites

Site	Platform	Model	Software	Coil
National Institutes of Health	Siemens	Skyra	VD11	32 channel
Brigham and Women’s Hospital/Harvard	Siemens	Skyra	VD13	32 channel
Cedars-Sinai Medical Center	Siemens	Verio	VB17	32 channel
Oregon Health and Science University	Siemens	Trio	VB17	32 channel
University of California at San Francisco	Siemens	Skyra	VD13	64 channel
Johns Hopkins University	Siemens	Trio	VB17	32 channel
Yale University	Siemens	Skyra	VD13	32 channel

Figure 1. Sagittal FLAIR Sequence Acquired in Single MS Subject Across NAIMS Sites (manually-segmented T2-lesion volumes are presented beside each center name).

A: Harvard University (19.1 mL); B: Cedars-Sinai Medical Center (17.2 mL); C: Johns Hopkins University (16.5 mL); D: National Institutes of Health (17.9 mL); E: Oregon Health and Science University (16.3 mL); F: University of California at San Francisco (19.3 mL); G: Yale University (19.4 mL)

Detailed comparisons of various quantitative MRI metrics from this effort have resulted in two published scientific studies,^5,6 in addition to other works in progress.^7–9 To summarize the results to date (e-table 1): magnetization-transfer ratio (MTR), DTI-derived and quantitative R1-derived metrics demonstrated excellent intra- and inter-site reliability,^7,8 while volumetrics and lesion volumes demonstrated excellent intra-site, but relatively poorer inter-site reliability.⁵ Resting state fMRI-derived metrics in the brain demonstrated adequate intra-site reliability, but suboptimal inter-site reliability.⁹ Upper cervical cord area measurements extracted from volumetric brain scans demonstrated significant intra- and inter-site variability, mainly related to inconsistencies in subject positioning and gradient non-linearity effects.⁶ These results have served as a basis for future collaborations, including one submitted multi-site federal grant application.

In conclusion, the NAIMS pilot study demonstrates the feasibility of utilizing advanced, quantitative MRI measures across the NAIMS network, and in multicenter settings in MS. These findings highlight the importance of careful standardization of scanning protocols and procedures, central image processing, intrinsic differences across acquisition types, and analytic strategies to account for inter-site variability, which remains an issue for some quantitative metrics, even in carefully controlled settings. These results have informed the design of a larger collaboration, setting the stage for future multi-center studies in MS utilizing a variety of state-of-the-art, quantitative MRI techniques. Ultimately, applying reliable, quantitative MRI measures in multi-center settings will facilitate the development of valid disease outcome measures that will support the evaluation of novel and emerging therapeutic agents for MS.