Abstract
Objective
Patients with multiple sclerosis (MS) routinely undergo serial contrast enhanced MRIs. Given concerns regarding tissue deposition of gadolinium-based contrast agents (GBCAs), and evidence that enhancement of lesions is only seen in patients with new disease activity on noncontrast imaging, we set out to implement a prospective quality improvement project whereby IV contrast would be reserved only for patients with evidence of new disease activity on noncontrast images.
Methods
In order to prospectively implement such a protocol, we leveraged our in-house computer-assisted-detection (CAD) software and 3-D laboratory radiology technologists to perform real-time preliminary assessments of the CAD-processed T2/FLAIR noncontrast images as a basis for deciding whether to inject contrast. Prior to implementation, we held multidisciplinary meetings with neurology, neuroradiology and MR technologists, and distributed surveys to objectively assess opinions and obstacles to clinical implementation. We evaluated reduction in GBCA utilization and technologist performance relative to final neuroradiologist interpretations.
Results
During a 2-month trial period, 153 patients were imaged under the new protocol. Technologists using the CAD software were able to identify patients with new or enlarging lesions on FLAIR images with 95% accuracy and 97% negative predictive value relative to final neuroradiologist interpretations, which allowed us to avoid the use of contrast and additional imaging sequences in 87% of patients.
Discussion
A multidisciplinary effort to implement a quality improvement project to limit contrast in MS patients receiving follow up MRIs allowed for improved safety and cost by targeting patients that would benefit from the use of intravenous contrast in real-time.
Keywords: Gadolinium, IV contrast, Multiple Sclerosis, Brain MRI, Quality Improvement, Precision Diagnostics
Summary Sentence
Through a multidisciplinary quality improvement project, we were able to drastically limit the use of gadolinium-based contrast in multiple sclerosis patients undergoing follow-up imaging.
Introduction
Multiple sclerosis (MS) is an inflammatory demyelinating and neurodegenerative disease of the central nervous system that affects approximately 400,000 Americans, and is the leading cause of non-traumatic disability in young-adults (1,2). Magnetic resonance imaging (MRI) is the mainstay modality for the diagnosis and disease monitoring of patients with MS (3). Focal demyelinating lesions accumulate over time and total T2 lesion volumes increase, but contrast enhancement within a lesion occurs early in lesion formation and is transient, only lasting an average of 4 weeks (4). MRI is more sensitive to detecting MS disease activity than clinical assessment, as clinically asymptomatic MS lesions are common. Therefore, patients typically receive imaging of the brain and spine with and without gadolinium-based contrast agents (GBCA) every 6–12 months to assess response to disease modifying therapies. All of the 13 current MS disease modifying therapies are approved to reduce clinical relapses and reduce the likelihood of new lesion formation and many MS patients have either no or minimal evidence of disease activity at any given time (5).
Given that MS is a disease of younger adults (peak onset of 30 years of age) (6), patients can receive >60 contrast-enhanced MR exams during the course of their lifetime. Growing evidence suggests that there is deposition of free gadolinium in the brain and other organs with GBCA (7), even in patients with normal renal function. In fact, the FDA recently issued a warning on all GBCA (8), although the clinical significance still remains unknown. Intravenous contrast also carries small risks related to physiologic and allergic reactions, as well as IV placement (9).
In order to start to address the concerns related to serial use of GBCAs in patients with MS at our institution, we recently performed a retrospective analysis of patients with MS follow-up imaging to determine whether we could predict which patients would or would not benefit from receiving IV contrast (10). Of all the patients who received IV contrast, only 10% of patients were found to have enhancing lesions. Of the 138 patients evaluated, 24% were found to have new or enlarging lesions on noncontrast FLAIR imaging. Interestingly, patients found to have enhancing lesions were only found within the group of patients with new disease activity seen on FLAIR imaging. Thus the 76% of patients undergoing MS follow-up MRs that had stable disease on FLAIR imaging did not exhibit enhancing lesions. An additional, recent, retrospective study also found that high resolution noncontrast FLAIR and double inversion recovery sequences were sufficient to assess for new disease activity in MS follow-up patients (11). These retrospective findings suggest that GBCA may not be always necessary for MS follow-up. In particular, patients without evidence of new disease activity on noncontrast imaging may be ideal candidates for MRs without GBCA.
Given these findings and concerns about GBCA deposition, we sought to implement a quality improvement project to limit the use of GBCAs in MS follow up imaging to only those with evidence of new disease activity on noncontrast FLAIR imaging. In the implementation of this project, we sought to make the decision of whether patients had new disease activity in “real-time” while the patient was still on the MRI scanner table, prior to making the decision to inject contrast. In order to achieve our goal, we utilized our routine in-house computer-assisted detection (CAD) software for evaluating changes in MS FLAIR MRs (12,13). This software is run through in our 3-D and advanced imaging laboratory (14) where radiology technologists have been trained to do advanced image processing on clinical examinations and generate results on a variety of different types of studies. We have also previously utilized radiology technologists (i.e., radiology extenders) at our institution to help with reading musculoskeletal radiographs, which has been shown to have substantial workflow and economic benefits (15).
Materials and Methods
Multidisciplinary Meetings
Prior to implementing a new protocol, several multidisciplinary meetings were held within the radiology department and with the MS neurology division to discuss how to exactly move forward with this quality improvement project to limit IV contrast in MS follow-up patients. It was agreed to pilot the new protocol during a two-month period to determine actual real-world performance to assess the benefits and disadvantages of such a new protocol.
Pre-Implementation Surveys
In order to more objectively assess the opinions and concerns of neurologists and neuroradiologists, we created and distributed pre-implementation surveys sent to all neuroradiologists and all neurologists who take care of MS patients. The surveys asked about willingness implement a new protocol to reduce contrast administration, their concerns about gadolinium contrast use, as well as several more general questions regarding satisfaction about the current MS MRI protocol and radiology dictations. We also had a subset of patients receiving the new protocol answer a survey regarding their own opinions about the protocol and concerns about IV’s and IV contrast.
Additional Training of 3-D Laboratory Technologists
A training module was created to help the 3-D Lab technologists learn the MS CAD software, with multiple examples and explanations of typical true positives and false positives that could result. This included artifacts as a result of field inhomogeneity, arterial and cerebrospinal fluid pulsations, motion and imperfect co-registrations. Each of the 3-D Lab technologists took part in this training module prior to starting the prospective pilot.
Automated MS change detection software
We utilized in-house MS CAD software for automated comparison of two timepoints of 3D FLAIR MRs designed for multiple sclerosis follow-up evaluations (12,13), which has been used as part of our standard clinical practice for the past 7 years. Briefly, the software works by performing skull stripping, intensity normalization, rigid registration between two 3D FLAIR images, subtraction between the two datasets and thresholding to generate clusters signifying differences between the two time points. The processed images are sent directly back to the PACS within 10 minutes of processing. An example of the layout of the results of this software displayed in our PACS is shown in Figure 1.
Figure 1.
MS CAD software results layout. The prior 3D FLAIR MRI shown on the top left and with the current scan on the bottom left. The larger composite scan on the right is shown with overlaid areas of new/growing lesions in red.
Pilot Protocol
It was decided that patients who agreed to participate in this new protocol would not receive an IV prior to being placed in the MRI scanner. After the 3D FLAIR sequence was obtained following a localizer scan, the scanning MRI technologist would send the exam to PACS and let the 3-D Lab know it could be processed. The 3-D Lab technologist would then run the CAD software, which takes ~10 minutes to process an exam. This step was performed while additional noncontrast brain and spine sequences were acquired, resulting in no dead time during the scan. The 3-D Lab technologist would review the output of the CAD software for the presence or absence of new lesions in the brain, and let the MRI technologist know whether the patient needed contrast-enhanced imaging or not. If the patient needed contrast, an IV would be placed by the technologist. If no contrast was needed, the MR technologist would omit the post contrast sequences of the brain and spine, which consisted of 15 minutes of additional sequences when the patient was ordered for imaging of the brain, cervical spine and thoracic spine. During the first week of the pilot a neuroradiologist was made available to review each of the cases. After the first week, if the technologists were unsure about any particular case they had the option to call a neuroradiologist for their opinion of whether there were new lesions or not in the brain as shown by CAD, and thus whether to inject or not to inject contrast. In order to limit bias during this pilot period, the neuroradiologist was not the same neuroradiologist that made the final interpretation.
Subjects
We prospectively implemented the new protocol, starting with a 2-month period between December 14th 2018 and February 11th 2019. The protocol was first implemented on two of the 3T scanners and then extended to all four MRI scanners at our outpatient center after the first month. Exclusion criteria included no prior 3D FLAIR sequence (required for the CAD software), refusal to enroll in study, off hours scanning (Monday to Friday 8-5pm), or not being scheduled on the scanner where the pilot protocol was being implemented.
Analysis of performance of CAD and 3D technologist assessment of CAD results
The decision to inject contrast was compared to “ground truth”, which consisted of whether the interpreting neuroradiologists final report mentioned whether or not there was a new or growing lesion present. A 2×2 contingency table was generated and relevant statistics were used to evaluate the performance of the 3D lab technologist’s decision to inject or not inject contrast at the time of the scan. A false positive was defined as a patient that had contrast injected but did not have any new or growing lesions per the final radiology report. A false negative was a study where the patient had a new or growing lesion identified on the final radiology report but a decision was made not to inject contrast. In addition, we evaluated for false negative studies in which although there was no new lesion in the brain a new lesion was identified in the spinal cord.
Longitudinal evaluation of patients with missed lesions
For all of the false negative cases, we performed chart reviews and discussed outcomes with neurologists taking care of the patients to determine whether bringing the patients back for additional imaging with contrast was necessary.
Results
Pre-implementation Surveys
Neuroradiologists (n = 14, 56% response rate) were much more agreeable to implementing this new protocol with 78.6% agreeing or strongly agreeing with the new protocol, compared to 33.3% of MS neurologists (n = 6, 60% response rate) that agreed with implementing the new protocol (Figure 2A). With regards to how concerned these groups were about giving contrast unnecessarily to MS patients, 57% of neuroradiologists were “slightly concerned” and 43% were “moderately concerned” (Figure 2B). Only one of the six neurologists (16.7%) was “slightly concerned” about giving contrast unnecessarily. Both groups were also specifically asked what percentage of patients they thought received contrast unnecessarily. Neuroradiologists estimated contrast to be unnecessary in 74% (±14% SD) of patients, whereas neurologists estimated it to be unnecessary 20% (± 18% SD) of patients.
Figure 2.
Pre-implementation survey results. A) Neuroradiologists were more likely to agree with implementing the new protocol. B) Neurologists were much less concerned about patients receiving contrast unnecessarily. C) A vast majority of patients surveyed were comfortable with the change in protocol and relatively of them had concerns about receiving an IV or IV contrast.
The first 56 patients involve in the pilot were surveyed regarding their opinions of this new protocol and their concerns about IV contrast (Figure 2C). Patients were largely comfortable with the change in protocol with 43% “definitely” agreeing and 41% agreeing with “minimal reservations”. There were 4 patients (7.1%) that were unsure about the change in protocol. In terms of IV contrast use, 10.7% of patients reported that receiving IV contrast concerned them and 6.2% of patients reported that placement of an IV itself concerned them. When asked if they were satisfied with their brain MR imaging experience, 94.2% agreed or strongly agreed.
Patients Included in Study
Total number of subjects included in the study were 153. This was out of a possible 424 patients that received the MS protocol during that same time period, but did not participate in the study due to one of the exclusion criteria. Of the patients included in the study 83.0% had the brain, cervical spine and thoracic spine imaged, 16.4% had the brain and cervical spine imaged, and a single patient (0.6%) only had imaging of the brain.
Performance of CAD and technologists relative to final read
The 2×2 contingency table and performance measures the MRI technologists using the CAD system relative final signed radiologists report are shown in Table 1. Overall accuracy was 94.8% with a sensitivity of 80.0% (positive predictive value of 80.0%) for detecting new lesions and specificity of 97.0% (negative predictive value of 97.0%) to rule out the presence of new lesions relative to the final neuroradiologist reports.
Table 1.
Performance measures of CAD system and MRI technologist during the pilot period. The 2×2 contingency table is shown in the upper portion of the table. The sensitivity, specificity, positive predictive value, negative predictive value with confidence intervals are shown below the contingency table.
| New or Growing | No New or Growing | ||
| Lesion on Final Read | Lesion on Final Read | ||
| Contrast Given | 16 | 4 | 20 |
| Contrast Not Given | 4 | 129 | 133 |
| 20 | 133 | 153 | |
| Sensitivity | 80.0% | CI: 62.5% to 97.5% | |
| Specificity | 97.0% | CI: 94.1% to 99.9% | |
| Positive Predictive Value | 80.0% | CI: 62.5% to 97.5% | |
| Negative Predictive Value | 97.0% | CI: 94.1% to 99.9% | |
| Accuracy | 94.8% | CI: 91.2% to 98.3% | |
There were three cases in which no lesions were seen in the brain but a new lesion was identified in the spinal cord, which resulted in an accuracy of 98.0% for excluding new spinal cord lesions on the basis of not identifying any new or growing lesions in the brain. Therefore, the overall negative predictive value of the workflow for excluding new lesions in either the brain or spinal cord was 95.4%.
There were a total four false positives, which consisted of giving contrast given when there were actually no new or growing lesions. Reasons for false positives included misregistration/motion artifact (n = 2) and increased signal at skull base (n = 2). There were four false negatives, which consisted of contrast not being given when there was a new/growing lesion in the brain identified on the final radiology report. Reasons for false negatives included lesions not detected due to small size (n = 1), location in posterior fossa (n = 1), and large amounts of motion/misregistration artifact resulting in many false positives obscuring a real new/growing lesion (n = 2).
As the patients with false positives were essentially given a routine protocol with contrast there was no need to perform a chart review. For the four patients with missed lesions in the brain and the three patients with new spinal cord lesions, follow up chart review revealed that none of them were brought back for additional imaging with contrast; all of these patients were managed routinely with appropriate modification in clinical management.
Discussion
Through a multidisciplinary quality improvement project, we were able to drastically limit the use of gadolinium-based contrast in multiple sclerosis patients undergoing follow-up imaging. In order to target GBCA use only for MS patients that show new disease activity on noncontrast imaging, we trained 3-D laboratory technologists to make preliminary assessments from the results of our in-house CAD software (12,13) that has been used clinically for the past 7 years to assess changes between two 3D FLAIR timepoints. The real-world performance of these preliminary assessments, while imperfect, showed high specificity for appropriately selecting patients that would likely receive little benefit from undergoing contrast enhanced MR sequences.
As the treatment of patients with chronic medical conditions requires a multidisciplinary team of physicians and healthcare workers, we sought to move forward with this quality improvement project by including the treating neurologists in order to have a collaborative environment to provide the best care for patients with MS. Interestingly, opinions regarding the potential concern for unnecessary use of GBCA were quite different between neuroradiologists and neurologists. Neuroradiologists were much more concerned about unnecessary contrast use and more likely to agree with a new protocol to limit contrast use, whereas to neurologists were much more hesitant to change and were not as concerned about unnecessary contrast use. Patients themselves generally approved of the new protocol as it allowed the possibility of avoiding receiving an IV and IV contrast, and a shortened time in the scanner, noting that relatively few expressed strong concerns about receiving contrast.
Through training and initial supervision in assessing the results of our CAD software, the 3-D Lab technologists (who already routinely process these exams using the CAD software) were able to achieve high accuracy (94.8%) compared to final neuroradiologist interpretations. There were 2.6% false positives, where the technologist thought a new lesion was present and which contrast was injected but no new disease activity was found on final neuroradiologist interpretation, resulting in a specificity of 97.0% in rule out the presence of new lesions. There were also 2.6% of cases that represented false negatives, in which a new lesion was identified by the final radiologist interpretation but not by technologist’s initial assessment of the CAD results. Given the relatively low percentage of patients with evidence of new disease activity on noncontrast imaging, these false negatives amounted to a modest 80% sensitivity for the technologists for detecting new lesions in the brain using the CAD system. In addition, in 2.4% of patients a new lesion in the spinal cord was identified despite there not being any new or enlarging lesions in the brain. Based on chart review and follow up discussions with the treating neurologists, the clinical course of these patients with evidence of new disease activity did not warrant them to return for additional imaging with contrast. Although the additional information regarding whether there is enhancement associated with new disease activity may provide some additional temporal information, for clinically asymptomatic patients the additional temporal information typically does not result in a change in acute management. Thus, we believe that this level moderate sensitivity in the technologist plus CAD step is unlikely to have a significant negative impact on patients, and thus may be outweighed by the benefit of targeted imaging with more limited contrast in vast majority of patients. Half of the missed new lesions were due to technologist assessment errors, which could be improved with further training of the technologists. The other half of errors were due to limitations in the software itself. Thus, it is also likely that more advanced software, such as deep learning-based methods could further improve the performance of technologists in implementing this sort of targeted diagnostic strategy as it could reduce positives and false negatives.
This pilot protocol resulted in a drastic reduction in contrast use, whereby only 13.1% of patients received contrast, helping mitigate concerns regarding the potential negative effects of gadolinium deposition or other risks related to IV contrast. Furthermore, this approach allowed for significant savings in terms of the costs to the healthcare system related to the contrast agent itself, time to administer contrast, and time for image generation and interpretation. These results are still considered preliminary given the relatively small time period, thus continued careful evaluation of the advantages and disadvantages of this protocol on patient outcomes is warranted.
A limitation of more widespread clinical implementation of our selective protocol is that the vast majority of radiology departments do not use CAD or AI systems for assistance in interpretation, which makes the task of comparing two 3D MRI timepoints much slower and challenging, making it nearly impossible for a technologists or radiologists to routinely do in a short period of time. in more hospital systems, our general approach should become a feasible solution for more departments. However, as more artificial intelligence tools become available and utilized
Take Home Points:
Multiple sclerosis patients without evidence of new disease activity on noncontrast imaging are unlikely to benefit from receiving IV contrast.
It is feasible for 3-D Lab technologists using automated computer software to perform an accurate real-time assessment of new disease activity on noncontrast imaging thus determining the need for contrast use while the patient is still in the scanner.
This sort of multidisciplinary quality improvement initiative has the potential to drastically improve patient care and healthcare system costs as part of a future utilizing precision diagnostics.
Acknowledgments:
The authors thank all of the neuroradiologists, neurologists and patients who participated in the surveys, as well as the radiology technologists who participated in the study.
Grant Support: T32-EB004311-10
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Conflict of Interest: The authors declare no conflict of interest related to the current manuscript.
References
- 1.Dilokthornsakul P, Valuck RJ, Nair KV, Corboy JR, Allen RR, and Campbell JD. Multiple sclerosis prevalence in the United States commercially insured population. Neurology. United States; 2016;86(11):1014–21. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.GBD 2015 Neurological Disorders Collaborator Group. Global, regional, and national burden of neurological disorders during 1990–2015: a systematic analysis for the Global Burden of Disease Study 2015. Lancet Neurol. England; 2017;16(11):877–897. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Rovira À, Wattjes MP, Tintoré M, Tur C, Yousry TA, Sormani MP, De Stefano N, Filippi M, Auger C, Rocca MA, Barkhof F, Fazekas F, Kappos L, Polman C, Miller D, Montalban X, and MAGNIMS study group. Evidence-based guidelines: MAGNIMS consensus guidelines on the use of MRI in multiple sclerosis-clinical implementation in the diagnostic process. Nat Rev Neurol. England; 2015;11(8):471–82. [DOI] [PubMed] [Google Scholar]
- 4.Cotton F, Weiner HL, Jolesz FA, and Guttmann CR. MRI contrast uptake in new lesions in relapsing-remitting MS followed at weekly intervals. Neurology. United States; 2003;60(4):640–6. [DOI] [PubMed] [Google Scholar]
- 5.Gasperini C, Prosperini L, Tintoré M, Sormani MP, Filippi M, Rio J, Palace J, Rocca MA, Ciccarelli O, Barkhof F, Sastre-Garriga J, Vrenken H, Frederiksen JL, Yousry TA, Enzinger C, Rovira A, Kappos L, Pozzilli C, Montalban X, De Stefano N, and and the MAGNIMS Study Group. Unraveling treatment response in multiple sclerosis: A clinical and MRI challenge. Neurology. United States; 2019;92(4):180–192. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Milo R, and Kahana E. Multiple sclerosis: geoepidemiology, genetics and the environment. Autoimmun Rev. Netherlands; 2010;9(5):A387–94. [DOI] [PubMed] [Google Scholar]
- 7.Gulani V, Calamante F, Shellock FG, Kanal E, Reeder SB, and International Society for Magnetic Resonance in Medicine. Gadolinium deposition in the brain: summary of evidence and recommendations. Lancet Neurol. England; 2017;16(7):564–570. [DOI] [PubMed] [Google Scholar]
- 8.FDA Drug Safety Communication: FDA warns that gadolinium-based contrast agents (GBCAs) are retained in the body; requires new class warnings. https://www.fda.gov/Drugs/DrugSafety/ucm589213.htm. Accessed January 28th, 2019
- 9.Rose TA, and Choi JW. Intravenous Imaging Contrast Media Complications: The Basics That Every Clinician Needs to Know. Am J Med. United States; 2015;128(9):943–9. [DOI] [PubMed] [Google Scholar]
- 10.Mattay RR, Davtyan K, Bilello M, and Mamourian AC. Do All Patients with Multiple Sclerosis Benefit from the Use of Contrast on Serial Follow-Up MR Imaging? A Retrospective Analysis. AJNR Am J Neuroradiol. United States; 2018;1–6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Eichinger P, Schön S, Pongratz V, Wiestler H, Zhang H, Bussas M, Hoshi MM, Kirschke J, Berthele A, Zimmer C, Hemmer B, Mühlau M, and Wiestler B. Accuracy of Unenhanced MRI in the Detection of New Brain Lesions in Multiple Sclerosis. Radiology. United States; 2019;:181568. [DOI] [PubMed] [Google Scholar]
- 12.Bilello M, Suri N, Krejza J, Woo JH, Bagley LJ, Mamourian AC, Vossough A, Chen JY, Millian BR, Mulderink T, Markowitz CE, and Melhem ER. An approach to comparing accuracies of two FLAIR MR sequences in the detection of multiple sclerosis lesions in the brain in the absence of gold standard. Acad Radiol. United States; 2010;17(6):686–95. [DOI] [PubMed] [Google Scholar]
- 13.Bilello M, Arkuszewski M, Nucifora P, Nasrallah I, Melhem ER, Cirillo L, and Krejza J. Multiple sclerosis: identification of temporal changes in brain lesions with computer-assisted detection software. Neuroradiol J. United States; 2013;26(2):143–50. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Cook TS, Steingall SJ, Steingall SR, and Boonn WW. Establishing and Running a Three-dimensional and Advanced Imaging Laboratory. Radiographics. United States; 2018;38(6):1799–1809. [DOI] [PubMed] [Google Scholar]
- 15.Borthakur A, Kneeland JB, and Schnall MD. Improving Performance by Using a Radiology Extender. J Am Coll Radiol. United States; 2018;15(9):1300–1303. [DOI] [PubMed] [Google Scholar]