Abstract
Objective
Neuro-imaging studies are frequently ordered to investigate neuro-ophthalmic symptoms. When misused these studies are expensive and time-consuming. This study aimed to describe the type and frequency of neuro-imaging errors in patients referred to an academic neuro-ophthalmology service and to measure how frequently these neuro-imaging studies were re-interpreted.
Design
Prospective cohort study
Participants
84 consecutive patients referred to an academic neuro-ophthalmology practice
Methods
From November 2009 through July 2010 we prospectively enrolled 84 consecutive new patients who had received a neuro-imaging study in the last 12 months specifically in evaluation of their presenting neuro-ophthalmic symptoms. Participants then underwent a complete neuro-ophthalmic evaluation followed by a review of prior neuro-imaging. Questions regarding appropriateness of the most recent imaging, concordance of radiological interpretation, and re-evaluation of referring diagnoses were answered by the attending physician.
Main Outcome Measures
1. The frequency and types of errors committed in the utilization of neuro-imaging. 2. The frequency of re-interpretation of pre-referral neuro-imaging studies following neuro-ophthalmic history and examination.
Results
Most study participants (84.5%; 71/84) underwent magnetic resonance imaging (MRI) prior to referral; 15.5% (13/84) underwent only computed tomography (CT). The rate of sub-optimal neuro-imaging studies was 38.1% (32/84). The three most common reasons for sub-optimal studies were incomplete area of imaging (34.4%; 11/32), wrong study type (28.1%; 9/32), and poor image quality (21.9%; 7/32). 24 of 84 subjects (28.6%) required additional neuro-imaging. We agreed with the radiology interpretation of the prior neuro-imaging studies in the majority (77.4%; 65/84) of patients. The most common anatomic locations for discordance in interpretation were the intraorbital optic nerve (35%; 7/20) and the brainstem (20%; 4/20).
Conclusions
There was a high rate of sub-optimal neuro-imaging studies performed in patients referred to neuro-ophthalmology. These findings have significant implications given the increasing attention to resource utilization currently and in the near future.
INTRODUCTION
Computerized tomography (CT) scans and magnetic resonance imaging (MRI) scans are widely used to evaluate patients with neuro-ophthalmic signs and symptoms. The increasing use of these tests has paralleled the increasing availability of the technology. As government, insurance agencies, and physicians struggle with appropriate and efficient allocation of resources, , there is increasing attention to the cost-effectiveness of these studies and their appropriate use by clinicians.1-6 As tertiary care providers, neuro-ophthalmologists frequently review and re-interpret images that were obtained on patients prior to referral. The evaluation of these studies provides an opportunity to determine if a study confirms the suspecteddiagnosis or adequately excludes alternative diagnoses. The aim of our prospective investigation was to learn the type and frequency of errors committed in the use of neuro-imaging in a series of consecutive patients referred to an academic neuro-ophthalmology service, and to measure how frequently these studies were re-interpretted.
METHODS
We performed a prospective study of 84 consecutive new patients referred to our university-based neuro-ophthalmology clinic from November 2009 through July 2010. Institutional Review Board approval was obtained, Health Insurance Portability and Accountability Act (HIPAA) regulations were followed, and all patients gave written informed consent prior to enrollment. To be eligible for participation, new patients must have received a neuro-imaging study (in all cases either an MRI or CT ) in the preceding 12 months specifically related to their presenting neuro-ophthalmic symptom(s) that prompted referral; patients presenting with neuro-imaging studies ordered for other reasons unrelated to his/her neuro-ophthalmic symptom(s) were excluded. Minors, patients previously evaluated by a neuro-ophthalmologist, and patients referred specifically for surgical correction of strabismus were excluded. A complete neuro-ophthalmic evaluation was performed, with review of all prior neuro-imaging. The attending neuro-ophthalmologist (GPV or JBS) completed a data collection form (see appendix A available online at http://aaojournal.org) to record demographic data, the appropriateness of the most recent neuro-imaging, reasons for inappropriate studies, concordance or discordance with radiological interpretation, location of pathology when there was disagreement with the prior radiological interpretation, and re-evaluation of referring diagnoses. For study purposes, appropriateness was defined as whether the neuro-imaging study in question adequately confirmed the suspected diagnosis (based upon history and examination) or adequately excluded alternate plausible diagnoses. Inappropriate, or sub-optimal, imaging could result from one or more of the following errors: the wrong study type was performed in evaluation of the patient at presentation (MRI vs. CT); the correct neuro-imaging was performed but the image quality was too poor to accurately interpret; there was a failure to administer intravenous contrast when indicated clinically (in patients without a contraindication to contrast); the neuro-imaging did not sufficiently visualize the region of pathology suspected clinically (incomplete neuro-imaging); the optimal sequences were not performed to evaluate for suspected pathology (e.g. fat suppression); or the neuro-imaging was not necessary based on the patients clinical presentation.
Concordance in neuro-imaging interpretation was assessed after we examined the patient and reviewed both the radiology report and images. Concordance was defined as agreement with the neuro-ophthalmologically relevant findings in the prior radiology report while discordance was defined as disagreement with the relevant findings of the radiology report. The decision of whether to review the imaging studies with an expert neuro-radiologist at our institution was left to the discretion of the attending neuro-ophthalmologist in order to approximate “real world” clinical practice patterns. Main outcome measures included the frequency and types of errors committed in the utilization of neuro-imaging and the frequency of re-interpretation of pre-referral neuro-imaging studies following neuro-ophthalmic history and examination.
RESULTS
In total 84 subjects were included (table 1). Male (48.8%; 41/84) and female (51.2%; 43/84) participants were nearly balanced and the average age of participants was 51.3 ± 17.4 (range: 18-84). The majority of participants were referred by ophthalmologists (53.6%; 45/84) (table 2).
Table 1. Participant demographics (n=84).
| total | % | |
|---|---|---|
| Race | ||
| Caucasian | 75 | 89.3% |
| African American | 7 | 8.3% |
| Asian | 1 | 1.2% |
| Other | 1 | 1.2% |
| Gender | ||
| Male | 41 | 48.8% |
| Female | 43 | 51.2% |
Table 2. Referring care provider (n=84).
| total | % | |
|---|---|---|
| Ophthalmologist | 45 | 53.6% |
| Neurologist | 12 | 14.3% |
| Optometrist | 9 | 10.7% |
| Primary care provider | 6 | 7.1% |
| Neurosurgeon | 2 | 2.4% |
| Self-referred | 2 | 2.4% |
| Other | 8 | 9.5% |
Most study participants (84.5%; 71/84) underwent MRI prior to referral while 15.5% (13/84) underwent only CT. The rate of sub-optimal neuro-imaging studies was 38.1% (32/84). The three most common reasons for sub-optimal neuro-imaging studies (table 3) were incomplete area of imaging (34.4%; 11/32), wrong study type (28.1%; 9/32), and poor image quality (21.9%; 7/32). 24 of 84 subjects (28.6%) required additional neuro-imaging. In 21/24 cases (87.5%) the indication was inadequacy of the initial neuro-imaging studies.
Table 3. Characterization of sub-optimal neuro-imaging.
| total | % | |
|---|---|---|
| Most recent neuro-imaging study | ||
| Sub-optimal | 32/84 | 38.1% |
| Optimal | 52/84 | 61.9% |
| Reasons for sub-optimal imaging | ||
| Incomplete area of imaging | 11/32 | 34.4% |
| Wrong study type | 9/32 | 28.1% |
| Poor image quality | 7/32 | 21.9% |
| Imaging not necessary | 4/32 | 12.5% |
| Lack of IV contrast | 3/32 | 9.4% |
| Missing / inappropriate windows | 1/32 | 3.1% |
| Other | 1/32 | 3.1% |
| Additional studies required after evaluation | 24/84 | 28.6% |
| Required: Prior imaging study sub-optimal | 21/24 | 87.5% |
| Required: Clinically warranted | 3/24 | 12.5% |
IV= intravenous
Re-interpretation of pre-referral neuro-imaging agreed with the official radiology interpretation in the majority of cases (77.4%; 65/84) (table 4). When discordance occurred (20 events), a previously normal radiology interpretation was re-interpreted as abnormal by the neuro-ophthalmologist 65% of the time (13/20). The most common anatomic locations for discordant interpretation were the intraorbital optic nerve (35%; 7/20) and the brainstem (20%; 4/20).
Table 4. Discordance in interpretation of neuro-imaging.
| total | % | |
|---|---|---|
| Most recent neuro-imaging study (n = 84) | ||
| Agree with radiology interpretation | 65 | 77.4% |
| Disagree with radiology interpretation | 19 | 22.6% |
| Events of disagreement (n = 20) | ||
| False Positive (radiology over read) | 7 | 35% |
| False Negative (radiology missed finding) | 13 | 65% |
| Anatomic location of discordant findings (n=20) | ||
| Intraorbital optic nerve | 7 | 35% |
| Brainstem | 4 | 20% |
| Dural venous sinuses | 2 | 10% |
| Optic chiasm | 2 | 10% |
| Intracranial optic nerve | 2 | 10% |
| Intracanalicular optic nerve | 2 | 10% |
| Arterial system | 1 | 5% |
| Optic tract | 1 | 5% |
| Cortical parenchyma | 1 | 5% |
| Other | 1 | 5% |
The diagnosis suspected by the referring care provider was available in 68/84 participants (81%) (table 5). Of those patients with a known suspected diagnosis before referral, the diagnosis following neuro-ophthalmic examination and review of all prior neuro-imaging was changed in 47 of 68 cases (69.1%).
Table 5. Discordance in diagnosis.
| total | % | |
|---|---|---|
| Participants with known referring diagnosis | 68/84 | 81% |
| Discordance between referring diagnosis and neuro- ophthalmology diagnosis |
47/68 | 69.1% |
DISCUSSION
Imaging of the visual pathways is often critical in the diagnosis and management of neuro-ophthalmic disorders.7-12 Although it seems intuitive that the best imaging study needs to match the clinical findings, previous reviews have suggested that this does not always occur.7-9 Improper neuro-imaging is often multi-factorial; errors may occur due to any combination of inaccurate clinical localization, sub-optimal image quality, incorrect imaging modality, incomplete imaging region / dedication, failure to utilize contrast, or failure of communication between the radiologist and clinician.7,8,10,13,14 The misuse of neuro-imaging is burdensome, resulting in lost time for both physicians and patients, costs of unnecessary or repetitive imaging studies, and delay in diagnosis. Better recognition and characterization of common errors in neuro-imaging may help to alleviate this burden. Through neuro-ophthalmic case analysis, Wolintz divided common errors in the utilization of MRI into “prescriptive errors”(failure to properly prescribe the correct study) and “interpretive errors” (failure to properly interpret the correct study).7 Prior literature has emphasized the importance of effective communication between clinician and radiologist as a common cause of interpretive errors in MRI utilization.7,14,19 Elmalem recently reviewed 17 cases of isolated third nerve palsy due to posterior communicating artery aneurysm and found that in 6/8 cases with misinterpreted non-invasive neurovascular imaging, the radiologist was given vague or wrong clinical history.14 The first step in optimizing the utilization of neuro-imaging modalities may be understanding the frequency and type of errors committed.
In our study, neuro-imaging was judged to be sub-optimal in 38.1% (32/84) of subjects. The most common reason for inadequate neuro-imaging prior to referral was an incomplete area of imaging (34.4%; 11/32). In these cases the area of pathology localized clinically was not adequately assessed with the imaging study chosen. Fault for such errors is often difficult to assign but can arise from a lapse in prescriber knowledge base, incomplete examination, or poor communication with ordering ancillary staff. Despite the known benefit of fat suppression sequences and dedicated orbital slices,7-8 most of the cases of inadequate imaging region omitted fat-saturated orbital views when they were required based on clinical localization.
We also found a high rate (28.1%; 9/32) of “wrong study types”, which most frequently occurred when a CT rather than an MRI was ordered. Expert reviews caution against the routine use of a screening head CT or “orbitobrainogram” for neuro-ophthalmic complaints,13 and suggest that MR scanning is the imaging modality of choice for virtually all neuro-ophthalmic evaluations.9 In this particular setting, however, judging clinical “appropriateness” of such studies retrospectively is challenging, given variables such as the exact patient presentation and the availability of MRI in the referring providers’ practice.
The relatively high rate of suboptimal imaging in our patients (38.1%; 32/84) contributed to a high rate of patients requiring additional neuro-imaging due to inadequacy of the original study (25%; 21/84). This translates to increasing cost, lost patient and provider time, and delay in final diagnosis. Although the importance of optimizing initial neuro-imaging studies to avoid waste and improve patient care has been advocated for some time,7,13 our study is the first to our knowledge that quantifies the extent of suboptimal imaging and frequency of image re-interpretation in patients referred to a university-based neuro-ophthalmology practice.
Overall we discovered a high rate of agreement between initial interpretation by the radiologist and subsequent re-interpretation by our neuro-ophthalmologists (77.4%; 65/84). Prior studies evaluating the accuracy of radiologic interpretation in patients with intracranial aneurysms highlight the importance of neuro-radiology fellowship training in interpreting radiologists.14,17,18 The relatively high rate of agreement in interpretation of neuro-imaging may reflect the abundance of neuro-radiologists in our academic institution. Our data on the training background of interpreting radiologists for neuro-imaging films performed prior to referral to our clinic was incomplete, however, so this analysis was not included.
Among cases where there was disagreement between the radiology and neuro-ophthalmology film interpretation, we investigated the anatomic locations of disagreement. Notably, the anatomic locations with the greatest number of discordant interpretations (optic nerve and brainstem) are sites where pathology can be difficult to visualize, yet often localizes clinically in a characteristic and specific fashion. This observation, along with a 69.1% (47/68) rate of change in the referral diagnosis following neuro-ophthalmic examination, supports the need for improved communication between clinicians and interpreting radiologists. This data would suggest that modern neuro-imaging serves best as an adjuvant to, and not a replacement for, proper clinical exam.
We acknowledge the limitations of our study, including relatively small sample size, inherent subjectivity in defining appropriateness, and inherent bias in re-interpretation of imaging studies. Study recruitment lasted nine months although the number enrolled was limited by a high percentage of patients who presented without undergoing prior neuro-imaging studies, without a copy of their prior radiological images, and/or without a copy of the radiological interpretation of prior neuro-imaging. Detailed information including the training background of the interpreting radiologist and data on the extent / accuracy of communication between the clinician who ordered pre-referral neuro-imaging and the interpreting radiologist would have been relevant to our study but proved logistically difficult to obtain. In accord with our standard practice procedure, we did consult neuroradiologists to confirm our interpretation of select, challenging neuro-imaging cases, although not all cases were reviewed. The lack of a masked, independent neuroradiologist to confirm our interpretation and serve as a gold standard could have lead to bias in interpretation. Statistical analysis is limited given the descriptive nature of the study. Previous studies have evaluated the utility of imaging in general2 and neuro-imaging in specific settings.14-16 To our knowledge, however, this is the first study to systematically review the rate of sub-optimal neuro-imaging and the rate of re-interpretation of neuro-imaging studies in patients referred to a neuro-ophthalmology clinic. This information may have important implications considering current and future attention to cost-effectiveness and resource utilization in healthcare.
Supplementary Material
Acknowledgments
Research supported by DOVS Core Grant 5 P30 EY02687, Institute for Clinical and Translational Sciences Grant RR023496, Biostat Core Grant U54 RR023496, and an Unrestricted Grant from Research to Prevent Blindness and NIH Core Vision Grant P30 EY02687
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Data presented as a poster at the 2011 NANOS meeting in Vancouver, Canada
This article contains online-only material. The following should appear online-only: Appendix A
No conflicting relationship exists for any author.
REFERENCES
- 1.Sistrom CL. The appropriateness of imaging: a comprehensive conceptual framework. Radiology. 2009;251:637–49. doi: 10.1148/radiol.2513080636. [DOI] [PubMed] [Google Scholar]
- 2.Schaefer PW, Miller JC, Singhal AB, et al. Headache: when is neurologic imaging indicated? J Am Coll Radiol. 2007;4:566–9. doi: 10.1016/j.jacr.2006.10.001. [DOI] [PubMed] [Google Scholar]
- 3.Blackmore CC, Magid DJ. Methodologic evaluation of the radiology cost-effectiveness literature. Radiology. 1997;203:87–91. doi: 10.1148/radiology.203.1.9122421. [DOI] [PubMed] [Google Scholar]
- 4.Mushlin AI, Ruchlin HS, Callahan MA. Costeffectiveness of diagnostic tests. Lancet. 2001;358:1353–5. doi: 10.1016/S0140-6736(01)06417-0. [DOI] [PubMed] [Google Scholar]
- 5.Mushlin AI, Mooney C, Holloway RG, et al. The cost-effectiveness of magnetic resonance imaging for patients with equivocal neurological symptoms. Int J Technol Assess Health Care. 1997;13:21–34. doi: 10.1017/s0266462300010205. [DOI] [PubMed] [Google Scholar]
- 6.Bettmann MA. The ACR Appropriateness Criteria: view from the committee chair. J Am Coll Radiol. 2006;3:510–2. doi: 10.1016/j.jacr.2006.03.024. [DOI] [PubMed] [Google Scholar]
- 7.Wolintz RJ, Trobe JD, Cornblath WT, et al. Common errors in the use of magnetic resonance imaging for neuro-ophthalmic diagnosis. Surv Ophthalmol. 2000;45:107–14. doi: 10.1016/s0039-6257(00)00147-8. [DOI] [PubMed] [Google Scholar]
- 8.Lee AG, Johnson MC, Policeni BA, Smoker WR. Imaging for neuro-ophthalmic and orbital disease - a review. Clin Experiment Ophthalmol. 2009;37:30–53. doi: 10.1111/j.1442-9071.2008.01822.x. [DOI] [PubMed] [Google Scholar]
- 9.Lee AG, Brazis PW, Garrity JA, White M. Imaging for neuro-ophthalmic and orbital disease. Am J Ophthalmol. 2004;138:852–62. doi: 10.1016/j.ajo.2004.06.069. [DOI] [PubMed] [Google Scholar]
- 10.Bose S. [Accessed January 21, 2012];Neuroimaging in neuroophthalmology. Neurosurg Focus. 2007 23:E9. doi: 10.3171/FOC-07/11/E9. serial online. Available at: http://thejns.org/doi/pdf/10.3171/FOC-07/11/E9. [DOI] [PubMed] [Google Scholar]
- 11.Levin LA. Clinical signs and symptoms requiring computed tomography and magnetic resonance imaging evaluation. Neuroimaging Clin N Am. 1996;6:1–14. [PubMed] [Google Scholar]
- 12.Tamraz J. Neuroradiologic investigation of the visual system using magnetic resonance imaging. J Clin Neurophysiol. 1994;11:500–18. doi: 10.1097/00004691-199409000-00005. [DOI] [PubMed] [Google Scholar]
- 13.Trobe JD, Gebarski SS. Looking behind the eyes: the proper use of modern imaging. Arch Ophthalmol. 1993;111:1185–6. doi: 10.1001/archopht.1993.01090090037016. [DOI] [PubMed] [Google Scholar]
- 14.Elmalem VI, Hudgins PA, Bruce BB, et al. Underdiagnosis of posterior communicating artery aneurysm in noninvasive brain vascular studies. J Neuroophthalmol. 2011;31:103–9. doi: 10.1097/WNO.0b013e3181f8d985. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Vernooij MW, Ikram MA, Tanghe HL, et al. Incidental findings on brain MRI in the general population. N Engl J Med. 2007;357:1821–8. doi: 10.1056/NEJMoa070972. [DOI] [PubMed] [Google Scholar]
- 16.Manolakaki D, Velmahos GC, Spaniolas K, et al. Early magnetic resonance imaging is unnecessary in patients with traumatic brain injury. J Trauma. 2009;66:1008–12. doi: 10.1097/TA.0b013e31819adba2. discussion 1012-4. [DOI] [PubMed] [Google Scholar]
- 17.White PM, Wardlaw JM, Lindsay KW, et al. The non-invasive detection of intracranial aneurysms: are neuroradiologists any better than other observers? Eur Radiol. 2003;13:389–96. doi: 10.1007/s00330-002-1520-1. [DOI] [PubMed] [Google Scholar]
- 18.Chaudhary N, Davagnanam I, Ansari SA, et al. Imaging of intracranial aneurysms causing isolated third cranial nerve palsy. J Neuroophthalmol. 2009;29:238–44. doi: 10.1097/WNO.0b013e3181b415f4. [DOI] [PubMed] [Google Scholar]
- 19.Lessell S. Pitfalls in imaging. J Neuroophthalmol. 2011;31:101–2. doi: 10.1097/WNO.0b013e318219c906. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.