Diagnostic Errors in Cerebrovascular Pathology: Retrospective Analysis of a Neuroradiology Database at a Large Tertiary Academic Medical Center

Most cerebrovascular diagnostic errors were perceptual and clinically significant, occurred in the emergency/inpatient setting, and were associated with higher-volume shifts. Diagnostic errors could be minimized by adjusting search patterns to ensure vigilance on the sites of the frequently missed pathologies.

Abstract

BACKGROUND AND PURPOSE:

Diagnostic errors affect 2%–8% of neuroradiology studies, resulting in significant potential morbidity and mortality. This retrospective analysis of a large database at a single tertiary academic institution focuses on diagnostic misses in cerebrovascular pathology and suggests error-reduction strategies.

MATERIALS AND METHODS:

CT and MR imaging reports from a consecutive database spanning 2015–2020 were searched for errors of attending physicians in cerebrovascular pathology. Data were collected on missed findings, study types, and interpretation settings. Errors were categorized as ischemic, arterial, venous, hemorrhagic, and “other.”

RESULTS:

A total of 245,762 CT and MR imaging neuroradiology examinations were interpreted during the study period. Vascular diagnostic errors were present in 165 reports, with a mean of 49.6 (SD, 23.3) studies on the shifts when an error was made, compared with 34.9 (SD, 19.2) on shifts without detected errors (P < .0001). Seventy percent of examinations occurred in the hospital setting; 93.3% of errors were perceptual; 6.7% were interpretive; and 93.9% (n = 155) were clinically significant (RADPEER 2B or 3B). The distribution of errors was arterial and ischemic each with 33.3%, hemorrhagic with 21.8%, and venous with 7.5%. Most errors involved brain MR imaging (30.3%) followed by head CTA (27.9%) and noncontrast head CT (26.1%). The most common misses were acute/subacute infarcts (25.1%), followed by aneurysms (13.7%) and subdural hematomas (9.7%).

CONCLUSIONS:

Most cerebrovascular diagnostic errors were perceptual and clinically significant, occurred in the emergency/inpatient setting, and were associated with higher-volume shifts. Diagnostic errors could be minimized by adjusting search patterns to ensure vigilance on the sites of the frequently missed pathologies.

Diagnostic errors are associated with significant morbidity and mortality.^1,2 In the United States alone, it is estimated that each year, 10% of deaths and $29 billion dollars in wasteful medical spending can be directly attributed to diagnostic errors in medicine; furthermore, postmortem studies have suggested that up to 20% of deaths result from a different diagnosis than that made premortem.^3,4 Diagnostic errors in radiology have been defined as major discrepancies between an interpreting radiologist’s findings or impressions on a particular study versus the consensus opinion of radiologist peers.⁵ Factors that predispose to diagnostic errors in radiology are overall categorized as radiologist-related errors and system-related errors;⁶ of the former, perceptual errors are the most important (60%–80%), while interpretation errors constitute the remainder.^7,8 System-related errors involve the reading room environment, workload, communication issues, and technical and equipment problems.⁹ Current reported errors pertaining to the neuroradiology subspecialty range between 1.7% and 7.7% overall error rates per study.^10-17 While there are qualitative articles discussing common blind spots in neurology imaging in general,¹⁸ clinical research on the quantification of diagnostic errors in neurovascular imaging is currently lacking. Increased awareness of factors that promote errors in diagnosing vascular problems on neuroradiology studies may direct more targeted search patterns, potentially leading to reduced error rates, including perceptual errors. The purpose of our study was to quantify vascular errors made by neuroradiology attending physicians at a single tertiary academic center (University of California, Davis Medical Center) and to offer potential strategies to decrease error rates, particularly by identifying common blind spots.

MATERIALS AND METHODS

Institutional review board approval was obtained for this retrospective study with a waiver of informed consent. The neuroradiology quality assurance (QA) database of diagnostic errors made by attending neuroradiologists at our institution was searched from January 2015 through March 2020 (63 months). The database was searched for vascular diagnostic errors by the 16 current and recently employed full-time diagnostic neuroradiologists (see below for details regarding the QA process). Vascular diagnostic errors were categorized as follows: ischemic (acute/subacute infarcts, chronic infarcts, multiple infarcts, hypoxic-ischemic encephalopathy), hemorrhagic (subdural, epidural, subarachnoid, contusion, intraventricular, retroclival, quantitative expanding hemorrhage), arterial (aneurysm, large-vessel occlusion, significant stenosis [>70% narrowing], dissection, AVM-AVF, vasospasm, subclavian steal), venous (abnormalities of the dural sinuses, cortical veins, or deep veins), and other (overcalls, posterior reversible encephalopathy syndrome). A broad definition of large-vessel occlusions was used including the intracranial ICA, M1, and M2 of the MCA, anterior cerebral artery (ACA), intracranial vertebral artery, basilar artery, and proximal posterior cerebral artery (PCA).¹⁹

The primary radiologic peer-review method used in the United States is the American College of Radiology RADPEER (https://www.acr.org/Clinical-Resources/RADPEER) scoring system.²⁰ Revised most recently in 2016, RADPEER uses a 3-point scoring scale (1, concur with interpretation; 2, discrepancy in interpretation not ordinarily expected to be made; and 3, discrepancy in an interpretation that should be made most of the time) with the option to designate a discrepancy as likely to be clinically insignificant (option A) or likely to be clinically significant (option B).²¹ Clinically significant errors are defined in this article as findings that were erroneously characterized or not observed that affected the patient’s treatment or follow-up if the findings had been accurately reported. Thus, we collected data on the following variables associated with the studies: clinically significant errors (RADPEER 2B, 3B) determined at the time of the peer-review process, interpretation setting (inpatient/emergency versus outpatient), mean number of CT and MR imaging scans read during the shift when the diagnostic error was made, study type, error type (perceptual, interpretive), and mean patient age.

The mean volume of interpreted studies per shift was extracted electronically from the department of radiology database, limiting the search to the shifts that were staffed by a neuroradiologist and MR imaging/CT examinations in neuroradiology. The total volume and mean number of interpreted neuroradiologic studies during the 2015–2020 study period were collected. A shift was defined as daily output of interpreted CT or MR imaging studies by an attending neuroradiologist. Shifts with ≤10 interpreted studies per day were excluded as outliers, as many of those likely pertain to reports generated on days off, administrative or other, in order to catch up with overflow work from the day before. The mean volume of interpreted studies per shift for studies containing a vascular diagnostic error was compared with the mean number of interpreted studies per shift for studies that did not have a documented error, using the Welch t test.

Neuroradiology Division and the QA Process

Our neuroradiology QA database includes only CT and MR imaging examinations that are collected from several sources. As part of the QA process, each radiologist in our department is required to evaluate 3 random (software-generated) studies on the days they are assigned to the clinical service and assign a RADPEER score (1, 2A, 2B, 3A, or 3B) to each reviewed study. RADPEER scores 2 and 3 are flagged for further review. Also, in an effort to capture as many diagnostic errors as possible, all addended neuroradiology reports (approximately 1000 addenda per year) are reviewed. Most reports are addended for technical reasons (ie, documentation of contrast or radiation dose, comparison with outside studies not available at the time of the original interpretation) and, therefore, automatically eliminated from the review process. Reports that contain potential diagnostic errors are flagged for further review. Most (approximately 90%) flagged cases are screened through 2 mechanisms: reviews of random, and addended studies. In addition, reports that are submitted by clinicians or radiologists because of disagreement with the original interpretation are flagged for further review. All flagged studies are either reviewed by 2 attending neuroradiologists, or, during our quarterly QA conference, assigned a consensus RADPEER score and entered in our QA database. Study findings are correlated against histologic findings or clinical follow-up, when available. Our neuroradiology division does not currently support subspecialized rotations such as vascular, head and neck, skull base, brain, or pediatric subdivisions, but rather functions as a general neuroradiology practice wherein every subspecialist reads all types of studies. All neuroradiologists included in this study are attending radiologists, full-time employees, and have similar yearly productivity relative value unit rates per full-time employee effort.

RESULTS

During the study period, a total of 245,762 CT and MR imaging studies were interpreted by our neuroradiology section, with a mean volume of 35.2 interpreted studies per shift. Vascular errors made up 25.8% of documented errors in this data set, with a total of 175 errors seen in 165 studies that contained a vascular diagnostic error, with a mean of 49.6 (SD, 23.3) interpreted CT and MR imaging studies on shifts when errors were made, totaling 158 shifts. Shifts in which no errors were documented had a mean volume of 34.9 (SD, 19.2) interpreted CT and MR imaging studies, totaling 7231 shifts. The Welch t test was used to look for significant differences between the mean volume of interpreted studies per shift for the 2 groups. The results suggest a highly significant difference (t 162 = 7.9, P < .0001), with studies containing an error interpreted during higher-volume shifts.

The mean patient age in this study was 54.9 (SD, 23.8) years; 96 patients (58.2%) were men and 69 (41.8%) were women. Of the 165 studies with an error, 116 (70%) involved patients in the hospital setting (emergency department or inpatient), while the remaining occurred in the ambulatory setting. Table 1 shows the breakdown of errors relative to the examination type, with MR imaging of the brain being the most common with 30.3% (n = 50), followed by CTA of the head (27.9%, n = 46) and noncontrast head CT (26.1%, n = 43). The remaining study types for which an error was detected had ≤7 cases each. Errors were considered perceptual in 154 (93.3%) and interpretive in the remaining 11 (6.7%) cases. Clinically significant errors were found in 155 (93.9%) cases (RADPEER score of 2B or 3B). All non-clinically significant errors were found to be perceptual.

Table 1:

Diagnostic error versus imaging technique

Study	Total Examinations (n = 165)	Percentage (%)
MR imaging brain	50	30.3%
CTA head	46	27.9%
Noncontrast head CT	43	26.1%
CTA neck	7	4.2%
MRA brain	5	3.0%
CT neck	4	2.4%
CT cervical spine	4	2.4%
MR imaging cervical spine	2	1.2%
MRA neck	1	0.6%
MR imaging lumbar spine	1	0.6%
MR imaging total spine	1	0.6%
CT sinus	1	0.6%

The total distribution of errors in this study is given in Table 2. Figure 1 shows that ischemic and arterial errors were equally prevalent with 58 errors each (33.3%), followed by hemorrhage (21.8%, n = 38) and venous abnormalities (7.5%, n = 13). Eight errors categorized as “other” (4.6%) included a single case of a posterior reversible encephalopathy syndrome as well as 7 overcalls consisting of 2 aneurysms, a pseudoaneurysm, a carotid injury, a vertebral artery occlusion, an epidural bleed, and a penumbra called an infarct. The breakdown of 58 missed ischemic injuries included 44 that were acute/subacute, 7 attributed to hypoxic-ischemic encephalopathy, 5 chronic ischemic insults, as well as 2 multiple, thought to be embolic. Figure 2 demonstrates the specific locations of acute infarcts, with MCA territory being the most common, followed by thalamic and cerebellar territories. The average size of infarct errors was: MCA (1.6 cm; range, 0.5–3.5 cm); cerebellar (2.0 cm; range, 0.4 –3.9 cm); thalamic (1.4 cm; range, 0.6–2.4 cm), while the less frequently seen errors involved the ACA at 2.6 cm (range, 0.7–4.5 cm), brainstem at 0.8 cm (range 0.4–1.1 cm), and embolic at 0.6 cm (range, 0.4 –0.8 cm).

Table 2:

Error type based on pathology

Pathology	Total Errors (n = 175)	Percentage (%)
Acute/subacute infarct	44	25.1%
Aneurysm	24	13.7%
Subdural hematoma	17	9.7%
Significant arterial stenosis	11	6.3%
Dural sinus occlusion	11	6.3%
Epidural hematoma	8	4.6%
Large-vessel occlusion	8	4.6%
Overcall	7	4.0%
Arterial dissection	7	4.0%
Hypoxic-ischemic encephalopathy	7	4.0%
Chronic infarct	5	2.9%
AVM-AVF	5	2.9%
Subarachnoid hemorrhage	5	2.9%
Hemorrhagic contusion	3	1.7%
Embolic infarct	2	1.1%
Intraventricular hemorrhage	2	1.1%
Retroclival bleed	2	1.1%
Vasospasm	2	1.1%
Subclavian steal	1	0.6%
Sinus pericranii	1	0.6%
Deep neck vein occlusion	1	0.6%
Growing hemorrhage	1	0.6%
PRES	1	0.6%

Note:—PRES indicates posterior reversible encephalopathy syndrome.

FIG 1.

Categories of errors.

FIG 2.

Errors in acute/subacute infarct location.

Figure 3 shows that of 58 cases of arterial pathology, 24 were attributed to cerebral aneurysms; 11, to significant stenoses; 8, to large-vessel occlusion (5 located at the M2 segment, as well as 1 each in the M1, PCA, and basilar artery, respectively); 7, to arterial dissection; 5, to AVM/AVF; 2, to vasospasm, and 1, to subclavian steal. With the exception of a 5.3-cm abdominal aortic aneurysm that was not discussed on MR imaging of the lumbar spine, the average size of aneurysms was 0.5 cm (range, 0.2–0.7 cm). Table 3 shows the location of missed aneurysms: 14 of the 25 aneurysmal errors (56.0%) involved the ICA, with 5 in the cavernous segment, 4 in the clinoid and supraclinoid segments, 3 in the terminus, and 2 paraophthalmic aneurysms. There were 4 MCA aneurysms as well as 2 aneurysms seen in the ACA and posterior communicating artery, while the abdominal aorta and PCA had 1 aneurysm. Figure 4 demonstrates the location of the combined 27 cases of significant stenosis, large-vessel occlusion, and arterial dissection, with the ICA and MCA, specifically the distal M2 segment, being the most common. Of the 8 large-vessel occlusions, 5 were of the M2 segment of the MCA, with single cases involving the M1 segment of the MCA, the PCA, and the basilar artery respectively.

FIG 3.

Errors in overall vascular pathology.

Table 3:

Location of missed aneurysm

Aneurysm Location	Total (n = 24)	Percentage
ICA, cavernous	5	20.8%
MCA	4	16.7%
ICA, clinoid	4	16.7%
ICA, terminus	3	12.50%
ICA, paraophthalmic	2	8.3%
AcomA	2	8.3%
PcomA	2	8.3%
Abdominal aorta	1	4.2%
PCA	1	4.2%

Note:—AcomA indicates anterior communicating artery; PcomA, posterior communicating artery.

FIG 4.

Errors in arterial pathology.

The most common missed hemorrhagic lesions were subdural hematomas (n = 17), followed by epidural hematomas (n = 8) and subarachnoid hemorrhage (n = 5). We found ≤3 errors of the following conditions: hemorrhagic contusion (n = 3), intraventricular hemorrhage (n = 2), retroclival bleed (n = 2), and expanding parenchymal hemorrhage (n = 1) using a standard 3-orthogonal-plane measurement and comparison per routine interpretation determined at the time of the RADPEER process (Fig 5). The average size of hemorrhagic lesions included 0.5 cm for subdural hematoma (range, 0.3–1.0 cm), 0.4 cm for epidural hematomas (range, 0.3–0.5 cm), 0.6 cm for contusions (range, 0.2–0.9 cm), and 0.3 cm for retroclival bleed (range, 0.2–0.4 cm). The growing hemorrhage on the follow-up CT 12 hours later increased to a volume of 10.9 mL from an initial 5.7 mL, which was not reported. The most common error involving the venous system was dural sinus occlusion, which was seen in 11 cases; the remaining 2 cases were sinus pericranii and a deep neck vein thrombosis.

FIG 5.

Errors in hemorrhage detection.

The imaging technique with errors was further categorized for the top 3 most common pathologies. For the 44 acute/subacute ischemic strokes, 21 were on MR imaging of the brain, 12 were on CTA of the head, and 11 were on noncontrast head CT. Of the 24 aneurysms (seen on 23 examinations; 1 CTA study contained bilateral missed aneurysms), 14 were on CTA of the head, 5 were on MR imaging of the brain, 3 were on MRA of the brain, and 1 abdominal aortic aneurysm was seen on MR imaging of the lumbar spine. Seventeen subdural hematomas were on 15 noncontrast head CTs, 1 on MR imaging of the brain, and 1 on CT of the sinus and face.

Although random, our method of collecting cases has yielded a very narrow and consistent range of total diagnostic neuroradiology error rates shown to range between 0.20% and 0.25% (2014–2020, internal records), which may be interpreted as indicative of reproducibility and reliability.

DISCUSSION

In this series, most cerebrovascular diagnostic errors were perceptual and clinically significant and occurred in the emergency/inpatient setting and were associated with higher-volume shifts. Although uncommon, cerebrovascular diagnostic errors had a significant clinical impact on most patients because 54.8% of errors involved acute/subacute ischemic injuries, cerebral aneurysms, subdural hematomas, and significant arterial stenosis. Every year in the United States, >215,000 deaths are thought to result from medical errors, third in prevalence only after cancer and heart disease,²² for which the impact of radiologic diagnostic errors is considered major.^5,23 The current standard used to define errors in radiology interpretation is a discrepancy in interpretation that differs substantially from the consensus of peers, for which the current evaluation mechanism in the United States is the aforementioned RADPEER process.^20,21 In that sense, a criterion standard is lacking; objective reference standards such as postmortem-proved or surgically confirmed diagnoses are, indeed, very uncommonly available.²⁴

Because most cerebrovascular conditions have a poor natural history, particularly severe consequences can be expected to result from an erroneous or delayed diagnosis.²⁵ Therefore, diagnostic errors in this organ system have relatively heightened relevance in comparison with other, more forgiving anatomic areas and disease processes.^26,27 For instance, the reported prognosis of untreated acute ischemic stroke from large-vessel occlusion is consistent with high rates (up to 64%) of severe functional dependence or death (measured by the 90-day mRS, with scores of >2); for non-large-vessel occlusion strokes, the reported average expected rates of functional dependence or death are around 24%.^19,28 Our study contained 44 missed, acute/subacute ischemic injuries.

The natural history of intracranial aneurysms suggests annual rupture rates around 1.1%–1.3%.^29,30 Although the prevalence of intracranial aneurysms worldwide may be far larger than previously thought, possibly ranging between 5% and 8%,^31,32 hemorrhagic rates may be high in certain populations with specific risk factors.^33-35 A cumulative hemorrhagic rate of 10.5% at 10 years has been estimated for previously unruptured aneurysms,³⁰ and the 10-year mortality rate for ruptured, untreated aneurysms was reported at around 76%.³⁶ Our study contained 24 missed aneurysms.

Subdural and epidural hematomas can have devastating morbidity and mortality if not recognized and acted on in a timely manner.³⁷ Subdural hematomas may often be treated conservatively on the basis of objective criteria such as the Glasgow Coma Scale score, the width of the hematoma, or the amount of midline shift.^38,39 A study by Ryan et al⁴⁰ published in 2012 in the Journal of Trauma and Acute Care Surgery explored mortality and functional outcomes in adult patients at a Level 1 Trauma Center. That study demonstrated that inpatient mortality was 16% (15% for surgical patients, 17% for the nonsurgical group), lower than that in previous studies from the 1970s to 2000s, which reported 22%–57% mortality ranges.^37,41,42 The authors credited the widespread and liberal use of CT, which allows detecting patients with less severe hematomas, and recent advances in surgical techniques as the main reasons for improved outcomes. Our study contained 17 missed subdural hematomas. Epidural hematomas may have better prognoses compared with subdural hemorrhage with prompt surgical evacuation, with a 17% reported mortality; however, that figure rises to 65% if surgery is performed >2 hours from the onset of coma.³⁷ Our study contained 8 missed epidural hematomas.

The prognosis of large intracranial artery stenotic disease was evaluated in multiple trials,⁴³ which have found an annual risk of death around 11.2% and an annual risk of stroke of 12.5%–17.1%.^44,45 Our study contained 11 missed large-artery stenoses.

Cerebral venous thrombosis, which encompasses both dural sinus and cortical vein thrombosis, accounts for 0.5% of all strokes.⁴⁶ Common predisposing factors are hypercoagulable disorders (known or cryptic), trauma, and infection, though rare occurrences have recently been shown to be linked to some coronavirus disease 2019 (COVID-19) vaccines.⁴⁶ Despite being uncommon, cerebral venous thrombosis is potentially fatal if diagnosis and treatment are delayed,⁴⁷ with median reported mortality rates of 5.6% (range, 0%–15.2%).⁴⁸ Our study contained 11 missed venous thromboses.

Although relatively benign compared with other causes of stroke, possibly due to the younger age of patients, the natural history of cervical artery dissection includes neurologic injury and severe hemodynamic compromise.⁴⁹ Recurrent TIA, stroke, or death may be seen in up to 15% within 2 weeks following intimal injury,⁵⁰ and recurrence of dissection is seen in roughly 1%–8% of patients.^51,52 Our study contained 7 cases of missed dissections.

The mean number of interpreted studies per shift was a factor that reached statistical significance, because more errors were made on busier shifts, showing a higher propensity for diagnostic errors with higher-volume reads, as suggested by the literature.^{5,7,10,11,25-27} Again, most errors in our series were perceptual errors, ie, radiologists did not identify the abnormality. Causes of errors may include type I heuristic thinking (rapid problem-solving based on presumptions and previous experiences), cognitive biases (satisfaction of search, availability, anchoring and framing biases), as well as systemic causes such as increased workload, understaffing, workplace interruptions, software failure, and insufficient clinical information.^1,6,8,9 Therefore, our response to the study findings included heightened attention paid to identifying the perceptual errors in structured radiology reports to quantify anatomic blind spots in vascular pathology to help create a checklist and minimize future errors. This checklist of anatomic regions requiring hypervigilance included areas susceptible to ischemic injury, locations of commonly missed cerebral aneurysms, and stenosis, as well as the large and small extra-axial spaces to look for extra-axial hematomas (subdural and epidural). The MCA and distal branches, ICA, thalamus, as well as the cerebellum made up the location of 57 (32.6%) errors in this article. Adding these areas to a checklist of items for which additional scrutiny is required would have a significant impact on reducing errors. Having such systems in place for error identification and reduction/elimination is not just important for patient care but also serves to decrease medicolegal exposure.

Potential avenues toward error reduction indeed include reporting template adjustments to include areas affected by frequent errors, additional educational initiatives, multidisciplinary/multispecialty conferences, subspecialty morbidity and mortality conferences, and occasionally double reading.^8,25 Although recent evidence exists that double reading of CTA might reduce error rates,¹⁷ as suggested by Garland⁵³ as far back as 1949, there is generally strong structural resistance to double reading in the United States because the second read is not reimbursed and for concern about the additional time commitment in a productivity-based system.⁵⁴

Limitations to our study include those inherent to a retrospective design and a relatively low rate of detected misses, given the volume of interpreted examinations. No standardized methodology is currently available for the recording of errors in QA databases, which has to be taken into account when interpreting our findings. Our neuroradiology section reads a large number of studies (>240,000 studies in this article), so a second reading cannot be reasonably used widely; however, we used the double-reading method in collecting cases for our QA database and when assessing addended reports, which has proved efficient in uncovering a large number of diagnostic errors that could then be further analyzed. Our robust quality-improvement program methods used to identify errors uncovered approximately 10% of our total errors (internal data), and any strategy aimed at reducing errors based on the findings in our sample of discovered errors is expected to be applicable to our studies. Although our QA methods may be biased toward detecting clinically significant errors, we believe that such methodology does the following: 1) serves to collect as many cases as feasible in a random manner, 2) allows root-cause analysis, and 3) provides an opportunity for the design of interventional strategies aimed at reducing diagnostic errors.

A direct comparison of our neurovascular error rates with those of other practices is not possible because no relevant literature exists that discusses vascular error rates. Why our perceptual error rates are at the high end of the spectrum of published literature is currently being evaluated, with possible factors including technical issues in vascular imaging, volumes of interpreted examinations per shift, significant underappreciation of neuroradiology error rates in the literature, and, possibly, our unique methodology of collecting cases for the QA database. Error identification and implementation of corrective systems significantly impact patient care and medicolegal exposure. We are also actively investigating other variables that may show potential correlations with diagnostic errors. For example, there is evidence that high participation rates at multidisciplinary tumor boards is strongly correlated with low attending errors.⁵⁵ These data may offer additional guidance on future interventions toward error reduction.

CONCLUSIONS

Most vascular errors noted in our series were perceptual and clinically significant, occurring in the emergency/inpatient setting and associated with higher-volume shifts. Acute/subacute ischemia, aneurysms, and subdural hematomas represent more than half of all errors. Hopefully, errors in vascular neuroradiology could be minimized if search patterns were altered to include hypervigilance of the sites most frequently affected by disease.

ABBREVIATIONS:

ACA

anterior cerebral artery

PCA

posterior cerebral artery

QA

quality assurance