Carotid Artery Stenosis: Wide Variability in Reporting Formats—A Review of 127 Veterans Affairs Medical Centers

The reporting of clinically significant carotid stenosis varied widely in the largest integrated health care system in the United States.

Abstract

Purpose:

To determine whether radiology reports describe clinically significant carotid arterial stenosis in a consistent format that is actionable by ordering clinicians.

Materials and Methods:

This study was HIPAA compliant. Informed consent was waived. Institutional review board approval was obtained for this retrospective chart review, which included radiology reports of carotid artery imaging for patients hospitalized with ischemic stroke at 127 Veterans Affairs medical centers in 2006–2007. “Clinically significant results” were defined as results with at least 50% stenosis or at least moderate stenosis, excluding complete occlusion. How often clinically significant results were reported as an exact percentage stenosis (such as 60%), range (such as 50%–69%), or category (such as moderate) was determined. Among results reported as a range, how often the range bracketed clinical thresholds of 50% and 70% (typically used to determine appropriateness of carotid arterial revascularization) was determined.

Results:

Among 2675 patients, there were 6618 carotid imaging results, of which 1015 (15%) were considered clinically significant. Among 695 clinically significant results at ultrasonography (US), 348 (50%) were described as a range, and another 314 (45%) were reported as an exact percentage stenosis. Among the 348 clinically significant US results reported as a range, 259 (74%) bracketed the thresholds of 50% or 70%. For magnetic resonance angiographic results, 48% (106 of 221) qualitatively described clinically significant results as a category, 38% (84 of 221) as an exact percentage stenosis, and 14% (31 of 221) as a range.

Conclusion:

In this national health care system, the manner in which clinically significant carotid arterial stenosis was reported varied widely.

© RSNA, 2012

Introduction

The landmark North American Symptomatic Carotid Endarterectomy Trial (NASCET) established the efficacy of carotid endarterectomy for patients with symptomatic carotid stenosis. For patients with severe symptomatic carotid artery stenosis, defined as 70%–99%, carotid endarterectomy greatly decreased the 2-year risk for stroke compared with medical therapy (26% vs 9%) (1). For patients with moderate symptomatic carotid artery stenosis, defined as 50%–69%, carotid endarterectomy modestly decreased the 5-year risk for stroke compared with medical therapy (20% vs 15%) (2). Once clinicians know the magnitude of carotid stenosis, they can discuss the natural history, as well as the risks and benefits of carotid revascularization. National guidelines provide therapeutic recommendations on the management of symptomatic carotid stenosis on the basis of the NASCET ranges of carotid stenosis (3,4).

To facilitate decision-making based on this evidence, the results of clinically significant carotid artery stenosis should conform to the format used in these trials and guidelines. The value of a carotid imaging modality is based on how accurately it can help identify patients with stenosis within these NASCET ranges (5). However, the reporting of carotid stenosis remains largely unstudied.

The Veterans Health Administration performed a comprehensive chart review of veterans admitted in 2007 with an ischemic stroke. We determined whether radiology reports describe clinically significant carotid stenosis in a consistent format that is actionable by ordering clinicians.

Materials and Methods

This retrospective chart review was approved by the Veterans Affairs Los Angeles and the Veterans Affairs Indianapolis Institutional Review Boards and was compliant with the Health Insurance Portability and Accountability Act. The institutional review boards waived the need to obtain informed consent for this retrospective analysis of data.

Setting and Patient Sample

The Department of Veterans Affairs (VA) includes the Veterans Health Administration, the largest health care system in the United States. In 2009, the VA Offices of Quality and Performance, Patient Care Services, and Stroke Quality Enhancement Research Initiative collaborated to conduct the Office of Quality and Performance Stroke Special Study (6). The inclusion criteria consisted of veterans admitted to any of 133 acute care VA medical centers in fiscal year 2007 (October 1, 2006 to September 30, 2007) with a diagnosis code of ischemic stroke. A sample of 5000 medical records was identified by including all veterans at low-volume facilities (≤55 patients with ischemic stroke in fiscal year 2007) and an 80% random sampling of veterans at high-volume facilities (>55 patients with ischemic stroke in fiscal year 2007).

Data Collection

Trained abstractors at the West Virginia Medical Institute reviewed the charts. The Institute is a federally designated Medicare quality improvement organization contractor and provides quality assurance services to the VA. All VA medical centers use an enterprise-wide electronic health care record system, thus allowing the abstractors to remotely retrieve and review medical charts in the VA. We excluded the following from analyses of the Offices of Quality and Performance cohort: records for patients determined not to have a diagnosis of acute stroke (n = 534), those admitted for carotid revascularization (n = 89), those admitted for rehabilitation (n = 190), those who sustained a stroke while an inpatient (n = 200), or those for whom an error occurred during abstraction (n = 22). After exclusions, the sample size was 3965 patients. Chart abstractors then performed a chart review of 307 data elements on those patients.

Carotid Artery Imaging Modalities

The abstractors obtained all results of carotid artery testing from a period of 12 months before admission to 2 months after admission. Carotid imaging tests consisted of any of the following modalities: carotid artery ultrasonography (US), magnetic resonance (MR) angiography of the neck, computed tomographic (CT) angiography of the neck, and conventional angiography of the neck (7).

Carotid Artery Stenosis

Abstractors were instructed to first look for numeric results of carotid artery tests. If the result was documented as an exact percentage stenosis, they recorded the single number. If the result was documented as a range, they recorded both the lower and the upper limits of that range. If no numeric results were available, they recorded whether one of following qualifiers was considered to be consistent with the result: no stenosis, mild or clinically insignificant stenosis, moderate stenosis, severe or clinically significant stenosis, or occlusion. Abstractors were also given separate options for recording “no results found” or “test not performed.”

Analysis

The unit of analysis was the carotid artery. Most text reports contained two results: one for the right and one for the left carotid artery. We excluded results of “no stenosis” because these could be documented equivalently as an exact percentage of “0% stenosis” or a category of “no stenosis.” Similarly, we excluded results of occlusion because these could be documented equivalently as an exact percentage of “100% stenosis” or a category of “occlusion.” We then labeled the remaining test results as “clinically significant” if the exact percentage stenosis was 50% or greater, any part of a range was 50% or greater, or the category was reported as “moderate” or “severe or clinically significant” because such persons met the inclusion criteria for carotid artery revascularization. Among the clinically significant results, we reported how often the result was reported in the formats of exact percentage stenosis, range, or category for each imaging modality.

Among the clinically significant US test results reported as a range, we evaluated how often the range matched that used in the NASCET studies and how often it bracketed the key thresholds of 50% and 70%. We focused particularly on US results because US is the most commonly used carotid imaging modality, and an existing guideline provides a table to convert US findings to ranges used in NASCET (8).

Finally, we analyzed the variation of reporting within sites. We restricted the analysis to sites that had at least five clinically significant results for a particular imaging modality. We then calculated the proportion of the clinically significant results at the site that used an exact percentage stenosis, range, or category.

Results

A total of 6618 internal carotid arteries were imaged in 68% (2675 of 3965) of patients in the sample. After exclusion of carotid artery imaging results showing no stenosis, unremarkable stenosis, and carotid occlusion, 15% (1015 of 6618) showed clinically significant stenosis (Figure). Among clinically significant US results, 50% (348 of 695) were reported as a range, 45% (314 of 695) were reported as an exact percentage stenosis, and 5% (33 of 695) were reported as a category. Among clinically significant MR angiographic results, 48% (106 of 221) were reported as a category, 38% (84 of 221) as an exact percentage stenosis, and 14% (31 of 221) as a range. Combining clinically significant results from CT angiography and conventional imaging, 31% (31 of 99) of the clinically significant results were reported in a format other than an exact percentage stenosis.

Flowchart identifies clinically significant carotid imaging results. CTA = CT angiography, MRA = MR angiography, U/S = ultrasonography.

When ranges were used in US reports to describe clinically significant carotid stenosis (n = 348), they were usually different from those used in NASCET (Table 1). Only 15% (53 of 348) of the ranges exactly conformed to those used in NASCET. Forty-six were in the 50%–69% range, and seven were in the 70%–99% range. A further 11% (39 of 348) had intervals entirely within the 50%–69% or the 70%–99% range; although these do not conform to NASCET, an ordering clinician could categorize the patient as having moderate or severe stenosis.

Table 1.

US Ranges of at Least 50% Stenosis

^*Data in parentheses are percentages.

On the other hand, 74% (259 of 348) crossed the 50% or 70% threshold used in NASCET (Table 1). All the 151 ranges that crossed the 50% threshold used 50% as the upper limit of their range. In addition, 31% (108 of 348) of the ranges crossed the threshold of 70% that separates moderate stenosis and severe stenosis. Only 20 of these ranges had an upper limit of exactly 70%; the other 88 ranges had a considerable proportion of their interval in the 50%–69% and in the 70%–99% ranges, thus preventing an ordering clinician from determining whether the patient possessed moderate or severe stenosis.

Despite variability in the use of formats across the VA system, most individual VA medical centers consistently used a predominant format at their site (Table 2). Of the 46 VA medical centers that had five or more clinically significant US results, 59% (27 of 46) used just one of the three formats—exact percentage stenosis, range, or category—to describe all results of clinically significant carotid stenosis.

Table 2.

Consistency of Reporting within a VA Medical Center

Note.—NA = not applicable.

^*Formats included exact percentage stenosis, range, or category.

Discussion

In this study, we demonstrated considerable variability in how clinically significant carotid stenosis was reported within the VA system. Although ordering clinicians use results in NASCET ranges to guide decision making on carotid revascularization, many of the carotid imaging reports are not presented as a NASCET range.

The variability of formats suggests that different algorithms are used to translate findings into results. Several criteria have been proposed to translate US findings to a NASCET range (8–11). The widespread use of non-NASCET ranges, exact percentage stenosis, and categories suggests that a wide set of criteria are used to interpret findings.

The importance of standardizing radiology reports through a common structure and language has been identified by a recent Intersociety Conference (12). Several professional recommendations and policy initiatives have successfully standardized mammography reports. To address concerns about variations of mammography reports, the Breast Imaging Reporting and Data System initiative began 20 years ago to bring uniformity in the reporting of mammography results. One recommendation in the Breast Imaging Reporting and Data System that is particularly relevant to our study is an agreement to use a restricted set of options in the impressions section of a report so that ordering clinicians are given clear recommendations (13). Use of the lexicon from the Breast Imaging Reporting and Data System has facilitated clinical care, education, and research. A set of standards has also been proposed for cardiovascular imaging (14).

We were surprised to find that many clinically significant US results were reported as an exact percentage stenosis. The precision of US has been described as 10% at best, so reporting results as an exact percentage stenosis implies a level of precision that is not achievable (8). In addition, we were likewise surprised to find that a considerable number of clinically significant CT angiographic and conventional results were reported in a format other than an exact percentage stenosis. Such results reduce the precision that is possible with these imaging modalities (15). Finally, when ranges were used to describe stenosis in a US report, many did not conform to NASCET ranges.

This study had several limitations. First, it is possible that abstractors erred in their chart review. For example, a text report of “up to 50% stenosis” may have been recorded as an exact percentage of 50% stenosis instead of a range of 0%–50% stenosis. Standardized report formatting would make it less likely that such errors of interpretation would occur. Second, we do not have any information on the interpreting clinician. US vascular laboratories may reside within radiology or vascular surgery departments. This could affect the way in which results are reported, with results possibly tailored to the individual facility’s criteria for surgery. In addition, we do not have any information on whether the interpreting clinician is directly employed by the VA or working on a fee-basis contract; enforcing standardized reporting would be more difficult among clinicians who practice in multiple settings that use different formats.

One cause of the observed variability in reporting could be insufficient training of the interpreting clinician. In our study, however, we did not have indicators of that training, such as specialty, board certification, or annual number of studies, to analyze this relationship. A very different cause could be that interpreting clinicians are conforming to preferences of ordering clinicians at a site. For example, surgeons at some sites may prefer the 80% US threshold as suggested by the European Carotid Surgery Trial instead of the 70% threshold used by NASCET (16). If a standardized reporting template can be identified, then several key stakeholders can enforce implementation of that standard. Further enforcement would probably occur through pay-for-performance programs, wherein payers such as Medicare are attempting to standardize key processes of care, including the format of radiology reports (17). Similarly, accreditation organizations could consider the format of radiology reports as part of their checklist in evaluating or renewing a facility (18,19). However, we also acknowledge that use of a structured reporting system has not always shown that reports are more accurate and clear, as was recently reported in a study of brain imaging (20). Therefore, activities to standardize carotid imaging reports need to be carefully studied to ensure accuracy and clarity (17–19).

In summary, we found substantial variation in the reporting of clinically significant carotid stenosis in the largest integrated health care system in the United States. Because reporting appears consistent within individual sites, the next step would be to obtain consensus among ordering and treating clinicians across all sites in this national system toward a standardized report that facilitates evidence-based decision making.

Advances in Knowledge.

• • There is substantial variation in the format of reporting clinically significant (>50% or at least moderate stenosis) carotid stenosis in the Veterans Health Administration, the largest integrated health care system in the United States.

• • Clinically significant US results of greater than 50% or at least moderate stenosis were reported as a range approximately half of the time.

• • Even when clinically significant US results were reported as a range, these results usually did not conform with those used in the North American Symptomatic Carotid Endarterectomy Trial and recommended in national guidelines.

Implication for Patient Care.

• • Radiologists may be using different criteria to translate imaging findings into results.

Disclosures of Conflicts of Interest: E.M.C. No relevant conflicts of interest to disclose. D.M.B. No relevant conflicts of interest to disclose. S.E. No relevant conflicts of interest to disclose. S.D.V. No relevant conflicts of interest to disclose. S.O. Financial activities related to the present article: institution receives fees for participation in review activities and payment for writing or reviewing manuscript from Research and Development, VA Medical Center, Indianapolis, Indiana. Financial activities not related to the present article: none to disclose. Other relationships: none to disclose. L.S.W. No relevant conflicts of interest to disclose. S.K. No relevant conflicts of interest to disclose.