Abstract
Rationale and Objectives
As breast MRI use grows, benchmark performance parameters are needed for auditing and quality assurance purposes. We describe the variation in breast MRI abnormal interpretation rates (AIRs) by clinical indication among a large sample of U.S. community practices.
Materials and Methods
We analyzed data from 41 facilities across five Breast Cancer Surveillance Consortium imaging registries. Each registry obtained IRB approval for this HIPAA compliant analysis. We included 11,654 breast MRI exams conducted in 2005–2010 among women aged 18–79 years. We categorized clinical indications as: 1) screening; 2) extent of disease; 3) diagnostic (e.g., breast symptoms); and 4) other (e.g., short-interval follow-up). We characterized assessments as positive (i.e., BI-RADS 0, 4, and 5) or negative (i.e., BI-RADS 1, 2, 6), and provide results with BI-RADS 3 categorized as positive and as negative. We tested for differences in AIRs across clinical indications both unadjusted and adjusted for patient characteristics and registry, and assessed for changes in AIRs by indication over time.
Results
When categorizing BI-RADS 3 as positive, AIRs were 21.0% (95% CI: 19.8, 22.3) for screening, 31.7% (95% CI: 29.6, 33.8) for extent of disease, 29.7% (95% CI: 28.3, 31.1) for diagnostic, and 27.4% (95% CI: 25.0, 29.8) for other indications (p<0.0001). When categorizing BI-RADS 3 as negative, AIRs were 10.5% (95% CI: 9.5, 11.4) for screening, 21.8% (95% CI: 19.9, 23.6) for extent of disease, 17.7% (95% CI: 16.5, 18.8) for diagnostic, and 13.3% (95% CI: 11.6, 15.2) for other indications (p<0.0001). The significant differences in AIRs by indication persisted even after adjusting for patient characteristics and registry (p<0.0001). In addition, for most indications, there were no significant changes in AIRs over time.
Conclusion
Breast MRI AIRs differ significantly by clinical indication. Practices should stratify breast MRI exams by indication for quality assurance and auditing purposes.
Keywords: breast magnetic resonance imaging, audit, quality assurance
INTRODUCTION
Breast magnetic resonance imaging (MRI) is the most sensitive modality for detecting breast cancer, often identifying malignancy otherwise occult by mammography, ultrasound, and clinical breast examination (1). As the technology improves and the interpretation and reporting by radiologists becomes standardized, breast MRI is used for an increasing number of purposes, including high-risk screening, evaluation of extent of malignancy, evaluation of patients with metastatic axillary adenopathy and unknown primary cancer, and surveillance after cancer treatment (2–9). Moreover, the technology has now become readily available in community settings throughout the U.S. with interpretation and reporting completed by both subspecialty-trained breast imagers and general radiologists (10).
To facilitate consistent reporting of and management recommendations for breast MRI findings, the American College of Radiology published the first edition of the Breast Imaging Reporting and Data System (11) MRI lexicon in 2003, with the most recently revised edition published in 2013 (12). Similar to the previously established BI-RADS mammography lexicon, the breast MRI lexicon provides common terminology for describing MRI findings. Standardized use of the lexicon and BI-RADS assessment categories allows for improved communication among radiologists and clinicians with regards to suspicious imaging findings and clinical recommendations (13). Recent studies have shown that the MRI BI-RADS assessment categories can accurately predict the risk of malignancy (14, 15).
Under the Medicare Improvement for Patients and Providers Act (MIPPA) of 2008, all radiology practices that bill for the technical component of breast MRI under part B of the Medicare Physician Fee Schedule must be accredited as of January 1, 2012 to qualify for reimbursements (16). To be awarded accreditation, practices must meet minimum quality standards, including mandatory use of MRI BI-RADS lexicon in reporting. In addition, imaging centers must maintain a medical outcomes audit program to follow-up positive BI-RADS assessments and correlate pathology results with suspicious imaging findings (17). In general, medical audits are widely recognized as important and effective quality assurance tools for improved patient care (18, 19).
Creating and maintaining a medical outcomes audit program for mammography can be difficult for community radiology practices without robust linkages to pathology and oncology databases (19). It is expected that similar challenges will affect the development of breast MRI outcomes audits, and practices will need to rely upon data that are readily available, such as clinical indications and image-guided biopsy results, to begin developing medical audit programs. Given the requirement for standardized use of MRI BI-RADS assessments, overall abnormal interpretation (i.e., recall) rates for breast MRI (recorded as a proportion of MRI exams with positive BI-RADS assessments) are realistic audit parameters readily determined by most community radiology practices developing breast MRI quality assurance programs.
Our study objective was to estimate abnormal interpretation rates (AIRs) overall and by clinical indications for breast MRI encountered in routine community practice. We provide a descriptive analysis of all breast MRI exams performed across a geographically diverse set of radiology practices over a six-year period. Based on our experience with mammography audits, we hypothesized that the proportion of positive BI-RADS assessments differs for screening versus diagnostic MRI exams.
MATERIALS AND METHODS
Data Source
Each of the National Cancer Institute-funded Breast Cancer Surveillance Consortium (BCSC) registries sends data for breast MRI exams to a central Statistical Coordinating Center (SCC) for pooled analyses. Each registry and the SCC follow previously reported data management and quality control procedures to ensure accurate data collection across registries (20). Each registry and the SCC obtain institutional review board approval for either passive or active patient consent or waiver of consent, linkage of patient characteristics to imaging-related outcomes, and performance of statistical analyses and results reporting. The SCC and each registry have a Federal Certificate of Confidentiality and other protections for the identity of individual women, physicians, and practices that are subjects of this research. All study procedures were compliant with the Health Insurance Portability and Accountability Act.
For this descriptive analysis, we used data from five breast imaging registries of the BCSC: the San Francisco Mammography Registry, Vermont Breast Cancer Surveillance System, New Hampshire Mammography Network, Carolina Mammography Registry, and Group Health Cooperative (Washington State). These registries comprise a geographically diverse group of breast imaging facilities in U.S. community settings that prospectively collect patient demographic and clinical information and breast imaging interpretation data as part of routine clinical care. A total of 41 individual imaging facilities across the five registries provided breast MRI data.
Study Population
We included data from all breast MRI exams conducted in 2005–2010 among women aged 18–79 years with reported clinical indication(s) and final BI-RADS assessment across the five BCSC registries. The registries collected standardized data on breast MRI exams, including the clinical indication(s) for the exam and the final BI-RADS assessments for each breast, from electronic data systems, billing information, and abstraction of radiology reports. Patient risk factor information was obtained at the time of the MRI exam or from the most recent mammogram within one year prior to the breast MRI exam, including: the patient’s age, race/ethnicity, any personal history of breast cancer (self-reported or via linkage with tumor registries), family history (first degree relative) of breast cancer, and the reported mammographic BI-RADS breast density.
Clinical Indication Categorization
We stratified reported clinical indications for breast MRI into one of the following four categories: 1) screening (i.e., asymptomatic); 2) extent of disease; 3) diagnostic; and 4) other. Our “diagnostic” indication category included MRI exams performed for additional evaluation of a recent abnormality identified by mammography or ultrasound, evaluation of specific breast symptoms, and differentiation of cancer recurrence from post-surgical scar. Our “other” indication category included MRI exams performed for short interval follow-up of a probably benign MRI finding, evaluation of treatment response to neoadjuvant chemotherapy, and all other recorded indications not conforming to any other indication category. For the minority of exams with multiple reported clinical indications (11%), we categorized clinical indication using the following hierarchy to isolate true screening breast MRI examinations in asymptomatic women from examinations performed for other reasons: 1) extent of disease; 2) evaluation of treatment response to neoadjuvant chemotherapy (= other); 3) axillary adenopathy of unknown primary (= other); 4) additional evaluation of recent abnormality on another breast imaging modality (= diagnostic); 5) evaluation of breast problem (= diagnostic); 6) differentiation of cancer recurrence from post-surgical scar (= diagnostic); 7) short interval follow-up of prior breast MRI examination (= other); 8) screening; and 9) other.
BI-RADS Assessment Categorization
We characterized overall assessments as either positive (i.e., needing further initial or short-term evaluation; BI-RADS 0, 3, 4, and 5) or negative (i.e., no further imaging evaluation needed; BI-RADS 1, 2, 6). We present results by categorizing BI-RADS 3 assessments as both positive and negative. The dichotomous or binary categorization of BI-RADS assessments allows for calculation of AIRs defined as the proportion of examinations with positive assessments by study clinical indication.
We included both unilateral and bilateral breast MRI exams in our analysis. Each breast in bilateral breast MRI exams was given a separate BI-RADS assessment. We categorized BI-RADS assessment as the highest order for the examination, and used the following BI-RADS assessment hierarchy: 5 > 4 > 3 > 0 > 6 > 2 > 1. Therefore, if a suspicious abnormality (BI-RADS 4) was identified in one breast but no abnormality was identified in the other (BI-RADS 1), then the overall exam assessment would be BI-RADS 4. Similarly, if an additional suspicious abnormality (BI-RADS 4) was identified in a patient with known, biopsy-proven malignancy (BI-RADS 6), then the overall exam assessment would be BI-RADS 4 since additional diagnostic work-up is recommended (it is assumed that BI-RADS 6 assessments do not require additional diagnostic work-up since the malignancy has already been proven by tissue sampling).
Statistical Analysis
We performed all analyses using SAS® Version 9.2 (SAS Institute, Cary, NC). Our univariate descriptive statistics included frequencies of the patient characteristics (age, race/ethnicity, family history of breast cancer, personal history of breast cancer, and BI-RADS breast density on mammography) stratified by four clinical indications. We calculated 95% exact confidence intervals for the BI-RADS assessment and used the Pearson chi-square test to identify statistically significant differences in positive versus negative BI-RADS assessment across clinical indications. In multivariate analyses, we used a logistic regression model to adjust for BCSC registry as well as patient age, race/ethnicity, family history of breast cancer, personal history of breast cancer, and mammographic breast density. Finally, we performed the Cochran-Armitage test for trend to identify significant changes in breast MRI AIRs by clinical indication over time. We considered a two-sided p<0.05 as statistically significant.
RESULTS
Population Characteristics
Between 2005 and 2010, a total of 15,242 breast MRI exams were performed across five BCSC breast imaging registries. Of these, 3,533 exams (23%, 3,533/15,242) did not meet our inclusion criteria and we excluded these from our analysis (589 exams due to patient age > 79 years-old, 2,451 exams due to missing clinical indication, 493 exams due to missing BI-RADS assessments, and 55 exams performed to evaluate for implant rupture). Thus, we included a total of 11,654 breast MRI exams performed between 2005 and 2010 in our analysis.
The characteristics of patients undergoing breast MRI exams during the study period are described by clinical indication in Table 1. Briefly, 36.9% (4300/11654) of breast MRI exams were performed for diagnostic purposes, 34.2% (3989/11654) were performed for screening, 16.8% (1954/11654) were performed to evaluate extent of disease among newly diagnosed breast cancer patients, and 12.1% (1411/11654) were performed for other clinical indications. The majority of breast MRI exams were performed among women who are in the routine age-based mammographic screening population, aged 40 to 79 years (82.4%, 9599/11654). The majority of breast MRI exams were performed among Caucasian women (83.9%, 8973/10689), women with a personal history of breast cancer (74.9%, 6114/8166), women without a first-degree family history of breast cancer (63.8%, 7054/11050), and women with heterogeneously or extremely dense breasts on mammography (65.7%, 5414/8242).
Table 1.
Study Population Characteristics by Breast MRI Clinical Indication
| Patient Characteristic | Screening | Extent of Disease | Diagnostic | Other | Total | p-value | |||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| N | (%) | N | (%) | N | (%) | N | (%) | N | (%) | ||
| 3989 | 1954 | 4300 | 1411 | 11654 | |||||||
| Age | |||||||||||
| <40 | 494 | (12.4) | 172 | (8.8) | 426 | (9.9) | 163 | (11.6) | 1255 | (10.8) | p<0.0001 |
| 40–49 | 1256 | (31.5) | 494 | (25.3) | 1208 | (28.1) | 432 | (30.6) | 3390 | (29.1) | |
| 50–59 | 1378 | (34.5) | 626 | (32.0) | 1328 | (30.9) | 486 | (34.4) | 3818 | (32.8) | |
| 60–69 | 707 | (17.7) | 452 | (23.1) | 940 | (21.9) | 247 | (17.5) | 2346 | (20.1) | |
| 70–79 | 154 | (3.9) | 210 | (10.7) | 398 | (9.3) | 83 | (5.9) | 845 | (7.3) | |
| Race | |||||||||||
| White, non- Hispanic | 3092 | (85.0) | 1479 | (80.9) | 3315 | (85.6) | 1087 | (80.6) | 8973 | (83.9) | p<0.0001 |
| Black, non- Hispanic | 53 | (1.5) | 63 | (3.4) | 174 | (4.5) | 35 | (2.6) | 325 | (3.0) | |
| Hispanic | 122 | (3.4) | 49 | (2.7) | 106 | (2.7) | 42 | (3.1) | 319 | (3.0) | |
| Asian or Pacific Islander | 256 | (7.0) | 170 | (9.3) | 185 | (4.8) | 139 | (10.3) | 750 | (7.0) | |
| Other | 115 | (3.2) | 68 | (3.7) | 94 | (2.4) | 45 | (3.3) | 322 | (3.0) | |
| Missing | 351 | 125 | 426 | 63 | 965 | ||||||
| Family history of breast cancer | |||||||||||
| No | 1844 | (48.3) | 1372 | (78.5) | 2992 | (72.0) | 846 | (63.5) | 7054 | (63.8) | p<0.0001 |
| Yes | 1973 | (51.7) | 376 | (21.5) | 1161 | (28.0) | 486 | (36.5) | 3996 | (36.2) | |
| Missing | 172 | 206 | 147 | 79 | 604 | ||||||
| Personal history of breast cancer | |||||||||||
| No | 489 | (21.4) | 26 | (1.4) | 1406 | (44.7) | 131 | (14.7) | 2052 | (25.1) | p<0.0001 |
| Yes | 1794 | (78.6) | 1822 | (98.6) | 1738 | (55.3) | 760 | (85.3) | 6114 | (74.9) | |
| Missing | 1706 | 106 | 1156 | 520 | 3488 | ||||||
| Yes, assuming Missing = No | (45.0) | (93.2) | (40.4) | (53.9) | (52.5) | p<0.0001 | |||||
| BI-RADS breast density | |||||||||||
| Almost entirely fat | 132 | (4.6) | 40 | (3.2) | 108 | (3.4) | 40 | (4.2) | 320 | (3.9) | p<0.0001 |
| Scattered fibroglandular densities | 885 | (30.7) | 402 | (31.8) | 980 | (31.1) | 241 | (25.5) | 2508 | (30.4) | |
| Heterogeneously dense | 1287 | (44.7) | 631 | (49.9) | 1612 | (51.2) | 452 | (47.8) | 3982 | (48.3) | |
| Extremely dense | 576 | (20.0) | 192 | (15.2) | 451 | (14.3) | 213 | (22.5) | 1432 | (17.4) | |
| Missing | 1109 | 689 | 1149 | 465 | 3412 | ||||||
Note: Missing information assumed to be no personal history of breast cancer
MRI BI-RADS Assessments by Clinical Indication
We provide a detailed distribution of MRI BI-RADS assessments by clinical indication in Table 2. The overall proportion of breast MRI exams with positive assessments differed across clinical indications. When categorizing BI-RADS 3 assessments as positive, AIRs were 21.0% (839/3989; 95% CI: 19.8, 22.3) for screening exams, 31.7% (619/1954; 95% CI: 29.6, 33.8) for extent of disease exams, 29.7% (1276/4300; 95% CI: 28.3, 31.1) for diagnostic exams, and 27.4% (386/1411; 95% CI: 25.0, 29.8) for exams done for other indications (p<0.0001). When categorizing BI-RADS 3 assessments as negative, the AIRs declined to 10.5% (417/3989; 95% CI: 9.5, 11.4) for screening exams, 21.8% (425/1954; 95% CI: 19.9, 23.6) for extent of disease exams, 17.7% (759/4300; 95% CI: 16.5, 18.8) for diagnostic exams, and 13.3% (188/1411; 95% CI: 11.6, 15.2) for exams done for other indications (p<0.0001). The significant association between AIR and clinical indication persisted even after adjusting for BCSC registry, patient age, race/ethnicity, family history of breast cancer, personal history of breast cancer, and mammographic breast density in our multiple logistic regression (p<0.0001).
Table 2.
Breast MRI BI-RADS Assessment by Clinical Indication
| Assessment | Clinical Indication | Total | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Screening | Extent of disease | Diagnostic | Other | |||||||
| N | % [95% CI] | N | % [95% CI] | N | % [95% CI] | N | % [95% CI] | N | [95% CI]% | |
| Total | 3989 | 1954 | 4300 | 1411 | 11654 | |||||
| BI-RADS 1, 2, or 6* | 3150 | (79.0) [77.7,80.2] | 1335 | (68.3) [66.2,70.4] | 3024 | (70.3) [68.9,71.7] | 1025 | (72.6) [70.2, 75.0] | 8534 | (73.2) [72.4,74.0] |
| BI-RADS 1 | 990 | (24.8) | 16 | (0.8) | 1249 | (29.0) | 235 | (16.7) | 2490 | (21.4) |
| BI-RADS 2 | 2116 | (53.0) | 235 | (12.0) | 1416 | (32.9) | 647 | (45.9) | 4414 | (37.9) |
| BI-RADS 6 | 44 | (1.1) | 1084 | (55.5) | 359 | (8.3) | 143 | (10.1) | 1630 | (14.0) |
| BI-RADS 3 | 422 | (10.6) [9.6,11.6] | 194 | (9.9) [8.6, 11.3] | 517 | (12.0) [11.1, 13.0] | 198 | (14.0) [ 12.3,16.0] | 1331 | (11.4) [10.8,12.0] |
| BI-RADS 0, 4, or 5* | 417 | (10.5) [9.5,11.4] | 425 | (21.8) [ 19.9,23.6] | 759 | (17.7) [16.5, 18.8] | 188 | (13.3) [11.6, 15.2] | 1789 | (15.4) [14.7,16.0] |
| BI-RADS 0 | 144 | (3.6) | 88 | (4.5) | 147 | (3.4) | 42 | (3.0) | 421 | (3.6) |
| BI-RADS 4 | 259 | (6.5) | 246 | (12.6) | 455 | (10.6) | 126 | (8.9) | 1086 | (9.3) |
| BI-RADS 5 | 14 | (0.4) | 91 | (4.7) | 157 | (3.7) | 20 | (1.4) | 282 | (2.4) |
| BI-RADS 0, 3, 4 or 5* | 839 | (21.0) [19.8,22.3] | 619 | (31.7) [29.6,33.8] | 1276 | (29.7) [28.3,31.1] | 386 | (27.4) [25.0, 29.8] | 3120 | (26.8) [26.0,27.6] |
Note: There was a statistically significant (p<0.0001) difference in positive versus negative assessment by clinical indication, regardless of the classification of BI-RADS 3 assessments.
We provide a trend analysis of breast MRI AIRs by indication over time in Table 3. We did not find any significant changes in AIRs for most indications over time regardless of whether BI-RADS 3 assessments were categorized as positive or negative. The exception was breast MRI AIR for diagnostic indications, which significantly decreased over time when BI-RADS 3 assessments were categorized as negative (p<0.0001). However, this trend bordered on significant when BI-RADS 3 assessments were re-categorized as positive (p=0.06).
Table 3.
Trends in Breast MRI Abnormal Interpretation Rate by Clinical Indication and Year
| Clinical Indication | Total | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Screening | Extent of Disease | Diagnostic | Other | |||||||
| Year(s) | # exams | % AIR | # exams | % AIR | # exams | % AIR | # exams | % AIR | # exams | % AIR |
| Abnormal BI-RADS assessments: 0, 3, 4, or 5 | ||||||||||
| 2005–6 | 655 | 19.1 | 361 | 27.4 | 1019 | 32.6 | 250 | 28.0 | 2285 | 27.4 |
| 2007 | 789 | 22.6 | 343 | 34.7 | 970 | 30.6 | 224 | 25.9 | 2326 | 28.0 |
| 2008 | 669 | 24.4 | 262 | 30.9 | 851 | 25.4 | 277 | 32.5 | 2059 | 26.7 |
| 2009 | 824 | 21.6 | 367 | 31.6 | 838 | 28.9 | 289 | 25.3 | 2318 | 26.3 |
| 2010 | 1052 | 18.5 | 621 | 32.9 | 622 | 30.4 | 371 | 25.6 | 2666 | 25.6 |
| p-value trend | 0.62 | 0.18 | 0.06 | 0.48 | ||||||
| Abnormal BI-RADS assessments: 0, 4, or 5 | ||||||||||
| 2005–6 | 655 | 10.4 | 361 | 19.1 | 1019 | 21.7 | 250 | 16.0 | 2285 | 17.4 |
| 2007 | 789 | 11.0 | 343 | 23.3 | 970 | 19.8 | 224 | 12.1 | 2326 | 16.6 |
| 2008 | 669 | 12.9 | 262 | 21.4 | 851 | 12.8 | 277 | 15.2 | 2059 | 14.2 |
| 2009 | 824 | 8.9 | 367 | 18.3 | 838 | 15.5 | 289 | 9.7 | 2318 | 12.9 |
| 2010 | 1052 | 9.8 | 621 | 24.6 | 622 | 17.2 | 371 | 13.7 | 2666 | 15.5 |
| p-value trend | 0.36 | 0.16 | <0.0001 | 0.27 | ||||||
Note: MRI exams for 2005 and 2006 were combined due to lower number of exams in the earliest years. AIR = abnormal interpretation rate.
DISCUSSION
Our study is the first to report proportions of positive BI-RADS assessments by clinical indication for breast MRI exams performed across a large, geographically diverse set of U.S. radiology practices over multiple years. Since positive BI-RADS assessments suggest that further imaging evaluation or work-up is required, our reported proportions of positive BI-RADS assessments also constitute overall average abnormal interpretation (i.e., recall) rates. We found that the rate was lower for breast MRI exams conducted for screening compared to exams conducted for diagnostic work-up or extent of disease. Given statistical significance in the difference in AIRs across clinical indications even after adjusting for multiple covariates, radiology practices should audit screening breast MRI exams separately from diagnostic and extent of disease breast MRI exams.
Our conclusion, that AIRs for breast MRI exams vary by indication, is consistent with previous studies demonstrating significantly different BI-RADS outcomes for diagnostic mammography compared with screening mammography, with diagnostic exams defined as those performed for indications other than screening asymptomatic women (21–23). Similar to mammography, AIRs for breast MRI are higher for non-screening examinations than screening examinations. However, unlike screening and diagnostic mammography exams, which are already categorized as different types of exams for administrative and billing purposes, there is currently no such delineation by clinical indication for breast MRI exams. Since the same MR sequences are obtained regardless of indication, all breast MRI exams are treated the same for scheduling and billing purposes. Our findings, therefore, suggest that capturing the clinical indication for breast MRI exams at the practice level will be critical to the development of accurate breast MRI audit and quality assurance programs.
Prior data regarding population-based breast MRI AIRs have been limited to those from Europe. For example, a population-based screening study from the UK demonstrated a 10% AIR for screening breast MRI (24). However, this study involved only 649 women age 35–49 years in a clinical trial setting, with inclusion of women with at least a 60% chance of carrying BRCA mutations and exclusion of women with a prior breast cancer history (25). In comparison, our study includes data from 11,654 MRI examinations in the U.S. among women aged 18 –79 years in real-world community practice settings, without rigid study inclusion and exclusion criteria. Moreover, we provide average AIRs among a large, national sample of practices for all other clinical indications for breast MRI exams beyond asymptomatic screening.
Our analysis is also the first to describe trends in breast MRI interpretation by clinical indication over several years. We found that the average annual breast MRI AIRs for most clinical indications did not change significantly over time. The one exception was the trend of AIRs for diagnostic indications, which decreased over time if BI-RADS 3 assessments were categorized as negative. This suggests that community radiologists in the U.S. may be improving in their interpretive performance of breast MRI exams indicated for diagnostic purposes with increasing experience.
While our reported proportion of positive BI-RADS assessments and AIRs represent performance in U.S. community settings, it should not be inferred that these are recommended targets for performance. The issues of defining standards of care or creating performance benchmarks and guidelines are beyond the scope of this work. Rather, our reported measures are a starting point for such discussions, and can be used as one source of information for examining variation in breast MRI BI-RADS assessments in U.S. community practice. Yet, given the sparse available data to which practices and radiologists may compare their breast MRI performance, our data can serve as “ballpark” figures from large aggregate averages that may be used by clinical practices to gauge their AIRs by clinical indication observed in their facility and individual audits, for the purpose of continuous quality improvement.
A major strength of our study is the large sample of breast MRI data from diverse community radiology practices across different geographic regions. Another major strength is that we consider scenarios where BI-RADS 3 assessments are classified as either positive or negative, accounting for their likely variable categorization across practices. Moreover, we adjust for patient demographic and clinical characteristics that are potential confounders for breast MRI AIR, including age, race/ethnicity, family history of breast cancer, personal history of breast cancer, and BI-RADS breast density (26–29). However, there are limitations to our analysis. First, we did not evaluate how physician characteristics such as interpretative volume and experience level or facility type influenced the proportion of positive MRI BI-RADS assessments. Second, we were unable to determine outcomes measures such as cancer detection rate or positive predictive value (PPV) as we do not yet have complete cancer capture for most of these exams due to lags in cancer registry reporting. Acceptability of reported AIRs may depend upon concomitant acceptability of these outcomes. Third, we did not evaluate the effect of technical parameters, such as magnetic field strength, on breast MRI AIR. However, a recent survey demonstrated that 94% of all BCSC facilities met both ACRIN and European Society of Breast Imaging technical standards for breast MRI equipment, including the use of ≥ 1.5 Tesla field strength for all breast MRI exams (30).
Finally, we assume that positive BI-RADS assessments were associated with the proper clinical management recommendations for suspicious imaging findings requiring further diagnostic work-up, and that women with negative assessments were not recalled for further workup. Yet, prior studies with regards to application of mammography BI-RADS shortly after its adoption into clinical practice demonstrated inconsistencies between the assessments and expected clinical management recommendations (31–33). Given the relatively recent adoption of breast MRI and the ACR breast MRI BI-RADS lexicon into clinical practice, there may be inconsistencies between MRI BI-RADS and clinical management recommendations in community practice. Therefore, as a next step, we are planning an analysis of the agreement between recorded MRI BI-RADS assessments and recorded clinical management recommendations.
In conclusion, we present data on the proportion of positive BI-RADS assessments by clinical indication for breast MRI examinations among a geographically diverse set of radiology practices across the U.S. These measures can be used as first available figures by individual breast imaging practices and radiologists to compare with their own performance as part of periodic mandatory audits and quality improvement initiatives. Our report, based on a large national sample of imaging practices, can also be used as a starting point by governing bodies and breast imaging experts who aim to determine performance benchmarks for breast MRI exams. Based on our statistically significant findings, radiology practices should determine and report their breast MRI AIRs stratified by clinical indications.
Acknowledgments
This work was supported by National Cancer Institute-funded grants (RC2CA148577, P01CA154292) and the Breast Cancer Surveillance Consortium (HHSN261201100031C). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Cancer Institute or the National Institutes of Health. We thank the participating women, mammography facilities, and radiologists for the data they have provided for this study. A list of the BCSC investigators and procedures for requesting BCSC data for research purposes are provided at: http://breastscreening.cancer.gov/.
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Contributor Information
Christoph I. Lee, Email: stophlee@uw.edu.
Laura Ichikawa, Email: ichikawa.l@ghc.org.
Michele C. Rochelle, Email: Michele.christina.rochelle@gmail.com.
Karla Kerlikowske, Email: Karla.Kerlikowske@ucsf.edu.
Diana L. Miglioretti, Email: dmiglioretti@phs.ucdavis.edu.
Brian L. Sprague, Email: brian.sprague@med.uvm.edu.
Wendy B. DeMartini, Email: WDeMartini@uwhealth.org.
Karen J. Wernli, Email: wernli.k@ghc.org.
Bonnie N. Joe, Email: Bonnie.Joe@ucsf.edu.
Bonnie C. Yankaskas, Email: bcy@med.unc.edu.
Constance D. Lehman, Email: clehman@seattlecca.org.
References
- 1.Liberman L. Breast cancer screening with MRI--what are the data for patients at high risk? N Engl J Med. 2004;351:497–500. doi: 10.1056/NEJMe048117. [DOI] [PubMed] [Google Scholar]
- 2.Elmore L, Margenthaler JA. Breast MRI surveillance in women with prior curative-intent therapy for breast cancer. J Surg Res. 2010;163:58–62. doi: 10.1016/j.jss.2010.03.016. [DOI] [PubMed] [Google Scholar]
- 3.Lehman CD, DeMartini W, Anderson BO, Edge SB. Indications for breast MRI in the patient with newly diagnosed breast cancer. J Natl Compr Canc Netw. 2009;7:193–201. doi: 10.6004/jnccn.2009.0013. [DOI] [PubMed] [Google Scholar]
- 4.DeMartini W, Lehman C. A review of current evidence-based clinical applications for breast magnetic resonance imaging. Top Magn Reson Imaging. 2008;19:143–150. doi: 10.1097/RMR.0b013e31818a40a5. [DOI] [PubMed] [Google Scholar]
- 5.Stout NK, Nekhlyudov L. Early uptake of breast magnetic resonance imaging in a community-based medical practice, 2000–2004. J Womens Health (Larchmt) 2011;20:631–634. doi: 10.1089/jwh.2010.2159. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Sommer CA, Stitzenberg KB, Tolleson-Rinehart S, Carpenter WR, Carey TS. Breast MRI utilization in older patients with newly diagnosed breast cancer. J Surg Res. 2011;170:77–83. doi: 10.1016/j.jss.2011.04.038. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Saslow D, Boetes C, Burke W, Harms S, Leach MO, Lehman CD, et al. American Cancer Society guidelines for breast screening with MRI as an adjunct to mammography. CA Cancer J Clin. 2007;57:75–89. doi: 10.3322/canjclin.57.2.75. [DOI] [PubMed] [Google Scholar]
- 8.Lehman CD. Role of MRI in screening women at high risk for breast cancer. J Magn Reson Imaging. 2006;24:964–970. doi: 10.1002/jmri.20752. [DOI] [PubMed] [Google Scholar]
- 9.Kuhl CK. Current status of breast MR imaging. Part 2. Clinical applications Radiology. 2007;244:672–691. doi: 10.1148/radiol.2443051661. [DOI] [PubMed] [Google Scholar]
- 10.Kuhl C. The current status of breast MR imaging. Part I. Choice of technique, image interpretation, diagnostic accuracy, and transfer to clinical practice. Radiology. 2007;244:356–378. doi: 10.1148/radiol.2442051620. [DOI] [PubMed] [Google Scholar]
- 11.Molleran V, Mahoney MC. The BI-RADS breast magnetic resonance imaging lexicon. Magn Reson Imaging Clin N Am. 2010;18:171–185. doi: 10.1016/j.mric.2010.02.001. [DOI] [PubMed] [Google Scholar]
- 12.American College of Radiology. ACR Breast Imaging Reporting and Data System (BI-RADS) breast imaging atlas. 5. Reston, VA: American College of Radiology; 2013. [Google Scholar]
- 13.Gutierrez RL, DeMartini WB, Eby PR, Kurland BF, Peacock S, Lehman CD. BI-RADS lesion characteristics predict likelihood of malignancy in breast MRI for masses but not for nonmasslike enhancement. AJR Am J Roentgenol. 2009;193:994–1000. doi: 10.2214/AJR.08.1983. [DOI] [PubMed] [Google Scholar]
- 14.Mahoney MC, Gatsonis C, Hanna L, DeMartini WB, Lehman C. Positive predictive value of BI-RADS MR imaging. Radiology. 2012;264:51–58. doi: 10.1148/radiol.12110619. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Congress US. [Accessed May 29, 2013];H.R. 6331: Medicare Improvements for Patients and Providers Act of 2008. Available at: http://www.govtrack.us/congress/bills/110/hr6331.
- 16.American College of Radiology. [Accessed May 29, 2013];Breast MRI Accreditation. Available at: http://www.acr.org/Quality-Safety/Accreditation/BreastMRI.
- 17.Spring DB, Kimbrell-Wilmot K. Evaluating the success of mammography at the local level: how to conduct an audit of your practice. Radiol Clin North Am. 1987;25:983–992. [PubMed] [Google Scholar]
- 18.Murphy WA, Jr, Destouet JM, Monsees BS. Professional quality assurance for mammography screening programs. Radiology. 1990;175:319–320. doi: 10.1148/radiology.175.2.2326453. [DOI] [PubMed] [Google Scholar]
- 19.Ballard-Barbash R, Taplin SH, Yankaskas BC, Ernster VL, Rosenberg RD, Carney PA, et al. Breast Cancer Surveillance Consortium: a national mammography screening and outcomes database. AJR Am J Roentgenol. 1997;169:1001–1008. doi: 10.2214/ajr.169.4.9308451. [DOI] [PubMed] [Google Scholar]
- 20.Carney PA, Geller BM, Moffett H, Ganger M, Sewell M, Barlow WE, et al. Current medicolegal and confidentiality issues in large, multicenter research programs. Am J Epidemiol. 2000;152:371–378. doi: 10.1093/aje/152.4.371. [DOI] [PubMed] [Google Scholar]
- 21.Dee KE, Sickles EA. Medical audit of diagnostic mammography examinations: comparison with screening outcomes obtained concurrently. AJR Am J Roentgenol. 2001;176:729–733. doi: 10.2214/ajr.176.3.1760729. [DOI] [PubMed] [Google Scholar]
- 22.Sohlich RE, Sickles EA, Burnside ES, Dee KE. Interpreting data from audits when screening and diagnostic mammography outcomes are combined. AJR Am J Roentgenol. 2002;178:681–686. doi: 10.2214/ajr.178.3.1780681. [DOI] [PubMed] [Google Scholar]
- 23.Sickles EA, Miglioretti DL, Ballard-Barbash R, Geller BM, Leung JW, Rosenberg RD, et al. Performance benchmarks for diagnostic mammography. Radiology. 2005;235:775–790. doi: 10.1148/radiol.2353040738. [DOI] [PubMed] [Google Scholar]
- 24.Warren RM, Pointon L, Caines R, Hayes C, Thompson D, Leach MO. What is the recall rate of breast MRI when used for screening asymptomatic women at high risk? Magn Reson Imaging. 2002;20:557–565. doi: 10.1016/s0730-725x(02)00535-0. [DOI] [PubMed] [Google Scholar]
- 25.Leach MO, Boggis CR, Dixon AK, Easton DF, Eeles RA, Evans DG, et al. Screening with magnetic resonance imaging and mammography of a UK population at high familial risk of breast cancer: a prospective multicentre cohort study (MARIBS) Lancet. 2005;365:1769–1778. doi: 10.1016/S0140-6736(05)66481-1. [DOI] [PubMed] [Google Scholar]
- 26.Kerlikowske K, Grady D, Barclay J, Sickles EA, Eaton A, Ernster V. Positive predictive value of screening mammography by age and family history of breast cancer. JAMA. 1993;270:2444–2450. [PubMed] [Google Scholar]
- 27.Frankel SD, Sickles EA, Curpen BN, Sollitto RA, Ominsky SH, Galvin HB. Initial versus subsequent screening mammography: comparison of findings and their prognostic significance. AJR Am J Roentgenol. 1995;164:1107–1109. doi: 10.2214/ajr.164.5.7717214. [DOI] [PubMed] [Google Scholar]
- 28.Kerlikowske K, Grady D, Barclay J, Sickles EA, Ernster V. Effect of age, breast density, and family history on the sensitivity of first screening mammography. JAMA. 1996;276:33–38. [PubMed] [Google Scholar]
- 29.Kerlikowske K, Grady D, Barclay J, Frankel SD, Ominsky SH, Sickles EA, et al. Variability and accuracy in mammographic interpretation using the American College of Radiology Breast Imaging Reporting and Data System. J Natl Cancer Inst. 1998;90:1801–1809. doi: 10.1093/jnci/90.23.1801. [DOI] [PubMed] [Google Scholar]
- 30.DeMartini WB, Ichikawa L, Yankaskas BC, Buist D, Kerlikowske K, Geller B, et al. Breast MRI in community practice: equipment and imaging techniques at facilities in the Breast Cancer Surveillance Consortium. J Am Coll Radiol. 2010;7:878–884. doi: 10.1016/j.jacr.2010.06.019. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Taplin SH, Ichikawa LE, Kerlikowske K, Ernster VL, Rosenberg RD, Yankaskas BC, et al. Concordance of breast imaging reporting and data system assessments and management recommendations in screening mammography. Radiology. 2002;222:529–535. doi: 10.1148/radiol.2222010647. [DOI] [PubMed] [Google Scholar]
- 32.Lehman C, Holt S, Peacock S, White E, Urban N. Use of the American College of Radiology BI-RADS guidelines by community radiologists: concordance of assessments and recommendations assigned to screening mammograms. AJR Am J Roentgenol. 2002;179:15–20. doi: 10.2214/ajr.179.1.1790015. [DOI] [PubMed] [Google Scholar]
- 33.Geller BM, Barlow WE, Ballard-Barbash R, Ernster VL, Yankaskas BC, Sickles EA, et al. Use of the American College of Radiology BI-RADS to report on the mammographic evaluation of women with signs and symptoms of breast disease. Radiology. 2002;222:536–542. doi: 10.1148/radiol.2222010620. [DOI] [PubMed] [Google Scholar]