Using a Tailored Web-based Intervention to Set Goals to Reduce Unnecessary Recall

Abstract

Purpose

To examine whether an intervention strategy consisting of a tailored web-based intervention, which provides individualized audit data with peer comparisons and other data that can affect recall can assist radiologists in setting goals for reducing unnecessary recall.

Methods

In a multi-site randomized controlled study, we used a tailored web-based intervention to assess radiologists' ability to set goals to improve interpretive performance. The intervention provided peer-comparison audit data, profiled breast cancer risk in each radiologist's respective patient populations, and evaluated the possible impact of medical malpractice concerns. We calculated the percentage of radiologists who would consider changing their recall rates, and examined the specific goals they set to reduce recall rates. We describe characteristics of radiologists who developed realistic goals to reduce their recall rates, and their reactions to the importance of patient risk factors and medical malpractice concerns.

Results

Forty-one of 46 radiologists (89.1%) who started the intervention completed it. Thirty-one (72.1%) indicated they would like to change their recall rates and 30 (69.8%) entered a text response about changing their rates. Sixteen of the 30 (53.3%) radiologists who included a text response set realistic goals that would likely result in reducing unnecessary recall. The actual recall rates of those who set realistic goals were not statistically different from those who did not (13.8% vs. 15.1% respectively). The majority of selected goals involved re-reviewing cases initially interpreted as BI-RADS category 0. Over half of radiologists who commented on the influence of patient risk (56.3%) indicated that radiologists planned to pay more attention to risk factors, and 100% of participants commented on concerns radiologists have about malpractice with the primary concern (37.5%) being fear of lawsuits.

Conclusions

Interventions designed to reduce unnecessary recall can succeed in assisting radiologists to develop goals that may ultimately reduce unnecessary recall.

INTRODUCTION

The full potential of mortality reduction by mammography may be limited by the current variability in radiologists’ interpretive performance (1–15). Sources of variability affecting performance have included fellowship training (15), which is associated with improved performance. Other studies have shown that, for some radiologists, inflated perceptions of breast cancer risk in their patient populations (7) and inflated perceptions of their own malpractice risk (6) are associated with higher than recommended recall rates, which cause harms such as patient anxiety, and excessive false positive biopsies. The potential harms of mammography were a consideration when the U.S. Preventive Services Task Force revised its recommendation for mammography frequency among average risk women aged 50 and older (16).

Prior research suggests that educational interventions may have value, but intervention research that includes rigorous study designs have limitations. The UK National Health Program evaluated a two-week multidisciplinary course with specialist training at high-volume screening sites, which included three sessions per week of interpreting screening mammograms. Radiologists additionally attended routine breast disease-related meetings and received personal and group audit reports that included data on cancer detection rate, recall rate and positive predictive value of biopsy recommendation (PPV₂) (17). Performance indices showed a reduction in the recall rate from 7 to 4%, and an increase in the small invasive cancer detection rate from 1.6/1,000 to 2.5/1,000. Unfortunately, mechanisms to track such indices in the U.S. do not typically exist. In another study, performed within a major US health maintenance organization (HMO) (18), 21 radiologists were provided with personal and group audit reports, participated in a self-assessment program, attended case review sessions, and were required to interpret 8,000 mammograms per year per radiologist. Sensitivity improved from 70% to 80%, with a mean cancer detection rate of 7.5/1,000 and a mean recall rate of 7%.

A limitation in the above research is that multiple intervention strategies, such as volume requirements, audit feedback, and case review sessions were tested, which does not allow for assessment of the impact of a single strategy. Other limitations in the HMO study include the small sample size, the lack of a randomized design, and restricted to a single healthcare setting, which reduces its generalizability.

We developed a single intervention strategy that included individualized web-based educational intervention designed to identify areas of interpretive weakness and to assist radiologists with setting goals to reduce unnecessary recall. The intervention has been tested in a randomized controlled design where follow-up data are now being collected. Here we describe characteristics of radiologists who developed realistic goals to reduce their recall rates, the goals they developed, and their reactions to the importance of patient risk factors and medical malpractice concerns when interpreting screening mammography.

METHODS

Four of seven mammography registries that are currently part of the Breast Cancer Surveillance Consortium (BCSC; http://breastscreening.cancer.gov) were included in this study: Group Health Breast Cancer Surveillance Project in Seattle, WA; the New Hampshire Mammography Network; the Vermont Breast Cancer Surveillance System; and the Carolina Mammography Registry. Patient demographic and clinical information is collected at the time mammography is performed at a participating facility. Radiologists' interpretation and follow-up recommendations according to the American College of Radiology's Breast Imaging Reporting and Data System (BI-RADS) (19) are also collected and linked to regional cancer registries and pathology databases to determine cancer outcomes. Data from the registries are annually pooled at the BCSC Statistical Coordinating Center (SCC) located in Seattle, WA for cleaning and analysis.

Each registry and the SCC received IRB approval for either active or passive consenting processes or a waiver of consent to enroll participants, link data, perform analytic studies, and for all activities related to the study described here. All procedures are Health Insurance Portability and Accountability Act (HIPAA) compliant and all registries and the SCC have received a Federal Certificate of Confidentiality and other protection for the identities of women, physicians, and facilities that are subjects of this research (20). In addition, all study-specific activities were reviewed and approved by IRBs associated with each participating site.

Performance data that automatically populated the web-based intervention were derived from the respective mammography registries. The study design included randomizing radiologists to either an early or late intervention group, with the late intervention group serving as controls for assessment of performance measures. Because the study is still in the follow-up data collection phase, data reported here do not include assessment of interpretive performance after the intervention. Thus, the intervention groups are combined in this report.

Radiologist Survey

We invited radiologists from four BCSC registries to complete a previously described mailed survey (15). The survey included questions about radiologist age, years of practice, affiliation with academic medical centers, completion of a breast imaging fellowship, estimates of annual volume of screening and diagnostic mammograms, and percentage of practice time spent in breast imaging. These data were used to characterize the radiologists, but completion of the survey was not required to participate in the web-based intervention.

Web-based Tailored Educational Intervention Data System

Radiologists eligible for the intervention were actively interpreting mammograms at a facility at one of the four participating BCSC registries between January 2006 and September 2007. Because only four of the BCSC registries were involved in the intervention portion of the study, the number of radiologists eligible to take part in the intervention was 196. Sixty-seven eligible radiologists did not complete the radiologist survey and did not consent to the intervention, leaving 129 radiologists who provided data for inclusion in this report. Of these, 55 completed the survey but did not consent to the intervention.

Each consenting radiologist (n=74) was mailed a unique log-in and password and participants were awarded two hours of Category I CME credit. Of the 74 who consented to the intervention, 64 (86.4%) completed the survey, 46 (62.2%) actually logged on to start the intervention and 41 (89.1%) of these completed it. The entire program took an average of 1 hour to complete. Among those radiologists who completed the intervention, 27 (65.9%) were randomized to the early group and 14 (34.2%) to the late group.

The intervention is described in detail elsewhere (21). Briefly, it was designed to encourage radiologists to change performance by illustrating the metrics in clinical performance that could be improved, to enable improvements by showing factors that may influence performance and then reinforce change by assisting radiologists to develop goals that would improve their performance. When a participating radiologist logged on, the system transparently combined mammography registry data provided by the BCSC SCC with content and questions from a machine-readable template prepared by our research team, to produce the final individually tailored intervention. The learning module had three components: 1) Understanding audit statistics for sensitivity, specificity, recall rates, positive predictive value, and cancer detection; 2) Misperceptions radiologists have about women’s risk of breast cancer; 3) Misperceptions about risk of medical malpractice lawsuits related to breast imaging.

Knowledge questions were imbedded into the system, which generated a data file we could use to award CME credits. Radiologists were able to insert their goals for changes they would like to make in their clinical practice, especially regarding recall rates, into a text field at the end of each module. We defined realistic goals as planned actions that, if implemented, would likely result in a change in recall rate toward alignment with national targets. Using this definition, two of the authors (PC and ES) classified each radiologist’s goals as realistic toward improving performance or not. Questions about radiologists' attitudes and perceptions of our web-based intervention program were also embedded into the system.

Data Analyses

Content analysis using the classical methods (22) was done to analyze the text responses of participating radiologists for goals they set to reduce unnecessary recall, and their reactions to the module on patient risk factors and their own risk of medical malpractice. Using a consensus approach, two observes (PAC and EAS) reviewed radiologists responses to goals participants identified as reducing unnecessary recall and classified these as being realistic or not using our agreed upon definition. Realistic goals were further classified using our study methods into four categories: 1) Change interpretive thresholds; 2) Re-review of certain images; 3) Change interpretive practices, and 4) Seek independent second review. We present the actual text for goals that were considered realistic along with radiologists' recall rates for a one-year period (2003–2004). Recall rates were determined by calculating the proportion of exams coded as BI-RADS 0, 4 or 5 or a 3 with a recommendation for immediate work-up among all exams that the radiologist indicated as screening.

We evaluated the distribution of the 41 radiologists’ demographic characteristics, practice types, breast imaging experience, and preferences/attitudes toward CME. We also assessed responses to questions about considering risk factors during mammography interpretation (based on the second module described above) pre- and post-intervention. Finally, we evaluated radiologists’ perceptions of medical malpractice on recall and breast biopsy rates pre- and post-intervention. All results were stratified by whether or not the goals developed by radiologists could realistically improve performance. We tested for statistically significant differences between the two groups using a chi-squared test and Fisher’s exact test when cell sizes were <5. All analyses were conducted using Stata, and p-values <0.05 were considered statistically significant.

RESULTS

Forty-one of 46 radiologists (89.1%) who started the intervention completed it. There were no differences between the characteristics of radiologists who completed the program compared to those who did not (data not shown). A complete assessment of characteristics of radiologists who did and did not participate in the survey is reported elsewhere (21). The demographic characteristics did not differ between radiologists who set goals to reduce their recall rates and those who did not (Table 1). The majority of participants were not affiliated with an academic medical center; only one had fellowship training in breast imaging; most had 10 or more years of experience interpreting breast imaging; and most spent at least 20% of their time in breast imaging. The actual recall rates of those who developed goals were not statistically different from those who did.

Table 1.

Radiologist Characteristics^* According to Whether or Not Realistic Goals Were Set to Reduce Unnecessary Recall

Characteristics	All Radiologists Combined (n=41)	Radiologists Who Set Realistic Goals (n=16)	Radiologists Who Did Not Set Realistic Goals (n=25)	p value
Demographics
Sex
Male	21 (52.5)	8 (50.0)	13 (54.2)	0.796
Female	19 (47.5)	8 (50.0)	11 (45.8)
Practice Type
Primary affiliation with academic medical center
No	34 (85.0)	13 (81.3)	21 (87.5)	1.00
Adjunct	2 (5.0)	1 (6.3)	1 (4.2)
Primary	4 (10.0)	2 (12.5)	2 (8.3)
Breast Imaging Experience
Fellowship training
No	39 (97.5)	15 (93.8)	24 (100.0)	0.400
Yes	1 (2.5)	1 (6.3)	0 (0.0)
Years of mammography interpretation
<10	8 (20.0)	4 (25.0)	4 (16.7)	0.837
10–19	16 (40.0)	6 (37.5)	10 (41.7)
≥20	16 (40.0)	6 (37.5)	10 (41.7)
Percent of time spent in breast imaging
<20%	10 (25.0)	2 (12.5)	8 (33.3)	0.214
20–39%	15 (37.5)	5 (31.3)	10 (41.7)
40–79%	9 (22.5)	6 (37.5)	3 (12.5)
80–100%	6 (15.0)	3 (18.8)	3 (12.5)
Actual Recall Rate
Mean	14.6	13.8%	15.1%	0.627
95% Confidence Intervals	12.0–17.2	10.8–16.9	11.2–19.1
Range	4.0–50.0	7.3–27.4	4.0–50.0

^*Those who completed the Radiologist survey

We also assessed participants' preferences and attitudes toward CME, and determined that the majority (87.2%) preferred instructor-led educational activities, 80% felt CME activities improve interpretive performance, and 88% indicated they would take a free CME course using the Internet (data not shown). No differences were noted for these variables according to whether or not respondents set realistic goals to improve performance.

After viewing their individual indices and the segment of the program illustrating that recall rates above 5–7% do not typically result in more cancer detection, participants were asked if they would consider changing their recall rates and 30 of 41 (73.2%) indicated they would. When asked what goals they would set to reduce their recall, 30 (73.2%) entered a text response in the space provided (Table 2). Fifteen of the 30 (50%) radiologists set realistic goals to reduce recall and one (3.3%) set a goal to increase recall. Thirty-one percent (n=5) indicated they planned to change interpretive thresholds, and 25% (n=4) indicated they would re-review of certain images. Nearly 19% (n=3) planned to change interpretive practices, and 12.5% (n=2) planned to seek an independent second review. Unrealistic goals (n=8, 57.1%) typically included a desire to reduce recall without a mechanism to do so.

Table 2.

Radiologists’ Recall Rates and Realistic Goals Set as Part of the Auditing Component of the Tailored Web-based Intervention

Radiologists Recall Rate (%)	Goals Set to Improve Recall Rates	Interpretive Themes
	Set Goal to Increase Recall
7.3	Recall more of the probably benign rather than following at six months.	Change Interpretive Threshold**
	Set Goal to Decrease Unnecessary Recall
9.2	I'd like it to be closer to 6–7% and could re-review some cases.	Re-review
9.3	Raise threshold of concern for asymmetry.	Change Interpretive Threshold
10.2	Double-check each possible recall before calling patient back.	Re-review
10.7	By reviewing the results/images of most/all of the cases that I have placed into a category 0.	Re-review
11.2	Less call back for lesions that are highly suggestive of a benign process cyst or LN [lymph node] that I would call back only because of double read and another radiologist would insist on calling back.	Change Interpretive Threshold
12.1	Decrease call-backs for densities that I am reasonably certain are summation densities.	Change Interpretive Threshold
12.3	Try to determine benign findings at the time of screening exam.	Re-review
13.3	Be sure to review more prior mammograms than are currently on the alternator.	Change Interpretive Practices
14.1	Change more zero codes to early follow up at one year.	Change Interpretive Practices
15.2	Trust my judgment, initially. Look at the mammogram less, so as not to 2nd guess. Ask for 2nd opinions from colleagues less often, as they often push my rate higher.	Change Interpretive Practices
17.2	Giving extra thought before call-backs, although my recall rate has been higher the last two years due to adjustment to a digital system.	Change Interpretive Threshold
19.8	I need to increase my sensitivity for small cancers and ask for 2nd read more often	Seek 2nd Independent Review
22.2	On uncertain cases, get 2nd opinion so decision-making is optimized.	Seek 2nd Independent Review
24.6	Try to limit unnecessary recalls with re-review	Re-review
27.4	High recall rate is related to our previous methodology of final reads. We didn't read out BI-RADS 0. I was reading many other radiologists callbacks. We have changed.	Change Interpretive Practices

^**for less concerning findings

We also assessed responses to the breast cancer risk component of the tailored intervention, according to whether or not realistic goals were set to reduce unnecessary recall. No statistically significant differences existed in how radiologists in these two groups consider risk factors when interpreting mammograms at the start of the intervention. This was true for considering age (77% consider among goal setters vs. 78% consider among non-goal setters), breast density (75% among goal setters vs. 44% among non-goal setters), family history (100% among goal setters vs. 94% among non-goal setters) and past breast procedures (58% among goal setters vs. 67% among non-goal setters) as risk factors pre-intervention (data not shown).

Post intervention, when pre-menopausal risk factors were considered, findings were similar in that we found no statistically significant differences between the two groups for the importance of a prior positive mammogram (87% among goal setters vs. 72% among non-goal setters), importance of younger age at menarche (56% among goal setters vs. 32% among non-goal setters), importance of race/ethnicity (80% among goal setters vs. 64% among non-goal setters), importance of higher breast density (100% among goal setters vs. 88% among non-goal setters), importance of family history (100% among goal setters vs. 88% among non-goal setters) and importance of prior benign breast biopsy (94% among goal setters vs. 80% among non-goal setters) as risk factors for premenopausal women. Post-intervention, when radiologists considered risk factors among postmenopausal women, they generally rated them as more important than the same risk factors for premenopausal women, though no significant differences were noted between the two groups (data not shown).

Fifteen radiologists commented on how the use of risk factors should be considered after this module was completed in response to the question: "Is there anything you would do differently as a result of what you have learned?" (Table 3). Six (19.4%) of these were radiologists who set reasonable goals for performance as part of the intervention. The majority of the comments indicated radiologists planned to pay more attention to risk factors (9/16, 56.3%). Another two participants (12.5%) planned to pay less attention to risk, three (18.8%) planned no practice change and two more (12.5%) commented to addressing risk in specific situations. Commenting on risk factors does not appear to be associated with setting specific goals for reducing unnecessary recall.

Table 3.

Responses for How Use of Risk Factors May Change Recall Rates As a Result of the Intervention

Radiologist’s Recall Rate (%)	Physicians Responses Regarding the Use of Risk Factors - Is there anything you would do differently as a result of what you have learned?	Interpretive Themes	Radiologist Set Realistic Goals
4.0	Pay more attention to age and race/ethnicity	Pay more attention to risk	No
5.0	Look at BMI.	Pay more attention to risk	No
5.7	Try not to be swayed by risk factors when dealing with equivocal findings	Pay less attention to risk	Yes
8.6	I currently review risk factors on all patients.	No Change in Practice	No
9.2	I generally review risk factors and would continue	No Change in Practice	No
9.4	Review risk factors more often	Pay more attention to risk	No
10.2	Yes, consider more select factors depending upon pre vs. postmenopausal status.	Pay more attention to risk	Yes
13.0	Not consider menarche and consider race	Pay less attention to risk	No
13.2	Awareness only.	No Change in Practice	No
13.3	Closer look at obese dense breasts	Pay more attention to risk	Yes
18.7	Worry a bit more about questionable findings in dense breasts; (be bit more aggressive) and same with older women	Consider risk in certain situations	No
18.8	Try not to overestimate a woman's risk for Breast Cancer, especially for premenopausal. Would also remember that breast density is a risk factor which is predictive of cancer in 1 yr.	Pay more attention to some risk factors and less to others	No
19.8	Put more weight on past negative benign biopsies and breast density as risk factors.	Pay more attention to risk	Yes
22.2	Only marginally. If I was unsure of the need for a possible callback, risk factors could possibly influence my decision.	Pay more attention to risk	Yes
24.6	Be more certain of checking for past procedures.	Pay more attention to risk	Yes

Prior to the intervention and using a scale where 0=No Change, 5=Work-up Moderately Increased and 10=Work-up Greatly Increased, radiologists who set realistic goals for reducing recall indicated that medical malpractice influences both their recall rates and biopsy recommendations to a greater extent than those who did not set goals to reduce unnecessary recall (Table 4). These scores dropped by half after the intervention, and all radiologists correctly indicated that the risk of a mammography related malpractice suit occurring in the next five years was less than 10% among radiologists working full time in breast imaging. Greater than 85% of radiologists commented that at least half of radiologists are concerned that fear about medical malpractice affects how they interpret mammograms (Table 4).

Table 4.

Responses to the Medical Malpractice Component of the Tailored Web-based Intervention According to Whether or Not Realistic Goals Were Set to Reduce Unnecessary Recall

Physicians Responses Regarding Medical Malpractice	Radiologists Who Set Realistic Goals (n=16)	Radiologists Who Did Not Set Realistic Goals (n=25)	p value
Pre-Intervention	Mean (SD)† Scale 0–10*	Mean (SD)† Scale 0–10*
To what extent do you think your medical malpractice concerns have influenced your recall rate	3.8 (2.2)	3.5 (2.4)	0.70
To what extent do you think your medical malpractice concerns have influenced your recommendation for breast biopsies?	3.9 (2.6)	3.4 (2.2)	0.62
Post-Intervention
To what extent do you think your medical malpractice concerns will influence your recall rate	2.0 (1.4)	1.6 (1.5)	0.45
To what extent do you think your medical malpractice concerns will influence your recommendation for breast biopsies?	1.9 (1.8)	1.6 (1.4)	0.52
What is the probable risk of a mammography related malpractice suit occurring in the next five years among radiologists working fulltime in breast imaging?  Less Than 10%	16 (100.0)	25 (100.0)	n/a
How many radiologists are concerned that fear about medical malpractice affects how they interpret mammograms?  25%  50%  75%	1 (6.3) 4 (25.0) 11 (68.8)	3 (12.0) 9 (36.0) 13 (52.0)	0.56

^†=Standard Deviation

^*0=No Change; 5=Work-up Moderately Increased; 10=Work-up Greatly Increased

All 41 radiologists who completed the intervention commented on concerns radiologists have about malpractice (Table 5). Of these, 16 (39.0%) set realistic goals, and of these 15 set goals for reducing unnecessary recall and one set a goal to increase recall. Forty of these comments fell into eight relevant categories including (one was considered not applicable): 1) fear of lawsuits 15/40 (37.5%), 2) media sensation 7/40 (17.5%), 3) media exposure 2/40 (5%), 4) media unspecified 3/40 (8%), 5) impact of hearsay 4/40 (10%), 6) fear unspecified 5/40 (12.5%), 7) fear for patients 1/40 (3%), and 8) fear of professional loss 1/40 (3%).

Table 5.

Responses for How Risk of Medical Malpractice Change Recall Rates As a Result of the Intervention

Radiologist’s Recall Rate	Reactions to Malpractice Content: Why do you think so many radiologists over estimate their own malpractice risk?	Interpretive Themes	Radiologist Set Realistic Goals
4.0	This seems to be much more prevalent in young, recently trained docs.	Not Applicable	No
5.0	They hear about the sentinel cases of large awards and they extrapolate.	Impact of Hearsay	No
5.7	Because it is in the news and the dollar amount is usually very high. We interpret a lot of mammograms and it is so subjective compared to other exams.	Media Sensation	No
6.7	Anecdotal cases leave deep impression on radiologists.	Impact of Hearsay	No
7.2	Fear. And the lack of accurate data to counteract lawsuits.	Fear of Lawsuit	No
7.3	Publicity regarding large settlements.	Media Sensation	Yes
8.6	Personal fear of being sued, high publicity in media of suits, work environment (e.g., being tired, being distracted while interpreting mammograms).	Fear of Lawsuit & Media Exposure	No
8.7	Breast cancer is so frequently in the media. Patients all have "a friend in whom the breast cancer was missed…"	Media Sensation	No
9.2	Because you never know when it will be your turn, and you assume all misses will result in a lawsuit.	Fear of Lawsuit	No
9.3	The press.	Media Unspecified	Yes
9.3	Fear of being sued, hearsay and gossip about suits.	Fear of Lawsuit	No
9.4	Reports in the literature of lawsuits.	Fear of Lawsuit	No
9.9	We are told that the most common reason for malpractice suit against a radiologist is failure to detect cancer on a mammogram.	Fear of Lawsuit	No
10.2	Because of the monetary, emotional and time cost of a malpractice suit.	Fear of Lawsuit	Yes
10.7	Do not know, perhaps because of hearsay information which is not based on fact.	Impact of Hearsay	Yes
11.2	Because of newspaper articles and non-verified data.	Media Sensation	Yes
11.9	It is everywhere in the media…also a recent study does show increasing malpractice suits in radiology.	Media Sensation	No
12.0	Probably cause media reports big cases and settlements, and malpractice premiums are higher if we read mammography.	Media Sensation	No
12.1	Fear - know a colleague who has been involved in a suit, do not wish to risk negative media exposure.	Media Exposure	Yes
13.0	The suits we do hear about are frightening and the awards are growing ever higher.	Impact of Hearsay	No
13.2	Fear, and horror stories.	Fear Unspecified	No
13.3	First because of the difficulty in mammography interpretation. Second, because of the very litigious society.	Fear of Lawsuit	Yes
13.3	Fear of litigious patients/lawyers.	Fear of Lawsuit	Yes
13.4	We don't understand how the world works. We are surprised to find out that the legal system protects physicians, attorneys, and insurers rather than patients.	Fear of Lawsuit	No
14.1	Fear	Fear Unspecified	No
14.1	Because risk is for general numbers…a lawsuit for them is 100%.	Fear of Lawsuit	Yes
15.2	Fear of being responsible for a woman's untimely death.	Fear for Patient	Yes
17.2	Fear of a trial is overwhelming and potentially devastating and therefore the risk is overestimated.	Fear of Lawsuit	Yes
18.7	Media attention. Traumatic effect of even one such case per career.	Media Exposure	No
18.8	Newspapers/TV articles.	Media Unspecified	No
19.4	Fear of the consequences	Fear Unspecified	No
19.4	Fear	Fear Unspecified	No
19.5	News media	Media Unspecified	No
21.5	Pervasive fear, attitudes of patients, lawsuits we have witnessed.	Fear of Lawsuit	No
22.2	You state that less than 10% of radiologists are sued… statistics are only statistics.	Fear of Lawsuit
24.4	Fear	Fear Unspecified	Yes
24.6	General publicity of malpractice cases. Perception of potential large monetary damages.	Media Sensation	Yes
26.3	Fear of lawsuits. Mammography is known as one of the highest classes of risk to radiologists in our country. Misdiagnosis of cancer, and misdiagnosis of fractures have been the highest.	Fear of Lawsuit	No
27.4	Every false negative case is surrounded by much concern and handwringing on part of referring physicians. We wonder what is said to patients behind closed doors.	Fear of Professional	Yes
50.0	Lawsuits are a nuisance.	Fear of Lawsuit	No

DISCUSSION

This study successfully recruited radiologists with recall rates that were, on average, between two and almost three times that of national recommended targets (5 –7%) and above the upper bound of 12% for recently set criteria for recall (23). Radiologists who consented to this study had recall rates that were almost 50% higher than the benchmark recall rate for screening mammography (24). This suggests that radiologists who agreed to take part in this study may have done so to reduce what they acknowledge to be excessive recall rates. In addition, after illustrating to participants during the one-hour CME intervention that recall rates beyond seven percent are not likely to identify many more cancers, nearly seventy-two percent of participants reported considering changing clinical practice to reduce unnecessary recall. Over half of the participants who considered changing their clinical practice developed specific goals that could realistically reduce unnecessary recall. This finding is important because research has shown that an important predictor of actual behavior change is intention to change behavior (25), and successfully setting realistic goals provides such an indication. The average recall rate for those who set realistic goals was 1.3% lower compared to those who did not set reasonable goals, which suggests that those closer to the benchmark were more highly motivated to make a change in practice relative to those whose recall rates were higher.

Among those who set realistic goals, the vast majority chose to undertake a re-review of cases where they felt the likelihood of a cancer being present was very low. Some considered re-reviewing cases they called BI-RADS category "0" or asymmetries. Only a few indicated they would obtain a second opinion as a goal to reduce recall, and, in fact, one participant indicated s/he would cease the practice of asking for a second opinion as s/he felt this was increasing unnecessary recall. This is a very different approach compared to how mammography is practiced outside the US. Interpretation in the UK and elsewhere includes routine double reading, and recall rates that are much lower than in the US (26, 27).

Interestingly, radiologists had strong reactions to the malpractice module that was part of the intervention. The majority of participating radiologists reported the perception that greater than 50% of radiologists are concerned that fear about medical malpractice affects how they interpret mammograms. Before the intervention, more participating radiologists who set realistic goals to reduce unnecessary recall reported that medical malpractice influences both their recall rates and biopsy recommendations than those who did not set goals. Importantly, these scores dropped by half after the intervention, and 100% of radiologists correctly indicated that the probable risk of a mammography related malpractice suit occurring in the next five years was less than 10% among radiologists working full time in breast imaging. In addition, many text comments provided by radiologists underscored the stress that radiologists experience when thinking about malpractice.

The module on breast cancer risk appeared to provide important information to radiologists, especially how risk factors differ among pre- versus postmenopausal women. The pre-intervention questions indicated that radiologists who set realistic goals were more likely to consider breast density a risk factor than those who did not set goals. Post intervention, radiologists who set realistic goals versus those who didn't were more likely to view older age at first birth and older age at mammography among pre-menopausal women as important risk factors. When radiologists considered risk factors among postmenopausal women, they generally rated them as more important than the same risk factors for premenopausal women, which was part of the knowledge content of the module. There were no significant differences in content knowledge between those who did and those who didn't set realistic goals. This finding suggests that the intention to change practice is not necessarily influenced by changes in knowledge. It may also be, as illustrated by the text responses to this module, that radiologists differ in their use of risk factors and any changes in knowledge may not greatly influence practice.

Our prior research on the use of risk factors in interpretation (28) indicates that the majority of radiologists (63.4%) report usually or always using clinical history when interpreting screening mammography; 29.2% reported using it only when they identified an abnormality, and 7.4% reported never or rarely using it. We found that sensitivity, false-positive rate, and PPV did not significantly differ by use of clinical history, however compared to radiologists who rarely use clinical history, radiologists who usually or always use it had a higher false-positive rate associated with either younger patient age, denser breast tissue, or longer screening interval. Participation in our intervention was likely the first time that our study radiologists were presented with the specific risk factors of their patient panel in combination with breast cancer rates showing how low the risk of breast cancer is in the population they actually serve.

The strengths of this study include that we were able to develop an intervention that was automatically populated with data tailored to the each individual radiologist. Over half of the radiologists who took part in the study and who developed goals for reducing unnecessary recall identified goals likely to improve practice. A weakness is that we have not yet assessed the impact of the intervention on actual recall rates, though we plan to do this when follow-up data collection is complete. In addition, recall rates may be overestimated for this group. The definition used to determine recall, though consistent with the ACR BI-RADS manual, may misclassify some diagnostic exams as screening exams because we only used the radiologist’s indication for exams.

In conclusion, interventions designed to reduce unnecessary recall can succeed in assisting radiologists to develop goals that may ultimately reduce unnecessary recall.