AI as a Second Reader Can Reduce Radiologists’ Workload and Increase Accuracy in Screening Mammography

See also the article by Elhakim et al in this issue.

Abhinav Suri, MPH, is a medical student at the David Geffen School of Medicine at the University of California Los Angeles. His research focuses on the intersection between artificial intelligence and radiology for opportunistic screening of diseases. He is a member of the trainee editorial board of Radiology: Artificial Intelligence and has authored a book titled Practical AI for Healthcare Professionals.

Imagine the following scenario: You or someone you know walks into a clinic for routine mammography. As you check in at the front desk, you’re asked, “Would you be comfortable with an AI system assisting in reading your mammogram?” Although this question might seem simple, its implications are complex. The role of artificial intelligence (AI) in health care, especially in radiology, is evolving rapidly, and we are only beginning to understand its potential impact. In the realm of breast cancer screening, AI may help reduce the burden of increasing workload, especially in many European countries where the standard practice for interpreting screening mammograms involves a double-reading system (1). For each mammogram, two radiologists independently review each scan, and a third radiologist serves as an arbitrator if there is any disagreement between the first two readings. This practice aims to increase diagnostic accuracy and reduce the likelihood of missed cancers, but it also substantially increases the workload for radiologists, who are already in short supply in many regions (2,3).

The advent of AI promises to alleviate some of these challenges by serving as an additional “pair of eyes” or even by replacing one of the human readers in this double-reading process. However, these roles for AI raise several important questions: Will AI actually improve diagnostic outcomes, or will it create new challenges and cognitive burdens for radiologists? Could an overreliance on AI lead to biases in clinical decision-making? As AI systems begin to be deployed in clinical environments, there is an urgent need to rigorously evaluate their performance using large-scale, representative datasets.

In response to this need, researchers have investigated what AI could look like when aiding in breast cancer screening. A study published last year in Lancet Oncology explored the effectiveness of AI in assisting with double reading in mammography screening (4). The study demonstrated that AI-assisted interpretations could match or even exceed the performance of traditional double-reading methods in terms of cancer detection rate, recall rate, and positive predictive value (PPV). Conducted across four screening sites in Sweden and involving more than 80 000 female individuals, the study made headlines and contributed to the growing body of evidence supporting the integration of AI into breast cancer screening. The findings suggested that AI could be used to support radiologists, either by acting as a first or second reader or by helping to prioritize scans that require more attention.

Building on this foundation, the study by Elhakim et al (5), published in this issue of Radiology: Artificial Intelligence, sought to provide a more comprehensive evaluation of AI’s role in mammography screening by simulating its use in three different scenarios within a large and diverse population of 249 402 mammograms collected between 2014 and 2018. This study represents one of the most extensive investigations to date into the use of AI in a real-world screening context. The authors aimed to determine whether AI could be integrated effectively into the double-reading process, either by replacing one of the human readers or by serving as a triage tool to help categorize mammograms into different risk levels.

The authors simulate the performance of AI and humans across three scenarios. In the first scenario, AI replaced the first reader of these mammograms (using an AI decision threshold set to match the specificity of the first reader across all the original mammography interpretations). The original second reader (human) readings were retained, and in the case of disagreement, the arbitration decision was simulated to match the level of accuracy of the original arbitrator. In the second scenario, the AI replaced the second reader (using the same threshold as the first scenario, arbitrations were handled in a similar manner as well). In the third scenario, the AI system was used to triage mammograms to low risk (no recall) versus high risk (recall), with the remainder of scans (targeted to be 50% of the original number of mammograms) being sent to radiologists to review in the standard double-reader fashion. The authors then looked at the sensitivity, specificity, PPV, negative predictive value (NPV), recall rate, and arbitration rate as measures of overall performance of each of the scenarios.

In aggregate, the authors found that replacing the first reader yielded a higher cancer detection rate (6.09 [AI] vs 6.03 [control]) at the cost of a slightly higher (but statistically significant) arbitration rate (+0.99%). Replacing the second reader led to a lower cancer detection rate (5.91 vs 6.03) and lower sensitivity (−1.58%) but a higher PPV (+0.03%). Using AI as a triage tool (third scenario) yielded a higher cancer detection rate (6.14 vs 6.03), sensitivity (+1.33%), PPV (+0.36%), and NPV (+0.01%) and a lower arbitration rate (−0.88%). Most notably, the number of mammograms requiring human review was almost halved across all three scenarios, demonstrating that AI can effectively prioritize cases without compromising diagnostic accuracy. This finding has important implications for health care systems, particularly those facing shortages of radiologists or high demands for mammography screening.

However, there are some important contextual considerations to keep in mind while evaluating the results of this study. The results in this study only pertain to one AI software product, which will limit the applicability of these results to other vendors. Additionally, the population on which the AI was simulated was a cohort of patients in southern Denmark, which may not apply to populations in other regions. Last, the results of this study are reported on a simulated screening scenario (with thresholds for recall based on retrospective data). While the simulated scenarios suggest that AI can function effectively as either a first or second reader or as a triage tool, the real-world application of these systems will require careful attention to workflow integration, training, and ongoing monitoring of AI performance.

In conclusion, Elhakim and colleagues demonstrate the feasibility of using AI to support and enhance mammography screening practices. This study provides compelling evidence that AI can be effectively integrated into double-reading workflows, either as a replacement for one of the readers or as a triage tool, without sacrificing diagnostic accuracy. These findings pave the way for broader adoption of AI in breast cancer screening, offering the potential for more efficient, accurate, and accessible care for patients around the world.

As health care systems continue to grapple with increasing demands and limited resources, the integration of AI into clinical practice represents a promising solution to some of these challenges, ultimately improving outcomes for patients and clinicians alike. The findings of this study are a crucial step forward in understanding the potential roles of AI in breast cancer screening. They suggest that AI can be a valuable tool to reduce the workload for radiologists, while maintaining and improving diagnostic accuracy and optimizing resource allocation in screening programs. As AI technology continues to advance and become more integrated into clinical workflows, ongoing research and rigorous evaluation will be essential to ensure that these systems are used safely and effectively. Future studies can focus on prospective deployments of AI in clinical settings across a diverse clinical population, assessing not only their impact on diagnostic outcomes but also their effects on radiologist workload, patient satisfaction, and health care costs.