Radiologists’ Imminent Demise at the Hand of Artificial Intelligence: Not Quite Yet!

See also article by Topff et al in this issue.

Brett M. Elicker, MD, is a clinical professor in the Department of Radiology and Biomedical Imaging at the University of California, San Francisco (UCSF). He did a radiology residency at Yale University and a thoracic imaging fellowship at UCSF. His clinical and research interests are in the areas of diffuse lung disease and lung cancer.

Few topics incite as much excitement and fear in radiologists as artificial intelligence (AI). The initial reports of radiology’s imminent demise at the hands of AI, however, seem to have been greatly exaggerated. Early predictions of the date on which radiologists would become superfluous have long since passed. While AI algorithms may eventually take over the primary interpretative role in radiology, and medicine in general, for now they remain an ancillary tool that have the potential to increase radiologists’ accuracy and efficiency. A variety of potential uses of AI have been suggested, including those related to interpretation, quantification, and workflow management. This last use in particular may have the greatest potential for near-term rollout in actual clinical practice. With the increased volumes of radiology examinations, particularly in thoracic imaging, and a lack of a subsequent rise in the supply of radiologists, the delay between the completion of CT examinations and interpretation has increased significantly in many practices. Thus, the ability to identify examinations that require more immediate attention in an automated fashion could have a significant impact for patients requiring urgent intervention. The study by Topff and colleagues in this issue of Radiology: Cardiothoracic Imaging addresses this idea in the context of incidental pulmonary emboli (PE) found at routine contrast-enhanced chest CT (1).

Incidental PE are estimated to be present in 1%–4% of routine chest CT (2) and are most prevalent in high-risk populations, such as those undergoing cancer staging. A variety of factors help determine the need for anticoagulation in patients with PE, including the size and number of PE, the patient’s underlying diagnosis, and the presence of comorbidities. Anecdotally, however, many of these incidentally discovered PE are treated with anticoagulation, even when they only involve small vessels. This is despite evidence that treatment of small emboli does not improve outcomes (3). While dedicated CT-PE protocols are generally given high priority for interpretation, routine chest CT examinations may have a significant delay between completion and interpretation given their nonurgent nature; thus, incidental PE may undergo delay in diagnosis. This is particularly problematic in practices with a significant backlog of cases. Additionally, it is more likely for PE to be missed at a routine chest CT for several reasons: (a) the contrast material bolus is not optimized for PE detection, (b) routine chest CT examinations may be performed with a thicker section reconstruction thickness, and (c) radiologists are focused on evaluating other abnormalities and are not specifically focused on a PE diagnosis.

AI applied to the detection of incidental PE at routine chest CT is a potential solution to these issues. It can flag cases as likely having incidental PE so that rapid interpretation by radiologists can be performed. It also has the potential to detect emboli that radiologists might not visualize because they are focusing on another specific clinical indication. To this end, Topff and colleagues investigated the detection of incidental PE at routine chest CT performed for cancer staging on three separate cohorts. The first cohort underwent routine interpretation without AI analysis (1). On the second cohort, radiologists were specifically instructed to look for incidental PE, but also did not have the aid of AI analysis. AI assistance was applied to the third cohort, the results of which were available to the radiologist at the time of interpretation. The performance of the AI algorithm was quite good, with a sensitivity of 91.6%, specificity of 99.7%, and accuracy of 99.6%. Most (>80%) of the false-negative results of AI were in small (segmental or subsegmental) vessels. The AI algorithm was retrospectively applied to the two cohorts that initially did not have AI support and detected 105 emboli, 44.8% of which were not visualized by the radiologist at the time of initial interpretation. It is important to note that only 16.7% of these were in proximal arteries, with the remainder being segmental or subsegmental in location. With regard to the time from examination completion to interpretation, the delay when using the AI algorithm was just over 2 hours; however, without the AI algorithm the delay averaged between 3.5 and 5.4 days.

The results of this study support the potential utility of AI in the detection of PE at routine chest CT as a means to decrease the time to detection and increase the detection rate. A few important caveats must be considered, however. First, the majority of PE detected by the AI algorithm, but not by radiologists, were in small vessels. The clinical significance of small emboli is questionable. Not only has a meta-analysis demonstrated a lack of benefit in patients treated for small emboli (3), but there is also indirect evidence that missed small emboli may not have a significant impact on outcomes. In a direct comparison between single- and multidetector CT, the rate of isolated subsegmental emboli was more than double in the multidetector CT group compared with the single-detector CT group. At the same time, the negative predictive value of clinical follow-up in these two cohorts was nearly identical (4), suggesting no added clinical benefit of detecting a greater number of small emboli. In a randomized controlled trial of CT versus ventilation-perfusion imaging for PE, the positive PE rate was 17.7% and 11.7% in the two groups, respectively (5). It is likely that small emboli accounted for a significant percentage of this difference. This study also demonstrated no difference in clinical outcomes in the CT versus ventilation-perfusion cohorts. Given that over 80% of the emboli missed by radiologists and detected by the AI algorithm in the study by Topff and colleagues were in small vessels, the clinical relevance of the increased rate of detection is questionable (1).

In general, the interpretation of a PE-CT is binary; PE is present or absent. On the other hand, there are a variety of artifacts that may mimic PE, which may lead to uncertainty in diagnosis. In one analysis (6), a group of experts reviewed a series of positive PE-CT examinations. In retrospect, it was thought that over 25% of cases initially interpreted as positive were false-positive. On one hand, radiologists need to report the presence of PE regardless of their size and number, and it is a clinician’s decision as to whether to treat the patient based on the specific risks and benefits of that particular patient. On the other hand, radiologists determine their own individual threshold for describing a study as positive or negative, and that threshold should probably differ depending on the size of suspected PE. In other words, the threshold for describing a central PE as being present should probably differ from that of a subsegmental PE. The addition of an AI algorithm as a “second reader” has a significant potential to bias the radiologist’s interpretation in this respect in one direction or the other. This bias might influence a radiologist to describe a small embolus as being present, when without the algorithm they would have described no PE. This is another factor that needs to be considered in the integration of an AI algorithm and has the potential to violate the “first do no harm” principle of medicine.

In summary, the study by Topff and colleagues provides thought-provoking results and suggests a clinically relevant use of AI in the workflow of practicing radiologists (1). It has the potential to decrease the delay in diagnosis of incidental PEs and decrease the rate of “missed” PEs. On the other hand, there are several complicating factors that should be considered before implementation, centered around the general concepts of maximizing specificity when interpreting PE studies and not succumbing to the potential of AI-induced bias.