Skip to main content
European Urology Open Science logoLink to European Urology Open Science
letter
. 2022 Dec 19;48:14–16. doi: 10.1016/j.euros.2022.09.024

Is Artificial Intelligence Replacing Our Radiology Stars? Not Yet!

Giovanni E Cacciamani a,b,c,d,e,, Daniel I Sanford a,b, Timothy N Chu a,b, Masatomo Kaneko a,b,e,f, Andre L De Castro Abreu a,b,c,d,e, Vinay Duddalwar c,d, Inderbir S Gill a,b,c,e
PMCID: PMC9794880  PMID: 36588775

Take Home Message

Radiologists assisted by artificial intelligence (AI) algorithms demonstrate superior performance for interpretation of prostate magnetic resonance imaging in comparison to radiologists or AI systems alone. A hybrid technology-human system could leverage the benefits of AI for radiologist performance while reducing burnout and enhancing patient care.

Keywords: Artificial intelligence, Machine learning, Deep learning, Prostate cancer, Radiology, Performance, Radiomics, Multiparametric magnetic resonance imaging

Abstract

Artificial intelligence (AI) is here to stay and will change health care as we know it. The availability of big data and the increasing numbers of AI algorithms approved by the US Food and Drug Administration together will help in improving the quality of care for patients and in overcoming human fatigue barriers. In oncology practice, patients and providers rely on the interpretation of radiologists when making clinical decisions; however, there is considerable variability among readers, and in particular for prostate imaging. AI represents an emerging solution to this problem, for which it can provide a much-needed form of standardization. The diagnostic performance of AI alone in comparison to a combination of an AI framework and radiologist assessment for evaluation of prostate imaging has yet to be explored. Here, we compare the performance of radiologists alone versus a combination of radiologists aided by a modern computer-aided diagnosis (CAD) AI system. We show that the radiologist-CAD combination demonstrates superior sensitivity and specificity in comparison to both radiologists alone and AI alone. Our findings demonstrate that a radiologist + AI combination could perform best for detection of prostate cancer lesions. A hybrid technology-human system could leverage the benefits of AI in improving radiologist performance while also reducing physician workload, minimizing burnout, and enhancing the quality of patient care.

Patient summary

Our report demonstrates the potential of artificial intelligence (AI) for improving the interpretation of prostate scans. A combination of AI and evaluation by a radiologist has the best performance in determining the severity of prostate cancer. A hybrid system that uses both AI and radiologists could maximize the quality of care for patients while reducing physician workload and burnout.


It is inevitable. Artificial intelligence (AI) will change health care as we know it. AI applications range from improved clinical decision-making to enhanced computer vision image analysis [1]. However, AI is not a “new kid on the block”. Alan Turing first mentioned AI in 1950 in his iconic essay The Imitation Game, in which he wondered, “Can a machine think? … What will happen when a machine takes the part of a human in a process?” The availability of big data and the increasing numbers of FDA-approved AI algorithms together will help health care professionals overcome human fatigue barriers and receive support for recurring tasks in daily practice. With these enhancements, physician performance and patient quality of care would certainly improve.

In oncology care, patients and providers rely on accurate interpretation of imaging studies during health care discussions and shared decision-making. In some cases, these critical imaging findings can be the difference between an invasive surgical intervention and watchful waiting. We rely on interpretation by radiologists; however, it is well documented that there is considerable variability among readers [2]. This discordance may be attributable to systematic differences in image acquisition methods or image quality, as well as radiologist-specific factors such as experience, internal bias, reader fatigue, and inconsistent reporting [2]. Despite improvements in imaging quality, discordance in interpretation has remained surprisingly consistent and exists across all domains of medicine [2], [3].

This discordance in interpretation has been consistently noted in the grading of Prostate Imaging-Reporting and Data System category 3 lesions and is often significant [4], [5], [6]. There are thus clear pitfalls in prostate magnetic resonance imaging (MRI), including inter-reader variability, difficulty in assessing benign “mimickers”, and imaging artifacts related to technical or physiological factors [7]. Awareness of these pitfalls is critical in prostate image assessment; however, there is a need to further optimize image analysis for accurate cancer detection.

AI could represent an emerging solution to this problem. Taking advantage of computerized algorithms and high-throughput learning, AI can provide a much-needed form of standardization in image analysis. Proponents argue that AI can help in delineating subtle imaging findings via powerful image analyses that can be used to predict odds of malignancy and anticipated tumor progression patterns with modern computer aided-diagnosis (CAD) systems. Historical CAD systems relied on data input by users without the ability to self-learn, while modern CAD systems incorporating AI can autonomously learn and adapt with new data that are presented [8]. In collaboration with expert radiologists, AI algorithms are trained on large “ground truth” data sets and used to reduce the uncertainty in image interpretation [9]. In addition, AI has many potential applications for health systems that can reduce the staffing shortages and the burnout that health care workers currently face [10]. For example, implementation of an automated image processing software that autonomously detects and flags suspicious lesions could expedite radiologist review and thus improve productivity, reduce workload, and enhance overall performance [10]. While these applications undoubtedly face ethical challenges and have liability concerns, potential solutions are realizable and of practical importance. When applied to urology, successful implementation of AI systems can have an important benefit in reducing unnecessary prostate biopsies while maintaining high accuracy in ruling out disease [11]. Efforts to develop AI systems that serve as diagnostic support aids and can be seamlessly integrated into a radiologist’s workflow are necessary for successful AI application [11].

A systematic review and meta-analysis of several medical specialties revealed that the diagnostic performance of deep learning models is equivalent to that of health care professionals, although it also noted that few studies presented externally validated results with poor reporting of methodology, limiting interpretation of the diagnostic accuracy [12]. Despite abundant, well-cited literature on the use of AI in prostate imaging, only one study looking at multiparametric MRI (mpMRI) scans of patients undergoing robot-assisted laparoscopic prostatectomy was included because of the rigid inclusion criteria [12]. Given this limitation, further assessment of the diagnostic performance of AI frameworks in comparison to health care professionals for prostate imaging evaluation is warranted.

In this context, we present preliminary findings from a project to address this deficiency in available data for detecting prostate cancer with and without AI models. The pilot cumulative analysis includes five studies (six references, seven contingency tables; Supplementary material) that used mpMRI for prostate cancer detection and compares the performance of radiologists alone versus a combination of radiologists aided by an AI-based CAD system. Our analysis showed pooled sensitivity of 89.1% (95% confidence interval [CI] 80.6–94.2%) for the radiologist-CAD combination and 79.5% (95% CI 75.5–84.3%) for radiologists alone (Fig 1). The pooled specificity was 78.1% (95% CI 64.8–87.4%) for the radiologist-CAD combination compared to 73.1% (95% CI 61.9–82.0%) for radiologists alone. A systematic review of AI system performance revealed average sensitivity of 84% and specificity of 61.5% [13]. Our pooled analysis for the radiologist-AI combination demonstrates superior performance in comparison to both radiologists alone and AI alone. Therefore, the real game-changing move could be a “partnership” between AI systems and health care professionals rather than “replacement” of the latter.

Fig. 1.

Fig. 1

Hierarchical summary receiver operating characteristic (HSROC) curves for studies assessing the performance of (A) radiologists alone and (B) a combination of artificial intelligence algorithms and radiologists (5 studies, 6 contingency tables) in the detection of any prostate cancer using multiparametric magnetic resonance imaging. CI = confidence interval; PI-RADS = Prostate Imaging-Reporting and Data System.

The current evidence demonstrates that a combination of radiologists and AI-based systems could perform best for the detection of prostate cancer lesions. Not surprisingly, we are still far from a totally automated diagnostic pathway. These fully automated systems are still limited by the requirement for professional oversight, as well as ethical considerations and liability concerns [14]. However, a hybrid technology-human system could leverage the benefits of AI in improving radiologist performance while also reducing physician workload, minimizing burnout, and enhancing the quality of patient care.



Conflicts of interest: The authors have nothing to disclose.

Footnotes

Appendix A

Supplementary data to this article can be found online at https://doi.org/10.1016/j.euros.2022.09.024.

Appendix A. Supplementary data

The following are the Supplementary data to this article:

Supplementary Data 1

Studies included in the analysis

mmc1.docx (15.3KB, docx)

References

  • 1.Chen A.B., Haque T., Roberts S., et al. Artificial intelligence applications in urology: reporting standards to achieve fluency for urologists. Urol Clin North Am. 2022;49:65–117. doi: 10.1016/j.ucl.2021.07.009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Schmid A.M., Raunig D.L., Miller C.G., et al. Radiologists and clinical trials: part 1. The truth about reader disagreements. Ther Innov Regul Sci. 2021;55:1111–1121. doi: 10.1007/s43441-021-00316-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Yoon S.H., Kim Y.J., Doh K., et al. Interobserver variability in Lung CT Screening Reporting and Data System categorisation in subsolid nodule-enriched lung cancer screening CTs. Eur Radiol. 2021;31:7184–7191. doi: 10.1007/s00330-021-07800-5. [DOI] [PubMed] [Google Scholar]
  • 4.Weinreb J.C., Barentsz J.O., Choyke P.L., et al. PI-RADS Prostate Imaging-Reporting and Data System: 2015, version 2. Eur Urol. 2016;69:16–40. doi: 10.1016/j.eururo.2015.08.052. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Smith C.P., Harmon S.A., Barrett T., et al. Intra- and interreader reproducibility of PI-RADS v2: a multireader study. J Magn Reson Imaging. 2019;49:1694–1703. doi: 10.1002/jmri.26555. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Rosenkrantz A.B., Ginocchio L.A., Cornfeld D., et al. Interobserver reproducibility of the PI-RADS version 2 lexicon: a multicenter study of six experienced prostate radiologists. Radiology. 2016;280:793–804. doi: 10.1148/radiol.2016152542. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Panebianco V., Giganti F., Kitzing Y.X., et al. An update of pitfalls in prostate mpMRI: a practical approach through the lens of PI-RADS vol 2 guidelines. Insights Imaging. 2018;9:87–101. doi: 10.1007/s13244-017-0578-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Oren O., Gersh B.J., Bhatt D.L. Artificial intelligence in medical imaging: switching from radiographic pathological data to clinically meaningful endpoints. Lancet Digit Health. 2020;2:e486–e488. doi: 10.1016/S2589-7500(20)30160-6. [DOI] [PubMed] [Google Scholar]
  • 9.Chen P.C., Mermel C.H., Liu Y. Evaluation of artificial intelligence on a reference standard based on subjective interpretation. Lancet Digit Health. 2021;3:e693–e695. doi: 10.1016/S2589-7500(21)00216-8. [DOI] [PubMed] [Google Scholar]
  • 10.Hazarika I. Artificial intelligence: opportunities and implications for the health workforce. Int Health. 2020;12:241–245. doi: 10.1093/inthealth/ihaa007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Penzkofer T., Padhani A.R., Turkbey B., et al. ESUR/ESUI position paper: developing artificial intelligence for precision diagnosis of prostate cancer using magnetic resonance imaging. Eur Radiol. 2021;31:9567–9578. doi: 10.1007/s00330-021-08021-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Liu X., Faes L., Kale A.U., et al. A comparison of deep learning performance against health-care professionals in detecting diseases from medical imaging: a systematic review and meta-analysis. Lancet Digit Health. 2019;1:e271–e297. doi: 10.1016/S2589-7500(19)30123-2. [DOI] [PubMed] [Google Scholar]
  • 13.Syer T., Mehta P., Antonelli M., et al. Artificial intelligence compared to radiologists for the initial diagnosis of prostate cancer on magnetic resonance imaging: a systematic review and recommendations for future studies. Cancers. 2021;13:3318. doi: 10.3390/cancers13133318. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.World Health Organization . WHO; Geneva: 2021. Ethics and governance of artificial intelligence for health: WHO guidance. [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary Data 1

Studies included in the analysis

mmc1.docx (15.3KB, docx)

Articles from European Urology Open Science are provided here courtesy of Elsevier

RESOURCES