Skip to main content
PMC home page
Elsevier - PMC COVID-19 Collection logoLink to Elsevier - PMC COVID-19 Collection
. 2023 Jan 31;218:1915–1925. doi: 10.1016/j.procs.2023.01.168

An Explainable AI driven Decision Support System for COVID-19 Diagnosis using Fused Classification and Segmentation

K Niranjan a, S Shankar Kumar a, S Vedanth a, Dr S Chitrakala a
PMCID: PMC9886321  PMID: 36743792

Abstract

The coronavirus has caused havoc on billions of people worldwide. The Reverse Transcription Polymerase Chain Reaction(RT-PCR) test is widely accepted as a standard diagnostic tool for detecting infection, however, the severity of infection can't be measured accurately with RT-PCR results. Chest CT Scans of infected patients can manifest the presence of lesions with high sensitivity. During the pandemic, there is a dearth of competent doctors to examine chest CT images. Therefore, a Guided Gradcam based Explainable Classification and Segmentation system (GGECS) which is a real-time explainable classification and lesion identification decision support system is proposed in this work. The classification model used in the proposed GGECS system is inspired by Res2Net. Explainable AI techniques like GradCam and Guided GradCam are used to demystify Convolutional Neural Networks (CNNs). These explainable systems can assist in localizing the regions in the CT scan that contribute significantly to the system's prediction. The segmentation model can further reliably localize infected regions. The segmentation model is a fusion between the VGG-16 and the classification network. The proposed classification model in GGECS obtains an overall accuracy of 98.51 % and the segmentation model achieves an IoU score of 0.595.

Keywords: COVID-19, Explainable AI, Segmentation, Convolutional Neural networks, Class Activavtion Mapping, Gradient weighted Class Activation Mapping (Grad-CAM), Guided Grad-CAM

References

  • 1.Abiyev Rahib H, Ma'aitaH Mohammad Khaleel Sallam. Deep convolutional neural networks for chest diseases detection. Journal of healthcare engineering. 2018:2018. doi: 10.1155/2018/4168538. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Afshar Parnian, Heidarian Shahin, Enshaei Nastaran, Naderkhani Farnoosh, Rafee Moezedin Javad, Oikonomou Anastasia, Fard Faranak Babaki, Samimi Kaveh, Plataniotis Konstantinos N, Mohammadi Arash. Covid-ct-md, covid-19 computed tomography scan dataset applicable in machine learning and deep learning. Scientific Data. 2021;8(1):1–8. doi: 10.1038/s41597-021-00900-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Afshar Parnian, Heidarian Shahin, Naderkhani Farnoosh, Oikonomou Anastasia, Plataniotis Konstantinos N, Mohammadi Arash. Covid-caps: A capsule network-based framework for identification of covid-19 cases from x-ray images. Pattern Recognition Letters. 2020;138:638–643. doi: 10.1016/j.patrec.2020.09.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Chollet François. Proceedings of the IEEE conference on computer vision and pattern recognition. 2017. Xception: Deep learning with depthwise separable convolutions; pp. 1251–1258. pages. [Google Scholar]
  • 5.Fan Deng-Ping, Zhou Tao, Ji Ge-Peng, Zhou Yi, Chen Geng, Fu Huazhu, Shen Jianbing, Shao Ling. Inf-net: Automatic covid-19 lung infection segmentation from ct images. IEEE Transactions on Medical Imaging. 2020;39(8):2626–2637. doi: 10.1109/TMI.2020.2996645. [DOI] [PubMed] [Google Scholar]
  • 6.Gao Kai, Su Jianpo, Jiang Zhongbiao, Zeng Ling-Li, Feng Zhichao, Shen Hui, Rong Pengfei, Xu Xin, Qin Jian, Yang Yuexiang, et al. Dual-branch combination network (dcn): Towards accurate diagnosis and lesion segmentation of covid-19 using ct images. Medical image analysis. 2021;67 doi: 10.1016/j.media.2020.101836. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Gilpin Leilani H, Bau David, Yuan Ben Z, Bajwa Ayesha, Specter Michael, Kagal Lalana. 2018 IEEE 5th International Conference on data science and advanced analytics (DSAA) IEEE; 2018. Explaining explanations: An overview of interpretability of machine learning; pp. 80–89. pages. [Google Scholar]
  • 8.Guidotti Riccardo, Monreale Anna, Ruggieri Salvatore, Turini Franco, Giannotti Fosca, Pedreschi Dino. A survey of methods for explaining black box models. ACM computing surveys (CSUR) 2018;51(5):1–42. [Google Scholar]
  • 9.Hasan Md Jahid, Alom Md Shahin, Ali Md Shikhar. 2021 International Conference on Information and Communication Technology for Sustainable Development (ICICT4SD) IEEE; 2021. Deep learning based detection and segmentation of covid-19 & pneumonia on chest x-ray image; pp. 210–214. pages. [Google Scholar]
  • 10.He Kaiming, Zhang Xiangyu, Ren Shaoqing, Sun Jian. Proceedings of the IEEE conference on computer vision and pattern recognition. 2016. Deep residual learning for image recognition; pp. 770–778. pages. [Google Scholar]
  • 11.He Tao, Guo Jixiang, Chen Nan, Xu Xiuyuan, Wang Zihuai, Fu Kaiyu, Liu Lunxu, Yi Zhang. Medimlp: using grad-cam to extract crucial variables for lung cancer postoperative complication prediction. IEEE Journal of Biomedical and Health Informatics. 2019;24(6):1762–1771. doi: 10.1109/JBHI.2019.2949601. [DOI] [PubMed] [Google Scholar]
  • 12.Holzinger Andreas, Biemann Chris, Pattichis Constantinos S, Kell Douglas B. What do we need to build explainable ai systems for the medical domain? arXiv preprint. 2017 arXiv:1712.09923. [Google Scholar]
  • 13.Krizhevsky Alex, Sutskever Ilya, Hinton Geofrey E. Vol. 25. 2012. Imagenet classification with deep convolutional neural networks. (Advances in neural information processing systems). [Google Scholar]
  • 14.Maftouni Maede, Law Andrew Chung Chee, Shen Bo, Grado Zhenyu James Kong, Zhou Yangze, Yazdi Niloofar Ayoobi. IIE Annual Conference. Proceedings. Institute of Industrial and Systems Engineers (IISE; 2021. A robust ensemble-deep learning model for covid-19 diagnosis based on an integrated ct scan images database; pp. 632–637. pages. [Google Scholar]
  • 15.Mahajan Arpana, Chaudhary Sanjay. 2019 3rd International conference on Electronics, Communication and Aerospace Technology (ICECA) IEEE; 2019. Categorical image classification based on representational deep network (resnet) pp. 327–330. pages. [Google Scholar]
  • 16.Ozturk Tulin, Talo Muhammed, Yildirim Eylul Azra, Baloglu Ulas Baran, Yildirim Ozal, Rajendra Acharya U. Automated detection of covid-19 cases using deep neural networks with x-ray images. Computers in biology and medicine. 2020;121 doi: 10.1016/j.compbiomed.2020.103792. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Pennisi Matteo, Kavasidis Isaak, Spampinato Concetto, Schinina Vincenzo, Palazzo Simone, Salanitri Federica Proietto, Bellitto Giovanni, Rundo Francesco, Aldinucci Marco, Cristofaro Massimo, et al. An explainable ai system for automated covid-19 assessment and lesion categorization from ct-scans. Artificial intelligence in medicine. 2021;118 doi: 10.1016/j.artmed.2021.102114. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Qin Chunli, Yao Demin, Shi Yonghong, Song Zhijian. Computer-aided detection in chest radiography based on artificial intelligence: a survey. Biomedical engineering online. 2018;17(1):1–23. doi: 10.1186/s12938-018-0544-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Rastogi Priyanka, Khanna Kavita, Singh Vijendra. Gland segmentation in colorectal cancer histopathological images using u-net inspired convolutional network. Neural Computing and Applications. 2022;34(7):5383–5395. [Google Scholar]
  • 20.Rastogi Priyanka, Singh Vijendra, Yadav Monika. 2018 Fifth International Conference on Parallel, Distributed and Grid Computing (PDGC) IEEE; 2018. Deep learning and big datatechnologies in medical image analysis; pp. 60–63. pages. [Google Scholar]
  • 21.Ribeiro Marco Tulio, Singh Sameer, ” Carlos Guestrin. Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. 2016. why should i trust you?” explaining the predictions of any classifier; pp. 1135–1144. pages. [Google Scholar]
  • 22.Ronneberger Olaf, Fischer Philipp, Brox Thomas. International Conference on Medical image computing and computer-assisted intervention. Springer; 2015. U-net: Convolutional networks for biomedical image segmentation; pp. 234–241. pages. [Google Scholar]
  • 23.Samek Wojciech, Wiegand Thomas, Müller Klaus-Robert. Explainable artificial intelligence: Understanding, visualizing and interpreting deep learning models. arXiv preprint. 2017 arXiv:1708.08296. [Google Scholar]
  • 24.Selvaraju Ramprasaath R, Cogswell Michael, Das Abhishek, Vedantam Ramakrishna, Parikh Devi, Batra Dhruv. Proceedings of the IEEE international conference on computer vision. 2017. Grad-cam: Visual explanations from deep networks via gradient-based localization; pp. 618–626. pages. [Google Scholar]
  • 25.Prabira Kumar Sethy and Santi Kumari Behera. Detection of coronavirus disease (covid-19) based on deep features. 2020.
  • 26.Simonyan Karen, Zisserman Andrew. Very deep convolutional networks for large-scale image recognition. arXiv preprint. 2014 arXiv:1409.1556. [Google Scholar]
  • 27.Singh Vijendra, Asari Vijayan K, Rajasekaran Rajkumar. A deep neural network for early detection and prediction of chronic kidney disease. Diagnostics. 2022;12(1):116. doi: 10.3390/diagnostics12010116. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Tena Alberto, Clarià Francesc, Solsona Francesc. Automated detection of covid-19 cough. Biomedical Signal Processing and Control. 2022;71 doi: 10.1016/j.bspc.2021.103175. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Wang Fei, Jiang Mengqing, Qian Chen, Yang Shuo, Li Cheng, Zhang Honggang, Wang Xiaogang, Tang Xiaoou. Proceedings of the IEEE conference on computer vision and pattern recognition. 2017. Residual attention network for image classification; pp. 3156–3164. pages. [Google Scholar]
  • 30.Zhou Bolei, Khosla Aditya, Lapedriza Agata, Oliva Aude, Torralba Antonio. Proceedings of the IEEE conference on computer vision and pattern recognition. 2016. Learning deep features for discriminative localization; pp. 2921–2929. pages. [Google Scholar]
  • 31.Zoabi Yazeed, Deri-Rozov Shira, Shomron Noam. Machine learning-based prediction of covid-19 diagnosis based on symptoms. npj digital medicine. 2021;4(1):1–5. doi: 10.1038/s41746-020-00372-6. [DOI] [PMC free article] [PubMed] [Google Scholar]

Articles from Procedia Computer Science are provided here courtesy of Elsevier

RESOURCES