The diagnostic accuracy of Artificial Intelligence-Assisted CT imaging in COVID-19 disease: A systematic review and meta-analysis

Abstract

Artificial intelligence (AI) systems have become critical in support of decision-making. This systematic review summarizes all the data currently available on the AI-assisted CT-Scan prediction accuracy for COVID-19. The ISI Web of Science, Cochrane Library, PubMed, Scopus, CINAHL, Science Direct, PROSPERO, and EMBASE were systematically searched. We used the revised Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) tool to assess all included studies' quality and potential bias. A hierarchical receiver-operating characteristic summary (HSROC) curve and a summary receiver operating characteristic (SROC) curve have been implemented. The area under the curve (AUC) was computed to determine the diagnostic accuracy. Finally, 36 studies (a total of 39,246 image data) were selected for inclusion into the final meta-analysis. The pooled sensitivity for AI was 0.90 (95% CI, 0.90–0.91), specificity was 0.91 (95% CI, 0.90–0.92) and the AUC was 0.96 (95% CI, 0.91–0.98). For deep learning (DL) method, the pooled sensitivity was 0.90 (95% CI, 0.90–0.91), specificity was 0.88 (95% CI, 0.87–0.88) and the AUC was 0.96 (95% CI, 0.93–0.97). In case of machine learning (ML), the pooled sensitivity was 0.90 (95% CI, 0.90–0.91), specificity was 0.95 (95% CI, 0.94–0.95) and the AUC was 0.97 (95% CI, 0.96–0.99). AI in COVID-19 patients is useful in identifying symptoms of lung involvement. More prospective real-time trials are required to confirm AI's role for high and quick COVID-19 diagnosis due to the possible selection bias and retrospective existence of currently available studies.

Abbreviations

2019-nCoVs

New Coronaviruses-2019

SARS-CoV-2

Severe Acute Respiratory Syndrome Coronavirus 2

COVID-19

Coronavirus Disease-2019

ALI

Acute Lung injury

ARDS

Acute Respiratory Distress Syndrome

HA

Hyaluronic Acid

ACE2

Angiotensin-Converting Enzyme 2

CXR

Chest X-ray Radiography

CT-Scans

Computed Tomography-Scans

GGO

Ground-Glass Opacity

AI

Artificial Intelligence

ML:

Machine Learning

DL:

Deep Learning

AUC

Area Under the Curve

CI

Confidence Interval

FN

False Negative

FT

False Positive

TN

True Negative

TP

True Positive

QUADAS-2

Quality Assessment of Diagnostic Accuracy Studies 2

HSROC:

Hierarchical Summary Receiver-Operating Characteristic

MOOSE

Meta-analyses Of Observational Studies in Epidemiology

PRISMA

Preferred Reporting Items for Systematic reviews and Meta-Analyses

1. Introduction

The 2019-new coronavirus (2019-nCoV, causing COVID-19 disease) was reported as the cause of the outbreak of pneumonia in Wuhan, Hubei province of China, at the end of 2019 [1]. This virus is associated with the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), a group of beta viruses that cause respiratory, gastrointestinal, neurological diseases in humans. The virus transmission appears to be done via respiratory droplets mainly [2].

COVID-19 patients usually present with trouble breathing, cough, and fever. The COVID-19- associated cytokine storms and innate immune system over-activation can lead to Acute Lung Injury (ALI) and induction of Acute Respiratory Distress Syndrome (ARDS), especially in patients with hypertension [3]. The cytokine storm induces the production of Hyaluronic Acid (HA) molecules in lung tissue, with consequent progressive fibrosis, tissue stiffness, and impaired lung function [4]. SARS-CoV-2 enters the cell by binding to spike (S) glycoproteins of the enzyme Angiotensin-Converting Enzyme 2 (ACE2) receptor [5,6]. Thus, pulmonary involvement is common in patients, and imaging techniques such as Chest X-ray Radiography (CXR) or Computed Tomography (CT-scans) are recommended as the first-line diagnostic tools [7].

Radiological manifestations clinically confirmed, such as unilateral or bilateral multilobar infiltration, Ground-Glass Opacity (GGO), and peripheral infiltration in chest CT-scan, have essential roles in the diagnosis of COVID-19 disease [8,9]. There is often no sign of lung involvement on a CT-scan in the early stages of the infection. In some cases, minimal involvement of up to two pulmonary lobes in the form of GGO, consolidation, or nodules less than one-third the volume of each lobe, especially in the peripheral areas [7,10]. Due to the removal and a high number of CT images of the lungs and its complex and uneven structure, it is challenging to diagnose vessels' nodules in patients' images [11]. Therefore, using computer-assisted techniques, especially Artificial Intelligence (AI) systems, has become more significant in supporting decision-making [12]. AI has great potential to improve clinical decisions; however, such systems' successful implementation requires careful attention to each information system's principles [13]. Due to the abundance and interference of variables in medical decisions, physicians can make faster and more efficient decisions using AI systems and spend more time evaluating decisions.

So far only two systematic reviews and meta-analyses have been performed on AI in the COVID-19 field. Li et al. conducted a systematic review and meta-analysis of 151 published studies to generate a more accurate diagnostic model of COVID-19 using correlations between clinical variables, clustering COVID-19 patients into subtypes, and generating a computational classification model for discriminating between COVID-19 patients and influenza patients based on clinical variables alone [14]. Michelson et al. proposed an approach to answer clinical queries, termed rapid meta-analysis (RMA). Unlike traditional meta-analysis, it is an AI-based method with rapid time to production and reasonable data quality assurances. They performed a RMA on 11 studies and estimated the incidence of ocular toxicity as a side effect of hydroxychloroquine in COVID-19 patients [15]. Thus, the purpose of this meta-analysis was to systematically assess and summarize all of the data currently available on the prediction accuracy of AI-assisted CT-Scanning for COVID-19.

2. Materials and methods

2.1. Protocol and registration

This study was done according to Meta-analyses Of Observational Studies in Epidemiology (MOOSE) [16] and Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) [17], and Synthesizing Evidence from Diagnostic Accuracy TEsts (SEDATE) [18] guidelines.

2.2. Eligibility criteria

Studies suggest that lung involvement in the confirmed cases of COVID-19 patients based on RT-PCR results without language limits were included. We excluded papers that did not fit into the study's conceptual framework focused on other types of infectious diseases.

2.3. Information sources

We systematically searched the ISI Web of Science, Cochrane Library, PubMed, Scopus, CINAHL, Science Direct, PROSPERO, and EMBASE for studies that evaluated the diagnostic accuracy of different models of AI-assisted CT-Scan for predict COVID-19 published between 2020 and 2021 years.

2.4. Search

Two reviewers (K.SH and F.R) performed the search using medical subject headings (MeSh) terms included “artificial neural network” OR “Artificial Intelligence” OR “Machine Learning” OR “expert system” OR “Deep Learning” OR “Supervised Machine Learning” OR “computer-aided” AND “Respiratory Tract Infections” OR “Respiratory System” OR “Coronavirus Infections” OR “COVID-19” OR “SARS COV 2 Infection” AND “Computed Tomography” OR “CT-Scan” and all possible combinations.

2.5. Summary measures

Our desired outcomes were sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV); studies that did not provide sufficient information to calculate true positive (TP, true COVID-19 predicted to be COVID-19 by AI), false positive (FP, non- COVID-19 predicted to be COVID-19), true negative (TN, non- COVID-19 predicted to be non- COVID-19 by AI) and false negative (FN, COVID-19 predicted to be non- COVID-19) values of AI on detection of COVID-19 in the patients, versus healthy control (HC). When the sensitivity and specificity were directly unavailable, we calculated them according to the following formulas: sensitivity = TP/ (TP + FN) and specificity = TN/ (FP + TN).

2.6. Risk of bias across studies

Data extraction for meta-analysis on detection of COVID-19 was based on the definition of criterion standard in the original study. Information including the year of publication, the country where the study was conducted, type of study, number of patients also retrieved. We used the revised Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) tool to assess the quality and potential bias of all studies by two independent reviewers (K.SH., F.R.)

Any disagreements were resolved with discussion and involvement of the third reviewer (B.A.), and reviewers [K.SH., F.R.] assessed the first included articles independently. Four domains, namely patient selection, index test, reference standard, and flow and timing, were assessed. Two categories, including the risk of bias and applicability, were assessed under the domain of patient selection, index test, and reference standard. The risk of bias was assessed in the domain of flow and timing.

2.7. Additional analyses

We used a bivariate model of random effects to estimate sensitivity, accuracy, and 95% confidence intervals (CI). A hierarchical summary receiver operating characteristic (HSROC) curve and a summary receiver operating characteristic (SROC) curve have been mounted. All experiments were viewed with the HSROC curve as a circle and plotted. The overview point was depicted by a dot surrounded by a 95% trust area (95% CI). The area under the curve (AUC) was computed to determine the diagnostic accuracy. Approaches 1.0 to the AUC would mean outstanding results, and impaired performance would be suggested if it approaches 0.5. Among numerous subgroups, we compared the 95% CI of the AUC. We used non-overlapping 95% CI between two subgroups to identify statistically relevant variations. The variability and threshold effects of the studies included were also measured. Generally, the Chi-Square test of p < 0.1 reveals substantial heterogeneity performed was Cochran's Q statistics and I2 test. Spearman's correlation coefficient with r ≥ 0.6 between sensitivity and FP rate typically suggests a substantial threshold influence. We conducted both statistical studies using version 1.4 of the Meta-DiSc software [19] and the quality and potential bias of all studies by using Review Manager 5.4 (RevMan 5.4) [20].

3. Results

3.1. Study selection and characteristics

Finally, 886 studies were retrieved on the initial search, and 223 duplicates were removed. After reviewing the title, abstract and full article, finally, 36 studies were selected for inclusion into the meta-analysis [[21], [22], [23], [24], [25], [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], [36], [37], [38], [39], [40], [41], [42], [43], [44], [45], [46], [47], [48], [49], [50], [51], [52], [53], [54], [55], [56], [57]] (Fig. 1 ). All included studies were retrospective, and all the studies were based on record images.

Fig. 1.

PRISMA 2009 flow diagram.

Based on the number of enrolled images, 32,857 images (19,623‬ COVID-19 images and 13,234 Healthy images) classified by analysis were included. The AI algorithm based on the neural network was established in a number of research articles [[21], [22], [23],[25], [26], [27],[29], [30], [31],[33], [34], [35], [36], [37],[41], [42], [43],47,48,[50], [51], [52], [53], [54], [55],57]. Among the included studies, twenty-nine models were selected for meta-analysis on DL assisted detection for predict COVID-19 [21,22,[25], [26], [27],30,[33], [34], [35], [36], [37],[40], [41], [42],46,47,[50], [51], [52], [53], [54],56,57] and fourteen models on ML assisted detection for predict COVID-19 [21,24,28,31,38,43,45,46,48,49] (Table 1 ).

Table 1.

Characteristics of included studies on various models in patients with COVID-19.

Country/ID	Country	Expert Radiologists involved as control	AI model	Reference standard	Chest CT images		Diagnosis factors
					Positive	Healthy samples	Accuracy, %	AUROC	PPV	NPV	Sen.	Spec.
Kelei He et al., 2021 [1]	China	Yes	DL	RT-PCR	666	NA	0.985	0.991	0.799	NA	0.783	NA
Ziwei Zhu et al., 2021 [2]	China	Yes	DL	RT-PCR	687	395	0.93	0.93	NA	NA	0.93	0.92
Vruddhi Shah et al., 2021 [3]	India	Yes	DL	RT-PCR	738	NA	0.821	NA	NA	NA	NA	NA
Carlos Quiroz et al., 2021 [4]	Australia	Yes	ML	RT-PCR	346	NA	NA	0.926	NA	NA	0.818	0.901
H Alshazly et al., 2021 [5]	Germany	Yes	DL	RT-PCR	1252	1230	0.994	NA	NA	NA	0.998	0.996
Mohit Agarwal et al., 2021 [6]	India	Yes	DL	RT-PCR	705	990	0.994	0.991	NA	NA	0.99	0.985
			ML				0.994	0.988	NA	NA	0.99	0.985
			DL				0.718	0.714	NA	NA	0.802	0.630
			DL				0.915	0.913	NA	NA	0.938	0.888
			DL				0.859	0.852	NA	NA	0.895	0.810
			DL				0.874	0.871	NA	NA	0.915	0.826
			DL				0.909	0.893	NA	NA	0.937	0.864
			DL				0.87	0.861	NA	NA	0.914	0.815
			ML				0.958	0.948	NA	NA	0.969	0.943
Xi Fang et al., 2021 [7]	USA	Yes	DL	RT-PCR	193	NA	NA	0.813	NA	NA	NA	NA
Kumar Mishra et al., 2020 [8]	India	Yes	DL	RT-PCR	360	397	0.8834	0.8832	NA	NA	0.8813	0.9051
Jun Chen et al., 2020 [9]	China	Yes	DL	RT-PCR	636	691	0.9524	NA	NA	NA	1	0.9355
Liang Sun et al., 2020 [10]	China	Yes	DL	RT-PCR	1495	1027	0.9179	0.9635	NA	NA	0.9305	0.8995
S Carvalho et al., 2020 [11]	Portugal	Yes	DL	RT-PCR	130	NA	0.82	0.90	NA	NA	0.80	0.86
Lu-Shan Xiao et al., 2020 [12]	China	Yes	DL	RT-PCR	408	NA	0.974	0.987	NA	NA	NA	NA
Kimura-Sandoval et al., 2020 [13]	Mexico	Yes	AI	RT-PCR	166	NA	NA	0.88	NA	NA	0.74	0.91
Hui-Bin Tan et al., 2020 [14]	China	Yes	ML	RT-PCR	NA	NA	NA	0.95	NA	NA	0.987	0.984
Liping Fu et al., 2020 [15]	China	Yes	ML	RT-PCR	64	NA	NA	0.833	NA	NA	0.8095	0.7442
Kang Zhang et al., 2020 [16]	China	Yes	AI	RT-PCR	752	697	.08411	0.9050	NA	NA	0.8667	0.8226
Quan Cai et al., 2020 [17]	China	Yes	ML	RT-PCR	81	122	0.709	0.811	NA	NA	0.765	0.625
D Javor et al., 2020 [18]	Austria	Yes	DL	RT-PCR	3102	NA	NA	0.956	NA	NA	0.844	0.933
Daowei Li et al., 2020 [19]	China	Yes	DL	RT-PCR	10	36	NA	0.68	NA	NA	NA	NA
Hoon Ko et al., 2020 [20]	Korea	Yes	DL	RT-PCR	337	998	0.9987	1	NA	NA	0.9958	1
Xueyan Mei et al., 2020 [21]	USA	Yes	DL	RT-PCR	419	486	0.796	0.86	NA	NA	0.836	0.759
Xinggang Wang et al., 2020 [22]	China	Yes	DL	RT-PCR	313	229	0.901	0.959	NA	NA	0.95	0.95
Xiangjun Wu et al., 2020 [23]	China	Yes	DL	RT-PCR	294	101	0.819	0.76	NA	NA	0.811	0.615
Shuo Wang et al., 2020 [24]	China	Yes	DL	RT-PCR	560	149	0.8124	0.90	NA	NA	0.7893	0.8993
Lin Li et al., 2020 [25]	China	Yes	DL	RT-PCR	1296	1325	NA	0.96	NA	NA	0.90	0.96
A. Harmon et al., 2020 [26]	USA	Yes	AI	RT-PCR	1029	1695	0.908	0.949	NA	NA	0.84	0.93
Chenglong Liu et al., 2020 [27]	China	Yes	ML	RT-PCR	73	27	0.9416	0.99	NA	NA	0.8862	1
Harrison X. Bai et al., 2020 [28]	China	Yes	AI	RT-PCR	521	665	0.96	0.95	NA	NA	0.95	0.96
A. Sakagianni et al., 2020 [29]	Greece	Yes	ML	RT-PCR	349	397	0.932	0.94	NA	NA	0.8831	0.8831
Deepika Selvaraj et al., 2020 [30]	India	Yes	ML	RT-PCR	50	NA	0.886	0.8723	NA	NA	0.5549	0.8988
			ML				0.833	0.9107	NA	NA	0.4025	0.9735
			ML				0.882	0.8187	NA	NA	0.5211	0.8950
			ML				0.93	0.94	NA	NA	0.756	0.9593
			DL				0.938	0.9427	NA	NA	0.7678	0.9285
Yuehua Li et al., 2020 [31]	China	Yes	DL	RT-PCR	148	NA	0.626	0.660	NA	NA	0.5897	0.6429
Fei Shan et al., 2020 [32]	China	Yes	ML	RT-PCR	249	NA	0.916	NA	NA	NA	NA	NA
Minghuan Wang et al., 2020 [33]	China	Yes	DL	RT-PCR	1647	800	NA	0.953	0.790	0.948	0.923	0.851
H–W Ren et al., 2020 [34]	China	Yes	AI	RT-PCR	58	NA	NA	0.740	NA	NA	0.912	0.588
Zhang Li et al., 2020 [35]	China	Yes	DL	RT-PCR	204	164	NA	0.97	NA	NA	NA	NA
Jiantao Pu et al., 2020 [36]	USA	Yes	DL	RT-PCR	151	498	NA	0.70	NA	NA	NA	NA
Fengjun Liu et al., 2020 [37]	USA	Yes	AI	RT-PCR	134	115	NA	0.84	NA	NA	NA	NA

False Positive (FP), False Negative (FN), True Negative (TN), True Positive (TP), Area Under the Curve (AUC), Deep Learning (DL), Machine Learning (ML), convolution neural network (CNN), artificial neural network (ANN), Decision tree (DT), and random forest (RF), artificial neural network (ANN), Tree-based pipeline optimization tool (TPOT), ensemble of bagged tree (EBT), support vector machine (SVM), Gaussian Naive Bayes (GNB), Logistic Regression (LR), Deep Neural Network (DNN),

3.2. Risk of bias within studies

In the final part, 31 studies had a low risk of bias in patient selection, while 5 studies had a high risk of bias (Supplementary Fig. 1). In terms of the patient selection, two studies [21,46] used multiple tests, including (DL, and ML). Overall, studies with high risk [39,44,48,55,58] in at least one of the seven domains were rated as low methodological quality in the subgroup analysis.

4. Diagnostic test accuracy (DTA)

4.1. Results of AI

Among the 37 studies [[21], [22], [23], [24], [25], [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], [36], [37], [38], [39], [40], [41], [42], [43], [44], [45], [46], [47], [48], [49], [50], [51], [52], [53], [54], [55], [56], [57]] of image-based analysis, the pooled sensitivity was 0.90 (95% CI, 0.90–0.91), specificity was 0.90 (95% CI, 0.90–0.91), the AUC was 0.96 (95% CI, 0.91–0.98), and diagnostic odds ratio (DOR) was 88.98 (95% CI, 56.38–140.44) as shown in (Fig. 2 ) (Supplementary Figs. 2–8).

Fig. 2.

The summary receiver-operating characteristic (SROC) curves of the diagnostic performance of AI and CT-Scan on detection. Significant difference was present when the 95% confidence regions.

4.2. Results of DL

Among the 23 studies [21,22,[25], [26], [27],30,31,[33], [34], [35], [36], [37],[40], [41], [42],46,47,[50], [51], [52], [53], [54],56,57] of image-based analysis, the pooled sensitivity was 0.91 (95% CI, 0.90–0.91), specificity was 0.88 (95% CI, 0.87–0.89), the AUC was 0.96 (95% CI, 0.93–0.97), and DOR was 99.04 (95% CI, 54.68–179.36) as shown in (Fig. 3 ) (Supplementary Figs. 3–8).

Fig. 3.

The summary receiver-operating characteristic (SROC) curves of the diagnostic performance of DL and CT-Scan on detection. Significant difference was present when the 95% confidence regions.

4.3. Results of ML

Among the 9 studies [21,24,28,38,43,45,46,48,49] of image-based analysis, the pooled sensitivity was 0.91 (95% CI, 0.90–0.91), specificity was 0.95 (95% CI, 0.94–0.95), the AUC was 0.97 (95% CI, 0.96–0.99), and DOR was 88.27 (95% CI, 29.52–263.96) as shown in (Fig. 4 ) (Supplementary Figs. 4–8).

Fig. 4.

The summary receiver-operating characteristic (SROC) curves of the diagnostic performance of ML and CT-Scan on detection. Significant difference was present when the 95% confidence regions.

5. Discussion

This meta-analysis study exhibited a satisfactory performance using the AI algorithm for AI assisted CT-Scan identification of COVID-19 vs. healthy samples. We showed that AI was accurate on the lung involvement in the COVID-19 with a pooled sensitivity was 0.90 (95% CI, 0.90–0.91), specificity was 0.90 (95% CI, 0.90–0.91) and the AUC was 0.96 (95% CI, 0.91–0.98). According to Table 2 , ResNet-50, ResNet101, ensemble of bagged tree (EBT), Tree-based pipeline optimization tool (TPOT), Gaussian Naive Bayes (GNB), random forest (RF), and convolution neural network (CNN) algorithms had performed good on the CT-based COVID-19 detection.

Table 2.

A detailed information of used AI-models to detect and Classified COVID- 19 by Compressed Chest CT Image.

Country/ID	Method	Input	Output	Algorithm names	Performance evaluation	Training/test splitting	Transfer learning / ab initio training	Network Architecture
Kelei He et al., 2021 [1]	DL	The raw 3D CT image	The lung segmentation and severity assessment of COVID19 patients	multi-task multi-instance U-Net (M2UNet)	A five-fold cross-validation strategy used	One subset as the testing set (20%)/ Four subsets are combined to construct the training set (70%) and validation set (10%)	Synergistic Learning	A bag (consisting of a set of 2D image patches) as the input data. M2UNet employs an encoding module for patch-level feature extraction
Ziwei Zhu et al., 2021 [2]	DL	The raw 3D CT image	The lung segmentation and severity assessment of COVID19 patients	Keras platform based on ResNet50 architecture	training set, validation set and testing set	One subset as the training set, one subset as validation set, and one subset as testing set	Transfer learning to detect the patients with COVID-19	Imagenet dataset, Newly initialized weights, Output
Vruddhi Shah et al., 2021 [3]	DL	The raw 3D CT image	The lung segmentation and severity assessment of COVID19 patients	ResNet-50	The confusion matrix	A training set, validation set, and test set with a split	A pre-trained network	VGG-19 architecture
Carlos Quiroz et al., 2021 [4]	ML	CT slices with <3 mm2 of lung tissue	The lung segmentation and severity assessment of COVID19 patients	EfficientNetB7 U-Net	5-fold repeated stratified cross-validation	-	-	A 4-layer, fully connected architecture
H Alshazly et al., 2021 [5]	DL	Chest CT scans	The lung segmentation and severity assessment of COVID19 patients	ResNet50 and ResNet101	K-fold cross-validation	About 600 images only, and the test fold has less than 200 images	Transfer learning to detect the patients with COVID-19; which data are scarce	The deep CNN architectures
Mohit Agarwal et al., 2021 [6]	DL, ML	Chest CT scans	The lung segmentation and severity assessment of COVID19 patients	CNN, RF, VGG16, DenseNet121, DenseNet169, DenseNet201, MobileNet, ANN, DT	K-fold cross-validation	K10 protocol (90% training and 10% testing)	VGG16, DenseNet121, DenseNet169, DenseNet201 and MobileNet	Based CNN thus has a total of 7 layers mainly adapting for simplicity
Xi Fang et al., 2021 [7]	DL	Chest CT scans	The lung segmentation and severity assessment of COVID19 patients	U-Net	Cross-dataset validation (training on Site A and testing on Site B; training on Site B and testing on Site A)	Labeled all five pulmonary lobes in 71 CT volumes from Site A using chest imaging platform	-	-
Kumar Mishra et al., 2020 [8]	DL	Chest CT scans	The lung segmentation and severity assessment of COVID19 patients	ResNet50	-	Split 80% of the data is kept for training purpose (training data) and the rest for testing (testing data)	-	Indicate the potential usage of various Deep CNN architectures
Jun Chen et al., 2020 [9]	DL	Chest CT scans	The lung segmentation and severity assessment of COVID19 patients	UNet++	-	35,355 images were selected and split into training and retrospectively testing datasets.	-	UNet++ consists of encoder and decoder connecting through a series of nested dense convolutional blocks.
Liang Sun et al., 2020 [10]	DL	Chest CT scans	The lung segmentation and severity assessment of COVID19 patients	VB-Net	-	Adaptive Feature Selection guided Deep Forest (AFS-DF)	-	Selection guided deep forest
S Carvalho et al., 2020 [11]	DL	Chest CT scans	The lung segmentation and severity assessment of COVID19 patients	ANN	Minimization of the cross-entropy	Validation (150 ROIs), and test (150 ROIs)	-	60 neurons in a single-hidden-layer architecture
Lu-Shan Xiao et al., 2020 [12]	DL	Chest CT scans	The lung segmentation and severity assessment of COVID19 patients	ResNet34	Five-fold cross-validation	Patch dataset with a size as large as 3 × 224 × 224 (z × y × x)	-	ResNet34, AlexNet, VGGNet, and DenseNet
Kimura-Sandoval et al., 2020 [13]	AI	Chest CT scans	The lung segmentation and severity assessment of COVID19 patients	Logistic	-	-	-	-
Hui-Bin Tan et al., 2020 [14]	ML	Chest CT scans	The lung segmentation and severity assessment of COVID19 patients	TPOT	Radiomics Auto-ML model in the first CT images	Training set and test set according to the proportion of 8:2	-	Auto-ML, each group's original data is imported into TPOT
Liping Fu et al., 2020 [15]	ML	Chest CT scans	The lung segmentation and severity assessment of COVID19 patients	K(K-1)/2 binary	-	One-leave-out cross-validation	-	-
Kang Zhang et al., 2020 [16]	AI	Chest CT scans	The lung segmentation and severity assessment of COVID19 patients	ResNet-18	A five-fold cross-validation test	Randomly assigned to a training set (80%), an internal validation set (10%) or a test set (10%)	-	A computer-aided diagnosis (CAD) system for detecting COVID-19 patients
Quan Cai et al., 2020 [17]	ML	Chest CT scans	The lung segmentation and severity assessment of COVID19 patients	-	-	7:3 ratio to either the training cohort or the testing cohort	-	-
D Javor et al., 2020 [18]	DL	Chest CT scans	The lung segmentation and severity assessment of COVID19 patients	ResNet50	-	Split for training the model and internal validation (20 % of the samples)	-	More layers (ResNet-101)
Daowei Li et al., 2020 [19]	DL	Chest CT scans	The lung segmentation and severity assessment of COVID19 patients	U-Net	-	-	-	-
Hoon Ko et al., 2020 [20]	DL	Chest CT scans	The lung segmentation and severity assessment of COVID19 patients	ResNet-50	5-fold cross-validation	Randomly split with a ratio of 8:2 into a training set and a testing set	On one of the following four pretrained CNN	Initially used the predefined weights for each CNN architecture
Xueyan Mei et al., 2020 [21]	DL	Chest CT scans	The lung segmentation and severity assessment of COVID19 patients	-	-	-	-	-
Xinggang Wang et al., 2020 [22]	DL	Chest CT scans	The lung segmentation and severity assessment of COVID19 patients	UNet	-	A simple 2D UNet using the CT images in our training set	-	3D deep convolutional neural Network to Detect COVID-19 (DeCoVNet) from CT volumes.
Xiangjun Wu et al., 2020 [23]	DL	Chest CT scans	The lung segmentation and severity assessment of COVID19 patients	ResNet50	The layer outputs the risk value of COVID-19 pneumonia	50 cases (10%, 37 of COVID-19, 13 of other pneumonia) of the validation set and 50 cases (10%, 37 of COVID-19, 13 of other pneumonia) of the testing set.	-	Modification of ResNet50 architecture
Shuo Wang et al., 2020 [24]	DL	Chest CT scans	The lung segmentation and severity assessment of COVID19 patients	COVID-19Net	Train and externally validate the performance	The auxiliary training set	The pre-trained COVID-19Net to the COVID-19 dataset to specifically	-
Lin Li et al., 2020 [25]	DL	Chest CT scans	The lung segmentation and severity assessment of COVID19 patients	COVID-19Net	Using an independent testing set. COVNet = COVID-19 detection neural network.	A ratio of 9:1 into a training set and an independent testing set at the patient level.	-	A supervised deep learning framework (COVNet) was developed to detect COVID-19 and community acquired pneumonia.
A. Harmon et al., 2020 [26]	AI	Chest CT scans	The lung segmentation and severity assessment of COVID19 patients	AH-Net	-	-	-	Densnet-121 architecture adapted to utilize 3D operations (i.e., 3D convolutions) compared to original 2D implementation
Chenglong Liu et al., 2020 [27]	ML	Chest CT scans	The lung segmentation and severity assessment of COVID19 patients	EBT	SVM, LR, DT, KNN are implemented with the same texture feature extraction	-	-	-
Harrison X. Bai et al., 2020 [28]	AI	Chest CT scans	The lung segmentation and severity assessment of COVID19 patients	EfficientNet B4	-	-	-	EfficientNet B4 deep neural network architecture
A. Sakagianni et al., 2020 [29]	ML	Chest CT scans	The lung segmentation and severity assessment of COVID19 patients	-	-	-	-	-
Deepika Selvaraj et al., 2020 [30]	DL, ML	Chest CT scans	The lung segmentation and severity assessment of COVID19 patients	SVM, GNB, LR, DT, DNN	50 images are used for testing the trained network	The dataset of training points is manually selected from the infected and background pixels from the 30 training images	-	The size of the input layer is 38 neurons (38 features), three hidden layers with 58 neurons per layer and binary classification output layer
Yuehua Li et al., 2020 [31]	DL	Chest CT scans	The lung segmentation and severity assessment of COVID19 patients	U-Net	The Dice coefficient	-	-	-
Fei Shan et al., 2020 [32]	ML	Chest CT scans	The lung segmentation and severity assessment of COVID19 patients	VB-Net	-	-	-	-
Minghuan Wang et al., 2020 [33]	DL	Chest CT scans	The lung segmentation and severity assessment of COVID19 patients	U-Net	-	Randomly split into a training set (1318 patients with COVID-19; 640 patients without COVID-19) and a testing set (329 patients with COVID-19; 160 patients without COVID-19)	-	-
H–W Ren et al., 2020 [34]	AI	Chest CT scans	The lung segmentation and severity assessment of COVID19 patients	-	-	-	-	-
Zhang Li et al., 2020 [35]	DL	Chest CT scans	The lung segmentation and severity assessment of COVID19 patients	U-Net	-	-	-	-
Jiantao Pu et al., 2020 [36]	DL	3D Chest CT scans	The lung segmentation and severity assessment of COVID19 patients	CNN	-	-	-	The CNN architectures used different numbers of filters at different layers.
Fengjun Liu et al., 2020 [37]	AI	Chest CT scans	The lung segmentation and severity assessment of COVID19 patients	-	-	-	-	-

False Positive (FP), False Negative (FN), True Negative (TN), True Positive (TP), Area Under the Curve (AUC), Deep Learning (DL), Machine Learning (ML), convolution neural network (CNN), artificial neural network (ANN), Decision tree (DT), and random forest (RF), artificial neural network (ANN), Tree-based pipeline optimization tool (TPOT), ensemble of bagged tree (EBT), support vector machine (SVM), Gaussian Naive Bayes (GNB), Logistic Regression (LR), Deep Neural Network (DNN),

The lesions could explain AI's excellent performance in detecting COVID-19 with the handle, bronchial vascularization, or lower extremities in bilateral lungs [59]. In contrast, AUC of ML detecting COVID-19 patients was 0.97 (95% CI, 0.96–0.99). However, the AUC of DL on detecting of COVID-19 patients was 0.96 (95% CI, 0.93–0.97). Thus, it may increase the AI, ML, and DL models' close diagnosis to detect COVID-19.

The AI system demonstrated performance comparable to senior practicing radiologists and can help to diagnose COVID-19 patients rapidly with 0.97 and 0.95 AUC [23,55]. Consequently, AI software expressing objective evaluations of the percentage of ventilated lung parenchyma compared to the affected one and can readily identify CT-scans with COVID-19 associated pneumonia [58,60]. Ilker Ozsahin et al., 2020, in the review study, showed that AI to be used in the clinic as a supportive system for physicians in detecting COVID-19 [61]. Also, pooled AUC in this study was 0.96 (95% CI, 0.91–0.98).

Lin Li et al., 2020, showed that the DL model with 0.96 AUC could accurately detect COVID-19 and differentiate it from Community-Acquired Pneumonia (CAP) and other lung conditions [35]. In contrast, Xiangjun Wu et al., 2020, Xueyan Mei et al., 2020, and Shuo Wang et al., 2020, showed that DL model with 0.732, 0.86, and 0.87 AUC could accurately detect COVID-19, respectively [51,53,62]. However, one study was showed that chest CT-Scan with AI could not replace molecular diagnostic tests with a nasopharyngeal swab (RT-PCR) or suspected for COVID-19 patients [63]. Overall, analysis shows that the DL model can classify the chest CT-Scan at a high accuracy rate and AUC values ranging from 0.90 to 1.00 [33,52,64,65]. At the same time, this study showed that the AUC of DL on detecting COVID-19 patients was 0.96 (95% CI, 0.93–0.97), which was near the same results with the research studies.

Daowei Li et al., 2020, showed that the AUR score of ML was 0.93 [34]. However, in our study, pooled AUC in ML was higher, 0.97 (95% CI, 0.96–0.99). Overall, ML's accuracy is almost achieved over 0.90 for COVID-19 classification [66], and Chenglong Liu et al., 2020, showed that AUC was 0.99 [38].

This meta-analysis has several limitations. 1. All studies were retrospective based on static images. 2. The selection bias of studies cannot be eliminated (shown in the QUADAS-2). 3. There were some heterogeneities in the CT-Scans equipment, images, and algorithm of AI, DL, and ML used. 4. Also, two studies used some algorithms and methods for AI, which was effect bias for this analysis.

6. Conclusion

Our findings revealed that AI-platforms based on the ResNet-50, ResNet101, an ensemble of the bagged tree, Tree-based pipeline optimization tool, Gaussian Naive Bayes, random forest, and convolution neural network algorithms perform well for CT-based COVID-19 detection. To confirm AI's role for rapid and accurate COVID-19 diagnosis, more prospective real-time trials are required due to reduce the possibility of selection bias and to compare with currently available studies.

Funding source, financial disclosures

Not have funding.

Contribute

Study concept and design: F.R, K.SH. Acquisition of data: F.R, K.SH. Analysis and interpretation of data. F.R. Drafting of the manuscript: K.SH, B.A. Critical review of manuscript: F.R, K.SH, H.K.SH.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Appendix A. Supplementary data

The following is the Supplementary data to this article: