Skip to main content
The Cochrane Database of Systematic Reviews logoLink to The Cochrane Database of Systematic Reviews
. 2022 May 16;2022(5):CD013639. doi: 10.1002/14651858.CD013639.pub5

Thoracic imaging tests for the diagnosis of COVID‐19

Sanam Ebrahimzadeh 1, Nayaar Islam 1,2, Haben Dawit 1,2, Jean-Paul Salameh 2, Sakib Kazi 2, Nicholas Fabiano 2, Lee Treanor 2, Marissa Absi 2, Faraz Ahmad 2, Paul Rooprai 2, Ahmed Al Khalil 2, Kelly Harper 2, Neil Kamra 2, Mariska MG Leeflang 3, Lotty Hooft 4, Christian B Pol 5, Ross Prager 6, Samanjit S Hare 7, Carole Dennie 2,8, René Spijker 4,9, Jonathan J Deeks 10,11, Jacqueline Dinnes 10,11, Kevin Jenniskens 12, Daniël A Korevaar 13, Jérémie F Cohen 14, Ann Van den Bruel 15, Yemisi Takwoingi 10,11, Janneke de Wijgert 12,16, Junfeng Wang 17, Elena Pena 2,8, Sandra Sabongui 18, Matthew DF McInnes 1,2,; Cochrane COVID-19 Diagnostic Test Accuracy Group 11
Editor: Cochrane Infectious Diseases Group
PMCID: PMC9109458  PMID: 35575286

Abstract

Background

Our March 2021 edition of this review showed thoracic imaging computed tomography (CT) to be sensitive and moderately specific in diagnosing COVID‐19 pneumonia. This new edition is an update of the review.

Objectives

Our objectives were to evaluate the diagnostic accuracy of thoracic imaging in people with suspected COVID‐19; assess the rate of positive imaging in people who had an initial reverse transcriptase polymerase chain reaction (RT‐PCR) negative result and a positive RT‐PCR result on follow‐up; and evaluate the accuracy of thoracic imaging for screening COVID‐19 in asymptomatic individuals. The secondary objective was to assess threshold effects of index test positivity on accuracy.

Search methods

We searched the COVID‐19 Living Evidence Database from the University of Bern, the Cochrane COVID‐19 Study Register, The Stephen B. Thacker CDC Library, and repositories of COVID‐19 publications through to 17 February 2021. We did not apply any language restrictions.

Selection criteria

We included diagnostic accuracy studies of all designs, except for case‐control, that recruited participants of any age group suspected to have COVID‐19. Studies had to assess chest CT, chest X‐ray, or ultrasound of the lungs for the diagnosis of COVID‐19, use a reference standard that included RT‐PCR, and report estimates of test accuracy or provide data from which we could compute estimates. We excluded studies that used imaging as part of the reference standard and studies that excluded participants with normal index test results.

Data collection and analysis

The review authors independently and in duplicate screened articles, extracted data and assessed risk of bias and applicability concerns using QUADAS‐2. We presented sensitivity and specificity per study on paired forest plots, and summarized pooled estimates in tables. We used a bivariate meta‐analysis model where appropriate.

Main results

We included 98 studies in this review. Of these, 94 were included for evaluating the diagnostic accuracy of thoracic imaging in the evaluation of people with suspected COVID‐19. Eight studies were included for assessing the rate of positive imaging in individuals with initial RT‐PCR negative results and positive RT‐PCR results on follow‐up, and 10 studies were included for evaluating the accuracy of thoracic imaging for imagining asymptomatic individuals.

For all 98 included studies, risk of bias was high or unclear in 52 (53%) studies with respect to participant selection, in 64 (65%) studies with respect to reference standard, in 46 (47%) studies with respect to index test, and in 48 (49%) studies with respect to flow and timing. Concerns about the applicability of the evidence to: participants were high or unclear in eight (8%) studies; index test were high or unclear in seven (7%) studies; and reference standard were high or unclear in seven (7%) studies.

Imaging in people with suspected COVID‐19

We included 94 studies. Eighty‐seven studies evaluated one imaging modality, and seven studies evaluated two imaging modalities. All studies used RT‐PCR alone or in combination with other criteria (for example, clinical signs and symptoms, positive contacts) as the reference standard for the diagnosis of COVID‐19.

For chest CT (69 studies, 28285 participants, 14,342 (51%) cases), sensitivities ranged from 45% to 100%, and specificities from 10% to 99%. The pooled sensitivity of chest CT was 86.9% (95% confidence interval (CI) 83.6 to 89.6), and pooled specificity was 78.3% (95% CI 73.7 to 82.3). Definition for index test positivity was a source of heterogeneity for sensitivity, but not specificity. Reference standard was not a source of heterogeneity.

For chest X‐ray (17 studies, 8529 participants, 5303 (62%) cases), the sensitivity ranged from 44% to 94% and specificity from 24 to 93%. The pooled sensitivity of chest X‐ray was 73.1% (95% CI 64.1 to 80.5), and pooled specificity was 73.3% (95% CI 61.9 to 82.2). Definition for index test positivity was not found to be a source of heterogeneity. Definition for index test positivity and reference standard were not found to be sources of heterogeneity.

For ultrasound of the lungs (15 studies, 2410 participants, 1158 (48%) cases), the sensitivity ranged from 73% to 94% and the specificity ranged from 21% to 98%. The pooled sensitivity of ultrasound was 88.9% (95% CI 84.9 to 92.0), and the pooled specificity was 72.2% (95% CI 58.8 to 82.5). Definition for index test positivity and reference standard were not found to be sources of heterogeneity.

Indirect comparisons of modalities evaluated across all 94 studies indicated that chest CT and ultrasound gave higher sensitivity estimates than X‐ray (P = 0.0003 and P = 0.001, respectively). Chest CT and ultrasound gave similar sensitivities (P = 0.42). All modalities had similar specificities (CT versus X‐ray P = 0.36; CT versus ultrasound P = 0.32; X‐ray versus ultrasound P = 0.89).

Imaging in PCR‐negative people who subsequently became positive

For rate of positive imaging in individuals with initial RT‐PCR negative results, we included 8 studies (7 CT, 1 ultrasound) with a total of 198 participants suspected of having COVID‐19, all of whom had a final diagnosis of COVID‐19. Most studies (7/8) evaluated CT. Of 177 participants with initially negative RT‐PCR who had positive RT‐PCR results on follow‐up testing, 75.8% (95% CI 45.3 to 92.2) had positive CT findings.

Imaging in asymptomatic PCR‐positive people

For imaging asymptomatic individuals, we included 10 studies (7 CT, 1 X‐ray, 2 ultrasound) with a total of 3548 asymptomatic participants, of whom 364 (10%) had a final diagnosis of COVID‐19. For chest CT (7 studies, 3134 participants, 315 (10%) cases), the pooled sensitivity was 55.7% (95% CI 35.4 to 74.3) and the pooled specificity was 91.1% (95% CI 82.6 to 95.7).

Authors' conclusions

Chest CT and ultrasound of the lungs are sensitive and moderately specific in diagnosing COVID‐19. Chest X‐ray is moderately sensitive and moderately specific in diagnosing COVID‐19. Thus, chest CT and ultrasound may have more utility for ruling out COVID‐19 than for differentiating SARS‐CoV‐2 infection from other causes of respiratory illness. The uncertainty resulting from high or unclear risk of bias and the heterogeneity of included studies limit our ability to confidently draw conclusions based on our results.

Keywords: Humans; COVID-19; COVID-19/diagnostic imaging; SARS-CoV-2; Sensitivity and Specificity; Tomography, X-Ray Computed; Ultrasonography

Plain language summary

How accurate is chest imaging for diagnosing COVID‐19?

Why is this question important?

People with suspected COVID‐19 need to know quickly whether they are infected, so they can receive appropriate treatment, self‐isolate, and inform close contacts.

Currently, a formal diagnosis of COVID‐19 requires a laboratory test (RT‐PCR) of nose and throat samples. RT‐PCR requires specialist equipment and takes at least 24 hours to produce a result. It is not completely accurate, and may require a second RT‐PCR or a different test to confirm diagnosis.

Clinicians may use chest imaging to diagnose people who have COVID‐19 symptoms, while awaiting RT‐PCR results or when RT‐PCR results are negative, and the person has COVID‐19 symptoms.

This is the fourth version of this review.

What did we want to find out?

We wanted to know whether chest imaging is accurate enough to diagnose COVID‐19 in people with suspected infection; we included studies in people with suspected COVID‐19 only and excluded studies in people with confirmed COVID‐19. We also wanted to assess the accuracy of chest imaging for screening asymptomatic people.

The evidence is up to date to 17 February 2021.

What are chest imaging tests?

X‐rays or scans produce an image of the organs and structures in the chest.

‐ X‐rays (radiography) use radiation to produce a 2‐D image. Usually done in hospitals, using fixed equipment by a radiographer; they can also be done on portable machines.

‐ Computed tomography (CT) scans use a computer to merge 2‐D X‐ray images and convert them to a 3‐D image. They require highly‐specialized equipment and are done in hospital by a specialist radiographer.

‐ Ultrasound scans use high‐frequency sound waves to produce an image. They can be done in hospitals or other healthcare settings, such as a doctor’s office.

What did we do?

We searched for studies that assessed the accuracy of chest imaging to diagnose COVID‐19 in people of any age with suspected COVID‐19. We included studies with ‘symptomatic' or 'mixed populations'.

What did we find?

We found 94 studies with 37,631 participants (of whom 19,768 (53%) had a final diagnosis of COVID‐19) for evaluating the diagnostic accuracy of thoracic imaging in the evaluation of people with suspected COVID‐19. Eighty‐seven studies evaluated one imaging modality, and seven studies evaluated two imaging modalities. All 94 studies used RT‐PCR either alone or in combination with other criteria (such as clinical signs and symptoms, or positive contacts) as the reference standard for the diagnosis of COVID‐19.

Chest CT: suspected people

Pooled results showed that chest CT (69 studies) correctly diagnosed COVID‐19 in 87% of people who had COVID‐19. However, it incorrectly identified COVID‐19 in 21% of people who did not have COVID‐19.

Chest X‐ray: suspected people

Pooled results showed that chest X‐ray (17 studies) correctly diagnosed COVID‐19 in 73 % of people who had COVID‐19. However, it incorrectly identified COVID‐19 in 27% of people who did not have COVID‐19.

Lung ultrasound: suspected people

Pooled results showed that lung ultrasound (15 studies) correctly diagnosed COVID‐19 in 87% of people with COVID‐19. However, it incorrectly diagnosed COVID‐19 in 24% of people who did not have COVID‐19.

Screening asymptomatic people

We included 10 studies (7 CT, 1 X‐ray, 2 ultrasound) with 3548 asymptomatic participants, of whom 364 (10%) had a final diagnosis of COVID‐19. Pooled results of seven studies, showed that CT correctly diagnosed COVID‐19 in 56% of people who had COVID‐19, and incorrectly identified COVID‐19 in 8% of people who did not have COVID‐19.

How reliable are the results?

The studies differed from each other and used different methods to report their results. Very few studies directly compared one type of imaging test with another. Also, the risk of bias was high or unclear in about half of all included studies. Therefore, it is difficult to draw confident conclusions.

What does this mean?

The evidence suggests that chest CT and ultrasound are better at ruling out COVID‐19 infection than distinguishing it from other respiratory problems. So, their usefulness may be limited to excluding COVID‐19 infection rather than differentiating it from other causes of lung infection. In addition, chest CT imaging had poor sensitivity and high specificity for detecting asymptomatic individuals.

Summary of findings

Summary of findings 1. Summary of findings table 1.

Question What is the diagnostic accuracy of chest imaging (computed tomography (CT), chest X‐ray and ultrasound) in the evaluation of people suspected of having COVID‐19?
Population Children or adults suspected of having COVID‐19
Index test Chest imaging tests used for the diagnosis of COVID‐19, including:
  • chest CT;

  • chest X‐rays;

  • ultrasound of the lungs.

Target condition COVID‐19, the illness following acute infection with SARS‐CoV‐2
Reference standard A positive diagnosis for COVID‐19 by one or a combination of the following.
  • A positive RT‐PCR test for SARS‐CoV‐2 infection, from any manufacturer in any country, from any source, including nasopharyngeal swabs or aspirates, oropharyngeal swabs, bronchoalveolar lavage fluid (BALF), sputum, saliva, serum, urine, rectal or faecal samples.

  • Positive on WHO criteria for COVID‐19 which includes some testing RT‐PCR negative.

  • Positive on China CDC criteria for COVID‐19 which includes some testing RT‐PCR negative.

  • Positive serology in addition to consistent symptomatology.

  • Positive on study specific list of criteria for COVID‐19 which includes some testing RT‐PCR negative.

  • Other criteria (symptoms, other tests, infected contacts).


A negative diagnosis for COVID‐19 by one or a combination of the following.
  • People with suspected COVID‐19 with negative RT‐PCR test results, whether tested once or more than once.

  • Currently healthy or with another disease (no RT‐PCR test)

Limitations in the evidence
Risk of bias
  • Participant selection: high in 10 (10%) studies and unclear in 42 (42%) studies

  • Application of index tests – chest CT: high in 6/73 (8%) studies and unclear in 27/73 (36%) studies

  • Application of index tests – chest X‐ray: unclear in 7/17 (41%) studies

  • Application of index tests – ultrasound of the lungs: unclear in 6/16 (37.5%) studies

  • Reference standard: high in 25 (26%) studies and unclear in 39 (39%) studies

  • Flow and timing: high in 9 (9%) studies and unclear in 39 (41%) studies

  • Repeat RT‐PCR testing objective: participant selection was high in 2/8 (25%) and unclear in 6/8 (75%) studies.

Concerns about applicability of the evidence
  • Participants: high in 3 (3%) and unclear in 5(5%) studies

  • Index test – chest CT: high in 1/73 (1.4%) and unclear in 2/73 (2.7%) studies

  • Index test – chest X‐ray: high in 2/17 (12%) and unclear in 1/17(5.9%)

  • Index test – ultrasound of the lungs: unclear in 1/16 (6%) study

  • Reference standard: high in 2 (2%) and unclear in 5(5%) studies

Findings
  • We included 94 studies for primary objective (37,631 participants suspected of COVID‐19, 19,768 (53%) cases).

  • Most studies (n = 69) evaluated the accuracy of chest CT scans. Chest X‐ray was evaluated in 17 studies and ultrasound of the lungs was evaluated in 15 studies.

  • Chest CT was sensitive and moderately specific in the diagnosis of COVID‐19 in suspected cases.

  • Chest X‐ray was moderately sensitive and moderately specific in the diagnosis of COVID‐19 in suspected cases.

  • Ultrasound of the lungs was sensitive and moderately specific in the diagnosis of COVID‐19 in suspected cases.

  • There was no statistical evidence indicating that reference standard conduct was a source of heterogeneity for chest CT studies. The definition used for index test positivity in chest CT studies appeared to impact sensitivity, as studies that used radiologists' impressions showed higher sensitivities than those that used formal scoring systems.However, the definition of index test positivity was not found to be a source of heterogeneity for chest CT specificity, chest X‐ray accuracy or ultrasound accuracy.

  • The ‘threshold’ effect in chest CT studies that used the CO‐RADS scoring system, or the RSNA scoring system demonstrated a trade‐off between sensitivity and specificity; as the threshold for index test positivity increased, sensitivity decreased, and specificity increased.

  • Indirect test comparisons showed that chest CT (69 studies) and ultrasound (15 studies) both gave higher sensitivity estimates than chest X‐ray (17 studies). Chest CT and ultrasound gave similar sensitivities. All modalities had similar specificities.

  • The rate of positive CT imaging in repeat RT‐PCR positive results (where initial RT‐PCR was negative), was 75.8% (95% CI 45.3 to 92.2).

  • Chest CT imaging had poor sensitivity and high specificity for detecting asymptomatic individuals.

Quantity of evidence for participants suspected of having COVID‐19
Imaging modality Sensitivity (95% CI) Specificity (95% CI) Number of participants (cases)
Chest CT 86.9% (83.6 to 89.6) 78.3% (73.7 to 82.3) 28,285 (14,342)
Chest X‐ray 73.1% (64.1 to 80.5) 73.3% (61.9 to 82.2) 8529 (5303)
Ultrasound of the lungs 88.9% (84.9 to 92.0) 72.2% (58.8 to 82.5) 2410 (1158)
Predicted outcomes
Given various prevalence settings, predicted outcomes for the number of individuals receiving a false positive result or a false negative (missed) result per 1000 people undergoing chest CT, chest X‐ray, and ultrasound of the lungs are outlined as follows.
Predicted outcomes per 1000 people undergoing chest CT
Prevalence of COVID‐19 True positive CT result, n (95% CI) False positive CT result, n (95% CI) True negative CT result, n (95% CI) False negative CT result, n (95% CI)
50% 435 (418 to 448) 109 (89 to 132) 392 (368 to 411) 65 (52 to 82)
20% 174 (167 to 179) 174 (142 to 210) 626 (590 to 658) 26 (21 to 33)
5% 43 (42 to 45) 206 (168 to 250) 744 (700 to 782) 7 (5 to 8)
Predicted outcomes per 1000 people undergoing chest X‐ray
Prevalence of COVID19 Positive CT result
n (95% CI)
False positive CT result
n (95% CI)
Negative CT result
n (95% CI)
False negative CT result
n (95% CI)
50% 366 (321 to 403) 133 (89 to 190) 367 (310 to 411) 134 (97 to 179)
20% 146 (128 to 161) 214 (142 to 305) 586 (495 to 658) 54 (39 to 72)
5% 37 (32 to 40) 254 (169 to 362) 696 (588 to 781) 13 (10 to 18)
Predicted outcomes per 1000 people undergoing ultrasound of the lungs
Prevalence of COVID19 Positive CT result
n (95% CI)
False positive CT result
n (95% CI)
Negative CT result
n (95% CI)
False negative CT result
n (95% CI)
50% 434 (397 to 459) 118 (66 to 194) 382 (306 to 434) 66 (41 to 103)
20% 174 (159 to 184) 190 (106 to 310) 610 (490 to 694) 26 (16 to 41)
5% 43 (40 to 46) 225 (126 to 369) 725 (581 to 824) 7 (4 to 10)

Abbreviations: CI: confidence interval; CT: computed tomography; n: number; RT‐PCR: reverse transcription polymerase chain reaction.

Background

The severe acute respiratory syndrome coronavirus 2 (SARS‐CoV‐2) infection and resulting coronavirus disease 2019 (COVID‐19) pandemic continue to present diagnostic evaluation challenges. While the World Health Organization (WHO) reports laboratory confirmation of COVID‐19 infection, such as a positive reverse transcriptase polymerase chain reaction (RT‐PCR) result as the standard for diagnosing COVID‐19, the value of imaging tests in the diagnostic pathway remains undefined (WHO 2020). Research on the role of imaging in COVID‐19 patients is evolving and more refined assessment methods for imaging tests, such as the COVID‐19 Reporting and Data System (CO‐RADS), are being investigated (Prokop 2020). Also, asymptomatic transmission of COVID‐19 is one of its biggest diagnostics challenges, with the WHO recently reminding the public of the distinction between asymptomatic patients and presymptomatic patients (Walker 2020). The role of imaging in the screening of asymptomatic patients remains undefined.

Decisions about patient and isolation pathways for COVID‐19 vary according to health services and settings, available resources, and outbreaks in different settings. They will change over time, as accurate tests, effective treatments, and vaccines are identified. The decision points between these pathways vary, but all include points at which knowledge of the accuracy of diagnostic information is needed to inform medical decisions. Therefore, it is essential to understand the accuracy of tests and diagnostic features to develop effective diagnostic and management pathways for different settings. This supports strategies aiming to identify those who are infected, and consequently the management of patients either through isolation precautions, contact tracing, quarantine, hospital admission or admission to a specialized facility, admission to the intensive care unit, or initiation of specific therapies, and implementation of mitigation strategies to limit the spread of the disease.

This review from the suite of Cochrane ‘living systematic reviews’ summarizes evidence on the accuracy of different imaging tests and diagnostic features in participants regardless of their symptoms. Estimates of accuracy from this review will help inform diagnostic, screening, isolation, and patient‐management decisions. We have included an explanation of terminology and acronyms in Appendix 1.

Target condition being diagnosed

The target condition being evaluated is COVID‐19, the illness following acute infection with SARS‐CoV‐2 (Datta 2020). People Infected with SARS‐CoV‐2 can be asymptomatic and can have a wide variety of symptoms, including fever, sore throat, diarrhoea, dyspnoea, headache, chest pain, stomach‐ache, nausea, loss of taste, loss of smell, myalgia (muscle pain), fatigue, runny nose, cough, aches, and lethargy (either without difficulty breathing at rest or with shortness of breath and increased respiratory rate potentially requiring supplemental oxygen or mechanical ventilation). Furthermore, in people diagnosed with a pulmonary condition (e.g. pulmonary embolism), symptoms could be indicative of COVID‐19, or could be a manifestation of the pre‐existing condition.

Index test(s)

Chest computed tomography (CT)

Chest CT refers to the acquisition of images of the chest using computed tomography. Typical imaging protocols would not use intravenous (IV) contrast; however, in this review we considered all variations of imaging protocols with the exception of studies specifically targeted at evaluating the coronary arteries or the heart, which did not include the entire lungs in the field of view. This includes, but is not limited to, non‐contrast chest CT, low‐dose chest CT (with or without contrast), high‐resolution chest CT, and chest CT with IV contrast (routine or pulmonary angiogram).

Chest radiographs/chest X‐rays

Chest radiography refers to the evaluation of the lungs using X‐rays. This often involves two orthogonal views, posterior‐anterior (PA) and lateral, but may be done by a portable machine and only acquire an anterior‐posterior (AP) view. In this review, we considered any and all variations of chest radiography protocols that evaluated the lungs. We did not include protocols that did not include the entire thorax and were done for reasons other than for assessment of pulmonary status (e.g. assessment of feeding tube position, which typically only includes the lower thorax, or dedicated evaluation of the ribs).

Ultrasound of the lungs

Ultrasound of the lungs refers to any ultrasound of the thorax done with the intention of evaluating the status of the lungs. This includes, but is not limited to, point‐of‐care ultrasound, done at the bedside by a physician, as well as what is often termed consultative’ ultrasound, which is done by a technologist and subsequently interpreted by a physician (typically a radiologist).

We considered all possible technical parameters (e.g. type of probe, transducer frequency, use of contrast). This did not include ultrasound done with the intended purpose of evaluating only the heart or vessels of the chest.

Clinical pathway

The optimal diagnostic pathway and the role of thoracic imaging for identifying people with COVID‐19 is unclear. Compared to RT‐PCR testing, a potential major advantage of thoracic imaging is that results are available faster and that it provides a better insight into the status of the lungs. However, chest CT imaging is typically only available in secondary and tertiary healthcare settings, and availability varies across these settings.

Role of index test(s)

  1. Thoracic imaging may play an integral role in ‘ruling out’ COVID‐19 pneumonia when RT‐PCR is unavailable, pending or negative, or when clinical suspicion is 'low' based on other signs, symptoms and routine laboratory tests. Role of test: triage for RT‐PCR, to make decisions about performing additional tests such as RT‐PCR.

  2. Thoracic imaging is used to rule in or rule out COVID‐19 when results from other tests (e.g. RT‐PCR) are not available in a timely manner.

  3. Concurrent/combination testing with other diagnostic tests (as part of a pair or group of tests) to improve diagnostic accuracy. For example, thoracic imaging could be used to identify false negatives of other tests (e.g. RT‐PCR), and to improve the overall accuracy of the testing strategy.

  4. Thoracic imaging used to detect COVID‐19 in asymptomatic patients.

Several diagnostic pathways have been proposed that provide guidance for physicians to identify people with COVID‐19. The order and components of these pathways differ with varying dependence on pre‐test probability, physical examination, laboratory tests and findings based on RT‐PCR results and availability. However, some professional organizations recommend imaging for patients with moderate or severe features of COVID‐19 (Rubin 2020). In some hospitals, the results of low‐dose chest CT are one of the many parameters (among molecular test results, routine laboratory results and clinical signs and symptoms) used to categorize patients as low risk, moderate to high risk, and proven COVID‐19 cases (China National Health Comission 2020).

Given the rapid progression of COVID‐19 and the constantly evolving evidence base, the diagnostic accuracy to inform the utility of thoracic imaging in these pathways is difficult to estimate. This ‘living systematic review' aims to identify and summarize evidence regarding the diagnostic accuracy of thoracic imaging in people with suspected COVID‐19. This represents our fourth version of this ‘living systematic review' (Islam 2021).

Alternative test(s)

Other Cochrane diagnostic test accuracy (DTA) reviews in the suite of reviews address the following tests.

  1. Signs and symptoms, which will be mainly used in primary care, including when presenting at the emergency department (Struyf 2020).

  2. Routine laboratory testing, such as for C‐reactive protein (CRP) and procalcitonin (PCT) (Stegeman 2020).

  3. Antibody tests (Deeks 2020).

  4. Laboratory‐independent point‐of‐care and near‐patient molecular and antigen tests (Dinnes 2020; Dinnes 2021).

  5. Electronic and animal noses (Leeflang 2021).

Summary of previous versions of the review

In Salameh 2020a, studies that only included confirmed cases of COVID‐19 reported high pooled sensitivities for chest CT and X‐ray: 93.1% (95% CI 90.2 to 95.0) and 82.1% (95% CI 62.5 to 92.7), respectively (Salameh 2020a). Thirteen studies that assessed chest CT in participants with suspected COVID‐19 demonstrated sensitivity of 86.2% (95% CI 71.9 to 93.8) but a low specificity of 18.1% (95% CI 3.71 to 55.8). This indicated a lack of discrimination, as the chances of getting a positive chest CT result are 86% in patients with a SARS‐CoV‐2 infection and 82% in patients without. We did not evaluate accuracy estimates for chest X‐ray and ultrasound of the lungs in participants with suspected COVID‐19 in the initial review as these data were not available.

Islam 2020 focused on people suspected of having COVID‐19 and excluded studies evaluating only confirmed cases of COVID‐19 (Islam 2020). Thirty‐one studies that evaluated chest CT in suspected participants demonstrated a pooled sensitivity of 89.9% (95% CI 85.7 to 92.9) and a pooled specificity of 61.1% (95% CI 42.3 to 77.1). We were not able to evaluate pooled accuracy estimates for chest X‐ray and ultrasound of the lungs in participants with suspected COVID‐19 due to limited data. We explored the value of formal scoring systems for the evaluation of index tests, and ‘threshold’ effects of index test positivity, however, we could not perform formal analyses due to the limited number of included studies.

Compared to Islam 2020, Islam 2021 had stricter inclusion criteria, excluding studies of case‐control design and those that reported an overview of index test findings without explicitly classifying the imaging test as either COVID‐19 positive or negative. Forty‐one studies evaluated chest CT in suspected participants, nine studies evaluated X‐ray and five studies evaluated ultrasound of the lungs in suspected participants. The pooled sensitivity of chest CT was 87.9% (95% CI 84.6 to 90.6) and the pooled specificity was 80.0% (95% CI 74.9 to 84.3). The pooled sensitivity of chest X‐ray was 80.6% (95% CI 69.1 to 88.6) and the pooled specificity was 71.5% (95% CI 59.8 to 80.8). The pooled sensitivity of ultrasound was 86.4% (95% CI 72.7 to 93.9) and the pooled specificity was 54.6% (95% CI 35.3 to 72.6). Definition of index test positivity and reference standard conduct were not found to impact accuracy of chest CT. Based on an indirect comparison using all included studies, chest CT had a higher specificity than ultrasound.

For this current update (fourth version of the review), we have further refined the inclusion criteria, excluding studies that used imaging as a reference standard and studies that excluded participants with normal index test results. We have also formally assessed the impact of definition of index test positivity on the accuracy of X‐ray and ultrasound, along with chest CT. We also assessed the rate of positive imaging in people who had an initial RT‐PCR negative result and a positive RT‐PCR result on follow‐up, and the accuracy of imaging for screening for COVID‐19 in asymptomatic individuals.

We do not have immediate future plans for this 'living systematic review'. Updates to the review and modifications to the protocol are made after discussion with many stakeholders including the author team, the Cochrane DTA COVID group, and the Cochrane Infectious Diseases Group (CIDG).

Changes in the evidence base since previous versions

Evolving research on imaging tests in COVID‐19 patients includes the use of formal scoring systems to evaluate imaging tests, which offer the potential for improved specificity. Formal scoring systems include CO‐RADS (Prokop 2020), the British Society of Thoracic Imaging (BSTI) COVID‐19 Reporting Templates (BSTI 2020), and the Radiological Society of North America (RSNA) Expert Consensus on Reporting Chest CT Findings for COVID‐19 (Simpson 2020). In Islam 2020, we explored the value of formal scoring systems, but we could not formally analyze them due to a limited number of studies that used these systems. In Islam 2021 we evaluated the value of formal scoring systems on accuracy estimates of imaging tests (Irwig 1995) and threshold effects of the CO‐RADS scoring system for chest CT studies. Since Islam 2021, more studies with comparative designs that compare different imaging modalities are available, as well as more studies that evaluate the rate of positive imaging in those with initial RT‐PCR negative results and positive RT‐PCR results on follow‐up, and the accuracy of imaging for screening asymptomatic individuals.

Objectives

The primary objectives are 1) to evaluate the diagnostic accuracy of thoracic imaging (computed tomography (CT), chest X‐ray and ultrasound) in the evaluation of people with suspected COVID‐19, 2) to assess the rate of positive imaging in individuals with initial RT‐PCR negative results and positive RT‐PCR results on follow‐up, and 3) to evaluate the accuracy of thoracic imaging for screening asymptomatic individuals. The secondary objective is to evaluate threshold effects of index test positivity on accuracy.

Methods

Criteria for considering studies for this review

Types of studies

We kept the eligibility criteria broad to be able to include all settings and all variations of a test. We included studies of all designs, with the exception of case‐control studies. Studies had to include participants suspected of having the target condition and produce estimates of test accuracy or provide 2x2 data (true positive (TP), true negative (TN), false positive (FP), false negative (FN)), from which we could compute estimates for the primary objective.

Studies with fewer than 10 participants who underwent the index test and reference standard were excluded.

Participants

Our focus was on studies that recruited participants suspected of having COVID‐19 as outlined in the Target condition being diagnosed section. We included studies with ‘symptomatic populations’ or 'mixed populations' (asymptomatic and symptomatic participants). There were no age or gender restrictions. We also included ‘asymptomatic populations’ for the objective on imaging of asymptomatic individuals in this review

To reduce the effect of selection bias, we excluded studies that excluded participants who had normal index test results.

Index tests

The index tests were chest CT, chest X‐ray, or ultrasound of the lungs, meeting the criteria described in the Index test(s) section. The roles of the test could have been a replacement of RT‐PCR, an add‐on test, a triage test, rapid testing, or used concurrently with other diagnostic tests.

We included only index tests interpreted by humans, and not an algorithm (machine learning/artificial intelligence (AI)). We included studies involving interpretation by an algorithm only if they provided data pertaining to diagnostic accuracy of human interpretation.

Definitions of imaging test positivity

Inclusion was limited to ‘diagnostic test accuracy studies’ in which the study authors explicitly indicated that the index test aims to distinguish between patients with and without COVID‐19. Specifically, studies with index test readers either (1) using a radiological scoring system (e.g. CO‐RADS), or (2) explicitly classifying patients as having a positive or negative imaging test were included. Studies that reported an overview of index test findings without explicitly classifying the imaging test as either COVID‐19 positive or negative were excluded.

There has been considerable heterogeneity and changes over time in the definitions used for positive imaging findings. Some groups have used constellations of specific findings (such as multiple peripheral ground‐glass opacities on CT), some have used an approach in which they consider the combined effect of specific findings (a ‘gestalt’ approach), and some have used formal scoring systems, such as CO‐RADS (5 categories Prokop 2020), the BSTI COVID‐19 Reporting Templates (four categories; BSTI 2020), and the RSNA Expert Consensus on Reporting Chest CT Findings for COVID‐19 (four categories; Simpson 2020). As such, we did not limit ourselves to a predefined definition or threshold for positivity. Instead, we extracted the definition for positivity used in each study, and the constellation of imaging features used to inform this definition. This offers an opportunity to determine if the definition of positivity contributes to variability in accuracy.

Target conditions

As explained above, our target condition is COVID‐19. However, we included all studies reporting data on COVID‐19 or COVID‐19 pneumonia that might provide data relevant to our objective.

Reference standards

A positive diagnosis for COVID‐19 by one or a combination of the following:

  1. a positive RT‐PCR test for SARS‐CoV‐2 infection, from any manufacturer in any country, and from any sample type, including nasopharyngeal swabs or aspirates, oropharyngeal swabs, bronchoalveolar lavage fluid, sputum, saliva, serum, urine, rectal or faecal samples;

  2. positive on WHO criteria for COVID‐19;

  3. positive on China CDC criteria for COVID‐19;

  4. positive serology for SARS‐CoV‐2 antibodies in addition to consistent symptomatology;

  5. positive on study‐specific list of criteria for COVID‐19 which includes other criteria (symptoms, other tests, infected contacts).

A negative diagnosis for COVID‐19 by one or a combination of the following:

  1. suspected COVID‐19 with negative RT‐PCR test results, whether tested once or more than once;

  2. currently healthy or with another disease (no RT‐PCR test).

Studies that used imaging as a part of the reference standard were excluded because of a risk of incorporation bias.

We assessed methodological quality based on our judgement of how likely it was that the reference standard definition used in each study would correctly classify individuals as positive or negative for COVID‐19. All reference standards are likely to be imperfect in some way; details of reference standard evaluation are provided in Appendix 2. We used a consensus process to agree on the classification of the reference standard as to what we regarded as good, moderate and poor. 'Good' reference standards need to have very little chance of misclassification; 'moderate', a small but acceptable risk; and 'poor', a larger and probably unacceptable risk.

Search methods for identification of studies

Electronic searches

We used three different sources for our electronic searches through 17 February 2021, which were devised with the help of an experienced Cochrane Information Specialist with DTA expertise (RSp). These searches aimed to identify all articles related to COVID‐19 and SARS‐CoV‐2 and were not restricted to those evaluating imaging tests. Thus, the searches used no terms that specifically focused on an index test, diagnostic accuracy or study methodology.

Due to the increased volume of published and preprint articles, we used artificial intelligence text analysis from 25 May 2020 and onwards to conduct an initial classification of documents, based on their title and abstract information, for relevant and irrelevant documents. See Appendix 3.

1. Living search from the University of Bern

We used the COVID‐19 living search results of the Institute of Social and Preventive Medicine (ISPM) at the University of Bern. This search includes PubMed, Embase and preprints indexed in bioRxiv and medRxiv databases. The strategies as described on the ISPM website (ispmbern.github.io/covid-19), are shown in Appendix 4.

2. Cochrane COVID‐19 Study Register searches

We also included searches undertaken by Cochrane to develop the Cochrane COVID-19 Study Register. These include searches of trials registers at ClinicalTrials.gov and the World Health Organization International Clinical Trials Registry Platform (WHO ICTRP), as well as PubMed (see Appendix 4 for details). Search strategies were designed for maximum sensitivity, to retrieve all human studies on COVID‐19. We did not apply any language limits.

3. The Stephen B. Thacker CDC Library, COVID‐19 Research Articles Downloadable Database

We included Embase records within the CDC library on COVID-19 research articles database (see Appendix 4 for details) and deduplicated these against the Cochrane COVID‐19 Study Register.

Searching other resources

We checked repositories of COVID‐19 publications against these search results including the following.

  1. EPPI centre eppi.ioe.ac.uk/COVID19_MAP/covid_map_v4.html.

  2. The Norwegian Institute of Public Health 'NIPH systematic and living map on COVID‐19 evidence www.nornesk.no/forskningskart/NIPH_diagnosisMap.html.

  3. From these websites we searched company and product websites for studies about test accuracy.

  4. We contacted companies to ask for further information about studies.

  5. We also contacted research groups that we were made aware of who are completing test evaluations (e.g. UK Public Health England‐funded studies, Foundation for Innovative New Diagnostics (FIND) studies).

Data collection and analysis

Selection of studies

The review authors screened studies independently, in duplicate. A third, experienced review author resolved disagreements about initial title and abstract screening. We resolved disagreements about eligibility assessments through discussion between three review authors.

Data extraction and management

The review authors performed data extraction independently, in duplicate. Three review authors discussed any disagreements to resolve them.

For each study, we extracted 2x2 contingency tables of the number of true positives, false positives, false negatives and true negatives. If a study reported accuracy data for more than one index test reader, we took the average of the data from all readers to compute the average 2x2 contingency table (McGrath 2017). If a study reported accuracy data for both an AI algorithm and one or more radiologists, we extracted only the 2x2 contingency table corresponding to the radiologist accuracy data. If a study used multiple reference standards, but we could determine 2x2 contingency tables that included only RT‐PCR as the reference standard, we extracted and analyzed these data. If a study reported accuracy data for multiple thresholds of index test positivity (e.g. studies that used the CO‐RADS scoring system, and/or the RSNA scoring system), we extracted the 2x2 contingency table for all available thresholds.

Two of the 11 studies that used the CO‐RADS scoring system did not report the 2x2 data for all five CO‐RADS thresholds. For these two studies, we contacted the corresponding authors but could not obtain the complete data; thus, we were only able to extract data for a CO‐RADS threshold of 3. One of the five studies that used the RSNA scoring system did not report the 2x2 data for all four RSNA thresholds. For this one study, we contacted the corresponding authors but could not obtain the complete data; thus we were only able to extract data for RSNA thresholds from 3 to 4 for this study.

In addition, we extracted the following items.

  1. Study setting (including country), age of study participants, study dates, disease prevalence at the time of acquisition (as reported in the study), number of participants, participant symptoms, number of imaging studies (and if more than one study was done per participant), participant outcomes and other relevant participant demographic parameters.

  2. Study design.

  3. Imaging timing relative to disease course.

  4. CT, chest X‐ray and ultrasound findings.

  5. Criteria for ‘positive’ diagnosis of COVID‐19 on imaging.

  6. Index test technical parameters.

  7. Reference standard results and details. If RT‐PCR was performed, timing of test, number of tests and method of acquisition (or similar details regarding other reference standards used).

  8. Details regarding interpretation of the index test (level of training, number of readers, the inter‐observer variability).

  9. The number of true positives, false positives, false negatives and true negatives or summary statistics from which they can be computed.

  10. Participant co‐morbidities as described in the studies.

Assessment of methodological quality

The review authors assessed the risk of bias and applicability concerns independently, in duplicate, using QUADAS‐2. Three review authors resolved any disagreements through discussion. See Appendix 2 for an explanation of the operationalization of the four QUADAS‐2 domains: participant selection, index test(s), reference standard(s), flow and timing.

Statistical analysis and data synthesis

We presented sensitivities and specificities per study using paired forest plots and we summarized pooled estimates in tables. We analyzed the data on a participant level, not a lesion on lung segment level, since this is what determines care.

We used a bivariate model for meta‐analyses, taking into account the within‐ and between‐study variance, and the correlation between sensitivity and specificity across studies (Chu 2006; Reitsma 2005). We performed meta‐analyses when four or more studies evaluated a given modality. We also performed sensitivity analyses by limiting inclusion in the meta‐analysis to studies published in peer‐reviewed journals. We undertook meta‐analyses using metandi in STATA (Harbord 2009; StataCorp 2019).

If a study reported accuracy data at multiple thresholds of index test positivity, we used the 2x2 contingency table corresponding to the threshold producing the highest Youden’s Index (YI) (YI = sensitivity + specificity – 1) for inclusion in the meta‐analysis. In addition, for studies that evaluated positive imaging chest CT imaging in repeat RT‐PCR positive results, we presented rates of positive imaging per study using forest plots. We used the same meta‐analysis methods for all primary and secondary objectives (metandi and meqrlogit in STATA, specifically).

Investigations of heterogeneity

We investigated heterogeneity by visual inspection of paired forest plots and summary receiver operating characteristics (SROC) plots. For chest CT studies, we evaluated the impact reference standard conduct (RT‐PCR performed at least twice in all participants with initial negative results versus RT‐PCR not done twice). For chest CT, chest X‐ray and ultrasound of the lungs, we evaluated the definition for index test positivity (radiologist impression versus formal scoring system). To investigate the impact of these factors on accuracy estimates, we used meta‐regression with the variable of interest added as a covariate to a bivariate model. Using the model parameters, we used a post estimation command to compute absolute differences in pooled sensitivity and specificity and we obtained their 95% CI using the delta method. We obtained P values using the Wald test. We performed meta‐regression when variables of interest consisted of subgroups with five or more studies in each subgroup, an arbitrary threshold chosen to facilitate convergence of the analyses using the bivariate model. We undertook meta‐regression using meqrlogit in STATA (StataCorp 2019).

Threshold effects

We performed meta‐analyses using a bivariate model for studies that used common thresholds for test positivity. (i.e. chest CT studies at CO‐RADS thresholds 2, 3, 4 and 5 and chest CT studies at RSNA thresholds 2, 3 and 4)

We used ggplot2 and ggforce in R to generate a plot displaying pooled accuracy estimates at varying CO‐RADS and RSNA thresholds (Wickham 2016; Pedersen 2020; R Core Team 2021).

Indirect test comparisons

We performed this using meta‐regression with modality type (i.e. chest CT, chest X‐ray, and ultrasound of the lungs) added as a covariate to a bivariate model. We obtained P values using the Wald test.

In future updates, as more data become available, we will also perform test comparisons that are restricted to only comparative studies (i.e. direct comparisons). It should be noted that there were not enough studies for direct comparisons.

We also generated a plot displaying meta‐analysis results across Salameh 2020a, Islam 2020, Islam 2021 and this version of this review (i.e. pooled sensitivity and specificity estimates from the Salameh 2020a published in September 2020, Islam 2020 published in November 2020, Islam 2021 published in February 2021, and this current version) using ggplot2 and ggforce in R (Wickham 2016; Pedersen 2020; R Core Team 2021).

Assessment of reporting bias

For this review, we did not undertake tests for publication bias and made no formal assessment of reporting bias.

Summary of findings

We provided a summary of the key findings of this review in Table 1, indicating the certainty of evidence for each finding and emphasizing the main gaps in our current level of available evidence.

Updating

Islam 2020 and Islam 2021 contained studies up to 22 June 2020 and up to 30 September 2020 respectively. This fourth version contains the results of an updated search performed on 17 February 2021.

Results

Results of the search

We identified 7734 search results and imported 976 studies for screening. Subsequently, we removed 11 duplicates. We then screened a total of 965 unique references (published or preprint studies) for inclusion; this is inclusive of the 773 references we screened in Salameh 2020a, Islam 2020, and Islam 2021. Of the 188 records selected for full‐text assessment, we included 98 studies in this review for all objectives. Of these 98 studies, 94 were included for evaluating the diagnostic accuracy of thoracic imaging in the evaluation of people with suspected COVID‐19; of these 94 studies, four have been included since our initial review(Salameh 2020a) and 12 have been included since the first update of this review (Islam 2020) and 29 have been included since the first update of this review (Islam 2021). Furthermore, 10 studies of the 98 included in this review were included for evaluating the accuracy of thoracic imaging for imagining asymptomatic individuals, and eight were included for assessing the rate of positive imaging in individuals with initial RT‐PCR negative results and positive RT‐PCR results on follow‐up.

Refer to Figure 1 for the PRISMA flow diagram of search and inclusion results (Salameh 2020b; Moher 2009). Exclusions were mainly due to ineligible study design, ineligible study outcomes, or ineligible patient populations; see Figure 1.

1.

1

Study flow diagram

Description of included studies (diagnostic accuracy in suspected participants)

We included 94 studies (64 CT, 12 X‐ray, 11 ultrasounds, three both CT and X‐ray, two both CT and ultrasound, and two both X‐ray and ultrasound) with a total of 37,631 participants suspected of having COVID‐19, of whom 19768 (53%) had a final diagnosis of COVID‐19. This could be on the basis of symptoms or epidemiological risk factors such as close contact with confirmed case.

The median sample size was 234 (interquartile range (IQR) 101.25 to 478.75). Sixty‐five studies were conducted in Europe (Italy 19, the Netherlands 9, France 9, Belgium 5, Turkey 6, Germany 7, UK 4, Switzerland 2, Czech Republic 1, Ireland 1, Spain 1, Denmark 1), 19 were conducted in Asia (China 9, Korea 1, India 4, Iran 2, Japan 1, Pakistan 1, United Arab Emirates 1), and the remaining studies were conducted in North America (USA 6, Canada 1) and South America (Brazil 3). Index test readings were performed by radiologists in 49 studies (52%), radiology residents in two studies (2%), both radiologists and residents in three (4%) study, and radiographers and radiologist in one study (1%); 39 studies (37%) did not clearly report the level of training of readers. Technical parameters regarding the protocol of chest CT used in 69 studies were not clearly reported in 31 (44%) studies, while non‐contrast CT was used in 25 (36%) studies, high‐resolution chest CT was used in eight (11%) studies, low‐dose CT with or without contrast was used in 11 (15%) studies and CT with IV contrast was used in five (7%) studies. Manuscripts of three (3%) of the studies were available only as preprints at the time of the search. Characteristics of the included studies are summarized in Table 2, and outlined in detail in the Characteristics of included studies.

1. Summary of included studies for diagnostic accuracy in suspected participants.
Study ID Country of corresponding author Study design Age group Setting Index test(s) Definition for index test positivity Level of training of readers Reference standard Prevalence
Ai 2020a China Suspected patients (unclear) Adults only Inpatient Chest CT Unclear Radiologist RT‐PCR, no other details provided 0.6
Aslan 2020 Turkey Suspected patients (all symptomatic) Adults only Outpatient Chest CT (non‐contrast, low dose) Pneumonia appeared to be radiologist's impression Radiologist RT‐PCR twice, in all with initial negative results 0.8
Bahrami‐Motlagh 2020 Iran Suspected patients (all symptomatic) Adults only Outpatient Chest CT (low dose) They reported negative or positive CT, according to previous reports on typical and atypical CT findings of COVID‐19 pneumonia. Unclear RT‐PCR, no other details provided 0.5
Barbosa 2020 Brazil Suspected patients (all symptomatic) Adults only Unclear Chest CT RSNA classification Radiologist RT‐PCR, no other details provided 0.3
Bellini 2020 Italy Suspected patients (all symptomatic) Children and adults Unclear Chest CT (non‐contrast) CO‐RADS classification Radiologist RT‐PCR twice, in some with initial negative results 0.2
Besutti 2020 Italy Suspected patients (all symptomatic) Adults, perhaps also children Outpatient Chest CT (non‐contrast) A structured report about the probability of COVID‐19 pneumonia Radiologist RT‐PCR twice, in some with initial negative results 0.8
Bock 2021 Denmark Suspected patients (all symptomatic) Adults only Outpatient Ultrasound of the lungs (POCUS) LUS was performed to determine the presence of the following predefined conditions: focal B‐lines, interstitial syndrome, lung consolidation, pleural effusion and pneumothorax. In all 14 zones, it was noted whether lung sliding, lung pulse, lung point, multiple B‐lines (≥ 3 per intercostal space), or thickened or fragmented visceral pleura were present. A normal LUS was defined as sufficient LUSinvestigation with none of the above‐mentioned findings. Unclear RT‐PCR, no other details provided 0.4
Bollineni 2021 Belgium Suspected patients (all symptomatic) Mix of children and adults Outpatient Chest CT (non‐contrast, low dose) Unclear Unclear RT‐PCR twice, in all with initial negative results 0.6
Borakati 2020 UK Suspected patients (symptomatic or asymptomatic) Adults, perhaps also children Outpatient Chest CT (non‐contrast, IV contrast)/ chest radiographs BSTI classification Radiologist RT‐PCR twice, in some with initial negative results 0.6
Bosso 2021 Italy Suspected patients (all symptomatic) Adults, perhaps also children Outpatient Ultrasound of the lungs (POCUS) Unclear Unclear RT‐PCR twice, in some with initial negative results 0.4
Boussouar 2020 France Suspected patients (all symptomatic) Adults only Outpatient Chest CT (non‐contrast) The conclusion was therefore one of the following: 1) imaging patterns suggesting the presence of COVID‐19; 2) imaging patterns suggesting an alternative diagnosis; 3) imaging patterns suggesting a combination of COVID‐19 with underlying lung disease; 4) CT considered normal Radiologists RT‐PCR twice, in all with initial negative results 0.5
Brun 2021 France Suspected patients (all symptomatic) Adults, perhaps also children Outpatient Chest CT (low dose) Highly probable, probable, and less probable of COVID‐19 pneumonia, alternative diagnosis, or normal. They established their diagnosis based on recent publications from China illustrating typical and atypical patterns in patients with COVID‐19 pneumonia (Pan 2020; Li 2020a; Ye 2020; Kanne 2020, Zhao 2020, Wang 2020a; Salehi 2020) and according to the Radiological Society of North America expert consensus statement (Zhou 2020) Unclear RT‐PCR, no other details provided 0.6
Caruso 2020 Italy Suspected patients(all symptomatic) Adults only Outpatient Chest CT (non‐contrast) Pneumonia Radiologist RT‐PCR twice, in all with initial negative results 0.4
Cengel 2021 Turkey Suspected patients (symptomatic or asymptomatic) Adults, perhaps also children Outpatient Chest CT (non‐contrast) RSNA classification Unclear RT‐PCR twice, in some with initial negative results 0.7
Colombi 2020a Italy Suspected patients (all symptomatic) Adults, perhaps also children Outpatient Chest CT (low dose)/ultrasound of lungs RSNA classification Unclear RT‐PCR twice, in some with initial negative results 0.7
Cozzi 2020 Italy Suspected patients (symptomatic or asymptomatic) Unclear Outpatient Chest radiographs/ Chest X‐rays The presence of interstitial infiltrates with predominantly bilateral and basal distribution Radiologist RT‐PCR, no other details provided 0.8
De Smet 2020 Belgium Suspected patients (all symptomatic) Children and adults Inpatient Chest CT CO‐RADS classification Unclear RT‐PCR, no other details provided 0.4
Debray 2020 France Suspected patients (unclear) Adults only Inpatient Chest CT (non‐contrast) “Evocative”: multifocal ground‐glass opacities, being nodular or not, or crazy‐paving with or without consolidations, with a bilateral, peripheral or mixed distribution and involvement of the posterior zones Radiologist RT‐PCR twice, in some with initial negative results 0.6
Deng 2020 China Suspected patients (all symptomatic) Children and adults Inpatient Chest CT (high resolution) Any one of the following: a) Single, multiple, or diffuse ground‐glass opacity, with thickened blood vessels and thickened bronchial shadows passing through, with or without localized lobular septal grid thickening; b) Single or multiple real shadows, (2) Reexamination 3 to 5 days later showed that the original ground‐glass opacity or consolidation range increased, the number increased, or accompanied by pleural effusion on one or both sides Radiologist RT‐PCR once 0.7
Dimeglio 2021 France Suspected patients (all symptomatic) Unclear Outpatient Chest CT Following the recommendation of the French Society of Radiology Unclear RT‐PCR once 0.4
Dini 2020 Italy Suspected patients (symptomatic or asymptomatic) 70 years of age and older Outpatient(LTC) Ultrasound of lungs(POCUS) Scoring system: non‐coalescent B‐lines, coalescent and with iperdensed non‐consolidated state. Unclear RT‐PCR once 0.6
Djangang 2020 Belgium Suspected patients (all symptomatic) Adults only Outpatient Chest CT CT‐scan was suggestive or not for COVID‐19 (i.e., ground‐glass opacities, consolidation or crazy‐paving patterns) (Ai 2020a; Zhang 2020) Radiologist RT‐PCR twice, in some with initial negative results 0.5
Dofferhoff 2020 The Netherlands Suspected patients (symptomatic or asymptomatic) Adults only Inpatient Chest CT (low dose) CO‐RADS classification; threshold not pre‐specified Unclear RT‐PCR twice, in some with initial negative results 0.5
Dogan 2020 Turkey Suspected patients (symptomatic or asymptomatic) Adults only Outpatient Chest CT (non‐contrast) RSNA criteria: typical, indeterminate, atypical, negative Radiologist RT‐PCR twice, in all with initial negative results 0.5
Ducray 2020 France Suspected patients (symptomatic or asymptomatic) Adults only Outpatient Chest CT (non‐contrast, IV contrast) On the final report, patients were rated as “Surely COVID+” when presenting with peripheral, bilateral, or multifocal GGO of rounded morphology ± consolidation or crazy paving, reversed halo sign, or sub‐pleural bands of consolidations. Patients were rated as “Possible COVID+” when presenting with multifocal, diffuse, peripheral, or unilateral GGO ± consolidation lacking a specific distribution and non‐rounded or non‐peripheral or with only few very small GGO with a non‐rounded and non‐peripheral distribution or with atypical findings: large pleural effusion, major lymph node size increase, or bronchiolitis pattern. Patients were rated as “COVID−” when the chest CT was normal or demonstrating another pathology Radiologist RT‐PCR twice, in some with initial negative results 0.4
Erxleben 2021 Germany Suspected patients (all symptomatic) Adults, perhaps also children Outpatient Chest CT (low dose) Unclear: "All CT images were evaluated manually and data on presence/absence of COVID‐19 was assessed" Unclear RT‐PCR twice, in some with initial negative results 0.1
Falaschi 2020 Italy Suspected patients (all symptomatic) Adults only Outpatient Chest CT (non‐contrast) RSNA classification Radiologist RT‐PCR twice, in some with initial negative results 0.6
Ferda 2020 Czech Republic Suspected patients (all symptomatic) Mix of children and adults Outpatient Chest CT(IV contrast) Groundglass opacities, mixed ground‐glass opacities, thickening of intra‐lobular septa, negative bronchogram, reverse halo sign, and dilatation of the vascular structures. Predominant peripheral, bilateral and caudal distributions were suspected to be COVID‐19 pneumonia. Radiologist RT‐PCR twice, in some with initial negative results 0.1
Fink 2021 Germany Suspected patients (all symptomatic) Adults only Outpatient Chest CT (High resolution)/ Chest X‐rays CT scans were classified according to two different reading scores: 1) presence of pneumonic features (0 – absent, 1 – present) and 2) presence of COVID‐19 typical features (0 – not typical, 1 – possible, 2 – highly suspicious). According to the current literature, COVID‐19 typical features were defined as ground glass opacities (GGO) with or without “crazy paving” and/or consolidations with peripheral emphasis. Radiologist RT‐PCR twice, in some with initial negative results 0.3
Fonsi 2020 Italy Suspected patients (all symptomatic) Adults only Outpatient Chest CT (non‐contrast) Ground glass opacities (GGOs); consolidation; a mixed GGO and consolidation pattern; single or multiple solid nodules surrounded by GGOs; a focal or multifocal distribution; GGO and consolidation location; multilobe involvement; a bilateral distribution; interlobular septal thickening; an air bronchogram; the presence of cavitation; bronchial wall thickening; bronchiectasis; mediastinal lymph node enlargement ; pleural effusion; and pericardial effusion. Radiologist RT‐PCR once 0.7
Fujioka 2020 Japan Suspected patients (all symptomatic) Adults only Unclear Chest CT CO‐RADS classification Radiologist RT‐PCR once 0.5
Gaia 2020 Italy Suspected patients (all symptomatic) Adults only Outpatient Chest CT Simpson 2020 Radiologist RT‐PCR once 0.5
Giannitto 2020 Italy Suspected patients (all symptomatic) Adults only Outpatient Chest CT (non‐contrast) Unclear Radiologist RT‐PCR twice, in all with initial negative results 0.3
Gietema 2020 The Netherlands Suspected patients (all symptomatic) Adults only Outpatient Chest CT (non‐contrast) Reporting scheme Resident RT‐PCR twice, in some with initial negative results 0.4
Gil‐Rodrigo 2020 Spain Suspected patients (all symptomatic) Adults only Outpatient Ultrasound of the lungs (POCUS) Scoring system by Soldati 2020 Unclear RT‐PCR once 0.4
Grando 2020 Brazil Suspected patients (all symptomatic) Adults only Outpatient Chest CT (non‐contrast) CT features were classified as "typical," "indeterminate," "atypical," and "negative" for COVID‐19 pneumonia", according to RSNA expert consensus Radiologist. RT‐PCR twice, in some with initial negative results 0.5
Gross 2021 Germany Suspected patients (all symptomatic) Adults only Outpatient Chest CT (low dose) CO‐RADS classification Radiologists RT‐PCR twice, in all with initial negative results 0.2
Guillo 2020 France Suspected patients (all symptomatic) Adults only Outpatient Chest CT (non‐contrast, IV contrast) A structured report about the probability of COVID‐19 pneumonia Resident RT‐PCR twice, in some with initial negative results 0.6
Haak 2021 The Netherlands Suspected patients (all symptomatic) Adults only Outpatient Ultrasound of the lungs (POCUS) Score of >/= 2 based on (Peng 2020b; 4 Lung ultrasound in COVID‐19 2020; Focus met POCUS op COVID‐19 2020) Unclear RT‐PCR twice, in all with initial negative results 0.3
Hanif 2021 Pakistan Suspected patients (all symptomatic) Adults only Outpatient Chest CT (high resolution) Positive findings for COVID‐19 defined as bilateral, multifocal, multilobar ground glass opacities with or without sub‐segmental consolidations or crazy paving pattern in a peripheral distribution (Han 2020; Lee 2020; Simpson 2020) Negative findings defined as presence of isolated lobar consolidation, pleural effusion, nodularity and absence of the positive findings of COVID‐19. Indeterminate cases defined as having multilobar ground glass opacities or consolidation with central or diffuse distribution lacking subpleural pattern or unilateral ground glass opacities; these were further categorized as positive or negative for COVID‐19 on the basis of clinical history, mutual consensus and RT‐PCR results, if available. Radiologists RT‐PCR twice, in some with initial negative results 0.8
He 2020 China Suspected patients (unclear) Children and adults Inpatient Chest CT (high resolution) Ground‐glass opacity with or without consolidation, crazy paving patten, peripheral and diffuse distribution, and bilateral/multilobular involvement Radiologist RT‐PCR twice, in some with initial negative results 0.4
Hermans 2020 The Netherlands Suspected patients (symptomatic or asymptomatic) Adults only Outpatient Chest CT CO‐RADS classification Radiologist RT‐PCR once 0.4
Hernigou 2020 Belgium Suspected patients (symptomatic or asymptomatic) Adults only Inpatient Chest CT (low dose) Unclear Radiologist RT‐PCR twice, in some with initial negative results 0.3
Herpe 2020 France Suspected patients (all symptomatic) Children and adults Unclear Chest CT Bilateral ground glass opacities with peripheral distribution, bilateral crazy paving appearance with intralobular thickening, reverse halo sign, or other signs compatible with organizing pneumonia. Radiologist RT‐PCR twice, in some with initial negative results 0.1
Hwang 2020 Korea Suspected patients (symptomatic or asymptomatic) Adults, perhaps also children Unclear Chest radiographs / chest X‐rays Abnormality suggesting pneumonia Radiologists and Resident RT‐PCR, no other details provided 0.05
Ippolito 2020 Italy Suspected patients (all symptomatic) Children and adults Inpatient Chest radiographs / chest X‐rays Reticulations, alveolar opacities or both Radiologist RT‐PCR, no other details provided 0.4
Jalil 2020 USA Suspected patients (all symptomatic) Adults, perhaps also children Outpatient Ultrasound of the lungs (POCUS) Unclear Unclear RT‐PCR twice, in all with initial negative results 0.5
Krdzalic 2020 The Netherlands Suspected patients (all symptomatic) Adults only Unclear Chest CT CO‐RADS classification Radiologist RT‐PCR twice, in some with initial negative results 0.5
Kuzan 2020 Turkey Suspected patients (all symptomatic) Adults only Outpatient Chest CT (non‐contrast) BSTI classification Radiologist RT‐PCR twice, in some with initial negative results 0.6
Lieveld 2021a The Netherlands Suspected patients (all symptomatic) Adults only Outpatient Chest CT CO‐RADS classification Radiologists RT‐PCR twice, in all with initial negative results 0.3
Lieveld 2021b The Netherlands Suspected patients (all symptomatic) Adults only Outpatient Ultrasound of the lungs (POCUS) CO‐RADS classification Unclear RT‐PCR twice, in some with initial negative results 0.4
Luo 2020a China Suspected patients (all symptomatic) Children and adults Inpatient Chest CT Scoring system was developed; threshold not pre‐specified Radiologist RT‐PCR twice, in all with initial negative results 0.4
Majeed 2020 UK Suspected patients (symptomatic or asymptomatic) Adults only Outpatient Chest CT BSTI classification and RSNA classification unclear RT‐PCR twice, in some with initial negative results 0.3
Mei 2020 USA Suspected patients (symptomatic or asymptomatic) Children and adults Unclear Chest CT Unclear Radiologist RT‐PCR twice, in all with initial negative results 0.5
Miranda Magalhaes Santos 2020 Brazil Suspected patients (all symptomatic) Children and adults Outpatient Chest CT RSNA classification Radiologist RT‐PCR, no other details provided 0.5
Moroni 2021 Italy Suspected patients (all symptomatic) Adults only Outpatient Chest radiographs / Chest X‐rays Unclear Unclear RT‐PCR, no other details provided 0.3
Murphy 2020 The Netherlands Suspected patients (all symptomatic) Children and adults Outpatient Chest radiographs / Chest X‐rays Readers assigned each image a category, sensitivities matched to AI reading Radiologist RT‐PCR, no other details provided 0.5
Narinx 2020 Belgium Suspected patients (all symptomatic) Adults, perhaps also children Outpatient Chest CT (low dose, with or without contrast)/ultrasound of lungs (POCUS) For Ultrasound: POCUS lung positive if one or more BLUE points showed a positive B‐line parameter.
For chest CT: Scored as suggestive for or inconsistent with COVID‐19 infection based on the presence of clinical manifestations as presented by Ng 2020 and Shi 2020
Radiologist RT‐PCR, no other details provided 0.2
Nivet 2021 France Suspected patients (all symptomatic) Adults only Outpatient Chest CT (non‐contrast) Each reading was categorized using a five‐point score, adapted from the recommendations of the Société Française de Radiologie (SFR). (1) normal; (2) non‐infectious findings; (3) infectious findings but not consistent with COVID‐19 infection; (4) consistent with COVID‐19 infection; (5) typical appearance of COVID‐19 infection. Residents and radiologist RT‐PCR twice, in some with initial negative results 0.4
Ohana 2021 France Suspected patients (all symptomatic) Adults only Outpatient Chest CT (non‐contrast) CT with typical COVID‐19 appearance, i.e. bilateral and predominantly peripheral and sub‐pleural ground glass opacities and/or alveolar consolidations, were classified as positive AB65 Radiologists RT‐PCR twice, in some with initial negative results 0.5
O'Neill 2020 Canada Suspected patients (symptomatic or asymptomatic) Adults only Outpatient Chest CT RSNA classification and CO‐RADS classification Radiologists RT‐PCR twice, in all with initial negative results 0.7
Pagano 2021 USA Suspected patients (symptomatic or asymptomatic) Adults only Outpatient Chest radiographs/chest X‐rays Unclear Unclear RT‐PCR, no other details provided 0.8
Palmisano 2021 Italy Suspected patients (all symptomatic) Adults, perhaps also children Outpatient Chest CT (non‐contrast) RSNA classification Unclear RT‐PCR twice, in some with initial negative results 0.6
Pare 2020 USA Suspected patients (all symptomatic) Adults, perhaps also children Outpatient Chest radiographs /chest X‐rays/Ultrasound of lungs (POCUS) Classified CXRs as positive if the report included infection in the differential. Unclear RT‐PCR twice, in some with initial negative results 0.8
Patel 2020 USA suspected patients (symptomatic or asymptomatic) Adults, perhaps also children Outpatient Chest CT (high resolution) Category 1 – consistent with multifocal pneumonia; Category 2 – indeterminate for multifocal pneumonia; Category 3 – not consistent with multifocal pneumonia Unclear RT‐PCR, no other details provided 0.5
Patrucco 2021 Italy Suspected patients (symptomatic or asymptomatic) Adults, perhaps also children Outpatient Chest CT RSNA classification and CO‐RADS classification Unclear RT‐PCR, no other details provided 0.4
Peng 2020a China Suspected patients (symptomatic or asymptomatic) Children only Inpatient Chest CT Ground glass opacity, consolidations with surrounding halo sign, nodules, residual fibre strips, lymphadenopathy Radiologist RT‐PCR, no other details provided; other (positive contacts) 0.5
Pivetta 2021 Italy Suspected patients (all symptomatic) Adults only Outpatient Ultrasound of the lungs (POCUS) Unclear Unclear RT‐PCR twice, in some with initial negative results 0.4
Ravikanth 2021 India Suspected patients (all symptomatic) Adults only Outpatient Chest CT (CT thorax with IV contrast ) Dichotomous ‐ suspicious or not suspicious for COVID‐19. Resident and radiologist RT‐PCR twice, in some with initial negative results 0.8
Reginelli 2021 Italy Suspected patients (symptomatic or asymptomatic) Adults only Outpatient Chest CT Radiologists observed according to localization and distribution of GGO and consolidations, crazy paving pattern, and presence of nodules Radiologist RT‐PCR twice, in some with initial negative results 0.8
Rona 2021 Turkey Suspected patients (all symptomatic) Children and young adults only Outpatient Chest CT (non‐contrast) Computed tomography images were divided into 3 groups: normal, consistent with COVID‐19, and inconsistent with COVID‐19. Multifocal consolidation, ground‐glass opacity, and reversed halo sign on CT were considered to be consistent with COVID‐19. Unclear RT‐PCR twice, in some with initial negative results 0.4
Roy Choudhury 2020 India Suspected patients (all symptomatic) Unclear Inpatient Chest radiographs/chest X‐rays Simpson 2020 Unclear RT‐PCR, no other details provided 0.3
Saeed 2020 United Arab Emirates Suspected patients (all symptomatic) Adults only Outpatient Chest CT (high resolution) RSNA classification radiologists RT‐PCR twice, in all with initial negative results 0.7
Salehi‐Pourmehr 2020 Iran Suspected patients (all symptomatic) Adults only Outpatient Chest CT Unclear Unclear RT‐PCR, no other details provided 0.3
Schalekamp 2020 The Netherlands Suspected patients (all symptomatic) Adults only Outpatient Chest CT (non‐contrast) CO‐RADS classification radiologists RT‐PCR twice, in some with initial negative results 0.5
Schmid 2020 Germany Suspected patients (all symptomatic) Adults only Inpatient Ultrasound of the lungs (POCUS) Unclear unclear RT‐PCR, no other details provided 0.3
Schulze‐hagen 2020 Germany Suspected patients (all symptomatic) Adults only Unclear Chest CT (non‐contrast, Low dose) COV‐Rads classification Radiologist RT‐PCR twice, in some with initial negative results 0.4
Shah 2021 India Suspected patients (all symptomatic) Not Reported Outpatient Chest CT (high resolution) Evaluated for ground‐glass opacities (GGOs), reticular thickening, focal consolidations, fibrosis, pleural effusion, nodules, and hilar lymphadenopathy Radiologists RT‐PCR twice, in some with initial negative results 0.9
Skalidis 2020 Switzerland Suspected patients (all symptomatic) Adults only Outpatient Chest CT (low dose) Each specialist classified the abnormal CT according to GGO distribution of the affected lung parenchyma graded on a 3‐point scale: 1 = light <30%, 2 = moderate 30–60%, 3 = severe >60%. Finally, the results of the classification were merged by consensus and the specialists classified the CT on positive or negative for COVID‐19. Unclear RT‐PCR twice, in some with initial negative results 0.4
Song 2020a China Suspected patients (all symptomatic) Adults only Inpatient Chest CT Viral pneumonia according to: multiple bilateral, ill‐defined ground glass opacities (GGOs) or mixed consolidation with diffuse peripheral distribution or bilateral pulmonary consolidation Radiologist RT‐PCR twice, in all with initial negative results 0.5
Sorlini 2021 Italy Suspected patients (all symptomatic) Adults, perhaps also children Outpatient Chest X‐rays/ Ultrasound of the lungs (POCUS) Interstitial lung syndrome: two or more positive regions bilaterally with irregular pleural line.
• Interstitial lung pattern: two or more positive regions with irregular pleural line, with focal/unilateral distribution.
• White lung (coalescent B lines) in two or more zones.
• Subpleural consolidations. Unclear RT‐PCR twice, in some with initial negative results 0.7
Speidel 2021 Switzerland Suspected patients (all symptomatic) Adults only Inpatient Ultrasound of the lungs (POCUS) Unclear Unclear RT‐PCR, no other details provided 0.2
Steuwe 2020 Germany Suspected patients (all symptomatic) Adults only Unclear Chest CT (Non‐contrast, Low dose) Unclear Unclear RT‐PCR twice, in some with initial negative results 0.2
Stevens 2020 UK Suspected patients (all symptomatic) Adults only Outpatient Chest radiographs/ Chest X‐rays BSTI classification Radiographer and Radiologist RT‐PCR twice, in some with initial negative results 0.8
Sukhija 2021 India Suspected patients (all symptomatic) Adults only Unclear Chest X‐rays Unclear Unclear RT‐PCR, no other details provided 0.6
Sverzellati Nicola 2021 Italy Suspected patients (all symptomatic) Adults only Inpatient Chest CT (High resolution)/ Chest X‐rays 4 CT categories: normal, alternative diagnosis, indeterminate, or typical for COVID‐19 pneumonia. Visual analysis: extent of combined GGO and consolidation was visually scored at the nearest 5% on the whole lungs. Distribution of findings, bilateral or unilateral involvement also considered in scoring. Radiologist RT‐PCR twice, in all with initial negative results 0.7
Teichgraber 2021 Germany Suspected patients (all symptomatic) Adults only Outpatient Chest CT (Low dose) RSNA classification Unclear RT‐PCR twice, in all with initial negative results 0.1
Tsakok 2020 UK Suspected patients (all symptomatic) Adults only Outpatient Chest X‐rays Unclear Unclear RT‐PCR, no other details provided 0.4
Wang 2020a China Suspected patients (symptomatic or asymptomatic) Children and adults Unclear Chest CT Standardized imaging reporting system Unclear RT‐PCR twice, in all with initial negative results 0.1
Wehbe 2021 USA Suspected patients (all symptomatic) Adults only Mixed Chest X‐rays Point scoring system based on overall impression of "positive for COVID‐19" or "negative for COVID‐19" radiologist RT‐PCR twice, in some with initial negative results 0.4
Xiaocheng 2020 China Suspected patients (all symptomatic) Adults only Outpatient Chest CT Unclear Unclear RT‐PCR, no other details provided 0.1
Xiong 2020 China Suspected patients (unclear) Children and adults Inpatient Chest CT Subpleural ground glass opacity without pleural effusion, bronchial changes or lymphadenopathy Radiologist RT‐PCR, no other details provided 0.4
Yassa 2020 Turkey Suspected patients (symptomatic or asymptomatic) Adults only Inpatient Ultrasound of the lungs (POCUS) 4 categories: characteristic changes, ordinary inflammation, other changes, normal Unclear RT‐PCR twice, in some with initial negative results 0.08
Yates 2021 Ireland Suspected patients (all symptomatic) Adults, perhaps also children Outpatient Chest X‐rays Unclear Unclear RT‐PCR twice, in all with initial negative results 0.2

CO‐RADS: COVID‐19 Reporting and Data System; CT: computed tomography; RSNA: Radiological Society of North America; RT‐PCR: reverse transcriptase polymerase chain reaction.

Participant characteristics (diagnostic accuracy in suspected participants)

All participants were suspected of having COVID‐19. Seventy (74%) studies involved only symptomatic participants, 20 (21%) studies involved both symptomatic and asymptomatic participants, and four (4%) studies did not clearly report participants’ symptom status. Fifty‐seven studies included only adult participants (aged 16 years and over), 32 studies included both children and adults (although in most cases, only a minority of included patients were children), one study included only children, one study included participants aged 70 years and older, and the remaining three studies did not clearly report the age range of participants.

All 94 studies used RT‐PCR as the reference standard for the diagnosis of COVID‐19, with 82 studies using only RT‐PCR as the reference standard and seven studies using a combination of RT‐PCR and other criteria (laboratory tests 2, clinical signs and symptoms 2, clinical signs on follow‐up 1, positive contacts 1, and follow‐up phone calls 1) as the reference standard.

With respect to RT‐PCR testing, eight studies tested each participant once, 42 studies tested some participants with initial negative RT‐PCR results at least twice, 19 studies tested all participants with initial negative RT‐PCR results at least twice, and 25 studies did not report on the frequency of testing per participant.

Seventeen studies included inpatients, 65 studies included outpatients, one study included both in‐ and outpatients, while the remaining 23 studies were conducted in unclear settings. Thirty‐three (35%) studies described the co‐morbidities of the study population, which commonly included hypertension, cardiovascular disease, and diabetes; however, the overall presence of co‐morbidities in the participant groups of these studies was unclear.

Description of included studies (positive imaging in repeat RT‐PCR positive results)

We included eight studies (Besutti 2020; Bollineni 2021; Debray 2020; Giannitto 2020; Herpe 2020; Pivetta 2021; Reginelli 2021; Song 2020a) (seven CT, and one ultrasound), with a total of 198 participants suspected of having COVID‐19, all of whom had a final diagnosis of COVID‐19. All studies were also included for the primary objective.

Seven studies were conducted in Europe (Italy 4, France 2, Belgium 1), and one was conducted in Asia (China). Index test readings were performed by radiologists in five studies (62%), while three studies (37%) did not clearly report the level of training of readers.

Technical parameters regarding the protocol of chest CT used in seven studies were not clearly reported in two (29%) studies, while non‐contrast CT was used in four (57%) studies, low‐dose CT with or without contrast was used in one (14%) study. Characteristics of the included studies are summarized in Table 3, and outlined in detail in the Characteristics of included studies.

2. Characteristics of the included studies summarized for rate of positive imaging in repeat RT‐PCR positive results.
Study ID Country of corresponding author Study design Age group Setting Index test(s) Definition for index test positivity Level of training of readers Reference standard Prevalence
Besutti 2020 Italy Suspected patients (all symptomatic) Adults, perhaps also children Outpatient Chest CT (non‐contrast ) A structured report about the probability of COVID‐19 pneumonia Radiologist RT‐PCR once; twice in some 0.8
Bollineni 2021 Belgium Suspected patients (all symptomatic) Mix of children and adults Outpatient Chest CT (non‐contrast, Low dose) Unclear Unclear RT‐PCR twice, in all with initial negative results 0.6
Debray 2020 France Suspected patients (unclear) Adults only Inpatient Chest CT (non‐contrast) Evocative: multifocal ground‐glass opacities, being nodular or not, or crazy‐paving with or without consolidations, with a bilateral, peripheral or mixed distribution and involvement of the posterior zones Radiologist RT‐PCR once; twice in some 0.7
Giannitto 2020 Italy Suspected patients (all symptomatic) Adults only Outpatient Chest CT (non‐contrast) Unclear Radiologist RT‐PCR twice, if necessary 0.3
Herpe 2020 France Suspected patients (all symptomatic) Children and adults Unclear Chest CT Bilateral ground glass opacities with peripheral distribution, bilateral crazy paving appearance with intralobular thickening, reverse halo sign, or other signs compatible with organizing pneumonia. Radiologist RT‐PCR once; twice in some 0.1
Pivetta 2021 Italy Suspected patients (all symptomatic) Adults only Outpatient Ultrasound of the lungs (POCUS) Presence of focal or diffuse interstitial syndrome associated with spared areas, subpleural consolidations, and irregular or thickened pleural line was considered suggestive of SARS‐CoV2–related pneumonia Unclear RT‐PCR twice, in some with initial negative results 0.4
Reginelli 2021 Italy Suspected patients (symptomatic or asymptomatic) Adults only Outpatient Chest CT Radiologists observed according to localization and distribution of GGO and consolidations, crazy paving pattern, and presence of nodules Unclear RT‐PCR twice, in some with initial negative results 0.8
Song 2020a China Suspected patients (all symptomatic) Adults only Inpatient Chest CT Viral pneumonia according to: multiple bilateral, ill‐defined ground glass opacities (GGOs) or mixed consolidation with diffuse peripheral distribution or bilateral pulmonary consolidation Radiologist RT‐PCR twice, if necessary 0.5

Abbreviations: CT: computed tomography; GGO: ground glass opacity; POCUS: Point‐of‐Care Ultrasound; RSNA: Radiological Society of North America; RT‐PCR: reverse transcriptase polymerase chain reaction

Participant characteristics (positive imaging in repeat RT‐PCR positive results)

Five studies included only adult participants (aged 16 years and over), three studies included both children and adults. This covers the fact that most were symptomatic and so relatively high pre‐test probability of COVID‐9. All the studies used RT‐PCR as the reference standard for the diagnosis of COVID‐19. With respect to RT‐PCR testing, one study tested all participants with initial negative RT‐PCR results at least twice, and seven studies tested some participants with initial negative RT‐PCR results at least twice.

Five studies included outpatients, two studies included inpatients, while the remaining one study was conducted in an unclear setting. Three (37%) studies described the co‐morbidities of the study population, which included hypertension, cardiovascular disease, diabetes, and asthma. However, the overall presence of co‐morbidities in the participant groups of these studies was unclear.

Description of included studies (imaging asymptomatic individuals)

We included 10 studies (Dafydd 2021; De Smet 2020; Dini 2020; Dogan 2020; Gumus 2020; Hernigou 2020; Hwang 2020; Ooi 2021; Puylaert 2020; Yassa 2020) (seven CT, one X‐ray, two ultrasound) with a total of 2007 participants suspected of having COVID‐19, of whom 127 (6%) had a final diagnosis of COVID‐19. For example, patients who had preoperative chest CT included in a study (Gumus 2020). Of these 10 studies, six were also included for the primary objective. Eight studies were conducted in Europe (Italy 1, UK 2, Belgium 2, the Netherlands 1, Turkey 3), and one was conducted in Korea.

Index test readings were performed by radiologists in three studies (30%), one study by radiologist and resident (10%) and other six studies (60%) did not clearly report the level of training of readers.

Technical parameters regarding the protocol of chest CT used in three studies were not clearly reported in six (60%) studies, while non‐contrast CT was used in two (20%) studies, low‐dose CT with or without contrast was used in one (10%) study and high resolution in one (10%) study. Characteristics of the included studies are summarized in Table 4, and outlined in detail in the Characteristics of included studies.

3. Characteristics of the included studies summarized for asymptomatic studies.
Study ID Country of corresponding author Study design Age group Reason for screening asymptomatic patients Setting Index test(s) Definition for index test positivity Level of training of readers Reference standard Prevalence
Dafydd 2021 UK Asymptomatic participants Adults only Asymptomatic patients referred for elective oncological surgery underwent chest CT within 2 days of surgery in high risk surgical cases. Inpatient Chesr CT (High resolution) Unclear Radiologist RT‐PCR twice, in some with initial negative results 0.02
De Smet 2020 Belgium Asymptomatic participants Children and adults Asymptomatic patients admitted for COVID‐19‐unrelated urgent medical needs were screened by chest CT Inpatient Chest CT CO‐RADS classification Unclear RT‐PCR, no other details provided 0.05
Dogan 2020 Turkey Asymptomatic participants Adults only Asymptomatic individuals who were suspected to have COVID‐19 based on suspected contact underwent CT chest. Unclear Chest CT (non‐contrast CT thorax) RSNA classification Radiologist RT‐PCR twice, in all with initial negative results 0.3
Dini 2020 Italy Asymptomatic participants 70 years of age and older Asymptomatic patients institutionalized in residential age care facilities who were exposed to the infection underwent chest imaging. Outpatient (LTC) Ultrasound of lungs (POCUS) Scoring system: non‐coalescent B‐lines, coalescent and with iperdensed non‐consolidated state. Unclear RT‐PCR once 0.6
Gumus 2020 Turkey Asymptomatic participants Adults only Asymptomatic patients scheduled for any surgery were eligible for preoperative chest CT Inpatient Chest CT (non‐contrast) RSNA classification Unclear RT‐PCR twice, in some with initial negative results 0.01
Hernigou 2020 Belgium Asymptomatic participants Adults only Asymptomatic patients institutionalized in residential age care facilities who were exposed to the infection underwent chest imaging Inpatient Chest CT (low dose) Unclear Radiologist RT‐PCR twice, in some with initial negative results 0.3
Hwang 2020 Korea Asymptomatic participants Adults, perhaps also children Unclear Unclear Chest radiographs/Chest X‐rays Abnormality suggesting pneumonia Radiologists and Resident RT‐PCR, no other details provided 0.05
Ooi 2021 UK Asymptomatic participants Adults, perhaps also children Asymptomatic patients scheduled for elective surgery were eligible for preoperative chest CT. Outpatient Chest CT Each area was given a score between 0 and 3 Unclear RT‐PCR twice, in some with initial negative results 0.1
Puylaert 2020 The Netherlands Asymptomatic participants Adults only Asymptomatic patients scheduled for an elective or emergency surgery or interventional procedure under general anaesthesia were eligible for preoperative chest CT Inpatient Chest CT (low dose) CO‐RADS classification Unclear RT‐PCR once 0.01
Yassa 2020 Turkey Asymptomatic participants Adults only Asymptomatic pregnant women admitted to the hospital underwent radiologic imaging Inpatient Ultrasound of the lungs (POCUS) Unclear Unclear RT‐PCR twice, in some with initial negative results 0.04

Abbreviations: CO‐RADS: COVID‐19 Reporting and Data System; CT: computed tomography; LTC: long‐term care; POCUS: point‐of‐care Ultrasound; RSNA: Radiological Society of North America; RT‐PCR: reverse transcriptase polymerase chain reaction.

Participant characteristics (imaging asymptomatic individuals)

Six studies included only adult participants (aged 16 years and over), three studies included both children and adults, and one study included 70 years of age and older. All the studies used RT‐PCR as the reference standard for the diagnosis of COVID‐19. With respect to RT‐PCR testing, two studies tested each participant once, one study tested all participants with initial negative RT‐PCR results at least twice, five studies tested some participants with initial negative RT‐PCR results at least twice, and two studies did not report on the frequency of testing per participant.

Three studies included outpatients, five studies included inpatients, while the remaining two studies were conducted in unclear settings. One study (10%) described the co‐morbidities of the study population, which included hypertension, kidney disease, heart failure, and diabetes; however, the overall presence of co‐morbidities in the participant groups of these studies was unclear

Index tests

Our primary objective was to evaluate the diagnostic accuracy of thoracic imaging (computed tomography (CT), X‐ray and ultrasound) in people with suspected COVID‐19. Also, we assessed the rate of positive imaging in people who had an initial RT‐PCR negative result and a positive RT‐PCR result on follow‐up, and the diagnostic accuracy of thoracic imaging for screening COVID‐19 in asymptomatic individuals

With respect to the primary objective, 87 studies evaluated a single imaging modality and seven studies evaluated two imaging modalities. In total, the 94 studies reported a total of 101 imaging modality evaluations for the diagnostic accuracy of thoracic imaging in people with suspected COVID‐19. Chest CT was evaluated in 69 studies, chest X‐ray was evaluated in 17 studies, and ultrasound of the lungs was evaluated in 15 studies.

For the objective for positive imaging in repeat RT‐PCR positive results, all studies evaluated a single imaging modality. Chest CT was evaluated in seven studies and ultrasound of the lungs was evaluated in one study.

For the objective for asymptomatic screening, all studies evaluated a single imaging modality. Chest CT was evaluated in seven studies, chest X‐ray was evaluated in one study, and ultrasound of the lungs was evaluated in two studies.

Methodological quality of included studies

Figure 2 provides a summary of the overall methodological quality assessment using the QUADAS‐2 tool for all 98 included studies. Figure 3 displays a study‐level quality assessment (see Figure 3 for details).

2.

2

Risk of bias and applicability concerns graph: review authors' judgements about each domain presented as percentages across included studies (n = 98).

3.

3

Risk of bias and applicability concerns summary: review authors' judgements about each domain for each included study.

Across all 98 included studies, we found risk of bias based on concerns about the selection of participants to be high in 10 (10%) and unclear in 42 (42%) studies; the main concern in this domain was high risk of bias due to inappropriate exclusions (n = 10).

Risk of bias for chest CT (73 studies) was high in six (8%) and unclear in 27 (36%) studies; risk of bias because of concerns regarding application of chest X‐ray (17 studies) was unclear in seven (41%) studies, and risk of bias because of concerns regarding application of ultrasound of the lungs (15 studies) was unclear in six (37%) studies. The six CT studies with a high risk of bias did not predefine the positivity criteria for index tests or did not blind index test readers to reference standard results (n = 1).

Risk of bias based on concerns about the reference standard was high in 25 (26%) and unclear in 39 (39%) studies; the 25 studies with a high risk of bias used an single RT‐PCR protocol that was not likely to correctly classify the target condition.

Risk of bias based on concerns related to participant flow and timing was high in nine (9%) and unclear in 39 (41%) studies; the nine studies with a high risk of bias did not provide the same reference standard to all participants (n = 3), or did not have an appropriate time interval between the reference standard and index test (n = 6).

Concerns about the applicability of the evidence to participants were high in three studies (3%) and unclear in five (5%) studies. Concerns about the applicability of the evidence to the index test were high in one (1.4%) and unclear in two (2.7%) studies in 73 chest CT studies, high in two (12%) and unclear in one (6%) chest X‐ray study (17 studies), and unclear in one (6%) ultrasound studies (15 studies). Concerns about the applicability of the evidence to the reference standard were high in two (2%) studies and unclear in five (5%) studies. Additional details about risk of bias and applicability assessment are presented in Figure 3.

For rate of positive imaging in repeat RT‐PCR positive results (eight studies), most studies had selection bias when describing the implications of this finding, so strength of these results is limited. For selection of participants, there was high risk of bias in 2/8 and unclear risk of bias in 6/8 studies. For chest CT (seven studies), 2/7 had a high risk of bias and 5/7 had an unclear risk of bias for participant selection.

Findings

Pooled estimates in suspected individuals

The sensitivity of CT in 69 studies (involving 14,342 (51%) cases in 28,285 participants) ranged from 45% to 100%, and the specificity ranged from 10% to 99% (Figure 4). The pooled sensitivity for chest CT was 86.9% (95% CI 83.6 to 89.6), and the pooled specificity was 78.3% (95% CI 73.7 to 82.3). The scatter of the study points in ROC space on the SROC plot (Figure 5) shows substantial variability in sensitivity and specificity.

4.

4

Forest plot of chest CT in suspected cases.

5.

5

Summary ROC plot of chest CT in suspected cases.The summary point is indicated by the solid black circle, individual studies are indicated by outlined circles (scale=study sample size). The dotted border and the dashed border represent 95% confidence regions and 95% prediction regions, respectively.

The forest plots for chest X‐ray and ultrasound of the lungs are presented in Figure 6. The sensitivity of chest X‐ray in 17 studies (including 5303 (62%) cases in 8529 participants) ranged from 44% to 94% and the specificity ranged from 24% to 93%. The pooled sensitivity for chest X‐ray was 73.1% (95% CI 64.1 to 80.5) and the pooled specificity was 73.3% (95% CI 61.9 to 82.2). The scatter of the study points in ROC space on the SROC plot (Figure 7) shows substantial variability in sensitivity, and specificity for chest X‐ray.

6.

6

7.

7

Summary ROC plot of chest X‐ray in suspected cases.The summary point is indicated by the solid black circle, individual studies are indicated by outlined circles (scale=study sample size). The dotted border and the dashed border represent 95% confidence regions and 95% prediction regions, respectively.

The sensitivity of ultrasound of the lungs in 15 studies (including 1158 (49%) cases in 2410 participants) ranged from 73% to 94% and the specificity ranged from 21% to 98%. The pooled sensitivity for ultrasound was 88.9% (95% CI 84.9 to 92.0), and the pooled specificity was 72.2% (95% CI 58.8 to 82.5). The scatter of the study points in ROC space on the SROC plot (Figure 8) shows substantial variability in sensitivity and specificity for ultrasound of the lungs.

8.

8

Summary ROC plot of ultrasound of the lungs in suspected cases.The summary point is indicated by the solid black circle, individual studies are indicated by outlined circles (scale=study sample size). The dotted border and the dashed border represent 95% confidence regions and 95% prediction regions, respectively.

Sensitivity analyses

For CT studies with suspected participants, we excluded the three studies published as preprints and found this did not affect summary sensitivity and specificity; studies published in peer‐reviewed journals (n = 66) had a pooled sensitivity of 87.5% (95% CI 84.3 to 90.1) and a pooled specificity of 78.0% (95% CI 72.9 to 82.4). These results are outlined in Table 5. The publication status of studies has been updated as of 17 February 2021.

4. Sensitivity analyses for chest CT of suspected cases.
Analysis Studies (n) Number of participants (cases) Sensitivity (95% CI) Specificity (95% CI)
Published in peer‐reviewed journalsa 66 27812 (14078) 87.5% (95% CI 84.3 to 90.1) 78.0% (95% CI 72.9 to 82.4)

Abbreviations: CI: confidence interval; CT: computed tomography

aThe publication status of studies has been updated as of 17 February 2021.

Investigations of heterogeneity

Investigations of heterogeneity found that reference standard conduct did not have an impact on accuracy of chest CT. Definition for index test positivity impacted the sensitivity, but not specificity, of chest CT. Definition for index test positivity did not impact the accuracies of chest X‐ray or ultrasound. The results of the investigations of heterogeneity are outlined in Table 6.

5. Meta‐regression analyses for chest CT, X‐ray, and US of suspected cases.
Test, analysis group Studies (n) Number of participants (cases) Sensitivity (95% CI) Specificity (95% CI)
Reference standard conduct (chest CT)
RT‐PCR testing at least twice for all initial negative results 17 5515 (2665) 88.4% (95% CI 79.4 to 93.8) 72.7% (95% CI 62.0 to 81.3)
RT‐PCR testing not done twice for all initial negatives 39 19102 (9909) 86.9% (95% CI 82.9 to 90.2) 81.2% (95% CI 75.8 to 85.6)
P value     0.71 0.13
Definition for index test positivity (chest CT)
Radiologist impression 27 14266 (7307) 90.4% (95% CI 84.9 to 94.0) 72.4% (95% CI 62.8 to 80.3)
Formal scoring system 42 14019 (7035) 84.3% (95% CI 80.3 to 87.5) 81.5% (95% CI 76.8 to 85.4)
P value     0.037 0.070
Definition for index test positivity (chest X‐ray)
Radiologist impression 6 4489 (3246) 76.2% (62.5 to 85.9) 64.5% (44.0 to 80.8)
Formal scoring system 11 4040 (2057) 71.8% (59.7 to 81.4) 77.7% (65.0 to 86.7)
P value     0.60 0.24
Definition for index test positivity (chest US)
Radiologist impression 9 1704 (974) 88.6% (95% CI 77.9 to 94.4) 73.8% (95% CI 49.0 to 89.1)
Formal scoring system 6 706 (208) 80.7% (95% CI 74.3 to 85.9) 79.9% (95% CI 64.8 to 89.6)
P value     0.12 0.62

Abbreviations: CI: confidence interval; CT: computed tomography; US: ultrasound ; RT‐PCR: reverse transcription polymerase chain reaction.

Stratification by reference standard for chest CT studies resulted in pooled sensitivity of 88.4% (95% CI 79.4 to 93.8) for studies that performed RT‐PCR testing at least twice for all participants with initial negative results versus 86.9% (95% CI 82.9 to 90.2) for studies that did not perform twice for all participants with initial negative results versus (P = 0.71). Pooled specificity estimates were 72.7% (95% CI 62.0 to 81.3) for studies that performed RT‐PCR testing at least twice for all participants with initial negative results versus 81.2% (95% CI 75.8 to 85.6) for studies that did not perform repeat RT‐PCR testing for all participants with initial negative results (P = 0.13).

Stratification by definition used for index test positivity for chest CT studies gave pooled sensitivity estimates of 90.4% (95% CI 84.9 to 94.0) for studies that defined index test positivity based on radiologist's impressions versus 84.3% (95% CI 80.3 to 87.5) for studies that used a formal scoring system to define index test positivity (P = 0.037). Pooled specificity estimates were 72.4% (95% CI 62.8 to 80.3) for studies that used radiologist's impressions versus 81.5% (95% CI 76.8 to 85.4) for studies that used a formal scoring system (P = 0.070). For studies that used a formal scoring system, we used the threshold demonstrating the highest Youden’s index in each study (or as in the cases of two studies that did not report data at all thresholds, the only threshold that was available) in the analysis.

Stratification by definition used for index test positivity for chest X‐ray studies gave pooled sensitivity estimates of 76.2% (95% CI 62.5 to 85.9) for studies that defined index test positivity based on radiologist's impressions versus 71.8% (95% CI 59.7 to 81.4) for studies that used a formal scoring system to define index test positivity (P = 0.60). Pooled specificity estimates were 64.5% (95% CI 44.0 to 80.8) for studies that used radiologist's impressions versus 77.7% (95% CI 65.0 to 86.7) for studies that used a formal scoring system (P = 0.24).

Stratification by definition used for index test positivity for ultrasound studies gave pooled sensitivity estimates of 88.6% (95% CI 77.9 to 94.4) for studies that defined index test positivity based on radiologist's impressions versus 80.7% (95% CI 74.3 to 85.9) for studies that used a formal scoring system to define index test positivity (P = 0.12). Pooled specificity estimates were 73.8% (95% CI 49.0 to 89.1) for studies that used radiologist's impressions versus 79.9% (95% CI 64.8 to 89.6) for studies that used a formal scoring system (P = 0.62).

Threshold effects (CO‐RADS)

Eleven studies that evaluated CT used the CO‐RADS scoring system to define index test positivity. We obtained the 2x2 data at all five CO‐RADS thresholds for nine studies; two studies only reported 2x2 data at a CO‐RADS threshold of 3, and the authors could not provide any additional data. The forest plots of chest CT studies that used CO‐RADS and reported 2x2 data for CO‐RADS thresholds >=2, >=3, >=4 and = 5 are presented in Figure 9Table 7 and Figure 10 summarize the results.

9.

9

Forest plot of chest CT studies in suspected cases that used the CO‐RADS scoring system at varying thresholds: A) CO‐RADS 5, B) CO‐RADS 4, C) CO‐RADS 3, and D) CO‐RADS 2.

6. Analyses of ‘threshold’ effects for chest CT studies of suspected cases that used the COVID‐19 Reporting and Data System (CO‐RADS).
CO‐RADS threshold Studies (n) Number of participants (cases) Sensitivity (95% CI) Specificity (95% CI)
5 9 4169 (1672) 67.3% (95% CI 57.9 to 75.6) 92.2% (95% CI 89.3 to 94.3)
4 9 4169 (1672) 83.3% (95% CI 76.1 to 88.7) 84.0% (95% CI 81.3 to 86.4)
3 11 4416 (1769) 90.3% (95% CI 85.9 to 93.5) 69.7% (95% CI 64.3 to 74.6)
2 9 4169 (1672) 94.0% (95% CI 89.8 to 96.6) 45.4% (95% CI 38.4 to 52.5)
1a

Abbreviations: CI: confidence interval; CT: computed tomography.

aMeta‐analysis was not performed for a CO‐RADS threshold of 1 since at this threshold all sensitivity values are equal to one, and all specificity values are equal to zero.

10.

10

Pooled sensitivity and specificity estimate and 95% confidence intervals at varying CO‐RADS thresholds: CO‐ RADS 2 (n = 9), CO‐RADS 3 (n = 11), CO‐RADS 4 (n = 9), and CO‐RADS 5 (n = 9).

  • At a CO‐RADS threshold of 5 (9 studies), the sensitivity ranged from 42% to 80% and the specificity ranged from 84% to 99%; the pooled sensitivity was 67.3% (95% CI 57.9 to 75.6) and the pooled specificity was 92.2% (95% CI 89.3 to 94.3).

  • At a CO‐RADS threshold of 4 (9 studies), the sensitivity ranged from 56% to 90% and the specificity ranged from 68% to 91%; the pooled sensitivity was 83.3% (95% CI 76.1 to 88.7) and the pooled specificity was 84.0% (95% CI 81.3 to 86.4).

  • At a CO‐RADS threshold of 3 (11 studies), the sensitivity ranged from 65% to 95% and the specificity ranged from 54 % to 87%; the pooled sensitivity was 90.3% (95% CI 85.9 to 93.5) and the pooled specificity was 69.7% (95% CI 64.3 to 74.6).

  • At a CO‐RADS threshold of 2 (9 studies), the sensitivity ranged from 75% to 100% and the specificity ranged from 11% to 57%; the pooled sensitivity was 94.0% (95% CI 89.8 to 96.6) and the pooled specificity was 45.4% (95% CI 38.4 to 52.5).

  • We did not perform meta‐analysis for a CO‐RADS threshold of 1, since at this threshold, all sensitivity values are equal to 1, and all specificity values are equal to 0.

Threshold effects (RSNA)

Five studies that evaluated CT used the RSNA scoring system to define index test positivity. We obtained the 2x2 data at all four RSNA thresholds for four studies; one study did not report 2x2 data at a RSNA threshold of 1 or 2, and the authors could not provide any additional data. The forest plots of chest CT studies that used RSNA and reported 2x2 data for RSNA thresholds 2, 3, and 4 are presented in Figure 11. Table 8 and Figure 12 summarize the results.

11.

11

Forest plot of chest CT studies in suspected cases that used the RSNA scoring system at varying thresholds: A) RSNA 4, B) RSNA 3, and C) RSNA 2.

7. Analyses of ‘threshold’ effects for chest CT studies of suspected cases that used the RSNA Reporting and Data System.
RSNA threshold Studies (n) Number of participants (cases) Sensitivity (95% CI) Specificity (95% CI)
4 5 1162 (601) 68.9% (47.1 to 84.7) 90.1% (79.4 to 94.4)
3 5 1162 (601) 87.6% (69.4 to 95.7) 63.4% (57.1 to 69.2)
2 4 1071 (576) 91.6% (67.1 to 98.3) 27.9% (17.0 to 42.1)
1a

Abbreviations: CI: confidence interval; CT: computed tomography.

aMeta‐analysis was not performed for a RSNA threshold of 1 since at this threshold all sensitivity values are equal to one, and all specificity values are equal to zero.

12.

12

Pooled sensitivity and specificity estimate and 95% confidence intervals at varying RSNA thresholds: RSNA 3 (n = 4), and RSNA 4 (n = 4).

  • At an RSNA threshold of 4 (5 studies), the sensitivity ranged from 34% to 88% and the specificity ranged from 74% to 97%; the pooled sensitivity was 68.9% (95% CI 47.1 to 84.7) and the pooled specificity was 90.1% (95% CI 79.4 to 94.4).

  • At an RSNA threshold of 3 (5 studies), the sensitivity ranged from 50% to 97% and the specificity ranged from 57% to 80%; the pooled sensitivity was 87.6% (95% CI 69.4 to 95.7) and the pooled specificity was 63.4% (95% CI 57.1 to 69.2).

  • At an RSNA threshold of 2 (4 studies), the sensitivity ranged from 55% to 100% and the specificity ranged from 10.7% to 43.6%; the pooled sensitivity was 91.6% (95% CI 67.1 to 98.3) and the pooled specificity was 27.9% (95% CI 17.0 to 42.1).

  • We did not perform meta‐analysis for a RSNA threshold of 1, since at this threshold, all sensitivity values are equal to 1, and all specificity values are equal to 0.

Indirect test comparisons in suspected individuals

Indirect comparisons of modalities evaluated across all 94 studies in suspected participants indicated that chest CT (69 studies) and ultrasound (15 studies) gave higher sensitivity estimates than X‐ray (P = 0.0003 and P = 0.001, respectively). Chest CT and ultrasound gave similar sensitivities (P = 0.42). All modalities had similar specificities (CT versus X‐ray P = 0.36; CT versus ultrasound P = 0.32; X‐ray versus ultrasound P = 0.89).

Pooled rates of positive imaging in individuals with initial RT‐PCR negative results

For rate of positive imaging in repeat RT‐PCR positive results (where initial RT‐PCR was negative), we included eight studies for rate of positive imaging in repeat RT‐PCR positive results (7 CT, 1 ultrasound) with a total of 198 participants suspected of having COVID‐19, who had an initial negative RT‐PCR test result, and a positive result on repeat RT‐PCR testing. For chest CT (7 studies, 177 participants), rate of positive imaging in repeat RT‐PCR positive results (where initial RT‐PCR was negative) ranged from 21% to 100%, and the pooled rate was 75.8% (95% CI 45.3 to 92.2). For ultrasound of the lungs (one study, 21 participants), the sensitivity was 90.4%. The forest plot of chest CT studies for repeat RT‐PCR positive results where initial RT‐PCR was negative is presented in Figure 13.

13.

13

Forest plot of positive chest CT imaging in participants with repeat RT‐PCR positive results where initial RT‐PCR was negative. N positive = number of participants with an initial negative RT‐PCR test and a positive result on repeat RT‐PCR testing, who had chest CT imaging positive for COVID‐19. N negative = number of participants with an initial negative RT‐PCR test result and a positive result on repeat RT‐PCR testing, who had chest CT imaging negative for COVID‐19. Rate = N positive / (N positive + N negative).

Pooled estimates in asymptomatic individuals

We included 10 studies for imaging asymptomatic individuals (7 CT, 1 X‐ray, 2 ultrasound).

For chest CT (7 studies, 3134 participants, 315 (10%) cases), the sensitivity ranged from 20.7% to 80%, and specificity ranged from 68.4% to 100%. The pooled sensitivity of chest CT was 55.7% (95% CI 35.4 to 74.3) and the pooled specificity was 91.1% (95% CI 82.6 to 95.7). For chest X‐ray (one study, 85 participants, 4 cases) the sensitivity was 75.0% and the specificity was 74.0%. For ultrasound of the lungs (2 studies, 329 participants, 45 cases) the sensitivity was 50.0% and 69.7%, and specificity was 98.8% and 68.0%, respectively. The SROC and forest plots of chest CT studies for asymptomatic screening are presented in Figure 14 and Figure 15.

14.

14

Forest plot of positive chest CT imaging in asymptomatic participants.

15.

15

Summary ROC plot of chest CT in asymptomatic cases.The summary point is indicated by the solid black circle, individual studies are indicated by outlined circles (scale=study sample size). The dotted border and the dashed border represent 95% confidence regions and 95% prediction regions, respectively.

Changes across review versions

Figure 16 displays the pooled sensitivity and specificity estimates with 95% CIs from all four versions of this review (i.e. Salameh 2020a published in September 2020, Islam 2020 published in November 2020, Islam 2021 published in March 2021, and this current version). The sensitivity estimates of chest CT appear to be similar across McInnes 2020, Islam 2020, Islam 2021 and this current version, while the specificity estimates of chest CT appear to increase from Salameh 2020a to Islam 2021, and then remain similar between version 3 and the current version.

16.

16

Pooled sensitivity and specificity estimate and 95% confidence intervals across all review versions (Salameh 2020a (Version 1); Islam 2020 (Version 2); Islam 2021 (Version 3); and this review update version (Version 4)) for chest CT, chest X‐ray and ultrasound of the lungs.

With respect to chest X‐ray, which was evaluated only in Islam 2021 and the current version, the specificities appear to be similar, while the sensitivity appears to slightly increase in the current version. With respect to ultrasound of the lungs, which was evaluated only in Islam 2021 and the current version, the sensitivities appear to be similar, while the specificity appears to increase in the current version.

Discussion

This is the fourth version of a Cochrane living systematic review evaluating the diagnostic accuracy of thoracic imaging (computed tomography (CT), chest X‐ray and ultrasound) in the evaluation of people suspected to have COVID‐19. This version of the review is based on published studies and preprints up to 17 February 2021.

Summary of main results

Chest CT (69 studies, 28,285 participants, 14342 (51%) cases) demonstrated a sensitivity of 86.9% (95% CI 83.6 to 89.6), and a specificity of 78.3% (95% CI 73.7 to 82.3) for the diagnosis of COVID‐19 in suspected participants. Compared with the findings of Islam 2021 in which we determined that chest CT had a sensitivity of 87.9% (95% CI 84.6 to 90.6), and specificity of 80.0% (95% CI 74.9 to 84.3), our current update demonstrates similar sensitivity and specificity of chest CT for diagnosing suspected patients. It should be mentioned that changes to inclusion criteria mean that while summary results are not vastly different, confidence in results has further improved on the prior version.

There was no statistical evidence of the effect of reference standard conduct on the sensitivity or specificity of chest CT; studies that performed reverse transcriptase polymerase chain reaction (RT‐PCR) testing at least twice for all initial negative results and studies that did not perform repeat RT‐PCR testing for all initial negative results had similar sensitivities and specificities. These findings align with those of Salameh 2020a, Islam 2020 and Islam 2021.

The definition used for index test positivity in chest CT studies appeared to impact sensitivity not specificity, as studies that used radiologists' impressions showed higher sensitivities than those that used formal scoring systems. A possible explanation is that a 'threshold effect' seems to apply to the different definitions for index test positivity. Thus, there are differences in the interpretation of chest CT between the formal scoring system and radiologist impression groups.

Chest X‐ray (17 studies, 8529 participants with 5303 (62%) cases) demonstrated a sensitivity of 73.1% (95% CI 64.1 to 80.5),and a specificity of 73.3% (95% CI 61.9 to 82.2) for the diagnosis of COVID‐19 in suspected participants. Compared to Islam 2021, the specificities appear to be similar, while the sensitivity appears to slightly increase in the current version.

Ultrasound of the lungs (15 studies, 2410 participants with 1158 (49%) cases) demonstrated a sensitivity of 88.9% (95% CI 84.9 to 92.0), and a specificity of 72.2% (95% CI 58.8 to 82.5). Compared to Islam 2021, the sensitivities appear to be similar, while the specificity appears to increase in the current version.

Threshold effects (CO‐RADS and RSNA)

In chest CT studies that used the CO‐RADS scoring system to define index test positivity (11 studies), as expected, when the threshold for index test positivity increased (i.e. from 2 to 5), sensitivity decreased and specificity increased. The same pattern can be seen for the RSNA scoring system. In chest CT studies that used the RSNA scoring system to define index test positivity (5 studies), when the threshold for index test positivity increased (i.e. from 2 to 4), sensitivity decreased and specificity increased.

Indirect test comparisons

Based on indirect comparisons of all included studies, chest CT and ultrasound gave higher sensitivity estimates than X‐ray. Chest CT and ultrasound gave similar sensitivities. All modalities had similar specificities.

Rate of positive imaging in individuals with initial RT‐PCR negative results

The pooled rate of positive chest CT imaging (7 studies, 177 participants all of whom had a final diagnosis of COVID‐19) in repeat RT‐PCR positive results where initial RT‐PCR was negative, was 75.8% (95% CI 45.3 to 92.2). We were unable to derive pooled rates for X‐ray and ultrasound due to insufficient available data.

Asymptomatic screening

Chest CT (8 studies, 3548 participants, 364 (10%) cases) demonstrated a sensitivity of 55.7% (95% CI 35.4 to 74.3), and a specificity of 91.1% (95% CI 82.6 to 95.7) for detecting COVID‐19 in asymptomatic participants. We were unable to derive pooled accuracy estimates for screening with X‐ray and ultrasound due to insufficient available data. Our findings show that imaging is not useful for screening asymptomatic patients.

Changes across review versions

Based on the visual assessments of the ggplot graphs, with respect to the four versions of this review, the sensitivity estimates of chest CT appear to remain similar across Salameh 2020a, Islam 2020, Islam 2021, and this current version, while the specificity estimates of chest CT appear to increase with Islam 2020 and Islam 2021. However, the specificity estimates of chest CT appear to remain similar between Islam 2021 and current versions. Given the large number of chest CT studies included in the prior review, which provided sensitivity and specificity estimates with narrow confidence intervals, we had expected that sensitivity and specificity estimates of chest CT will not notably differ in future updates of this review. The results of the current review align with this expectation.

For chest X‐ray, the specificities between Islam 2021 and this current version appear to be similar, while the sensitivity appears to have slightly increased in the current version. For ultrasound of the lungs the sensitivities between Islam 2021 and this current version appear to be similar, while the specificity appears to have increased in the current version.

Strengths and weaknesses of the review

Our search strategy was broad and allowed for identification of a wide range of articles about COVID‐19 diagnosis. The review authors screened records, extracted data, and assessed study methodology independently and in duplicate. Though we are relatively confident in the accuracy and completeness of our findings, please inform us at mmcinnes@toh.ca should errors be found so that we can address them in a future update. Furthermore, compared to Salameh 2020a, Islam 2020, and Islam 2021, this current update includes a greater number of studies that evaluated accuracy estimates of imaging tests in the diagnosis of suspected COVID‐19 participants.

We included studies that involved only symptomatic participants, as well as studies that had a mixed population (i.e. symptomatic and asymptomatic participants). Thus, there may be situations when asymptomatic individuals are suspected of having COVID‐19, such as if they have infected contacts or other risk factors for infection. However, not all the studies clearly reported information on participants’ symptoms.

We identified that how index test positivity is defined impacts on chest CT sensitivity but not any other modality. These findings may suggest that the variables we investigated did not significantly contribute to variability; alternatively, there may be unmeasured confounding variables blurring our analyses. Due to insufficient granularity of data, we were unable to investigate additional potential sources of variability, particularly participant setting (inpatient versus outpatient). We plan to perform these analyses in future updates, when sufficient data become available.

In this update, we addressed additional objectives of evaluating the rate of positive imaging in repeat RT‐PCR positive results where initial RT‐PCR was negative. Furthermore, we evaluated the diagnostic accuracy of thoracic imaging (CT, chest X‐ray and ultrasound) in asymptomatic individuals.

We explored indirect comparisons of chest CT, chest X‐ray and ultrasound of the lungs. Due to the limited number of studies that evaluated multiple imaging modalities in the same population, we did not formally evaluate direct comparisons of different imaging tests at this stage. We plan to conduct formal analyses of direct comparisons of imaging tests in future updates, as more studies with comparative designs become available.

We were not able to evaluate accuracy estimates based on specific findings of imaging tests (e.g. ground‐glass, consolidation, pleural effusion) or combinations of such findings because of the lack of data granularity reported in included studies; however, we will consider this in future updates of the review.

We hope that in future versions of this review we will be able to evaluate these associations as research on the role of imaging tests in the diagnosis of COVID‐19 evolves. It should be noted that any association between number of days after symptom onset, symptom severity and the findings on chest imaging for patients with COVID‐19 might impact the diagnostic performance of chest CT in the future versions.

The quality of the primary studies included in this review continues to impact the overall robustness of the review. Several studies failed to describe their participants (e.g. recruitment method), the details of reference standard conduct used for identifying COVID‐19 cases, and the definition used for positivity of the imaging tests. In this version, half of all studies seemed to have low risk of bias data, while, in Islam 2021, most were high or unclear.

Of the studies that did report recruitment methods, most reported including ‘consecutive’ participants. However, many of these studies did not actually recruit ‘consecutive’ participants that represent the target population (i.e. individuals suspected of having COVID‐19), but instead included all consecutive participants that underwent an imaging test and RT‐PCR testing. These studies did not describe whether all suspected patients in the recruitment setting underwent both an imaging test and RT‐PCR as a part of standard practice (which would result in a true ‘consecutive’ recruitment), or whether imaging tests were only performed in patients with specific clinical signs (e.g. severe symptoms). In studies where the latter situation is present, included participants may not represent the target population, and this could create selection bias.

We recommend that the accuracy estimates reported in this review are interpreted with caution because of the use of RT‐PCR as the reference standard. The results of RT‐PCR are not always sensitive, and it is possible that chest CT may be more sensitive than the reference standard in some patients. However, our investigations of heterogeneity for chest CT studies did not identify different accuracy estimates between studies that used at least two RT‐PCR test results to define disease‐negative status versus studies that used only one RT‐PCR test result to define disease‐negative status. At this stage, despite its limitations, RT‐PCR remains the best tool for diagnosing COVID‐19. However, the best reference standard may vary across clinical questions, settings, and populations (Korevaar 2020).

In future updates of this review, we may consider the use of a latent‐class bivariate model for meta‐analysis, which adjusts for the imperfect accuracy of the reference standard (Butler‐Laporte 2021).

Three out of 98 included studies (3%) were only available as preprints at the time of the search. We will update data extracted from these studies in future versions of our review as these studies become published in peer‐reviewed journals.

Applicability of findings to the review question

As the studies in our cohort included suspected COVID‐19 participants, our findings are applicable to individuals suspected to have COVID‐19. Our search did not identify many studies that evaluated the accuracy of chest CT, ultrasound of the lungs, and chest X‐ray for the diagnosis of COVID‐19 in paediatric populations. Thus, the diagnostic accuracy of these modalities in children is not as well‐established. In addition, the lack of data available in the included studies pertaining to signs and symptoms of presenting cases, the severity of the symptoms, as well as timing of symptom onset adds complexity to the interpretation of the findings in this review. It should be noted that the results apply mostly to imaging interpreted by radiologists.

Authors' conclusions

Implications for practice.

Our findings indicate that chest computed tomography (CT), chest X‐ray and ultrasound all give higher proportions of positive results for individuals with COVID‐19 as compared to those without. For chest CT, the chances of getting a positive result are 86.9% (95% CI 83.6 to 89.6) in individuals with COVID‐19 and 21.7% (95% CI 17.7 to 26.3) in those without. For chest X‐ray, the chances of getting a positive result are 73.1% (95% CI 64.1 to 80.5) in individuals with COVID‐19 and 26.7% (95% CI 17.8 to 38.1) in those without.

For ultrasound of the lungs, the chances of getting a positive result are 88.9% (95% CI 84.9 to 92.0) in individuals with COVID‐19 and 23.7% (95% CI 13.3 to 33.8) in those without. Due to the limited availability of data, accuracy estimates of chest X‐ray and ultrasound of the lungs for the diagnosis of COVID‐19 in suspected participants should be carefully interpreted.

Implications for research.

From our current pool of included studies, we can draw limited conclusions regarding the diagnostic performance of thoracic imaging modalities. Additional studies evaluating the accuracy of chest X‐ray and ultrasound of the lungs for diagnosis COVID‐19 in suspected patients are needed to allow for more reliable findings.

In this update, we were unable to assess several objectives of interest due to the lack of available data required to formally evaluate direct comparisons of different imaging modalities, and the effect of time since onset of symptoms on the diagnostic performance of various index tests. Future studies should ideally pre‐define positive imaging findings and include direct comparisons of the various modalities of interest on the same participant population in order to provide robust and reliable data. Furthermore, improved transparency and reporting is necessary for more efficient data extraction in our updated versions of this review. We encourage authors and investigators to refer to the STARD 2015 checklist (Bossuyt 2015; Hong 2018) to ensure that any relevant information is clearly reported in their studies. Also, the uncertainty resulting from high or unclear risk of bias of included studies limit our ability to confidently draw conclusions based on our results.

We hope that future updates of this review include more informative studies to allow for additional investigations of variability with improved power and further evaluations of additional objectives.

What's new

Date Event Description
27 May 2022 Amended Corrected minor typo in Abstract

History

Protocol first published: Issue 6, 2020
Review first published: Issue 9, 2020

Date Event Description
14 April 2022 New search has been performed The author team updated the date of search to 17 February 2021, and included all new studies identified. Changes to methods in this review update version are outlined in the 'Differences between protocol and review' section.
14 April 2022 New citation required and conclusions have changed The results for chest X‐ray and ultrasound have changed.
10 March 2021 New citation required and conclusions have changed The results for chest X‐ray and ultrasound have changed.
9 February 2021 New search has been performed This is a 'living' systematic review'; searches are run and screened every few months. The last search date was 30 September 2020. Results of all new studies identified have been incorporated. The conclusions of this Cochrane Review are therefore considered up to date.
23 October 2020 New search has been performed This is a 'living' systematic review'; searches are run and screened monthly. The last search date was 22 June 2020. Results of all new studies identified have been incorporated. The conclusions of this Cochrane Review are therefore considered up to date.
23 October 2020 New citation required and conclusions have changed The results for chest computed tomography (CT) have changed.

Acknowledgements

Members of the Cochrane COVID‐19 Diagnostic Test Accuracy Review Group include the following.

  • The project team (Deeks JJ, Dinnes J, Takwoingi Y, Davenport C, Leeflang MMG, Spijker R, Hooft L, Van den Bruel A, McInnes MDF, Emperador D, Dittrich S);

  • The systematic review teams for each review

    • Molecular, antigen, and antibody tests (Adriano A, Beese S, Dretzke J, Ferrante di Ruffano L, Harris I, Price M, Taylor‐Phillips S)

    • Signs and symptoms (Stuyf T, Domen J, Horn S)

    • Routine laboratory markers (Yang B, Langendam M, Ochodo E, Guleid F, Holtman G, Verbakel J, Wang J, Stegeman I)

    • Imaging tests (Islam N, Ebrahimzadeh S, Dawit H, Salameh JP, Kazi S, Fabiano N, Treanor L, Absi M, Ahmad F, Rooprai P, Al Khalil A, Harper K, Kamra N, Van Der Pol CB, Prager R, Hare SS, Dennie C, Jenniskens K, Korevaar DA, Cohen JF, van de Wijgert J, Wang J, Pena E, Sabongui S)

  • The wider team of systematic reviewers from University of Birmingham, UK who assisted with title and abstract screening across the entire suite of reviews for the diagnosis of COVID‐19 (Agarwal R, Baldwin S, Berhane S, Herd C, Kristunas C, Quinn L, Scholefield B).

We thank Dr Jane Cunningham (World Health Organization) for participation in technical discussions and comments on the manuscript.

The Cochrane Infectious Diseases Group (CIDG) managed the editorial process on this review update version. The CIDG Editor was Professor Mical Paul, DTA Editor was Professor Gianni Virgili (Contact Editor), and the Sign‐off Editor was Professor Paul Garner. We thank Heather Maxwell (Cochrane Copy Edit Support) who copyedited this review update version.

For previous published review versions, we thank the Cochrane Central Editorial Service for managing the editorial process (Salameh 2020a; Islam 2020; Islam 2021). We thank Helen Wakeford and Anne‐Marie Stephani (Managing Editors, Central Editorial Service, Cochrane); Gianni Virgili (Contact Editor), Sophie Beese and Bella Harris (Managing Editors) and Marta Roqué (statistical peer reviewer) from the Cochrane Diagnostic Test Accuracy Reviews Editorial Team; Robin Featherstone (Central Editorial Service, Cochrane) for search peer review; peer reviewers Paul Garner and Robert Walton; consumer referee Shirley Hall; and Denise Mitchell (Central Editorial Service, Cochrane).

The CIDG editorial base is funded by UK aid from the UK government for the benefit of low‐ and middle‐income countries (project number 300342‐104). The views expressed do not necessarily reflect the UK government’s official policies.

Jonathan Deeks is a UK National Institute for Health Research (NIHR) Senior Investigator Emeritus. Yemisi Takwoingi is supported by a NIHR Postdoctoral Fellowship. Jonathan Deeks, Jacqueline Dinnes, and Yemisi Takwoingi are supported by the NIHR Birmingham Biomedical Research Centre. The views expressed are those of the authors and not necessarily those of the NHS, the NIHR, or the Department of Health and Social Care.

Appendices

Appendix 1. Glossary

Terminology/acronyms

  • COVID‐19: coronavirus disease 2019, the clinical manifestations/symptoms caused by infection with SARS‐CoV‐2, name given to the disease associated with the virus SARS‐CoV‐2

  • COVID‐19 pneumonia: COVID‐19 that presents as infection‐inflammation of the lungs

  • Index test: the test that is being assessed (the index test will often be a new test)

  • False negative: the test does not detect a condition in someone when it is present

  • False positive: the test detects a condition in someone when it is not present

  • Negative predictive value: the probability that someone who has tested negative for the target condition with the index test will really not have it (a true negative)

  • Positive predictive value: the probability that someone who has tested positive for the target condition with the index test will actually have it (a true positive)

  • Reference standard: the most reliable method for determining if the target condition is present or absent, used to verify index test results. This could be a combination of tests.

  • RT‐PCR: reverse transcription polymerase chain reaction (RT‐PCR) is a laboratory technique that combines reverse transcription of RNA into DNA and amplification of specific DNA targets using polymerase chain reaction. In this context it is used to detect the presence of SARS‐CoV‐2 RNA

  • SARS‐CoV‐2: severe acute respiratory syndrome coronavirus 2, the name given to the 2019 novel coronavirus

  • SARS‐CoV‐2 infection: people infected with severe acute respiratory syndrome coronavirus 2, but who may or may not have any clinical manifestations of infection

  • Secondary care: medical care that is provided by a specialist or facility upon referral by a primary care physician and that requires more specialized knowledge, skill, or equipment than the primary care physician can provide

  • Sensitivity: the proportion of people with the target condition (with disease) that are correctly identified by the index test

  • Specificity: the proportion of people without the target condition (without disease) that are correctly identified by the index test

  • Tertiary care: specialized care, usually for inpatients and on referral from a primary or secondary health professional, in a facility that has personnel and facilities for advanced medical investigation and treatment

  • Target condition: the disease or condition of interest

  • True negative: a correct diagnosis of a condition being absent

  • True positive: a correct diagnosis of a condition being present

Appendix 2. QUADAS‐2

QUADAS‐2
Index test(s): Imaging studies of the chest (computed tomography (CT), chest X‐ray and ultrasound) for diagnosis of COVID‐19
Participants (setting, intended use of index test, presentation, prior testing): People with suspected COVID‐19
All settings, in particular secondary care, emergency care and ICUs
In people presenting with suspected COVID‐19; suspicion may be based on prior testing, such as general lab testing.
Signs and symptoms often used for triage or referral
Reference standard and target condition: A positive diagnosis for COVID‐19 by the following.
  1. A positive reverse transcriptase polymerase chain reaction (RT‐PCR) test for SARS‐CoV‐2 infection, from any manufacturer in any country, from any source, including nasopharyngeal swabs or aspirates, oropharyngeal swabs, bronchoalveolar lavage fluid (BALF), sputum, saliva, serum, urine, rectal or faecal samples.

  2. Positive on WHO criteria for COVID‐19 which includes some testing RT‐PCR negative.

  3. Positive on China CDC criteria for COVID‐19 which includes some testing RT‐PCR negative.

  4. Positive serology in addition to consistent symptomatology.

  5. Positive on study specific list of criteria for COVID‐19 which includes some testing RT‐PCR negative.

  6. Other criteria (symptoms, imaging findings, other tests).


A negative diagnosis for COVID‐19 by the following.
  1. COVID suspects with negative RT‐PCR test results, whether tested once or more than once.

  2. Current healthy or with another disease (no RT‐PCR test).


This list is not exhaustive, as we anticipate that studies will use a variety of reference standards and we plan to include all of them, at least for Salameh 2020a, Islam 2020, and Islam 2021. Although RT‐PCR is considered the best available test, it is suspected of missing a substantial proportion of cases, and thus may not be the ideal reference standard if used as a standalone test (Li 2020b; Loeffelholz 2020). Therefore, we are likely to use alternative reference standards, such as a combination of RT‐PCR, and symptoms or imaging findings, or both.
We will judge how likely each reference standard definition is to correctly classify individuals in the assessment of methodological quality. All reference standards are likely to be imperfect in some way; details of reference standard evaluation are provided in the 'Risk of bias' tool below. We will use a consensus process to agree the classification of the reference standard as to what we regard as good, moderate and poor. 'Good' reference standards need to have very little change of misclassification, 'moderate', a small but acceptable risk, 'poor', a larger and probably unacceptable risk.
Participant selection
Was a consecutive or random sample of patients enrolled? YES: if a study explicitly states that all participants within a certain time frame were included; that this was done consecutively; or that a random selection was done.
NO: if it is clear that a different selection procedure was employed; e.g. selection based on clinician’s preference, or based on institutions (i.e. ‘convenience’ series)
UNCLEAR: if the selection procedure is not clear or not reported at all.
Was a case‐control design avoided? YES: if a study explicitly states that all participants came from the same group of (suspected) patients.
NO: if it is clear that a different selection procedure was employed for the participants depending on their COVID‐19 status (e.g. proven infected patients in one group and proven non‐infected patients in the other group).
UNCLEAR: if the selection procedure is not clear or not reported at all.
Did the study avoid inappropriate in‐ or exclusions? This needs to be addressed on a case‐to‐case basis.
YES: if all eligible patients were more or less equally suspected of having COVID‐19 and were included and if the numbers in the flow chart show not too many excluded participant (a maximum of 20% of eligible patients excluded without reasons).
NO: if over 20% of eligible patients were excluded without providing a reason; if only proven patients were included, or only proven non‐patients were included; if in a retrospective study participants without index test or reference standard result were excluded; if exclusion was based on severity assessment post‐factum or comorbidities (cardiovascular disease, diabetes, immunosuppression). If the study oversampled patients with particular characteristics likely to affect estimates of accuracy.
UNCLEAR: if the exclusion criteria are not reported.
Could the selection of patients have introduced bias? HIGH: if one or more signalling questions were answered with NO, as any deviation from the selection process may lead to bias.
LOW: if all signalling questions were answered with YES.
UNCLEAR: all other instances
Is there concern that the included patients do not match
the review question?
This needs to be addressed on a case‐to‐case basis, based on the objective the included study answers to.
HIGH: if accuracy was assessed in a case‐control design, or the study was able to only estimate sensitivity or specificity.
LOW: any situation where imaging is generally available.
UNCLEAR: if a description about the participants is lacking.
For studies included for rate of positive imaging in repeat RT‐PCR+ results objective: Could the selection of patients have introduced bias? YES: if only some (and not all) included participants underwent repeat RT‐PCR testing, and it is clear that a non‐consecutive or non‐random selection procedure was employed; e.g. based on symptom status, or based on index test findings
NO: if participants who underwent repeat RT‐PCR testing were selected in a random or consecutive manner from the total included participants
UNCLEAR: if the selection method was unclearly reported.
Index tests
Were the index test results interpreted without knowledge of the results of the reference standard? YES: if blinding was explicitly stated or index test was recorded before the results from the reference standard were available
NO: if it was explicitly stated that the index test results were interpreted with knowledge of the results of the reference standard
UNCLEAR: if blinding was unclearly reported.
If a threshold was used, was it prespecified? YES: for any of these index tests it is highly unlikely that any numerical threshold is used. Still we expect studies to report their criteria for test‐positivity (e.g. the constellation of imaging findings used). If these criteria are reported in the methods section, we will score ‘YES’ for this question.
NO: if the optimal criterion for test‐positivity was based on the reported data (for example, different scores on a quantitative scoring system) we will score ‘NO’.
UNCLEAR: if the criteria for test positivity were not or unclearly reported.
Could the conduct or interpretation of the index test have introduced bias? HIGH: if one or more signalling questions were answered with NO.
LOW: if all signalling questions were answered with YES.
UNCLEAR: all other instances
Note: For studies that use formal scoring systems with clearly defined thresholds, even if the signalling question about using a ‘prespecified threshold’ is 'unclear' or ‘no’, this domain should not be considered as having a ‘unclear’ or ‘high’ risk of bias based on the aforementioned question.
Is there concern that the index test, its conduct, or
interpretation differ from the review question?
There is not a huge amount of variability from a technical perspective. Therefore, this question will probably be answered ‘LOW’ in all cases except when assessments are made using personnel not available in practice, or personnel not trained for the job, or using modalities that are uncommon in practice. We will consult expert clinicians on a case‐to‐case basis to judge this question.
Reference standard
Is the reference standard likely to correctly classify the target
condition?
YES: for COVID‐19: RT‐PCR, done by trained personnel, and repeated after a first negative RT‐PCR, following guidelines for confirmed cases and done with an assay targeting minimum 2 targets in the genes N, E, S or RdRP (one target even acceptable in zone with known transmission). To clarify, a low risk of bias reference standard for true negative would require 2 (or more) negative RT‐PCR results.
NO: any other test
UNCLEAR: if no reference standard was reported, or if it was just reported that RT‐PCR was done.
Were the reference standard results interpreted without
knowledge of the results of the index test?
YES: if it was explicitly stated that the reference standard results were interpreted without knowledge of the results of the index test, or if the result of the index test was obtained after the reference standard.
NO: if it was explicitly stated that the reference standard results were interpreted with knowledge of the results of the index test or if the index test was used to make the final diagnosis (incorporation bias).
UNCLEAR: if blinding was unclearly reported.
Could the conduct or interpretation of the reference standard have introduced bias? HIGH: if one or more signalling questions were answered with NO.
LOW: if all signalling questions were answered with YES.
UNCLEAR: all other instances
Note: For studies that use RT‐PCR testing as the reference standard, even if this signalling question about 'blinding' is 'unclear' or ‘no’, this domain should not be considered as having a ‘unclear’ or ‘high’ risk of bias based on the aforementioned question.
Is there concern that the target condition as defined by the reference standard does not match the review question? HIGH: there is a high concern regarding applicability of the reference standard if the reference standard actually measures a different target condition than the one we are interested in for the review. For example, if the diagnosis is only based on clinical picture, without excluding other possible causes of this clinical picture (e.g. other respiratory pathogens), then there is considerable concern that the reference standard is actually measuring something else than COVID‐19. In addition, a positive RT‐PCR only measures SARS‐CoV‐2 infection and not COVID‐19 and therefore the reference standard for COVID‐19 is a combination of positive RT PCR and symptoms and/or imaging findings.
LOW: if above situations not present
UNCLEAR: if intention for testing is not reported in the study
Flow and timing
Was there an appropriate interval between index test(s)
and reference standard?
YES: as the situation of a patient, including clinical presentation and disease progress, evolves rapidly and new/ongoing exposure can result in case status change. On the other hand, negative PCR results need to be repeated for several days. Therefore, an appropriate time interval will be within 7 days.
NO: if there is more than 7 days between the index test and the reference standard or if patients are otherwise reported to be assessed with the index versus reference standard test at moments of different severity.
UNCLEAR: if the time interval is not reported
Did all participants receive a reference standard? YES: if all patients received a reference standard (clearly no partial verification)
NO: if only (part of) the index test positives or index test negatives received the complete reference standard
UNCLEAR: if it is not reported.
Did all participants receive the same reference standard? YES: if all patients received the same reference standard (clearly no differential verification). Verification of negative PCR result with a second PCR measurement is considered to be one reference
standard.
NO: if (part of) the index test positives or index test negatives received a different reference standard
UNCLEAR: If it is not reported.
Were all participants included in the analysis? YES: if all included participants were included in the analyses as well
NO: if after the inclusion/exclusion process, participants were removed from the analyses for different reasons: no reference standard done, no index test done, intermediate results of both index test or reference standard, indeterminate results of both index test or reference standard, samples unusable.
UNCLEAR: If this is not clear from the reported numbers.
Could the patient flow have introduced bias? HIGH: if one or more signalling questions were answered with NO, or if one question answered with NO was judged to have little impact on the methodological quality of the study (this should be justified in the scoring).
LOW: if all signalling questions were answered with YES.
UNCLEAR: all other instances

Abbreviations: CT: computed tomography; CXR: chest X‐ray; ICU: intensive care unit; RTPCR: reverse transcriptase polymerase chain reaction; SARSCoV2: severe acute respiratory syndrome coronavirus 2; US: ultrasound

Appendix 3. Search classification model

A more efficient approach was required to keep up with the rapidly increasing volume of COVID‐19 literature. A classification model for COVID‐19 diagnostic studies was built with the model building function within Eppi Reviewer, which uses the standard SGCClassifier in Scikit‐learn on word trigrams. As outputs, new documents receive a percentage (from the predict_proba function) where scores close to 100 indicate a high probability of belonging to the class ‘relevant document’ and scores close to 0 indicate a low probability of belonging to the class ‘relevant document’. We used three iterations of manual screening (title and abstract screening, followed by full‐text review) to build and test classifiers. The final included studies were used as relevant documents, while the remainder of the COVID‐19 studies were used as irrelevant documents. The classifier was trained on the first round of selected articles, and tested and retrained on the second round of selected articles. Testing on the second round of selected articles revealed poor positive predictive value but 100% sensitivity at a cut‐off of 10. The poor positive predictive value is mainly due to the broad scope of our topic (all diagnostic studies in COVID‐19), poor reporting in abstracts, and a small set of included documents. The model was retrained using the articles selected for the second and third rounds of screening, which added a considerable number of additional documents. This led to a large increase in positive predictive value, at the cost of a lower sensitivity, which led us to reduce the cut‐off to 5. The largest proportion of documents had a score between 0‐5. This set did not contain any of the relevant documents. This version of the classifier with a cut‐off 5 was used in subsequent rounds and accounted for approximately 80% of the screening burden.

Appendix 4. Search strategies

1. Living search from the University of Bern

27 April 2020

From 27 April 2020, we retrieved the curated bioRxiv/medRxiv dataset link

26 March 2020 to 27 April 2020

MEDLINE: (\"Wuhan coronavirus\" [Supplementary Concept] OR \"COVID‐19\" OR \"2019 ncov\"[tiab] OR ((\"novel coronavirus\"[tiab] OR \"new coronavirus\"[tiab]) AND (wuhan[tiab] OR 2019[tiab])) OR 2019‐nCoV[All Fields] OR (wuhan[tiab] AND coronavirus[tiab])))))

Embase: (nCoV or 2019‐nCoV or ((new or novel or wuhan) adj3 coronavirus) or covid19 or covid‐19 or SARS‐CoV‐2).mp

bioRxiv/medRxiv: ncov or corona or wuhan or COVID or SARS‐CoV‐2

With the kind support of the Public Health & Primary Care Library PHC, and following guidance of the Medical Library Association

01 January 2020 to 27 April 2020

MEDLINE: ("Wuhan coronavirus" [Supplementary Concept] OR "COVID‐19" OR "2019 ncov"[tiab] OR (("novel coronavirus"[tiab] OR "new coronavirus"[tiab]) AND (wuhan[tiab] OR 2019[tiab])) OR 2019‐nCoV[All Fields] OR (wuhan[tiab] AND coronavirus[tiab])))))

Embase: ncov OR (wuhan AND corona) OR COVID

bioRxiv/medRxiv: ncov or corona or wuhan or COVID

2. Cochrane COVID‐19 Study Register searches

Source Strategy
ClinicalTrials.gov COVID‐19 OR 2019‐nCoV OR SARS‐CoV‐2 OR 2019 novel coronavirus OR severe acute respiratory syndrome coronavirus 2 OR Wuhan coronavirus OR coronavirus
WHO International Clinical Trials Registry Platform We screen the entire COVID‐19.csv file available from
www.who.int/emergencies/diseases/novel-coronavirus-2019
PubMed (2019 nCoV[tiab] OR 2019nCoV[tiab] OR corona virus[tiab] OR corona viruses[tiab] OR coronavirus[tiab] OR coronaviruses[tiab] OR COVID[tiab] OR COVID19[tiab] OR nCov 2019[tiab] OR SARS‐CoV2[tiab] OR SARS CoV‐2[tiab] OR SARSCoV2[tiab] OR SARSCoV‐2[tiab] OR "Coronavirus"[Mesh:NoExp] OR "COVID‐19"[nm] OR "COVID‐19 drug treatment"[nm] OR "COVID‐19 diagnostic testing"[nm] OR "COVID‐19 serotherapy"[nm] OR "COVID‐19 vaccine"[nm] OR "LAMP assay"[nm] OR "severe acute respiratory syndrome coronavirus 2"[nm] OR "spike protein, SARS‐CoV‐2"[nm]) NOT ("animals"[mh] NOT "humans"[mh]) NOT (editorial[pt] OR newspaper article[pt])

3. CDC Library, COVID‐19 Research Articles Downloadable Database

Embase records from the Stephen B. Thacker CDC Library, Covid‐19 Research articles Downloadable database.

Records were obtained by the CDC Library by searching Embase through Ovid using the following search strategy.

Source Strategy
Embase (coronavir* OR corona virus* OR betacoronavir* OR covid19 OR covid 19 OR nCoV OR novel CoV OR CoV 2 OR CoV2 OR sarscov2 OR 2019nCoV OR wuhan virus*).mp. OR ((wuhan OR hubei OR huanan) AND (severe acute respiratory OR pneumonia*) AND outbreak*).mp. OR Coronavirus infection/ OR coronavirinae/ OR exp betacoronavirus/
Limits: 2020‐
OR
(novel coronavir* OR novel corona virus* OR covid19 OR covid 19 OR nCoV OR novel CoV OR CoV 2 OR CoV2 OR sarscov2 OR 2019nCoV OR wuhan virus*).mp. OR ((wuhan OR hubei OR huanan) AND (severe acute respiratory OR pneumonia*) AND outbreak*).mp. OR ((wuhan OR hubei OR huanan) AND (coronavir* OR betacoronavir*)).mp.
Limits: 2019‐

Data

Presented below are all the data for all of the tests entered into the review.

Tests. Data tables by test.

1. Test.

1

Chest CT in suspected cases

2. Test.

2

Chest X‐ray in suspected cases

3. Test.

3

Ultrasound of the lungs in suspected cases

4. Test.

4

CT CO‐RADS 2

5. Test.

5

CT CO‐RADS 3

6. Test.

6

CT CO‐RADS 4

7. Test.

7

CT CO‐RADS 5

8. Test.

8

RT‐PCR (Chest CT)

9. Test.

9

RT‐PCR (US of the lungs)

10. Test.

10

Asymptmotic (Chest CT)

11. Test.

11

Asymptomatic (X‐ray)

12. Test.

12

Asymptomatic (US of the lungs)

13. Test.

13

CT‐RSNA 2

14. Test.

14

CT‐RSNA 3

15. Test.

15

CT RSNA 4

Characteristics of studies

Characteristics of included studies [ordered by study ID]

Ai 2020a.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, unclear symptom status
Patient characteristics and setting Age group: adults only
Setting: unclear
Index tests Index test(s): chest CT
Definition for positive diagnosis on CT: unclear
Level of training of readers: radiologist
Prevalence: 0.6
Target condition and reference standard(s) Reference standard: RT‐PCR, no other details provided
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Unclear    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Unclear    
Could the selection of patients have introduced bias?   Unclear risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? No    
If a threshold was used, was it pre‐specified? Unclear    
Could the conduct or interpretation of the index test have introduced bias?   High risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Unclear    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Unclear risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Yes    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Low risk  

Aslan 2020.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: adults only
Setting: outpatient
Index tests Index test(s): chest CT (non‐contrast, low dose)
Definition for positive diagnosis on CT: radiological evidence of COVID‐19 pneumonia, including presence of ground glass opacity (GGO), mixed GGO (GGO and consolidation), consolidation, distribution and number of lobes and segment affected by GGO and/or consolidation, etc.
Level of training of readers: radiologist
Prevalence: 0.8
Target condition and reference standard(s) Reference standard: RT‐PCR twice, if necessary
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Unclear    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Yes    
Could the selection of patients have introduced bias?   Unclear risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Unclear    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Unclear risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Yes    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Low risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Yes    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Low risk  

Bahrami‐Motlagh 2020.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: adults only
Setting: unclear
Index tests Index test(s): chest CT (low dose CT)
Definition for positive diagnosis on CT: according to previous reports on typical and atypical CT findings of COVID‐19 pneumonia
Level of training of readers: unclear
Prevalence: 0.55
Target condition and reference standard(s) Reference standard: RT‐PCR, no further details provided or further details are unclear
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Yes    
Could the selection of patients have introduced bias?   Unclear risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Low risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Unclear    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Unclear risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Unclear
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Yes    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Low risk  

Barbosa 2020.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: adults only
Setting: unclear
Index tests Index test(s): chest CT, no further details provided
Definition for positive diagnosis on CT: RSNA classification
Level of training of readers: radiologist
Prevalence: 0.3
Target condition and reference standard(s) Reference standard: RT‐PCR, no other details provided
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Yes    
Could the selection of patients have introduced bias?   Low risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Unclear    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Unclear risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Unclear    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Unclear risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Yes    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Low risk  

Bellini 2020.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: children and adults
Setting: unclear
Index tests Index test(s): chest CT (non‐contrast)
Definition for positive diagnosis on CT: CO‐RADS
Level of training of readers: radiologist
Prevalence: 0.2
Target condition and reference standard(s) Reference standard: RT‐PCR once; twice in some; other (clinical signs on follow‐up)
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Yes    
Could the selection of patients have introduced bias?   Low risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? No    
Could the conduct or interpretation of the index test have introduced bias?   Low risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? No    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   High risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Yes    
Did all patients receive the same reference standard? Unclear    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Unclear risk  

Besutti 2020.

Study characteristics
Patient Sampling Study design: suspected patients, all symptomatic
Patient characteristics and setting Age group: adults, perhaps also children
Setting: outpatient
Index tests Index test(s): chest CT (non‐contrast)
Definition for positive diagnosis on CT: a structured report about the probability of COVID‐19 pneumonia
Level of training of readers: radiologist
Prevalence: 0.9
Target condition and reference standard(s) Reference standard: RT‐PCR once; twice in some
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Yes    
Could the selection of patients have introduced bias?   Low risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Low risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? No    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   High risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Yes    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Low risk  

Bock 2021.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: adults only
Setting: outpatient
Index tests Index test(s): ultrasound of the lungs (POCUS)
Definition for positive diagnosis on US:unclear
Level of training of readers: unclear
Prevalence: 0.43
Target condition and reference standard(s) Reference standard: RT‐PCR, no further details provided or further details are unclear
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Unclear    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Yes    
Could the selection of patients have introduced bias?   Unclear risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified?      
Could the conduct or interpretation of the index test have introduced bias?   Low risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Unclear    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Unclear risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Unclear
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Unclear    
Did all patients receive the same reference standard? Unclear    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Unclear risk  

Bollineni 2021.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19
Patient characteristics and setting Age group: mix of children and adults
Setting: outpatient
Index tests Index test(s): chest CT (with or without contrast)
Definition for positive diagnosis on CT: unclear
Level of training of readers: unclear
Prevalence: 0.6
Target condition and reference standard(s) Reference standard: RT‐PCR twice, in all with initial negative results
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Unclear    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Yes    
Could the selection of patients have introduced bias?   Low risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Low risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Yes    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Unclear risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Yes    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Low risk  

Borakati 2020.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, symptomatic or asymptomatic
Patient characteristics and setting Age group: adults, perhaps also children
Setting: outpatient
Index tests Index test(s): chest CT (non‐contrast, IV contrast); chest x‐rays
Definition for positive diagnosis (both CT and x‐ray): BSTI template
Level of training of readers: radiologist
Prevalence: 0.6
Target condition and reference standard(s) Reference standard: RT‐PCR once; twice in some
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Yes    
Could the selection of patients have introduced bias?   Low risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Unclear    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Unclear risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Unclear risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Unclear    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Unclear risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Unclear    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Unclear risk  

Bosso 2021.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: adults, perhaps also children
Setting: outpatient
Index tests Index test(s): Ultrasound of the lungs (POCUS)
Definition for positive diagnosis on US: unclear
Level of training of readers: unclear
Prevalence: 0.4
Target condition and reference standard(s) Reference standard:RT‐PCR twice, in some with initial negative results
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Unclear    
Could the selection of patients have introduced bias?   Unclear risk  
Are there concerns that the included patients and setting do not match the review question?     Unclear
DOMAIN 2: Index Test (Chest CT)
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? Unclear    
Could the conduct or interpretation of the index test have introduced bias?   Low risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Yes    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Low risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Yes    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Low risk  

Boussouar 2020.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: adults only
Setting: outpatient
Index tests Index test(s): chest CT (Non contrast CT)
Definition for positive diagnosis on CT: 1) imaging patterns suggesting the presence of COVID‐19; 2) imaging patterns suggesting an alternative diagnosis; 3) imaging patterns suggesting a combination of COVID‐19 with underlying lung disease; 4) CT considered normal
Level of training of readers: radiologists
Prevalence: 0.51
Target condition and reference standard(s) Reference standard: RT‐PCR twice, in all with initial negative results
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Yes    
Could the selection of patients have introduced bias?   Low risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Low risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Yes    
Were the reference standard results interpreted without knowledge of the results of the index tests? No    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Unclear risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Unclear    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Low risk  

Brun 2021.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: adults, perhaps also children
Setting: outpatient
Index tests Index test(s): chest CT (low dose)
Definition for positive diagnosis on CT: highly probable, probable, and less probable of COVID‐19 pneumonia, alternative diagnosis, or normal
Level of training of readers: unclear
Prevalence: 0.6
Target condition and reference standard(s) Reference standard: RT‐PCR, no other details provided
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Yes    
Could the selection of patients have introduced bias?   Low risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Low risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Yes    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Low risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Unclear    
Did all patients receive the same reference standard? No    
Were all patients included in the analysis? No    
Could the patient flow have introduced bias?   High risk  

Caruso 2020.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: adults only
Setting: outpatient
Index tests Index test(s): chest CT (non‐contrast)
Definition for positive diagnosis on CT: pneumonia
Level of training of readers: radiologist
Prevalence: 0.4
Target condition and reference standard(s) Reference standard: RT‐PCR twice, if necessary
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Unclear    
Could the selection of patients have introduced bias?   Unclear risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Unclear    
If a threshold was used, was it pre‐specified? Unclear    
Could the conduct or interpretation of the index test have introduced bias?   Unclear risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Yes    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Unclear risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Yes    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Low risk  

Cengel 2021.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, symptomatic or asymptomatic
Patient characteristics and setting Age group: adults, perhaps also children
Setting: outpatient
Index tests Index test(s): chest CT (non contrast)
Definition for positive diagnosis on CT:RSNA
Level of training of readers: unclear
Prevalence: 0.7
Target condition and reference standard(s) Reference standard: RT‐PCR twice, in some with initial negative results
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Yes    
Could the selection of patients have introduced bias?   Low risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Low risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Yes    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Low risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Yes    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Low risk  

Colombi 2020a.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: adults, perhaps also children
Setting: outpatient
Index tests Index test(s): chest CT (low dose)/ Ultrasound of lungs (POCUS)
Definition for positive diagnosis on CT: RSNA
Level of training of readers: unclear
Prevalence: 0.42
Target condition and reference standard(s) Reference standard: RT‐PCR twice, in some with initial negative results
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Yes    
Could the selection of patients have introduced bias?   Low risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Low risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Low risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Yes    
Were the reference standard results interpreted without knowledge of the results of the index tests? Yes    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Low risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Yes    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Low risk  

Cozzi 2020.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, symptomatic or asymptomatic
Patient characteristics and setting Age group: unclear
Setting: outpatient
Index tests Index test(s): chest X‐rays
Definition for positive diagnosis on X‐ray: the presence of interstitial infiltrates with predominantly bilateral and basal distribution
Level of training of readers: radiologist
Prevalence: 0.8
Target condition and reference standard(s) Reference standard: RT‐PCR, no other details provided; other (follow‐up phone call)
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Unclear    
Could the selection of patients have introduced bias?   Unclear risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
DOMAIN 2: Index Test (Chest X‐ray)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Low risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? No    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   High risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Yes    
Did all patients receive the same reference standard? Unclear    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Unclear risk  

Dafydd 2021.

Study characteristics
Patient Sampling Study design: suspected patients, symptomatic or asymptomatic
Patient characteristics and setting Age group: adults
Setting: inpatient
Index tests Index test(s): chest CT(high resolution)
Definition for positive diagnosis on CT: unclear
Level of training of readers: radiologist
Prevalence: 0.01
Target condition and reference standard(s) Reference standard: RT‐PCR twice, in some with initial negative results
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Unclear    
Could the selection of patients have introduced bias?   Unclear risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Unclear    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Unclear risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Unclear    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Unclear risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Yes    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Low risk  

Debray 2020.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, unclear symptom status
Patient characteristics and setting Age group: adults only
Setting: outpatient
Index tests Index test(s): chest CT (non‐contrast)
Defintion for positive diagnosis on CT: quote: “evocative”: multifocal ground‐glass opacities, being nodular or not, or crazy‐paving with or without consolidations, with a bilateral, peripheral or mixed distribution and involvement of the posterior zones
Level of training of readers: radiologist
Prevalence: 0.7
Target condition and reference standard(s) Reference standard: RT‐PCR once; twice in some
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Yes    
Could the selection of patients have introduced bias?   Low risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Low risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? No    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   High risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Unclear    
Did all patients receive the same reference standard? Unclear    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Unclear risk  

Deng 2020.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: children and adults
Setting: unclear
Index tests Index test(s): chest CT (high resolution)
Defintion for positive diagnosis on CT:
  1. any one of the following:

    1. single, multiple, or diffuse GGO, with thickened blood vessels and thickened bronchial shadows passing through, with or without localised lobular septal grid thickening

    2. single or multiple real shadows

  2. re‐examination 3‐5 days later showed that the original GGO or consolidation range increased, the number increased, or accompanied by pleural effusion on one or both sides


Level of training of readers: radiologist
Prevalence: 0.7
Target condition and reference standard(s) Reference standard: RT‐PCR once
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Unclear    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Yes    
Could the selection of patients have introduced bias?   Unclear risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Low risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Yes    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Low risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Unclear    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Unclear risk  

De Smet 2020.

Study characteristics
Patient Sampling Study design: suspected patients, all symptomatic
Patient characteristics and setting Age group: children and adults
Setting: outpatient
Index tests Index test(s): chest CT, no further details provided
Defintion for positive diagnosis on CT: CO‐RADS
Level of training of readers: unclear
Prevalence: 0.4 for primary objective, 0,05 for secondary objective.
Target condition and reference standard(s) Reference standard: RT‐PCR, no other details provided
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Unclear    
Could the selection of patients have introduced bias?   Unclear risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Unclear    
If a threshold was used, was it pre‐specified? No    
Could the conduct or interpretation of the index test have introduced bias?   High risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Unclear    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Unclear risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Unclear    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Unclear risk  

Dimeglio 2021.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: unclear
Setting: outpatient
Index tests Index test(s): chest CT
Defintion for positive diagnosis on CT:following the recommendation of the French Society of Radiology
Level of training of readers: unclear
Prevalence: 0.4
Target condition and reference standard(s) Reference standard: RT‐PCR once
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Yes    
Could the selection of patients have introduced bias?   Low risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Unclear    
If a threshold was used, was it pre‐specified? Unclear    
Could the conduct or interpretation of the index test have introduced bias?   Unclear risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Yes    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Low risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Unclear    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Unclear risk  

Dini 2020.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, symptomatic or asymptomatic
Patient characteristics and setting Age group: ≥ 70 years of age
Setting: outpatient
Index tests Index test(s): ultrasound of lungs (POCUS); no further details provided
Definition for positive diagnosis on ultrasound: scoring system: non‐coalescent B‐lines, coalescent and with hyperechoic non‐consolidated state
Level of training of readers: unclear
Prevalence: 0.6
Target condition and reference standard(s) Reference standard: RT‐PCR, no other details provided
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Yes    
Could the selection of patients have introduced bias?   Low risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
Were the index test results interpreted without knowledge of the results of the reference standard? Unclear    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Unclear risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Unclear    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Unclear risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Yes    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Low risk  

Djangang 2020.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: adults only
Setting: outpatient
Index tests Index test(s): chest CT
Defintion for positive diagnosis on CT: ground‐glass opacities, consolidation or crazy‐paving patterns
Level of training of readers: radiologist
Prevalence: 0.5
Target condition and reference standard(s) Reference standard: RT‐PCR twice, in some with initial negative results
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Unclear    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Yes    
Could the selection of patients have introduced bias?   Unclear risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Unclear    
If a threshold was used, was it pre‐specified? Unclear    
Could the conduct or interpretation of the index test have introduced bias?   Unclear risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Unclear    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Unclear risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Unclear    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Unclear risk  

Dofferhoff 2020.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, symptomatic or asymptomatic
Patient characteristics and setting Age group: adults only
Setting: outpatient
Index tests Index test(s): chest CT (low dose)
Defintion for positive diagnosis on CT: CO‐RADS
Level of training of readers: unclear
Prevalence: 0.5
Target condition and reference standard(s) Reference standard: RT‐PCR once; twice in some
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Unclear    
Could the selection of patients have introduced bias?   Unclear risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Unclear    
If a threshold was used, was it pre‐specified? No    
Could the conduct or interpretation of the index test have introduced bias?   High risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? No    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   High risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Unclear    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Unclear risk  

Dogan 2020.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, symptomatic or asymptomatic)
Patient characteristics and setting Age group: adults only
Setting: unclear
Index tests Index test(s): chest CT (Non contrast)
Definition for positive diagnosis on CT: RSNA
Level of training of readers: unclear
Prevalence: 0.55
Target condition and reference standard(s) Reference standard: RT‐PCR twice, in all with initial negative results
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Yes    
Could the selection of patients have introduced bias?   Low risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Low risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Yes    
Were the reference standard results interpreted without knowledge of the results of the index tests? Yes    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Low risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Yes    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Low risk  

Ducray 2020.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, symptomatic or asymptomatic
Patient characteristics and setting Age group: adults only
Setting: outpatient
Index tests Index test(s): chest CT (IV contrast)
Defintion for positive diagnosis on CT: classification system: surely COVID+, possible COVID+, COVID‐
Level of training of readers: radiologist
Prevalence: 0.4
Target condition and reference standard(s) Reference standard: RT‐PCR once; twice in some
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Yes    
Could the selection of patients have introduced bias?   Low risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? No    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   High risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? No    
Were the reference standard results interpreted without knowledge of the results of the index tests? Yes    
Could the reference standard, its conduct, or its interpretation have introduced bias?   High risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Unclear    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Unclear risk  

Erxleben 2021.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: adults, perhaps also children
Setting: outpatient
Index tests Index test(s): chest CT (Low‐dose CT)
Definition for positive diagnosis on CT: "All CT images were evaluated manually and data on presence/absence of COVID‐19 was assessed"
Level of training of readers: radiograph
Prevalence: 0.13
Target condition and reference standard(s) Reference standard: RT‐PCR twice, in some with initial negative results
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Unclear    
Was a case‐control design avoided? Unclear    
Did the study avoid inappropriate exclusions? Unclear    
Could the selection of patients have introduced bias?   Unclear risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Unclear    
If a threshold was used, was it pre‐specified? No    
Could the conduct or interpretation of the index test have introduced bias?   High risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     High
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Yes    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   High risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Yes    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Unclear risk  

Falaschi 2020.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: adults only
Setting: outpatient
Index tests Index test(s): chest CT (non‐contrast)
Defintion for positive diagnosis on CT: STR/ACR/RSNA
Level of training of readers: radiologist
Prevalence: 0.6
Target condition and reference standard(s) Reference standard: RT‐PCR once; twice in some
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Unclear    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Yes    
Could the selection of patients have introduced bias?   Unclear risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Low risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? No    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   High risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Yes    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Low risk  

Ferda 2020.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19 (all symptomatic)
Patient characteristics and setting Age group: mix of children and adults
Setting: unclear
Index tests Index test(s): chest CT (with IV contrast)
Definition for positive diagnosis on CT: ground glass opacities, mixed ground‐glass, opacities, thickening of intra‐lobular septa, negative bronchogram, reverse
Level of training of readers: radiologist
Prevalence: 0.1
Target condition and reference standard(s) Reference standard: RT‐PCR twice, in some with initial negative results
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Unclear    
Could the selection of patients have introduced bias?   Unclear risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Low risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Yes    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Unclear risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Unclear    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Unclear risk  

Fink 2021.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: adults only
Setting: outpatient
Index tests Index test(s): chest CT (high‐resolution CT)/ X‐ray
Definition for positive diagnosis on CT: CT scans were classified according to two different reading scores
Definition for positive diagnosis on X‐ray:
Level of training of readers: unclear
Prevalence: 0.29
Target condition and reference standard(s) Reference standard: RT‐PCR twice, in some with initial negative results
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Unclear    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Yes    
Could the selection of patients have introduced bias?   Unclear risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Unclear risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Unclear risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Yes    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Unclear risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     High
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Unclear    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Unclear risk  

Fonsi 2020.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: adults only
Setting: outpatient
Index tests Index test(s): chest CT (non‐contrast)
Defintion for positive diagnosis on CT: GGOs; consolidation; a mixed GGO and consolidation pattern; single or multiple solid nodules surrounded by GGOs; a focal or multifocal distribution; GGO and consolidation location; multilobe involvement; a bilateral distribution; interlobular septal thickening; an air bronchogram; the presence of cavitation; bronchial wall thickening; bronchiectasis; mediastinal lymph node enlargement; pleural effusion; and pericardial effusion
Definition for positive diagnosis on ultrasound: not reported
Level of training of readers: radiologist
Prevalence: 0.7
Target condition and reference standard(s) Reference standard: RT‐PCR once; twice in some
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Yes    
Could the selection of patients have introduced bias?   Low risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? Unclear    
Could the conduct or interpretation of the index test have introduced bias?   Unclear risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
Were the index test results interpreted without knowledge of the results of the reference standard? Unclear    
If a threshold was used, was it pre‐specified? Unclear    
Could the conduct or interpretation of the index test have introduced bias?   Unclear risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Unclear    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Unclear risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Yes    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Low risk  

Fujioka 2020.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: adults only
Setting: unclear
Index tests Index test(s): chest CT, no further details provided
Definition for positive diagnosis on CT: CO‐RADS
Level of training of readers: radiologist
Prevalence: 0.5
Target condition and reference standard(s) Reference standard: RT‐PCR once; twice in some
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Unclear    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Unclear    
Could the selection of patients have introduced bias?   Unclear risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? No    
Could the conduct or interpretation of the index test have introduced bias?   Low risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Unclear    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Unclear risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Unclear    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Unclear risk  

Gaia 2020.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: adults only
Setting: outpatient
Index tests Index test(s): chest CT
Definition for positive diagnosis on CT: Simpson 2020
Level of training of readers: radiologist
Prevalence: 0.5
Target condition and reference standard(s) Reference standard: RT‐PCR once
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Unclear    
Was a case‐control design avoided? Unclear    
Did the study avoid inappropriate exclusions? Unclear    
Could the selection of patients have introduced bias?   Unclear risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Unclear    
If a threshold was used, was it pre‐specified? Unclear    
Could the conduct or interpretation of the index test have introduced bias?   Unclear risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Unclear    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Unclear risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Yes    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Unclear    
Could the patient flow have introduced bias?   Low risk  

Giannitto 2020.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: adults only
Setting: outpatient
Index tests Index test(s): chest CT (non‐contrast)
Definition for positive diagnosis on CT: classification system: suspected COVID‐19 pneumonia, non‐COVID‐19 pneumonia, negative CT
Level of training of readers: radiologist
Prevalence: 0.3
Target condition and reference standard(s) Reference standard: RT‐PCR twice, if necessary
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Unclear    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? No    
Could the selection of patients have introduced bias?   High risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Unclear risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Yes    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Low risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Yes    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Low risk  

Gietema 2020.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: adults only
Setting: outpatient
Index tests Index test(s): chest CT (non‐contrast)
Definition for positive diagnosis on CT: standardized imaging reporting system (typical for COVID‐19, equivocal, non COVID‐19)
Level of training of readers: resident
Prevalence: 0.4
Target condition and reference standard(s) Reference standard: RT‐PCR once; twice in some
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Yes    
Could the selection of patients have introduced bias?   Low risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Low risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? No    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   High risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Yes    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Low risk  

Gil‐Rodrigo 2020.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: adults only
Setting: outpatient
Index tests Index test(s): ultrasound of the lungs (POCUS)
Definition for positive diagnosis on US:Scoring system by Soldati 2020
Level of training of readers: unclear
Prevalence: 0.42
Target condition and reference standard(s) Reference standard: RT‐PCR once
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Unclear    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Yes    
Could the selection of patients have introduced bias?   Unclear risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Low risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Unclear    
Were the reference standard results interpreted without knowledge of the results of the index tests? Yes    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Unclear risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Unclear    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Unclear risk  

Grando 2020.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: adults only
Setting: outpatient
Index tests Index test(s): Chest CT (non contrast)
Definition for positive diagnosis on CT: RSNA
Level of training of readers: radiologist
Prevalence: 0.57
Target condition and reference standard(s) Reference standard: RT‐PCR twice, in some with initial negative results
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? No    
Could the selection of patients have introduced bias?   High risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Low risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Unclear    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   High risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Unclear    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Low risk  

Gross 2021.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: adults only
Setting: outpatient
Index tests Index test(s): chest CT(Low dose CT)
Definition for positive diagnosis on CT: CO‐RADS
Level of training of readers: radiologist
Prevalence: 0.21
Target condition and reference standard(s) Reference standard: RT‐PCR twice, in all with initial negative results
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Unclear    
Could the selection of patients have introduced bias?   Unclear risk  
Are there concerns that the included patients and setting do not match the review question?     Unclear
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Low risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? No    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   High risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Unclear    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Unclear risk  

Guillo 2020.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: adults only
Setting: outpatient
Index tests Index test(s): chest CT (IV contrast)
Definition for positive diagnosis on CT: a structured report about the probability of COVID‐19 pneumonia based on the presence of GGOs with or without crazy‐paving pattern, isolated or admixed with perilobular or linear consolidation, their peripheral or central distribution, etc.
Level of training of readers: resident
Prevalence: 0.6
Target condition and reference standard(s) Reference standard: RT‐PCR once; twice in some
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? No    
Could the selection of patients have introduced bias?   High risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Low risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? No    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   High risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Yes    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Low risk  

Gumus 2020.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all asymptomatic
Patient characteristics and setting Age group: adults only
Setting: inpatient
Index tests Index test(s): chest CT(Low‐dose CT)
Definition for positive diagnosis on CT: RSNA
Level of training of readers: unclear
Prevalence: 0.01
Target condition and reference standard(s) Reference standard: RT‐PCR twice, in some with initial negative results
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Yes    
Could the selection of patients have introduced bias?   Low risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Unclear    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Unclear risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Unclear    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Unclear risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Unclear    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Unclear risk  

Haak 2021.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: adults only
Setting: outpatient
Index tests Index test(s): ultrasound of the lungs (POCUS)
Definition for positive diagnosis on US: unclear
Level of training of readers: unclear
Prevalence: 0.3
Target condition and reference standard(s) Reference standard: RT‐PCR twice, in all with initial negative results
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Unclear    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Yes    
Could the selection of patients have introduced bias?   Unclear risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Low risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Unclear    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Unclear risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? No    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Unclear    
Could the patient flow have introduced bias?   Unclear risk  

Hanif 2021.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: adults only
Setting: outpatient
Index tests Index test(s): chest CT (high‐resolution CT)
Definition for positive diagnosis on CT: positive HRCT chest findings for COVID‐19 were defined as bilateral, multifocal, multilobar ground glass opacities with or without sub‐segmental consolidations or crazy paving pattern in a peripheral distribution.
Level of training of readers: radiologist
Prevalence: 0.83
Target condition and reference standard(s) Reference standard: RT‐PCR twice, in some with initial negative results
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Unclear    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Unclear    
Could the selection of patients have introduced bias?   Unclear risk  
Are there concerns that the included patients and setting do not match the review question?     High
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? Unclear    
Could the conduct or interpretation of the index test have introduced bias?   Unclear risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Unclear    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Unclear risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Yes    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Low risk  

He 2020.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, unclear symptom status
Patient characteristics and setting Age group: children and adults
Setting: unclear
Index tests Index test(s): chest CT (high‐resolution)
Defintion for positive diagnosis on CT: GGO with or without consolidation, crazy paving patten, peripheral and diffuse distribution, and bilateral/multilobular involvement
Level of training of readers: radiologist
Prevalence: 0.4
Target condition and reference standard(s) Reference standard: RT‐PCR once; twice in some
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Unclear    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Yes    
Could the selection of patients have introduced bias?   Unclear risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Low risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? No    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   High risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Yes    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Low risk  

Hermans 2020.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, symptomatic or asymptomatic
Patient characteristics and setting Age group: adults only
Setting: outpatient
Index tests Index test(s): chest CT, no further details provided
Defintion for positive diagnosis on CT: CO‐RADS
Level of training of readers: radiologist
Prevalence: 0.4
Target condition and reference standard(s) Reference standard: RT‐PCR once
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Yes    
Could the selection of patients have introduced bias?   Low risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Unclear    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Unclear risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? No    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   High risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Yes    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Low risk  

Hernigou 2020.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, symptomatic or asymptomatic
Patient characteristics and setting Age group: adults only
Setting: outpatient
Index tests Index test(s): chest CT (low dose)
Defintion for positive diagnosis on CT: unclear
Level of training of readers: radiologist
Prevalence: 0.3
Target condition and reference standard(s) Reference standard: RT‐PCR once; twice in some
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Unclear    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? No    
Could the selection of patients have introduced bias?   High risk  
Are there concerns that the included patients and setting do not match the review question?     High
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Unclear    
If a threshold was used, was it pre‐specified? Unclear    
Could the conduct or interpretation of the index test have introduced bias?   Unclear risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Unclear    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Unclear risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Yes    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Low risk  

Herpe 2020.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: children and adults
Setting: unclear
Index tests Index test(s): chest CT, no further details provided
Definition for positive diagnosis on CT: bilateral GGO with peripheral distribution, bilateral crazy paving appearance with intralobular thickening, reverse halo sign, or other signs compatible with organising pneumonia
Level of training of readers: radiologist
Prevalence: 0.5
Target condition and reference standard(s) Reference standard: RT‐PCR once; twice in some
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Unclear    
Could the selection of patients have introduced bias?   Unclear risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Low risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? No    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   High risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Yes    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Low risk  

Hwang 2020.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, symptomatic or asymptomatic
Patient characteristics and setting Age group: adults, perhaps also children
Setting: unclear
Index tests Index test(s): chest X‐rays
Definition for positive diagnosis on X‐ray: abnormality suggesting pneumonia
Level of training of readers: radiologists and resident
Prevalence: 0.05
Target condition and reference standard(s) Reference standard: RT‐PCR, no other details provided
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Unclear    
Could the selection of patients have introduced bias?   Unclear risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
DOMAIN 2: Index Test (Chest X‐ray)
Were the index test results interpreted without knowledge of the results of the reference standard? Unclear    
If a threshold was used, was it pre‐specified? Unclear    
Could the conduct or interpretation of the index test have introduced bias?   Unclear risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     High
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Unclear    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Unclear risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Yes    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Low risk  

Ippolito 2020.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: children and adults
Setting: outpatient
Index tests Index test(s): chest X‐rays
Defintion for positive diagnosis on X‐ray: reticulations, alveolar opacities or both
Level of training of readers: radiologist
Prevalence: 0.4
Target condition and reference standard(s) Reference standard: RT‐PCR, no other details provided
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Unclear    
Could the selection of patients have introduced bias?   Unclear risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
DOMAIN 2: Index Test (Chest X‐ray)
Were the index test results interpreted without knowledge of the results of the reference standard? Unclear    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Unclear risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Unclear    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Unclear risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Unclear    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Unclear risk  

Jalil 2020.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: adults, perhaps also children
Setting: outpatient
Index tests Index test(s): ultrasound of the lungs (POCUS)
Definition for positive diagnosis on US: unclear
Level of training of readers: unclear
Prevalence: 0.52
Target condition and reference standard(s) Reference standard: RT‐PCR twice, in all with initial negative results
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Unclear    
Could the selection of patients have introduced bias?   Unclear risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
Were the index test results interpreted without knowledge of the results of the reference standard? Unclear    
If a threshold was used, was it pre‐specified? Unclear    
Could the conduct or interpretation of the index test have introduced bias?   Unclear risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Yes    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Low risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Unclear    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Unclear risk  

Krdzalic 2020.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: adults only
Setting: unclear
Index tests Index test(s): chest CT
Defintion for positive diagnosis on CT: CO‐RADS
Level of training of readers: radiologist
Prevalence: 0.5
Target condition and reference standard(s) Reference standard: RT‐PCR twice, if necessary
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? No    
Could the selection of patients have introduced bias?   High risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Unclear    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Unclear risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Yes    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Low risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Yes    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Low risk  

Kuzan 2020.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: adults only
Setting: outpatient
Index tests Index test(s): chest CT (non‐contrast)
Defintion for positive diagnosis on CT: BSTI version 2
Level of training of readers: radiologist
Prevalence: 0.6
Target condition and reference standard(s) Reference standard: RT‐PCR twice, if necessary
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Yes    
Could the selection of patients have introduced bias?   Low risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Low risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Yes    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Low risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Yes    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Low risk  

Lieveld 2021a.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: adults only
Setting: outpatient
Index tests Index test(s): chest CT
Defintion for positive diagnosis on CT: CO‐RADS
Level of training of readers: radiologists
Prevalence: 0.3
Target condition and reference standard(s) Reference standard: RT‐PCR twice, in all with initial negative results
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Yes    
Could the selection of patients have introduced bias?   Low risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Low risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Yes    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Low risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Unclear    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Unclear risk  

Lieveld 2021b.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: adults only
Setting: outpatient
Index tests Index test(s): ultrasound of the lungs (POCUS)
Definition for positive diagnosis on US: CO‐RADS
Level of training of readers: unclear
Prevalence: 0.4
Target condition and reference standard(s) Reference standard: RT‐PCR twice, in some with initial negative results
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Yes    
Could the selection of patients have introduced bias?   Low risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Low risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Yes    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Low risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Unclear    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Unclear risk  

Luo 2020a.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: children and adults
Setting: outpatient
Index tests Index test(s): chest CT, no further details provided
Defintion for positive diagnosis on CT: scoring system was developed (with scores from −4 to +7)
Level of training of readers: radiologist
Prevalence: 0.4
Target condition and reference standard(s) Reference standard: RT‐PCR twice, if necessary
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Unclear    
Was a case‐control design avoided? Unclear    
Did the study avoid inappropriate exclusions? Yes    
Could the selection of patients have introduced bias?   Unclear risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? No    
Could the conduct or interpretation of the index test have introduced bias?   High risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Yes    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Low risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Unclear    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Unclear risk  

Majeed 2020.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, symptomatic or asymptomatic
Patient characteristics and setting Age group: adults only
Setting: outpatient
Index tests Index test(s): chest CT (non‐contrast)
Definition for positive diagnosis on CT: BSTI and RSNA
Level of training of readers: unclear
Prevalence: 0.33
Target condition and reference standard(s) Reference standard: RT‐PCR twice, in some with initial negative results
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? No    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? No    
Could the selection of patients have introduced bias?   High risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Unclear    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Unclear risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Yes    
Were the reference standard results interpreted without knowledge of the results of the index tests? Yes    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Low risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Yes    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Low risk  

Mei 2020.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, symptomatic or asymptomatic
Patient characteristics and setting Age group: children and adults
Setting: unclear
Index tests Index test(s): chest CT, no further details provided
Defintion for positive diagnosis on CT: unclear
Level of training of readers: radiologist
Prevalence: 0.5
Target condition and reference standard(s) Reference standard: RT‐PCR twice, if necessary
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Yes    
Could the selection of patients have introduced bias?   Low risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Unclear    
If a threshold was used, was it pre‐specified? Unclear    
Could the conduct or interpretation of the index test have introduced bias?   Unclear risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Yes    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Low risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Unclear    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Unclear risk  

Miranda Magalhaes Santos 2020.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: children and adults
Setting: outpatient
Index tests Index test(s): chest CT, no further details provided
Defintion for positive diagnosis on CT: RSNA classification
Level of training of readers: radiologist
Prevalence: 0.5
Target condition and reference standard(s) Reference standard: RT‐PCR, no other details provided
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Yes    
Could the selection of patients have introduced bias?   Low risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Low risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Unclear    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Unclear risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Yes    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Low risk  

Moroni 2021.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: adults only
Setting: outpatient
Index tests Index test(s): chest radiographs/chest X‐rays
Definition for positive diagnosis on X‐rays: unclear
Level of training of readers: unclear
Prevalence: 0.31
Target condition and reference standard(s) Reference standard: RT‐PCR, no further details provided or further details are unclear
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Unclear    
Could the selection of patients have introduced bias?   Unclear risk  
Are there concerns that the included patients and setting do not match the review question?     Unclear
DOMAIN 2: Index Test (Chest CT)
DOMAIN 2: Index Test (Chest X‐ray)
Were the index test results interpreted without knowledge of the results of the reference standard? Unclear    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Low risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Yes    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Low risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Yes    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Low risk  

Murphy 2020.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: children and adults
Setting: outpatient
Index tests Index test(s): chest X ‐rays
Defintion for positive diagnosis on X‐ray: classification system: normal, no finding (category 0); abnormal but no lung opacity consistent with pneumonia (category 1); lung opacity consistent with pneumonia (unlikely COVID‐19) (category 2); lung opacity consistent with pneumonia (consistent with COVID‐19) (category 3). Sensitivities matched to AI reading.
Level of training of readers: radiologist
Prevalence: 0.5
Target condition and reference standard(s) Reference standard: RT‐PCR, no other details provided
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Unclear    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Yes    
Could the selection of patients have introduced bias?   Low risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
DOMAIN 2: Index Test (Chest X‐ray)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Low risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Unclear    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Unclear risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Unclear    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Unclear    
Could the patient flow have introduced bias?   Unclear risk  

Narinx 2020.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: adults, perhaps also children
Setting: outpatient
Index tests Index test(s): chest CT (low dose); ultrasound of lungs (POCUS)
Defintion for positive diagnosis on CT: scored as suggestive for or inconsistent with COVID‐19 infection based on the presence of clinical manifestations as presented by Ng 2020 and Shi 2020
Defintion for positive diagnosis on ultrasound: positive if one or more BLUE points showed a positive B‐line parameter
Level of training of readers: radiologist
Prevalence: 0.2
Target condition and reference standard(s) Reference standard: RT‐PCR, no other details provided
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Unclear    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Yes    
Could the selection of patients have introduced bias?   Unclear risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Low risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Low risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Unclear    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Unclear risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Yes    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Low risk  

Nivet 2021.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: adults only
Setting: outpatient
Index tests Index test(s): chest CT (non contrast)
Definition for positive diagnosis on CT: each reading was categorised using a five‐point score, adapted from the recommendations of the Société Française de Radiologie (SFR)
Level of training of readers: radiologist
Prevalence: 0.4
Target condition and reference standard(s) Reference standard: RT‐PCR twice, in some with initial negative results
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Yes    
Could the selection of patients have introduced bias?   Low risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Low risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Yes    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Low risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Unclear    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Unclear risk  

O'Neill 2020.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19 (all symptomatic)
Patient characteristics and setting Age group: adults only
Setting: outpatient
Index tests Index test (s): chest CT
Definition for positive diagnosis on CT: RSNA and CO‐RADS
Level of training of readers: radiologists
Prevalence:0.5
Target condition and reference standard(s) Reference standard: RT‐PCR twice, in all with initial negative results
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Yes    
Could the selection of patients have introduced bias?   Low risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Low risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Yes    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Low risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Yes    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Low risk  

Ohana 2021.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: adults only
Setting: outpatient
Index tests Index test(s): chest CT(non contrast)
Definition for positive diagnosis on CT: chest CT with typical COVID‐19 appearance
Level of training of readers: radiologists
Prevalence: 0.5
Target condition and reference standard(s) Reference standard: RT‐PCR twice, in some with initial negative results
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Yes    
Could the selection of patients have introduced bias?   Low risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Unclear    
If a threshold was used, was it pre‐specified? Unclear    
Could the conduct or interpretation of the index test have introduced bias?   Unclear risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? No    
Were the reference standard results interpreted without knowledge of the results of the index tests? No    
Could the reference standard, its conduct, or its interpretation have introduced bias?   High risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Yes    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Low risk  

Ooi 2021.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic or asymptomatic
Patient characteristics and setting Age group: adults, perhaps also children
Setting: outpatient
Index tests Index test(s): chest CT
Definition for positive diagnosis on CT: each area was given a score between 0 and 3
Level of training of readers: unclear
Prevalence: 0.1
Target condition and reference standard(s)  
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Unclear    
Did the study avoid inappropriate exclusions? Unclear    
Could the selection of patients have introduced bias?   Unclear risk  
Are there concerns that the included patients and setting do not match the review question?     High
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Unclear    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Low risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Unclear    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   High risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Unclear    
Did all patients receive the same reference standard? No    
Were all patients included in the analysis? Unclear    
Could the patient flow have introduced bias?   High risk  

Pagano 2021.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic or asymptomatic
Patient characteristics and setting Age group: adults only
Setting: outpatient
Index tests Index test(s): chest radiographs/chest X‐rays
Definition for positive diagnosis on CT: unclear
Level of training of readers: unclear
Prevalence: 0.8
Target condition and reference standard(s) Reference standard: RT‐PCR, no other details provided
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Unclear    
Could the selection of patients have introduced bias?   Unclear risk  
Are there concerns that the included patients and setting do not match the review question?     Unclear
DOMAIN 2: Index Test (Chest CT)
DOMAIN 2: Index Test (Chest X‐ray)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Unclear risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     High
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Yes    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Low risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Unclear
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Unclear    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   High risk  

Palmisano 2021.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: adults, perhaps also children
Setting: outpatient
Index tests Index test(s): chest CT (non contrast)
Definition for positive diagnosis on CT: RSNA
Level of training of readers: unclear
Prevalence: 0.68
Target condition and reference standard(s) Reference standard: RT‐PCR twice, in some with initial negative results
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Unclear    
Could the selection of patients have introduced bias?   Unclear risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Unclear    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Unclear risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Yes    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Low risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Unclear    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Low risk  

Pare 2020.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: adults, perhaps also children
Setting: outpatient
Index tests Index test(s): chest X‐rays; ultrasound of lungs (POCUS)
Defintion for positive diagnosis on X‐ray: if the report included infection in the differential, as defined by words such as opacity, consolidation, or airspace disease; negative if no abnormality was noted, an abnormality was noted but attributed to a non‐infectious aetiology, or was inconclusive for infectious process
Definition for positive diagnosis on ultrasound: positive if any B‐lines were detected.
Level of training of readers: unclear
Prevalence: 0.6
Target condition and reference standard(s) Reference standard: RT‐PCR once; twice in some
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Unclear    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? No    
Could the selection of patients have introduced bias?   High risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
DOMAIN 2: Index Test (Chest X‐ray)
Were the index test results interpreted without knowledge of the results of the reference standard? Unclear    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Unclear risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Ultrasound of the lungs)
Were the index test results interpreted without knowledge of the results of the reference standard? Unclear    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Unclear risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? No    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   High risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? No    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   High risk  

Patel 2020.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: children and adults
Setting: outpatient
Index tests Index test(s): chest CT (high resolution)
Defintion for positive diagnosis on CT: scoring system: consistent with multifocal pneumonia (category 1); indeterminate for multifocal pneumonia (category 2); not consistent with multifocal pneumonia (category 3)
Level of training of readers: radiologist
Prevalence: 0.5
Target condition and reference standard(s) Reference standard: RT‐PCR once; twice in some
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Yes    
Could the selection of patients have introduced bias?   Low risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? No    
Could the conduct or interpretation of the index test have introduced bias?   Low risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? No    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   High risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Unclear    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Low risk  

Patrucco 2021.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, symptomatic or asymptomatic
Patient characteristics and setting Age group: adults, perhaps also children
Setting: outpatient
Index tests Index test(s): chest CT
Definition for positive diagnosis on CT: RSNA system and CO‐RADS system
Level of training of readers: unclear
Prevalence: 0.4
Target condition and reference standard(s) Reference standard: RT‐PCR, no further details provided or further details are unclear
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Unclear    
Could the selection of patients have introduced bias?   Unclear risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Unclear    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Unclear risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Unclear
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Yes    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Low risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Unclear    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Low risk  

Peng 2020a.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, symptomatic or asymptomatic
Patient characteristics and setting Age group: children only
Setting: unclear
Index tests Index test(s): chest CT
Definition for positive diagnosis on CT: GGO, consolidations with surrounding halo sign, nodules, residual fibre strips, lymphadenopathy
Level of training of readers: radiologist
Prevalence: 0.5
Target condition and reference standard(s) Reference standard: RT‐PCR, no other details provided; other (positive contacts)
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Unclear    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Yes    
Could the selection of patients have introduced bias?   Unclear risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Unclear    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Unclear risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Unclear    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Low risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Yes    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Low risk  

Pivetta 2021.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: adults only
Setting: outpatient
Index tests Index test(s): ultrasound of the lungs (POCUS)
Definition for positive diagnosis on US: unclear
Level of training of readers:unclear
Prevalence: 0.47
Target condition and reference standard(s) Reference standard: RT‐PCR twice, in some with initial negative results
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Yes    
Could the selection of patients have introduced bias?   Low risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
Were the index test results interpreted without knowledge of the results of the reference standard? Unclear    
If a threshold was used, was it pre‐specified? Unclear    
Could the conduct or interpretation of the index test have introduced bias?   Unclear risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Yes    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Low risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Unclear    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Unclear risk  

Puylaert 2020.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: adults only
Setting: inpatient
Index tests Index test(s): chest CT (low dose)
Definition for positive diagnosis on US: CO‐RADS
Level of training of readers:unclear
Prevalence: 0.01
Target condition and reference standard(s) Reference standard: RT‐PCR once
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Yes    
Could the selection of patients have introduced bias?   Low risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Low risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Unclear    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Low risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Yes    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Low risk  

Ravikanth 2021.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19
Patient characteristics and setting Age group: adults only
Setting: outpatient
Index tests Index test(s): chest CT (with IV contrast)
Definition for positive diagnosis on CT: dichotomous ‐ suspicious or not suspicious for COVID‐19
Level of training of readers: resident and radiologist
Prevalence: 0.8
Target condition and reference standard(s) Reference standard: RT‐PCR twice, in some with initial negative results
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Yes    
Could the selection of patients have introduced bias?   Low risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Low risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Unclear    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Unclear risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Yes    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Low risk  

Reginelli 2021.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, symptomatic or asymptomatic
Patient characteristics and setting Age group: adults only
Setting: outpatient
Index tests Index test(s): chest CT
Definition for positive diagnosis on CT: radiologists observed according to localization and distribution of GGO and consolidations, crazy paving pattern, and presence of nodules
Level of training of readers: radiologist
Prevalence: 0.8
Target condition and reference standard(s) Reference standard: RT‐PCR twice, in some with initial negative results
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Unclear    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Unclear    
Could the selection of patients have introduced bias?   Unclear risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Low risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Unclear    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Unclear risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Yes    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Low risk  

Rona 2021.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: children and young adults only
Setting: outpatient
Index tests Index test(s): chest CT (non contrast CT)
Definition for positive diagnosis on CT: computed tomography images were divided into 3 groups: normal, consistent with COVID‐19, and inconsistent with COVID‐19.
Level of training of readers: unclear
Prevalence: 0.45
Target condition and reference standard(s) Reference standard: RT‐PCR twice, in some with initial negative results
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Unclear    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? No    
Could the selection of patients have introduced bias?   High risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Low risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? No    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   High risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Unclear    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Unclear risk  

Roy Choudhury 2020.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: unclear
Settinng: inpatient
Index tests Index test(s): chest X‐rays, no further details provided
Defintion for positive diagnosis: a previously unvalidated Likert score (scores 1 to 5) based on radiographic features thought to be related to COVID‐19, based on format reported by Simpson 2020
Level of training of readers: unclear
Prevalence: 0.3
Target condition and reference standard(s) Reference standard: RT‐PCR, no other details provided
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Unclear    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Unclear    
Could the selection of patients have introduced bias?   Unclear risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
DOMAIN 2: Index Test (Chest X‐ray)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? No    
Could the conduct or interpretation of the index test have introduced bias?   Low risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Unclear    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Unclear risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Yes    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Low risk  

Saeed 2020.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: adults only
Setting: outpatient
Index tests Index test(s): chest CT (high resolution)
Definition for positive diagnosis on CT: RSNA
Level of training of readers: radiologist
Prevalence: 0.76
Target condition and reference standard(s) Reference standard:RT‐PCR twice, in all with initial negative results
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Yes    
Could the selection of patients have introduced bias?   Low risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Unclear    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Low risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Yes    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Low risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Unclear    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Unclear risk  

Salehi‐Pourmehr 2020.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: adults only
Setting: outpatient
Index tests Index test(s): chest CT
Definition for positive diagnosis on CT: unclear
Level of training of readers: unclear
Prevalence: 0.35
Target condition and reference standard(s) Reference standard: RT‐PCR, no further details provided or further details are unclear
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Yes    
Could the selection of patients have introduced bias?   Low risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Unclear risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Unclear    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Unclear risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Yes    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Low risk  

Schalekamp 2020.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: adults only
Setting: outpatient
Index tests Index test(s): chest CT (non contrast)
Definition for positive diagnosis on CT: CO‐RADS
Level of training of readers: radiologists
Prevalence: 0.5
Target condition and reference standard(s) Reference standard: RT‐PCR twice, in some with initial negative results
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Yes    
Could the selection of patients have introduced bias?   Low risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Low risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Unclear    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   High risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Yes    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Unclear risk  

Schmid 2020.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: adults only
Setting: inpatient
Index tests Index test(s): ultrasound of the lungs (POCUS)
Definition for positive diagnosis on US: unclear
Level of training of readers: unclear
Prevalence: 0.3
Target condition and reference standard(s) Reference standard: RT‐PCR, no other details provided
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Yes    
Could the selection of patients have introduced bias?   Low risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Low risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Yes    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Low risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Unclear    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Unclear risk  

Schulze‐hagen 2020.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: adults only
Setting: unclear
Index tests Index test(s): chest CT (low dose)
Defintion for positive diagnosis on CT: CO‐RADS
Level of training of readers: radiologist
Prevalence: 0.4
Target condition and reference standard(s) Reference standard: RT‐PCR once; twice in some
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Unclear    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Yes    
Could the selection of patients have introduced bias?   Unclear risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Low risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Unclear    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Unclear risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Yes    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Low risk  

Shah 2021.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: unclear
Setting: outpatient
Index tests Index test(s): chest CT (non contrast, low dose)
Definition for positive diagnosis on CT: COV‐Rads
Level of training of readers: radiologist
Prevalence: 0.4
Target condition and reference standard(s) Reference standard: RT‐PCR twice, in some with initial negative results
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Unclear    
Was a case‐control design avoided? Unclear    
Did the study avoid inappropriate exclusions? Unclear    
Could the selection of patients have introduced bias?   Unclear risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Unclear    
If a threshold was used, was it pre‐specified? Unclear    
Could the conduct or interpretation of the index test have introduced bias?   Unclear risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Unclear    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Unclear risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Yes    
Did all patients receive the same reference standard? No    
Were all patients included in the analysis? No    
Could the patient flow have introduced bias?   High risk  

Skalidis 2020.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: adults only
Setting: outpatient
Index tests Index test(s): chest CT (low dose CT thorax)
Definition for positive diagnosis on CT: the results of the classification were merged by consensus and the specialists classified the CT on positive or negative for COVID‐19.
Level of training of readers: unclear
Prevalence: 0.42
Target condition and reference standard(s) Reference standard: RT‐PCR twice, in some with initial negative results
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Unclear    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Yes    
Could the selection of patients have introduced bias?   Unclear risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Low risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Unclear    
Were the reference standard results interpreted without knowledge of the results of the index tests? Yes    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Unclear risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Unclear    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Low risk  

Song 2020a.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: adults only
Setting: unclear
Index tests Index test(s): chest CT, no further details provided
Defintion for positive diagnosis on CT: diagnosis of viral pneumonia according to: multiple bilateral, ill‐defined GGOs or mixed consolidation with diffuse peripheral distribution or bilateral pulmonary consolidation
Prevalence: 0.5
Target condition and reference standard(s) Reference standard: RT‐PCR twice, if necessary
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Yes    
Could the selection of patients have introduced bias?   Low risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Low risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Yes    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Low risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Yes    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Low risk  

Sorlini 2021.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: adults, perhaps also children
Setting: outpatient
Index tests Index test(s): chest X‐rays/Ultrasound of the lungs (POCUS)
Definition for positive diagnosis on chest X‐rays/Ultrasound of the lungs (POCUS): unclear
Level of training of readers: unclear
Prevalence: 0.75
Target condition and reference standard(s) Reference standard: RT‐PCR twice, in some with initial negative results
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Yes    
Could the selection of patients have introduced bias?   Low risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
DOMAIN 2: Index Test (Chest X‐ray)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Low risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Ultrasound of the lungs)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Low risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Yes    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Low risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Yes    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Low risk  

Speidel 2021.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: adults only
Setting: inpatient
Index tests Index test(s): ultrasound of the lungs (POCUS)
Definition for positive diagnosis on US: unclear
Level of training of readers: unclear
Prevalence: 0.22
Target condition and reference standard(s) Reference standard: RT‐PCR, no further details provided or further details are unclear
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Yes    
Could the selection of patients have introduced bias?   Low risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Low risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? No    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   High risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Unclear    
Did all patients receive the same reference standard? No    
Were all patients included in the analysis? No    
Could the patient flow have introduced bias?   High risk  

Steuwe 2020.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: adults only
Setting: unclear
Index tests Index test(s): chest CT (low dose)
Defintion for positive diagnosis on CT: unclear; based on typical COVID‐19 findings reported by Salehi 2020.
Level of training of readers: unclear
Prevalence: 0.2
Target condition and reference standard(s) Reference standard: RT‐PCR once; twice in some
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Unclear    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Unclear    
Could the selection of patients have introduced bias?   Unclear risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Low risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? No    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   High risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Unclear    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Unclear risk  

Stevens 2020.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: adults only
Setting: outpatient
Index tests Index test(s): chest X‐rays
Defintion for positive diagnosis on X‐ray: BSTI template
Level of training of readers: radiologist
Prevalence: 0.8
Target condition and reference standard(s) Reference standard: RT‐PCR once; twice in some
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? No    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? No    
Could the selection of patients have introduced bias?   High risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
DOMAIN 2: Index Test (Chest X‐ray)
Were the index test results interpreted without knowledge of the results of the reference standard? Unclear    
If a threshold was used, was it pre‐specified? No    
Could the conduct or interpretation of the index test have introduced bias?   Unclear risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Unclear
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? No    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   High risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Unclear    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Unclear risk  

Sukhija 2021.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: adults only
Setting: unclear
Index tests Index test(s): chest X‐rays
Definition for positive diagnosis on X‐rays: unclear
Level of training of readers: unclear
Prevalence: 0.6
Target condition and reference standard(s) Reference standard: RT‐PCR, no further details provided or further details are unclear
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Unclear    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Unclear    
Could the selection of patients have introduced bias?   Unclear risk  
Are there concerns that the included patients and setting do not match the review question?     Unclear
DOMAIN 2: Index Test (Chest CT)
DOMAIN 2: Index Test (Chest X‐ray)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Low risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Unclear    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Unclear risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Unclear
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Unclear    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Unclear risk  

Sverzellati Nicola 2021.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19(all symptomatic)
Patient characteristics and setting Age group: adults only
Setting: inpatient
Index tests Index test(s): chest CT (high resolution) and X‐ray
Definition for positive diagnosis on CT: For CT, 4 CT categories: normal, alternative diagnosis, indeterminate, or typical for COVID‐19 pneumonia
Level of training of readers: radiologist
Prevalence: 0.77
Target condition and reference standard(s) Reference standard: RT‐PCR twice, in all with initial negative results
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Yes    
Could the selection of patients have introduced bias?   Low risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? No    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Low risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? Unclear    
Could the conduct or interpretation of the index test have introduced bias?   Low risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Yes    
Were the reference standard results interpreted without knowledge of the results of the index tests? No    
Could the reference standard, its conduct, or its interpretation have introduced bias?   High risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     High
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Yes    
Did all patients receive the same reference standard? No    
Were all patients included in the analysis?      
Could the patient flow have introduced bias?   High risk  

Teichgraber 2021.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: adults only
Setting: outpatient
Index tests Index test(s): chest CT (low‐dose CT)
Definition for positive diagnosis on CT: structured reporting was conducted according to the RSNA expert consensus statement on reporting chest CT findings related to COVID‐19.
Level of training of readers: unclear
Prevalence: 0.01
Target condition and reference standard(s) Reference standard: RT‐PCR twice, in all with initial negative results
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Yes    
Could the selection of patients have introduced bias?   Low risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Low risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Yes    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Low risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Yes    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Unclear risk  

Tsakok 2020.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: adults only
Setting: outpatient
Index tests Index test(s): chest X‐rays
Definition for positive diagnosis on CT: unclear
Level of training of readers: unclear
Prevalence: 0.4
Target condition and reference standard(s) Reference standard: RT‐PCR, no further details provided or further details are unclear
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Yes    
Could the selection of patients have introduced bias?   Low risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
DOMAIN 2: Index Test (Chest X‐ray)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Low risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Unclear    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Unclear risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Unclear
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Yes    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   High risk  

Wang 2020a.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, symptomatic or asymptomatic
Patient characteristics and setting Age group: children and adults
Setting: unclear
Index tests Index test(s): chest CT (no further details provided)
Defintion for positive diagnosis on CT: standardised imaging reporting system: infectious disease, viral pneumonia is highly likely (class 1), infectious lesions, viral pneumonia (class 2), infectious lesions, pathogens to be investigated (class 3), infectious lesions (class 4)
Level of training of readers: unclear
Prevalence: 0.2
Target condition and reference standard(s) RT‐PCR twice, if necessary
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? No    
Could the selection of patients have introduced bias?   High risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Unclear    
If a threshold was used, was it pre‐specified? No    
Could the conduct or interpretation of the index test have introduced bias?   Unclear risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Yes    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Low risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Unclear    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Unclear risk  

Wehbe 2021.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: adults only
Setting: mixed
Index tests Index test(s): chest X‐ray
Definition for positive diagnosis on X‐ray: 6‐point scoring system based on overall impression of "positive for COVID‐19" or "negative for COVID‐19"
Level of training of readers: radiologist
Prevalence: 0.4
Target condition and reference standard(s) Reference standard: RT‐PCR twice, in some with initial negative results
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Yes    
Could the selection of patients have introduced bias?   Low risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
DOMAIN 2: Index Test (Chest X‐ray)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Low risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Unclear    
Were the reference standard results interpreted without knowledge of the results of the index tests? Yes    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Unclear risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Unclear    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Low risk  

Xiaocheng 2020.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: adults only
Setting: outpatient
Index tests Index test(s): chest CT
Definition for positive diagnosis on CT: unclear
Level of training of readers: unclear
Prevalence: 0.1
Target condition and reference standard(s) Reference standard: RT‐PCR, no further details provided or further details are unclear
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Yes    
Could the selection of patients have introduced bias?   Low risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Unclear    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Unclear risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Unclear
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Yes    
Were the reference standard results interpreted without knowledge of the results of the index tests? No    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Low risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Unclear    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Unclear risk  

Xiong 2020.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, unclear symptom status
Patient characteristics and setting Age group: children and adults
Setting: inpatient
Index tests Index test(s): chest CT, no further details provided
Definition for positive diagnosis on CT: subpleural GGO without pleural effusion, bronchial changes or lymphadenopathy
Level of training of readers: radiologist
Prevalence: 0.4
Target condition and reference standard(s) Reference standard: RT‐PCR, no other details provided
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Unclear    
Could the selection of patients have introduced bias?   Low risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
Were the index test results interpreted without knowledge of the results of the reference standard? Unclear    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Unclear risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Unclear    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Unclear risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Unclear    
Did all patients receive the same reference standard? No    
Were all patients included in the analysis? Unclear    
Could the patient flow have introduced bias?   High risk  

Yassa 2020.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, symptomatic or asymptomatic
Patient characteristics and setting Age group: adults only
Setting: inpatient
Index tests Index test(s): ultrasound of the lungs (POCUS)
Definition for positive diagnosis on US: 4 categories: characteristic changes, ordinary inflammation, other changes, normal
Level of training of readers: unclear
Prevalence: for primary objective: 0.08; for secondary objective: 0.04
Target condition and reference standard(s) Reference standard: RT‐PCR twice, in some with initial negative results
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Yes    
Could the selection of patients have introduced bias?   Low risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
DOMAIN 2: Index Test (Chest X‐ray)
DOMAIN 2: Index Test (Ultrasound of the lungs)
Were the index test results interpreted without knowledge of the results of the reference standard? Unclear    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Unclear risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Unclear
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Unclear    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Unclear risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Unclear    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Unclear risk  

Yates 2021.

Study characteristics
Patient Sampling Study design: patients with suspected COVID‐19, all symptomatic
Patient characteristics and setting Age group: adults, perhaps also children
Setting: outpatient
Index tests Index test(s): chest X‐rays
Definition for positive diagnosis on X‐rays: unclear
Level of training of readers: unclear
Prevalence: 0.25
Target condition and reference standard(s) Reference standard: RT‐PCR twice, in all with initial negative results
Flow and timing  
Comparative  
Notes  
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? Yes    
Was a case‐control design avoided? Yes    
Did the study avoid inappropriate exclusions? Yes    
Could the selection of patients have introduced bias?   Low risk  
Are there concerns that the included patients and setting do not match the review question?     Low concern
DOMAIN 2: Index Test (Chest CT)
DOMAIN 2: Index Test (Chest X‐ray)
Were the index test results interpreted without knowledge of the results of the reference standard? Yes    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Low risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Low concern
DOMAIN 2: Index Test (Ultrasound of the lungs)
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? Yes    
Were the reference standard results interpreted without knowledge of the results of the index tests? Unclear    
Could the reference standard, its conduct, or its interpretation have introduced bias?   Low risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Yes    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? Yes    
Could the patient flow have introduced bias?   Low risk  

Abbreviations: ACR: American College of Radiology; AI: artificial intelligence; BSTI: British Society of Thoracic Imaging; CO‐RADS: COVID‐19 Reporting and Data System; CT: computed tomography; GGO: ground‐glass opacity; IV: intravenous; POCUS: point‐of‐care ultrasound; RSNA: Radiological Society of North America; RT‐PCR: reverse transcriptase polymerase chain reaction; STR: Society of Thoracic Radiology; US: ultrasound

Characteristics of excluded studies [ordered by study ID]

Study Reason for exclusion
Ai 2020b Ineligible study design
Ai 2020c Ineligible setting
Arentz 2020 Ineligible patient population
Bai 2020a Ineligible study design
Bai 2020b Ineligible study design
Chang 2020 < 10 participants
Chen 2020a Ineligible outcomes
Chen 2020b Ineligible outcomes
Chen 2020c Ineligible patient population
Cheng 2020 Ineligible outcomes
Çinkooğlu 2020 Ineligible study design
Colombi 2020b Ineligible outcomes
Dai 2020 Ineligible outcomes
Ding 2020 Ineligible outcomes
Dong 2020 Ineligible study design
Guan 2020 < 10 participants
Hao 2020 < 10 participants
Himoto 2020 Ineligible study design
Huang 2020 < 10 participants
Liang 2020 Ineligible study design
Lu 2020 Ineligible patient population
Mao 2020 Ineligible study design
Miao 2020a Ineligible study design
Miao 2020b Ineligible study design
Pakray 2020 Ineligible study design
Poggiali 2020 Ineligible outcomes
Pu 2020 Ineligible study design
Siegel 2020 Ineligible study design
Song 2020b Ineligible outcomes
Tavare 2020 Ineligible study design
Wang 2020b Ineligible patient population
Wu 2020a Ineligible setting
Wu 2020b Ineligible setting
Wu 2020c Ineligible patient population
Wu 2020d Ineligible patient population
Xie 2020 Ineligible study design
Xu 2020a Ineligible outcomes
Xu 2020b < 10 participants
Yang 2020a Ineligible setting
Yang 2020b Ineligible study design
Yuan 2020 Ineligible target condition
Zhifeng 2020 Ineligible study design

Differences between protocol and review

Inclusion criteria

The exclusion of case‐control studies, as well as studies that report an overview of index test findings in participants with and without the target condition, without explicitly classifying the imaging test as either COVID‐19 positive or negative, are modifications from the study protocol and Salameh 2020a, Islam 2020, and Islam 2021. These changes were made prior to initiating the update with approval by the Cochrane COVID‐19 Diagnostic Test Accuracy Group, as well as all of the review authors.

Risk of bias assessment

The criteria for the index test and reference standard domains of the QUADAS‐2 tool were modified for this update (Appendix 2). For studies that used formal scoring systems with clearly defined thresholds, even if the signalling question about using a ‘prespecified threshold’ was 'unclear' or ‘no’, the index test domain was not considered to have a ‘unclear’ or ‘high’ risk of bias based on the ‘prespecified threshold’ signalling question. For studies that used RT‐PCR testing as the reference standard, even if this signalling question about 'blinding' was 'unclear' or ‘no’, the reference standard domain was not considered to have a ‘unclear’ or ‘high’ risk of bias based on the 'blinding' signalling question. These changes were approved by the Cochrane COVID‐19 Diagnostic Test Accuracy Group, as well as all of the review authors.

Secondary objectives

We did not address several planned secondary objectives due to insufficient available data (McInnes 2020). These objectives include: evaluating the rate of positive imaging in patients with initial RT‐PCR‐negative results who have a positive result on a follow‐up RT‐PCR test; determining if there is an association between number of days after symptom onset, symptom severity and the findings on thoracic imaging for patients with COVID‐19; and determining the rate of alternative diagnoses identified by thoracic imaging.

Sensitivity analyses

We had planned to undertake additional sensitivity analyses to determine whether low risk of bias for all QUADAS‐2 domains had an effect on findings. However, since most included studies had an overall high or unclear risk of bias due to study design and only two studies had an overall low risk of bias, it was not possible to undertake these analyses.

Investigations of heterogeneity

Our protocol included additional sources of heterogeneity to be evaluated, such as disease prevalence, participant symptoms (severity), timing of symptom onset, participant co‐morbidities and other potential candidate variables. Due to the lack of available data, we did not investigate these covariates.

Limitations of previous review and changes in this update

Islam 2021 included studies of cross‐sectional or case‐control designs that either:

  1. reported specific criteria for index test positivity (i.e. used a scoring system, such as CO‐RADS);

  2. did not report specific criteria, but had the index test reader(s) explicitly classify the imaging test result as either COVID‐19 positive or negative; or

  3. reported an overview of index test findings, without having the index test reader(s) explicitly classify index tests as either COVID‐19 positive or negative.

The inclusion of case‐control studies may have been a source of bias as the disease prevalence in the sample of these types of studies do not represent the prevalence in the target population. The inclusion of studies that only reported an overview of index test findings (i.e. studies not intended to be ‘diagnostic test accuracy studies’) was a possible source of bias identified by sensitivity analysis in Islam 2021 and may have limited our ability to evaluate the sensitivity and specificity of chest CT, chest X‐ray and ultrasound. In this update, we excluded studies with case‐control designs, and studies that only reported an overview of index test findings without having the index test reader(s) explicitly classify index tests as either COVID‐19 positive or negative. The body of evidence has grown to the point that sufficient studies that meet these preferred criteria are now available.

Investigations of variability were limited in Islam 2021 due to limited available data. The assessment of secondary objectives such as the association between number of days after symptom onset, symptom severity and the findings on thoracic imaging for patients with COVID‐19 was also not possible. In this update, we evaluated the impact of reference standard conduct (RT‐PCR, performed at least twice in all initial negative results versus RT‐PCR, not performed at least twice in all initial negative results) and definition used for index test positivity (formal scoring system versus radiologist impression), but we were unable to conduct further investigations of variability due to limited available data. We also formally evaluated the impact of threshold effects on accuracy estimates in this update, particularly for studies that used the CO‐RADS scoring system. We were unable to evaluate threshold effects in other types of formal scoring systems due to the limited number of included studies that used other systems.

Of the studies included in Islam 2021, several failed to clearly report key information about their study design, as well as their methods for recruiting participants and delivering the reference standard. Therefore, data derived from these studies may have a high risk of bias and this quality of reporting and weaknesses in the primary studies reflected the overall degree of robustness of our study. In this update, several included studies also failed to report key information and had a high or unclear risk of bias with respect to participant selection, index test, reference standard, and participant flow.

The interpretation of the accuracy estimates in Islam 2021 involved several uncertainties. While RT‐PCR is considered the best available test, the results of the RT‐PCR are not always sensitive; sensitivity depends on the timing of specimen collection, with high sensitivity around the onset of symptoms and during the symptomatic period but lower sensitivity before and after that window (Kucirka 2020), and collection of an appropriate specimen for testing can also be challenging. RT‐PCR alone may not be the ideal reference standard (Li 2020b; Loeffelholz 2020), and it is possible that chest CT may be more sensitive than the reference standard in some patients, as some patients identified as having a false‐positive diagnosis on CT may have been missed by the RT‐PCR test. In this update, similar uncertainties with respect to the use of RT‐PCR as the reference standard exist. However, our meta‐regression analyses for studies that performed RT‐PCR testing at least twice for all participants with initial negative results (i.e. studies that addressed, to some extent, the low sensitivity of RT‐PCR testing by conducting at least two RT‐PCR tests to define disease‐negative status) compared with studies that did not perform repeat RT‐PCR testing for all participants with initial negative results, did not identify significantly different accuracy estimates between the groups. The quality of reporting and the design of the included studies also affected the generalizability and ability to assess the validity of our findings.

About a quarter of the studies (9/34; 26%) included in Islam 2021 were only available as preprints at the time of the search and had not yet been through the peer‐review process; of the four preprint studies that were included in Islam 2021 and also included in this update, two have since been published (publication statuses are updated as of 1 November 2020). Compared to Islam 2021, this update includes a notably smaller proportion of preprint studies (3/51; 6%). We will update data extracted from these studies and include them in future versions of our review as these studies become published in peer‐reviewed journals.

Changes to author list

The list of authors has changed between the protocol and the first review version, and has also changed with each update version. Changes to the author list since the protocol to the current review version are outlined below:

  • Added authors: Sanam Ebrahimzadeh; Nayaar Islam; Haben Dawit; Sakib Kazi; Nicholas Fabiano; Lee Treanor; Marissa Absi; Faraz Ahmad; Paul Rooprai; Ahmed Al Khalil; Kelly Harper; Neil Kamra; Junfeng Wang; Elena Pena; and Sandra Sabongui.

  • Removed authors: Trevor A McGrath and Johanna AAG Damen.

Contributions of authors

All authors reviewed, edited, contributed to, and approved this review update.

The search was performed by RS, MMGL, and LH.

Sources of support

Internal sources

  • Liverpool School of Tropical Medicine, UK

External sources

  • Foreign, Commonwealth and Development Office (FCDO), UK

    Project number: 300342‐104

  • National Institute for Health Research (NIHR), UK

  • Government of Ontario Ministry of Health COVID‐19 Rapid Response Research Grant program, Canada

  • University of Ottawa Faculty of Medicine COVID‐19 Pandemic Response Funding Program, Canada

Declarations of interest

Sanam Ebrahimzadeh has no known conflicts of interest.

Nayaar Islam has no known conflicts of interest.

Haben Dawit has no known conflicts of interest.

Jean‐Paul Salameh has no known conflicts of interest.

Sakib Kazi has no known conflicts of interest.

Nicholas Fabiano has no known conflicts of interest.

Lee Treanor has no known conflicts of interest.

Marissa Absi has no known conflicts of interest.

Faraz Ahmad has no known conflicts of interest.

Paul Rooprai has no known conflicts of interest.

Ahmed Al Khalil has no known conflicts of interest.

Kelly Harper has no known conflicts of interest.

Neil Kamra has no known conflicts of interest.

Mariska MG Leeflang has no known conflicts of interest.

Lotty Hooft has no known conflicts of interest.

Christian B van der Pol has no known conflicts of interest.

Ross Prager has no known conflicts of interest.

Samanjit S Hare has no known conflicts of interest.

Carole Dennie has no known conflicts of interest.

René Spijker: the Dutch Cochrane Centre (DCC) has received grants for performing commissioned systematic reviews. In no situation did the commissioner have any influence on the results of the work.

Jonathan J Deeks has no known conflicts of interest.

Jacqueline Dinnes has no known conflicts of interest.

Kevin Jenniskens has no known conflicts of interest.

Daniel Korevaar has no known conflicts of interest.

Jérémie F Cohen has no known conflicts of interest.

Ann Van den Bruel has no known conflicts of interest.

Yemisi Takwoingi has no known conflicts of interest.

Janneke van de Wijgert has no known conflicts of interest.

Junfeng Wang received a consultancy fee from Biomind, an Artificial Intelligence (AI) company providing machine intelligence solutions in medical imaging. The consultancy service was about design of clinical studies, not related to this review. The company had no influence on the results of the work.

Elena Pena has no known conflicts of interest.

Sandra Sabongui has no known conflicts of interest.

Matthew McInnes has no known conflicts of interest.

These authors contributed equally to this work

These authors contributed equally to this work

Edited (no change to conclusions)

References

References to studies included in this review

Ai 2020a {published data only}

  1. Ai T, Yang Z, Hou H, Zhan C, Chen C, Lv W, et al.Correlation of chest CT and RT-PCR testing for coronavirus disease 2019 (COVID-19) in China: a report of 1014 cases. Radiology 2020;296(2):E32-40. [DOI: 10.1148/radiol.2020200642] [DOI] [PMC free article] [PubMed] [Google Scholar]

Aslan 2020 {published data only}

  1. Aslan S, Bekci T, Cakir IM, Ekiz M, Yavuz I, Sahin AM.Diagnostic performance of low-dose chest CT to detect COVID-19: a Turkish population study. Diagnostic and Interventional Radiology 2020 Mar;27(2):181-7. [DOI: 10.5152/dir.2020.20350] [DOI] [PMC free article] [PubMed] [Google Scholar]

Bahrami‐Motlagh 2020 {published data only}

  1. Bahrami-Motlagh H, Abbasi S, Haghighimorad M, Salevatipour B, Alavi Darazam I, Sanei Taheri M, et al.Performance of Low-Dose Chest CT Scan for Initial Triage of COVID-19. Iranian Journal of Radiology 2020 Oct 11;17(4):e104950. [DOI: 10.5812/iranjradiol.104950] [DOI] [PMC free article] [PubMed] [Google Scholar]

Barbosa 2020 {published data only}

  1. Barbosa PN, Bitencourt AG, Miranda GD, Almeida MF, Chojniak R.Chest CT accuracy in the diagnosis of SARS-CoV-2 infection: initial experience in a cancer center. Radiologia Brasileira 2020;53(4):211-5. [DOI: 10.1590/0100-3984.2020.0040] [DOI] [PMC free article] [PubMed] [Google Scholar]

Bellini 2020 {published data only}

  1. Bellini D, Panvini N, Rengo M, Vicini S, Lichtner M, Tieghi T, et al.Diagnostic accuracy and interobserver variability of CO-RADS in patients with suspected coronavirus disease-2019: a multireader validation study. European Radiology 2020 Sep 23;31(4):1932-40. [DOI: 10.1007/s00330-020-07273-y] [DOI] [PMC free article] [PubMed] [Google Scholar]

Besutti 2020 {published data only}

  1. Besutti G, Giorgi Rossi P, Iotti V, Spaggiari L, Bonacini R, Nitrosi A, et al.Accuracy of CT in a cohort of symptomatic patients with suspected COVID-19 pneumonia during the outbreak peak in Italy. European Radiology 2020 Dec;30(12):6818-27. [DOI: 10.1007/s00330-020-07050-x] [DOI] [PMC free article] [PubMed] [Google Scholar]

Bock 2021 {published data only}

  1. Bock A, Lassen AT, Laursen CB, Posth S.Lung ultrasound as a prognostic tool in emergency patients clinically suspected of COVID-19. Danish Medical Journal 2021 Jan 7;68(2):A07200551. [PubMed] [Google Scholar]

Bollineni 2021 {published data only}

  1. Bollineni VR, Nieboer KH, Döring S, Buls N, Mey J.The role of CT imaging for management of COVID-19 in epidemic area: early experience from a University Hospital. Insights into Imaging 2021 Jan 29;12(1):10. [DOI: 10.1186/s13244-020-00957-5] [DOI] [PMC free article] [PubMed] [Google Scholar]

Borakati 2020 {published data only}

  1. Borakati A, Perera A, Johnson J, Sood T.Chest X-ray has poor diagnostic accuracy and prognostic significance in COVID-19: a propensity matched database study. medRxiv [Preprint] 2020. [DOI: 10.1101/2020.07.07.20147934] [DOI] [PMC free article] [PubMed]

Bosso 2021 {published data only}

  1. Bosso G, Allegorico E, Pagano A, Porta G, Serra C, Minerva V, et al.Lung ultrasound as diagnostic tool for SARS-CoV-2 infection. Internal and Emergency Medicine 2021 Oct 03;16(2):471-6. [DOI: 10.1007/s11739-020-02512-y] [DOI] [PMC free article] [PubMed] [Google Scholar]

Boussouar 2020 {published data only}

  1. Boussouar S, Wagner M, Donciu V, Pasi N, Salem JE, Renard-Penna R, et al.Diagnostic performance of chest computed tomography during the epidemic wave of COVID-19 varied as a function of time since the beginning of the confinement in France. PLOS One 2020 Nov 23;15(11):e0242840. [DOI: 10.1371/journal.pone.0242840] [DOI] [PMC free article] [PubMed] [Google Scholar]

Brun 2021 {published data only}

  1. Brun AL, Gence-Breney A, Trichereau J, Ballester MC, Vasse M, Chabi ML, et al.COVID-19 pneumonia: high diagnostic accuracy of chest CT in patients with intermediate clinical probability. European Radiology 2021 Apr;31(4):1969-77. [DOI: 10.1007/s00330-020-07346-y] [DOI] [PMC free article] [PubMed] [Google Scholar]

Caruso 2020 {published data only}

  1. Caruso D, Zerunian M, Polici M, Pucciarelli F, Polidori T, Rucci C, et al.Chest CT features of COVID-19 in Rome, Italy. Radiology 2020;296(2):E79-E85. [DOI: 10.1148/radiol.2020201237] [DOI] [PMC free article] [PubMed] [Google Scholar]

Cengel 2021 {published data only}

  1. Cengel F, Gurkan O, Calik M, Demirkol MA, Sargin Altunok E, Kaya MF, et al.Diagnosis of the coronavirus disease 2019 with chest computed tomography: a retrospective inter-observer agreement study between radiologists and clinicians. Hong Kong Journal of Emergency Medicine 2021 Jan 1;28(1):15-21. [DOI: 10.1177/1024907920968648] [DOI] [Google Scholar]

Colombi 2020a {published data only}

  1. Colombi D, Petrini M, Maffi G, Villani GD, Bodini FC, Morelli N, et al.Comparison of admission chest computed tomography and lung ultrasound performance for diagnosis of COVID-19 pneumonia in populations with different disease prevalence. European Journal of Radiology 2020 Dec 1;133:109344. [DOI: 10.1016/j.ejrad.2020.109344] [DOI] [PMC free article] [PubMed] [Google Scholar]

Cozzi 2020 {published data only}

  1. Cozzi A, Schiaffino S, Arpaia F, Della Pepa G, Tritella S, Bertolotti P, et al.Chest x-ray in the COVID-19 pandemic: radiologists' real-world reader performance. European Journal of Radiology 2020 Nov;132:109272. [DOI: 10.1016/j.ejrad.2020.109272] [DOI] [PMC free article] [PubMed] [Google Scholar]

Dafydd 2021 {published data only}

  1. Ap Dafydd D, O'Mahony M, Jhanji S, Devaraj A, Allum W, Nicol D, et al.The role of CT chest in screening for asymptomatic COVID-19 infection in self-isolating patients prior to elective oncological surgery: findings from a UK Cancer Hub. British Institute of Radiology 2021 Jan 1;94(1117):20200994. [DOI: 10.1259/bjr.20200994] [DOI] [PMC free article] [PubMed] [Google Scholar]

Debray 2020 {published data only}

  1. Debray M-P, Tarabay H, Males L, Chalhoub N, Mahdjoub E, Pavlovsky T, et al.Observer agreement and clinical significance of chest CT reporting in patients suspected of COVID-19. European Society of Radiology 2020;31(2):1081–9. [DOI: 10.1007/s00330-020-07126-8] [DOI] [PMC free article] [PubMed] [Google Scholar]

Deng 2020 {published data only}

  1. Deng Z, Zhang X, Li Y, Xu H, Gang Y, Wang H, et al.[The value of chest CT screening in the early stage of a new coronaviruspneumonia outbreak]. Chinese Journal of Radiology 2020 Mar 01;5(54):430-4. [DOI: 10.3760/cma.j.cn112149-20200218-00187] [DOI] [Google Scholar]

De Smet 2020 {published data only}

  1. Smet KD, Smet DD, Demedts I, Bouckaert B, Ryckaert T, Laridon E, et al.Diagnostic power of chest CT for COVID-19: to screen or not to screen. medRxiv [Preprint] 2020. [DOI: 10.1101/2020.05.18.20097444] [DOI]

Dimeglio 2021 {published data only}

  1. Dimeglio C, Collot S, Abravanel F, Sauné K, Lhomme S, Faruch M, et al.Diagnosis options in patients suffering from COVID-19-like symptoms. Journal of Medical Virology 2021;93(7):4076-7. [DOI: 10.1002/jmv.26724] [DOI] [PubMed] [Google Scholar]

Dini 2020 {published data only}

  1. Dini FL, Bergamini C, Allegrini A, Scopelliti M, Secco G, Miccoli M, et al.Bedside wireless lung ultrasound for the evaluation of COVID-19 lung injury in senior nursing home residents. Monaldi Archives for Chest Disease 2020;90(3):523-7. [DOI: 10.4081/monaldi.2020.1446] [DOI] [PubMed] [Google Scholar]

Djangang 2020 {published data only}

  1. Ndieugnou Djangang N, Peluso L, Talamonti M, Izzi A, Gevenois PA, Garufi A, et al.Eosinopenia in COVID-19 Patients: a Retrospective Analysis. Microorganisms 2020 Dec;8(12):1929. [DOI: 10.3390/microorganisms8121929] [DOI] [PMC free article] [PubMed] [Google Scholar]

Dofferhoff 2020 {published data only}

  1. Dofferhoff AS, Swinkels A, Sprong T, Berk Y, Spanbroek M, Nabuurs-Franssen MH, et al.Diagnostic algorithm for COVID-19 at the ER. Nederlands Tijdschrift voor Geneeskunde 2020;164:D5042. [PubMed] [Google Scholar]

Dogan 2020 {published data only}

  1. Doğan D, Cüce F, Akay S, Ayaz T, Kinik D, Savaşçi Ü et al.Normal chest CT prevalence in coronavirus disease 2019 (covid-19) patients: A report of 791 cases. Acta Medica Mediterranea 2020 Sep 21;36(5):2917-21. [DOI: 10.19193/0393-6384_2020_5_447] [DOI] [Google Scholar]

Ducray 2020 {published data only}

  1. Ducray V, Vlachomitrou AS, Bouscambert-Duchamp M, Si-Mohamed S, Gouttard S, Mansuy A, et al.Chest CT for rapid triage of patients in multiple emergency departments during COVID-19 epidemic: experience report from a large French university hospital. European Radiology 2020;31(2):795-803. [DOI: 10.1007/s00330-020-07154-4] [DOI] [PMC free article] [PubMed] [Google Scholar]

Erxleben 2021 {published data only}

  1. Erxleben C, Adams LC, Albrecht J, Petersen A, Vahldiek JL, Thieß H-M, et al.Improving CT accuracy in the diagnosis of COVID-19 in a hospital setting. Clinical Imaging 2021 Aug;76:1-5. [DOI: 10.1016/j.clinimag.2021.01.026] [DOI] [PMC free article] [PubMed] [Google Scholar]

Falaschi 2020 {published data only}

  1. Falaschi Z, Danna PS, Arioli R, Pasché A, Zagaria D, Percivale I, et al.Chest CT accuracy in diagnosing COVID-19 during the peak of the Italian epidemic: a retrospective correlation with RT-PCR testing and analysis of discordant cases. European Journal of Radiology 2020;130:109192. [DOI: 10.1016/j.ejrad.2020.109192] [DOI] [PMC free article] [PubMed] [Google Scholar]

Ferda 2020 {published data only}

  1. Ferda J, Vítovec M, Baxa J, Sedláček D, Havel D, Topolčan O, et al.Chest ct: A valuable tool in discrimination of covid-19 pneumonia, community acquired pneumonia and the other pathologies in slow epidemic phase. Ceska Radiologie 2020;74(3):171-9. [Google Scholar]

Fink 2021 {published data only}

  1. Fink N, Rueckel J, Kaestle S, Schwarze V, Gresser E, Hoppe B, et al.Evaluation of patients with respiratory infections during the first pandemic wave in Germany: characteristics of COVID-19 versus non-COVID-19 patients. BMC Infectious Diseases 2021 Feb 10;21(1):167. [DOI: 10.1186/s12879-021-05829-x] [DOI] [PMC free article] [PubMed] [Google Scholar]

Fonsi 2020 {published data only}

  1. Fonsi GB, Sapienza P, Brachini G, Andreoli C, De Cicco ML, Cirillo B, et al.Is lung ultrasound imaging a worthwhile procedure for severe acute respiratory syndrome coronavirus 2 pneumonia detection? Journal of Ultrasound in Medicine 2020 Sep 07;40(6):1113-23. [DOI: 10.1002/jum.15487 ] [DOI] [PubMed] [Google Scholar]

Fujioka 2020 {published data only}

  1. Fujioka T, Takahashi M, Mori M, Tsuchiya J, Yamaga E, Horii T, et al.Evaluation of the usefulness of CO-RADS for chest CT in patients suspected of having COVID-19. Diagnostics 2020;10(9):608. [DOI: 10.3390/diagnostics10090608] [DOI] [PMC free article] [PubMed] [Google Scholar]

Gaia 2020 {published data only}

  1. Gaia C, Maria Chiara C, Silvia L, Chiara A, Maria Luisa DC, Giulia B, et al.Chest CT for early detection and management of coronavirus disease (COVID-19): a report of 314 patients admitted to Emergency Department with suspected pneumonia. Radiologia Medica 2020;125(10):931-42. [DOI: ] [DOI] [PMC free article] [PubMed] [Google Scholar]

Giannitto 2020 {published data only}

  1. Giannitto C, Sposta FM, Repici A, Vatteroni G, Casiraghi E, Casari E, et al.Chest CT in patients with a moderate or high pretest probability of COVID-19 and negative swab. Radiologia Medica 2020;125(12):1260-70. [DOI: 10.1007/s11547-020-01269-w] [DOI] [PMC free article] [PubMed] [Google Scholar]

Gietema 2020 {published data only}

  1. Gietema HA, Zelis N, Nobel JM, Lambriks LJ, Van Alphen LB, Oude Lashof AM, et al.CT in relation to RT-PCR in diagnosing COVID-19 in The Netherlands: a prospective study. PLOS One 2020;15(7):e0235844. [DOI: 10.1371/journal.pone.0235844] [DOI] [PMC free article] [PubMed] [Google Scholar]

Gil‐Rodrigo 2020 {published data only}

  1. Gil-Rodrigo A, Llorens P, Martínez-Buendía C, Luque-Hernández M-J, Espinosa B, Ramos-Rincón JM.Diagnostic yield of point-of-care ultrasound imaging of the lung in patients with COVID-19. Emergencias 2020 Sep;32(5):340-4. [PubMed] [Google Scholar]

Grando 2020 {published data only}

  1. Grando RD, Brentano VB, Zanardo AP, Hertz FT, Júnior LC, Prietto dos Santos JF, et al.Clinical usefulness of tomographic standards for COVID-19 pneumonia diagnosis: Experience from a Brazilian reference center. Brazilian Journal of Infectious Diseases 2020 Nov;24(6):524-33. [DOI: 10.1016/j.bjid.2020.10.002] [DOI] [PMC free article] [PubMed] [Google Scholar]

Gross 2021 {published data only}

  1. Gross A, Heine G, Schwarz M, Thiemig D, Gläser S, Albrecht T.Structured reporting of chest CT provides high sensitivity and specificity for early diagnosis of COVID-19 in a clinical routine setting. British Journal of Radiology 2021 Jan 1;94(1117):20200574. [DOI: 10.1259/bjr.20200574] [DOI] [PMC free article] [PubMed] [Google Scholar]

Guillo 2020 {published data only}

  1. Guillo E, Bedmar Gomez I, Dangeard S, Bennani S, Saab I, Tordjman M, et al.COVID-19 pneumonia: diagnostic and prognostic role of CT based on a retrospective analysis of 214 consecutive patients from Paris, France. European Journal of Radiology 2020;131:109209. [DOI: 10.1016/j.ejrad.2020.109209] [DOI] [PMC free article] [PubMed] [Google Scholar]

Gumus 2020 {published data only}

  1. Gümüs T, Kabaoglu ZU, Coskan B, Kartal F, Artukoglu F, Atasoy KC.Preoperative computerized tomography screening for COVID‑19 pneumonia in asymptomatic patients: experiences from two centers. Japanese Journal of Radiology 2021 Mar;39(3):240-5. [DOI: 10.1007/s11604-020-01061-w] [DOI] [PMC free article] [PubMed] [Google Scholar]

Haak 2021 {published data only}

  1. Haak SL, Renken IJ, Jager LC, Lameijer H, Kolk BY.Diagnostic accuracy of point-of-care lung ultrasound in COVID-19. Emergency Medicine Journal 2021 Feb 1;38(2):94-9. [DOI: 10.1136/emermed-2020-210125] [DOI] [PubMed] [Google Scholar]

Hanif 2021 {published data only}

  1. Hanif N, Rubi G, Irshad N, Ameer S, Habib U, Zaidi SRH.Comparison of HRCT Chest and RT-PCR in Diagnosis of COVID-19. Journal of the College of Physicians and Surgeons Pakistan 2021 Jan;30(1):S1-6. [DOI: 10.29271/jcpsp.2021.01.S1] [DOI] [PubMed] [Google Scholar]

He 2020 {published data only}

  1. He J-L, Luo L, Luo Z-D, Lyu J-X, Ng M-Y, Shen X-P, et al.Diagnostic performance between CT and initial real-time RT-PCR for clinically suspected 2019 coronavirus disease (COVID-19) patients outside Wuhan, China. Respiratory Medicine 2020 Jul;168:105980. [DOI: 10.1016/j.rmed.2020.105980] [DOI] [PMC free article] [PubMed] [Google Scholar]

Hermans 2020 {published data only}

  1. Hermans JJ, Groen J, Zwets E, Boxma-De Klerk BM, Van Werkhoven JM, Ong DS, et al.Chest CT for triage during COVID-19 on the emergency department: myth or truth? Emergency Radiology 2020;27(6):641-51. [DOI: 10.1007/s10140-020-01821-1] [DOI] [PMC free article] [PubMed] [Google Scholar]

Hernigou 2020 {published data only}

  1. Hernigou J, Cornil F, Poignard A, El Bouchaibi S, Mani J, Naouri JF, et al.Thoracic computerised tomography scans in one hundred eighteen orthopaedic patients during the COVID-19 pandemic: identification of chest lesions; added values; help in managing patients; burden on the computerised tomography scan department. International Orthopaedics (SICOT) 2020 Aug;44(8):1571–80. [DOI: 10.1007/s00264-020-04651-5] [DOI] [PMC free article] [PubMed] [Google Scholar]

Herpe 2020 {published data only}

  1. Herpe G, Lederlin M, Naudin M, Ohana M, Chaumoitre K, Gregory J, et al.Efficacy of chest CT for COVID-19 pneumonia in France. Radiology 2020 Sep 1;298(2):E81-E87. [DOI: 10.1148/radiol.2020202568] [DOI] [PMC free article] [PubMed] [Google Scholar]

Hwang 2020 {published data only}

  1. Hwang EJ, Kim H, Yoon SH, Goo JM, Park CM.Implementation of a deep learning-based computer-aided detection system for the interpretation of chest radiographs in patients suspected for COVID-19. Korean Journal of Radiology 2020 Oct;21(10):1150-60. [DOI: 10.3348/kjr.2020.0536] [DOI] [PMC free article] [PubMed] [Google Scholar]

Ippolito 2020 {published data only}

  1. Ippolito D, Pecorelli A, Maino C, Capodaglio C, Mariani I, Giandola T, et al.Diagnostic impact of bedside chest X-ray features of 2019 novel coronavirus in the routine admission at the emergency department: case series from Lombardy region. European Journal of Radiology 2020;129:109092. [DOI: 10.1016/j.ejrad.2020.109092] [DOI] [PMC free article] [PubMed] [Google Scholar]

Jalil 2020 {published data only}

  1. Jalil BA, Khan A, Kugasia IR, Ijaz M.Lung ultrasound in early SARS-CoV-2 pneumonia and the LUS-CoV criteria. Baylor University Medical Center Proceedings 2020 Oct 26;34(1):1-4. [DOI: 10.1080/08998280.2020.1834658] [DOI] [PMC free article] [PubMed] [Google Scholar]

Krdzalic 2020 {published data only}

  1. Krdzalic J, De Jaegere TM, Kwee RM.Diagnostic performance of chest CT in screening patients with suspected COVID-19 infection in a Western population. Brtish Jounal of Radiology 2020 Sep 1;93(1113):20200643. [DOI: 10.1259/bjr.20200643] [DOI] [PMC free article] [PubMed] [Google Scholar]

Kuzan 2020 {published data only}

  1. Kuzan TY, Murzoğlu Altıntoprak K, Çiftçi HÖ, Ergül U, Ünal Özdemir NB, Bulut M, et al.A comparison of clinical, laboratory and chest CT findings of laboratory-confirmed and clinically diagnosed COVID-19 patients at first admission. Diagnostic and Interventional Radiology 2020 May;27(3):336-43. [DOI: 10.5152/dir.2020.20270] [DOI] [PMC free article] [PubMed] [Google Scholar]

Lieveld 2021a {published data only}

  1. Lieveld AW, Azijli K, Teunissen BP, Haaften RM, Kootte RS, van den Berk IA, et al.Chest CT in COVID-19 at the ED: validation of the COVID-19 Reporting and Data System (CO-RADS) and CT Severity Score: a prospective, multicenter, observational study. Chest 2021 Mar 1;159(3):1126-35. [DOI: 10.1016/j.chest.2020.11.026] [DOI] [PMC free article] [PubMed] [Google Scholar]

Lieveld 2021b {published data only}

  1. Lieveld AW, Kok B, Schuit FH, Azijli K, Heijmans J, Laarhoven A, et al.Diagnosing COVID-19 pneumonia in a pandemic setting: Lung Ultrasound versus CT (LUVCT) – a multicentre, prospective, observational study. ERJ Open Research 2020 Oct;6(4):00539-2020. [DOI: 10.1183/23120541.00539-2020] [DOI] [PMC free article] [PubMed] [Google Scholar]

Luo 2020a {published data only}

  1. Luo L, Luo Z, Jia Y, Zhou C, He J, Lyu J, et al.CT differential diagnosis of COVID-19 and non-COVID-19 in symptomatic suspects: a practical scoring method. BMC Pulmonary Medicine 2020 May 07;20(1):129. [DOI: 10.1186/s12890-020-1170-6] [DOI] [PMC free article] [PubMed] [Google Scholar]

Majeed 2020 {published data only}

  1. Majeed T, Ali RS, Solomon J, Mesri M, Sharma S, Shamim S, et al.The role of the computed tomography (CT) thorax in the diagnosis of COVID-19 for patients presenting with acute surgical emergencies. a single institute experience. Indian Journal of Surgery 2020 Oct 20;82(6):1005-10. [DOI: 10.1007/s12262-020-02626-9] [DOI] [PMC free article] [PubMed] [Google Scholar]

Mei 2020 {published data only}

  1. Mei X, Lee H-C, Diao K, Huang M, Lin B, Liu C, et al.Artificial intelligence-enabled rapid diagnosis of COVID-19 patients. Nature Medicine 2020 Aug;26(8):1224-8. [DOI: 10.1038/s41591-020-0931-3] [DOI] [PMC free article] [PubMed]

Miranda Magalhaes Santos 2020 {published data only}

  1. Miranda Magalhães Santos JM, Paula Alves Fonseca A, Pinheiro Zarattini Anastacio E, Formagio Minenelli F, Furtado de Albuquerque Cavalcanti C, Borges da Silva Teles G.Initial results of the use of a standardized diagnostic criteria for chest computed tomography findings in coronavirus disease 2019. Journal of Computer Assisted Tomography 2020;44(5):647-51. [DOI: 10.1097/RCT.0000000000001054] [DOI] [PubMed] [Google Scholar]

Moroni 2021 {published data only}

  1. Moroni C, Cozzi D, Albanesi M, Cavigli E, Bindi A, Luvarà S, et al.Chest X-ray in the emergency department during COVID-19 pandemic descending phase in Italy: correlation with patients' outcome. Radiologia Medica 2021 May;126(5):661-8. [DOI: 10.1007/s11547-020-01327-3] [DOI] [PMC free article] [PubMed] [Google Scholar]

Murphy 2020 {published data only}

  1. Murphy K, Smits H, Knoops AJ, Korst MB, Samson T, Scholten ET, et al.COVID-19 on chest radiographs: a multireader evaluation of an artificial intelligence system. Radiology 2020;296(3):E166-72. [DOI: 10.1148/radiol.202020187] [DOI] [PMC free article] [PubMed] [Google Scholar]

Narinx 2020 {published data only}

  1. Narinx N, Smismans A, Symons R, Frans J, Demeyere A, Gillis M.Feasibility of using point-of-care lung ultrasound for early triage of COVID-19 patients in the emergency room. Emergency Radiology 2020;27(6):663-70. [DOI: 10.1007/s10140-020-01849-3] [DOI] [PMC free article] [PubMed] [Google Scholar]

Nivet 2021 {published data only}

  1. Nivet H, Crombé A, Schuster P, Ayoub T, Pourriol L, Favard N, et al.The accuracy of teleradiologists in diagnosing COVID-19 based on a French multicentric emergency cohort. European Radiology 2021 May 1;31(5):2833-44. [DOI: 10.1007/s00330-020-07345-z] [DOI] [PMC free article] [PubMed] [Google Scholar]

O'Neill 2020 {published data only}

  1. O'Neill SB, Byrne D, Müller NL, Jalal S, Parker W, Nicolaou S, et al.Radiological Society of North America (RSNA) Expert consensussStatement related to chest CT findings in COVID-19 versus CO-RADS: comparison of reportingsSystem performance among chest radiologists and end-user preference. Canadian Association of Radiologists' Journal 2020;0846537120968919:Epub 2020 Nov 3. PMID: 33138634. [DOI: 10.1177/0846537120968919] [DOI] [PubMed] [Google Scholar]

Ohana 2021 {published data only}

  1. Ohana M, Muller J, Severac F, Bilbault P, Behr M, Oberlin M, et al.Temporal variations in the diagnostic performance of chest CT for Covid-19 depending on disease prevalence: Experience from North-Eastern France. European Journal of Radiology 2021 Jan;134:109425. [DOI: 10.1016/j.ejrad.2020.109425] [DOI] [PMC free article] [PubMed] [Google Scholar]

Ooi 2021 {published data only}

  1. Ooi MW, Liong SY, Baguley N, Sharman A, Tuck J.Role of complementary Ct chest in patients presenting with acute abdominal symptoms during covid-19 pandemic: a UK experience. Clinical Imaging 2021 Jan;69:289-92. [DOI: 10.1016/j.clinimag.2020.09.00] [DOI] [PMC free article] [PubMed] [Google Scholar]

Pagano 2021 {published data only}

  1. Pagano A, Finkelstein M, Overbey J, Steinberger S, Ellison T, Manna S, et al.Portable chest radiography as an exclusionary test for adverse clinical outcomes during the COVID-19 pandemic. Chest 2021 Jul;160(1):238-48. [DOI: 10.1016/j.chest.2021.01.053] [DOI] [PMC free article] [PubMed] [Google Scholar]

Palmisano 2021 {published data only}

  1. Palmisano A, Scotti GM, Ippolito D, Morelli MJ, Vignale D, Gandola D, et al.Chest CT in the emergency department for suspected COVID-19 pneumonia. Radiologia Medica 2021 Mar 1;126(3):498-502. [DOI: 10.1007/s11547-020-01302-y] [DOI] [PMC free article] [PubMed] [Google Scholar]

Pare 2020 {published data only}

  1. Pare J, Camelo I, Mayo K, Leo M, Dugas J, Nelson K, et al.Point-of-care lung ultrasound is more sensitive than chest radiograph for evaluation of COVID-19. Western Journal of Emergency Medicine 2020 Jun 19;21(4):771-8. [DOI: 10.5811/westjem.2020.5.47743] [DOI] [PMC free article] [PubMed] [Google Scholar]

Patel 2020 {published data only}

  1. Patel M, Chowdhury J, Zheng M, Abramian O, Verga S, Zhao H, et al.High Resolution CHEST CT(HRCT) evaluation in patients hospitalized with COVID-19 infection. medRxiv [Preprint] 2020;203(4). [DOI: 10.1101/2020.05.26.20114082] [DOI]

Patrucco 2021 {published data only}

  1. Patrucco F, Carriero A, Falaschi Z, Paschè A, Gavelli F, Airoldi C, et al.COVID-19 diagnosis in case of two negative nasopharyngeal swabs: association between chest CT and bronchoalveolar lavage results. Radiology 2021 Jan 5;298(3):E152-5. [DOI: 10.1148/radiol.2020203776] [DOI] [PMC free article] [PubMed] [Google Scholar]

Peng 2020a {published data only}

  1. Peng D, Zhang J, Xu Y, Liu Z, Wu P.Clinical analysis and early differential diagnosis of suspected pediatric patients with 2019 novel coronavirus infection. medRxiv [Preprint] 2020. [DOI: 10.1101/2020.04.07.20057315] [DOI]

Pivetta 2021 {published data only}

  1. Pivetta E, Goffi A, Tizzani M, Locatelli SM, Porrino G, Losano I, et al.Lung ultrasonography for the diagnosis of SARS-CoV-2 pneumonia in the emergency department. Annals of Emergency Medicine 2021 Apr 1;77(4):385-94. [DOI: 10.1016/j.annemergmed.2020.10.008] [DOI] [PMC free article] [PubMed] [Google Scholar]

Puylaert 2020 {published data only}

  1. Puylaert CA Scheijmans JC, Borgstein AB, Andeweg CS, Bartels-Rutten A, Beets GL, et al.Yield of Screening for COVID-19 in asymptomatic patients prior to elective or emergency surgery using chest CT and RT-PCR (SCOUT): multicenter study. Annals of Surgery 2020 Dec;272(6):919-24. [DOI: 10.1097/SLA.0000000000004218] [DOI] [PMC free article] [PubMed] [Google Scholar]

Ravikanth 2021 {published data only}

  1. Ravikanth R.Diagnostic accuracy and false-positive rate of chest CT as compared to RT-PCR in coronavirus disease 2019 (COVID-19) pneumonia: a prospective cohort of 612 cases from India and review of literature. Indian Journal of Radiology & Imaging 2021 Jan;31(Suppl 1):S161-9. [DOI: 10.4103/ijri.IJRI_377_20] [DOI] [PMC free article] [PubMed] [Google Scholar]

Reginelli 2021 {published data only}

  1. Reginelli A, Grassi R, Feragalli B, Belfiore MP, Montanelli A, Patelli G, et al.Coronavirus disease 2019 (COVID-19) in Italy: double reading of Ccest CT examination. Biology (Basel) 2021 Jan 25;10(2):89. [DOI: 10.3390/biology10020089] [DOI] [PMC free article] [PubMed] [Google Scholar]

Rona 2021 {published data only}

  1. Rona G, Voyvoda N, Arifoğlu M, Karaaslan A, Çetin C.The efficacy of chest computed tomography in pediatric patients with suspected COVID-19. Journal of Computer Assisted Tomography 2021 Mar 1;45(2):337-41. [DOI: 10.1097/RCT.0000000000001127] [DOI] [PubMed] [Google Scholar]

Roy Choudhury 2020 {published data only}

  1. Choudhury RS, Shahi PK, Sharma S, Dhar R.Utility of chest radiography on admission for initial triaging of COVID-19 in symptomatic patients. ERJ Open Research 2020 Jul;6(3):00357-2020. [DOI: 10.1183/23120541.00357-2020] [DOI] [PMC free article] [PubMed] [Google Scholar]

Saeed 2020 {published data only}

  1. Saeed GA, Helali AA, Almazrouei S, Shah A, Ahmed LA.Chest CT features of COVID-19 in the region of Abu Dhabi, UAE: a single institute study. Chinese Journal of Academic Radiology 2021;4:248–56. [DOI: 10.1007/s42058-021-00075-1] [DOI] [PMC free article] [PubMed] [Google Scholar]

Salehi‐Pourmehr 2020 {published data only}

  1. Salehi-Pourmehr H, Pourfathi H, Tarzamni MK, Ghojazadeh M, Naghili B, Zarrintan A, et al.Diagnostic value of chest CT in Iranian patients with suspected COVID-19. Caspian Journal of Internal Medicine 2020;11(Suppl 1):527-30. [DOI: 10.22088/cjim.11.0.527] [DOI] [PMC free article] [PubMed] [Google Scholar]

Schalekamp 2020 {published data only}

  1. Schalekamp S, Bleeker-Rovers CP, Beenen LF, Quarles van Ufford HM, Gietema HA, Stöger JL, et al.Chest CT in the emergency department for diagnosis of COVID-19 pneumonia: Dutch experience. Radiology 2020 Nov 17;298(2):E98-106. [DOI: 10.1148/radiol.2020203465] [DOI] [PMC free article] [PubMed] [Google Scholar]

Schmid 2020 {published data only}

  1. Schmid B, Feuerstein D, Lang CN, Fink K, Steger R, Rieder M, et al.Lung ultrasound in the emergency department - a valuable tool in the management of patients presenting with respiratory symptoms during the SARS-CoV-2 pandemic. BMC Emergency Medicine 2020 Dec 7;20(1):96. [DOI: 10.1186/s12873-020-00389-w] [DOI] [PMC free article] [PubMed] [Google Scholar]

Schulze‐hagen 2020 {published data only}

  1. Schulze-Hagen M, Hübel C, Meier-Schroers M, Yüksel C, Sander A, Sähn M, et al.Low-dose chest CT for the diagnosis of COVID-19—a systematic, prospective comparison with PCR. Deutsches Aerzteblatt Online 2020;117(22-23):389-95. [DOI: 10.3238/arztebl.2020.0389] [DOI] [PMC free article] [PubMed] [Google Scholar]

Shah 2021 {published data only}

  1. Shah JV, Shah C, Shah S, Gandhi N, Dikshit NA, Patel P, et al.HRCT chest in COVID-19 patients: An initial experience from a private imaging center in western India. Indian Journal of Radiology & Imaging 2021 Jan;31(Suppl 1):S182-6. [DOI: 10.4103/ijri.IJRI_405_20] [DOI] [PMC free article] [PubMed] [Google Scholar]

Skalidis 2020 {published data only}

  1. Skalidis I, Nguyen VK, Bothorel H, Poli L, Da Costa RR, Younossian AB, et al.Unenhanced computed tomography (CT) utility for triage at the emergency department during COVID-19 pandemic. American Journal of Emergency Medicine 2021 Aug;46:260-5. [DOI: 10.1016/j.ajem.2020.07.058] [DOI] [PMC free article] [PubMed] [Google Scholar]

Song 2020a {published data only}

  1. Song S, Wu F, Liu Y, Jiang H, Xiong F, Guo X, et al.Correlation between chest CT findings and clinical features of 211 COVID-19 suspected patients in Wuhan, China. Open Forum Infectious Diseases 2020 Jun;7(6):ofaa171. [DOI: 10.1093/ofid/ofaa171] [DOI] [PMC free article] [PubMed] [Google Scholar]

Sorlini 2021 {published data only}

  1. Sorlini C, Femia M, Nattino G, Bellone P, Gesu E, Francione P, et al.The role of lung ultrasound as a frontline diagnostic tool in the era of COVID-19 outbreak. Internal and Emergency Medicine 2021 Apr;16(3):749-56. [DOI: 10.1007/s11739-020-02524-] [DOI] [PMC free article] [PubMed] [Google Scholar]

Speidel 2021 {published data only}

  1. Speidel V, Conen A, Gisler V, Fux CA, Haubitz S.Lung assessment with point-of-care ultrasound in respiratory coronavirus disease (COVID-19): a prospective cohort study. Ultrasound in Medicine & Biology 2021 Apr 1;47(4):896-901. [DOI: 10.1016/j.ultrasmedbio.2020.12.021] [DOI] [PMC free article] [PubMed] [Google Scholar]

Steuwe 2020 {published data only}

  1. Steuwe A, Rademacher C, Valentin B, Köhler M-H, Appel E, Keitel V, et al.Dose-optimised chest computed tomography for diagnosis of coronavirus disease 2019 (COVID-19) - evaluation of image quality and diagnostic impact. Journal of Radiological Protection 2020;40(3):877-91. [10.1088/1361-6498/aba16a] [DOI] [PubMed] [Google Scholar]

Stevens 2020 {published data only}

  1. Stevens BJ.Reporting radiographers' interpretation and use of the British Society of Thoracic Imaging's coding system when reporting COVID-19 chest X-rays. Radiography 2020 Feb;27(1):90-4. [DOI: 10.1016/j.radi.2020.06.010] [DOI] [PMC free article] [PubMed] [Google Scholar]

Sukhija 2021 {published data only}

  1. Sukhija A, Mahajan M, Joshi PC, Dsouza J, Seth ND, Patil KH.Radiographic findings in COVID-19: Comparison between AI and radiologist. Indian Journal of Radiology and Imaging 2021 Jan;31(Suppl 1):S87-93. [10.4103/ijri.IJRI_777_20] [DOI] [PMC free article] [PubMed] [Google Scholar]

Sverzellati Nicola 2021 {published data only}

  1. Sverzellati N, Ryerson CJ, Milanese G, Renzoni EA, Volpi A, Spagnolo P, et al.Chest x-ray or CT for COVID-19 pneumonia? Comparative study in a simulated triage setting. European Respiratory Journal 2021 Sep 9;58(3):2004188. [DOI: 10.1183/13993003.04188-2020] [DOI] [PMC free article] [PubMed] [Google Scholar]

Teichgraber 2021 {published data only}

  1. Teichgräber U, Malouhi A, Ingwersen M, Neumann R, Reljic M, Deinhardt-Emmer S, et al.Ruling out COVID-19 by chest CT at emergency admission when prevalence is low: the prospective, observational SCOUT study. Respiratory Research 2021 Jan 12;22(1):13. [10.1186/s12931-020-01611-w] [DOI] [PMC free article] [PubMed] [Google Scholar]

Tsakok 2020 {published data only}

  1. Tsakok M, Shaw R, Murchison A, Ather S, Xie C, Watson R, et al.Diagnostic accuracy of initial chest radiograph compared to SARS-CoV-2 PCR in patients with suspected COVID-19. BJR Open 2020 Aug 5;2(1):20200034. [DOI: 10.1259/bjro.20200034] [DOI] [PMC free article] [PubMed] [Google Scholar]

Wang 2020a {published data only}

  1. Wang T, Xiong Z, Zhou H, Luo W, Tang H, Liu J.Design, validation, and clinical practice of standardized imaging diagnostic report for COVID-19. Zhong Nan da Xue Xue Bao Yi Xue Ban [Journal of Central South University Medical Sciences] 2020 Mar 28;45(3):229-35. [DOI: 10.11817/j.issn.1672-7347.2020.200152] [DOI] [PubMed] [Google Scholar]

Wehbe 2021 {published data only}

  1. Wehbe RM, Sheng J, Dutta S, Chai S, Dravid A, Barutcu S, et al.DeepCOVID-XR: an artificialiIntelligence algorithm to detect COVID-19 on chest radiographs trained and tested on a large U.S. clinical data set. Radiology 2021 Apr;299(1):E167-76. [DOI: 10.1148/radiol.2020203511] [DOI] [PMC free article] [PubMed] [Google Scholar]

Xiaocheng 2020 {published data only}

  1. Xiaocheng X, Haiyan J, Xiaoping C, Yi Z, Huiyuan S.Preliminary analysis of visits to fever clinics during the epidemic of new coronavirus pneumonia. Academic Journal of Second Military Medical University 2020;41(8):828-31. [DOI: ] [DOI] [Google Scholar]

Xiong 2020 {published data only}

  1. Xiong Z, Fu L, Zhou H, Liu JK, Wang AM, Huang Y, et al.Construction and evaluation of a novel diagnosis pathway for 2019-Corona virus disease. Zhonghua Yi Xue za Zhi 2020 Apr 28;100(16):1223-9. [DOI: 10.3760/cma.j.cn112137-20200228-00499] [DOI] [PubMed] [Google Scholar]

Yassa 2020 {published data only}

  1. Yassa M, Yirmibes C, Cavusoglu G, Eksi H, Dogu C, Usta C, et al.Outcomes of universal SARS-CoV-2 testing program in pregnant women admitted to hospital and the adjuvant role of lung ultrasound in screening: a prospective cohort study. Journal of Maternal-Fetal & Neonatal Medicine 2020 Nov;33(22):3820-6. [DOI: 10.1080/14767058.2020.1798398] [DOI] [PubMed] [Google Scholar]

Yates 2021 {published data only}

  1. Yates A, Dempsey PJ, Vencken S, MacMahon PJ, Hutchinson BD.Structured reporting in portable chest radiographs: An essential tool in the diagnosis of COVID-19. European Journal of Radiology 2021 Jan;134:109414. [DOI: 10.1016/j.ejrad.2020.109414] [DOI] [PMC free article] [PubMed] [Google Scholar]

References to studies excluded from this review

Ai 2020b {published data only}

  1. Ai J, Gong J, Xing L, He R, Tian F, Wang J, et al.Analysis of factors associated early diagnosis in coronavirus disease 2019 (COVID-19). medRxiv [Preprint] 2020. [DOI: 10.1101/2020.04.09.20059352] [DOI]

Ai 2020c {published data only}

  1. Ai J, Chen J, Wang Y, Liu X, Fan W, Qu G, et al.The cross-sectional study of hospitalized coronavirus disease 2019 patients in Xiangyang, Hubei province. medRxiv [Preprint] 2020. [DOI: 10.1101/2020.02.19.20025023] [DOI]

Arentz 2020 {published data only}

  1. Arentz M, Yim E, Klaff L, Lokhandwala S, Riedo FX, Chong M, et al.Characteristics and outcomes of 21 critically ill patients with COVID-19 in Washington State. JAMA 2020;323(16):1612-4. [DOI] [PMC free article] [PubMed] [Google Scholar]

Bai 2020a {published data only}

  1. Bai HX, Hsieh B, Xiong Z, Halsey K, Choi JW, Tran TM, et al.Performance of radiologists in differentiating COVID-19 from viral pneumonia on chest CT. Radiology 2020;296(2):E46-E54. [DOI] [PMC free article] [PubMed] [Google Scholar]

Bai 2020b {published data only}

  1. Bai HX, Wang R, Xiong Z, Hsieh B, Chang K, Halsey K, et al.Artificial intelligence augmentation of radiologist performance in distinguishing COVID-19 from pneumonia of other origin at chest CT. Radiology 2020;296(3):E156-65. [DOI] [PMC free article] [PubMed] [Google Scholar]

Chang 2020 {published data only}

  1. Chang D, Lin M, Wei L, Xie L, Zhu G, Dela Cruz CS, et al.Epidemiologic and clinical characteristics of novel coronavirus infections involving 13 patients outside Wuhan, China. JAMA 2020;323(11):1092-3. [DOI: 10.1001/jama.2020.1623 10.1001/jama.2020.1623.] [DOI] [PMC free article] [PubMed] [Google Scholar]

Chen 2020a {published data only}

  1. Chen S, Wu JJ, Li MZ, Xu DZ, Zhu YZ, Wang HC, et al.Clinical features of 109 cases of novel coronavirus pneumonia. Chinese Journal of Infectious Diseases 2020;38(12):E015. [DOI: 10.3760 / cma.j.issn.1000-6680.2020.0015] [Google Scholar]

Chen 2020b {published data only}

  1. Chen X, Tang Y, Mo Y, Li S, Lin D, Yang Z, et al.A diagnostic model for coronavirus disease 2019 (COVID-19) based on radiological semantic and clinical features: a multi-center study. European Radioliology 2020;30(9):4893-902. [DOI: 10.1007/s00330-020-06829-2] [DOI] [PMC free article] [PubMed] [Google Scholar]

Chen 2020c {published data only}

  1. Chen Z, Fan H, Cai J, Li Y, Wu B, Hou Y, et al.High-resolution computed tomography manifestations of COVID-19 infections in patients of different ages. European Journal of Radiology 2020;126:108972. [DOI] [PMC free article] [PubMed] [Google Scholar]

Cheng 2020 {published data only}

  1. Cheng Z, Lu Y, Cao Q, Qin L, Pan Z, Yan F, et al.Clinical features and chest CT manifestations of coronavirus disease 2019 (COVID-19) in a single-center study in Shanghai, China. American Journal of Roentgenology 2020;215(1):121-6. [DOI: 10.2214/ajr.20.22959 10.2214/AJR.20.22959.] [DOI] [PubMed] [Google Scholar]

Çinkooğlu 2020 {published data only}

  1. Çinkooğlu A, Bayraktaroğlu S, Savaş R.Lung changes on chest CT during 2019 novel coronavirus (COVID-19) pneumonia. European Journal of Breast Health 2020;16(2):89-90. [DOI: 10.5152/ejbh.2020.010420] [DOI] [PMC free article] [PubMed] [Google Scholar]

Colombi 2020b {published data only}

  1. Colombi D, Bodin FC, Petrini M, Maffi G, Morelli N, Milanese G, et al.Well-aerated lung on admitting chest CT to predict adverse outcome in COVID-19 pneumonia. Radiology 2020;296(2):E86-E96. [DOI: 10.1148/radiol.2020201433] [DOI] [PMC free article] [PubMed] [Google Scholar]

Dai 2020 {published data only}

  1. Dai H, Zhang X, Xia J, Zhang T, Shang Y, Huang R, et al.High-resolution chest CT features and clinical characteristics of patients infected with COVID-19 in Jiangsu, China. International Journal of Infectious Diseases 2020;95:106-12. [DOI: ] [DOI] [PMC free article] [PubMed] [Google Scholar]

Ding 2020 {published data only}

  1. Ding X, Xu J, Zhou J, Long Q.Chest CT findings of COVID-19 pneumonia by duration of symptoms. European Journal of Radiology 2020;127:109009. [DOI: 10.1016/j.ejrad.2020.109009] [DOI] [PMC free article] [PubMed] [Google Scholar]

Dong 2020 {published data only}

  1. Dong J, Wu L, Jin Q, Chen J, He J.Chest CT scan of hospitalized patients with COVID-19: a case-control study. medRxiv [Preprint] 2020. [DOI: 10.1101/2020.04.07.20056762] [DOI]

Guan 2020 {published data only}

  1. Guan W, Liu J, Yu C.CT findings of coronavirus disease (COVID-19) severe pneumonia. American Journal of Roentgenology 2020;5:W85-W6. [DOI: 10.2214/AJR.20.23035] [DOI] [PubMed] [Google Scholar]

Hao 2020 {published data only}

  1. Hao W, Li M.Clinical diagnostic value of CT imaging in COVID-19 with multiple negative RT-PCR testing. Travel Medicine and Infectious Disease 2020;34:101627. [DOI: 10.1016/j.tmaid.2020.101627] [DOI] [PMC free article] [PubMed] [Google Scholar]

Himoto 2020 {published data only}

  1. Himoto Y, Sakata A, Kirita M, Hiroi T, Kobayashi K I, Kubo K, et al.Diagnostic performance of chest CT to differentiate COVID-19 pneumonia in non-high-epidemic area in Japan. Japanese Journal of Radiology 2020;38(5):400-6. [DOI] [PMC free article] [PubMed] [Google Scholar]

Huang 2020 {published data only}

  1. Huang P, Liu T, Huang L, Liu H, Lei M, Xu W, et al.Use of chest CT in combination with negative RT-PCR assay for the 2019 novel coronavirus but high clinical suspicion. Radiology 2020;295(1):22-3. [DOI: 10.1148/radiol.2020200330 10.1148/radiol.2020200330. Epub 2020 Feb 12.] [DOI] [PMC free article] [PubMed] [Google Scholar]

Liang 2020 {published data only}

  1. Liang Y, Liang J, Zhou Q, Li X, Lin F, Deng Z, et al.Prevalence and clinical features of 2019 novel coronavirus disease (COVID-19) in the Fever Clinic of a teaching hospital in Beijing: a single-center, retrospective study. medRxiv [Preprint] 2020. [DOI: 10.1101/2020.02.25.20027763] [DOI]

Lu 2020 {published data only}

  1. Lu X, Zhang L, Du H, Zhang J, Li Y, Qu J, et al.SARS-CoV-2 infection in children. New England Journal of Medicine 2020;382(17):1663-5. [DOI: 10.1056/NEJMc2005073] [DOI] [PMC free article] [PubMed] [Google Scholar]

Mao 2020 {published data only}

  1. Mao X, Liu X-P, Xiong M, Yang X, Jin X, Li Z, et al.Development and validation of chest CT-based imaging biomarkers for early stage COVID-19 screening. medRxiv [Preprint] 2020. [DOI: 10.1101/2020.05.15.20103473] [DOI] [PMC free article] [PubMed]

Miao 2020a {published data only}

  1. Miao C, Zhuang J, Jin M, Xiong H, Huang P, Zhao Q, et al.A comparative multi-centre study on the clinical and imaging features of confirmed and unconfirmed patients with COVID-19. medRxiv [Preprint] 2020. [DOI: 10.1101/2020.03.22.20040782] [DOI]

Miao 2020b {published data only}

  1. Miao C, Jin M, Miao L, Yang X, Huang P, Xiong H, et al.Early chest computed tomography to diagnose COVID-19 from suspected patients: a multicenter retrospective study. American Journal of Emergency Medicine 2020;S0735-6757(20):30281-3. [DOI] [PMC free article] [PubMed] [Google Scholar]

Pakray 2020 {published data only}

  1. Pakray A, Walker D, Figacz A, Kilanowski S, Rhodes C, Doshi S, et al.Imaging evaluation of COVID-19 in the emergency department. Emergency Radiology 2020;27:579-88. [DOI] [PMC free article] [PubMed] [Google Scholar]

Poggiali 2020 {published data only}

  1. Poggiali E, Dacrema A, Bastoni D, Tinelli V, Demichele E, Mateo Ramos P, et al.Can lung US help critical care clinicians in the early diagnosis of novel coronavirus (COVID-19) pneumonia? Radiology 2020;295(E6):200847. [DOI: 10.1148/radiol.2020200847] [DOI] [PMC free article] [PubMed] [Google Scholar]

Pu 2020 {published data only}

  1. Pu J, Leader J, Bandos A, Shi J, Du P, Yu J, et al.Any unique image biomarkers associated with COVID-19? European Radiology 2020;30(NA):6221–7. [DOI] [PMC free article] [PubMed] [Google Scholar]

Siegel 2020 {published data only}

  1. Siegel A, Chang PJ, Jarou ZJ, Paushter DM, Harmath CB, Arevalo JB, et al.Lung base findings of coronavirus disease (COVID-19) on abdominal CT in patients with predominant gastrointestinal symptoms. American Journal of Roentgenology 2020;215(3):607-9. [DOI: 10.2214/AJR.20.23232] [DOI] [PubMed] [Google Scholar]

Song 2020b {published data only}

  1. Song F, Shi N, Shan F, Zhang Z, Shen J, Lu H, et al.Emerging 2019 novel Coronavirus (2019-nCoV) pneumonia. Radiology 2020;295(1):210-7. [DOI: 10.1148/radiol.2020200274 10.1148/radiol.2020200274. Epub 2020 Feb 6.] [DOI] [PMC free article] [PubMed] [Google Scholar]

Tavare 2020 {published data only}

  1. Tavare AN, Braddy A, Brill S, Jarvis H, Sivaramakrishnan A, Barnett J, et al.Managing high clinical suspicion COVID-19 inpatients with negative RT-PCR: a pragmatic and limited role for thoracic CT. Thorax 2020;75(7):537-8. [DOI: ] [DOI] [PMC free article] [PubMed] [Google Scholar]

Wang 2020b {published data only}

  1. Wang Y, Dong C, Hu Y, Li C, Ren Q, Zhang X, et al.Temporal changes of CT findings in 90 patients with COVID-19 pneumonia: a longitudinal study. Radiology 2020;296(2):E55-E64. [DOI: 10.1148/radiol.2020200843 10.1148/radiol.2020200843.] [DOI] [PMC free article] [PubMed] [Google Scholar]

Wu 2020a {published data only}

  1. Wu J, Feng CL, Xian XY, Qiang J, Zhang J, Mao QX, et al.Novel coronavirus pneumonia (COVID-19) CT distribution and sign features. Zhonghua Jie He He Hu Xi za Zhi [Chinese Journal of Tuberculosis and Respiratory Diseases] 2020;43(4):321-6. [DOI: 10.3760/cma.j.cn112147-20200217-00106 10.3760/cma.j.cn112147-20200217-00106.] [DOI] [PubMed] [Google Scholar]

Wu 2020b {published data only}

  1. Wu Q, Xing Y, Shi L, Li W, Gao Y, Pan S, et al.Epidemiological and clinical characteristics of children with coronavirus disease 2019. medRxiv [Preprint] 2020. [DOI: 10.1101/2020.03.19.20027078] [DOI]

Wu 2020c {published data only}

  1. Wu X, Sun R, Chen J, Xie Y, Zhang S, Wang X.Radiological findings and clinical characteristics of pregnant women with COVID-19 pneumonia. International Journal of Gynaecology and Obstetrics 2020;150(1):58-63. [DOI: 10.1002/ijgo.13165] [DOI] [PMC free article] [PubMed] [Google Scholar]

Wu 2020d {published data only}

  1. Wu X, Fu B, Chen L, Feng Y.Serological tests facilitate identification of asymptomatic SARS-CoV-2 infection in Wuhan, China. Journal of Medical Virology 2020;10:1002/jmv.25904. [DOI: 10.1002/jmv.25904] [DOI] [PMC free article] [PubMed] [Google Scholar]

Xie 2020 {published data only}

  1. Xie S, Lei Z, Chen X, Liu W, Wang X, Dong Y, et al.Chest CT-based differential diagnosis of 28 patients with suspected corona virus disease 2019 (COVID-19). British Journal of Radiology 2020;93(1112):20200243. [DOI] [PMC free article] [PubMed] [Google Scholar]

Xu 2020a {published data only}

  1. Xu X, Yu C, Qu J, Zhang L, Jiang S, Huang D, et al.Imaging and clinical features of patients with 2019 novel coronavirus SARS-CoV-2. European Journal of Nuclear Medicine and Molecular Imaging 2020;47(5):1275-80. [DOI: 10.1007/s00259-020-04735-9 10.1007/s00259-020-04735-9.] [DOI] [PMC free article] [PubMed] [Google Scholar]

Xu 2020b {published data only}

  1. Xu X, Yu C, Zhang L, Luo L, Liu J.Imaging features of 2019 novel coronavirus pneumonia. European Journal of Nuclear Medicine and Molecular Imaging 2020;47(5):1022-23. [DOI: 10.1007/s00259-020-04720-2] [DOI] [PMC free article] [PubMed] [Google Scholar]

Yang 2020a {published data only}

  1. Yang S, Shi Y, Lu H, Xu J, Li F, Qian Z, et al.Clinical and CT features of early-stage patients with COVID-19: a retrospective analysis of imported cases in Shanghai, China. European Respiratory Journal 2020;55(4):2000407. [DOI: 10.1183/13993003.00407-2020] [DOI] [PMC free article] [PubMed] [Google Scholar]

Yang 2020b {published data only}

  1. Yang W, Cao Q, Qin L, Wang X, Cheng Z, Pan A, et al.Clinical characteristics and imaging manifestations of the 2019 novel coronavirus disease (COVID-19): a multi-center study in Wenzhou city, Zhejiang, China. Journal of Infection 2020;80(4):388-93. [DOI] [PMC free article] [PubMed] [Google Scholar]

Yuan 2020 {published data only}

  1. Yuan M, Yin W, Tao Z, Tan W, Hu Y.Association of radiologic findings with mortality of patients infected with 2019 novel coronavirus in Wuhan, China. PlLOS One 2020;15(3):e0230548. [DOI: 10.1371/journal.pone.0230548 10.1371/journal.pone.0230548. eCollection 2020.] [DOI] [PMC free article] [PubMed] [Google Scholar]

Zhifeng 2020 {published data only}

  1. Zhifeng J, Feng A, Li T.Consistency analysis of COVID-19 nucleic acid tests and the changes of lung CT. Journal of Clinical Virology 2020;127:104359. [DOI] [PMC free article] [PubMed] [Google Scholar]

Additional references

Bossuyt 2015

  1. Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig L, et al.STARD 2015: An Updated List of Essential Items for Reporting Diagnostic Accuracy Studies. Radiology 2015 Oct 28;277(3):826–32. [DOI: 10.1148/radiol.2015151516] [DOI] [PubMed] [Google Scholar]

BSTI 2020

  1. British Society of Thoracic Imaging.COVID-19 BSTI Reporting Templates and Codes. bsti.org.uk/covid-19-resources/covid-19-bsti-reporting-templates/ 2020 (accessed 4 January 2020).

Butler‐Laporte 2021

  1. Butler-Laporte G, Lawandi A, Schiller I, Yao MC, Dendukuri N, McDonald E, et al.Comparison of saliva and nasopharyngeal swab nucleic acid amplification testing for detection of SARS-CoV-2: a systematic review and meta-analysis. JAMA Internal Medicine 2021;18(3):353-60. [DOI: 10.1001/jamainternmed.2020.8876] [DOI] [PMC free article] [PubMed] [Google Scholar]

China National Health Comission 2020

  1. China National Health Comission.Chinese Clinical Guidance for COVID-19 Pneumonia Diagnosis and Treatment (7th Edition). kjfy.meetingchina.org/msite/news/show/cn/3337.html 2020 (accessed 24 January 2020).

Chu 2006

  1. Chu H, Cole SR.Bivariate meta-analysis of sensitivity and specificity with sparse data: a generalized linear mixed model approach. Journal of Clinical Epidemiology 2006;59(12):1331-2; author reply 1332-3. [DOI: 10.1016/j.jclinepi.2006.06.011] [DOI] [PubMed] [Google Scholar]

Datta 2020

  1. Datta SD, Talwar A, Lee JT.A proposed framework and timeline of the spectrum of disease due to SARS-CoV-2 infection: illness beyond acute infection and public health implications. JAMA 2020;324(22):2251-2. [DOI: 10.1001/jama.2020.22717] [DOI] [PubMed] [Google Scholar]

Deeks 2020

  1. Deeks JJ, Dinnes J, Takwoingi Y, Davenport C, Spijker R, Taylor-Phillips S, et al.Antibody tests for identification of current and past infection with SARS-CoV-2. Cochrane Database of Systematic Reviews 2020, Issue 6. Art. No: CD013652. [DOI: 10.1002/14651858.CD013652] [DOI] [PMC free article] [PubMed] [Google Scholar]

Dinnes 2020

  1. Dinnes J, Deeks JJ, Adriano A, Berhane S, Davenport C, Dittrich S, et al.Rapid, point-of-care antigen and molecular-based tests for diagnosis of SARS-CoV-2 infection. Cochrane Database of Systematic Reviews 2020, Issue 8. Art. No: CD013705. [DOI: 10.1002/14651858.CD013705] [DOI] [PMC free article] [PubMed] [Google Scholar]

Dinnes 2021

  1. Dinnes J, Deeks JJ, Berhane S, Taylor M, Adriano A, Davenport C, et al.Rapid, point-of-care antigen and molecular-based tests for diagnosis of SARS-CoV-2 infection. Cochrane Database of Systematic Reviews 2021, Issue 3. Art. No: CD013705. [DOI: 10.1002/14651858.CD013705] [DOI] [PMC free article] [PubMed] [Google Scholar]

Han 2020

  1. Han R, Huang L, Jiang H, Dong J, Peng H, Zhang D.Early clinical and CT manifestations of coronavirus disease 2019 (COVID-19) pneumonia. American Journal of Roentgenology 2020 Aug;215(2):338-43. [DOI: 10.2214/AJR.20.22961] [DOI] [PubMed] [Google Scholar]

Harbord 2009

  1. Harbord RM, Whiting P.metandi: meta-analysis of diagnostic accuracy using hierarchical logistic regression. Stata Journal 2009;9(2):211-29. [DOI: 10.1177/1536867X0900900203] [DOI] [Google Scholar]

Hong 2018

  1. Hong PJ, Korevaar DA, McGrath TA, Ziai H, Frank R, Alabousi M, et al.Reporting of imaging diagnostic accuracy studies with focus on MRI subgroup: adherence to STARD 2015. Journal of Magnetic Resonance Imaging 2018;47(2):523-44. [DOI: 10.1002/jmri.25797] [DOI] [PubMed] [Google Scholar]

Irwig 1995

  1. Irwig L, Macaskill P, Glasziou P, Fahey M.Meta-analytic methods for diagnostic test accuracy. Journal of Clinical Epidemiology 1995;48(1):119-30; discussion 131-2. [DOI: 10.1016/0895-4356(94)00099-c] [DOI] [PubMed] [Google Scholar]

Kanne 2020

  1. Kanne J.Chest CT findings in 2019 novel Coronavirus (2019-nCoV) infections from Wuhan, China: kkey points for the radiologist. Radiology 2020 Apr;295(1):16-7. [DOI: 10.1148/radiol.2020200241] [DOI] [PMC free article] [PubMed] [Google Scholar]

Korevaar 2020

  1. Korevaar DA, Kootte RS, Smits LP, Van den Aardweg JG, Bonta PI, Schinkel J, et al.Added value of chest computed tomography in suspected COVID-19: an analysis of 239 patients. European Respiratory Journal 2020;56(2):2001377. [DOI: 10.1183/13993003.01377-2020] [DOI] [PMC free article] [PubMed] [Google Scholar]

Kucirka 2020

  1. Kucirka LM, Lauer SA, Laeyendecker O, Boon D, Lessler J.Variation in false-negative rate of reverse transcriptase polymerase chain reaction-based SARS-CoV-2 tests by time since exposure. Annals of Internal Medicine 2020;173(4):262-7. [DOI: 10.7326/M20-1495] [DOI] [PMC free article] [PubMed] [Google Scholar]

Lee 2020

  1. Lee EY, Ng MY, Khong PL.COVID-19 pneumonia: what has CT taught us? Lancet Infectious Diseases 2020 Apr;20(4):384-5. [DOI: 10.1016/S1473-3099(20)30134-1] [DOI] [PMC free article] [PubMed] [Google Scholar]

Leeflang 2021

  1. Leeflang MMG, Bell K, Deeks JJ, Dinnes J, Doust J, Korevaar DA, et al.Electronic and animal noses for detecting SARS-CoV-2 infection. Cochrane Database of Systematic Reviews 2021 Jun 28, Issue 6. Art. No: CD015013. [DOI: 10.1002/14651858.CD015013] [DOI] [Google Scholar]

Li 2020a

  1. Li Y, Xia L.Coronavirus disease 2019 (COVID-19): role of chest CT in diagnosis and management. American Journal of Roentgenology 2020;214(6):1280-6. [DOI: 10.2214/AJR.20.22954] [DOI] [PubMed] [Google Scholar]

Li 2020b

  1. Li Y, Yao L, Li J, Chen L, Song Y, Cai Z, et al.Stability issues of RT-PCR testing of SARS-CoV-2 for hospitalized patients clinically diagnosed with COVID-19. Journal of Medical Virology 2020;92(7):903-8. [DOI: 10.1002/jmv.25786] [DOI] [PMC free article] [PubMed] [Google Scholar]

Loeffelholz 2020

  1. Loeffelholz MJ, Tang YW.Laboratory diagnosis of emerging human coronavirus infections - the state of the art. Emerging Microbes and Infections 2020;9(1):747-56. [DOI: 10.1080/22221751.2020.1745095] [DOI] [PMC free article] [PubMed] [Google Scholar]

McGrath 2017

  1. McGrath TA, McInnes MD, Langer FW, Hong J, Korevaar DA, Bossuyt PM.Treatment of multiple test readers in diagnostic accuracy systematic reviews-meta-analyses of imaging studies. European Journal of Radiology 2017;93:59–64. [DOI: 10.1016/j.ejrad.2017.05.032] [DOI] [PubMed] [Google Scholar]

Moher 2009

  1. Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group.Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Journal of Clinical Epidemiology 2009;62(10):1006-12. [DOI: 10.1371/journal.pmed.1000097] [DOI] [PubMed] [Google Scholar]

Ng 2020

  1. Ng M-Y, Lee EY, Yang J, Yang F, Li X, Wang H, et al.Imaging profile of the COVID-19 infection: radiologic findings and literature review. Cardiothoracic Imaging 2020 Feb 1;2(1):e200034. [DOI: 10.1148/ryct.2020200034] [DOI] [PMC free article] [PubMed] [Google Scholar]

Pan 2020

  1. Pan Y, Guan H, Zhou S, Wang Y, Li Q, Zhu T et al.Initial CT findings and temporal changes in patients with the novel coronavirus pneumonia (2019-nCoV): a study of 63 patients in Wuhan, China. European Radiology 2020 Jun;30(6):3306-9. [DOI: 10.1007/s00330-020-06731-x] [DOI] [PMC free article] [PubMed] [Google Scholar]

Pedersen 2020

  1. Pedersen, TL.Package ‘ggforce’. www.ggforce.data-imaginist.com 2020 (accessed 5 January 2020).

Peng 2020b

  1. Peng QY, Wang XT, Zhang LN.Findings of lung ultrasonography of novel corona virus pneumonia during the 2019-2020 epidemic. Intensive Care Medicine 2020;46(5):849-50. [DOI: 10.1007/s00134-020-05996-6] [DOI] [PMC free article] [PubMed] [Google Scholar]

Prokop 2020

  1. Prokop M, Van Everdingen W, Van Rees Vellinga T, Quarles van Ufford H, Stöger L, Beenen L, et al.CO-RADS: a categorical CT assessment scheme for patients suspected of having COVID-19—definition and evaluation. Radiology 2020;296(2):E97-104. [DOI: 10.1148/radiol.2020201473] [DOI] [PMC free article] [PubMed] [Google Scholar]

R Core Team 2021 [Computer program]

  1. R: A language and environment for statistical computing.R Core Team. Vienna, Austria: R Foundation for Statistical Computing, 2020.

Reitsma 2005

  1. Reitsma JB, Glas AS, Rutjes AW, Scholten RJ, Bossuyt PM, Zwinderman AH.Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. Journal of Clinical Epidemiology 2005;58(10):982-90. [DOI: 10.1016/j.jclinepi.2005.02.022] [DOI] [PubMed] [Google Scholar]

Rubin 2020

  1. Rubin GD, Ryerson CJ, Haramati LB, Sverzellati N, Kanne JP, Raoof S, et al.The role of chest imaging in patient management during the COVID-19 pandemic: a multinational consensus statement from the Fleischner Society. Radiology 2020;296(1):172–80. [DOI: 10.1148/radiol.2020201365] [DOI] [PMC free article] [PubMed] [Google Scholar]

Salameh 2020b

  1. Salameh J-P, Bossuyt PM, McGrath TA, Thombs BD, Hyde CJ, Macaskill P, et al.Preferred reporting items for systematic review and meta-analysis of diagnostic test accuracy studies (PRISMA-DTA): explanation, elaboration, and checklist. BMJ 2020;370:m2632. [DOI: 10.1136/bmj.m2632] [DOI] [PubMed] [Google Scholar]

Salehi 2020

  1. Salehi S, Abedi A, Balakrishnan S, Gholamrezanezhad A.Coronavirus disease 2019 (COVID-19): a systematic review of imaging findings in 919 patients. American Journal of Roentgenology 2020;215(1):87-93. [DOI: 10.2214/AJR.20.23034] [DOI] [PubMed] [Google Scholar]

Shi 2020

  1. Shi H, Han X, Jiang N, Cao Y, Alwalid O, Gu J, et al.Radiological findings from 81 patients with COVID-19 pneumonia in Wuhan, China: a descriptive study. Lancet Infectious Diseases 2020 Apr;20(4):425–34. [DOI: 10.1016/S1473-3099(20)30086-4] [DOI] [PMC free article] [PubMed] [Google Scholar]

Simpson 2020

  1. Simpson S, Kay FU, Abbara S, Bhalla S, Chung JH, Chung M, et al.Radiological Society of North America expert consensus document on reporting chest CT findings related to COVID-19: endorsed by the Society of Thoracic Radiology, the American College of Radiology, and RSNA. Radiology: Cardiothoracic Imaging 2020;2(2):e200152. [DOI: 10.1148/ryct.2020200152] [DOI] [PMC free article] [PubMed] [Google Scholar]

Soldati 2020

  1. Soldati G, Smargiassi A, Inchingolo R, Buonsenso D, Perrone T, Briganti DF, et al.Proposal for international Ssandardization of the use oflLung ultrasound for patients with COVID-19: a simple, quantitative, reproducible method. Journal of Ultrasound in Medicine 2020 Jul;39(7):1413-9. [DOI: 10.1002/jum.15285] [DOI] [PMC free article] [PubMed] [Google Scholar]

StataCorp 2019 [Computer program]

  1. Stata Statistical Software.StataCorp, Version Release 16. College Station, TX: StataCorp LLC, 2019.

Stegeman 2020

  1. Stegeman I, Ochodo EA, Guleid F, Holtman GA, Yang B, Davenport C, et al.Routine laboratory testing to determine if a patient has COVID-19. Cochrane Database of Systematic Reviews 2020, Issue 11. Art. No: CD013787. [DOI: 10.1002/14651858.CD013787] [DOI] [PMC free article] [PubMed] [Google Scholar]

Struyf 2020

  1. Struyf T, Deeks JJ, Dinnes J, Takwoingi Y, Davenport C, Leeflang MM, et al.Signs and symptoms to determine if a patient presenting in primary care or hospital outpatient settings has COVID-19 disease. Cochrane Database of Systematic Reviews 2020, Issue 7. Art. No: CD013665. [DOI: 10.1002/14651858.CD013665] [DOI] [PMC free article] [PubMed] [Google Scholar]

Walker 2020

  1. Walker M.What does asymptomatic COVID-19 look like under the surface? www.medpagetoday.com/infectiousdisease/covid19/87168 2020 (accessed 25 July 2021).

WHO 2020

  1. World Health Organization (WHO).WHO COVID-19 Case definition. WHO/2019-nCoV/Surveillance_Case_Definition/2020.1 2020 (accessed 17 October 2020).

Wickham 2016

  1. Wickham H.ggplot2: elegant graphics for data analysis. New York: Springer-Verlag, 2016. [Google Scholar]

Ye 2020

  1. Ye Z, Zhang Y, Wang Y, Huang Z, Song B.Chest CT manifestations of new coronarovirus disease 2019 (COVID-19): a pictorial review. European Radiology 2020;30(8):4381-9. [DOI: 10.1007/s00330-020-06801-0] [DOI] [PMC free article] [PubMed] [Google Scholar]

Zhang 2020

  1. Zhang JJ, Dong X, Cao YY, Yuan YD, Yang YB, Yan YQ, et al.Clinical characteristics of 140 patients infected with SARS-CoV-2 in Wuhan, China. Allergy 2020 Jul;75(7):1730-41. [DOI: 10.1111/all.14238] [DOI] [PubMed] [Google Scholar]

Zhao 2020

  1. Zhao W, Zhong Z, Xie X, Yu Q, Liu J.Relation between chest CT findings and clinical conditions of coronavirus disease (COVID-19) pneumonia: a multicenter study. American Journal of Roentgenology 2020 May;214(5):1072-7. [DOI: 10.2214/AJR.20.22976] [DOI] [PubMed] [Google Scholar]

Zhou 2020

  1. Zhou Z, Guo D, Li C, Fang Z, Chen L, Yang R, et al.Coronavirus disease 2019: initial chest CT findings. European Radiology 2020 Aug;30(8):4398-406. [DOI: 10.1007/s00330-020-06816-7] [DOI] [PMC free article] [PubMed] [Google Scholar]

References to other published versions of this review

Islam 2020

  1. Islam N, Salameh J-P, Leeflang MM, Hooft L, McGrath TA, Pol CB, et al.Thoracic imaging tests for the diagnosis of COVID-19. Cochrane Database of Systematic Reviews 2020, Issue 11. Art. No: CD013639. [DOI: 10.1002/14651858.CD013639.pub3] [DOI] [PubMed] [Google Scholar]

Islam 2021

  1. Islam N, Ebrahimzadeh S, Salameh J-P, Kazi S, Fabiano N, Treanor L, et al.Thoracic imaging tests for the diagnosis of COVID-19. Cochrane Database of Systematic Reviews 2021, Issue 3. Art. No: CD013639. [DOI: 10.1002/14651858.CD013639.pub4] [DOI] [PMC free article] [PubMed] [Google Scholar]

McInnes 2020

  1. McInnes MD, Leeflang MM, Salameh J-P, McGrath TA, Pol CB, Frank RA, et al.Imaging tests for the diagnosis of COVID-19. Cochrane Database of Systematic Reviews 2020, Issue 6. Art. No: CD013639. [DOI: 10.1002/14651858.CD013639] [DOI] [PubMed] [Google Scholar]

Salameh 2020a

  1. Salameh J-P, Leeflang MM, Hooft L, Islam N, McGrath TA, Pol CB, et al.Thoracic imaging tests for the diagnosis of COVID-19. Cochrane Database of Systematic Reviews 2020, Issue 9. Art. No: CD013639. [DOI: 10.1002/14651858.CD013639.pub2] [DOI] [PubMed] [Google Scholar]

Articles from The Cochrane Database of Systematic Reviews are provided here courtesy of Wiley

RESOURCES