Abstract
Objectives
To identify the diagnostic accuracy of common imaging modalities, chest X-ray (CXR) and CT, for diagnosis of COVID-19 in the general emergency population in the UK and to find the association between imaging features and outcomes in these patients.
Design
Retrospective analysis of electronic patient records.
Setting
Tertiary academic health science centre and designated centre for high consequence infectious diseases in London, UK.
Participants
1198 patients who attended the emergency department with paired reverse transcriptase PCR (RT-PCR) swabs for SARS-CoV-2 and CXR between 16 March and 16 April 2020.
Main outcome measures
Sensitivity and specificity of CXR and CT for diagnosis of COVID-19 using the British Society of Thoracic Imaging reporting templates. Reference standard was any RT-PCR positive naso-oropharyngeal swab within 30 days of attendance. ORs of CXR in association with vital signs, laboratory values and 30-day outcomes were calculated.
Results
Sensitivity and specificity of CXR for COVID-19 diagnosis were 0.56 (95% CI 0.51 to 0.60) and 0.60 (95% CI 0.54 to 0.65), respectively. For CT scans, these were 0.85 (95% CI 0.79 to 0.90) and 0.50 (95% CI 0.41 to 0.60), respectively. This gave a statistically significant mean increase in sensitivity with CT of 29% (95% CI 19% to 38%, p<0.0001) compared with CXR. Specificity was not significantly different between the two modalities.
CXR findings were not statistically significantly or clinically meaningfully associated with vital signs, laboratory parameters or 30-day outcomes.
Conclusions
Computed tomography has substantially improved diagnostic performance over CXR in COVID-19. CT should be considered in the initial assessment for suspected COVID-19 instead of CXR if capacity allows and balanced against radiation exposure risk.
Keywords: COVID-19, Chest imaging, accident & emergency medicine, general medicine (see Internal Medicine), diagnostic radiology
Strengths and limitations of this study.
Large, appropriately powered, study population consisting of all patients attending the emergency department rather than those solely with confirmed COVID-19; this allowed assessment of specificity for the imaging modalities and applicability to the general population who may attend medical personnel with other complaints, but have underlying SARS-CoV-2 infection.
Comprehensive statistical analyses were conducted to address confounding in reporting of X-rays including propensity score matching and logistic regression to give a ‘doubly robust’ model.
Low amount of missing data and for secondary covariates only; multiple imputation was performed with a good fit, however, observed data would be preferable to imputed data.
Single centre, retrospective study; potential for inter-reporter and intercentre variability in reporting.
Large proportion of patients excluded due to not having a reverse transcriptase PCR swab, predominantly, those with imaging reported as negative; this may bias the results towards increased sensitivity and specificity.
Introduction
SARS-CoV-2 and its resulting disease, COVID-19, have propagated exponentially worldwide, with over 10 million cases in 188 countries at the time of writing.1 2
The gold standard for diagnosis of the virus is the detection of viral RNA through reverse transcriptase PCR (RT-PCR) of respiratory tract samples. However, this method has several limitations including (1) low sensitivity at 59%–71%;3 4 (2) relatively slow turnaround times ranging from a few hours to several days;5 (3) high expense; and (4) limited capacity for testing in many countries.
CT has been shown to be more sensitive than RT-PCR for diagnosis of COVID-19,3 4 while being significantly faster and cheaper. This comes with a large radiation dose and capacity is still lacking in many countries.
Plain film chest X-ray (CXR) is ubiquitous worldwide, with a 30–70 times lower dose of radiation6 and is commonly performed as an initial investigation in COVID-19.
Studies have so far only evaluated imaging in those with confirmed infection; it is, therefore, not possible to calculate the specificity of these modalities. In the context of the global pandemic, infection may be widespread in the community, often with subclinical infection.7 8 A reliable and rapid method to detect infection in the general population, who may present to medical personnel with other complaints, is needed.
Despite its extensive use, the specificity and sensitivity of CXR in the general emergency population for diagnosis of COVID-19 is unknown, nor how imaging features correlate with severity.
This study evaluated the performance of CXR in diagnosing COVID-19 in the emergency department (ED) of a tertiary care hospital.
Methods
This study was conducted at the Royal Free Hospital, London, UK, an academic health science centre and nationally designated centre for high consequence infectious diseases.9
All individuals attending the ED who had paired posterior–anterior chest radiographs and RT-PCR nasopharyngeal swabs for COVID-19 at the time of initial attendance between 16 March 2020 and 16 April 2020 were included.
All chest radiographs were reported by a consultant radiologist and rated on an ordinal scale for probability of COVID-19: alternative pathology identified, not COVID-19; clear chest, unlikely COVID-19; indeterminate findings for COVID-19; classical findings of COVID-19, based on the British Society of Thoracic Imaging’s (BSTI) reporting templates (table 1).10 These were reported prior to RT-PCR results being available.
Table 1.
Ordinal scale used in this study based on the British Society of Thoracic Imaging (BSTI) reporting template10
| Ordinal scale for study | BSTI grade | Features on X-ray |
| 0 | CVCX3-Non-COVID-19 | Alternative pathology such as CVCX3—Non-COVID-19 pneumothorax with no features of COVID-19 identified |
| 1 | CVCX0—Normal | No pathology seen |
| 2 | CVCX2—Indeterminate for COVD-19 or atypical features | Poor quality film or central/basal consolidation |
| 3 | CVCX1—Classic findings of COVID-19 | Peripheral ground glass opacities |
RT-PCR of swabs were performed in laboratories either at our centre or at a public health laboratory (PHE Collindale, UK), according to published national standard operating procedures
.11 Subsequent RT-PCR swabs taken within 30 days of initial ED attendance were also included.
CT scans performed within 30 days of attendance were retrieved. These were also reported according to the BSTI template. CT pulmonary angiogram was performed in the ED if the D-dimer was >5000 to exclude pulmonary emboli (PE) as per the locally agreed protocol. Subsequent CT chest imaging (whether pulmonary angiogram, contrast or non-contrast) was performed on the basis of clinical suspicion.
Prospectively recorded data were extracted from the Cerner Millennium electronic patient record system (Cerner, Kansas City, Missouri, USA).
Primary outcome
The primary outcome is sensitivity and specificity of initial CXR, where it is reported as having classic COVID-19 features in the ED. This is compared with RT-PCR swab as the reference standard for diagnosis of COVID-19.
In the event of multiple RT-PCR swabs during one attendance, a single positive swab was taken as an overall positive test during one admission.
Secondary outcomes
In those patients who also had CT scans of the thorax, the diagnostic accuracy was compared with CXR, with RT-PCR again as the reference standard. Sensitivity and specificity of CXR when X-rays reported as indeterminate or atypical for COVID-19 were classed as positive was also calculated.
CXR findings were correlated with vital signs at attendance and blood results, including neutrophil counts, D-dimer and C reactive protein, which have been associated with poor prognosis in COVID-19.12 HRs for clinical outcomes including direct admission to the intensive treatment unit (ITU) from ED and 30-day mortality rates were also calculated for CXR reporting categories.
Statistical analysis
In the event of missing data, multiple imputation was conducted using a Predictive Mean Matching algorithm, via the MICE R package, as described previously.13 Briefly, this uses a linear regression model (or logistic regression model for categorical data) to find a random value based on already observed data, to replace missing fields.14 Variables without missing data fields were not modified. The number of imputed datasets was similar in number to the percentage of missing data, as suggested by White and colleagues.15 Balance diagnostics with density plots are available in online supplemental file 1, adequate balance was assessed via visual inspection of imputed distributions with respect to the original dataset.
bmjopen-2020-042946supp001.pdf (1MB, pdf)
The propensity for a CXR being reported as positive or negative for COVID-19 was calculated for several plausible covariates that may influence image characteristics such as age, gender, ethnicity, pre-existing morbidities and the respiratory rate of the patient using a generalised linear model.16 X-ray positive and negative groups were then matched in each imputed dataset using the nearest neighbour algorithm, with a calliper of 0.2 of the propensity score SD, without replacement and in random sequential order to obtain a 1:1 match as described elsewhere.17
The balance of the match data was assessed quantitatively with mean differences of covariates in each of the X-ray groups before and after matching, with a difference of less than 0.1% considered a good match (online supplementary figures 1 and 2). Visual inspection of matches was also conducted to ensure balance (online supplementary figure 2-4).
After matching, outcome data were adjusted for covariates including age, gender, ethnicity and presence of comorbidities, as well as C reactive protein, D-dimer, troponin and vital signs. This was achieved by generalised linear regression for continuous outcome data, binomial logistic regression for binary categorical outcomes or ordinal logistic regression in the case of CXR where it is the outcome variable.
These regression models were run on each imputed dataset and outcomes were pooled together across each imputed dataset according to Rubin’s rules18 to give an overall estimate.
Diagnostic accuracy statistics
CXRs reported as classical for COVID-19 as per the BSTI guidelines were considered a positive test in the primary analysis. In a secondary analysis, X-rays reported as ‘Indeterminate’ or ‘Atypical’ for COVID-19 were also considered positive. All other reports were classified as a negative test. These were compared with nasopharyngeal aspirate RT-PCR results, which were taken as the gold standard for diagnosis of COVID-19. Where more than one swab was taken during the study period (up to 30 days after initial attendance), a single positive result was taken as a positive result for calculation of diagnostic accuracy statistics.
Sensitivity, specificity, predictive values and diagnostic accuracy were calculated using the propensity-matched data after imputation and pooled across imputed datasets with 95% CIs. Apparent and true prevalence based on this dataset are also given for interpretation of the predictive values.
Chest CTs were also reported according to the BSTI guidelines as with X-ray. Diagnostic statistics were calculated on raw, unmatched and non-imputed data (due to a low volume of data for imputation and matching) in the same manner as X-ray. Mean differences and 95% CIs between CT and X-ray for each of the diagnostic statistics are given, with a p value calculated from the CIs.
Agreement between the modalities was assessed on the unmatched dataset, in the sample where CT, CXR and RT-PCR were all available using Cohen’s (for two group agreement) and Fleiss’ kappa (when all three are compared).
Data presentation
Descriptive statistics are given as means and SD for normally distributed data and as medians and IQRs for non-normally distributed data, before and after matching and multiple imputation (for the latter, these statistics are pooled across imputations).
Association of explanatory variables with SARS-CoV-2 and CXR findings are given as ORs in univariate and multivariate configurations.
Data were considered statistically significant if p<0.05. Given the large number of analyses in this paper, data are separately highlighted if p<0.001 as a secondary threshold to address the potential for false positives with multiple testing.
Analyses were conducted using R V.4.0.0 (R Foundation for Statistical Computing, Vienna, Austria) and code for the analyses is given in online supplemental file 2.
bmjopen-2020-042946supp002.pdf (1.3MB, pdf)
Sample size calculation
In this study, the lower CI for sensitivity of CXR as reported by Wong et al19 (56%) was used as an estimate of likely sensitivity for COVID-19. A power of 80% at an alpha of 0.05 was used to calculate the sample size for sensitivity and specificity of 56%. This gave an estimated sample size of 165 in each of the COVID-19 negative and positive groups by RT-PCR (total of 330).
Reporting guidelines
This study is reported according to the STARD guidelines20 for diagnostic accuracy studies.
Patient and public involvement
Patients and the public were not involved in the design, conduct or dissemination of this study.
Results
A total of 1198 eligible patients with both CXR and RT-PCR were identified in the study period (figure 1). Their characteristics, stratified by positivity for SARS-CoV-2 infection by RT-PCR, are summarised in table 2. This showed that those with confirmed SARS-CoV-2 infection were more likely to be male, older (mean age 66.2 vs 62.7), have lower saturations, higher respiratory rates, while being more likely to be admitted and die within 30 days. There was a significant association with X-ray images and SARS-CoV-2 at baseline, with 59.6% having classic imaging features of COVID-19 in those with positive swabs versus 39.1% in those with negative swabs. There was 8.6% missing data overall in the dataset when variables with >50% missing data were removed and 15 imputations were performed on these remaining variables only.
Figure 1.
Inclusion and exclusion of patients during study period with test results. RT-PCR, reverse transcriptase PCR.
Table 2.
Baseline characteristics of dataset stratified by overall SARS-CoV-2 RT-PCR status, including subsequent swabs during the study period
| SARS-CoV-2 RT-PCR | P value | Missing (%) | ||
| Negative | Positive | |||
| n (%) | 435 (36.3) | 763 (63.7) | ||
| No of swabs (%) | 810 (48.3) | 868 (51.7) | ||
| Age (mean (SD)) | 62.74 (17.72) | 66.18 (17.58) | 0.001* | 0 |
| Ethnicity | 0.097 | 19 | ||
| Other Asian (%) | 29 (8.0) | 72 (11.8) | ||
| South Asian (%) | 27 (7.5) | 38 (6.2) | ||
| Black (%) | 41 (11.4) | 91 (14.9) | ||
| Mixed (%) | 6 (1.7) | 6 (1.0) | ||
| Other (%) | 56 (15.5) | 105 (17.2) | ||
| White (%) | 202 (56.0) | 297 (48.8) | ||
| Sex—Male (%) | 233 (53.6) | 480 (62.9) | 0.002* | 0 |
| Oxygen saturation (median (IQR)) | 95 (6) | 93 (8) | <0.001** | 6.3 |
| Respiratory rate (median (IQR)) | 22 (8) | 26 (12) | <0.001** | 6.3 |
| Glasgow Coma Scale (median (IQR)) | 15 (0) | 15 (0) | 0.043* | 6.6 |
| Systolic BP (median (IQR)) | 134 (32) | 130 (30) | 0.009* | 15.8 |
| Heart rate (median (IQR)) | 96 (27) | 94 (27) | 0.092 | 6.4 |
| Temperature (median (IQR)) | 37.1 (1.4) | 37.7 (1.4) | <0.001** | 6.7 |
| Chest X-ray report | <0.001** | 0 | ||
| Alternative pathology (%) | 4 (0.9) | 3 (0.4) | ||
| No abnormalities (%) | 178 (4.9) | 136 (17.8) | ||
| Indeterminate (%) | 83 (19.1) | 169 (22.1) | ||
| Classic COVID-19 (%) | 170 (39.1) | 455 (59.6) | ||
| Presence of comorbidities (%) | 297 (79.0) | 482 (80.3) | 0.669 | 18.5 |
| Dyspnoea (%) | 274 (69.4) | 497 (75.5) | 0.034 | 12.1 |
| Neutrophils (median (IQR)) | 6.42 (4.56) | 5.25 (3.92) | <0.001** | 2.3 |
| D-dimer (median (IQR)) | 1250 (2440) | 1105 (1803) | 0.204 | 23.2 |
| Albumin (median (IQR)) | 39 (7) | 37 (6) | <0.001** | 10 |
| C reactive protein (median (IQR)) | 91.0 (115) | 146.5 (264.8) | <0.001** | 3 |
| Creatine kinase (median (IQR)) | 51 (104) | 145 (260) | <0.001** | 23.3 |
| Troponin (median (IQR)) | 19 (46) | 20 (44) | 0.278 | 19.1 |
| Admitted (%) | 331 (76.0) | 635 (83.2) | 0.003* | 0.1 |
| Admitted to ITU (%) | 5 (1.3) | 32 (4.8) | 0.005* | 12.4 |
| 30-day follow-up status | <0.001** | 24 | ||
| Discharged (%) | 219 (78.2) | 367 (58.3) | ||
| On ambulatory follow-up (%) | 14 (5.0) | 49 (7.8) | ||
| Admitted (%) | 18 (6.4) | 60 (9.5) | ||
| Died (%) | 29 (10.4) | 154 (24.4) | ||
| CT report | <0.001** | 0 | ||
| No pathology identified (%) | 23 (22.1) | 6 (3.3) | ||
| Classic COVID-19 findings (%) | 52 (50.0) | 157 (85.8) | ||
| Indeterminate for COVID-19 (%) | 14 (13.5) | 14 (7.7) | ||
| Alternative pathology identified (%) | 15 (14.4) | 6 (3.3) | ||
| Day of symptoms (mean (SD)) | 9.84 (9.63) | 8.56 (15.80) | 0.368 | 69.2 |
There were 480 additional swabs on 399 unique patients with a median of 2 and mean of 3.5 per patient.
*Significant at p<0.05; **significant at p<0.001.
BP, blood pressure; RT-PCR, reverse transcriptase PCR.
After multiple imputation for missing data and pooled propensity score matching for plausible covariates that may affect CXR reporting, there were 430 patients in each of the X-ray positive and X-ray negative groups, for a total of 860 patients. Adequate balance was achieved for relevant covariates with a mean difference of <0.1 between groups (online supplemental file 1, table 2).
CT was performed in 302 patients with paired RT-PCR during the same time period, with a median serial interval of 4.5 days (IQR 0–17) after the initial attendance in ED, and of these, 30.1% were within 1 day of attendance.
Diagnostic accuracy
The pooled sensitivity and specificity of CXR was 0.56 (95% CI 0.51 to 0.60) and 0.60 (95% CI 0.54 to 0.65), respectively (table 3). This gave an overall diagnostic accuracy of 0.57 (95% CI 0.54 to 0.61) for CXR.
Table 3.
Diagnostic accuracy metrics for CXR and CT chest with RT-PCR for SARS-CoV-2, as the reference standard
| Chest X-ray | CT chest | Mean difference | P value | |
| Total (n) | 860 | 302 | ||
| True positives (n) | 305 | 162 | – | – |
| False positives (n) | 125 | 55 | – | – |
| True negatives (n) | 187 | 56 | – | – |
| False negatives (n) | 243 | 29 | – | – |
| Apparent prevalence (95% CI) | 0.50 (0.47 to 0.53) | 0.72 (0.66 to 0.77) | 0.22 (0.04 to 0.21) | <0.0001** |
| True prevalence (95% CI) | 0.64 (0.60 to 0.67) | 0.63 (0.58 to 0.69) | −0.00 (−0.09 to 0.03) | 0.111 |
| Sensitivity (95% CI) | 0.56 (0.51 to 0.60) | 0.85 (0.79 to 0.90) | 0.29 (0.19 to 0.38) | <0.0001** |
| Specificity (95% CI) | 0.60 (0.54–0.65) | 0.50 (0.41 to 0.60) | −0.10 (−0.25 to 0.04) | 0.119 |
| Positive predictive value (95% CI) | 0.71 (0.66 to 0.75) | 0.75 (0.68 to 0.80) | 0.04 (−0.06 to 0.14) | 0.492 |
| Negative predictive value (95% CI) | 0.43 (0.39 to 0.48) | 0.66 (0.55 to 0.76) | 0.22 (0.06 to 0.37) | 0.005* |
| Positive likelihood ratio (95% CI) | 1.39 (1.19 to 1.62) | 1.71 (1.41 to 2.08) | 0.32 (−0.22 to 0.89) | 0.258 |
| Negative likelihood ratio (95% CI) | 0.74 (0.64 to 0.84) | 0.30 (0.21 to 0.44) | −0.44 (−0.64 to −0.21) | 0.022* |
| Diagnostic accuracy (95% CI) | 0.57 (0.54 to 0.61) | 0.72 (0.66 to 0.77) | 0.15 (0.06 to 0.23) | <0.0001** |
*Significant difference at the <0.05 level; **significant difference at the <0.0001 level.
CXR, chest X-ray; RT-PCR, reverse transcriptase PCR.
In comparison, sensitivity and specificity for CT was 0.85 (95% CI 0.79 to 0.90) and 0.50 (95% CI 0.41 to 0.60), respectively. This gave a statistically significant mean increase in sensitivity with CT compared with CXR by 29% (95% CI 19% to 38%, p<0.0001). Specificity was not significantly different between the two modalities. Diagnostic accuracy and negative predictive values were also significantly increased with CT at 0.15 and 0.22, respectively, while the negative likelihood ratio was significantly decreased at −0.44. This shows that the post-test odds of being negative for SARS-CoV-2 by RT-PCR with a negative CT is significantly lower.
Taking X-rays reported as indeterminate as positive increased the sensitivity of CXR to 0.80 (95% CI 0.77 to 0.84), however, specificity was reduced to 0.40 (95% CI 0.35 to 0.46). When CT scans reported as indeterminate are also considered positive, the sensitivity of CT increased to 0.93 (95% CI 0.89 to 0.96), while mean specificity reduced to 0.37 (95% CI 0.28 to 0.47), although this was not statistically different from when indeterminate CTs are considered negative. Sensitivity of CT remained significantly higher than CXR (when indeterminates are considered positive for both) by 0.13 (95% CI 0.05 to 0.19, p<0.001), but specificity was not significantly different between the two.
When comparing only the unimputed, unmatched subset of data where CT, RT-PCR and CXR were all performed (n=287), the agreement between CT and CXR was poor (Cohen’s kappa 0.406). Agreement between all three modalities was also poor (Fleiss’ kappa 0.361).
Association of CXR with markers of severity and outcomes
Association of covariates with RT-PCR results is shown in table 4 and figure 2. Those who tested positive for SARS-CoV-2 by RT-PCR were significantly more likely to have a classical X-ray (OR 1.79 95% CI 1.25 to 2.56, p<0.002) as would be expected by the diagnostic accuracy statistics (table 4). When the CXR report is considered as an ordered scale, worsening grades of report were associated more strongly with RT-PCR positivity, with a 1.94 times increase in odds for each grade.
Table 4.
Association of covariates with RT-PCR status for SARS-CoV-2, following propensity score matching and binomial logistic regression
| SARS-CoV-2 RT-PCR | |||||||
| Negative | Positive | OR (univariable) | OR (multivariable) | ||||
| n | 312 | 548 | |||||
| Chest X-ray report | Alternative pathology (%) | 3 (0.8) | 3 (0.5) | – | – | ||
| No abnormalities (%) | 123 (39.6) | 104 (19.1) | 0.76 | (0.08–6.82, p=0.801) | 0.48 | (0.03–8.82, p=0.620) | |
| Indeterminate/atypical findings (%) |
61 (19.5) | 136 (4.8) | 1.99 | (0.22–17.81, p=0.535) | 0.92 | (0.05–16.88, p=0.952) | |
| Classic COVID-19 (%) | 125 (40.1) | 305 (55.6) | 2.17 | (0.24–19.19, p=0.484) | 1.14 | (0.06–20.98, p=0.927) | |
| Age | Mean (SD) | 61.8 (17.9) | 67.0 (17.7) | 1.02 | (1.01–1.02, p<0.001)** | 1.02 | (1.00–1.03, p=0.028)* |
| Sex | Female (%) | 138 (44.3) | 212 (38.7) | – | – | ||
| Male (%) | 174 (55.7) | 336 (61.3) | 1.26 | (0.93–1.70, p=0.137) | 1.19 | (0.83–1.71, p=0.340) | |
| Ethnicity | Other Asian (%) | 31 (9.9) | 66 (12.0) | – | |||
| White (%) | 164 (52.7) | 270 (49.2) | 0.76 | (0.44–1.31, p=0.326) | 0.73 | (0.38–1.40, p=0.339) | |
| Black (%) | 39 (12.4) | 84 (15.3) | 1.01 | (0.52–1.98, p=0.974) | 0.92 | (0.43–1.97, p=0.827) | |
| Mixed (%) | 6 (1.8) | 4 (0.8) | 0.36 | (0.08–1.62, p=0.184) | 0.74 | (0.11–4.94, p=0.754) | |
| South Asian (%) | 22 (7.0) | 36 (6.6) | 0.77 | (0.34–1.76, p=0.531) | 0.68 | (0.28–1.65, p=0.390) | |
| Other (%) | 51 (16.2) | 89 (16.2) | 0.82 | (0.43–1.55, p=0.535) | 0.88 | (0.45–1.74, p=0.716) | |
| Comorbidity | No (%) | 65 (20.8) | 95 (17.4) | – | – | ||
| Yes (%) | 247 (79.2) | 453 (82.6) | 1.25 | (0.82–1.89, p=0.296) | 1 | (0.53–1.88, p=0.993) | |
| Dyspnoea on attendance | No (%) | 90 (28.8) | 139 (25.4) | – | – | ||
| Yes (%) | 222 (71.2) | 409 (74.6) | 1.19 | (0.82–1.73, p=0.356) | 0.84 | (0.53–1.32, p=0.447) | |
| Oxygen saturation | Median (IQR) | 96 (6) | 93 (8) | 0.94 | (0.91–0.97, p<0.001** | 0.97 | (0.93–1.00, p=0.072) |
| Respiratory rate | Median (IQR) | 23 (8) | 25 (8) | 1.04 | (1.01–1.07, p=0.002)* | 1.01 | (0.98–1.05, p=0.462) |
| Glasgow Coma Scale | Median (IQR) | 15 (0) | 15 (0) | 1.02 | (0.89–1.17, p=0.819) | 1.21 | (0.98–1.48, p=0.073) |
| Temperature | Mean (SD) | 37.2 (1.4) | 37.7 (1.1) | 1.48 | (1.26–1.73, p<0.001)** | 1.44 | (1.20–1.74, p<0.001)** |
| Heart rate | Mean (SD) | 96.7 (20.5) | 94.9 (21.5) | 1 | (0.99–1.00, p=0.305) | 1 | (0.99–1.01, p=0.702) |
| Systolic blood pressure | Mean (SD) | 136.2 (25.8) | 132.6 (24.5) | 0.99 | (0.99–1.00, p=0.086) | 0.99 | (0.98–1.00, p=0.097) |
| Neutrophils | Median (IQR) | 6.26 (4.52) | 5.05 (3.93) | 0.92 | (0.89–0.96, p<0.001)** | 0.87 | (0.82–0.91, p<0.001)** |
| D-dimer | Median (IQR) | 1220 (2343) | 1061 (1814) | 1 | (1.00–1.00, p=0.403) | 1 | (1.00–1.00, p=0.419) |
| C reactive protein | Median (IQR) | 45 (100) | 77 (107) | 1 | (1.00–1.01, p<0.001)** | 1 | (1.00–1.01, p=0.021)* |
| Troponin | Median (IQR) | 20 (55) | 21 (46) | 1 | (1.00–1.00, p=0.890) | 1 | (1.00–1.00, p=0.667) |
| Albumin | Median (IQR) | 39 (7) | 37 (6) | 0.97 | (0.94–1.00, p=0.071) | 1.02 | (0.98–1.06, p=0.432) |
| Creatine kinase | Median (IQR) | 94 (131) | 145 (263) | 1 | (1.00–1.00, p=0.119) | 1 | (1.00–1.00, p=0.152) |
| Admitted from ED | Admitted (%) | 235 (75.2) | 453 (82.7) | – | – | ||
| Discharged (%) | 77 (24.8) | 95 (17.3) | 1.56 | (1.06–2.33, p=0.022)** | 1.35 | (0.79–2.30, p=0.272) | |
| Admitted to ITU from ED | No (%) | 307 (98.5) | 532 (97.1) | – | – | ||
| Yes (%) | 5 (1.5) | 16 (2.9) | 1.92 | (0.60–6.18, p=0.274) | 1.06 | (0.25–4.40, p=0.940) | |
| 30-day follow-up status | Discharged (%) | 259 (83.0) | 368 (67.1) | – | – | ||
| Admitted (%) | 22 (6.9) | 47 (8.5) | 1.53 | (0.82–2.87, p=0.181) | 1.64 | (0.77–3.51, p=0.198) | |
| Dead (%) | 31 (10.1) | 133 (24.4) | 3 | (1.86–4.84, p<0.001)** | 2.81 | (1.22–6.50, p=0.017)* | |
*p<0.05; **p<0.001.
ED, emergency department; ITU, intensive treatment unit; RT-PCR, reverse transcriptase PCR.
Figure 2.
Forest plot of ORs of variables associated with reverse transcriptase PCR positivity for SARS-CoV-2 following multiple imputation, propensity score matching and binomial logistic regression. *Significant difference at the <0.05 level; **significant difference at the <0.001 level. ED, emergency department; ITU, intensive treatment unit.
Positive CXRs for COVID-19 were significantly associated with lower oxygen saturations (OR 0.94 95% CI 0.92 to 0.97, p<0.001) and temperatures (2.30 95% CI 1.46 to 3.63, p<0.001) in the ED following propensity score matching and multivariate regression (table 5 and figure 3).
Table 5.
Association of covariates with CXR report following propensity score matching and either binomial or ordinal logistic regression
| X-ray report | OR (univariable) | OR with XR as binary | OR with XR as ordinal | ||||||
| Other X-ray | Classical | ||||||||
| Findings | COVID-19 | Variable (multivariable) | Outcome (multivariable) | ||||||
| n | 430 | 430 | |||||||
| RT-PCR for SARS-CoV-2 | Negative (%) | 187 (43.4) | 125 (29.1) | – | – | – | |||
| Positive (%) | 243 (56.6) | 305 (7.9) | 1.85 | (1.36–2.56, p<0.001)** | 1.79 | (1.25–2.56, p<0.002)* | 1.94 | (1.37–2.76, p<0.001)** | |
| Age | Mean (SD) | 65.0 (18.9) | 65.3 (16.9) | 1 | (0.99–1.01, p=0.849) | 0.99 | (0.98–1.00, p=0.164) | 1 | (0.99–1.01, p=0.542) |
| Sex | Female (%) | 176 (40.9) | 175 (40.6) | – | – | ||||
| Male (%) | 254 (59.1) | 255 (59.3) | 1.01 | (0.75–1.37, p=0.940) | 0.87 | (0.63–1.20, p=0.400) | 1.02 | (0.49–2.09, p=0.967) | |
| Ethnicity | Other Asian (%) | 49 (11.4) | 48 (11.2) | – | – | ||||
| South Asian (%) | 29 (6.7) | 29 (6.7) | 1.04 | (0.52–2.04, p=0.912) | 1.02 | (0.47–2.17, p=0.965) | 1.02 | (0.49–2.09, p=0.967) | |
| Black (%) | 61 (14.2) | 61 (14.2) | 1.02 | (0.55–1.85, p=0.957) | 0.88 | (0.46–1.69, p=0.719) | 0.92 | (0.52–1.65, p=0.789) | |
| Mixed (%) | 5 (1.2) | 5 (1.2) | 0.92 | (0.21–4.00, p=0.911) | 0.86 | (0.18–4.17, p=0.853) | 0.85 | (0.17–4.30, p=0.838) | |
| Other (%) | 70 (16.3) | 70 (16.3) | 1.02 | (0.58–1.79, p=0.943) | 0.98 | (0.52–1.82, p=0.942) | 0.93 | (0.53–1.64, p=0.810) | |
| White (%) | 216 (50.2) | 217 (50.5) | 1.03 | (0.63–1.67, p=0.913) | 0.97 | (0.57–1.67, p=0.926) | 0.9 | (0.55–1.47, p=0.666) | |
| Comorbidity | No (%) | 82 (19.1) | 78 (18.1) | – | – | ||||
| Yes (%) | 348 (80.9) | 352 (81.9) | 0.95 | (0.66–1.36, p=0.777) | 0.93 | (0.59–1.49, p=0.782) | 0.88 | (0.57–1.37, p=0.592) | |
| Dyspnoea | No (%) | 191 (29.3) | 103 (24.0) | – | – | ||||
| Yes (%) | 304 (70.7) | 327 (76.0) | 1.31 | (0.92–1.88, p=0.123) | 1.2 | (0.80–1.82, p=0.380) | 1.22 | (0.83–1.80, p=0.301) | |
| Oxygen saturation | Median (IQR) | 95 (7) | 93 (7) | 0.94 | (0.91–0.96, p<0.001)** | 0.94 | (0.92–0.97, p<0.001)** | 0.94 | (0.91–0.97, p<0.001)** |
| Respiratory rate | Median (IQR) | 24 (10) | 24 (10) | 1.01 | (0.99–1.02, p=0.570) | 0.97 | (0.94–1.00, p=0.063) | 0.98 | (0.96–1.01, p=0.157) |
| Glasgow Coma Scale | Median (IQR) | 15 (0) | 15 (0) | 1.04 | (0.92–1.19, p=0.524) | 1.05 | (0.90–1.23, p=0.503) | 1.05 | (0.92–1.21, p=0.464) |
| Temperature | Mean (SD) | 37.6 (1.1) | 37.5 (1.3) | 0.93 | (0.83–1.06, p=0.297) | 0.79 | (0.67–0.93, p=0.006)* | 0.85 | (0.73–0.99, p=0.031)* |
| Heart rate | Mean (SD) | 95.7 (21.4) | 95.5 (21.0) | 1 | (0.99–1.01, p=0.888) | 1 | (0.99–1.01, p=0.864) | 1 | (0.99–1.01, p=0.872) |
| Systolic blood pressure | Mean (SD) | 133.8 (25.0) | 134.0 (25.6) | 1 | (0.99–1.01, p=0.907) | 1 | (0.99–1.01, p=0.335) | 1 | (1.00–1.01, p=0.478) |
| Neutrophils | Median (IQR) | 5.44 (4.54) | 5.67 (4.03) | 1 | (0.97–1.04, p=0.892) | 0.96 | (0.92–1.01, p=0.143) | 0.96 | (0.92–1.01, p=0.115) |
| D-dimer | Median (IQR) | 1119 (2221) | 1119 (1850) | 1 | (1.00–1.00, p=0.513) | 1 | (1.00–1.00, p=0.568) | 1 | (1.00–1.00, p=0.385) |
| C reactive protein | Median (IQR) | 46 (93) | 88 (110) | 1 | (0.99–1.00, p<0.001)** | 1 | (1.00–1.01, p<0.001)** | 1 | (1.00–1.01, p<0.001)** |
| Troponin | Median (IQR) | 23 (54) | 20 (46) | 1 | (1.00–1.00, p=0.231) | 1 | (1.00–1.00, p=0.277) | 1 | (1.00–1.00, p=0.059) |
| Albumin | Median (IQR) | 39 (7) | 37 (6) | 0.93 | (0.90–0.96, p<0.001)** | 0.93 | (0.90–0.97, p=0.001)* | 0.94 | (0.91–0.97, p=0.001)* |
| Creatine kinase | Median (IQR) | 110 (183) | 134 (239) | 1 | (1.00–1.00, p=0.535) | 1 | (1.00–1.00, p=0.242) | 1 | (1.00–1.00, p=0.186) |
| Admitted from ED | Admitted (%) | 315 (73.3) | 373 (86.7) | 2.37 | (1.63–3.46, p<0.001)** | 2.3 | (1.46–3.63, p<0.001)** | 2.22 | (1.47–3.33, p<0.001)** |
| Discharged (%) | 115 (26.7) | 57 (13.3) | – | – | – | ||||
| Admitted to ITU from ED | No (%) | 423 (98.4) | 416 (96.7) | – | – | ||||
| Yes (%) | 7 (1.6) | 14 (3.3) | 2.17 | (0.69–6.67, p=0.181) | 1.27 | (0.32–5.00, p=0.732) | 1.34 | (0.36–5.00, p=0.653) | |
| 30-day follow-up status | Discharged (%) | 316 (73.5) | 311 (72.3) | – | – | ||||
| Admitted (%) | 34 (7.9) | 34 (7.9) | 1.31 | (0.81–2.13, p=0.282) | 1.32 | (0.69–2.53, p=0.392) | 1.43 | (0.78–2.63, p=0.653) | |
| Dead (%) | 80 (18.6) | 85 (19.8) | 1.03 | (0.73–1.45, p=0.886) | 1.38 | (0.80–2.37, p=0.247) | 1.41 | (0.87–2.27, p=0.157) | |
*p<0.001.
ED, emergency department; ITU, intensive treatment unit; RT-PCR, reverse transcriptase PCR.
Figure 3.
Forest plot of ORs of variables associated with classical chest X-ray features COVID-19 following propensity score matching and binomial logistic regression. *Significant difference at the <0.05 level; **significant difference at the <0.001 level. ED, emergency department; ITU, intensive treatment unit; RT-PCR, reverse transcriptase PCR.
They also had higher rates of admission to a general ward from the ED (OR 2.30 95% CI 1.46 to 3.63, p<0.001) but no significant association with 30-day outcomes. There was a statistically significant increase in C reactive protein with a positive X-ray; however, this is unlikely to be clinically meaningful due to the minimal association (OR 1.00 95% CI 1.00 to 1.01).
Discussion
This study is the first to report the diagnostic accuracy of CXR and CT in the general emergency population during the COVID-19 pandemic.
We show that CXR has poor sensitivity and specificity for diagnosis of COVID-19, while CT has 29% higher sensitivity. Many international radiological guidelines advise against CT scanning for the initial assessment of COVID-1921–23 or where there are equivocal CXRs, while in other countries CT scanning is performed as a routine first-line investigation. Our results suggest that CT should be considered in the initial assessment of COVID-19 and that CXR findings poorly correlate with CT findings in this setting. We also show that indeterminate and non-classical features of COVID-19 significantly increase the sensitivity of these imaging modalities, without a significant decrease in specificity. Further, we demonstrate the limited prognostic value of CXR in COVID-19.
These findings mirror what has previously been reported in the literature on individuals with confirmed COVID-19. Wong et al19 showed a sensitivity of 59% for initial X-ray in confirmed COVID-19 infection, similarly initial case series in China also reported a sensitivity of 59.1%.12
A recent press article from Italy reported a much higher sensitivity of 89% for CXR in a smaller general emergency population (n=535) without confirmed COVID-19 at attendance.24 However, this used telephone follow-up for clinical symptoms of COVID-19 as a reference standard in individuals with an initial negative RT-PCR swab and appeared to classify any abnormal X-ray as positive, which may inflate this figure. When indeterminate CXRs are counted as positive in this study, the sensitivity would be in line with these Italian data. In the USA, a study of patients attending an urgent care centre with confirmed COVID-19 showed a much lower sensitivity at 41.7% for CXR where any abnormality was found on the images.25 In this study, 97 out of 636 reports were reclassified from ‘possible pneumonia’ to ‘normal’ on a second reading from a radiologist, highlighting the importance of inter-rater agreement and possibly explaining this low estimate.
CT has been reported in previous studies as being up to 98% sensitive for the diagnosis of COVID-19 in confirmed patients, when RT-PCR is used as the reference standard in confirmed patients.3 4 These studies used any potential features of COVID-19 (eg, ground glass opacification, crazy paving) as a positive scan, regardless of spatial distribution or features more characteristic of alternate pathology, unlike the BSTI guidelines used in this study. When we classified indeterminate CTs as positive like these latter studies, our estimates match their sensitivity values.
Consequently, a much lower specificity of 25% was found with initial RT-PCR in the previous literature; however, it is reported that 10 out of 15 (67%) of these negatives subsequently tested positive. This would give an adjusted specificity of 75%, considering subsequent swabs as a reference standard, which, combined with the wider CIs in these smaller studies, would bring estimates in line with the specificity in this paper. More recent meta-analyses have placed the pooled sensitivity of CT in populations with confirmed COVID-19 only, at 89.76% (95% CI 84.42% to 93.84%),26 in line with the estimates identified here.
There is limited coverage in the literature on association of X-ray findings with clinical and laboratory parameters and outcomes in the COVID-19 pandemic. This study demonstrates that classic appearances of COVID-19 were associated with initial lower saturations and lower temperature. Volume opacification of the lung fields was not quantified as a surrogate of severity; however, the use of the BSTI grading templates does this somewhat. When the X-ray report is considered as a graded scale from low likelihood of COVID-19 and severity to high likelihood and severity of disease, there was no significant difference in association with vital signs or laboratory parameters compared with when the X-ray report is merely considered as a binary positive and negative outcome for COVID-19.
Borghesi and colleagues have devised an X-ray grading system, the Brixia score, for severity in admitted patients with confirmed SARS-CoV-2 infection.27 They further found a significant increase in the severity of CXR by this scoring system in those who were discharged versus those who died.28 29
Here, there were no relevant associations between CXR and laboratory values. This analysis also found no association with positive X-rays and 30-day outcomes after multivariate analyses, unlike Borghese et al. This is also in contrast to Guan et al who found higher rates of ITU admission and death in those with positive imaging findings. However, these studies analysed only those with confirmed SARS-CoV-2 infection. The divergence observed in this study may be due to classifying those with ‘Alternate pathology/Indeterminate’ or ‘CVXC3/CVXC2’ as per the BSTI templates, negative for COVID-19 in these analyses. Other studies classified X-rays with any abnormality as a positive for COVID-19. These alternate distributions may still be reflective of underlying COVID-19 and we show significantly higher sensitivity for both CT and CXR when these are classed as positive. It may be that correlating indeterminate X-rays (in addition to classical images) with vitals, laboratory markers and 30-day outcomes would yield significant associations. However this may be unlikely, Xu and Zhang et al found that those with classical bilateral and diffuse involvement in upper and lower lobes had more severe disease than those without.30 31
There were a total of 70 confirmed PE in our dataset out of 114 CT pulmonary angiograms (61.0%, 5.84% of all patients attending) performed in the ED. The incidence of venous thromboembolism is reported as ranging from 20% to 30% in admitted confirmed SARS-CoV-2 positive patients.32 Although we have not focused on this cohort of patients in this paper for the sake of brevity and simplicity, this high incidence represents a further advantage for CT over CXR.
CT, even with the absence of contrast, has been shown to have strong accuracy in the diagnosis of PE and many imaging features correlate with the presence of PE. Sensitivities of non-contrast CT for diagnosis of PE have been reported at 96.9% and specificity at 71.9%.33 34
We therefore see the advantages of CT scanning in COVID-19 as threefold over other diagnostic techniques: (1) the rapid turnaround; (2) increased sensitivity; and (3) the possibility to identify PE in COVID-19, which are a significant burden in this group.
This must be balanced against the excess radiation exposure with CT. Radiation from CT and its association with carcinogenesis is difficult to quantify and no definitive epidemiological studies have confirmed excess risk of cancer.35 Modern CT scanners and software reconstruction techniques continue to minimise radiation exposure and many ways of shielding parts of the body from radiation also exist. Nevertheless, the excess risk of lifetime cancer is estimated at 1 per 5000 CT examinations.36
Strengths and limitations
This study is the largest conducted on imaging during the COVID-19 pandemic and one of the only studies conducted in the general population during the pandemic rather than only in confirmed patients. This enables greater applicability to the clinical setting where the diagnosis is uncertain, in addition to being able to calculate specificity, which is not possible in most studies. This study was planned to be powered to detect a sensitivity and specificity of 56% for CXR and greatly exceeded the sample size necessary for this.
Comprehensive statistical analyses were conducted to account for confounders in both factors influencing reporting of CXR and in factors affecting outcomes. The data were collected from prospectively maintained electronic records; however, the retrieval took place retrospectively with its inherent disadvantages. We were not able to collect data on several relevant covariates such as specific comorbidities or markers of severity such as lymphocytes. Furthermore, there was a significant amount of missing data that required multiple imputation to replace, although the fit of these imputed data was good, actual, observed data would be ideal.
Inter-rater reliability of imaging reports was not analysed in this paper and there was the potential for individual radiologists to have greater or lesser accuracy in the diagnosis of COVID-19. The literature has so far suggested a strong degree of agreement between radiologists in reporting of COVID-19 images.28
The single centre nature of this study further limits generalisability and the potential for interhospital disagreement in imaging, in addition to inter-rater disagreement.
Finally, the median time for patients to receive a CT scan was 4.5 days following initial attendance to ED. Thus, the scans may not have been directly comparable to the initial CXR, both because of the progression of disease and because the SARS-CoV-2 status may have been confirmed at this point, biassing the reporting of these scans.
Future research
Although this study used RT-PCR of nasopharyngeal swabs as a reference standard, newer methods exist for diagnosis of the disease. Serological assays for antibodies against SARS-CoV-2 are increasingly available and may represent a better gold standard in diagnosis for future research.37 RT-PCR is limited by swabbing technique for nasopharyngeal samples and the fact that the virus is more avid in the lower respiratory tract.38 However, many patients may not seroconvert prior to death, limiting this test to survivors only.
Point-of-care lung ultrasound is a new technique for diagnosis of COVID-19 which may mitigate many of the issues noted with the modalities discussed so far. It has no radiation, is fast, cheap and may be able to detect lower respiratory tract disease unlike nasopharyngeal swab.
However, there is limited evidence beyond small case series on its diagnostic accuracy.39–41 Further, like other ultrasound techniques, accuracy will likely be operator dependent42 and experience will need to be built up for robust results in evaluating suspected COVID-19.
Finally, much research has been conducted in the use of artificial intelligence techniques to correctly diagnose COVID-19 based on imaging.43–45 These techniques would obviate capacity limitations in reporting imaging as well as eliminate inter-reporter variability. However, as with any supervised machine learning technique, large, generalisable datasets, with correctly preclassified positive and negative cases (which in turn will depend on a truly accurate reference standard), are needed.46
Conclusion
CXR has poor sensitivity and specificity in diagnosing COVID-19 in the general population during the pandemic. CT scanning has demonstrated excellent sensitivity and should strongly be considered during the pandemic in the initial assessment of COVID-19. This needs to be balanced against the risk of excess radiation with CT, where capacity allows.
Supplementary Material
Acknowledgments
We would like to thank Scott Wilson from the Royal Free Hospital’s clinical practice group analytics department for retrieving the data from the hospital’s data warehouse. We would like to thank Dr Federico Ricciardi of the Department of Statistical Science and PRIMENT Clinical Trials Unit at University College London for reviewing the statistical methods in this study.
Footnotes
Correction notice: This article has been corrected since it was first published. Counclusion section of abstract has been corrected.
Funding: The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests: None declared.
Patient consent for publication: Not required.
Ethics approval: This study was registered with the local institutional review board as a service evaluation using anonymised data only. No formal ethics committee review was required.
Provenance and peer review: Not commissioned; externally peer reviewed.
Data availability statement: Anonymised data is available on reasonable request from the corresponding author. Analysis scripts are attached as a supplementary file.
Supplemental material: This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.
References
- 1.COVID-19 map. Johns Hopkins coronavirus Resour. Cent https://coronavirus.jhu.edu/map.html (accessed 30 Jun 2020). [Google Scholar]
- 2.Dong E, Du H, Gardner L. An interactive web-based dashboard to track COVID-19 in real time. Lancet Infect Dis 2020;0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Ai T, Yang Z, Hou H, et al. Correlation of chest CT and RT-PCR testing for coronavirus disease 2019 (COVID-19) in China: a report of 1014 cases. Radiology 2020;296:E32–40. 10.1148/radiol.2020200642 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Fang Y, Zhang H, Xie J, et al. Sensitivity of chest CT for COVID-19: comparison to RT-PCR. Radiology 2020;296:E115–7. 10.1148/radiol.2020200432 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Konrad R, Eberle U, Dangel A, et al. Rapid establishment of laboratory diagnostics for the novel coronavirus SARS-CoV-2 in Bavaria, Germany, February 2020. Eurosurveillance 2020;25. 10.2807/1560-7917.ES.2020.25.9.2000173 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Lin EC. Radiation risk from medical imaging. Mayo Clin Proc 2010;85:1142–6. 10.4065/mcp.2010.0260 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Mizumoto K, Kagaya K, Zarebski A, et al. Estimating the asymptomatic proportion of coronavirus disease 2019 (COVID-19) cases on board the diamond Princess cruise SHIP, Yokohama, Japan, 2020. Eurosurveillance 2020;25:2000180. 10.2807/1560-7917.ES.2020.25.10.2000180 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Arons MM, Hatfield KM, Reddy SC, et al. Presymptomatic SARS-CoV-2 infections and transmission in a skilled nursing facility. N Engl J Med 2020;382:2081–90. 10.1056/NEJMoa2008457 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.GOV.UK . High consequence infectious diseases (HCID). Available: https://www.gov.uk/guidance/high-consequence-infectious-diseases-hcid [Accessed 24 May 2020].
- 10.Desai S. COVID-19 BSTI reporting templates | the British Society of thoracic imaging. Br. Soc. Thorac. Imaging 2020. https://www.bsti.org.uk/covid-19-resources/covid-19-bsti-reporting-templates/ (accessed 29 Apr 2020). [Google Scholar]
- 11.NHS England . Guidance and standard operating procedure: COVID-19 virus testing in NHS laboratories, 2020. Available: https://www.england.nhs.uk/coronavirus/wp-content/uploads/sites/52/2020/03/guidance-and-sop-covid-19-virus-testing-in-nhs-laboratories-v1.pdf [Accessed 24 May 2020].
- 12.Guan W, Ni Z, Hu Y, et al. Clinical characteristics of coronavirus disease 2019 in China. N Engl J Med 2020;0:null.10/ggm6dh. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Honaker J, King G, Blackwell M. Amelia II: A Program for Missing Data. J Stat Softw 2011;45. 10.18637/jss.v045.i07 [DOI] [Google Scholar]
- 14.van Ginkel JR, Linting M, Rippe RCA, et al. Rebutting existing misconceptions about multiple imputation as a method for handling missing data. J Pers Assess 2020;102:297–308. 10.1080/00223891.2018.1530680 [DOI] [PubMed] [Google Scholar]
- 16.White IR, Royston P, Wood AM. Multiple imputation using chained equations: issues and guidance for practice. Stat Med 2011;30:377–99.10.1002/sim.4067. 15. [DOI] [PubMed] [Google Scholar]
- 16.He H, McDermott MP. A robust method using propensity score stratification for correcting verification bias for binary tests. Biostat Oxf Engl 2012;13:32–47.10/c4jzn6. [DOI] [PMC free article] [PubMed]
- 17.DE H, Imai K, King G, et al. MatchIt : Nonparametric Preprocessing for Parametric Causal Inference. J Stat Softw 2011;42. [Google Scholar]
- 18.Marshall A, Altman DG, Holder RL, et al. Combining estimates of interest in prognostic modelling studies after multiple imputation: current practice and guidelines. BMC Med Res Methodol 2009;9:57. 10.1186/1471-2288-9-57 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Wong HYF, HYS L, Fong AH-T, et al. Frequency and distribution of chest radiographic findings in COVID-19 positive patients. Radiology 2020:201160. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Cohen JF, Korevaar DA, Altman DG, et al. Stard 2015 guidelines for reporting diagnostic accuracy studies: explanation and elaboration. BMJ Open 2016;6:e012799. 10.1136/bmjopen-2016-012799 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Rubin GD, Ryerson CJ, Haramati LB, et al. The role of chest imaging in patient management during the COVID-19 pandemic: a multinational consensus statement from the Fleischner Society. Radiology 2020:201365. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.ACR recommendations for the use of chest radiography and computed tomography (CT) for suspected COVID-19 infection. Available: https://www.acr.org/Advocacy-and-Economics/ACR-Position-Statements/Recommendations-for-Chest-Radiography-and-CT-for-Suspected-COVID19-Infection [Accessed 5 Jun 2020].
- 23.British Society of Thoracic Imaging . COVID-19: BSTI statement and guidance, 20201. Available: https://www.bsti.org.uk/media/resources/files/COVID11.3.20_2.pdf [Accessed 5 Jun 2020].
- 24.Schiaffino S, Tritella S, Cozzi A, et al. Diagnostic performance of chest X-ray for COVID-19 pneumonia during the SARS-CoV-2 pandemic in Lombardy, Italy. J Thorac Imaging 2020. [DOI] [PubMed] [Google Scholar]
- 25.Weinstock MB, Echenique A, Russell JW, et al. Chest X-ray findings in 636 ambulatory patients with COVID-19. 10. Presenting to an Urgent Care Center: A Normal Chest X-Ray Is no Guarantee. [Google Scholar]
- 26.Bao C, Liu X, Zhang H, et al. Coronavirus disease 2019 (COVID-19) CT findings: a systematic review and meta-analysis. Journal of the American College of Radiology 2020;17:701–9. 10.1016/j.jacr.2020.03.006 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Borghesi A, Zigliani A, Masciullo R, et al. Radiographic severity index in COVID-19 pneumonia: relationship to age and sex in 783 Italian patients. Radiol Med 2020;125:461–4. 10.1007/s11547-020-01202-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Borghesi A, Maroldi R. COVID-19 outbreak in Italy: experimental chest X-ray scoring system for quantifying and monitoring disease progression. Radiol Med 2020;125:509–13. 10.1007/s11547-020-01200-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Borghesi A, Zigliani A, Golemi S, et al. Chest X-ray severity index as a predictor of in-hospital mortality in coronavirus disease 2019: a study of 302 patients from Italy. International Journal of Infectious Diseases 2020;96:291–3. 10.1016/j.ijid.2020.05.021 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Y-H X, Dong J-H, W-M A, et al. Clinical and computed tomographic imaging features of novel coronavirus pneumonia caused by SARS-CoV-2. J Infect 2020;80:394–400. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Zhang J-J, Dong X, Cao Y-Y, et al. Clinical characteristics of 140 patients infected with SARS-CoV-2 in Wuhan, China. allergy (Published Online First: 19 February 2020). [DOI] [PubMed]
- 32.Lodigiani C, Iapichino G, Carenzo L, et al. Venous and arterial thromboembolic complications in COVID-19 patients admitted to an academic hospital in Milan, Italy. Thromb Res 2020;191:9–14. 10.1016/j.thromres.2020.04.024 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Chien C-H, Shih F-C, Chen C-Y, et al. Unenhanced multidetector computed tomography findings in acute central pulmonary embolism. BMC Med Imaging 2019;19:65. 10.1186/s12880-019-0364-y [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Mohamed N, Othman MM, Hassan L, et al. The accuracy of non-contrast chest computed tomographic scan in the detection of pulmonary thromboembolism. J Curr Med Res Pract 2019;4:61. 10.4103/JCMRP.JCMRP_13_19 [DOI] [Google Scholar]
- 35.McCollough CH, Bushberg JT, Fletcher JG, et al. Answers to common questions about the use and safety of CT scans. Mayo Clinic Proceedings 2015;90:1380–92. 10.1016/j.mayocp.2015.07.011 [DOI] [PubMed] [Google Scholar]
- 36.Moser JB, Sheard SL, Edyvean S, et al. Radiation dose-reduction strategies in thoracic CT. Clin Radiol 2017;72:407–20. 10.1016/j.crad.2016.11.021 [DOI] [PubMed] [Google Scholar]
- 37.Long Q-X, Liu B-Z, Deng H-J, et al. Antibody responses to SARS-CoV-2 in patients with COVID-19. Nat Med 2020:1–4. [DOI] [PubMed] [Google Scholar]
- 38.Wang W, Xu Y, Gao R, et al. Detection of SARS-CoV-2 in different types of clinical specimens. JAMA 2020;323:1843–4. 10.1001/jama.2020.3786 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Smith MJ, Hayward SA, Innes SM, et al. Point-of-care lung ultrasound in patients with COVID-19 - a narrative review. Anaesthesia 2020;75:1096–1041. 10.1111/anae.15082 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Haaksma ME, Heldeweg MLA, Matta JEL, et al. Lung ultrasound findings in patients with novel SARS-CoV2. medRxiv 2020;2020:05.18.20105775. [Google Scholar]
- 41.Benchoufi M, Bokobza J, Chauvin AA, et al. Lung injury in patients with or suspected COVID-19 : a comparison between lung ultrasound and chest CT-scanner severity assessments, an observational study. medRxiv 2020;2020:04.24.20069633. [Google Scholar]
- 42.Fine D, Perring S, Herbetko J, et al. Three-Dimensional (3D) ultrasound imaging of the gallbladder and dilated biliary tree: reconstruction from real-time B-scans. Br J Radiol 1991;64:1056–7. 10.1259/0007-1285-64-767-1056 [DOI] [PubMed] [Google Scholar]
- 43.Shi F, Wang J, Shi J, et al. Review of artificial intelligence techniques in imaging data acquisition, segmentation and diagnosis for COVID-19. IEEE Rev Biomed Eng 2020:1. 10.1109/RBME.2020.2987975 [DOI] [PubMed] [Google Scholar]
- 44.Li L, Qin L, Xu Z, et al. Artificial intelligence distinguishes COVID-19 from community acquired pneumonia on chest CT. radiology 2020;:200905.10/ggpdgp.
- 45.Wang L, Wong A. COVID-Net: a tailored deep Convolutional neural network design for detection of COVID-19 cases from chest X-ray images. ArXiv200309871 Cs Eess. Available: http://arxiv.org/abs/2003.09871 [Accessed 13 Jun 2020]. [DOI] [PMC free article] [PubMed]
- 46.Kotsiantis SB. Use of machine learning techniques for educational proposes: a decision support system for forecasting students’ grades. Artif Intell Rev 2012;37:331–44.10/fmbng4 19. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
bmjopen-2020-042946supp001.pdf (1MB, pdf)
bmjopen-2020-042946supp002.pdf (1.3MB, pdf)