Abstract
Objective
To describe the characteristics and outcomes of patients with a clinical diagnosis of COVID-19 and false-negative SARS-CoV-2 reverse transcription-PCR (RT-PCR), and develop and internally validate a diagnostic risk score to predict risk of COVID-19 (including RT-PCR-negative COVID-19) among medical admissions.
Design
Retrospective cohort study.
Setting
Two hospitals within an acute NHS Trust in London, UK.
Participants
All patients admitted to medical wards between 2 March and 3 May 2020.
Outcomes
Main outcomes were diagnosis of COVID-19, SARS-CoV-2 RT-PCR results, sensitivity of SARS-CoV-2 RT-PCR and mortality during hospital admission. For the diagnostic risk score, we report discrimination, calibration and diagnostic accuracy of the model and simplified risk score and internal validation.
Results
4008 patients were admitted between 2 March and 3 May 2020. 1792 patients (44.8%) were diagnosed with COVID-19, of whom 1391 were SARS-CoV-2 RT-PCR positive and 283 had only negative RT-PCRs. Compared with a clinical reference standard, sensitivity of RT-PCR in hospital patients was 83.1% (95% CI 81.2%–84.8%). Broadly, patients with false-negative RT-PCR COVID-19 and those confirmed by positive PCR had similar demographic and clinical characteristics but lower risk of intensive care unit admission and lower in-hospital mortality (adjusted OR 0.41, 95% CI 0.27–0.61). A simple diagnostic risk score comprising of age, sex, ethnicity, cough, fever or shortness of breath, National Early Warning Score 2, C reactive protein and chest radiograph appearance had moderate discrimination (area under the receiver–operator curve 0.83, 95% CI 0.82 to 0.85), good calibration and was internally validated.
Conclusion
RT-PCR-negative COVID-19 is common and is associated with lower mortality despite similar presentation. Diagnostic risk scores could potentially help triage patients requiring admission but need external validation.
Keywords: COVID-19, molecular diagnostics, epidemiology
Strengths and limitations of this study.
Large cohort of consecutive acute medical admissions in two hospitals covering a diverse population in London, UK, during first COVID-19 ‘peak’.
Assessment of ‘real-world’ performance of SARS-CoV-2 reverse transcription-PCR (RT-PCR) from nasopharyngeal swabs for diagnosis of COVID-19.
Inherent limitations of retrospective cohort study design, including some missing data.
Not all patients had SARS-CoV-2 RT-PCR testing.
Introduction
The COVID-19 global pandemic, caused by infection with theSARS-CoV-2, has led to unprecedented numbers of unwell and infectious patients requiring admission to hospital. The symptoms of COVID-19 can be nonspecific, so diagnostic confirmation in hospital is often sought by detection of SARS-CoV-2 RNA sequences by reverse transcription-PCR (RT-PCR) from a clinical specimen.
Since the beginning of the pandemic, the standard sample for PCR testing has been a nasopharyngeal swab (NPS) or aspirate, but there are concerns that a significant proportion of cases tests negative on initial RT-PCR of an NPS sample, with many patients having repeated sampling to confirm the diagnosis.1 A systematic review of real-world diagnostic sensitivity of SARS-CoV-2 RT-PCR reports that up to 33% of patients with COVID-19 may have an initial false-negative NPS result despite a compatible clinical illness, consistent thoracic imaging and/or subsequent positive antibodies to COVID-19.2–5 False-negative RT-PCR may result from inadequate nasopharyngeal sampling technique, delayed time to analysis, ineffective sample storage, variable gene targets in RT-PCR assays leading to imperfect analytic sensitivity or if a patient is tested at a point when viral throat carriage is absent or below the detectable threshold (either too early or too late).6 7 This high false-negative rate complicates both hospital infection control and clinical decision-making. Being able to identify patients with a high probability of COVID-19 despite a negative RT-PCR is crucial for effective clinical care.
The clinical characteristics and outcomes of hospitalised patients with COVID-19 have been well described globally, but these studies are limited to patients with RT-PCR-confirmed COVID-19.8–10 The pattern of disease and outcomes of patients with false-negative COVID-19 tests has neither been well reported to date nor the diagnostic accuracy of RT-PCR assays in secondary care settings in the UK. Several studies have derived and validated risk scores to assess severity and prognosis among patients with COVID-19; however, few risk scores focus on identifying patients with COVID-19 among those needing hospital admission and those that do are from outside the UK, do not consider all hospital admissions, rely on high-resolution CT scanning of the lungs and exclude patients without RT-PCR-confirmed disease.11
We, therefore, aim to describe the characteristics and outcomes of patients with a clinical diagnosis of COVID-19 but with negative RT-PCR from NPS and the real-world sensitivity of RT-PCR for COVID-19. Second, we describe predictors of COVID-19 among general medical admissions, including assessing whether a simple diagnostic risk score could be derived, internally validated and used to predict which patients admitted to medical wards will have COVID-19.
Methods
Study design
This is a retrospective observational cohort study of consecutive admissions in London North West University Healthcare NHS Trust, comprising two hospitals, Northwick Park and Ealing. Patients were included in this study if they were admitted via the acute medical team between 2 March and 3 May 2020 inclusive.
Data collection
Cases were identified retrospectively through electronic medical admission lists. Deidentified data on patient demographics, comorbidities, clinical characteristics, vital signs, routine biochemical, haematological and microbiological tests, diagnosis and clinical outcomes were extracted from routinely collected clinical data using electronic patient record systems and other NHS Trust health information systems. Physiological observations were those first recorded on admission to the emergency department. All biochemical and haematological data were from the first samples taken within 48 hours of admission. Thoracic imaging (chest radiographs and CT) were reported by consultant radiologists and coded based on COVID-19 guidelines from the British Society of Thoracic Imaging (BSTI).12
RT-PCR of a clinical specimen from NPS was the only SARS-CoV-2 testing available during the study period. The decision to test was based on a clinical suspicion of COVID-19. Testing was performed at the point of admission or as soon as possible afterwards. Due to high demand and limited capacity, some patients with high clinical suspicion did not undergo SARS-CoV-2 testing. Routine testing for all admissions was introduced after the study period. Most SARS-CoV-2 testing was done using Panther Fusion (Hologic; ORF1ab Region 1/2 target) or Abbott RealTime (RNA-dependent RNA polymerase, nucleocapsid target) assays on NPS.
Approval for this study was provided by London North West University Healthcare NHS Trust research and governance department, and the NHS Health Regulatory Authority (IRAS ID 285815). Written informed consent from participants was not obtained in compliance with Secretary of State for Health and Social Care ‘Notice’ under Regulation 3 (4) of the Health Service Control of Patient Information Regulations 20 021 requiring health providers to process confidential patient and Control of Patient Information Regulations due to the COVID-19 pandemic.
Definitions
Patients were assigned as having RT-PCR-confirmed COVID-19 if they had a positive SARS-CoV-2 RT-PCR within 7 days before or after the date of admission and had a discharge diagnosis of COVID-19 recorded by the clinical team. False-negative RT-PCR COVID-19 was defined as patients with a discharge diagnosis of COVID-19 made by the clinical team and one or more negative SARS-CoV-2 RT-PCR within 48 hours of admission in the absence of any positive SARS-CoV-2 RT-PCR results. Patients with evidence of alternative diagnoses (ie, not COVID-19) made by the clinical team and no positive SARS-CoV-2 RT-PCR results were defined as not having COVID-19. Medical records were reviewed for patients with positive SARS-CoV-2 tests greater than 7 days after admission but before discharge and a diagnosis of COVID-19, to assess whether the admission was likely to represent a missing or delayed SARS-CoV-2 RT-PCR result (ie, patients with community-acquired COVID-19) or nosocomial COVID-19 transmission. Mortality was assessed at discharge from hospital.
Statistical methods
Basic descriptive statistics were performed, with continuous data presented as median (IQR) and categorical data as frequency (%). Comparisons were made using χ2 tests for proportions, t-tests for means and Wilcoxon rank sum for medians. Logistic regression was used to assess associations between variables and diagnosis of COVID-19. In exploratory analyses to assess association between RT-PCR-negative COVID-19 and mortality, a multivariable logistic regression model was used adjusting for other variable associated with poor outcomes in COVID-19.13
Sensitivity and false-negative RT-PCR
The real-world sensitivity of SARS-CoV-2 RT-PCR from NPS against a reference standard of a clinical diagnosis of COVID-19 was estimated as the proportion of patients positive from any RT-PCR, excluding those without any valid RT-PCR results. Sensitivity was also calculated by restricting analyses to patients with two or more RT-PCR results from NPS taken in a 24-hour and 48-hour period. The reference standard was patients with at least one positive RT-PCR in the time period. Incremental yield of a second RT-PCR following an initial negative result in patients was also calculated. Specificity of SARS-CoV-2 RT-PCR was assumed to be 100%.
Diagnostic risk score
In development of a score to predict COVID-19 among medical admissions, candidate predictor variables were selected based on a priori knowledge, published literature, clinical reasoning and the need for variables to be objective, reproducible, available in the emergency department soon after presentation. We considered demographic characteristics (age, sex, ethnicity), clinical symptoms associated with COVID-19 (cough, fever or shortness of breath), vital signs (including National Early Warning Score 2 (NEWS 2)) and laboratory bloods (including C reactive protein (CRP) and arterial/venous blood gas) at the time of presentation to hospital.
Continuous variables were assessed for non-linearity using fractional polynomials and categorised based on established cut-off values and/or fractional polynomials. Complete case analysis was chosen for derivation and internal validation of the score, given most key variables had fewer than 10% missing data. To derive a prediction model, we undertook univariable logistic regression analysis assessing associations between candidate variables and COVID-19 diagnosis (including all COVID-19 irrespective of RT-PCT status). We then used a backward elimination approach to create a multivariable predictive model, with stepwise elimination of variables, using likelihood ratio tests and Akaike information criterion to compare models. Interaction in the model was also assessed using likelihood ratio testing.
Points were assigned to each variable by identifying clusters of regression coefficients from the final model, then taking the median of those clustered coefficients and scaling, so the lowest point score is at least 1, and then rounding to the nearest integer.14 A COVID-19 diagnostic risk score was then derived by combining the points based on patient characteristics. Performance of both the full predictive model and risk score was assessed using the area under the receiver operating characteristic (AUROC) curve (also known as concordance statistic) for discrimination and plots of predicted probability of COVID-19 against observed risk of COVID-19 for calibration (calibration plots). Decision curve analysis was also conducted to help weigh benefits of using the model, compared with assuming all or no patients were diagnosed with COVID-19, and comparison with other single variables with strong associations with COVID-19.
Internal model validation was done using the bootstrap procedure, with final model applied to each bootstrap sample (n=200), and an optimism-corrected AUROC curve was calculated.15 A prediction model was also generated using bootstrap samples and tested on the original data set. Cut-off thresholds were defined to identify patients at high-risk and low-risk of COVID-19 after plotting risk score against observed COVID-19 risk such that the high-risk group accounted for as many COVID-19 cases as the low risk as few as possible. Sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) were calculated for each threshold, and NPV and PPV were calculated for varying prevalence of COVID-19 among medical admissions. Sensitivity analysis used multivariate multiple imputation with chained equations for missing data, assuming they were missing at random. Imputation was done for missing candidate predictor variables using 20 imputations, and model generation and performance were repeated. All analyses were done using Stata V.16 (StataCorp 2019). Predictive modelling elements are presented in accordance with TRIPOD guidance.16
Results
Patient characteristics
Between 2 March and 3 May 2020, 4008 patients were admitted (2536 at Northwick Park Hospital and 1472 at Ealing Hospital), with 1792 (44.7%) diagnosed with COVID-19 (figure 1). There were a median of 65 (IQR 57–76) admissions daily, including median daily admission of 47 (IQR 28–56) patients diagnosed with COVID-19 (online supplemental figure 1). COVID-19 diagnoses of 1391 (77.6%) had at least one positive SARS-CoV-2 RT-PCR, 283 (15.8%) had at least one negative and no positive RT-PCR, and 119 (6.6%) did not have an RT-PCR result.
Figure 1.
Patient flow diagram by final diagnosis and SARS-CoV-2 RT-PCR status with outcomes. Note ‘presumed COVID-19’ includes patients who were RT-PCR negative (n=293) and those who did not have a valid RT-PCR results (n=109). RT-PCR, reverse transcription-PCR.
bmjopen-2020-047110supp005.pdf (82KB, pdf)
There were several differences between patients with and without a COVID-19 diagnosis at discharge (including those with false-negative RT-PCR results, table 1, online supplemental table 1). Most notably patients with COVID-19 were more likely to be men, be more unwell at admission (NEWS score 6 vs 2 for patients without COVID-19) and more likely to need supplementary oxygen. On chest radiograph, patients with COVID-19 were more likely to have lung infiltrates (79% vs 48%) and less likely to have clear lung fields (7% vs 33%).
Table 1.
Baseline characteristics and outcomes for patients, including demographics, co-morbidities, admission vital signs and laboratory blood tests, stratified by diagnosis and SARS- CoV-2 RT-PCR status
| Not diagnosed with COVID | All COVID diagnoses | P value | COVID diagnosis PCR negative | COVID diagnosis PCR positive | P value | ||
| n=2215 | n=1793 | n=283 | n=1391 | ||||
| Age at admission, median years (IQR) | 71 (51, 82) (n=2215) | 69 (56, 81) (n=1793) | 0.44 | 70 (54, 79) (n=283) | 70 (57, 81) (n=1391) | 0.27 | |
| Age 65 years or older | 1266 (57.2%) | 1005 (56.1%) | 0.48 | 154 (54.4%) | 800 (57.5%) | 0.34 | |
| Sex | Female | 1021 (46.1%) | 651 (36.3%) | <0.0001 | 112 (39.6%) | 498 (35.8%) | 0.23 |
| Male | 1193 (53.9%) | 1142 (63.7%) | 171 (60.4%) | 893 (64.2%) | |||
| Ethnicity | South Asian | 486 (21.9%) | 447 (24.9%) | <0.0001 | 57 (20.1%) | 362 (26.0%) | 0.15 |
| Asian Other | 174 (7.9%) | 211 (11.8%) | 30 (10.6%) | 162 (11.6%) | |||
| Black African or Caribbean | 212 (9.6%) | 224 (12.5%) | 33 (11.7%) | 181 (13.0%) | |||
| Mixed ethnicity | 6 (0.3%) | 10 (0.6%) | 2 (0.7%) | 8 (0.6%) | |||
| Unknown | 330 (14.9%) | 318 (17.7%) | 53 (18.7%) | 233 (16.8%) | |||
| White European | 890 (40.2%) | 458 (25.5%) | 81 (28.6%) | 361 (26.0%) | |||
| Other | 117 (5.3%) | 125 (7.0%) | 27 (9.5%) | 84 (6.0%) | |||
| Index of multiple deprivation decile, median (IQR) | 5 (3, 7) (n=2105) | 5 (3, 6) (n=1743) | 0.048 | 4 (3, 6) (n=277) | 5 (3, 6) (n=1366) | 0.043 | |
| Diabetes | 563 (25.7%) | 599 (33.6%) | <0.0001 | 81 (28.9%) | 482 (34.8%) | 0.059 | |
| Hypertension | 825 (37.7%) | 739 (41.5%) | 0.015 | 110 (39.3%) | 590 (42.6%) | 0.31 | |
| Ischaemic heart disease | 413 (18.9%) | 309 (17.3%) | 0.21 | 44 (15.7%) | 247 (17.8%) | 0.40 | |
| Heart failure | 156 (7.1%) | 70 (3.9%) | <0.0001 | 14 (5.0%) | 53 (3.8%) | 0.36 | |
| Chronic obstructive pulmonary disease | 185 (8.5%) | 112 (6.3%) | 0.010 | 21 (7.5%) | 88 (6.3%) | 0.48 | |
| Asthma | 200 (9.1%) | 165 (9.3%) | 0.89 | 19 (6.8%) | 133 (9.6%) | 0.14 | |
| Cancer | 169 (7.7%) | 78 (4.4%) | <0.0001 | 11 (3.9%) | 65 (4.7%) | 0.58 | |
| HIV | 21 (1.0%) | 14 (0.8%) | 0.56 | 3 (1.1%) | 11 (0.8%) | 0.64 | |
| Cerebrovascular disease | 110 (5.0%) | 96 (5.4%) | 0.61 | 15 (5.4%) | 75 (5.4%) | 0.97 | |
| Dementia | 156 (7.1%) | 188 (10.5%) | <0.0001 | 29 (10.4%) | 153 (11.0%) | 0.74 | |
| Chronic kidney disease | 263 (12.0%) | 233 (13.1%) | 0.31 | 33 (11.8%) | 182 (13.1%) | 0.54 | |
| Cough | 537 (24.5%) | 1114 (62.5%) | <0.0001 | 177 (63.2%) | 865 (62.4%) | 0.80 | |
| Shortness of breath | 687 (31.4%) | 1171 (65.7%) | <0.0001 | 203 (72.5%) | 886 (63.9%) | 0.006 | |
| Fever | 547 (25.0%) | 1117 (62.7%) | <0.0001 | 184 (65.7%) | 860 (62.0%) | 0.25 | |
| Confusion | 241 (11.0%) | 195 (10.9%) | 0.95 | 30 (10.7%) | 153 (11.0%) | 0.87 | |
| Symptom duration (days), median (IQR) | 4 (2, 12) (n=592) | 7 (3, 10) (n=1083) | 0.010 | 7 (3, 12) (n=163) | 6 (3, 10) (n=844) | 0.021 | |
| Observations | |||||||
| Pulse >120 bpm | 203 (10.3%) | 241 (14.3%) | <0.0001 | 41 (15.4%) | 177 (13.4%) | 0.39 | |
| Respiratory rate >30 per minute | 175 (8.9%) | 568 (33.6%) | <0.0001 | 90 (33.8%) | 439 (33.3%) | 0.87 | |
| Temperature >38°C | 180 (9.2%) | 605 (35.9%) | <0.0001 | 72 (27.0%) | 495 (37.7%) | <0.001 | |
| Systolic blood pressure <100 mm Hg | 108 (5.5%) | 101 (6.1%) | 0.51 | 16 (6.1%) | 78 (6.0%) | 0.97 | |
| Consciousness level | Alert | 646 (95.1%) | 596 (96.0%) | 0.93 | 101 (97.1%) | 449 (95.5%) | 0.47 |
| Confusion | 13 (1.9%) | 11 (1.8%) | 3 (2.9%) | 8 (1.7%) | |||
| Verbal | 8 (1.2%) | 5 (0.8%) | 0 (0.0%) | 4 (0.9%) | |||
| Pain | 5 (0.7%) | 3 (0.5%) | 0 (0.0%) | 3 (0.6%) | |||
| Unresponsive | 7 (1.0%) | 6 (1.0%) | 0 (0.0%) | 6 (1.3%) | |||
| O2 saturations<94% | 198 (10.1%) | 543 (32.2%) | <0.0001 | 79 (29.8%) | 430 (32.6%) | 0.37 | |
| NEWS 2 Score, median (IQR) | 2 (1, 4) (n=1951) | 6 (3, 8) (n=1666) | <0.0001 | 6 (4, 7) (n=264) | 6 (3, 8) (n=1299) | 0.73 | |
| NEWS 2 Score ≥5 | 477 (24.4%) | 1084 (65.1%) | <0.0001 | 176 (66.7%) | 840 (64.7%) | 0.53 | |
| Supplementary oxygen | 169 (8.8%) | 529 (33.1%) | <0.0001 | 96 (37.9%) | 404 (32.4%) | 0.091 | |
| PO2 <8 mm Hg | 127 (35.4%) | 251 (36.2%) | 0.79 | 34 (27.9%) | 205 (38.7%) | 0.025 | |
| PCO2 >6 mm Hg | 124 (34.5%) | 75 (10.8%) | <0.0001 | 12 (9.8%) | 59 (11.1%) | 0.68 | |
| Neutrophils>10×109/L | 361 (17.8%) | 250 (15.6%) | 0.083 | 52 (19.0%) | 183 (14.7%) | 0.078 | |
| Lymphocytes <1 x109/L | 509 (25.1%) | 736 (46.1%) | <0.0001 | 107 (39.1%) | 594 (47.8%) | 0.009 | |
| Platelet count x109/L, median (IQR) | 246.0 (193.0, 317.0) (n=2025) | 231.0 (177.0, 306.0) (n=1597) | <0.0001 | 263.0 (206.0, 343.0) (n=274) | 226.0 (172.0, 297.0) (n=1242) | <0.0001 | |
| Creatinine >120 mmol/L | 507 (25.2%) | 426 (26.9%) | 0.24 | 64 (23.8%) | 338 (27.4%) | 0.23 | |
| CRP μg/mL, median (IQR) | 16.1 (3.4, 66.9) (n=1928) | 98.7 (46.0, 175.3) (n=1590) | <0.0001 | 86.2 (41.7, 170.1) (n=272) | 101.5 (48.3, 180.2) (n=1237) | 0.15 | |
| Influenza RT-PCR | Influenza A | 11 (2.3%) (n=490) | 1 (0.2%) (n=528) | <0.0001 | 0 (n=72) | 1 (0.2%) (n=445) | 0.31 |
| Influenza B | 9 (1.9%) | 2 (0.4%) | 1 (1.4%) | 1 (0.2%) | |||
| Admitted to ICU | 21 (1.0%) | 227 (12.7%) | <0.0001 | 22 (7.8%) | 192 (13.8%) | 0.006 | |
| Died during hospital admission | 142 (6.4%) (n=2202) | 482 (26.9%) (n=1789) | <0.0001 | 47 (16.6%) | 408 (29.3%) (n=1387) | <0.0001 | |
Data on co-morbidities represent number with each condition. Where data are missing, total numbers in each category are presented in brackets. P values are calculated using Χ2 for proportions, t-tests for means and Wilcoxon rank sum for medians.
CRP, C reactive protein; ICU, intensive care unit; IQR, inter quartile range; NEWS, National Early Warning Score; PCO2, partial pressure of carbon dioxide; PO2, partial pressure of oxygen.
bmjopen-2020-047110supp001.pdf (59.7KB, pdf)
Outcomes
Overall, 248 (6.2%) of medical admissions were admitted to intensive care unit (ICU) for level 2 or 3 support. Patients with COVID-19 diagnosis were more likely to be admitted to ICU (12.7% compared with 1.0%, p<0.0001). Median time to intensive care admission was 1 day (IQR 0–3) from admission. Inpatient mortality was 15.6% overall with substantially higher mortality in patients with COVID-19 diagnosis (26.9% compared with 6.4%). Of 0.4% (n=16) remained admitted at the time of data extraction or were missing mortality status. Inpatient death occurred a median of 5 (IQR 2–10) days after admission for patients with COVID-19, and hospital stay was longer than for those without COVID-19 (median 5 (IQR 3–11) days compared with median 3 (IQR 1–7) days, p<0.0001).
Sensitivity of SARS-CoV-2 RT-PCR
Based on patients with COVID-19 with at least one valid SARS-CoV-2 RT-PCR result (n=1674), 16.9% (n=283) diagnosed with COVID-19 had at least one false-negative RT-PCR. Of 217 patients had a single negative result, with 66 having two or more negative results. Median time from admission to negative swab was 0 (IQR 0–1) days. Based on a clinical COVID-19 reference standard, the sensitivity of PCR was 83.1% (95% CI 81.2% to 84.8%). The diagnostic yield (ie, including those without SARS-CoV-2 PCR results) of SAR-CoV-2 PCR testing of NPS was 77.6% (95% CI 75.6% to 79.5%). If restricted to patients with chest radiology suggestive of COVID-19, 198/968 patients with COVID-19 were RT-PCR negative, giving a sensitivity of 79.6%.
A total of 185 patients with COVID-19 had two RT-PCR tests within 24 hours, at least one of which was positive. Of 35/185 had a false-negative RT-PCR, giving a sensitivity of 81.1% (95% CI 74.7% to 86.5%). Of 62/254 patients with COVID-19 and two or more RT-PCR tests within 48 hours, giving a sensitivity of 75.6% (95% CI 70.0% to 80.5%). A total of 557 patients with two RT-PCR tests within 24 hours had an initial negative test, of whom 17 had a second test that was positive, giving an incremental yield of 3.1% (95% CI 1.9% to 4.8%). Of 36/669 patients with an initial negative RT-PCR had a second test that was positive within 48 hours, giving an incremental yield of 5.4% (95% CI 3.9% to 7.4%).
False-negative COVID-19 RT-PCR
Of patients with RT-PCR-negative COVID-19, 70.0% (198/283) had chest radiography or chest CT suggestive of COVID-19 based on BSTI coding, 80.2% (227/283) had lung infiltrates on chest imaging, and only 6.7% (19/283) had normal lung fields on chest radiography; 88.0% reported cough, fever or shortness of breath at admission. Broadly, patients with false-negative RT-PCR COVID-19 and those confirmed by positive PCR had similar demographic and clinical characteristics. Distribution of NEWS score and CRP were similar to RT-PCR-confirmed patients with COVID-19 and differed from those without COVID-19 diagnosis (online supplemental figure 2). Notable differences include false-negative patients with RT-PCR COVID-19 being more likely to report shortness of breath, slightly longer duration of symptoms (median of 7 (IQR 3–12) days compared with 6 (IQR 3–10) days for PCR-positive patients) (table 1). False-negative patients with RT-PCR also had higher median lymphocyte and platelet counts.
bmjopen-2020-047110supp006.pdf (61.9KB, pdf)
Importantly, outcomes were worse for patients with RT-PCR-confirmed COVID-19 compared with those who were had a false-negative RT-PCR, with a higher proportion admitted to ICU (13.8 (95% CI 12.1 to 15.7 vs 7.8 (95% CI 5.2 to 11.5)%, p=0.006), and more patients dying during admission (29.3 (95% CI 27.0 to 31.8)% vs 16.6 (95% CI 12.7 to 21.4)%, p<0.0001). When limited to patients with chest radiology suggestive of COVID-19, patients with false-negative RT-PCR disease still had better outcomes than PCR-confirmed COVID-19 (ICU admission 8.4%, mortality 16.3%, n=227). In exploratory analyses adjusted for age, sex, comorbidities, admission oxygen saturation and admission urea, OR for mortality was 0.41 (95% CI 0.27 to 0.61) for RT-PCR-negative compared with RT-PCR-positive COVID-19 (see online supplemental figure 2).
Predictors of COVID-19 and diagnostic model
Several demographic and clinical variables were strongly associated with a diagnosis of COVID-19, both in univariable and multivariable analysis (table 2). Abnormal chest radiography with infiltrates (OR 7.8, 95% CI 6.3 to 9.6), CRP over 50 (OR 6.0, 95% CI 5.2 to 6.9) and NEWS 2 score 5 or more (OR 5.2, 95% CI 5.0 to 6.6) had the strongest associations with COVID-19 diagnosis.
Table 2.
Univariable and multivariable logistic regression analysis for risk of COVID-19 diagnosis
| Univariable regression | Multivariable regression | |||||
| Variable | N | OR (95% CI) | P | OR (95% CI) | P | |
| Age | Increase 10 years | 4008 | 1.05 (1 to 1.08) | 0.015 | ||
| 50–70 | 4008 | 1.62 (1.4 to 1.86) | <0.0001 | 1.7 (1.4 to 2.08) | <0.0001 | |
| Sex | Male | 4008 | 1.5 (1.3 to 1.71) | <0.0001 | 1.26 (1.1 to 1.52) | <0.0001 |
| Index of multiple deprivation decile | 3848 | 0.97 (0.9 to 1) | 0.013 | |||
| Diabetes | 3971 | 1.46 (1.3 to 1.68) | <0.0001 | |||
| Hypertension | 3971 | 1.17 (1 to 1.33) | 0.007 | |||
| Ethnicity | 4008 | |||||
| White | 1348 | 1 | <0.0001 | 1 | <0.0001 | |
| Asian | 1318 | 1.94 (1.7 to 2.26) | 1.82 (1.5 to 2.27) | |||
| Black | 436 | 2.05 (1.6 to 2.56) | 1.85 (1.4 to 2.53) | |||
| Mixed/ Other | 258 | 2.13 (1.6 to 2.79) | 2.25 (1.5 to 3.33) | |||
| Unknown | 648 | 1.87 (1.5 to 2.27) | 1.77 (1.3 to 2.34) | |||
| Symptoms | 3971 | |||||
| Cough | 5.13 (4.5 to 5.88) | <0.0001 | ||||
| Shortness of breath | 4.19 (3.7 to 4.79) | <0.0001 | ||||
| Fever | 5.04 (4.4 to 5.78) | <0.0001 | ||||
| Respiratory rate | Any of above | 4008 | 6.29 (5.4 to 7.36) | <0.0001 | 3.11 (2.5 to 3.85) | <0.0001 |
| Oxygen saturations | 3654 | 1.14 (1.1 to 1.15) | <0.0001 | |||
| NEWS Score | Continuous (linear) | 3647 | 0.89 (0.9 to 0.9) | <0.0001 | ||
| Continuous (linear) | 3617 | 1.39 (1.3 to 1.42) | <0.0001 | |||
| CRP | >5 | 5.76 (5 to 6.65) | <0.0001 | 2.39 (2 to 2.87) | <0.0001 | |
| Every 10 increase | 3518 | 1.01 (1 to 1.01) | <0.0001 | |||
| Lymphocytes | >50 | 5.99 (5.2 to 6.93) | <0.0001 | 3.11 (2.6 to 3.75) | <0.0001 | |
| Continuous (linear) | 3624 | 0.66 (0.6 to 0.72) | <0.0001 | |||
| Chest X-ray | <1 | 2.54 (2.2 to 2.93) | <0.0001 | 1.72 (1.4 to 2.08) | <0.0001 | |
| 3581 | ||||||
| Normal | 718 | 1 | <0.0001 | 1 | <0.0001 | |
| Lung infiltrates | 2262 | 7.79 (6.3 to 9.65) | 3.75 (2.9 to 4.91) | |||
| Other abnormality | 601 | 3.56 (2.8 to 4.6) | 1.94 (1.4 to 2.68) | |||
| CVCX0 | 424 | 1 | <0.0001 | |||
| CVCX1 | 1040 | 25.85 (18.7 to 35.66) | ||||
| CVCX2 | 435 | 2.98 (2.3 to 3.93) | ||||
| CVCX3 | 129 | 1.64 (1.1 to 2.44) | ||||
P values calculated using likelihood ratio tests. There was no evidence of interaction between variables in the final multivariable model. n=2490 for multivariable model. CVCX represents British Society of Thoracic Imaging (BSTI) classification of chest X-ray.
CRP, C reactive protein; IMD, Index of multiple deprivation; NEWS, National Early Warning Score.
The final multivariable diagnostic model included age (modelled as a binary variable being between 50 and 70 years old), sex, ethnicity, reporting anyone of cough, fever or shortness of breath, NEWS 2, CRP and chest radiograph appearance (n=2940 table 3). Discrimination of the full model was moderate (AUROC curve 0.83, 95% CI 0.82 to 0.85), with good calibration (see figure 2). A simplified risk score was constructed based on β coefficients (table 3), with similar calibration and discrimination to the full model (AUROC curve 0.83, 95% CI 0.81 to 0.84). Internal validation using bootstrap samples (n=200) generated an optimism-corrected AUC 0.82 (95% CI 0.80 to 0.84), AUC for internal validated model 0.83 (95% CI 0.81 to 0.85). Decision curve analysis showed that the diagnostic risk score model had better clinical utility across a range of thresholds than treating all or no patients as having COVID-19, using a CRP of >50 or a NEWS score ≥5 (see figure 2). The model and risk score performed similarly in sensitivity analyses using multiple imputation instead of complete case analysis and assessing the risk score using the whole patient population (see online supplemental table 3).
Table 3.
Multivariable logistic regression diagnostic model for COVID-19, with regression (β) coefficients and diagnostic score points
| Variable | Coefficient | SE | Diagnsotic score points | |
| Age | 50–70 | 0.53 (0–0.41) | 0.09 | 1 |
| Sex | Male | 0.23 (0.3–0.73) | 0.10 | 1 |
| Ethnicity | Asian | 0.6 (0.4–0.82) | 0.11 | 1 |
| Black | 0.62 (0.3–0.93) | 0.16 | 1 | |
| Mixed/other | 0.81 (0.4–1.2) | 0.20 | 1 | |
| Unknown | 0.57 (0.3–0.85) | 0.14 | 1 | |
| Cough, fever or shortness of breath | 1.13 (0.9–1.35) | 0.11 | 2 | |
| NEWS2 Score | >5 | 0.87 (0.7–1.05) | 0.09 | 2 |
| CRP | >50 | 1.13 (1–1.32) | 0.09 | 2 |
| Lymphocytes | <1 | 0.54 (0.4–0.73) | 0.10 | 1 |
| Chest X-ray | Lung infiltrates | 1.32 (1.1–1.59) | 0.14 | 2 |
| Other abnormality | 0.66 (0.3–0.98) | 0.16 | 1 |
The constant (intercept) was −4.0 (95% CI −4.4 to −3.6). n=2940.
CRP, C reactive protein; NEWS2, National Early Warning Score 2.
Figure 2.
(A) ROC for the full diagnostic predictive model. AUC 0.839 (95%CI 0.824 to 0.853), n=2940. (B) Calibration plot showing observed compared with predicted risk of COVID-19 diagnosis as deciles, with 95% CI. The dashed green line shows perfect calibration. (C) Decision curve analysis showing standardised net benefit at different threshold probabilities for diagnosing patients with COVID-19, comparing diagnosing all patients as COVID-19 (blue solid line), diagnosing no patients with COVID-19 (solid red line), and various diagnostic risk models, including the COVID-19 diagnostic score (full model and simplified risk score), CRP over 50, and NEWS of 5 or more. AUC, area under the curve; CRP, C reactive protein; NEWS, National Early Warning Score; ROC, receiver operating characteristic curve.
bmjopen-2020-047110supp003.pdf (40.5KB, pdf)
The number and proportion of patients with or without COVID-19 diagnosis based on the risk score are shown in figure 3. Of 446 (15%) patients had a score of <4, of whom 10.9% (49/446) were diagnosed with COVID-19. Using this threshold to identify patients without COVID-19 had a 26.6% sensitivity, but 96.6% specificity, with an 89.0% PPV(table 4). A total of 594 (20.2%) patients were above the high-risk threshold, set at a diagnostic risk score >9. At high COVID-19 prevalence (50%), this threshold had a good PPV (>90%), and at a low prevalence (<5%), it had a high NPV; however, most patients fell in between both thresholds. Potential uses for such a clinical score are highlighted in online supplemental table 4.
Figure 3.
(A) Overlaid histogram of COVID-19 diagnostic risk score and number of patients with COVID-19 (white) and alternative (not COVID-19) diagnoses. (B) Proportion (%) of patients with COVID-19 (orange) or alternative (not COVID-19, blue) diagnoses by COVID-19 diagnostic risk score. N=2940.
Table 4.
Diagnostic performance of a low COVID-19 risk threshold (less than four points on the diagnostic score) and high-risk threshold (greater than nine points)
| Prevalence | ||||||
| Low-risk diagnostic score threshold (<4) | Study population | 0.5 | 0.2 | 0.1 | 0.05 | 0.01 |
| Sensitivity | 26.6% | – | – | – | – | – |
| Specificity | 96.6% | – | – | – | – | – |
| PPV | 89.0% | 88.7% | 66.2% | 46.6% | 29.2% | 7.3% |
| NPV | 56.0% | 56.8% | 84.0% | 92.2% | 96.2% | 99.2% |
| High-risk diagnostic score threshold (>9) | ||||||
| Sensitivity | 37.0% | – | – | – | – | – |
| Specificity | 96.1% | – | – | – | – | – |
| PPV | 90.1% | 90.4% | 70.1% | 51.0% | 33.0% | 8.6% |
| NPV | 61.2% | 60.4% | 85.9% | 93.2% | 96.7% | 99.3% |
Low-risk threshold diagnostic accuracy is for identifying patients without COVID-19, whereas high-risk threshold is for identifying patients with COVID-19.
NPV, negative predictive value; PPV, positive predictive value.
bmjopen-2020-047110supp004.pdf (82KB, pdf)
Discussion
The key findings of this study are that SARS-CoV-2 RT-PCR-negative COVID-19 is common among patients admitted to hospital, with real-life sensitivity of RT-PCR testing from NPS being 83% compared with a clinical reference standard of clinical diagnosis of COVID-19. Patients with RT-PCR-negative COVID-19 had similar clinical characteristics to RT-PCR-positive patients in this and other cohorts,17 although significantly better outcomes (lower risk of mortality and ICU admission).14 17 The proportion and number of COVID-19 admissions were increased during a 3-week period from the 22 March to 11 April 2020, and patients with COVID-19 were substantially more unwell than patients without COVID-19, with implications for service delivery. Mortality in patients admitted without COVID-19 was also high at 6.4%.
The current gold-standard diagnostic test for COVID-19, SARS-CoV-2 PCR from NPS, has several limitations, which are challenging health systems and healthcare facilities management. We demonstrate, despite high analytical sensitivity, the real-life sensitivity of PCR is inadequate (around 80% at best).18 Repeat testing of patients with an initial negative RT-PCR only increased yield by 3%–5% within 48 hours. In addition to slow turnaround times, and resource and logistical challenges, there is an urgent need for alternative rapid and accurate methods to triage and stratify patient’s risk of COVID-19, to allow appropriate infection control measures and safe patient flow to cohort areas or isolation rooms, without overwhelming hospital infrastructure. CT imaging of lungs can lack specificity for COVID-19, and rapid RT-PCR platforms are expensive and have inadequate throughput for future peaks of COVID-19.19 20 Few studies have assessed pragmatic tools to assess risk of COVID-19 based on readily available clinical or laboratory variables.21 22
We found several clinical, radiological and laboratory blood factors that were associated with COVID-19. Our diagnostic score had moderate performance for discriminating COVID-19 from other diagnoses (AUROC curve 0.83). A low-risk threshold had a good specificity and PPV, therefore, it could be used to identify patients with a low COVID-19 risk for transfer to a low-risk cohort area. Similarly, the high-risk score had a good PPV and specificity, therefore, these patients could be managed as having COVID-19 and cared for in isolation rooms or cohorts if necessary. Those patients in neither high-risk nor low-risk group may benefit from rapid COVID-19 RT-PCR or antigen testing, depending on capacity; however, this score would need external validation before use. Although derived from a cohort including unselected acute medical admissions, the higher prevalence of other respiratory viral pathogens may impact performance, especially specificity.23 Furthermore, this score does not account for the vulnerability of individual patients for severe COVID-19 (eg, based on age or comorbidities), which would also impact decisions on isolation and testing.23
This is the first study, to our knowledge, reporting lower ICU admissions and mortality in RT-PCR-negative patients with COVID-19, despite similar markers of disease severity at admission (NEWS, CRP, oxygen saturations and requirement for supplementary oxygen), and in multivariable adjusted model. Interestingly, the median duration of symptoms was slightly longer, and median lymphocyte count was slightly higher in PCR-negative patients, suggesting they presented slightly later in their disease course, and, therefore, may be at a phase of illness with lower viral burden in the upper respiratory tract.24–26 This may also be associated with their better prognosis. Other potential reasons for better outcomes in PCR-negative patients with COVID-19 include misclassification bias, where other respiratory conditions may have been classified as COVID-19. However, sensitivity analysis in patients with chest radiology suggestive of COVID-19 had similar findings, and a small number of misclassifications are unlikely to lead to such substantial differences in mortality. RT-PCR result may, therefore, be important in prognostic scores for COVID-19, especially as its association with mortality was independent of other key predictors such as age and sex. Patients with RT-PCR-negative COVID-19 should also be included in treatment trials, and the efficacy of treatment could be analysed separately given their different outcomes.
During the study period, the overall number of daily admissions did not increase substantially. However, the proportion of admissions that were related to COVID-19 increased substantially in late March and early April, with a fall in non-COVID-19 admissions, as previously documented.27 This has implications for planning for future COVID-19 peaks. Another important finding was the high mortality in patients without COVID-19, an over twofold increase from mortality in the previous year (2.4% compared with 6.4%).28 While we were unable to describe the causes of death among these patients, the increased mortality may result from late presentation to hospital due to national government-mandated ‘lockdown’ COVID-19 control measures and fear of nosocomial transmission risk. This has been previously documented in patients with paediatric, cardiology and oncology-related issues but not among acute medical admissions.28 29
This study has several strengths. The cohort is in a large acute hospital trust with two sites covering a diverse population, and all consecutive medical admissions were included. This is one of the first large cohorts to report data on unselected acute medical admissions, and one of the largest cohorts of patients with RT-PCR-negative results with COVID-19. There are also several limitations. The retrospective nature of the study has inherent limitations, including missing data. Although we included consecutive admitted patients, not all patients had SARS-CoV-2 testing, and two different RT-PCR assays were used, which may have slightly different primer targets and analytical sensitivities and may impact the generalisability. The decision to repeat tests on patients with negative RT-PCR results was made by the responsible clinical team. The absence of serology or other confirmatory testing introduces a risk of misclassification bias and RT-PCR inclusion in the reference standard, and the influence of variables including in the diagnostic risk score on clinical diagnosis of COVID-19 introduces incorporation bias. However, there remains no perfect reference standard for COVID-19 diagnosis and these biases are unlikely to significantly impact our findings. Our diagnostic risk model needs external validation, only has moderate discrimination, and is at risk of overfitting. Systematic reviews have struggled to identify other diagnostic clinical scores with high discrimination, and effective patient management is likely to involve a combination of clinical features, radiology and rapid PCR testing.11
In conclusion, we demonstrate that RT-PCR-negative COVID-19 is common among patients admitted to hospital and is associated with a better outcome despite similar severity at presentation. We derived and internally validated a diagnostic risk score with potential utility to help triage patients admitted from the emergency department, although prospective trials of different approaches are warranted in future peaks of COVID-19.
bmjopen-2020-047110supp002.pdf (38.3KB, pdf)
Supplementary Material
Acknowledgments
The authors would like to acknowledge all staff at London North West University Healthcare Trust who were involved in the care of patients during the study period, staff supporting the provision of clinical care, and all patients and their families.
Footnotes
Twitter: @ankurgw
AG-W and CKM contributed equally.
Contributors: AG-W, CKM, TC, VP, GS, RT, NV, SD, AW, AM, MH and PP made substantial contribution to the conception of the work. AG-W, CKM, AW, AM and PP made substantial contribution to the design of the work. AG-W, CKM, JB, SF, GS, JT, NG, HC, MH contributed to data acquisition. AG-W and CKM analysed the data. AG-W, CKM, AW, AM, PP contributed to data interpretation. AG-W and CKM drafted the manuscript. All authors contributed to revising the manuscript critically for important intellectual content, approved the final manuscript and are accountable for all aspects of the work.
Funding: The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests: None declared.
Patient and public involvement statement: Due to the retrospective nature of this study, undertaken during the COVID-19 pandemic, patients or the public were not involved in the design, or conduct, or reporting, or dissemination plans of our research.
Patient consent for publication: Not required.
Provenance and peer review: Not commissioned; externally peer reviewed.
Data availability statement: Data are available upon reasonable request, subject to approval by the London North West University Healthcare NHS Trust Research and Governance Department and approval from relevant ethics and regulatory bodies.
Supplemental material: This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.
References
- 1.Long C, Xu H, Shen Q, et al. . Diagnosis of the coronavirus disease (COVID-19): rRT-PCR or CT? Eur J Radiol 2020;126:108961. 10.1016/j.ejrad.2020.108961 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Arevalo-Rodriguez I, Buitrago-Garcia D, Simancas-Racines D, et al. . False-Negative results of initial RT-PCR assays for COVID-19: a systematic review. PLoS One 2020;15:e0242958. 10.1371/journal.pone.0242958 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Woloshin S, Patel N, Kesselheim AS. False negative tests for SARS-CoV-2 infection - challenges and implications. N Engl J Med 2020;383:e38. 10.1056/NEJMp2015897 [DOI] [PubMed] [Google Scholar]
- 4.Xie X, Zhong Z, Zhao W. Chest CT for typical 2019-nCoV pneumonia: relationship to negative RT-PCR testing. Radiology 2020;296:E41–5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Brogna B, Bignardi E, Brogna C, et al. . Typical CT findings of COVID-19 pneumonia in patients presenting with repetitive negative RT-PCR. Radiography 2020;52. 10.1016/j.radi.2020.09.012 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Watson J, Whiting PF, Brush JE. Interpreting a covid-19 test result. BMJ 2020;369:1–7. 10.1136/bmj.m1808 [DOI] [PubMed] [Google Scholar]
- 7.Jang S, Rhee J-Y, Wi YM, et al. . Viral kinetics of SARS-CoV-2 over the preclinical, clinical, and postclinical period. Int J Infect Dis 2021;102:561–5. 10.1016/j.ijid.2020.10.099 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Guan W-J, Ni Z-Y, Hu Y, et al. . Clinical characteristics of coronavirus disease 2019 in China. N Engl J Med 2020;382:1708–20. 10.1056/NEJMoa2002032 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Docherty AB, Harrison EM, Green CA, et al. . Features of 20 133 UK patients in hospital with covid-19 using the ISARIC WHO Clinical Characterisation Protocol: prospective observational cohort study. BMJ 2020;369:m1985. 10.1136/bmj.m1985 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Richardson S, Hirsch JS, Narasimhan M, et al. . Presenting characteristics, comorbidities, and outcomes among 5700 patients hospitalized with COVID-19 in the new York City area. JAMA 2020;323:2052–8. 10.1001/jama.2020.6775 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Wynants L, Van Calster B, Collins GS, et al. . Prediction models for diagnosis and prognosis of covid-19 infection: systematic review and critical appraisal. BMJ 2020;369:m1328. 10.1136/bmj.m1328 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.British Society of Thoracic Imaging Version 2 BSTI COVID-19 guidance for the reporting radiologist, 2020. Available: https://www.bsti.org.uk/standards-clinical-guidelines/clinical-guidelines/bsti-covid-19-guidance-for-the-reporting-radiologist/ [Accessed 30th June 2020].
- 13.Knight SR, Ho A, Pius R, et al. . Risk stratification of patients admitted to hospital with covid-19 using the ISARIC who clinical characterisation protocol: development and validation of the 4C mortality score. BMJ 2020;370:m3339–13. 10.1136/bmj.m3339 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Baik Y, Rickman HM, Hanrahan CF, et al. . A clinical score for identifying active tuberculosis while awaiting microbiological results: development and validation of a multivariable prediction model in sub-Saharan Africa. PLoS Med 2020;17:e1003420. 10.1371/journal.pmed.1003420 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Altman DG, Vergouwe Y, Royston P, et al. . Prognosis and prognostic research: validating a prognostic model. BMJ 2009;338:b605–5. 10.1136/bmj.b605 [DOI] [PubMed] [Google Scholar]
- 16.Collins GS, Reitsma JB, Altman DG, et al. . Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BMC Med 2015;13:1. 10.1186/s12916-014-0241-z [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.De Angelis G, Posteraro B, Biscetti F, et al. . Confirmed or unconfirmed cases of 2019 novel coronavirus pneumonia in Italian patients: a retrospective analysis of clinical features. BMC Infect Dis 2020;20:775. 10.1186/s12879-020-05504-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Tang Y-W, Schmitz JE, Persing DH, et al. . Laboratory diagnosis of COVID-19: current issues and challenges. J Clin Microbiol 10.1128/JCM.00512-20 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Gibani MM, Toumazou C, Sohbati M, et al. . Assessing a novel, lab-free, point-of-care test for SARS-CoV-2 (CovidNudge): a diagnostic accuracy study. Lancet Microbe 2020;1:e300–7. 10.1016/S2666-5247(20)30121-X [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Salameh J-P, Leeflang MM, Hooft L, et al. . Thoracic imaging tests for the diagnosis of COVID-19. Cochrane Database Syst Rev 2020;9:CD013639. 10.1002/14651858.CD013639.pub2 [DOI] [PubMed] [Google Scholar]
- 21.Wake RM, Morgan M, Choi J, et al. . Reducing nosocomial transmission of COVID-19: implementation of a COVID-19 triage system. Clin Med 2020;20:e141–5. 10.7861/clinmed.2020-0411 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Patterson B, Marks M, Martinez-Garcia G, et al. . A novel cohorting and isolation strategy for suspected COVID-19 cases during a pandemic. J Hosp Infect 2020;105:632–7. 10.1016/j.jhin.2020.05.035 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Pormohammad A, Ghorbani S, Khatami A, et al. . Comparison of influenza type A and B with COVID-19: a global systematic review and meta-analysis on clinical, laboratory and radiographic findings. Rev Med Virol 2020:e2179. 10.1002/rmv.2179 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Zheng S, Fan J, Yu F, et al. . Viral load dynamics and disease severity in patients infected with SARS-CoV-2 in Zhejiang Province, China, January-March 2020: retrospective cohort study. BMJ;15:m1443 10.1136/bmj.m1443 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.To KK-W, Tsang OT-Y, Leung W-S, et al. . Temporal profiles of viral load in posterior oropharyngeal saliva samples and serum antibody responses during infection by SARS-CoV-2: an observational cohort study. Lancet Infect Dis 2020;20:565–74. 10.1016/S1473-3099(20)30196-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Xiao AT, Tong YX, Gao C, et al. . Dynamic profile of RT-PCR findings from 301 COVID-19 patients in Wuhan, China: a descriptive study. J Clin Virol 2020;127:104346. 10.1016/j.jcv.2020.104346 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.NHS Digital Summary Hospital-level Mortality Indicator (SHMI) - Deaths associated with hospitalisation, England, March 2019 - February 2020, 2020. Available: https://digital.nhs.uk/data-and-information/publications/statistical/shmi/2020-07/shmi-march-2019-february-2020(accessedOct14,2020)
- 28.Lynn RM, Avis JL, Lenton S, et al. . Delayed access to care and late presentations in children during the COVID-19 pandemic: a snapshot survey of 4075 paediatricians in the UK and ireland. Arch Dis Child 2021;106:e8. 10.1136/archdischild-2020-319848 [DOI] [PubMed] [Google Scholar]
- 29.Pessoa-Amorim G, Camm CF, Gajendragadkar P, et al. . Admission of patients with STEMI since the outbreak of the COVID-19 pandemic: a survey by the European Society of cardiology. Eur Heart J Qual Care Clin Outcomes 2020;6:210–6. 10.1093/ehjqcco/qcaa046 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
bmjopen-2020-047110supp005.pdf (82KB, pdf)
bmjopen-2020-047110supp001.pdf (59.7KB, pdf)
bmjopen-2020-047110supp006.pdf (61.9KB, pdf)
bmjopen-2020-047110supp003.pdf (40.5KB, pdf)
bmjopen-2020-047110supp004.pdf (82KB, pdf)
bmjopen-2020-047110supp002.pdf (38.3KB, pdf)