Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2 or COVID-19) is frequently encountered in the emergency department (ED). Over a hundred COVID-19 clinical diagnosis and prediction rules have been published in the past two years, but many have incomplete results and are at high/uncertain risk for bias [1,2]. The reported C index estimates ranged between 0.71 and 0.99 for general population prediction models 0.65 to >0.99 in diagnostic models, and 0.54 to 0.99 for prognostic models [1]. Most diagnostic models use a combination of vital signs, comorbidities, age, and imaging [1]. The objective of this study was to assess ED clinician's pretest probability for COVID-19 against the institution's gold-standard SARS-CoV-2 RT-PCR as the reference.
After receiving Mayo Clinic institutional review board approval, ED attending physicians (N = 20) and credentialed advanced practice providers (N = 4) working in the Mayo Clinic ED in Jacksonville, FL, were asked to complete a REDCap survey of their pretest probability (0 (no chance) to 100 (certain)) that their patient had COVID-19 after treating the patient but before the COVID-19 test had resulted. Per institutional guidelines, all patients requiring hospital admission were tested for COVID-19, and all other ED patients had testing done at the provider's discretion. The study took place between June 9, 2020, and November 22, 2020, during which the ED tested 4839 patients, of whom 459 (9.5%) were positive, and 3 were indeterminant. Two hundred sixty-nine surveys (5.6% of tested patients) were completed, with only 245 patients (26 COVID-19 positives, 219 COVID-19 negatives) included in the analysis. Subjects were excluded for the following reasons: seven did not have a COVID-19 test result recorded at the index visit, four had an erroneous date of the ED encounter recorded, three had repeat visits to the ED during the study period (repeat encounters removed), two subject encounters were entered twice, two did not acknowledge informed provider consent, three because the medical record number (MRN) was incorrect or missing, and three had administrative errors. Information on the clinical encounter was collected through retrospective chart review by a study investigator and included demographic and triage data, relevant past medical history, documented COVID-19 test in previous 14 days, laboratory tests, radiology obtained in the ED, critical care consults, and the ED disposition.
Continuous variables were summarized with median and range and compared between COVID-19 negative and positive groups using a Wilcoxon rank sum test. Categorical variables were summarized with number and percentage of patients and were compared between COVID-19 negative and positive groups using Fisher's exact test. For the COVID-19 pretest probability, we also estimated the area under the ROC curve (AUC) along with a 95% confidence interval (CI) regarding ability to predict COVID-19 positivity. We estimated the magnitude of association between COVID-19 pretest probability and COVID-19 positivity by calculating an odds ratio (OR) and 95% CI from an unadjusted logistic regression model. P-values <0.05 were considered statistically significant. Statistical analysis was performed using R Statistical Software.
Baseline patient characteristics are in Table 1 . Compared to COVID-19 negative patients, COVID positive patients were more often non-white (36% vs. 15%, P = 0.010), had a higher temperature (Median: 37.2 vs. 36.8 degree Celsius, P = 0.002), had a higher hemoglobin (Median: 13.5 vs. 12.5, P = 0.045), had a lower WBC (Median: 5.8 vs. 7.8, P = 0.003), and more often had a chest x-ray (92% vs. 58%, P < 0.001). COVID-19 pretest probabilities between COVID negative and positive patients are shown in Table 1. Compared to COVID-19 negative patients, COVID-19 positive patients had a significantly higher COVID-19 pretest probability (Median: 72 vs. 10, P < 0.001, Fig. 1 ), and this corresponded to an AUC of 0.81 (95% CI: 0.72–0.91), indicating relatively good predictive ability. For each 10-unit increase in COVID-19 pretest probability, the odds of a positive COVID-19 test increased multiplicatively by 1.49 (95% CI: 1.30–1.74). Our results are similar to an ED study reporting clinical gestalt for COVID-19 with an AUC of 80.8%, which increased to 91.6% with medical imaging [3]. Another ED study found that clinical gestalt improved their COVID-19 predictive modeling [4].
Table 1.
Comparisons of patients positive and negative for COVID-19
| COVID-19 positive patients (N = 26) |
COVID-19 negative patients (N = 219) |
||||
|---|---|---|---|---|---|
| Variable | N | Median (minimum, maximum) or (%) of patients | N | Median (minimum, maximum) or No. (%) of patients | P-value |
| COVID-19 pretest probability (continuous) | 26 | 72 (5, 100) | 219 | 10 (0, 98) | <0.001 |
| COVID-19 pretest probability (categorical) | |||||
| 0–20 | 5 | (19.2%) | 145 | (66.2%) | |
| 21–40 | 1 | (3.8%) | 28 | (12.8%) | |
| 41–60 | 4 | (15.4%) | 11 | (5.0%) | |
| 61–80 | 5 | (19.2%) | 24 | (11.0%) | |
| 81–100 | 11 | (42.3%) | 11 | (5.0%) | |
| Sex (Male) | 26 | 15 (57.7%) | 219 | 121 (55.3%) | 0.84 |
| Age (years) | 26 | 60 (18, 86) | 219 | 67 (18, 97) | 0.18 |
| Ethnicity (not Hispanic or Latino) | 26 | 25 (96.2%) | 211 | 206 (97.6%) | 0.56 |
| Race (non-White) | 25 | 9 (36.0%) | 212 | 31 (14.6%) | 0.010 |
| Tobacco use | 26 | 8 (30.8%) | 215 | 106 (49.3%) | 0.18 |
| History of Asthma | 26 | 2 (7.7%) | 219 | 17 (7.8%) | 1.00 |
| History of CAD | 26 | 5 (19.2%) | 219 | 46 (21.0%) | 1.00 |
| History of CHF | 26 | 2 (7.7%) | 219 | 27 (12.3%) | 0.75 |
| History of COPD | 26 | 1 (3.8%) | 219 | 25 (11.4%) | 0.33 |
| History of DM | 26 | 7 (26.9%) | 219 | 41 (18.7%) | 0.31 |
| History of HTN | 26 | 16 (61.5%) | 219 | 121 (55.3%) | 0.68 |
| History of Dx Dialysis | 26 | 0 (0.0%) | 219 | 14 (6.4%) | 0.37 |
| Temperature (Celsius) | 26 | 37.2 (36.2, 37.8) | 217 | 36.8 (35.7, 39.6) | 0.002 |
| Heart rate | 26 | 83 (48, 113) | 219 | 77 (37, 138) | 0.19 |
| Systolic blood pressure | 26 | 124 (104, 163) | 219 | 128 (85, 191) | 0.42 |
| Diastolic blood pressure | 26 | 75.5 (62, 111) | 219 | 75 (40, 123) | 0.86 |
| Respiratory rate | 26 | 18 (12, 27) | 216 | 18 (12, 24) | 0.20 |
| Spo2 | 26 | 95 (92, 100) | 219 | 96 (90, 100) | 0.25 |
| Acuity | 26 | 218 | 0.13 | ||
| 1 | 0 (0.0%) | 5 (2.3%) | |||
| 2 | 3 (11.5%) | 69 (31.7%) | |||
| 3 | 23 (88.5%) | 143 (65.6%) | |||
| 4 | 0 (0.0%) | 1 (0.5%) | |||
| COVID test in previous 14 days | 26 | 0 (0.0%) | 219 | 25 (11.4%) | 0.086 |
| Hemoglobin | 21 | 13.5 (3, 16.9) | 210 | 12.5 (3.9, 18.3) | 0.045 |
| WBC | 21 | 5.8 (2, 13.7) | 210 | 7.8 (1.3, 86.4) | 0.003 |
| Platelets | 21 | 185 (64, 436) | 210 | 221 (12, 965) | 0.11 |
| Sodium | 21 | 135 (122, 142) | 209 | 137 (122, 145) | 0.074 |
| Potassium | 21 | 4.1 (3.3, 5.1) | 209 | 4.0 (2.2, 7.1) | 0.56 |
| Chloride | 21 | 97 (87, 111) | 209 | 100 (86, 112) | 0.27 |
| BUN | 21 | 17 (6, 102) | 209 | 17 (4, 122) | 0.71 |
| Creatinine | 21 | 0.86 (0.44, 4.67) | 209 | 0.97 (0.23, 15.98) | 0.39 |
| eGFR | 21 | 79 (15, 90) | 209 | 65 (0, 90) | 0.24 |
| Chest x-ray | 26 | 24 (92.3%) | 219 | 127 (58.0%) | <0.001 |
| CT-Chest/Angio/PE study | 26 | 2 (7.7%) | 219 | 24 (11.0%) | 1.00 |
| Critical care consult in ED | 26 | 2 (7.7%) | 218 | 13 (6.0%) | 0.67 |
| ED disposition | 26 | 218 | 0.076 | ||
| Admit | 10 (38.5%) | 124 (56.9%) | |||
| Admit-OR | 0 (0.0%) | 1 (0.5%) | |||
| AMA | 0 (0.0%) | 1 (0.5%) | |||
| Discharge | 14 (53.8%) | 59 (27.1%) | |||
| Observation | 2 (7.7%) | 33 (15.1%) | |||
P-values result from a Wilcoxon rank sum test (continuous and ordinal variables) or Fisher's exact test (categorical variables). COVID-19 probability assigned by physicians was able to discriminate between COVID-19 positive and negative patients with an AUC of 0.81 (95% CI: 0.72–0.91).
Fig. 1.
Boxplots of COVID pretest probability COVID-19 negative and positive patients.
Clinician gestalt, independent of specific formal scoring tools and specific ancillary testing algorithms, was good for predicating COVID-19 in ED patients. As the ED provider pretest probability for COVID-19 increased, so did the patient's likelihood to test positive. Our results are similar to other more conventional COVID-19 clinical prediction tools and close to ED provider accuracy for pneumonia (80%), group A beta-hemolytic streptococcal pharyngitis (73%), acute coronary syndrome (75%), acute heart failure (86%), pulmonary embolism (81%), and appendicitis (84%) [[5], [6], [7], [8], [9]]. Our study was limited by a small sample size, that data was collected from a single institution from a small percentage of eligible patients—suggesting selection bias, and had a limited number of participating clinicians. We did not discriminate between patients reinfected with COVID-19 or asymptomatic patients testing positive with persistent viral shedding from a recent infection.
Reprints
Johnathan M. Sheele, MD, MHS, MPH, Department of Emergency Medicine, Mayo Clinic, 4500 San Pablo Rd, Jacksonville, FL 32224 (Sheele.Johnathan@Mayo.edu).
Author contributions
J.M.S.: Study concept and design; acquisition, analysis, and interpretation of the data; drafting of the manuscript; critical revision of the manuscript for important intellectual content; study supervision.
S·F: Drafting of the manuscript.
A.V.L.: Analysis and interpretation of the data; critical revision of the manuscript for important intellectual content.
M.H.: Analysis and interpretation of the data; drafting of the manuscript; critical revision of the manuscript for important intellectual content; statistical expertise.
A.H.: Analysis and interpretation of the data; drafting of the manuscript; critical revision of the manuscript for important intellectual content; statistical expertise.
L.S.: Study concept and design; critical revision of the manuscript for important intellectual content.
Declaration of Competing Interest
All authors report no conflicts of interest.
External funding.
None.
References
- 1.Wynants L., Van Calster B., Collins G.S., Riley R.D., Heinze G., Schuit E., et al. Prediction models for diagnosis and prognosis of covid-19: systematic review and critical appraisal. BMJ. 2020 Apr;7:369. doi: 10.1136/bmj.m1328. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Shamsoddin E. Can medical practitioners rely on prediction models for COVID-19? A systematic review. Evid Based Dent. 2020 Sep;21(3):84–86. doi: 10.1038/s41432-020-0115-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Nazerian P., Morello F., Prota A., Betti L., Lupia E., Apruzzese L., et al. ED COVID-19 investigators. Diagnostic accuracy of physician’s gestalt in suspected COVID-19: prospective bicentric study. Acad Emerg Med. 2021 Apr;28(4):404–411. doi: 10.1111/acem.14232. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Peyrony O., Marbeuf-Gueye C., Truong V., Giroud M., Rivière C., Khenissi K., et al. Accuracy of emergency department clinical findings for diagnosis of coronavirus disease 2019. Ann Emerg Med. 2020 Oct 1;76(4):405–412. doi: 10.1016/j.annemergmed.2020.05.022. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Dale A.P., Marchello C., Ebell M.H. Clinical gestalt to diagnose pneumonia, sinusitis, and pharyngitis: a meta-analysis. British J Gen Pract. 2019 Jul 1;69(684):e444–e453. doi: 10.3399/bjgp19X704297. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Lee W.H., O’Brien S., Skarin D., Cheek J.A., Deitch J., Nataraja R., et al. Accuracy of clinician gestalt in diagnosing appendicitis in children presenting to the emergency department. Emerg Med Australas. 2019 Aug;31(4):612–618. doi: 10.1111/1742-6723.13220. [DOI] [PubMed] [Google Scholar]
- 7.Oliver G., Reynard C., Morris N., et al. Can emergency physician gestalt “rule in” or “rule out” acute coronary syndrome: validation in a multicenter prospective diagnostic cohort study. Acad Emerg Med. 2020;27:24–30. doi: 10.1111/acem.13836. [DOI] [PubMed] [Google Scholar]
- 8.Roncalli J., Picard F., Delarche N., et al. Predictive criteria for acute heart failure in emergency department patients with acute dyspnoea: the PREDICA study. Eur J Emerg Med. 2019;26:400–404. doi: 10.1097/MEJ.0000000000000622. [DOI] [PubMed] [Google Scholar]
- 9.Penaloza A., Verschuren F., Meyer G., et al. Comparison of the unstructured clinician gestalt, the Wells score, and the revised Geneva score to estimate pretest probability for suspected pulmonary embolism. Ann Emerg Med. 2013;62 doi: 10.1016/j.annemergmed.2012.11.002. 117–124.e2. [DOI] [PubMed] [Google Scholar]