Abstract
Background
Isolation of hospitalized persons under investigation (PUIs) for coronavirus disease 2019 (COVID-19) reduces nosocomial transmission risk. Efficient evaluation of PUIs is needed to preserve scarce healthcare resources. We describe the development, implementation, and outcomes of an inpatient diagnostic algorithm and clinical decision support system (CDSS) to evaluate PUIs.
Methods
We conducted a pre-post study of CORAL (COvid Risk cALculator), a CDSS that guides frontline clinicians through a risk-stratified COVID-19 diagnostic workup, removes transmission-based precautions when workup is complete and negative, and triages complex cases to infectious diseases (ID) physician review. Before CORAL, ID physicians reviewed all PUI records to guide workup and precautions. After CORAL, frontline clinicians evaluated PUIs directly using CORAL. We compared pre- and post-CORAL frequency of repeated severe acute respiratory syndrome coronavirus 2 nucleic acid amplification tests (NAATs), time from NAAT result to PUI status discontinuation, total duration of PUI status, and ID physician work hours, using linear and logistic regression, adjusted for COVID-19 incidence.
Results
Fewer PUIs underwent repeated testing after an initial negative NAAT after CORAL than before CORAL (54% vs 67%, respectively; adjusted odd ratio, 0.53 [95% confidence interval, .44–.63]; P < .01). CORAL significantly reduced average time to PUI status discontinuation (adjusted difference [standard error], −7.4 [0.8] hours per patient), total duration of PUI status (−19.5 [1.9] hours per patient), and average ID physician work-hours (−57.4 [2.0] hours per day) (all P < .01). No patients had a positive NAAT result within 7 days after discontinuation of precautions via CORAL.
Conclusions
CORAL is an efficient and effective CDSS to guide frontline clinicians through the diagnostic evaluation of PUIs and safe discontinuation of precautions.
Keywords: COVID-19 diagnosis, electronic health record, diagnostic algorithm, clinical decision support system
The COvid Risk cALculator diagnostic algorithm and clinical decision support system substantially reduced duration of transmission-based precautions for persons under investigation for coronavirus disease 2019, and the time infectious diseases physicians spent evaluating them in a large academic medical center.
Isolation of persons under investigation (PUIs) for coronavirus disease 2019 (COVID-19) in healthcare settings is essential to prevent nosocomial transmission of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). Once COVID-19 infection has been excluded, prompt discontinuation of transmission-based precautions is critical to preserve hospital bed capacity, conserve personal protective equipment (PPE), and reduce adverse effects associated with isolation [1–3].
SARS-CoV-2 nucleic acid amplification tests (NAATs) are the reference standard for COVID-19 diagnosis, but they have imperfect sensitivity dependent on specimen source, collection technique, and symptom duration [4]. The Infectious Diseases Society of America recommends a repeated NAAT 24–48 hours after the first negative test result among symptomatic patients when clinical suspicion for COVID-19 remains intermediate or high [4]. Several reports describe COVID-19 diagnosed after 2 negative nasopharyngeal (NP) NAAT results [5, 6]. However, repeating a NAAT extends the duration of isolation and may delay treatment, and standardized frameworks to categorize patients by clinical suspicion have not been developed.
Among hospitalized PUIs, the stakes for accurate diagnoses are high, given the potential for nosocomial transmission [7–9]. Many medical centers have turned to infectious diseases (ID) physicians, infection preventionists, and hospital epidemiologists to guide COVID-19 evaluations and make case-by-case determinations regarding discontinuation of transmission-based precautions [10–12]. These specialists are in limited supply [13–17] and may not be available for 24/7 evaluation [13]. Approaches to increase the efficiency and accessibility of COVID-19 diagnostic evaluation are needed.
Clinical decision support systems (CDSSs) improve adherence to guideline-based care [18–20]. SARS-CoV-2 testing criteria, diagnostics, and literature on evaluation of PUIs have evolved rapidly; CDSSs can standardize evaluations through adoption of best practices. We created the COvid Risk cALculator (CORAL) CDSS, using a modified Delphi method informed by >4500 person-hours of ID physician–led case-by-case evaluation of PUIs. We describe the development and implementation of CORAL and outcomes from its use in >2000 inpatient encounters.
METHODS
Study Setting and Design
We conducted a pre-post intervention study of COVID-19 PUIs hospitalized at Massachusetts General Hospital during March–July 2020. Throughout the study period, all NAATs were performed on emergency use authorized assays, including a laboratory-developed quantitative polymerase chain reaction test (turnaround time 8–36 hours; through 23 March), Roche COBAS SARS-CoV-2 (6–18 hours; used from 23 March onward), and the Cepheid Xpert Xpress SARS-CoV-2 (2–4 hours; used from 30 March onward). (See full details regarding test availability in the Supplementary Methods and Supplementary Figure 1.)
In early March, SARS-CoV-2 NAAT was limited to inpatients with symptoms consistent with COVID-19. Admission NAAT screening for all patients began on 23 April. Patients were considered PUIs if they reported COVID-19 symptoms, had epidemiologic risk factors (ie, living in a congregate setting, experiencing homelessness, or receiving hemodialysis treatment), or if a history could not reliably be obtained. PUIs were isolated with enhanced respiratory isolation (ERI; gown, gloves, N95 or powered air purifying respirator, and eye protection), admitted to single-occupancy rooms, and marked as “CoV-Risk” in the electronic health record (EHR; Supplementary Table 1). Discontinuation of CoV-Risk status indicated that the patient was no longer a PUI and allowed for discontinuation of ERI. If clinical suspicion of COVID-19 remained high despite negative NAAT results, PUIs were labeled “CoV-Presumed” and maintained with ERI.
Pre-CORAL: ID physician review of PUIs
From 18 March to 23 April, ID physicians actively evaluated all hospitalized PUIs who had an initial negative NAAT before discontinuation of ERI. These physicians iteratively reviewed a list of PUIs in the EHR and proactively contacted frontline providers with diagnostic recommendations. From 24 April to 19 May, ID physicians passively evaluated all PUIs; frontline clinicians conducted guideline-based COVID-19 workup (Supplementary Figure 2) and requested ID physician review once complete. A team of 6–12 ID physicians a day, each contributing 4–12 hours daily, reviewed the EHR and, if needed, spoke to care teams to examine clinical details regarding the PUI’s exposure history, epidemiologic risk factors, symptom types and duration, laboratory values, imaging studies, and response to treatment for alternative diagnoses. Based on this information, the ID physician recommended discontinuation of CoV-Risk status or additional work-up (eg, repeated SARS-CoV-2 NAAT from an NP or lower respiratory tract [LRT] specimen, serology, and chest computed tomography [CT]) (Supplementary Methods and Supplementary Table 2).
CORAL Components and Derivation
CORAL was developed as a diagnostic algorithm and clinical scoring system to incorporate symptoms, epidemiologic risk factors, and imaging findings (Supplementary Tables 3 and 4). It was designed to (1) guide licensed independent practitioners through the diagnostic workup of PUIs after an initial negative NAAT, (2) indicate when transmission-based precautions and isolation orders could be discontinued, and (3) triage more complex PUIs to ID physician review in the setting of clinical or radiographic features highly concerning for COVID-19 despite 2 negative NAAT results.
To be eligible for CORAL, patients must be adult (≥19-years-old) PUIs with ≥1 negative SARS-CoV-2 NAAT result and have chest imaging performed within the past 3 days. Classification of CT chest findings are based on Radiological Society of North America standardized categories, which were widely used at Massachusetts General Hospital during the study period [21]. There are 4 possible CORAL pathways, depending on the number of negative NAATs and available imaging: 1 NAAT plus chest radiography, 1 NAAT plus chest CT, ≥2 NAATs plus chest radiography, and ≥2 NAATs plus chest CT (Supplementary Figure 3). Scoring systems for these pathways were developed using a modified Delphi method (Supplementary Methods) [22, 23].
Technical Development of CORAL
CORAL, launched on 20 May, was developed using native EHR (Epic™ Inc, Verona, WI) functionality and can be used iteratively as the diagnostic evaluation progresses (Figure 1). Clinicians load CORAL into a SmartForm (ie, advanced note template) and then select answers, which generates a standardized note for the record (Supplementary Figure 4). Responses are scored once CORAL is completed; scores are not visible to the clinician. If the score is below a preset threshold, CORAL discontinues CoV-Risk status (Supplementary Figure 5). If the score is above the preset threshold, CORAL displays orders for recommended diagnostics. If additional workup cannot be obtained and clinical suspicion for COVID-19 remains high, CORAL prompts ID physician review.
Figure 1.
Overview of COvid Risk cALculator (CORAL) workflow. Persons under investigation for coronavirus disease 2019 (COVID-19) (PUIs) are eligible for CORAL if they have ≥1 nucleic acid amplification test (NAAT) with negative results and 1 imaging study performed. On completion of CORAL, PUIs are given a risk score which leads to either a prompt for discontinuation of PUI status with discontinuation of enhanced respiratory isolation (ERI) precautions, or a prompt for repeated nasopharyngeal NAAT, further diagnostics for COVID-19 workup, or infectious diseases (ID) physician review. (a If CORAL cannot be performed, clinicians are instructed to contact the ID physician for review.) See Study Setting and Design for explanation of how CoV-Risk and CoV-Presumed status was assigned. Abbreviations: CT, computed tomography; LRT, lower respiratory tract.
Pre- and Post-CORAL Outcomes
We evaluated pre-CORAL outcomes among hospitalized patients with new CoV-Risk status from 18 March to 19 May. Allowing for a 1-week transition period for a hospital-wide CORAL education campaign, we evaluated post-CORAL outcomes from 27 May to 28 July. During both periods, we examined the proportion of PUIs with an initial negative NAAT result who underwent repeated NAAT and who required additional diagnostics beyond 2 negative NAAT results. We evaluated the time from the first (if only a single test was performed) or second negative NAAT result to discontinuation of CoV-Risk status, total duration of CoV-Risk status during the hospitalization, and average ID physician person-hours dedicated to PUI evaluations. After CORAL, we described provider-documented patient characteristics from the first use of CORAL and CORAL recommendations.
After discontinuation of CoV-Risk status, repeated SARS-CoV-2 NAAT was recommended for patients with new or worsening symptoms concerning for COVID-19. We performed record review for patients with a positive NAAT result within 14 days after discontinuation of CoV-Risk in the same hospital encounter. This study was approved by the Mass General Brigham Institutional Review Board (no. 2012P002359).
Statistical Analysis
We conducted unadjusted and adjusted analyses for each outcome measure, comparing pre-CORAL with post-CORAL values. The pre-CORAL period included both the passive PUI evaluation and active PUI evaluation periods, which were analyzed together. We used χ 2 tests to compare unadjusted pre- and post-CORAL proportions of patients with repeated testing after an initial negative NAAT result and additional workup after 2 negative NAAT results. We used logistic regression to adjust these comparisons for daily COVID-19 incidence among hospitalized PUIs as a surrogate for pretest probability. COVID-19 incidence was defined as the proportion of daily new COVID-19 diagnoses among all newly admitted PUIs. For continuous outcomes, we used t tests to compare unadjusted mean time from NAAT result return to discontinuation of CoV-Risk status and total duration of CoV-Risk status among PUIs. Pre- and post-CORAL means for these continuous outcomes were compared using a generalized linear model, adjusted for daily COVID-19 incidence.
RESULTS
COVID-19 Incidence and Demographics of Inpatient PUIs
On average, the proportion of new COVID-19 diagnoses among all newly admitted PUIs was 29% from 18 March to 19 May (before CORAL) and 3% from 27 May to 28 July (after CORAL) as the initial COVID-19 surge subsided. At the height of the surge in early April, >100 PUIs per day required ID review. Median age (interquartile range) among PUIs was similar before and after CORAL (63 [50–75]) vs 64 [51–77] years, respectively), as was the proportion of female PUIs (44% vs 40%). We report the epidemiology, symptoms, imaging findings, and additional diagnostics recommended for post-CORAL PUIs (Table 1); these structured EHR data were not available before CORAL.
Table 1.
Characteristics of Patients Evaluated With the COvid Risk cALculator (CORAL)
| Outcome of 1st CORAL Use Within Encounter, No. (%) | |||||
|---|---|---|---|---|---|
| 1 Negative NAAT Result | ≥2 Negative NAAT Results | ||||
| Characteristic | Resolved (n = 1034) | Repeated NAAT (n = 546) | Resolved (n = 647) | Further Chest Imaging ± LRT Sampling (n = 49) | ID Physician Review ± LRT Sampling (n = 27) |
| Epidemiologya | |||||
| Cannot obtain history | 53 (5.1) | 108 (19.8) | 100 (15.5) | 12 (24.5) | 5 (18.5) |
| Close contact with confirmed COVID-19 in last 14 d | 5 (0.5) | 3 (0.5) | 2 (0.3) | 3 (6.1) | 1 (3.7) |
| Preadmission circumstances | |||||
| Congregate setting | 110 (10.6) | 69 (12.6) | 77 (11.9) | 8 (16.3) | 4 (14.8) |
| Experiencing homelessness | 48 (4.6) | 25 (4.6) | 36 (5.6) | 8 (16.3) | 1 (3.7) |
| Private home | 818 (79.1) | 341 (62.5) | 433 (66.9) | 18 (36.7) | 16 (59.3) |
| Receives hemodialysis | 35 (3.4) | 24 (4.4) | 15 (2.3) | 4 (8.2) | 1 (3.7) |
| Symptomsb | |||||
| Respiratory symptoms | |||||
| Shortness of breath | 158 (15.3) | 280 (51.3) | 143 (22.1) | 20 (40.8) | 16 (59.3) |
| Cough | 45 (4.4) | 132 (24.2) | 74 (11.4) | 19 (38.8) | 6 (22.2) |
| Viral symptoms | |||||
| Fever (objective or subjective) | 195 (18.9) | 96 (17.6) | 113 (17.5) | 13 (26.5) | 8 (29.6) |
| Chills | 45 (4.4) | 46 (8.4) | 29 (4.5) | 5 (10.2) | 2 (7.4) |
| Headache | 26 (2.5) | 17 (3.1) | 18 (2.8) | 0 (0) | 2 (7.4) |
| Sore throat | 15 (1.4) | 18 (3.3) | 23 (3.6) | 2 (4.1) | 1 (3.7) |
| Muscle aches | 21 (2.0) | 24 (4.4) | 19 (2.9) | 2 (4.1) | 3 (11.1) |
| Loss of taste (ageusia) | 2 (0.2) | 3 (0.5) | 4 (0.6) | 0 (0) | 0 (0) |
| Loss of smell (anosmia) | 3 (0.3) | 4 (0.7) | 0 (0) | 0 (0) | 0 (0) |
| Other symptoms | 15 (1.5) | 3 (0.5) | 6 (0.9) | 0 (0) | 1 (3.7) |
| Chest radiographic findings | |||||
| Clear lungs | 610 (59.0) | 99 (18.1) | 211 (32.6) | 10 (20.4) | 0 (0) |
| Focal consolidation, lobar collapse, likely atelectasis, or any changes that are stable to improved from prior imaging >2 wk earlier | 128 (12.4) | 141 (25.8) | 122 (18.9) | 6 (12.2) | 0 (0) |
| Any other abnormality | 52 (5.0) | 219 (40.1) | 112 (17.3) | 33 (67.3) | 0 (0) |
| Chest CT findingsc | |||||
| No findings suspicious for COVID-19 | 234 (22.6) | 1 (0.2) | 134 (20.7) | 0 (0) | 0 (0) |
| Atypical or indeterminate for COVID-19 | 10 (1.0) | 76 (13.9) | 68 (10.5) | 0 (0) | 7 (25.9) |
| Typical for COVID-19 | 0 (0) | 10 (1.8) | 0 (0) | 0 (0) | 20 (74.1) |
| Mitigating factors | |||||
| Alternative diagnosis | NA | NA | 429 (66.3) | 13 (26.5) | 7 (25.9) |
| Sputum NAAT negative | NA | NA | 88 (13.6) | 0 (0) | 2 (7.4) |
| Tracheal aspirate NAAT negative | NA | NA | 24 (3.7) | 0 (0) | 0 (0) |
Abbreviations: COVID-19, coronavirus disease 2019; CT, computed tomography; ID, infectious diseases; LRT, lower respiratory tract; NA, not applicable; NAAT, nucleic acid amplification test.
aQuestions cascaded based on previous answers; for example, if a patient had close contact, questions about living situation and dialysis were not asked (see Supplementary Material).
bPatients could report >1 symptom.
cCategories were based on Radiological Society of North America Reporting criteria [21].
Outcomes of PUIs
During the pre-CORAL period, 2443 of 3358 inpatient PUI encounters (73%) with ≥1 SARS-CoV-2 NAAT had a first negative NAAT result (Figure 2A). Of these, CoV-Risk status was discontinued without further testing in 764 of 2443 (31%), not discontinued owing to death or discharge in 39 of 2443 (1.6%), and converted to CoV-Presumed status in 4 of 2443 (0.2%). A second NAAT was performed in the remaining 1636 of 2443 encounters (67%) with a first negative NAAT result; 38 of 1636 (2.3%) repeated NAATs had newly positive results. Of the remaining 1598 encounters with 2 negative NAAT results, CoV-Risk status was discontinued without further workup in 1042 of 1598 (65%), and additional diagnostics were pursued in 485 of 1598 (30%). Additional testing resulted in positive NAATs results in 18 of 485 encounters (3.7%), conversion to CoV-Presumed status in 31 of 485 (6.4%), and discontinuation of CoV-Risk status in 383 of 485 (79%).
Figure 2.
Outcomes before and after implementation of the COvid Risk cALculator (CORAL). Testing and CoV-Risk status discontinuation outcomes are demonstrated for the pre-CORAL (A) and post-CORAL (B) periods. In the post-CORAL period, outcomes are shown for persons under investigation for coronavirus disease 2019 (COVID-19) (PUIs) for whom CORAL was initially used after 1 (green) or 2 (blue) negative nucleic acid amplification test (NAAT) results. See Study Setting and Design for explanation of how CoV-Risk and CoV-Presumed status was assigned. Abbreviation: ID, infectious diseases physician.
During the post-CORAL study period, the initial NAAT was negative in 2411 of 2485 inpatient encounters (97%) in which PUIs underwent SARS-CoV-2 NAATs (Figure 2B). CORAL was used at least once in 2303 of 2411 encounters (96%); its first use was after the first negative NAAT in 1580 of 2303 encounters (69%) and after a second negative NAAT in 723 of 2303 (31%). CoV-Risk status was discontinued with CORAL after 1 negative NAAT in 1034 of 2303 encounters (45%) and after ≥2 negative NAATs in 1198 of 2303 (52%). PUIs underwent additional workup in 224 of 1245 encounters (18%) in which CORAL was used after 2 negative NAATs; of these, CoV-Risk status was discontinued with CORAL in 190 of 224 encounters (85%), discontinued with ID physician review in 21 of 224 (9.4%), and converted to CoV-Presumed status in 4 of 224 (1.8%). In encounters with ≥1 repeated NAAT after an initial negative result, subsequent NAAT yielded a new COVID-19 diagnosis in 12 of 1310 encounters (0.9%).
In the unadjusted analysis, repeated NAAT after 1 initial negative result was less frequent after CORAL than before CORAL (54% vs 67%, respectively; odds ratio [OR], 0.59 [95% confidence interval [CI]: .52–.66]; P < .01) (Table 2) and persisted after adjustment for COVID-19 incidence (adjusted OR [aOR], 0.53 [.44–.63]; P < .01). Additional diagnostics for COVID-19 beyond 2 negative NAATs were also less frequent after CORAL than before CORAL (19% vs 30%, respectively) in both unadjusted (OR, 0.53 [95% CI: .45–.63]) and adjusted (aOR, 0.42 [.33–.54]) analyses (both P < .01). These declines in repeated testing and additional diagnostics beyond 2 NAATs were not evident during the transition from active to passive ID physician review (24 April through 19 May); repeated testing and additional diagnostics were actually more frequent during this period despite decreasing COVID-19 incidence, which then led to lower ORs after adjustment for COVID-19 incidence (Figure 3A).
Table 2.
COVID-19 Diagnostic Testing Utilitization Before and After Implementation of the COvid Risk cALculator
| PUIs, % | OR (95% CI) | ||||
|---|---|---|---|---|---|
| Outcome | Before CORAL | After CORAL | Unadjusted | Adjusteda | P Value |
| Underwent repeated testing after initial negative NAAT result and repeated testing | 67.0 | 54.3 | 0.59 (.52–.66) | 0.53 (.44–.63) | <.01 |
| Underwent additional diagnostic testing after 2 Negative NAAT results with additional diagnostic testing | 30.4 | 18.8 | 0.53 (.45–.63) | 0.42 (.33–.54) | <.01 |
Abbreviations: CI, confidence interval; CORAL, COvid Risk cALculator; NAAT, nucleic acid amplification test; OR, odds ratio; PUIs, persons under investigation for coronavirus disease 2019 (COVID-19).
aWe adjusted all analyses for daily COVID-19 incidence among hospitalized PUIs.
Figure 3.
Daily changes in key outcomes in the periods before and after implementation of the COvid Risk cALculator (CORAL) relative to coronavirus disease 2019 (COVID-19) incidence. The pre-CORAL period (left) is shown with the active evaluation window for persons under evaluation for COVID-19 (PUIs), from 18 March to 23 April 2020, and the passive PUI evaluation window from 24 April to 19 May (Methods). CORAL was launched on 20 May 2020, followed by a 1-week transition period, with the post-CORAL period spanning 27 May to 28 July 2020. The incidence of new COVID-19 diagnoses among hospitalized PUIs, shown as a 5-day moving average, is shown in black (left y-axis) in all panels. (Dates are given in month/date format.) A, Proportion of PUIs with an initial negative nucleic acid amplification test (NAAT) result who underwent repeated testing (solid red line; right y-axis) and the proportion of PUIs with 2 negative NAAT results who underwent additional testing (dashed red line; right y-axis). B, Mean total duration of PUI status (solid blue line; right y-axis) and the mean time from final negative NAAT result return to PUI status discontinuation (dashed blue line; right y-axis). C, Mean infectious diseases (ID) physician person-hours/day dedicated to PUI evaluations (solid green line; right y-axis). Of note, we excluded patients who became PUIs during the pre-CORAL or wash-in period, but who had resolution of CoV-Risk status with CORAL, owing to the duration of their PUI status; the greater fluctuation in results around the end of the pre-CORAL and wash-in period is likely because fewer patients were contributing to the data set in those weeks.
Duration of CoV-Risk Status
Among patients who had CoV-Risk status discontinued while in the hospital, the mean time in hours (standard deviation [SD]) from negative NAAT result return to discontinuation of CoV-Risk status was 5.3 (2.3) hours after CORAL, compared with 11.5 (3.6) hours before CORAL (unadjusted difference [standard error (SE)], −6.2 [0.5] hours; adjusted difference, −7.4 [0.8] hours; P < .01) (Table 3). The mean (SD) total duration of CoV-Risk status was 19.0 (5.9) hours after versus 36.1 (8.7) hours before CORAL (unadjusted difference [SE], −17.2 [1.3] hours; adjusted difference, −19.5 [1.9] hours; P < .01) (Table 3). In contrast, the mean time from negative NAAT result return to discontinuation of CoV-Risk status and the mean total duration of CoV-Risk status increased during the pre-CORAL period when transitioning from active to passive ID physician review and then declined in the post-CORAL period (Figure 3B).
Table 3.
PUI Status Duration and ID Physician Work Hours Before and After Implementation of the COvid Risk cALculator
| Mean (SD) | Absolute Difference (SE) | ||||
|---|---|---|---|---|---|
| Outcome | Before CORAL | After CORAL | Unadjusted | Adjusteda | P Value |
| Time from final NAAT result return to PUI status discontinuation, h | 11.5 (3.6) | 5.3 (2.3) | −6.2 (0.5) | −7.4 (0.8) | <.01 |
| Total duration of PUI status, hb | 36.1 (8.7) | 19.0 (5.9) | −17.2 (1.3) | −19.5 (1.9) | <.01 |
| ID physician work hours, person-hours/d | 68.7 (12.1) | 3.1 (2.0) | −65.6 (1.5) | −57.4 (2.0) | <.01 |
Abbreviations: CORAL, COvid Risk cALculator; ID, infectious diseases; NAAT, nucleic acid amplification test; PUI, person under investigation for coronavirus disease 2019 (COVID-19); SD, standard deviation; SE, standard error.
aWe adjusted all analyses for daily COVID-19 incidence among hospitalized PUIs. For total duration of PUI status, we were unable to additionally adjust for changes in test result return time owing to the introduction of additional in-house assays.
bPatients who did not have PUI status discontinued owing to death or discharge, or who were converted to CoV-Presumed status (defined in Study Setting and Design) or confirmed to have COVID-19 by subsequent NAAT were excluded from the 2 analyses regarding the duration of PUI status.
ID Physician Workload
Despite declining COVID-19 incidence in late April and early May, daily numbers of PUIs remained high, as did ID physician workload (Figure 3C). Implementation of CORAL resulted in an immediate and dramatic reduction in ID physician workload. Before CORAL, PUI evaluations required a mean (SD) of 68.7 (12.1) person-hours/d (Table 2), which decreased to a mean of 3.1 (2.0) person-hours/d after CORAL (adjusted difference [SE], −57.4 [2.0] person-hours/d; P < .01).
Diagnosis After Discontinuation of CoV-Risk Status
In 360 of 2189 pre-CORAL encounters (16%), patients underwent repeated NAAT within 14 days of ERI discontinuation owing to new symptoms or required testing for discharge settings; 4 of 360 (1.1%) had COVID-19 diagnosed, all within 7 days of ERI discontinuation. Among 2232 encounters after discontinuation of CoV-Risk status with CORAL, 433 patients (19%) had clinician-directed repeated testing within 14 days of ERI discontinuation. One patient of 433 (0.2%) had a positive NP NAAT result 9 days after initial negative NAAT result and ERI discontinuation by CORAL, as well as a positive total SARS-CoV-2 antibody test 11 days after admission. Details of the 5 patients are provided (Supplementary Table 5); no onward nosocomial transmission from these individuals to other patients was identified.
DISCUSSION
We describe the development and implementation of CORAL, a novel real-time diagnostic algorithm and CDSS to guide frontline clinicians through the evaluation of PUIs and assist with safe and efficient discontinuation of transmission-based precautions. As a structured diagnostic algorithm, CORAL decreased the use of repeated SARS-CoV-2 NAAT after a first negative NAAT result and the use of additional COVID-19 diagnostics beyond 2 negative NAAT results. CORAL reduced the duration of transmission-based precautions for PUIs compared with individualized ID physician review, supporting efforts to conserve PPE, preserve hospital capacity, and advance patient care.
Before CORAL implementation, individualized inpatient PUI evaluations were very resource-intensive [10]. Alternatives to case-by-case ID physician review such as indefinite continuation of ERI precautions, or algorithmic discontinuation of PUI status after 2 NAATs, were neither feasible nor an efficient use of laboratory or other resources. Sustaining individualized ID physician PUI evaluations required a considerable proportion of the ID workforce, at a time when demand for ID consultation was also high [24]. CORAL solved this critical problem, because it guided diagnostic evaluation and discontinuation of the PUI status 24 hours a day. CORAL was quickly adopted by clinicians, resulting in a significant decrease in ID physician person-power required and serving important roles in laboratory stewardship, PPE conservation, and improved bed allocation.
Overall COVID-19 incidence declined before CORAL implementation, as public health interventions were implemented statewide [25]. Declining incidence may have contributed to decreased need for repeated NAATs and additional testing beyond 2 negative NAAT results, as pretest probability of COVID-19 declined. However, the difference in these key outcomes before and after CORAL was sustained after adjustment for COVID-19 incidence. Furthermore, in the late pre-CORAL period, we also observed paradoxical increases in these outcomes, as well as time from final result return to discontinuation of CoV-Risk status and overall duration of CoV-Risk status, despite declining COVID-19 incidence. These increases corresponded to a shift from a system of active to passive PUI evaluation, which makes it more likely that the decreased testing use and duration of CoV-Risk status were directly attributable to the standardization of and 24/7 access to diagnostic guidance through the CORAL CDSS, rather than declining COVID-19 incidence.
Implementation of a CDSS has benefits beyond direct clinical guidance. The CDSS captures structured data entered by clinicians, which can be used to refine the scoring algorithm. Second, as frontline clinicians must answer structured questions to complete CORAL, it also serves as an educational tool to teach clinicians about the diagnostic workup of COVID-19 [4]. In addition, the platform is easily extendable and could be modified to follow local ID expert and stakeholder consensus, provide differential guidance based on population prevalence, or incorporate emerging laboratory testing modalities, such as SARS-CoV-2 serology, which may also help assess patient transmissibility [26, 27].
Several other COVID-19 diagnostic risk scores based on combinations of patient epidemiology, laboratory values, imaging findings, symptoms, and time from symptom onset have been validated [28–31]. For example, the Corona-score algorithm relies on age, sex, chest imaging, and laboratory values; it demonstrates 82%–96% sensitivity to predict COVID-19 infection among patients presenting to the emergency department [30, 32]. While other scoring systems have demonstrated reasonable sensitivity for COVID-19 infection compared with a single NAAT, CORAL remains unique as the only algorithm, to our knowledge, that has been incorporated into routine, prospective clinician workflow via an EHR CDSS. CORAL guides clinicians through additional diagnostic recommendations after a negative test result, which other published tools do not incorporate [28–32]. Finally, since CORAL is embedded into Epic, it could be adopted by other health systems in the United States using this EHR or incorporated into other EHRs [33]. Alternatively, clinicians can apply the scoring system independently (Supplementary Methods).
The current study had several limitations. COVID-19 incidence declined before CORAL was implemented; however, differences in outcomes before and after CORAL were sustained after adjustment for COVID-19 incidence to capture pretest probability. Delays in test turnaround time may have contributed to longer duration of CoV-Risk status in the early pre-CORAL period, but that duration was shorter post-CORAL compared with the late pre-CORAL period. Our evaluation of patients with COVID-19 diagnosed in the 14 days after discontinuation of CoV-Risk status is limited by clinician-directed testing of patients and may have been influenced by the lower COVID-19 incidence in the post-CORAL period. CORAL requires recent imaging and accurate history taking by frontline clinicians. Owing to changes in PUI definitions over time, patients in the pre-CORAL period were more likely to be at higher risk for COVID-19, given that testing was initially focused on symptomatic individuals; however, testing of all admitted patients regardless of symptoms started midway through the pre-CORAL period. Finally, this analysis was conducted at a single center with specific testing protocols and guidelines for discontinuation of transmission-based precautions. However, other facilities have similar protocols and guidelines [34, 35], and CORAL has since been implemented in 8 other affiliated facilities and used >19 000 times.
CORAL is an effective, efficient, and safe method to guide clinicians through the diagnostic steps of evaluating COVID-19 PUIs, enabling 24/7 discontinuation of transmission-based precautions in PUIs, and triaging complex patients for ID physician review. CORAL implementation markedly reduced ID physician-time for COVID-19 PUI evaluations and assisted in laboratory stewardship and PPE conservation. Healthcare facilities should consider implementing CDSS for PUI evaluations.
Supplementary Data
Supplementary materials are available at Clinical Infectious Diseases online. Consisting of data provided by the authors to benefit the reader, the posted materials are not copyedited and are the sole responsibility of the authors, so questions or comments should be addressed to the corresponding author.
Notes
Acknowledgments. We thank the many other infectious diseases (ID) physicians who volunteered their time on the Massachusetts General Hospital (MGH) Biothreats Results Review Team, as well as the MGH Infection Control Unit infection preventionists. We also thank Elizabeth M. Barks, MBA, for helping to coordinate the administrative elements of the Biothreats Results Review Team and for providing information about ID physician staffing during the spring 2020 coronavirus disease 2019 (COVID-19) pandemic. We would like to thank Rachel Mayer for her help collecting the dates of the policy and testing changes at MGH and Christopher Alba, BS, for help with proofreading and formatting the manuscript. We also acknowledge the important contributions that George A. Alba, MD, Joshua J. Baugh, MD, MPP, Nancy C. Haff, MD, MPH, Kathryn A. Hibbert, MD, Melissa Mattison, MD, Nino Mihatov, MD, Amber Moore, MD, MPH, Benjamin A. White, MD, and Brian Yun, MD, MBA, MPH, made to help implement the CORAL (COvid Risk cALculator) clinical decision support system, as well as the many frontline clinicians who cared for patients with confirmed or suspected COVID-19 during the spring 2020 surge.
Disclaimer. The findings and conclusions in this report are those of the authors. Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the funders.
Financial support. This work was supported by Massachusetts General Hospital Infection Control Unit departmental funds, the National Institute of Allergy and Infectious Diseases (grants R37AI058736-16S1 to A. L. C. and E. P. H., T32AI007061 to J. S. A., and K08AI14755 to J. L. R.), the Eunice Kennedy Shriver National Institute of Child Health and Human Development (grant K08HD101342 to C. M. D.), the Centers for Disease Control and Prevention (grant U01CK000490 to E. T. R.), Harvard Catalyst (J. E. Lazarus), the Cystic Fibrosis Foundation (J. S. A.), the Sullivan Family Foundation (L. C. I.), and the Roger I. and Ruth B. MacFarlane Foundation (E. S. S.).
Potential conflicts of interest. C. M. D. reports grants from IMPAACT Network and National Institutes of Health, consulting fees from UNAIDS, and travel reimbursement from Infectious Diseases Society of America, outside the submitted work. S. M. M. reports Gilead Sciences Research Scholars Program in HIV Award to her institution, outside the submitted work. N. B. was on the board of Allergan, until 8 May 2020. J. E. L. has received personal fees from Sherlock Biosciences for consultant work, outside the submitted work. R. P. W. reports a Steve and Deborah Gorlin MGH research scholar award. E. P. H. reports an honorarium from the American College of Cardiology. E. S. S. reports a license on a clinical decision support tool for β-lactam allergies from Persistent Systems and lecture fees from Vertex Pharmaceuticals, outside the submitted work.
All other authors report no potential conflicts. All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed.
References
- 1. Purssell E, Gould D, Chudleigh J. Impact of isolation on hospitalised patients who are infectious: systematic review with meta-analysis. BMJ Open 2020; 10:e030371. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Tran K, Bell C, Stall N, et al. . The effect of hospital isolation precautions on patient outcomes and cost of care: a multi-site, retrospective, propensity score-matched cohort study. J Gen Intern Med 2017; 32:262–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Morgan DJ, Diekema DJ, Sepkowitz K, Perencevich EN. Adverse outcomes associated with contact precautions: a review of the literature. Am J Infect Control 2009; 37:85–93. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Hanson KE, Caliendo AM, Arias CA, et al. . Infectious Diseases Society of America guidelines on the diagnosis of COVID-19. Infectious Diseases Society of America, 2020. Available at: https://www.idsociety.org/practice-guideline/covid-19-guideline-diagnostics/. Accessed 5 July 2020. [Google Scholar]
- 5. Lee TH, Junhao Lin R, Lin RTP, et al. ; National Centre for Infectious Diseases COVID-19 Outbreak Research Team . Testing for SARS-CoV-2: can we stop at 2? Clin Infect Dis 2020; 71:2246–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Dugdale CM, Anahtar MN, Chiosi JJ, et al. . Clinical, laboratory, and radiologic characteristics of patients with initial false-negative SARS-CoV-2 nucleic acid amplification test results. Open Forum Infect Dis 2020; 8:ofaa559. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Biernat MM, Zińczuk A, Biernat P, et al. . Nosocomial outbreak of SARS-CoV-2 infection in a haematological unit–high mortality rate in infected patients with haematologic malignancies. J Clin Virol 2020; 130:104574. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Jewkes SV, Zhang Y, Nicholl DJ. Nosocomial spread of COVID-19: lessons learned from an audit on a stroke/neurology ward in a UK district general hospital. Clin Med (Lond) 2020; 20:e173–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Rickman HM, Rampling T, Shaw K, et al. . Nosocomial transmission of COVID-19: a retrospective study of 66 hospital-acquired cases in a London teaching hospital. Clin Infect Dis 2021; 72:690–3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Dugdale CM, Turbett SE, McCluskey SM, et al. . Outcomes from an infectious disease physician-guided evaluation of hospitalized persons under investigation for coronavirus disease 2019 (COVID-19) at a large US academic medical center. Infect Control Hosp Epidemiol 2020; 1–4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Rearigh LM, Hewlett AL, Fey PD, et al. . Utility of repeat testing for COVID-19: laboratory stewardship when the stakes are high. Infect Control Hosp Epidemiol 2020; 1–3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Doll ME, Pryor R, Mackey D, et al. . Utility of retesting for diagnosis of SARS-CoV-2/COVID-19 in hospitalized patients: impact of the interval between tests. Infect Control Hosp Epidemiol 2020; 41:859–61. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Walensky RP, McQuillen DP, Shahbazi S, Goodson JD. Where is the ID in COVID-19? Ann Intern Med 2020; 173:587–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Walensky RP, Del Rio C, Armstrong WS. Charting the future of infectious disease: anticipating and addressing the supply and demand mismatch. Clin Infect Dis 2017; 64:1299–301. [DOI] [PubMed] [Google Scholar]
- 15. Pogorzelska-Maziarz M, Gilmartin H, Reese S. Infection prevention staffing and resources in U.S. acute care hospitals: results from the APIC megasurvey. Am J Infect Control 2018; 46:852–7. [DOI] [PubMed] [Google Scholar]
- 16. Bryant KA, Harris AD, Gould CV, et al. . Necessary infrastructure of infection prevention and healthcare epidemiology programs: a review. Infect Control Hosp Epidemiol 2016; 37:371–80. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Bartles R, Dickson A, Babade O. A systematic approach to quantifying infection prevention staffing and coverage needs. Am J Infect Control 2018; 46:487–91. [DOI] [PubMed] [Google Scholar]
- 18. Kawamoto K, Houlihan CA, Balas EA, Lobach DF. Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ 2005; 330:765. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Bernasconi A, Crabbé F, Raab M, Rossi R. Can the use of digital algorithms improve quality care? an example from Afghanistan. PLoS One 2018; 13:e0207233. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. Garg AX, Adhikari NK, McDonald H, et al. . Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. JAMA 2005; 293:1223–38. [DOI] [PubMed] [Google Scholar]
- 21. Simpson S, Kay FU, Abbara S, et al. . Radiological Society of North America expert consensus statement on reporting chest CT findings related to COVID-19: endorsed by the Society of Thoracic Radiology, the American College of Radiology, and RSNA—secondary publication. J Thorac Imaging 2020; 35:219–27. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Dreesens D, Kremer L, Burgers J, van der Weijden T. Lost in definitions: reducing duplication and clarifying definitions of knowledge and decision support tools. A RAND-modified Delphi consensus study. Health Policy 2020; 124:531–9. [DOI] [PubMed] [Google Scholar]
- 23. Jandhyala R. Delphi, non-RAND modified Delphi, RAND/UCLA appropriateness method and a novel group awareness and consensus methodology for consensus measurement: a systematic literature review. Curr Med Res Opin 2020; 36:1873–87. [DOI] [PubMed] [Google Scholar]
- 24. Nguyen CT, Olson G, Pho MT, et al. . Automatic ID consultation for inpatients with COVID-19: point, counterpoint, and a single-center experience. Open Forum Infect Dis 2020; 7:ofaa318. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25. Massachusetts Department of Health. COVID-19 information for local boards of health. 2020. Available at: https://www.mass.gov/covid-19-information-for-local-boards-of-health. Accessed 31 October 2020.
- 26. Theel ES, Slev P, Wheeler S, Couturier MR, Wong SJ, Kadkhoda K. The role of antibody testing for SARS-CoV-2: is there one? J Clin Microbiol 2020; 58:e00797-20. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27. Espejo AP, Akgun Y, Al Mana AF, et al. . Review of current advances in serologic testing for COVID-19. Am J Clin Pathol 2020; 154:293–304. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28. Borghetti A, Ciccullo A, Paratore M, et al. . Derivation and validation of a scoring system to assess pre-test probability of being COVID-19 positive. J Infect 2021; 82:159–98. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29. Huang D, Wang T, Chen Z, Yang H, Yao R, Liang Z. A novel risk score to predict diagnosis with coronavirus disease 2019 (COVID-19) in suspected patients: a retrospective, multicenter, and observational study. J Med Virol 2020; 92:2709–17. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30. Kurstjens S, van der Horst A, Herpers R, et al. . Rapid identification of SARS-CoV-2-infected patients at the emergency department using routine testing. Clin Chem Lab Med 2020; 58:1587–93. [DOI] [PubMed] [Google Scholar]
- 31. Qin L, Yang Y, Cao Q, et al. . A predictive model and scoring system combining clinical and CT characteristics for the diagnosis of COVID-19. Eur Radiol 2020; 30:6797–807. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32. Lippi G, Henry BM, Hoehn J, Benoit S, Benoit J. Validation of the Corona-score for rapid identification of SARS-CoV-2 infections in patients seeking emergency department care in the United States. Clin Chem Lab Med 2020; 58:e311–3. [DOI] [PubMed] [Google Scholar]
- 33. Epic UserWeb. 2020. Available at: https://userweb.epic.com. Accessed 31 October 2020.
- 34. Patterson B, Marks M, Martinez-Garcia G, et al. . A novel cohorting and isolation strategy for suspected COVID-19 cases during a pandemic. J Hosp Infect 2020; 105:632–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35. Ridgway JP, Pisano J, Landon E, Beavis KG, Robicsek A. Clinical sensitivity of severe acute respiratory syndrome coronavirus 2 nucleic acid amplification tests for diagnosing coronavirus disease 2019. Open Forum Infect Dis 2020; 7:ofaa315. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.