Modified PRIEST score for identification of very low-risk COVID patients

Abstract

Background

COVID-19 transmission remains high around the world, and severe local outbreaks continue to occur. Prognostic tools may be useful in crisis conditions as risk stratification can help determine resource allocation. One published tool, the Pandemic Respiratory Infection Emergency System Triage Severity Score, seems particularly promising because of its predictive ability and ease of application at the bedside. We sought to understand the performance of a modified version of this score (mPRIEST) in our institution for identifying patients with a greater than minimal risk for adverse outcome (death or organ support) at 30 days after index visit.

Methods

Consecutive visits at two northern Manhattan EDs with a new diagnosis of symptomatic COVID-19 were identified between November and December of 2020. Demographic variables and clinical characteristics were obtained from chart review. Outcomes were obtained from chart review and follow-up phone call.

Results

Outcomes were available on 306 patients. The incidence of death or mechanical ventilation at 30 days for patients in patients with mPRIEST above the threshold value was 43/181 (23.8%), and for patients below 1/125 (0.8%). The sensitivity of the score for adverse outcome was 97.7% (95% CI: 93.3% to 100%).

Conclusions

This data suggests the mPRIEST score, which can be calculated from clinical variables alone, has potential for use in EDs to identify patients at very low risk for adverse outcomes within 30 days of COVID diagnosis. This should be confirmed in larger formal validation studies in diverse settings.

1. Introduction

Emergency Departments (EDs) in New York City operated under de facto crisis standards of care during the first surge of COVID-19 infection in the spring of 2020. [1] Our own ED deployed a crisis clinical pathway that attempted to stratify care of patients with COVID-19 by their risk of deterioration based on the best available evidence and simple clinical criteria in order to more effectively allocate resources. [2]

Since that time, numerous prediction models for COVID-19 have been published. [3] We desired to improve our approach to prognostic stratification to use as part of a crisis pathway to use if we again reached disaster conditions during a second wave. We found one tool, the Pandemic Respiratory Infection Emergency System Triage (PRIEST) Severity Score, which seemed promising in its predictive power and applicability at the bedside. This score, which consists of the NEWS2 score with the addition of age, sex, and a measure of activity level, can be calculated without laboratory tests or diagnostic imaging. The authors reported that PRIEST scores >4 predicted 30 day mortality or need for organ support with 98% sensitivity. [4]

We undertook a project to validate the performance of this for identifying patients with a greater than minimal risk for adverse outcome (death or organ support) at 30 days after index visit.

2. Methods

Consecutive patients with a new diagnosis of symptomatic COVID-19 were collected from two northern Manhattan EDs from November 21st to December 15th, 2020. These EDs, part of the same academic medical center and hospital system, serve a largely urban Hispanic population and see 100,000 and 45,000 visits per year. The larger ED sees adult patients and is staffed with residents; the smaller ED sees both adult and pediatric patients; the same attending physicians cover both. Potential subjects were identified by a report in the electronic medical record that identified all ED patients who had a positive nucleic acid amplification test for SARS-CoV2 collected on that visit with no history of prior positive SARS-CoV2 test result in the system. Patients were included if they were adults (age 18 years or older) who had at least one documented symptom compatible with active infection, such as fever, cough, dyspnea, sore throat, malaise, vomiting, diarrhea, weakness, or fatigue. Asymptomatic patients (such as those tested after an exposure or as standard screening for hospital admission for other reasons) and patients who presented in cardiopulmonary arrest were excluded. This was initially conceived and conducted as a quality and operations project and no formal sample size estimation was performed a priori. We requested and received institutional-review board approval to analyze and report our data as research.

Baseline data was obtained by chart review. Abstraction was performed by the principal investigator and the two co-investigators, both of whom received over an hour of training; abstractors were not blinded to the objectives. All abstractors had significant previous experience with chart review for research. Variables and outcomes were defined in a data dictionary and a written protocol for both chart abstraction and structured follow-up phone calls was produced in advance. Data was abstracted into a structured worksheet on a secure cloud-based platform. Each investigator received one set of charts for review and there was no performance monitoring, but testing of 10% of the charts for inter-rater reliability between the co-investigators and the principal investigator revealed complete agreement. This concordance is not surprising as the majority of the variables were taken from structured fields in the medical record.

The component variables of the original PRIEST Severity Score are given in Fig. 1 . As one variable in the original score, “performance status”, could not be reliably obtained from chart review, this was excluded in advance to create a modified PRIEST score (mPRIEST). There were no missing clinical variables otherwise in the data set.

Fig. 1.

PRIEST score components.

The outcomes of interest were death or mechanical ventilation within 30 days of diagnosis or documented clinical resolution of acute illness after 10 days of diagnosis. Outcomes were largely determined from chart review, as both outpatient visit notes and post-discharge clinical follow-up program notes were often available. When a definitive outcome could not be ascertained from the EMR, follow-up phone calls were attempted using the phone number listed in the chart. Charts were re-reviewed for patients who had non-functional numbers or were otherwise unreachable; patients were considered lost to follow-up if no further visits were noted.

Statistical analysis was conducted using R, version 3.6.0 (R Core Team, Vienna, Austria). Confidence intervals were calculated with a publically available package, ‘pROC’, using DeLong's method for AUC comparison and stratified bootstrap at 2000 replicates for estimation at the pre-determined mPRIEST threshold. [5]

3. Results

A total of 392 visits were identified and screened. 27 patients were excluded - 4 patients who presented in cardiopulmonary arrest, 22 patients who had no active COVID symptoms, and 1 child; this left 365 subjects who met inclusion and exclusion criteria. Baseline characteristics for all patients are available in Table 1 . Primary outcomes could be ascertained for 306 patients, 200 patients who were admitted and 106 patients who were discharged. The discriminatory performance of mPRIEST was excellent, with an AUC of 0.86 (95% CI: 0.81 to 0.91). The rate of adverse outcome in patients with mPRIEST >4 was 23.8% (43/181), with 36 deaths and 7 patients on mechanical ventilation but still alive at 30 days. For mPRIEST ≤ 4, the rate of adverse outcome was 0.8% (1/125) with no deaths at 30 days. Fig. 2 : The sensitivity of mPRIEST >4 for 30-day mortality or mechanical ventilation was 97.7% (CI: 93.2% to 100%). Given the specificity of 47.2% (95% CI: 41.1% to 53.2%) at that threshold, the likelihood ratios positive and negative are 1.86 (1.64 to 2.10) and 0.05 (0.01 to 0.33), respectively.

Table 1.

Baseline clinical and demographic characteristics.

	Admitted	Discharged	Total
N	200	164	365a
Age, mean (IQR)	67.4 (58.5, 79.5)	50.7 (35, 62)	59.8 (47, 75)
Sex	105 Male (52.5%)	65 Male (39.6%)	171 Male (46.8%)
Ethnicityb	53.5% Hispanic, 29.5% White, 15.5% Black or African American, 16% Declined	54.2% Hispanic, 18.3% White, 16.5% Black or African American, 14.6% Declined	54% Hispanic, 24.4% White, 15.9% Black or African American, 15.3% Declined
Temperature, mean (IQR)	37.4C (36.8, 37.9)	37.2C (36.9, 37.4)	37.4C (36.8, 37.6)
Heart rate, mean (IQR)	97 bpm (83, 109)	93 bpm (85, 101)	96 bpm (85, 106)
Systolic blood pressure, mean (IQR)	129 mmHg (114, 145)	131 mmHg (115, 145)	130 mmHg (114, 145)
Respiratory rate, mean (IQR)	20 (18, 22)	18 (18, 18)	19 (18, 20)
Oxygen saturation, mean (IQR)	91% (90, 96)	97% (96, 99)	94% (93, 98)
Supplemental oxygen use, n (%)	121 (60.5%)	2 (1.2%)	123 (33.7%)
Abnormal mental status, n (%)	26 (13%)	0 (0%)	26 (7.1%)
mPRIEST score, median (IQR)	8 (5, 11)	3 (2, 4)	5 (3, 9)

^aOne patient left without completing evaluation.

^bNon-exclusive designations for race & ethnicity. The predominant race indicated was “Other Combination Not Described” at 43.5% of all patients; the vast majority of those who indicated “Other” chose “Hispanic or Latino or Spanish Origin” as ethnicity. There were only 3 total patients who indicated their race as Asian, Native Hawaiian, or other Pacific Islander.

Fig. 2.

Adverse outcome by mPRIEST score and diagram.

4. Discussion

Efficient prognostic systems are always of great interest to EDs but more so in the setting of a COVID surge. An instrument that can quickly identify patients who are unlikely to progress to severe adverse outcome would be very useful in a crisis situation.

Many prognostic instruments have been proposed in the past year. Several are promising; and a few draw from tens of thousands of patient encounters and show significant ability to predict clinical deterioration. [6] Many models were derived only from patients who clinicians had already decided to hospitalize, and often employ laboratory measurements that are unlikely to be collected on every patient with COVID-like illness. However, the PRIEST score was developed out of an ED populations and has wide adaptability, as it does not rely on diagnostic tests or special software. Although the NEWS2 score, from which Goodacre et al. derived their tool, also shows fair correlation with outcomes, the additional elements of the PRIEST score improve performance. [7]

We found that a mPRIEST score at ≤ 4 had similar sensitivity in our population as Goodacre et al. reported for PRIEST ≤4 in their derivation and validation cohorts, 97.7% versus 98%. We found a higher specificity, 47.2% compared to 34%. Significantly, 27% of patients in the PRIEST cohort fell under the threshold; but over 40% of patients in our study did.

In normal conditions, the direct applicability of this information is limited, as the 30-day timeframe is longer than the usual acute course of illness that is the pressing concern for most ED clinicians, and because the particular outcomes measured do not take into account the potential benefit of admission on outcomes through therapeutics interventions or more timely identification of deterioration. However, in a situation where crisis standards of care are being considered, we believe being able to estimate such a low rate of moderate-term mortality for a significant portion of potential patients would be of immense utility in resource allocation decisions.

This represents to our knowledge the first description of attempted external validation of the performance of this score, and the first description of its potential use outside of the United Kingdom where it was originally derived. Strengths of this data include the inclusion of consecutive visits and following outcomes of both admitted and discharged patients in an American, urban, largely Hispanic patient population. The unknown effect of emerging variants and widespread vaccination, the necessary modification made by removal of performance status due to the study design, as well as the relatively small sample size and substantial loss to follow-up represent the major limitations of our study. Our findings should be confirmed in formal validation studies in a variety of ED settings among diverse patient populations. The performance of the tool in the setting of coronavirus variants of concern should also be evaluated.

Presentations

None.

Sources of support

None.

Declaration of Competing Interest

None.