Pulmonary Embolism Risk Stratification: Pulse Oximetry and Pulmonary Embolism Severity Index

Abstract

Background

Risk stratification of pulmonary embolism (PE) patients is important to determine appropriate management.

Objectives

We evaluated two published risk-stratification tools in emergency department (ED) PE patients: a pulse oximetry cutoff below 92.5% oxygen (at 5280 feet elevation) and the Pulmonary Embolism Severity Index (PESI).

Methods

Electronic medical records of all patients diagnosed with PE were abstracted to identify their triage vital signs, co-morbidities, and adverse short-term outcomes (AO) either requiring interventions (defined as respiratory failure, hypotension requiring pressors, and hemodynamic impairment requiring thrombolytics) or resulting in death. We applied these models to our ED PE patients and assessed their performance.

Results

There were 168 PE patients identified, with an overall AO rate of 7.1% (12/168), including a 3.0% mortality rate. A room-air pulse oximetry cutoff of 92.5% for values measured at 5280 feet classified 89/136 patients as low risk, 1.1% of which had an AO, and 47/136 patients as high risk, of which 10.6% had AO. This pulse oximetry cutoff had a sensitivity of 83% (95% confidence interval [CI] 36–99%), specificity of 68% (95% CI 58–76%), and a negative predictive value (NPV) of 99% (95% CI 93–100%). PESI classified 91/168 patients as low risk (class I or II): 2.2% had AO but none died, and 77/168 were classified as high risk (class III, IV, or V), with an AO rate of 13.0%. A PESI cutoff score of II had a sensitivity of 83% (95% CI 52–98%), specificity of 57% (95% CI 49–65%), and NPV of 98% (95% CI 92–100%).

Conclusion

Both PESI and pulse oximetry measurements are moderately accurate identifiers of low-risk patients with PE.

Introduction

Evidence suggests that some patients with pulmonary embolism (PE) may be candidates for treatment in a less intensive, non-monitored setting using low molecular-weight heparin (1,2). However, non-monitored therapy is not often used because there is no well-accepted method that allows early identification of patients at low risk for short-term adverse outcomes. Several risk-stratification tools have recently been proposed that involve biomarkers, echocardiography, electrocardiography, or other computed tomography (CT) technology (3–8). However, these tools are not practical in all emergency department (ED) settings (9).

There are three clinically based prognostic models for PE patients. In a multicenter study in the United States, Kline et al. demonstrated that a pulse oximetry cutoff of 94.5% room air oxygen saturation at sea level can effectively differentiate patients with PE into high-risk (< 95% saturation) and low-risk (≥ 95% saturation) groups (10). Aujesky et al. derived and validated the Pulmonary Embolism Severity Index (PESI), which assesses 11 factors (age, gender, cancer, heart failure, chronic obstructive pulmonary disease [COPD], pulse rate ≥ 110 beats/min, systolic blood pressure < 100 mm Hg, respiratory rate ≥ 30 beats/min, temperature < 36°C, altered mental status, and pulse oximetry < 90%) to stratify PE patients into five risk classes for 30-day mortality (11,12). A third prognostic model, the Geneva Risk Score developed by Wicki et al., requires arterial blood gas and lower extremity ultrasound data along with clinical information to compute a score (13).

Both the PESI and pulse oximetry stratification models have been validated as valuable predictors of short-term mortality (10,12,14). Our inquiry expanded this evaluation beyond predicting mortality alone and assessed the ability of these clinically based prognostic models to identify PE patients at low risk for any adverse outcome requiring inpatient management. Furthermore, we assessed the performance of these models at our altitude of > 5000 feet above sea level.

Methods

Study Design

This was a single-center retrospective study of patients admitted from the ED with a diagnosis of PE.

Study Setting

Patients enrolled in this study were evaluated in an ED at approximately 5000 feet above sea level. It is an urban academic center staffed by board-certified emergency physicians 24 h a day and has an established nationally accredited residency in emergency medicine. This study was approved by the local Institutional Review Board and informed consent was waived.

Study Protocol

Two trained investigators searched the admitting diagnosis field of the ED electronic medical database (Picis PulseCheck, Wakefield, MA) for the diagnoses containing “pulm,” “PE,” “DVT,” and “deep vein thrombosis.” The ED records of all cases identified from this search were reviewed to identify suspected PE or DVT (deep vein thrombosis) patients from June 2004 to June 2008. The researchers then abstracted the following data from patients’ electronic inpatient medical records:

• First documented vital signs in the ED (heart rate, systolic blood pressure, respiratory rate, temperature, and pulse oximetry [either room air or if on supplemental oxygen])

• The patient’s mental status at presentation

• Interventions performed during the patient’s hospital stay (heparin, echocardiogram, inferior vena cava filter placement)

• Comorbidities and alternative causes of hypoxia (asthma, home oxygen requirement, cancer, COPD, smoking)

• Morbidity and mortality outcomes (death, cardiac arrest, hypotension requiring pressor administration, respiratory failure requiring endotracheal intubation or positive pressure ventilation, and hemodynamic impairment requiring thrombolytic administration)

To limit bias, the clinical variables required to determine the risk-stratification scores were abstracted by a reviewer blinded to outcome (treatment, complications, mortality), and outcome data were abstracted by a reviewer blinded to clinical variables. To determine reliability of data abstraction, each researcher then independently abstracted data on 10% of the entire cohort recorded by the other researcher.

Data Analysis

Each patient’s medical information determined their risk classification according to the criteria for each predictive model. Although initial intent was to calculate the Geneva Risk Score, this could not be calculated because neither a blood gas nor lower extremity ultrasound is routinely obtained at our hospital (13). Consequently, the PaO₂ and ultrasound data were unavailable for the majority of our cohort and it was not statistically valuable to evaluate those with complete data.

Pulse oximetry data and PESI scores were calculated for all patients and the proportion of patients with a defined adverse outcome (AO) was determined. We determined the sensitivity, specificity, and developed receiver operator characteristic curves to describe the performance of the two systems. When presenting continuous data, we used means and standard deviations, whereas categorical data are presented as proportions with 95% confidence intervals. Statistical analysis was performed using JMP 7 (SAS Institute, Cary NC) and PRISM 4.0 (Graph-Pad, San Diego, CA).

Results

A total of 300 patients were identified on the initial ED electronic database search using the described search criteria. Eleven patients were excluded, as 3 subjects had missing inpatient records, 3 subjects left against medical advice before hospital care was completed, 4 patients transferred to another hospital or hospice care, and 1 patient was entered twice for the same visit. Analysis of the electronic medical records revealed that of the remaining 289 patients, only 168 had a final diagnosis of PE, and these were our study cohort. Of the 121 patients who did not have PE according to complete medical records, 80 were diagnosed with DVT only and others had the search term “pulm” as a diagnosis code due to some other pulmonary complaint. The Kappa coefficients between the data abstractors for outcome data ranged from 0.84 to 1. The correlation for the predictor variables (temp, RR, HR, shock index, pulse ox) was 1 for all variables, indicating perfect agreement between the abstractors.

Final diagnosis of PE was determined by a positive contrast-enhanced helical CT scan, positive pulmonary angiography, high-probability ventilation-perfusion scan, or non-diagnostic scanning associated with synchronous DVT or high clinical suspicion due to symptoms or past medical history.

Of the 168 patients entered into the cohort, 5 (3.0%) died during their hospitalization. Additionally, 1 patient required vasopressor infusion for hypotension, 2 patients suffered from respiratory failure, and 4 patients required thrombolytic administration. Therefore, a total of 12 patients (7.1%) suffered an AO as defined by our criteria. Patients who suffered multiple AO were counted only under their “worst” outcome, in the order listed in the Methods section. Table 1 shows the demographics of the cohort.

Table 1.

Demographics of Patient Cohort

	No Adverse Outcome	Adverse Outcome
Demographics
Total number	156	12
Male, n (%)	67 (43%)	5 (42%)
Age (median, IQR)	53 (40–66)	58 (41–72)
Comorbidities
Cancer (%)	52 (33%)	4 (33%)
Heart failure	5 (3%)	1 (8%)
COPD	4 (4%)	5 (42%)
Smoker	56 (36%)	5 (42%)
Clinical findings
Heart rate (median, IQR)	96 (84–110)	106 (99–119)
Systolic BP mm Hg (median, IQR)	135 (119–145)	113 (97–149)
Respiratory rate (median, IQR)	18 (16–22)	23(19–26)
Altered mental status	9 (6%)	1 (8%)
Pulse oximetry < 90%	15 (10%)	3 (25%)
Interventions
Heparin	154 (99%)	12 (100%)
Echocardiogram	45 (29%)	7 (58%)
IVC filter	14 (9%)	2 (17%)

IQR = interquartile range; COPD = chronic obstructive pulmonary disease; BP = blood pressure; IVC = inferior vena cava.

Pulse Oximetry

PE patients were classified according to their first documented ED oxygen saturations (not prehospital). This was almost universally documented in the triage vital signs but otherwise was the first recorded pulse oximetry documented in the examination room. We used an a priori oxygen saturation of 92.5% as a cutoff because 93% saturation is considered the clinical cutoff for hypoxia at our altitude. This also gave the best stratification into low- and high-risk groups at our altitude. Only 136 of the PE patients had a pulse oximetry reading taken while breathing room air. Within this subset, 89/136 patients were classified as low risk. Only one of these patients (1.1%) suffered an AO. Five of the remaining 47 high-risk patients suffered from an AO, a rate of 10.6%. Notably, the AO rate increased to 3/20 (15%; 3.2–38%) at oxygen saturations lower than 88% on room air.

Results changed when we included the pulse oximetry data from the 32 patients in whom pulse oximetry was measured only on supplemental oxygen. Three of the additional 20 patients with a “low risk” pulse oximetry reading > 92.5% suffered from an AO, bringing the total AO rate to 3.7% (4/109) among low-risk patients. Twelve additional patients were still hypoxic despite receiving supplemental oxygen during their pulse oximetry reading, and 3 of them suffered an AO. Therefore, the overall AO rate for all PE patients (regardless of whether on supplemental oxygen or not) with a pulse oximetry reading lower than 92.5% was 8/59 (13.6%, 95% CI 6–25%). However, the true room air pulse oximetry reading cannot be known for a patient on supplemental oxygen, whereas it is reasonable to assume that a patient who is hypoxic on supplemental oxygen would be hypoxic on room air. Therefore, we also analyzed the data eliminating those 20 patients who were non-hypoxic on supplemental oxygen. This analysis gives a low-risk AO rate of 1.1% (1/89) and a high-risk AO rate of 13.6% (8/59), and improved the sensitivity of this pulse oximetry cutoff to 88.9%, with a specificity of 63.3%. The pulse oximetry data accounting for the varying concentrations of inspired oxygen are illustrated in Table 2 and the receiver operating curve for all of the pulse oximetry data is shown in Figure 1.

Table 2.

Pulse Oximetry Groups

	Room Air Only (136 Total)					Supplemental O2 Only (32 Total)					All Patients (168 Total)
	< 88%	88–90%	91–92%	93–94%	≥ 95%	< 88%	88–90%	91–92%	93–94%	≥ 95%	< 88%	88–90%	91–92%	93–94%	≥ 95%
Total no. patients	20	14	13	24	65	2	4	6	7	13	22	18	19	31	78
Death	1	0	1	0	1	0	1	0	1	0	1	1	1	1	1
Cardiac arrest	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
Pressors	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
Thrombolytics	0	1	0	0	0	0	1	1	0	1	0	2	1	0	1
ETT/resp failure	2	0	0	0	0	0	0	0	0	1	2	0	0	0	1
Total AO	3	1	1	0	1	0	2	1	1	2	3	3	2	1	3
% AO	15.0%	7.1%	7.7%	0.0%	1.5%	0.0%	50.0%	16.7%	14.3%	15.4%	13.6%	16.7%	10.5%	3.2%	3.8%
Sensitivity for AO*	83.3% (36–99%)					50% (14–86%)					66.6% (35–89%)
Specificity for AO*	67.7% (58–75%)					65.4% (44–82%)					67.3% (59–74%)

^*Using 92.5% saturation as the cutoff point. ETT = endotracheal tube; AO = adverse outcomes.

Figure 1.

Receiver operating curve for pulse oximetry.

PESI

The PESI categorizes those in classes I and II as low risk, and those in classes III–V as high risk for 30-day mortality. Using PESI, 91 of 168 patients were placed in the low-risk category. PESI adverse outcomes were as follows: 2.1% in Class I, 2.3% in Class II, 15.6% in Class III, 8.0% in Class IV, and 15.0% in Class V. Thrombolytics were given to 2 of the low-risk patients, but no other low-risk patients suffered AO, giving an AO rate of 2.2% (0.3 to 8%) in the low-risk group. Notably, both of these patients were on supplemental oxygen, possibly raising their oxygen saturations to non-hypoxic levels (the use of supplemental oxygen is not considered when calculating the PESI score). This raises the possibility that they may have been in a higher-risk PESI category if oxygen saturations had been measured on room air. The only patient categorized as PESI class I that suffered an AO was a 35-year-old woman with a heart rate of 122 beats/min who received thrombolytics. Seventy-seven of the 168 patients were classified as high risk, and 10 of these patients (13.0%) suffered from one or more AO. Only the worst AO was tallied: deaths (n = 5), requirement for vasopressors (n = 1), respiratory failure (n = 2), or thrombolytics (2). Overall, a high-risk PESI score had a sensitivity of 83% (95% CI 52–98%) and a specificity of 57% (95% CI 48–65%) for predicting AO in the entire cohort. The PESI data accounting for the varying concentrations of inspired oxygen are illustrated in Table 3, and Figure 2 demonstrates the receiver operating curve for the PESI score with all patients included.

Table 3.

PESI Groups

	Room Air Only (136 Total)					Supplemental O2 Only (32 Total)					All Patients (168 Total)
	V	IV	III	II	I	V	IV	III	II	1	V	IV	III	II	1
Total no. patients	15	21	26	37	37	5	4	6	6	11	20	25	32	43	48
Death	1	1	1	0	0	1	0	1	0	0	2	1	2	0	0
Cardiac arrest	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
Pressors	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
Thrombolytics	0	0	1	0	0	0	0	1	1	1	0	0	2	1	1
ETT/resp failure	0	1	1	0	0	1	0	0	0	0	1	1	1	0	0
Total AO	1	2	3	0	0	2	0	2	1	1	3	2	5	1	1
% AO	6.7%	9.5%	11.5%	0.0%	0.0%	40.0%	0.0%	33.3%	16.7%	9.1%	15.0%	8.0%	15.6%	2.3%	2.1%
Sensitivity for AO*	100% (51–100%)					66.7% (24–94%)					83.3% (51–97%)
Specificity for AO*	56.9% (48–65%)					57.7% (37–76%)					57.1% (49–65%)

^*Using PESI = II as cutoff for low risk.

PESI = Pulmonary Embolism Severity Index; ETT = endotracheal tube; AO = adverse outcomes.

Figure 2.

Receiver operating curve for the PESI (Pulmonary Embolism Severity Index).

Discussion

Our study aimed to retrospectively evaluate how prognostic models risk-stratify our single hospital’s ED PE patient population. Both a room air pulse oximetry cutoff and PESI performed fairly well and were practical tools with criteria based on readily available clinical factors. Respectively, their sensitivities were 83% and 83%, specificities were 68% and 57%, and negative predictive values (NPVs) were 99% and 98%. However, this study was not powered to compare the two risk-stratifying tools.

A pulse oximetry cutoff (regardless of supplemental or room air oxygen) at approximately 5280 feet above sea level of 92.5%, classified 63.9% of our patients as low risk (109/168), with an AO rate of 3.7% (4/109). Kline et al. used a room air pulse oximetry cutoff of 94.5% at sea level to predict a similar array of in-hospital adverse outcomes: death, cardiac arrest, development of a systolic blood pressure < 90 mm Hg persisting long enough to require vasopressor infusion, or respiratory arrest or respiratory distress requiring endotracheal intubation. Kline’s study of normotensive PE patients had a sensitivity of 90% and a specificity of 64%. This is a higher sensitivity than our overall pulse oximetry sensitivity of 83%; however, there are important differences between these studies. Unlike Kline, we did not include inpatients that later developed a PE and we did not exclude hypotensive patients or those with a high short-term mortality rate. We included all patients diagnosed in the ED with PE, regardless of their hemodynamic status. Therefore we used slightly different outcome measures, such as the administration of thrombolytics as an AO endpoint. Furthermore Kline obtained 30-day follow-up, whereas we followed only the inpatient course, so these endpoint differences may also lead to differences in test performance.

Unfortunately, in our study not all pulse oximetry readings were taken on room air, and this complicates our analysis. Including the oxygen saturation values from patients on supplemental oxygen weakens the power of pulse oximetry as a tool for risk stratification. When including all pulse oximetry readings, regardless of supplemental oxygenation, a pulse oximetry cutoff of 92.5% had a sensitivity of only 66.7% (35–89%) and a specificity of 67.3% (59–74%). However, when excluding PE patients on supplemental oxygen whose pulse oximetry readings were above 92.5%, the sensitivity for AO increases to 88.9% (50–99%), with a specificity of 63.3% (54–72%). With these parameters, 89/136 patients were classified as low risk for AO. Only one of these patients (1.1%) suffered an adverse outcome. This lone false-negative case involved a terminally ill patient with a room air pulse oximetry of 95% who died of multiple cancer metastases to the brain (glioblastoma, astrocytoma). Therefore one would not suspect hypoxia to be a presenting vitals sign abnormality. Furthermore, Kline et al. would have excluded this patient from their data set based on short-term high mortality rate. Our data suggest that a pulse oximetry cutoff is a more sensitive predictor of AO when only considering measurements made while the patient is breathing room air and in patients still hypoxic despite supplemental oxygenation. However, this study is not powered to definitively declare this.

Aujesky et al. were able to classify 47% of their European PE patients into a low-risk group, and these patients had an overall mortality of 1.2% and a PE-specific mortality of 0.7%. Their PESI sensitivity was 91% (81–97%) and NPV was 99% (97–100%). In our American study, PESI identified a slightly greater percentage of PE patients as low risk (91/168) (54.2%), with an AO rate of 2.2% (2/91). Our overall PESI sensitivity was also lower at 83%. There were 5/168 (3%) deaths. However, Aujesky et al. used only mortality as an endpoint and did not consider the other AO that we assessed in our study. We believe that these other outcomes are also important as they require further monitoring and in-hospital interventions.

The 2 PESI Class I and Class II patients who suffered an AO were on supplemental oxygen while their triage vitals were taken. The patient in Class I had an oxygen saturation reading of 95% while on 3 L, and the Class II patient had a reading of 90% while on 3 L. The current PESI model disregards whether or not a pulse oximetry reading was taken on supplemental oxygen. If these patients had a pulse oximetry below 90% on room air, each score would have each been increased one additional PESI class due to the additional 20 points. This would have resulted in zero patients in our cohort defined as Class I and only 1 patient defined as Class II. The PESI score may benefit from revision to accommodate for supplemental oxygen or should be consistent in scoring only room air oxygen saturation values.

Using pulse oximetry values for patients receiving supplemental oxygen significantly weakens both pulse oximetry and the PESI as prognostic indicators. Including the supplemental oxygen data in our study lowers the sensitivity of a 92.5% pulse oximetry cutoff from 83.3% to 50%, and lowers the sensitivity of PESI from 100% to 66.7%. Including those patients who received supplemental oxygen without adjusting for their supplemental oxygen lowers the NPV from 98.9% to 85% for the pulse oximetry tool and lowers the NPV for the PESI tool from 100% to 88.2%.

There are other limitations of the PESI score. Aujesky et al. acknowledge that the original data used to “validate PESI externally were not originally designed for that task and the mental status was not explicitly recorded.” We agree that mental status data are difficult to abstract retrospectively from chart review and are given a heavy weight of 60 points in the PESI. Aujesky et al. also acknowledge that PESI may have performed differently in a sicker population. Our study attempts to overcome this limitation by applying the PESI to all PE patients diagnosed in the ED and we feel that it performed quite well across all levels of acuity. We differed from the original PESI study in that we did not perform 3-month follow-up and only assessed in-hospital AO and death.

Although we initially intended to evaluate the Geneva Risk Score, the additional clinical data required are not routinely gathered in our ED and therefore could not be applied to the majority of our patients. These additional assessments are expensive, require specialized personnel and equipment not easily accessible to all EDs, and are potentially time-consuming. However, pulse oximetry and PESI require no laboratory tests or additional radiographic procedures, are more functional in our typical ED setting, and were easily applied. Furthermore, when PESI was compared with the Geneva Risk Score, Jimenez et al. found that the PESI quantified the prognosis of PE patients more effectively than Geneva criteria (14).

The precision of our estimates of test performance was limited by the very small number of adverse outcomes. Recent reports have shown that the mortality rates from PE have declined significantly in the last few years, from 10% to 1.68% in one study (15). This more closely matches our mortality rate of 3.0%. Still, although the power of our study was not high, the data do suggest that both pulse oximetry and the PESI have promise as stratification tools to identify patients at high risk for adverse outcomes, particularly if patient oxygenation data are obtained on room air.

These prognostic models could be very useful in providing clinicians with a set of specific criteria to guide their medical decision-making in the ED. Other studies have validated the PESI as effective in identifying patients with low risk of 30-day mortality, significant bleeding, or recurrent venous thromboembolism (5). Our study expands on this by demonstrating that the PESI also stratifies patients according to whether the severity of their disease requires any procedures necessitating inpatient hospitalization. Low-risk patients might be candidates for early discharge and outpatient therapy. The PESI identified 54.1% of our patient cohort as low risk. However, further studies are needed to determine whether hospitalization of low-risk patients lowers their morbidity and mortality when compared to treatment as an outpatient.

Limitations

Several limitations to our study should be acknowledged. First, this was a retrospective study of a relatively small cohort of 168 PE patients from a single hospital at altitude since June 2004, when our electronic medical record system was established. We acknowledge the very small absolute number of AOs, which leads to large confidence intervals around the sensitivities of each model. However, our study period represents a recent PE patient population with the potential for improved diagnosis and lower mortality rates using current standard of care. Second, this study followed patients only during their hospital course and did not seek to obtain a 3-month follow-up, which the derivation studies for the risk-stratification models did. However, we felt that our main goal was to evaluate the course of that specific hospital visit to determine if there were any interventions that could not be performed in an outpatient setting.

Third, this study included only patients with suspected PE or DVT in the ED setting and not those admitted who were later diagnosed with PE as an inpatient. Therefore, our study could have missed patients with milder PE or DVT symptoms who were not suspected or diagnosed to have PE as well as those with other severe medical or surgical issues such as trauma or heart failure in whom PE was not diagnosed in the ED setting.

Fourth, some patients came into the ED on supplemental oxygen, distorting the pulse oximetry data; although we attempted to delineate those who were still hypoxic despite oxygen, it is impossible to know if those documented only on oxygen may have been hypoxic. Finally, it is possible that the same clot burden was evaluated on more than one occasion. Four patients had multiple visits entered into our study, although all but one had visits separated by at least 2 months. However, because each visit represents a discrete patient encounter, the vital signs determined at triage presentation are still relevant for risk stratification for that patient on that ED visit.

Conclusion

Using risk-stratification tools, specific criteria may be helpful to identify a subset of PE patients at low risk for adverse outcomes. The Pulmonary Embolism Severity Index cutoff score of II or a room air pulse oximetry cut off of 92.5% oxygenation at our altitude of 5280 feet proved to be a modestly reliable predictor of short-term morbidity and mortality. The sensitivities of both tools may be improved by measuring room air oxygenation values. Larger prospective studies are needed to confirm this.