PERFORM: Pulmonary embolism risk score for mortality in computed tomographic pulmonary angiography-confirmed patients

Abstract

Background

Current prognostic scores for pulmonary embolism (PE) were partly based on patients without PE confirmation via computed tomographic pulmonary angiography (CTPA), involving subjective parameters and complicated scoring methods. Therefore, we sought to develop an objective, accurate, and simple prognostic model in CTPA-confirmed patients to predict the risk of 30-day mortality.

Methods

We retrospectively evaluated 509 patients with objectively confirmed PE by CTPA from 2010 to 2017 in the Minhang Hospital, which is affiliated to Fudan University. Patients were randomly divided into the training and validation cohorts. The primary end point was 30-day mortality. The secondary end points were the time to recovery in 30 days and mortality in 15 days. We compared the predictive performance of Pulmonary Embolism Severity Index (PESI), simplified PESI (sPESI), and the PE risk score we developed, called PERFORM.

Findings

PERFORM (ranging from 0 to 12 score) is based on the patient's age, heart rate, and partial pressure of arterial oxygen. The area under the curve was 0.718 (95% confidence interval [CI], 0.627–0.809) for the training cohort and 0.906 (95% CI, 0.846–0.966) for the validation cohort. PERFORM was as good as PESI and sPESI in predicting mortality. Patients in the low-risk group (PERFORM score < 5) had a shorter time to recovery, whereas those in the high-risk group (PERFORM score ≥ 5) had a high mortality.

Interpretation

PERFORM in CTPA-confirmed patients is an objective, accurate, and simple tool to predict the risk of 30-day mortality.

Funding

Research Project of Shanghai Municipal Commission of Health and Family Planning (201740127), Shanghai Medical Key Subject Construction Project (ZK2019B08).

Research in context.

Evidence before this study

We conducted an extensive search of previous studies on prognostic scores for pulmonary embolism (PE), using the PubMed and Google Scholar, with the terms “pulmonary embolism”, “risk score”, “mortality”, “computed tomographic pulmonary angiography (CTPA)”, etc. We found that current PE risk scores partly included patients without PE confirmation through CTPA and comprised many subjective variables on the basis of medical history and inquiry. Therefore, they cannot accurately predict the risk of 30-day mortality in CTPA-confirmed patients.

Added value of this study

In order to improve the accuracy of PE risk scores, we identified patients with PE using International Classification of Diseases code combined with CTPA. Finally, our score for PE is entirely based on patients with a confirmed PE diagnosis by CTPA. Our score consists of objective, routinely measurable predictors within a short period of time after admission and does not rely on past medical history and inquiry. It may accurately predict the risk of 30-day mortality in CTPA-confirmed patients in different clinical settings. PERFORM only consists of three variables categorized using clinically meaningful cutoff points that are commonly used in clinical practice and are easily remembered by physicians. It is more convenient to calculate in busy clinical settings. In the study, PERFORM showed similar predictive performance to the Pulmonary Embolism Severity Index (PESI) and simplified PESI (sPESI) scores for 30-day mortality.

Implications of all the available evidence

Early identification and timely therapeutic strategies for patients with PE are of great importance to reduce mortality. The PERFORM score may be determined and implemented as an objective, accurate, and simple tool for early risk stratification of patient, not only by specialists but also by physicians with diverse backgrounds and specialties. Low-risk patients could be considered for outpatient therapy and early hospital discharge, while high-risk patients could be admitted for surveillance in an intensive care setting. Our PE score may help frontline clinicians in optimizing medical treatment with limited resources.

Alt-text: Unlabelled box

1. Introduction

Pulmonary embolism (PE) is a common and potentially lethal condition in the emergency department requiring early and accurate management [1]. Deaths from PE usually occur within weeks after the diagnosis is made [2]. The short-term mortality rate of PE varies widely and ranges from less than 2% in many patients with nonmassive PE to more than 95% in patients who experience cardiorespiratory arrest [3], [4], [5].

Although several prognostic models of acute PE are currently used, all of them have practical limitations [6], [7], [8], [9], [10]. Of all clinical scores integrating PE severity and comorbidity, the Pulmonary Embolism Severity Index (PESI) and its simplified version, the simplified Pulmonary Embolism Severity Index (sPESI) have been most extensively validated to date [11], [12], [13], [14]. Computed tomographic pulmonary angiography (CTPA) is the gold standard for PE diagnosis. However, in some studies, an International Classification of Diseases (ICD) code was the only criteria for identifying patients with PE [6, 15, 16]. These studies may include some patients with suspected PE and non-PE who had similar signs and symptoms to PE. Therefore, they cannot accurately predict the risk of 30-day mortality in CTPA-confirmed patients. In addition, existing PE risk scores comprise many subjective variables on the basis of medical history and inquiry, such as history of cancer and chronic cardiopulmonary diseases. In countries and regions where the electronic medical systems of each hospital cannot be shared, the past medical history of the patients cannot be accurately evaluated by physicians. Some patients may not know that they have chronic cardiopulmonary diseases or occult cancer or they may be unable to describe diseases clearly because of factors, such as psychological stress and low education. Therefore, deviation is present in the PE risk assessment for mortality. A study of 17 clinical prognostic models of PE reported that the current prediction models have disadvantages [13]. Moreover, under a busy clinical working environment with a heavy workload, too many variables increase computational complexity and are not convenient for daily clinical practice.

Considering the limitations of the current prognostic models, an objective, accurate, and simple clinical prognostic model for PE is needed to help clinicians assess patients’ risks and improve therapeutic decision-making, such as the early discharge from the hospital or complete outpatient management for patients at low risk or closer monitoring and aggressive therapy for patients at high risk [16], [17], [18], [19].

Here, we report a simple clinical prognostic model, which does not rely on past medical history and inquiry, to assess the risk of 30-day mortality. Only patients with confirmed PE using CTPA were considered for inclusion to improve the accuracy of PE risk scores.

2. Materials and methods

2.1. Data collection

According to the diagnosis strategy of “2019 ESC Guidelines for the diagnosis and management of acute PE developed in collaboration with the European Respiratory Society (ERS)” [20], we retrospectively analyzed patients who were hospitalized at the Minhang Hospital, which is affiliated to Fudan University, from January 2010 to December 2017. We used ICD discharge diagnosis codes, including I26.0 or I26.9, to identify patients with PE [15]. Clinical electronic medical records, laboratory findings, nursing records, and radiological reports for all patients with PE were reviewed. All patients received standardized treatment. Only patients with PE confirmed by CTPA were considered for inclusion in the study. Detailed admission data, including demographic information, past medical history, signs and symptoms, laboratory test results, and imaging reports, of each patient were collected.

2.2. Study design

Using a computer-generated randomization list, the study cohorts were randomly divided into two groups (the proportion was approximately 2:1). One group was used to construct the model (training cohort, n = 339), while the other group was used to validate the model (validation cohort, n = 170). The primary end point was 30-day mortality. The secondary end points were the time to recovery in 30 days and mortality in 15 days. The time to recovery was defined by discharge from the hospital. The study was conducted in accordance with the Declaration of Helsinki and was approved by the Ethics Review Committee of the Minhang Hospital. This study adheres to RECORD guidelines.

2.3. Statistical analysis

Statistical analysis was performed using the SAS statistical software version 9.4 (SAS Institute, Cary, NC, USA). Descriptive data were expressed as absolute numbers, percentages or means ± standard deviation (means ± SD), or medians (interquartile range). The normally distributed continuous variables were compared using the Student's t-test or ANOVA, whereas the non-Normally distributed variables were compared using the Mann–Whitney U test. Comparisons between categorical variables were performed using Chi-square test or Fisher's exact test.

To develop our prediction score, we used the clinical data of the training cohort to assess the effects of different clinical variables upon admission (sex, age, symptoms, signs, echocardiography, blood gas analysis, blood routine, blood biochemical examination, coagulation function, and treatment) on mortality using a univariate logistic regression [2, [21], [22], [23], [24]]. Variables with an unadjusted P-value of < 0.10 were potential risk factors or protective factors. In the multivariate logistic regression analysis, stepwise regression (P < 0.05) was performed to screen potential variables for inclusion in the final model and the odds ratio (OR), and 95% confidence interval (CI) were calculated. Finally, three variables associated with the risk of 30-day mortality were incorporated into the regression model. To simplify clinical application, continuous variables were categorized using clinically meaningful cutoff points that are commonly used in clinical practice and are easily remembered by physicians. Then, we established a clinical risk score ranging from 0 to 12 points in combination with the OR value of each variable, with higher scores representing worse prognosis.

The diagnostic value of the final model was tested in a validation cohort of 170 patients with objectively confirmed PE. Receiver operating characteristic (ROC) curves were constructed to compare the area under curve (AUC) of three different scoring systems (PESI, sPESI, and PERFORM) to assess the accuracy of the prognostic models. We estimated the optimal cut-off values of each scoring system based on the Youden index (sensitivity + specificity-1) and the corresponding sensitivity, specificity, and positive and negative predictive values. Patients were divided into two groups (high-and low-risk groups) following the cut-off value of ROC curve. Survival curves were estimated using the Kaplan–Meier method and compared using log-rank test. A P-value < 0.05 was considered statistically significant.

2.4. Role of the funding source

The funding source had no role in the study design, collection, analysis, or interpretation of the data, the writing of this manuscript, or the decision to submit the manuscript for publication. All authors had full access to the full data in the study and accepted responsibility to submit for publication.

3. Results

3.1. Characteristics of patients

From January 2010 to December 2017, a total of 612 patients with PE, identified using ICD codes, were admitted to Minhang Hospital. Of those, 31 patients who did not undergo CTPA and 63 patients with negative CTPA were excluded from the study. Of the 518 patients who were objectively confirmed with PE, nine were excluded from the study because they left the hospital prior to discharge by a physician. Hence, 509 patients with objectively confirmed PE were ultimately included in the analysis (Fig. 1). Among them, 36 inpatients died. A total of 24 and 12 deaths were recorded in the training and validation cohorts, respectively. The baseline characteristics of the training and validation cohorts are outlined in Table 1. Overall, the two groups had no significant differences (P > 0.05) in terms of age, sex, symptoms, physical signs, and past medical history except for chronic heart disease (P = 0.0178).

Fig. 1.

Eligibility of patients for inclusion in this study.

Table 1.

Comparison of baseline characteristics of patients in the training and validation cohorts (N = 509).

Index	Training (n = 339)	Validation (n = 170)	P
Age (y), median (IQR)	79 (69–85)	76.5 (66–85)	0.1910
Male, n (%)	176 (51.92%)	80 (47.09%)	0.7772
Symptoms
Dyspnea, n (%)	26 (7.67%)	14 (8.24%)	0.8230
Chest pain, n (%)	44 (12.98%)	29 (17.06%)	0.2155
Cough, n (%)	195 (57.52%)	96 (56.47%)	0.8211
Fever, n (%)	53 (15.63%)	36 (21.18%)	0.1205
Hemoptysis, n (%)	5 (1.47%)	6 (3.53%)	0.2379
Syncope, n (%)	24 (7.08%)	14 (8.24%)	0.6399
Altered mental status, n (%)	52 (15.34%)	25 (14.71%)	0.8508
Unilateral lower limb pain, n (%)	7 (2.06%)	5 (2.94%)	0.5389
Signs
Respiratory rate (beats/min), median (IQR)	20 (18–20)	20 (18–20)	0.9458
Heart rate (beats/min), median (IQR)	82 (75–94)	81 (76–96)	0.8744
Systolic pressure (mmHg), median (IQR)	130 (120–140)	130 (120–140)	0.7951
Jugular vein filling, n (%)	89 (26.25%)	35 (20.59%)	0.1602
Lung wet rales, n (%)	153 (45.13%)	76 (44.71%)	0.9273
P2 hyperfunction, n (%)	7 (2.06%)	3 (1.76%)	0.8180
Bilateral Lower limb edema, n (%)	38 (11.21%)	20 (11.76%)	0.8525
Lower limb asymmetric edema, n (%)	13 (3.83%)	7 (4.12%)	0.8769
Past medical history
Cancer, n (%)	10 (2.95%)	4 (2.35%)	0.9195
Chronic heart disease, n (%)	43 (12.68%)	10 (5.88%)	0.0178
Chronic lung disease, n (%)	25 (7.37%)	6 (3.53%)	0.0871
Pulmonary embolism, n (%)	13 (3.83%)	7 (4.12%)	0.8769
Deep vein thrombosis, n (%)	21 (6.16%)	10 (5.88%)	0.8895
Immobilization, n (%)	177 (52.21%)	83 (48.82%)	0.4707
Surgery, n (%)	113 (33.33%)	52 (30.59%)	0.5326
Hypertension, n (%)	129 (38.05%)	63 (37.06%)	0.8272
Diabetes, n (%)	54 (15.93%)	35 (20.59%)	0.1918

3.2. Predictors of PERFORM

Using data from 339 patients with objectively confirmed PE in the training cohort, we explored predictors of death in patients with PE. Univariate logistic regression analysis showed that age, heart rate, bilateral lower limb edema, lower limb asymmetric edema, neutrophil percentage, lymphocyte percentage, red blood cell distribution width, blood potassium content, partial pressure of arterial oxygen (PO₂), and TnI (cardiac troponin) were significantly associated with mortality (Table 2). After nine variables obtained from the univariate regression analysis were incorporated into a multivariate logistic regression model, the results showed that age (OR: 1.060; 95% CI: 1.012–1.111), heart rate (OR: 1.030; 95% CI: 1.006–1.055), and PO₂ (OR: 0.529; 95% CI: 0.335–0.835) were significantly associated with the risk of PE death (Table 2).

Table 2.

Univariate and multivariate analyses of variables associated with the 30-day mortality in the training cohort.

Variables	Univariate		Multivariate final model (stepwise)
	P-value	OR (95% CI)	P-value	OR (95% CI)
Age	0.010	1.060 (1.014–1.109)	0.015	1.060 (1.012–1.111)
Heart rate	0.006	1.031 (1.009–1.054)	0.015	1.030 (1.006–1.055)
Bilateral lower limb edema	0.033	2.948 (1.091–7.962)		Not included
Lower limb asymmetric edema	0.034	4.357 (1.114–17.042)		Not included
Neutrophil percentage	0.005	1.063 (1.019–1.110)		Not included
Lymphocyte percentage	0.027	0.944 (0.897–0.994)		Not included
Red blood cell distribution width	0.036	1.067 (1.004–1.133)		Not included
K+	0.030	1.015 (1.001–1.029)		Not included
PO2	0.001	0.477 (0.311–0.732)	0.006	0.529 (0.335–0.835)
TNI	0.032	9.079 (1.203–68.532)		Not included

3.3. Comparison of three scoring systems in the validation cohort

We used the three variables associated with PE death to develop a clinical prognostic model (Table 3). Next, ROC curves were constructed to compare the AUCs of the three different scoring systems (PESI, sPESI, and PERFORM). The differences in AUC of each scoring system in the whole cohort and the training cohort were not significant. In the validation cohort, the AUC was higher for the PERFORM (0.906; 95% CI: 0.846–0.966) than for the sPESI (0.820; 95% CI: 0.733–0.907) (P = 0.022) and the PESI scoring systems (0.818; 95% CI: 0.741–0.895) (P = 0.020). This finding indicated that the PERFORM score may have a higher predictive value for PE death than the other two scoring systems (Table 4 and Fig. 2). In addition, we estimated the optimal cut-off values of the three scoring systems based on the maximum Youden index and compared the corresponding sensitivity, specificity, positive and negative predictive values. In the training, validation, and whole cohorts, the optimal cut-off values of the scoring systems were similar. Compared with the two other scoring systems, PERFORM had a higher specificity and a relatively lower sensitivity (Table 5).

Table 3.

PERFORM score based on variables associated with pulmonary embolism death.

Index	Point
Age (y)
<65	0
≥65 and <75	1
≥75 and <85	2
≥85	4
Heart rate (beats/min)
<75	0
≥75 and <85	1
≥85 and <95	2
≥95	4
PO2 (mm Hg)
≥80	0
≥60 and <80	1
≥40 and <60	2
<40	4

Table 4.

AUC of ROC curves of PERFORM, sPESI, and PESI.

Group	AUC	SE	P	95% CI
Whole
PERFORM	0.780	0.035	–	0.711–0.849
sPESI	0.732	0.039	0.188	0.656–0.809
PESI	0.734	0.039	0.211	0.657–0.810
Training
PERFORM	0.718	0.046	–	0.627–0.809
sPESI	0.688	0.054	0.552	0.582–0.793
PESI	0.692	0.056	0.618	0.582–0.801
Validation				–
PERFORM	0.906	0.031	–	0.846–0.966
sPESI	0.820	0.044	0.022	0.733–0.907
PESI	0.818	0.039	0.020	0.741–0.895

Fig. 2.

Comparison of ROC curves of PERFORM, sPESI, and PESI in the whole (a), training (b), and validation cohorts (c) for predicting 30-day mortality.

Table 5.

Prognostic performance of PERFORM, sPESI, and PESI.

Index	Cut-off value	The maximum Youden's index	Sensitivity	Specificity	PPV	NPV
Whole
PERFORM	5	0.450	0.857	0.593	0.142	0.981
sPESI	1	0.328	0.914	0.414	0.109	0.984
PESI	92	0.415	0.914	0.501	0.125	0.987
Training
PERFORM	5	0.379	0.783	0.596	0.129	0.973
sPESI	1	0.260	0.870	0.391	0.098	0.975
PESI	92	0.343	0.870	0.474	0.112	0.979
Validation
PERFORM	7	0.730	0.833	0.897	0.400	0.985
sPESI	1	0.462	1	0.462	0.133	1
PESI	100	0.697	1	0.697	0.214	1

3.4. Kaplan–Meier survival analysis based on the PERFORM score

According to ROC analysis, the optimal cut-off value of the PERFORM score was 5. Therefore, patients were divided into high-risk (≥ 5) and low-risk (< 5) groups. Patients in the high-risk group had worse clinical outcomes than those in the low-risk group. Among patients in our analysis, 1%, and 1% of the patients in the low-risk group had died by days 10, 20, and 30, as compared with 5%, 8%, and 11% of patients in the high-risk group, respectively. In the low-risk group, 17%, 74%, and 91% of the patients had been discharged from the hospital by days 10, 20, and 30 compared with 12%, 63%, and 78% of the patients in the high-risk group, respectively (Fig. 3a). Patients in the low-risk group had a shorter time to recovery than those in the high-risk group (median, 15 days compared with 17 days; rate ratio for recovery, 1.40; 95% CI, 1.15–1.70; P = 0.0006; 482 patients). Among the patients in the training cohort (325 patients), the rate ratio for recovery was 1.28 (95% CI, 1.02–1.62). Among patients in the validation cohort (157 patients), the rate ratio for recovery was 1.68 (95% CI, 1.20–2.35) (Fig. 3b). Mortality was numerically higher in the high-risk group than in the low-risk group, and the difference was significant (hazard ratio for death, 5.76; 95% CI, 2.21–14.99; 482 patients). The Kaplan–Meier estimates of mortality by 15 days were 1.6% and 5.6% in the low- and high-risk groups, respectively (Fig. 3c).

Fig. 3.

Clinical outcomes in 30-day follow-up and Kaplan–Meier estimates of the time to recovery in 30 days and mortality in 15 days in the whole, training, and validation cohorts according to the PERFORM score.

4. Discussion

We present a clinical prediction score based on a large cohort of patients with confirmed PE by CTPA. The PERFORM score only consisted of three variables, which are categorized using clinically meaningful cutoff points that are commonly used in clinical practice and are easily remembered by physicians. Through simple calculation, it showed similar predictive performance to the PESI and sPESI score for 30-day mortality. Besides, PERFORM has advantages over PESI and sPESI as it is calculated using objective variables readily available at initial examination and is not subject to patient recall error regarding past medical history. Therefore, PERFORM has a better specificity compared with PESI or sPESI.

According to the score, the severity of PE is classified as high or low risk. Patients in the low-risk group had a shorter time to recovery, while those in the high-risk group had a high mortality. This simple means of scoring could be useful for physicians in the emergency department as it may allow for early risk identification in a single patient, thereby favoring an efficient management and substantially reducing the use of healthcare resources [25]. Using < 5 points as cut-off, the PERFORM score allows identification of an important subgroup of patients in the low-risk group who may be considered for outpatient therapy and early hospital discharge. Conversely, patients in the high-risk group according to the PERFORM score (≥5 points) may require surveillance in an intensive care setting. Therefore, using the objective, accurate, and simple prognostic model including only three objective parameters readily available at initial examination in CTPA-confirmed patients, individualized treatment may be an important step forward for management of PE.

To evaluate the patients comprehensively, we considered for inclusion in the model all of the known risk factors from PESI and sPESI combined with the laboratory indicators obtained easily within minutes of a patient's arrival to the emergency department. We believe that the PERFORM is useful because it includes one variable that quantifies the age of the patients, and two variables that express the cardiopulmonary consequences of PE. Age, heart rate, and oxygen partial pressure are routinely available parameters in all hospital settings and were previously shown to be associated with adverse outcomes among patients with PE [8, 22, 23, 26]. Elevated heart rate in settings of acute PE is associated with a more severe PE stage and poorer outcomes [27]. Tachycardia was associated with a seven times higher risk of in-hospital death [28]. The pathophysiology of PE is associated with cardiovascular and pulmonary gas exchange abnormalities [29]. The resulting hypoxemia related to the increase in right ventricular pressure initiates a cascade of right ventricular injury and dysfunction that results in death [20, 30]. Decreased oxygen partial pressure may be correlated with increased severity of PE [31].

This study has several limitations. First, given the retrospective design of our study, some patients with confirmed PE may be excluded from the analysis due to missing data, which may cause selection bias. In addition, we could also not accurately determine when heart rate and oxygen partial pressure were recorded and what duration was an average pulse taken in the patients of the present study, which may be the problem of PESI, sPESI and PERFORM. However, this may not affect the overall study trend because these parameters remain relatively stable within a short period after admission. Moreover, it is a retrospective study from 2010 to 2017. Although the influence of changes in techniques, diagnostics and treatment are inevitable, overall, the changes are relatively small and may not lessen its applicability. The obtained outcomes need to be assessed in a prospective validation study. Second, the number of deaths caused by PE and the size of the cohort are relatively small; thus, larger prospective cohort studies may be required for validation. Third, our study cohort was drawn from the same hospital, and the results may be different in other settings. PERFORM only underwent internal validation with a small cohort, which also may cause a bias. External validation through cases from other hospitals or regions is needed to confirm the universal applicability of PERFORM.

The study retrospectively developed PERFORM, which is a clinical prediction score for estimating the risk of 30-day mortality in a cohort of patients with objectively confirmed PE in Minhang Hospital Affiliated to Fudan University for 8 years. Our PERFORM score uses fewer predictor variables than the existing scores and may have a higher predictive value. As such, it may provide clinicians with an explicit tool for risk identification, thus supporting appropriate treatment and optimizing the use of medical resources. Whether this score remains accurate and useful in clinical practice should be determined in a prospective validation study. On this basis, we will further increase the number of patients included, and perform a multicenter, prospective study to further evaluate its clinical utility.

Data sharing statement

Data will be available from the corresponding authors upon reasonable request.

CRediT authorship contribution statement

Shuili Yu: Writing – original draft, Writing – review & editing, Validation, Data curation. Honglu Zhou: Writing – original draft, Writing – review & editing, Validation, Data curation. Yang Li: Data curation. Jianfeng Song: Data curation. Jinyan Shao: Writing – original draft, Writing – review & editing. Xuanyi Wang: Writing – review & editing. Zichen Xie: Conceptualization, Writing – original draft, Writing – review & editing, Supervision, Validation, Data curation. Chao Qiu: Conceptualization, Writing – original draft, Writing – review & editing, Supervision, Validation, Data curation. Keyu Sun: Conceptualization, Writing – original draft, Writing – review & editing, Supervision, Validation.

Declaration of Competing Interest

All authors who have taken part in this study declare that they have nothing to disclose.