Pulmonary embolism risk stratification: external validation of the 4-level Clinical Pretest Probability Score (4PEPS)

Abstract

Background

The 4-level clinical pretest probability score (4PEPS) was recently introduced as a clinical decision rule for the diagnosis of pulmonary embolism (PE). Based on the score, patients are classified into clinical pretest probability categories (c-PTP). The “very low” category aims at excluding PE without further testing; “low” and “moderate” categories require D-dimer testing with specific thresholds, while patients with a “high” pretest directly proceed to imaging.

Objectives

To provide further external validation of the 4PEPS model.

Methods

The 4PEPS was applied to a previously collected prospective database of 756 patients with clinically suspected PE enrolled from European emergency departments in 2002 to 2003. The safety threshold for the failure rate in our study was calculated at 1.95% based on a 26% prevalence of PE in our study, as per the International Society on Thrombosis and Haemostasis Scientific and Standardization Committee guidance.

Results

Patients were classified as follows: 90 (12%) in the very low c-PTP group, of whom 5 (5.6%; 95% CI, 2.4%-12.4%) had PE; 363 (49%) in the low c-PTP group, of whom 34 had PE (9.4%); 246 (34%) in the moderate c-PTP group, of whom 124 (50%) had PE; and 35 (5%) in the high c-PTP group of whom 30 (86%) had PE. Overall, the failure rate of the 4PEPS was 9/734 (1.2%; 95% CI, 0.59%-2.23%) Overall, 9 out of 734 patients (1.2%; 95% CI, 0.59%-2.23%) were diagnosed with PE despite a negative 4PEPS rule; 5 (5.6%) from the very low c-PTP group, 3 (1.4%) in the low c-PTP group, and 1 (3.2%) in the moderate c-PTP group.

Conclusion

We provide external validation data of the 4PEPS. In this high-prevalence cohort (26% prevalence), PE prevalence in the very low-risk group was higher than expected. A prospective validation study is needed before implementing the 4PEPS model in routine clinical practice.

Essentials

• •4-level clinical pretest probability score (4PEPS) is a clinical decision rule to help diagnose pulmonary embolism.
• •We externally validated the 4PEPS in a population with high prevalence of pulmonary embolism.
• •4PEPS reduced D-dimer and computed tomography scan tests, with a low failure rate of 1.2% in 734 patients.
• •Before implementing 4PEPS in clinical practice, a prospective validation study is needed.

1. Introduction

The clinical diagnosis of pulmonary embolism (PE) remains a challenge that requires clinical gestalt, D-dimer testing, and imaging [1]. The challenge is hinted at by the plethora of clinical decision rules and strategies available. Some of these include the Wells score, Revised Geneva score, pulmonary embolism rule-out criteria (PERC), simplified diagnostic management of pulmonary embolism (YEARS), or the Pulmonary Embolism Graduated D-dimer (PEGeD) strategies [[2], [3], [4], [5], [6]]. These scores include various combinations of factors such as age, risk factors, symptoms, and physical exam findings. The aim for these scores is to reduce the need for computed tomography (CT) imaging in the emergency department (ED), which can lead to unwarranted radiation risks, incidental findings, and costs.

Roy et al. [7] proposed to integrate previous knowledge and derived a rule combining existing scores into a single clinical decision rule called the 4-level clinical pretest probability score (4PEPS). In their study, a Bayesian approach was used to predetermine 4 clinical pretest probability levels (c-PTP), aiming to achieve a posttest probability of PE less than 2%. For the derivation of the score, 3 prospectively collected databases from EDs in France, Belgium, and the US were used. To determine which variables were strong predictors of PE, variables from patients in these databases and the individual PE scores were included in a univariate analysis. Those significant variables were then included in a multivariate logistic regression model, and remaining significant variables were included in the final score. Of note, certain variables were not included as predetermined by the authors, such as if their database had more than 2% of missing data of a certain variable, which consisted of mostly chronic medical conditions and the main symptoms of dyspnea or chest pain. In addition, the combination of dyspnea and chest pain was included in their initial analysis and ended up in the final score. The final 4PEP score is displayed in Table 1.

Table 1.

Four-level clinical pretest probability score.

Four-level pulmonary embolism Clinical Probability Score
Variable	Regression coefficient	Points
Age, y
<50	-0.993	-2
50-64	-0.656	-1
Chronic respiratory disease	-0.570	-1
Heart rate <80 beats per minute	-0.406	-1
Chest pain and acute dyspnea	0.297	1
Male	0.472	2
Hormonal estrogenic treatment	0.608	2
Personal history of VTE	0.711	2
Syncope	0.504	2
Immobility within the last 4 weeks	0.509	2
Pulse oxygen saturation <95%	0.832	3
Calf pain and/or unilateral lower limb edema	1.009	3
PE is the most likely diagnosis	1.860	5

c-PTP, total
Very low c-PTP (<2%): PE can be ruled out	<0
Low c-PTP (20%-65%): PE can be ruled out if D-dimer <1000 ng/mL	0-5
Moderate c-PTP (20%-65%): PE can be ruled out if D-dimer <500 ng/mL or <age × 10 ng/mL	6-12
High c-PTP (>65%): PE cannot be ruled out without imaging testing	≥13

c-PTP, clinical pretest probability; PE, pulmonary embolism; VTE, venous thromboembolism.

Ultimately, Roy et al. [7] conducted internal and external validation in separate cohorts and found that the 4PEPS was accurate, safe, and efficient to use in the ED in clinical decision-making for PE assessment [7]: the 4PEPS had an acceptable failure rate of 0.71% (95% CI, 0.37%-1.23%) and 0.89% (95% CI, 0.53%-1.49%) in 2 different cohorts, with a reduction in imaging testing by -22% (95% CI, -26% to -19%) and -19% (95% CI, -22% to -16%), respectively. A formal outcome study was proposed as the next step. While these findings are promising, we have taken the opportunity to conduct a separate independent external validation of the 4PEPS in order to reassess these findings in a different prospectively obtained population.

2. Methods

2.1. 4PEPS derivation

The derivation of the 4PEPS is summarized above. The full methodology has been described in the original paper [7]. A 4PEPS of less than 0 corresponds to a very low c-PTP (less than 2%), a 4PEPS of 0 to 5 corresponds to a low c-PTP (less than 20%), a 4PEPS of 6 to 12 corresponds to a moderate c-PTP (less than 65%), and a 4PEPS greater than 12 corresponds to a high c-PTP (65% or greater) (Table 1).

2.2. Source of patients

We used the database from Perrier et al.’s [8] study titled “Multidetector-Row Computed Tomography in Outpatients with Suspected Pulmonary Embolism.” The methods are detailed in the original paper. This database was not included in the original 4PEPS study. In brief, these included patients presenting to the ED with a clinical suspicion of PE with prospectively collected data. For example, variables such as “most likely diagnosis of PE” were collected at the time of the original. Exclusions included contraindication to CT contrast, creatinine clearance < 30 mL/min by Cockcroft–Gault, pregnancy, ongoing anticoagulation, life expectancy less than 3 months, unable to have follow-up, known diagnosis prior to presentation, absence of peripheral venous access, inability to undergo CT scan due to hemodynamic instability, and other reasons. The study was conducted between 2002 and 2003 in France and Switzerland and was designed as a prospective management trial with a 3-month follow-up. Ethics approval was previously obtained in the original study.

2.3. Study design

The 4PEPS was applied to this prospectively collected patient database. Patients with missing data required for calculating the 4PEPS were excluded, which included variables such as oxygen, heart rate, and oxygen saturation.

The safety threshold of the 4PEPS strategy was defined as a function of PE prevalence as per the International Society on Thrombosis and Haemostasis recommendations (1.82 + [0.00528 × prevalence]) [9]. PE prevalence in our external validation cohort was 26%; therefore, the upper limit of our CI for failure rate was 1.95%. Failure of the 4PEPS was defined as a confirmed PE at initial testing or during follow-up in a patient with a negative 4PEPS rule.

We also retroactively applied the revised Geneva and age-adjusted D-dimer (ADJUST-PE) strategies to our external validation cohort in the same manner as the original 4PEPS study [7]. Additionally, we applied the PERC and YEARS study strategies [3,10]. We compared the rates of negative D-dimer tests and the rate of CT pulmonary angiography (CTPA) use to determine the relative efficacy of the 4PEPS.

The application of the revised Geneva score was done by computing the score as per the original paper; those with a score of 0 to 10 required a D-dimer with a cutoff of <500 ng/mL [4]. The application of the age adjusted D-dimer strategy was done using the revised Geneva score, but the D-dimer cutoff was calculated as patient’s age × 10 (in ng/mL) in patients aged more than 50 years [11].

2.4. Statistical analysis

Continuous variables were reported using mean and SD, and categorical variables as number of patients and proportions. The 95% CIs using the mid-P exact value were calculated using OpenEpi version 2, an open-source calculator. The receiver operative characteristic curve analysis of the 4PEPS for PE diagnosis was done using IBM SPSS version 29.0.

3. Results

There were 756 patients in our database, of whom 22 (2.9%) were excluded due to missing variables, leading to a total of 734 patients. The mean (± SD) age was 60 ± 19 years, and 292 (40%) were males. The characteristics of the patients are displayed in Table 2. The overall prevalence of PE was 26% after exclusion.

Table 2.

Characteristics of patients.

Demographics	n (%)a
Total, N	734
Age, y, mean ± SD	60 ± 19
Male sex	292 (40)
History
Hormone replacement therapy	51 (7)
History of venous thromboembolism	140 (19)
Active malignancy	73 (10)
Chronic respiratory disease	77 (10)
Chronic heart failure	56 (8)
Immobility within 4 weeks	50 (7)
Pregnancy or peripartum within 4 weeks	13 (2)
Signs and symptoms
Chest pain	466 (63)
Dyspnea	524 (71)
Chest pain and dyspnea	299 (41)
Syncope	182 (25)
Suspected deep vein thrombosis	71 (10)
Hemoptysis	35 (5)
Heart rate, beats per minute, mean ± SD	88 ± 20
Systolic blood pressure, mm Hg, mean ± SD	138 ± 24
Oxygen saturation, %, mean ± SD	95 ± 5
Temperature, °C, mean ± SD	37.3 ± 1
PE most likely diagnosis	287 (39)
Prevalence of PE	193 (26)

PE, pulmonary embolism.

^aUnless otherwise specified.

In Table 3, each probability group is displayed with further subdivisions with their exact score, PE prevalence, and rate of false negatives. Figure 1 shows a graphical representation of each group. Altogether, in the very low c-PTP group (score < 0), there were 5 out of 90 (5.6%) patients diagnosed with PE. Within this group, 2 patients had a subsegmental PE, 2 a segmental PE, and 1 had a central PE. Detailed clinical characteristics of these patients are displayed in Appendix Table. In the low c-PTP group (score 0-5), there were 34 out of 363 patients (9.4%) diagnosed with PE. There were 142 (39%) patients in this low c-PTP group with a D-dimer >1000 ng/mL. In the moderate c-PTP group (score 6-12), there were 246 patients, of which 124 (50%) were diagnosed with PE. There were 215 (87%) patients with a D-dimer greater than the age-adjusted cutoff. Finally, in the high c-PTP group (score ≥ 13), there were 35 patients, of which 30 (86%) were diagnosed with PE.

Table 3.

4PEPS score of patients with pulmonary embolism prevalence.

4PEPS category	Score	N	PE prevalence	PE prevalence (%)	False negatives
Very low c-PTP (PE can be ruled out)	≤-3	10	0	0	-
	-2	26	1	4	1
	-1	54	4	7	4
Low c-PTP (PE ruled out if D-dimer <1.0 μg/mL)	0	78	2	3	2
	1	54	5	9	-
	2	61	4	7	-
	3	60	5	8	1
	4	51	7	14	-
	5	59	11	19	-
Moderate c-PTP (PE ruled out if D-dimer <0.5 μg/mL or age adjusted <age × 0.01 μg/mL)	6	55	11	20	-
	7	47	20	43	-
	8	42	28	67	-
	9	32	15	47	-
	10	38	27	71	1
	11	12	7	58	-
	12	20	16	80	-
High c-PTP (requires further imaging)	13	15	12	80	-
	14	11	10	91	-
	≥15	9	8	89	-

4PEPS, 4-level clinical pretest probability score; c-PTP, clinical pretest probability; PE, pulmonary embolism.

Figure 1.

Four-level clinical pretest probability score (4PEPS) score stratified by D-dimer. PE, pulmonary embolism.

Overall, 9 out of 734 patients (1.2%; 95% CI, 0.59%-2.23%) were diagnosed with PE despite a negative 4PEPS rule: 5/90 in the very low c-PTP group (5.6%; 95% CI, 2.4%-12.4%), 3/363 in the low c-PTP group (0.8%; 95% CI, 0.2%-2.2%), 1/246 in the moderate c-PTP group (0.4%; 95% CI, 0.0%-2.0%), and none in the high c-PTP group.

A nonhigh revised Geneva score in combination with a D-dimer below 500 ng/mL had a failure rate of 0% (95% CI, 0.0%-0.4%). A nonhigh revised Geneva score, in combination with a negative age-adjusted D-dimer, had a failure rate of 0.1% (95% CI, 0.0%-0.7%).

Table 4 displays the comparison between the 4PEPS score and other clinical strategies. There was a reduction in the number of required D-dimer tests with 4PEPS compared with the strategy using the revised Geneva score: 609 (83%) vs 677 (92%), a difference of -9%. There was also a reduction in the number of required CTPA when using the 4PEPS. The revised Geneva score required 516 (70%) CTPA, the adjusted D-dimer strategy required 476 (65%) CTPA, the PERC score required 479 (65%) CTPA, the YEARS strategy required 437 (59%) CTPA, and 4PEPS required 392 (53%) CTPA. The absolute difference in reduction of utilizing CTPA between 4PEPS and revised Geneva score was -17%.

Table 4.

Comparison of clinical strategies between 4PEPS, revised Geneva, PERC, YEARS, and age-adjusted D-dimer.

Strategy	D-dimer (%)a	CTPA (%)	Failure rate (%)	95% CI of failure rate
Revised Geneva	677 (92)b	516 (70)	0 (0)c	0%-0.4%
ADJUST-PE	677 (92)	476 (65)	1 (0.1)	0.0%-0.7%
PERC	648 (88)	479 (65)	7 (0.9)	0.4%-1.9%
YEARS	734 (100)	437 (59)	2 (0.3)	0.05%-0.9%
4PEPS	609 (83)	392 (53)	9 (1.2)	0.6%-2.2%

4PEPS, 4-level clinical pretest probability score; ADJUST-PE; age-adjusted D-dimer cutoff to rule out pulmonary embolism study; CTPA, computed tomography pulmonary angiography; PERC, pulmonary embolism rule-out criteria study; YEARS, simplified diagnostic management of pulmonary embolism study.

^aPercentages refer to the total population, N = 734.

^bThe total number of D-dimers was lower than the total sample because a revised Geneva score of >10 or 4PEPS with ≥13 points went directly to imaging.

^cThe revised Geneva score had no false-negative rates because, in our retrospective application, all patients with a score of 0 had further investigations, such as a D-dimer.

The area under the curve for the receiver operating characteristic curve analysis of the 4PEPS for PE diagnosis was 0.85 (95% CI, 0.82-0.88) (Figure 2).

Figure 2.

Area under the receiver operating characteristic (ROC) curve for the 4-level clinical pretest probability score (4PEPS) for pulmonary embolism (PE) diagnosis.

4. Discussion

Our study provides further external validation of the recently published 4PEPS strategy. We confirm a strong correlation between 4PEPS and PE prevalence. In addition, the 4PEPS was shown to reduce the amount of D-dimer or CT testing when compared to other strategies. The overall failure rate, when applied to our high PE prevalence population, was 1.2% (95% CI, 0.6%-2.2%). However, we observed a notably high failure rate of 5.6% (95% CI, 2.4%-12.4%) in patients among the very low c-PTP group.

There are a few potential reasons why our results differ from the original 4PEPS study, in particular, the higher rate of PE diagnosis in the very-low c-PTP group. Further characteristics of these false-negative patients are depicted in the Supplementary Table. The 4PEPS would have “missed” a main pulmonary artery PE, 2 segmental PE, and 2 subsegmental PE in the very low c-PTP group. Admittedly, given the retrospective analysis of our analysis, we cannot predict the outcome of the patients with missed PE if they had not been scanned. Interestingly, the 3 patients with a segmental or more proximal PE were PERC positive, while the 2 with subsegmental PE were PERC negative. Although during the derivation of 4PEPS, variables from the PERC score were considered, they were used in a different way where all variables need to be absent for a patient to be PERC negative. Another point to consider is that in our study, the likelihood of an alternative diagnosis to PE, which carries an important weight in the 4PEPS, was prospectively collected at inclusion but was not “decisional,” meaning that it had no impact on patients’ management. One could hypothesize that physicians using the 4PEPS will pay more attention to this item when using their “gestalt” assessment, given that no testing is to be performed on very low-risk patients. Physicians might be reluctant/selective in judging that an alternative diagnosis is more likely than PE given the impact on further testing. Of note, PE was not considered as the most likely diagnosis in any of our 5 false-negative patients. Lastly, our cohort was recruited in 2002 to 2003 when the prevalence of PE was higher—26% in our study—compared to 21% and 11% in the 2 validation cohorts used by Roy et al. [7] and a current average prevalence of PE of 14% in Europe [[12], [13], [14]]. Despite the high prevalence in our cohort, our study can provide valuable insights regarding the suitability of using the 4PEPS in high-prevalence settings. Furthermore, considering the lower disease prevalence in certain populations, a negative test result using the 4PEPS would yield an improved negative predictive value.

The 4PEPS publication and our external validation raise again an emerging debate in the literature on the reporting of the efficiency and safety of newer diagnostic strategies. Indeed, recent publications on PE diagnostic algorithms have reported the overall failure rate of their strategy rather than failure rates in patients in whom the diagnostic management was modified by the new strategy, eg, the subgroup of patients with a low c-PTP and a D-dimer between 500 ng/mL and 1000 ng/mL in the PEGeD study or the subgroup of patients with no YEARS item and a D-dimer between 500 ng/mL and 1000 ng/mL in the YEARS study. In the case of the 4PEPS study, subgroups or particular interest would be the very-low c-PTP group as a whole and the subgroup of low c-PTP patients with a D-dimer between the age-adjusted cutoff and 1000 ng/mL. In the correspondence following publication of the 4PEPS, Freund et al. [15] suggested that the safety of the 4PEPS should be tested within each probability category, commenting particularly on the subgroup of patients with a very low c-PTP. The overall failure rate “dilutes” the actual impact of the strategy by including many patients in whom: 1) the new strategy is not used, 2) the safety has already been established (for example, patients with nonhigh PTP and D-dimer below 500 ng/mL). A working group for the International Society of Thrombosis and Hemostasis Sub Standardization Committee on Diagnostic and Predictive Variables for VTE is currently drafting guidelines for the reporting of diagnostic management studies.

There are other limitations in our study. Our study sample was relatively limited, resulting in wide CIs, particularly when looking at subgroups. There were 22 of 756 patients (2.9%) with missing variables required in calculating the 4PEPS who were excluded. As already mentioned, this was a retrospective analysis of a former diagnostic cohort constituted 20 years ago, and the clinical outcome of false-negative patients, should they have not been diagnosed and treated, cannot be predicted. We also recognize a 19% prior history of VTE in our dataset, with 39% of these patients diagnosed with a recurrent VTE at inclusion. The proportion of patients with a previous VTE among those presenting with a suspected PE is likely lower nowadays (eg, it was 8% in the PEGeD study) due to broadening of indications for extended duration of anticoagulation after a first VTE episode [6]. None of these were among the false negatives, likely due to the inclusion of a history of VTE in the 4PEPS.

It is important to note that during the period when this study was conducted in Europe, regulations prohibited the collection of information on race or ethnicity as part of research involving human subjects. Consequently, we were unable to include data on the race/ethnicity of the study participants in Table 2. This limitation is reflective of the legal and ethical constraints that were in place at that time. The absence of race/ethnicity information is acknowledged as a limitation, as these socio-cultural determinants of health can play a significant role in influencing health outcomes and patterns of disease. We encourage future studies to consider the inclusion of race/ethnicity information where possible, as it can contribute to a more nuanced understanding of health disparities and outcomes.

In conclusion, the 4PEPS aims to refine the process in how we diagnose PE in the ED setting and has an initially promising outlook. Our data provide further validation of the 4PEPS; however, the higher-than-expected failure rate observed among patients in the very-low c-PTP group raises concerns about its applicability in this specific subset of the population. A prospective management study is currently being planned and will provide more definitive validation data on the role of the 4PEPS in our toolkit for PE diagnosis.