Performance of the ESC/ERS 4-strata risk stratification model for pulmonary arterial hypertension with missing variables

Extract

The 2022 European Society of Cardiology (ESC)/European Respiratory Society (ERS) pulmonary hypertension (PH) guidelines recommend risk stratification to guide treatment decisions in patients with pulmonary arterial hypertension (PAH) [1, 2]. Risk, referring to the likelihood of death within 12 months, is estimated based on World Health Organization (WHO) functional class (FC), 6-min walk distance (6MWD), and serum levels of brain natriuretic peptide (BNP) or N-terminal pro-BNP (NT-proBNP), using prespecified thresholds. A 3-strata model (categorising risk as low, intermediate or high) is recommended at time of diagnosis, whereas a 4-strata model (categorising risk as low, intermediate–low, intermediate–high or high) is recommended during the course of the disease. The 4-strata model was developed from the Comparative, Prospective Registry of Newly Initiated Therapies for Pulmonary Hypertension (COMPERA) and cross-validated by the French PH Registry [3, 4]. Both studies included patients when all three variables, i.e. WHO-FC, 6MWD and BNP/NT-proBNP were available at baseline. No formal assessment of the predictive value of the 4-strata model has been performed when one of these components was missing.

Tweetable abstract

In patients with PAH, the ESC/ERS 4-strata risk model maintains some of its discriminative power when one of the three parameters (WHO-FC, 6MWD, and BNP/NT-proBNP) is missing, although with an increasing likelihood of overestimating risk https://bit.ly/46mz83d

To the Editor:

The 2022 European Society of Cardiology (ESC)/European Respiratory Society (ERS) pulmonary hypertension (PH) guidelines recommend risk stratification to guide treatment decisions in patients with pulmonary arterial hypertension (PAH) [1, 2]. Risk, referring to the likelihood of death within 12 months, is estimated based on World Health Organization (WHO) functional class (FC), 6-min walk distance (6MWD), and serum levels of brain natriuretic peptide (BNP) or N-terminal pro-BNP (NT-proBNP), using prespecified thresholds. A 3-strata model (categorising risk as low, intermediate or high) is recommended at time of diagnosis, whereas a 4-strata model (categorising risk as low, intermediate–low, intermediate–high or high) is recommended during the course of the disease. The 4-strata model was developed from the Comparative, Prospective Registry of Newly Initiated Therapies for Pulmonary Hypertension (COMPERA) and cross-validated by the French PH Registry [3, 4]. Both studies included patients when all three variables, i.e. WHO-FC, 6MWD and BNP/NT-proBNP were available at baseline. No formal assessment of the predictive value of the 4-strata model has been performed when one of these components was missing.

In the present study, we sought to determine the prognostic performance of the 4-strata model when any of the three variables (WHO-FC, 6MWD and BNP/NT-proBNP) were unavailable. From the COMPERA database, we selected patients based on the following criteria: 1) diagnosis of PAH within 6 months prior to inclusion, confirmed by right heart catheterisation showing a mean pulmonary arterial pressure (mPAP) ≥25 mmHg, a pulmonary artery wedge pressure ≤15 mmHg and a pulmonary vascular resistance (PVR) >3 Wood units (WU); and 2) data on WHO-FC, 6MWD and BNP/NT-proBNP available at baseline. The dataset as of 1 May 2023 was analysed. No imputations were made for missing data.

Risk was calculated at baseline and first follow-up based on all three parameters (WHO-FC, 6MWD and BNP/NT-proBNP) as described previously [3, 4]. To assess the impact of missing data, each of the three parameters was set to missing once for all patients. Risk scores were calculated as the mean of the assigned points of the available parameters, rounded to the nearest integer. Vital status was ascertained by on-site visits or phone calls to the patients or their caregivers. Patients who underwent lung transplantation and patients who were lost to follow-up were censored at the date of the last contact. Cox proportional hazards models were calculated with risk as predictor variable. Harrell's concordance statistics (C-index) were used to evaluate the impact of missing parameters for risk score calculation. In addition, we calculated the proportion of patients who were reclassified, i.e. who had a change in risk category when one of the parameters was missing [5]. The Kaplan–Meier method was used to estimate survival according to risk at baseline and first follow-up (with survival time starting at first follow-up for the latter analysis). All statistical analyses were performed using R version 4.1.3.

The present analysis included baseline data of 1976 patients who were enrolled between 1 January 2009 and 31 December 2022. Mean±sd age was 65±16 years, 64.6% were female; 70.5% of the patients were diagnosed with idiopathic, heritable or drug-associated PAH, 19.8% with connective tissue disease-associated PAH, and the remaining patients with other forms of PAH. At baseline, mPAP was 43.4±12.1 mmHg, pulmonary artery wedge pressure 9.3±3.4 mmHg, cardiac index 2.2±0.7 L·min⁻¹·m⁻² and PVR 9.3±5.0 WU. The 6MWD was 293±129 m, and 72.4% of the patients presented in WHO-FC III. Initial PAH therapy consisted of monotherapy (77.8%) or combination therapy (22.2%).

For risk assessment at baseline, Harrell's C-index was 0.636 (95% CI 0.618–0.653). When WHO-FC, 6MWD or BNP/NT-proBNP were excluded, the C-indices were 0.644 (95% CI 0.626–0.662), 0.603 (95% CI 0.586–0.621) and 0.629 (95% CI 0.611–0.647), respectively.

The median interval between baseline assessment and first follow-up was 4.1 months (interquartile range 3.4–5.5 months). Risk assessment at first follow-up was based on a subset of 958 patients, for whom all three parameters were available. The baseline characteristics of the patients in this subset were similar to those of the complete baseline cohort (data are not shown but can be obtained from the authors upon request). Harrell's C-index at first follow-up was 0.691 (95% CI 0.664–0.717). When WHO-FC, 6MWD or BNP/NT-proBNP were excluded, the C-indices were 0.699 (95% CI 0.671–0.726), 0.672 (95% CI 0.644–0.701) and 0.668 (95% CI 0.641–0.696), respectively. Discrimination between the 4 risk strata remained similar to the complete dataset, except for the distinction between the intermediate–low and intermediate–high risk cohorts, which was less clear when 6MWD data were missing (figure 1).

FIGURE 1.

Kaplan–Meier survival estimates based on four risk strata obtained at first follow-up when a) all parameters (World Health Organization (WHO) functional class (FC), 6-min walk distance (6MWD) and brain natriuretic peptide (BNP)/N-terminal pro-BNP (NT-proBNP)) were available, b) when WHO-FC was missing, c) when 6MWD was missing and d) when BNP/NT-proBNP was missing.

Risk category was reclassified in a sizable proportion of patients when one of the three parameters were missing, although by no more than one category. At baseline, risk category was modified in 32.0%, 45.4% and 26.3%, respectively, of the patients, when WHO-FC, 6MWD or BNP/NT-proBNP were missing. The risk category increased in 30.0%, 44.4% and 20.3%, respectively, of the patients, and decreased in 2.1%, 1.0%, and 6.0%, respectively, of the patients. At first follow-up, the risk category changed in 38.4%, 35.2% and 29.2%, respectively, of the patients, when WHO-FC, 6MWD or BNP/NT-proBNP were missing. The risk category increased in 36.4%, 32.8% and 21.3%, respectively, of the patients, and decreased in 2.0%, 2.4% and 7.9%, respectively, of the patients. Thus, missing variables increase the likelihood of overestimating risk, while the likelihood of underestimating risk was relatively low, which can be explained mathematically as the mean value of two variables often ends with .5, which is rounded to the next integer.

Limitations of this study include the selection of patients who had all three risk parameters available at baseline and a subset of these patients in whom these parameters were also available at first follow-up, which could have created a selection bias. In addition, this analysis included only patients with a mPAP ≥25 mmHg and a PVR >3 WU, not considering the recent revision of the haemodynamic criteria for precapillary PH [1, 2].

Our data demonstrate that the ESC/ERS 4-strata risk model maintains some of its discriminative power when one of the three parameters (WHO-FC, 6MWD or BNP/NT-proBNP) is unavailable. Furthermore, our findings confirm previous observations suggesting that risk stratification during follow-up holds greater predictive value than risk prediction at baseline. This is likely due to the impact of PAH therapy on survival. While these results align with previous observations from the REVEAL risk scores [6], we noted an increased tendency to overestimate risk in a significant proportion of patients when one of the key variables was missing. It is important to underscore that risk stratification based on WHO-FC, 6MWD and BNP/NT-proBNP allows only for a basic assessment that needs to be complemented by additional variables to guide individual decision-making. This is particularly crucial when not all parameters are available.

Shareable PDF

This one-page PDF can be shared freely online.

Shareable PDF ERJ-01023-2023.Shareable ^{(280.7KB, pdf)}