Abstract
Background
Achievement of low-risk status is a treatment goal in pulmonary arterial hypertension (PAH). Risk assessment often is performed using multiparameter tools, such as the Registry to Evaluate Early and Long-Term PAH Disease Management (REVEAL) risk calculator. Risk calculators that assess fewer variables without compromising validity may expedite risk assessment in the routine clinic setting. We describe the development and validation of REVEAL Lite 2, an abridged version of REVEAL 2.0.
Research Question
Can a simplified version of the REVEAL 2.0 risk assessment calculator for patients with PAH be developed and validated?
Study Design and Methods
REVEAL Lite 2 includes six noninvasive variables—functional class (FC), vital signs (systolic BP [SBP] and heart rate), 6-min walk distance (6MWD), brain natriuretic peptide (BNP)/N-terminal prohormone of brain natriuretic peptide (NT-proBNP), and renal insufficiency (by estimated glomerular filtration rate [eGFR])—and was validated in a series of analyses (Kaplan-Meier, concordance index, Cox proportional hazard model, and multivariate analysis).
Results
REVEAL Lite 2 approximates REVEAL 2.0 at discriminating low, intermediate, and high risk for 1-year mortality in patients in the REVEAL registry. The model indicated that the most highly predictive REVEAL Lite 2 parameter was BNP/NT-proBNP, followed by 6MWD and FC. Even if multiple, less predictive variables (heart rate, SBP, eGFR) were missing, REVEAL Lite 2 still discriminated among risk groups.
Interpretation
REVEAL Lite 2, an abridged version of REVEAL 2.0, provides a simplified method of risk assessment that can be implemented routinely in daily clinical practice. REVEAL Lite 2 is a robust tool that provides discrimination among patients at low, intermediate, and high risk of 1-year mortality.
Trial Registry
ClinicalTrials.gov; No.: NCT00370214; URL: www.clinicaltrials.gov;
Key Words: calculator, pulmonary arterial hypertension, risk, risk score
Take-home Points.
Study Question: To develop and validate a simplified version of the REVEAL 2.0 risk assessment calculator for patients with PAH.
Results: REVEAL Lite 2, an abridged version of REVEAL 2.0 that uses six rather than 13 variables, approximates REVEAL 2.0 at discriminating low, intermediate, and high risk for 1-year mortality in patients in the REVEAL Registry.
Interpretation: REVEAL Lite 2 provides a simplified and robust method of risk assessment for implementation in routine clinical practice.
FOR EDITORIAL COMMENT, SEE PAGE 14
Despite advances in the treatment of pulmonary arterial hypertension (PAH; World Health Organization [WHO] group 1 pulmonary hypertension), no cure exists for this progressive and ultimately fatal disease. However, with timely and effective clinical intervention, clinical status and survival are improved. The current goal of PAH treatment is to enable patients to achieve a low mortality risk status, which has been associated with improved outcomes.1, 2, 3, 4, 5 To enable such outcomes, assessments of mortality risk should be made at PAH diagnosis and at regular intervals during follow-up. The results of these assessments should be used to guide management, including proactive adjustment of treatment if a low mortality risk status is not achieved.1 , 5
Current best practice is for risk assessments to be made using multiparameter risk assessment tools, such as the Registry to Evaluate Early and Long-Term PAH Disease Management (REVEAL) risk calculator versions 1.0 or 2.0,6 , 7 the Comparative, Prospective Registry of Newly Initiated Therapies for Pulmonary Hypertension (COMPERA) method,8 the Swedish PAH Register method,9 the French Pulmonary Hypertension Registry (FPHR) method,10 and the Bologna strategy.11 REVEAL 1.0 and 2.0 estimate PAH mortality risk by assigning scores using up to 12 or 13 variables, respectively. The scores are used to categorize patients into specific risk strata.6 , 7 REVEAL 2.0 incorporates new variables and expanded thresholds from REVEAL 1.0 to improve risk discrimination. The COMPERA, FPHR, and Bologna methods use data from up to six variables and assign mortality risk based on thresholds published in the European Society of Cardiology/European Respiratory Society (ESC/ERS) pulmonary hypertension guidelines.8 , 10 , 11 Of clinical importance, REVEAL 2.0, when compared with COMPERA and FPHR, showed greater risk discrimination than either of the two ESC/ERS-based risk assessment strategies.7
The need for timely and regular risk assessment in PAH is acknowledged widely1 , 5 , 10 , 12 , 13; however, real-world evidence indicates that risk assessment in the clinical setting is suboptimal.14 Several barriers to practical implementation have been documented, including the complexity of tools,15 the number of parameters that need to be included (with a reported 41% of patients excluded from risk calculation analysis because of insufficient measurements), and a desire to avoid potentially unnecessary invasive procedures.14
To expedite risk assessment in the clinic, where comprehensive data for all patients may be lacking and time constrained, risk assessment tools using fewer variables may be preferable. To this end, we developed two simplified risk calculators, REVEAL Lite 1 and REVEAL Lite 2. Both are based on the recently developed and validated REVEAL 2.0 risk calculator,7 , 16 , 17 but in an abridged format. REVEAL Lite 1 uses only nine noninvasive variables, whereas REVEAL Lite 2 uses only six modifiable and noninvasive variables.18 Herein, we present results from analyses conducted during development and internal validation of REVEAL Lite 2.
Methods
Patient Characteristics
REVEAL Lite 2 was developed using data from patients enrolled in the REVEAL (final database lock, February 4, 2013). The same patient population used for REVEAL 2.0 development was used for the current REVEAL Lite 2 analysis. Patients enrolled in REVEAL were eligible if they were 18 years of age or older at diagnosis, met hemodynamic criteria for PAH (ie, pulmonary capillary wedge pressure ≤ 15 mm Hg), and had ≥ 12 months of follow-up data available. This enabled the capture of all-cause hospitalization data from the previous 6 months for development of the REVEAL 2.0 tool. One year after enrollment was considered baseline for these analyses. Patients were excluded from the analyses if they were participating in a blinded clinical trial at enrollment or if they received a lung transplant within 1 year of enrollment (e-Appendix 1; e-Fig 1).
Risk Assessment Tools
The relevant parameters and variables and associated scoring included in the REVEAL 2.0 and REVEAL Lite 2 risk calculators are presented in Table 1 . REVEAL Lite 2 is based on REVEAL 2.0, but includes only six noninvasive and modifiable parameters: New York Heart Association (NYHA) or WHO functional class (FC), vital signs (systolic BP [SBP] and heart rate), 6-min walk distance (6MWD), brain natriuretic peptide (BNP)/N-terminal prohormone of brain natriuretic peptide (NT-proBNP), renal insufficiency (if estimated glomerular filtration rate [eGFR] <60 mL/min/1.73 m2 or reported as “renal insufficiency,” as assessed by the principal investigator when eGFR was unavailable). For both REVEAL 2.0 and REVEAL Lite 2, patients were grouped into three risk categories according to ESC/ERS guidelines.1 , 7 For the REVEAL 2.0 assessment, and based on the 1-year mortality outcomes in the REVEAL derivation data (with scores ranging from 0 to 23), a score between 0 and 6 was considered low risk, a score of 7 or 8 was considered intermediate risk, and a score of 9 or higher was considered high risk.7 For the REVEAL Lite 2 assessment (with scores ranging from 1 to 14), a score between 1 and 5 was considered low risk, a score of 6 or 7 was considered intermediate risk, and a score of 8 or higher was considered high risk. Risk was calculated for both risk calculators using data from a subpopulation of the REVEAL who had survived ≥ 1 year after enrollment. To provide proper reference for REVEAL Lite 2 to REVEAL 2.0, the same dataset used for REVEAL 2.0 was also used for REVEAL Lite 2. REVEAL Lite 2 scores at time of enrollment were recalculated for patients included in this analysis. A correction factor of 6 was used for REVEAL Lite 2 calculations.
Table 1.
Variables Included in the REVEAL 2.0 and REVEAL Lite 2 Risk Calculators and Associated Risk Scores
| Parameter | REVEAL 2.0 (13 Variables) | REVEAL Lite 2 (6 Variables) |
|---|---|---|
| Cause | Connective tissue disease: +1 Portopulmonary hypertension: +3 Heritable: +2 |
— |
| Demographics | Men > 60 y: +2 | — |
| Renal insufficiency | eGFR < 60 mL/min/1.73 m2 or defined by clinical judgment if eGFR is not available: +1 | |
| NYHA or WHO FC | FC I: −1 FC III: +1 FC IV: +2 |
|
| All-cause hospitalization within the previous 6 mo | +1 | — |
| Vital signs | SBP < 110 mm Hg: +1 HR > 96 bpm: +1 |
|
| 6MWD | ≥ 440 min: −2 320-< 440 min: −1 < 165 min: +1 |
|
| BNP/NT-proBNP | BNP < 50 pg/mL OR NT-proBNP < 300 pg/mL: −2 BNP 200-< 800 pg/mL: +1 BNP ≥800 pg/mL OR NT-proBNP ≥1100 pg/mL: +2 |
|
| Echocardiogram | Pericardial effusion: +1 | — |
| Pulmonary function test | % predicted Dlco < 40%: +1 | — |
| RHC within 1 y | mRAP > 20 mm Hg: +1 PVR < 5 Wood units: −1 |
— |
| Total score | Sum of above scores +6 | Sum of above scores +6 |
Em dashes denote parameter not included in REVEAL Lite 2.
6MWD = 6-min walk distance; BNP = brain natriuretic peptide; bpm = beats per minute; Dlco = diffusing capacity of the lungs for carbon monoxide; eGFR = estimated glomerular filtration rate; FC = functional class; HR = heart rate; mRAP = mean right atrial pressure; NT-proBNP = N-terminal prohormone of brain natriuretic peptide; NYHA = New York Heart Association; PAH = pulmonary arterial hypertension; PVR = pulmonary vascular resistance; REVEAL = Registry to Evaluate Early and Long-Term PAH Disease Management; RHC = right heart catheterization; SBP = systolic BP; WHO = World Health Organization.
Statistical Methods
REVEAL Lite 2 is based on earlier versions of the REVEAL risk assessment tools 1.0 and 2.0. Detailed descriptions of the statistical methods used in their development have been described previously for REVEAL 1.06 and REVEAL 2.0.7 Patient data, definitions, and algorithm of derivations from the development of REVEAL 2.0 were used in the current analysis, in which baseline risk was calculated based on the last available assessment at 12 months’ follow-up or an earlier time point, starting from enrollment. A score of zero was assigned for missing individual assessments.
The six noninvasive and modifiable parameters were classified onto categorical values according to the REVEAL 2.0 risk calculator: NYHA FC (−2, 0, 1, 2), SBP (0, 1), heart rate (0, 1), 6MWD (−2, −1, 0, 1), BNP/NT-proBNP (−2, 0, 1, 2), renal insufficiency (0, 1). The Cox proportional hazard model with the six parameters as independent variables and survival time as the dependent variable was used to derive the prognostic equation, in which stepwise selection was used to rank the impact of these prognostic parameters. The Cox proportional hazard model was used to compare the survival rate between risk groups, Harrell’s concordance statistic (c-index) was used as a goodness-of-fit measure, and the associated 95% CIs were used to evaluate the discrimination of the risk assessment tools.
The Kaplan-Meier method was used to estimate 1-year survival from baseline for each of the risk score groups for both risk calculators (REVEAL 2.0 and REVEAL Lite 2). Simple κ values were calculated to examine the agreement between REVEAL 2.0 and REVEAL Lite 2 on risk group classifications. C-indexes were used to evaluate the impact of missing factors when one or more individual factors were missing from the model on the discrimination of REVEAL Lite 2. All analyses were conducted using SAS version 9.4 software (SAS Institute).
Results
In total, 2,529 of the 3,515 patients enrolled in REVEAL Registry were eligible for inclusion in our analyses (e-Fig 1). Proportions of patients with available data for each variable and handling of missing data were reported previously.7 Patient demographics and clinical characteristics for these patients at 1 year after enrollment are presented in Table 2 . Approximately 50% of patients had idiopathic PAH (IPAH) and 25% had connective tissue-associated PAH (CTD-PAH). Most patients (approximately 87%) were classified as NYHA FC II/III.
Table 2.
Patient Demographics and Clinical Characteristics at 1 Year After Enrollment
| Characteristic | Patients With 1 y of Follow-up (N = 2,529) |
|---|---|
| Age, mean (SD), y | 53.6 (14.3) |
| Sex, No. (%) | … |
| Male | 505 (20.0) |
| Female | 2,024 (80.0) |
| Race, No. (%) | … |
| White | 1,809 (71.5) |
| Black | 330 (13.0) |
| Hispanic | 228 (9.0) |
| Asian or Pacific Islander | 85 (3.4) |
| Native American or Native Alaskan | 16 (0.6) |
| Other | 22 (0.9) |
| Unknown | 39 (1.5) |
| WHO group I PAH subgroup, No. (%) | … |
| Idiopathic | 1,171 (46.3) |
| Heritablea | 74 (2.9) |
| Other | 18 (0.7) |
| PAH associated with | … |
| Connective tissue disease | 649 (25.7) |
| Congenital heart disease | 244 (9.6) |
| Portal hypertension | 139 (5.5) |
| HIV | 48 (1.9) |
| Other | 186 (7.4) |
| Modified NYHA or WHO FC, No. (%)b | … |
| I | 203 (8.4) |
| II | 1,003 (41.3) |
| III | 1,116 (45.9) |
| IV | 108 (4.4) |
FC = functional class; NYHA = New York Heart Association; PAH = pulmonary arterial hypertension; WHO = World Health Organization.
Some, but not all, had confirmed BMPR2 or ALK1 mutations.
Data were missing for 99 patients.
Estimation of 1-Year Mortality
Kaplan-Meier survival curves to 5 years by REVEAL 2.0 and REVEAL Lite 2 are shown in Figure 1A and 1B , respectively. Both demonstrate clear separation of risk between each risk stratum. The results of the Kaplan-Meier, hazard ratio, and c-index calculations for 1-year survival are presented in Table 3 . These data show that REVEAL Lite 2 approximates the “parent” REVEAL 2.0 risk calculator at discriminating among patients at low, intermediate, or high risk for 1-year mortality (based on c-index). This was the case regardless of whether the data were compared using categorical or numerical values. The c-indexes using categorical values were 0.73 (95% CI, 0.71-0.75) and 0.70 (95% CI, 0.68-0.72) for REVEAL 2.0 and REVEAL Lite 2, respectively. The c-indexes, using the original numerical values, were 0.76 (95% CI, 0.74-0.78) and 0.73 (95% CI, 0.71-0.75) for REVEAL 2.0 and REVEAL Lite 2, respectively. Because REVEAL 2.0 was developed based on data at 12 months of follow-up (as baseline), we also examined whether REVEAL Lite 2 provides consistent discrimination at the time of enrollment. When we applied REVEAL Lite 2 to value at enrollment (N = 3,046 PAH patients), the c-index was 0.71 (95% CI, 0.69-0.73), indicating good discrimination. We calculated the c-index for IPAH (n = 1,171) and CTD-PAH (n = 649) subgroups separately using REVEAL Lite 2. C-indexes, using the original numerical values, were 0.74 (95% CI, 0.71-0.77) and 0.76 (95% CI, 0.73-0.79) and, using categorical values, were 0.71 (95% CI, 0.68-0.74) and 0.72 (95% CI, 0.69-0.75) for IPAH and CTD-PAH, respectively.
Figure 1.
A, B, Kaplan-Meier survival curves obtained after baseline for Registry to Evaluate Early and Long-Term PAH Disease Management (REVEAL) 2.0 (A) and REVEAL Lite 2 (B). HR = hazard ratio vs low-risk group; int = intermediate; PCWP = pulmonary capillary wedge pressure.
Table 3.
Hazard Ratios and Concordance Indexes for Estimation of 1-Year Mortality
| Risk Assessment Strategy and Risk Group | No. of Patients (%) | Kaplan-Meier Estimated Mortality at 1 y, % (95% CI) | HR (95% CI) Compared With Low-Risk Group | C-Index (95% CI), Three-Category/Original |
|---|---|---|---|---|
| REVEAL 2.0 (N = 2,529) | ||||
| Low (score, ≤ 6) | 1,073 (42.4) | 1.9 (1.1-2.7) | NA | 0.73 (0.71-0.75)/0.76 (0.74-0.78) |
| Intermediate (score, 7-8) | 692 (27.4) | 6.5 (4.7-8.4) | 2.73 (2.2-3.4) | |
| High (score, ≥ 9) | 764 (30.2) | 25.8 (22.7-28.9) | 8.09 (6.6-9.9) | |
| REVEAL Lite 2 (N = 2,529) | ||||
| Low (score, ≤ 5) | 960 (38.0) | 2.9 (1.8-3.9) | NA | 0.70 (0.68-0.72)/0.73 (0.71-0.75) |
| Intermediate (score, 6-7) | 883 (34.9) | 7.1 (5.4-8.8) | 2.27 (1.8-2.8) | |
| High (score, ≥ 8) | 686 (27.1) | 25.1 (21.9-28.4) | 6.35 (5.2-7.8) | |
c-index = Harrell’s concordance statistic; HR, hazard ratio; NA = not applicable. See Table 1 legend for expansion of other abbreviation.
Prognostic Equation
Cox proportional hazard multivariate analysis showed that all variables and scores were independent prognosticators of survival and that higher risk score was associated with higher risk of death (e-Table 1). Predicted 1-year survival was computed as follows: S0(1)exp(Z′βγ), where S0(1) is the baseline survivor function (0.925), Z′β is the linear component, and γ is the shrinkage coefficient (0.976). The core of the prognostic equation is Z, the linear component of the Cox model (e-Table 2).
All parameters were found to be highly predictive (based on χ 2 value for individual variables), with the exceptions of heart rate and NYHA or WHO FC I (e-Table 1). However, it is important to note that only 203 patients (8.4%) in the analysis sample had NYHA or WHO FC I disease. The model indicated that the most highly predictive parameter was BNP/NT-proBNP (χ 2 value for high level, 56.4254; P < .0001), followed by 6MWD (χ 2 value for ≥ 440 m, 48.1825; P < .0001), and NYHA or WHO FC IV (χ 2 value, 38.0737; P < .0001).
Agreement Between Risk Calculators
Agreement between REVEAL 2.0 and REVEAL Lite 2 was examined using the simple κ method and is presented in Table 4 . With all patients included and missing variables scored as zero (analysis 1; N = 2,529), the agreement between the two methods using the simple κ method was good at 0.62 (good, 0.61-0.80). When risk scores were calculated based on patients without missing BNP/NT-proBNP or 6MWD values (analysis 2; n = 1,505), the κ value improved to 0.67 (and the lower boundary of the CI increased to 0.64). With either analysis 1 or 2, no patient was assessed as low risk using REVEAL 2.0 who was found to be at high risk using REVEAL Lite 2.
Table 4.
Effect of Missing Values on Agreement Between REVEAL 2.0 and REVEAL Lite 2
| Analysis | REVEAL Lite 2 |
Simple κ Value (95% CI) | ||
|---|---|---|---|---|
| Low | Intermediate | High | ||
| Analysis 1a | ||||
| REVEAL 2.0 | … | … | … | … |
| Low | 844 | 229 | 0 | |
| Intermediate | 102 | 477 | 113 | 0.62 (0.60-0.65) |
| High | 14 | 177 | 573 | |
| Analysis 2b | ||||
| REVEAL 2.0 | … | … | … | … |
| Low | 565 | 102 | 0 | |
| Intermediate | 61 | 242 | 58 | 0.67 (0.64-0.71) |
| High | 7 | 92 | 378 | |
See Table 1 legend for expansion of abbreviations.
All patients included (n = 2,529); missing values counted as 0.
Only patients with all values present for both BNP/NT-proBNP and 6MWD included (n = 1,505).
Handling of Missing Values
The impact of missing variables was evaluated using c-index. Scenarios of missing one, two, and three variables were used. The results are presented in Table 5 . REVEAL Lite 2 provided good discrimination between risk groups missing any one variable. As presented in the earlier section on prognostic equation, not all variables within REVEAL Lite 2 are of equal prognostic value; the three variables of most prognostic value are BNP/NT-proBNP, 6MWD, and NYHA or WHO FC. At baseline, BNP/NT-proBNP was missing in 797 patients, 6MWD was missing in 317 patients, and NYHA or WHO FC was missing in 99 patients (Table 5). When any one of six variables were missing, the c-index was ≥ 0.70, indicating good discrimination. Missing one of the variables having higher prognostic value was associated with greater reduction in discrimination. When one to three variables from the three least important variables were missing (ie, eGFR, SBP, heart rate, or a combination thereof), REVEAL Lite 2 still provided good discrimination between risk groups. However, when two variables from the top three variables (BNP/NT-proBNP, 6MWD, and NYHA or WHO FC) were missing, the c-index would be < 0.70, which is below the threshold for good discrimination.
Table 5.
Effect of Missing Values on Risk Discrimination Using REVEAL Lite 2a
| Lite 2 Model Description | Variables Included in the Model (N = 2,529; Missing Value Assumed of 0 Risk Score) | C-Index (95% CI) |
|---|---|---|
| All 6 parameters/variables included | BNP/NT-proBNP, 6MWD, NYHA or WHO FC, SBP < 110 mm Hg, HR >96 bpm, renal insufficiency | 0.73 (0.71-0.75) |
| Missing 1 parameter/variable | ||
| No BNP/NT-proBNP | 6MWD, NYHA/WHO FC, SBP <110 mm Hg, HR > 96 bpm, renal insufficiency | 0.70 (0.68-0.72) |
| No 6MWD | BNP/NT-proBNP, NYHA or WHO FC, SBP < 110 mm Hg, HR > 96 bpm, renal insufficiency | 0.71 (0.69-0.73) |
| No NYHA or WHO FC | BNP/NT-proBNP, 6MWD, SBP < 110 mm Hg, HR > 96 bpm, renal insufficiency | 0.72 (0.70-0.74) |
| No HR | BNP/NT-proBNP, 6MWD, NYHA or WHO FC, SBP < 110 mm Hg, renal insufficiency | 0.73 (0.71-0.75) |
| No renal insufficiency | BNP/NT-proBNP, 6MWD, NYHA or WHO FC, SBP < 110 mm Hg, HR > 96 bpm | 0.73 (0.71-0.75) |
| No SBP | BNP/NT-proBNP, 6MWD, NYHA or WHO FC, HR > 96 bpm, renal insufficiency | 0.73 (0.71-0.75) |
| Missing 2 parameters from top 3 parameters or variables BNP/NT-proBNP, 6MWD, and NYHA or WHO FC | ||
| Lite 2 (no BNP and 6MWD) | NYHA/WHO FC, SBP < 110 mm Hg, HR > 96, renal insufficiency | 0.67 (0.65-0.69) |
| Lite 2 (no NYHA and 6MWD) | BNP, SBP < 110 mm Hg, HR > 96, renal insufficiency | 0.68 (0.66-0.70) |
| Lite 2 (no NYHA and BNP) | 6MWD, SBP < 110 mm Hg, HR > 96, renal insufficiency | 0.69 (0.67-0.71) |
| Missing 2 parameters or variables from SBP, renal insufficiency, and HR | ||
| Missing SBP and renal insufficiency | BNP/NT-proBNP, 6MWD, NYHA or WHO FC, HR > 96 bpm | 0.72 (0.70-0.74) |
| Missing HR and SBP | BNP/NT-proBNP, 6MWD, NYHA or WHO FC, renal insufficiency | 0.73 (0.71-0.75) |
| Missing HR and renal insufficiency | BNP/NT-proBNP, 6MWD, NYHA or WHO FC, SBP < 110 mm Hg | 0.72 (0.70-0.74) |
| Missing SBP, renal insufficiency, and HR (3 parameters or variables) | BNP/NT-proBNP, 6MWD, NYHA or WHO FC | 0.72 (0.70-0.74) |
Discussion
REVEAL Lite 2 is a multiparameter risk assessment tool for patients with PAH that provides a simplified and robust risk calculator for routine clinical implementation. It is an abridged version of the REVEAL 2.0 risk calculator that uses six (rather than 13) exclusively noninvasive and modifiable variables. REVEAL Lite 2 closely approximates REVEAL 2.0 at discriminating among patients in the REVEAL Registry at low, intermediate, and high risk for 1-year mortality (based on the c-index), using either categorical values (low, intermediate, or high risk) or the original numerical values (risk scores). The model indicated that the most highly predictive parameter included in REVEAL Lite 2 (based on the χ 2 value) was BNP/NT-proBNP, followed by 6MWD and NYHA or WHO FC. Assessment of the level of agreement between REVEAL Lite 2 and REVEAL 2.0 was found to be good (when missing variables were scored as zero); agreement increased when recalculated using data only from patients without missing BNP/NT-proBNP or 6MWD values.
During the development of REVEAL Lite 2, we conducted a series of rigorous analyses to validate the results obtained using this abridged risk calculator by comparing them with those produced by the parent risk calculator, REVEAL 2.0. In the analysis of agreement between REVEAL 2.0 and REVEAL Lite 2, using data from all patients in the REVEAL cohort, including those with missing values, one of the most important findings was that no patients assessed as being at low risk using REVEAL 2.0 were found to be at high risk using REVEAL Lite 2; furthermore, only a small number of patients found to be at low risk using REVEAL 2.0 were assessed as being at intermediate risk using REVEAL Lite 2 (n = 229). Therefore, REVEAL Lite 2 has clinical usefulness in screening patients because it can be used as a relatively quick and simple method for accurately identifying patients predicted to have a low risk for 1 year of mortality.
Risk discrimination of patients enrolled in REVEAL using REVEAL Lite 2 seems more accurate than that obtained using risk assessment methods based on ESC/ERS guidelines thresholds. For example, when two versions of the FPHR method (four-variable and three-variable formats) were used to assess risk in the same subpopulation of patients from REVEAL, the c-indexes obtained were 0.61 (95% CI, 0.59-0.63) and 0.67 (95% CI, 0.65-0.68)11; these are both lower than the c-index obtained using REVEAL Lite 2 (0.70 [95% CI, 0.68-0.72]) (e-Table 3). This improved discrimination may be because REVEAL Lite 2 variables are weighted. In addition, neither the FPHR nor COMPERA method fully account for missing data: the FPHR method cannot be used if one variable is missing, whereas COMPERA takes the average for missing variables. In general, REVEAL Lite 2 provides the highest discrimination when all six variables are measured. When missing values exist, REVEAL Lite 2 still provides good discrimination if ≥ 50% of six variables are measured, including at least two variables from the three top prognostic variables. Missing variables are imputed as zero when calculating REVEAL Lite 2 risk score. The finding that having the three least discriminating variables missing still allows REVEAL Lite 2 to assess risk in patients is consistent with the FPHR approach (ie, having only FC, 6MWD, and BNP/NT-proBNP is sufficient); however, REVEAL Lite 2 also provides the possibility to discriminate among low, intermediate, and high risk. Another notable finding from our analyses is that the c-index for risk discrimination obtained using REVEAL Lite 2 was good when based on the three risk categories (low, intermediate, and high), but was equally robust when the risk scores were used. Our findings also should prompt discussion around how risk categories are assigned and whether any additional categories are needed to represent patients with risk scores that do not fall into existing categories. Although categorizing risk can provide clearer guidance to associate treatment with risk and change in risk, the increase in the 1-year death rate between intermediate-risk and high-risk groups from 7.1% to 25.1% suggests that more risk groups have yet to be identified or that using continuous variables (ie, risk scores) rather than risk category should be strongly considered.
In clinical settings, limitations in data availability, as well as time constraints, may make a risk assessment strategy that assesses fewer variables (such as REVEAL Lite 2) more practical than existing methods. In our analyses, we looked at the impact of missing data on REVEAL Lite 2 and found that even if all three of the variables classed as least valuable were missing (ie, heart rate, SBP, and eGFR), REVEAL Lite 2 still provided good discrimination among risk groups. However, in clinical practice it would be very rare for patients to be missing many of the six variables used in REVEAL Lite 2. Furthermore, REVEAL Lite 2 achieved good discrimination when applied at enrollment, demonstrating that it can be used at the appropriate time point in routine clinical practice when patients begin therapy. Another potential advantage of using REVEAL Lite 2 is that like its parent risk calculator (REVEAL 2.0), REVEAL Lite 2 uses weighting of variables. Weighting is achieved by assigning an integer score to a risk factor; this score is proportional to its contribution to the overall risk rating. The use of weighting improves the degree of agreement between predicted and observed risk.7 Other risk assessment tools, such as the FPHR, COMPERA, and Bologna methods,8, 9, 10 either do not use weighting of variables (FPHR and Bologna) or use only equal weighting (COMPERA), rather than assigning different weight based on predictive value and performance by statistical model incorporated in the REVEAL risk assessment tool, thus impacting their ability to assign risk accurately. Previous analyses conducted in a subpopulation of patients in the REVEAL Registry demonstrated that the parent risk calculator REVEAL 2.0 has greater discrimination than COMPERA and FHPR7 and that REVEAL Lite 2 has greater discrimination than both FPHR and Bologna risk assessment strategies.18
In the clinical setting, we recommend that the full REVEAL 2.0 score be used for baseline, 4- to 6-month, and yearly evaluations in treatment-naïve patients. The abridged REVEAL Lite 2 form could be used in between these time points to project a trajectory using a simpler three-component system. In established patients, the full score should be used for yearly assessments and the abridged three-component score should be used in between to monitor trajectory and to ensure that patients remain in the low-risk category. We further recommend that if the abridged three-component score indicates an increase in risk status, a full REVEAL 2.0 score should be completed, additional studies should be completed (ie, imaging), or both. REVEAL Lite 2 is intended to complement rather than replace REVEAL 2, and our recommendations aim to facilitate appropriate incorporation of REVEAL Lite 2 into routine clinical practice.
Our analyses do have some limitations. Because we used a derivative cohort (ie, a REVEAL cohort) to confirm our findings, REVEAL Lite 2 must be validated in a nonderivative cohort and as needed in other WHO group populations. In subgroup analysis, the discrimination applied in IPAH and CTD-PAH causes were as consistent as in the overall patient population with c-indexes of more than 0.7. Patients in REVEAL were treated at specialized PAH centers within the United States; therefore, our results (using data exclusively from REVEAL patients) may not be applicable to PAH patients who receive treatment in different clinical settings. The results of the present study may be subject to survival bias because we used data from patients in REVEAL who had survived for ≥ 1 year from enrollment (to account for all-cause hospitalization data in the previous 6 months). Although the time frame for enrollment in REVEAL was limited to a population from 2013, the parent REVEAL 2.0 risk calculator has been tested in large randomized trials of the four agents approved for PAH since 201319, 20, 21, 22 and registries.16 , 17 C-indexes were 0.7 or more in these analyses, indicating good discrimination of the REVEAL 2.0 risk calculator in contemporary trials and registries. A potential and timely advantage of REVEAL Lite 2 is its applicability to remote telehealth, especially in the current coronavirus disease 2019 environment. Efforts to improve further the usability of REVEAL Lite 2 are ongoing, including potential incorporation of the tool into electronic medical records.23
Conclusions
REVEAL Lite 2 is an abridged version of REVEAL 2.0 that was developed to provide clinicians with a simplified method of risk calculation that can be implemented routinely in clinical practice. When compared with REVEAL 2.0, REVEAL Lite 2 is robust, providing good discrimination between patients at low, intermediate, and high 1-year mortality risk. After being validated independently, REVEAL Lite 2 could be used for screening patients with PAH because it readily discriminates between patients at low risk compared with those not at low risk. A detailed analysis of the six variables used in the REVEAL Lite 2 risk calculator found that they were not equally valuable. Ideally, missing values should be avoided whenever possible. However, assessment of 1-year mortality risk based on at least three REVEAL Lite 2 variables (including at least two of the three most valuable variables [ie, BNP/NT-proBNP, 6MWD, and WHO FC]) seems to be accurate.
Acknowledgments
Author contributions: R. L. B. takes responsibility for the content of the manuscript, including the data and analysis. R. L. B., M. K. K., A. R., J. S., C. L. Z., M. S., C. G. E., and H. W. F. were involved in study concept and design, preparation of the draft manuscript, and critical revision and approval of the final manuscript for interpretation of the data and important intellectual input.
Financial/nonfinancial disclosures: The authors have reported to CHEST the following: R. L. B.’s institution has received grants from The American Heart Association, Bayer, The National Heart, Lung, and Blood Institute of National Institutes of Health (NHLBI/NIH), and United Therapeutics. M. M. K. is an advisory board member, speaker bureau for Bayer, Inc. A.R. has been a consultant and is a principle investigator for Actelion, is a consultant and principle investigator for United Therapeutics, is a principle investigator for Liquidia, Bellerophon, and Complexa and has received speaking honoraria from Bayer. J. S has received research support from the United States Food and Drug Administration and Instrumentation Laboratory Company. C. L. Z. and M. S. are employees of Janssen Pharmaceuticals, Inc. H. W. F. is a speaker for Bayer; a scientific advisory board member for Janssen Pharmaceuticals, Inc, Acceleron, Altavant, and United Therapeutics; and has received research support from Janssen Pharmaceuticals, Inc, Eiger, Reata, and United Therapeutics. C. G. E. has served as a consultant for Bellerophon Therapeutics, Inc., Janssen Pharmaceuticals, Inc, United Therapeutics, and Bayer Corporation. C. G. E. has received grant/research support from Janssen Pharmaceuticals, Inc, Gilead Sciences, Inc, United Therapeutics, and Intermountain Research and Medical Foundation.
Role of sponsors: Janssen Pharmaceuticals, Inc (a Janssen Pharmaceutical Company of Johnson & Johnson) is the sponsor of the REVEAL Registry and provided funding and support for the analysis presented. Medical writing support was provided by Twist Medical LLC and was funded by Janssen Pharmaceuticals, Inc.
Additional information: The e-Appendix, e-Figure, and e-Tables can be found in the Supplemental Materials section of the online article.
Footnotes
FUNDING/SUPPORT: This study was funded by Janssen Pharmaceuticals, Inc (a Janssen Pharmaceutical Company of Johnson & Johnson), the sponsor of the REVEAL Registry. J. S. and C. G. E. have received research support from the National Heart, Lung, and Blood Institute, The National Institutes of Health.
Supplementary Data
References
- 1.Galiè N., Humbert M., Vachiery J.L., et al. 2015 ESC/ERS guidelines for the diagnosis and treatment of pulmonary hypertension: the Joint Task Force for the Diagnosis and Treatment of Pulmonary Hypertension of the European Society of Cardiology (ESC) and the European Respiratory Society (ERS): endorsed by: Association for European Paediatric and Congenital Cardiology (AEPC), International Society for Heart and Lung Transplantation (ISHLT) Eur Heart J. 2016;37(1):67–119. doi: 10.1093/eurheartj/ehv317. [DOI] [PubMed] [Google Scholar]
- 2.Benza R.L., Farber H.W., Selej M., Gomberg-Maitland M. Assessing risk in pulmonary arterial hypertension: what we know, what we don’t. Eur Respir J. 2017;50(2):1701353. doi: 10.1183/13993003.01353-2017. [DOI] [PubMed] [Google Scholar]
- 3.Farber H.W., Benza R.L. Risk assessment tools in pulmonary arterial hypertension. Prognosis for prospective trials? Am J Respir Crit Care Med. 2018;197(7):843–845. doi: 10.1164/rccm.201801-0042ED. [DOI] [PubMed] [Google Scholar]
- 4.Weatherald J., Boucly A., Sahay S., Humbert M., Sitbon O. The low-risk profile in pulmonary arterial hypertension. Time for a paradigm shift to goal-oriented clinical trial endpoints? Am J Respir Crit Care Med. 2018;197(7):860–868. doi: 10.1164/rccm.201709-1840PP. [DOI] [PubMed] [Google Scholar]
- 5.Galiè N., Channick R.N., Frantz R.P., et al. Risk stratification and medical therapy of pulmonary arterial hypertension. Eur Respir J. 2019;53(1):1801889. doi: 10.1183/13993003.01889-2018. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Benza R.L., Gomberg-Maitland M., Miller D.P., et al. The REVEAL Registry risk score calculator in patients newly diagnosed with pulmonary arterial hypertension. Chest. 2012;141(2):354–362. doi: 10.1378/chest.11-0676. [DOI] [PubMed] [Google Scholar]
- 7.Benza R.L., Gomberg-Maitland M., Elliott C.G., et al. Predicting survival in patients with pulmonary arterial hypertension: the REVEAL risk score calculator 2.0 and comparison with ESC/ERS-based risk assessment strategies. Chest. 2019;156(2):323–337. doi: 10.1016/j.chest.2019.02.004. [DOI] [PubMed] [Google Scholar]
- 8.Hoeper M.M., Kramer T., Pan Z., et al. Mortality in pulmonary arterial hypertension: prediction by the 2015 European pulmonary hypertension guidelines risk stratification model. Eur Respir J. 2017;50(2):1700740. doi: 10.1183/13993003.00740-2017. [DOI] [PubMed] [Google Scholar]
- 9.Kylhammar D., Kjellström B., Hjalmarsson C., et al. A comprehensive risk stratification at early follow-up determines prognosis in pulmonary arterial hypertension. Eur Heart J. 2018;39(47):4175–4181. doi: 10.1093/eurheartj/ehx257. [DOI] [PubMed] [Google Scholar]
- 10.Boucly A., Weatherald J., Savale L., et al. Risk assessment, prognosis and guideline implementation in pulmonary arterial hypertension. Eur Respir J. 2017;50(2):1700889. doi: 10.1183/13993003.00889-2017. [DOI] [PubMed] [Google Scholar]
- 11.Dardi F, Palazzini M, Gotti E, et al. Simplified table for risk stratification in patients with different types of pulmonary arterial hypertension. Poster presented at: European Society of Cardiology Annual Meeting; August 25-29, 2018; Munich, Germany. Poster P4538.
- 12.Benza R.L., Lohmueller L.C., Kraisangka J., Kanwar M. Risk assessment in pulmonary arterial hypertension patients: the long and short of it. Adv Pulm Hyperten. 2018;16:125–135. [Google Scholar]
- 13.Raina A., Humbert M. Risk assessment in pulmonary arterial hypertension. Eur Respir Rev. 2016;25(142):390–398. doi: 10.1183/16000617.0077-2016. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Simons J.E., Mann E.D., Pierozynski A. Assessment of risk of disease progression in pulmonary arterial hypertension: insights from an international survey of clinical practice. Adv Ther. 2019;36:2351–2363. doi: 10.1007/s12325-019-01030-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Wilson M, Keeley J, Kingman M, Rogers F. Risk assessment tools in pulmonary arterial hypertension (PAH): a survey of real-world practices and barriers to use. Paper presented at: PAH PHPN Symposium; September 5-7, 2019; Washington, DC. Abstract 1001.
- 16.Anderson J.J., Lau E.M., Lavender M., et al. Retrospective validation of the REVEAL 2.0 risk score with the Australian and New Zealand Pulmonary Hypertension Registry Cohort. Chest. 2020;157(1):162–172. doi: 10.1016/j.chest.2019.08.2203. [DOI] [PubMed] [Google Scholar]
- 17.Kanwar M.K., Gomberg-Maitland M., Hoeper M., et al. Risk stratification in pulmonary arterial hypertension using Bayesian analysis. Eur Respir J. 2020;56(2):2000008. doi: 10.1183/13993003.00008-2020. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Benza RL, Kanwar M, Raina A, et al. Comparison of risk discrimination between the REVEAL 2.0 calculators, the French Pulmonary Registry algorithm, and the Bologna Method in patients with pulmonary arterial hypertension. Poster presented at: American Thoracic Society International Conference; May 17-22, 2019; Dallas, TX. Poster 11945.
- 19.Pulido T., Adzerikho I., Channick R.N., et al. Macitentan and morbidity and mortality in pulmonary arterial hypertension. N Engl J Med. 2013;369:809–818. doi: 10.1056/NEJMoa1213917. [DOI] [PubMed] [Google Scholar]
- 20.Sitbon O., Channick R., Chin K.M., et al. Selexipag for the treatment of pulmonary arterial hypertension. N Engl J Med. 2015;373:2522–2533. doi: 10.1056/NEJMoa1503184. [DOI] [PubMed] [Google Scholar]
- 21.White R.J., Jerjes-Sanchez C., Bohns Meyer G.M., et al. FREEDOM-EV Investigators Combination therapy with oral treprostinil for pulmonary arterial hypertension: a double-blind placebo-controlled clinical trial. Am J Respir Crit Care Med. 2020;201:707–717. doi: 10.1164/rccm.201908-1640OC. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Ghofrani H.-A., Nazzareno G., Grimminger F., et al. Riociguat for the treatment of pulmonary arterial hypertension. N Engl J Med. 2013;369:330–340. doi: 10.1056/NEJMoa1209655. [DOI] [PubMed] [Google Scholar]
- 23.Sitbon O, Nikkho S, Benza RL, et al. Novel composite clinical endpoints and risk scores used in clinical trials in PAH. Pulm Circ. 2020;2045894020962960. [DOI] [PMC free article] [PubMed]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.