Comparative Analysis of Established Risk Scores and Novel Hemodynamic Metrics in Predicting Right Ventricular Failure in Left Ventricular Assist Device Patients

Abstract

Background

Right ventricular failure (RVF) portends poor outcomes after left ventricular assist device (LVAD) implantation. Although numerous RVF predictive models have been developed, there are few independent comparative analyses of these risk models.

Methods and Results

Patients implanted with LVADs from 2011 to 2016 at the University of Virginia were retrospectively reviewed. RVF was defined as use of inotropes for >14 days, inhaled pulmonary vasodilators for >48 hours or unplanned right ventricular mechanical support post-operatively during the index hospitalization. Risk models were evaluated for the primary outcome of RVF using logistic regression and receiver operating curves. Among 93 LVAD patients with complete data, the Michigan RVF score (c-statistic 0.7374) compared favorably with newer RVF risk scores (Utah, Pitt, Euromacs) and was also superior to individual hemodynamic/echocardiographic metrics including pulmonary artery pulsatility index (PAPi), pre-operative right ventricular dysfunction (RVD), right atrial pressure, and pulmonary vascular resistance. The Michigan RVF score was also the best predictor of in-hospital mortality (c-statistic 0.6729) and long-term survival (Kaplan-Meier log-rank 0.0135).

Conclusions

While several new models and metrics provide predictive value, the more established Michigan RVF score – which emphasizes pre-operative hemodynamic instability and target end-organ dysfunction – remains a superior predictor of postoperative RVF as well as short-term and long-term mortality.

Introduction:

Left ventricular assist devices (LVADs) have become an important cornerstone in the management of advanced heart failure patients. These devices have alleviated the unacceptable mortality that was previously observed in advanced heart failure, particularly in those waiting on the heart transplant list.(1) Despite the widespread adoption and success of LVADs for bridge-to-transplant or destination therapy indications, several attendant complications continue to limit the efficacy of LVADs.(2,3) The prevention and management of certain complications such as bleeding, hemolysis, stroke and driveline infections has fortunately improved, particularly with the newer generation LVADs.(4–7) Conversely, post-operative right ventricular failure (RVF) has persisted as a perennial vexing challenge to LVAD success. RVF in the post-operative period is not only difficult to predict but also a management conundrum. In the current era of continuous-flow LVADs, RVF affects ~15–40% of LVAD implants.(8–15) It is an ominous and often intractable complication that is highly associated with increased mortality and morbidity.(8,16,17)

The pathophysiology of RVF post-LVAD is complex and heterogeneous, thus making risk stratification and prediction a challenging task.(17) Accordingly, numerous pre-operative factors have been studied and several models developed in an effort to predict post-operative RVF.(13,18–22) More recently, several advanced hemodynamic metrics and echocardiographic findings have also been explored as predictors of RVF.(14,23–26) Despite the ubiquity of RVF predictors and scores, there have been few independent, comparative analyses of these models in the modern era of continuous-flow LVADs.(27). We therefore performed a comparative analysis of commonly employed RVF predictive models in our cohort of LVAD patients undergoing primary implants.

Materials and Methods:

Study Design and Patient Cohort

Patients implanted with LVADs at a single academic center (University of Virginia) from March 2011 to June 2016 were retrospectively reviewed. Baseline characteristics and hospitalization data were obtained through the Clinical Data Repository. Patient charts were individually reviewed from the electronic medical record system. We abstracted data that included clinical, pre-operative laboratory data, medications, hemodynamic data, echocardiographic information, known heart failure etiology and cardiac catheterization data, and requirement for temporary mechanical circulatory support. Data points closest to the time of LVAD surgery were recorded, typically from the morning of surgery, but up to 14 days prior to surgery as necessary. Patients in this study underwent device implantation with HeartMate II®, HeartMate 3®, (both manufactured by Thoratec Corp, now Abbott Laboratory, Pleasanton, CA) or Heartware® (HVAD, manufactured by HeartWare Corp, now Medtronic, Framingham, MA).

Right Ventricular Failure and Predictive Models:

The respective RVF predictive models were calculated as specified in the original studies (Table 1).(18–20,22) RVF was defined as use of inotropes for >14 days, inhaled pulmonary vasodilators for >48 hours or unplanned need for right ventricular mechanical support post-operatively during the index hospitalization.(18,19) Calculated scores for each predictive model were then plotted against the outcome of RVF (as a binary outcome) using logistic regression analysis, and receiver operating curves (ROC) were computed to generate C-statistics for model comparison. For the Pitt algorithm, there was no range of ‘score’ outcomes provided so the score was recorded as binary (1 for patients who required a RVAD and 0 for those who did not).

Table 1.

Established models for right ventricular failure prediction in LVAD patients

Study	Patient population (n, % continuous-flow, % destination therapy)	RVF rate (%)	C- statistic	Score components (points)
Michigan RVF risk score(18) (2008)	• 197	35	0.73	• Vasopressor requirement (4)
	• 14% CF			• AST § ≥80 IU/l (2)
	• 6% DT			• Bilirubin ≥2.0 mg/dl (2.5)
				• Creatinine ≥2.3 mg/dl (3)
Utah RVF risk score(19) (2010)	• 175	44	0.74	• Destination therapy (3.5)
	• 14% CF			• IABP⸸ (4)
	• 42% DT			• Pulmonary vascular resistance
				• ≤1.7 Wood units (1)
				• 1.8–2.7 Wood units (2)
				• 2.8–4.2 Wood units (3)
				• ≥4.3 Wood units (4)
				• Inotrope dependency (2.5)
				• Obesity (2)
				• ACEi and/or ARB ƪ (−2.5)
				• Beta blocker (2)
Pittsburgh Decision Tree(20) (2012)	• 183	15*	0.87	• Transpulmonary gradient¶
	• 21.9% CF			• Age
	• DT % not reported			• Right atrial pressure
				• International normalized ratio
				• Heart rate
				• White blood cell count
				• ALT §
				• Number of inotropic agents
EUROMACS-RHF risk score(22) (2017)	• 2000ǂ	21.7	0.70ǂ	• INTERMACS class
	• 100% CF			• Use of multiple inotropes
	• 17% DT			• Severe RV dysfunction on echocardiography
				• Ratio of right atrial/ pulmonary capillary wedge pressure
				• Hemoglobin

^*Defined only as RVAD requirement; all other reviewed models utilized RVF definition cited in Methods section

^§AST - aspartate aminotransferase; ALT - alanine aminotransferase

^⸸Intra-aortic balloon counterpulsation

^ƪAngiotensin-converting enzyme inhibitor and/or angiotensin II receptor blocker

^¶No point system utilized in Pittsburgh Decision Tree model

^ǂ2000 patients included in derivation cohort. Validation cohort included additional 988 patients with c-statistic of 0.67

Hemodynamic and echocardiographic metrics including pulmonary artery pulsatility index [PAPi, calculated as (systolic pulmonary artery pressure – diastolic pulmonary artery pressure)/right atrial pressure], pre-operative right ventricular dysfunction (RVD), pre-operative tricuspid regurgitation (TR), right atrial pressure (RAP), pulmonary vascular resistance (PVR), RV stroke work index (RVSWI), and RA to pulmonary capillary wedge pressure ratio (RA:PCW) were initially compared in a separate logistic regression with RVF as the primary outcome.

Study End Points and Statistical Analysis:

Comparisons of categorical and continuous variables were performed using Chi-square and Wilcoxon rank-sum tests, respectively. The Fisher’s exact test was utilized for categorical variables with small (<5) expected values. The respective RVF models were evaluated for the association with the primary outcome of incident post-implantation RVF during the implant hospitalization. Comparison of the RVF models were assessed by inputting the respective model score into a logistic regression model with RVF as the outcome. Additionally, based on the pattern of association with RVF, the best-performing individual hemodynamic metrics were converted from continuous to categorical variables (if necessary) to enable appropriate comparison with respective RVF models. Using a logistic regression model, receiver operating characteristics (ROC) were then constructed with RVF as a binary outcome in order to estimate C-statistics. Best-performing RVF predictors were also examined for association with in-hospital mortality and long-term survival using Kaplan-Meier analyses. All statistical analysis was performed using SAS, version 9.4 (Cary, North Carolina).

Results:

Study Cohort and Outcomes:

There were 128 LVAD consecutive patients reviewed between March 2011 and June 2016. Of these patients, 93 had complete hemodynamic and echocardiographic data (Table 2). RVF was identified in 16 patients (17.2%). Baseline clinical and demographic characteristics of the study cohort are shown in Table 2. The median age was 60 years (interquartile range 53–65 years). 82.8% of patients were white, 14.0% black, and 3.2% Hispanic; 21.5% of patients were women. The implantation strategy was DT in 47.3% of patients. The etiology for HF was primarily ischemic in 41.9% of patients. Patients with RVF had a significantly longer length of stay (median 36.5 days vs. 29 days, p=0.040). Pre-operative RVF predictors and model scores are compared in Table 3. Patients who developed RVF had significantly higher pre-operative creatinine and bilirubin as well as more pre-operative RV dysfunction by echocardiography. There was a trend towards greater pre-operative vasopressor requirement, lower PAPi, and higher acuity INTERMACS profiles in patients with RVF. All-cause in-hospital mortality occurred in 11 patients (11.8%) with a mortality rate of 37.5% (n=6) in the RVF group compared to 6.5% (n=5) in patients without RVF (p= 0.003).

Table 2.

Participant demographics and pre-operative comorbidities

	RV failure
	Yes (n= 16)	No (n=77)
Age	60 (52.5–65.5)	60 (54–65)
Sex – Female	1 (6.3%)	19 (24.7%)
Race/Ethnicity
Caucasian	12 (75.0%)	65 (84.4%)
African-American	3 (18.8%)	10 (13.0%)
Hispanic	1 (6.3%)	2 (2.6%)
Body mass index	25.3 (20.6–29.7)	26.8 (23.3–30.2)
Coronary artery disease	9 (56.3%)	43 (55.8%)
Hypertension	7 (43.8%)	37 (48.1%)
Diabetes	7 (43.8%)	39 (50.7%)
Chronic kidney disease	10 (62.5%)	42 (54.6%)
Chronic obstructive pulmonary disease	3 (18.8%)	11 (14.3%)
Charlson Index	3.5 (2–5)	4 (2–5)
Length of hospitalization (days)	36.5 (27.5–53)	29 (24–38)

Values are median (IQR) or count (percentage).

Table 3.

Pre-operative RV failure predictors and model scores

	RV failure
	Yes (n= 16)	No (n=77)	p-value
Heart rate	84.5 (78–91.5)	84 (76–96)	0.955
Hemoglobin	11.1 (9.3–13.2)	11.4 (10.2–12.8)	0.695
White blood cell count	9.4 (6.3–12.0)	8.1 (6.3–10.0)	0.334
Creatinine	1.7 (1.4–2.0)	1.4 (1.0–1.5)	0.028
Aspartate aminotransferase (AST)	33.0 (24.5–54.5)	33.0 (22.0–46.0)	0.524
Alanine aminotransferase (ALT)	31.5 (17.0–54.5)	32.0 (19.0–68.0)	0.593
Bilirubin	1.6 (1.1–2.4)	1.0 (0.8–1.5)	0.026
International normalized ratio (INR)	1.2 (1.1–1.4)	1.2 (1.1–1.3)	0.756
Vasopressor requirement	4 (25.0%)	6 (7.8%)	0.066
Inotrope requirement	13 (81.3%)	64 (83.1%)	1.00
Mechanical circulatory support pre-opǂ	5 (31.3%)	14 (18.2%)	0.306
Mechanical ventilation pre-op	3 (18.8%)	7 (9.1%)	0.368
Severe RV dysfunction	4 (25.0%)	3 (3.9%)	0.016
Severe tricuspid regurgitation	2 (12.5%)	7 (9.1%)	0.650
Right atrial pressure	15.0 (12.0–19.5)	12.0 (9.0–17.0)	0.103
Transpulmonary gradient	10.0 (7.5–15.0)	11.0 (7.0–14.0)	0.980
Pulmonary vascular resistance	2.7 (2.3–3.7)	2.5 (1.9–3.4)	0.238
PAPi§	1.7 (1.2–2.7)	2.3 (1.7–3.2)	0.078
RA:PCW	0.5 (0.4–0.7)	0.5 (0.4–0.7)	0.776
RVSWI	0.5 (0.3–0.7)	0.6 (0.4–0.8)	0.227
Pre-operative ACEi/ARB	3 (18.8%)	24 (31.2%)	0.381
Pre-operative BB	9 (56.3%)	60 (77.9%)	0.113
Destination therapy	8 (50.0%)	36 (46.8%)	0.813
INTERMACS	2.5 (2.0–3.0)	3.0 (2.0–3.0)	0.099
Michigan RVF risk score	2.5 (0–4.0)	0 (0–0)	0.0002
Pittsburgh RVF decision tree	2 (12.5%)	7 (9.1%)	0.650
Utah RVF risk score	8.0 (7–11.3)	7.5 (6.0–10.5)	0.406
EUROMACS-RHF risk score	4.0 (2.0–5.0)	3.0 (2.0–4.0)	0.054

Values are median (IQR) or count (percentage).

^ǂIncludes intra-aortic balloon pump (n=10), TandemHeart (n=8), and extracorporeal membrane oxygenation (ECMO, n=1)

^§Pulmonary artery pulsatility index

Right Ventricular Failure and Risk Models:

The Michigan RVF score (c-statistic 0.7374) compared favorably with newer RVF risk scores – Utah (0.5666), Pitt (0.5170), and Euromacs (0.6489). The unadjusted c-statistics for individual hemodynamic/echocardiographic metrics were as follows: pre-operative RVD (0.7139), pre-operative TR (0.6420), PAPi (0.6408), right atrial pressure (0.6303), pulmonary vascular resistance (0.5946), PA mean pressure (0.6080), RVSWI (0.5974), and RA:PCWP (0.5231). PAPi and pre-operative RV dysfunction were then selected as the best performing hemodynamic and echocardiographic findings, respectively. PAPi was converted to a categorical variable for comparison using the optimal cutoff value of 1.5, generating two categories (with higher PAPi associated with favorable outcomes). Pre-operative RVD was analyzed as a categorical variable with three categories: none-mild, moderate, and severe. The established models and best-performing, categorized metrics are compared by ROC curve in Figure 1. Adding the Michigan RVF score to the categorized PAPi and pre-operative RVD to develop a combination score marginally improved predictive value (c-statistic 0.7849).

Figure 1.

ROC curves for RV failure

The Michigan RVF score also performed favorably (c-statistic 0.6729) in predicting all-cause in-hospital mortality when compared to the newer RVF prediction scores and metrics (Figure 2). When compared to other top performing scores/metrics (Euromacs and RV dysfunction), the Michigan RVF score was also more closely associated with long-term survival. (Kaplan-Meier log-rank 0.0135) (Figure 3).

Figure 2.

ROC curves for all-cause in-hospital mortality

Figure 3.

Discussion:

In this contemporary cohort of patients undergoing primary implantation of continuous-flow LVADs, we found that the more established Michigan RV failure score provided significantly greater predictive ability for RVF and survival compared to newer RVF predictive scores/metrics. We also noted that, overall, pre-operative metrics and scoring systems for RVF demonstrated modest predictive ability (c-statistics ranging 0.5170–0.7374).

As the use of mechanical circulatory support for the treatment of advanced heart failure has increased over the past decade, RVF has unfortunately remained a notoriously troubling challenge in the patient selection process for LVAD implantation. RVF also continues to be a leading cause of morbidity and mortality following LVAD surgery. Patients who develop RVF also have a greater risk of developing concomitant complications remote from the failing ventricle. For example, patients with pre- and/or post-operative RV dysfunction/failure have significantly higher rates of adverse events such as multi-organ system failure, gastrointestinal/post-operative bleeding, pulmonary complications, and thromboembolic events.(8,28–30) Importantly, forecasting which patient subsequently develops RVF post-LVAD implantation can be a challenging undertaking, even for the most experienced clinicians. The scope of the problem of RVF risk prediction is underscored by the numerous risk scores proposed to aid in the risk stratification process. Most of the risk scores have been developed from small single center studies with heterogeneity in definitions of RVF, type of LVAD (pulsatile vs. continuous flow), as well as the era of derivation. Moreover, there is a paucity of externally validation studies evaluating these risk prediction models.(27) Some of well-known risk scores include the Michigan RV failure score,(18) the Penn RVAD risk score,(31) Heartmate II bridge-to-transplantation RV failure analysis,(8) the Utah RV failure risk score,(19) the Pittsburgh decision tree,(20) Penn’s CRITT score,(13) the EUROMACS score,(22) and Pittsburgh’s Bayesian models.(21) These models have included PVR, RV dysfunction, TR, RA pressure, RA/CVP to PCW ratio, and RVSWI as echocardiographic and hemodynamic predictors of RVF. Additionally, other metrics have been explored including PAPi, RV-to-LV end-diastolic diameter ratio, RV free wall peak longitudinal strain, RV fractional area change, RV end-systolic and end-diastolic volume indexes, left atrial volume index, and LV volume and function.(9,10,14,23,24,32,33) These studies have shown a wide range of ability to predict RVF with c-statistics ranging mostly 0.6–0.8 and over 0.9 in some studies.(24,33) The reproducibility of these predictive metrics remains sparse and underwhelming. For example, several studies have found substantially lower predictive ability for the Michigan RVF score (c-statistic 0.54–0.61)(13,20,21) compared to the original Michigan study (c-statistic 0.73)(18); these discrepancies highlight the need for external validation research. Many of these studies have also demonstrated a close association between RVF and mortality in LVAD patients, underscoring the importance of identifying patients at risk for RVF.

In comparison to the original studies, this analysis demonstrates lower predictive ability for the Utah risk score, the Pittsburgh decision tree, and the Euromacs score. The Michigan risk score, on the other hand, retained its strength of prediction in this analysis as compared to the original development study. In assessing the differences between the risk models, it is evident that the Michigan score exclusively emphasizes severity of pre-operative hemodynamic instability and clinical end-organ injury. On the other hand, the Pitt, Utah, and Euromacs models include a range of variables that includes cardiopulmonary hemodynamic metrics, patient characteristics, and pre-operative medical management. Notably, previous work has shown that clinical variables reflecting hemodynamic instability and end-organ dysfunction (and thereby severity of RV dysfunction) drive much of the ability of scores to predict RVF.(34,35) The incidence of RVF in our study was lower than that reported in some other published studies,(18,19,27) but was similar to the recently published Euromacs study.(22) The wide variation in the reported incidence of RVF in previous studies has been attributed to heterogeneities in RVF definitions.(15) Notably, our definition of RVF was limited to the immediate post-operative period of the index hospitalization. Other studies have included RVF up to 90 days post LVAD implantation – a definition that may include late onset RVF.(27) The phenomenon of late onset RVF that occurs weeks to months following the initial post-operative period is well documented in the literature.(36) Even so, our findings are concordant with the only other large independent comparison of RVF predictive models in which the Michigan score outperformed the Utah risk score, the Pittsburgh decision tree, and Penn’s CRITT score, among others.(27) Our study validates this finding in a modern LVAD population and expands the comparative analysis to include newer predictive models/metrics. Further, this study demonstrates that combining the Michigan risk score (reflective of pre-operative hemodynamic instability and clinical end-organ injury) with pre-operative echocardiographic and hemodynamic RV dysfunction (i.e. PAPi) may have incremental prognostic yield.

Judicious patient selection is vital to preventing RVF in patients undergoing LVAD surgery. This is even more pertinent among patients whose treatment strategy is DT in whom biventricular support is practically tenuous. Considering RVF post-LVAD remains a common and significant driver of morbidity and mortality,(8–15) robust risk stratification tools are needed to mitigate this complication. At a bare minimum, risk stratification for patients undergoing LVAD implantation provides prognostic information that may be valuable in guiding the shared decision-making process between patients and clinicians prior to the LVAD surgery. Further, RVF risk modeling identifies patients that may benefit from a planned upfront biventricular support strategy – either as bridge-to-transplant or as temporary peri-operative RV support – both of which may be associated with favorable outcomes.(37–39) In this study, the superior predictive strength of the Michigan risk score highlights the importance of pre-operative target end-organ dysfunction in the risk of developing RVF post-LVAD implantation. Admittedly, multiple end-organ dysfunction is not confined to indicating impending right ventricular failure and can also be reflective of a more global critical illness such as severe sepsis. In this regard, risk modeling that focuses on the right ventricle itself may be more specific and attractive. For example, advanced imaging of the right ventricle-using cardiovascular magnetic resonance imaging (CMR) and speckle tracking imaging are promising modalities that may enhance patient selection.(40,41). Unfortunately, these modalities are yet to achieve wider application in routine patient care as they require specialized skills and may not be widely available. In the current study, pre-operative RV dysfunction by echocardiography provided nearly as much predictive power for RVF as the Michigan score (although notably less correlation with survival).

Limitations:

This study carries the inherent limitations of a retrospective, single-center study. The sample size is fairly modest and the analyzed population was further reduced by missing hemodynamic metrics in a minority of patients. If there were systematic differences between the patients with and without complete data, then this could limit the ability to extrapolate these results to all LVAD patients without further evaluation. Additionally, this study was unable to compare the CRITT model due to the non-availability of reliable pertinent variables. This study focused primarily on acute RVF during the index hospitalization rather than late-onset RVF that occurs in the weeks to months after LVAD implantation. Late RVF is increasingly being recognized in the weeks to months following LVAD implantation and has been estimated to occur in about 10–35% of patients.(8,36,42–44)

Conclusions:

Among several new models and metrics for predicting RVF in patients undergoing LVAD implantation, the more established Michigan RVF score – which emphasizes pre-operative hemodynamic instability and target end-organ dysfunction – remains a superior predictor of postoperative RVF as well as short-term and long-term mortality.