A pragmatic lab-based tool for risk assessment in cardiac critical care: data from the Critical Care Cardiology Trials Network (CCCTN) Registry

Abstract

Aims

Contemporary cardiac intensive care unit (CICU) outcomes remain highly heterogeneous. As such, a risk-stratification tool using readily available lab data at time of CICU admission may help inform clinical decision-making.

Methods and results

The primary derivation cohort included 4352 consecutive CICU admissions across 25 tertiary care CICUs included in the Critical Care Cardiology Trials Network (CCCTN) Registry. Candidate lab indicators were assessed using multivariable logistic regression. An integer risk score incorporating the top independent lab indicators associated with in-hospital mortality was developed. External validation was performed in a separate CICU cohort of 9716 patients from the Mayo Clinic (Rochester, MN, USA). On multivariable analysis, lower pH [odds ratio (OR) 1.96, 95% confidence interval (CI) 1.72–2.24], higher lactate (OR 1.40, 95% CI 1.22–1.62), lower estimated glomerular filtration rate (OR 1.26, 95% CI 1.10–1.45), and lower platelets (OR 1.18, 95% CI 1.05–1.32) were the top four independent lab indicators associated with higher in-hospital mortality. Incorporated into the CCCTN Lab-Based Risk Score, these four lab indicators identified a 20-fold gradient in mortality risk with very good discrimination (C-index 0.82, 95% CI 0.80–0.84) in the derivation cohort. Validation of the risk score in a separate cohort of 3888 patients from the Registry demonstrated good performance (C-index of 0.82; 95% CI 0.80–0.84). Performance remained consistent in the external validation cohort (C-index 0.79, 95% CI 0.77–0.80). Calibration was very good in both validation cohorts (r = 0.99).

Conclusion

A simple integer risk score utilizing readily available lab indicators at time of CICU admission may accurately stratify in-hospital mortality risk.

Key points

• Laboratory measures which are often obtained routinely at the time of patient admission to cardiac intensive care units (CICUs) are associated with risk of in-hospital mortality.

• Among these, lower pH, higher lactate, lower estimated glomerular filtration rate, and lower platelets are the top four independent lab predictors of in-hospital mortality across a broad range of CICU patients.

• A simple integer risk score comprised of only these four independent lab predictors identifies an over 20-fold gradient of risk for in-hospital mortality, with consistently good performance and excellent calibration in an external validation cohort.

Introduction

Patients admitted to cardiac intensive care units (CICUs) present with a wide variety of diagnoses and mortality risk.^1–6 While tools validated in general ICUs, such as the Sequential Organ Failure Assessment (SOFA) score, offer good discrimination in the CICU, they require serial data, rely on a large number of variables, and are sub-optimally calibrated.^6–10 Emerging CICU-specific risk tools using data from the in-hospital course robustly discriminate mortality risk.¹¹ Much of this discriminatory capacity derives from laboratory data obtained at presentation. We hypothesized that a pragmatic approach using only limited laboratory data would offer acceptable performance for rapid initial risk stratification among CICU admissions. We sought to derive a simple integer risk score for mortality using routinely collected lab markers which could be applied at time of admission.

Methods

Study population

The Critical Care Cardiology Trials Network (CCCTN) is a network of tertiary CICUs in North America coordinated by the TIMI Study Group (Boston, MA, USA). Methods for the CCCTN Registry are published.³ This analysis encompassed three annual collection campaigns (2017–2020) of all consecutive CICU admissions during each site’s (n = 25) collection period. The first and second campaigns were derivation and validation cohorts, respectively. The third campaign was used for a sensitivity analysis based on timing of pH ascertainment. External validation was performed in a previously reported CICU population (Mayo Clinic, 2007–2015, Rochester, MN, USA).¹¹

Statistical analysis

Candidate lab indicators were prospectively selected for clinical relevance: haemoglobin, creatinine, lactate, alanine aminotransferase, aspartate aminotransferase, bilirubin, glucose, platelets, and pH (venous or arterial). pH values collected in the initial two campaigns reflected the lowest or ‘worst’ values. In the third campaign, initial and ‘worst’ pH values were separately collected. Continuous variables were log-transformed where appropriate.

Multivariable logistic regression was performed using forward stepwise-selection with α ≤ 0.1 for inclusion and α ≤ 0.05 for selection to identify the top independent predictors in the derivation cohort. For ease of use, continuous variables were categorized based on clinically relevant pre-specified cut-offs. Points were allocated for each independent predictor with simple weighting guided by beta-coefficients. Labs not measured (or missing) were assigned 0 points. Discrimination was assessed using the C-index and contrasted with the SOFA score.¹⁰ Calibration was assessed in the validation cohorts by comparing observed mortality rates by risk category with predicted rates in the derivation cohort (Pearson r). P-values were two-sided. Statistics were performed with SAS v9.4.

Results

Patient demographics and intensive care unit indications

A total of 4352 admissions comprised the derivation cohort (Table 1). The most common primary diagnoses were acute coronary syndrome (28.1%) and heart failure (13.5%). Cardiac intensive care unit indications included shock in 26.0% and cardiac arrest in 10.0%.

Table 1.

Baseline characteristics of derivation cohort

Characteristic	Overall, % (n)	Alive at discharge	Death in hospital
	(n = 4352)	(n = 3757)	(n = 595)
Demographics
Age, median (IQR), years (n = 4351)	65 (55–75)	65 (55–75)	68 (58–77)
BMI, median (IQR), kg/m2 (n = 4339)	28.0 (24.1–32.9)	28.1 (24.2–33.0)	27.2 (23.4–32.4)
Female	38.3 (1666)	38.0 (1428)	40.0 (238)
Caucasian (n = 3808)	73.3 (2791)	73.1 (2397)	74.3 (394)
General medical problems and risk factors
Smoking status (n = 4311)
Current	17.2 (742)	18.2 (676)	11.2 (66)
Ex-smoker	37.4 (1613)	37.5 (1396)	36.9 (217)
Unknown	6.7 (289)	5.8 (215)	12.6 (74)
Hypertension	65.6 (2854)	66.2 (2488)	61.5 (366)
Diabetes mellitus	34.4 (1495)	33.8 (1268)	38.2 (227)
Chronic kidney disease	26.3 (1143)	24.8 (932)	35.5 (211)
Dialysis dependent (n = 1142)	21.1 (241)	20.5 (191)	23.7 (50)
Significant pulmonary disease	16.8 (729)	16.0 (601)	21.5 (128)
Significant liver disease	3.7 (160)	3.5 (132)	4.7 (28)
Cardiovascular history
Coronary artery disease	40.8 (1774)	40.7 (1530)	41.0 (244)
Cerebrovascular disease	10.5 (459)	10.0 (374)	14.3 (85)
Peripheral artery disease	9.9 (431)	9.4 (355)	12.8 (76)
Heart failure	39.3 (1711)	37.9 (1425)	48.1 (286)
LVEF < 40% (n = 1667)	60.5 (1009)	59.8 (832)	64.4 (177)
Atrial fibrillation	25.2 (1097)	24.5 (921)	29.6 (176)
Ventricular arrhythmia	7.0 (303)	6.7 (250)	8.9 (53)
Severe valvular disease	16.5 (720)	16.1 (605)	19.3 (115)
Pulmonary hypertension	6.4 (278)	5.9 (221)	9.6 (57)
Congenital heart disease	2.6 (111)	2.6 (98)	2.2 (13)

All data are reported as column % (n), unless otherwise specified.

For rows that are limited to subgroups or have missing data, available n is specified.

BMI, body mass index; LVEF, left ventricular ejection fraction.

Lab indicators

All nine candidate lab indicators from the derivation cohort were associated with in-hospital mortality on univariable analysis; on multivariable assessment, lower pH, higher lactate, lower estimated glomerular filtration rate (eGFR), and lower platelets were the top four variables independently associated with in-hospital mortality (Supplementary material online, Table S1 and Figure S1). Categorical modelling of each risk indicator revealed a significant relationship with mortality and guided point allocation for the CCCTN Lab-Based Risk Score (Figure 1; Supplementary material online, Table S2).

Figure 1.

Adjusted odds ratios for in-hospital mortality in the derivation cohort by lab indicator and point allocation in the CCCTN Lab-Based Risk Score. Estimated glomerular filtration rate (eGFR) was calculated using serum creatinine in the Chronic Kidney Disease Epidemiology Collaboration (CKD-EPI) equations. Odds ratios displayed are in reference to pH >7.3, lactate <2 mmol/L, eGFR ≥45 mL/min/1.73 m², and platelets >100 K/µL, respectively. pH used in the analysis reflects the ‘worst’ pH value as was captured in the derivation cohort. ABG, arterial blood gas; CCCTN, Critical Care Cardiology Trials Network; eGFR, estimated glomerular filtration rate; VBG, venous blood gas.

Lab-based risk score

In the derivation cohort, in-hospital mortality was 13.7%. The CCCTN Lab-Based Risk Score identified a strong gradient of risk for in-hospital mortality (3.2–73.5%, P < 0.001, Figure 2A). This gradient of risk was maintained in higher-risk subgroups (shock and cardiac arrest; P < 0.001, Figure 2B). The risk score demonstrated good discrimination [C-index 0.82, 95% confidence interval (CI) 0.80–0.84] in the overall cohort and higher-risk patient subgroups (cardiac arrest: C-index 0.79, 95% CI 0.75–0.83; shock: C-index 0.72, 95% CI 0.69–0.75). Continuous modelling of the lab indicators did not improve performance (C-index 0.82, 95% CI 0.80–0.84). Inclusion of all clinical factors from Table 1, including age and sex, also did not substantially improve discrimination (C-index 0.84, 95% CI 0.82–0.86). Performance was good across subgroups (Supplementary material online, Figures S2–S4).

Figure 2.

Performance of the CCCTN Lab-Based Risk Score with respect to in-hospital mortality. (A) CCCTN derivation cohort, (B) high-risk subgroups (derivation cohort), (C) CCCTN validation cohort, (D) correlation of mortality rates by CCCTN Lab-Based Risk Score in the CCCTN validation and derivation cohorts, (E) external validation cohort using initial pH, and (F) correlation of mortality rates by CCCTN Lab-Based Risk Score in the external validation cohort. CCCTN, Critical Care Cardiology Trials Network.

Admission eGFR and platelets were available in >99% of patients indicating a high degree of completeness of data capture. Lactate and pH were measured selectively per the clinician’s discretion and available in 59.3% and 49.9% of patients, respectively. Although discrimination was lower, performance remained acceptable when applied to only patients with data for all four labs (C-index 0.77, 95% CI 0.75–0.80).

Validation and sensitivity analyses

Validation in the second campaign (n = 3888, Supplementary material online, Table S3) demonstrated a similarly strong, graded relationship with in-hospital mortality (Figure 2C; C-index 0.82; 95% CI 0.80–0.84). Assessment of calibration revealed very good qualitative agreement between the predicted risk and the observed mortality rates in the validation set (r = 0.99, Figure 2D). Discrimination was qualitatively similar to the SOFA score (C-index 0.85, 95% CI 0.83–0.87; Supplementary material online, Table S4). Subgroup analyses revealed good performance (Supplementary material online, Figure S4).

In the third campaign, initial pH was collected in 731 patients. Re-assessment of the score using initial pH yielded overall good performance (C-index 0.79, 95% CI 0.77–0.81; r = 0.93; Supplementary material online, Figure S5 and Table S4).

The external validation cohort included 9716 patients with characteristics and demographics that have been previously reported.¹¹ Stable discrimination and excellent calibration were apparent using initial pH (C-index 0.79, 95% CI 0.77–0.80; r = 0.99; Figure 2E and F; Supplementary material online, Table S4).

Discussion

We developed a pragmatic lab-based risk score in a large well-characterized cohort of CICU patients. Using only four variables routinely available at the time of admission, the CCCTN Lab-Based Risk Score identified a 20-fold gradient in mortality risk with very good discrimination and calibration in two separate validation cohorts. Importantly, performance of the score was consistent across commonly encountered CICU diagnoses and in higher-risk subgroups (i.e., shock, cardiac arrest). This risk score can be easily calculated at the bedside or incorporated into electronic medical record systems. As such, the CCCTN Lab-Based Risk Score could serve to complement pre-existing, more complex risk assessment tools which are effective but require serial measures or are applied after diagnostic testing for specific CICU subpopulations (e.g. SOFA, APACHE II, M-CARS, or IABP-SHOCK II scores).^7–13

Early risk stratification using such a tool could serve to (i) guide triage and treatment decisions, (ii) optimize resource allocation, and (iii) inform initial goals of care discussions.¹³ From a research perspective, such a tool may be useful for guiding entry into clinical trials, controlling for differences in patient acuity, and facilitating risk-adjusted quality assessment.¹⁴ The novelty of this risk score is its effective risk stratification that might be easily applied on the ‘first call’ from the Emergency Department to provide an immediate gauge of mortality risk using only four lab-based variables.

Limitations

First, our analysis cohort was predominantly from tertiary care centres; calibration may be different when applied to other environments. Second, the availability of lactate and pH integrates the initial clinical assessment of the ordering clinician and contributes to the discriminatory performance of the score. Third, worst pH was used in the first two campaigns. However, use of initial pH in the third campaign and a separate external validation cohort demonstrated consistent performance. Fourth, performance was contrasted only with the SOFA score and merits comparison with other risk models that were unable to be tested in our dataset. Fifth, cardiac biomarkers (e.g. troponin, brain natriuretic peptide) were not captured and may further complement risk prediction.

Conclusion

A simple integer risk score utilizing readily available lab indicators at time of CICU admission may accurately stratify in-hospital mortality risk.

Supplementary material

Supplementary material is available at European Heart Journal: Acute Cardiovascular Care online.