Prediction of in-hospital mortality after acute upper gastrointestinal bleeding: cross-validation of several risk scoring systems

Abstract

Objective

We aimed to identify the clinical, biochemical, and endoscopic features associated with in-hospital mortality after acute upper gastrointestinal bleeding (AUGIB), focusing on cross-validation of the Glasgow-Blatchford score (GBS), full Rockall score (RS), and Cedars-Sinai Medical Center Predictive Index (CSMCPI) scoring systems.

Methods

Our prospective cross-sectional study included 156 patients with AUGIB. Several statistical approaches were used to assess the predictive accuracy of the scoring systems.

Results

All three scoring systems were able to accurately predict in-hospital mortality (area under the receiver operating characteristic curve [AUC] > 0.9); however, the multiple logistic model separated the presence of hemodynamic instability (state of shock) and the CSMCPI as the only significant predictive risk factors. In compliance with the overall results, the CSMCPI was consistently found to be superior to the other two systems (highest AUC, highest sensitivity and specificity, highest positive and negative predictive values, highest positive likelihood ratio, lowest negative likelihood ratio, and 1-unit increase in CSMCPI associated with 6.3 times higher odds of mortality), outperforming the GBS and full RS.

Conclusions

We suggest consideration of the CSMCPI as a readily available and reliable tool for accurately predicting in-hospital mortality after AUGIB, thus providing an essential backbone in clinical decision-making.

Introduction

Acute upper gastrointestinal bleeding (AUGIB), a common medical emergency, is a life-threatening event associated with substantial morbidity, mortality, and medical care costs. Because of its dramatic clinical picture and necessity for urgent diagnostic and therapeutic procedures, AUGIB has high priority in pre-hospital and hospital treatments. The in-hospital mortality rate after AUGIB is usually in the range of 5% to 15%, but it may also reach 35% in patients hospitalized for other health problems.^1–3 Common etiologies of AUGIB include peptic ulcers (duodenal or gastric ulcers followed by erosive gastritis and duodenitis), bleeding esophageal varices, Mallory–Weiss tears, angiodysplasias or other vascular malformations, and tumors; less common causes are hemobilia and aortoenteric fistulas. ⁴ Efficient localization of the bleeding site and causes as well as proper management of the bleeding activity in the shortest possible time period from symptom onset seem fundamentally important. Urgent esophagogastroduodenoscopy can significantly reduce the morbidity, mortality, and hospitalization costs, thus distinguishing between high-risk and low-risk patients.^5,6

The etiology of the disease, endoscopic signs of acute and recent bleeding or persistent bleeding, presence of shock, age, and comorbidities might be significant risk factors for adverse outcomes after AUGIB. Several risk scoring systems (endoscopic-based and non-endoscopic-based) for AUGIB have been recently developed as tools for assessment of the clinical outcomes to support clinical decision-making; among them, the validated full Rockall score (RS) and the Glasgow-Blatchford score (GBS) seem to be the best known and most widely used.^7–9 Generally, the GBS is considered more feasible because it requires only clinical and laboratory data, allowing to be applied immediately without endoscopic findings. In contrast, the full RS is not suitable for decision-making regarding urgent endoscopy because it requires endoscopic criteria. Other endoscopic-based scoring systems have also been introduced in the past few decades, such as the Cedars-Sinai Medical Center Predictive Index (CSMCPI), ¹⁰ and their predictive capacity for prognostication of mortality should be cross-validated.

In this study, we aimed to identify the key risk factors associated with in-hospital mortality after AUGIB. Particular focus was given to cross-validation and exploration of the reliability, discrimination power, and predictive capacity of three risk scoring systems in relation to the in-hospital mortality after AUGIB: the GBS as a tool for clinical decision-making based on clinical and laboratory findings, and the full RS and CSMCPI as endoscopy-based systems.

Materials and methods

Patients and study

From January 2016 to January 2017, we performed a prospective, cross-sectional study focused on detailed investigations of the clinical, endoscopic, and biochemical features of patients with AUGIB and the relationship of these features with the in-hospital mortality risk. During the defined period, all patients who were suspected to have AUGIB (bleeding from the proximal segment of the digestive tract, anatomically defined as proximal to the ligament of Treitz) and clearly exhibited symptoms such as hematemesis and/or melena were consecutively included in the initial screening. All patients suspected to have AUGIB were admitted to the Clinic of Gastroenterology and Hepatology, Clinical Center Niš, where urgent esophagogastroduodenoscopy was performed for diagnostic purposes (as well as to assess the etiology of the bleeding) by using the Pentax EPK-1000 endoscope (Pentax Medical, Tokyo, Japan) and Olympus Exera II CLV-180 endoscope (Olympus, Tokyo, Japan). Clinical examinations to evaluate disseminated malignity were also conducted in all patients by using echo sonography of the abdomen and chest X-ray radiography.

During hospitalization, patients received their final diagnosis based on their detailed clinical evaluation findings. Because the study specifically focused on patients with AUGIB, the following study inclusion criteria were implemented: diagnosis of AUGIB caused by esophageal varices with bleeding, acute esophageal ulcer with hemorrhage, Mallory–Weiss tear, acute gastric ulcer with hemorrhage, acute duodenal ulcer with hemorrhage, acute erosive hemorrhagic gastritis, malignant esophageal tumors with hemorrhage, malignant gastric tumors with hemorrhage, and gastric leiomyoma with hemorrhage.

The exclusion criteria were negative findings of proximal endoscopy, small and medium-sized esophageal varices without signs of acute and recent bleeding, congestive gastropathy, chronic erosive gastritis, and esophageal and gastric polyps without bleeding.

Experimental procedures

Clinical, endoscopic, and biochemical analyses were performed in all patients during hospitalization. The following clinical parameters were evaluated: age, sex, admission symptoms (melena/hematemesis), systolic blood pressure, pulse rate, hemodynamic stability, presence of comorbidities (ischemic heart disease, heart failure, liver failure, kidney disease, disseminated malignancy), and time of bleeding onset (≥48 hours before admission, <48 hours before admission, in-hospital bleeding). Moreover, risk assessment was performed by using the GBS, ¹¹ RS, ¹² and CSMCPI ¹⁰ in accordance with the instructions established for these scoring systems. The following endoscopic features were analyzed: the presence of gastric and duodenal ulcers according to the Forrest’ classification, ⁵ the presence of erosions with or without stigmata of recent hemorrhage, the size of the esophageal varices, the presence of esophagitis, Mallory–Weiss tears, benign and malignant tumors, and the presence of persistent gastrointestinal bleeding. Several biochemical parameters were also analyzed, including the serum urea concentration, serum creatinine concentration, serum bilirubin concentration, blood hemoglobin concentration, and international normalized ratio (INR).

After discharge from the Clinic of Gastroenterology and Hepatology, the final disease outcome (survival/death) was recorded for all patients. In-hospital mortality was defined as mortality during the hospitalization period (up to 25 days), which in this study usually occurred between the 3rd and 10th day of hospitalization (i.e., in 67% of the patients with a fatal outcome). All patients’ details were de-identified during preparation of the final database. Multiple people coded the data to maintain objectivity and avoid bias, and patients with incomplete data were excluded from the study to avoid bias caused by missing data.

The reporting of this study conforms to the STROBE guidelines. ¹³

Ethics statement

This research complied with the principles of the Declaration of Helsinki and with the latest Good Clinical Practice guideline. ¹⁴ Approval was obtained from our local institutional review board (Ethics Committee of the Medical Faculty, University of Niš, No. 12-14250-2/1, issued in Niš on 18 December 2018). All patients provided written informed consent for participation in the study.

Statistical analysis

All statistical analyses were performed using IBM SPSS Statistics for Windows, Version 21.0 (IBM Corp., Armonk, NY, USA). Categorical variables are expressed as the percentage of individuals, and the differences in the frequencies between the groups were calculated with the χ² test. Because the distribution of the values of all continuous variables differed significantly from normality (Shapiro–Wilk test, p < 0.05), these results are expressed as median and interquartile range (values of Q1–Q3). The Mann–Whitney U-test was used to compare the differences in the central tendencies between the groups. The non-parametric Levene’s test was used to compare the variances between the groups. The bivariate statistical analyses were performed using non-parametric correlation with Spearman’s coefficient. Receiver operating characteristic (ROC) curves were generated for all of the scoring systems to assess the sensitivity and specificity of the discrimination between the two outcomes (the area under the ROC curve [AUC] was used as a measure of the discrimination power). Univariate and multivariate nominal logistic regression modeling was used to estimate the significant independent predictors of the fatal outcome, and their sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), positive likelihood ratio (LR+), negative likelihood ratio (LR−), and diagnostic odds ratio (OR) were calculated based on a classification 2 × 2 table (true positives, false positives, false negatives, true negatives).

In all cases, the level of statistical significance was defined as p < 0.05, and p < 0.001 was deemed highly significant.

Results

Clinical characteristics of study population

During the defined 1-year cross-sectional period, only 216 of 369 potentially eligible patients fulfilled the initial screening criteria and were examined for eligibility (i.e., screened for AUGIB); the remaining 153 excluded patients in this phase had a medical history of chronic gastrointestinal disorders with bleeding or did not give informed consent. After implementation of the inclusion and exclusion criteria, only 158 of the 216 patients were confirmed to have AUGIB and were included in the study (the remaining 58 parents had negative findings on proximal endoscopy). During the in-hospital follow-up, two patients were excluded from the study because of incomplete data. Finally, 156 patients completed the study and were included in the statistical analyses.

The patients’ median age was 68 years (interquartile range, 57–78 years). Most of the patients were male (69.2%), and most had a bleeding duration of <48 hours at the time of admission (67.3%). Upon admission, the AUGIB presented as melena alone in 52.6% of all patients and as both hematemesis and melena in 47.4% of the patients. Only 14.1% of all patients had severe hemostatic instability and were admitted in a state of shock. Liver cirrhosis was detected in 7.7% of all patients. According to the endoscopic findings, the main cause of AUGIB was peptic ulcers, more precisely duodenal ulcers (42.9%) and gastric ulcers (34.6%). Gastric and duodenal erosions were detected in 15.4% of the patients (10.3% and 5.2%, respectively), esophagitis in 7.7% of the patients, large esophageal varices in 7.1% of the patients, and Mallory–Weiss tears in 7.7% of the patients (non-bleeding in 3.2% and bleeding in 4.5%). The cause of AUGIB in the remaining patients was a malignant tumor (gastric malignant tumor in 4.5% of patients, esophageal malignant tumor in 0.64%, and gastric leiomyoma in 0.64%). After triage in the emergency department, 37.8% of the patients were treated by endoscopic hemostasis and 5.1% by surgical intervention. Most of the patients included in this study had a good outcome; the overall in-hospital mortality rate was 14.7%.

Comparison of clinical, biochemical, and endoscopic features between survivors and non-survivors

A comparison of the basic clinical, biochemical, and endoscopic features between survivors and non-survivors is shown in Table 1. Non-survivors were significantly older than survivors (p < 0.001), but sex did not affect the mortality outcome. The concurrent presence of both hematemesis and melena on admission (p < 0.001), in-hospital bleeding onset (p < 0.001), and especially the presence of severe hemodynamic instability (most of the non-survivors (82%) were in a state of shock, p < 0.001) were significantly associated with fatal outcomes. Overall, 59.6% of the patients in this study had comorbidities, and comorbidities were found significantly more frequently in non-survivors (95.7%) than in survivors (p < 0.001). Further analyses revealed that all comorbidities were significantly associated with in-hospital mortality after AUGIB, specifically the presence of liver failure (26.1% vs. 4.5%, p < 0.001), kidney failure (34.8% vs. 10.5%, p < 0.05), disseminated malignancy (17.4% vs. 3.7%, p < 0.05), and especially ischemic heart disease (87% of all non-survivors had ischemic heart disease, p < 0.001) and heart failure (65.2% vs. 12.8%, p < 0.001).

Table 1.

Clinical, biochemical, and endoscopic features between survivors and non-survivors.

Variable	Survivors	Non-survivors	p
Age, years	67 (56–76)	78 (64–80)	0.008*
Sex
Male	70.7%	60.9%	0.374 (ns)
Female	29.3%	39.1%
Symptoms			0.0003*
Both hematemesis and melena	41.4%	82.6%
Only melena	58.6%	17.4%
Hemodynamic stability			1.25 × 10−23**
Stable	50.4%	0.0%
Moderate instability	47.4%	17.4%
State of shock	2.3%	82.6%
Presence of comorbidities	53.4%	95.7%	0.00005**
Liver cirrhosis	4.5%	21.1%	0.0003**
Bleeding onset			1.52 × 10−7**
≥48 hours before admission	30.8%	0.0%
<48 hours before admission	66.9%	69.6%
In-hospital	2.3%	30.4%
Systolic blood pressure, mmHg	120 (100–130)	90 (80–103)	7.73 × 10−8**
Pulse, beats/minute	85 (75–100)	110 (100–120)	5.25 × 10−9**
Blood hemoglobin, g/L	89 (78–107)	74 (55–93)	0.005*
Serum urea, mmol/L	12.2 (8.2–19.5)	25.6 (15.2–30.1)	0.00002**
Serum creatinine, mg/dL	1.08 (0.89–1.43)	1.60 (1.09–2.25)	0.009*
Serum bilirubin, mg/dL	0.64 (0.43–0.89)	0.65 (0.49–1.39)	0.197 (ns)
INR	1.2 (1.1–1.3)	1.6 (1.3–2.2)	0.000001**
Gastric ulcers
Presence	36.1%	26.1%	0.352 (ns)
Forrest classification
Active spurting bleed Ia	0.0%	0.0%	0.999 (ns)
Active oozing bleed Ib	2.3%	4.3%	0.558 (ns)
Non-bleeding visible vessel IIa	3.8%	4.3%	0.892 (ns)
Adherent clot IIb	7.5%	4.3%	0.583 (ns)
Dark base IIc	13.5%	13.0%	0.949 (ns)
Lesions without active bleeding	9.0%	0.0%	0.134 (ns)
Duodenal ulcers
Presence	42.9%	43.5%	0.956 (ns)
Forrest Classification
Active spurting bleed Ia	0.0%	4.3%	0.147 (ns)
Active oozing bleed Ib	4.5%	4.3%	0.972 (ns)
Non-bleeding visible vessel IIa	4.5%	8.7%	0.401 (ns)
Adherent clot IIb	6.0%	17.4%	0.059 (ns)
Dark base IIc	18.8%	8.7%	0.237 (ns)
Lesions without active bleeding	9.0%	0.0%	0.134 (ns)
Presence of gastric erosions	12.0%	0.0%	0.079 (ns)
Presence of duodenal erosions	6.0%	0.0%	0.605 (ns)
Presence of Mallory–Weiss tear	7.5%	8.7%	0.691 (ns)
Presence of esophagitis	8.3%	4.3%	0.514 (ns)
Presence of large esophageal varices	3.8%	26.1%	0.0001**
Tumors			0.950 (ns)
Absent	94.0%	95.7%
Malignant esophageal tumor	0.8%	0.0%
Malignant gastric tumor	4.5%	4.3%
Gastric leiomyoma	0.8%	0.0%
Persistent GI bleeding	1.5%	56.5%	3.37 × 10−11**
Recurrent bleeding	5.3%	13.0%	0.167 (ns)
Intervention
Endoscopic hemostasis	36.8%	47.8%	0.284 (ns)
Surgical intervention	6.0%	5.1%	0.227 (ns)

Categorical variables are expressed as percentage of patients and were calculated with the χ² test. Continuous variables are expressed as median (interquartile range, Q1–Q3) and were compared with the Mann–Whitney U-test.

ns, not significant; INR, international normalized ratio; GI, gastrointestinal.

*p < 0.05, **p < 0.001.

Liver cirrhosis was significantly more frequent in non-survivors than in survivors (21.1% vs. 4.5%, p < 0.001). Compared with survivors, non-survivors also had a significantly lower systolic blood pressure (p < 0.001) and higher pulse rate (p < 0.001) at admission. A lower hemoglobin concentration (p < 0.05) and a higher serum urea concentration (p < 0.001), creatinine concentration (p < 0.05), and INR (p <0.001) were all significantly associated with fatal outcomes. None of the endoscopic findings (presence of gastric or duodenal ulcers in compliance with the Forrest classification, presence of gastric or duodenal erosions, Mallory–Weiss tears, esophagitis, or presence of tumors) was significantly different between the two outcomes. However, detection of large esophageal varices (p < 0.001) and persistent gastrointestinal bleeding (p < 0.001) were highly associated with fatal outcomes. Finally, in-hospital mortality was not affected by the type of treatment intervention (endoscopic hemostasis or surgical intervention).

Comparison of risk scoring systems between survivors and non-survivors

The results obtained from each risk scoring system are shown in Figure 1. Non-survivors had a significantly higher GBS (p < 0.001) (Figure 1(a)), full RS (p < 0.001) (Figure 1(b)), and CSMCPI (p < 0.001) (Figure 1(c)) at admission. We observed significantly lower variability and spread of the values for all of the examined scoring systems among the non-survivors (p < 0.001 for each system, non-parametric Levene’s test for equality of variances). Visual analyses of the histograms confirmed more or less precise separation of the values for all of the scoring systems between the two outcomes, suggesting that the non-survivors could be distinguished on the basis of their admission scores.

Figure 1.

Scores between survivors and non-survivors. All three scoring systems (GBS, full RS, and CSMCPI) showed significantly higher values for non-survivors than survivors. A precise separation of the values for all three analyzed scoring systems in relation to the outcomes was observed from the generated histograms.

**p < 0.001; GBS, Glasgow-Blatchford score; full RS, full Rockall score; CSMCPI , Cedars-Sinai Medical Center Predictive Index.

The relationships among the different scoring systems were analyzed by bivariate statistical analyses. The GBS showed a positive and moderate correlation with the full RS (ρ = 0.681, p < 0.001) and the CSMCPI (ρ = 0.574, p < 0.001); nevertheless, the highest correlation was detected between the two endoscopy-based systems (i.e., the full RS and CSMCPI) (ρ = 0.724, p < 0.001). The bivariate analyses concerning the outcomes showed that the finest separation of the outcomes could be judged based on the full RS and CSMCPI (Figure 2).

Figure 2.

Bivariate statistical analyses between the values of the analyzed scoring systems. A linear trend of the relationship is represented, with shaded ellipses corresponding to the 95% confidence intervals for the two different outcomes (blue, survivors; red, non-survivors). The finest separation of the outcomes can be judged on the basis of the full RS and CSMCPI.

GBS, Glasgow-Blatchford score; full RS, full Rockall score; CSMCPI, Cedars-Sinai Medical Center Predictive Index.

ROC curves were generated to estimate the optimal cutoff point for discrimination between the two outcomes (Figure 3). The ROC curves showed that a GBS of 13.5 was associated with a risk of in-hospital mortality with 95.7% sensitivity and 82.7% specificity (AUC = 0.979). A full RS of 6.5 was determined to be the optimal cutoff point associated with a fatal outcome with 100% sensitivity and 84.2% specificity (AUC = 0.977). The CSMCPI was estimated to have the highest capacity for discrimination between the outcomes (AUC =0.991), with a value of 5.5 as the optimal cutoff point for detection of a fatal outcome (100% sensitivity and 96.2% specificity). The precise calculations for the performance of these tests based on above-defined cutoffs are shown in Table 2.

Figure 3.

Receiver operating characteristic curve analyses for assessment of the optimal cutoff points for discrimination between survivors and non-survivors.

GBS, Glasgow Blatchford score; full RS, full Rockall score; CSMCPI, Cedars-Sinai Medical Center Predictive Index.

Table 2.

Comparison of sensitivity, specificity, PPV, NPV, LR+, LR−, and DOR for prediction of mortality in patients with AUGIB among the scoring systems.

Scoring system	Sensitivity	Specificity	PPV	NPV	Likelihood ratio		DOR (LR+/LR−)	False positives(type I error)	False negatives (type II error)
					LR+	LR−
GBS (>13.5)	95.7%	82.7%	48.9%	99.1%	5.53	0.05	110.6	17.3%	4.3%
Full RS (>6.5)	100%	84.2%	53.2%	100%	6.33	0.0	∞	15.8%	0.0%
CSMCPI (>5.5)	100%	96.2%	81.2%	100%	26.3	0.0	∞	3.8%	0.0%

AUGIB, acute upper gastrointestinal bleeding; GBS, Glasgow-Blatchford score; Full RS, full Rockall score; CSMCPI, Cedars-Sinai Medical Center Predictive Index; PPV, positive predictive value; NPV, negative predictive value; LR+, positive likelihood ratio; LR−, negative likelihood ratio; DOR, diagnostic odds ratio.

The calculations show that although the GBS and full RS had high sensitivity (i.e., the proportion of patients with a positive GBC or RS test among all patients with an actual fatal outcome was 95.7% and 100%, respectively), their PPV (probability of a fatal outcome after a positive GBC or RS test) was low. The CSMCPI test based on the defined cutoff of 5.5 outperformed the previous tests, having a high PPV and NPV. The LR+ was also high (26.3), suggesting that the odds of a fatal outcome increased with a positive CSMCPI test. Overall, the CSMCPI was shown to be superior to the GBS and full RS in predicting in-hospital morality.

Nominal logistic model for prediction of fatal outcome

A nominal logistic regression was performed to estimate which of the analyzed variables represented the most important significant predictive risk factors for a fatal outcome in patients with AUGIB. The results of the univariate logistic regression are shown in Table 3 (only statistically significant predictors are shown).

Table 3.

Univariate logistic regression and ORs for mortality risk assessment after AUGIB.

Variables	OR	95% CI for OR		p
		Lower	Upper
Age, years	1.051	1.010	1.094	0.014
Presence of both hematemesis and melena as symptoms	6.736	2.172	20.897	0.001
Systolic blood pressure, mmHg	0.935	0.908	0.962	<0.001
Pulse, beats/minute	1.079	1.046	1.113	<0.001
Moderate hemodynamic instability	0.234	0.076	0.725	0.012
Hemodynamic instability – state of shock	205.833	42.718	991.790	<0.001
Presence of comorbidities	19.211	2.516	146.681	0.004
Diagnosed ischemic heart disease	16.068	4.514	57.195	<0.001
Diagnosed heart failure	12.794	4.718	34.697	<0.001
Diagnosed liver failure	7.471	2.163	25.806	0.001
Diagnosed kidney failure	4.533	1.633	12.588	0.004
Diagnosed disseminated malignancy	5.389	1.329	21.862	0.018
In-hospital bleeding onset	18.958	4.453	80.721	<0.001
Serum urea, mmol/L	1.084	1.042	1.129	<0.001
Blood hemoglobin, g/L	0.967	0.946	0.989	0.003
INR	3.525	1.654	7.513	0.001
Liver cirrhosis	7.471	2.163	25.806	0.001
Presence of large esophageal varices	9.035	2.487	32.829	0.001
Persistent GI bleeding	85.150	16.824	430.974	<0.001
GBS	3.344	1.954	5.724	<0.001
Full RS	30.262	4.016	228.031	0.001
CSMCPI	7.011	2.812	17.482	<0.001

AUGIB, acute upper gastrointestinal bleeding; OR, odds ratio; CI, confidence interval; INR, international normalized ratio; GI, gastrointestinal; GBS, Glasgow-Blatchford score; Full RS, full Rockall score; CSMCPI, Cedars-Sinai Medical Center Predictive Index.

Variables marked with dark orange: OR of >80.

Variables marked with pale orange: OR of >10.

In the univariate logistic regression analyses, the following variables were not significant predictors of mortality: sex, presence of hemodynamic stability, bleeding onset of ≥48 hours before admission, bleeding onset of <48 hours before admission, creatinine and bilirubin concentrations, and most of the endoscopic variables (presence of gastric and duodenal ulcers as well as their identification according to the Forrest classification, presence of gastric erosions, duodenal erosions, presence of Mallory–Weiss tears, presence of esophagitis, presence of tumors, and recurrent bleeding). Nevertheless, 22 variables were found to be significant predictors of in-hospital mortality in the univariate logistic model (Table 3), with the following variables estimated as the main drivers of fatal outcomes: hemodynamic instability (state of shock; OR = 205.8, 95% confidence interval (CI) = 42.7–991.8), persistent gastrointestinal bleeding (OR = 85.1, 95% CI = 16.8–430.9), full RS (OR = 30.3, 95% CI = 4.0–228.0), presence of comorbidities (OR = 19.2, 95% CI = 2.5–146.7), and in-hospital bleeding onset (OR = 18.9, 95% CI = 4.5–80.7).

The sensitivity, specificity, PPV, NPV, LR+, and LR− of these most important predictors are summarized in Table 4. As shown in the table, only hemodynamic instability (i.e., state of shock) was estimated as a valuable predictor of mortality, with a satisfactory PPV and NPV and the highest LR+ and diagnostic OR among all variables.

Table 4.

Sensitivity, specificity, PPV, NPV, LR+, LR−, and DOR of variables with highest effect and OR in univariate logistic regression.

Variables	Sensitivity	Specificity	PPV	NPV	Likelihood ratio		DOR (LR+/LR−)	False positives (type I error)	False negatives (type II error)
					LR+	LR−
Hemodynamic instability (state of shock)	82.6%	97.7%	86.4%	97.0%	35.9	0.18	199.4	2.3%	17.4%
Persistent GI bleeding	56.5%	98.5%	86.7%	92.9%	37.7	0.44	85.7	1.5%	43.5%
Presence of comorbidities	95.7%	53.4%	76.3%	98.4%	2.1	0.08	26.3	46.6%	4.3%
In-hospital bleeding onset	30.4%	97.7%	70.0%	89.0%	13.2	0.71	18.6	2.3%	69.6%
Diagnosed ischemic heart disease	87.0%	70.7%	33.9%	96.9%	2.9	0.18	16.1	29.3%	13.0%
Diagnosed heart failure	65.2%	87.2%	46.9%	93.5%	5.1	0.40	12.8	12.8%	34.8%

GI, gastrointestinal; PPV, positive predictive value; NPV, negative predictive value; LR+, positive likelihood ratio; LR−, negative likelihood ratio; DOR, diagnostic odds ratio; OR, odds ratio.

For the multivariate logistic regression modeling, we built up several models (Forward: conditional, Forward: LR, and Forward: Wald methods) after all significant covariates associated with the fatal outcome (Table 3) were included in the models. All three methods resulted in the same final, optimal multiple regression model in which the presence of hemodynamic instability (state of shock) and the CSMCPI were estimated as the only significant predictive risk factors for in-hospital mortality (Table 5).

Table 5.

Multiple logistic model with significant prognostic risk factors for fatal outcome after AUGIB.

Independent predictors	p	Odds ratio	Lower 95% CI	Upper 95% CI
Presence of hemodynamic instability (state of shock)	0.026 (*)	14.541	1.381	153.126
CSMCPI	0.002 (*)	6.311	2.003	19.888
Constant	0.001 (*)	0	/	/

AUGIB, acute upper gastrointestinal bleeding; CI, confidence interval; CSMCPI, Cedars-Sinai Medical Center Predictive Index.

*p < 0.05.

According to the model, the presence of hemodynamic instability (state of shock) was associated with 14.5 times higher odds of a fatal outcome, and every 1-point increase in the CSMCPI was associated with a 6.3 times higher odds of in-hospital mortality, stressing the importance of these predictors for the prognosis after AUGIB. The developed model was statistically significant (F = 110.4, p < 0.001), with an estimated 99.2% sensitivity, 91.3% specificity, overall classification accuracy rate of 98.1%, PPV of 98.5%, NPV of 95.5%, LR+ of 11.4, and LR− of 0.008. According to the Cox-Snell R2 and Nagelkerke’s R2 values, the independent predictors included in this model were able to explain 50.7% to 89.5% of the variation in the risk of the occurrence of death.

Discussion

Risk stratification after AUGIB has been a focus of clinical research during the last several years. Assessing the risk of mortality might be useful in clinical decision-making regarding the need for a specific intervention (transfusion, endoscopic or surgical treatment). Nevertheless, risk stratification, the role of endoscopic therapy, the requirement for endoscopic hemostasis, the need for blood transfusion, and the indications for medical and surgical treatment have all been found to be notoriously confounding and controversial, ⁷ resulting in the absence of a consensus on a specific approach to patients with AUGIB. ¹⁵

In this study, we identified several important clinical, biochemical, and endoscopic features that affected the in-hospital mortality outcome. The following features were significantly associated with fatal outcomes after AUGIB: age, concurrent hematemesis and melena on admission, moderate hemodynamic instability, lower systolic blood pressure and higher pulse at admission, lower hemoglobin concentration, higher serum urea and creatinine concentrations, higher INR, and the presence of large esophageal varices. The following were especially associated with the risk of mortality as significant predictors in the univariate logistic regression, suggesting their role as possible key indicators for risk stratification after AUGIB: severe hemodynamic instability (state of shock), persistent gastrointestinal bleeding, the presence of comorbidities, in-hospital bleeding onset, the presence of ischemic heart disease, and diagnosed heart failure. Some of these findings are highly congruent with previously published data. For instance, Corzo et al. ¹⁶ found that age, hematemesis, in-hospital bleeding, and a state of shock-trauma were significantly related to mortality, and Klebl et al. ¹⁷ showed that variceal bleeding was an important risk factor for mortality. Nevertheless, after performing the multiple logistic regression, we found that hemodynamic instability (state of shock) was the only significant predictive risk factor for in-hospital mortality among these variables (OR = 14.5, 95% CI = 1.4–153.1), with the highest capacity for prognostication of the outcome (PPV = 86.4%, NPV = 97.0%, LR+ = 35.9, LR− = 0.18, diagnostic OR = 199.4). This finding is supported by several studies examining the risk of mortality,^1–3,16 suggesting the crucial role of hemodynamic instability in the outcome after AUGIB.

Risk scoring systems, which take into consideration the specific combination of admission clinical, biochemical, and endoscopic features, are of great importance as risk adjusters for prognostication and risk assessment of the poor outcome after AUGIB.^18–22 The clinical use of risk scoring systems may lead to improved patient risk stratification, triage, and management, thus providing more accurate guidance for emergency physicians for making final decisions (hospital admission, intensive care unit admission, discharge from the emergency department). Unfortunately, no specific scoring system has been adopted in routine clinical practice for patients presenting to the emergency department. ²³ For instance, Laursen et al. ²⁴ reported that no scoring system seems to accurately predict 30-day mortality or re-bleeding. The GBS was shown to be beneficial for risk stratification in patients with UGIB before the performance of endoscopy, ⁷ thus providing an important tool for triage in the emergency department. The GBS can be easily calculated, ⁷ helping to determine the need for urgent endoscopy and endoscopic treatment ²³ as well as to assess the need for intervention and transfusion, ²⁵ determine the re-bleeding rate, and assess the need for intensive care unit admission. ⁸ However, it is not accurate enough to stratify the mortality risk. ²⁵

This study was performed to assess the predictive capacity of the GBS, full RS, and CSMCPI and their cross-validation for predicting the outcomes in patients with AUGIB. Our results revealed that non-survivors had significantly lower variability of the obtained scores, mainly obtaining only the highest scores in all of the examined systems (thus having significantly higher values when compared with survivors). Precise separation of the values for all three analyzed scoring systems (GBS, full RS, and CSMCPI) with the outcomes was observed from the generated histograms, suggesting that non-survivors can be distinguished based on their admission scores. The ROC curve analyses confirmed the high discrimination capacity of the outcomes for all of the examined systems (AUC values of >0.9), fairly identifying non-survivors with high sensitivity and specificity. Nevertheless, the bivariate statistical analyses in relation to the outcomes showed that the finest discrimination between the outcomes could be judged according to the two endoscopy-based systems (i.e., the full RS and CSMCPI), suggesting that endoscopy-based scoring systems might be more suitable for assessment of mortality. Similar findings have been reported by several research groups, showing that the GBS was inferior to the full RS in terms of 1-month mortality assessment.^8,26,27 The obtained results might be partially explained by the fact that the GBS may be confounded by its subjective components because the GBS system can predict outcomes of patients with UGIB based on subjective signs. ²⁸

According to the multiple logistic regression, the CSMCPI was the only significant independent predictor of in-hospital mortality among the three systems, thus outperforming the predictive capacity of the GBD and full RS. Actually, consistent with the overall results, the CSMCPI was consistently found to be superior to the other systems in several characteristics, such as having the highest discrimination power for correctly classifying patients (AUC = 0.991), highest sensitivity and specificity for prediction of a fatal outcome after AUGIB (100% and 96.2%, respectively), highest PPV and NPV (81.2% and 100%, respectively), highest LR+ (26.3), and clear separation as a significant risk factor for mortality (1-unit increase was associated with a 6.3 times higher odds of death). These results suggest the utility of the CSMCPI as an easily obtainable and reliable tool that can assist in treatment recommendations for patients with life-threatening bleeding disorders.

Like most studies, our study has several limitations. All data were obtained from a single center during a defined (cross-sectional) period. Replicating our results by further metacentric studies would be beneficial in terms of assessing the utility of the CSMCPI for mortality forecasting on a larger scale. Additionally, our results are restricted to patients with AUGIB; patients with chronic bleeding were not taken into consideration.

Conclusion

The presence of severe hemodynamic instability (state of shock), the state of persistent gastrointestinal bleeding, the presence of comorbidities, in-hospital bleeding onset, the presence of ischemic heart disease, and diagnosed heart failure might significantly increase the risk of mortality after AUGIB.

In contrast to the results obtained by Laursen et al., ²⁴ our cross-validation study of three risk scoring systems (GBS, full RS, and CSMCPI) showed that all scoring systems may accurately predict mortality, pointing to their validity. Nevertheless, the finest discrimination between the outcomes was judged on the basis of the full RS and CSMCPI, suggesting that endoscopy-based scoring systems might be more sensitive for assessment of mortality. The CSMCPI was the most accurate system for prognostication of mortality after AUGIB, being superior to the GBS and full RS in several characteristics and outperforming their discrimination power and predictive capacity.

Because of the high overall classification accuracy, discrimination power, and statistical significance, our developed logistic model that takes into consideration the presence of hemodynamic instability (state of shock) and the CSMCPI might be appropriate for prediction of mortality in patients with life-threatening AUGIB, thus providing an important backbone in clinical decision-making and treatment recommendations.

Research Data

sj-pdf-1-imr-10.1177_03000605221086442 - Research Data for Prediction of in-hospital mortality after acute upper gastrointestinal bleeding: cross-validation of several risk scoring systems

This article is distributed under the terms of the Creative Commons Attribution 4.0 License (http://www.creativecommons.org/licenses/by/4.0/) which permits any use, reproduction and distribution of the work without further permission provided the original work is attributed as specified on the SAGE and Open Access page (https://us.sagepub.com/en-us/nam/open-access-at-sage).

Research Data, sj-pdf-1-imr-10.1177_03000605221086442 for Prediction of in-hospital mortality after acute upper gastrointestinal bleeding: cross-validation of several risk scoring systems by Daniela Benedeto-Stojanov, Milica Bjelaković, Dragan Stojanov and Boris Aleksovski in Journal of International Medical Research

sj-pdf-2-imr-10.1177_03000605221086442 - Research Data for Prediction of in-hospital mortality after acute upper gastrointestinal bleeding: cross-validation of several risk scoring systems

This article is distributed under the terms of the Creative Commons Attribution 4.0 License (http://www.creativecommons.org/licenses/by/4.0/) which permits any use, reproduction and distribution of the work without further permission provided the original work is attributed as specified on the SAGE and Open Access page (https://us.sagepub.com/en-us/nam/open-access-at-sage).

Research Data, sj-pdf-2-imr-10.1177_03000605221086442 for Prediction of in-hospital mortality after acute upper gastrointestinal bleeding: cross-validation of several risk scoring systems by Daniela Benedeto-Stojanov, Milica Bjelaković, Dragan Stojanov and Boris Aleksovski in Journal of International Medical Research

Data availability statement

The original research data are available upon request.