Abstract
Introduction
Several prediction scores for triaging patients with upper gastrointestinal (GI) bleeding have been developed, yet these scores have never been compared to the current gold standard, which is the clinical evaluation by a gastroenterologist. The aim of this study was to assess the added value of prediction scores to gastroenterologists’ Gut Feeling in patients with a suspected upper GI bleeding.
Methods
We prospectively evaluated Gut Feeling of senior gastroenterologists and asked them to estimate: (1) the risk that a clinical intervention is needed; (2) the risk of rebleeding; and (3) the risk of mortality in patients presenting with suspected upper GI bleeding, subdivided into low, medium, or high risk. The predictive value of the gastroenterologists’ Gut Feeling was compared to the Blatchford and Rockall scores for various outcomes.
Results
We included 974 patients, of which 667 patients (68.8%) underwent a clinical intervention. During the 30-day follow up, 140 patients (14.4%) developed recurrent bleeding and 44 patients (4.5%) died. Gut Feeling was independently associated with all studied outcomes, except for the predicted mortality after endoscopy. Predictive power, based on the AUC of the Blatchford and Rockall prediction scores, was higher than the Gut Feeling of the gastroenterologists. However, combining both the Blatchford and Rockall scores and the Gut Feeling yielded the highest predictive power for the need of an intervention (AUC 0.88), rebleeding (AUC 0.73), and mortality (AUC 0.71 predicted before and 0.77 predicted after endoscopy, respectively).
Conclusions
Gut Feeling is an independent predictor for the need of a clinical intervention, rebleeding, and mortality in patients presenting with upper GI bleeding; however, the Blatchford and Rockall scores are stronger predictors for these outcomes. Combining Gut Feeling with the Blatchford and Rockall scores resulted in the most optimal prediction.
Keywords: Prediction scores, prognosis, risk, upper gastrointestinal bleeding
Introduction
There is an increasing role for evidence-based medicine in clinical practice. This is accompanied by the development of prediction scores and their use is increasingly being recommended and adopted in clinical guidelines.1 A prediction score (or risk score/decision rule) is a tool for physicians based on several predictors – such as patients’ history, physical examination, test results, and other disease characteristics – which give an estimation on the probability of a likely diagnosis, prognosis, or response to treatment.2 Such tools can be of added value for the physician in daily clinical practice.
Upper gastrointestinal (GI) bleeding is a common clinical problem and accounts for 25–35 hospitalizations per 100,000 person-years.3,4 The severity of the disease may vary from no active bleeding to rapid exsanguinations, and yet the course remains difficult to predict. Almost all patients suspected for upper GI bleeding are therefore admitted to the hospital and endoscopy is being performed within 24 hours after hospitalization.5 This results in a high pressure on hospital capacity, possibly unnecessary discomfort for the patient, and high healthcare costs. Accurate predicting of the course and outcome of upper GI bleeding should ideally facilitate triage into low- and high-risk groups and would thus help clinical management.
Several prediction scores for upper GI bleeding have been developed.6 The most commonly used scores are the Blatchford and Rockall scores.7,8 The Blatchford score is a validated score using pre-endoscopic variables, such as clinical and laboratory data, and has the primary goal to predict the need for an intervention, such as an upper endoscopy with a haemostatic procedure. The Rockall score is a validated score based on clinical, laboratory, and endoscopic variables and primarily predicts mortality. Although these scores are validated and recommended by international guidelines,5 it seems that gastroenterologists confronted with upper GI bleeding do not often incorporate these scores into clinical practice.
From previous studies we have learned that scores are more likely to be implemented if they are easy to use, if recommendations are being made based on the score (instead of just assessment), if they can be incorporated in the normal daily usual workflow, and if they are computerized.9 However, the willingness of a physician to use scores is also important. The reasons for a physician not using scores may be: they are difficult to calculate, they take time, and, most importantly, they do not add to their own clinical knowledge or ‘Gut Feeling’. Moreover, it has been reported that clinical decision making may be even better than prediction scores in predicting whether patients with upper GI bleeding should be admitted to the intensive care unit.10
In the current study, we assessed the added value of prediction scores to the Gut Feeling of gastroenterologists in patients with suspected upper GI bleeding presenting to the accident and emergency department (A&E).
Methods
Patients and outcomes
All patients of 18 years or older that were admitted to A&E for suspected upper GI bleeding (i.e. presentation with self-reported melaena or haematemesis) between October 2009 and April 2012 were included in eight participating hospitals in the Netherlands. Patients were treated according the treatment protocols of the participating centres and no interference was made with regard to patient management. Patients were followed for 30 days after presentation by the study coordinator of the participating hospital.
Upper GI bleeding was defined as ‘confirmed’ if patients with suspected upper GI bleeding met the criteria shown in Table 1. Upper GI bleeding included all haemorrhages of the upper GI tract, including peptic ulcer bleeding, variceal bleeding, Mallory-Weiss lesions, severe reflux oesophagitis and gastritis with haemorrhage, Dieulafoy’s lesions, neoplastic lesions, and angiodysplasia. Data were systematically collected using a dedicated case report form, including demographic features, data from the medical history (presenting signs or symptoms) and physical examination (blood pressure, heart rate), medication use (e.g. nonsteroidal anti-inflammatory drugs, proton pump inhibitors, and anticoagulants), comorbidities, biochemical (haemoglobin, platelet count, urea, creatinine, and International Normalized Ratio [INR]) and endoscopic findings. Comorbidities included chronic heart disease, liver cirrhosis, previous history of GI haemorrhage, presence of cancer in the GI tract or any other site, lung emphysema, renal failure (creatinine >200 µl/l or dialysis), endovascular prosthesis, diabetes mellitus, and ongoing chemotherapy or radiotherapy. Endoscopic findings included location and number of lesions, stigmata of recent haemorrhage, and Forrest classification of ulcers. Procedure-related factors included time from presentation to endoscopy, need to perform endoscopic haemostasis as judged by the endoscopist, and number of units blood transfused before and after endoscopy.
Table 1.
Diagnostic criteria for upper gastrointestinal bleeding
| Combination of reported signs of melaena and/or haematemesis with: |
| Anaemia (Hb <13.0 g/dl for men or <12.0 g/dl for women), or |
| Haemodynamic instability (a state requiring pharmacological or mechanical support to maintain a normal blood pressure or adequate cardiac output), or |
| Discrepant increased urea |
| Confirmed bleeding during endoscopy or manifest old/fresh blood |
The primary outcomes were: (1) need for clinical intervention; (2) 30-day mortality (in- and out-hospital); and (3) 30-day rebleeding rate. A clinical intervention was defined as a blood transfusion or any operative, radiological, or endoscopic intervention to control the haemorrhage and/or the occurrence of rebleeding or mortality.7 Mortality was defined as all-cause mortality in- and out-hospital. Rebleeding was defined according to the criteria set by the Peptic Ulcer Bleed study as recurrent haematemesis of fresh blood (>200 ml), active bleeding or fresh blood found during endoscopy, or two of the following: (1) Hb drop >20 g/l within 24 h; (2) Hb increase <10 g/l after adequate blood transfusion; (3) systolic RR <90 mmHg (after being higher initially); or pulse rate >110/min (after being lower initially) within 30 days after initial stabilization.11
Gut Feeling and prediction scores
Prior to upper endoscopy, the treating consultant gastroenterologists filled out a questionnaire with two questions regarding the probability of the patient needing an intervention to control the bleeding and the risk that the patient would die. After endoscopy, another two questions were filled out regarding risk of rebleeding and mortality. The questions and probabilities are shown in Table 2. The gastroenterologists estimated whether the patient was at a low, medium or high risk for these endpoints. In this study, we refer to this risk estimation by the gastroenterologists as ‘Gut Feeling’.
Table 2.
Questions regarding the Gut Feeling of the gastroenterologist
| Time point | Question | Risk category |
|---|---|---|
| At presentation at A&E | What is the risk for current bleeding requiring endoscopic treatment (or surgery/angiography) or transfusion in this patient | Low risk (<1%) |
| Medium risk (1–10%) | ||
| High risk (>10%) | ||
| What is the mortality risk (<30 days) for this patient | Low risk (<1%) | |
| Medium risk (1–5%) | ||
| High risk (>5%) | ||
| After upper endoscopy | What is the risk for continued bleeding or rebleeding requiring additional endoscopic treatment (or surgery/angiography) or transfusion in this patient | Low risk (<1%) |
| Medium risk (1–10%) | ||
| High risk (>10%) | ||
| What is the mortality risk (<30 days) for this patient | Low risk (<1%) | |
| Medium risk (1–5%) | ||
| High risk (>5%) |
The full Rockall score and the Blatchford score were calculated for each patient. The Rockall score consists of both clinical and laboratory variables, and endoscopic findings and the Blatchford score consists only of clinical and laboratory variables (Table 3). The Rockall score was used for predicting rebleeding and mortality; the Blatchford score was used for predicting the need of intervention and mortality. For predicting or excluding an intervention we used a cut off of <1 for the Blatchford score as this is the cut-off level mostly used in literature and validation studies.7,12 For predicting rebleeding and mortality for both scores a cut off of >2 points was used to compare the performance of the scores with Gut Feeling. Patients with ≤2 points were classified as low risk and >2 points as medium to high risk.7,8,12,13
Table 3.
Variables tested according to use of Rockall and Blatchford scores
| Score | Variable |
|---|---|
| Rockall | Age |
| Shock (blood pressure and heart rate) | |
| Comorbidities | |
| Diagnosis post upper gastrointestinal endoscopy | |
| Stigmata of recent haemorrhage | |
| Blatchford | Blood urea |
| Haemoglobin men/women | |
| Systolic blood pressure | |
| Other markers (pulse/melaena/syncope/hepatic disease/heart failure) |
Statistical analysis
Patient characteristics, comorbidities, and endoscopic findings were analysed using standard descriptive statistics. Sensitivity and specificity rates and negative and positive predictive values were calculated for both Gut Feeling (medium/high risk compared to low risk and high risk compared to low/medium risk) and the prediction scores at a cut-off level of <1 for predicting/excluding the need of an intervention and >2 for predicting rebleeding and mortality. Logistic regression analyses were performed to assess the association between Gut Feeling and each individual outcome (need for intervention, rebleeding, and mortality predicted before and after endoscopy), as well for the prediction scores and the various outcomes. The area under receiver operating curve (AUC) was calculated to compare the discriminative power of the Gut Feeling and the prediction scores, with an AUC of 0.5 indicating no and a value of 1.0 indicating perfect discrimination between high and low risk. Ordinal logistic regression analyses were performed to identify risk factors associated with Gut Feeling. Statistical analysis was performed with SPSS 14.0 (SPSS, Chicago, IL, USA).
Ethical considerations
This was a prospective observational study, in which patient data were entered anonymously in a central database. The protocol did not include any (additional) interventions and no additional testing was performed. Therefore, the Dutch Law on Medical Research on Humans did not apply here and approval by a medical ethical committee was not required, as agreed by the Medical Ethics Committee of the Sint Antonius Hospital, Nieuwegein, the Netherlands (20 July 2009).
Results
In total, 1001 patients were included, with Gut Feeling being completed for 974 patients (Figure 1). The mean age of the studied population was 65 (range 18–99) years and 37% of the patients were female. Other baseline characteristics are shown in Table 4.
Figure 1.
Flowchart of study population.
Table 4.
Baseline characteristics for different outcomes
| Characteristic | All patients (n = 974) |
|---|---|
| Age (years, mean and range) | 66 (18–99) |
| Sex (female) | 355 (36.5) |
| Medical history | |
| Melaena | 614 (63.1) |
| Haematemesis | 442 (45.4) |
| Rectal blood loss | 145 (14.9) |
| Collapse | 153 (15.7) |
| Medication use | |
| Oral anti-coagulants | 245 (25.2) |
| Corticosteroids | 54 (5.5) |
| NSAID | 103 (10.6) |
| Acetylsalicylic acid | 309 (31.7) |
| Clopidogrel | 87 (8.9) |
| PPI | 330 (33.9) |
| SSRI | 42 (4.3) |
| Physical examination | |
| Systolic BP (mm Hg) | 127 ± 25 |
| Diastolic BP (mm Hg) | 69 ± 17 |
| HR (beats per min) | 90 ± 19 |
| Laboratory results | |
| Hb level (mmol/L) | 6.3 ± 1.9 |
| Platelet count (*109) | 246 ± 111 |
| Creatinine (μmol) | 108 ± 80 |
| Urea (mmol/L) | 13.8 ± 11.0 |
| International Normalized Ratio | 1.90 ± 2.15 |
| Comorbidities | |
| Liver cirrhosis | 105 (10.8) |
| History of upper GI bleeding | 214 (22.0) |
| Presence of GI cancer | 41 (4.2) |
| Chronic heart disease | 331 (34.0) |
| Lung emphysema | 122 (12.5) |
| Renal failure | 74 (7.6) |
| Endovascular prosthesis | 92 (9.4) |
| Diabetes mellitus | 170 (17.5) |
| Active chemo- or radiotherapy | 17 (1.7) |
| Active cancer on other site than GI tract | 54 (5.5) |
| Endoscopic findings | |
| Confirmed upper GI bleeding | 733 (76.4) |
| Variceal bleeding | 75 (7.5) |
| Causes of nonvariceal bleeding | |
| Peptic ulcer bleed | 352 (36.1) |
| Oesophagitis | 90 (9.5) |
| Malignancy | 25 (2.6) |
| Other (e.g. Mallory Weiss tear, Dieulafoy lesion, hypertensive gastropathy) | 266 (36.3) |
| Other | |
| Duration of admission (days, median and interquartile range) | 4 (2–7) |
| Do not resuscitate status | 163 (17.0) |
| Patients with one or more blood transfusion | 569 (58.9) |
| Patients receiving surgery | 16 (1.6) |
| Patients receiving angiography | 22 (2.3) |
Values are n (%) or mean ± SD unless otherwise stated.
BP, Blood pressure; GI, gastrointestinal; NSAID, nonsteroidal anti-inflammatory drug; PPI, proton pump inhibitor; SSRI, selective serotonin re-uptake inhibitor.
Prediction of need for clinical intervention
In total, 667 patients (69%) underwent a clinical intervention (e.g. blood transfusion, operative or endoscopic procedure to control the haemorrhage). The number of interventions increased significantly with higher estimated risks by the Gut Feeling, as well as with increasing Blatchford scores (Figure 2). The Blatchford score showed high sensitivity rates while the Gut Feeling revealed higher specificity rates (Table 5). After correcting for the Blatchford score, the Gut Feeling was still independently associated with the need for an intervention (odds ratio, OR, 1.5, 95% CI 1.0–2.3 for medium risk; OR 4.9, 95% CI 3.0–8.2 for high risk). However, the Blatchford score had a better predictive power than the Gut Feeling of the gastroenterologists (AUC 0.86 vs. 0.78, respectively). Combining the Gut Feeling with the Blatchford score improved the predictive power to 0.88 (Figure 3). Regression analyses showed that haemoglobin and urea levels, haematemesis, and a history of collapse had the highest effect on the Gut Feeling of gastroenterologists.
Figure 2.
Association Gut Feeling and prediction scores with outcome. (a) Prediction of clinical intervention. (b) Prediction of rebleeding. (c) Prediction of mortality before endoscopy. (d) Prediction of mortality after endoscopy. Left Y-axes illustrate the percentage of patients with outcome predicted by the Gut Feeling; right Y-axes illustrate the mean prediction score for every group of the Gut Feeling.
Table 5.
The sensitivity, specificity and predictive values of Gut Feeling and the prediction scores for various outcomes
| Outcome | Gut Feeling |
Prediction score | |
|---|---|---|---|
| Low risk vs. medium/high risk | High risk vs. low/medium risk | ||
| Intervention needed: n = 667 (68.8%) | Blatchford score: cut off <1 | ||
| Sensitivity | 551/667 (82.6) | 345/667 (51.7) | 663/667 (99.4) |
| Specificity | 181/303 (59.7) | 273/303 (90.5) | 41/262 (13.5) |
| NPV | 181/297 (60.9) | 273/595 (45.9) | 41/45 (91.1) |
| PPV | 551/673 (81.9) | 345/375 (92.0) | 663/925 (71.7) |
| Rebleeding: n = 140 (14.4%) | Rockall score: cut off <2 | ||
| Sensitivity | 99/140 (70.7) | 54/140 (38.6) | 133/140 (95.0) |
| Specificity | 479/801 (59.8) | 696/801 (86.9) | 190/816 (23.3) |
| NPV | 479/520 (92.1) | 696/782 (89.0) | 190/197 (96.4) |
| PPV | 99/421 (23.5) | 54/159 (34.0) | 133/759 (17.5) |
| Mortality predicted before endoscopy: n = 44 (4.5%) | Blatchford score: cut off <2 | ||
| Sensitivity | 34/44 (77.3) | 18/44 (40.9) | 41/43 (95.3) |
| Specificity | 488/924 (52.8) | 785/924 (85.0) | 121/900 (13.4) |
| NPV | 488/498 (98.0) | 785/811 (96.8) | 121/123 (98.4) |
| PPV | 34/470 (7.2) | 18/157 (11.5) | 41/820 (5.0) |
| Mortality predicted after endoscopy: n = 44 (4.5%) | Rockall score: cut off <2 | ||
| Sensitivity | 25/41 (61.0) | 11/41 (26.8) | 43/44 (97.7) |
| Specificity | 611/896 (68.2) | 811/896 (90.5) | 196/908 (21.6) |
| NPV | 611/627 (97.4) | 811/841 (96.4) | 196/197 (99.5) |
| PPV | 25/310 (8.1) | 11/96 (11.5) | 43/755 (5.7) |
Values are n/total (%).
NPV, negative predictive value; PPV, positive predictive value.
Figure 3.
ROC curves for various outcomes.
Prediction of rebleeding
In total, 140 patients (14.4%) developed a rebleeding. The rebleeding rate and the mean Rockall score increased significantly with higher estimated risks by the Gut Feeling (Figure 2). Sensitivity rates were highest for the Rockall score and specificity rates for the Gut Feeling (Table 5). The Gut Feeling was independently of the Rockall score associated with rebleeding (OR 1.7, 95% CI 1.1–2.7 for medium risk; OR 3.4, 95% CI 2.1–5.7 for high risk). Rebleeding was slightly better predicted by the Rockall score compared to the Gut Feeling, but both predict rebleeding only to a moderate degree (AUC 0.69 vs. 0.68, respectively). Combining the Rockall score and the Gut Feeling improved the predictive power to an AUC of 0.73 (Figure 3). An intervention and, to a lesser extent, haematemesis and a history of a collapse had the highest effect on the Gut Feeling of gastroenterologists.
Prediction of mortality
Forty-four patients (4.5%) died. The intra-observer agreement between the Gut Feeling before and after endoscopy was low (kappa 0.41), suggesting that the results of endoscopy significantly changed the gastroenterologists’ views on the prognosis. Mortality rates increased with higher Gut Feeling risk estimations before and after endoscopy (Figure 2). The sensitivity of the Gut Feeling was highest before endoscopy, while the specificity was highest after endoscopy. Both Blatchford and Rockall scores showed high sensitivity rates (95.6% and 95.7%, respectively) (Table 5). A high Gut Feeling risk estimation before endoscopy was a significant predictor for mortality, also after adjustment for the risk scores (OR 1.8, 95% CI 0.8–4.1 for medium risk; OR 3.3, 95% CI 1.4–8.0 for high risk); however, the Gut Feeling after endoscopy was no predictor after correcting for the Rockall score (OR 1.4, 95% CI 0.6–3.0 for medium risk; OR 2.0, 95% CI 0.8–4.7 for high risk). The predictive power of the Gut Feeling and the risk scores are shown in Figure 3. The Rockall score combined with the Gut Feeling provided the most optimal prediction for mortality (AUC 0.77). The Gut Feeling before endoscopy was mostly affected by haemoglobin level, liver disease, haematemesis, and a do not resuscitate status of the patient. The Gut Feeling after endoscopy was mostly affected by a do not resuscitate status, liver disease, and whether an intervention was performed.
Discussion
We found that the Gut Feeling of gastroenterologists was a good predictor for the need of a clinical intervention, rebleeding, and mortality in patients presenting with upper GI bleeding; however, the Blatchford and Rockall prediction scores had overall higher predictive values. Prediction scores had a higher sensitivity and thus performed better in excluding an unfavourable outcome, while the Gut Feeling showed a higher specificity and thus had a better performance in predicting an unfavourable outcome.
We observed an overall tendency in gastroenterologists to overestimate the risk of an adverse outcome (positive predictive value), especially for the risk of rebleeding and mortality. An explanation could be that the estimation of gastroenterologists of the risk of rebleeding and mortality is based on older literature citing high rebleeding and mortality rates while a considerable reduction in rebleeding and mortality has been observed over the last decade as a result of better acid suppression, advanced endoscopy haemostatic techniques, and improvement of radiological haemostatic interventions.14,15
With regard to the outcomes in this study, both the Gut Feeling and the prediction scores performed well in excluding mortality and rebleeding as high negative predictive values were observed for those endpoints. For predicting the need of a clinical intervention, high positive predictive values were observed, especially for the Gut Feeling, meaning that gastroenterologists were able to predict the need for an intervention. The high sensitivity rates of the Blatchford and Rockall scores can be used to select patients who need closer monitoring. A high specificity – such as we found for the Gut Feeling – is however also of clinical importance because hospital admission and endoscopy are both a burden to patients, but to some extent also to the hospital, and a decision to do so should be based on well established risk factors.
These risk factors used in the prediction scores and the risk factors gastroenterologists based their Gut Feeling on are different for the various outcomes. For predicting the need of a clinical intervention, gastroenterologists mainly based their Gut Feeling on low haemoglobin levels and presentation with haematemesis and collapse. In the Blatchford score, a collapse and haemoglobin levels also play an important role, but other parameters used in this score, such as blood urea levels, tachycardia, and blood pressure were not used to a large extent by gastroenterologists.7 A recent systematic review by Srygley et al.16 identified tachycardia and haemoglobin level as important predictors for a clinical intervention, while they also identified a history of cirrhosis, malignancy, and nasogastric lavage with red blood as risk factors. For rebleeding, the most important predictors identified in the literature are age, comorbidities, active bleeding, and location of the bleeding.17–19 The Gut Feeling of gastroenterologists was, however, mainly based on whether an intervention was performed (which might be a reflection of active bleeding) and to a smaller extent on haematemesis and collapse. Lastly, important predictors of mortality found in literature are age, comorbidities, and haemodynamic instability.8,20–23 Gastroenterologists found haemoglobin level, liver disease, haematemesis, and a do not resuscitate status the most important indicators for mortality, which partly overlap with the predictors known from the literature.
This study has several strengths and limitations. This is the first study that shows that prediction scores are better than clinical estimation of experienced gastroenterologists in risk classification of patients with upper GI bleeding. Secondly, not only patients with an established upper GI bleeding but also patients with a suspected upper GI bleeding were included, as this is clearly a better reflection of the population actually presenting to the A&E and requiring risk classification. Moreover, the predictive power of the Blatchford and Rockall scores may increase somewhat after excluding variceal bleeding as these scores were originally developed for patients with nonvariceal bleeding, resulting in a larger difference than Gut Feeling. A limitation of this study was that due to the prospective design, we were unable to compare Gut Feeling to other more recently developed risk scores (e.g. PNED, AIM65)21,23 as we did not include variables such as time from onset of symptoms to admission and albumin. Retrospective collection of these parameters would likely have resulted in missing values and bias.
The results of our study may have important implications for clinical practice. We have shown that the use of prediction scores increases the predictive power for all clinical relevant outcomes of upper GI bleeding over Gut Feeling. The use of these scores will thus lead to a better prediction of the need for a clinical intervention. The risk scores were superior to the clinical estimation of experienced gastroenterologists, and given that A&Es are usually manned by less experienced registrars and junior house officers, there is definitely a reason for incorporation of these scores in clinical practice. However, we have also shown that combining prediction scores with the Gut Feeling of a gastroenterologist predicts even more accurately. This emphasizes the need for gastroenterologists always to be included in clinical decision making and triaging of patients presenting with upper GI bleeding. Based on our results, we propose to combine Gut Feeling and the established risk scores, balancing both sensitivity and specificity; however, this needs be confirmed in a prospective follow-up study.
In conclusion, this is the first study in which the added value of prediction scores was compared to Gut Feeling of gastroenterologists in predicting the outcome in patients with upper GI bleeding. We found that Gut Feeling is an independent predictor for adverse outcome; however, prediction scores have higher sensitivity and better predictive power compared to Gut Feeling. We therefore suggest using prediction scores in combination with Gut Feeling of gastroenterologists in making clinical decisions regarding treatment and monitoring of patients with upper GI bleeding.
Funding
This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.
Conflict of interest
The authors declare that there is no conflict of interest.
References
- 1.Toll DB, Janssen KJ, Vergouwe Y, et al. Validation, updating and impact of clinical prediction rules: a review. J Clin Epidemiol 2008; 61: 1085–1094 [DOI] [PubMed] [Google Scholar]
- 2.Laupacis A, Sekar N, Stiell IG. Clinical prediction rules. A review and suggested modifications of methodological standards. JAMA 1997; 277: 488–494 [PubMed] [Google Scholar]
- 3.Lanas A, Garcia-Rodriguez LA, Polo-Tomas M, et al. The changing face of hospitalisation due to gastrointestinal bleeding and perforation. Aliment Pharmacol Ther 2011; 33: 585–591 [DOI] [PubMed] [Google Scholar]
- 4.Ahsberg K, Ye W, Lu Y, et al. Hospitalisation of and mortality from bleeding peptic ulcer in Sweden: a nationwide time-trend analysis. Aliment Pharmacol Ther 2011; 33: 578–584 [DOI] [PubMed] [Google Scholar]
- 5.Barkun AN, Bardou M, Kuipers EJ, et al. International consensus recommendations on the management of patients with nonvariceal upper gastrointestinal bleeding. Ann Intern Med 2010; 152: 101–113 [DOI] [PubMed] [Google Scholar]
- 6.de Groot NL, Bosman JH, Siersema PD, et al. Prediction scores in gastrointestinal bleeding: a systematic review and quantitative appraisal. Endoscopy 2012; 44: 731–739 [DOI] [PubMed] [Google Scholar]
- 7.Blatchford O, Murray WR, Blatchford M. A risk score to predict need for treatment for upper-gastrointestinal haemorrhage. Lancet 2000; 356: 1318–1321 [DOI] [PubMed] [Google Scholar]
- 8.Rockall TA, Logan RF, Devlin HB, et al. Risk assessment after acute upper gastrointestinal haemorrhage. Gut 1996; 38: 316–321 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Kawamoto K, Houlihan CA, Balas EA, et al. Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ 2005; 330: 765–765 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Farooq FT, Lee MH, Das A, et al. Clinical triage decision vs risk scores in predicting the need for endotherapy in upper gastrointestinal bleeding. Am J Emerg Med 2012; 30: 129–134 [DOI] [PubMed] [Google Scholar]
- 11.Sung JJ, Mossner J, Barkun A, et al. Intravenous esomeprazole for prevention of peptic ulcer re-bleeding: rationale/design of Peptic Ulcer Bleed study. Aliment Pharmacol Ther 2008; 27: 666–677 [DOI] [PubMed] [Google Scholar]
- 12.Stanley AJ, Dalton HR, Blatchford O, et al. Multicentre comparison of the Glasgow Blatchford and Rockall scores in the prediction of clinical end-points after upper gastrointestinal haemorrhage. Aliment Pharmacol Ther 2011; 34: 470–475 [DOI] [PubMed] [Google Scholar]
- 13.Enns RA, Gagnon YM, Barkun AN, et al. Validation of the Rockall scoring system for outcomes from non-variceal upper gastrointestinal bleeding in a Canadian setting. World J Gastroenterol 2006; 12: 7779–7785 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Crooks C, Card T, West J. Reductions in 28-day mortality following hospital admission for upper gastrointestinal hemorrhage. Gastroenterology 2011; 141: 62–70 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Button LA, Roberts SE, Evans PA, et al. Hospitalized incidence and case fatality for upper gastrointestinal bleeding from 1999 to 2007: a record linkage study. Aliment Pharmacol Ther 2011; 33: 64–76 [DOI] [PubMed] [Google Scholar]
- 16.Srygley FD, Gerardo CJ, Tran T, et al. Does this patient have a severe upper gastrointestinal bleed? JAMA 2012; 307: 1072–1079 [DOI] [PubMed] [Google Scholar]
- 17.Garcia-Iglesias P, Villoria A, Suarez D, et al. Meta-analysis: predictors of rebleeding after endoscopic treatment for bleeding peptic ulcer. Aliment Pharmacol Ther 2011; 34: 888–900 [DOI] [PubMed] [Google Scholar]
- 18.Lanas A, Aabakken L, Fonseca J, et al. Clinical predictors of poor outcomes among patients with nonvariceal upper gastrointestinal bleeding in Europe. Aliment Pharmacol Ther 2011; 33: 1225–1233 [DOI] [PubMed] [Google Scholar]
- 19.Saeed ZA, Winchester CB, Michaletz PA, et al. A scoring system to predict rebleeding after endoscopic therapy of nonvariceal upper gastrointestinal hemorrhage, with a comparison of heat probe and ethanol injection. Am J Gastroenterol 1993; 88: 1842–1849 [PubMed] [Google Scholar]
- 20.Almela P, Benages A, Peiro S, et al. A risk score system for identification of patients with upper-GI bleeding suitable for outpatient management. Gastrointest Endosc 2004; 59: 772–781 [DOI] [PubMed] [Google Scholar]
- 21.Marmo R, Koch M, Cipolletta L, et al. Predicting mortality in non-variceal upper gastrointestinal bleeders: validation of the Italian PNED Score and Prospective Comparison with the Rockall Score. Am J Gastroenterol 2010; 105: 1284–1291 [DOI] [PubMed] [Google Scholar]
- 22.Chiu PW, Ng EK, Cheung FK, et al. Predicting mortality in patients with bleeding peptic ulcers after therapeutic endoscopy. Clin Gastroenterol Hepatol 2009; 7: 311–316 [DOI] [PubMed] [Google Scholar]
- 23.Saltzman JR, Tabak YP, Hyett BH, et al. A simple risk score accurately predicts in-hospital mortality, length of stay, and cost in acute upper GI bleeding. Gastrointest Endosc 2011; 74: 1215–1224 [DOI] [PubMed] [Google Scholar]