Do Scoring Systems Help in Predicting Survival Following Ruptured Abdominal Aortic Aneurysm Surgery?

Abstract

INTRODUCTION

The aim of this study was to assess the value of the Hardman Index and the Glasgow Aneurysm Score in predicting postoperative mortality in patients with ruptured abdominal aortic aneurysm (rAAA), and to assess the correlation between the two.

PATIENTS AND METHODS

Patients admitted with rAAA were identified from a hospital database. Hospital records were reviewed and a retrospective Hardman Index and Glasgow Aneurysm Score was calculated. Poor postoperative prognosis was considered at a Glasgow Aneurysm Score > 95 or a Hardman Index ≥ 3.

RESULTS

A total of 96 patients with a median age of 77.5 years (interquartile range, 71–83 years) and a male:female ratio of 2:1 were identified. Of these, 37 patients were not offered surgery and this was associated with 100% mortality. Of the 59 operated patients, 36 (61%) patients died postoperatively. Operated patients had a median Glasgow Aneurysm Score of 91 (interquartile range, 77–101) and a Hardman Index of 2 (interquartile range, 1–2). In this group, a Glasgow Aneurysm Score > 95 or a Hardman Index ≥ 3 was not associated with mortality (P = 0.10 and P = 0.79, respectively). Correlation between the scoring systems was poor (+0.42 τ_b).

CONCLUSIONS

The scoring systems assessed did not help predict the outcome of rAAA surgery, and correlated poorly with each other. They do not aid clinical judgement.

Ruptured abdominal aortic aneurysms (rAAA) are a major cause of death in the Western world and account for 2% (6000 per year) of all deaths in men in the UK.¹ Despite an increase in early detection, as well as an upsurge in elective repairs, the incidence of rAAA may still be rising.^2,3 Less than 50% manage to survive long enough to be hospitalised, of whom a further 30–70% succumb despite undergoing surgery.^2,4–6 The cumulative mortality for this vascular catastrophe has been reported to be around 80–88%,⁷ and as high as 95% in some series.^8,9

With the continued emphasis on efficiency of service provision and cost-containment, the possibility of withholding repair from some patients if their death was predictable has been entertained by some authors.^10–17 A number of scoring systems have been developed with the aim of identifying patients less likely to survive surgery based on a number of pre-operative factors. Two such statistical models are the Hardman Index¹⁶ and the Glasgow Aneurysm Score.¹⁷ The aim of this report was to assess the value of these two scoring systems in predicting postoperative mortality in patients with rAAA.

Patients and Methods

As part of an on-going departmental audit, all patients admitted to one district general hospital with a diagnosis of rAAA between February 1998 and February 2003 were identified from a computerised patient administration hospital database. This hospital admitted all surgical patients in the catchment area. The medical records for these patients were reviewed individually and basic demographics were recorded. A Hardman Index and a Glasgow Aneurysm Score were calculated retrospectively based on criteria set down in the original description of these statistical models.^16,17 Individual scores were related to postoperative mortality and other clinical outcomes.

The Hardman Index was calculated from five pre-operative variables (Table 1). A point was awarded for an age > 76 years, a haemoglobin < 9.0 g/dl, a serum creatinine > 190 μmol/l, for ischaemic changes on an electrocardiogram, and a history of loss of consciousness.¹⁶ Patients could score a minimum of 0 and a maximum of 5. An Hardman Index equal to or greater than 3 was considered to indicate a patient at high risk of surgical mortality since this has been reported to be almost universally fatal.¹⁸ The Glasgow Aneurysm Score was also calculated from five pre-operative variables (Table 2). Points were awarded for age, shock, myocardial disease, cerebrovascular disease and renal impairment using the formula: Glasgow Aneurysm Score = age (in years) + 17 for shock + 7 for myocardial disease + 10 for cerebrovascular disease + 14 for renal disease.¹⁷ A Glasgow Aneurysm Score of greater than 95 was considered to indicate a patient at high risk of surgical mortality since this has been associated with a mortality of 80% or more.¹⁹

Table 1.

The Hardman index

Parameter	Points
Age > 76 years	1
Loss of consciousness	1
Haemoglobin < 9 g/dl	1
Serum creatinine >μ190 μmol/l	1
Ischaemic changes on ECG	1

Table 2.

The Glasgow Aneurysm Score

Parameter	Points awarded
Age	(years)
Shock	17
Myocardial disease	7
Cerebrovascular disease	10
Renal disease	14

For purposes of this paper, rAAA was defined as the presence of intraperitoneal or retroperitoneal blood in the presence of an abdominal aortic aneurysm. Surgery was defined as the administration of an anaesthetic with the aim of attempting rAAA repair. Postoperative death was recorded for all in-hospital fatalities to include death up to 1 month after surgery in patients that had been discharged home. Exclusion criteria included patients in whom the medical notes could not be located or in whom these were unhelpful to calculate an Hardman Index or a Glasgow Aneurysm Score. Patients were also excluded if they had been deemed unfit for elective repair or were dead on arrival. In the index institution, chronological age is not considered as an independent contra-indication for surgery.

Results were tabulated on an Excel® spreadsheet (Excel for Windows®, Microsoft Corporation, Redmond, WA, USA) and then analysed using SPSS for Windows® v.11.5 (SPSS, Chicago, IL, USA). Results of non-parametric data were presented as median and interquartile ranges. Relationships between groups were assessed using χ² test for binary outcomes, and Fisher's exact test for small cohorts.

Continuous variables were compared using the Mann– Whitney U-test for skewed data. Correlation between scoring systems was explored using Kendall's tau-b (τ_b) correlation coefficient for data which is not normally distributed or has ordered categories. The sensitivity, specificity, positive and negative predictive values together with the respective 95% confidence intervals (95% CI) of the two scoring systems was estimated. The relationship between the specificity and sensitivity of continuous variables was explored by plotting receiver operator curves (ROC). Statistical significance was considered at the 5% level. All results are quoted to the closest 2 decimal places.

Results

A total of 99 patients with a diagnosis of rAAA were identified. Of these, three had to be excluded from data analysis, two because of missing information precluding full Glasgow Aneurysm Score or Hardman Index scoring, and one because the medical notes were not available. The 96 patients finally included for data analysis had a median age of 77.5 years (interquartile range, 71–83 years) and a M:F ratio of 2:1. Of these 96 patients, 37 were treated conservatively with a mortality of 100%. Eighteen patients had previously been deemed unfit for elective repair because of significant co-existing morbidity (8 cardiac, 4 respiratory, 3 malignant and 3 advanced dementia), 15 patients did not consent for surgery, and 4 were deemed clinically unfit for surgery by an attending vascular surgeon.

Males patients were younger than females (median, 76 years; interquartile range, 69–82 years versus 81 years; interquartile range, 75.5–85.5 years; P < 0.01). More males underwent surgery than females (46 of 65 [71%] males versus 13 of 31 [42%] females; P < 0.01). Patient characteristics are summarised in Table 3.

Table 3.

Basic characteristics of the different patient cohorts

	Whole cohort (n = 96)	Patients treated conservatively (n = 37)	Operated patients(n = 59)	P-valueb
M:F (ratio)	65:31 (2:1)	19:18 (1:1)	46:13 (3.5:1)	< 0.01
Age (years)a	77.5 (71–83)	83 (75–87)	76 (69–80.5)	< 0.01
Hardman Indexa	2 (1–2)	2 (1–3)	2 (1–2)	0.03
n < 3	73	24	49
n ≥ 3	23	13	10
Glasgow Aneurysm Scorea	97 (84–104)	104 (96–116)	91 (78–101)	< 0.01
n ≤ 85	26	3	23
n > 85	70	34	36
n > 95	51	28	23
Mortality (%)	73/97 (75%)	37/37 (100%)	36/59 (61%)	< 0.01
Hardman Index < 3	53/73 (73%)	24/24 (100%)	29/49 (59%)
Hardman Index ≥ 3	20/23 (87%)	13/13 (100%)	7/10 (70%)
Glasgow Aneurysm Score ≤ 85	15/26 (58%)	3/3 (100%)	12/23 (52%)
Glasgow Aneurysm Score > 85	58/70 (83%)	34/34 (100%)	24/36 (67%)
Glasgow Aneurysm Score > 95	45/51 (88%)	28/28 (100%)	17/23 (74%)

^aValues are median (interquartile ranges).

^bComparisons between the conservatively-treated and operated groups.

A total of 59 patients underwent emergency rAAA surgery. These had a median age of 76 years (interquartile range, 69–80.5 years), and a M:F ratio of 3.5:1. Operated patients were significantly younger than those treated conservatively (P < 0.01). The two groups also differed significantly in their gender distribution (P<0.01). The overall postoperative mortality was 61%.

The sensitivity, specificity, positive predictive value and negative predictive value for different Hardman Index and Glasgow Aneurysm Score estimates are summarised in Table 4. A ROC curve was plotted to analyse the relationship between the specificity and sensitivity of the Glasgow Aneurysm Score and postoperative mortality (Fig. 1). The area under the curve was 0.62 (SE 0.08; 95% CI, 0.48–0.77; P = 0.11). In this cohort of patients, a Glasgow Aneurysm Score of more than 95 compared poorly with an Hardman Index of ≥ 3 (P < 0.01, χ² test) as suggested in the literature. The correlation between the two scoring systems was +0.42 (τ_b). Ten (43%) of the 23 patients with a Glasgow Aneurysm Score > 95 simultaneously had an Hardman Index ≥ 3 while all the 10 patients with a Hardman Index ≥ 3 had a Glasgow Aneurysm Score > 95. Further, as summarised in Table 4, an Hardman Index of ≥ 3, a Glasgow Aneurysm Score of > 85, or a Glasgow Aneurysm Score of > 95 were not associated with an increased risk of postoperative death (70% mortality, P = 0.79; 67% mortality, P = 0.27; 74% mortality, P = 0.10, respectively).

Table 4.

Association of Hardman Index and Glasgow Aneurysm Score with mortality in 59 operated patients

Score	Mortality (%)	Sensitivity (95% CI)	Specificity (95% CI)	Positive predictive value (95% CI)	Negative predictive value (95% CI)	P-value
HI 0	4/9 (44)	0.11 (0.05–0.18)	0.78 (0.69–0.89)	0.44 (0.20–0.72)	0.36 (0.32–0.41)	0.46*
HI 1	10/18 (56)	0.28 (0.19–0.37)	0.65 (0.51–0.79)	0.56 (0.37–0.73)	0.37 (0.29–0.44)	0.57
HI 2	15/22 (68)	0.42 (0.32–0.50)	0.70 (0.54–0.83)	0.68 (0.52–0.82)	0.43 (0.34–0.52)	0.38
HI ≥ 3	7/10 (70)	0.19 (0.12–0.25)	0.87 (0.75–0.95)	0.70 (0.42–0.89)	0.41 (0.35–0.45)	0.79*
GAS ≤ 85	12/23 (52)	0.33 (0.24–0.43)	0.52 (0.38–0.67)	0.52 (0.38–0.67)	0.33 (0.24–0.43)	0.27
GAS >85	24/36 (67)	0.67 (0.57–0.76)	0.48 (0.33–0.63)	0.67 (0.57–0.76)	0.48 (0.33–0.63)	0.27
GAS > 95	17/23 (74)	0.47 (0.37–0.55)	0.74 (0.58–0.86)	0.74 (0.58–0.86)	0.47 (0.37–0.55)	0.10

HI = Hardman Index; GAS = Glasgow Aneurysm Score;

^*Fisher's exact test.

Figure 1.

The ROC curve describing the sensitivity against the specificity of the Glasgow Aneurysm Score in predicting postoperative death. AUC = 0.62; SE 0.08; 95% CI, 0.48–0.77; P = 0.11.

Discussion

The ability to select patients that are likely to survive surgery for ruptured aneurysms has its appeal. Indeed, the high mortality of this condition has both ethical and resource implications; resources which could otherwise be directed at other patient groups with similar, albeit possibly less immediate, needs.²⁰ A number of statistical models have been formulated to aid in the identification of such patients. This paper considers the merit of the Glasgow Aneurysm Score and Hardman Index systems in predicting postoperative mortality.

The results of this study indicate that both the Hardman Index and the Glasgow Aneurysm Score are poor predictors of postoperative mortality following ruptured abdominal aneurysm repair. The two scoring systems also correlated poorly with each other. Of the 10 patients that scored 3 or more on the Hardman Index, three (30%) survived surgery despite suggestions in the literature that this cut-off value is almost universally fatal.¹⁸ Six (26%) of the 23 patients that scored more than 95 on the Glasgow Aneurysm Score had the same favourable result. Whilst the potential of such statistical models to predict outcomes may seem appealing, our results suggest otherwise. Refusing patients potentially life-saving surgery based solely on inaccurate statistical models may be questionable.

Retrospective studies are often fraught with problems of incomplete data.²¹ In our cohort, the number of exclusions necessary because of missing data was small. The reasons for this may include the computerisation of haematological results, a protocol-driven admission policy for rAAA patients in the index institution, as well as the active involvement of only two vascular consultant surgeons (NELB and EPP) in a relatively small unit and the insistence on the documentation of the relevant variables prior to the consideration for surgery.

The authors acknowledge the limitations of interpretations based on retrospective data collections. Whilst the possibility of bias cannot be fully excluded, we feel that there is mounting evidence to support our findings of this trial. Other investigators have raised doubt on the robustness of both the Hardman Index as well as the Glasgow Aneurysm Score in predicting postoperative mortality after rAAA surgery. Tambyraja et al.²⁰ have recently published an article comparing the two scoring systems with astonishingly near-identical findings to this series. Doubts raised by this and other similar studies stress the importance of clinical assessment by an experienced surgeon in favour of reliance on statistical models. In our cohort, the only way to predict a patient's outcome with some certainty was not to offer surgery.

Despite recent advances in patient care, poor outcomes following rAAA remain prohibitively high. Most centres quote a surgical mortality of 30–70% and an overall mortality of 80–90%.^{2–7,9,10,12} Numerous factors are known to effect postoperative mortality.^16,17,22 These include pre-operative cardiac, renal and neurological failure. Increased age²³ and the female gender²⁴ have also been associated with poorer outcomes. The postoperative mortality in this cohort was 61% rising to 75% when all patients, including those treated conservatively, are considered. Similar to many series, a conservative approach for true rupture was associated with a universal mortality. The authors feel that the relatively high postoperative mortality in this cohort may, in part, be explained by the methodology employed to include 30-day mortality, even in those patients that had been successfully discharged home. Further, the authors noted that the cohort of patients in this series comprised a relatively older population than many other published studies and there was also a comparatively higher proportion of female patients. These high mortality statistics may have influenced the performance of both the Hardman Index and the Glasgow Aneurysm Score, but much larger patient numbers would be required to investigate this. However, with reference to the Glasgow Aneurysm Score, ROC curve analysis suggests that the Glasgow Aneurysm Score is a poor test for predicting postoperative outcome, with an area under the curve of only 0.624. This is in accordance with the results of other investigators who have critically assessed the Glasgow Aneurysm Score.²⁰

Proportionally fewer women underwent surgery and these were of an older age group. Whilst the authors do not intentionally preclude surgery based solely on a patient's chronological age or indeed gender, these findings are once again consistent with numerous other studies in the literature. In a study of 692 patients with a rAAA, Evans et al.²⁴ reported that females were less likely to be selected for an operation than their male counterparts, and that women who were not operated on were significantly older than men. Similarly, from their series of 3570 patients, Dueck et al.²⁵ reported that men were more likely to undergo surgical repair than women, at 80% versus 58%. The reasons for this are not entirely clear. Multivariate analysis, however, did not support the observation that increased age in the female cohort was an independent contributing factor for this. In a trial of 873 patients, Semmens et al.²⁶ concluded that women were more likely to die of rAAA than males; however, whether gender alone is causal to poorer outcomes or simply a result of a more conservative approach to the female patient remains unclear.

Conclusions

Scoring systems may have their usefulness in assisting surgeons' clinical decisions when alternatives to patient management exist. There are, however, a number of problems with such statistical models. First, they all accept a degree of error based solely on statistical chance.²⁷ This raises serious concern, particularly when the error directly affects patients' chances of survival. Second, statistical models can only be assumed to be applicable to the population from which they were originally drawn.²⁷ No two populations are ever identical as is clear from the original articles on the Glasgow Aneurysm Score and the Hardman Index.^16,17 Indeed, the two populations differed greatly even in respect of basic demographic proportions. Only a process of thorough validation may permit models to be used outside the remit of the population from which they were originally drawn. As such, these two models appear to have no role in the authors' catchment population. Finally, it must be emphasised that statistical models predict risk not survival. As such, some high-risk patients will survive and conversely some low-risk patients will die; this is the nature of risk. When using statistical models, clinicians need to appreciate that results may support, but can never replace, clinical judgement. In agreement with current literature,^28,29 our results suggest that the Glasgow Aneurysm Score and Hardman Index are poor predictors of outcome after rAAA surgery and more accurate models need to be developed.