Sampling time error in EuroSCORE II

Abstract

Seasonal variation in mortality after cardiac surgery exists. EuroSCORE II accrued data over a 12-week period from May to July 2010. We investigated whether the accrual period for EuroSCORE II had a different mortality rate compared with the rest of the year. We found in a study population of 18 706 that the accrual period of EuroSCORE II may introduce bias into the predicted mortality, potentially reducing the accuracy of the new model.

INTRODUCTION

Numerous publications have documented the inaccuracy of EuroSCORE as a predictive model for estimating risk of death post-cardiac surgery, hence the development of EuroSCORE II [1–3]. EuroSCORE II accrued data over a 12-week period from May to July 2010.

Seasonal variation in mortality after cardiac surgery has previously been described [4]. This work divided the year into four seasons: winter, autumn, spring and summer. In the northern hemisphere, the dates of accrual for EuroSCORE II overlap spring and summer.

We investigated whether the mortality during the period of EuroSCORE II data collection was different to that at other times of the year.

METHODS

Patients

All patients from a single institution who had undergone cardiac surgery were included, n = 18 706, between 1 April 1997 and 31 March 2010. Mortality was defined as in-hospital.

Benchmarking

We benchmarked our in-hospital mortality figures against the UK national results.

Analysis

The observed, expected and the ratio of observed to expected mortality was plotted by the month of year. Logistic regression was performed with EuroSCORE and month of year as independent variables. The month of year was dichotomized into 0 for May to July inclusive (EuroSCORE II sampling period) and 1 for the rest of the year.

RESULTS

Benchmarking failed to reveal any significant difference in our in-hospital mortality rates compared with the rest of the UK. The overall institutional mortality rate for all cardiac surgery was 2.6%. In total, 71.5% had undergone isolated coronary artery bypass grafts (CABG), 11.9% underwent aortic valve replacement (AVR), 5.3% underwent mitral valve replacement (MVR), 8.8% underwent AVR and CABG, and 2.6% underwent MVR and CABG. The number of cases broken down by the month period was 1567, 1506, 1629, 1490, 1498, 1683, 1626, 1506, 1535, 1662, 1645 and 1359 from January to December, respectively. This equated to a study sample of 4807 during the same monthly time period of EuroSCORE II recruitment.

Logistic regression demonstrated that the month of year being May to July was a significant independent variable affecting in-hospital mortality, P = 0.004 (odds ratio 0.72, 95% confidence interval 0.57 to 0.90).

The effect of time period of the year on observed verses expected mortality is shown in Fig. 1. The observed mortality in May to July was 2.0%, which was 2.8% for the rest of the year. This was a significantly lower observed mortality in May to July verses the rest of the year, P = 0.005. The expected mortality however was not significantly different, P = 0.9.

Figure 1:

Observed, expected and the ratio of observed to expected in-hospital mortality.

Analysis by weekday of surgery failed to reveal any association after adjusting for risk profile.

DISCUSSION

The variation in observed to expected mortality in the period from May to July inclusive is greater than for any other time period of the year. The observed risk of death in the May to July period was 38% less than that was observed for the rest of the year. It should be noted that the estimated risk profile of our patients showed no significant difference over 12 months. This finding was confirmed by both univariate and multivariate analysis.

The finding that the risk of death would seem to be lower in the period May to July 2011 implies that EuroSCORE II may underestimate the risk of death for patients operated outside this seasonal period. Of note the original EuroSCORE system recruited patients in autumn (September to November 1995) [5].

This is the second large study to document the time of year effect on mortality post-cardiac surgery, the previous one however only analysed by season, and not by month. As EuroSCORE II straddles two seasons, summer and spring, direct application of this previous work to assess the potential effect of the accrual date period of EuroSCORE II is impossible.

The demonstration that the observed mortality varies significantly by the time period of the year, whereas the expected mortality does not, inferring that there is a direct causal relationship. Direct causality is imposable to confirm or refute in a retrospective analysis such as this. As two independent studies have both identified the time of year of surgery as a significant factor, it may need to be incorporated into any future risk scoring system.

As EuroSCORE II recruited data in a time period with a reduced observed versus expected mortality rate, its accuracy and applicability for annual institutional and individual risk-adjusted comparisons and clinical practice need to be investigated.

Limitations

Our institutional mortality has fallen over the period of this study; however, it is unlikely that the improvements will disparately affect the months May to July.

The use of the term in-hospital mortality has limitations as patients may be transferred to another unit and subsequently die, and death in the immediate period after discharge is not accounted for. This is a limitation of all current risk models for mortality prediction post-cardiac surgery.

Conflict of interest: none declared.