Survival after surgery or therapeutic catheterisation for congenital heart disease in children in the United Kingdom: analysis of the central cardiac audit database for 2000-1

Abstract

Objectives To analyse simple national statistics and survival data collected in the central cardiac audit database after treatment for congenital heart disease and to provide long term comparative statistics for each contributing centre.

Design Prospective, longitudinal, observational, national cohort survival study.

Setting UK central cardiac audit database.

Main outcome measures Survival at 30 days and one year after treatment in the year April 2000-March 2001, assessed by using both volunteered life status and independently validated life status through the Office for National Statistics, using the patient's unique NHS number, or the general register offices of Scotland and Northern Ireland. Institutional results following a group of six benchmark operations and three benchmark catheterisation procedures.

Results Since April 2000 data have been received from all 13 UK tertiary centres performing cardiac surgery or therapeutic cardiac catheterisation in children with congenital heart disease. Altogether 3666 surgical procedures and 1828 therapeutic catheterisations were performed. Central tracking of mortality identified 469 deaths, 194 occurring within 30 days and 275 later. Forty two of the 194 deaths within 30 days were detected by central tracking but not by volunteered data. For surgery overall, survival at 30 days was 94.9%, falling to 91.2% at one year; this effect was most marked for infants. For therapeutic catheterisation survival at 30 days was 99.1%, falling to 98.1% at one year. Survival of individual centres or individual operators did not differ from the national average after benchmark procedures.

Conclusions Independent data validation is essential for accurate survival analysis. One year survival gives a more realistic view of outcome than traditional perioperative mortality. Currently no detectable difference exists in survival between any of the 13 UK tertiary congenital heart disease centres, but confidence intervals for small centres are wide, limiting our power to detect underperformance from analysis of a single year's data. Appropriately resourced, focused national audit is capable of accurate data collection on which nationwide, long term quality control can be based.

Introduction

Monitoring of survival rates after cardiac surgery was introduced in the United Kingdom in 1977 with voluntary submission of data to the Society of Cardiothoracic Surgeons of Great Britain and Ireland. The central cardiac audit database was established by the British Cardiac Society, the Society of Cardiothoracic Surgeons, and the British Paediatric Cardiac Association to provide national analysis of outcomes of cardiac surgery and therapeutic cardiac catheterisation. It differs in three major aspects from previous national audit projects: data are collected electronically in a secure format; mortality and reintervention are tracked centrally by using a unique patient identifier (the NHS number); and independent data validation is used. In 2000 the Department of Health funded the central cardiac audit database to collate data from all centres for congenital heart disease in the United Kingdom. This report contains the first year's data (1 April 2000 to 31 March 2001), with centrally tracked one year survival. The results are presented on behalf of the Society of Cardiothoracic Surgeons, the British Paediatric Cardiac Association, and all contributing centres, each of which gave consent to publication of identifiable, centre specific data.

Methods

Data collection

We designed a minimum dataset of 20 fields with the simple aims of the project in mind. All 13 congenital heart disease centres in England, Scotland, and Northern Ireland participated. To ensure patient confidentiality the central cardiac audit database employs advanced data encryption technology to control access to data through a secure key system. We used lists with fixed choices consisting of all but the rarest and most complex combinations of diagnoses and procedures to minimise the potential complexities of diagnostic and procedural coding for congenital heart disease.

Data validation

The minimum dataset included date of death, but we linked with the Office for National Statistics by using NHS numbers to assess mortality wherever possible. We compared volunteered mortality data with centrally tracked data. In Northern Ireland and Scotland we used the general register offices to track mortality centrally.

The central cardiac audit database includes other forms of independent data validation carried out by visiting centres, when two weeks' submitted data, chosen at random, are compared with hospitals' written medical records, with operating theatres' records, and with laboratory records on cardiac catheterisation. We checked entries in the logbooks for operating theatres and catheter laboratories for the entire year in each centre, to ensure complete ascertainment of caseload. We also compared submitted data with nationally held hospital episode statistics whenever these were accessible.

Data analysis

We used the online Lotus Domino version of the central cardiac audit database to collect data and transferred these for analysis to SPSS 10.0 for Microsoft Windows. We used Wilson's score method to calculate confidence intervals for survival.¹,² We used 99% confidence intervals (table) to make allowances for the high number of multiple comparisons, to minimise false positive results. We did not consider an individual survival value to be significantly different from the mean if the confidence intervals overlapped the mean.

Table 1.

Survival for neonates, infants, all children under 1 year, and children between 1 year and 16 years undergoing surgery or therapeutic catheterisation

Age	No of procedures	% survival at 30 days (99% CI)	% survival at 1 year (99% CI)
All surgery (bypass and non-bypass)
All ages	3666	94.9 (94.5 to 95.1)	91.2 (90.6 to 91.5)
Neonate and infant:	2073	93.1 (92.4 to 93.4)	87.7 (85.2 to 89.8)
Neonate	780	90.9 (89.4 to 91.5)	86.1 (84.1 to 87.2)
Infant	1293	94.5 (93.6 to 94.7)	88.7 (87.4 to 89.4)
Child	1561	97.7 (97.0 to 97.7)	96.1 (95.4 to 96.2)
Bypass surgery
All ages	2664	94.7 (94.2 to 94.9)	91.8 (91.1 to 92.1)
Neonate and infant:	1292	92.1 (91.1 to 92.5)	87.8 (84.6 to 90.3)
Neonate	383	87.1 (84.3 to 88.4)	82.8 (79.4 to 84.8)
Infant	909	94.4 (93.3 to 94.7)	90.0 (88.4 to 90.6)
Child	1353	97.5 (96.8 to 97.5)	96.0 (95.1 to 96.1)
Non-bypass surgery
All ages	1002	95.5 (94.6 to 95.7)	89.7 (88.3 to 90.4)
Neonate and infant:	781	94.6 (93.4 to 94.9)	87.6 (83.4 to 90.8)
Neonate	397	94.7 (92.5 to 95.0)	89.3 (86.5 to 90.3)
Infant	384	94.6 (92.5 to 94.9)	85.9 (82.7 to 87.3)
Child	208	98.8 (95.0 to 98.8)	97.4 (93.2 to 97.5)
Catheter intervention
All ages	1828	99.1 (98.7 to 99.1)	98.1 (97.6 to 98.1)
Neonate and infant:	472	98.3 (96.7 to 98.3)	96.1 (92.5 to 97.9)
Neonate	178	98.8 (94.9 to 98.8)	97.3 (93.0 to 97.4)
Infant	294	98.0 (95.4 to 98.0)	95.3 (92.4 to 95.6)
Child	1320	99.4 (98.8 to 99.4)	98.8 (98.1 to 98.7)

Survival is calculated based only on cases where follow up reaches 30 days or one year and no further intervention has taken place. The Wilson score method was used to calculate 99% confidence intervals for survival.

We used analysis of survival after “benchmark” operations to compare results from different centres, to eliminate the effect of different case mix. We chose six benchmark procedures for surgery (repairs of atrial septal defect, ventricular septal defect, atrioventricular septal defect, tetralogy of Fallot, simple transposition of the great arteries, and coarctation) and three for therapeutic catheterisation (atrial septal defect closure, arterial duct closure, and pulmonary balloon valvoplasty). We did not undertake detailed risk stratification as no validated method exists.

We calculated 30 day postoperative survival to facilitate comparison with results from previous UK registry data and to comply with practice in the United States.³ Central tracking of mortality has, however, also allowed us to plot one year survival curves, in contrast to previous registry analyses. We included foreign nationals but censored them from survival analysis after the perioperative period unless specific follow up data were available (central tracking was not possible for this group). We analysed individual operators' results anonymously, but each centre agreed to be identified. We have not calculated freedom from reintervention statistics for this report as follow up is currently too short to allow meaningful interpretation of results.

Results

Data collection and quality

Overall completeness of the dataset was 96.8%, with completeness for individual data fields ranging from 75% (for NHS number) to 100%. Data were received for a total of 5494 procedures, of which 3666 were surgical and 1828 were therapeutic catheterisations.

We have reported all cause mortality, choosing not to attempt detailed investigation of cause of death and its relation to treatment. We found substantial differences in volunteered and centrally tracked mortality, with seven of 11 centres in England under-reporting death within 30 days. Central tracking of mortality identified 469 deaths, 194 occurring within 30 days and 275 later. Forty two of the 194 deaths within 30 days were detected by central tracking but were not in the volunteered data. Nineteen of these patients were discharged alive but subsequently died within 30 days of the operation. The remainder had been incorrectly coded as alive at discharge; using reported discharge status would have underestimated 30 day mortality by 22%. Data on hospital episode statistics were available for 2716 patients and under-reported death within 30 days by 9%, classifying 1% of surviving patients as deceased. Hospital episode statistics data also under-reported the total number of procedures by 10%. During validation visits we found a total of 143 procedures to be missing from the data submissions to the central cardiac audit database, predominantly related to systematic errors in data collection. The visits resulted in submission of missing or revised data from all of the 13 centres.

Survival

Figures 1, 2, 3 show national survival curves after cardiopulmonary bypass surgery, non-cardiopulmonary bypass surgery or therapeutic catheterisation. The table shows survival at 30 days and one year after all procedures. We assessed benchmark procedure survival anonymously for individual operators (41 surgeons and 63 cardiologists) as well as for different centres. No significant difference from the national mean survival was detectable for any individual. Figures 4 and 5 and figures A-I (see bmj.com), respectively, show individual centre's survival data for pooled and individual benchmark procedures.

Fig 1.

One year survival after cardiopulmonary bypass surgery for age groups <1 month, 1 month-1 year, and 1-16 years for the United Kingdom

Fig 2.

One year survival for non cardiopulmonary bypass surgery for age groups <1 month, 1 month-1 year, and 1-16 years for the United Kingdom

Fig 3.

One year survival for therapeutic catheterisation for age groups <1 month, 1 month-1 year, and 1-16 years for the United Kingdom

Fig 4.

Survival at 30 days and at one year reported by individual centre, with 99% confidence intervals for all benchmark surgical procedures. The shaded areas represent the national means with 99% confidence intervals. If a centre's confidence intervals overlap the shaded area their survival does not differ statistically from the national mean. For a list of the abbreviations see bmj.com

Fig 5.

Survival at 30 days and at one year reported by individual centre, with 99% confidence intervals for all benchmark catheter procedures. The shaded areas represent the national means with 99% confidence intervals. If a centre's confidence intervals overlap the shaded area their survival does not differ statistically from the national mean. For a list of the abbreviations see bmj.com

Discussion

Data quality

We found a striking difference in deaths identified by centres' own records and by central tracking. This was most marked when death occurred after the perioperative period, but deaths were missing from submitted data even when death occurred within 30 days. It seems inevitable that previous registry reports (including the register of the Society of Cardiothoracic Surgeons, used in the Bristol inquiry) that have relied on voluntary reporting of death have also under-reported mortality, casting doubt on their validity. The introduction of the NHS number as a unique and permanent patient identifier and the ability to establish electronic linkage with the Office for National Statistics is a major advance in tracking patients' outcomes.

Risk stratification

The use of benchmark procedures minimises the effect of varying case mix for the purposes of comparison of outcomes in different centres. Attempts have been made to establish a consensus view of risk assessment in the United States⁴ and case complexity in the United Kingdom.⁵ These protocols, applied to the data in the central cardiac audit database that have been accumulated over several years, should facilitate development and validation of risk stratification for the future.

Patient confidentiality

We did not include patients' consent for data submission to the central cardiac audit database in our protocol. Our current understanding of the Data Protection Act 1998 is that patients' consent is not required if anonymised data are used for the purpose of research or audit. The exception to anonymising data is the NHS number, which we have protected by encryption with a key held only by the data managers and used only for record linkage. In the United Kingdom we have an almost unique opportunity to carry out effective and believable national audit because we have a single healthcare system with an Office for National Statistics where the life status of an individual patient, based on their NHS number, is known.

With appropriate precautions central tracking is possible while maintaining patient confidentiality.

Diagnostic and procedure coding

Several groups have devised coding systems for congenital heart disease.³,⁶,⁷ The central cardiac audit database plans to adopt the coding system of the Association for European Paediatric Cardiology⁷ from 2004, to facilitate future international compatibility of data.

Outcomes after treatment

Centre specific data analysis shows that quality of treatment is high throughout the United Kingdom; no centre and no individual operator has detectably different survival from the national mean after benchmark procedures. We have been unable to assess accurately how our results compare with those of other nations, although our data seem to compare favourably with unvalidated registry reports from North America and Europe.

We believe that this study is the first to present validated, centre specific survival data for nationwide treatment of congenital heart disease. Population based, 45 year, actuarial survival has been reported for the whole of Finland,⁸ but individual centres' performance was not included. Most previous reports have concentrated on multicentre, mean perioperative mortalities (defined as death within 30 days of operation),⁹ and a similar approach was used for the register of the Society of Cardiothoracic Surgeons.¹⁰ Although this simplistic approach may be convenient, our data show how misleading 30 day results may be as a descriptor of overall outcome.

We calculated 99% confidence intervals for the purpose of assessing survival in different centres. Even at this level, having performed a total of 178 comparisons, we think that it is likely that we will generate spuriously significant results: we calculate that the chance of at least one spuriously significant difference in survival is 83% (this would have been 99.99% had we used 95% confidence intervals).

What is already known on this topic

The validity of previous voluntary registers of survival after surgery has long been held to be potentially inaccurate

The Bristol inquiry report highlighted the inadequacy of current national audit, particularly for the treatment of congenital heart disease

What this study adds

Volunteered survival data are of little value, sometimes overestimating survival by as much as 20%

Data validation is essential for national or local audit of survival and has been made far easier by the introduction of the NHS number and the ability to use it to create electronic links to the Office for National Statistics

Traditionally reported perioperative (30 day) mortality can give a misleadingly optimistic view of prognosis to both professionals and the public; for infants mortality after treatment for heart disease at one year was double that at 30 days

The central cardiac audit database places validated, centre specific survival results for treatment of children with congenital heart disease in the public domain

This early data analysis has concentrated on survival, which is a crude indicator of overall performance. For smaller centres, as well as for individual operators, analysis of a single year's data has limited power to detect underperformance by institutions or individual operators and year on year analysis will be necessary to provide more robust reassurance. Freedom from reintervention is likely to be a powerful indicator of overall performance, but several years' data will be required before our capability to track reintervention can be put to use.

Conclusions

Independent validation of data is essential for accurate survival analysis. One year survival statistics give a more realistic view of outcome than traditional perioperative mortality. At present survival is no different between any of the 13 UK tertiary centres for congenital heart disease, but confidence intervals are wide, limiting our power to detect underperformance from analysis of a single year's data. Appropriately resourced, focused, national audit is capable of accurate data collection on which nationwide, long term, quality control can be based.