Surgery for congenital heart conditions in Oxford

The NHS in the United Kingdom fears another “Bristol,” where preventable or excess adverse outcomes may go undetected and unresolved for a prolonged period. In Bristol, reports of excessive surgical mortality among children having heart surgery in the early 1990s led to an independent public inquiry, which concluded that the mortality in children younger than 1 year who were having open cardiac surgery was unacceptably high.¹ In October 2004 the BMJ published an article saying that similar problems may be emerging in relation to surgery for congenital heart disease in Oxford.² Is this really the case?

The Bristol Royal Infirmary inquiry had explored several sources of data that may have detected this inferior performance, and all were found deficient in one way or another. The UK cardiac surgical register run by the Society of Cardiothoracic Surgeons was not only anonymised, but it was based on diagnosis rather than operation and so could not detect differences in outcome where more than one operation existed to treat a single diagnosis. Hospital episode statistics, compiled from routine administrative data, were also not designed for assessment of surgical outcomes for complex congenital conditions and so have codes for some common cardiac operations. Nevertheless, using both sources of data, Aylin et al developed an innovative method for the Bristol inquiry that clearly showed statistically significant excess mortality in Bristol for children younger than 1 year.³ More recently Aylin et al applied the same method to more recent data from hospital episode statistics to see how practice had changed since the inquiry.² They reported that results had shown a steady improvement over the past decade, with overall mortality in hospital for surgery on infants falling from 12% to 4%. However, they identified Oxford as having a significantly higher mortality of 11% throughout this time frame. This was at odds with an earlier publication from the central cardiac audit database, which, after the Bristol inquiry, had superseded the now defunct congenital UK cardiac surgical register.⁴ This database, which contains data on cardiological and surgical interventions on children, is administered by the NHS Information Authority under the jurisdiction of the Healthcare Commission. It is driven by clinicians, validated by peer visits, and linked to the Office for National Statistics to track mortality indefinitely.

These conflicting reports published in the BMJ prompted the Department of Health to instruct Thames Valley Strategic Health Authority to undertake a critical analysis of both studies and their most up to date data. Its report was released on 2 February 2005 and showed that in a comparison of contemporary data from 11 paediatric units in England between 2000-2, hospital episode statistics recorded 20% fewer cases than the central cardiac audit database (5-38% less per unit).⁵ Hospital episode statistics (HES) also recorded fewer deaths because only in-hospital mortality was available, whereas the tracking facility in the central cardiac audit database (CCAD) allowed 30 day mortality in or out of hospital to be recorded, which both standardised and elevated death rates (HES 4%, CCAD 8%).

Using even more recent three year data from central cardiac audit database (2000-3), the report shows that Oxford's 30 day mortality for children younger than 1 year is in the middle of the national range. Interestingly, in three of the 11 hospitals, analysed data from hospital episode statistics and the central cardiac audit database correlated quite well. These included Oxford, and its good data from hospital episode statistics may have paradoxically contributed to its outlier status in the hospital episode statistics analysis. The report concludes that the central cardiac audit database has come of age and now represents a more reliable and useful dataset for analysing trends and performance in this complex arena.

Coming of age brings responsibilities. For the guardians of clinical databases it brings a responsibility for transparency, reflected in an open, agreed, and understood strategy to access data and for publication. The Freedom of Information Act will certainly act as an accelerant in this regard. Properly conducted third party analysis of such data will strengthen the value of these databases, but the risk is that misuse of the data will discourage voluntary submission of data, destroy a valuable asset, and discourage others from starting on the journey.

This raises the issue of how analyses that identify potential outliers should be handled. Is a peer reviewed journal the right place for publication? What are the relative responsibilities of authors, reviewers, and editors—particularly when a review process may take several months and delay remedial action?

In his response to the Bristol Royal Infirmary inquiry, the secretary of state for health told parliament that, “For data on surgical outcomes to be published, of course, it needs to be robust, rigorous, and risk-adjusted. That will inevitably take time.” The paediatric cardiac surgeons now have data that are robust and rigorous. Unravelling the complexities of events in Bristol was bedevilled by issues of coding and muddled accountability for the surveillance of performance. Paediatric cardiac surgical coding is now agreed, and surveillance clearly falls to the independent Healthcare Commission through the central cardiac audit database, hospital episode statistics, and other corroborating sources including peer review, because absence of evidence of a difference does not always mean evidence of no difference.

In time, clinical and administrative datasets should function as one, and this is already on the agenda in England. With the advent of performance monitoring and payment by results, all clinicians must be prepared to take an active role in institutional data collection.