Public confidence and cardiac surgical outcome

The General Medical Council has recently been grappling with the problem of measuring and comparing surgical outcomes after complex surgery in a heterogeneous patient population with differing severities of illness.¹ Cardiothoracic surgery, with its immediate, and sometimes catastrophic outcomes, is the first surgical specialty to come under such scrutiny. Inevitably the media coverage has dented public confidence in the ability of the medical profession to police itself, and in particular this has been focused on cardiothoracic surgery.¹ Yet, the irony is that in the United Kingdom cardiothoracic surgery has better data and is more subject to internal scrutiny than perhaps any other specialty.

The Society of Cardiothoracic Surgeons has a long history of audit. In 1977 Sir Terence English established the United Kingdom cardiac surgical register,² which collects activity and mortality data on all cardiac surgical procedures performed in each NHS cardiac surgical unit, amounting to 35 000 procedures a year. Although apparently simple in concept, the process represented the first attempt in Britain by any specialty to collect national activity and outcome data.

All data are anonymised, since this was a prerequisite for encouraging voluntary data submission from all units. Similarly the United Kingdom heart valve registry has collected national valve surgery data since 1986. Linkage of this registry to the Office for National Statistics means we now have unique 10 year survival data following heart valve replacements in the NHS.³,¹⁸

Both registries return aggregated data to each member of the society as an annual report containing national activity and mortality data for a wide range of cardiac operations. Since inception the presumption has been that access to national information would draw each surgeon’s attention to his or her own performance and encourage local introspection and action. So what has gone wrong? Why have we apparently failed to identify those few surgeons whose performance has fallen below acceptable standards?

Firstly, the data in the cardiac surgical register relate to individual units, not individual surgeons. Hence, a unit’s figures can easily camouflage an errant performer. Secondly, poor individual performance could be dismissed as a casemix problem, since risk stratification algorithms were not available. Thirdly, reliable data collection facilities have not been available in every unit, and failure to track every death may have resulted in the reporting of unrealistically low operative mortalities for some procedures.

Nevertheless, the register represented a spearhead endeavour both internationally and within Britain and provided a reasonable indication of national activity and mortality. Even so, the Society of Cardiothoracic Surgeons recognised the shortcomings of the system, particularly in the light of transatlantic developments. The Freedom of Information Act in the United States had forced individual cardiac surgeons’ outcomes into the public domain,⁴,⁵ and the release of raw mortality data by public health agencies had caused considerable alarm within the specialty. This stimulated interest in understanding outcome measures and developing risk stratification algorithms⁶,⁷ and prompted a reappraisal of our own national system.

Acknowledging the need to be able to measure casemix and severity of illness, the society established a national database in 1994, to run in parallel with the existing, simpler, register. This database collects some 150 datapoints on all adults undergoing cardiac surgery in selected units across Britain, with the aim of developing reliable comparative UK oriented risk stratification models in conjunction with the MRC Biostatistics Unit in Cambridge. This year the national database accepted data from just over half of all British units. It now provides a unique repository of comprehensive data on 30 000 patients for risk stratification modelling and which is available to contributing units. At present the database does not collect surgeon identifiers, since in 1993-4 this would have been an insurmountable stumbling block to its launch.

However, the tide of public and professional opinion is changing rapidly and the society has this year added a paediatric surgical database to its endeavours. The chief medical officer has made it clear that the public have a right to know that standards are under scrutiny and that the profession cannot hide behind anonymity. The society supports this stance but believes that measurement and interpretation should be governed by the specialty, and most members consider that comparisons between surgeons should be risk stratified to take account of casemix. At the same time the society recognises a conflict. A mechanism for professional assurance needs to be put in place promptly to reassure the public, but not all units have the information technology or staff to collect the detailed information required for risk stratification. To force these units to collect complex data in the absence of adequate facilities would either dilute the reliability of the data or risk reducing the dataset to the lowest common denominator and thereby reduce its value.

To balance these apparently conflicting aspirations the society has asked all NHS units to return annual, raw, surgeon specific mortality data on marker operations for adult cardiac surgery, thoracic surgery, and paediatric cardiac surgery from 1 April 1997 as an extension to the cardiac surgical register. These data will be analysed independently and the results scrutinised through an internal mechanism within the society. Individual surgeons will be notified and required to respond if their performance appears to be outside predetermined limits. This will provide an effective, specialty driven, early warning system at little or no additional cost to individual units.

The comprehensive data collection required for risk stratification may be intimidating to some. However, surgeons from a unit with risk stratification in place will find themselves in a stronger position to respond to the society’s new early warning system. Herein lies an inequity even for those committed to good data collection. The standard NHS patient management systems are generally not sophisticated enough to process these types of data. Most cardiac surgical units have already demonstrated commitment by either developing a bespoke system or purchasing a proprietary system capable of performing benchmarking against national standards by both simple⁸,⁹ and complex risk modelling¹⁰ with logistic regression,¹¹ Bayesian analyses,¹²,¹³ and individual risk adjusted CUSUM.¹⁴,¹⁵ The changing climate is encouraging the remaining units to do the same and submit comprehensive data to the national database. The limiting feature is that good data collection requires local resources in the form of appropriate software and staff together with commitment from consultants.

In parallel with the society’s initiative the department of health is exploring the feasibility of centralised online data collection and warehousing for all interventional cardiology and cardiac surgical procedures through the central cardiac audit database project. This is entering its third year, in six pilot centres, and will be reporting soon.¹⁶

Most cardiac surgeons have long recognised their responsibility to collect reliable and comprehensive data on their performance. This will be facilitated by the development of an international cardiac surgical dataset currently being drawn up between the Society of Thoracic Surgeons in the United States and the European Association for Cardiothoracic Surgery. This will help to standardise risk factor data collection and facilitate the development of robust comparative risk modelling between populations, procedures, institutes, and individual surgical teams.

Evidence based medicine indicates that those patients most likely to benefit from cardiac surgery are usually the sickest, with the most damaged hearts, who therefore have the greatest surgical risk. So auditing performance without correction for casemix will subject the surgeon to unfair comparisons and ensure that some patients who might otherwise benefit will be denied surgery.¹⁷ Good risk stratification, however, will reduce the chances of high risk patients being turned down for surgery and encourage fully informed preoperative consent. Furthermore, although operative mortality is always attributed to the surgeon, this ignores the subtle but important influences of cardiological management and referral, anaesthetic care, and intensive care resources.

Although the climate is changing, these remain complex and sensitive issues, but these new mechanisms should go some way to restoring public confidence. Our outcome statistics have been in the public domain for many years and are now published on our web page¹⁸ along with American outcome data.¹⁹ We will soon be adding a coronary surgery risk calculator based on UK data which will introduce the concept of operative risk calculation into the public domain. This year the Society of Cardiothoracic Surgeons has gone a stage further and democratically assumed responsibility for quality control of individual surgical practices—a new role for any specialist society within the United Kingdom. However, our specialty represents the tip of the iceberg in medical quality assurance, and the major challenge will be determining realistic, measurable, and auditable outcomes for other medical and surgical specialties, where poor outcomes also occur but the process is less transparent.