Table 1.

Summary of studies including evidence of the effect of consultant outcome publication on surgeon behaviour.

Study	Location	Specialty	Survey (response)	Database	Included (n)	Outcomes/comments
Risk-averse behaviour
Evidence for
Schneider 19966	Pensylvannia	CTS/IC	Survey(64%/74%)		612/85	59% of cardiologists found that referring a high-risk patient for surgery had become more difficult since introduction of COP; 63% of cardiac surgeons were less willing to perform surgery on high-risk patients since introduction of COP.
Hannan 199724	New York	IC	Survey (36%)		450	14% of free text comments related to high-risk patients being turned down for surgery; 28% stated that report cards should be discontinued or not publically reported, with reasons given including that it deters surgeons from taking on high-risk patients
Burack 19997	New York	CTS	Survey (69%)		104	67% of respondents had refused surgery to at least one high-risk patient in the previous 12 months. 18% had refused surgery to 5 or more. Surgeons who had refused a high-risk patient due to concerns on the effect on their published outcomes were significantly more likely to have been practising as a consultant for less than 10 years, to have performed less than 100 cardiac procedures/year or to have less than 50% adult cardiothoracic practice.
Dranove 20028	New York and Pensylvannia	CTS		Medicare/AHD		Following COP, the health expenditure per patient in the year prior to intervention decreased suggesting that CABG was being performed on healthier patients.
Narins 200515	New York	IC	Survey		186	79% of cardiologists agreed or strongly agreed that COP had affected their decision to offer intervention to high-risk patients. Suggested not to include the highest-risk patients in the outcome data to help avoid risk averse behaviour.
Moscucci 20059	New York	IC		Regional PCI data	38,066	Patients from Michigan (non-reporting) were significantly more likely to undergo PCI for acute myocardial infarction and cardiogenic shock. Subsequently, patients from a public reporting state had a lower unadjusted in hospital mortality but this difference was not seen once risk adjustment for comorbidities was applied.
Apolito 200812	New York	CTS/IC		SHOCK registry	545	NY (public reporting) patients with AMI complicated with cardiac shock less likely to receive angiography, PCI and CABG compared with a non-reporting state. Lower prevalence of revascularisation in these patients was associated with a higher mortality (1.5-fold higher). NY patients had to wait 10 times longer to receive CABG (101.2 vs 10.3 hours).
Romano 201125	California	CTS		CCMRP/ CCORP	131,986	Following COP, high-mortality outlier hospitals were found to operate on patients with lower risk-adjusted mortality suggesting lower-risk patients were being selected.
Joynt 201210	New York, Pensylvannia, Massachussets	IC		Medicare		Patients with myocardial infarction were less likely to undergo PCI in states with public reporting with the greatest difference in intervention being seen in high risk patients (ST elevation myocardial infarction, cardiogenic shock or cardiac arrest). Following 5 years of COP, Massachusetts patients with cardiogenic shock had significant reduction in odds ratio for undergoing PCI compared with non-COP states.
Sherman 201326	Northwestern	Multi- specialty	Survey/ interview		122/20	78% were concerned that outcome publication of hospital data would lead to high-risk patients being turned down for surgery. 68% were concerned that outcome publication at hospital level may lead to the shifting of high-risk patients to safety-net hospitals.
Waldo 201513	New York/ Massachussets	CTS/IC		US NIS	84,121	Significantly fewer revascularisation procedures were performed in states with public reporting. Patients who underwent revascularisation procedures in public reporting states had a lower mortality compared with those in non-reporting states but were associated with a higher adjusted in-hospital mortality rate, with a higher proportion of mortality being associated with patients who did not undergo intervention.
Boyden 201511	New York and Michigan	IC		NCDR CathPCI registry	105,511	New York (public reporting) patients found to have a lower proportion of patients with high risk factors such as ST elevation myocardial infarction, non-ST elevation myocardial infarction, cardiac arrest and cardiogenic shock when compared to Michigan (non-public reporting). Significantly fewer New York patients had an extremely high risk (20%) of predicted risk of mortality, suggesting a degree of risk aversion.
Evidence against
Hannan 19944	New York	CTS		CSRS	57,187	COP associated with improved patient outcomes. Number of high-risk patients increased over first 4 years.
Hannan 199520	New York	CTS		CSRS	15,637	Actual mortality rate decreased during study period despite the fact that the average severity of illness of patients undergoing surgery increased (expected mortality 2.62–3.5%). 7.6% of patients with the highest risk were operated on by low-volume surgeons, which then dropped to 4.9% in 1991.
Peterson 199814	New York	CTS		Medicare	42,047	NY patients undergoing CABG were slightly older, more likely to have DM and to have had a preprocedural myocardial infarction when compared with non-PR states. From 1987 to 1992 there was an increase in percentage of patients 65–70 years receiving surgery from 3.4% to 8.4% which paralleled national trends.
Hannan 200323	US	CTS		Medicare	911,407	No evidence of risk averse behaviour or outmigration of patients. Some states with public reporting it was found to offer intervention for higher-risk patients.
Bridgewater 20075	UK	CTS		NWQI	25,730	Progressive increase in predicted mortality rate from 3.0 in 1997–98 to 3.5 in 2004–05 (P < 0.001). Median age of patients increased from 63 to 65 years (P < 0.001). Significant increases in the proportion of patients aged 80+ years, with renal dysfunction, recent myocardial infarction and peripheral vascular disease. Significantly more high-risk and very-high-risk patients underwent CABG after COP compared with before.
Li 201027	California	CTS		CCORP	36,923	Most hospitals had a higher or unchanged predicted risk of mortality suggesting a higher proportion of high-risk patients were being operated on. Proportion of high-risk patients undergoing CABG did not decline post COP.
Shahian 201519	US	CTS		STS	8,929	No change was seen across the preoperative risk factors) before and after the implementation of public reporting.
Gaming
Evidence for
Omoigui 199616	Cleveland	CTS		CCIR	42,027	Patients referred to Cleveland from New York were significantly more likely to have had previous open-heart surgery and to be of NYHA heart failure III or IV. There was a trend towards the transfer of a higher proportion of older patients from Cleveland to New York. Subsequently, there was a significantly higher postoperative morbidity and mortality in patients from New York. These findings were not seen between 1980 and 1988, the period prior to the introduction of COP.
Burack 19997	New York	CTS	Survey (69%)		104	30% of respondents admitted to a significant change in their practice, including altered profiling of patients (gaming), moving to thoracic non-cardiac surgery, relocation to a non-COP state or retirement from surgery. 40% of respondents stated that risk-factor gaming was the area requiring most improvement on the database.
Narins 200515	New York	IC	Survey (65%)		186	88% agreed or strongly agreed that physicians may report higher risk conditions for their patients features to improve risk-adjusted mortality outcome statistics.
Guru 200917	Ontario	CTS	Survey (95%)		52	84% of surgeons thought that upcoding of risk factors took place in Ontario following the introduction of COP.
Evidence against
Peterson 199814	New York	CTS		Medicare	42,047	No sign of outmigration over the time of the study with the percentage of 11.3% in 1992 being significantly less than the respective proportion from 1987 (P < 0.001). NY patients being referred out of state for CABG had similar demographics to those who remained in state. There was a significant decline for out of state referral from NY between 1987 and 1992. Elderly patients were less likely to be referred out of state for their procedure.
Hannan 200323	US	CTS		Medicare	911,407	No sign of outmigration.
Guru 200618	Ontario	CTS		CCN	67,693	No sharp increase in comorbidity following public reporting, suggesting no upcoding took.
Shahian 201519	US	CTS		STS	8,929	No change was seen across the preoperative risk factors before and after the implementation of public reporting. Across the 9 years, the expected mortality rate remained stable or increased slightly across both reporting and non-reporting regions.
Cessation of practice by low volume/poorly performing surgeons
Evidence for
Hannan 199520	New York	CTS		CSRS	15,637	Following COP, total number of surgeons performing less than 50 procedures per year fell from 39 to 33 (31% to 24%). The number of surgeons in the high-volume group increased from 30 to 41 (24–30%). Percentage of cases performed by low-volume surgeons decreased from 7.6% to 5.7% over the study period. Low-volume surgeons had a higher risk-adjusted mortality for each year studied.
Jha 200621	New York	CTS		CSRS		Poorer performing surgeons were more likely to leave CABG surgery in New York within 2 years of the publication of the report cards. More than 20% of those in the lowest quartiles stopped practising within this time frame. In contrast, only approx. 5% of the other quartiles left practice within the same time frame. 31 surgeons left practice between 1989 and 1999. Of the 25 who could be contacted, 18 responded. 10/18 stated COP had no effect on their decision to leave. 4 of the 10 had been in the lowest quartile prior to leaving. 2/18 said COP had a minimal impact. 6/18 said it had a moderate or substantial impact on decision to leave. Of these 8, 4 were in the lowest quartile.
Evidence against
Chen 201222	New York	IC		NY PCI	351	Cardiologists with poor performance were no more likely to leave practice than those with good performance.

AHD, American Hospital Association database; AMI, acute myocardial infarction; CABG, coronary artery bypass graft; CCIR, Cleveland Cardiovascular Information Registry; CCMRP, California CABG Mortality Reporting Programme; CCN, Cardiac Care Network Database; CCORP, California CABG Outcome Reporting Programme; COP, consultant outcome publication ; CSRS, Cardiac Surgery Reporting System; CTS, cardiothoracic surgery; IC, interventional cardiology; NCDR, National Cardiovascular Data Registry CathPCI Registry; NWQI, North West Quality Improvement Programme in Cardiac Interventions; NY PCI, New York PCI Reports; PCI, percutaneous coronary intervention; SHOCK, Should we emergently revascularise occluded coronaries for cardiogenic shock; STS, Society of Thoracic Surgeons Clinical Registry; US NIC, US National Inpatient Sample Database.