Table 1.
Characteristics of the included studies and effects of PR on clinical outcomes
| First author, year | Objective | Study design | Subject, Setting and Location | Public Reporting mechanism | Clinical outcomes | Key findings | Effect on reported outcome |
|---|---|---|---|---|---|---|---|
| Dziuban et al., 1994 [21] | To evaluate the impact of NYS CSRS program in CABG related mortality in one hospital identified as poor performing | Cohort study for the same facility before and after the introduction of a PR mechanism | One poorly performing, high risk hospital in New York (1992–1993) | NYS CSRS | Cardiac mortality | The NYS CSRS program has been associated with a reduction in the actual CABG-related mortality from 3.52 % in 1989 to 2.78 % in 1992. The risk-adjusted mortality, using pooled data from 1989 to 1992, decreased from 4.17 % in 1989 to 2.45 % in 1992 | Positive |
| Hannan et al., 1994 [14] | To examine changes in the risk-adjusted mortality and operation volume associated with CABG procedures performed during the first 4 years of NYS CSRS in three groups of hospitals | Cohort study for the same facility before and after the introduction of a PR mechanism | New York Hospital cardiac surgeons performing CABG (1989–1992) | NYS CSRS | Cardiac mortality and CABG operation volume | During 4 years of NYS CSRS program, the risk-adjusted mortality decreased from 2.72 to 2.19 % for group1, from 4.24 to 2.51 % for group 2 and from 7.12 to 2.77 % for group 3. The groups of providers that showed the highest initial mortalities manifested the most improvement. The volume of operations performed by the various provider groups did not change substantially in the 4-year period | Positive |
| Rosenthal et al., 1997 [22] | To measure changes in hospital mortality that occurred after implementation of the CHQC, which publicly released in-hospital mortality rates | Cohort study for the same facility before and after the introduction of a PR mechanism | Discharges with 8 diagnosis from Northeastern Ohio hospitals (1992–1993) | CHQC | Cardiac, Respiratory, Neurologic, Gastro-Intestinal mortality | Risk-adjusted mortality for most conditions declined for 3 subsequent periods after publication of mortality data (July-December 1992/January-June 1993/ July-December 1993). Decreases in mortality rates were statistically significant in weighted linear regression analyses for heart failure (0.50 % per period) and pneumonia (0.38 % per period). | Positive |
| Peterson et al., 1998 [23] | To examine the impact of the NYS CSRS on in-hospital mortality rates by comparing mortality rates in New York to those in other states | Cohort study among different facilities with and without PR mechanisms, over the same period of time | New York Hospital Medicare beneficiaries aged 65 years and older who underwent bypass surgery between (1987–1992) | NYS CSRS | Cardiac mortality | After NYS CSRS program initiation, unadjusted 30-day mortality rates following bypass declined by 33 % in NY Medicare patients compared with a 19 % decline nationwide. Risk-adjusted 30-day mortality of bypass surgery in NY patients declined an average of 10.30 % per year (1987–1992) compared with 5.80 % for patients in the rest of the nation. | Positive |
| Baker et al., 2002 [24] | To examine mortality trends during a period (1991–1997) when the CHQC program was operational in Cleveland Hospitals | Cohort study for the same facility before and after the introduction of a PR mechanism | Cleveland Hospital Medicare patients for cardiac, respiratory, neurologic or gastro-intestinal diseases (1991–1997) | CHQC | Cardiac, Respiratory, Neurologic and Gastro-intestinal mortality (GIH) | During CHQC program risk-adjusted in-hospital mortality declined for all conditions except stroke and GIH. The 30-day mortality declined significantly only for CHF to 1.40 %, and COPD to 1.60 %. For stroke, risk-adjusted 30-day mortality actually increased by 4.30 %. | Mixed |
| Chassin, 2002 [25] | To examine the impact of NYS CSRS implementation on mortality rate outlier status and CABG mortality | Cohort study for the same facility before and after the introduction of a PR mechanism | Lowest And Highest CABG Mortality Hospitals In New York, (1989–1995) | NYS CSRS | Cardiac mortality | After NYS CSRS program was implemented risk-adjusted mortality fell 41 % statewide in New York. Mortality statewide continued to fall in the next period; the crude mortality reached 2.15 % in 1998 from 3.52 % in 1989. | Positive |
| Clough et al., 2002 [26] | To verify the decline in in-patient mortality in Cleveland Hospitals and to better understand relationship with CHQC project | Cohort study among different facilities with and without PR mechanisms, over the same period of time | Hospitals included in the Ohio Hospital Association’s in-patient discharge data (1992–1995) | CHQC | Cardiac, Respiratory, Neurologic and Gastro-intestinal mortality | No significant beneficial effect of the CHQC project on hospital mortality in Cleveland was demonstrated. The rate of decline in mortality in Cleveland (−0.218 % per six months) was statistically indistinguishable from that in the rest of the state. | None |
| Baker et al., 2003 [27] | To describe trends in risk-adjusted mortality for six acute conditions for hospitals that were identified as outliers by CHQC compared with other hospitals. | Cohort study for the same facility before and after the introduction of a PR mechanism | Medicare patients with AMI, heart failure, gastrointestinal hemorrhage, obstructive pulmonary disease, pneumonia, or stroke receiving care at Cleveland-area hospitals (1991–1997) | CHQC | Cardiac, Respiratory, Neurologic and Gastro-intestinal mortality | Hospital outlier status was not significantly related to changes in risk-adjusted 30-day mortality. During CHQC reporting period, the absolute decline in risk-adjusted 30-day mortality at “average” hospitals was 0.50 %. | None |
| Dranove et al., 2003 [28] | To study the effects of PR in New York and Pennsylvania on health care providers and patient outcomes | Cohort study for the same facility before and after the introduction of a PR mechanism | Medicare beneficiaries and hospitals found in a Medicare claims data set and hospitals participating in the American Hospital Association annual survey (1987–1994) | New York and Pennsylvania CABG report card | Cardiac, Respiratory, Neurologic and Gastro-intestinal mortality and readmission | Report card provided statistically marginal evidence that the average mortality rate in NY and PA increased by 0,45 % point on a base of 33 %. Report cards increased significantly the average rate of readmission with heart failure by approximately 0,50 % point. | Negative |
| Moscucci et al., 2005 [29] | To compare in-hospital mortality from large multicenter PCI databases in Michigan, where PR is not mandated, and in New York where PR of PCI data is mandatory | Cohort study among different facilities with and without PR mechanisms, over the same period of time | Patients included in a multicenter PCI database in Michigan Hospitals and statewide PCI database in New York Hospitals (1998–2000) | PCI database in Michigan and New York | Cardiac mortality | The unadjusted in-hospital mortality rate was significantly lower in New York than in Michigan (0.83 % vs. 1.54 %, OR = 0.54). However, after adjustment for comorbidities, there was no significant difference in mortality between the two groups (adjusted OR = 1.05). | None |
| Carey et al., 2006 [30] | To examine the relationship between CCSIP and cardiac surgery mortality in California Hospital | Cohort study for the same facility before and after the introduction of a PR mechanism | Cardiac surgery patients (CABG, PCI, Valve) in California Hospital (1998–2004) | CCSIP | Cardiac mortality | The risk-adjusted in-hospital mortality for CABG decreased and PCI mortality remained unchanged. Combining the two procedural groups, the average annual mortality was 1.88 % (1998–2002) compared with 1.67 % (2003–2004) | Positive |
| Guru et al., 2006 [31] | To evaluate the differences in clinical outcomes observed during the transition from no reporting to confidential, and ultimately PR cards for CABG surgery in a public health system in Ontario | Cohort study for the same facility before and after the introduction of a PR mechanism | CABG surgery patients in Ontario Hospitals (1 September 1991–31 March 2002) | Ontario institution-level performance report cards on outcomes of CABG surgery | Cardiac mortality | The risk-adjusted 30-day mortality rate decreased 29 % from the era of no reporting (1991–1993) to confidential reporting (1994–1998). There was no further decrease with PR (1999–2001). In-hospital mortality fell significantly faster in Ontario during the period of confidential reporting than in other parts of Canada | Positive |
| Jha et al., 2006 | To examine the impact of NYS CSRS fifteen years after its launch on cardiac surgery mortality | Cohort study for the same facility before and after the introduction of a PR mechanism | All New York Hospital cardiac surgeons performing CABG (1989–2002) | NYS CSRS | Cardiac mortality | Users who picked a top-performing hospital or surgeon from the latest available report had approximately half the chance of dying (risk-adjusted mortality rate = 1.59) as did those who picked a hospital or surgeon from the bottom quartile (risk-adjusted mortality rate = 2.78). | Positive |
| Hollenbeak et al., 2008 [33] | To assess effect of intensive PHC4 on hospital mortality for 6 high-frequency, high-mortality medical conditions | Cohort study among different facilities with and without PR mechanisms, over the same period of time | Cardiac surgery patients in Pennsylvania Hospitals (1997–2003) | PHC4 | Cardiac, Respiratory, Neurologic and Sepsis mortality | Patients treated at hospitals subjected to intensive PR had significantly lower odds of in-hospital mortality when compared with similar patients treated at hospitals in environments with no PR or only limited reporting. The 2000–2003 in-hospital mortality OR for Pennsylvania patients versus non-Pennsylvania patients ranged from 0.59 to 0.79 across 6 clinical conditions. For the same comparison using the 1997–1999 period, OR ranged from 0.72 to 0.90. | Positive |
| Friedberg et al., 2009 [34] | To determine association of PR with over diagnosis of pneumonia, excessive antibiotic use, or inappropriate prioritization of patients with respiratory symptoms | Cohort study for the same facility before and after the introduction of a PR mechanism | Patients with respiratory symptoms in the National Hospital Ambulatory Medical Care Survey (2001–2005) | Hospital Quality Alliance data on antibiotic timing in pneumonia | Rates of pneumonia diagnosis, antibiotic use, and waiting times to see a physician | Public reporting of hospital antibiotic timing scores has not led to increased pneumonia diagnosis, antibiotic use, or change in patient prioritization. Comparing outcomes before and after antibiotic timing score reporting, there were no differences in rates of pneumonia diagnosis (10 % vs. 11 %) or antibiotic administration (34 % vs. 35 %). | None |
| Ryan, 2009 [35] | To evaluate the effects of the PHQID, a public quality reporting and P4P program, on Medicare patient mortality | Cohort study among different facilities with and without PR mechanisms, over the same period of time | Medicare patients with AMI, heart failure, pneumonia, or a CABG procedure from acute care hospitals (2000–2006). | PHQID program | Cardiac and Respiratory mortality | No evidence that the PHQID had a significant effect on risk-adjusted 30-day mortality for AMI, heart failure, pneumonia, or CABG. | None |
| Li et al., 2010 [36] | To evaluate the impact of PR by comparing CABG volume and mortality for hospitals and surgeons in the first year of state-mandated PR (2003), and with the most recent data available (2006) | Cohort study for the same facility before and after the introduction of a PR mechanism | Cardiac surgery patients from the California Hospital CABG Outcomes Reporting Program database for 2003 and 2006 | California CABG Outcomes Reporting Program | Cardiac mortality | The statewide observed mortality declined from 2.90 % in 2003 to 2.22 % in 2006. Overall, the empiric odds ratio of operative death for 2006 patients was 24 % lower than for 2003 patients. Total CABG volume decreased from 2003 to 2006 by almost 27 %. | Positive |
| Werner et al., 2010 [37] | To estimate changes in cardiac and respiratory mortality, length of stay and readmission rate after Hospital Compare was initiated | Cohort study for the same facility before and after the introduction of a PR mechanism | Patients with AMI, heart failure, pneumonia from 3,476 acute care, nonfederal U.S. hospitals that publicly reported quality information on the CMS Hospital Compare Web site (2004–2006) | The Centers for Medicare and Medicaid Services and other health care organizations participate in the Hospital Quality Alliance | Cardiac and Respiratory mortality, length of stay, readmission rate | There was a decline in mortality rates (0.6 % points), lengths-of-stay (0.19 days), and readmission rates (0.5 % points) for acute myocardial infarction from 2004 to 2006. Changes in outcomes for heart failure and pneumonia were less consistent and smaller, when present at all. | Positive |
| Jha et al., 2012 [38] | To assess the long-term effect of the Medicare PHQID on cardiac and respiratory mortality at Premier versus Non Premier hospitals | Cohort study among different facilities with and without PR mechanisms, over the same period of time | Patients with AMI, CHF, CABG, pneumonia in New England Hospital (2003–2009) | Premier Healthcare Informatics program | Cardiac and Respiratory 30-day mortality | No evidence that the largest hospital-based pay-for-performance program led to a decrease in 30-day mortality. The rates of decline in mortality per quarter at Premier and Non Premier hospitals were also similar (0.04 and 0.04 %, respectively; and mortality remained similar after 6 years under the pay-for-performance system (11.82 % for Premier hospitals and 11.74 % for non-Premier hospitals) | None |
| Joynt et al., 2012 [39] | To evaluate PCI mortality in PR states versus non-reporting states in USA | Cohort study among different facilities with and without PR mechanisms, over the same period of time | Medicare patients admitted with acute MI to US acute care hospitals (2002–2010) | Mandatory state PR programs (NY, MA and PA) for PCI | Cardiac mortality | There were no differences in overall mortality among patients with acute MI in reporting vs non reporting states. In Massachusetts, odds of PCI for acute MI were comparable with odds in non reporting states prior to PR (40.6 % vs 41.8 %; OR, 1.00). Among Medicare beneficiaries with acute MI, the use of PCI was lower for patients treated in states with PR compared with patients treated in states without PR | Mixed |
| Renzi et al., 2012 [40] | To evaluate association between public reporting of hospital performance and PCI rates, hip fractures, cesarean deliveries in Lazio versus other regions of Italy | Cohort study for the same facility before and after the introduction of a PR mechanism and among different facilities with and without PR mechanisms over the same period of time | Patients with acute MI, hip fractures, and maternity patients discharged from any hospital within the Italian National Health Service (2006–2009) | Regional Outcome Evaluation Program P.Re. Val.E. | PCI rates, hip fractures, cesarean deliveries | In Lazio PCI within 48 h, changed from 22.49 to 29.43 % following reporting of the P.Re.Val.E results. In the other regions this proportion increased from 22.48 to 27.09 % during the same time period. Hip fractures operated on within 48 h increased from 11.73 to 15.78 % in Lazio, and not in other regions (from 29.36 to 28.57 %). Cesarean deliveries did not decrease in Lazio (from 34.57 to 35.30 %, and only slightly decreased in the other regions (from 30.49 to 28.11 %). | Positive |
| Ryan et al., 2012 [41] | To estimate the effect of Hospital Compare, Medicare’s PR initiative on 30-day mortality for heart attack, heart failure, and pneumonia | Cohort study for the same facility before and after the introduction of a PR mechanism | Medicare patients in USA Hospitals with heart attack, heart failure, pneumonia, stroke, gastrointestinal hemorrhage, and hip fracture (2000–2008) | Hospital Compare, Medicare’s PR initiative | Cardiac and Respiratory 30-day mortality | Hospitals that reported quality data under Hospital Compare had no reductions in mortality beyond existing trends for heart attack and pneumonia and led to a modest reduction in mortality for heart failure (RR = 0.92) | None |
| Linkin et al., 2013 [42] | To evaluate the association between state-legislated PR of hospital-acquired infection with infection control process | Cohort study among different facilities with and without PR mechanisms, over the same period of time | Patients from 137 eligible US hospitals in 35 states (2008–2011) | Medicare’s Hospital Compare website reports | Improvements in infection prevention | There is not estimated improvement in infection prevention program or hospital-acquired infection rates in hospitals in states legislating mandatory PR | None |
| McCabe et al., 2013 [43] | To evaluate the impact of PR of hospitals as negative outliers, on PCI risk adjusted mortality and case mix selection | Cohort study among different facilities with and without PR mechanisms, over the same period of time | Cardiac patients at all non-federally funded Massachusetts hospitals performing PCI (2003–2010) | National Cardiovascular Data Registry and Massachusetts Data Analysis Center model | In-hospital cardiac mortality | After public identification as a negative outlier institution, there was an 18 % relative reduction in predicted mortality among PCI patients at outlier institutions compared with non-outlier institutions | None |
| Marsteller et al., 2014 [44] | To evaluate mandatory reporting in participation and performance in reducing CLABSI in a national patient safety collaborative | Cohort study among different facilities with and without PR mechanisms, over the same period of time | Patients of intensive care units participating in the US national Comprehensive Unit-based Safety Program: Stop Bloodstream Infections (2009–2011) | Comprehensive Unit-based Safety Program: Stop Bloodstream Infections | CLABSI mortality | There was a reductions in CLABSI rates in the first 6 months compared with the units in states with no reporting requirement. During months 13–18, both state groups with mandatory PR of CLABSI showed a trend toward greater reduction in CLABSI compared with states with no requirement. | Positive |
| Wang et al., 2014 [45] | To evaluate the effect of publicly reporting performance data of medicine use on the injection prescribing rate | Cohort study for the same facility before and after the introduction of a PR mechanism | Effective electronic injection prescriptions in Primary healthcare institutions of Hubei province (China 2013–2014) | Database of electronic prescriptions of the local health bureau | Injection prescribing rates | PR led to a reduction of approximately 4 % in the injection prescribing rate four months after intervention (OR = 0.96). The intervention effect was inconsistent in each month after intervention, and it was most positive in the second month after intervention (OR = 0.90) | Positive |
| Yang et al., 2014 [46] | To evaluate the impact of PR on antibiotic prescribing for URTI in a sample of primary care institutions | Cohort study among different facilities with and without PR mechanisms, over the same period of time | URTI patients in Primary healthcare institutions of Hubei province (China 2013–2014) | Electronic health information system | Antibiotic prescribing | PR interventions reduced the incidence of oral antibiotic prescription (9 % point reduction adjusted RR = 39 %) and slowed down the increase of combined use of antibiotics for URTIs (7 % point reduction (adjusted RR = 36 %), while the use of injectable antibiotics remained unchanged. The intervention had little impact on the use of IV injections or infusions, or the total prescription expenditure | Mixed |
Abbreviations: AMI Acute Myocardial Infarction, CABG Coronary Artery Bypass Graft, CCSIP California Cardiac Surgery and Intervention Project, CHF Congestive Heart Failure, CHQC Cleveland Health Quality Choice, CLABSI Central Line-Associated BloodStream Infections, NYS CSRS New York State Cardiac Surgery Reporting System, P4P Pay-for-Performance, PCI Percutaneous Coronary Intervention, PHC4 Pennsylvania Health Care Cost Containment Council, PHQID Premier Hospital Quality Incentive Demonstration, PR Public Reporting, URTI Upper Respiratory Tract Infections