Performance indicators that "name and shame" NHS organisations or individuals should be scrapped because they "demoralise and antagonise" staff and damage the health service as a whole, the Royal Statistical Society said in a report released last week.
The society says that public sector employees and the public are suspicious of government performance statistics and see them as being used to meet political ends and as likely to be misreported by the media. The society wants the government to take a "much wider consideration of the ethics and cost-effectiveness of performance monitoring" and to look at alternatives.
Launching the report, Professor Sheila Bird, of the Medical Research Council's Biostatistics Unit, said the NHS's star ratings system was "in essence" misleading and called for a radical overhaul of performance monitoring.
She said that in spite of some good examples statistical standards had "largely been ignored" and that the criticism applied not just to target setting by the government but to the design, analysis, and media reporting of performance indicators.
Professor Bird said that when publishing league tables or star ratings the authors and the media should present the data in the context of measures of uncertainty to "avoid over-interpretation and the false labelling of performance."
The society is also calling for strict protocols to ensure that statistical standards are met and for a new independent body to scrutinise the "wider... public interest, the individuals and institutions being monitored and the methodological rigour."
Asked to give an example of a performance measure that should be abolished, Professor Bird said that mandatory drugs tests of prisoners since 1995 and drug treatment and testing orders, which were piloted in 1998, had been rolled out without proper randomised trials.
While intended to cut rates of reoffending among users of hard drugs, the tests do not discriminate between testing positive for cannabis and testing positive for class A drugs. "As a result UK judges prescribe sentences on less evidence than doctors prescribe medicines," she said.
While the society concedes that performance monitoring has brought about particular changes, such as reductions in waiting times, its report raises questions about the high cost of data capture and analysis and says that "unmeasured" parts of the NHS suffer.
It also says that performance monitoring can lead to "unintended consequences such as manipulation of data, gaming or fraud by service providers" and that it could "inhibit new approaches to service delivery."
The society is also concerned that presentation of data on the performance of cardiac surgeons in England could prove counterproductive.
"Among cardiac surgeons in New York whose individual unadjusted patient death rates have been published regularly, there has been a tendency to avoid taking high risk cases, with a subsequent increase in the mortality of Medicare patients at risk for cardiac surgery," the report says. The society would prefer that any data on English surgeons be for a five year period rather than an unrepresentative snapshot.
The society's president, Andy Grieve, said he wants to work with the government and parliament to foster good practices and that he will "engage with journalists in pursuit of better reporting standards."
Performance Indicators: The Good, the Bad and the Ugly is available at www.rss.org.uk