Is the NHS getting better or worse?

The NHS is a shambles, and you are too much of a coward to say so. This was the gist of an email I received from an NHS consultant a few weeks ago. I answered—weakly in his eyes—that I couldn't be sure that the NHS was collapsing. I met many people who agreed with him but also many who thought otherwise. I didn't see clear evidence. Yet whether the NHS is improving may be the most important political question in Britain. The government, which has increased NHS expenditure by billions and launched into a 10 year modernisation plan, insists that it is improving. The opposition alleges that the money is being wasted. The people want a better health service, and a billion pound investment that came to nothing would be a national tragedy. So what is the answer? The main conclusion of an extensive, independent review funded by the Nuffield Trust and published this week is that we don't have the data to answer the question reliably.¹ This in itself is an indictment—particularly when the NHS is awash with bodies auditing and inspecting it.

The review—which is of quality in the NHS in England not the other three home countries—has been conducted by Sheila Leatherman, an American professor with appointments in both the United States and the United Kingdom, and Kim Sutherland from the Judge Institute in Cambridge. They describe the review as a mid-term evaluation of the Labour government's 10 year agenda described in The New NHS—Modern, Dependable: a National Framework for Assessing Performance, published in 1997.² That document introduced the National Institute for Clinical Excellence (NICE), the Commission for Health Improvement (CHI), national service frameworks, primary care groups, and the concept of clinical governance. In January 2000—under great political pressure—the prime minister announced that expenditure on health care would be brought up to the average in the European Union, and in July the NHS plan was launched, which among other initiatives announced the creation of the Modernisation Agency.³ All of these policy developments have come against the backcloth of the inquiries into the excess deaths in children's heart surgery in Bristol and general practitioner Harold Shipman murdering dozens of his patients. In addition, the General Medical Council has been reformed, and revalidation is being introduced for all doctors.

Is this more than an uncoordinated orgy of reform driven by panic? Leatherman and Sutherland think so. They describe it as “the most ambitious, comprehensive and intentionally funded national initiative to improve health care quality in the world.” This bold statement stands up to close examination. Even the most shameless politician in Britain could not any longer boast that the country has the world's best healthcare system—but it seems to have the most ambitious programme of reform. It is impossible to reform healthcare systems with single initiatives. Multifaceted strategies are needed, and some parts of the programmes will fail. Leatherman and Sutherland think that the reforms lack coherence, and so far there has been much greater impact at national and regional levels than at institutional or individual levels. And it is the individual level that matters. Elegant and imaginative reforms at higher levels are worthless if they do not translate into an improved experience for individual patients and a public perception that things are better.

What does the evidence show on NHS quality?

Leatherman and Sutherland scoured the NHS for data to show what is happening with quality of care. They may have assembled more data than anybody else, but the evidence still falls far short of a complete, validated, interpretable, uncontested picture—even though they entered rooms where people had for years been collecting data that nobody before had ever asked to see. The data that are available show a mixed picture of improvement, stasis, and deterioration.

Access to care is the most politically contentious aspect of quality, and the number of patients waiting 12 months or longer for admission to hospital has fallen from 50 000 in 1999 to only 73 in the fourth quarter of 2002-3. But a fifth of people still wait for more than six months. The number of operations cancelled at the last minute, which is frustrating for both patients and staff, increased from around 56 000 in 1998-9 to 68 000 in 2002-3. Waiting times in emergency departments have improved, but a third of patients still wait more than two hours. Access to general practitioners seems to have deteriorated, with 13% of patients waiting more than two days in 1998 and 23% in 2003. Meanwhile, calls to NHS Direct increased from 110 000 in 19989-9 to over 6 million in 2002-3.

Effectiveness also shows a mixed picture. The percentage of children being immunised—particularly against measles, mumps, and rubella—has declined, but the proportion of hospitals giving thrombolysis to 75% of patients with heart attacks has increased from 24% to 45%. Other targets of the national service frameworks for heart disease and cancer are being met, and mortality from both circulatory disease and cancer is falling. We don't know, of course, whether these changes—both favourable and unfavourable—would have occurred without billions of pounds of quality initiatives. Nor do we have good data on what is happening with conditions not covered by national service frameworks. There must be anxiety that services that were not prioritised may have deteriorated.

The capacity of the system has generally improved. The number of nurses increased from 256 000 in 1997 to 291 000 in 2002, but the numbers of general practitioners and consultants have not increased anywhere near as fast. Facilities for caring for patients who have strokes have increased, but three quarters of patients still spend less than half their time in a stroke unit (down from 83% in 1998).

Ultimately of course it must be patients and the public who determine the quality of care. Data purporting to show improvement will be worthless if it doesn't feel that way to patients and the public. The percentage of the public who think that the NHS needs a “complete rebuild” or “fundamental change” fell from 78% in 1988 through 69% in 1998 to 72% in 2001. Dissatisfaction with the system (measured in this way) thus seems to be high and has increased since the quality initiative began. Complaints to the health service ombudsman have increased from 2500 in 1999-2000 to 4000 in 2002-3—although the number of investigations has declined and the overall number is a minute proportion of all patient encounters with the NHS. Few other patient measures allow comparison across time, but a quarter of patients did not feel that “hospital staff did everything they could to control pain,” and 40% said they were not told about the danger signals regarding their illness or treatment to watch for after they went home. Nevertheless, 80% or more of patients felt that they were “always” treated with respect and dignity when in hospital or seeing their general practitioners.

Leatherman and Sutherland not only reviewed published reports and gathered data but also spoke to over 50 “experts and leaders” (I was one of them), and their conclusion is cautiously positive. They see a willingness to admit problems, a favourable context for policy, reasonable resources and organisational capacity for improving quality, and a multipronged strategy. They don't, however, see a common understanding of the “state of quality” (hence the NHS consultant who emailed me, insisting it is lamentable), sufficient involvement of public and patients, adequate leadership at all levels, and enough involvement of the clinical professions. Greater involvement of public, patients, and professions will be essential for long term success.

Recommendations for mid-term change

The review's first recommendation is for a quality information centre, which is already blessed with the essential acronym—QuIC. The authors are surprised that this turned out to be their first recommendation, and many may groan at the thought of yet another institution. But reliable measurement is essential for improvement. Otherwise, we can never know whether changes are making things better or worse. It might be that this body could be absorbed into the new Commission for Healthcare Audit and Inspection (CHAI), particularly as its chairman, Ian Kennedy, says that he wants to replace the “men in bowler hats” of the old CHI with “a mirror” that can be held up for trusts and government to assess their performance. It will be essential, however, that the data can be believed by the public and professions. Any sense of the data being spun will be disastrous. Leatherman and Sutherland say that the body must be credible, independent, dispassionate, “deeply” competent, stable, and longstanding, and “serving in the interest of the public.” These criteria may be hard to achieve for a health service that is one of the most politicised in the world. One product of QuIC might be an annual report on the state of quality.

Another recommendation is to engage the public and patients. Angela Coulter, director of Picker Institute Europe (which specialises in measuring patients' experiences and using their feedback to improve the quality of health care), contributed a chapter to the review and concluded that a critical stocktaking of achievements to date (in the strategy to put patients at the centre of the NHS) reveals a collection of disconnected initiatives rather than a coherent joined up strategy.⁴ Leatherman and Sutherland present many proposals on how to engage the public and patients but had much greater difficulty with knowing how to implement their recommendation to engage the professions. A fundamental problem with the quality initiative is that it isn't owned by the professions. Many clinicians are involved in many improvement projects, but the initiative belongs to the government and those directly in its thrall. Yet real improvements can be delivered only with the full participation of clinicians and their institutions. There must be a role here for the royal colleges and specialist associations, but few have risen comprehensively to the challenge.

The final conclusion of Leatherman and Sutherland is that the quality initiative is moving in the right direction and that incremental refinements are needed not a complete redesign. Nobody in the service could stomach a complete change in direction, but producing reliable evidence on the quality of the NHS and fully engaging the public, patients, and the professions are major challenges. I am not confident that a state of quality and grace will be achieved in the NHS in another five years. Further muddling through seems more likely.