Public views on healthcare performance indicators and patient choice

Abstract

Patients on certain waiting lists in the UK National Health Service (NHS) are now offered the choice of persevering with their home hospital or switching to another hospital where they will be treated on a guaranteed date. Such decisions require knowledge of performance. We used facilitated focus groups to investigate the views of patients and members of the public on publication of information about the performance of healthcare providers. Six groups with a total of 50 participants met in six different locations in England.

Participants felt that independent monitoring of healthcare performance is necessary, but they were ambivalent about the value of performance indicators and hospital rankings. They tended to distrust government information and preferred the presentational style of 'Dr Foster', a commercial information provider, because it gave more detailed locally relevant information. Many participants felt the NHS did not offer much scope for choice of provider.

If public access to performance information is to succeed in informing referral decisions and raising quality standards, the public and general practitioners will need education on how to interpret and use the data.

INTRODUCTION

The British Government is keen to promote choice and accountability in the National Health Service (NHS) by measuring performance and publishing performance indicators and by offering patients on the waiting list a choice of provider. The Patient Choice initiative, launched last year, was initially confined to patients with coronary heart disease but now extends to patients waiting for a wide range of elective surgical procedures. Certain patients who have been on the waiting list for six months are offered a choice of having their treatment in a different hospital on a guaranteed date or continuing to wait for treatment at their 'home' hospital. The NHS Performance Ratings are now published regularly, and over the next few years these will be refined and expanded. In January 2001 'Dr Foster', an independent organization set up to provide patients and the public with healthcare performance information, was launched on the internet and in the media. 'Dr Foster' hospital guides include both narrative descriptions of facilities and outcome assessments, including mortality rates.¹

Little is known about the extent or nature of public demand for this type of information, or what patients think about using it to choose providers. We report a focus group study to investigate views on the existing range of performance indicators and to find out what type of information patients and members of the public want.

METHODS

Six focus groups were conducted in different locations around England involving a total of 50 people. Locations (Sheffield, Brighton, Huntingdon, Coventry, Northampton and Banbury) were chosen to include areas where the local acute trusts had 3-star or 0-star ratings. The groups had 7–10 participants and each discussion lasted around two hours (see Table 1). One group involved members of a carers' group; one was made up entirely of members of ethnic minorities; and all included people with recent inpatient experience of the NHS as well as individuals with chronic conditions and healthy members of the public.

Table 1.

Focus group participants

Group	Age	Gender	Social class	No. in group	Details	Location
1	49-68	F	BC1E	7	Carers	Sheffield
2	23-67	M/F	BC1	9	Healthy/disabled/outpatient/inpatient	Brighton
3	19-51	M/F	BC1	10	Healthy/disabled/outpatient/inpatient/carer	Huntingdon
4	28-57	M/F	C2DE	9	Healthy/disabled/outpatient/inpatient/carer	Coventry
5	18-69	M/F	BC1C2	8	Healthy/outpatient/inpatient/carer	Northampton
6	18-69	M/F	C1DE	7	Afro-Caribbean/Asian/Chinese: healthy/outpatient	Banbury

Participants for five of the six groups were recruited by an agency using street recruiters who provided their homes as venues. The carers' group met in a hospital. Participants were paid £30. A topic guide was used to focus the discussion. Participants were asked for their views on measuring and comparing performance in the NHS, what should be measured and who should be responsible for monitoring performance. The study predated the launch of the Patient Choice schemes, but groups were conducted a few weeks after publication of the Mail on Sunday's 'Good Hospital Guide' which was produced by Dr Foster. This was used together with other press cuttings and material downloaded from the Department of Health (DH) and Dr Foster websites, to stimulate discussion. Each group discussion was recorded and the transcripts were read and analysed independently by the focus group facilitator (HM) and another researcher (LD), who identified and agreed on the main themes.

RESULTS

Effects of measuring and comparing performance

There was wide agreement that performance should be monitored in some way. Some participants were familiar with performance indicators and saw publication of this type of comparative information as inevitable and potentially beneficial (Box 1). There was recognition that monitoring performance was increasingly part of modern life and participants singled out other areas such as education and the police service where these methods were used. But experience of other forms of performance measurement did not always result in a positive attitude towards them and many participants reacted negatively to the notion of league tables.

Some felt that publication of hospital rankings could lead to an improvement in standards, but others highlighted the possible negative results of 'naming and shaming.' Of particular concern was the morale of staff working in poorly performing hospitals and the consequences of heavy demand on successful hospitals. Fears were expressed that publication could raise alarm, particularly amongst more vulnerable people. Several participants commented negatively about the cost of gathering performance information, saying the money could be better spent elsewhere in the NHS.

Reactions to Department of Health performance ratings

Before being shown examples of the performance ratings, most participants had only a vague awareness of the recently published hospital league tables. There was a good deal of confusion between the DH ratings and the Dr Foster guides. Hardly anyone knew exactly what star rating their local hospital trust had been awarded.

Box 1 Effects of measuring and comparing performance

'Because I use performance indicators in my job and it is a way of seeing what people are doing, I think it is a good idea.'

'It would make whoever is at the bottom of the list pull their socks up.'

'You have got to shame them into doing something about it.'

'It can dishearten them, can't it... demoralize the staff.'

'Nurses could look at this and say, well I don't want to work there, it's not a very good hospital.'

'It might lead to overcrowding in say the better hospitals: oh we'll go there.'

'I think to target individual hospitals is sort of scaremongering really for the people who live around those areas.'

'It might make the managers manipulate the figures so they don't look so bad next time.'

Box 2 Reactions to Department of Health's performance ratings

'I just remember hearing something on the radio and I thought my God, how quick to damn a hospital which has probably got superb staff in there.'

'I get so angry with league tables because we don't know the full facts. I'm sorry but they would have to print a 50-page document for me.'

'Government figures don't mean anything to me because government is baloney.'

'I think that if the Government sent them out then maybe you've got a better chance of believing it.'

'I am not political, but I think most of them massage figures to suit themselves.'

'I don't trust this waiting list thing because I know for a fact they don't put you on the waiting list until they know you can be treated.'

'It depends how much it's costing really. If it's costing a couple of million to get it all put out, that to me is wasting money because two million can go into a kiddies' hospital.'

There was considerable mistrust of the DH ratings (Box 2). In many cases this stemmed from a general distrust of any Government statistics. There was an expectation of manipulation by Government and hospital managers. Several participants were particularly sceptical about the figures on waiting times and cancelled operations. Participants were confused about how the data for performance ratings were collected and assessed.

Reactions to Dr Foster information

Several participants had seen the 'Good Hospital' and 'Good Consultant' guides produced by Dr Foster (Box 3). Participants generally found these more user-friendly than the DH ratings, although some were confused about how to interpret part of the data. Basic information in these guides about hospital facilities, public transport etc. was felt to be useful for patients and their visitors. The detailed information on waiting times in specific clinical areas was also considered useful.

Box 3 Reactions to Dr Foster information

'It seems to be better laid out and a little write-up on each hospital which is analysing the figures they have found out rather than just slapped down on a page.'

'Something like this [Dr Foster's Good Hospital Guide] would be good if it was delivered with your free papers.'

'This is much more useful because it is down to basic questions with basic answers, like does the hospital have a newsagent. Little things like that are the sort of things that it's handy to find out about.'

'But who pays Dr Foster then?'

'You don't know who's been the financial backer of something like this. I don't take much information as read. I'm a “why” girl, I'm afraid.'

'So if you look at it like that, it is saying that it [Walsgrave Hospital] is a good hospital... and we know it isn't.'

Wary of vested interests affecting the reliability of the information, some participants wanted to know who funded Dr Foster. The news that Dr Foster relied on the DH for some of its information generated scepticism about its reliability.

Participants wanted information about specific local services rather than generalized, comparative information. Priorities included information on waiting times, consultants' special interests, their clinical experience and their success rates. This led most to prefer the Dr Foster approach because it was more detailed and locally relevant.

Choice

The extent to which participants thought performance information was useful depended on their perception of the choice available to them: most were not confident that they had any. Even if choice were possible, many participants did not like the idea of 'shopping around' and expected a high standard everywhere (Box 4).

Participants were divided over whether they would be prepared to travel to a hospital outside their area if they had the opportunity to do so. Some were willing to accept the idea even if it meant going abroad for treatment; others were opposed. Most participants relied heavily on their general practitioner to make decisions on their behalf and they expected the GP to understand and use performance information when making referral decisions.

Accountability

Although initially there was little enthusiasm for performance indicators, on being shown the material most participants thought it should be publicly available (Box 5). Some felt that as taxpayers they were entitled to this information and many supported the notion that healthcare providers should be required to account for their performance. Others felt this type of information would be relevant only when they needed it as a patient.

Box 4 Choice

'What is the benefit of this to us when we are probably stuck with the hospital anyway.'

'If you are going into a maternity ward, you want information about maternity. You don't want anything generalized because it clouds the issue.'

'If you were told you have got the choice between two and one had a 9-week waiting list and one had a week's waiting list, to be honest I'd go for the 9-week waiting list because you know that everybody is going there because he's a good guy.'

'If I needed a serious operation and I knew another hospital could do it quicker than 6 months, I would go.'

'I think it would be a good idea if you had like a portfolio on the doctor, like he's been successful in this and he specializes in that.'

'If the specialist treatment you are going to get is 100 miles away and is better than any treatment available locally, you are going to go for the specialist.'

'If you need to travel to get the best treatment, surely that's better than getting a sub-standard level of treatment.'

'I'd hate it, it's totally inconvenient to have to travel.'

'You rely on your GP to direct you to the consultant he thinks is the best one, most likely a pal of his.'

'I shouldn't have to go shopping around to get healthcare for my son. I would hope that my GP would have that sort of information.'

'You shouldn't have to be surfing the net or looking all over the country for the best place to get treatment, you should be able to get it anywhere.'

Box 5 Accountability

'It's our National Health, we should know it's working and where it's going wrong.'

'We all have to do it nowadays in our jobs. We have to prove what we are doing.'

'But what can you do about it? Tax payments and insurance payments are automatically deducted from your pay; you have no say in how it's spent, so it's a case of getting in touch with your MP.'

'If you are a general Joe off the street you don't look in the papers at these things that are going on in hospitals.'

'I think the people who need it are the people who are going to be able to make changes. To highlight massive problems through the media isn't going to help anybody diddly squat.'

'Doctors can do what they want and I don't even know who they are accountable to. Perhaps the surgery needs to be accountable to someone.'

'Ultimately they [the Government] have got to be the ones responsible because that's where the buck stops.'

'An independent commission because the minute it is in the Government you don't know if that is the real version, if it is the truth.'

'A hospital watch type set-up. I mean we have it on the railways, don't we?'

'The user is the patient. The patients should be able to assess the hospitals.'

'With the advent of computers your GP should have that sort of information at the touch of a button.'

Both the DH performance ratings and Dr Foster provide information on mortality rates but very few participants found this information useful. Some questioned its relevance to clinical quality, others found it frightening; but a few felt that publication of such information is essential, citing the Bristol Inquiry into child deaths. There was general agreement that independent assessment is important. Many participants did not trust the Government to arrange this, believing the task should be assigned to an independent commission or consumer 'watchdog'.

DISCUSSION

Patients are only one among several potential audiences for performance information, but evidence on what type of information British patients want, and what they might use it for, is sparse. This issue has been studied more extensively in the USA, particularly in relation to choice of health plan.²

Despite the longer history in the USA of publishing information on the quality of care among different providers, there is as yet little evidence that American patients are using this on a major scale to make choices.^3–9 The explanations that have been offered include the following: consumers are not aware of variations in quality so do not seek information about 'the best' providers;⁵ consumers do not believe they have a choice or prefer to leave it to their employer to choose a plan;¹⁰ relevant information is not available at the time it is needed;⁹ healthcare report cards are badly designed and consumers find them hard to understand;^{11, 12, 13} consumers do not trust the information or its source.⁵

Our findings suggest that the British public is likewise ambivalent about the value of performance indicators. Awareness and comprehension of these is currently quite limited, but there is a strong sense that some form of public monitoring is necessary and desirable. Meanwhile the Government is pressing ahead with its plans for wider dissemination of performance information. The justification for this policy may be different in the UK from the USA. While Americans are accustomed to the idea of market competition in healthcare, this notion is more foreign to the British public. British patients have historically tended to rely on their GPs to make choices on their behalf, so it will be interesting to see if the Patient Choice schemes result in greater willingness to choose providers and greater demand for comparative information on the quality of care. Nevertheless, our study suggests that many British people do feel that performance information ought to be available and that the NHS should be seen to be accountable to the taxpayer.

The establishment of the Office of Information on Health Care Performance within the Commission for Health Improvement goes some way towards meeting the demand identified here for an independent source of performance information. It remains to be seen if it will be viewed as more trustworthy than the Department of Health in the eyes of the public.

A key issue is whether greater transparency will enhance or undermine public confidence in the NHS.^14,15 To increase the likelihood of beneficial impact, it will be necessary to educate the public and GPs about how to interpret and use the data, and to ensure that choices are available for those who want to exercise them.