Development of an information source for patients and the public about general practice services: an action research study

Abstract

Objective  The publication of information about the performance of health‐care providers is regarded as central to promoting greater accountability and empowering patients to exercise choice. The evidence suggests that the public is not very interested in accessing or using current sources of information. This study aimed to explore the information needs of patients in the context of UK primary care and to develop an information source about general practice services, designed to be usable by and useful to patients.

Design  An action research study making use of data from formal and informal interviews, focus groups, participant observation and document review.

Setting  The geographical areas covered by two Primary Care Trusts in the north of England and two Local Health Boards in south Wales.

Participants  A partnership between 103 members of the public, general practice staff from 19 practices, NHS managers from four Primary Care Organizations and the research team.

Results  The public would like to know more about the quality and range of general practice services but current sources of information do not meet their needs. The public do not like league tables comparing the performance of practices and only a small number of people want to use comparative information to choose between practices. They seem to be more interested in the context and availability of services and the willingness of practices to improve, than in the practice's absolute or relative performance. They want to be clear about the source of the information so that they can make personal judgements about its veracity. Information is most likely to be useful if it adheres to the basic principles of cognitive science in terms of its structure, content and presentation format. Using these findings, paper and electronic prototype versions of a guide to general practice services have been developed.

Conclusions  In order to maximize the potential use of performance information by the public it is necessary to move beyond provider‐led and professionally constructed approaches to information provision and ensure that the public is actively involved in the development of information sources. Such involvement produces a different kind of information to that currently available to the public. The findings of this study have important implications for policy. Most importantly, it seems that the traditional consumerist model underlying a policy of making comparative performance information available to the public to enable them to exercise choice between primary care providers may not be appropriate. An alternative model of information provision, which recognizes the public's commitment to their practice and is integrated with ‘soft’ sources of knowledge is more likely to engage and be of use to the public.

Introduction

Providing better and more accessible information about the performance of health‐care providers is regarded as essential if health services are to become more orientated around the needs of patients and members of the public. ¹ Better access to information is advocated on the grounds that it promotes accountability and encourages providers to improve their performance. ² Comparative information also provides patients with an opportunity to choose between different providers, an issue which, though controversial in the National Health service (NHS), ³ is seen as key to public sector reform by the UK government ⁴ , ⁵ and a prerequisite for patient choice in market‐based health systems. ⁶

Despite these perceived benefits, practical initiatives in the United Kingdom have, until recently, been slow to follow the policy rhetoric. High‐level data, for example, about the performance of Health Authorities or hospitals, have been published since 1983. ⁷ , ⁸ Recently, there have been attempts to make this information more accessible to the public in the form of organizational star ratings, ⁹ the Good Hospital and Good Birth guides produced by the drfoster group ¹⁰ and by the development of local information sources called Your Guide to Local Health Services. ¹¹ These developments have largely failed to address the fundamental problem that the evidence suggests that a large proportion of the public is at best disinterested in accessing or using information. ¹²

Some commentators have argued that this lack of interest represents an unwillingness on the part of the public to behave like ‘consumers’ of health services. ¹³ Others have claimed that it results from inadequacies in the content, presentation format or timeliness of the information. ⁶ , ¹⁴ Others still have argued that it is just a question of time and that more recent evidence indicates that public attitudes towards performance data, and their willingness to exercise choice, are becoming more positive as the information becomes more familiar. ¹⁵ , ¹⁶ In the United States at least, a small proportion of patients do seem to be starting to make use of comparative information. ¹⁷ , ¹⁸ Despite this emerging evidence, the overall picture is a disappointing one for those who see an informed public as a key lever for health system improvement.

The difficulties being encountered in developing high quality and usable sources of information presents challenges to policy‐makers and to front line staff. The former will fail to realize the potential benefits associated with a more informed public if people remain disinterested in information. Clinicians and managers are under pressure to publish more information but there is little useful and rigorous evidence to help them to do so in a cost‐effective way. This is particularly true in relation to general practice, the part of the health system with which patients are most familiar and have most contact but about which the public have little hard performance information.

This study describes the first stage of an action research project which aimed to improve the level of public engagement with information about health services. By actively involving patients, and by drawing on expertise from fields such as cognitive psychology and marketing, we aimed to clarify the factors influencing the use of performance information about general practice services by the public and then to use this information to develop an information source designed to be usable by and useful to patients and the public.

Methods

Rationale for design

We chose an action research‐based approach because it is compatible with the participative and developmental nature of the project and with our desire to empower service users and generate a tangible product. ¹⁹ Action research explicitly acknowledges the nature of the project as a complex social process and the role of the researcher as a facilitator of change. ²⁰ Using an action research approach enabled the research team to act as a partner in the process, with all of the participants sharing views and contributing to the change processes, according to their knowledge and expertise.

Setting and participants

The 3‐year study received ethics approval and data collection for the part of the project presented in this study took place between September 2002 and April 2004. We worked with 103 members of the public, general practice staff and NHS managers in the areas covered by four purposefully sampled Primary Care Organizations (PCOs) located in the north of England and in south Wales.

The PCOs were selected on the basis of their geographical proximity to the research bases, their willingness to participate in the project, and their contrasting demographic and organizational characteristics (Table 1). Within each of the PCOs we worked with a senior member of the management team, who ‘championed’ the project and recruited up to six volunteer practices. Each of these practices agreed to work with their patients, PCO managers and the research team to develop and publish information about their services and performance. Patient participants were drawn from established practice‐based patient participation groups, or from individuals who responded to advertisements in the practice waiting rooms. Although they were all volunteers, they included representatives from both genders, all social classes and adults from all age groups. Project steering groups were established in each of the four sites, comprising patient, clinical and support staff representatives from each of the participating practices and clinical governance or performance managers from the PCOs. The local project champion and the lead researcher for each site co‐chaired the steering group meetings.

Table 1.

 Information about the study sites

	Site 1	Site 2	Site 3	Site 4
Demographic characteristics of PCO population	Rural and semi‐urban. Mixed socio‐economic status	Inner city. Mostly socio‐economically deprived	Urban. Mixed socio‐economic status	Urban, most economically deprived
Number of participating practices	6	5	2	6
Number of focus/working groups (total number of participants)
Patient groups	6 (43)	5 (21)	1 (5)	2 (14)
Practice staff/PCO or steering groups	11 (67)	10 (53)	12 (62)	16 (89)
Number of one‐to‐one interviews (formal and informal)
With patients	8	9	3	0
With practice staff	15	12	0	10
With PCO managers	23	12	5	5

PCO, Primary Care Organizations.

Data collection

The multimethod process was based on frameworks devised by Bate and Stringer, ²¹ , ²² which emphasize iteration between defining the issues, developing solutions and evaluation. Table 1 shows the number and nature of the contacts between the participants in each of the four PCO sites and Table 2 describes the purposes of the phases of data collection, and the data sources and approaches used in each of the phases. This study presents data from the first four phases; the findings of the final evaluative phase are available elsewhere. ²³

Table 2.

 Project phases and methods of data collection

Phase	Purpose	Data sources and approaches
1	To build relationships with the stakeholders, understand the local context, and gain information about current and previous public reporting initiatives	In‐depth interviews with PCO Board members, managers and practice staff Focus groups of patients registered with the practices Focus groups of practice staff Informal meetings with practice staff and PCO managers Participant observation of PCO and practice meetings, including patient participation/support group meetings, backed up by field notes and researcher diaries Review of relevant documentation, such as annual reports and minutes of meetings
2	To analyse the facilitators and barriers to developing the information source	Qualitative analysis of the data derived from Phase 1 to examine the strength and nature of forces acting in favour of the aims of the project, and those acting against Comparisons drawn between data derived from different stakeholders and from different localities
3	To feedback the results of Phase 2 to the participants and to devise and agree possible formats for the information source and individuals responsible for implementation	Formal presentations, seminars, written reports and informal conversations to feedback results of Phase 2 to all stakeholders
4	To implement the action plan agreed in Phase 3	Establishment of PCO‐based steering groups bringing together members of the public and practice staff from all participating practices with PCO managers Formal and informal discussions with members of the public, practice and PCO staff to negotiate responsibilities for progress, content and reporting format of the information and modes of dissemination Sharing of information about progress between sites Development of ‘prototype’ information sources Publication of the final prototype and development of the website
5	To evaluate the use and impact of the information source on the participants	Process diaries kept by research staff Surveys of public response to prototype information source Interviews and focus groups with members of the public, practice staff and PCO managers Website usage analysis and ‘talk‐through’ interviews

The participants varied in their level of participation in the project. Some of the PCO managers, practice staff and members of the public played a significant role in the conduct of the project, for example, by initiating new ideas or solutions, recruiting other participants and helping to interpret the data. Others responded more reactively to specific questions from the research team or other participants. The research team was responsible for overseeing the project, suggesting solutions, and maintaining and feeding back progress. They also encouraged and facilitated interaction and sharing of ideas between localities and practices. The role played by the researchers varied between sites and at different stages of the project. Sometimes they had to function as the main driver for the project whilst at other times they were able to largely withdraw from the process, with leadership provided by the practices, PCO managers and patient groups.

Data analysis and guide development

We analysed the field notes, interview transcripts, reflective diaries and documents using a constant comparative approach. ²⁴ We identified and explored emergent themes arising from the participants' discussions describing factors influencing the public's use of information and their information needs. Data were collected and analysed until stable ideas or themes were developed. These themes were explored and interpreted in an iterative way with the project participants and were triangulated between the different stakeholder groups and sites. The findings were then used to guide the development of an information source for patients and the public about general practice services.

Results

Five key themes were identified from the data which helped to guide the production of an information source of interest and use to patients.

Theme 1: The importance of designing information specifically for the public

Whilst the interests of the clinical staff and the managers participating in the study were primarily around information about clinical performance, it became clear at any early stage of the discussions that this was not the main area of interest for members of the public. When this finding was highlighted within the steering groups, the participants agreed that the lack of attention given to what patients want to see, as opposed to what the service or policy‐makers want to provide, might represent a significant explanation for the lack of patient engagement with current sources of information. All of the participants therefore decided to focus the development process and the final product explicitly on the needs of the public.

Theme 2: The influence of performance information on patients’ judgements and decisions

Some of the members of the public expressed scepticism about the usefulness of information about the performance of practices. They were unsure whether providing more ‘hard’ information would replace their reliance on informal sources, such as their own personal experiences and those of trusted family members and friends. As one person commented:

‘Patients do their research in the pub (patient, site 2)’.

and another expressed a view that:

‘It's my own experience of the practice which matters the most, not figures and statistics and comparisons (patient, site 1)’.

This finding led to a desire on the part of the steering groups to produce an information source which made use of, and could be integrated with, the personal experience of individuals. Different ways of achieving this were discussed. Most importantly, the participants wanted to give a high profile in the guides to patient feedback on practice services using information collected from standardized patient experience surveys. In addition, the patient participants liked the idea of personalizing the tone of the guides and using informal qualitative descriptions of practices alongside more conventional information about performance. As patient commented:

‘It will be set in the right tone … I mean (using the word) ‘we’ in the first person gives a nice friendly feel about it, so you feel, oh well, they're nice folk, so maybe we'll get along (patient, site 4)’.

Theme 3: Attitudes to comparative performance information

Whilst the patients who participated in the project were clear that they wanted to know more about their general practice, they were less sure that they wanted comparative information. In particular, they disliked the idea of presenting information in the form of ‘league tables’. Three possible explanations for this finding emerged from the discussions. First, most patients felt neither qualified nor did they regard it as their responsibility to monitor or make judgements about what they saw as being the doctor's job. Secondly, many of the participants expressed scepticism about the reliability and validity of quantitative performance data.

‘How much information that is fed to us in bar form or any other statistical form, how much do we believe in them (patient, site 3)’?

Thirdly, patients expressed significant concerns about the possible negative impact of publishing practice performance data on the practices themselves. In part this reflected a practical concern about the extra work entailed in producing the guides.

‘What worried me when I first heard about this was the doctors have enough forms and things to fill in; will this entail my doctor to do more? Because I feel sorry for them, they have that much to do, don't they (patient, site 2)’?

In addition, they were concerned that ranking performance would lead to competition between practices and they regarded this as neither desirable nor practical.

‘I don't think we should set up one practice against another (patient, site 4)’.

‘We're not going to sort of transfer, are we? Oh, they're better than them, so we'll go there now; your doctor isn't that kind of business is it (patient, site 1)’?

Underlying this view there appeared to be a strong sense of social solidarity with, and responsibility for, the NHS as an institution and particularly for their local practice and the clinical staff who work in it.

Theme 4: Knowing the source of information

Some patients expressed the view that the perceived source of information had an influence on its credibility. They felt that performance league tables were highly politicized and that they were used by politicians and managers to be critical of practices. At the same time, they thought that information produced solely by the practice staff might also be biased by their desire to present themselves in a good light. One patient commented:

‘Everyone has vested interests (patient, site 3)’.

To overcome this problem, the participants together came up with the idea of presenting the information in the form of different opinions or ‘voices’, in which the source of any claims was clearly identified and the reader could then make a personal judgement about its veracity. Producing different voices required co‐operation between the key stakeholders and the patients like the idea of information being produced in partnership.

Theme 5: Expectations in terms of the content of the information source

Patients expressed their information needs in terms of priorities. First, they wanted to understand how the NHS was structured, where general practices fitted into this and how the health system worked. It was clear from the discussions, which took place that many patients felt ill‐informed about how the NHS is organized. Secondly, they wanted to know more about the people providing the service – their gender, how old they were and whether they had any special qualifications or areas of interest. They also wanted more personal information, such as a photograph and whether the doctor was married or had children. Most of the participating practice staff felt uncomfortable about providing these kinds of information. Next, they wanted to know what services were available within the practice, when they were available and whether they could be accessed directly or via a general practitioner.

Patients were least interested in clinical performance data. Practice‐level comparative performance information appeared to only be of interest to the minority of participants who had already decided to change their practice because they had moved house or because they were highly dissatisfied with their current care. A marginally larger proportion of participants were happy to see how their preferred practice (usually chosen on the basis of convenience) compared with a national or local mean level of performance, rather than to specified individual practices. However, most people were more interested in whether their practice was committed to improving what they did, rather than in their performance relative to other practices.

How the findings were used

These findings were used to create a Guide to General Practice, produced in paper copy for each practice individually, and in a common web‐based format for all participating practices (http://www.yourGPguide.org.uk). The development of the guide was an iterative process, testing out several versions with the participants before a final prototype was agreed upon.

The prototype was based on the above findings. First, we developed it clearly with the needs of the public in mind, rather than those of clinicians or managers. Secondly, it was designed in a way, which attempted to built on the informal sources of information used by patients, rather than attempting to replace them. Thirdly, we did not rank practices using a single aggregate performance measure, though we designed the website in a way which allowed limited comparisons in specific areas to be made by those who wanted them. Fourthly, we made sure that the source of any information given was clearly labelled.

Finally, we presented the kinds of information that patients said they wanted and we drew on the cognitive science and social marketing literatures to guide the presentation format of the publication. For example, information was presented in a hierarchical fashion, starting with background contextual information, then information about the services on offer and the people providing them, and then more limited data about performance and outcomes. Knowing that information is usually scanned by readers, rather than read in detail, the text was broken up using subheadings, frequently asked questions and bullet points. The overall volume of information, for the non‐web‐based version, was kept to a minimum. Pictures and simple graphics were used but complicated tables of numerical information were not.

Discussion

This study suggests that it is necessary to think and act differently in order to maximize the potential of information as a catalyst for public engagement with health system performance. It seems that the public have more modest demands for information about general practice services, the part of the health service that they use most frequently, than some policy‐makers might think (or clinicians might fear). They have different information needs from managers, clinicians and regulators and are most likely to simply ignore routine performance data that are put into the public domain. They are not clamouring for lots of complex performance or outcome data. They do not want to be treated like, or behave in, a traditional consumerist way which expects them to exercise power over service providers, and expressing choice of primary care provider does not seem to be a high priority.

This does not mean that the public want to be passive or ignorant about the quality of general practice services available to them. On the contrary, they are clear what they want in terms of the provision of information and the need for service improvement. To date their needs have been largely ignored. To our knowledge this study represents the first attempt to systematically respond to these information needs in the primary care sector. First, we have confirmed that patients want to be able to integrate hard performance data with softer sources of information such as their individual beliefs and personal experiences. ²⁵ Secondly, at the current time the majority of people want contextual and structural information, as well as information relating to processes of care that they have experienced; the public is not asking for practices to be ranked in performance league tables and complex outcomes data are more likely to turn people off than to engage them. Thirdly, the public want to feel they can trust the source of the information and to use the development of information sources as an opportunity to exercise their sense of responsibility and social solidarity with their local services. Finally, they want information, which adheres to the basic principles of cognitive science in terms of its structure, content and presentation format.

The traditional approach to publishing information is in the ‘information telling’ mode. ²⁶ This reflects the rational model of decision‐making – that people balance the pros and cons and then make their decisions in apparently rational and ego‐centric ways. In contrast, the social processes model sees decision‐making as a complex, iterative social process in which ‘soft information’ may be more important than ‘hard data’. ²⁵ Performance information may therefore be used as one small part of a ‘knowledge construction’ process. ²⁷ The model underlying making judgements about performance and decisions about choice of general practice appears to be more aligned to the social processes model than the rational decision‐making model.

This study suggests two limitations to a consumerist model of general practice and should temper the commonly held conceptualization of choice as requiring a market basis in the UK primary care sector. Comparative performance information intended to assist the choice of practice may only be relevant to the small proportion of the population who have moved into a new area, or are highly dissatisfied with their current practice. For the majority of people, information about service availability seems to be of greater interest than information about service quality. Secondly, consumerism overlooks the personal, even emotional, ties that many patients have to a particular doctor or practice. This study suggests that information may be a more useful catalyst for public engagement in the form of ‘voice’ (i.e. getting opinions heard) within a chosen practice, rather than ‘exit’ (i.e. re‐registration) from one practice to another. ²⁸

There were several benefits to adopting an action research‐type approach in this study. The participative ethos encouraged a consensual development of grounded solutions to practical problems – and solutions developed by those who use the service appear to be different from those currently being advanced by policy‐makers and those working in the service. Formative evaluation of the progress being made in each site and the rapid feedback into the process ensured that evidence and experience swiftly influenced practice. However, there are a number of limitations. First, there are questions about the reproducibility of the process in other sites, particularly in the absence of the research team. In line with the principles of qualitative methodologies, the transferability of the results can be maximized by detailed description of the context and process ²⁴ and we have attempted to do this in the final project report. ²³ Secondly, this study has only described the development of a new source of information and not its utility or impact. The evaluation of the product represents the second phase of this study and is described elsewhere. ²³ Thirdly, the resources available dictated that we focus on generic issues, rather than on the information needs of specific groups. We know from other studies that information is more likely to be used if it is adapted to the needs of the audience, such as ethnic minorities, people with specific health needs, or those with special educational needs. ²⁹ , ³⁰

The principle that information can be a useful catalyst for public engagement and that the public have a right to access this information is slowly becoming accepted within the health service. This study, which moves beyond the provider‐led and professionally constructed approaches to informing patients, provides empirical evidence and practical guidance to front line clinicians and managers about how to publish information which has the best chance of engaging and being useful to the public.

Author contributions

MM had the original idea for the study and all authors developed the proposal. JN and HD led the fieldwork. All authors and the study participants contributed to the analysis and interpretation of the data. MM wrote the first draft of the paper and all authors contributed to subsequent drafts. MM acts as guarantor.