Methods for incorporating patients' views in health care

Efforts to improve health care will be wasted unless they reflect what patients want from the service. But to be sure that surveys of patients' views are valid and have an effect on care, the methods used must be evaluated rigorously

Society now acknowledges the importance of the views of users in developing services, and the healthcare sector has used a range of methods to identify the views of patients and the public. Examples are questionnaires to assess patients' needs before a consultation with the clinician, shared decision making, focus groups with patients to include their views in clinical guidelines, and surveys among patients to provide feedback to care providers or the public. Such methods need to be examined in terms of validity, effectiveness, and implementation.¹ We describe some of the important issues related to measuring patients' views and evaluating their use in improving health care.

Summary points

• Patients can contribute to debates on health care by giving their preferences for care, evaluations of what occurred, or factual reports of care

• Measures of patients' views should be assessed for validity, preferably by rigorous qualitative studies

• Methods to include patients' views must be shown to affect the processes and outcomes of health care; possible negative consequences should also be considered

Types of measures

The methods used to determine patients' views can be divided into three types: measures of preferences, evaluations by users, and reports of health care (box). The types of measure used will depend on what aspect of health care is being assessed, but all have limitations.

Preferences

One problem with assessing preferences is that patients' decisions about what is important in health care often reflect their individual experience rather than a general view. Interaction between patients in focus groups can help overcome this.

Another issue with measures of preference is deciding what options are presented. Qualitative research methods, such as individual interviews and focus groups, use open ended approaches such as topic lists rather than structured questionnaires. These give the greatest scope for expressing different preferences. Quantitative methods for eliciting preferences include surveys and consensus methods, such as Delphi and nominal group techniques. These techniques ask individuals to rate, rank, or vote for different types of care (such as, general practice or hospital) or attributes of care providers (short waiting list, adequate information). It is unclear whether the different methods of rating produce comparable results.⁶,⁷

Definitions of preferences, evaluations, and reports

Preferences are ideas about what should occur in healthcare systems.² Preference is often used to refer to individual patients' views about their clinical treatment, and the term priorities is used to describe the preferences of a population³

Evaluations are patients reactions to their experience of health care—for example, whether the process or outcome of their care was good or bad⁴

Reports represent objective observations of organisation or process of care by patients, regardless of their preferences or evaluations.⁵ Patients can, for instance, register how long they had to wait in the waiting room, irrespective of whether this was too long

Several models have been developed to collect and analyse preference data, including the expectancy-value model, multi-attribute utility models, and conjoint analysis models (discrete choice experiments).⁸,⁹ The choice of methods will influence the results.³ Patients should contribute to the development of preference frameworks.

Decisions on prioritisation in healthcare systems inevitably involve a wide array of factors, and instruments have to be able to incorporate this multidimensionality. The most realistic methods present constrained choices, in which trade-offs have to be made between different attributes or choices.

Evaluations

Questionnaires that ask for evaluations of health care in terms of satisfaction or dissatisfaction show less discrimination than questionnaires that use terms such as good and bad or agree and disagree with concrete aspects of care.¹⁰ Some questionnaires measure preferences and experiences and derive evaluations from the two by calculating difference or ratio scores.¹¹ There is some evidence that patients distinguish between preferences and experiences,¹² but there is no validated framework for deriving evaluations from preferences and experiences.¹³

Patients are not always satisfied with their experiences of health care,¹⁴ and qualitative methods can be used to examine their experiences in more depth. Qualitative approaches are particularly useful for exploring patients' views in areas that have not been previously studied.¹⁵ Pragmatic approaches to qualitative analysis, such as logging key themes without full transcription analyses, have been used, but the reliability and validity of such approaches has not been assessed.

Reports

Although reports reflect patients' observations, they do not necessarily imply a patients' perspective on the quality of care. In some situations, patients' reports are the most accurate method of observation—for example, if data are required about a patient's pathway through different healthcare institutions.

Quality of measurement instruments

Assessments of patients' views cannot be considered representative unless the measure has been properly evaluated. Various aspects need to be considered.

SUE SHARPLES

Validity

In a review of patient satisfaction studies, only 46% reported some validity or reliability data.¹⁶ Ideally, the instrument used should be compared with a criterion measure—that is, a measure with established validity. For instance, patients' reports of their care can be compared with the medical records or clinicians' reports of the care delivered.¹⁵

Criterion measures are often not available, however, so other approaches are needed. In this case, the validity of instruments should be based on a conceptual framework that describes a specific domain (the relevant aspects of health care). Ideally, patients should be consulted about the selection and description of the included aspects. Qualitative studies are particularly suitable for this purpose. Europep, an international instrument for obtaining patients' evaluations of general practice care, was based on systematic literature studies and qualitative and quantitative studies of patients' priorities.¹⁷ Sometimes it is also possible to verify that patients' views are associated with other factors as predicted by theory. This is known as construct validity.

Psychometrics

Quantitative instruments should have adequate psychometric features.¹⁸ High response rates to an item usually indicate that the question is relevant and understandable. However, this does not apply to instruments that examine rare events, such as medical errors (complaint procedures) or side effects of drugs (surveys among people taking the drug).

Instruments designed to measure aspects of quality should also show good variation across patients (discrimination) and variation between measurements at different points in time (responsiveness). If several indicators are supposed to assess one dimension of care, validity is supported by high internal consistency in the responses to indicators in that dimension. Ideally, instruments will also have good test-retest reliability.

The most often used reliability coefficients (such as α) refer to the internal consistency of items within a dimension per patient. In the context of quality improvement, however, aggregated scores per care provider are often important. These figures are based on several indicators and a number of patients or events. Generalisability analyses can be used to calculate reliability coefficients for the aggregated scores.¹⁹

Sampling

Non-responders are more likely to be ill, less satisfied with care provided, and less frequent users of health care than responders, although this isn't always the case.²⁰,²¹ Surveys or interview methods need to consider the effects of such patients being excluded or dropping out. Response rates for surveys of patients vary considerably. A literature review reported a mean response rate of 60% and a standard deviation of 21%.²² Many factors can influence the response rate of a survey. These include:

• Motivation of the clinician to recruit patients

• Attractiveness of the layout of a questionnaire

• Method of administering the questionnaire to patients

• Use of monetary incentives

• Use of information technology for administering questionnaires.²³

Effectiveness

Methods to identify and use patient views to improve health care need to be shown to be effective. The best way to show this is by randomised trials. It is important that the outcomes chosen are relevant. We suggest that outcome measures for the evaluation are derived from the underlying objectives of the quality improvement exercise.

Ethical and legal perspective

—It is an ethical and legal rule that patients should be informed and involved in their health care, at least to minimal standards. Many patients wish to take part in the decision processes.²⁴ When the aim is to include patients in decision making, it is the process of involvement rather than its outcome that is crucial. The criteria of effectiveness are therefore defined by the ethical principles and patients' preferences. For instance, shared decision making can be evaluated in terms of information delivered on treatment options, checking of understanding and preferences, and making a shared decision.²⁵

Quality of care

—Patient involvement can also result in better processes and outcomes of care. It could, for instance, make clinicians more responsive to patients' preferences, contribute to better implementation of clinical guidelines, improve safety by engaging patients in redesigning processes, and result in better satisfaction with care. Patients can be seen as co-producers of health care, because their decisions and behaviour influence healthcare provision and its outcomes. Outcome measures should reflect the effects on process or outcomes of care that are expected.

Strategic perspective

—Integration of patients' views may be driven by political and strategic motivations, such as protecting a company's a position in a competitive healthcare market, the wish to have democratic control in the healthcare organisation, or the perceived need to do something for underserved populations. Such aims can be difficult to assess, but measurable outcome measures can be found in some cases. For instance, position in the healthcare market can be evaluated in terms of attendance rates and turnover of patients.

Finally, evaluations should consider possible unintended consequences. These include unrealistic patient expectations of what health care can deliver; defensive behaviour of care providers, resulting in higher numbers of unnecessary clinical procedures; undermining of professional moral; and increased costs. Such consequences are not imaginary. Conflict between public health policy and the rights of individuals to exercise choice are examples of irresolvable dilemmas. One recent example is the refusal of parents to have their children immunised with the MMR vaccine.

Implementation

As well as studying the effects of specific methods, it is important to know whether they are actually used in health care. Clinicians, patients, and the public may lack the skills to use specific instruments or have negative attitudes about specific approaches. Incentives built into employment frameworks can lead to important shifts in attitudes— for example, the new general practice contract in the United Kingdom mentions the evaluation of “patient experience” as one of three areas for measuring quality.²⁶ Such strategies should be evaluated in terms of uptake of the instruments.

Conclusions

Increased participation of patients and the public in health care is desirable. Considering patients' views can improve processes and outcomes as well as satisfaction. However, many of the methods used have not been shown to be valid or effective. The evaluation of specific methods to obtain the views of patients therefore requires urgent action.

This is the third of three articles on research into improving the quality of health care