The use of evidence by health care user organizations

Abstract

Objectives To explore the use of research evidence by consumer and patient organizations and the extent to which their goals and activities are consistent with evidence‐based health care and patient‐centred care.

DesignA mailed survey, telephone and face‐to‐face interviews of leaders of organizations representing health care users.

SettingNorway.

Participants Sixty‐nine of 109 questionnaires that were mailed were included in our analysis and approximately 20 interviews were conducted with representatives of general consumer and patient advocacy groups and interest groups that focus on particular diseases or disabilities.

MeasurementsInformation was collected on the goals of the organizations, the nature of their everyday work, the extent to which research information is required in this work and how research information is accessed and appraised.

ResultsAn important focus of many user groups is peer support. They tend to emphasize experience‐based knowledge. A total of 82% of the respondents said that they often or sometimes had use for research results in their work. Research‐based information is most often obtained indirectly through physicians or researchers.

ConclusionsNorwegian health care user organizations do not appear to promote evidence‐based health care. To the extent that they help to disseminate scientific information, they appear to do so uncritically, relying on few sources and traditional authorities.

Introduction

The concepts of evidence‐based medicine and patient‐centred care have both become popular in recent years. While evidence based medicine emphasizes the importance of basing health care decisions on valid evidence rather than authoritarian opinion, patient centred care focuses on the patient's views and values and draws attention to the potential for patients to become active participants in decisions about their health rather than simply passive recipients of authoritarian advice. ¹ Health care users are likely to benefit if decision‐making is improved by the use of both valid evidence and better information about patients values and preferences. ²

While both providers and recipients of health care require access to valid information about their options, the recipients, in particular, often find it difficult to get this information. Organizations representing health care users are possible sources of assistance. Norwegian user organizations are varied and offer a number of services. It has been observed that these organizations tend to be built up around the use of experience‐based knowledge, ³ and that the collective experience of members is often their main asset. The role that scientific knowledge plays in these organizations is not clear. In this paper we examine the use of scientific information by user organizations in Norway and discuss the extent to which these organizations might offer assistance to individuals who wish to access and critically assess research‐based information about health care effectiveness.

Methods

We conducted a mailed survey and held telephone and face‐to‐face interviews with representatives of organizations representing health care users in Norway. The term `user' can be defined as `those who use, are affected by, and who are entitled or compelled to use (health) services'. ^{2 , 4} For this study we included any organization that represents health care users. We searched for such organizations in the membership lists of the Norwegian Organization for the Disabled, the Norwegian League of Handicap Organizations, and the Norwegian Association for Rare Diseases all of which are umbrella organizations. We also searched for organizations with the use of telephone catalogues and a list of self‐help organizations that receive funding from the Ministry of Health and Social Services. Finally, a representative from the government‐appointed Consumer Council as well as government‐appointed patient ombudsmen from two counties were included in the survey.

A total of 109 questionnaires were mailed. The questionnaires asked for information about the overall goals of the organization, the nature of their everyday work, and the extent to which research information was required in this work. Respondents were also asked what type of information they regarded as best suited for decision‐making, from which sources they obtained health care information, and whether increased access to different types of medical information would be of interest. Respondents were also asked to send in brochures describing their organizations. One of us (CG) attended the annual meeting of FOSS, the umbrella organization for 20 small interest groups that focus on rare diseases. Informal conversations were held with representatives of roughly half of the organizations present at this meeting, and face to face interviews and telephone interviews with another 10 organizations. These interviews allowed responses to the questionnaire to be better interpreted and the issues touched upon to be considered in a larger context.

Results

Of the 109 questionnaires that were mailed, six were returned unopened, while 71 were filled out and returned. Two of these were excluded because they did not state the name of their organization, leaving a useable response rate of 63%. A list of respondents and non‐respondents is available from the authors.

The organizations included in the survey can be roughly divided into three categories, although several of the organizations had mixed characteristics making them difficult to categorize. The first category is made up of general consumer organizations and patient advocacy groups. These include the Consumer Council, the county councils' patient ombudsmen (both of which are government funded), and the Norwegian Patient Association (a private organization that receives some government support). The Consumer Council represents consumers in general, and the latter two represent health care consumers.

The second category is made up of small interest groups that focus on particular diseases or disabilities. Often `kitchen‐table' organizations run by volunteers, they rarely have more than a few hundred members, and the smallest one in our sample had only 12. In most of these organizations, membership is only open to those with the diagnosis in question and their families. In some organizations these rules of eligibility also apply to leadership positions.

The third category of organizations is made up of the large interest groups that focus on particular types of disease or disability. These include the National Association for Heart and Lung Disease, with 52 000 members, and a handful of other organizations with several thousand members. Many of these organizations are run by professionals and while members are predominantly those with the ailment, some of these organizations also offer membership to others, including health professionals.

None of the organizations in the first category responded to our mailed questionnaire and one ombudsman declined to discuss the issues we raised over the telephone, stating that it was not relevant to her work. Telephone interviews were carried out with one other ombudsman and a representative of the Norwegian Patient Association. A face‐to‐face interview was conducted with a representative of the Consumer Council. Brochures were received from all three organizations.

Goals and activities

The general consumer organizations and patient advocacy groups acknowledge access to information as a patient's right, but the dissemination of health care information is not their primary concern. Their main activity involves responding to applications for help about legal issues and instances of malpractice, and they are staffed by people with legal and health care backgrounds. The Consumer Council has on occasion lobbied the government about consumer interests in health care legislation, but this is not currently a priority area for them.

For the organizations that focus on particular diseases or disabilities, the quality of professional health care is an area of interest, but not a major concern. Despite the wide range of diseases and disabilities covered and the differences in their size, these organizations share many of the same goals and activities, although with varying emphasis.

Peer support

An important activity for many user organizations, especially the smaller groups, is establishing communication between people with similar health problems so that they can benefit from each other's experiences. Many organizations arrange regular meetings for this purpose.

Provision of information

Apart from the information spread through peer support, information is also provided through telephone inquiries, newsletters, magazines and seminars. Many respondents provided information about the medical, practical and social implications of the disease or disability to health workers and the general public as well as to their members.

Advocacy and lobbying

Many organizations spend a great deal of time and energy in disputes and discussions about welfare benefits with local authorities. The larger organizations especially also lobby the national government around issues such as the official recognition of sign language or prisoners' rights to psychiatric treatment.

Treatment/service provision

Many of the smaller organizations work with government‐funded treatment facilities, while some of the larger organizations run their own treatment facilities and schools.

Research funding

At least five of the larger organizations in our sample fund research.

The need for and use of scientific information

Twenty‐seven (39%) of the respondents reported that they often had use for research results in their work. Thirty (43%) said that they had use for research results sometimes. Five (7%) seldom or never had use for research results in their work and another six did not answer this question.

When asked what type of information would be most suited for forming an opinion about important issues, almost two‐thirds of the respondents replied that information from researchers and research reports was very important ( Table 1), and roughly half replied that information from doctors and from systematic reviews was very important. Popular science was regarded as the least important type of information. Despite the emphasis placed upon the importance of scientific research, many potential sources of research based information are left unused by user organizations. Very few respondents made use of libraries, either medical or otherwise, and almost all never or seldom used MEDLINE or EMBASE or had never heard of these sources ( Table 2). Information from doctors and researchers was used far more commonly. Twenty‐four (34%) of the organizations obtained information from researchers often while 27 (39%) received information from this source sometimes. Forty (59%) organizations obtained information from doctors often and 18 (26%) sometimes. Information obtained from brochures and interviews also indicated that several organizations include doctors or researchers on their advisory boards or otherwise rely on established contacts with specific doctors or researchers.

Table 1.

What type of information would be most useful when forming an opinion about important matters?

Table 2.

How do you get information about the medical or health care issues you are interested in?

Thirty (43%) of the user organizations sometimes made use of their own surveys to collect information and another 12 (17%) often made use of this source of information. Journal articles were also popular. Thirty‐eight (55%) of the responding organizations often gained information from this source and another 21 (30%) sometimes did.

When asked if more contact with relevant medical speciality centres would be of interest, 59 (85%) of the respondents answered that they would be interested in written information, 58 (84%) in lists of experts that could be contacted, and 54 (78%) in lectures or teaching ( Table 3). Thirty‐six (52%) indicated an interest in advice as to how to critically appraise health information.

Table 3.

Would further contact with relevant medical speciality centres be of interest?

Some organizations may obtain scientific information from sources not included in our questionnaire. For example, some receive information from sister organizations abroad and some obtain information from the Internet. At the national meeting of interest groups for rare diseases, several delegates mentioned that they had more or less coincidentally found information about their diagnoses on the World Wide Web. One respondent, who stumbled upon MEDLINE during a long hospital stay, noted: `I had no idea how easy it was to get hold of all of this information!'. The idea of searching for information in this way, however, still appears to be very new to most user organizations.

Discussion

How is scientific information assessed?

How critical are the user organizations of the scientific information they do seek and receive? When research results are used, the organizations appear to rely on secondary sources of information, particularly expert opinions. Over half of the respondents receive information from doctors and researchers often, while little use is made of other sources. Actual research reports are seldom read, despite the fact that over 60% of the respondents see them as very important sources of information. And while 55% of respondents stated that they often obtained information from journal articles, the interviews suggest that many of them were referring to any journal articles that shed light on their situation (including commentaries, letters and articles from non‐scientific journals), and not just research‐based articles.

The organizations apparently prefer to rely on experts to access, critically appraise and interpret research evidence. One organization we expected to differ in this respect was the national HIV organization (PLUSS) which receives frequent inquiries from members about reports of new cures for AIDS. The organization's usual response to the latest wonder drug is only to establish if and when Norwegian doctors will adopt it. They do not attempt to assess the drug's effectiveness. PLUSS suggested that they respond in this way because they trust the doctors treating them. Our interviews suggest that this trust is shared by other organizations.

In general, the organizations appear to have very little confidence in their own abilities to appraise medical information, a common refrain being: `We don't want to play at being experts'. In many ways, this attitude and approach follows traditional patterns of patient‐doctor relationships in that it appears to be based on trust. ⁵ A trust in professionals and distrust of their own abilities is perhaps further illustrated by the fact that only half of the respondents were interested in learning more about how to critically appraise scientific information, whereas over 80% were interested in written information, lists of experts and lectures. The trust in professionals could be a consequence of the fact that Norway has for decades had a well‐functioning publicly funded health service. Private health care is still a novel alternative for most Norwegians, who are unused to choosing the type of treatment or the provider they are exposed to. However, more choices are available to Norwegian patients than in the past, both within and outside of the publicly funded system and there are signs that patients are beginning to question their doctors' decisions. ⁶

One possible sign of change is the increase in the use of the Internet, a source of information where medical professionals do not serve as gate‐keepers. This source does have some problems, however. The Internet `allows anyone to become a publisher' and `has no mechanisms to refute scientific information'. ^{7 , 8} Since our survey was concluded, FFO, the umbrella organization to which 36 (52%) of our respondents belong, has established it's own home page on the Internet. FFO sees the Internet as another opportunity to share experiences, but also as a source of medical information. The quality of this medical information, however, does not as yet appear to have become an issue among user organizations.

How is research evidence put to use?

The ways in which user organizations seek and use research evidence is influenced by several factors. Many organizations appear to rely on a very small number of experts for information and advice, which potentially limits their access to information as well as reducing their inclination to assess research results critically. It seems that experts and advisory board members are often chosen because they share the views of the organization, particularly if the organizations focus on ailments for which medical consensus is lacking. For some of the larger organizations, potential conflicts of interest arise because they aim to represent both consumer and professional interests and some are themselves health care providers.

User organizations' assessments of research results are influenced by their goals. Some respondents stated quite clearly that if a research report could be used to support their political goals they would use it to do so. Research that shows a high frequency of an illness, for instance, is a powerful tool for organizations when lobbying for more funds. Similarly, research that acknowledges the existence or severity of ailments unrecognized by welfare authorities or doctors is also likely to be welcomed and used uncritically. The implications of research findings for members may also be considered. For the Norwegian Whiplash Association, for instance, the contents of research results could seriously affect members' insurance claims.

Another goal that might influence the use of research, but one that is not mentioned explicitly by the organizations, is the creation or protection of the group's social identity. ⁹ Some diseases and disabilities are more socially acceptable than others, and it is to be expected that patient organizations welcome research that sheds the most positive light on their ailment. For example, the Tourette Syndrome Association emphasizes that `Tourette Syndrome is an organic condition. It is not caused by psychological problems'. An underlying assumption here might be that `mental illness' is less socially acceptable than `physical illness'. For those diseases that are generally acknowledged as having psychological causes, research that points to a high frequency of such diseases may serve to normalize these and make them more socially acceptable.

A debate concerning the use of ear implants in deaf children also illustrates the connection between medical treatment and the social status of a group. The Norwegian Association for the Deaf wish their members to be acknowledged as a language minority rather than a handicapped group. The organization is critical of the use of ear implants, arguing that children with implants may loose their social identity as deaf without gaining full membership among the hearing.

Medical research may have a number of legal, financial, or social implications that in certain contexts are as important as the medical implications it intends to describe. With this in mind, it is reasonable to conclude that many organizations are biased both in the type of information they seek and the ways in which they appraise and use the information they receive.

Evidence and experience

The questionnaire asked what type of information would be best suited for forming an opinion about important issues, but neglected to acknowledge that different types of information are needed for different purposes. The type of information most commonly provided by user organizations has been described as experience‐based knowledge, as opposed to knowledge based on scientific evidence or deduction, ³ and also in contrast to folk or lay information, ¹⁰ , i.e. knowledge held by lay people with no first‐hand experience of the diagnosis in question. Experience‐based knowledge capitalises on practical experience and takes account of the social context in which people exist. It can be seen as one of the prime assets of many user organizations and provides the basis for many of their activities, particularly that of peer support.

While experience‐based knowledge meets the information needs of many important activities, it relies largely on anecdotal experience and data that are collected and analysed non‐systematically. It is therefore much more open to bias, spurious conclusions and errors of inference than evidence from well‐designed research. Experience based knowledge is arguably not sufficient for people participating in decisions about their medical care. For these types of decisions, valid information about the effects of different alternatives is necessary.

The apparent neglect of dissemination of valid scientific information by user organizations may be due to a number of factors. One obvious factor is the organizations' lack of resources. The smaller organizations, at least, neither have the time nor the knowledge to make use of sources of information such as MEDLINE, EMBASE and medical libraries. Many of them feel obliged to spend their time working on matters perceived to be of more immediate need, such as supporting members' rights to welfare benefits. This does not imply that practical or financial solutions alone will automatically lead to an increase in the use of scientific information. An emphasis upon experience‐based knowledge can also be seen as a conscious decision to represent an alternative to traditional medical knowledge. This emphasis is also encouraged by the Norwegian government, which has, in recent years, actively supported peer‐support groups. In 1996 one million US dollars were handed out to user organizations in Norway for peer‐support projects. The apparent lack of confidence that many organizations have in their ability to use scientific knowledge also serves to encourage such an emphasis. While many individuals are intimidated by scientific terms and phrases, and see science as belonging to scientists, experience‐based knowledge is `their own', and the sharing of personal experiences may offer sufferers `a sense of worth, a sense of meaningfulness and purpose'. ⁹

Potential roles for user organizations

Disseminating quality information

It could be argued that improving access to and use of reliable information by health care users is the responsibility of health care professionals, and possibly the mass media and the government. All of these have interests that influence the type and quality of information they provide, and that may or may not coincide with the interests of users.

There is no shortage of patient information and information about health care in the mass media. However, the scientific quality of the material that is available is often lacking. ¹¹ , 12, ^–13

User organizations already provide important practical and emotional support to their members. For user organizations interested in ensuring the effectiveness of health care and in supporting shared decision‐making, the ability to provide understandable and reliable scientific information about the effects of medical treatment would represent an important step towards this goal.

It could be argued, however, that user organizations may not be the most appropriate vehicles for promoting the critical use of research evidence because of an inherent bias towards promoting their own interests as an organization. In addition, for the small organizations, adoption of such activities could lead to a neglect of other important areas of work. For the larger organizations and umbrella organizations, however, activities to promote critical use of research evidence are not as unrealistic. In other countries, user organizations have attempted to increase user access to scientific information, for example, through the establishment of telephone information services, ^{14 , 15} although few of these services assess this information in terms of scientific quality and applicability for the user in question.

Influencing research

While the flow of high quality information from research environments to health care users is restricted by a number of circumstances, perhaps even less information travels in the other direction. Researchers rarely appear to incorporate the experiences of user groups into their research agendas in any systematic way and few user groups attempt to influence these agendas. One exception is the Norwegian Diabetes Association which is closely involved in the research programmes it funds. In the United States, the National Breast Cancer Coalition (NBCC), having successfully campaigned for an increase in the amount of money spent on breast cancer research, also demanded a say in how this money is spent. ¹⁶ To implement this goal, NBCC members are trained in the language and concepts of science, and then ask for membership on committees dealing with breast cancer research. In the UK, an NHS consumer information services' guide to the effectiveness of treatments for depression was assembled, evaluated and improved through cooperation with relevant self‐help and advocacy groups and particular attention was paid to the quality and accessibility of the information. ¹⁷ The Cochrane Collaboration, responsible for the Cochrane Database of Systematic Reviews, also attempts to include user representatives in its review groups, and the Cochrane Consumer Network works towards ensuring that Cochrane reviews of the effects of health care are accessible and understandable to users as well as health professionals.

A number of possibilities exist for those wishing to break down the barriers between research environments and health care users. User organizations could become more involved in the dissemination of research results and in making the implications of these results more understandable to lay individuals. In addition, they could play a more active role in the research process and involve themselves in decisions about the type of research that is carried out. It is not clear whether user organizations themselves wish to participate in these areas to a greater extent than is the case today.

Conclusion

Thirty years ago, Fuchs said of the American health system that `very few industries could be named where the consumer is so dependent upon the producer for information concerning the quality of the product'. ¹⁸ Health care users are still largely dependent on medical authorities and health care providers. When authorities disagree, as they often do, users are left with little basis for deciding what to believe.

This survey suggests Norwegian user groups do not see this as a problem that they should be tackling, and there are important limitations to their ability to promote critical use of research evidence as an alternative to relying on experts. Their most important work may be the portrayal of their members as `whole' individuals and their emphasis upon the non‐medical aspects of their lives. On the other hand, most members must relate to health care information in one way or another, and users should have access to information about the effects of health care and should be able to assess the validity and applicability of the information they receive.

The critical use of information can sometimes make decision making complex and difficult. However, the alternative is uninformed or misinformed decisions, which in the long run are far less likely to result in effective, efficient and equitable health services. Patient involvement in evidence‐based decision making would seem to be a much more desirable option than authoritarian edicts or uninformed decisions.

Improving use of scientific information will require the efforts of several groups including patients, health care professionals, purchasers, policy makers and educators. A number of circumstances, both practical and ideological, may limit the extent to which user organizations can support the critical user of research evidence. Providing that these circumstances are taken into consideration, however, user organizations could potentially play an important role in the promotion of evidence based health care and patient centred care.

Acknowledgements

We would like to thank Liv Welde Johansen for the original design of the survey. We would also like to thank Hilda Bastian, Iain Chalmers, Kay Dickersin, Vikki Entwistle, Jean Jones, Alessandro Liberati and Sandy Oliver for helpful comments on earlier versions of this manuscript, and apologize to them if we still have not got it right. Finally, we would like to thank the organizations included in our survey.