Involving consumers in designing, conducting, and interpreting randomised controlled trials: questionnaire survey

Abstract

Objective

To assess the extent to which consumers are involved in the work of clinical trial coordinating centres in the United Kingdom and the nature of consumers' involvement in randomised trials coordinated by these centres.

Design

National surveys using structured questionnaires with some open ended sections.

Setting

103 clinical trial coordinating centres in the United Kingdom identified through a database assembled in 1997 by the NHS clinical trials adviser.

Participants

Named contacts at 62 coordinating centres and investigators in 60 trials that were identified as involving consumers.

Main outcome measures

Number of coordinating centres and number of trials in which consumers were involved and the nature of consumers' involvement.

Results

Of the 62 eligible centres, 23 reported that consumers had already been involved in their work, and most respondents were positive about this involvement. 17 centres planned to involve consumers. 15 centres had no plans to involve consumers, but only four of these considered such involvement irrelevant. Responses from investigators about the 48 individual trials were mostly positive, with respondents commenting that input from consumers had helped refine research questions, improve the quality of patient information, and make the trial more relevant to the needs of patients.

Conclusions

Consumer involvement in the design and conduct of controlled trials seems to be growing and seems to be welcomed by most researchers. Such involvement seems likely to improve the relevance to consumers of the questions addressed and the results obtained in controlled trials.

Introduction

There is substantial evidence that there are mismatches between the research that gets done and the research that patients would like to see done.^1–3 This has led some to call for greater involvement of patients in the research process.⁴,⁵ Research designed to assess the effects of treatments and randomised controlled trials in particular seem especially likely to benefit from the involvement of consumers.

Both consumers and researchers are interested in involving consumers in clinical trials, but there has been little formal advocacy of such involvement in the United Kingdom. The 1998 guidelines on clinical trials from the Medical Research Council referred to the involvement of consumers only in an appendix,⁶ and the guidelines of the Association of the British Pharmaceutical Industry made no mention of consumer involvement.⁷ Most reports of trials do not make clear whether consumers have been involved in designing or conducting the trial or in interpreting or disseminating the results, thus making it difficult to learn from the experience of others how to involve consumers.

Because there does not seem to be any publicly available information on the extent of consumer involvement in trials in the United Kingdom, we were commissioned to conduct a survey by Consumers in NHS Research (formerly the Standing Advisory Group on Consumer Involvement in the NHS Research and Development Programme), a group that advises the director of research and development for the NHS in England.

Participants and methods

Most large non-pharmaceutical trials in the United Kingdom are coordinated by specialised trial centres. We concentrated our survey on these centres, using as a sampling frame a national register of clinical trials offices and support units established by the NHS clinical trials adviser in 1997 (R Lilford, personal communication, 1998). Fieldwork began in February 1999 and ended in May 2000. The period investigated was 1990 to 1998. Centres were considered ineligible for the survey if they had coordinated fewer than two trials during this period.

We adopted the 1998 definition of consumers used by Consumers in NHS Research, which defines consumers as “patients and potential patients, carers, organisations representing consumers' interests, members of the public who are the targets of health promotion programmes and groups asking for research because they believe they have been exposed to potentially harmful circumstances, products or services.”⁸ We also adopted the group's definition of involvement, which specifies that involvement is: “the active involvement of consumers in the research process, rather than the use of consumers as the ‘subjects’ of research.”⁹

Survey

Trial coordinating centres

The survey instruments and procedures were piloted, and the person in each coordinating centre who was named on the clinical trials register was sent a letter explaining the survey's aims and explaining that the survey had been initiated by Consumers in NHS Research. A leaflet about the organisation was also enclosed. A list of the coordinating centres to which we were writing was also attached. Respondents were asked to inform us about any other centres. If these centres seemed to be eligible for the survey, they were also sent a letter.

Data were collected using structured questionnaires with some open ended sections. Identification details were requested to allow further contact from the survey office, but assurance was given that all information would be treated confidentially and no named person, centre, or trial would be identified in any published material without explicit consent. The questionnaire requested information about the involvement of consumers in trials that had been coordinated by the centre as well as any future plans for such involvement. Centres were also asked to explain why consumers had not been involved. Those centres that had involved consumers were asked to describe the nature of this involvement in general terms (that is, not in relation to specific trials). Information was also sought on the background of the consumers who had been involved and the respondent's views on the impact of that involvement.

Individual trials

A second part of the questionnaire asked for brief details of specific trials in which consumers had been involved so that the principal investigators (or their representatives) could be contacted. Using these responses, a letter and questionnaire were sent to the contact person for the individual trials. This questionnaire asked for brief details of each trial including the main source of funding and the clinical specialty involved. Other questions sought information on the nature of consumers' involvement, the background of the consumers who had been involved, and the researcher's views about the impact that the consumers had had on the trial. Trials were considered ineligible for inclusion if they had not been randomised, had not involved consumers, had not been conducted within the specified time frame, or had been coordinated by a centre that had conducted only one trial or had not existed during the period studied. Telephone calls, email messages, and personal visits were used if responses were incomplete or if there was no response to the postal questionnaires.

Data analysis

Two members of the research team (AT and DE) made decisions about eligibility separately. If there was disagreement, clarification was sought from the respondent. Four members of the research team drafted coding frames for answers to open ended questions; disagreements were resolved by consensus. Data entered into a Microsoft Access database were analysed using Stata software (Stata Corporation, College Station, TX).

Results

Survey of the centres

Letters and questionnaires were sent to 82 centres on the database, and respondents identified a further 21 centres. Of these 103 centres 10 did not reply, and 31 replied but were deemed ineligible, leaving 62 centres. Of the 62 eligible centres, 23 reported that consumers had already been involved in their work, and most respondents were positive about this involvement. Another 17 centres reported that they planned to involve consumers; 15 centres had no plans to involve consumers, but only four of these considered such involvement irrelevant (see table on the BMJ 's website).

When asked an open ended question about their perceptions of the impact of involving consumers the respondents were largely positive (box). There was, however, recognition that there were some difficulties with the process. Principal reasons given for not involving consumers were that centres did not know who to involve (8 respondents) and that it had not been considered (5 respondents). Four centres did not see the involvement of consumers as relevant.

Nineteen of the 62 centres identified eligible trials that had involved consumers. Of these, 10 reported that there had been only one eligible trial and one reported that there had been 14. The remaining eight centres reported between two and seven eligible trials.

Survey of the trials

In all, 60 trials were identified. Of the 58 questionnaires returned by investigators 48 were eligible for this part of the survey (nine trials had not involved consumers and one had not been randomised). Consumers were most commonly reported to have been involved in drafting or reviewing information for patients (32 trials). In 22 trials they promoted recruitment, and in 19 they were on the steering committee for the trial. No consumers were involved in monitoring data. (Additional information about the trials is available on the BMJ's website.)

Positive comments about consumers' input into trials

“Pivotal”

“It gave a patient view on what it was like to be involved in trials with varying designs . . . could help design trials which are patient friendly”

“Of great value at every stage . . . discussion on feasibility and acceptability . . . and of results so that the community [the group affected by the disease] understands”

 Negative comments about consumers' input

“Time consuming. Although wanting to be involved, they were extremely naive about the research process and funding problems”

Many of the responses to open ended questions about the impact of consumer involvement were positive (box). Overall, 46 respondents noted some positive impact and 9 noted some negative impact. These responses were not mutually exclusive. Respondents commented that input from consumers helped refine the research question, improved the quality of information provided to patients, and helped make trials more relevant to consumers' needs. Respondents also drew attention to the impact on the recruitment of trial participants, dissemination of information, and feeling of ownership of the trial that was enhanced by the participation of consumers.

Positive comments from investigators about involving consumers in trials

Setting the question

“They pushed hard for the trial”

“[The consumers] helped convince researchers and funders that the trial was possible and ethical”

“[They were] important in helping to refine questions”

“More relevant and clearer questions were. . .asked”

“Useful in developing patient centred outcome measures”

“[They] provided important insights into how to make the trial work”

Informing participants

“[They were] important in helping refine information”

“[They] helped make a complex trial comprehensible to most patients”

“The backing and input of the range of relevant consumer groups undoubtedly improved the quality of information given to potential participants”

“The consumers had an impact on the type of information about the trial; the leaflet was produced to fully inform patients about risks associated with their treatment”

“Consumers were able to increase their knowledge of the rationale for the trial”

 Recruiting participants

“[They] provided insights into issues [important to the] community and patients”

“[Their participation] led to improved recruitment”

“They played a pivotal role in providing ‘front line’ intelligence on how the trial was being received during its development and execution”

 Advocating for the trial

“A similar US trial was stopped prematurely and we [national steering committee that included three consumers] felt it important to continue with the trial”

 Disseminating information

“[They provided a] link to consumer networks which helped publicise the trial”

 Owning the trial

“[They brought a sense of] ownership of the concept and design of the trial to all who were involved and affected”

“[They] helped build relationships that have enabled more proactive involvement [of consumers] in a trial that followed on from a particular study”

Some comments were more negative (box). There were concerns about conflicts of interest between the needs of the trial and those of a patients' group and about the representativeness of the consumers who participated. Some respondents commented that the trial took longer and response rates were reduced as a result of input from consumers.

Negative comments from investigators

“[There is] a conflict between [the] role of a patient advocate and [the] need for reliable assessment of the cost effectiveness of expensive new drugs”

“[There need to be] clear guidelines as to the remit of a consumers' group so expectations are not disappointed”

“The problem is that there is no such thing as a ‘consumer representative’: they are individuals often with totally conflicting viewpoints (from supporting the trial to trying to stop it!). Also, their knowledge and understanding of trials varies greatly”

“At the moment [there is] no obvious impact”

“The role of consumers in this particular project was not great”

“The whole process took much longer”

“The involvement of the Community Health Council somewhat jeopardised the usefulness of the data. Their insistence that patients should not be sent a reminder letter resulted in a low response rate and poor representativeness of our sample”

Discussion

We believe this is the first national survey of consumer involvement in randomised controlled trials. Nearly two thirds of the centres that responded said that they either already involved consumers or that they intended to do so. However, the involvement of consumers is still comparatively uncommon. Respondents gave a number of reasons for not involving consumers, in particular they indicated that they had not thought about encouraging them to participate and that they had not known how to involve them. We were surprised at the lack of involvement of community health councils, since these organisations were set up to act as watchdogs for patients' interests and are active in other areas of health research.¹⁰ It could be that most community health councils act locally and generally, rather than in clinical settings or by focusing on specific health problems or interventions.

In centres or specific trials where consumers were involved, there were examples of them contributing to nearly all stages of the research process. Respondents mentioned consumers' roles in developing trial protocols, drafting information for patients, and participating on steering committees. These activities were perceived as having helped ensure that trials addressed questions relevant to consumers. The involvement of consumers was also felt to have resulted in trials being more user friendly.

Consumers sometimes became actively involved in promoting recruitment to trials, as has been noted by Edwards and colleagues.¹¹ We did not ask whether consumers' involvement actually led to increased recruitment into trials. There was at least one example in which the involvement of consumers led to reduced recruitment because the consumers felt it was unacceptable to send continued reminders to potential participants who had not responded to earlier approaches.

This study has some limitations. Our sample was restricted to trials that were associated with United Kingdom coordinating centres, and the findings from our survey of individual trials were strongly influenced by the work of one particular coordinating centre. Replication of this study later and elsewhere would allow temporal and international comparisons to be made. Although the survey identified trials that involved consumers in a number of specialties (most commonly perinatal medicine, cardiovascular medicine, HIV/AIDS, and cancer), the information on each specialty is difficult to interpret without data on the number of trials in these fields.

Because of limited resources we used a postal survey, and answers to some open ended questions often raised additional questions. We would have been able to address these in more depth if we had interviewed respondents. Our conclusions have, however, benefited from discussions after presentation of our preliminary findings at a conference on consumers' involvement in health research (Research. Who's Learning? 28 January 2000, London).

Perhaps the most important limitation of this study is that the survey was sent only to researchers and not to the consumers who had been involved with the coordinating centres or the individual trials. Thus only the perceptions of researchers have been obtained. Future studies should address this deficiency and compare the perceptions of consumers and those of researchers about the impact of consumers' involvement in trials. Certainly—in a previous study examining a variety of trials and systematic reviews in fields as diverse as subfertility, pregnancy and childbirth, and Alzheimer's disease¹²— the views of consumers about the usefulness of particular research questions and the implementation of studies' findings in practice differed markedly from those in the papers published.

It has been suggested that if consumers become more involved in randomised controlled trials they will become “knowledge-empowered”¹³ and therefore better able to challenge the beliefs and practices of researchers. Thornton, a consumer who declined an invitation to enter a particular trial,¹⁴ has become a passionate advocate of the importance of trials and believes that there is a need for professionals and the public to share responsibility at every stage of the trial process and collaborate in finding solutions to problems when they arise.¹⁵ Jadad has argued that “the clinical relevance of randomised controlled trials could be increased, easily and substantially, if researchers and funding agencies were willing to involve consumers . . . as active members of research teams.”¹⁶

Some funders are already promoting further involvement of consumers in research. For example, both the Medical Research Council and the United Kingdom Co-ordinating Committee on Cancer Research have established consumer liaison groups. The Medical Research Council's trial management guide (B Farrell, personal communication June 2000) now includes guidelines on involving consumers that were written with help from consumers. The challenge now is to conduct research to identify whether their involvement leads to actual, rather than merely perceived, benefits for research processes and output.

Our survey has shown that consumers are working in partnership with researchers to promote participation in trials. Now that an international meta-register of controlled trials has been established (www.controlled-trials.com), it should be possible to create a consumer led, electronic guide to running good controlled trials.¹⁷ This could help consumers make informed choices about whether to enter a trial. The mobilisation of consumer influence in this way might help to reorient the clinical research agenda to serve the interests of patients better.

What is already known on this topic

There is no publicly available information on the nature and extent of the involvement of consumers in randomised controlled trials in the United Kingdom

What this study adds

This is the first national survey of the involvement of consumers in these trials

More consumers are becoming involved in designing controlled trials, and this seems to be welcomed by researchers

This involvement will likely increase the relevance to consumers of the questions addressed and the results obtained in controlled trials