Cluster randomized trials of prescription medicines or prescribing policy: public and general practitioner opinions in Scotland

Abstract

AIMS

To understand public and general practitioner (GP) opinion on the acceptability of randomized policy design (RPD) studies (cluster randomized trials) of prescription medicines in Scotland.

METHODS

We surveyed public opinion on the concept of RPD studies in a sample of 1040 adults to determine acceptability and understand how people feel when changes are made to their medicines. We also surveyed GPs (n = 1034) about the concept of RPD studies as a tool for improving understanding of comparative effectiveness and safety of medicines in the ‘usual care’ setting.

RESULTS

Thirty per cent of people would be happy to receive a letter about randomized policy changes to their therapy, 31% would not mind or had no opinion and 39% would be unhappy. This view was sensitive to the reason for change; effectiveness and safety reasons were most acceptable (96%) and cost saving least acceptable (39%). Only 19% thought randomized policy change was not an acceptable method of determining the best treatments. Eighty-one per cent of respondents were willing for their medical data to be followed up to compare drug treatments (further 10% undecided). Participants reporting long-term medical conditions and those reporting previous changes to drug therapy were more in favour of RPD studies than other participants.

Thirty-three per cent (n = 341) of GPs responded to our survey. Of these, 45% were in favour of RPD studies, 19% were undecided and 36% not in favour.

CONCLUSIONS

The public in Scotland is broadly supportive of the concept of randomized policy design studies of medicines, while there is a spread of opinion among GPs.

WHAT IS ALREADY KNOWN ABOUT THIS SUBJECT

• Cluster randomized trials (randomized policy design studies) can be used to compare the safety and effectiveness of different interventions.

• Cluster randomized trials are relatively novel in the context of studying drug interventions.

• Using cluster randomized trials to investigate drug safety and effectiveness would require public and primary-care practitioner co-operation.

WHAT THIS STUDY ADDS

• The general public in Scotland is broadly supportive of the concept of cluster randomized trials (randomized policy design studies) to improve understanding of comparative effectiveness and safety of medicines within the National Health Service.

• There is a spread of opinion among general practitioners in Scotland regarding the acceptability of the concept of cluster randomized trials in the field of drug safety and effectiveness.

• Of the general public participants, 97% agreed that the National Health Service has a duty to determine the safety and effectiveness of the medicines its doctors prescribe.

Introduction

Obtaining data on the comparative effectiveness and safety of different drug therapies within the National Health Service (NHS) is essential to make correct and cost-effective decisions regarding treatment recommendations. The ideal way to gather this information is to use routinely collected data in real-life patients who are representative of the whole population of patients within the NHS. Clinical trial populations sometimes have poor external validity due to selection of subjects using inclusion and exclusion criteria and the different trial participation behaviours of different types of patients [1], while observational data are often confounded by other factors that are difficult to control for [2, 3]. One way of collecting data inexpensively and with good external validity for extrapolation to the general population is to introduce policy changes that randomize groups of patients (i.e. at practice level) within the usual NHS system to different treatments, then compare outcomes using routinely collected data; so-called ‘randomized policy design’ or ‘cluster randomized studies’[4]. The advantages of cluster randomized studies are that they take account of between-patient variability within a cluster. They also better reflect usual care interventions and thus have better external validity [5]. The downside of cluster randomization is that there is loss of statistical power because the unit of analysis must be the unit of randomization; thus, they usually need to be much larger studies. There has been an increasing interest in cluster randomized trials over the past 20 years [6]. Using this model, two established therapies can be compared, or a new therapy can be compared with an established therapy using effective record-linkage techniques. This is done by randomizing certain primary-care practices to use one therapy in their patients, while other practices continue to use the usual existing therapy in their patients. Alternatively, the same intervention can be introduced to all practices, but the implementation delayed in half of the practices [7]. Patients in all of the participating practices are then followed up for the events of interest, and the rates of events in the two therapy groups are compared. If wide participation of general practices and patients were achieved, research questions could be answered quickly and inexpensively.

While we believe that randomized policy design studies would be ideal for studying the comparative effectiveness and safety of drug therapies within the UK NHS system, we felt that it was important to gather the opinions of the general public and general practitioners (GPs) on this subject. To assess public opinion, a survey was conducted in the Scottish general population. Participants were asked questions about their views on randomized policy design studies and changes being made to their prescribed medicines. We also surveyed general practitioners in Scotland to ascertain their attitudes towards the concept of randomized policy design studies to improve understanding of comparative effectiveness and safety of medicines.

Methods

Two separate surveys were carried out in May 2010.

A general population survey was conducted by mruk research as part of a monthly Scottish Consumer Omnibus Survey. mruk research is a market research agency that conducts surveys in samples of the general population. The distribution of sample points was in line with the geographic spread of the population and was collected from 52 constituency-based sample points. Interviews were conducted using CAWI (computer-aided web interviewing) technology. One interview was conducted per household, and quotas were imposed on age and gender to reflect the population. One thousand and forty people took part in the survey. This sample size allowed proportions to be estimated within a margin of error of 3%. The structured sample in terms of gender and age distribution is presented in Table 1. In order to understand the nature of the population surveyed better, participants were also asked to report whether they had any long-term medical condition. Interviews were conducted between 10 and 20 May 2010. The questions included in the survey are listed in Table 2.

Table 1.

Population distribution by age and gender in the general population survey

Age group (years)	Male [n (%)]	Female [n (%)]	Total [n (%)]
16–24	43 (8.0)	101 (20.0)	144 (13.9)
25–34	59 (11.0)	144 (28.5)	203 (19.5)
35–44	107 (20.0)	99 (19.6)	206 (19.8)
45–54	123 (23.0)	57 (11.3)	180 (17.3)
55–64	111 (20.8)	24 (4.8)	135 (13.0)
65+	92 (17.2)	80 (15.8)	172 (16.5)
Total	535 (100.0)	505 (100.0)	1040 (100.0)

Table 2.

Questions included in the general population survey

Scenario
Imagine you are a patient with a medical condition for which you take Drug A long term.
Another drug, Drug B, is also available for this condition. It is very similar to Drug A and also works well for this medical condition.
The NHS in Scotland wants to know if there is any difference between Drug A and Drug B.
Letter
The following letter from NHS Scotland comes to you in the post, via your GP:
NHS Scotland is working to find the best treatment for a range of medical conditions. In line with this policy, your prescription for Drug A will be changed to Drug B at your next prescription renewal. Your care will otherwise remain unchanged.
Thank you for your co-operation in this matter.
Q1: What do you think would be your reaction to such a letter?
1. I would be happy
2. I would be unhappy
3. I would not mind/have no opinion on it
Q2: Irrespective of your response to Q1, how acceptable would each of the following reasons for changing your medicine be to you?
1. A change to a more effective drug	Acceptable/not acceptable/no opinion
2. A change to a safer drug	Acceptable/not acceptable/no opinion
3. A change to a cheaper drug	Acceptable/not acceptable/no opinion
4. To find out which drug works better	Acceptable/not acceptable/no opinion
5. To find out which drug is safer	Acceptable/not acceptable/no opinion
6. To compare two older drugs to find the best one	Acceptable/not acceptable/no opinion
7. To compare a new drug with an older drug to find the best one	Acceptable/not acceptable/no opinion
Q3: Do you agree with the following statement?
‘The NHS has a duty to determine the safety and effectiveness of the drugs its doctors prescribe’.
1. Agree
2. Disagree
3. No opinion
Q4: Who do you feel has the most responsibility to find out which are the best drug treatments? You may choose more than one option.
1. The NHS in general
2. My GP practice
3. My individual GP
4. Drug companies
5. A UK agency such as the Medicines and Healthcare products Regulatory Agency (MHRA)
6. A European agency such as the European Medicines Agency
Q5: One way to compare the safety and effectiveness of two similar drugs is to divide the people who take them into two groups. Each group takes one of the drugs and the results of treatment are then followed up. Would this be acceptable to you as a way of comparing drugs?
1. Acceptable
2. Not acceptable
3. No opinion
Q6: To find out the results of treatment, doctors need to follow up what happens to large numbers of individual patients. Provided adequate protection was in place, would you be willing for your medical data to be followed up by NHS Scotland so that drug treatments can be compared?
1. Yes
2. No
3. Unsure
Q7: Do you have a medical condition for which you take prescription medicines on a regular basis?
1. Yes
2. No (skip to end)
3. Declined (skip to end)
Q8: Have you ever had one of your regular medicines changed by your GP surgery or pharmacist as part of a general policy?
1. Yes
2. No
3. Unsure
Q9: Were you asked first?
1. Yes
2. No
3. Unsure
Q10: Why do you think the change was made? (Choose the most likely)
1. The new drug was better
2. The new drug was safer
3. The new drug was cheaper
4. Don't know
Q11: Which of the following statements do you most agree with?
1. I was unhappy for my drug to be changed but there were no problems afterwards
2. I was unhappy for my drug to be changed and there were problems afterwards
3. I was happy for my drug to be changed and there were no problems afterwards
4. I was happy for my drug to be changed but there were problems afterwards
5. None of these
Data to be recorded for each respondent
• Age
• Sex
• Presence of a long-term medical condition (free text entry if willing to give any details)

The general practitioner survey was conducted by writing to a 20% random sample of general practitioners within Scotland (n = 1034) in May 2010 with the single question, ‘Assuming that the clinical topics are well chosen and that patient follow-up can be automated as much as possible, what is your view on randomized drug formulary policy changes as a tool for improving drug prescribing?’

General practitioners were asked to indicate on a form whether they were ‘in favour’, ‘undecided’ or ‘not in favour’ of this concept. They were also invited to make any additional free-text comments on the concept, then to return the form to us in a reply-paid envelope.

Data are reported as numbers and percentages. Chi-squared test was used to determine the statistical difference between the groups.

Results

General population survey

One thousand and forty people were asked the questions listed in Table 2. When asked what would be their reaction to receiving a letter regarding NHS Scotland changing their prescription drug therapy at next prescription renewal in line with a policy of finding the best treatment for a range of medical conditions (Q1), 30% of people said they would be happy, 39% unhappy and 31% would not mind or had no opinion on it. There was a significant difference between women and men, with a higher proportion of women saying they would be unhappy (44% women vs. 35% men, P < 0.01).

Participants were then asked how acceptable different reasons for changing their drug therapy would be to them (irrespective of their answer to the first question; Q2). The vast majority of people (96%) found the reasons of ‘a change to a more effective drug’ or ‘a change to a safer drug’ to be acceptable. Men tended to find ‘a change to a cheaper drug’ more acceptable than women (42% men vs. 35% women, P < 0.01) and there were no significant differences in other answers between men and women (Table 3).

Table 3.

Acceptability of different reasons for changing medicine

	Total [n (%)]	Male [n (%)]	Female [n (%)]	P value
A change to a more effective drug
Acceptable	1000 (96.2)	509 (95.1)	491 (97.2)	0.17
Not acceptable	14 (1.4)	8 (1.5)	6 (1.2)
No opinion	26 (2.5)	18 (3.4)	8 (1.6)
A change to a safer drug
Acceptable	1001 (96.3)	507 (94.8)	494 (97.8)	<0.01
Not acceptable	16 (1.5)	9 (1.7)	7 (1.4)
No opinion	23 (2.2)	19 (3.5)	4 (0.8)
A change to a cheaper drug
Acceptable	404 (38.8)	226 (42.2)	178 (35.3)	<0.01
Not acceptable	423 (40.7)	187 (35.0)	236 (46.7)
No opinion	213 (20.5)	122 (22.8)	91 (18.0)
To find out which drug works better
Acceptable	608 (58.5)	306 (57.2)	302 (59.8)	0.27
Not acceptable	343 (33.0)	176 (32.9)	167 (33.1)
No opinion	89 (8.5)	53 (9.9)	36 (7.1)
To find out which drug is safer
Acceptable	507 (48.8)	257 (48.0)	250 (49.5)	0.26
Not acceptable	457 (43.9)	232 (43.4)	225 (44.6)
No opinion	76 (7.3)	46 (8.6)	30 (5.9)
To compare two older drugs to find the best one
Acceptable	530 (51.0)	269 (50.3)	261 (51.7)	0.57
Not acceptable	394 (37.9)	201 (37.6)	193 (38.2)
No opinion	116 (11.1)	65 (12.1)	51 (10.1)
To compare a new drug with an older drug to find the best one
Acceptable	616 (59.2)	311 (58.1)	305 (60.4)	0.12
Not acceptable	337 (32.4)	170 (31.8)	167 (33.1)
No opinion	87 (8.4)	54 (10.1)	33 (6.5)

The next two questions were based around who has responsibility to determine drug safety and effectiveness. Participants were asked whether they agreed that ‘the NHS has a duty to determine the safety and effectiveness of the drugs its doctors prescribe’ (Q3). Ninety-seven per cent of respondents agreed with this statement. Participants were then asked whom they felt had the most responsibility to find out which are the best drug treatments from the options: the NHS in general, my GP practice, my individual GP, drug companies, a UK agency such as the Medicines and Healthcare products Regulatory Agency (MHRA), a European agency such as the European Medicines Agency or other (Q4). The most popular answers were the NHS in general (62%), a UK agency such as MHRA (57%) and drug companies (48%). Men were more likely than women to agree with the answer of ‘my individual GP’ (25% men vs. 17% women, P < 0.01), while women tended to agree more with the answer of ‘drug companies’ than men (54% women vs. 43% men, P < 0.01; Figure 1).

Figure 1.

Participants were asked their opinion on whom has the most responsibility to find out which are the best drug treatments. Results are presented as the percentage of positive responses by women (F) and men (M). Abbreviations: EMA, European Medicines Agency; MHRA, Medicines and Healthcare products Regulatory Agency

The principle of comparing the safety and effectiveness of two similar drugs by dividing people who take them into two groups, then following up the results of treatment, was then explained to participants and they were asked whether this would be acceptable or not acceptable to them as a way of comparing drugs (Q5). Seventy-two per cent said that this would be acceptable, 19% not acceptable and 9% had no opinion. There was no gender difference in the answers to this question.

Participants were asked whether they would be willing for NHS Scotland to follow up their medical data, with adequate protection in place, so that drug treatments could be compared (Q6). Eighty-one per cent were willing, 9% were not and 10% were unsure. There were no age or gender differences in the answers to this question.

In order to correct for any differences in responses to the earlier questions based on personal experience, participants were asked whether they had a medical condition for which they take prescription medicines on a regular basis (Q7; yes 43%, no 56% and declined to answer 1%). If they answered ‘yes’, they were also asked whether they had ever had one of their regular medicines changed by their GP surgery or pharmacist as part of a general policy (Q8; yes 31%, no 60% and unsure 9%) and whether they were asked first (Q9; yes 58%, no 39% and unsure 5%).

Those who reported that they had previously had a regular medication changed (n = 136) were asked why they think the change was made. They reported the following reasons: ‘the new drug was better’ 44%, ‘the new drug was safer’ 10%, ‘the new drug was cheaper’ 34% and ‘don't know’ 11%. These participants were also asked about their reaction to this change (Q11). The majority (58%) reported that they were ‘happy for their drug to be changed and there were no problems afterwards’.

There were significant differences in the answers given to some of the earlier questions (Q1, Q2 and Q4) between the 136 subjects reporting a previous change to their regular medication and the remainder of the study subjects. The subjects reporting a previous change to their regular medication were more likely to be ‘happy’ to receive the letter about prescribing policy change (Q1), more likely to find some of the reasons for changing their medicine ‘acceptable’, namely ‘to find out which drug works better’, ‘to find out which drug is safer’, ‘to compare two older drugs to find the best one’ and ‘to compare a new drug with an older one to find the best one’ (Q2, responses 4–7), and were more likely to nominate ‘my individual GP’ or ‘drug companies’ as having the most responsibility to find out which are the best drug treatments (Q4, responses 3–4).

Finally, subjects were asked whether they had a long-term medical condition (Q12) and, if so, to give a brief description. Overall, 58% of subjects said they did not have a long-term condition (69% of women and 49% of men, P < 0.01). The types and frequencies of conditions in the participants by percentage are listed in Table 4. The five most common types of conditions were blood pressure problems, mental health problems, arthritis, asthma/chronic obstructive pulmonary disease/breathing problems and diabetes mellitus. Men had more blood pressure problems, stomach/bowel/reflux problems, cholesterol problems, heart problems/angina and diabetes mellitus than women (P < 0.05). Subjects not reporting any long-term conditions were more likely to be ‘unhappy’ than subjects reporting long-term conditions about receiving a letter explaining about randomized policy changes (Q1; 41.8% in the no long-term condition group vs. 32.5% in the long-term condition group, P < 0.01).

Table 4.

Types and frequencies of long-term medical conditions in participants

Condition	Percentage
Blood pressure problems	6.3*
Mental health problems	5.6
Arthritis	5.0
Asthma/chronic obstructive pulmonary disease/breathing problems	4.6
Diabetes mellitus	4.0*
Stomach/bowel/reflux problems	3.6*
Cholesterol problems	2.7*
Heart problems/angina	2.7*
Mobility problems	2.0
Thyroid/endocrine problems	1.8
Chronic pain	1.0

^*P < 0.05 between men and women.

General practitioner survey

We received 341 replies (33% response rate) from the 1034 general practitioners approached. Of these replies, 154 (45%) general practitioners were ‘in favour’, 65 (19%) were ‘undecided’ and 122 (36%) were ‘not in favour’ of the concept of randomized drug formulary policy changes.

Twenty-seven general practitioners returned additional comments on the proposal (six of those in favour of the concept, seven of those undecided and 14 of those not in favour). The majority of these comments, especially from the group of general practitioners not in favour of the concept (11 of 14), referred to the potential extra workload involved for general practitioners. Other negative comments included problems with penalties due to noncompliance with formularies, the lack of electronic formularies, the potential to cause confusion, questionable benefit for patients and consent issues. Two general practitioners expressed a preference to randomize only newly treated patients to therapies rather than change established therapies in treated patients. Positive comments included the need for robust primary-care-based research to underpin treatments and question the risk-to-benefit ratio of long-established treatments, support for using the best and most effective drugs and general indications of support for the idea.

Discussion

The vast majority of the general public participants agreed that the NHS has a duty to determine the safety and effectiveness of the drugs its doctors prescribe. Traditionally, randomized controlled clinical trials have been seen as the gold-standard way of comparing different therapies. However, these can be very expensive and time consuming and may include a nonrepresentative sample of the population and therefore have poorer external validity than the type of approach described in this paper. Randomized policy design studies to compare different treatments within the NHS could be useful in a number of settings, including the comparison of new treatments with established treatments or the comparison of existing treatments which are widely used but for which we have limited evidence of safety or effectiveness. In some cases, this might be the only realistic way to compare treatments within the real-life NHS setting; for example, for older, cheaper therapies where there is no commercial interest in finding differences in safety or effectiveness, but which are widely used and could therefore have enormous implications for overall patient outcomes and NHS expenditure, e.g. comparing different thiazide diuretics in patients with hypertension. A recent review by Johnston et al. showed that substitution strategies for antihypertensives have not been tested in large outcome trials and there is little available clinical or economic evidence on which to base decisions to switch drugs [8]. We believe improved knowledge of effectiveness will translate into improved assessment of cost-effectiveness.

Such a mechanism would also be of use in the appraisal of newly marketed medicines. Increasingly, pharmaceutical companies are being asked to carry out postlicensing research to determine the safety and effectiveness of their newly marketed medicines. As cost-effectiveness appraisals drive the prescribing reimbursement of new medicines, postlicensing observational studies cannot be done because the medicine is not prescribed to any extent. Cluster randomization of practices to new medicines vs. standard therapy would allow a potential solution to this problem, because they could be done relatively quickly and inexpensively. In addition, 50% of practices who participated would be able to give their patients the opportunity to get the most recent medicine. Pharmaceutical companies would be expected to reimburse the cost of the medicines used in such an appraisal phase. This type of design would generate data on the safety and effectiveness vs. standard therapy and would allow the generation of data that could provide reassurance on safety and robust comparative effectiveness in the setting of normal care. These data would facilitate the capture of cost-effectiveness data that would assist the decision on reimbursement.

Recruitment to clinical trials is always difficult [9]. In randomized policy design studies, patients would be informed that their general practice was participating in a policy change study and given the option to opt out if desired. Performing research of this type driven by randomized policy changes could also encourage participation of more general practices in research. At present, some general practices participate in much research, while others participate in none. In a recent study with minimal input required at the practice level, only one-third of general practices participated [10]. In more complex studies, requiring additional workload, fewer practices generally choose to participate.

In order to gain the answers to important health-related questions in the real-life setting, it is important to achieve widespread participation of primary-care practices in research, with appropriate support for the additional workload involved, as this is usually the main concern preventing participation. Some steps have been taken towards this already, with the introduction of bodies such as the Primary Care Research Networks in the UK. Perhaps further incentives, such as Quality Outcomes Framework points for research activity, should be considered to improve research participation further. However, involvement in medical research is a responsibility of all medical practitioners according to General Medical Council Good Medical Practice guidance, which states that ‘research involving people directly or indirectly is vital in improving care and reducing uncertainty for patients now and in the future, and improving the health of the population as a whole’[11]. It also states that doctors ‘must help to resolve uncertainties about the effects of treatments’[12]. However, a major factor in implementing such studies is that they require to be resourced adequately. Changing patient prescription medicines often results in questions, intolerance, requests to stay on current therapy and a host of administrative and other issues that require time and manpower to address. Primary-care physicians will wish to be convinced that the introduction of such evaluations will not increase further their already saturated workload or that extra resources are available to deal with this.

Randomized policy studies raise certain ethical issues, which have been discussed in detail elsewhere [4]. Mainly, these involve the lack of individual informed consent and the use of opt-out systems instead. However, issues such as these have previously been addressed to enable effective research to take place using primary-care data, e.g. General Practice Research Database in the UK (http://www.gprd.com). As the results of our survey show, many patients taking long-term prescription medicines are familiar with policy changes being made to their regular medications by their practices (often as a result of formulary initiatives), and it seems from our results that many patients do not object to changes being made for a good reason, particularly if it is to improve safety or effectiveness. In fact, there is little evidence in the literature to assess patient views on such formulary changes to date [13].

The main concerns from general practitioners were around the potential additional workload involved. The results of this survey are broadly in keeping with our experience of involving GP practices in formal randomized controlled trials.

Conclusions

The results of the surveys suggest that the general population within Scotland is broadly supportive of the concept of randomized policy design studies to determine the safety and effectiveness of drug treatments within the NHS, while there is a spread of opinion among general practitioners. The next step may be to perform one of these types of studies, to answer an important and widely relevant question in medicine, then further determine the acceptability of this method of studying drug effectiveness and safety with both the public and health professionals.

Competing Interests

There are no competing interests to declare.