Recruiting patients to medical research: double blind randomised trial of “opt-in” versus “opt-out” strategies

Abstract

Objective To evaluate the effect of opt-in compared with opt-out recruitment strategies on response rate and selection bias.

Design Double blind randomised controlled trial.

Setting Two general practices in England.

Participants 510 patients with angina.

Intervention Patients were randomly allocated to an opt-in (asked to actively signal willingness to participate in research) or opt-out (contacted repeatedly unless they signalled unwillingness to participate) approach for recruitment to an observational prognostic study of patients with angina.

Main outcome measures Recruitment rate and clinical characteristics of patients.

Results The recruitment rate, defined by clinic attendance, was 38% (96/252) in the opt-in arm and 50% (128/258) in the opt-out arm (P = 0.014). Once an appointment had been made, non-attendance at the clinic was similar (20% opt-in arm v 17% opt-out arm; P = 0.86). Patients in the opt-in arm had fewer risk factors (44% v 60%; P = 0.053), less treatment for angina (69% v 82%; P = 0.010), and less functional impairment (9% v 20%; P = 0.023) than patients in the opt-out arm.

Conclusions The opt-in approach to participant recruitment, increasingly required by ethics committees, resulted in lower response rates and a biased sample. We propose that the opt-out approach should be the default recruitment strategy for studies with low risk to participants.

Introduction

Recruiting unbiased patient samples with high response rates is vital for the scientific rigour of much medical research. The traditional means of participant recruitment assumes that patients are potentially willing to participate, and non-response to an initial approach can be followed up with further communication (the so called “opt-out” approach). However, research ethics committees and institutional review bodies increasingly stipulate that investigators refrain from repeated contact with potential participants, unless these patients actively signal willingness to consider participation (the so called “opt-in” approach). The opt-in approach is increasingly applied to different aspects of study design, including consent for use of routine patient data and patient recruitment to studies.¹ Opting in is deemed ethically more defensible, as it relies on active participation of individuals, and some evidence shows that this is what patients expect.² The opt-out method has come under scrutiny as it relies on both inertia and the moral assumption that most people are willing to help researchers in principle.³ Whether patients find continued communication from researchers acceptable if they do not respond to an initial enquiry is unclear.

No evidence exists to show that the traditional opt-out approach for initial patient contact is in any way harmful. On the other hand, a low response rate and selection bias have obvious implications for study design, cost, and applicability of results. No study has examined the impact on response rate and patients' characteristics when the opt-in method is used to approach patients for recruitment into a research study. In medical record research, in which no new data are collected, the opt-in requirement has been associated with a poor response rate and a selected study population.⁴,⁵ A poor response rate has also been found with an opt-in approach in screening clinics and when asking for direct consent to participation in a study.^6-8 In this study, we compared the opt-in and opt-out methods of approaching patients for research in a randomised controlled trial of recruitment to a pilot study of patients with angina.

Methods

Design and randomisation

We obtained approval in 2002 to carry out a randomised controlled trial of patient recruitment comparing opt-in and opt-out approaches in a pilot of an observational study investigating the prognosis of angina. Patients were eligible for the pilot cohort study if they had angina within the previous three years. These patients were also eligible for this trial. We recruited patients from two general practices with comparable list sizes in Barking and Havering and in east London. We identified 899 patients by using search terms for prescription of nitrate, diagnosis of angina, or referral to a cardiologist within the previous three years on the Egton Medical Information System (EMIS) general practice computer systems. Two clinicians (GF and MJ) independently verified the computer search by checking the case notes, using a low threshold for inclusion. A total of 527 patients were verified, of whom 17 were deemed unsuitable for research by their general practitioner because of death, important comorbidity, severe mental illness, or personal reasons such as recent bereavement.

A research assistant used minimisation software to randomise the 510 remaining patients to opt-in or opt-out recruitment, balanced by sex and age (four age bands: ≤ 49, 50-59, 60-69, ≥ 70).⁹ The identity of the trial arm was kept in a sealed envelope and was known only to the research assistant, who used it to send out the appropriate invitation pack. Where people from the same household happened to be randomised to two different recruitment methods, we used the recruitment method of the husband for all people in the household (two participants changed to opt-in). After randomisation to either recruitment method, patients were invited by a letter from their general practitioner to participate in a pilot for a longitudinal cohort study of patients with angina at a clinic at their local hospital (London) or their own surgery (Barking and Havering). All patients received an information leaflet and a reply card, but with different formats depending on which recruitment strategy they had been allocated to. Patients (in agreement with the ethics committee) were unaware of the recruitment trial but fully informed about the cohort study. At the clinic, patients gave consent for the completion of a questionnaire, cardiovascular examination, blood tests, investigations, and follow-up. The clinics ran from November 2002 to June 2003.

Intervention

We asked patients in the opt-in arm to return the reply card or to phone if they wished to participate. We then telephoned patients who opted in to arrange an appointment. We sent a reminder letter to the opt-in group after two weeks if they had not contacted the research team, after which we made no further contact unless the patient contacted us. We told patients in the opt-out arm that they would be contacted by a researcher after two weeks unless they declined participation by returning their reply card or phoning the researchers. We phoned patients who did not opt out to ask if they wanted to participate and, if appropriate, to arrange an appointment.

Sample size and outcome

The primary endpoint was the proportion of patients seen who had given written informed consent at the clinic. We did additional analyses to compare patients' characteristics in the two arms with respect to factors that might influence prognosis. On the basis of reported response rates in recent studies recruiting from primary care,¹⁰ we assumed a conservative recruitment rate of 40%. We needed 250 patients in each arm to detect a minimum 5% difference in recruitment rate with 95% power (P = 0.05, two sided). In the trial reported here, we tested the null hypothesis that recruitment rates did not differ according to the method of patient approach (opt-in versus opt-out).

Statistical analysis

We analysed all data according to the pre-specified study protocol submitted for approval. We used the χ² statistic to compare proportions. We presented continuous variables as medians with interquartile range and used Student's t test to compare them. We calculated 95% confidence intervals for proportions.

Results

The figure shows the flow of patients through the trial. We recruited 239 patients from east London and 271 patients from Barking and Havering. We randomised 252 patients to the opt-in arm and 258 to the opt-out arm. Comparison of baseline characteristics of the 510 randomised patients showed that both arms had similar proportions of women (103 (41%) opt-in v 109 (42%) opt-out) and patients aged over 70 (127 (50%) opt-in v 134 (52%) opt-out).

Figure 1.

Flow of participants through trial

One hundred and three patients from both practices (East London (EL) 29, Barking and Havering (BH) 74) opted in after the first letter in the opt-in arm, and 50 patients opted out in the opt-out arm (EL 20, BH 30). Twelve patients in the opt-out arm sent back their reply card or phoned to actively opt-in. After the second and final mail out in the opt-in arm, 120 appointments had been made (EL 38, BH 82). In the opt-out arm, 151 appointments were made after patients were phoned two weeks after the letter had been sent out (EL 56, BH 95). Fourteen patients were not contactable by phone despite repeated attempts. The recruitment rate, defined by clinic attendance, was 38% (96/252) in the opt-in arm and 50% (128/258) in the opt-out arm (P = 0.014). Once an appointment had been made, no statistically significant difference occurred in attendance at the clinic between the two arms (80% opt-in v 83% opt-out; P = 0.86). All patients who were seen at the clinic gave consent for examination, tests, and further follow-up.

The table shows clinical characteristics of the pilot study population of participants with angina. Although participants did not differ by age, sex, ethnicity, or previous myocardial infarction, participants from the opt-in arm had less treatment for angina (69% v 82%; P = 0.010); less functional impairment as defined by the Canadian Cardiovascular Society angina classification (9% v 20%; P = 0.023); and fewer risk factors for coronary disease, defined by smoking status, South Asian ethnicity, hypertension, hypercholesterolaemia, diabetes, peripheral artery disease, and a body mass index ≥ 30 (44% v 60%; P = 0.053) than participants from the opt-out arm.

Table 1.

Comparison of characteristics relevant to target angina population in clinic attenders recruited by opt-in and opt-out methods. Values are numbers (percentages, 95% confidence intervals)

Characteristics	Opt-in (n=96)	Opt-out (n=128)	P value
Age over 70 years	49 (51, 41 to 61)	70 (55, 46 to 63)	0.50
Women	43 (45, 35 to 55)	56 (44, 35 to 52)	0.96
Previous myocardial infarction	25 (26, 18 to 36)	37 (29, 22 to 37)	0.56
Abnormal resting ECG	56 (58, 48 to 68)	73 (57, 48 to 65)	0.45
Maximal anti-anginal treatment*	66 (69, 59 to 77)	105 (82, 74 to 88)	0.010
Severe functional impairment†	9 (9, 5 to 17)	26 (20, 14 to 28)	0.023
Three or more coronary risk factors‡	42 (44, 34 to 54)	77 (60, 52 to 68)	0.053

ECG=electrocardiogram.

^*Combination of nitrates, β blocker, and calcium channel blocker.

^†Canadian Cardiovascular Society class III or IV.

^‡Current smoker, blood pressure >140/90 mm Hg, total cholesterol >6 mmol/l, diabetes, South Asian ethnicity, peripheral arterial disease, body mass index ≥30.

Discussion

Recruitment requiring patients to opt in was associated with a sample of healthier participants and a significantly lower response rate than the opt-out method. Our findings from a randomised comparison agree with descriptive accounts that modification of study protocols by ethics committees may compromise the integrity of research.¹¹,¹² The difference in recruitment between the two arms could be explained by the fact that patients in the opt-out arm were contacted directly by phone, whereas those in the opt-in arm were contacted only by letter, unless they responded to the invitation. Increased intensity of recruitment and a more personal approach are known to improve response rates.¹³,¹⁴ This does not, however, invalidate our finding that the opt-out approach achieves a better response rate, as the very nature of the opt-in approach prohibits contacting the patient by phone.

Another reason for the better response rate achieved in the opt-out arm could be the fact that patients were coerced into participation by the more aggressive opt-out approach, an explanation that would support the ethical requirement to use the opt-in method. However, we did not observe a higher rate of non-attendance among patients in the opt-out arm, which might be expected if this method had been perceived as coercive. Rather than feeling coerced, people willing to participate may find it burdensome to opt in. This possibility is supported by our findings that a higher proportion of functionally impaired and high risk patients were recruited in the opt-out arm. Although we believe that this is a plausible explanation backed up by individual feedback at the clinic, we did not systematically record patients' perceptions of the two recruitment methods directly. In addition, we do not know whether the higher response rate seen in the opt-out arm is a function of the ability to comply with recruitment requirements, such as travelling to the clinic, or whether the opt-out method in itself is more appealing. If there was no difference in functional impairment between the groups it may be that the response rate in the opt-out arm would be even greater than shown in our study.

As is commonly the case in randomised trials, our ability to assess how representative the sample of recruited patients was of the target population in either arm (external validity) is limited by the small number of characteristics for which data were available on all eligible patients. Whether our findings represent selection bias is difficult to determine, because the few characteristics we had of non-responders, such as age and sex, cannot be sensitive enough to measure differences between those groups. The opt-in arm recruited participants who were at lower risk of an adverse event than those in the opt-out arm. These differences in characteristics have important implications for sample size calculations, as study size and cost are largely determined by the baseline risk of patients and expected accrual of events. The recruitment of more patients at lower risk of events translates into a substantial increase in study cost and may render observational studies unfeasible.

We used a randomised trial design for our study to ensure that important baseline characteristics were balanced in each arm and that any results seen were not an artefact of the selection process (internal validity). However, in comparatively small studies such as ours, randomisation may not always achieve balance between the arms. We therefore included minimisation in our randomisation algorithm to ensure balanced age and sex distribution in the two arms.

Strengths and weaknesses

Our study is the first randomised comparison of opt-in and opt-out recruitment methods. One limitation is its focus on one condition (angina) and one study design (observational). Patients' responses to screening services differ widely by condition,¹⁵ which may well result in different responses to the opt-in or opt-out approach. Our study was recruiting patients to a cohort study, but patients are probably more concerned with the ease of participation and the trade-off between perceived benefits (for example, closer clinical monitoring) and personal sacrifices rather than the broad study method, and it is therefore plausible to assume that the low response rate seen with the opt-in approach would also be reflected in other study designs such as randomised controlled trials.

Finally, our study was carried out in two practices in the London area, so our findings may not be generalisable to other settings. However, our response rate is similar to that of comparable studies of patients recruited from general practice lists,¹⁰ and our findings were consistent in both practices despite the very different socioeconomic and ethnic composition of the practice populations and the fact that one clinic was held at the local hospital and one directly at the surgery.

In this context it is worth mentioning that the absence of a difference in ethnic backgrounds in the trial arms (P for difference = 0.16) may still conceal a bias towards English speaking patients among the 18% of patients for whom data on ethnicity was missing. In addition, ethnicity may not be able to distinguish between native and non-native English speakers. In both arms, 79% of clinic attenders were recorded as white. We did not have data on ethnicity or ability to speak English available for the population invited to participate and are unable to elucidate this issue. Future studies on selection bias and health inequalities are needed to investigate this.

Implications

The question remains whether and to what extent the opt-out method of recruitment diminishes personal autonomy and whether personal autonomy should be placed in the context of societal benefit and harm. We believe that the opt-out method is a justifiable infringement of personal autonomy if the patient is informed, every effort is made not to pressurise patients during the phone conversation, and refusal is accepted without question. Currently, the two recruitment approaches rest on opposing assumptions: that non-response is an indication of refusal (opt-in) or an indication of willingness (opt-out) to participate. The impact of either approach on the individual and society is unknown. However, the consequences for the validity and potential societal benefits of future research are potentially large: a 10% lower response, for example, might have serious scientific and financial implications for the recruitment of half a million patients to the UK Biobank programme.¹⁶ Given our findings, we believe that the opt-out approach to study recruitment should be the default strategy for studies that pose a low risk to patients.

What is already known on this topic

Ethics committees are seeking to tighten the rules governing recruitment of patients to research studies, with an opt-in method increasingly required

Opting in is associated with poor response rates and selection bias in medical record linkage studies

How an opt-in method for patient contact affects response rates and the characteristics of the recruited study population is not known

What this study adds

Patients contacted with an opt-in method had a greater than 10% lower response rate than patients contacted with an opt-out method

Patients recruited with the opt-in method were healthier and less relevant to the study than patients recruited with the opt-out method

The opt-out approach should be the default recruitment strategy for studies that pose a low risk to patients