Strategies to improve retention in randomised trials

Valerie C Brueton; Jayne Tierney; Sally Stenning; Seeromanie Harding; Sarah Meredith; Irwin Nazareth; Greta Rait

doi:10.1002/14651858.MR000032.pub2

. 2013 Dec 3;(12):1–126. doi: 10.1002/14651858.MR000032.pub2

Strategies to improve retention in randomised trials

Valerie C Brueton ¹, Jayne Tierney ², Sally Stenning ³, Seeromanie Harding ⁴, Sarah Meredith ³, Irwin Nazareth ⁵, Greta Rait ⁵

PMCID: PMC4470347 PMID: 24297482

Abstract

Background

Loss to follow-up from randomised trials can introduce bias and reduce study power, affecting the generalisability, validity and reliability of results. Many strategies are used to reduce loss to follow-up and improve retention but few have been formally evaluated.

Objectives

To quantify the effect of strategies to improve retention on the proportion of participants retained in randomised trials and to investigate if the effect varied by trial strategy and trial setting.

Search methods

We searched the Cochrane Central Register of Controlled Trials (CENTRAL), MEDLINE, PreMEDLINE, EMBASE, PsycINFO, DARE, CINAHL, Campbell Collaboration's Social, Psychological, Educational and Criminological Trials Register, and ERIC. We handsearched conference proceedings and publication reference lists for eligible retention trials. We also surveyed all UK Clinical Trials Units to identify further studies.

Selection criteria

We included eligible retention trials of randomised or quasi-randomised evaluations of strategies to increase retention that were embedded in 'host' randomised trials from all disease areas and healthcare settings. We excluded studies aiming to increase treatment compliance.

Data collection and analysis

We contacted authors to supplement or confirm data that we had extracted. For retention trials, we recorded data on the method of randomisation, type of strategy evaluated, comparator, primary outcome, planned sample size, numbers randomised and numbers retained. We used risk ratios (RR) to evaluate the effectiveness of the addition of strategies to improve retention. We assessed heterogeneity between trials using the Chi² and I² statistics. For main trials that hosted retention trials, we extracted data on disease area, intervention, population, healthcare setting, sequence generation and allocation concealment.

Main results

We identified 38 eligible retention trials. Included trials evaluated six broad types of strategies to improve retention. These were incentives, communication strategies, new questionnaire format, participant case management, behavioural and methodological interventions. For 34 of the included trials, retention was response to postal and electronic questionnaires with or without medical test kits. For four trials, retention was the number of participants remaining in the trial. Included trials were conducted across a spectrum of disease areas, countries, healthcare and community settings. Strategies that improved trial retention were addition of monetary incentives compared with no incentive for return of trial-related postal questionnaires (RR 1.18; 95% CI 1.09 to 1.28, P value < 0.0001), addition of an offer of monetary incentive compared with no offer for return of electronic questionnaires (RR 1.25; 95% CI 1.14 to 1.38, P value < 0.00001) and an offer of a GBP20 voucher compared with GBP10 for return of postal questionnaires and biomedical test kits (RR 1.12; 95% CI 1.04 to 1.22, P value < 0.005). The evidence that shorter questionnaires are better than longer questionnaires was unclear (RR 1.04; 95% CI 1.00 to 1.08, P value = 0.07) and the evidence for questionnaires relevant to the disease/condition was also unclear (RR 1.07; 95% CI 1.01 to 1.14). Although each was based on the results of a single trial, recorded delivery of questionnaires seemed to be more effective than telephone reminders (RR 2.08; 95% CI 1.11 to 3.87, P value = 0.02) and a 'package' of postal communication strategies with reminder letters appeared to be better than standard procedures (RR 1.43; 95% CI 1.22 to 1.67, P value < 0.0001). An open trial design also appeared more effective than a blind trial design for return of questionnaires in one fracture prevention trial (RR 1.37; 95% CI 1.16 to 1.63, P value = 0.0003).

There was no good evidence that the addition of a non-monetary incentive, an offer of a non-monetary incentive, 'enhanced' letters, letters delivered by priority post, additional reminders, or questionnaire question order either increased or decreased trial questionnaire response/retention. There was also no evidence that a telephone survey was either more or less effective than a monetary incentive and a questionnaire. As our analyses are based on single trials, the effect on questionnaire response of using offers of charity donations, sending reminders to trial sites and when a questionnaire is sent, may need further evaluation. Case management and behavioural strategies used for trial retention may also warrant further evaluation.

Authors' conclusions

Most of the retention trials that we identified evaluated questionnaire response. There were few evaluations of ways to improve participants returning to trial sites for trial follow-up. Monetary incentives and offers of monetary incentives increased postal and electronic questionnaire response. Some other strategies evaluated in single trials looked promising but need further evaluation. Application of the findings of this review would depend on trial setting, population, disease area, data collection and follow-up procedures.

PLAIN LANGUAGE SUMMARY

Methods that might help to keep people in randomised trials

Background

Most trials follow people up to collect data through personal contact after they have been recruited. Some trials get data from other sources, such as routine collected data or disease registers. There are many ways to collect data from people in trials, and these include using letters, the internet, telephone calls, text messaging, face-to-face meetings or the return of medical test kits. Most trials have missing data, for example, because people are too busy to reply, are unable to attend a clinic, have moved or no longer want to participate. Sometimes data has not been recorded at study sites, or are not sent to the trial co-ordinating centre. Researchers call this 'loss to follow-up', 'drop out' or 'attrition' and it can affect the trial's results. For example, if the people with the most or least severe symptoms do not return questionnaires or attend a follow-up visit, this will bias the findings of the trial. Many methods are used by researchers to keep people in trials. These encourage people to send back data by questionnaire, return to a clinic or hospital for trial-related tests, or be seen by a health or community care worker.

Study characteristics

This review identified methods that encouraged people to stay in trials. We searched scientific databases for randomised studies (where people are allocated to one of two or more possible treatments in a random manner) or quasi-randomised studies (where allocation is not really random, e.g. based on date of birth, order in which they attended clinic) that compared methods of increasing retention in trials. We included trials of participants from any age, gender, ethnic, cultural, language and geographic groups.

Key results

The methods that appeared to work were offering or giving a small amount of money for return of a completed questionnaire and enclosing a small amount of money with a questionnaire with the promise of a further small amount of money for return of a filled in questionnaire. The effect of other ways to keep people in trials is still not clear and more research is needed to see if these really do work. Such methods are shorter questionnaires, sending questionnaires by recorded delivery, using a trial design where people know which treatment they will receive, sending specially designed letters with a reply self addressed stamped envelope followed by a number of reminders, offering a donation to charity or entry into a prize draw, sending a reminder to the study site about participants to follow-up, sending questionnaires close to the time the patient was last followed-up, managing peoples' follow-up, conducting follow-up by telephone and changing the order of questionnaire questions.

Quality of evidence

The methods that we identified were tested in trials run in many different disease areas and settings and, in some cases, were tested in only one trial. Therefore, more studies are needed to help decide whether our findings could be used in other research fields.

Background

Description of the problem or issue

Randomised trials are the gold standard for evaluating the effectiveness and efficacy of interventions. Non-response or loss to follow-up within study groups in randomised trials can compromise study findings by reducing the power of a study to detect a true difference between the control and the intervention group. Differential loss to follow-up may lead to bias through exaggerated effects in favour of one of the groups. This can affect the generalisability and internal validity of the trial and the results (Fewtrell 2008; Schulz 2002).

Missing data from loss to follow-up can be dealt with statistically by various methods including, for example, imputing values based on valid assumptions about the missing data to give a conservative estimate of the treatment effect. However, the risk of bias still remains when trials do not collect adequate data to give accurate estimates (Hollis 1999). Schulz and colleagues suggested that less than 5% loss to follow-up may lead to minimum bias, while 20% loss to follow-up can threaten trial validity, although the pattern of loss to follow-up by treatment may also be an important factor (Schulz 2002). Loss to follow-up from randomised trials can sometimes go unreported and using different, but plausible, assumptions about outcomes for participants lost to follow-up can change the results of randomised trials.

A number of trials have retrospectively examined the predictors of loss to follow-up in different disease areas (Arnow 2007; Snow 2007; Villarruel 2006). In a trial for the treatment of chronic major depression, Arnow examined the predictors of time to, and reason for, dropout of participants (Arnow 2007). Ethnic minorities and participants with comorbid anxiety were more likely to drop out. In a randomised trial of a human immunodeficiency virus (HIV) prevention intervention for Latino youths, English speakers were more likely to attend follow-up (Villarruel 2006). Snow examined the predictors of clinic attendance and dropout at the 11-year follow-up of the Lung Health study (Snow 2007). Age, gender, number of cigarettes smoked per day, marital status and whether participant's children smoked were predictors of clinic attendance. These analyses showed that attendance for follow-up can be trial and disease specific. An awareness of these factors can help trialists decide which strategies to adopt to improve retention in their randomised trial.

Description of the methods being investigated

Strategies to improve trial retention include those designed to generate maximum data return or compliance to follow-up procedures. These can include frequency and timing of follow-up visits (follow-up shortly after randomisation versus long-term follow-up), nature of the outcome to be measured (survey based self reported outcomes versus morbidity or mortality reporting), target of the intervention (participants versus providers versus trial sites), and type of intervention (incentives versus communication strategies versus participant case management).

How these methods might work

These retention strategies are designed to motivate participants (Leathem 2009), or the trial site to continue participating in a trial once they have been recruited and randomised. Some strategies are designed to encourage participants to identify with the trial and to promote a sense of value and belonging, for example, using trial identity cards. Other strategies are designed to keep participants engaged in the trial, for example, by sending participant newsletters. To encourage a proactive approach to trial retention, strategies can be designed to target participants directly through letters, emails, telephone calls or to target them via the clinicians involved in participant follow-up, for example, through regular communication with trial sites. Strategies have been specifically developed to promote retention in areas of research where it is particularly challenging, such as mental health (Furimsky 2008; Loue 2008), weight loss ( Couper 2007; Goldberg 2005), rare diseases (McKinstry 2007), substance abuse (El Khorazaty 2007), research involving minority ethnic groups (Eakin 2007; Loftin 2005; Villacorta 2007), and vulnerable groups such as older people (Burns 2008) or people with HIV (Anastasi 2005).

Why it is important to do this review

As drop-out or incomplete data causes problems in the conduct, analysis and interpretation of randomised trials, it is important to identify retention strategies that minimise this loss as far as possible.

Davis and colleagues conducted a review of community-based trials published from 1990 to 1999 and described retention strategies and retention outcomes for this area (Davis 2002). Robinson and colleagues conducted a systematic review of strategies for retaining study participants (Robinson 2007). While both reviews identified studies providing data on retention rates from primary studies and strategies used to promote retention, these were not evaluated quantitatively in either review.

A systematic review of strategies to retain participants in population-based cohort studies found that providing incentives was consistently associated with retention in these studies and that response generally increased with increasing incentive value (Booker 2011). Reminder letters, repeat questionnaires and reminder calls also increased response rates. Furthermore, the Edwards et al. Cochrane methodology review on methods to increase response rates to postal and electronic questionnaires found that including monetary incentives, keeping the questionnaire short and contacting people before sending the questionnaire were ways to increase response rates (Edwards 2009). That review was not restricted to research exclusively within randomised trials and covered both healthcare and non-healthcare settings and it is difficult to know which of these strategies would be applicable to randomised trials in health care. Reasons for drop-out in cohort studies and surveys may differ from those in randomised trials. For example, in trials, participants may be randomised to a study group that is not their preferred choice and factors around randomisation and the type of intervention mean that strategies increasing retention in cohort studies and surveys cannot necessarily be extrapolated to randomised trials.

The challenges of boosting recruitment to randomised trials is often described alongside retention in the literature. Some similar strategies may be used in an attempt to both increase recruitment and improve retention, such as giving incentives together with extra information. Rendell et al. assessed the evidence for the effect of disincentives and incentives on the extent to which clinicians invite eligible people to participate in randomised trials of healthcare interventions (Rendell 2007). No randomised trials of interventions were identified and the authors concluded that some aspects of the conduct of the trial might affect a clinician's willingness to invite people to participate, for example, the way the clinician is invited to take part and the availability of support staff. In another Cochrane methodology review, Treweek et al. assessed strategies to improve recruitment to research studies (Treweek 2010), but recruitment to trials presents different challenges to participant engagement and follow-up. For example, strategies to market a trial and win over participants during the recruitment phase may be different to strategies to keep participants engaged in a trial (Francis 2007).

Many untested strategies are used by researchers to try to improve retention in randomised trials. Therefore, because loss to follow-up can compromise the validity of a trial's findings, delay results and, in some circumstances, increase the costs of the research, a systematic review is needed to assess the effect of strategies to improve retention in randomised trials.

Objectives

To quantify the effect of strategies to improve retention in randomised trials.

To investigate if the effect varies by the type of strategy, trial setting and healthcare area.

Methods

Criteria for considering studies for this review

Types of studies

We included completed randomised trials that compared strategies to increase retention embedded in host randomised trials (hereafter referred to as retention trials). The retention trials were embedded in real trials (host trials) and not hypothetical trials. The retention trials included at least one randomised comparison of two or more strategies to improve retention, or compared one or more strategies with no strategy. In anticipation of few trials, we included retention trials if they were randomised or quasi-randomised (e.g. had used alternation, date of birth or case record number as a method of allocating participants) (Lefebvre 2008).

Strategies to improve retention were designed for impact after participants were recruited and randomised to either the intervention or control group of the main and the retention trial. We included trials to increase response to postal and electronic questionnaire. We excluded trials to increase recruitment only. We excluded cohort studies with embedded randomised retention trials, which were the subject of a separate systematic review (Booker 2011).

Types of data

We included randomised and quasi-randomised retention trials within the context of a host randomised trial with participants from any age, gender, ethnic, cultural, language and geographic groups. We included unpublished and published participant retention data from randomised trials addressing healthcare (including all disciplines and disease areas) and non-healthcare (education, social sciences) topics. We also included trials set in the community that were healthcare related.

Types of methods

We considered any strategy aimed at increasing retention, directed towards the clinician, researcher or participant. We included strategies compared with each other or with usual study procedures. We also included trials with any combination of strategies to increase retention. Strategies could be participant or trial management focused and include any of the following:

strategies to motivate participants and clinicians (e.g. incentives or gifts);
strategies to improve communication with participants or trial sites (e.g. enhanced letters);
methodology strategies (e.g. shorter length of follow-up or variation in follow-up visit frequency);
strategies to improve social support for participant retention.

Types of outcome measures

Primary outcomes

We used retention (the proportion of participants retained) at the primary analysis point as defined in each individual retention trial as the primary outcome because it is easier to interpret than attrition/loss to follow-up (i.e. the proportion lost or not retained). In cases where the time point for measurement of the primary outcome was not predefined, we took the first time point reported for analysis. In most cases, this was final response. If retention at a number of time points was reported and no clear time point for the primary outcome for the retention trial was stated, we took data for the nearest time point to the intervention in the retention trial analyses.

Secondary outcomes

Retention of participants at secondary analysis points.

Search methods for identification of studies

We designed a search strategy to identify published and unpublished randomised and quasi-randomised trials that assessed strategies to improve retention in randomised trials in healthcare, education and social science settings. We searched bibliographic databases for published trials and trial registers for trials that had not been fully published, or were unpublished or ongoing. We applied no language restrictions.

Electronic searches

Each search comprised an established filter to identify randomised trials plus free-text terms and database subject headings relating to reducing loss to follow-up or increasing retention (Appendix 1). Electronic databases searched included:

the Cochrane Central Register of Controlled Trials (CENTRAL) (to May 2012);

PreMEDLINE (to April 2010);
MEDLINE (1950 to May 2012) (Appendix 2), EMBASE (1980 to May 2012) (Appendix 3) and PsycINFO (1806 to May 2012) (Appendix 4), searched using an Ovid platform;
Database of Abstracts of Reviews of Effects (DARE, in The Cochrane Library May 2012);
CINAHL (Cumulative Index to Nursing and Allied Health; 1981 to May 2012) (Appendix 5), using the EBSCOHost platform;
Campbell Collaboration's Social, Psychological, Educational and Criminological Trials Register (C2-SPECTR http://geb9101.gse.upenn.edu/: searched May 2009 (website no longer accessible)) (Appendix 6);
Education Resource Information Centre (ERIC) 1966 to May 2009) (Appendix 7), using Dialog Datastar.

Searching other resources

We handsearched the reference lists of relevant publications and reviews to identify further trial reports (Horsley 2011) (Appendix 8). We also searched the abstracts of Society for Clinical Trials (SCT) meetings from 1980 to 2012, the Current Controlled Trials metaRegister of Controlled Trials (mRCT) (http://www.controlled-trials.com/mrct), the Cochrane Methodology Register (in The Cochrane Library to April 2012) and the World Health Organization (WHO) trials platform (http://apps.who.int/trialsearch). We conducted a survey of Clinical Trial Units in the UK to identify further eligible trials not identified through other sources and the review was presented at the Society for Clinical Trials 31st Conference in Baltimore, USA in May 2010 and advertised on the Conference notice board with the aim of identifying potentially eligible trials from outside the UK.

Data collection and analysis

Selection of studies

One review author (VB) selected potentially eligible trials from the titles and abstracts retrieved by the searches, using a predesigned study eligibility screening form. We were over inclusive when screening, 0.7% (168/24,304) of records identified were sent for screening to a second review author (GR), which is 23% (168/735) of all potentially eligible records identified. We obtained full-text papers and two review authors (VB, GR) reviewed potentially eligible trials for inclusion. We contacted study authors for electronic copies of papers that we could not access through library sources. We were able to obtain copies of all the potentially eligible papers that we wanted to screen. We resolved disagreements by discussion with a third review author (SS). When necessary, we sought information from the original investigators for potentially eligible trials where we wished to clarify eligibility.

Data extraction and management

One review author (VB) extracted data from eligible retention trial and associated host trial papers and a second review author (JT) checked the entries. We reached consensus on any disparities by discussion with a third review author (SS). Data extracted for the host trial were aim, setting, disease area, comparators, primary outcome, sample size calculation, inclusion exclusion criteria, sequence generation and allocation concealment, and numbers randomised to each group. For the embedded retention trial, we extracted data for onset in relation to the host trial, source of the sample, aim, primary outcome and type of follow-up. The retention strategy details included type, frequency and timing of administration method of randomisation, numbers randomised, included and retained at primary analysis, and data required for the risk of bias assessment.

Assessment of risk of bias in included studies

To assess the validity of each retention trial we judged them against the four domains of the Cochrane 'Risk of bias' tool (Higgins 2008a). To assess selection bias, we recorded how the allocation sequence was generated at study level and the methods used to conceal the allocation. We assessed performance bias by recording methods used to blind participants if considered appropriate to do so. For some interventions, participants could not be blinded to the intervention (e.g. where vouchers, cash or gifts were administered). However, in these cases, study personnel could be blinded to the allocation if administration of the intervention was carried out by someone unaware of the allocation.

As retention is the subject of our review, and retention of participants is the primary outcome, attrition from the trials does not constitute a bias and has not been included in the 'Risk of bias' tables. We assessed each included retention trial for selective outcome reporting by recording the primary outcome for the trial and the outcomes for which results were reported. A judgement was made about each trial for each risk of bias domain assessed. For completed host trials (within which retention trials were embedded), we only assessed sequence generation and allocation concealment, in order to ensure the host trial was randomised.

Measures of the effect of the methods

We calculated risk ratios (RR) and their 95% confidence intervals (CI) for retention to determine the effect of strategies on this outcome.

Unit of analysis issues

For retention trials that randomised individuals and clusters, the unit of analysis was the participant. For cluster randomised trials that ignored clustering in the analysis, we inflated the standard errors (SE) to avoid overprecise estimates of effect as follows (Higgins 2008b).

We calculated the RR, 95% CI and SE based on participants in the usual way (i.e. ignoring clustering).
This standard error was then inflated using the design effect to get an adjusted SE: adjusted SE = SE X√ design effect. With the design effect calculated as follows: design effect = 1 + (M - 1) ICC where M = mean cluster size, ICC = the intracluster correlation coefficient.
Where published ICCs were not available, we used the mean ICC from appropriate external estimates for Land 2007. This was the mean of estimates for the return of EuroQol questionnaires (ICC = 0.054) from a source recommended by the Cochrane Handbook for Systematic Reviews of Interventions (Section 16.3.4) (Higgins 2008b) and http://www.abdn.ac.uk/hsru/documents/iccs-web.xls (last accessed 27 September 2013).
We entered the effect estimate and the new updated SE into Review Manager 5 using the generic inverse variance (RevMan 2012).

Where the number of participants randomised was not clearly stated in the included study report, we contacted the study authors for this information.

Dealing with missing data

We contacted study authors for data for the risk of bias assessment, numbers randomised to each group and numbers retained in each group at the primary endpoint. We described outcomes with insufficient data qualitatively. For time-to-event outcomes, we used the time point of the host study primary outcome, taking account of censoring if necessary and if the data were available.

Assessment of heterogeneity

We measured heterogeneity of the intervention effect using the Chi² statistic at a significance level of 0.10 and the I² statistic (Higgins 2003), and explored through subgroup analyses.

Assessment of reporting biases

We would have investigated reporting bias using tests for funnel plot asymmetry if sufficient data had been available (Egger 1997; Sterne 2008).

Data synthesis

If there was no substantial heterogeneity, we pooled RRs using the fixed-effect model. If heterogeneity was detected and could not be explained by subgroup or sensitivity analyses, we used the random-effects model or did not pool results.

For factorial trials (Sharp 2006a-h, Kenton 2007a-d), all main effects were included as separate trial comparisons if they addressed different categories of strategies. Where the main effects addressed two or more strategies within the same category (e.g. Bowen 2000abc), we combined the relevant intervention groups and compared them with the control group. We also compared each intervention group with the control group, as separate trial comparisons, in exploratory analyses. For one 2 x 2 x 2 x 2 factorial trial (Renfroe 2002a-d), the numbers randomised for each group were not available at the time of analysis so comparison groups were collapsed as far as possible and then treated as separate trial comparisons in the appropriate analyses. For two three-armed trials that compared two similar intervention groups with one control group, we combined the intervention groups and compared it with the control group for the main analyses (Bauer 2004ab, Khadjesari 2011 1abc). We also compared each intervention group as separate trial comparisons in exploratory analyses.

These approaches allowed full exploration of the data and also avoided double counting and over-precise pooled estimates of effect in our main analyses. However, this also meant that there were occasionally a greater number of trial comparisons than trials.

Computations for the absolute benefits of effective strategies on questionnaire response and trial retention were based on absolute risk reductions derived from meta-analysis RRs (Cochrane Handbook for Systematic Reviews of Interventions, Section 12.5.4.2: Schünemann 2008).

Subgroup analysis and investigation of heterogeneity

To explore the effect of different strategies on trial retention, we planned the following subgroup analyses by the type of strategy used in included retention trials.

Whether the strategy was compared with usual follow-up or other strategies.
Whether in healthcare or non-healthcare settings.
Whether assessment of retention was immediate or longer term (e.g. if a response to a questionnaire was expected immediately or at later time points).
Whether the strategy was participant or management focused.

However, we identified such a diversity of retention trials and interventions that these analyses were inappropriate or not possible. Therefore, different types of strategies were analysed separately and new subgroups were defined within these before we conducted the analyses.

(a) Incentives

We subgrouped retention trials or trial comparisons evaluating the addition of an incentive strategy versus none as follows for analysis.

Monetary incentives given upfront, defined as money given to the trial participant prior to data collection in cheque, cash or voucher format.
Non-monetary incentives, defined as gifts, for example, pens or certificates.
Offers of monetary incentives after data collection, defined as a promise of the incentive after return of outcome data through attendance for scheduled follow-up or receipt of follow-up questionnaires.
Offers of non-monetary incentives defined as a promise of the non-monetary incentive after return of outcome data through attendance for scheduled follow-up or receipt of follow-up questionnaires.

We subgrouped retention trials or trial comparisons comparing different values of monetary incentives into:

those offering incentives;
those both giving and offering an incentive for any subsequent data (e.g. sending GBP5 with a questionnaire with an offer of GBP5 if the questionnaire is returned).

We analysed retention trials evaluating the addition of a monetary incentive versus either an offer of a monetary incentive or follow-up by telephone separately.

(b) Communication

We grouped retention trials or trial comparisons of the effect of different communication strategies into letter, post and reminder strategies for analysis as follows.

Enhanced versus standard cover letter.
Total design method versus standard postal communication strategy.
Priority versus regular post.
Additional reminders versus usual reminders to trial sites.
Additional reminders versus usual follow-up to trial participants.
Early versus late administration of questionnaire (i.e. sending questionnaires two to three weeks after a follow-up visit versus one to four months after a follow-up visit).
Recorded delivery versus telephone reminder.

(c) Questionnaire structure

We subgrouped trials of questionnaire strategies into length of questionnaire, clarity of meaning, order of questions and layout as follows.

Short versus long questionnaire.
Long and clear questionnaire versus short and condensed questionnaire.
Medical condition questions first versus generic questions first.
Relevance of questionnaires: alcohol versus mental health questionnaires.

There were no subgroups for behavioural, case management and methodology retention trials.

Our analyses focused on the primary endpoint of retention. We initially pooled retention trials within subgroups using the fixed-effect model and quantified heterogeneity. We assessed whether these subgroups had a differential impact on retention using the test for interaction. We did not pool trials if results were inconsistent or heterogeneity was excessive.

Sensitivity analysis

To assess the robustness of the results we planned sensitivity analyses that excluded quasi-randomised retention trials.

Results

Description of studies

The studies are described in the Characteristics of included studies, Characteristics of studies awaiting classification, and Characteristics of excluded studies tables.

Results of the search

We identified 24,304 abstracts, titles and other records from database searches to May 2012, handsearches of reviews, lists of references in included papers, SCT conference abstracts (to 2012), personal contact with trialists, and the survey of UK Clinical Trials Units (Figure 1). We screened 735 full-text papers, reports and manuscripts for eligible studies. Of 68 potentially eligible studies, we found 30 to be subsequently ineligible. This left 38 retention trials for inclusion in the review. The retention trials were embedded in real trials (host trials). We identified 11retention trials from CENTRAL, MEDLINE and CINAHL; 14 from handsearching reviews, conference abstracts, and references lists of eligible papers; and 13 through personal communications or correspondence with clinical trials units. We evaluated six broad types of strategy to improve retention in randomised trials. Most strategies were targeted at increasing questionnaire response. The strategies used for this were incentives, communication, methodology and questionnaire design strategies. There was minimal evidence for the use of behavioural and case management strategies to improve retention.

Included studies

Of the 38 eligible retention trials, 28 were published in full, one as an abstract (Kenton 2007a-d), and one as part of a PhD thesis (Nakash 2007). Four retention trial publications contained two trials each (Khadjesari 2011; McCambridge 2011; McColl 2003; Severi 2011). Eight retention trials are unpublished as of June 2013 (Bailey 1; Bailey 2; Edwards 2001; Land 2007; Letley 2000; MacLennan; Marson 2007; Svoboda 2001).

Host trials

Twenty-two host trials included a single retention trial (AVID investigators 1997; Boyd 2002; Chaffin 2009; Cooke 2009; Cox 2008; Gail 1992; Dennis 2009; Hughes 1984; International Stroke Trial Group 1997; Kenyon 2001; Lamb 2007; Leigh Brown 2001; Marson 2007 (2); Omenn 2006; Porterhouse 2005; Rothert 2006; Tai 1999; Tilbrook 2011; TOMBOLA 2009a; TOMBOLA 2009b; UK BEAM 2004). Two host trials from this group were unpublished (for the retention trials by Ashby 2011 and Land 2007).

The other host trials included multiple retention trials (one unpublished for the retention trials by Bailey 1 and Bailey 2). Two retention trials (Ford 2006; Subar 2001) were embedded in the US-based Prostate, Lung, Colorectal, Ovarian (PLCO) screening trial of Prorok 2000; two (Avenell 2004; MacLennan) in the RECORD fracture prevention trial (RECORD 2007); two (Edwards 2001; Svoboda 2001) in the CRASH trial (CRASH Trial collaborators 2004); four (Khadjesari 2011 1abc; Khadjesari 2011 2; McCambridge 2011 1; McCambridge 2011 2) in the Down your Drink Trial (Murray 2007); two (Bailey 1; Bailey 2) in a feasibility study for the Sex unzipped website (unpublished); two (Severi 2011 1; Severi 2011 2) in the Text to Stop smoking cessation trial (Free 2011); and two (McColl 2003 1; McColl 2003 2) in the COGENT trial (Eccles 2002).

Participants and settings

Included retention trials were conducted in a broad spectrum of clinical conditions and geographical settings (see Appendix 9). Eight included retention trials were embedded in trials for the treatment of alcohol and smoking dependency (Bauer 2004ab; Hughes 1989; Khadjesari 2011 1abc; Khadjesari 2011 2; McCambridge 2011 1; McCambridge 2011 2; Severi 2011 1; Severi 2011 2), and four in trials investigating treatments for injuries (Edwards 2001; Gates 2009; Nakash 2007; Svoboda 2001). Six retention trials were set in treatment trials for cancer, cardiovascular disease, epilepsy and back pain (Dorman 1997; Land 2007; Letley 2000; Man 2011; Marson 2007; Renfroe 2002a-d), and four were embedded in screening trials for cancer, postnatal depression, and elderly diseases (Ford 2006; Kenton 2007a-d; Sharp 2006a-h; Subar 2001). Seven retention trials were embedded in prevention trials, which included two cancer prevention trials for lung and breast cancer (Bowen 2000abc; Sutherland 1996), one migraine prevention trial (Ashby 2011), and three fracture prevention trials (Avenell 2004; Cockayne 2005; MacLennan). Four retention trials were conducted in clinical management trials for orthopaedics, asthma, diabetes and angina (Leigh Brown 1997; McColl 2003 1; McColl 2003 2; Tai 1997). Six retention trials were conducted in other areas: exercise (Cox 2008), parenting (Chaffin 2009), weight management (Couper 2007), neonatal medicine (Kenyon 2005), and sexual health promotion (Bailey 1; Bailey 2).

Twenty-five retention trials were UK based, nine were USA based and two were set in Canada. The remainder were set in Czech Republic and Australia (see Characteristics of included studies table).

Retention trials were embedded in host trials that recruited participants from different settings. Five trials recruited participants directly from the community. Sixteen trials were conducted through secondary care facilities. One trial recruited participants through a combination of state workers compensation programmes, occupational and physician clinic, a surveillance programme and union records. Six UK trials recruited solely through general practitioner (GP) practices and two used a combination of recruitment through GP practices and the media. Seven trials recruited participants via the Internet, six of these were UK based and the other was US based. For one US-based smoking cessation trial, it was unclear how participants were recruited (see Characteristics of included studies table).

Design of included retention trials

One trial was hosted in a clustered randomised trial and used this design to evaluate a strategy to improve retention (Land 2007). Four retention trials used different factorial designs (Bowen 2000abc; Kenton 2007a-d; Renfroe 2002a-d; Sharp 2006a-h). There was also one three-armed trial (Bauer 2004ab), and three four-armed trials (Khadjesari 2011 1abc; McCambridge 2011 1; McCambridge 2011 2).

Five trials used quasi-randomisation to allocate participants (Bowen 2000abc; Ford 2006; Gates 2009; McColl 2003 1; McColl 2003 2). Two used participant identification numbers (Ford 2006; Gates 2009), and two allocated the first half of a simple random sample of participants to receive one version of a questionnaire, while the remaining half was allocated to a second version (McColl 2003 1; McColl 2003 2). One retention trial used day of clinic visit to allocate participants (Bowen 2000abc).

All trials targeted individual trial participants, except one that targeted trial sites (Land 2007).

We recorded the timing of randomisation in the host trial versus the timing of randomisation in the retention trial. Four trials commenced during a randomised pilot study for the host trial (Khadjesari 2011 1abc; Letley 2000; McCambridge 2011 1; Sutherland 1996). One study started before the host trial (Chaffin 2009). Twenty-nine trials commenced during follow-up for the host trial (Ashby 2011; Avenell 2004; Bailey 1; Bailey 2; Bowen 2000abc; Cockayne 2005; Couper 2007; Cox 2008; Dorman 1997; Edwards 2001; Ford 2006; Gates 2009; Khadjesari 2011 2; Land 2007; Leigh Brown 1997; MacLennan; Man 2011; Marson 2007; McCambridge 2011 2; McColl 2003 1; McColl 2003 2; Nakash 2007; Renfroe 2002a-d; Severi 2011 1; Severi 2011 2; Sharp 2006a-h; Subar 2001; Svoboda 2001; Tai 1997). For one trial, it was unclear when the retention trial started in relation to the host trial (Kenton 2007a-d). Three retention trials started after the host trial had finished (Bauer 2004ab; Hughes 1989; Kenyon 2005): Kenyon 2005 followed-up seven-year-old children of mothers enrolled in the ORACLE trial (Kenyon 2001), Bauer 2004ab followed up participants in the COMMIT smoking cessation trial (Gail 1992), eight years after the original trial was completed and Hughes 1989 followed up participants in a smoking cessation trial six months after that study finished (Hughes 1984).

Strategies to improve retention

Retention in trials and response to questionnaires were the outcomes measured for all included trials. The included trials evaluated six different types of strategies to improve response or retention. Incentives, communication strategies, variation in questionnaire design, methodology strategies, and combinations of communication and incentive strategies evaluated improving response to postal and electronic questionnaires. Behavioural strategies, case management and some non-monetary incentives were used to encourage participants to return to trial sites for follow-up visits. Each type of strategy is described separately below.

Outcome measures in the included trials

Thirty-four retention trials measured response to questionnaires. Among these, the questionnaires were by post in 26 trials, electronically in four and one was done by interview. For another three retention trials, response was return of biomedical kits or biomedical kits plus a questionnaire (see Characteristics of included studies table).

Four included trials measured the number of participants remaining in the trial (Bowen 2000abc; Chaffin 2009; Cox 2008; Ford 2006)

Ten included trials specified that their primary outcome was questionnaire response at a particular time point: McCambridge 2011 1 measured response at one and three months, McCambridge 2011 2 measured response at three and 12 months, and Khadjesari 2011 1abc and Khadjesari 2011 2 measured response within 40 days of the first reminder. For Severi 2011 1, the primary outcome was completed follow-up at 30 weeks from randomisation, Severi 2011 2 used return of specimens one month after a telephone call, Avenell 2004 used retention at one year measured by questionnaire return but also reported retention at four and eight months. Cockayne 2005 and Sharp 2006a-h had final follow-up questionnaire response at any time as their primary outcome.

Two included trials reported questionnaire response at one time point only but without specifying that this was the primary outcome for the trial (Edwards 2001; Svoboda 2001). These trials measured response at three months from the questionnaire being sent. One trial reported trial retention at one time point only (three years) but without specifying that this was the primary outcome for the trial (Ford 2006). This was measured as completing the next cancer screening in a cancer screening trial. In each of these three trials, we used these data for analyses.

Two trials recorded questionnaire response at two time points without stating which was the primary outcome (Dorman 1997; Gates 2009). One trial recorded retention at two time points without stating which was the primary outcome (Cox 2008). We used data for response/retention after the first contact with respondents as the primary outcome for analyses. One trial reported response at three time points (4 weeks, 12 weeks and 9 months), which were all stated as the primary outcome (Nakash 2007). We used the data for week four in our main analysis.

Five trials reported data in survival curves. For these, we used the final analysis point (Ashby 2011; Bowen 2000abc; Chaffin 2009; Land 2007; Sutherland 1996). Authors confirmed data when it had been extracted. Fifteen trials reported the number of questionnaires returned with no time point specified (Bauer 2004ab; Couper 2007; Hughes 1989; Kenton 2007a-d; Kenyon 2005; Leigh Brown 1997; Letley 2000; MacLennan; Man 2011; Marson 2007; McColl 2003 1; McColl 2003 2; Renfroe 2002a-d; Subar 2001; Tai 1997).

Addition of incentive versus none

There were 14 retention trials of incentives and 19 trial comparisons (Table 1). Thirteen trials were aimed at improving questionnaire response in trials and one trial was aimed at improving return for follow-up at trial site (Bowen 2000abc). The different incentive strategies aimed at improving questionnaire response were vouchers, cash, a charity donation, entry to prize draws, cheques, a certificate of appreciation and offers of study results. Incentive strategies aimed at improving retention were: certificates of appreciation and lapel pins. The value of incentives used in UK evaluations ranged from GBP5 to GBP20 and were in cash, cheque or voucher format. The value of incentives used in US-based studies was USD2 to USD10. For offers of entries into prize draws, the values were higher, ranging from GBP25 to GBP250 for UK prize draws and USD50 for US-based prize draws. One trial evaluated giving a monetary incentive with a promise of a further incentive for return of trial data (Bailey 2).

Table 1.

Trials evaluating incentive strategies

Trial or trial comparison	Incentive groups	Control group	Outcome type
Addition of incentive vs. none

Bauer 2004ab	a) USD10 cheque b) USD2 cheque Arms combined for main analysis	No incentive	DNA specimen kit return plus postal questionnaire response

Gates 2009	GBP5 voucher	No incentive	Postal questionnaire response

Kenyon 2005	GBP5 voucher	No incentive	Postal questionnaire response

Khadjesari 2011 1ac	a) Offer GBP5 voucher c) Offer of entry into GBP250 prize draw, groups combined for main analysis	No incentive	Internet-based questionnaire response

Khadjesari 2011 2	Offer of GBP10 Amazon.co.uk voucher	No incentive	Internet-based questionnaire response

Bowen 2000abc	a) Certificate b) Pin c) Pin and certificate groups combined for main analysis	No incentive	Participants retention

Renfroe 2002a	Certificate of appreciation	No certificate of appreciation	Postal questionnaire response

Sharp 2006a	Pen	No pen	Postal questionnaire response

Sharp 2006b	Pen	No pen	Postal questionnaire response

Sharp 2006c	Pen	No pen	Postal questionnaire response

Sharp 2006d	Pen	No pen	Postal questionnaire response

Cockayne 2005	Offer of study results	No offer	Postal questionnaire response

Hughes 1989	Offer of free reprint of results	No offer	Postal questionnaire response

Khadjesari 2011 1b	Offer of GBP5 charity donation	No offer	Internet-based questionnaire response

Addition of monetary incentive to both groups

Bailey 1 unpublished	Offer of GBP20 shopping voucher	Offer of GBP10 shopping voucher	Postal questionnaire response

Bailey 2 unpublished	Shopping voucher: GBP10 in advance and GBP10 on data return	Shopping voucher: GBP5 in advance and GBP5 on data return	Questionnaire response and chlamydia kit return

Addition of monetary incentive vs. offer of incentive

Kenton 2007a	USD2 coin	Draw for USD50 gift voucher	Postal questionnaire response

Kenton 2007b	USD2 coin	Draw for USD50 gift voucher	Postal questionnaire response

Offer of prize draw vs. no offer

Leigh Brown 1997	Aware of monthly prize draw of GBP25 gift voucher	No offer of draw	Postal questionnaire response

Trial or trial comparison	Communication strategy	Control arm	Outcome type
Enhanced letter vs. standard letter

Renfroe 2002c	Cover letter signed by physician	Cover letter signed by co-ordinator	Postal questionnaire response

Marson 2007	Letter explaining the approximate length of time to complete questionnaire	Standard letter	Postal questionnaire response

Total design method vs. customary method

Sutherland 1996	Total design method for postal follow-up	Standard method for postal follow-up	Postal questionnaire response

Priority vs. regular post

Renfroe 2002b	Express delivery	Standard delivery	Postal questionnaire response

Sharp 2006e	Despatch first-class stamp	Despatch second-class stamp	Postal questionnaire response

Sharp 2006f	Despatch first-class stamp	Despatch second-class stamp	Postal questionnaire response

Sharp 2006g	Second-class return envelope	Free post return envelope	Postal questionnaire response

Sharp 2006h	Second-class return envelope	Free post return envelope	Postal questionnaire response

Kenton 2007c	Priority mail	Standard mail	Postal questionnaire response

Kenton 2007d	Priority mail	Standard mail	Postal questionnaire response

Additional reminder vs. usual follow-up

Ashby 2011	Electronic reminder	No electronic reminder	Postal questionnaire response

MacLennan unpublished	Telephone reminder	No telephone reminder	Postal questionnaire response

Nakash 2007	Trial calendar given at recruitment with questionnaire due dates	No calendar	Postal questionnaire response

Severi 2011 1	Text message and fridge magnet both emphasising social benefits of study participation.	Text message reminder sent 3 days after questionnaire	Postal questionnaire response

Severi 2011 2	Telephone reminder from principle investigator	Standard procedures.	Return of cotinine samples

Man 2011	SMS text message as follow-up questionnaire sent out	No SMS text message	Postal questionnaire response

Additional trial site reminder vs. usual reminder

Land 2007	Prospective monthly reminder of upcoming assessments to trial sites	No extra reminder to trial sites	Postal questionnaire response

Early vs. late administration of questionnaire

Renfroe 2002d	Questionnaire sent 2-3 weeks after last AVID follow-up visit	Questionnaire sent 1-4 months after last AVID follow-up visit	Postal questionnaire response

Recorded delivery vs. telephone reminder

Tai 1997	Recorded delivery reminder	Telephone reminder	Postal questionnaire response

Addition telephone follow-up vs. incentive

Couper 2007	Telephone survey by trained interviewer	Postal questionnaire and USD5 bill	Post and questionnaire response

Trial or trial comparison	Questionnaire strategy	Control arm	Outcome type
Short vs. long

Dorman 1997	Short EuroQol	Long SF-36 questionnaire.	Postal questionnaire response

Edwards 2001 unpublished	1-page, 7-question functional dependence questionnaire	3-page, 16-question functional dependence questionnaire	Postal questionnaire response

Svoboda 2001 unpublished	1-page, 7-question functional dependence questionnaire	3-page, 16-question functional dependence questionnaire	Postal questionnaire response

McCambridge 2011 1b	AUDIT Short + LDQ	APQ	Internet-based questionnaire response

McCambridge 2011 2b	AUDIT Short + LDQ	APQ	Internet-based questionnaire response

Long and clear vs. short and condensed

Subar 2001	DHQ (36-page food frequency questionnaire)	PLCO (16-page food frequency questionnaire)	Postal questionnaire response and onsite completion

Question order

McColl 2003 1	Asthma condition-specific questions first followed by generic	Generic questions followed by condition specific	Postal questionnaire response

McColl 2003 2	Angina condition-specific questions followed by generic	Generic questions followed by condition specific	Postal questionnaire response

Letley 2000 unpublished	RDQ at front and SF-36 at back	SF-36 at front RDQ at back	Postal questionnaire response

Relevance of questionnaire

McCambridge 2011 1a	APQ23 items	CORE-OM Mental health assessment 23/34 items	Internet-based questionnaire response

McCambridge 2011 2a	AUDIT Short + LDQ	CORE-OM Mental health assessment 10 items	Internet-based questionnaire response

Example of proportion of questionnaires returned in control arm			30%	40%	50%	60%	70%	80%	90%
Strategy to improve retention	RR	¹ _RR

Addition of monetary incentive versus none	1.18	0.847	107	92	76	61	5	3	2

Addition of offer of monetary incentive/prize draw versus none	1.25	0.800	140	120	100	80	60	40	20

Addition of higher value monetary incentive versus addition of lower amount	1.12	0.890	77	66	55	44	33	22	11

Clinical area main trial	Condition	Participants	Setting	Attrition Study
Treatment of dependence	Alcohol	Adults scoring +5 on AUDIT-C, mean age 37 years in an online trial comparing interactive computer intervention plus website information vs. website information for modifying alcohol intake Murray 2007	Community: online	Khadjesari 2011 1abc

	Alcohol	Adults scoring +5 on AUDIT-C mean age 37 years in an online trial comparing interactive computer intervention plus website information vs. website information for modifying alcohol intake Murray 2007	Community: online	Khadjesari 2011 2

	Alcohol	Adults scoring +5 on AUDIT-C mean age 37 years in an online trial comparing interactive computer intervention plus website information vs. website information for modifying alcohol intake Murray 2007	Community: online	McCambridge 2011 1

	Alcohol	Adults scoring +5 on AUDIT-C mean age 37 years in an online trial comparing interactive computer intervention plus website information vs. website information for modifying alcohol intake Murray 2007	Community: online	McCambridge 2011 2

	Smoking	Adult smokers aged 38-77 years in a smoking cessation trial of public education through media and community wide events, healthcare providers work sites and other organisations vs. no intervention Gail 1992	USA community	Bauer 2004ab

	Smoking	Adult smokers mean age 36.7 years in a trial of nicotine gum vs. placebo gum. Smokers for = 1 year Hughes 1984	USA community	Hughes 1989

	Smoking	Adult smokers willing to quit aged = 16 years in a trial comparing Txt2stop motivational messages and behaviour change support vs. text messages unrelated to quitting Free 2011	UK community	Severi 2011 1

	Smoking	Adult smokers willing to quit aged = 16 years in a trial comparing Txt2stop motivational messages and behaviour change support vs. text messages unrelated to quitting Free 2011	UK community	Severi 2011 2

Treatment of injury	Neck	MINT trial: adults with whiplash injury 18-87 years in a 2 x 2 cluster randomised trial comparing whiplash book vs. usual advice. Individuals randomised to physiotherapy vs. single advice session reinforcing advice given Lamb 2007	UK hospital trusts	Gates 2009

	Ankle	Cast trial: adults aged 16-57 years with acute severe ankle sprain in a trial comparing tubular bandage vs. below knee cast vs. Aircast® ankle brace vs. Bledsoe® boot Cooke 2009	UK accident and emergency departments	Nakash 2007*

	Head	Adults with head injury aged = 16 years in trial of 48-hour infusion of methylprednisolone vs. placebo CRASH Trial collaborators 2004	UK hospital intensive care units	Edwards 2001*

	Head	Adults with head injury = 16 years CRASH trial: 48-hour infusion of methylprednisolone vs. placebo CRASH Trial collaborators 2004	Czech republic hospital intensive care units	Svoboda 2001*

Treatment of disease	Breast cancer	Women with ductal carcinoma in situ aged = 49 years in a trial comparing anastrozole vs. tamoxifen (unpublished)	USA, Canada, Puerto Rico hospitals	Land 2007

	Stroke	Acute stroke patients 50-80 years in an international stroke trial of heparin 125,000 IU twice daily + aspirin 300 mg daily vs. heparin 125,000 IU twice vs. heparin 5000 IU twice daily + aspirin 300 mg daily, heparin 5000 IU twice daily vs. aspirin 300 mg daily vs. no heparin or aspirin International Stroke Trial Group 1997	UK hospital	Dorman 1997

	Ventricular fibrillation, ventricular tachycardia	Adults cardioverted from ventricular tachycardia or resuscitated from ventricular fibrillation aged 54-76 years participating in the AVID trial comparing an implanted cardioverter defibrillator vs. antiarrhythmic drugs AVID investigators 1997	USA hospital	Renfroe 2002a-d

	Epilepsy	Adults with epilepsy mean 38.3 years in the SANAD trial. Arm a: carbamazepine vs. gabapentin vs. lamotrigine vs. oxcarbazepine vs. topiramate. Arm b: valproate vs. lamotrigine (LTG) vs. vs topiramate (TPM) Marson 2007 (2)	UK hospital outpatient departments	Marson 2007†

	Back pain	Adults with low back pain aged 18-65 years in a trial comparing exercise manipulation vs. exercise plus manipulation UK BEAM 2004	UK primary care	Letley 2000‡

	Back pain	Adults aged 18-65 years in a trial comparing yoga vs. usual care Tilbrook 2011	UK primary care	Man 2011

Screening	Prostate, lung, ovarian, colorectal cancer	Adults aged 55-74 years in PLCO trial comparing PSA and CA125 at baseline, and annually for 5 years. Digital rectal examination, transvaginal ultrasound and chest x-ray at baseline and 5 years vs. usual follow-up Prorok 2000	USA trial sites	Subar 2001, Ford 2006 (African American men aged ≥ 55 years only from PLCO)

	Cervical	Women with low-grade abnormal cervical smear aged 20-59 years in the TOMBOLA trial: Colposcopy vs. 6-monthly smears TOMBOLA 2009a TOMBOLA 2009b	UK	Sharp 2006a-h

	Postnatal depression	Women childbearing aged = 18 years < 2 weeks' postpartum at high risk of postnatal depression in a trial comparing proactive individualised telephone-based peer support vs. standard postpartum care Dennis 2009	Canada community	Kenton 2007a-d

Prevention	Fracture	Adults with history of osteoporotic fracture = 70 years in the RECORD trial: oral calcium + vitamin D vs. oral calcium vs. vitamin D vs. placebo RECORD 2007	UK hospital	MacLennan*

	Fracture	Adults with history of osteoporotic fracture aged = 70 years in the RECORD trial: oral calcium + vitamin D vs. oral calcium vs. vitamin D vs. placebo RECORD 2007	UK hospital	Avenell 2004

	Fracture	Women with hip fracture risk factors aged = 70 years in a fracture prevention trial of calcium 1000 mg plus vitamin D₃ 800 IU plus information sheet on dietary calcium intake and falls prevention vs. information sheet Porterhouse 2005	UK primary care	Cockayne 2005

	Migraine	Adults history of 2 migraine attacks aged 18-65 years with migraine randomised to true diet vs. sham diet (unpublished)	UK community	Ashby 2011

	Lung cancer	Adults exposed to smoking and asbestos aged = 45 years in the CARET trial 2 x 2: beta-carotene + retinol daily vs. beta-carotene vs. retinol vs. placebo Omenn 2006	USA trial sites	Bowen 2000abc

	Breast cancer	Women with 50% of breast volume dysplasia = 30 years in Canadian diet and cancer prevention trial. Counselling and individualised dietary prescription vs. taught principals of a healthy diet not counselled to change fat content Boyd 2002	Canada Hospital clinic	Sutherland 1996

Clinical management	Asthma	Adult with asthma = 70 years in COGENT trial: computerised decision support guidelines for asthma vs. angina care Eccles 2002	UK primary care	McColl 2003

	Asthma and diabetes	Adults with asthma mean age 47 years. Study template for diabetes vs. study template for asthma Tai 1999	UK primary care	Tai 1997

	Angina	Adult with asthma aged = 70 years in the COGENT trial: computerised decision support guidelines for asthma vs angina care Eccles 2002	UK primary care	McColl 2003

	Orthopaedics	Adults with non-surgical musculoskeletal condition ≥ 18 years in OMENS trial: orthopaedic patients management from single musculoskeletal medicine physician vs. orthopaedic surgeon-led management Leigh Brown 1997	UK Hospital othhopaedic outpatient department	Leigh Brown 1997

Other areas	Exercise	Women sedentary 50-70 years. SWEAT2 trial: moderate walking programme vs. swimming programme Cox 2008	Australia community	Cox 2008

	Parenting	Adults referred for parenting mean age 29 years. Parent-child interactive therapy vs. standard didactic parenting condition Chaffin 2009	USA community	Chaffin 2009

	Weight management	Adults with BMI = 25 aged = 18 years. Internet-based tailored weight management materials vs. Internet-based non-tailored user navigated weight management materials Rothert 2006	USA community	Couper 2007

	Effect of antibiotics on neonatal outcomes	Women < 37 weeks' gestation in ORACLE 1 + 2 trial: 2 x 2 factorial: co-amoxiclav + erythromycin vs. co-amoxiclav vs. erythromycin vs. placebo; 4 times daily x 10 days or until birth Kenyon 2005	UK secondary care/community	Kenyon 2005

	Health promotion	People aged 16-20 years in the sex unzipped pilot feasibility trial: interactive intervention website vs. information only website (main trial unpublished)	UK on line	Bailey 1*

	Health promotion	People aged 16-20 years in the sex unzipped pilot feasibility trial: interactive intervention website vs. information only website (main trial unpublished)	UK on line	Bailey 2*

Methods	2-arm randomised trial

Data	Adults aged 18-65 years who provided email or mobile telephone contact details for receiving electronic reminders for follow-up in a migraine prevention trial

Comparisons	1. Electronic reminder: either SMS text message, email message, or both sent after the 4-week follow-up study questionnaire sent 2. No electronic reminder sent

Outcomes	Primary: questionnaire response rate defined as proportion of questionnaires returned by participants at final analysis at 40 days Secondary: time to response

Notes	Retention trial embedded in a randomised trial evaluating the effectiveness of food elimination diet based on the ELISA test for food sensitivity for prevention of migraine. Primary outcome for the migraine prevention trial (host trial): change in the number of headache days over 12 weeks using the migraine disability assessment questionnaire (MIDAS). Retention trial identified through mail out to UK clinical trials units

*Risk of bias*

Item	Authors' judgement	Description

Allocation concealment?	Yes	Authors response "an independent data manager at the trials unit was responsible for generating the allocation sequence and assigning participants"

Adequate sequence generation?	Unclear	"Randomly generated numbers used to list all participants by ID [identification] number who had provided a mobile phone number and/or and email address. The first half of listed participants were allocated to the intervention group the remaining participants were allocated to the control group"

Blinding?	Unclear	No reference to blinding of either participants or outcome assessors in the study report

Free of selective outcome reporting?	Yes	All defined outcomes reported

Other sources of bias?	Unclear	Insufficient information to make a judgement

Methods	Randomised trial

Data	Adults aged ≥ 70 years with a low trauma osteoporotic fracture in the past 10 years recruited in 1 centre of the RECORD trial

Comparisons	1. An open version of the RECORD trial otherwise identical in design 2. RECORD trial, a randomised double-blind placebo-controlled factorial design of oral calcium 1 g daily and or vitamin D 800 IU/20 µg supplementation

Outcomes	Proportion of eligible participants recruited Proportion remaining in the trial at 1 year Proportion compliant on pill counts at 8 months

Notes	Open version of the Randomised Evaluation of Calcium or Vitamin D (RECORD) Trial treated as the intervention group in the analysis. Proportion retained at 4, 8 and 12 months were reported. Primary outcome for the randomised double-blind placebo-controlled version of the RECORD trial was all new low-energy fractures

*Risk of bias*

Item	Authors' judgement	Description

Allocation concealment?	Yes	"Nurse used a pre programed laptop computer"

Adequate sequence generation?	Yes	"A pre programed laptop computer to generate random allocation"

Blinding?	Yes	Double-blind randomised trial design compared with an open trial design. For the double-blind randomised trial "allocation remained concealed until the final analyses". "All outcomes were reported or verified by people who were masked to the allocation scheme"

Free of selective outcome reporting?	Yes	All defined outcomes reported

Other sources of bias?	Unclear	Insufficient information to make a judgement

Methods	Randomised trial embedded in the sex unzipped website feasibility trial

Data	UK English speaking people aged 16-20 years

Comparisons	1. Offer of GBP20 voucher 2. Offer of GBP10 voucher

Outcomes	Retention of participants at 3-month follow-up, i.e. completion of sexual health survey

Notes	Retention trial identified through personal correspondence with the author Sexunzipped website evaluated in an online trial

*Risk of bias*

Item	Authors' judgement	Description

Allocation concealment?	Yes	Authors response "the trial statisticians generated the randomisation sequence with participants identified by ID [identification] number only and the trial manager implemented it manually"

Adequate sequence generation?	Yes	Authors response "participants were randomised after recruitment but before follow-up to a GBP10 or GBP20 incentive. Randomisation to increased incentive was through simple permutation of the list of remaining recruits"

Blinding?	Unclear	Authors response "allocation sequences were generated without participants' knowledge". "For those allocated to the increased amount of GBP20, this was revealed in a 3 month follow-up email. Those allocated to GBP10 were not aware that others were offered GBP20 (unless friends had enrolled and had discussed the study). Since the trial recruited participants online from all over the UK, this will have reduced the chance of bias due to contamination"

Free of selective outcome reporting?	Yes	Reports the primary outcome

Methods	3-arm randomised trial (first incentive vs. no incentive)

Data	Pilot study of 15 randomly selected participants from each of 20 communities participating in COMMIT trial

Comparisons	Enclosed with mouthwash swish collection kits sent to participants "subjects were further randomised to receive" either: 1. USD10 cheque told to keep the cheque whether or not they participated, sent with covering letter and prepaid envelope 2. No incentive, kit sent with covering letter and prepaid envelope All sent 2 weeks after an advance letter with a professionally rendered brochure

Outcomes	Percentage of mouthwash kits returned reported. No time point given

Notes	Study embedded in the cluster randomised Community Intervention Trial for Smoking Cessation (COMMIT) trial. Primary outcome: quit rate among heavy smokers. Several attempts to contact authors regarding allocation sequence. Data extracted from Edwards Cochrane review on response to postal questionnaires

*Risk of bias*

Item	Authors' judgement	Description

Allocation concealment?	Unclear	No reply from author

Adequate sequence generation?	Unclear	"Subjects were further randomised to receive an incentive of….."

Blinding?	Unclear	Insufficient information to make a judgement about this, no reply from author

Free of selective outcome reporting?	Unclear	No reply from author

Other sources of bias?	Unclear	Insufficient information to make a judgement

Methods	3-arm randomised trial (combined incentive vs. no incentive)

Data	Pilot study of 15 randomly selected participants from each of 20 communities participating in COMMIT trial

Comparisons	Enclosed with mouthwash swish collection kits sent to participants "subjects were further randomised to receive" either: 1. USD10 cheque told to keep the cheque whether or not they participated sent with covering letter and prepaid envelope or 2. USD2 cheque with covering letter and prepaid envelope or 3. No incentive, kit sent with covering letter and prepaid envelope All sent 2 weeks after an advance letter with a professionally rendered brochure

Outcomes	Percentage of mouthwash kits returned reported. No time point given

Notes	Study embedded in the cluster randomised Community Intervention Trial for Smoking Cessation (COMMIT) trial. Primary outcome: quit rate among heavy smokers. Several attempts to contact authors regarding allocation sequence. Data extracted from Edwards Cochrane review on response to postal questionnaires

*Risk of bias*

Item	Authors' judgement	Description

Allocation concealment?	Unclear	No reply from author

Adequate sequence generation?	Unclear	"Subjects were further randomised to receive an incentive of….."

Blinding?	Unclear	Insufficient information to make a judgement about this no reply from author

Free of selective outcome reporting?	Unclear	No reply from author

Other sources of bias?	Unclear	Insufficient information to make a judgement

Methods	3-arm randomised trial (second incentive vs. no incentive)

Data	Pilot study of 15 randomly selected participants from each of 20 communities participating in COMMIT trial

Comparisons	Enclosed with mouthwash swish collection kits sent to participants, "subjects were further randomised to receive" either: 1. USD2 check with covering letter and prepaid envelope 2. No incentive, kit sent with covering letter and prepaid envelope All sent 2 weeks after an advance letter with a professionally rendered brochure

Outcomes	Percentage of mouthwash kits returned reported. No time point given

Notes	Study embedded in the cluster randomised Community Intervention Trial for Smoking Cessation (COMMIT) trial. Primary outcome: quit rate among heavy smokers. Several attempts to contact authors regarding allocation sequence. Data extracted from Edwards Cochrane review on response to postal questionnaires

*Risk of bias*

Item	Authors' judgement	Description

Allocation concealment?	Unclear	No reply from author

Adequate sequence generation?	Unclear	"Subjects were further randomised to receive an incentive of….."

Blinding?	Unclear	Insufficient information to make a judgement about this

Free of selective outcome reporting?	Unclear	No reply from author

Other sources of bias?	Unclear	Insufficient information to make a judgement

Methods	Randomised 2 x 2 factorial trial (first incentive vs. no incentive)

Data	Adults aged 50-69 years recruited from 2 CARET trial sites, participating in the Participant Retention Item Distribution Evaluation (PRIDE) trial

Comparisons	1. Certificate of appreciation preprinted on an 8.5 x 11 inch (21.59 x 27.94 cm)bond off-white paper with gold trim and bold, black lettering. The certificate read "[participant's name] in recognition of your contribution to an important national study for the prevention of lung cancer, CARET, CancerPrevention Study, sponsored by The National Cancer Institute". The participant's name was computer printed in an attractive font on the certificate. Each certificate had the signatures of the Co-ordinating Center's principal investigator, study centre investigator and CARET's project officer from the National Cancer Institute, given during a visit for randomisation or follow-up 2. No incentive

Outcomes	Primary: time of first inactivation (stop taking vitamins or placebos) during the 2-year follow-up period of PRIDE

Notes	Primary disease outcome for the Carotene and Retinol Efficacy Trial (CARET): lung cancer

*Risk of bias*

Item	Authors' judgement	Description

Allocation concealment?	Unclear	Author response "the Coordination Center supplied one date-labelled envelope for each day in the enrolment period to the two participating study centres. The envelopes contained the randomisation assignment for the given day. At the end of each day during the randomisation period, study centre staff opened the envelope containing the intervention assignment for the next day. Study centre staff members were otherwise blinded to the allocation sequence"

Adequate sequence generation?	Yes	Author response "we used a block randomisation approach (stratified by study centre) with a 1:1:1:1 intervention arm allocation ratio, where the randomisation unit was the date of enrolment". Note: treated this as quasi-randomised in the analysis

Blinding?	Yes	Author response "with IRB [Institutional Review Board] approval, the study was conducted without participants' knowledge of this research. Thus, participants were blinded to their own intervention only in the sense that they were unaware they were randomised to receive particular item(s)"

Free of selective outcome reporting?	Yes	Cumulative incidence of individuals who became inactive during 2-year follow-up reported

Other sources of bias?	Unclear	Insufficient information to make a judgement

Methods	Randomised 2 x 2 factorial trial (combined incentive vs. no incentive)

Data	Adults aged 50-69 years recruited from 2 CARET trial sites, participating in the Participant Retention Item Distribution Evaluation (PRIDE) trial

Comparisons	1. Certificate of appreciation preprinted on an 8.5 x 11 inch (21.59 x 27.94 cm)bond off-white paper with gold trim and bold, black lettering. The certificate read '[participant's name] in recognition of your contribution to an important national study for the prevention of lung cancer, CARET, CancerPrevention Study, sponsored by The National Cancer Institute'. The participant's name was computer printed in an attractive font on the certificate. Each certificate had the signatures of the Co-ordinating Center's principal investigator, study centre investigator, and CARET's project officer from the National Cancer Institute, given during a visit for randomisation or follow-up (arm a) 2. Lapel pin 1 inch (2.5 cm) in size and designed in cloisonne. Choice between a pin with 6 colours with inscription 'CARET NCI prevention study' and an orange carrot in the middle of the pin or a pin with 5 colours with inscription 'PARTICIPNAT CARET Cancer Prevention Study' and 'Sponsored by NCI' and given during a visit for randomisation or follow-up (arm b) 3. Certificate of appreciation (details as before) and lapel pin (details as before) and given during a visit for randomisation or follow-up (arm c) 4. No incentive

Outcomes	Primary: time of first inactivation (stop taking vitamins or placebos) during the 2-year follow-up period of PRIDE

Notes	Primary disease outcome for the Carotene and Retinol Efficacy Trial (CARET): lung cancer

*Risk of bias*

Item	Authors' judgement	Description

Allocation concealment?	Unclear	Author response "the Coordination Center supplied one date-labelled envelope for each day in the enrolment period to the two participating study centres. The envelopes contained the randomisation assignment for the given day. At the end of each day during the randomisation period, study centre staff opened the envelope containing the intervention assignment for the next day. Study centre staff members were otherwise blinded to the allocation sequence"

Adequate sequence generation?	Yes	Author response "we used a block randomisation approach (stratified by study centre) with a 1:1:1:1 intervention arm allocation ratio, where the randomisation unit was the date of enrolment"

Blinding?	Yes	Author response "with IRB [Institutional Review Board] approval, the study was conducted without participants’ knowledge of this research. Thus, participants were blinded to their own intervention only in the sense that they were unaware they were randomised to receive particular item(s)"

Free of selective outcome reporting?	Yes	Cumulative incidence of individuals who became inactive during 2 year follow-up reported

Other sources of bias?	Unclear	Insufficient information to make a judgement

Methods	Randomised 2 x 2 factorial trial (second incentive vs. no incentive)

Data	Adults aged 50-69 years recruited from 2 CARET trial sites, participating in the Participant Retention Item Distribution Evaluation (PRIDE) trial

Comparisons	1. Lapel pin 1 inch (2.5 cm) in size and designed in cloisonne. Choice between a pin with 6 colours with inscription 'CARET NCI prevention study' and an orange carrot in the middle of the pin or a pin with 5 colours with inscription 'PARTICIPNAT CARET Cancer Prevention Study' and "Sponsored by NCI" and given during a visit for randomisation or follow-up 2. No incentive

Outcomes	Primary: time of first inactivation (stop taking vitamins or placebos) during the 2-year follow-up period of PRIDE

Notes	Primary disease outcome for the Carotene and Retinol Efficacy Trial (CARET): lung cancer

*Risk of bias*

Item	Authors' judgement	Description

Allocation concealment?	Unclear	Author response "the Coordination Center supplied one date-labelled envelope for each day in the enrolment period to the two participating study centres. The envelopes contained the randomisation assignment for the given day. At the end of each day during the randomisation period, study centre staff opened the envelop containing the intervention assignment for the next day. Study centre staff members were otherwise blinded to the allocation sequence"

Adequate sequence generation?	Yes	Author response "we used a block randomisation approach (stratified by study centre) with a 1:1:1:1 intervention arm allocation ratio, where the randomisation unit was the date of enrolment"

Blinding?	Yes	Author response "with IRB [Institutional Review Board] approval, the study was conducted without participants’ knowledge of this research. Thus, participants were blinded to their own intervention only in the sense that they were unaware they were randomised to receive particular item(s)"

Free of selective outcome reporting?	Yes	Cumulative incidence of individuals who became inactive during 2-year follow-up reported

Other sources of bias?	Unclear	Insufficient information to make a judgement

Methods	Randomised 2 x 2 factorial trial (third incentive vs. no incentive)

Data	Adults aged 50-69 years recruited from 2 CARET trial sites, participating in the Participant Retention Item Distribution Evaluation (PRIDE) trial

Comparisons	1. Certificate of appreciation (details as before) and lapel pin (details as before) and given during a visit for randomisation or follow-up 2. No incentive

Outcomes	Primary: time of first inactivation (stop taking vitamins or placebos) during the 2-year follow-up period of PRIDE

Notes	Primary disease outcome for the Carotene and Retinol Efficacy Trial (CARET): lung cancer

*Risk of bias*

Item	Authors' judgement	Description

Allocation concealment?	Unclear	Author response "the Coordination Center supplied one date-labelled envelope for each day in the enrolment period to the two participating study centres. The envelopes contained the randomisation assignment for the given day. At the end of each day during the randomisation period, study centre staff opened the envelop containing the intervention assignment for the next day. Study centre staff members were otherwise blinded to the allocation sequence"

Adequate sequence generation?	Yes	Author response "we used a block randomisation approach (stratified by study centre) with a 1:1:1:1 intervention arm allocation ratio, where the randomisation unit was the date of enrolment"

Blinding?	Yes	Author response "with IRB [Institutional Review Board] approval, the study was conducted without participants’ knowledge of this research. Thus, participants were blinded to their own intervention only in the sense that they were unaware they were randomised to receive particular item(s)"

Free of selective outcome reporting?	Yes	Cumulative incidence of individuals who became inactive during 2-year follow-up reported

Other sources of bias?	Unclear	Insufficient information to make a judgement

Methods	2-arm randomised trial (incentive plus postal questionnaire vs. telephone survey)

Data	Adults ≥ 18 years, BMI of ≥ 25 participating in an Internet-based weight management trial who did not respond to the 12-month questionnaire

Comparisons	1. Telephone call and survey by trained interviewers. Repeated up to 15 times. Attempts made on various days and at various times of the day 2. Postal questionnaire with return address and covering letter signed by directors plus a USD5 bill

Outcomes	Questionnaire response. No time point given

Notes	Internet-based weight management trial compared Internet-based tailored weight management materials with Internet-based non-tailored user navigated weight management materials. Primary outcome for the Internet-based trial: percentage of baseline weight lost

*Risk of bias*

Item	Authors' judgement	Description

Allocation concealment?	Unclear	Not described

Adequate sequence generation?	Unclear	Author response "randomly assigned"

Blinding?	Unclear	Not described

Free of selective outcome reporting?	Unclear	Unclear at the outset what is to be reported

Other sources of bias?	Unclear	Insufficient information to make a judgement

Methods	2-arm quasi-randomised trial

Data	African American men aged 55 years enrolled in the intervention screening arm of the PLCO trial

Comparisons	1. Indepth case management. Case management monthly telephone calls to participants, assisted making medical appointments, helped participants obtain health insurance information, legal aid, transportation services, food programmes, financial support, medication assistance, free medical care, information related to health risks facing African Americans. Provision of PLCO Cancer Screening Trial screening information and the scheduling of annual screening appointments. 2. Regular trial screening procedures. Participants called annually to schedule screening examinations

Outcomes	Number completing the next scheduled PLCO cancer screen at 3 years

Notes	PLCO trial compared digital rectal examination, transvaginal ultrasound and chest x-ray at baseline and 5 years to usual follow-up. PSA and cancer antigen CA125 at baseline, and annually for 5 years. Primary outcome mortality from 4 PLCO cancers

*Risk of bias*

Item	Authors' judgement	Description

Allocation concealment?	Unclear	Not described

Adequate sequence generation?	No	"Randomised by participant id number"

Blinding?	Unclear	Insufficient information to make a judgement about this

Free of selective outcome reporting?	Yes	All defined outcomes reported

Other sources of bias?	Unclear	Insufficient information to make a judgement on this

Methods	Randomised 2 x 2 factorial trial (incentive vs. offer of incentive)

Data	New mothers with Edinburgh postnatal depression scale score = 9 participating in postpartum depression peer support trial

Comparisons	1. USD2 coin mailed with questionnaire 2. Offer of entry into prize draw for USD50 gift certificate

Outcomes	Questionnaire response. No time point given

Notes	The postpartum depression peer support trial enrolled women < 2 weeks' postpartum at high risk for postnatal depression. New proactive individualised telephone-based peer support with standard postpartum care. The primary outcome was postnatal depression measured using the Edinburgh postnatal depression scale and Structured Clinical Interview Depression (SCID)

*Risk of bias*

Item	Authors' judgement	Description

Allocation concealment?	Yes	Author response "allocation sequence stored in a password protected file only accessed by an external researcher, computer generated"

Adequate sequence generation?	Unclear	Author response "computer generated"

Blinding?	Yes	Author response "participants were not aware of the sub-study and didn't know other participants were receiving different types of mailings. The trial coordinator was not blinded to the study group"

Free of selective outcome reporting?	Yes	Defined outcomes reported

Other sources of bias?	Unclear	Insufficient information to make a judgement on this

Methods	Randomised 2 x 2 factorial trial

Data	New mothers with Edinburgh, postnatal depression scale score = 9 participating in postpartum depression peer support trial

Comparisons	1. USD2 coin mailed with questionnaire 2. Offer of entry into prize draw for USD50 gift certificate 3. USD2 coin with questionnaire sent with high-priority postage stamp 4. Offer of entry into lottery draw for USD50 gift certificate with high-priority postage stamp

Outcomes	Questionnaire response. No time point given

Notes	The postpartum depression peer support trial enrolled women < 2 weeks' postpartum at high risk for postnatal depression. New proactive individualised telephone-based peer support with standard postpartum care. The primary outcome was postnatal depression measured using the Edinburgh postnatal depression scale and Structured Clinical Interview Depression (SCID)

*Risk of bias*

Item	Authors' judgement	Description

Allocation concealment?	Yes	Author response "allocation sequence stored in a password protected file only accessed by an external researcher, computer generated"

Adequate sequence generation?	Unclear	Author response "computer generated"

Blinding?	Yes	Author response "participants were not aware of the sub-study and didn't know other participants were receiving different types of mailings. The trial coordinator was not blinded to the study group"

Free of selective outcome reporting?	Yes	Defined outcomes reported

Other sources of bias?	Unclear	Insufficient information to make a judgement on this

Methods	Randomised 2 x 2 factorial trial (priority mail vs. no priority mail)

Data	New mothers with Edinburgh postnatal depression scale score = 9 participating in postpartum depression peer support trial

Comparisons	1. USD2 coin sent by high-priority postage stamp plus questionnaire 2. USD2 coin plus questionnaire

Outcomes	Questionnaire response. No time point given

Notes	The postpartum depression peer support trial enrolled women < 2 week' postpartum at high risk for postnatal depression. New proactive individualised telephone-based peer support with standard postpartum care. The primary outcome was postnatal depression measured using the Edinburgh postnatal depression scale and Structured Clinical Interview Depression (SCID)

*Risk of bias*

Item	Authors' judgement	Description

Allocation concealment?	Yes	Author response "allocation sequence stored in a password protected file only accessed by an external researcher, computer generated"

Adequate sequence generation?	Unclear	Author response "computer generated"

Blinding?	Yes	Author response "participants were not aware of the sub-study and didn't know other participants were receiving different types of mailings. The trial coordinator was not blinded to the study group"

Free of selective outcome reporting?	Yes	Expected outcomes reported

Other sources of bias?	Unclear	Insufficient information to make a judgement on this

Methods	Randomised 2 x 2 factorial trial (offer of entry into prize draw plus high-priority postage stamp vs. offer of entry into prize draw)

Data	New mothers with Edinburgh postnatal depression scale score = 9 participating in postpartum depression peer support trial

Comparisons	1. Offer of entry into prize draw for USD50 gift certificate plus high-priority postage stamp 2. Offer of entry into prize draw for USD50 gift certificate

Outcomes	Questionnaire response. No time point given

Notes	The postpartum depression peer support trial enrolled women < 2 weeks' postpartum at high risk for postnatal depression. New proactive individualised telephone-based peer support with standard postpartum care. The primary outcome was postnatal depression measured using the Edinburgh postnatal depression scale and Structured Clinical Interview Depression (SCID)

*Risk of bias*

Item	Authors' judgement	Description

Allocation concealment?	Yes	Author response "allocation sequence stored in a password protected file only accessed by an external researcher, computer generated"

Adequate sequence generation?	Unclear	Author response "computer generated"

Blinding?	Yes	Author response "participants were not aware of the sub-study and didn't know other participants were receiving different types of mailings. The trial coordinator was not blinded to the study group"

Free of selective outcome reporting?	Yes	Expected outcomes reported

Other sources of bias?	Unclear	Insufficient information to make a judgement on this

PERMALINK

Strategies to improve retention in randomised trials

Valerie C Brueton

Jayne Tierney

Sally Stenning

Seeromanie Harding

Sarah Meredith

Irwin Nazareth

Greta Rait

Abstract

Background

Objectives

Search methods

Selection criteria

Data collection and analysis

Main results

Authors' conclusions

PLAIN LANGUAGE SUMMARY

Methods that might help to keep people in randomised trials

Background

Description of the problem or issue

Description of the methods being investigated

How these methods might work

Why it is important to do this review

Objectives

Methods

Criteria for considering studies for this review

Types of studies

Types of data

Types of methods

Types of outcome measures

Primary outcomes

Secondary outcomes

Search methods for identification of studies

Electronic searches

Searching other resources

Data collection and analysis

Selection of studies

Data extraction and management

Assessment of risk of bias in included studies

Measures of the effect of the methods

Unit of analysis issues

Dealing with missing data

Assessment of heterogeneity

Assessment of reporting biases

Data synthesis

Subgroup analysis and investigation of heterogeneity

(a) Incentives

(b) Communication

(c) Questionnaire structure

Sensitivity analysis

Results

Description of studies

Results of the search

Figure 1.

Included studies

Host trials

Participants and settings

Design of included retention trials

Strategies to improve retention

Outcome measures in the included trials

Addition of incentive versus none

Table 1.

Communication strategies

Table 2.

Questionnaire format

Table 3.

Behavioural strategies

Case management

Methodology strategies

Studies excluded from analyses

Excluded studies

Risk of bias in included studies

Allocation

Blinding

Incomplete outcome data

Selective reporting

Other potential sources of bias

Effect of methods

1. Incentive strategies

Methods	Khadjesari 1: 4-arm randomised trial Khadjesari 2: 2-arm randomised trial

Data	See 'Table of characteristics' for Khadjesari 2011 1abc; Khadjesari 2011 2

Comparisons	See 'Table of characteristics' for Khadjesari 2011 1abc; Khadjesari 2011 2

Outcomes	See 'Table of characteristics' for Khadjesari 2011 1abc; Khadjesari 2011 2

Notes	See 'Table of characteristics' for Khadjesari 2011 1abc; Khadjesari 2011 2

*Risk of bias*

Item	Authors' judgement	Description

Allocation concealment?	Yes	"Allocation was concealed", "randomisation could not be subverted by the study team"

Adequate sequence generation?	Yes	Computer-generated randomisation sequence

Blinding?	Unclear	Unclear

Free of selective outcome reporting?	Yes	All outcomes reported

Other sources of bias?	Unclear	Insufficient information to make a judgement about this

Methods	4-arm randomised trial (offer of incentives vs. no offer)

Data	Non-responders at 1 week to the pilot DYD trial 3-month online follow-up questionnaire. Adults aged ≥ 18 scoring 5 or more on the AUDIT-C test

Comparisons	1. Offer of GBP5 Amazon.co.uk voucher. Emailed voucher code on receipt of response (arm a) 2. Email with no offer of incentive (control)

Outcomes	Proportion completing questionnaire after 40 days

Notes	Primary outcome for DYD: total past week alcohol consumption measured using the TOT-AL measure of total alcohol consumption. Trial identified through personal communication

*Risk of bias*

Item	Authors' judgement	Description

Allocation concealment?	Yes	"Allocation was concealed randomisation could not be subverted by the study team"

Adequate sequence generation?	Yes	Computer-generated randomisation sequence

Blinding?	Unclear	Unclear

Free of selective outcome reporting?	Yes	Expected outcomes reported

Other sources of bias?	Unclear	Insufficient information to make a judgement about this

Methods	4-arm randomised trial

Data	Non-responders at 1 week to the pilot DYD trial 3-month online follow-up questionnaire. Adults aged ≥ 18 scoring 5 or more on the AUDIT-C test

Comparisons	1. Offer of GBP5 Amazon.co.uk voucher. Emailed voucher code on receipt of response (arm a) 2. Offer of GBP5 donation to Cancer Research UK. Emailed hyperlink to charity's website showing donation when response received (arm b) 3. Offer of entry into GBP250 prize draw emailed confirmation of entry when response received (arm c) 4. Email prompt for completion of questionnaires with no offer of incentive (control arm)

Outcomes	Proportion completing questionnaire after 40 days

Notes	Primary outcome for DYD: total past week alcohol consumption measured using the TOT-AL measure of total alcohol consumption. Trial identified through personal communication

*Risk of bias*

Item	Authors' judgement	Description

Allocation concealment?	Yes	"Allocation was concealed randomisation could not be subverted by the study team"

Adequate sequence generation?	Yes	Computer-generated randomisation sequence

Blinding?	Unclear	Unclear

Free of selective outcome reporting?	Yes	Expected outcomes reported

Other sources of bias?	Unclear	Insufficient information to make a judgement about this

Methods	4-arm randomised trial (incentives combined vs. no incentive)

Data	Non-responders at 1 week to the pilot DYD trial 3-month online follow-up questionnaire. Adults aged ≥ 18 scoring 5 or more on the AUDIT-C test

Comparisons	1. Offer of GBP5 Amazon.co.uk voucher. Emailed voucher code on receipt of response (arm a) 2.Offer of entry into GBP250 prize draw emailed confirmation of entry when response received (arm c) 3. Email with no offer of incentive (control arm)

Outcomes	Proportion completing questionnaire after 40 days

Notes	Primary outcome for DYD: total past week alcohol consumption measured using the TOT-AL measure of total alcohol consumption. Trial identified through personal communication Separate comparison for arm b of attrition trial see Khadjesari 2011 1b

*Risk of bias*

Item	Authors' judgement	Description

Allocation concealment?	Yes	"Allocation was concealed randomisation could not be subverted by the study team"

Adequate sequence generation?	Yes	Computer-generated randomisation sequence

Blinding?	Unclear	Unclear

Free of selective outcome reporting?	Yes	Expected outcomes reported

Other sources of bias?	Unclear	Insufficient information to make a judgement about this

Methods	4-arm randomised trial (offer of donation to charity vs. no offer)

Data	Non-responders at 1 week to the pilot DYD trial 3-month online follow-up questionnaire. Adults aged ≥ 18 scoring 5 or more on the AUDIT-C test

Comparisons	1. Offer of GBP5 donation to Cancer Research UK. Emailed hyperlink to charity's website showing donation when response received (arm b) 2. Email with no offer of incentive (control)

Outcomes	Proportion completing questionnaire after 40 days

Notes	Primary outcome for DYD: total past week alcohol consumption measured using the TOT-AL measure of total alcohol consumption. Trial identified through personal communication

*Risk of bias*

Item	Authors' judgement	Description

Allocation concealment?	Yes	"Allocation was concealed randomisation could not be subverted by the study team"

Adequate sequence generation?	Yes	Computer-generated randomisation sequence

Blinding?	Unclear	Unclear

Free of selective outcome reporting?	Yes	Expected outcomes reported

Other sources of bias?	Unclear	Insufficient information to make a judgement about this

Methods	4-arm randomised trial (offer of entry into prize draw vs. no offer)

Data	Non-responders at 1 week to the pilot DYD trial 3-month online follow-up questionnaire. Adults aged ≥ 18 scoring 5 or more on the AUDIT-C test

Comparisons	1. Offer of entry into GBP250 prize draw emailed confirmation of entry when response received (arm c) 2. Email with no offer of incentive (control arm)

Outcomes	Proportion completing questionnaire after 40 days

Notes	Primary outcome for DYD: total past week alcohol consumption measured using the TOT-AL measure of total alcohol consumption. Trial identified through personal communication

*Risk of bias*

Item	Authors' judgement	Description

Allocation concealment?	Yes	"Allocation was concealed randomisation could not be subverted by the study team"

Adequate sequence generation?	Yes	Computer-generated randomisation sequence

Blinding?	Unclear	Unclear

Free of selective outcome reporting?	Yes	Defined outcomes reported

Other sources of bias?	Unclear	Insufficient information to make a judgement about this

Methods	McCambridge 1: 4-arm randomised trial in the pilot phase of the DYD trial; McCambridge 2: 4-arm randomised trial in the host DYD trial

Data	See 'Table of characteristics' for McCambridge 2011 1; McCambridge 2011 1a; McCambridge 2011 1b; McCambridge 2011 2; McCambridge 2011 2a; McCambridge 2011 2b

Comparisons	See 'Table of characteristics' for McCambridge 2011 1; McCambridge 2011 1a; McCambridge 2011 1b; McCambridge 2011 2; McCambridge 2011 2a; McCambridge 2011 2b

Outcomes	See 'Table of characteristics' for McCambridge 2011 1; McCambridge 2011 1a; McCambridge 2011 1b; McCambridge 2011 2; McCambridge 2011 2a; McCambridge 2011 2b

Notes	See 'Table of characteristics' for McCambridge 2011 1; McCambridge 2011 1a; McCambridge 2011 1b; McCambridge 2011 2; McCambridge 2011 2a; McCambridge 2011 2b

*Risk of bias*

Item	Authors' judgement	Description

Allocation concealment?	Yes	See 'Risk of bias' table McCambridge 2011 1; McCambridge 2011 1a; McCambridge 2011 1b; McCambridge 2011 2; McCambridge 2011 2a; McCambridge 2011 2b

Adequate sequence generation?	Yes	See 'Risk of bias' table McCambridge 2011 1; McCambridge 2011 1a; McCambridge 2011 1b; McCambridge 2011 2; McCambridge 2011 2a; McCambridge 2011 2b

Blinding?	Unclear	See 'Risk of bias' table McCambridge 2011 1; McCambridge 2011 1a; McCambridge 2011 1b; McCambridge 2011 2; McCambridge 2011 2a; McCambridge 2011 2b

Free of selective outcome reporting?	Yes	See 'Risk of bias' table McCambridge 2011 1; McCambridge 2011 1a; McCambridge 2011 1b; McCambridge 2011 2; McCambridge 2011 2a; McCambridge 2011 2b

Other sources of bias?	Unclear	See 'Risk of bias' table McCambridge 2011 1; McCambridge 2011 1a; McCambridge 2011 1b; McCambridge 2011 2; McCambridge 2011 2a; McCambridge 2011 2b

Methods	4-arm randomised trial in the pilot phase of the DYD trial

Data	Adults aged ≥ 18 years scoring ≥ 5 on the AUDIT-C test participating in the pilot phase of the DYD trial 1-month data used

Comparisons	1. APQ 23 items 2. AUDIT 10 3. LDQ 10 4. CORE-OM 23/34 items (mental health assessment)

Outcomes	Response to electronic questionnaires

Notes	Primary outcome for DYD: total past week alcohol consumption measured using the TOT-AL measure of total alcohol consumption. Trial identified through personal communication

*Risk of bias*

Item	Authors' judgement	Description

Allocation concealment?	Yes	"Allocation was concealed randomisation could not be subverted by the study team"

Adequate sequence generation?	Yes	Computer-generated randomisation sequence

Blinding?	Unclear	Unclear

Free of selective outcome reporting?	Yes	Defined outcomes reported

Other sources of bias?	Unclear	Insufficient information to make a judgement about this

Methods	4-arm randomised trial ('relevance' of questionnaire: alcohol vs. mental health data)

Data	Adults aged ≥ 18 years scoring 5 or more on the AUDIT-C test participating in the pilot phase of the DYD trial 1-month data used

Comparisons	1. APQ 23 items 2. CORE-OM 23/34 items (Mental health assessment)

Outcomes	Response to electronic questionnaires

Notes	Primary outcome for DYD: total past week alcohol consumption measured using the TOT-AL measure of total alcohol consumption. Trial identified through personal communication

*Risk of bias*

Item	Authors' judgement	Description

Allocation concealment?	Yes	"Allocation was concealed randomisation could not be subverted by the study team"

Adequate sequence generation?	Yes	Computer-generated randomisation sequence

Blinding?	Unclear	Unclear

Free of selective outcome reporting?	Yes	Defined outcomes reported

Other sources of bias?	Unclear	Insufficient information to make a judgement about this

Methods	4-arm randomised trial (short vs. long questionnaire comparison)

Data	Adults aged ≥ 18 years scoring 5 or more on the AUDIT-C test participating in the DYD trial. For this comparison 1-month follow-up data from McCambridge 2011 1 used

Comparisons	1. AUDIT 10 items + LDQ 10 items 2. APQ 23 items

Outcomes	Response to electronic questionnaires

Notes	Primary outcome for DYD: total past week alcohol consumption measured using the TOT-AL measure of total alcohol consumption. Trial identified through personal communication

*Risk of bias*

Item	Authors' judgement	Description

Allocation concealment?	Yes	"Allocation was concealed randomisation could not be subverted by the study team"

Adequate sequence generation?	Yes	Computer-generated randomisation sequence

Blinding?	Unclear	Unclear

Free of selective outcome reporting?	Yes	Defined outcomes reported

Other sources of bias?	Unclear	Insufficient information to make a judgement about this