Strategies to improve retention in randomised trials

Katie Gillies; Anna Kearney; Ciara Keenan; Shaun Treweek; Jemma Hudson; Valerie C Brueton; Thomas Conway; Andrew Hunter; Louise Murphy; Peter J Carr; Greta Rait; Paul Manson; Magaly Aceves-Martins

doi:10.1002/14651858.MR000032.pub3

. 2021 Mar 6;2021(3):MR000032. doi: 10.1002/14651858.MR000032.pub3

Strategies to improve retention in randomised trials

Katie Gillies ^1,^✉, Anna Kearney ², Ciara Keenan ³, Shaun Treweek ¹, Jemma Hudson ¹, Valerie C Brueton ⁴, Thomas Conway ⁵, Andrew Hunter ⁶, Louise Murphy ⁶, Peter J Carr ⁶, Greta Rait ⁷, Paul Manson ⁸, Magaly Aceves-Martins ¹

Editor: Cochrane Methodology Review Group

PMCID: PMC8092429 PMID: 33675536

Abstract

Background

Poor retention of participants in randomised trials can lead to missing outcome data which can introduce bias and reduce study power, affecting the generalisability, validity and reliability of results. Many strategies are used to improve retention but few have been formally evaluated.

Objectives

To quantify the effect of strategies to improve retention of participants in randomised trials and to investigate if the effect varied by trial setting.

Search methods

We searched the Cochrane Central Register of Controlled Trials (CENTRAL), MEDLINE, Scopus, PsycINFO, CINAHL, Web of Science Core Collection (SCI‐expanded, SSCI, CPSI‐S, CPCI‐SSH and ESCI) either directly with a specified search strategy or indirectly through the ORRCA database. We also searched the SWAT repository to identify ongoing or recently completed retention trials. We did our most recent searches in January 2020.

Selection criteria

We included eligible randomised or quasi‐randomised trials of evaluations of strategies to increase retention that were embedded in 'host' randomised trials from all disease areas and healthcare settings. We excluded studies aiming to increase treatment compliance.

Data collection and analysis

We extracted data on: the retention strategy being evaluated; location of study; host trial setting; method of randomisation; numbers and proportions in each intervention and comparator group. We used a risk difference (RD) and 95% confidence interval (CI) to estimate the effectiveness of the strategies to improve retention. We assessed heterogeneity between trials. We applied GRADE to determine the certainty of the evidence within each comparison.

Main results

We identified 70 eligible papers that reported data from 81 retention trials. We included 69 studies with more than 100,000 participants in the final meta‐analyses, of which 67 studies evaluated interventions aimed at trial participants and two evaluated interventions aimed at trial staff involved in retention. All studies were in health care and most aimed to improve postal questionnaire response. Interventions were categorised into broad comparison groups: Data collection; Participants; Sites and site staff; Central study management; and Study design.

These intervention groups consisted of 52 comparisons, none of which were supported by high‐certainty evidence as determined by GRADE assessment. There were four comparisons presenting moderate‐certainty evidence, three supporting retention (self‐sampling kits, monetary reward together with reminder or prenotification and giving a pen at recruitment) and one reducing retention (inclusion of a diary with usual follow‐up compared to usual follow‐up alone). Of the remaining studies, 20 presented GRADE low‐certainty evidence and 28 presented very low‐certainty evidence.

Our findings do provide a priority list for future replication studies, especially with regard to comparisons that currently rely on a single study.

Authors' conclusions

Most of the interventions we identified aimed to improve retention in the form of postal questionnaire response. There were few evaluations of ways to improve participants returning to trial sites for trial follow‐up. None of the comparisons are supported by high‐certainty evidence. Comparisons in the review where the evidence certainty could be improved with the addition of well‐done studies should be the focus for future evaluations.

Plain language summary

Strategies that might help to encourage people to continue to participate in a randomised trial (a type of scientific study)

Why is this review important?

Randomised trials are a type of scientific study typically used to test new healthcare treatments. In a randomised trial, people who agree to take part are randomly (by chance) put into one of two or more treatment groups and then studied for a period of time. The research team try to keep in touch with them to collect information about how they are doing. This 'follow up' can last from days to years depending on the trial, but the longer the trial lasts, the more difficult it can be. This might be because people are too busy to reply, are unable to come to a clinic, or just do not want to participate any longer. Keeping people in a trial is called 'retention'. If retention is poor, it can make the trial results less certain but most trials do not get data from all the people who started out in the trial.

The information gathered during follow‐up, sometimes called data, helps the trial team to determine which of the treatments being tested works the best. Often this information is collected directly from patients by asking them to complete a questionnaire or by asking them to come back for a clinic visit.

There are many ways to collect data from people in trials. These include using letters, the internet, telephone calls, text messaging, face‐to‐face meetings or the return of medical test kits. Research teams use different methods to try to collect data and it's important to know which strategies are effective and worthwhile, which is why we did this review to compare the success of different strategies.

How did we identify and evaluate the evidence?

We searched scientific databases for studies that compared strategies that research teams use to improve trial retention against each other or against not using such a strategy.We looked for studies that included participants from any age, gender, ethnic, language or geographic group. We then compared the results of the studies, and summarised the evidence that we had found. Finally, we rated our confidence in this evidence, based on factors such as the methods used in the studies and their size, and the consistency of findings across studies.

What did we find?

We identified 70 relevant articles, which reported 81 retention studies involving more than 100,000 participants, that had investigated different ways of trying to encourage randomised trial participants to provide data and stay in the trial. We organised these into broad comparison groups but, unfortunately, we are not able to say with confidence that any of the results we found is a true effect and not caused by other factors, such as flaws with the design of the studies. As such, the effect of ways to encourage people to stay involved in trials is still not clear and more research is needed to see if these retention methods really do work.

How certain is the evidence and how up‐to‐date is this review?

The strategies we identified were tested in randomised trials run in many different disease areas and settings but, in some cases, were tested in only one trial. None of the comparisons we made provided high quality evidence and more studies are needed to help provide more confidence for the results we did find. The evidence in this Cochrane Review is current to January 2020.

Summary of findings

Background

Randomised trials are considered the gold standard for evaluating the effectiveness and efficacy of interventions. Poor retention (or high attrition) in randomised trials has serious consequences for the validity, reliability and usability of their results. Missing data, resulting from poor retention, are of particular concern if the data that are not missing are not at random. In other words, if there is a difference in the amount of missing data between the trial arms or amongst people who are more unwell. However, even if data are missing at random, this is also a potential problem because it will weaken the power of the trial and mean that more participants are needed to achieve a satisfactory sample size. It has been proposed that loss of less than 5% is not problematic but that more than 20% is a serious threat to validity, with anything in between also requiring attention (Fewtrell 2008; Schulz 2002). Recent work suggests that up to 50% of all trials have loss to follow‐up of more than 11% (Walters 2016).

Missing data from loss to follow‐up can be dealt with statistically by various methods including, for example, imputing values based on assumptions about the missing data to give a conservative estimate of the treatment effect (methods such as maximum likelihood estimation routines or multiple imputation). However, the risk of bias still remains when trials do not collect adequate data to give accurate estimates (Hollis 1999). Loss to follow‐up from randomised trials can sometimes go unreported and using different, but plausible, assumptions about outcomes for participants lost to follow‐up can change the results of randomised trials (Walsh 2014). However, rather than adjusting for missingness in the analysis of a trial, and inflating the sample size during recruitment, it seems much more sensible to mitigate the problem of poor retention by designing and evaluating approaches and strategies to maximise data collection. Not knowing how best to retain people in trials means trials will take longer (and cost more) and may expose additional patients to unnecessary risk or forgo the opportunity for others to receive effective treatments. Evidence for effective retention strategies would enable trial teams to include strategies in their trial which are likely to maximise trial design, efficiency and reduce research waste.

This is a substantially revised update of the first full version of this Cochrane Methodology Review (Brueton 2013). The scope of this review is restricted to interventions that are designed to maximise data collection from trial participants once they have been recruited and randomised. The Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) guidelines define non‐retention as instances in which participants are prematurely ’off‐study’ (i.e. consent withdrawn or lost to follow‐up), and therefore outcome data cannot be obtained (Chan 2013). However, participants can still be 'on‐study' but not provide outcome data. Trial non‐retention is distinct from non‐adherence to the trial intervention, which refers to the degree to which the behaviour of trial participants corresponds to the intervention assigned to them. There are Studies Witihin A Trial (SWAT) for this (Bensaaud 2020), but it is not within the scope of this review.

Description of the methods being investigated

Strategies to improve trial retention include those designed to generate maximum data return or compliance and follow‐up procedures that aim to collect data from participants (e.g. weight measurements, blood tests). These strategies can include how outcomes are collected (e.g. postal or telephone); who collects outcomes (e.g. participant‐reported or routine data), when outcomes are collected and also consider, where outcomes are collected (e.g. postal questionnaire or clinic visits).

Objectives

To quantify the effects of strategies for improving retention of participants in randomised trials. A secondary objective is to investigate if the effects vary by trial setting.

Methods

Criteria for considering studies for this review

Types of studies

Randomised and quasi‐randomised trials of interventions to improve retention of participants in randomised trials (hereafter referred to as retention trials).

Strategies to improve retention are designed to have an impact after participants are randomised to one of the intervention groups of the host and the retention trial, however, they could be delivered at any point (including at the time of recruitment, for example by modifying the information that focuses on retention that is presented to potential participants). Participants in the host trials cover a range of groups and can include (but not be limited to): patients, public, healthcare professionals, etc, and likewise the retention trials might include a range of designs such as individually‐randomised, cluster‐randomised, etc. We excluded trials of strategies that were intended to increase recruitment only, because these are covered by a complementary Cochrane Methodology Review (Treweek 2018). We excluded cohort studies with embedded randomised retention trials, which are the subject of a separate systematic review (Booker 2011).

When referring to embedded trials, we mean randomised trials of retention interventions (e.g. monetary incentives to improve response to postal questionnaires) that are set within a clinical trial (e.g. drug treatment for stroke). Clinical trials that embed retention trials are sometimes referred to as the host trial. Embedded trials are also sometimes referred to as Studies Within A Trial or SWATs. As per guidance by Treweek 2018, ‘a SWAT is a self‐contained research study that has been embedded within a host trial with the aim of evaluating or exploring alternative ways of delivering or organising a particular trial process’.

Types of data

We included retention trials within the context of a host randomised trial with participants from any age, gender, ethnic, language and geographic groups. We included unpublished and published participant retention data from randomised trials addressing health care (including all disciplines and disease areas) and non‐healthcare (education, social sciences) topics. We also included trials set in the community that were healthcare‐related. However, whilst the setting could be non‐health care, the outcomes being measured in the host randomised trial were required to be clinical‐ or health‐related. The retention trials were embedded in real trials (host trials) and not hypothetical trials.

Types of methods

Any intervention that aimed to improve retention of participants to a randomised trial. We considered any strategy aimed at increasing retention, whether it was directed towards the clinician, researcher or participant. The retention trials included at least one randomised comparison of two or more strategies to improve retention, or compared one or more strategies with usual study procedures. We also included trials with any combination of strategies to increase retention. Strategies could include any of the following:

data collection (e.g. shorter length of follow‐up or variation in follow‐up visit frequency);
participant strategies (e.g. monetary incentives, non‐monetary incentives, reminders, behavioural strategies, etc);
sites and site staff (e.g. monitoring approaches);
central study management (e.g. patient and public involvement);
study design (e.g. blinding and treatment preference).

For trials that simultaneously evaluated more than one intervention, unless designed as a factorial trial, or interaction effects were accounted for in the analysis, interventions had to be separated by at least six months to be considered eligible for the review. The reason for this was to account for contamination effects from carry over of previous intervention effects.

Types of outcome measures

Primary outcomes

The proportion of participants retained at the primary analysis point as defined in each individual retention trial is our primary outcome. If the primary outcome was not predefined in a retention trial, we took the first time point reported for analysis. In most cases, this was final response. If retention at a number of time points was reported and no clear time point for the primary outcome for the retention trial was stated, we took data for the nearest time point to the intervention in the retention trial analyses. For studies that reported data captured 'without additional chasing' (i.e. no further standard follow‐up processes such as telephone calls were included before data collection), this was selected as the primary analysis point for data to be included in this review. For studies that delivered an intervention at trial recruitment, we took the total number of participants in the intervention trial as the number who consented as the denominator and the number retained as the numerator. All decisions about primary outcome timing were based on the retention trial publication and discussion within our team; we did not check study protocols or contact authors for clarification.

Secondary outcomes

This update includes no secondary outcomes. This is a change from the previous version of the review (Brueton 2013) which stated "Retention of participants at secondary analysis points" as a secondary outcome. However, because this is rarely reported, we decided to no longer include it as a secondary outcome.

Search methods for identification of studies

We used the Online Resource for Recruitment research in Clinical triAls (ORRCA, www.orrca.org.uk) database to search for studies that had been published up to the end of December 2017. As the scope of this update had changed from that of the original review (Brueton 2013), we re‐ran the full search from database inception rather than only for the period required for the update (which would have been 2013 to 2020). The ORRCA database provides a comprehensive online database of published research (empirical and non‐empirical) about recruitment and/or retention to clinical research. ORRCA is populated from an extensive systematic search of the Cochrane Library, MEDLINE (Ovid), SCOPUS, CINAHL, psycINFO, and SCI‐EXPANDED and SSCI (via ISI Web of Science). The search strategy used to populate ORRCA was based on the original Cochrane Review of strategies to improve trial retention (Brueton 2013), but updated and extended to ensure capture of all relevant studies in this area (see below and Appendices for details). Eligible articles are categorised on the ORRCA database according to research methods and host study characteristics. We searched the ORRCA retention database in April 2020 to identify randomised evaluations of retention strategies that were nested within randomised trials (including factorial, cluster and cross‐over trials), patient preference studies, registries or where the host study type was unknown.

The search strategy used to develop ORRCA aimed to identify published research addressing retention challenges in healthcare and social science settings involving any method of follow‐up. At the time of updating this review, ORRCA only captured studies published until January 2018. Therefore we also ran the search strategy across all platforms described above, to capture studies published between January 2018 and January 2020.

Electronic searches

Each search comprised a filter to identify randomised trials plus free‐text terms and database subject headings relating to reducing loss to follow‐up or increasing retention (Appendix 1). Electronic databases that we searched included the following.

Cochrane Central Register of Controlled Trials (CENTRAL) (to January 2020)
MEDLINE (OVID) (1950 to January 2020) (Appendix 1)
CINAHL (Cumulative Index to Nursing and Allied Health; 1981 to January 2020) (Appendix 1)
PsycINFO (1806 to January 2020) (Appendix 1)
SCOPUS (to January 2020)
Web of Science Core Collection (SCI‐expanded, SSCI, CPSI‐S, CPCI‐SSH and ESCI) (1900 to January 2020)

Searching other resources

We also searched the SWAT repository (SWAT) to identify retention trials that were unpublished or ongoing.

Data collection and analysis

Selection of studies

All review authors were involved in the screening of titles and abstracts retrieved by the searches (in batches of 600) using a predesigned study eligibility screening form. A random 10% of each batch and all potentially eligible titles and abstracts were double screened by one of the review team (KG). We obtained full‐text papers for all potentially eligible studies for inclusion. All review authors were involved in independently assessing full‐text articles to determine if they fulfilled the inclusion criteria, with two review authors allocated to each full‐text article. We contacted study authors for electronic copies of papers that we could not access through library sources. We were able to obtain copies of all the potentially eligible papers, or abstracts, that we wanted to screen. When necessary, we sought information from the original investigators for potentially eligible trials where we wished to clarify eligibility. We resolved disagreements by discussion with a third review author (MAM or KG).

Data extraction and management

All review authors were involved in independently extracting data from included studies using a prespecified data extraction form, with two review authors allocated to each study. A third review author (MAM) checked the extractions for inconsistencies and any discrepancies were resolved by discussion with another review author (KG). Data extracted for the host trial were: design, location, setting, population, intervention, and comparator. For the embedded retention trial, we extracted data on: randomised or quasi‐randomised; design; aim; definition of retention used; retention period; the source of the retention trial sample (e.g. all host participants, participants lost to follow‐up, etc), and participant details. The retention strategy details extracted included: type, theoretically based; description; frequency and timing; mode of delivery; co‐interventions; economic information; resource requirements, numbers and proportions of participants in the intervention and comparator groups of the retention trial.

Assessment of risk of bias in included studies

All included studies (from previous version of review (n = 32) and this update (n = 39)) were assessed independently by two review authors (KG and MAM or ST) for risk of bias using the Cochrane 'Risk of bias' tool (Higgins 2008a), with any disagreements being resolved by a third member of the review team (ST or MAM). Information on risk of bias for all included studies is presented in the Characteristics of included studies table. When assessing studies on 'Blinding of participants and/or personal', we determined that if study authors noted that participants/personnel were not able to be blinded but that they were not given explicit knowledge of the retention trial and/or there was no way staff could use this knowledge to influence the objective outcome of retention, we determined these to be of low risk of bias for this domain. Likewise, when assessing 'Blinding of outcome assessment' we made a judgement as to whether the lack of blinding of outcome assessors would impact on their assessment of the objective outcome of retention. For the majority assessed, we considered studies to be low risk of bias on this domain. If studies were scored as low risk of bis on any one element, or unclear on any one element, this was the corresponding overall risk of bias rating.

We applied GRADE to all comparisons, including when only one study was available for a comparison (Guyatt 2008). For meta‐analyses, GRADE assessment data for the relevant meta‐analyses are provided in the relevant 'Summary of findings' table.

For single studies, we used the rules applied in the Cochrane recruitment review (Treweek 2018), with all studies initially assigned a high GRADE rating of certainty, with the following rules then applied to determine the overall rating.

Study limitations: downgrade all studies at high risk of bias by two levels; downgrade all studies at uncertain risk of bias by one level.
Inconsistency: assume no serious inconsistency.
Indirectness: assume no serious indirectness (all studies provided direct retention data).
Imprecision: downgrade all single studies by one level because of the sparsity of data; downgrade by a further level if the confidence interval is wide and includes a risk difference of zero.
Reporting bias: assume no serious reporting bias.

We provided an informative statement with each GRADE assessment following the guidance in GRADE Guideline 26 (Santesso 2020). This uses both the GRADE assessment and the effect size to produce an informative statement. We used the following rules regarding effect size.

Large effect: 10% or over
Moderate effect: 5% to 9%
Small important effect: 2% to 4%
Trivial, small unimportant effect or no effect: 0% to‐ 1%

We applied the same rules to effect size, regardless of whether the effect was an increase or a decrease in retention.

As per the Cochrane recruitment review (Treweek 2018), we did not exclude studies that were assessed to be at high risk of bias. However, where a high risk of bias study is the only study in a comparison, we do not describe them in the Results or Discussion sections due to the low confidence we have in their findings. We encourage more rigorous evaluations of these interventions but would discourage interpretation about their effects on retention from existing evaluations. The exception to this is if the data from high risk of bias studies could be included in a meta‐analysis alongside data from other studies and where a cumulative judgement on the certainty of the body of evidence using GRADE (as described above) could then be done.

Measures of the effect of the methods

We calculated risk difference (RD) and 95% confidence intervals (CIs) for retention to determine the effect of strategies on this outcome.

Unit of analysis issues

For retention trials that randomised individuals and clusters, the unit of analysis was the participant. For cluster‐randomised trials that ignored clustering in the analysis, we inflated the standard errors (SEs) to avoid over precise estimates of effect as follows (Higgins 2008b).

We calculated the RD, 95% CI and SE based on participants in the usual way (i.e. ignoring clustering).
This SE was then inflated using the design effect to get an adjusted SE: adjusted SE = SE X√ design effect. With the design effect calculated as follows: design effect = 1 + (M ‐ 1) Intra‐cluster coefficient (ICC) where M = mean cluster size, ICC = the intracluster correlation coefficient.
Where published ICCs were not available, we used the mean ICC from appropriate external estimates for Land 2007. This was the mean of estimates for the return of EuroQol questionnaires (ICC = 0.054) from a source recommended by the Cochrane Handbook for Systematic Reviews of Interventions (Section 16.3.4) (Higgins 2008b) and www.abdn.ac.uk/hsru/documents/iccs-web.xls (last accessed 24 November 2020).
We entered the effect estimate and the new updated SE into Review Manager 5 using the generic inverse variance (RevMan 2012).

Where the number of participants randomised was not clearly stated in the included study report, we contacted the study authors for this information.

Dealing with missing data

For unpublished studies, we contacted study authors for data for the'R risk of bias' assessment, numbers randomised to each group and numbers retained in each group at the primary endpoint.

Assessment of heterogeneity

We agreed the presence of heterogeneity of the intervention effect where the Chi² statistic has a significance level of 0.10 (representing a 10% chance of a Type I error). This figure was chosen as it counterbalances the relatively low power of the test. We also used the I² test (Higgins 2003). It represents the total variation across studies and is unlike the Chi² test in that it is independent from the number of studies. Instead theI² is based on treatment effect. Heterogeneity was also explored through subgroup analyses.

Assessment of reporting biases

We would have assessed reporting bias using tests for funnel plot asymmetry if sufficient data were available (Egger 1997; Sterne 2008).

Data synthesis

We grouped included trials based on the type of intervention under investigation with groups directly informed by the ORRCA retention domains (https://www.orrca.org.uk/Uploads/ORRCA_Retention_Domains.pdf). We added a further domain within the 'Participants' domain to allow separate consideration of prompts and reminders targeting retention. This classification resulted in five broad categories with intervention functions grouped within them.

Data collection (Category A), interventions include:
1. questionnaire design;
2. data collection frequency/timing;
3. data collection location and method.
Participants (Category B), interventions include:
1. reminders ‐ intention to be received after a retention time point is reached;
2. prompts ‐ intention to be received before a retention time point is reached;
3. monetary incentives and rewards‐ includes both incentives (i.e. not conditional on behaviour) and rewards (i.e. is conditional on behaviour);
4. non‐monetary incentives;
5. maintaining participant engagement;
6. behavioural intervention.
Sites and site staff (Category C), interventions include:
1. prompt;
2. monitoring visits.
Central Study Management (Category D), interventions include:
1. patient public involvement.
Study design (Category E), interventions include:
1. impact of recruitment;
2. blinding and treatment preference.

We present the results as RD, pooled using a random‐effects model for all meta‐analyses with more than one included study,and with associated CIs where sufficient data were available. If heterogeneity was detected and could not be explained by subgroup or sensitivity analyses, we did not pool results.

For factorial trials, the data for different categories of interventions were included as separate trial comparisons. For multiple retention trials conducted within the same host trial that were not designed to allow for interaction effects between interventions (i.e. did not stratify at randomisation or account for interaction effects in analysis), we pre‐specified the requirement for interventions to be delivered at least six months apart in order to minimise the potential for any intervention interaction effects. In order to minimise interaction effects, we chose not to include data in the meta‐analyses from trials that had evaluated interventions within six months of each other that had not accounted for the interaction effects in the analysis. This resulted in three studies and data from a further four studies (with multiple evaluations) being omitted from our analyses.

Subgroup analysis and investigation of heterogeneity

We planned to explore the following factors in subgroup analyses assuming enough studies were identified within each comparison.

Type of design used to evaluate the retention strategy (randomised versus quasi‐randomised)
Setting of the host trial (e.g. primary versus secondary care, healthcare setting versus non‐healthcare setting)
Disease area of the host trial (e.g. oncology versus ante‐natal)
Duration of follow‐up (e.g. short versus long term)
Value of monetary incentive (e.g. £5 versus £10 etc)

Sensitivity analysis

To assess the robustness of the results, we planned sensitivity analyses that excluded quasi‐randomised retention trials.

Results

Description of studies

The studies are described in the Characteristics of included studies, Characteristics of studies awaiting classification, Characteristics of ongoing studies, and Characteristics of excluded studies tables. We identified 18,756 abstracts, titles and other records and sought the full text for 150 records to confirm eligibility. In total, 70 papers (reporting data from 81 retention trials) were considered eligible for inclusion (Figure 1). The studies were conducted in eight countries with two multi‐national studies. The majority of studies (n = 53) were conducted in the UK followed by the USA (n = 10) (Table 17). Of these 70 papers, 68 evaluated interventions targeting trial participants and two evaluated interventions targeting individuals involved in trial retention. A total of 101,689 participants were included across the retention trials, which included all participants originally randomised to the retention trial. Included retention trials were conducted in a broad spectrum of clinical conditions across a range of different settings including primary care, secondary care, and community settings. However, similar to the previous version of this review (Brueton 2013), the included studies were predominantly composed of studies evaluating interventions to improve questionnaire return (n = 70) rather than clinic attendance (n = 2).

1. Countries where the included studies took place.

Country	Number of studies
Australia	1
Canada	2
Denmark	1
France	1
Norway	1
UK	53
USA	10
Multinational	2 (one involving UK and Ireland and one involving the USA and Canada).

Sub‐domains	Study ID	Intervention	Control
1A. Questionnaire design: Questionnaire length
	Dorman 1997	New questionnaire (Shorter version)	Standard questionnaire
	Edwards 2004	New questionnaire (Shorter version)	Standard questionnaire
	Subar 2001	New questionnaire (Shorter version)	Standard questionnaire
2A. Questionnaire design: Addition of a diary to usual follow‐up
	Griffin 2019	Diaries follow‐up	Postal questionnaires follow‐up
	Marques2013	Resource use log to prospectively record their use of health services	No resource use log
3A. Questionnaire design: Question order, condition first vs generic first question
	McColl 2003	Condition‐specific measures of quality of life preceded generic instruments	Questionnaires in a reverse order
4A. Data collection frequency and timing: Timing of questionnaire delivery
	Renfroe 2002	Timing of postal questionnaire, cover letter signatory, express	Regular mail, non‐monetary incentive
5A. Data Collection Location and Method: Postal follow‐up vs clinic follow‐up
	Greig 2017	Postal follow‐up	Clinic follow‐up
6A. Data Collection Location and Method: Telephone follow‐up vs postal questionnaire
	Couper 2007	Telephone follow‐up	Postal questionnaire
	Marsh 1999 (Postal trial)	Postal follow‐up with an incentive	Postal follow‐up without incentive
	Marsh 1999 (Telephone trail)	Telephone follow‐up with an incentive	Telephone follow‐up without incentive
7A. Data Collection Location and Method: First class vs second class outward mailing
	Sharp 2006	First‐class post	Second class
8A. Data Collection Location and Method: Return postage
	Sharp 2006	Preaddressed second class stamped envelope	Business reply envelope
	Kenton 2007	'high priority' stamp to the mailing	Business format mailing
	Dinglas 2015 (Mail trial)	Personalised postal follow‐up	Generic postal follow‐up
9A. Data Collection Location and Method: Use of self‐sampling kits
	Tranberg 2018	Received a modified second reminder, a leaflet, and a self‐sampling kit.	received the same material as those in the directly mailed group but received no kit

Short questionnaire compared with long questionnaire for trial retention
Patient or population: trial participants being followed up for data collection Settings: any setting Intervention: short questionnaire Comparison: usual questionnaire
Outcomes	*Illustrative comparative risks (95% CI)**		Relative effect (95% CI)	No of Participants (studies)	Certainty of the evidence (GRADE)
	Assumed risk	Corresponding risk
	Short questionnaire	Usual questionnaire
Retention [follow‐up]	As measured

	Low^a		RR 1.01 (0.89 to 1.14)	3252 (3)	⊕⊝⊝⊝ very low
	25 per 100	25 per 100 (22 to 29 )
	Medium^a
	50 per 100	51 per 100 (45 to 57)
	High^a
	80 per 100	81 per 100 (71 to 91)
The basis for the assumed risk* (e.g. the median control group risk across studies) is provided in footnotes. The effect of a short questionnaire (and its 95% confidence interval) is based on the assumed risk in the comparison group and the relative effect of the intervention (and its 95% CI). CI: confidence interval; RR: risk ratio.
GRADE Working Group grades of evidence High certainty: we are very confident that the true effect lies close to that of the estimate of the effect. Moderate certainty: we are moderately confident in the effect estimate; the true effect is likely to be close to the estimate of the effect, but there is a possibility that it is substantially different. Low certainty: our confidence in the effect estimate is limited; the true effect may be substantially different from the estimate of the effect. Very low certainty: we have very little confidence in the effect estimate; the true effect is likely to be substantially different from the estimate of effect.
^a We selected low, medium and high illustrative retention levels of 25%, 50% and 80% based on prior experience with trial retention and evidence from the literature. For example, it has been previously stated that it is common for up to 20% trial participants to drop out before the trial finishes (Walsh 2015), as such we set the upper limit of good retention as 80%. The other extreme of 25% was informed by evidence that some trials (largely internet based) can have retention as low as 10% to 25% (Murray 2009). The mid point of 50% was a judgement made by the review team and was deemed appropriate given the evidence for the other parameters.

Addition of diary to usual follow‐up compared with usual follow‐up for trial retention
Patient or population: trial participants being followed up for data collection Settings: any setting Intervention: diary Comparison: no diary
Outcomes	*Illustrative comparative risks (95% CI)**		Relative effect (95% CI)	No of Participants (studies)	Certainty of the evidence (GRADE)
	Assumed risk	Corresponding risk
	Diary	No diary
Retention [follow‐up]	Low^a		RR 0.97 (0.96 to 0.98)	9906 (2)	⊕⊕⊕⊝ moderate
	25 per 100	24 per 100 (24 to 25)
	Medium^a
	50 per 100	49 per 100 (48 to 49)
	High^a
	80 per 100	78 per 100 (77 to 78)
The basis for the assumed risk* (e.g. the median control group risk across studies) is provided in footnotes. The effect of not including a diary (and its 95% confidence interval) is based on the assumed risk in the comparison group and the relative effect of the intervention (and its 95% CI). CI: confidence interval; RR: risk ratio.
GRADE Working Group grades of evidence High certainty: we are very confident that the true effect lies close to that of the estimate of the effect. Moderate certainty: we are moderately confident in the effect estimate; the true effect is likely to be close to the estimate of the effect, but there is a possibility that it is substantially different. Low certainty: our confidence in the effect estimate is limited; the true effect may be substantially different from the estimate of the effect. Very low certainty: we have very little confidence in the effect estimate; the true effect is likely to be substantially different from the estimate of effect.
^aWe selected low, medium and high illustrative retention levels of 25%, 50% and 80% based on prior experience with trial retention and evidence from the literature. For example, it has been previously stated that it is common for up to 20% trial participants to drop out before the trial finishes (Walsh 2015), as such we set the upper limit of good retention as 80%. The other extreme of 25% was informed by evidence that some trials (largely internet based) can have retention as low as 10% to 25% (Murray 2009). The mid point of 50% was a judgement made by the review team and was deemed appropriate given the evidence for the other parameters.

Telephone follow‐up compared with postal questionnaire for trial retention
Patient or population: trial participants being followed up for data collection Settings: any setting Intervention: telephone follow‐up Comparison: postal questionnaire
Outcomes	*Illustrative comparative risks (95% CI)**		Relative effect (95% CI)	No of Participants (studies)	Certainty of the evidence (GRADE)
	Assumed risk	Corresponding risk
	Telephone follow‐up	Postal questionnaire
Retention [follow‐up]	Low^a		RR 1.04 (0.94 to 1.17)	1006 (2)	⊕⊝⊝⊝ very low
	25 per 100	26 per 100 (24 to 29)
	Medium^a
	50 per 100	52 per 100 (47 to 59)
	High^a
	80 per 100	83 per 100 (75 to 94)
The basis for the assumed risk* (e.g. the median control group risk across studies) is provided in footnotes. The effect of telephone follow‐up (and its 95% confidence interval) is based on the assumed risk in the comparison group and the relative effect of the intervention (and its 95% CI). CI: confidence interval; RR: risk ratio.
GRADE Working Group grades of evidence High certainty: we are very confident that the true effect lies close to that of the estimate of the effect. Moderate certainty: we are moderately confident in the effect estimate; the true effect is likely to be close to the estimate of the effect, but there is a possibility that it is substantially different. Low certainty: our confidence in the effect estimate is limited; the true effect may be substantially different from the estimate of the effect. Very low certainty: we have very little confidence in the effect estimate; the true effect is likely to be substantially different from the estimate of effect.
^a We selected low, medium and high illustrative retention levels of 25%, 50% and 80% based on prior experience with trial retention and evidence from the literature. For example, it has been previously stated that it is common for up to 20% trial participants to drop out before the trial finishes (Walsh 2015), as such we set the upper limit of good retention as 80%. The other extreme of 25% was informed by evidence that some trials (largely internet based) can have retention as low as 10% to 25% (Murray 2009). The mid point of 50% was a judgement made by the review team and was deemed appropriate given the evidence for the other parameters.

Return postage compared with control intervention for trial retention
Patient or population: trial participants being followed up for data collection Settings: any setting Intervention: various return postage strategies Comparison: control intervention
Outcomes	*Illustrative comparative risks (95% CI)**		Relative effect (95% CI)	No of Participants (studies)	Certainty of the evidence (GRADE)
	Assumed risk	Corresponding risk
	Various return postage strategies (such as free post versus second class stamp; high priority mail stamp versus usual postage; and personal form)	Standard return postage
Retention [follow‐up]	Low^a		RR 1.06 (0.99 to 1.15)	1543 (3)	⊕⊕⊝⊝ low
	25 per 100	27 per 100 (25 to 29)
	Medium^a
	50 per 100	53 per 100 (50 to 58)
	High^a
	80 per 100	85 per 100 (79 to 92)
The basis for the assumed risk* (e.g. the median control group risk across studies) is provided in footnotes. The corresponding risk (and its 95% confidence interval) is based on the assumed risk in the comparison group and the relative effect of the intervention (and its 95% CI). CI: confidence interval; RR: risk ratio.
GRADE Working Group grades of evidence High certainty: we are very confident that the true effect lies close to that of the estimate of the effect. Moderate certainty: we are moderately confident in the effect estimate; the true effect is likely to be close to the estimate of the effect, but there is a possibility that it is substantially different. Low certainty: our confidence in the effect estimate is limited; the true effect may be substantially different from the estimate of the effect. Very low certainty: we have very little confidence in the effect estimate; the true effect is likely to be substantially different from the estimate of effect.
^a We selected low, medium and high illustrative retention levels of 25%, 50% and 80% based on prior experience with trial retention and evidence from the literature. For example, it has been previously stated that it is common for up to 20% trial participants to drop out before the trial finishes (Walsh 2015), as such we set the upper limit of good retention as 80%. The other extreme of 25% was informed by evidence that some trials (largely internet based) can have retention as low as 10% to 25% (Murray 2009). The mid point of 50% was a judgement made by the review team and was deemed appropriate given the evidence for the other parameters.

Electronic reminder compared with usual follow‐up for trial retention
Patient or population: trial participants being followed up for data collection Settings: any setting Intervention: electronic reminder Comparison: usual follow‐up
Outcomes	*Illustrative comparative risks (95% CI)**		Relative effect (95% CI)	No of Participants (studies)	Certainty of the evidence (GRADE)	Comments
	Assumed risk	Corresponding risk
	Electonic reminder	Usual follow‐up
Retention [follow‐up]	Low^a		RR 1.01 (0.95 to 1.09)	790 (3)	⊕⊕⊝⊝ low
	25 per 100	25 per 100 (24 to 27)
	Medium^a
	50 per 100	51 per 100 (48 to 55)
	High^a
	80 per 100	81 per 100 (76 to 87)
The basis for the assumed risk* (e.g. the median control group risk across studies) is provided in footnotes. The effect of an electronic reminder (and its 95% confidence interval) is based on the assumed risk in the comparison group and the relative effect of the intervention (and its 95% CI). CI: confidence interval; RR: risk ratio.
GRADE Working Group grades of evidence High certainty: we are very confident that the true effect lies close to that of the estimate of the effect. Moderate certainty: we are moderately confident in the effect estimate; the true effect is likely to be close to the estimate of the effect, but there is a possibility that it is substantially different. Low certainty: our confidence in the effect estimate is limited; the true effect may be substantially different from the estimate of the effect. Very low certainty: we have very little confidence in the effect estimate; the true effect is likely to be substantially different from the estimate of effect.
^a We selected low, medium and high illustrative retention levels of 25%, 50% and 80% based on prior experience with trial retention and evidence from the literature. For example, it has been previously stated that it is common for up to 20% trial participants to drop out before the trial finishes (Walsh 2015), as such we set the upper limit of good retention as 80%. The other extreme of 25% was informed by evidence that some trials (largely internet based) can have retention as low as 10% to 25% (Murray 2009). The mid point of 50% was a judgement made by the review team and was deemed appropriate given the evidence for the other parameters.

Electronic prompt compared with no prompt for trial retention
Patient or population: trial participants being followed up for data collection Settings: any setting Intervention: electronic prompt Comparison: no prompt
Outcomes	*Illustrative comparative risks (95% CI)**		Relative effect (95% CI)	No of Participants (studies)	Certainty of the evidence (GRADE)	Comments
	Assumed risk	Corresponding risk
	Electronic prompt	No prompt
Retention [follow‐up]	Low^a		RR 1.03 (0.98 to 1.08)	2897 (5)	⊕⊝⊝⊝ very low
	25 per 100	26 per 100 (25 to 27)
	Medium^a
	50 per 100	52 per 100 (49 to 54)
	High^a
	80 per 100	82 per 100 (78 to 86)
The basis for the assumed risk* (e.g. the median control group risk across studies) is provided in footnotes. The effect of electronic prompts (and its 95% confidence interval) is based on the assumed risk in the comparison group and the relative effect of the intervention (and its 95% CI). CI: confidence interval; RR: risk ratio.
GRADE Working Group grades of evidence High certainty: we are very confident that the true effect lies close to that of the estimate of the effect. Moderate certainty: we are moderately confident in the effect estimate; the true effect is likely to be close to the estimate of the effect, but there is a possibility that it is substantially different. Low certainty: our confidence in the effect estimate is limited; the true effect may be substantially different from the estimate of the effect. Very low certainty: we have very little confidence in the effect estimate; the true effect is likely to be substantially different from the estimate of effect.
^a We selected low, medium and high illustrative retention levels of 25%, 50% and 80% based on prior experience with trial retention and evidence from the literature. For example, it has been previously stated that it is common for up to 20% trial participants to drop out before the trial finishes (Walsh 2015), as such we set the upper limit of good retention as 80%. The other extreme of 25% was informed by evidence that some trials (largely internet based) can have retention as low as 10% to 25% (Murray 2009). The mid point of 50% was a judgement made by the review team and was deemed appropriate given the evidence for the other parameters.

Telephone prompt compared with usual follow‐up for trial retention
Patient or population: trial participants being followed up for data collection Settings: any setting Intervention: telephone prompt Comparison: usual follow‐up
Outcomes	*Illustrative comparative risks (95% CI)**		Relative effect (95% CI)	No of Participants (studies)	Certainty of the evidence (GRADE)	Comments
	Assumed risk	Corresponding risk
	Telephone prompt	Usual follow‐up
Retention [follow‐up]	Low^a		RR 1.02 (0.85 to 1.22)	943 (2)	⊕⊝⊝⊝ very low
	25 per 100	26 per 100 (21 to 31)
	Medium^a
	50 per 100	51 per 100 (43 to 61)
	High^a
	80 per 100	82 per 100 (68 to 98)
The basis for the assumed risk* (e.g. the median control group risk across studies) is provided in footnotes. The effect of telephone prompts (and its 95% confidence interval) is based on the assumed risk in the comparison group and the relative effect of the intervention (and its 95% CI). CI: confidence interval; RR: risk ratio.
GRADE Working Group grades of evidence High certainty: we are very confident that the true effect lies close to that of the estimate of the effect. Moderate certainty: we are moderately confident in the effect estimate; the true effect is likely to be close to the estimate of the effect, but there is a possibility that it is substantially different. Low certainty: our confidence in the effect estimate is limited; the true effect may be substantially different from the estimate of the effect. Very low certainty: we have very little confidence in the effect estimate; the true effect is likely to be substantially different from the estimate of effect.
^a We selected low, medium and high illustrative retention levels of 25%, 50% and 80% based on prior experience with trial retention and evidence from the literature. For example, it has been previously stated that it is common for up to 20% trial participants to drop out before the trial finishes (Walsh 2015), as such we set the upper limit of good retention as 80%. The other extreme of 25% was informed by evidence that some trials (largely internet based) can have retention as low as 10% to 25% (Murray 2009). The mid point of 50% was a judgement made by the review team and was deemed appropriate given the evidence for the other parameters.

Personalised prompt compared with usual follow‐up for trial retention
Patient or population: trial participants being followed up for data collection Settings: any setting Intervention: personalised prompt Comparison: usual follow‐up
Outcomes	*Illustrative comparative risks (95% CI)**		Relative effect (95% CI)	No of Participants (studies)	Certainty of the evidence (GRADE)	Comments
	Assumed risk	Corresponding risk
	Personalised prompt	Usual follow‐up
Retention [follow‐up]	Low^a		RR 0.97 (0.89 to 1.07)	701 (2)	⊕⊕⊝⊝ low
	25 per 100	24 per 100 (22 to 27)
	Medium^a
	50 per 100	49 per 100 (45 to 54)
	High^a
	80 per 100	78 per 100 (71 to 86)
The basis for the assumed risk* (e.g. the median control group risk across studies) is provided in footnotes. The effect of personalised prompts (and its 95% confidence interval) is based on the assumed risk in the comparison group and the relative effect of the intervention (and its 95% CI). CI: confidence interval; RR: risk ratio.
GRADE Working Group grades of evidence High certainty: we are very confident that the true effect lies close to that of the estimate of the effect. Moderate certainty: we are moderately confident in the effect estimate; the true effect is likely to be close to the estimate of the effect, but there is a possibility that it is substantially different. Low certainty: our confidence in the effect estimate is limited; the true effect may be substantially different from the estimate of the effect. Very low certainty: we have very little confidence in the effect estimate; the true effect is likely to be substantially different from the estimate of effect.
^a We selected low, medium and high illustrative retention levels of 25%, 50% and 80% based on prior experience with trial retention and evidence from the literature. For example, it has been previously stated that it is common for up to 20% trial participants to drop out before the trial finishes (Walsh 2015), as such we set the upper limit of good retention as 80%. The other extreme of 25% was informed by evidence that some trials (largely internet based) can have retention as low as 10% to 25% (Murray 2009). The mid point of 50% was a judgement made by the review team and was deemed appropriate given the evidence for the other parameters.

Addition of monetary incentives compared with usual follow‐up for trial retention
Patient or population: trial participants being followed up for data collection Settings: any setting Intervention: monetary incentives Comparison: usual follow‐up
Outcomes	*Illustrative comparative risks (95% CI)**		Relative effect (95% CI)	No of Participants (studies)	Certainty of the evidence (GRADE)	Comments
	Assumed risk	Corresponding risk
	Monetary incentives	Usual follow‐up
Retention [follow‐up]	Low^a		RR 1.20 (1.06 to 1.36)	3166 (3)	⊕⊕⊝⊝ low
	25 per 100	30 per 100 (27 to 34)
	Medium^a
	50 per 100	60 per 100 (53 to 68)
	High^a
	80 per 100	96 per 100 (85 to [109)
The basis for the assumed risk* (e.g. the median control group risk across studies) is provided in footnotes. The effect of monetary incentives (and its 95% confidence interval) is based on the assumed risk in the comparison group and the relative effect of the intervention (and its 95% CI). CI: confidence interval; RR: risk ratio.
GRADE Working Group grades of evidence High certainty: we are very confident that the true effect lies close to that of the estimate of the effect. Moderate certainty: we are moderately confident in the effect estimate; the true effect is likely to be close to the estimate of the effect, but there is a possibility that it is substantially different. Low certainty: our confidence in the effect estimate is limited; the true effect may be substantially different from the estimate of the effect. Very low certainty: we have very little confidence in the effect estimate; the true effect is likely to be substantially different from the estimate of effect.
^a We selected low, medium and high illustrative retention levels of 25%, 50% and 80% based on prior experience with trial retention and evidence from the literature. For example, it has been previously stated that it is common for up to 20% trial participants to drop out before the trial finishes (Walsh 2015), as such we set the upper limit of good retention as 80%. The other extreme of 25% was informed by evidence that some trials (largely internet based) can have retention as low as 10% to 25% (Murray 2009). The mid point of 50% was a judgement made by the review team and was deemed appropriate given the evidence for the other parameters.

STUDY ID	Clinical area main trial	Condition	Host trial	Participants	Overall characteristic
AMBER trial	Treatment	Multiple sclerosis	Abdominal Massage for Bowel Dysfunction Effectiveness Research (AMBER) trial	≥18 years participants, with a diagnosis of multiple sclerosis (in a stable phase, i.e. no multiple sclerosis relapse for 3 months)	Multi‐centred patient‐randomised superiority trial comparing an experimental strategy of once daily abdominal massage for 6 weeks against a control strategy of no massage in people with multiple sclerosis who have stated that their constipation is troublesome.
Arundel 2019	Smoking Cessation	Smoking	Smoking Cessation Intervention for Severe Mental Ill Health Trial (SCIMITAR+)	≥18 years participants, with a documented diagnosis of bipolar disorder, schizophrenia or schizoaffective disorder who smoke were included in this study.	The intervention group were assigned a mental health professional trained to deliver smoking‐cessation interventions who worked with the participant and participant's GP or mental health specialist to provide an individually tailored smoking‐cessation service. The control group received usual care (following NICE guidelines for smoking cessation)
Ashby 2011	Treatment	Migraine	No name provided for the host trial.	18‐65 years with a self‐reported diagnosis of migraine for at least 12 months, with no evidence of any other significant co‐existing pathology and experienced two or more migraine‐like attacks (or four or more headache days) in the previous 4‐week period were included in this study.	The intervention consisted of a diet based on eliminating foods to which participants exhibited IgG antibodies vs eliminating the same number of foods from the diet but not those for which the participant exhibited IgG antibodies.
Avenell 2004	Prevention	Fractures	UK RECORD trial	Participants ≥70 years old that had an osteoporotic fracture within the last ten years identified from the hospital notes and seen either in a fracture clinic or on an orthopaedic ward were included in this study.	Randomised double‐blind placebo‐controlled, factorial design, evaluation of oral calcium (1g/day) or vitamin D (800 IU/20 Mg) supplementation in the secondary prevention of osteoporotic fractures.
Bailey 2013	Prevention	Sexual health	“Sexunzipped” website	Youth 16‐20 years recruited online.	Theory‐based website that aims to give young people the tools to make informed decisions about their sexual well‐being.
Bauer 2004	Smoking Cessation	Smoking	Community Intervention Trial for Smoking Cessation (COMMIT) study	A cohort of smokers in each community and administered a detailed telephone‐based questionnaire about their tobacco use in 1988 and 1993 were included in this study.	The design was a matched pair, randomised, control trial, which involved 11 pairs of small‐ to medium‐sized communities in the USA and Canada. Each pair of communities contained one intervention site and one control site. The study was conducted as a community‐level intervention to help smokers achieve and maintain cessation. Within each community, the smoking behaviours and habits of the population were monitored over five years.
Bean 2018	Treatment	Childhood obesity	Nourishing Our Understanding of Role‐modelling to Improve Support and Health (NOURISH+) study	Parents/caregivers were eligible if parent age ≥18 years; child age 5–11 years; the child had overweight, or obesity and the child primarily reside in the caregiver's home were included in this study.	The intervention group received a culturally tailored parent‐based treatment for parents of children with overweight or obesity issues. Participants received eight sessions of group‐based parent‐based treatments (6 core group sessions with two adjunctive experiential sessions (a group cooking and an individual dietician visit)). This treatment was targeted at parents as the agent of change of their children with overweight or obesity issues. The control group received an educational control intervention.
Bell 2016	Prevention	Fracture	Women for Prevention of Fracture Trial (SCOOP)	70‐85‐year old women at risk of fracture were included in this study.	Those in the intervention group received a 10‐year fracture risk assessment calculated using a WHO risk algorithm computed from baseline questionnaire data and bone mineral density values measured via a Dual‐energy X‐ray absorptiometry scan in selected participants. For women in the control group, fracture risk was not calculated, and participants continued to receive usual care.
Bradshaw 2020	Diagnosis	Eczema	Barrier Enhancement for Eczema Prevention (BEEP) randomised trial	Infants at high risk of developing atopic eczema and with no more than 21 days old at the point of randomisation, the mother must be aged at least 16 years, and the consenting adult must be able to understand English were included in this study.	This trial investigated the effect of applying emollient for 12 and 24 months from birth on the development of eczema in high‐risk infants. Both groups received standard skincare advice. The intervention group receives additional advice to apply emollient daily for the first year of life and are supplied with emollient free of charge.
Brubaker 2019	Treatment	Urinary incontinence	Extended Operations and Pelvic Muscle Training in the Management of Apical Support Loss (E‐OPTIMAL) study	Women undergoing vaginal apical prolapse repair with a mid‐urethral sling for stress urinary incontinence were included in this study.	Multi‐centre randomised to (1) perioperative behavioural therapy with pelvic floor muscle training vs usual care and (2) surgical intervention or (3) usual care. The extended version was designed to lengthen the follow‐up of these women and compare 5‐ year success and complication rates.
Clark 2015	Smoking Cessation	Smoking in participants with chronic obstructive pulmonary disease	Determining the Optimal approach to identify individuals with Chronic obstructive pulmonary disease (DOC) study	Smokers aged ≥35 years undertaking lung function tests and symptom‐based questionnaires were included in this study.	DOC is a case‐finding study for chronic obstructive pulmonary disease and a randomised trial of the impact of case‐finding on smoking cessation. The intervention group received lung function tests and symptom‐based questionnaires. The control group was in a 6‐month waitlist for tests and symptom questionnaires.
Cochrane 2020	Prevention	Risk of falling	Occupational Therapist Intervention Study (OTIS)	Participants ≥65 years, community‐dwelling, currently able to walk 10 feet (with a walking aid if needed) were included in this study.	Participants were randomised to either home environmental assessment and modification, led by an occupational therapist or usual care from GP or other healthcare professional.
Cockayne 2005	Prevention	Fracture prevention	No name provided for the Host trial	Community‐dwelling women aged ≥70 years, living in the York and Cumbria area were included in this study.	The intervention group received daily oral supplementation of 1,000 mg of calcium with 800 IU vitamin D3 with a patient information leaflet on dietary calcium intake and falls prevention. The control group received the patient information leaflet only.
Cockayne 2017	Prevention	Risk of falls	REducing Falls with ORthoses and a Multifaceted podiatry intervention (REFORM) study	Participants ≥65 years from routine podiatry clinics in the UK and Ireland to the REFORM cohort were included in this study. Participants had one fall in the past 12 months: or one fall in the past 24 months requiring hospital attention were included in this study.	Podiatry intervention for the prevention of falls in older people. Participants were randomised to receive routine podiatry care, and a falls prevention leaflet or routine podiatry care, a falls prevention leaflet, and a multifaceted podiatry intervention.
Cook 2020	Treatment	Influenza	Antivirals for influenza‐Like Illness, An rCt of Clinical and Cost ‐effectiveness in primary CarE (ALIC4E) Trial	Participants ≥1 year presenting with influenza‐like illness, with symptom duration ≤72 hours in primary care over three consecutive periods of confirmed high influenza incidence were included in this study.	European multinational, multi‐centre, open‐labelled, non‐industry funded, pragmatic, adaptive‐platform, RCT. Control group received the best usual primary care, and the intervention group received the best usual primary care plus treatment with oseltamivir for five days.
Couper 2007	Treatment	Obesity	No name provided for the host trial.	Adult participants with overweight and obesity (BMI 27 to 40 kg/m2) members from four regions of Kaiser Permanente's integrated healthcare delivery system were included in this study.	Participants were randomly assigned to one of two Web‐based treatments for weight management: the expert system materials or information‐only materials.
Cunningham 2004	Treatment	Alcohol Consumption	No name provided for the host trial.	Participants were recruited through a random digit dialling telephone survey conducted by the Centre for Addiction and Mental Health were included in this study.	Participants were randomly assigned to one of four conditions in a two‐by‐two factorial design: no‐intervention control group, personalised feedback only, self‐help book only and both personalised feedback and self‐help book.
Cunningham‐Burley 2020	Prevention	Risk of falls	The Stopping Slips among Healthcare Workers (SSHeW) trial	NHS staff, aged ≥18 years, who adhere to a dress code policy and work in a clinical, catering, or general hospital environment were included in this study.	This trial evaluated the effectiveness of slip‐resistant footwear to reduce slips in NHS staff.
Dinglas 2015	Treatment	Acute lung injury survivors	ARDS Network Long Term Outcomes Study (ALTOS)	Participants with an acute lung injury survivor, recruited from 41 hospital sites at 12 centres across the US were included in this study.	Participants were enrolled in randomised trials of novel interventional therapies initial trophic vs full enteral feeding.
Dorman 1997	Prevention	Stroke	International Stroke Trial	Patients who have a clinical diagnosis of acute ischaemic stroke, with onset within the previous 48 hours and no clear indication for, or clear contraindication to, treatment with aspirin or subcutaneous heparin were included in this study.	This was an international trial involving around 36 countries. The intervention group received early administration of aspirin or heparin or both. The analysis was structured as immediate heparin (low or medium dose) vs avoiding heparin, and immediate aspirin vs avoiding aspirin.
Edwards 2004	Treatment	Head injury	CRASH Trial	All head‐injured adults who were observed while in the hospital to have GCS of 14 or less (out of a maximum score of 15), and who were within eight hours of the injury were included in this study.	This was a multi‐centre RCT evaluating the efficacy and safety of a high dose of corticosteroid infusion after head injury.
Edwards 2016	Treatment	Depression & cardiovascular health	The Healthlines Study	Participants with depression aged between 40 and 74 years old recruited from 42 general practices in or near Bristol, Sheffield and Southampton were included in this study.	Two linked, parallel RCTs of patients with depression and raised risk of cardiovascular disease who were allocated to a telehealth intervention plus usual care or usual care alone.
Fouad 2014	Diagnosis	Cancer	ASCUS‐LSIL Triage Study (ALTS)	Women residing in Jefferson County, Alabama were included in this study.	This was a multi‐centre clinical trial to evaluate the optimal clinical management of low‐grade cervical cytologic abnormalities. ALTS participants were randomised to three management strategies: 1) immediate colposcopy; 2) human papillomavirus (HPV) DNA testing, which triaged to colposcopy only participants with oncogenic HPV type; and 3) conservative management followed with serial Pap smears and colposcopy if Pap smear progressed to high grade.
Gattellari 2004	Diagnosis	Cancer	No name provided for the host trial.	Men recruited from local general practices, aged 40–70 years, fluent in English and who had not been diagnosed with prostate cancer were included in this study.	Participants were randomised to receive a 32‐page evidence‐based booklet contained content previously identified by experts as essential to informed screening or conventional information about prostate cancer screening (pamphlet had been published by the Australian government).
Gates 2009	Treatment	Acute whiplash injuries	Managing Injuries of the Neck Trial (MINT) study	Patients presenting with acute whiplash injuries, in which eligible patients were identified in emergency departments were included in this study.	Cluster randomised trial of advice interventions given in emergency departments. Participants were then followed up by postal questionnaires sent from the study office, 4, 8 and 12 months after their injury. The intervention group received advice (psycho‐educational intervention, including Whiplash book advice/active management advice) and the control group received usual care advice.
Glassman 2020	Treatment	Diabetic retinopathy	Diabetic Retinopathy Clinical Research Retina Network Protocol S randomised trial	Participants ≥18 years with at least one eye with Proliferative diabetic retinopathy were included in this study.	Participants were randomly assigned to receive either laser treatment or injections of a drug (ranibizumab) into the study eye.
Goulao 2020	Prevention	Bleeding on probing	IQuaD trial & INTERVAL trial	Adults with good oral health who are regular attendees to the United Kingdom's National Health System primary care dental services were included in this study.	Intervention measuring if providing no scale and polish or 12‐month was compared with the standard 6‐month scale and polish. Personalized (intervention) vs standard oral hygiene advice (control) was also compared.
Greig 2017	Treatment	Nail‐bed injury	The Nail bed INJury Analysis (NINJA RCT) study	Participants <16 years with acute nail bed injury within the last 48 hrs and requiring surgery were included in this study.	Children were randomised for the surgeon to either replace or discard the nail plate after nail‐bed repair.
Griffin 2009	Prevention	Falls	the Prevention of Falls Injury Trial (Pre‐FIT)	Community‐dwelling adults aged ≥70 years from 63 practices across England were included in this study.	Cluster‐randomised pragmatic design to test alternative falls prevention interventions (to receive one of three fall prevention interventions) of advice, exercise, and multifactorial assessment, on outcomes of falls and fractures.
Guarino 2006	Treatment	War illnesses	No name provided for the Host trial	American Gulf War veterans were included in this study.	2 x 2 factorial randomised clinical trial of exercise and cognitive behavioural therapy for the treatment of Gulf War veterans' illnesses.
Hardy 2016	Treatment	Women in labour	Birth in Upright Maternal Positionvslying down position, in women with a low‐dose Epidural, in the Second stage of labour (BUMPES) trial	Women ≥16 years, having their first child, who was admitted to a labour ward, ≥37 weeks gestation and with low dose epidural in situ were included in this study.	This was a multi‐centre RCT investigating the effect of maternal position during the late stages of labour in women with an epidural. The intervention group were upright during the second stage of labour. The control group were Laying down during the second stage of labour.
Henderson 2010	Prevention	unsafe sexual behaviours among youth.	The Sexual Health and Relationships: Safe, Happy and Responsible (SHARE) intervention	Young people aged 14 to 20, starting in school and following them into the community were included in this study. Pupils were from 47 non‐Catholic state schools within 24 km of the main cities in Tayside and Lothian regions in Scotland, UK.	This was a five‐day teacher training programme plus a 20‐session pack: 10 sessions in the third year of secondary school (at 13‐14 years) and 10 in the fourth year (at 14‐15 years). In the 12 control schools, sex education for third and fourth years varied from seven to 12 lessons in total and was primarily devoted to the provision of information and discussion.
James 2020	Prevention	Risk of falling	Occupational Therapist Intervention Study (OTIS)	Participants ≥ 65 years, community‐dwelling, currently able to walk 10 feet (with a walking aid if needed) were included in this study.	Participants were randomised to either home environmental assessment and modification, led by an occupational therapist or usual care from GP or other healthcare professional.
Dorling 2020	Treatment	Very preterm or very low‐birthweight infants	Speed of Increasing milk Feeds Trial (SIFT)	Infants born at <32 weeks' gestation or who had a birth weight of <1500 g, who were receiving <30 ml/kg/day of milk recruited from 78 UK and Republic of Ireland neonatal units were included in this study.	This was a multi‐centre, two‐arm, parallel‐group, RCT in very preterm or very low‐birthweight infants. When advancing feed volumes, participants were randomised to receive daily increments in feed volume of 30 ml/kg or 18 ml/kg.
Keding 2016	Treatment	Depression	Acupuncture and Depression (ACUDep) trial	Patients with depression from Yorkshire and northern England were included in this study.	Participants were randomised to receive either acupuncture, counselling, or usual care.
Kenton 2007	Prevention	Postnatal depression	Postpartum Depression Peer Support Trial.	Women were recruited from seven large health regions and their corresponding public health departments across Ontario, Canada were included in this study.	Study evaluating the effect of telephone‐based peer (mother to mother) support on preventing postnatal depression among women identified as high risk within the first two weeks postpartum. Participants support initiated within 48‐72 hours of randomisation, provided by a volunteer recruited from the community who had previously experienced and recovered from self‐reported postnatal depression and attended a four‐hour training session.
Kenyon 2005	Prevention	Neonatal outcomes	The MRC ORACLE Children Study (MOCS)	Participants were seven‐year‐old children whose mothers joined the MRC ORACLE Trial (Broad‐spectrum antibiotics for spontaneous preterm labour)	The original trial evaluated the use of antibiotics to improve neonatal outcome after preterm labour or preterm rupture of the membranes. Women were randomly assigned to one of four possible treatments: 325 mg co‐amoxiclav (250 mg amoxicillin and 125 mg clavulanic acid) plus 250 mg erythromycin; co‐amoxiclav plus erythromycin placebo; erythromycin plus co‐amoxiclav placebo; or co‐amoxiclav placebo plus erythromycin placebo.
Khadjesari 2011	Prevention	Alcohol consumption	Down your Drink randomised controlled trial (DYD‐RCT)	Participants were people who came across DownYourDrink while browsing the web were included in this study. Eligible participants were people drinking potentially unhealthy levels of alcohol who were also willing to consider changing their behaviour.	A two‐arm individually RCT for people with hazardous alcohol consumption. It was conducted in three phases: pilot, main trial and main trial extension. The intervention website was a theoretically informed programme based on brief intervention and psychological treatment principles. The control website used a similar graphical design and style to present simple, text‐based information about the harms caused by excess alcohol consumption.
Land 2007	Treatment	Cancer	The Study of Raloxifene and Tamoxifen (STAR).	Postmenopausal women were included in this study.	This study compared raloxifene vs tamoxifen for the prevention of breast cancer in high‐risk postmenopausal women.

Outcome or subgroup title	No. of studies	No. of participants	Statistical method	Effect size
6.1 Retention	2	1006	Risk Difference (M‐H, Random, 95% CI)	0.02 [‐0.04, 0.09]
6.2 Retention ‐ sensitivity analysis removing quasi‐RCTs	1	672	Risk Difference (M‐H, Fixed, 95% CI)	0.04 [‐0.04, 0.11]

Outcome or subgroup title	No. of studies	No. of participants	Statistical method	Effect size
8.1 Retention	3	1543	Risk Difference (M‐H, Random, 95% CI)	0.04 [‐0.00, 0.09]
8.1.1 Freepost vs second class stamp	1	930	Risk Difference (M‐H, Random, 95% CI)	0.04 [‐0.02, 0.10]
8.1.2 High priority mail stamp vs usual postage	1	281	Risk Difference (M‐H, Random, 95% CI)	0.02 [‐0.08, 0.13]
8.1.3 Personal format mailing vs business format mailing	1	332	Risk Difference (M‐H, Random, 95% CI)	0.08 [‐0.02, 0.19]

Outcome or subgroup title	No. of studies	No. of participants	Statistical method	Effect size
9.1 Retention	1	19582	Risk Difference (M‐H, Random, 95% CI)	0.09 [0.04, 0.13]
9.1.1 Directly mailed self‐sampling kit vs invitation to order salf‐sampling kit	1	6529	Risk Difference (M‐H, Random, 95% CI)	0.07 [0.05, 0.09]
9.1.2 Directly mailed self‐sampling kit vs usual follow up	1	6527	Risk Difference (M‐H, Random, 95% CI)	0.13 [0.11, 0.15]
9.1.3 Invitation to order self‐sampling vs usual follow up	1	6526	Risk Difference (M‐H, Random, 95% CI)	0.06 [0.04, 0.08]

Addition of monetary reward compared with usual follow‐up for trial retention
Patient or population: trial participants being followed up for data collection Settings: any setting Intervention: monetary reward Comparison: usual follow‐up
Outcomes	*Illustrative comparative risks (95% CI)**		Relative effect (95% CI)	No of Participants (studies)	Certainty of the evidence (GRADE)
	Assumed risk	Corresponding risk
	Monetary reward	Usual follow‐up
Retention [follow‐up]	Low^a		RR 1.02 (0.96 to 1.09)	1159 (3)	⊕⊝⊝⊝ very low
	25 per 100	26 per 100 (24 to 27)
	Medium^a
	50 per 100	51 per 100 (48 to 55)
	High^a
	80 per 100	82 per 100 (77 to 87)
The basis for the assumed risk* (e.g. the median control group risk across studies) is provided in footnotes. The effect of monetary reward (and its 95% confidence interval) is based on the assumed risk in the comparison group and the relative effect of the intervention (and its 95% CI). CI: confidence interval; RR: risk ratio.
GRADE Working Group grades of evidence High certainty: we are very confident that the true effect lies close to that of the estimate of the effect. Moderate certainty: we are moderately confident in the effect estimate; the true effect is likely to be close to the estimate of the effect, but there is a possibility that it is substantially different. Low certainty: our confidence in the effect estimate is limited; the true effect may be substantially different from the estimate of the effect. Very low certainty: we have very little confidence in the effect estimate; the true effect is likely to be substantially different from the estimate of effect.
^a We selected low, medium and high illustrative retention levels of 25%, 50% and 80% based on prior experience with trial retention and evidence from the literature. For example, it has been previously stated that it is common for up to 20% trial participants to drop out before the trial finishes (Walsh 2015), as such we set the upper limit of good retention as 80%. The other extreme of 25% was informed by evidence that some trials (largely internet based) can have retention as low as 10% to 25% (Murray 2009). The mid point of 50% was a judgement made by the review team and was deemed appropriate given the evidence for the other parameters.

Pen compared with no pen for trial retention
Patient or population: trial participants being followed up for data collection Settings: any setting Intervention: pen Comparison: no pen
Outcomes	*Illustrative comparative risks (95% CI)**		Relative effect (95% CI)	No of Participants (studies)	Certainty of the evidence (GRADE)
	Assumed risk	Corresponding risk
	Pen	No pen
Retention [follow‐up]	Low^a		RR 1.02 (1.00 to 1.05)	13013 (5)	⊕⊕⊝⊝ low
	25 per 100	26 per 100 (25 to 26)
	Medium^a
	50 per 100	51 per 100 50 to 53)
	High^a
	80 per 100	82 per 100 (80 to 84)
The basis for the assumed risk* (e.g. the median control group risk across studies) is provided in footnotes. The corresponding risk (and its 95% confidence interval) is based on the assumed risk in the comparison group and the relative effect of the intervention (and its 95% CI). CI: confidence interval; RR: risk ratio.
GRADE Working Group grades of evidence High certainty: we are very confident that the true effect lies close to that of the estimate of the effect. Moderate certainty: we are moderately confident in the effect estimate; the true effect is likely to be close to the estimate of the effect, but there is a possibility that it is substantially different. Low certainty: our confidence in the effect estimate is limited; the true effect may be substantially different from the estimate of the effect. Very low certainty: we have very little confidence in the effect estimate; the true effect is likely to be substantially different from the estimate of effect.
^a We selected low, medium and high illustrative retention levels of 25%, 50% and 80% based on prior experience with trial retention and evidence from the literature. For example, it has been previously stated that it is common for up to 20% trial participants to drop out before the trial finishes (Walsh 2015), as such we set the upper limit of good retention as 80%. The other extreme of 25% was informed by evidence that some trials (largely internet based) can have retention as low as 10% to 25% (Murray 2009). The mid point of 50% was a judgement made by the review team and was deemed appropriate given the evidence for the other parameters.

Newsletter compared with usual follow‐up for trial retention
Patient or population: trial participants being followed up for data collection Settings: any setting Intervention: newsletter Comparison: usual follow‐up
Outcomes	*Illustrative comparative risks (95% CI)**		Relative effect (95% CI)	No of Participants (studies)	Certainty of the evidence (GRADE)
	Assumed risk	Corresponding risk
	Newsletter	Usual follow‐up
Retention [follow‐up]	Low^a		RR 0.99 (0.95 to 1.04)	5622 (4)	⊕⊝⊝⊝ very low
	25 per 100	25 per 100 (24 to 26)
	Medium^a
	50 per 100	50 per 100 (48 to 52)
	High^a
	80 per 100	79 per 100 (76 to 83)
The basis for the assumed risk* (e.g. the median control group risk across studies) is provided in footnotes. The corresponding risk (and its 95% confidence interval) is based on the assumed risk in the comparison group and the relative effect of the intervention (and its 95% CI). CI: confidence interval; RR: risk ratio.
GRADE Working Group grades of evidence High certainty: we are very confident that the true effect lies close to that of the estimate of the effect. Moderate certainty: we are moderately confident in the effect estimate; the true effect is likely to be close to the estimate of the effect, but there is a possibility that it is substantially different. Low certainty: our confidence in the effect estimate is limited; the true effect may be substantially different from the estimate of the effect. Very low certainty: we have very little confidence in the effect estimate; the true effect is likely to be substantially different from the estimate of effect.
^a We selected low, medium and high illustrative retention levels of 25%, 50% and 80% based on prior experience with trial retention and evidence from the literature. For example, it has been previously stated that it is common for up to 20% trial participants to drop out before the trial finishes (Walsh 2015), as such we set the upper limit of good retention as 80%. The other extreme of 25% was informed by evidence that some trials (largely internet based) can have retention as low as 10% to 25% (Murray 2009). The mid point of 50% was a judgement made by the review team and was deemed appropriate given the evidence for the other parameters.

Post‐it note compared with usual follow‐up for trial retention
Patient or population: trial participants being followed up for data collection Settings: any setting Intervention: post‐it note Comparison: usual follow‐up
Outcomes	*Illustrative comparative risks (95% CI)**		Relative effect (95% CI)	No of Participants (studies)	Certainty of the evidence (GRADE)
	Assumed risk	Corresponding risk
	Post‐it note	Usual follow‐up
Retention [follow‐up]	Low^a		RR 1.00 (0.99 to 1.01)	4698 (3)	⊕⊕⊝⊝ low
	25 per 100	25 per 100 (25 to 25)
	Medium^a
	50 per 100	50 per 100 (50 to 51)
	High^a
	80 per 100	80 per 100 (79 to 81)
The basis for the assumed risk* (e.g. the median control group risk across studies) is provided in footnotes. The effect of a post‐it note (and its 95% confidence interval) is based on the assumed risk in the comparison group and the relative effect of the intervention (and its 95% CI). CI: confidence interval; RR: risk ratio.
GRADE Working Group grades of evidence High certainty: we are very confident that the true effect lies close to that of the estimate of the effect. Moderate certainty: we are moderately confident in the effect estimate; the true effect is likely to be close to the estimate of the effect, but there is a possibility that it is substantially different. Low certainty: our confidence in the effect estimate is limited; the true effect may be substantially different from the estimate of the effect. Very low certainty: we have very little confidence in the effect estimate; the true effect is likely to be substantially different from the estimate of effect.
^a We selected low, medium and high illustrative retention levels of 25%, 50% and 80% based on prior experience with trial retention and evidence from the literature. For example, it has been previously stated that it is common for up to 20% trial participants to drop out before the trial finishes (Walsh 2015), as such we set the upper limit of good retention as 80%. The other extreme of 25% was informed by evidence that some trials (largely internet based) can have retention as low as 10% to 25% (Murray 2009). The mid point of 50% was a judgement made by the review team and was deemed appropriate given the evidence for the other parameters.

Theory informed cover letter compared with usual cover letter for trial retention
Patient or population: trial participants being followed up for data collection Settings: any setting Intervention: theory informed cover letter Comparison: usual cover letter
Outcomes	*Illustrative comparative risks (95% CI)**		Relative effect (95% CI)	No of Participants (studies)	Certainty of the evidence (GRADE)
	Assumed risk	Corresponding risk
	Theory informed cover letter	Usual cover letter
Retention [follow‐up]	Low^a		RR 1.05 (0.98 to 1.12)	3343 (4)	⊕⊝⊝⊝ very low
	25 per 100	26 per 100 (25 to 28)
	Medium^a
	50 per 100	53 per 100 (49 to 56)
	High^a
	80 per 100	84 per 100 (78 to 90)
The basis for the assumed risk* (e.g. the median control group risk across studies) is provided in footnotes. The effect of a theory informed cover letter (and its 95% confidence interval) is based on the assumed risk in the comparison group and the relative effect of the intervention (and its 95% CI). CI: confidence interval; RR: risk ratio.
GRADE Working Group grades of evidence High certainty: we are very confident that the true effect lies close to that of the estimate of the effect. Moderate certainty: we are moderately confident in the effect estimate; the true effect is likely to be close to the estimate of the effect, but there is a possibility that it is substantially different. Low certainty: our confidence in the effect estimate is limited; the true effect may be substantially different from the estimate of the effect. Very low certainty: we have very little confidence in the effect estimate; the true effect is likely to be substantially different from the estimate of effect.
^a We selected low, medium and high illustrative retention levels of 25%, 50% and 80% based on prior experience with trial retention and evidence from the literature. For example, it has been previously stated that it is common for up to 20% trial participants to drop out before the trial finishes (Walsh 2015), as such we set the upper limit of good retention as 80%. The other extreme of 25% was informed by evidence that some trials (largely internet based) can have retention as low as 10% to 25% (Murray 2009). The mid point of 50% was a judgement made by the review team and was deemed appropriate given the evidence for the other parameters.

Addition of optimised information compared with standard information for trial retention
Patient or population: trial participants being followed up for data collection Settings: any setting Intervention: optimised patient information leaflet (PIL) Comparison: standard PIL
Outcomes	*Illustrative comparative risks (95% CI)**		Relative effect (95% CI)	No of Participants (studies)	Certainty of the evidence (GRADE)	Comments
	Assumed risk	Corresponding risk
	Optimised PIL	Standard PIL
Retention [follow‐up]	Low^a		RR 0.96 (0.85 to 1.09)	1285 (2)	⊕⊝⊝⊝ very low
	25 per 100	24 per 100 (21 to 27)
	Medium^a
	50 per 100	48 per 100 (43 to 55)
	High^a
	80 per 100	77 per 100 (68 to 87)
The basis for the assumed risk* (e.g. the median control group risk across studies) is provided in footnotes. The corresponding risk (and its 95% confidence interval) is based on the assumed risk in the comparison group and the relative effect of the intervention (and its 95% CI). CI: confidence interval; RR: risk ratio.
GRADE Working Group grades of evidence High certainty: we are very confident that the true effect lies close to that of the estimate of the effect. Moderate certainty: we are moderately confident in the effect estimate; the true effect is likely to be close to the estimate of the effect, but there is a possibility that it is substantially different. Low certainty: our confidence in the effect estimate is limited; the true effect may be substantially different from the estimate of the effect. Very low certainty: we have very little confidence in the effect estimate; the true effect is likely to be substantially different from the estimate of effect.
^a We selected low, medium and high illustrative retention levels of 25%, 50% and 80% based on prior experience with trial retention and evidence from the literature. For example, it has been previously stated that it is common for up to 20% trial participants to drop out before the trial finishes (Walsh 2015), as such we set the upper limit of good retention as 80%. The other extreme of 25% was informed by evidence that some trials (largely internet based) can have retention as low as 10% to 25% (Murray 2009). The mid point of 50% was a judgement made by the review team and was deemed appropriate given the evidence for the other parameters.

Date	Event	Description
22 January 2021	New citation required and conclusions have changed	Based on the revisions made during this update, the overall findings of the review and its conclusions have changed substantially.
22 January 2021	New search has been performed	Review updated. Significant amendments during update include the following. Revision of overall scope to focus and restrict to interventions that are designed to maximise data collection from trial participants once they have been recruited and randomised. It does not include interventions that aim to improve adherence to trial interventions. A new search was run from data base inception to January 2020. Based on this refinement in scope some studies that were included in the original review have now been excluded (Bowen 2000, Chaffin 2009, Cox 2008, Ford 2004, Hughes 1989, Leigh Brown 1997, Svodoba 2001) and one study that was excluded from the original review has now been included (Marsh 1999). We used the ORRCA (https://www.orrca.org.uk/) domains to inform categorisation of interventions and comparisons. The lack of any secondary outcomes is a change from the previous version of the review which stated "Retention of participants at secondary analysis points": as a secondary outcome. Due to these data being rarely reported this outcome has been removed as a secondary outcome. The review has now applied GRADE to all comparisons that include more than one study

Outcome or subgroup title	No. of studies	No. of participants	Statistical method	Effect size
21.1 Retention	3	3166	Risk Difference (M‐H, Random, 95% CI)	0.07 [0.04, 0.11]
21.2 Retention‐ sensitivity anlysis removing quasi‐RCTs	2	1022	Risk Difference (M‐H, Random, 95% CI)	0.09 [0.02, 0.16]

Outcome or subgroup title	No. of studies	No. of participants	Statistical method	Effect size
24.1 Retention	3	1159	Risk Difference (M‐H, Random, 95% CI)	0.02 [‐0.03, 0.06]
24.1.1 Return of postal questionnaires	2	725	Risk Difference (M‐H, Random, 95% CI)	0.00 [‐0.06, 0.07]
24.1.2 Attendance at follow up visits	1	434	Risk Difference (M‐H, Random, 95% CI)	0.03 [‐0.04, 0.10]
24.2 Retention ‐ sensitivity analysis removing quasi‐RCTs	2	955	Risk Difference (M‐H, Random, 95% CI)	0.01 [‐0.04, 0.06]
24.2.1 Return of postal questionnaires	1	521	Risk Difference (M‐H, Random, 95% CI)	‐0.02 [‐0.09, 0.06]
24.2.2 Attendance at follow up visits	1	434	Risk Difference (M‐H, Random, 95% CI)	0.03 [‐0.04, 0.10]

Outcome or subgroup title	No. of studies	No. of participants	Statistical method	Effect size
39.1 Retention	3	4698	Risk Difference (M‐H, Random, 95% CI)	0.00 [‐0.01, 0.01]
39.1.1 Printed post‐it note vs no post‐it note	2	1165	Risk Difference (M‐H, Random, 95% CI)	0.00 [‐0.02, 0.03]
39.1.2 Handwritten post‐it note vs no post‐it note	2	1051	Risk Difference (M‐H, Random, 95% CI)	‐0.00 [‐0.03, 0.02]
39.1.3 Printed post‐it note vs handwritten post‐it note	1	546	Risk Difference (M‐H, Random, 95% CI)	0.00 [‐0.02, 0.03]
39.1.4 Post‐it note vs no post‐it note	3	1936	Risk Difference (M‐H, Random, 95% CI)	‐0.00 [‐0.02, 0.02]

PERMALINK

Strategies to improve retention in randomised trials

Katie Gillies

Anna Kearney

Ciara Keenan

Shaun Treweek

Jemma Hudson

Valerie C Brueton

Thomas Conway

Andrew Hunter

Louise Murphy

Peter J Carr

Greta Rait

Paul Manson

Magaly Aceves-Martins

Abstract

Background

Objectives

Search methods

Selection criteria

Data collection and analysis

Main results

Authors' conclusions

Plain language summary

Summary of findings

Background

Description of the methods being investigated

Objectives

Methods

Criteria for considering studies for this review

Types of studies

Types of data

Types of methods

Types of outcome measures

Primary outcomes

Secondary outcomes

Search methods for identification of studies

Electronic searches

Searching other resources

Data collection and analysis

Selection of studies

Data extraction and management

Assessment of risk of bias in included studies

Measures of the effect of the methods

Unit of analysis issues

Dealing with missing data

Assessment of heterogeneity

Assessment of reporting biases

Data synthesis

Subgroup analysis and investigation of heterogeneity

Sensitivity analysis

Results

Description of studies

1.

1. Countries where the included studies took place.

Risk of bias in included studies

2.

3.

Effect of methods

2. Data collection (Category A).

3. Participants (Category B).

4. Sites and site staff (Category C).

5. Central Study Management (Category D).

6. Study design (Category E).

Summary of findings 1. Questionnaire design: short vs usual questionnaire.

Summary of findings 2. Questionnaire design: addition of diary to usual follow‐up vs usual follow‐up.

Summary of findings 3. Data collection location and method: telephone follow‐up vs postal questionnaire.

Summary of findings 4. Data collection location and method: return postage.

Summary of findings 5. Reminders: electronic reminder vs usual follow‐up.

Summary of findings 6. Prompts: Electronic prompt vs no prompt.

Summary of findings 7. Prompts: Telephone prompt vs usual follow‐up.

Summary of findings 8. Prompts: personalised prompt vs usual follow‐up.

Summary of findings 9. Monetary incentives: addition of monetary incentives vs usual follow‐up.

Summary of findings 10. Monetary incentives: addition of monetary incentives vs addition of a monetary reward.

Summary of findings 11. Monetary incentives: addition of monetary reward vs usual follow‐up.

Summary of findings 12. Non‐monetary incentives: addition of pen vs usual follow‐up.

Summary of findings 13. Maintaining participant engagement: newsletter vs usual follow‐up.

Summary of findings 14. Maintaining participant engagement: post‐it note vs usual follow‐up.

Summary of findings 15. Behavioural interventions: theory informed cover letter vs usual cover letter.

Summary of findings 16. Impact of recruitment: optimised information vs standard information.

Outcome or subgroup title	No. of studies	No. of participants	Statistical method	Effect size
42.1 Retention	1	408	Risk Difference (M‐H, Fixed, 95% CI)	0.07 [‐0.01, 0.16]
42.1.1 Request for a collateral vs no request	1	100	Risk Difference (M‐H, Fixed, 95% CI)	0.12 [‐0.04, 0.28]
42.1.2 Request for a collateral with 50% chance of use vs no request	1	152	Risk Difference (M‐H, Fixed, 95% CI)	0.06 [‐0.09, 0.21]
42.1.3 Request for a collateral vs Request for a collateral with 50% chance of use	1	156	Risk Difference (M‐H, Fixed, 95% CI)	0.06 [‐0.07, 0.18]

*Study characteristics*
Methods	Parallel RCT, individuals randomised
Data	UK, secondary care settings. Not all participants in the host trial that took part in this embedded trial. Only those who had yet to complete the study at the time of the SWAT set up. Total n = 64, age NR; sex NR.
Comparisons	Intervention group received a tested a theoretically informed letter sent with the questionnaire Control group received a standard letter
Outcomes	Questionnaires returned
Notes	6 months
*Risk of bias*
Item	Authors' judgement	Support for judgement
Allocation concealment?	Unclear	No information of the nested RCT was provided.
Adequate sequence generation?	No	According to the authors, the randomisation list was generated and was not concealed.
Blinding of participants and personnel?	Unclear	Participants were unaware if they were receiving a standard or theory‐based cover letter but may have noticed from earlier letters that it had a different “tone”
Blinding of outcome assessment?	Unclear	No information of the nested RCT was provided.
Incomplete outcome data addressed? All outcomes	Unclear	No concerns raised.
Free of selective outcome reporting?	Unclear	No concerns raised.
Other sources of bias	No	No further concerns raised.
Overall Risk of Bias	Unclear	Unclear

*Study characteristics*
Methods	RCT, individuals randomised
Data	UK, secondary care setting. Participants were those approached about the host trial and willing to receive further information about taking part. Total n = 180; mean age 77 (SD 5) years; 82.8% females
Comparisons	Intervention group received full information provided to eligible people, who were approached by a nurse and informed they would know their treatment allocation Control group received conventional trial methods, comprising randomised blinded, placebo‐controlled.
Outcomes	Participants remaining in the study (i.e. not withdrawn).
Notes	Retention period: 12 months.
*Risk of bias*
Item	Authors' judgement	Support for judgement
Allocation concealment?	Yes	The study nurse then used a pre‐programmed laptop computer to generate random allocation to either the open‐trial design or the blinded, placebo‐controlled trial design in a 1: 2 ratio.
Adequate sequence generation?	Yes	Computer used to generate random allocation to either the open‐trial design or the blinded.
Blinding of participants and personnel?	Unclear	Participant unblinded but unblinding not likely to impact objective outcome.
Blinding of outcome assessment?	Yes	Not reported in the paper.However, objective outcome, staff have no plausible additional opportunity to influence postal response rate once questionnaires sent.
Incomplete outcome data addressed? All outcomes	Yes	No concerns raised.
Free of selective outcome reporting?	Yes	No concerns raised.
Other sources of bias	Yes	No further concerns raised.
Overall Risk of Bias	Unclear	Unclear

*Study characteristics*
Methods	R 2x2 factorial RCT, individuals randomised
Data	UK, community setting (online) Host trial respondents who agreed to be contacted. However, 15 were randomly selected from each of the 20 original US communities to participate. Total n= 1030; range age 18‐20 years; sex NR
Comparisons	Interventions group 1 were allocated to a postal request for a urine sample for genital chlamydia testing and receipt for £10 shopping voucher Interventions group 2 Interventions group 1 were allocated to a postal request for a urine sample for genital chlamydia testing and receipt for £20 shopping voucher Control group no request for a sample nor vouchers
Outcomes	Completion of sexual health survey and return of kits
Notes	Retetnion period: 3 months
*Risk of bias*
Item	Authors' judgement	Support for judgement
Allocation concealment?	Yes	At recruitment, all participants were randomised in a factorial (2x2) design to either the intervention or control website and to receive or not receive a urine sample kit for chlamydia testing at follow‐up. In addition, the final 902 participants were randomised after recruitment to a £10 or £20 voucher to complete follow‐up as requested.
Adequate sequence generation?	Yes	The first two randomisations were performed using an automated computer algorithm and the third was performed off‐site by random permutation of participant identifiers. Participants were automatically allocated by computer to control or intervention after submitting baseline data.
Blinding of participants and personnel?	Unclear	Not reported in the paper.
Blinding of outcome assessment?	Unclear	Not reported in the paper.
Incomplete outcome data addressed? All outcomes	Unclear	Not reported in the paper.
Free of selective outcome reporting?	Unclear	Not reported in the paper.
Other sources of bias	Unclear	No further concerns raised.
Overall Risk of Bias	Unclear	Unclear

*Study characteristics*
Methods	Parallel factorial RCT, individuals randomised
Data	UK, mainly secondary care centres with a smaller number of primary care centres. All host trial participants, as a source of the retention trial sample. Total n = 1394; age NR; sex NR
Comparisons	Intervention groups 1 received an SMS message (text message) was sent the day before the e‐mail with the link to the questionnaire. Intervention groups 2 received a further £10 high‐street shopping voucher sent by post to parents at around 22 months before the 24‐month visit. Intervention groups 3 received a further £10 high‐street shopping voucher given at the visit. Control group received no SMS or voucher.
Outcomes	Collection of outcome data
Notes	Retention periods: 3‐, 6‐, 12‐, 18‐ and 24‐months
*Risk of bias*
Item	Authors' judgement	Support for judgement
Allocation concealment?	Yes	Using an allocation schedule created by the Nottingham Clinical Trials Unit
Adequate sequence generation?	Yes	Research nurses randomised participants to the BEEP host trial by accessing an online system provided by the co‐ordinating centre. The second randomisation was then automatically performed for the SWAT to each of the retention strategies (1:1).
Blinding of participants and personnel?	No	Participants were informed in the host trial information sheet about the SWAT for SMS notification for questionnaires and timing of the voucher for the 24‐month visit but were not informed at the time of the randomisation of their allocated groups for the SWAT. Research nurses were not blinded
Blinding of outcome assessment?	Yes	The Trial Management Team and the research nurses were not blinded to the allocations for the SWAT. The sequence of allocations for the SWAT was concealed from the statisticians until the database was locked in the host trial. However, objective outcome, participants blind (did not know there was a study) staff have no plausible additional opportunity to influence postal response rate once questionnaires sent.
Incomplete outcome data addressed? All outcomes	Yes	No concerns raised.
Free of selective outcome reporting?	Yes	No concerns raised.
Other sources of bias	Yes	No further concerns raised.
Overall Risk of Bias	No	No

*Study characteristics*
Methods	RCT, individuals randomised.
Data	US, multi‐centre setting. Participants who completed 24‐month follow‐up outcomes were included. Total n= 305, mean age 57.4 (SD 10.8) years; 100% females; 83.9% White.
Comparisons	Intervention group received a standardised video followed by the usual consent process. Control group received the usual consent process alone.
Outcomes	Follow‐up data collection.
Notes	Retention period: 36‐, 48‐ and 60‐months (a window of 3 months on each side of the visit was allowed for data collection).
*Risk of bias*
Item	Authors' judgement	Support for judgement
Allocation concealment?	Unclear	The study co‐ordinator was not masked to the enrolment intervention
Adequate sequence generation?	Unclear	Not discussed in the paper.
Blinding of participants and personnel?	Unclear	Only mentions that the study co‐ordinator was not masked to the enrolment intervention. However, unclear for participants or other personnel.
Blinding of outcome assessment?	Yes	Evaluators of outcome assessments were masked to surgical, behavioural therapy with pelvic floor muscle training and enrolment process randomisations
Incomplete outcome data addressed? All outcomes	Yes	The detail provided on lost and withdrawn participants
Free of selective outcome reporting?	Yes	They present and describe additional outcomes as 'exploratory'.
Other sources of bias	Unclear	No further concerns raised.
Overall Risk of Bias	Unclear	Unclear

*Study characteristics*
Methods	Parallel RCT, clusters randomised
Data	UK, community setting. Participants who were due to be sent a final follow‐up questionnaire as part of the host trial. Total n = 1038; mean age 76.4 (SD 4.6); 100% females.
Comparisons	Intervention group participants were offered the results of the trial in a postal questionnaire. Control group participants did not receive the offer of knowing the results of the trial.
Outcomes	Return of final follow‐up questionnaire.
Notes	Retention period: Three weeks. Those participants nor returning questionnaires within three weeks were sent up to two reminder letters, questionnaires and business reply envelopes, three and six weeks after the initial mailing.
*Risk of bias*
Item	Authors' judgement	Support for judgement
Allocation concealment?	Unclear	Not reported in the paper.
Adequate sequence generation?	Yes	An independent researcher from the York Trials Unit randomised eligible women in a 3:1 ratio in favour of offering the results of the trial, by computer
Blinding of participants and personnel?	Yes	Not clear if participants were blinded to effect of embedded trial intervention on retention but unblinding not likely to impact objective outcome.
Blinding of outcome assessment?	Yes	Statistical analysis was not undertaken blind to group allocation. However, objective outcome, participants blind (did not know there is a study) staff have no plausible additional opportunity to influence postal response rate once questionnaires sent.
Incomplete outcome data addressed? All outcomes	Yes	No concerns raised.
Free of selective outcome reporting?	Yes	No concerns raised.
Other sources of bias	Yes	No further concerns raised.
Overall Risk of Bias	Unclear	Unclear

*Study characteristics*
Methods	Parallel RCT, cluster randomisation
Data	UK, primary care setting. Host trial was a multinational trial. Only UK sites were included. 42 out of the 43 participant GP surgeries were included. Total n = 345; median age of intervention group 26 (IQR 12,46) years and control group median 36 (IQR 23,53) years; 60.4% females.
Comparisons	Intervention group received a £20 gift voucher given to study at the end of the recruitment visit. Control group were informed that a £20 gift voucher would be given upon the return of the trial symptom diary, and voucher then sent to those who returned a diary by the trial team.
Outcomes	Proportion of participants returning a symptom diary.
Notes	Retention period: 14 days.
*Risk of bias*
Item	Authors' judgement	Support for judgement
Allocation concealment?	Unclear	Not discussed in the paper.
Adequate sequence generation?	Yes	The trial sites were the unit of randomisation. The sites were cluster‐randomised to keep the process as straight forward as possible for the recruiters. Randomisation was performed in two waves (before the start of seasons 2 and 3) using computer‐generated random numbers carried by one of the investigators.
Blinding of participants and personnel?	Yes	GP practices were not blinded to their allocation due to the necessity of them either distributing the incentives initially or not. Participants were unaware that there was a SWAT taking place as it was thought this might influence whether or not they returned their diary.Unblinding not likely to impact objective outcome
Blinding of outcome assessment?	Yes	Not discussed in the paper. However, objective outcome, staff have no plausible additional opportunity to influence postal response rate once questionnaires sent.
Incomplete outcome data addressed? All outcomes	Yes	Whether the Symptom diary was returned, when it was received and how many pages were completed. There was no imputation of outcome data for sites where no‐one was recruited, or where a participant did not return their Symptom diary.
Free of selective outcome reporting?	Yes	All data reported for pre‐specified objectives.
Other sources of bias	Yes	No further concerns raised.
Overall Risk of Bias	Unclear	Unclear

*Study characteristics*
Methods	Parallel RCT, individuals randomised.
Data	Canada, community setting. Participants who provided address to participate in the 6‐month follow‐up. Total n = 204, age NR, sex NR.
Comparisons	Intervention group1 were asked to provide a collateral. Intervention group2 asked to provide collateral and told that there was a 50% chance that the collateral would be contacted. All those respondents asked for collateral were told that the collateral would receive a CAN$20 payment for a brief telephone interview. Control group was not asked to provide a collateral.
Outcomes	Proportion of respondents who returned the survey at follow‐up.
Notes	Retention period: 6 months.
*Risk of bias*
Item	Authors' judgement	Support for judgement
Allocation concealment?	Unclear	Not discussed in the paper.
Adequate sequence generation?	Unclear	Not discussed in the paper.
Blinding of participants and personnel?	Unclear	Not discussed in the paper.
Blinding of outcome assessment?	Unclear	Not discussed in the paper.
Incomplete outcome data addressed? All outcomes	Yes	Outcomes seem to be clearly reported.
Free of selective outcome reporting?	Yes	No concerns raised.
Other sources of bias	Unclear	No further concerns raised.
Overall Risk of Bias	Unclear	Unclear

*Study characteristics*
Methods	Two RCTs, individuals randomised.
Data	US, secondary care setting. 1) Trial 1 (mail trial): participants who had been enrolled before the introduction of this survey were sequentially randomised. Total n = 332; mean age 48.8 (SD 15.0) years; 52% females; 20% of ethnic minorities (i.e. African American, Asian, American Indian and Alaskan Native), 24% unemployed; 46% retired or disabled. 2) Trial 2 (Phone trial): non‐responders from the prior mail and those excluded from mail trial due to lack of a correct mailing address were eligible. Total n = 171; mean age 46.0 (SD 14.7) years; 51% females; 26% of ethnic minorities (i.e. African American, Asian, American Indian and Alaskan Native), 26% unemployed; 43% retired or disabled.
Comparisons	1) Trial 1 (Mail trial) participants were randomised to mailed letters every two weeks until the survey was completed. Intervention group 1 received a "personal format letter" in which their mailing address and the return address were handwritten, and a traditional stamp was stamped using the envelope. Intervention group 2 received a "business format letter" in which the addresses were typed, and a commercial stamp‐machine affixed the postage. 2) Trial 2 (Phone trial): started 20 days after the end of the mail trial. These telephone calls were made once weekly by the same caller, for up to 4 weeks, until the participant was reached by telephone or the participant called back and completed the survey. Intervention group receive a personalised voice message. Control group received generic voice message.
Outcomes	Participant to complete the insurance survey.
Notes	Retention period: 1) Trial 1 (mail trial): unclear. However, every two weeks until the survey was completed, or the participant was sent a total of 4 mailings. Trial 2 (Phone trial): unclear. However, once weekly by the same caller, for up to 4 weeks
*Risk of bias*
Item	Authors' judgement	Support for judgement
Allocation concealment?	Unclear	Not discussed in the paper.
Adequate sequence generation?	Yes	In both the mail and the phone trials, randomisation was performed by a statistician using computer‐generated random numbers with an allocation ratio of 1:1.
Blinding of participants and personnel?	Yes	Given the nature of this study design, outcome assessment was not blinded, but participants were blinded. Unclear about the personnel. unblinding not likely to impact objective outcome
Blinding of outcome assessment?	Yes	Given the nature of this study design, outcome assessment was not blinded. However, objective outcome, participants blinded (did not know there was a study) telephone contact was only those participants who received the scripted message, not those who were spoken to, staff have no/very limited plausible additional opportunity to influence postal response rate once questionnaires sent.
Incomplete outcome data addressed? All outcomes	Yes	No concerns raised.
Free of selective outcome reporting?	Yes	No concerns raised.
Other sources of bias	Yes	No further concerns raised.
Overall Risk of Bias	Unclear	Unclear