Study within a trial (SWAT) protocol. Investigating the effect of personalised versus non-personalised study invitations on recruitment: An embedded randomised controlled recruitment trial

Abstract

Introduction

Recruitment into clinical trials is a common challenge experienced by healthcare researchers. Currently, there is little evidence regarding strategies to improve recruitment into clinical trials. However, preliminary research suggests the personalisation of study invitation letters may increase recruitment rates. As such, there is a need to investigate the effectiveness of personalisation strategies on trial recruitment rates. This study within a trial (SWAT) will investigate the effect of personalised versus non-personalised study invitation letters on recruitment rates into the host trial ENGAGE, a feasibility study of an internet-administered, guided, Cognitive-Behavioural Therapy (CBT) based self-help intervention for parents of children previously treated for cancer.

Methods

An embedded randomised controlled trial (RCT) will investigate the effectiveness of a personalised study invitation letter including the potential participant’s name and address compared with a standard, non-personalised letter without name or address, on participant recruitment rates into the ENGAGE study. The primary outcome is differences in the proportion of participants recruited, examined using logistic regression. Results will be reported as adjusted odds ratios with 95% confidence intervals.

Discussion

Even moderate effects of the personalisation of study invitation letters on recruitment rates could be of significant value by shortening study length, saving resources, and providing a faster answer to the clinical question posed by the study. This protocol can be used as a template for other researchers who wish to contribute to the evidence base for trial decision-making, by embedding a similar SWAT into their trial.

Trial registration

ISRCTN 57233429; ISRCTN 18404129; SWAT 112, Northern Ireland Hub for Trials Methodology Research SWAT repository (2018 OCT 1 1231).

1. Introduction

Successful recruitment of participants is a common challenge to clinical trial conduct. Indeed, a review of publicly funded randomised controlled trials (RCTs) in the United Kingdom identified that only 56% of trials recruited to target, 53% received a recruitment extension, and overall there was substantial variation in recruitment rates across trials [1]. Furthermore, a review of terminated trials in ClinicalTrials.gov indicated in 56.5% of clinical trials, inadequate accrual rate was the reason for early trial termination [2]. Difficulties in participant recruitment result in considerable research waste [3]. For example, successful recruitment is essential for clinical trials to reach statistical power and maximise internal and external validity [4,5]. Further, early trial termination due to poor recruitment may result in unnecessary inclusion of participants in studies that cannot answer important research questions [6] and recruitment delays may negatively affect the health of patients by preventing the realization of timely research impact [5].

In the context of the present host trial ENGAGE [7], the target population are parents of children previously treated for cancer who are experiencing psychological distress. Psychological distress is commonly experienced by parents after the child’s cancer treatment has ended, with parents commonly reporting symptoms of PTSS, depression, anxiety, fear of reoccurrence, and sleep difficulties [[8], [9], [10], [11]]. However, participant recruitment into clinical trials of psychological interventions appears to be particularly difficult. Research focusing on recruitment into depression trials has proposed stigma and barriers related to attitudes towards interventions and trust as contributing factors [12]. Further, depression symptomology (e.g., lack of energy, poor motivation, and difficulties concentrating) may hinder help seeking and partly explain poor recruitment into depression trials [13]. In addition, previous trials of psychological interventions for parents of children and adolescents with chronic health conditions, including cancer, have experienced difficulties with small sample sizes, thus indicating possible recruitment challenges [14].

Despite the significant negative impact of poor participant recruitment on clinical trials, there is little evidence to inform the utilisation of successful recruitment strategies [15], including within the field of mental health [16]. Recent review findings only identified two recruitment strategies with high-certainty evidence: (1) telephoning non-responders and (2) choosing an open rather than blind trial design [15]. Further, recruitment strategies are often poorly reported in trials, and thus it is difficult to identify potentially effective recruitment strategies from trial reports [17]. Given aforementioned difficulties, the identification of evidence-based strategies to improve clinical trial recruitment is urgently needed.

One suggested recruitment strategy is the personalisation of study invitation letters, whereby potential participants are referred to by their name, rather than receiving generic invitations [18]. Existing evidence from cognitive psychology recognises the use of a person’s name increases the likelihood of attracting attention [19] and helps people filter out competing stimuli and refocus attention [20]. Importantly, seeing one’s name in printed text heightens attention [21]. The personalisation of study invitations has been found to be successful for recruitment rates in survey research [22,23] and an RCT demonstrated breast cancer survivors were more likely to respond to a personalised study invitation e-mail, compared to a non-personalised e-mail [18]. Further, another RCT demonstrated payment of delinquent fines was increased when receiving a personalised text message compared to a non-personalised text message [24]. In addition, inclusion of the name of deceased cancer patients was associated with significantly improved proxy survey response rates from bereaved family members [25]. However, whilst research concerning the use of personalised study invitations shows promise, few studies utilising an RCT design or in the context of clinical healthcare research have been conducted. To the best of our knowledge, no RCT has examined personalisation of study invitations in the context of mental health research.

As such, the objective of the present study within a trial (SWAT) is to investigate the effect of personalised study invitation letters on recruitment rates compared with non-personalised study invitation letters. SWATs are designed to improve the evidence-base concerning trial processes, such as improving recruitment or retention [26] and are typically designed to be embedded into the context of a larger host trial [27]. The present SWAT utilises an RCT design and will be embedded in the host trial ENGAGE [7]. ENGAGE is a feasibility study of an internet-administered, guided, Cognitive-Behavioural Therapy (CBT) based self-help intervention, EJDeR, for parents of children previously treated for cancer. ENGAGE aims to recruit 50 participants during a six month period between May 2020 and November 2020.

2. Methods

This protocol is reported in accordance with guidelines for reporting embedded recruitment trials [28] based on the Consolidated Standards for Reporting Trials (CONSORT) statement 2010 [29].

2.1. Study design

A parallel group embedded RCT to investigate the effect of personalised study invitation letters compared with non-personalised study invitation letters on recruitment rates.

2.2. Participants

Participants eligible for inclusion in the ENGAGE host trial are: (1) a parent of a child diagnosed with cancer during childhood (0–18 years) who has completed cancer treatment three months to five years previously; (2) lives in Sweden; (3) able to read Swedish; (4) access to e-mail, the internet, and a mobile telephone and/or BankID (Swedish citizen authorisation system); and (5) self-report a need for psychological support. For the purposes of the present recruitment SWAT, all potential participants who are invited to participate in the ENGAGE host trial will be included (see Fig. 1).

Fig. 1.

Study flow of study within a trial (SWAT) participants in the ENGAGE host trial.

2.3. Setting

The ENGAGE intervention, EJDeR (Swedish acronym), is delivered online via the U-CARE-portal (Portal). EJDeR has been visually optimised for use on a computer/laptop screen, however it can also be accessed by participants on mobile devices. Therefore, participants are anticipated to use EJDeR both inside and outside of their own home, or other community settings. Participants will be supported by e-therapists located at the Department of Women’s and Children’s Health, Uppsala University. Data will be collected online, via the Portal, and/or over the telephone by members of the research team.

2.4. Recruitment

The ENGAGE host trial utilises two recruitment strategies: (1) postal study invitation letters; and (2) social media and patient organisation websites. For the purposes of the SWAT, only participants recruited via postal study invitation letters will be included. However, it is possible that participants recruited via postal study invitation letters may also come across study advertisements on social media and patient organisation websites. To examine this possibility, all participants will be asked about source ofrecruitment (e.g., how did they find out about the study).

Children’s personal identification details will be received from the Swedish Childhood Cancer Registry (National Quality Registry) and linked to parents’ names and addresses via the Swedish Tax Agency Registry NAVET (Population Registry). Potential participants will be invited in blocks of 100. Data concerning the child's age, gender, cancer diagnosis, and date of first diagnosis will be provided via the Swedish Childhood Cancer Registry. Still, upon invitation into the study, the mental health status of the parent will not be known. A study information pack will be sent to each potential participant's home address and will include: a study invitation letter, a study information sheet, a link to the study website and a reply slip with a stamped addressed envelope. As independent study invitation packs will be sent to parents, there is the possibility that two parents of the same child could participate in the study. Study invitation packs clearly state that the study is designed for parents who self-report a need for psychological support related to their child's cancer disease and treatment, moreover this will be asked during the trial recruitment process. Potential participants will also be able to opt out of further contact with the study team by either sending a paper opt out form via the post or completing an opt out form online. Opt out forms include an optional ‘reasons for non-participation’ questionnaire. Full ENGAGE host trial recruitment procedures can be found in the published protocol [7].

2.5. Intervention

Participants will be randomised to receive one of the following interventions:

(1) A personalised invitation letter (Supplementary File 1), including their name and address (intervention group); or (2) non-personalised study invitation letter (Supplementary File 2) not including their name or address (control group). Personalised invitation letters will not include participants’ titles given that titles are rarely used in Sweden. However, invitation letter signatures include the title of the Principal Investigator, as titles are more commonly used for academics in the Swedish context.

The wording of the personalised and non-personalised study invitation letters were designed in consultation with the ENGAGE Parent Research Partner group, consisting of four parents with lived experience of being a parent of a child treated for cancer (see 2.10 Patient and Public Involvement for further details).

2.6. Outcomes

The primary outcome is the effectiveness of the personalised study invitation letter in recruiting participants, in comparison with the non-personalised study invitation letter. As such, the primary outcome is the difference in proportion of participants in the intervention and control group respectively that are enrolled into the ENGAGE host trial. Secondary outcomes are the proportion of participants invited into the study in each group that:

• •Express initial interest in participating in the ENGAGE host trial
• •Opt out of the ENGAGE host trial
• •Complete a reasons for non-participation in the ENGAGE host trial questionnaire
• •Complete the eligibility interview for inclusion in the ENGAGE host trial
• •Complete the baseline assessment for inclusion in the ENGAGE host trial
• •Require a telephone reminder at baseline, post-treatment (12 weeks) and 6-month follow-up for the ENGAGE host trial
• •Are retained at post-treatment (12 weeks) and 6-month follow-up for the ENGAGE host trial

Study flow data (see Fig. 1) will be collected online via the Portal or over the telephone with data entered into paper case report forms. All study flow data will be entered into a Microsoft® Access database.

2.7. Sample size

The target sample size of the ENGAGE host trial is 50 participants [7]. As is common for a SWAT, no formal power calculation has been conducted to determine the SWAT sample size, given that the sample size is constrained by the number of participants approached in the ENGAGE host trial [16]. However, following reports recommending a sample size of 50–60 to assess feasibility outcomes and estimate sample size for a definite trial [30,31], we estimate inviting 600 potential participants into the ENGAGE host trial to meet the target sample size of 50, representing a recruitment rate of 8%. The sample size calculation has been outlined in the ENGAGE study protocol [7]. As such, we anticipate the sample of size of present SWAT to be 600, which would provide 90% power to identify a 7.5% difference between the personalised and non-personalised study invitation letter groups in recruitment rate at a two-sided alpha = 0.05.

2.8. Randomisation

Random allocation will be utilised in 1:1 ratio without stratification. Potential participants will be invited into the study using blocks of 100. A de-identified list of potential participants will be prepared by a member of the research team. To ensure allocation concealment, a member of the U-CARE web-development team, external to the research team and not involved in participant recruitment, will produce a computer-generated randomised sequence. The randomisation software has been written specifically for randomisation into the SWAT by a member of the U-CARE web-development team. The software is designed to read a de-identified text file-list of potential participants and outputs the participants in two randomised groups into a CSV file. The participant allocation list will be returned to the research team to implement. To ensure adherence to the randomisation sequence, a 10% sample of invitations after every 50 randomisations will be checked by a member of the research team not involved in participant recruitment. Potential participants will be blind to the SWAT hypothesis and unaware they are part of an embedded trial. It is not possible for the researcher administering the interventions (posting study invitation letters) to be blinded to intervention group status. Participants will be provided with a Recruitment ID number within the study invitation pack dependent on which SWAT intervention they are allocated to. Participants will be required to enter this Recruitment ID number when registering for the study on the Portal. In addition, an allocation list alongside participants’ personal identification number will be stored on a secure USB in a locked filing cabinet.

2.9. Statistical analysis

All analyses will be conducted in SPSS, on an intention-to-treat basis by a statistician blind to group allocation. A two-sided p value of <0.05 will be taken to indicate statistical significance. Numbers and percentages within the personalised and non-personalised study invitation letter groups will be reported for categorical outcomes. Differences in recruitment proportions between groups for the primary and secondary outcomes will be compared, using logistic regression. Logistic regression models stratified by parent gender (male/female) and child gender (male/female) will be constructed, with results reported as an adjusted odds ratio with 95% confidence intervals. If two parents of the same child participate this would cause some dependency in the data between the two parents. However, as the number of cases is expected to be very small, data will be analysed as independent. Anonymised data from the SWAT will ultimately be combined in a meta-analysis with data from similar host studies participating in the UK Medical Research Council-funded PROMoting THE USE of SWATs (PROMETHEUS) programme (https://www.york.ac.uk/healthsciences/research/trials/research/swats/prometheus/). However, to the best of our knowledge, no other similar SWATs are currently being conducted by other research groups.

2.10. Patient and Public Involvement

In accordance with Guidance for Reporting Involvement of Patients and the Public (GRIPP) – short form [32], the present study involved the ENGAGE Parent Research Partner group, which includes two mothers and two fathers, aged 45–54 years old, with lived experience of being a parent of a child previously treated for cancer. One Parent Research Partner was previously involved in the development of the Cognitive-Behavioural Therapy (CBT) based self-help intervention being tested in host trial ENGAGE [33]. A further Parent Research Partner is also an academic member of the department, but otherwise not associated with the present study. The further two Parent Research Partners were recruited via word-of-mouth.

Aim: To assess the appropriateness of suggested wording of the personalised study invitation letter.

Methods: Parent Research Partners were e-mailed three versions of the participant invitation letter drafted by the research team, alongside information concerning the rationale of the SWAT: (1) personalised with both the name of the parent and the name of the child; (2) personalised with only the name of the parent; and (3) non-personalised with no names mentioned. Parents were asked to provide opinions on: (1) including the first name of the child in the personalised invitation letter; (2) including the parent’s full name and address on the top of the personalised invitation letters; and (3) the general wording of the invitation letter. Study invitation letter content, format, and design, including the decision to include a parent's name on the letter signature, was made by the research group, with feedback requested from the Parent Research Partners.

Outcomes: Three out of the four Parent Research Partners advised not to include the child’s name as this could be considered an invasion of privacy. All Parent Research Partners advised to include the full name and address on the top of the personalised invitation letter. In addition, all Parent Research Partners reported that the letter was short, informative, and validating, with no specific word changes suggested.

Reflections: Parent Research Partner input was at the consultation level, whereby feedback was provided on materials already developed by the research team. This approach was invaluable concerning making a decision on which personalised invitation letter should be adopted in the present study. For example, members of the research team felt including the child’s name may have increased personalisation and potentially result in improved recruitment rates. However, feedback from Parent Research Partners was almost unanimous that this may be considered an invasion of privacy, and may have had a negative impact on recruitment. Nonetheless, Patient and Public Involvement may have been further improved by engaging the Parent Research Partners more closely in the drafting of the invitation letters, for example, wording, content, format and design.

3. Discussion

Few recruitment strategies are currently supported by high-quality evidence [16]. As such, researchers conducting clinical trials have little evidence to rely on when making decisions regarding recruitment strategies. The present SWAT protocol addresses this gap by investigating the effectiveness of personalising participant study invitation letters and provides a possible study design template for other researchers planning to embed a similar SWAT within their own clinical trials. This is of particular importance given the need to replicate SWATs, since individual SWATs are often limited by sample size [34] and thus evidence synthesis is required to provide a more precise effect estimate [35]. Even moderate effects of the personalisation of study invitation letters on recruitment rates may be of significant value by shortening study length, saving resources, and providing a faster answer to the clinical question posed by the study.

3.1. Limitations

This study has several limitations. First, results may be limited to trials with access to databases of potential participants’ names and addresses. Indeed, a general limitation of SWATs concerns results being potentially related to the specific context of the host trial, subsequently possibly limiting the generalisability of results [26]. However, mailing letters is a common recruitment strategy [36] and as further SWATs are undertaken, evidence concerning the underlying mechanisms of action behind SWAT recruitment interventions may increase with findings generalizable outside the context of the host trial [26].

Second, it is important to recognise that generating evidence concerning trial processes via SWATs, may be subject to limited time and financial resources [26]. Further, SWATs may experience a number of challenges, such as delays in approval processes concerning ethical aspects and anonymous data sharing regulations [37]. As such, whilst SWATs are able to address important unanswered questions concerning important trial processes, careful decisions should be made to determine the need for a SWAT, to avoid trial resource waste [26]. Third, the sample size of the present SWAT is dictated by the host trial and as such no formal sample size calculation was made. Indeed, small sample sizes constitute a common difficulty experienced by SWATs [34] with host trial sample sizes often not being adequate to detect small but important differences in recruitment rates [38]. Finally, it is currently unknown to what extent non-personalised study invitation letters are used in clinical trials. Indeed, given the accessibility of mail-merge software, inclusion of participants’ names may be common place in clinical trial study invitation letters. Nonetheless, few RCTs have examined the effectiveness of personalising study invitations and given that mail-merging letters can be both resource and time intensive [39], it is important to examine whether personalisation does indeed improve recruitment rates.

However, despite the aforementioned limitations, research to maximise recruitment into mental health trials is of significant importance, given significant difficulties with recruitment are common [12].

Ethics approval

The ENGAGE study has been approved by the Regional Ethical Review Board in Uppsala, Sweden (Dnr: 2017/527) and will be conducted in accordance with the Helsinki Declaration, ensuring the welfare and rights of all participants, and Good Clinical Practice (GCP) guidelines. Ethical amendment for the SWAT has been obtained from Swedish Ethical Review Authority August 07, 2019, ref: 2019–03083.

Funding statement

This work is supported by the Swedish Research Council (grant number K2015-99X-20836-08-4 and 2018–02578), the Swedish Cancer Society (grant number 150673 and 180,589) and the Swedish Childhood Cancer Foundation (grant number PR2017-0005). The funding bodies have not been involved in the development of the study protocol or in writing the manuscript. Author AP’s time was funded by the UK Medical Research Council (MRC) [grant number MR/R013748/1] and the views expressed are those of the author and not necessarily those of the NHS, the NIHR, the MRC or the Department of Health Sciences, University of York .

Declaration of competing interest

None declared.

Appendix A. Supplementary data

The following are the Supplementary data to this article: