Abstract
Background
Randomised controlled trials are often beset by problems with poor recruitment and retention. Information to support decisions on trial participation is usually provided as printed participant information sheets (PIS), which are often long, technical, and unappealing. Multimedia information (MMI), including animations and videos, may be a valuable alternative or complement to a PIS. The Trials Engagement in Children and Adolescents (TRECA) study compared MMI to PIS to investigate the effects on participant recruitment, retention, and quality of decision-making.
Methods
We undertook six SWATs (Study Within A Trial) within a series of host trials recruiting children and young people. Potential participants in the host trials were randomly allocated to receive MMI-only, PIS-only, or combined MMI + PIS. We recorded the rates of recruitment and retention (varying between 6 and 26 weeks post-randomisation) in each host trial. Potential participants approached about each host trial were asked to complete a nine-item Decision-Making Questionnaire (DMQ) to indicate their evaluation of the information and their reasons for participation/non-participation. Odds ratios were calculated and combined in a meta-analysis.
Results
Data from 3/6 SWATs for which it was possible were combined in a meta-analysis (n = 1758). Potential participants allocated to MMI-only were more likely to be recruited to the host trial than those allocated to PIS-only (OR 1.54; 95% CI 1.05, 2.28; p = 0.03). Those allocated to combined MMI + PIS compared to PIS-only were no more likely to be recruited to the host trial (OR = 0.89; 95% CI 0.53, 1.50; p = 0.67). Providing MMI rather than PIS did not impact on DMQ scores. Once children and young people had been recruited to host trials, their trial retention rates did not differ according to intervention allocation.
Conclusions
Providing MMI-only increased the trial recruitment rate compared to PIS-only but did not affect DMQ scores. Combined MMI + PIS instead of PIS had no effect on recruitment or retention. MMIs are a useful tool for trial recruitment in children and young people, and they could reduce trial recruitment periods.
Supplementary Information
The online version contains supplementary material available at 10.1186/s12916-023-02936-1.
Keywords: Children, Trial, Recruitment, Retention, Consent, Multimedia information
Background
High-quality randomised controlled trials involving children and young people (CYP) are essential to ensure that interventions are safe and effective [1–4]. However, there is a lack of trial evidence, partly caused by poor recruitment, causing delayed completion and/or discontinuation [5, 6], and participant attrition [7, 8]. This contributes to trial costs [9] and research waste [10, 11]. Recruitment of CYP to trials is complicated by issues with consent and assent, judgements of ‘competence’, and different legal requirements for Clinical Trial of an Investigational Medicinal Product (CTIMP) and non-CTIMP trials [12, 13]. CYP may often be in a better position than their parent/carer to envisage what participation will mean [14, 15] and so excluding them from decision-making may lead to misunderstandings and subsequent withdrawal. Finally, when parents or carers are consenting on behalf of a child, the consent threshold tends to be higher [16], and parents or carers and healthcare professionals both tend to be more risk-averse (opting for standard care rather than a trial more often than they would for an adult) [17]. In 2013, the UK Chief Medical Officer [18] called for researchers to ‘…work with children and young people to input to the design of clinical studies…to facilitate (their) increased participation in trials’, an approach that has been undertaken in this study.
Information plays a crucial role in decision-making about trial participation [19]. In most cases, information is provided to potential participants as a printed participant information sheet (PIS); these have received prolonged criticism for being lengthy, technical, unappealing, and hard to navigate [20–23]. People with lower levels of literacy can find them especially difficult [24]. The UK Health Research Authority has encouraged researchers to use shorter PIS in low-risk research and explore the use of non-print media [25]. Providing multimedia information (MMI) through digital platforms could include animations, ‘talking-head’ videos, diagrams, photos, and written text [24, 26, 27].
In healthcare practice, MMIs have mostly been more effective than print for informing patients’ knowledge [28–31] such as about medical procedures [32–40]. The evidence in research recruitment is limited [26, 27], with only one study in CYP, which reported a greater understanding of trials from MMI [24]. In adults, the evidence is mixed: studies have reported increased understanding and knowledge of the trial [41] and positive participant evaluations [42], but another study reported that MMI and PIS produced similar trial recruitment rates [43].
MMIs have several potential advantages over PIS, including the following:
Choice of the order in which aspects of the MMI are viewed
Increased content choice, allowing some user personalisation
Delivery by sound allows provision to those with sight impairment
Concurrent delivery in sound and vision, facilitating user engagement and understanding, and reduced cognitive load [44]
We suggest that positive effects of MMI on recruitment might be seen through more effective attention to, and use of the information, producing beneficial effects on knowledge, attitudes to trials, and decision confidence (Fig. 1).
Fig. 1.
Logic model for the projected effects of multimedia information in trial recruitment
The science of trial recruitment and retention is being developed through ‘Study Within A Trial’ (SWAT) methods, which are self-contained studies embedded within a host trial to evaluate alternative ways to deliver or organise trial processes [45–47]. The pre-planned coordination of SWATs is a recent development, first undertaken in MRC-START [48], and the approach has four main strengths:
Increased sample size and certainty of findings
Greater researcher control over participant sampling, outcomes, and (especially) interventions
Cost-efficiency, depending on the development time for the SWAT intervention(s)
Coordination of ‘recruitment science’, likely to accelerate evidence generation [45, 46]
The pre-planned approach to SWATs was undertaken in this study.
Objectives
To evaluate MMI templates in a series of SWATs set within trials in CYP, to test their effects on recruitment and retention, and participant decision-making, by comparing the provision of MMIs to PIS, and the provision of MMIs in addition to PIS.
Methods
SWATs were embedded in a series of six host randomised control trials recruiting CYP. Within each SWAT, potential trial participants were randomised to receive trial recruitment information in one of three formats: MMI-only, PIS-only, or combined MMI + PIS (Fig. 2). Separate randomisation was done by each host trial, whereby their respective participants or recruiting sites (if cluster) were randomised to receive the information they had chosen to use in their trial. Details of the randomisation (ratios, clustering, stratification), and the arms included in each SWAT, can be found in Tables 1 and 2. Data from the SWATs were analysed and combined in a pre-planned two-stage meta-analysis.
Fig. 2.
Pathway through each SWAT. Note: In the FORCE SWAT, only two allocations were used (MMI-only and PIS-only)
Table 1.
Summary of the six host trials
| Trial | Target sample size (expected number to be approached, i.e. TRECA sample size) | Summary of host trial (funder; centres; eligible age range; health condition; interventions; stratification factors; retention time point) |
|---|---|---|
| FORCE | 696 (1071) | NIHR HTA; multicentre; 4–16 years; Torus fracture; splint versus soft bandage immobilisation; age, centre; 6 weeks |
| CHAMP-UK | 289 (413) | NIHR EME; multicentre; 6–12 years; myopia; atropine versus placebo; centre, ethnicity, severity of myopia; 6 months |
| THERMIC-3 | 94 (118) | British Heart Foundation; single-centre; 0–18 years; cardiac surgery; warm versus cold cardioplegia; risk adjustment for congenital heart surgery score; 3 months |
| BALANCE | 66 (100) | Action for medical research; multicentre; 3–8 years; amblyopia; blurred movie watching versus standard patching/eye drops; type of amblyopia and centre; 16 weeks |
| BAMP | 60 (100) | NHS Trust funding; single centre; 14–18 years; ‘reverse bite’; surgery at age 11–14 versus surgery at age 17 + ; gender; 18 months |
| UKALL-2011 | 40 (50) |
Blood Cancer UK and Children with Cancer UK; multicentre; < 25 years; acute lymphoblastic leukaemia and lymphoblastic lymphoma; high-dose methotrexate versus standard maintenance therapy; maintenance with or without pulses R1: disease type (ALL/LBL), sex, age, white blood cell count R2: induction regimen, minimal residual disease risk group, disease type, sex, age, white blood cell count; 3 months |
Table 2.
Summary of the PIS and MMI used in each SWAT and the type of randomisation used
| Trial | PIS and MMI summary |
|---|---|
| FORCE |
2 arms—cluster randomised, PIS or MMI (1:1) PIS: 3 versions, one for parents/carers and two for CYP (younger and older). Developed with PPI representatives. The PIS for the parents/carers was 4 pages, and the CYP (younger and older) versions were 1 page and 2 pages, respectively MMI: 2 versions (6–11 years; adolescents and parents/carers). Contained all written content of the PIS, with text amended to improve clarity when required. Viewed on tablet computer at the hospital. Included 5 short video animations (each lasting 45–60 s) and 12 ‘talking head’ videos: 5 with a study investigator; 3 with a research nurse; plus 1 with an adolescent and 3 with parents/carers of CYP who had taken part in similar studies (each lasting 15–50 s) and describing different aspects of the trial. The MMI content was organised on 6 main webpages |
| CHAMP-UK |
3 arms—participant randomised, PIS, MMI, or PIS + MMI (1:1:1) PIS: Emailed to participants prior to attending clinic for a screening/eligibility appointment. Three versions, one for parents/carers and 2 for children (including 1 that was a 2 pages picture booklet for CYP aged 6–7). Developed with PPI representatives. The PIS for parents/carers was 7 pages MMI: Link to URL to access MMI emailed to participants prior to attending clinic for a screening/eligibility appointment. Two versions (CYP; parents/carers), included 5 short video animations, each lasting 45–80 s. MMI contents for parents/carers organised on 6 main web pages: ‘Home page’, ‘About the study’, ‘Taking part’, ‘After the trial’, ‘Questions’, and ‘Contacts’. MMI for CYP had similar but less content under the 6 headings. The MMI for CYP included 19 ‘talking head’ videos: from the study investigator (13), CYP (5), and a parent from a similar study (1). Parents’/carers’ MMI included 19 ‘talking head’ clips (all < 1 min): from the study investigator (14), parents/carers (3), and CYP from a similar study (2) |
| THERMIC-3 |
3 arms—participant randomised, PIS, MMI, or PIS + MMI (1:1:1) PIS: Developed by the Thermic-3 trial team with feedback from the Generation R Young People’s Advisory Group (YPAG). Four versions, one for parents/carers and 3 for CYP (7–10, 11–15, 16–17 years). Length 2–7 pages MMI: Three versions (for different ages of CYP). The MMI was provided as a link (on laminated card), for viewing at the hospital or at home. Included 5 short video animations (each 45–60 s) and 34 ‘talking head’ video clips. The number of ‘talking head’ clips varied by age of intended user. The ‘talking head’ video clips (each lasting 10–80 s) were recorded with 4 individuals: a trial investigator, a research nurse, a parent/carer, and a CYP involved in a similar cardiac surgery study |
| BALANCE |
3 arms—participant randomised, PIS, MMI, or PIS + MMI (1:1:1) PIS: Three versions (< 6 years, 6–8 years, parent/carers) and ranged from 1 to 9 pages according to age group MMI: Two versions (CYP; parents/carers). The MMIs included 5 short animation videos, each lasting 45–60 s. The parents’/carers’ MMI was organised on six main pages: ‘Home page’, ‘About the study’, ‘Taking part’, ‘After the trial’, ‘Questions’, and ‘Contacts’. The MMI for younger CYP had similar contents as the parents/carers. Both MMIs included 16 ‘talking head’ videos: study investigator (13), plus a young adult (1), and parent/carer (2) from a previous similar study (each lasting 20 s to 4 min) |
| BAMP |
3 arms—participant randomised, PIS, MMI, or PIS + MMI (1:1:1) PIS: Two versions (parents/carers; CYP). Developed with PPI representatives. The PIS for parents was 7 pages, and the CYP’s PIS was 8 pages (one page for on assent form) MMI: Text addressed the CYP rather than the parent/carer (e.g. ‘your treatment’ rather than ‘their treatment’). The resource also included five short animation videos, each lasting 45–60 s (‘Summary of the key aspects of the BAMP trial’, ‘Why do we do trials?’, ‘What are trials?’, ‘Who’s in a trial team’, and ‘Assent and consent’) and 17 short ‘talking head’ videos, featuring 3 individuals (10 with the trial principal investigator; 4 with an adolescent who had received bone anchored maxillary protraction; 3 with a parent of a child who had received bone anchored maxillary protraction), each lasting 15–50 s and describing different aspects of the trial and clinical procedures. The MMI content was organised on 6 main webpages with the following headings: ‘Home page (including summary animation)’, ‘About the trial’, ‘Taking part’, ‘After the trial’, ‘Questions’, and ‘Contacts’ |
| UKALL-2011 |
3 arms—participant randomised, PIS, MMI, or PIS + MMI (1:1:1) PIS: Five versions (under 8; 8–12 years; 13–15 years; 16 plus; parents/carers). Four to 9 pages according to age group MMI: The different versions of the MMIs were created for younger and older CYP/parents/carers and according to type of leukaemia or lymphoma. The MMIs also included 5 short animation videos, each lasting 45–80 s. The parents’/carers’ MMI contents were organised on 6 main web pages: ‘Home page’, ‘About the study’, ‘Taking part’, ‘After the trial’, ‘Questions’, and ‘Contacts’. The younger age group MMI had less content under each heading. The MMIs for younger patients included 17 ‘talking head’ videos: with the study investigator (16) and a parent of a child from a similar study (1). The MMIs for older patients and parents included 22 ‘talking head’ videos: from the study investigator (18) parents of a child from a similar study (4). Each clip lasted 20 s to 3 min |
Recruitment of host trials
We publicised the Trials Engagement in Children and Adolescents (TRECA) study through printed flyers and email to UK trial centres and the UK Trial Managers’ Research Network. We emphasised that the cost of developing the MMI would be borne by TRECA, and we also offered host trials modest financial support (£1500) to compensate for additional administrative work. Interested trials were selected based on the following:
Recruitment timing (within the TRECA period)
Not currently using MMI, website, or video in trial recruitment
Recruiting CYP capable of at least some involvement in consent decisions (set as age 6 years and above)
Ideally, we wanted host trials to cover a range of health conditions and intervention types and be recruiting CYP across a range of ages. Given the historical prominence of oncology trials in CYP trial research [49], we preferred at least one host trial to have that setting.
Finally, the sample size calculation for the meta-analysis (see below) meant that host trials needed to approach an average of at least 329 CYP about participation. It is important to note that in these recruitment SWATs, the outcome of interest is the number of participants recruited out of the number of people approached to take part in the host trial. The people to be approached were randomly allocated to different recruitment methods, and the outcome is calculated according to the proportion of people recruited to the host trial from each arm in the SWAT.
Host trials were offered SWATs with two arms (MMI-only versus PIS-only; or MMI-only versus PIS + MMI) or three arms (MMI-only versus PIS-only versus PIS + MMI).
Interventions
Interventions used in the TRECA study are listed in Table 2. The PIS was the Research Ethics Committee-approved information being used in each host trial; the PIS content for each trial was not modified. The decision on who read the PIS (i.e. parent, child, both together, both separately) was made by individual participants.
The MMI for each trial was based on a template developed through co-design and extensive empirical work during the first year of the TRECA study, comprising a qualitative study with stakeholder groups [50]; a user testing study with CYP, parents, and carers [51]; and close collaboration with the TRECA Patient and Parent Involvement group (PPI) [52]. Two templates were developed (one for children aged 6–11 years and a second for children aged 12–18 years and parents). The MMIs were developed by a website and video production company (Morph Studios Ltd.) and contained all the content from the PIS, organised into six sections within the MMI (home page, about the trial, taking part, after the trial, questions, contacts) plus five short video animations with voiceover (one that was trial-specific, summarising the trial; four that were trial-generic, explaining different aspects of trials), and a series of short ‘talking heads’ videos (e.g. the trial principal investigator or a young participant talking about aspects of the trial including the experience of participation). None of the video clips included subtitles. The number of ‘talking heads’ videos varied among the SWATs (see Table 3 for links to example MMIs). The written text of each MMI was amended for clarity through readability indices [53]. The decision on who viewed the MMI (i.e. parent, child, both together, both separately) was made by individual participants.
Table 3.
Links to MMIs tested in the SWATs
Outcome measures
The primary outcome was trial recruitment in the MMI-only versus the PIS-only arms, to test the effect of replacing printed information with MMI. Trial recruitment was assessed by the number of eligible participants who were recruited to the host trial, according to the allocated SWAT arm.
The secondary outcomes were as follows:
Trial recruitment in the MMI-only versus the combined PIS + MMI arms, to test the effect of providing MMI in addition to PIS
Trial retention in the MMI-only versus the PIS-only arms
- Trial retention in the MMI-only versus the combined PIS + MMI arms
- Trial retention was assessed by the number of participants in the host trial who were retained in the trial at follow-up. When a trial had multiple primary outcome time points, we selected one to maximise similarity across the host trials.
Decision-Making Questionnaire (DMQ) scores in the MMI-only versus the PIS-only arms
- DMQ scores in the MMI-only versus the combined PIS + MMI arms
- Decision-making (i.e. evaluation of the information plus reasons for participation/non-participation) was assessed by the mean score on the Decision-Making Questionnaire (DMQ) scale (see Supplementary Materials 4 for the three versions of the DMQ). The DMQ measured the quality of decision-making by potential participants and was developed for the study because currently available measures were intended for adults or measured decisions about treatments. When a parent/carer was involved in the decision, we also asked them to complete the scale separately. The scale was adapted to facilitate completion by younger CYP. We aimed to obtain DMQ scores both from individuals who decided to participate in the host trial and those who declined. In those who decided to take part, the CYP and/or parent/carer were asked to complete the DMQ once the host trial participation documentation was completed, or it was emailed to them. In CYP who declined participation, they were asked to complete the DMQ in the clinic or it was posted or emailed to home, as appropriate.
- The DMQ versions for older CYP (intended for ages 12 and over) and parents or carers contained the same number of questions, with slight changes in question phrasing. It contained nine questions (with five fixed-response options), and there were a further three free-text questions. The younger CYP version of the DMQ (intended for ages 6–11) was of a similar format, comprising three questions with fixed responses and three ‘free-text’ questions.
- Answers to fixed-response questions were allocated values of 0–4. The values for each question were summed to create an overall score out of 36 (or out of 12 for the younger CYP version), in which higher scores represented more positive evaluations. Up to three missing responses were allowed on the 9-question scale, and one was allowed on the three-question version. A total score was calculated by replacing missing values with the mean score from the completed responses given by the participant. Any questionnaires with more than three (older CYP/parent/carer version) or one (younger CYP version) missing values, were not scored.
Trial and SWAT registration
The TRECA study was registered on the ISRCTN registry (ISRCTN73136092) and the Northern Ireland Hub for Trials Methodology Research SWAT Repository (SWAT 97). The SWATs were undertaken through amendments to Research Ethics Committee approvals obtained by the host trial research teams.
Data analysis
Sample size calculation
The TRECA sample size was based on the meta-analysis of the recruitment data from each SWAT on the primary outcome (MMI-only versus PIS-only). We assumed 80% power at 5% type I error (alpha rate).
Assuming the baseline recruitment rate (in the PIS-only arm) was 80%, to detect an increase to 88% in the MMI arm in a single randomised controlled trial (RCT) with 1:1 randomisation between arms, n = 329 per group was needed. We multiplied this number by three to account for the 3-arm randomisation in the SWATs (n = 987). We assumed the heterogeneity in observed effect across the trials (I 2 statistic) would be 50%, having the effect of doubling the sample size, deriving an overall sample of 1974 across the six SWATs. This calculation has been updated from that included in the initial published protocol, due to an inability to reproduce the calculation. However, this does not impact on the validity of the results as the sample size of the overall project was driven by the individual SWATs that were undertaken, which could not be pre-determined, and which had their own recruitment targets.
Analysis
All analyses were conducted in STATA v16, following the principles of intention-to-treat (ITT) with participants’ outcomes analysed according to their original, randomised group. A modified ITT (mITT) approach was used, whereby any participants who were subsequently found to be ineligible for the host trial, were not included in the analysis. Where it was possible for a participant to receive a SWAT allocation different to what they were randomised to (for example, randomised to MMI, but received PIS), we also undertook a per-protocol analysis; these results are not presented and can be found in the main publication [54]. The analysis, outcomes, and significance levels were pre-specified in a statistical analysis plan (see Supplementary Material 1) before the analysis was conducted. Model assumptions were checked.
As there was a single primary outcome, we did not need to adjust for multiplicity issues arising from the PIS arm being in all comparisons.
Baseline data
All participant baseline data were summarised descriptively by the SWAT arm, and no formal statistical comparisons were undertaken. Continuous measures were reported as means and standard deviations (after normality was checked), and categorical data were reported as counts and percentages. Baseline data for the host trials varied due to different trial data collection; all data that were collected have been reported. Baseline data were only available for participants who were randomised into each host trial (see Supplementary Material 2).
Primary analysis
Recruitment: MMI-only versus PIS-only
Recruitment rates were compared using logistic regression, undertaken separately for each SWAT, with SWAT allocation (MMI or PIS) included as a covariate. The results from the regression have been presented as odds ratios, with associated 95% confidence intervals and p-values.
Secondary analyses
Recruitment: MMI + PIS versus PIS-only
This analysis was undertaken in the same way as the primary analysis, including only those trials which included a combined MMI + PIS arm.
Retention
Retention was measured as the proportion of completed follow-ups, i.e. the number of participants retained at the first follow-up measurement of the host trial primary outcome variable, divided by the number who were due a follow-up. The retention rate was compared using logistic regression, with SWAT allocation and host trial allocation (for FORCE and CHAMP-UK only) included as covariates. When a host trial used stratification variables (Table 1) in the randomisation, these were included as covariates wherever possible. As for the recruitment analyses, two pairwise comparisons were used: MMI-only versus PIS-only, and MMI + PIS versus PIS-only. The results have been presented as odds ratios, with associated 95% confidence intervals, and p-values.
Quality of decision-making questionnaires
The responses to each question (including the number of missing responses) and the calculated total scores of the DMQ were summarised descriptively overall and presented by host trial, SWAT allocation and type of questionnaire (younger CYP, older CYP, or parent/carer).
As CYP and their parent/carer may have both completed a questionnaire, data from all three questionnaires were not combined, due to the lack of independence. Hence, scores for CYP (younger or older) and parents/carer questionnaires were analysed separately using linear regression, with SWAT allocation and host trial status (whether the participant was recruited) included as covariates. The mean difference has been presented with 95% confidence intervals.
The DMQ results were compared using a regression model for each SWAT, adjusted for SWAT allocation, and whether CYP consented to participate in the host trial. To assess the robustness of the method used to replace the missing values, a sensitivity analysis was conducted, where the analysis was repeated using only the questionnaires in which all nine questions were answered.
For all the above analyses, if the SWAT was cluster randomised, the analysis included cluster as a random effect.
Meta-analyses
The results from each SWAT were combined in meta-analyses. A two-stage random-effects meta-analysis was used in each case, where the results from each model were combined using an inverse-variance approach; no further adjustments were made. For each of the outcomes, any available data were combined using a meta-analysis.
Results
Host trials
Six host trials were recruited to TRECA: FORCE, CHAMP-UK, Thermic-3, BALANCE, BAMP, and UKALL-2011 (Tables 1 and 2), recruiting CYP with a variety of ages and health conditions in the UK (Table 1). Three of the SWATs generated sufficient data for logistic regression models, which were combined in a pre-planned statistical meta-analysis (FORCE, CHAMP-UK, Thermic-3). Unfortunately, three SWATs were significantly affected by adverse circumstances: the BAMP trial closed early due to funding restrictions; the UKALL-2011 SWAT took many months to approve, which limited recruitment before the host trial closed; and recruitment to the BALANCE trial was closed for several months due to the COVID-19 pandemic. Consequently, it was not possible to run logistic regression models for these three SWATs, due to being too small (2 trials) or having insufficient variation in outcomes (1 trial). Descriptive results are provided for each of the trials. However, the meta-analysis was based on the three SWATs where models could be fitted (total n = 1758).
The pathway through each SWAT is displayed in Fig. 3 and the results of the six individual SWATs are reported in Tables 4, 5, and 6. The results of the FORCE and Thermic-3 SWATs have already been reported individually [55, 56], and the results of the CHAMP-UK SWAT are reported in Supplementary Material 3.
Fig. 3.
Flow of participants through each SWAT
Table 4.
Summary of the recruitment analysis across the six SWATs, including the modified ITT meta-analysis results
| Trial | PIS-only, n/n (%) | MMI-only, n/n (%) | MMI + PIS, n/n (%) | Overall, n/n (%) | MMI-only vs PIS-onlya | MMI + PIS vs PIS-onlya |
|---|---|---|---|---|---|---|
| FORCE | 484/729 (66.4%) | 475/681 (69.8%) | – | 959/1410 (68.0%) | OR: 1.35 (95% CI 0.76, 2.40), p = 0.30 | – |
| CHAMP-UK | 28/65 (43.1%) | 38/68 (55.9%) | 30/68 (44.1%) | 96/201 (47.8%) | OR: 1.67 (95% CI 0.84, 3.32), p = 0.14 | OR: 1.04 (95% CI 0.53, 2.07), p = 0.90 |
| THERMIC-3 | 24/47 (51.1%) | 32/49 (65.3%) | 22/51 (43.1%) | 77/139 (55.4%) | OR: 1.80 (95% CI 0.79, 4.10), p = 0.16 | OR: 0.73 (95% CI 0.33, 1.61), p = 0.43 |
| BALANCE | 4/4 (100.0%) | 8/8 (100.0%) | 8/9 (88.9%) | 20/21 (95.2%) | – | – |
| BAMP | 1/3 (33.3%) | 3/3 (100.0%) | 4/4 (100.0%) | 8/10 (80.0%) | – | – |
| UKALL-2011 | 2/2 (100.0%) | – | 1/1 (100.0%) | 3/3 (100.0%) | – | – |
| Overall | 543/850 (63.9%) | 556/809 (68.7%) | 65/133 (48.9%) | 1164/1792 (65.0%) | – | – |
| Meta-analysis results | OR: 1.54 (95% CI 1.05, 2.28), p = 0.03** | OR: 0.89 (95% CI 0.53, 1.50), p = 0.67*** |
**FORCE, CHAMP-UK, and THERMIC-3 are included
***CHAMP-UK and THERMIC-3 are included
aPIS-only is the reference class
Table 5.
Summary of the retention analysis across the six SWATs, including the meta-analysis results
| Trial | PIS-only, n/n (%) | MMI-only, n/n (%) | MMI + PIS, n/n (%) | Overall, n/n (%) | MMI-only vs PIS-onlya | MMI + PIS vs PIS-onlya |
|---|---|---|---|---|---|---|
| FORCE | 481/484 (99.4%) | 473/475 (99.6%) | – | 954/959 (99.5%) | OR: 1.14 (95% CI 0.11, 12.32), p = 0.91 | – |
| CHAMP-UK | 26/28 (92.9%) | 34/38 (89.5%) | 29/30 (96.7%) | 89/96 (93.0%) | OR: 1.11 (95% CI 0.12, 10.27), p = 0.92 | OR: 2.23 (95% CI 0.19, 26.06), p = 0.52 |
| THERMIC-3 | 21/24 (87.5%) | 30/32 (93.8%) | 21/22 (95.5%) | 72/77 (94.0%) | OR: 1.62 (95% CI 0.20, 12.98), p = 0.65 | OR: 2.05 (95% CI 0.17, 24.64), p = 0.57 |
| BALANCE | 3/4 (75.0%) | 8/8 (100.0%) | 7/8 (87.5%) | 18/20 (90.0%) | – | – |
| BAMP | 0/1 (0%) | 2/3 (66.7%) | 3/4 (75.0%) | 5/8 (63.0%) | – | – |
| UKALL-2011 | 2/2 (100.0%) | – | 1/1 (100.0%) | 3/3 (100.0%) | – | – |
| Overall | 533/543 (98.2%) | 547/556 (98.4%) | 61/65 (93.8%) | 1141/1164 (98.0%) | – | – |
| Meta-analysis results | OR: 1.29 (95% CI 0.36, 4.65), p = 0.70** | OR: 2.18 (95% CI 0.48, 10.00), p = 0.31*** |
**FORCE, CHAMP-UK, and THERMIC-3 are included
***CHAMP-UK and THERMIC-3 are included
aPIS-only is the reference class
Table 6.
DMQ mean and standard deviation for each trial, by TRECA allocation and DMQ version, and results of comparisons
| Trial (DMQ versions used) | PIS-only | MMI-only | MMI + PIS | Overall | MMI-only vs PIS-onlya | MMI + PIS vs PIS-onlya |
|---|---|---|---|---|---|---|
| FORCE (P/C) |
P/C: N = 157 31.2 (4.9) |
P/C: N = 151 31.3 (4.5) |
– |
P/C: N = 308 31.3 (4.7) |
P/C: N = 308 AMD: 0.07 (95% CI − 1.08, 1.22) P = 0.91 |
– |
| CHAMP-UK (young and P/C) |
Young: – P/C: N = 26 33.2 (3.4) |
Young: – P/C: N = 31 29.7 (4.6) |
Young: – P/C: N = 24 30.0 (3.8) |
Young: N = 1 P/C: N = 81 30.6 (4.1) |
Young: – P/C: N = 57 AMD: − 2.43 (95% CI − 4.61, − 0.24) P = 0.03 |
Young: – P/C: N = 50 AMD: − 2.11 (95% CI − 4.23, 0.01) P = 0.05 |
| THERMIC-3 (all three) |
Young: – Older: – P/C: N = 8 29.5 (6.9) |
Young: – Older: – P/C: N = 6 31.8 (4.4) |
Young: – Older: – P/C: N = 3 28.3 (4.9) |
Young: – Older: – P/C: N = 17 30.1 (5.7) |
Young: – Older: – P/C: N = 14 AMD: 0.73 (95% CI − 5.34, 6.80) |
Young: – Older: – P/C: N = 11 AMD: 0.72 (95% CI − 10.43, 7.53) |
| BALANCE (young and P/C) |
Young: – P/C: – |
Young: – P/C: – |
Young: – P/C: – |
Young: N = 4 8.8 (2.2) P/C: N = 9 30.1 (4.0) |
Young: – P/C: – |
Young: – P/C: – |
| BAMP (older) |
Older: N = 3 29.0 (8.9) |
Older: N = 3 33.7 (2.1) |
Older: N = 4 31.3 (4.4) |
Older: N = 10 31.3 (5.4) |
Older: N = 6 AMD: 1.67 (95% CI − 24.59, 27.92) P = 0.85 |
Older: N = 7 AMD: − 0.75 (95% CI − 22.88, 21.37) P = 0.93 |
| UKALL-2011 (all three) |
Young: – Older: – P/C: – |
Young: – Older: – P/C: – |
Young: – Older: – P/C: – |
Young: – Older: N = 2 19.5 (2.1) P/C: N = 3 21.3 (14.6) |
Young: – Older: – P/C: – |
Young: – Older: – P/C: – |
| Overall |
Young: – Older: N = 4 26.3 (9.1) P/F: N = 217 31.0 (4.9) |
Young: – Older: N = 3 33.7 (2.1) P/F: N = 166 31.0 (5.6) |
Young: – Older : N = 5 29.2 (6.0) P/F: N = 28 30.0 (4.0) |
Young: N = 5 9.4 (2.4) Older: N = 12 29.3 (6.7) P/F: N = 411 30.9 (5.1) |
||
| Meta-analysis results |
Young: – Older: – P/C AMD: − 0.79 (95% CI − 2.80, 1.22) P = 0.44** |
Young: – Older: – P/C AMD: − 2.07 (95% CI − 4.13, − 0.01) P = 0.05*** |
aPIS-only is the reference class
**FORCE, CHAMP-UK, and THERMIC-3 are included
***CHAMP-UK and THERMIC-3 are included
Meta-analysis of SWAT data
Trial recruitment
The pooled results of the mITT data show that those CYP who received MMI-only information were more likely to be recruited into a trial than those who received PIS-only information: pooled odds ratio (OR) = 1.54 (95% CI: 1.05, 2.28; p = 0.03). The width of the 95% confidence interval indicates considerable uncertainty about the true effect. The statistical heterogeneity (I 2) in the meta-analysis was 0% (Fig. 4).
Fig. 4.
Forest plot of trial recruitment in the MMI-only and PIS-only arms for the mITT
The meta-analysis comparing PIS-only versus the combined MMI + PIS arms included data from only two SWATs (CHAMP-UK and Thermic-3) because the FORCE SWAT did not include a combined MMI + PIS arm. There was no significant difference between the two arms: pooled OR = 0.89 (95% CI: 0.53, 1.50; p = 0.67).
Trial retention rates
Retention was recorded between 6 and 26 weeks post-randomisation, depending on the trial. The retention rates of CYP recruited to host trials in the MMI-only and PIS-only arms were similar: pooled OR = 1.29 (95% CI: 0.36, 4.65; p = 0.70). Statistical heterogeneity was 0% (Table 5).
The retention rate was higher in the MMI + PIS arm than in the PIS-only arm, but sample sizes were small and the difference was not statistically significant: pooled OR = 2.18 (95% CI: 0.48, 10.00; p = 0.31). Statistical heterogeneity was 0%.
Quality of decision-making questionnaires
The meta-analysis of the DMQs is limited to the parent/carer (P/C) version of the questionnaire because there were insufficient data for meta-analyses of the younger and older CYP versions. The results of the younger and older CYP versions are given descriptively where possible (see Table 6).
Overall, for the parent/carer version those in the MMI-only arm had similar (but slightly lower) DMQ scores to those in the PIS-only arm: pooled adjusted mean difference (AMD) = − 0.79(95% CI: − 2.80, 1.22; p = 0.44). Statistical heterogeneity was moderately high (53.6%).
Participants in the combined MMI + PIS arm had lower DMQ scores than those in the PIS-only arm: pooled AMD = − 2.07 (95% CI: − 4.13, 0.01; p = 0.05), which is of borderline statistical significance. Statistical heterogeneity was 0%.
Discussion
Brief summary of findings
MMI provision, rather than standard PIS, led to higher recruitment rates of CYP to trials. There was no effect on trial retention. Providing combined MMI + PIS resulted in no benefit on trial recruitment or retention rates compared to PIS alone.
The provision of MMI rather than PIS produced no effect on DMQ scores. However, the provision of combined MMI + PIS resulted in less favourable evaluations than from PIS alone.
Strengths and limitations
The MMIs were developed through extensive empirical work, to ensure that their content, layout, and appearance would best meet the preferences and needs of CYP, parents/carers, healthcare practitioners, and researchers. Two templates were developed (one for children aged 6–11 and one for children aged 12–18 and parents), based on estimated literacy and cognitive maturity. Finer-grained age distinctions would be possible in MMIs although there would be both financial and practical implications.
The six host trials recruited CYP with a range of ages and health conditions across the UK, although we found that many otherwise eligible host trials were too small to include in TRECA. Unfortunately, we were able to include data from only three of the SWATs in the meta-analysis. However, the pre-planned meta-analysis increased the available sample size for evaluating the effects of MMIs, and allowed the comparison of similar interventions (i.e. template-driven MMIs) across several trials.
The study faced several challenges [57], the main effect of which was to reduce the size of the meta-analysis dataset. There was a good response to publicity about TRECA, but some potential host trials did not meet our criteria. In addition, trial recruitment postponements due to COVID-19 affected three SWATs, reducing the dataset size in two of them (by around n = 200 in total) and delaying the meta-analysis. Unfortunately, any adverse circumstance affecting host trial progress will almost always affect SWATs. On the other hand, the sample size estimate assumed a meta-analysis heterogeneity value of 50%, but in fact, a value of 0% was obtained, increasing power.
The DMQ return rates were around half that anticipated, reducing statistical precision. The questionnaire return rate was much higher in CYP recruited to host trials than in those who declined participation. It constitutes a significant limitation to the findings; potentially, it was also a source of bias in the DMQ dataset. Sending questionnaires via website or email (as happened in the CHAMP-UK trial), rather than providing printed versions, could have increased return rates, but this approach has not been evaluated [58]. We could also have sought other data sources on recruitment. For example, we could have interviewed people who did or did not consent in host trials, or clinical and research staff responsible for trial recruitment, although this approach would have had its own challenges.
Finally, a common limitation with SWATs is the application of host trial entry criteria to patient eligibility after the SWAT random allocation of individuals has happened. The effect is to lose SWAT participants and reduce the fidelity of the SWAT randomisation; consequently, a modified intention-to-treat (mITT) analysis was required in this study. In all, 18 participants were omitted from the analysis for being ineligible for the host trial (7 CHAMP-UK, 1 UKALL, 4 BALANCE, 6 THERMIC-3).
What this study adds
This study contributes to the growing evidence base of RCT-level evidence for interventions targeting trial recruitment and retention [45], particularly in CYP, which is currently lacking.
The retention rates of CYP at follow-up in the trials are reassuring, indicating that increased recruitment through MMI was not achieved at the expense of understanding the host trial when being recruited.
One aim of the study was to produce an MMI template for trial recruitment, in part to ensure that all the TRECA SWATs would be testing similar multimedia information, and to allow future practitioners and researchers to use the template in trials if the MMI was shown to be effective and/or acceptable. That opportunity is now available, and access can be provided on request.
There is growing evidence on the use of MMI in research recruitment and healthcare, although few interventions have been evaluated within SWATs using random allocation. As such, this study makes a meaningful contribution.
Finally, this study shows that using MMI in research recruitment is achievable and acceptable. Our view is that a hesitancy among researchers towards digital or multimedia information stems from two concerns:
First, a view that Research Ethics Committees will be reluctant to approve non-print participant information. Notably, we did not encounter any such resistance during this study.
Second, that MMI is too difficult, expensive, or time-consuming to produce. Depending on its quality and complexity, a trial MMI currently costs £10,000–£15,000. The pooled absolute recruitment rate in this study was 4.8% higher in the MMI-only arm (68.7% versus 63.9%, see Table 4), which would equate to a relative reduction of 7% in the recruitment period in the host trials, potentially reducing study length and cost. If replicated, the use of MMIs in trials could be cost-neutral or even cost-saving. However, no formal health economic analysis was undertaken.
Implications of the study
This study demonstrated a benefit of MMI on trial recruitment in CYP, although the relevant evidence base is small and further research is needed. Furthermore, the effect of MMIs on recruitment generated an odds ratio statistic with the lower end of the confidence interval not far above 1 (with an associated probability of 0.03) and so there remains considerable uncertainty around its true effect. That is particularly the case for CYP, but it also applies to trials recruiting adults.
The combined provision of MMI + PIS resulted in no gains, and possibly less positive questionnaire evaluations. However, the small available dataset for this question greatly lessens the certainty of this finding. Given the time demands and complexity of generating participant information in two formats, this study does not suggest that using both MMI and PIS is a better option, although it would be helpful for further research to assess this point.
One SWAT could not provide participants with MMI access as planned due to poor hospital internet connectivity [57]. Consequently, in that SWAT, participants were given the MMI URL on a laminated card, to access at home, increasing the risk of non-access. A solution could be through social media or email/text message communication between the healthcare provider and patients, although there are several barriers (ethical and practical) that may need to be overcome.
One aim was to assess the relative impacts of MMI and PIS on more deprived and/or less health-literate populations. Suboptimal or complex patient information tends to have a disproportionately negative effect on people with low literacy (including low health literacy) [24]. Unfortunately, we were unable to examine these aspects. Demographic baseline data are generally only available for those who have consented to the host trial. Due to research ethics restrictions, data access is rarely permitted for people who have declined trial participation, placing a significant limit on assessing equity within ‘recruitment science’. There remains a need to assess whether carefully developed MMI has similar benefits on understanding and decision-confidence across population groups. Easy internet access in the population is commonplace but not universal. There is a likelihood of deepening of inequities if information is exclusively provided via MMIs, or both MMIs and PIS are provided but MMI content has greater clarity. MMIs may also have negative associations for some users: video and animations can appeal and engage, although these effects must not be achieved through superficiality or imbalance. Content in different formats may lead to duplication or ‘information redundancy’ [59]. Finally, a recent review identified that CYP with long-term health conditions may associate digital health technologies with concerns about privacy, trust, and confidentiality [60].
Information plays an essential role in research consent. Valid participation decisions (whether positive or negative) must be informed, and the person taking consent has an ethical duty to be sure that the participant is making an informed decision. The role of MMI in individual consent decisions, and in allowing recruiters to fulfil their ethical duty, is an area in need of research. However, the external context is changing: people are increasingly expecting digital provision of information, particularly younger people, and digital provision may also be altering content expectations [61, 62]. These are two of the factors which may increase participant acceptance and use of MMI in trial recruitment.
This study has generated some relevant findings from the DMQs, although low return rates (particularly from those declining trial recruitment) mean that there is considerable uncertainty about MMI evaluations. For example, the findings of a SWAT undertaken within a hypothetical trial were that the TRECA-generated MMI produced higher appraisals of ‘information was easy to understand’ and ‘I had confidence in decision-making’ than did PIS provision [63]. There is also a need for research with adults and CYP around the effects of MMI on communication between potential participants and those recruiting them; MMIs may impact recruiter-participant interaction, but this needs evaluation.
We should also know more about people’s use of MMIs to inform decisions, to ensure they are being informed about research and not just being entertained. Furthermore, it is unclear whether all the components of the MMI are required to achieve benefits. Two concerns are that:
Animations and video are relatively expensive components of MMIs, which may create a barrier in less well-funded trials.
The inclusion in MMIs of all the written PIS text plus information content within the animations and videos allows choice but may create information duplication [59]. Analysis of the use of the various components of MMIs could indicate whether duplication and/or redundancy are apparent.
The overall effect of the MMIs in TRECA was to increase recruitment in children and young people (indicated by a meta-analysis of three SWATs with negligible statistical heterogeneity), which indicates their potential as a recruitment tool. However, given the width of the confidence intervals, and a dataset based on just three SWATs, there remains a need for more research to produce a more precise estimate of how well MMIs work and in what settings. The TRECA dataset on combined MMI + PIS was small and further research would increase the certainty on its effects. Finally, we need to explore what components of MMIs are associated with benefits, and whether MMIs are cost-effective.
Conclusions
Providing only multimedia information increased the trial recruitment rate compared to providing only printed information, but it did not affect people’s evaluation of the information. Among children and young people recruited to host trials, trial retention rates did not differ according to intervention allocation. Providing combined multimedia and printed information instead of printed information only had no effect on recruitment or retention. Multimedia information is a useful tool for trial recruitment in children and young people, and it could reduce trial recruitment periods.
Supplementary Information
Additional file 1. Statistical Analysis Plan.
Additional file 2. Baseline characteristics for participants who consented to the FORCE, CHAMP-UK and Thermic-3 trials.
Additional file 3. Results of the CHAMP-UK SWAT.
Additional file 4. Decision-making Questionnaire.
Acknowledgements
The authors thank the CYP, their families/carers, investigators, study teams, and operational staff who contributed to the included studies. Per funder requirements, the full findings of this study will be published in Health and Social Care Delivery Research. The funder had no role in the study design; collection, analysis, and interpretation of the data; writing of the manuscript; or decision to submit this article for publication.
Abbreviations
- AMD
Adjusted mean difference
- CI
Confidence interval
- COVID-19
Coronavirus disease 2019
- CYP
Children and young people
- DMQ
Decision-Making Questionnaire
- HCP
Healthcare professional
- ITT
Intention-to-treat
- mITT
Modified intention-to-treat
- MMI
Multimedia information
- OR
Odds ratio
- PIS
Participant information sheet
- RCT
Randomised controlled trial
- SWAT
Study Within A Trial
- TRECA
Trials Engagement in Children and Adolescents
- UK
United Kingdom
- USA
United States of America
Authors’ contributions
PK (https://orcid.org/0000-0001-5904-8699) was the principal investigator, led the design of the study, contributed to the design of the multimedia information, and is the lead author of the report and its corresponding author (peter.knapp@york.ac.uk). JMK (https://orcid.org/0000-0002-9299-1360) was responsible for the day-to-day management of the project (until 2019), undertook the data collection, led the qualitative study, and was involved in the preparation of this report. RS (https://orcid.org/0000-0002-7715-1224) undertook the data collection, chaired the Patient and Parent Advisory Group, led the user testing study, and was involved in the preparation of this report. EC (https://orcid.org/0000-0003-4210-1865) led the data analysis and was involved in the preparation of this report. JR (https://orcid.org/0000-0003-1822-9519) led the data analysis and was involved in the preparation of this report. TMB (https://orcid.org/0000-0002-2827-9715) was responsible for the day-to-day management of the project (from 2019), undertook the data collection, and was involved in the analysis and preparation of this report. BY (https://orcid.org/0000-0001-6041-9901) contributed to the study design; advised on the study throughout, particularly the qualitative study; and was involved in the preparation of this report. CG (https://orcid.org/0000-0002-3021-1955) contributed to the study design, advised on the data analysis, and was involved in the preparation of this report. JG contributed to the study design. JP (https://orcid.org/0000-0003-4800-234X) contributed to the study design, advised on the PPI aspects of this study, contributed to the PPI data collection, and was involved in the preparation of this report. CS (https://orcid.org/0000-0002-9797-8587) contributed to the study design, advised on the MMI design, and was involved in the preparation of this report. PB (https://orcid.org/0000-0001-9558-3349) contributed to the study design; advised on the study throughout, particularly the phase 2 study; and was involved in the analysis and preparation of this report. SH (https://orcid.org/0000-0003-0314-484) contributed to the study design, led the readability revisions of text in the MMIs, advised on the study throughout, and was involved in the preparation of this report. DP, AD-N, MA, PK, SL, AA-B, EM, NM, AL, CAR, HS, and RH led or worked on the included SWATs. SRS, DHT, SA, and JA are patient or parent authors and contributed to the conduct of the research.
Funding
This study was funded by the NIHR Health and Social Care Delivery Research Programme (14/21/21). The views expressed are those of the authors and not necessarily those of the NIHR or the Department of Health and Social Care.
Availability of data and materials
Upon request, and subject to review, the authors will provide the data that support the findings of this study.
Declarations
Ethics approval and consent to participate
The study protocol, protocol amendments, and informed assent/consent documents were reviewed and approved by the NHS Yorkshire and the Humber – Bradford Leeds Research Ethics Committee (17/YH/0082) and the Health Research Authority (IRAS ID 212761). The TRECA study was conducted in accordance with the principles of the Declaration of Helsinki, and all participants provided written informed consent to the host trial. No specific consent was required for the TRECA study, as approved by the Research Ethics Committee (17/YH/0082).
Consent for publication
All authors reviewed and approved the manuscript for publication.
Competing interests
SRS is an employee of Envision Pharma Group, owns stock options in Envision Pharma Group, is a member of Savvy Cooperative, and is a trustee of RAiISE, a charitable incorporated organisation registered in England and Wales.
No other authors have any competing interests to add.
Footnotes
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
- 1.Joseph PD, Craig JC, Caldwell PH. Clinical trials in children. Br J Clin Pharmacol. 2013;79(3):357–369. doi: 10.1111/bcp.12305. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Chappuy H, Doz F, Blanche S, et al. Children’s views on their involvement in clinical research. Pediatr Blood Cancer. 2008;50(5):1043–1046. doi: 10.1002/pbc.21359. [DOI] [PubMed] [Google Scholar]
- 3.Rocchi F, Tomasi P. The development of medicines for children. Part of a series on Pediatric Pharmacology, guest edited by Gianvincenzo Zuccotti, Emilio Clementi, and Massimo Molteni. Pharmacol Res. 2011;64(3):169–75. doi: 10.1016/j.phrs.2011.01.016. [DOI] [PubMed] [Google Scholar]
- 4.Klassen TP, Hartling L, Craig JC, et al. Children are not just small adults: the urgent need for high-quality trial evidence in children. PLoS Med. 2008;5(8):e172. doi: 10.1371/journal.pmed.0050172. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Srivastava A, Bourgeois FT. Evaluation of publication of pediatric drug trials. JAMA Netw Open. 2021;4(4):e215829–e215929. doi: 10.1001/jamanetworkopen.2021.5829. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Pica N, Bourgeois F. Discontinuation and nonpublication of randomized clinical trials conducted in children. Pediatrics. 2016;138(3):e20160223. doi: 10.1542/peds.2016-0223. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Walters SJ, Bonacho dos Anjos Henriques-Cadby I, Bortolami O, et al. Recruitment and retention of participants in randomised controlled trials: a review of trials funded and published by the United Kingdom Health Technology Assessment Programme. BMJ Open. 2017;7(3):e015276. doi: 10.1136/bmjopen-2016-015276. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Bruzzese JM, Gallagher R, McCann-Doyle S, et al. Effective methods to improve recruitment and retention in school-based substance use prevention studies. J Sch Health. 2009;79(9):400–407. doi: 10.1111/j.1746-1561.2009.00427.x. [DOI] [PubMed] [Google Scholar]
- 9.Kitterman DR, Cheng SK, Dilts DM, et al. The prevalence and economic impact of low-enrolling clinical studies at an academic medical center. Acad Med. 2011;86(11):1360–1366. doi: 10.1097/ACM.0b013e3182306440. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Moher D, Glasziou P, Chalmers I, et al. Increasing value and reducing waste in biomedical research: who’s listening? Lancet. 2016;387(10027):1573–1586. doi: 10.1016/s0140-6736(15)00307-4. [DOI] [PubMed] [Google Scholar]
- 11.Ioannidis JP, Greenland S, Hlatky MA, et al. Increasing value and reducing waste in research design, conduct, and analysis. Lancet. 2014;383(9912):166–175. doi: 10.1016/s0140-6736(13)62227-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.The General Medical Council. 0–18 years: guidance for all doctors. 2007. https://www.gmc-uk.org/ethical-guidance/ethical-guidance-for-doctors/0-18-years (Accessed 4 Oct 2022).
- 13.UK Medical Research Council. MRC ethics guide - medical research involving children. 2004. https://mrc.ukri.org/documents/pdf/medical-research-involving-children/ (Accessed 5 Dec 2022).
- 14.Coyne I. Research with children and young people: the issue of parental (proxy) consent. Child Soc. 2010;24(3):227–237. [Google Scholar]
- 15.Luchtenberg M, Maeckelberghe E, Locock L, et al. Young people’s experiences of participation in clinical trials: reasons for taking part. Am J Bioeth. 2015;15(11):3–13. doi: 10.1080/15265161.2015.1088974. [DOI] [PubMed] [Google Scholar]
- 16.Caldwell PH, Butow PN, Craig JC. Parents’ attitudes to children’s participation in randomized controlled trials. J Pediatr. 2003;142(5):554–559. doi: 10.1067/mpd.2003.192. [DOI] [PubMed] [Google Scholar]
- 17.Caldwell PH, Murphy SB, Butow PN, et al. Clinical trials in children. Lancet. 2004;364(9436):803–811. doi: 10.1016/s0140-6736(04)16942-0. [DOI] [PubMed] [Google Scholar]
- 18.Health Do . Chief Medical Officer’s annual report 2012: our children deserve better: Prevention Pays. 2013. [Google Scholar]
- 19.Sheridan R, Martin-Kerry J, Hudson J, et al. Why do patients take part in research? An overview of systematic reviews of psychosocial barriers and facilitators. Trials. 2020;21(1):259. doi: 10.1186/s13063-020-4197-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Tarnowski KJ, Allen DM, Mayhall C, et al. Readability of pediatric biomedical research informed consent forms. Pediatrics. 1990;85(1):58–62. doi: 10.1542/peds.85.1.58. [DOI] [PubMed] [Google Scholar]
- 21.Ogloff JR, Otto RK. Are research participants truly informed? Readability of informed consent forms used in research. Ethics Behav. 1991;1(4):239–252. doi: 10.1207/s15327019eb0104_2. [DOI] [PubMed] [Google Scholar]
- 22.Eder ML, Yamokoski AD, Wittmann PW, et al. Improving informed consent: suggestions from parents of children with leukemia. Pediatrics. 2007;119(4):e849–e859. doi: 10.1542/peds.2006-2208. [DOI] [PubMed] [Google Scholar]
- 23.Caldwell PH, Dans L, de Vries MC, et al. Standard 1: consent and recruitment. Pediatrics. 2012;129(Suppl 3):S118–S123. doi: 10.1542/peds.2012-0055D. [DOI] [PubMed] [Google Scholar]
- 24.Tait AR, Voepel-Lewis T. Digital multimedia: a new approach for informed consent? JAMA. 2015;313(5):463–464. doi: 10.1001/jama.2014.17122. [DOI] [PubMed] [Google Scholar]
- 25.Health Research Authority . Applying a proportionate approach to the process of seeking consent: HRA Guidance. 2016. [Google Scholar]
- 26.Hutchison C, Cowan C, McMahon T, et al. A randomised controlled study of an audiovisual patient information intervention on informed consent and recruitment to cancer clinical trials. Br J Cancer. 2007;97(6):705–711. doi: 10.1038/sj.bjc.6603943. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Shneerson C, Windle R, Cox K. Innovating information-delivery for potential clinical trials participants. What do patients want from multi-media resources? Patient Educ Couns. 2013;90(1):111–7. doi: 10.1016/j.pec.2012.06.031. [DOI] [PubMed] [Google Scholar]
- 28.Savage I, Goodyer L. Providing information on metered dose inhaler technique: is multimedia as effective as print? Fam Pract. 2003;20(5):552–557. doi: 10.1093/fampra/cmg510. [DOI] [PubMed] [Google Scholar]
- 29.Hopper KD, Zajdel M, Hulse SF, et al. Interactive method of informing patients of the risks of intravenous contrast media. Radiology. 1994;192(1):67–71. doi: 10.1148/radiology.192.1.8208968. [DOI] [PubMed] [Google Scholar]
- 30.Krishna S, Francisco BD, Balas EA, et al. Internet-enabled interactive multimedia asthma education program: a randomized trial. Pediatrics. 2003;111(3):503–510. doi: 10.1542/peds.111.3.503. [DOI] [PubMed] [Google Scholar]
- 31.Wilson EA, Makoul G, Bojarski EA, et al. Comparative analysis of print and multimedia health materials: a review of the literature. Patient Educ Couns. 2012;89(1):7–14. doi: 10.1016/j.pec.2012.06.007. [DOI] [PubMed] [Google Scholar]
- 32.Schenker Y, Fernandez A, Sudore R, et al. Interventions to improve patient comprehension in informed consent for medical and surgical procedures: a systematic review. Med Decis Making. 2011;31(1):151–173. doi: 10.1177/0272989x10364247. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Kinnersley P, Phillips K, Savage K, et al. Interventions to promote informed consent for patients undergoing surgical and other invasive healthcare procedures. Cochrane Database Syst Rev. 2013;7:Cd009445. doi: 10.1002/14651858.CD009445.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Tuong W, Larsen ER, Armstrong AW. Videos to influence: a systematic review of effectiveness of video-based education in modifying health behaviors. J Behav Med. 2014;37(2):218–233. doi: 10.1007/s10865-012-9480-7. [DOI] [PubMed] [Google Scholar]
- 35.Dahodwala M, Geransar R, Babion J, et al. The impact of the use of video-based educational interventions on patient outcomes in hospital settings: a scoping review. Patient Educ Couns. 2018;101(12):2116–2124. doi: 10.1016/j.pec.2018.06.018. [DOI] [PubMed] [Google Scholar]
- 36.Dekkers T, Melles M, Groeneveld BS, et al. Web-based patient education in orthopedics: systematic review. J Med Internet Res. 2018;20(4):e143. doi: 10.2196/jmir.9013. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Knox ECL, Quirk H, Glazebrook C, et al. Impact of technology-based interventions for children and young people with type 1 diabetes on key diabetes self-management behaviours and prerequisites: a systematic review. BMC Endocr Disord. 2019;19(1):7. doi: 10.1186/s12902-018-0331-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Ciciriello S, Johnston RV, Osborne RH, et al. Multimedia educational interventions for consumers about prescribed and over-the-counter medications. Cochrane Database Syst Rev. 2013;4:Cd008416. doi: 10.1002/14651858.CD008416.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Nishimura A, Carey J, Erwin PJ, et al. Improving understanding in the research informed consent process: a systematic review of 54 interventions tested in randomized control trials. BMC Med Ethics. 2013;14:28. doi: 10.1186/1472-6939-14-28. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Friedlander JA, Loeben GS, Finnegan PK, et al. A novel method to enhance informed consent: a prospective and randomised trial of form-based versus electronic assisted informed consent in paediatric endoscopy. J Med Ethics. 2011;37(4):194–200. doi: 10.1136/jme.2010.037622. [DOI] [PubMed] [Google Scholar]
- 41.Afolabi MO, Bojang K, D’Alessandro U, et al. Multimedia informed consent tool for a low literacy African research population: development and pilot-testing. J Clin Res Bioethics. 2014;5(3):178. doi: 10.4172/2155-9627.1000178. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.McFarlane SJ, Morgan SE, Occa A, et al. An evaluation of clinical trial multimedia to support Hispanic cancer patients’ informational and decision-making needs. J Cancer Educ. 2021;36(1):110–117. doi: 10.1007/s13187-019-01606-2. [DOI] [PubMed] [Google Scholar]
- 43.Jolly K, Sidhu M, Bower P, et al. Improving recruitment to a study of telehealth management for COPD: a cluster randomised controlled ‘Study Within A Trial’ (SWAT) of a multimedia information resource. Trials. 2019;20(1):453. doi: 10.1186/s13063-019-3496-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Hermann M. Dreidimensionale Computeranimation - neues Medium zur Unterstutzung des Aufklarungsgesprachs vor Operationen Akzeptanz und Bewertung der Patienten anhand einer prospektiv randomisierten Studie - Bild versus Text, Der Chirug. 2002;73(5):500–507. doi: 10.1007/s00104-001-0416-y. [DOI] [PubMed] [Google Scholar]
- 45.Treweek S, Pitkethly M, Cook J, et al. Strategies to improve recruitment to randomised trials. Cochrane Database Syst Rev. 2018;2:Mr000013. doi: 10.1002/14651858.MR000013.pub6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46.Treweek S, Bevan S, Bower P, et al. Trial Forge Guidance 1: what is a Study Within A Trial (SWAT)? Trials. 2018;19(1):139. doi: 10.1186/s13063-018-2535-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47.National Institute for Health Research (NIHR). Why the NIHR’s new funding stream for ‘Studies Within A Trial’ (SWATs) is potentially game-changing. 2018. https://www.nihr.ac.uk/blog/why-the-nihrs-new-funding-stream-for-studies-within-a-trial-swats-is-potentially-game-changing/11101#:~:text=innovation%20in%20research-,Why%20the%20NIHR’s%20new%20funding%20stream%20for%20'Studies%20Within%20A,SWATs)%20is%20potentially%20game%2Dchanging&text=Good%20clinical%20trials%20are%20hard%20work.&text=This%20is%20what%20makes%20NIHR's,(HTA)%20Programme%20so%20fabulous.
- 48.Madurasinghe VW, Bower P, Eldridge S, et al. Can we achieve better recruitment by providing better information? Meta-analysis of ‘Studies Within A Trial’ (SWATs) of optimised participant information sheets. BMC Med. 2021;19(1):1–8. doi: 10.1186/s12916-021-02086-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49.EMA Regulation (EC) No 1901/2006 of the European Parliament and of the Council of 12 December 2006 on medicinal products for paediatric use and amending Regulation (EEC) No 1768/92, Directive 2001/20/EC, Directive 2001/83/EC and Regulation (EC) No 726/2004 (Official Journal L 378, 27/12/2006 p.1–19). http://ec.europa.eu/health/documents/eudralex/vol-1/index_en.htm (Accessed 5 Dec 2022).
- 50.Martin-Kerry JM, Knapp P, Atkin K, et al. Supporting children and young people when making decisions about joining clinical trials: qualitative study to inform multimedia website development. BMJ Open. 2019;9(1):e023984. doi: 10.1136/bmjopen-2018-023984. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51.Sheridan R, Martin-Kerry J, Watt I, et al. User testing digital, multimedia information to inform children, adolescents and their parents about healthcare trials. J Child Health Care. 2018:1367493518807325. 10.1177/1367493518807325 [DOI] [PubMed]
- 52.Sheridan R, Preston J, Stones S, et al. Patient and public involvement in a study of multimedia clinical trial information for children, young people and families. Research for All. 2020;4(1):47–65. doi: 10.18546/RFA.04.1.05. [DOI] [Google Scholar]
- 53.Readability formulas. https://readabilityformulas.com/free-readability-formula-tests.php (Accessed 22 Feb 2023).
- 54.Knapp P, Martin-Kerry J, Sheridan R, et al. The effectiveness and acceptability of multimedia information when recruiting children and young people to trials: pre-planned meta-analysis of SWATs. NIHR Journals Library (in press). [DOI] [PubMed]
- 55.Moe-Byrne T, Knapp P, Perry D, et al. Does digital, multimedia information increase recruitment and retention in a children’s wrist fracture treatment trial, and what do people think of it? A randomised controlled Study Within A Trial (SWAT) BMJ Open. 2022;12(7):e057508. doi: 10.1136/bmjopen-2021-057508. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 56.Knapp P, Heys R, Dabner L, et al. The effects of multimedia information on recruitment and retention in a children’s cardiac surgery trial: a randomised controlled SWAT (Study Within A Trial) [version 1; peer review: awaiting peer review]. F1000Research. 2022;11(340). 10.12688/f1000research.110083.1.
- 57.Martin-Kerry J, Parker A, Bower P, et al. SWATted away: the challenging experience of setting up a programme of SWATs in paediatric trials. Trials. 2019;20(1):141. doi: 10.1186/s13063-019-3236-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 58.Edwards P, Roberts I, Clarke M, et al. Increasing response rates to postal questionnaires: systematic review. BMJ. 2002;324(7347):1183. doi: 10.1136/bmj.324.7347.1183. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 59.Antal H, Bunnell HT, McCahan SM, et al. A cognitive approach for design of a multimedia informed consent video and website in pediatric research. J Biomed Inform. 2017;66:248–258. doi: 10.1016/j.jbi.2017.01.011. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 60.Blower S, Swallow V, Maturana C, et al. Children and young people’s concerns and needs relating to their use of health technology to self-manage long-term conditions: a scoping review. Arch Dis Child. 2020;105(11):1093–1104. doi: 10.1136/archdischild-2020-319103. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 61.Duffy B. Generations: does when you’re born shape who you are? London: Atlantic Books; 2021. [Google Scholar]
- 62.Katz R, Ogilvie S, Shaw J, et al. Gen Z explained: the art of living in a digital age. USA: University of Chicago Press; 2021. [Google Scholar]
- 63.Knapp P, Mandall N, Hulse W, et al. Evaluating the use of multimedia information when recruiting adolescents to orthodontics research: a randomised controlled trial. J Orthod. 2021;48:(4). 10.1177/14653125211024250. [DOI] [PMC free article] [PubMed]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Additional file 1. Statistical Analysis Plan.
Additional file 2. Baseline characteristics for participants who consented to the FORCE, CHAMP-UK and Thermic-3 trials.
Additional file 3. Results of the CHAMP-UK SWAT.
Additional file 4. Decision-making Questionnaire.
Data Availability Statement
Upon request, and subject to review, the authors will provide the data that support the findings of this study.