Abstract
Background: Studies within trials (SWATs) present an opportunity to examine design factors that may impact on the successful delivery of trials. One area in need of research is trial recruitment. Recruiting patients to trials is a major challenge facing trialists. Failure to meet recruitment targets can result in delays and underpowered studies. This SWAT evaluates the effectiveness of hand-held digital multimedia presentation of trial information and standard written patient information to potential participants on recruitment and retention to a host trial.
Methods: This is the protocol for SWAT 15, a two-group, embedded parallel randomised controlled trial (RCT) (ISRCTN12838042) designed within a host trial - the SATIN trial (ISRCTN88111427), a RCT designed for implementation in the Irish primary care setting. The SWAT eligibility criteria was determined by the host trial. General practices who agree to participate in the host trial will provide women (participants) who are willing to consider participating in the host trial with either a multimedia digital information resource facilitated through a handheld tablet device, plus a written participant information leaflet (Intervention) or a written participant information leaflet (comparator). Outcomes are recruitment and retention to the host SATIN trial and participant’s quality of decision-making.
Discussion: Although designed to be implemented in a host trial, the host trial, was suspended and therefore this SWAT was not implemented. The protocol and the lessons learnt whilst developing it offer guidance to researchers who wish to answer similar research questions in the future in a similar context or setting.
Trial registration: ISRCTN Registry ISRCTN12838042 (11/10/2017)
Keywords: Study Within A Trial, Recruitment, informed consent, primary care, Randomised Control Trial
Background
Rigorous research is essential to delivering and improving the quality of health care. Randomised controlled trials (RCT) provide reliable evidence on the benefits and harms of healthcare interventions 1. Although RCTs are accepted as the most appropriate method to evaluate the effects of health care interventions, there is strong evidence that recruiting clinicians and patient participants to trials presents a significant challenge 2. Across trials, it is estimated that less than 50% meet their recruitment target or do so only with an extension to the original trial duration 3, 4. For example, of 114 trials funded by the UK Medical Research Council (MRC) and the Health Technology Assessment (HTA) Programme that recruited participants between 1994 and 2002, only 31% met their recruitment targets and over half (53%) were given an extension 5. More recently, Sully and colleagues investigated the recruitment success of 73 trials funded by the same bodies between 2002 and 2008 and observed similar results. Only 55% of trials recruited to their pre-specified target sample size and nearly half (45%) received an extension 6. Similar issues have also been recognised in the United States. A study investigating the prevalence and associated economic impact of low-enrolling clinical studies at a single academic medical centre found that of the 837 clinical studies terminated during the study period, nearly a third (31.1%) were low-enrolling. Furthermore, primary care trials often fail to achieve adequate sample size as demonstrated in a recent study of primary care trials in which only 23% recruited successfully compared to 62% of mental health trials 6. These failures in meeting recruitment targets mean that overall trial findings are likely to be underpowered; with the studies delayed and falling short of answering their objectives 3, 6.
Understanding how to maximise the recruitment process will help to overcome these challenges in the future and would benefit trialists during the design and implementation phases of trials 4. Developing and evaluating interventions aimed at improving recruitment to trials may be a good investment, where even a small return could translate into avoidance of substantial additional costs whilst reducing the time to potential knowledge impact. However, there is, as identified in a recent Cochrane systematic review, limited high quality evidence evaluating the effectiveness of interventions to improve trial recruitment 7. Furthermore, over 35% (24/68) of trials included in this review evaluated the effectiveness of recruitment strategies to hypothetical trials meaning that the effectiveness of the strategies evaluated in real-life settings is further limited.
Methodological innovation is necessary to improve the science of recruitment and should be a focus when seeking to improve trial recruitment. The importance of establishing new methods of effective engagement among patients, practitioners, and the primary care research community is paramount; as there is a propensity among researchers to overestimate the degree to which research is viewed positively by practitioners and patients 8. It is acknowledged that one barrier to patient recruitment is inadequacy of trial information to meet the needs of potential participants as well as the ineffective mode of dissemination of this information. A failure to adequately explain what the trial is about, what participation involves, and the value of participation to potential participants has a direct impact on the informed consent process 9.
Studies Within A Trial (SWATs) have been developed as one method of gathering information on different design factors potentially impacting on the outcome of trials 10– 12. SWATs seek to “aid the development of such research by increasing awareness of, and stimulating interest in the need for this research and providing a framework and resource to inspire and generate ideas, and to store, disseminate and modify such research” 13. Researchers interested in conducting a SWAT are encouraged to register their SWAT in the SWAT repository.
This SWAT was registered as SWAT-15 in the SWAT repository. The host trial, SATIN (ISRCTN88111427); however, was stopped prior to recruitment of the first participant due to the emergence of new evidence on the treatment of urinary tract infections (UTI) and therefore this SWAT although designed was never implemented. The protocol and the lessons learnt whilst developing it could guide trialists who wish to answer similar research questions in the future. This protocol for SWAT-15, will be of use to researchers considering evaluating different ways to present information to potential trial participants and to those interested in SWATs in general.
Aim
To evaluate the effectiveness of presenting potential trial participants with trial information using hand-held digital multimedia and written information leaflet or a standard written information leaflet, alone, on recruitment and retention to a host trial.
Objectives
-
a)
To establish if (a) the proportion of patients willing to consider participating and (b) the proportion of participants recruited to the host trial (in case this differs from the number of participants willing to participate, due to e.g. exclusion criterion) is improved using a hand-held multimedia presentation of trial information plus a standard written participant information leaflet compared to a standard written participant information leaflet, alone;
-
b)
To explore whether a hand-held multimedia presentation of trial information plus a standard written participant information leaflet improves retention of patient participants to the end of the host trial;
-
c)
To establish if the quality of decision-making as measured through a decisional scale, adapted from one used within the REFORM trial 14 and drawing conceptually on the SURE 15 and DelibeRATE scales 16, 17 is affected by the presentation mode (multimedia and written -v- written only) of participant information to patients.
Methods
Study design
A two-group, parallel embedded RCT using the SATIN trial as an example of how SWAT-15 could be implemented.
Study population
The study population will be individuals who will be screened for and/or who are eligible to take part in the host SATIN trial.
Inclusion criteria
To participate in SWAT-15 individuals must, as determined by the host SATIN trial:
Be attending a general practice that is taking part in the trial;
Have a GP-diagnosed UTI, and at least one of the symptoms of dysuria, urinary frequency, or urgency with/without low abdominal pain;
Be a woman (non-pregnant) aged 18 years or above;
Be able and willing to give written informed consent;
Own a smartphone.
Exclusion criteria
Exclusion criteria are as per the SATIN trial i.e., any signs of complicated infection or any condition that may lead to complications, current or recent antibiotic use, recent UTI, current intake of NSAIDs, pregnancy or breastfeeding, non-use of highly effective contraception, previous adverse reaction to any of the study drugs, current intake of drugs potentially interacting with the trial drugs, diabetes mellitus, chronic kidney disease or any other previous illness related to kidney or urinary tract, history of gastro-intestinal ulcers, Glucose 6 phosphate Dehydrogenase deficiency or any other medical condition that may put the participant at risk or influence the study results in the investigators’ opinion.
Study setting
The SWAT will be carried out in general practices.
Assignment of interventions
Allocation will be performed at the individual patient level with half of the participants receiving:
-
a)
Multimedia digital information resource facilitated through a handheld tablet device, plus a written participant information leaflet, within the general practice (n=230) (Intervention) (see extended data 18)
OR
-
b)
Written participant information leaflet (n=230) (Comparator) (see extended data 19)
A cluster design could also be utilised but the resultant impact on study power would need to be considered. During the study period, potentially eligible participants (n=460) will be identified by the GP during their routine consultations based on the inclusion and exclusion criteria outlined above. Patients who are deemed eligible and willing to consider participating in the host study will be seen by a practice nurse or General Practitioner (GP). The practice nurse or GP will give consecutive eligible patients the next sequentially ordered, participant information leaflet, which will be taken from the top of the bundle of trial information pack. Attached to the participation information leaflet will be a sequentially numbered, sealed, opaque envelope with an ‘envelope ID’ number on it and a card inside with details of the woman’s group allocation. Group allocation will be determined by computer generation of a random allocation sequence with a 1:1 ratio and block sizes of 4, 8 and 8 (at random). As per allocation, the practice nurse or GP will give the potential participant either the patient information leaflet with a handheld tablet device and headphones to access the multimedia information (intervention) or no additional information i.e. the potential participant is given an information leaflet only (comparator). The practice nurse or GP will subsequently record the envelope ID number on the SATINs screening and enrolment log. The GP and/ or practice nurse will have been trained in the requirements of the host trial, and this SWAT, including in the use of the digital resource.
Sample size, estimated effect size and power
Primary analysis is the comparison of the differences in proportions of women recruited to the SATIN trial between intervention and comparator. The host trial aims to recruit 460 women. Based on a background proportion recruitment (i.e., randomised) of 64% of eligible women agreeing to participate in the SWAT using conventional written patient information (only) 20 and an acceptable error rate of α = 5%, this SWAT will have power of 80% to detect a 18% relative increase in recruitment proportions between control and intervention groups (i.e., 64% v 76%, absolute difference 12 percentage points).
We acknowledge that the effects we are assuming are relatively large for a test of two different ways of presenting trial information to potential participants. However, the sample sizes of SWATs addressing important methodology questions are limited by the size of the host trial. SWATs provide important data for appropriate pooling and meta-analysis, and the evidence base can be developed by encouraging other trials to run the same intervention in other contexts. This study will add to the limited evidence base in the area of trial recruitment and enable the development of pooled datasets capable of informing whether intervention effects vary by country, trial, or participant population.
Designing recruitment material
Both comparator and intervention arms focus on the provision of trial information to eligible participants. The decision to focus on presentation of patient information was based on the limited amount of empirical evidence available on how the quality of patient decision making is effected by the use of multimedia patient information and whether different modes of presentation can improve recruitment 21. Both trial arms will provide similar information and will conform to Good Clinical Practice guidance 22 and to the Declaration of Helsinki 23 for gaining informed consent.
Comparator: Written participant information leaflet
Participants in the comparator arm will receive written participant information. The design was informed by examples of similar patient information leaflets and the requirements of Good Clinical Practice 22. Information provided within the written participant information leaflet answer each of the following questions:
Can I stop taking part if I wish?
What is the purpose of this study?
Why is this study important?
Why have I been asked to take part in this study?
What does taking part mean for me?
What will I be asked to do?
Is my information confidential?
What are the benefits of taking part?
What are the risks of taking part?
What do I do if I feel worse or do not improve?
What happens if I suffer complications because of the study?
Compensation
Who should I contact if I’m concerned about the running of this study?
Where can I find more information?
Further queries
The content of the written participant information leaflet was reviewed by a practice nurse, host trial steering group members, the host trials Public and Patient Partnership in Research (PPP-R) group and the Health Research Board Clinical Research Facility, Galway, (HRB CRFG) representatives, who suggested changes to the content. The literacy level of the participant information leaflet was assessed using readability formulas (SMOG) available online 24.
Intervention design
The design of this SWAT intervention was informed by best practice approaches demonstrated by the MRC START (Medical Research Council Systematic techniques for assisting recruitment to trials) programme of recruitment research 25. The MRC START programme of recruitment research was developed based on relevant theoretical and empirical work about patient decision-making generally and in trials specifically. Our SWAT intervention design was also informed and constrained by the SATIN trial design and the complexity of the general practice setting in which we proposed it would be implemented. The multimedia digital information resource needed to be easily accessible through an electronic tablet device which will be provided to the potential participants within the general practices and provide adequate information in a relatively short period of time (e.g. 15–20 minutes). The researchers sought to make the resource as sustainable as possible therefore integrated the content into the same pre-existing website, which also supported the host trial support material.
The MRC START programme adopted a process for optimising readability and navigation of participant recruitment material 25, 26. A similar approach was adopted to design this SWAT intervention. The SWAT-15 core team members combined their expertise with findings from previous published trial recruitment research 3, 4, patient decision making research 27, and behavioural theory. The structure and content of the multimedia digital information intervention was informed by PPP-R forum members feedback, a steering group consisting of international experts in trial recruitment, and by the generic website template provided by the MRC START team 28. The written participant information leaflet (comparator) formed the basis for the development of this multimedia digital information intervention.
The multimedia intervention component was made up of six sections, which repeat the written patient information in text format and supplemented it with multimedia resources ( Figure 1).
-
1.
Why we need your help?- A video from the lead researcher describing the rationale for the trial (trial specific)
-
2.
What will happen during the study?- An infomercial explaining what the patient was expected to do during the trial (trial specific)
-
3.
Questions and Answers
-
4.
Why are we doing this study?
-
5.
What are randomised controlled trials? - An infomercial describing what random allocation is (universal generic information)
-
6.
Contact Us
The universal generic applicable content related to trial design was adopted from the MRC START resource 25.
During the design phase, the research team drafted an outline of the content of the multimedia website and wireframes of the proposed content and website navigation. The scripts for the generic and host trial video and infomercial were drafted; the text to accompany the videos came from the comparator patient information leaflet.
A ‘think aloud method’ was used to test the design and content of the website. This technique has been used previously to improve patient versions of clinical guidelines 29. The ‘think aloud’ method uses semi structured interview guide to explore first impressions of the multimedia digital information 30. The five semi structured interviews conducted explored six facets for the user experience namely credibility, usefulness, desirability, findability, and value 31. The seventh facet of Morville’s model, accessibility, was also explored as the intervention needed to be easily accessible on a tablet device 32. The process of integrating the multimedia digital information through a handheld tablet into practice will be pre-tested in one general practice prior to the launch of the host RCT. During this process, additional elements such as ease of use and the instructions of how to use the device will be tested and changes made as required.
Figure 1. Website template.
Informed consent
Informed consent will be sought for all women who decide to participate in the host trial, which was granted ethical approval from the Irish College of General Practitioners Ethics Committee (1 st December 2016). Formal consent to participate in the SWAT, at the individual, patient-participant level will not be sought as this embedded study is not withholding information; instead the focus is on how it is presented. All potential participants will receive the host trial participant information leaflet approved by the Research Ethics Committee. Further, by telling participants that they are being randomised to different recruitment strategies would not only contaminate the results and undermine the intervention being tested, as it would unduly draw potential trial participant’s attention to the recruitment process, it would introduce complexity and likely confusion for participants due to the double consent present. There is precedence for this, as a similar approach has been used previously within the MRC START programme of research on the basis that the embedded study is not withholding information – just changing the way it is presented (while also presenting in traditional information leaflet format) (NRES Committee Yorkshire and the Humber – South Yorkshire (REC Reference 11/YH/0271)). Permission to proceed with the current study, without obtaining formal consent, was approved by the ICGP Ethics Committee (1st December, 2016).
Outcomes
Primary outcome
The primary outcome will be proportion of potentially eligible participants willing to participate in the host trial.
The number of eligible participants willing to consider participation (numerator) will be calculated as the number of participants who sign the host trial informed consent form (including those that are subsequently found not to fulfil the eligibility criteria). The denominator will be the number of potential participants who are randomised to receive either the intervention or comparator.
Secondary outcomes
a) The proportion of participants recruited to the host trial (in case this differs from the number of participants willing to participate, due to, for example, an exclusion criterion).
This will be recorded in the host trial electronic case report form (eCRF) (or equivalent). The host trial research team will share the number of recruited participants to their study (i.e., the number of participants who sign a consent form and complete screening through the eCRF) with the SWAT research team. Demographic information (i.e., age, education, number of children, health insurance status) will be collected as part of the host study from participants who give informed consent and shared with the SWAT team. All data will be aggregated and anonymous and it will not be possible for the SWAT research team to identify individual participants.
The nurse/ GP will complete a screening and enrollment log for all patients who are willing to consider participating in the study. Any patient who receives the study intervention or comparator will be allocated an envelope ID number and this will be recorded on the screening and enrollment log. The envelope ID number is the key to identifying the intervention to which participants were randomised.
The practice nurse will also record the patient ID number on the screening and enrollment log form. A patient ID is generated within the eCRF when it is opened and the patient has given informed consent to participate. The patient ID will allow the SWAT team to identify if the demographic characteristics differ between consenting patients in the intervention and comparator groups.
b) The proportion of recruited participants who are retained to the end of the host trial;
Retention will be measured as the number of participants who complete outcome measures in the host trial.
c) The quality of decision-making
Quality of decision-making will measure the women’s understanding of participating in the host trial. The quality of decision-making by potential host trial participants will be measured through the completion of a decisional scale, adapted from one used within the REFORM trial 14 and drawing conceptually on the SURE 15 and DelibeRATE scales 16, 17. The SWAT research team will analyse these data after the host study has finished recruiting and the last patient has completed the study.
Statistical analysis plan
Descriptive statistics and correlations will be reported. The analysis plan will be as per stated in the study outcomes. The number of eligible participants willing to consider participation (numerator) will be calculated as the number of participants who sign the host trial informed consent form (including those that are subsequently found not to fulfil the eligibility criteria). The denominator will be the number of potential participants who are randomised to receive either the intervention or comparator. Number of patients retained will be calculated as a frequency.
Data management plan
The Data Management Plan was developed alongside the SATIN trial. If central unblinding for the host trial is necessary this will be provided by a specialised external company. All recruitment and consent data will be managed by the SATIN team as part of the host trial. The host trial will give the researchers access to anonpymised information in relation to recruitment and retention to the host trial. Data will be captured on an encrytpted electronic CRF and a survey included in the SATIN mobile App.
Dissemination of information
The findings of this SWAT were to be published in Peer review journals and presented at both international and national conferences. As this SWAT was terminated prior to recruitment, it is hoped that this protocol could be implemented in another setting.
Study status
The host trial, SATIN ( ISRCTN88111427); was stopped prior to recruitment of the first participant due to the emergence of new evidence on the treatment of urinary tract infections (UTI) and therefore this SWAT although designed was never implemented.
Discussion
Improving the efficiency of how randomised trials are planned, conducted, analysed and reported is an important area of research and one of increased interest by the trial community 33. Inadequate recruitment to trials has been identified as an important contributor to research waste that contributes to increased research costs and delayed information on the effectiveness of health care interventions 3, 4.
This protocol for SWAT-15 offers an opportunity to answer important questions on efficiencies in trial processes by embedding primary trial methodological studies within host randomised trials. This SWAT has the potential to evaluate the effectiveness of a handheld digital multimedia presentation of trial information and written participant information to potential participants on recruitment and retention to the host SATIN trial. This will help inform future recruitment and retention strategies to trials evaluating Investigational Medicinal Products in the primary care setting.
Ethical approval
Ethical approval was granted by the ICGP Ethics Committee on 1st December 2016
Data availability
Underlying data
No data are associated with this article
Extended data
Figshare: SWAT Patient Information Leaflet. https://doi.org/10.6084/m9.figshare.11894385.v2 19
This project contains the following extended data:
-
-
SATIN PIL Version 4.0 08062017.pdf (Patient information leaflet)
Figshare: SWAT Website. https://doi.org/10.6084/m9.figshare.11923146.v1 18
This project contains the following extended data:
-
-
SWAT Website.pdf (Series of screenshots of the SWAT website)
Reporting guidelines
SPIRIT checklist for ‘The effectiveness of digital multimedia presentation of trial information on recruitment and retention of patients: Protocol for a study within a trial (SWAT).’ https://doi.org/10.6084/m9.figshare.11894355.v1 34
Acknowledgements
This research was funded by the Health Research Board in Ireland as part of the HRB Primary Care Clinical Trials Network Ireland. The authors would like to acknowledge with thanks the international steering group who advised on the design of this SWAT, in particular, Professor Mike Clarke, Professor Peter Bower and Dr Jo Rick. We would also like to acknowledge and thank the MRC START team members who shared their resources and expertise with us during the design stages of this SWAT. We would also like to thank Prof Peter Knapp and colleagues who permitted the SWAT team to adapt their modified quality of decision-making questionnaire.
Funding Statement
Health Research Board Ireland [CTN-2014-011]
[version 1; peer review: 2 approved]
References
- 1. Moher D, Hopewell S, Schulz KF, et al. : Resources for authors of reports of randomized trials: harnessing the wisdom of authors, editors, and readers. Trials. 2011;12(1):98. 10.1186/1745-6215-12-98 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Bower P, Wilson S, Mathers N: Short report: how often do UK primary care trials face recruitment delays? Fam Pract. 2007;24(6):601–3. 10.1093/fampra/cmm051 [DOI] [PubMed] [Google Scholar]
- 3. Treweek S, Lockhart P, Pitkethly M, et al. : Methods to improve recruitment to randomised controlled trials: Cochrane systematic review and meta-analysis. BMJ Open. 2013;3(2): pii: e002360. 10.1136/bmjopen-2012-002360 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Treweek S, Mitchell E, Pitkethly M, et al. : Strategies to improve recruitment to randomised controlled trials. Cochrane Database Syst Rev. 2010; (1): MR000013. 10.1002/14651858.MR000013.pub4 [DOI] [PubMed] [Google Scholar]
- 5. McDonald AM, Knight RC, Campbell MK, et al. : What influences recruitment to randomised controlled trials? A review of trials funded by two UK funding agencies. Trials. 2006;7:9. 10.1186/1745-6215-7-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Sully BG, Julious SA, Nicholl J: A reinvestigation of recruitment to randomised, controlled, multicenter trials: a review of trials funded by two UK funding agencies. Trials. 2013;14(1):166. 10.1186/1745-6215-14-166 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Treweek S, Pitkethly M, Cook J, et al. : Strategies to improve recruitment to randomised trials. Cochrane Database Syst Rev. 2018;2(2): MR000013. 10.1002/14651858.MR000013.pub6 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Bower P, Wallace P, Ward E, et al. : Improving recruitment to health research in primary care. Fam Pract. 2009;26(5):391–7. 10.1093/fampra/cmp037 [DOI] [PubMed] [Google Scholar]
- 9. Shneerson C, Windle R, Cox K: Innovating information-delivery for potential clinical trials participants. What do patients want from multi-media resources? Patient Educ Couns. 2013;90(1):111–117. 10.1016/j.pec.2012.06.031 [DOI] [PubMed] [Google Scholar]
- 10. Smith V, Clarke M, Devane D, et al. : SWAT 1: what effects do site visits by the principal investigator have on recruitment in a multicentre randomized trial? J Evid Based Med. 2013;6(3):136–137. 10.1111/jebm.12049 [DOI] [PubMed] [Google Scholar]
- 11. Graffy J, Bower P, Ward E, et al. : Trials within trials? Researcher, funder and ethical perspectives on the practicality and acceptability of nesting trials of recruitment methods in existing primary care trials. BMC Med Res Methodol. 2010;10(1):38. 10.1186/1471-2288-10-38 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Treweek S, Bevan S, Bower P, et al. : Trial Forge Guidance 1: what is a Study Within A Trial (SWAT)? Trials. 2018;19(1):139. 10.1186/s13063-018-2535-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Clarke M, Savage G, Maguire L, et al. : The SWAT (study within a trial) programme; embedding trials to improve the methodological design and conduct of future research. Trials. 2015;16(Suppl 2):P209 10.1186/1745-6215-16-S2-P209 [DOI] [Google Scholar]
- 14. Knapp P, Graffy J, Bower P, et al. : Preliminary analysis of a new measure of quality of patient decision making about research participation.In: Presented at the 4th International Clinical Trials Methodology Conference.Liverpool;2017. [Google Scholar]
- 15. Ferron Parayre A, Labrecque M, Rousseau M, et al. : Validation of SURE, a four-item clinical checklist for detecting decisional conflict in patients. Med Decis Making. 2014;34(1):54–62. 10.1177/0272989X13491463 [DOI] [PubMed] [Google Scholar]
- 16. Gillies K, Elwyn G, Cook J: Making a decision about trial participation: the feasibility of measuring deliberation during the informed consent process for clinical trials. Trials. 2014;15(1):307. 10.1186/1745-6215-15-307 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Sivell S, Edwards A, Manstead AS, et al. : Increasing readiness to decide and strengthening behavioral intentions: evaluating the impact of a web-based patient decision aid for breast cancer treatment options (BresDex: www.bresdex.com). Patient Educ Couns. 2012;88(2):209–217. 10.1016/j.pec.2012.03.012 [DOI] [PubMed] [Google Scholar]
- 18. Duane S, Devane D: SWAT Website. figshare. 2020; Media. 10.6084/m9.figshare.11923146.v1 [DOI] [Google Scholar]
- 19. Vellinga A, Duane S, Devane D, et al. : SWAT Patient Information Leaflet. figshare. 2020; Book. 10.6084/m9.figshare.11894385.v2 [DOI] [Google Scholar]
- 20. Gágyor I, Haasenritter J, Bleidorn J, et al. : Predicting antibiotic prescription after symptomatic treatment for urinary tract infection: development of a model using data from an RCT in general practice. Br J Gen Pract. 2016;66(645):e234–e240. 10.3399/bjgp16X684361 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21. Flory J, Emanuel E: Interventions to improve research participants' understanding in informed consent for research: a systematic review. JAMA. 2004;292(13):1593–1601. 10.1001/jama.292.13.1593 [DOI] [PubMed] [Google Scholar]
- 22. International Conference on Harmonisation Working Group: ICH harmonised tripartite guideline: guideline for good clinical practice E6 (R1).In: International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Us: 1996. 1996;59 Reference Source [Google Scholar]
- 23. World Medical Association: World Medical Association Declaration of Helsinki. Ethical principles for medical research involving human subjects. Nurs Ethics. 2002;9(1):105–913. [DOI] [PubMed] [Google Scholar]
- 24. Readibility Formulas. Reference Source [Google Scholar]
- 25. Rick J, Graffy J, Knapp P, et al. : Systematic techniques for assisting recruitment to trials (START): study protocol for embedded, randomized controlled trials. Trials. 2014;15(1):407. 10.1186/1745-6215-15-407 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26. Man MS, Healthlines Study Group, . Rick J, et al. : Improving recruitment to a study of telehealth management for long-term conditions in primary care: two embedded, randomised controlled trials of optimised patient information materials. Trials. 2015;16(1):309. 10.1186/s13063-015-0820-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27. Charles C, Gafni A, Whelan T: Decision-making in the physician-patient encounter: revisiting the shared treatment decision-making model. Soc Sci Med. 1999;49(5):651–661. 10.1016/s0277-9536(99)00145-8 [DOI] [PubMed] [Google Scholar]
- 28. MRC START multimedia blank template. Reference Source [Google Scholar]
- 29. Fearns N, Graham K, Johnston G, et al. : Improving the user experience of patient versions of clinical guidelines: user testing of a Scottish Intercollegiate Guideline Network (SIGN) patient version. BMC Health Serv Res. 2016;16(1):37. 10.1186/s12913-016-1287-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30. Rosenbaum SE, Glenton C, Nylund HK, et al. : User testing and stakeholder feedback contributed to the development of understandable and useful Summary of Findings tables for Cochrane reviews. J Clin Epidemiol. 2010;63(6):607–619. 10.1016/j.jclinepi.2009.12.013 [DOI] [PubMed] [Google Scholar]
- 31. Morville P: User Experience Honeycomb.2006;15:2013 Reference Source [Google Scholar]
- 32. Morville P: User experience honeycomb. Web Log Post]2004;21:09–12. Reference Source [Google Scholar]
- 33. Treweek S: Trial forge: a systematic approach to making trials more efficient. Trials. 2013;14(S1):O121 10.1186/1745-6215-14-S1-O121 [DOI] [Google Scholar]
- 34. Duane S: SWAT Protocol_SPIRIT_Fillable-checklist-15-Aug-2013 (00000002).doc. figshare. 2020; Preprint. 10.6084/m9.figshare.11894355.v1 [DOI] [Google Scholar]