Abstract
Introduction
Patient-reported outcomes (PRO) are currently collected from trial participants using paper questionnaires by the Clinical Trials and Statistics Unit at The Institute of Cancer Research (ICR-CTSU). Streamlining PRO collection using electronic questionnaires (ePRO) may improve data collection and patient experience. Here, we outline our protocol for a Study within a trial of electronic versus paper-based Patient-Reported oUtcomes CollEction (SPRUCE), which investigates the acceptability of ePRO in oncology clinical trials.
Methods and analysis
SPRUCE was developed alongside patient and public contributors. SPRUCE runs in multiple host trials with a partially randomised patient preference design, allowing participants to be randomised or choose their preference of electronic or paper questionnaires. Questionnaires are scheduled in accordance with host trial follow-up. The primary objective will assess differences in return rates (compliance) between ePRO and paper PROs at the first timepoint post-host trial intervention in the randomised group. Paper PRO compliance is expected to be 90%. 244 randomised participants are required to exclude ≤80% compliance rates with ePRO (10% non-inferiority margin, with 80% power and one-sided alpha=0.05). SPRUCE aims to assess acceptability of ePRO in oncology clinical trials, establish whether ePRO is acceptable to ICR-CTSU trial participants and can capture complete PRO data, consistent with paper PROs.
Ethics and dissemination
The SPRUCE protocol (ICR-CTSU/2021/10074) was approved by the Coventry and Warwick Central Research Ethics Committee (21/WM/0223) on 21 October 2021. Results will be disseminated via presentations, publications and lay summaries. No participant identifiable data will be included.
Trial registration
SWAT169.
Keywords: patient reported outcome measures, quality of life, adult oncology
STRENGTHS AND LIMITATIONS OF THIS STUDY.
This Study within a trial of electronic versus paper-based Patient-Reported oUtcomes CollEction (SPRUCE) is exploring the impact of electronic collection of patient-reported outcomes in a randomised controlled setting, to obtain robust information about the impact of data collection modality on the data reported for randomised oncology treatment trials.
The study has been designed as a partially randomised patient preference study so those unwilling to be randomised can select their preferred questionnaire format at study entry, to prevent exclusion of patients with a strong preference for one questionnaire modality.
The patient preference aspect of SPRUCE allows exploration of demographic differences between groups selecting different modalities.
SPRUCE is being conducted in selected oncology randomised controlled trials so results may not be generalisable to other disease settings.
Introduction
Within healthcare and clinical trials, questionnaires can be used to collect information directly from patients on the impact that treatment and health conditions may be having on their quality of life. These data are known as patient-reported outcomes (PRO) and are defined as ‘any report of the status of a patient’s health condition that comes directly from the patient, without interpretation of the patient’s response by a clinician or anyone else’.1 A wide range of validated questionnaires are used to collect PRO, covering general health items as well as more disease specific factors.
Within oncology trials, the patient perspective and survivorship effects are crucial factors to consider in evaluation of new treatments2 and information on quality of life (QoL) is a key factor to consider in their adoption and implementation. At many academic clinical trials units, including the Clinical Trials and Statistics Unit at the Institute of Cancer Research (ICR-CTSU), QoL information is currently captured with paper PRO questionnaires to inform primary and key secondary end points of clinical trials. Collection of PROs can be a time-consuming and laborious process requiring significant input from both patients, hospital staff and clinical trials units often over an extended period of time. Streamlining this process by using technology in the form of electronic PRO (ePRO) questionnaires requires administrative and funding resources for implementation. The use of ePRO has the potential to increase patient convenience, improve patient experience, reduce administrative burden, save costs, increase patient compliance and avoid potential secondary data errors due to data transcription, leading to more accurate and complete data.3 4
ePRO questionnaires have been widely studied in the general clinical setting, establishing the intrapatient equivalence of scores on paper and electronic questionnaires.5 6 Current evidence for the use of ePRO demonstrates substantial variability in uptake, with consent rates to ePRO capture being only 37.8% compared with 75% for paper PROs in one study,7 compared with a second study showing 75% uptake in filling out electronic-reported questionnaires.8 Many studies evaluating the use of ePRO tools in a clinical setting have excluded participants without internet access.9 10
In the clinical trial setting, it is vital that all trial participants are able to complete PRO, including those without either access or ability to use the internet. To further evaluate the use of ePRO in clinical trials, with the associated additional research ethics, governance and regulatory requirements, we have designed the Study within a trial of electronic versus paper-based Patient-Reported oUtcomes CollEction (SPRUCE). The protocol is outlined below.
Study hypothesis
The collection of ePRO in clinical trials will be acceptable to participants and will provide a similar compliance level of data collection to paper PROs.
Methods and analysis
Study design and setting
SPRUCE is a partially randomised patient preference study within a trial (SWAT) investigating electronic versus paper PRO collection. The study design allows participants to either be randomised or choose their modality of preference. As a SWAT, SPRUCE will run in multiple actively enrolling ICR-CTSU host trials, with the aim of including participants affected by a range of cancers representative of those in ICR-CTSU’s trial portfolio.
Host trials will be identified within ICR-CTSU and approached for inclusion in SPRUCE. Trials with an anticipated recruitment period of at least 1 year will be considered. A minimum of two and a maximum of six host trials will be included. One host trial will be set up at a time and SPRUCE will be opened at the highest recruiting sites within the host trial.
Participants who are enrolled in a SPRUCE host trial will be approached to consider taking part in the SPRUCE study. Those who provide written informed consent will be either randomised 1:1 between electronic and paper PRO questionnaire completion or registered for electronic or paper PRO collection if they elect to join the patient preference arm of the study (figure 1).
Figure 1.
Study within a trial of electronic versus paper-based Patient-Reported oUtcomes CollEction (SPRUCE) schema.PRO, patient-reported outcomes.
The study will run in two formats depending on the host trial within which it is embedded:
-
Format A:
Applicable where a host trial includes PRO within the approved trial protocol. The instrument used to capture the host trials’ primary PRO endpoint will be the key questionnaire of interest in SPRUCE. PRO data for host trial participants who are also participating in SPRUCE will be collected within SPRUCE (and not within the host trial) and the questionnaire schedule and content will follow that of the host trial. Data will be shared with the host trial for the purpose of the host trial’s PRO analysis. Details of each host trial will be set out in an appendix to the SPRUCE protocol.
-
Format B:
Applicable where a host trial does not include PRO within the approved trial protocol. A relevant PRO hypothesis and questionnaire will be developed for the patient population in collaboration with host trial investigators. Questionnaires will be completed by participants at time points selected to fit the host trials’ patient pathway.
Objectives
The primary objective of SPRUCE is to assess whether there are differences in return rates (compliance) between electronic and paper PRO questionnaires at the first time point post-intervention within a host trial.
Secondary objectives are outlined below:
To investigate whether there are differences in response scores between the two modalities (electronic and paper) at key time points (eg, due to convenience of returning ePRO questionnaires, more severe PRO issues may be identified electronically).
To assess whether there are differences in return rates (compliance) between paper and electronic questionnaires at later questionnaire time points.
To investigate whether there are differences in number of items completed within a questionnaire (completeness) between electronic and paper questionnaires.
To investigate whether there are differences in satisfaction between participants completing electronic or paper questionnaires.
To investigate whether changes in response scores from baseline (paper questionnaire) to follow-up vary according to modality of follow-up questionnaire completion.
To investigate whether there are any demographic differences between people who agree to be randomised and those who choose to complete questionnaires electronically or on paper.
To investigate the time taken to distribute paper questionnaires compared with electronic questionnaires.
To assess the requirement to remind patients to complete their paper and electronic questionnaires.
To assess patient acceptance of randomisation between questionnaire modalities.
To assess overall feasibility of recruitment.
Participants (inclusion and exclusion criteria)
Inclusion criteria are outlined below:
Participation in a host trial.
Participation in host trial’s QoL/PRO substudy (if applicable, format A only).
Informed consent for participation in SPRUCE.
Ability to read English.
There are no exclusion criteria.
Procedure (recruitment, study schedule and interventions)
The procedure for SPRUCE is outlined in figure 1. Host trial participants providing written informed consent for SPRUCE will be asked if they agree to be randomised between electronic versus paper PRO. If they agree, they will be allocated 1:1 to receive either electronic or paper questionnaires using a minimisation algorithm with a random element and balancing factors of age, sex and host trial.
If they do not agree to be randomised, they can take part in the study in the group of their choice, completing either paper or electronic questionnaires, and the reason for refusing randomisation/preferring one format will be recorded.
Study assessments
Participants will complete an initial questionnaire booklet on paper at the time of host trial entry. They will also complete a demographic form specific to the SPRUCE study. Further questionnaires will be completed in accordance with their randomly allocated or preferred format.
Participants completing questionnaires electronically will receive email reminders at the required time points, with a personalised link to the electronic questionnaire for completion which can be completed on any electronic device. If the questionnaires are not completed within a timeframe appropriate to the host trial’s QoL schedule, a reminder email will be sent. ICR-CTSU will provide guidance to participants to aid the completion of questionnaires.
Participants completing questionnaires on paper will receive booklets by post directly to their home address sent by the ICR-CTSU following confirmation that the patient is able to complete the questionnaire. Participants will post completed booklets back to ICR-CTSU using a prepaid envelope and the responses will be entered into the study database by the SPRUCE team. If the questionnaires are not returned within a timeframe appropriate to the host trial’s QoL schedule, a reminder letter will be sent to participants. If required, one further reminder will be sent to participants in both groups and an option will be offered for participants to change method of questionnaire collection if they find it difficult to use their randomised or chosen modality.
Questionnaires will be administered for the purposes of the SPRUCE study up to 12 months post-study entry, after which PRO data will continue to be collected within the host trial as appropriate.
If a participant wishes to switch modality of questionnaire during the study, this is permitted and the reason for the change will be recorded.
Sample size calculation
Sample size estimates are based on numbers required for the randomised part of the study. Based on compliance reports from existing ICR-CTSU trials, return rates for paper questionnaires are expected to be in the region of 90% at the first post-intervention time point; 244 patients would therefore be required to be randomised (1:1) to exclude ≤80% compliance rates with ePRO (ie, 10% non-inferiority margin, with 80% power and 1-sided alpha=0.05).
We have assumed that approximately two-thirds of patients entering SPRUCE will agree to be randomised, and thus anticipate needing to recruit 366 patients overall (244 randomised and 122 preference).
The proportion of participants opting for allocation via randomisation versus preference will be monitored throughout SPRUCE. If the numbers in the randomisation cohort are lower than anticipated, then the study design may be changed to offer all patients the choice of electronic or paper questionnaires.
Statistical analysis
All participants who consent to enrol in SPRUCE will be included in the analysis. The randomised and patient preference cohorts will be analysed separately for all endpoints. Confounders, including patient baseline characteristics (eg, age and sex), will need to be taken into account for the comparisons between electronic and paper questionnaires in the non-randomised patients. A per protocol analysis of participants still receiving the same questionnaire as they were originally allocated will be performed for the primary endpoint.
Regression analyses will adjust for potential confounders, such as patient demographics and clinical characteristics, for the comparison of outcomes between the patient preference groups. Additionally, descriptive analyses will summarise demographic and clinical characteristics of patients opting for randomisation vs expressing a preference.
The primary outcome of compliance at the first post-intervention time point within the host trial will be calculated as a percentage of returned questionnaires out of those expected (ie, not withdrawn or died) for the electronic and paper questionnaire groups, and the difference calculated with a two-sided 90% CI. Non-inferiority for electronic questionnaires will be concluded if the lower confidence limit for the difference in compliance for electronic versus paper questionnaires is greater than −10%.
For the secondary outcomes of questionnaire compliance at further time-points and data completeness, descriptive analyses will compare percentages between groups at each time-point. Questionnaire domain scores will be calculated as per guidance for each specific measure (eg, the global health/overall QoL score from the European Organisation for Research and Treatment of Cancer Quality of Life Questionnaire comprises 30 items (EORTC QLQ-C30)), and compared between electronic questionnaires and paper questionnaires at each time-point using descriptive statistics appropriate for the distributions (eg, means or medians for numeric scales and percentages for categorical outcomes). Comparisons of ePRO and paper PRO will be undertaken, stratified for host trial and randomisation arm within the host trial. For continuous scores, a t-test or Mann-Whitney tests will be used as appropriate. For ordinal data, χ2 test for trend will be used. χ2 tests will be used for categorical data; Fisher’s exact test will be used if cell frequencies are small (<5).
A linear regression model (analysis of covariance) will analyse the QoL score change from baseline as the dependent variable with the baseline score and questionnaire modality included as covariates.
The proportion of participants opting for allocation via randomisation versus participant preference will be monitored in the secondary outcomes, with the number of participants opting for preference assessed after 50 participants have been recruited. If greater than 50% of participants have opted for the preference arm of the study, this will be reviewed by the study oversight committees and consideration will be given to ending the randomisation cohort.
Patient and public involvement
Patient and public involvement (PPI) is central to the work of the ICR-CTSU; design of the SPRUCE study was informed by undertaking a public survey on attitudes to electronic completion of health questionnaires and conducting a series of patient and public online focus groups. The survey protocol received Higher Education Institutional Ethics approval (CCR5447). The survey was conducted between 9 February 2021 and 16 June 2021. It was advertised both online and, with the preclusion of face-to-face contact as a result of the COVID-19 pandemic, we attempted to reach a wider audience of participants by publishing advertisements in offline mediums such as newspapers and newsletters. Similarly, participants in a routine telephone prostate cancer clinic were approached by the study lead (LP) and consented to receive the survey by post to ensure that it reached them with no need for internet access. Seventeen respondents completed the survey on paper and 33 online.
Of the 50 respondents who completed the survey, 47 had regular access to the internet either at home or on their mobile telephone; 72% of respondents stated they would prefer to complete a health questionnaire online rather than on paper. Of those who would prefer to complete it on paper, 6/11 (55%) would be happy to complete online if requested. Those who did not have access to the internet were all aged 71 or older and were in the lowest educational and income bracket.
These findings underline the requirement for maintaining the option of completing PRO on paper in cancer clinical trials, to prevent the exclusion of any trial participants.
Following on from the survey, respondents were asked if they would be prepared to be involved in a focus group on the SPRUCE study design, and eight took part in focus groups on 8 September 2021 and 20 September 2021. Recommendations from these groups were subsequently included in the study protocol. Focus group suggestions included ensuring that we monitor the impact of the study on host trial PRO study enrolment. Similarly, following end user testing by focus group contributors, recommendations were made to improve the user friendliness of the database design, including the ability for patients to return to previous questions in the questionnaire before submitting and including a comments box at the end of the questionnaire.
Although our survey found that over 90% of participants had access to the internet, there was likely to have been bias towards respondents having easy internet access as advertising the survey offline and outreach to less digitally literate groups was difficult due to the constraints on movement and in person gatherings during the pandemic. Further measures arranged at the focus group stage included offering the loan of an internet enabled tablet to join the focus group, which would have been held in person in the absence of the pandemic. No participants took up this offer and as such, we unfortunately did not manage to have a focus group including anyone with no previous access to the internet. It is a known difficulty within PPI work to engage a diverse group of participants; although we were not successful in this instance, we are aware of the limitation and have tried to consider this deficit of opinion when taking the study protocol plan forwards.
Those who joined the focus groups were invited to join the SPRUCE Patient and Public Oversight Committee, six of whom joined. Four also joined the Study Management Group. These representatives contributed to study design and are actively involved in the ongoing oversight of SPRUCE. The Patient and Public Oversight Committee members will also aid the trial team in preparing documents (such as presentations, publications and lay summaries) to disseminate the results to patients.
Study status
The first participant joined SPRUCE on 11 April 2022. The study is open in three host trials as of 16 August 2023. Four host trials in total were approached to open SPRUCE, however it was not possible to implement in one of them as the timelines for enrolment were not compatible with SPRUCE. SPRUCE target enrolment is expected to be reached in 2024.
Study governance
The study is sponsored by The Institute of Cancer Research and centrally managed by the ICR-CTSU. A study management group and patient and public oversight committee meet regularly to oversee progress and will advise on dissemination of results.
SPRUCE is registered on the SWAT Repository (SWAT 169) store, hosted by the Northern Ireland Hub for Trials Methodology Research.11
Ethics and dissemination
This study received ethics approval from the Coventry and Warwick Central Research Ethics Committee (21/WM/0223) on 21 October 2021. Informed consent is obtained from participants prior to study entry.
Study results will be shared as widely as possible and will be published in peer-reviewed journals and in conference proceedings. Plain English summaries of the results will be provided to study participants and disseminated via the ICR-CTSU website and other appropriate routes.
Discussion
This paper describes the protocol for the SPRUCE study, aiming to assess the impact of using ePRO for participant questionnaires within ICR-CTSU trials. While there is extensive data in the literature regarding intrapatient validity of electronic versus paper questionnaires, and on their use in a clinical setting, there is very little information about their validity within clinical trials.3 5 6 8 12 13 By designing a robust comparative study, we seek to assess whether electronic questionnaires can be considered equivalent to collecting data on paper. The main aim of the study is to verify that electronic questionnaire return rates are no lower than those observed on paper. However, we have also included a number of secondary endpoints which seek to further elucidate any potential impact on participants and the data collected.
By running SPRUCE as a SWAT, the study can be run across multiple clinical trials managed by the ICR-CTSU, providing the opportunity for patients with different types of cancer to be included. Results of the study should therefore provide a representative picture of the impact of any future ePRO rollout across ICR-CTSU’s portfolio of trials, and on trials conducted in similar settings by other groups. The SWAT has been designed to complement the host trials and aims to keep research burden for participants to a minimum, by using the host trials’ current PRO questionnaires where applicable.
In order to prevent the exclusion of participants with a strong preference for one questionnaire modality over another or those without internet access, we designed the study to include a patient preference group. The advantage of this approach is that the study will collect real-world data and no patient will be excluded from completing PROs due to lack of access to technology. However, it is possible that most participants may opt to select their preferred method of data collection, leading to failure to reach the randomised sample size required. This circumstance will necessitate a change in study design and subsequent analysis plan for this study.
The protocol described aims to provide evidence as to the feasibility of using ePROs in clinical trials and large clinical trial units. By running this study, we hope to gain robust data on whether the use of ePROs is acceptable to trial participants and provides good quality QoL data.
Supplementary Material
Footnotes
Contributors: The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted. LP was the previous Chief Investigator. RL is the current Chief Investigator. EH is the methodological lead. LP, DG, SF, JG, AG, JH, EH, DK, GM, MS, EH and RL contributed to the study design. LP, DG, SF, AG, JH, EH, GM, MS, EH and RL contributed to the development of the protocol. LP, JG, JH, DK, GM, MS, EH and LP are members of the SPRUCE Study Management Group, which contributes to study design and is responsible for oversight throughout the study. RL provides senior study management oversight. LP, EH, RL, GM and MS led manuscript writing; all other authors contributed to and reviewed the manuscript. EH and RL had the final responsibility for the decision to submit for publication. We thank our patient advisors who have contributed to study design and oversight.
Funding: This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors. The ICR-CTSU is supported by a programme grant from Cancer Research UK (CRUK) Programme Grant (C1491/A25351). LP is supported in her PhD by a CRUK clinical trials fellowship (A30384) and NHS funding to the NIHR Biomedical Research Centre at The Royal Marsden and the ICR. Patient and public involvement for the ePRO survey was supported by NHS funding to the NIHR Biomedical Research Centre at The Royal Marsden and the ICR. This report represents independent research supported by the National Institute for Health Research (NIHR) Biomedical Research Centre at The Royal Marsden NHS Foundation Trust and the Institute of Cancer Research, London. The views expressed are those of the authors and not necessarily those of the NIHR or the Department of Health and Social Care.
Competing interests: EH receives support from a Cancer Research UK (CRUK) core grant (C1491/A25351) and LP receives support from her PhD by a CRUK clinical trials fellowship (A30384); no other authors have any potential conflicts of interests to declare.
Patient and public involvement: Patients and/or the public were involved in the design, or conduct, or reporting or dissemination plans of this research. Refer to the Methods section for further details.
Provenance and peer review: Not commissioned; externally peer reviewed.
Ethics statements
Patient consent for publication
Not required.
Ethics approval
Coventry and Warwick Central Research Ethics Committee; 21/WM/0223; 21/10/2021
References
- 1.FDA . Patient-reported outcome measures: use in medical product development to support labeling claims. Clinical/Medical Federal Register 2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Eremenco S, Coons SJ, Paty J, et al. PRO data collection in clinical trials using mixed modes: report of the ISPOR PRO mixed modes good research practices task force. Value Health 2014;17:501–16. 10.1016/j.jval.2014.06.005 [DOI] [PubMed] [Google Scholar]
- 3.Coons SJ, Gwaltney CJ, Hays RD, et al. Recommendations on evidence needed to support measurement equivalence between electronic and paper-based patient-reported outcome (PRO) measures: ISPOR ePRO good research practices task force report. Value Health 2009;12:419–29. 10.1111/j.1524-4733.2008.00470.x [DOI] [PubMed] [Google Scholar]
- 4.Aiyegbusi OL. Key methodological considerations for usability testing of electronic patient-reported outcome (ePRO) systems. Qual Life Res 2020;29:325–33. 10.1007/s11136-019-02329-z [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Gwaltney CJ, Shields AL, Shiffman S. Equivalence of electronic and paper-and-pencil administration of patient-reported outcome measures: a meta-analytic review. Value Health 2008;11:322–33. 10.1111/j.1524-4733.2007.00231.x [DOI] [PubMed] [Google Scholar]
- 6.Muehlhausen W, Doll H, Quadri N, et al. Equivalence of electronic and paper administration of patient-reported outcome measures: a systematic review and meta-analysis of studies conducted between 2007 and 2013. Health Qual Life Outcomes 2015;13:167. 10.1186/s12955-015-0362-x [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Pugh SL, Rodgers JP, Yeager KA, et al. Characteristics of participation in patient-reported outcomes and electronic data capture components of NRG oncology clinical trials. Int J Radiat Oncol Biol Phys 2020;108:950–9. 10.1016/j.ijrobp.2020.06.030 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Basch E, Artz D, Dulko D, et al. Patient online self-reporting of toxicity symptoms during chemotherapy. J Clin Oncol 2005;23:3552–61. 10.1200/JCO.2005.04.275 [DOI] [PubMed] [Google Scholar]
- 9.Absolom K, Holch P, Warrington L, et al. Electronic patient self-reporting of adverse-events: patient information and aDvice (eRAPID): a randomised controlled trial in systemic cancer treatment. BMC Cancer 2017;17:318. 10.1186/s12885-017-3303-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Girgis A, Durcinoska I, Arnold A, et al. Web-based patient-reported outcome measures for personalized treatment and care (PROMPT-care): multicenter pragmatic Nonrandomized trial. J Med Internet Res 2020;22:e19685. 10.2196/19685 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.The SWAT Repository, Available: https://www.qub.ac.uk/sites/TheNorthernIrelandNetworkforTrialsMethodologyResearch/SWATSWARInformation/Repositories/SWATStore
- 12.Park JY, Kim BS, Lee HJ, et al. Comparison between an electronic version of the foot and ankle outcome score and the standard paper version: A randomized multicenter study. Medicine 2019;98:e17440. 10.1097/MD.0000000000017440 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Byrom B, Doll H, Muehlhausen W, et al. Measurement equivalence of patient-reported outcome measure response scale types collected using bring your own device compared to paper and a provisioned device: results of a randomized equivalence trial. Value Health 2018;21:581–9. 10.1016/j.jval.2017.10.008 [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.