Data collection methods for patient-reported outcome measures in cancer randomised controlled trials: a protocol for a rapid scoping review

Abstract

Background

There are different modes and ways to assess patient-reported outcomes (PROs) in clinical trials. However, there is little systematic information on how often different modes of assessment (MOA) are used in cancer clinical trials and how exactly assessments are conducted. The goal of this scoping review is to gain an understanding of the MOA and data management of PROs in cancer randomised controlled trials (RCTs) and the reporting quality thereof.

Methods and analysis

This scoping review protocol follows the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. Relevant trials will be identified via their indexed publications. We will search PubMed for RCTs conducted in cancer populations that evaluate a biomedical treatment with a PRO endpoint. Trials with publications published between January 2019 and November 2023 will be included. Two independent reviewers will review the references for both the abstract and full-text screening. We will extract data from the publications from a trial and the trial protocol if a protocol can be traced. Data will be summarised at the trial level. We will focus on a descriptive analysis of the MOA of PROs and on the relative frequencies of the different MOA. We will also evaluate the quality of reporting for the relevant SPIRIT and CONSORT guidelines that refer to the assessment of PROs in trials. Due to the scoping nature of our review, we will not perform a dedicated quality assessment of all trials.

Ethics and dissemination

The proposed review is based on secondary, published data. Hence, no ethics review is necessary. The review is part of an ongoing project on the use of electronic data capture methods in cancer clinical trials. The findings from the review will support the project and contribute to synthesising guidance to ultimately improve the (electronic) measurement of patient-reported outcome measures in clinical trials.

Strengths and limitations of this study.

• This scoping review protocol describes the methodology for a comprehensive exploration of how patient-reported outcome measures (PROMs) are assessed in cancer randomised controlled trials published on PubMed, with a particular emphasis on the mode of assessment (MOA) and the quality of reporting.

• The synthesis of how different MOAs are used in trials may further our understanding of how PROMs are assessed in trials and which trial characteristics are associated with different MOAs.

• The variability in how MOAs are reported (or not reported) may prevent quantitative analyses.

Background

Patient-reported outcomes (PROs) are increasingly assessed electronically in clinical trials, often referred to as ePRO data capture. Compared with traditional paper-based assessment, ePRO assessment holds the promise of improved data quality, more control over the correct assessment and timing of patient-reported outcome measures (PROMs) and shorter turnaround times, as well as improved audit capabilities.¹,⁵ Consequently, ePROs are the preferred assessment method as recommended by the Centre for Medical Technology Policy,⁶ in the European Society for Medical Oncology practice guidelines,⁷ and are discussed by regulatory bodies like the US Food and Drug Administration (FDA) and the European Medicines Agency.⁸ However, there are also potential drawbacks to ePRO data capture that may limit usefulness, including limited computer literacy in some populations, problems if the ePRO software is difficult to use for patients or study personnel or network dependency to function.

There is little systematic information on how often ePRO assessments are being used in cancer clinical trials and how exactly assessments are conducted. Trials may use paper-pencil PROMs, electronic PROMs through ePRO data capture (which can also include interactive voice response systems) or even a mixture of both. Systematic information on the mode of assessment (MOA) and data management of PROMs in clinical trials is missing to inform the ongoing discussions about how PROMs should or should not be used in different scenarios (eg, ⁹,¹⁴).

Information about data management and the MOA of PROMs are recommended reporting items in guidelines for trial protocols (Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) and Guidelines for Inclusion of Patient-Reported Outcomes in Clinical Trial Protocols (SPIRIT-PRO))[15,16,^{15 16} as well as trial publications (Consolidated Standards of Reporting Trials (CONSORT) and Reporting of patient-reported outcomes in randomised trials: the CONSORT-PRO extension^{17 18}). However, such information is often not sufficiently reported in protocols and publications, and there is ample evidence demonstrating a slow uptake of conformity with reporting guidelines in the literature in certain areas.^{19 20}

The goal of this scoping review is to develop a more comprehensive picture of the MOA and management of data derived from PROs in cancer randomised controlled trials (RCTs) and the reporting quality thereof. Using the scoping review methodology to explore existing literature, we aim to fill the current knowledge gap regarding the prevalence of different practices of PROM assessment in cancer clinical trials. Additionally, we seek to identify areas where adherence to established reporting guidelines, that is, SPIRIT-PRO and CONSORT-PRO, can be improved.

Methods

This paper is a protocol for a scoping review. The protocol is reported in accordance with the Preferred Reporting Items for Systematic reviews and Meta Analyses extension for Scoping Reviews checklist.²¹ Our approach is adapted from the framework originally proposed by Arksey and O'Malley,²² which has received widespread recognition and refinement in the methodological literature.²³,²⁵

A scoping review aims to explore and chart the fundamental concepts within a research field, along with the primary sources and various types of available evidence. Consequently, scoping reviews often investigate overarching research questions, and they do not engage in formal quality assessment. This review aims to gain a comprehensive picture of the MOA of PROMs and PRO data management in cancer RCTs and the reporting thereof. Hence, the methodology of a scoping review is followed to answer our research questions. Due to the nature of the research questions asked (see below), we will not conduct a systematic assessment of trial quality (eg, risk of bias). We argue that the reporting of the MOA of PROMs in trials is unlikely to be linked to a trial’s risk of bias and hence that this kind of assessment is unlikely to be relevant to our scope. However, the literature search approach is still systematic.

Identifying the research questions

Research question 1: What is the frequency of different MOA used to assess PROMs in cancer RCTs?

This review aims to provide an overview of the MOA of PROMs in cancer RCTs in light of existing guidance and regulatory recommendations. For example, the FDA generally recommends electronic source data capture, which includes PROMs, to harmonise and modernise clinical trials.²⁶ Hence, the first research question is descriptive and focuses on the frequency of different MOA for PROMs among trials that report assessing PROMs.

Research question 2: Are there study characteristics that are linked to different MOA?

Second, we aim to evaluate if any specific study characteristics are associated with different types of MOA. For example, multicentre and international studies might be more prone to rely on a centralised ePRO data collection. Similarly, it might be too costly to implement ePRO data collection for academic trials which commonly have less funding compared with industry trials.

Research question 3: To what level does the reporting of data management and MOA conform to the CONSORT and SPIRIT recommendations?

Third, we aim to assess the reporting conformity of publications and protocols to the relevant EQUATOR guidelines in terms of data management and the MOA. For protocols, these are the SPIRIT guideline¹⁵ and the SPIRIT-PRO extension.¹⁶ For publications, these are the CONSORT guideline¹⁷ and the CONSORT-PRO extension.¹⁸ The relevant items are listed in the data extraction sheet in online supplemental material 1.

Research question 4: Do any trials report using PRO-based alerts during the trial to support patients’ clinical care?

Finally, we aim to identify whether any trials using electronic PROM assessment used the PRO data during the trial. In other words, whether PROM data were actively reviewed by healthcare professionals involved in the treatment of the study patient, for example, using alerts. There are good (ethical) arguments for not only using PRO data for analyses after the trial but for also monitoring PRO data during trials to maximise patient safety and well-being.^{27 28} However, we estimate that the number of trials using these advanced levels of PROM-based clinical alerts will be few.

Study selection

We will search for RCTs published between January 2019 and November 2023 (date of protocol inception). Two independent reviewers will decide on the eligibility of references on two subsequent levels (ie, level 1: abstract screening and level 2: full-text screening) using the literature software DistillerSR.²⁹ Inconsistent ratings will be resolved by the two raters. In case of persistent disagreement, a third reviewer will be consulted. The following inclusion and exclusion criteria will be applied to decide on the eligibility of trial publications.

The inclusion criteria were as follows:

• Study design: RCT.

• Overall sample size (randomised patients) >50.

• Cancer population (bladder, breast, colorectal, gynaecological, lung and prostate).

• Biomedical treatment evaluated (eg, chemotherapy, radiotherapy, surgery, targeted therapy, immunotherapy and hormonal therapy), therefore, excluding psychosocial, educational and complementary medicine and exercise and behavioural interventions.

• At least one PRO endpoint (primary, secondary or exploratory) using any European Organization for Research and Treatment of Cancer (EORTC) or Functional Assessment of Chronic Illness Therapy (FACIT) questionnaire.

• Language: English.

The exclusion criteria were as follows:

• Mixed population (ie, no mix of cancer diagnoses within one trial).

• Screening study.

• Survivorship study.

• No full text available.

We focus on the abovementioned six tumour groups as they are the most common solid tumour types per incidence.³⁰ Further, we only include studies with a PRO endpoint using either an EORTC or a FACIT questionnaire, as they are, by far, the two most common questionnaires used in cancer clinical trials.³¹ The search strategy and search strings were adapted from the PROMOTION registry (https://promotion.gimema.it/, ³²); see online supplemental material 2 for the complete search string.

Data sources

Relevant trials will be identified via their publications. As outlined in the Study selection section, we will search for trials with a PRO endpoint. To keep the overall workload manageable, only trial publications indexed in PubMed will be considered. PubMed coverage of studies for lung, colorectal and gynaecological cancers is over 80% and even at 90% for breast cancer.³³

Detailed information on the MOA and data management is unlikely to be frequently found in trial publications only (eg, due to word count limitations). Moreover, the authors may sometimes only report that ‘a questionnaire was assessed’ but not report the MOA as paper-based assessment is often perceived as the norm. Trial protocols might be more explicit in describing the MOA and data management. Hence, we will undertake reasonable efforts to source trial protocols. For each trial that is included in the analysis, we check the following sources to trace a protocol:

1. Searching Clinicaltrials.gov and EduraCT for an uploaded protocol.

2. If not obtained in the previous step: simple web search of the trial using the trial acronym and the registration number where available (eg, Open Science Framework and other trial databases).

3. If not obtained in the previous step: contacting the corresponding author of the trial’s primary publication (once).

Extracting and charting the data

We will extract data from both the publications from a trial, as well as the protocol if a protocol can be traced, again applying a double-review procedure in DistillerSR.²⁹ Where feasible and of relevance, we will also distinguish between the origin of the data (from publication(s) or protocol). The table in online supplemental material 1 gives an overview of the information that will be assessed at the trial level.

Trial protocols will be charted according to relevant items regarding data collection and management from the SPIRIT checklist and the SPIRIT-PRO extension. To evaluate the quality of reporting with reference to the CONSORT and CONSORT-PRO, we will jointly refer to all publications from a single trial. If there are multiple publications from a single trial, we will not evaluate CONSORT conformity for each publication but rather consider the sum of all information extracted from all publications. While this approach may slightly overestimate CONSORT conformity for trials that have a lot of different publications, we argue that this approach is more true to the nature of the research question (ie, not all publications from a trial will focus on PROs).

Summarising and reporting the results

As outlined before, we will compare data on a trial level based on the information from all publications and the protocol of a single trial. The collected information will then be compared with regard to the research questions. Specifically, we will focus on a descriptive analysis of the MOA of PROMs (ie, how are PROMs assessed in trials that have information on their MOA) and on the relative frequencies of the different MOA and associated trial characteristics (ie, research questions 1 and 2). We will also evaluate the quality of reporting for the relevant SPIRIT and CONSORT guidelines that refer to the assessment of PROMs in trials (research question 3; see online supplemental material 1, table 1). Generally, we expect that the detailed reporting of the MOA will be the exception, rather than the norm, even in published trial protocols. Considering that the CONSORT and CONSORT-PRO guidelines were published in 2010 and 2013, respectively,^{16 17} this might indicate a lack of uptake of the MOA-related items or a perceived lack of importance by researchers. However, research shows that differences between MOA should be carefully considered, especially when mixing or comparing different MOA in the same trial.^{3 9 11} Hence, it would be important to increase awareness of (1) a potential lack of sufficient MOA reporting and (2) the challenges and benefits associated with different MOA in trials. Moreover, we will explore whether any trials report on using ePRO-based alerts during the trial (research question 4).

Ethics and dissemination of results

No ethics approval was sought for this study as the study is based on published literature only. This review is part of an ongoing EORTC project on the use of electronic data capture methods in cancer clinical trials. The findings from the review will be reviewed and discussed by a multidisciplinary expert committee to synthesise guidance to ultimately improve the (electronic) measurement of PROMs in clinical trials. Input from different stakeholders involved in the setup and use of ePRO solutions for trials will be sought to get further insight into the data.

Finally, the findings from the review will be presented at international meetings and published in a peer-reviewed journal following the PRIMSA-ScR guidelines.²¹

Patient and public involvement

Patients or the public were not involved in the design, conduct or reporting of this protocol. Patient (and other trial stakeholders) will however be involved in expert groups discussing the review’s findings and following works to draft recommendations around electronic data capture.

Limitations

First, a potential limitation is that our search strategy only includes trials with publications that include mentioning of PROs (or relevant synonyms). Possibly, there might be some trials that included a PRO in the protocol only but did not include any mention of PROs in any of the trial’s publications, and we would not find those trials with our search strategy. While a systematic review of trial protocols and publications between 2014 and 2017 found that about 38% of trials did not publish their PRO results,¹⁹ most trial publications commonly include at least the definitions of primary and secondary endpoints, thus allowing us to include them in our search.

Second, although restricting the search to a single database and studies from 2019 to 2023 helps keep the review workload manageable, it may also cause us to miss some potentially suitable publications.

Finally, focusing on trials that report some information on their PRO MOA might introduce sampling bias by picking trials with available information. However, it is unclear in what direction this may introduce bias (ie, in the direction of more or less electronic data capture).