Patient-reported outcomes in randomised controlled trials of colorectal cancer: an analysis determining the availability of robust data to inform clinical decision-making

Abstract

Purpose

Randomised controlled trials (RCTs) are the most robust study design measuring outcomes of colorectal cancer (CRC) treatments, but to influence clinical practice trial design and reporting of patient-reported outcomes (PROs) must be of high quality. Objectives of this study were as follows: to examine the quality of PRO reporting in RCTs of CRC treatment; to assess the availability of robust data to inform clinical decision-making; and to investigate whether quality of reporting improved over time.

Methods

A systematic review from January 2004–February 2012 identified RCTs of CRC treatment describing PROs. Relevant abstracts were screened and manuscripts obtained. Methodological quality was assessed using International Society for Quality of Life Research—patient-reported outcome reporting standards. Changes in reporting quality over time were established by comparison with previous data, and risk of bias was assessed with the Cochrane risk of bias tool.

Results

Sixty-six RCTs were identified, seven studies (10 %) reported survival benefit favouring the experimental treatment, 35 trials (53 %) identified differences in PROs between treatment groups, and the clinical significance of these differences was discussed in 19 studies (29 %). The most commonly reported treatment type was chemotherapy (n = 45; 68 %). Improvements over time in key methodological issues including the documentation of missing data and the discussion of the clinical significance of PROs were found. Thirteen trials (20 %) had high-quality reporting.

Conclusions

Whilst improvements in PRO quality reporting over time were found, several recent studies still fail to robustly inform clinical practice. Quality of PRO reporting must continue to improve to maximise the clinical impact of PRO findings.

Introduction

Well-designed and conducted randomised controlled trials (RCTs) can provide the highest quality evidence for novel cancer therapies (Sibbald and Roland 1998). Traditional outcomes include overall or progression-free survival, complications such as infection and side effects of chemotherapy or radiotherapy. In recent years, trials have incorporated the patients’ perspective describing patient-reported outcomes (PROs). A number of guidelines for reporting PROs in RCTs have been proposed in the last two decades (Efficace et al. 2003; Lee and Chi 2000; Staquet et al. 1996), and more recently, these recommendations have been updated by the International Society for Quality of Life Research (ISOQoL) and incorporated as a PRO extension to the Consolidated Standards of Reporting Trials (CONSORT) statement. Despite this guidance, evidence suggests that clinicians find interpreting PROs difficult (Greenhalgh 2009). Many RCTs fail to explain the relevance of PROs to the overall trial findings and often do not incorporate PROs into subsequent treatment recommendations (Cocks et al. 2008). If trials are to influence clinical care, they must be of high quality, consistently report both clinical and PROs, and base treatment recommendations on all relevant outcomes. This is particularly important when treatments may have different impacts on clinical and PROs, for worsening global quality of life (QoL) and symptoms but improved long-term survival. Fully understanding the overall effect of a novel therapy on both patient-reported and clinical outcomes will allow the clinician and patient to consider differing treatment effects and make decisions that trade off survival, symptoms, functional abilities and QoL during shared decision-making.

Colorectal cancer is common worldwide (Ferlay et al. 2010; UK 2009) and although survival rates of 60 % at 5 years are often achieved using a combination of surgery, pre-operative radiotherapy and post-operative chemotherapy (Coleman et al. 2011), functional deficits and severe symptoms due to these treatments are common. Problems related to bowel habit and sexual/urinary dysfunction are often reported in rectal cancer (Gujral et al. 2008). Fatigue, pain and reduced activity are common (Denlinger and Barsevick 2009; Thong et al. 2013), and problems with adaption to life with a stoma are frequently described (Peeters et al. 2005). The consequences of this disease and its subsequent treatment will impact upon a large number of survivors, and patients should be provided with the highest quality and most accurate information regarding optimal treatments and their effects on clinical and PROs. Information about the detrimental effects of treatment on functional abilities, QOL and symptoms will help inform patients about the anticipated effects of treatment and help inform decision-making. However, current evidence synthesis of RCTs describing colorectal cancer treatments that include both clinical and PROs is limited (de Kort et al. 2006; Gujral et al. 2008). In a previous systematic review of colorectal PRO–RCTs studies published between 1980 and 2003, a number methodological limitations were highlighted, which hampered critical appraisal of the PRO findings (Efficace et al. 2004).

The objective of this systematic review is to identify RCTs in colorectal cancer that report clinical and PROs and determine the quality of PRO reporting. Secondary objectives are to document the characteristics of the included studies, document the PRO tools used, record the effect of the experimental treatment on both clinical and PROs assess whether the reporting of key methodological aspects of PROs has improved over time and describe those studies most likely to robustly inform clinical decision-making.

Methods

Identification of studies

Systematic literature searches were undertaken between January 2004 and February 2012 using the OVID Gateway™ (http://ovidsp.ovid.com/) to access Medline, CINHAL, PsychInfo and the Cochrane Controlled Trials Register using an optimised highly sensitive search strategy. Inclusion criteria were as follows: RCTs enrolling at least 50 patients in total, with participants in the RCTs aged 18 years or older; diagnosed and treated for colorectal cancer (studies on anal cancer also considered eligible); treated with surgery, chemotherapy, radiotherapy or biological treatments; assessments including PROs, either as a primary or secondary endpoint. Only RCTs enrolling at least 50 patients, overall, were included as those with smaller samples were considered too limited to provide adequately powered PRO analyses.

If a second publication, related to the main RCT, highlighting additional details regarding PROs was identified, then the full text of this additional manuscript was also obtained. Studies reporting complementary, alternative or psychological interventions and those describing proxy-based questionnaires (i.e. completed by clinicians or other observers) were excluded. The results were reported according to the PRISMA guidelines (Liberati et al. 2009).

Relevant data were independently extracted by two reviewers to a standardised data extraction e-form hosted on a secure server with a password-protected database (http://promotion.gimema.it/). Supporting or additional publications directly relevant to the included RCTs were also examined. In face-to-face meetings when disagreement between reviewers occurred, the relevant trial manuscript was reviewed jointly by the two reviewers (VC and KW) and differences reconciled until consensus on the extracted data was achieved. If consensus could not be achieved, then a third-independent reviewer was consulted (FE).

Type of data collection

Details of the study design, the number of participants and their demographics, whether the study was performed by a co-operative group and if industry had a role in any funding were recorded. Information describing the disease stage “curable/local” or “advanced/metastatic” was noted. Details of the interventions or treatments employed in the control and experimental arm of the RCT were recorded, and information describing each outcome assessed in the trial was entered into the database including details of which outcome constituted the primary endpoint of the trial. When an endpoint was patient-reported, details of the PRO tool used to make the assessment were noted. The effect of the experimental intervention on PROs was recorded, and this was classified as an effect on symptoms alone, domains other than symptoms (functional scales and global quality of life) or both symptoms and other domains.

Quality of reporting of PROs and assessment of bias

Based on previous work (Efficace et al. 2013), the quality of PRO reporting was assessed by comparison with the criteria of the ISOQoL reporting standards (Brundage et al. 2012). For studies where PROs were the primary outcome, 29 criteria of the reporting standards were applicable, whilst only 18 criteria apply to trials where PROs were a secondary outcome (Brundage et al. 2012). A score of 1 was allocated to a study if a relevant reporting standard was met and 0 if it was not met. Therefore, in studies where PROs were a primary endpoint, a maximum score of 29 points could be achieved, whilst a reduced maximum score of 18 points was possible for trials where PROs were a secondary endpoint. Studies were considered to have “high-quality PRO reporting”, based on previously defined criteria (Efficace et al. 2013). Study bias was assessed using the Cochrane risk of bias tool specifically assessing adequacy of trial sequence generation, allocation concealment, blinding of trial patients and staff, blinding of outcome assessment, incomplete outcome data, selective outcome reporting and any other potential sources of bias (Higgins and Green 2011).

Assessment of changes in PRO reporting quality over time

Differences over time in the frequency of reporting of key selected PRO quality criteria (a priori hypothesis stated, rationale for the instrument reported, method of instrument administration reported, timing of the PRO assessments documented, missing data documented and clinical significance addressed) across the studies were determined by comparison with previously published data(Efficace et al. 2004).

Analysis of outcomes reporting, availability of robust data to inform decision-making and summary of studies with high-quality PRO reporting

For trials with “high-quality PRO reporting”, details of the main clinical outcomes and PROs were recorded, the results of the trials summarised and the data available to inform decision-making described. Trials with the greatest generalisability and those with the lowest risk of bias were further highlighted.

Data analysis

Findings are reported using descriptive statistics. It was specified that PROs from included studies would be synthesised in a meta-analysis using a random-effects model (Hedges and Olkin 1985) only if sufficient studies using consistent PRO measurements were identified. All analyses were performed with SAS version 9.2 (SAS Institute Inc., Cary, NC, USA).

Results

Identification of studies

Four thousand and nine hundred and seventy-eight abstracts were screened, 292 full-text manuscripts were retrieved and 66 RCTs were identified (Fig. 1), enrolling overall 36,367 patients. In addition, a further 63 manuscripts providing additional information regarding the included RCTs were obtained.

Fig. 1.

Schematic breakdown of literature search results of colorectal randomised controlled trials (preferred reporting items for systematic reviews and meta-analysis). PRO patient-reported outcomes

Study characteristics and PRO measures

Details of study characteristics are reported in Table 1. PROs were primary endpoints in 12/66 (18 %) studies of which three (25 %) were at least partially supported by industry. Overall, 33 (50 %) studies involved patients with advanced or metastatic disease, 22 (33 %) with localised disease and 11 (17 %) studies included patients both localised and advanced disease. The most commonly reported treatment type was chemotherapy (n = 45; 68 %) with smaller numbers of trials describing outcomes after surgery (n = 19; 29 %), targeted therapy (n = 12; 18 %) or radiotherapy (n = 9; 14 %). A number of studies included two or more treatment modalities. PROs were most commonly assessed with the EORTC QLQ-C30 questionnaire (n = 31; 47 %) used as a stand-alone measure or in conjunction with other PRO tools. Twenty-nine studies (44 %) used more than one PRO measure.

Table 1.

RCT demographic characteristics

Variable	PRO endpoint n (%)		Total n (%)
	Primary 12 (18 %)	Secondary 54 (82 %)
Basic RCT demographics
Industry supported (fully or in part)*
No	9 (75)	25 (46)	34 (52)
Yes	3 (25)	29 (54)	32 (48)
Overall study sample size (regardless of patients included in the PRO analysis)
≤200 patients	8 (67)	14 (26)	22 (33)
>200 patients	4 (33)	40 (74)	44 (67)
Disease stage
Advanced/metastatic	2 (17)	31 (57)	33 (50)
Loco-regional/no distant metastasis	6 (50)	16 (30)	22 (33)
Mixed disease stages	4 (33)	7 (13)	11 (17)
Broad treatment type
Radiotherapy	1 (8)	8 (15)	9 (14)
Surgery	7 (58)	12 (22)	19 (29)
Chemotherapy	2 (17)	43 (80)	45 (68)
Target therapy	0 (0)	12 (22)	12 (18)
Difference between treatment arms in the primary endpoint
No	4 (33)	29 (54)	33 (50)
Yes	8 (67)	25 (46)	33 (50)
Overall survival (OS) difference favouring experimental treatment
No	5 (42)	37 (68)	42 (63)
Yes	0 (0)	7 (13)	7 (11)
N/A (in case OS was not assessed)	7 (58)	10 (19)	17 (26)
PRO-related basic characteristics
PRO instrument used
EORTC Instruments	3 (26)	28 (52)	31 (47)
FACT Instruments	1 (8)	7 (13)	8 (12)
VAS	4 (33)	7 (13)	11 (17)
Others	4 (33)	12 (22)	16 (24)
PRO difference between treatment arms
No differences at all	4 (33)	26 (48)	30 (45)
Yes broadly favouring experimental treatment†	8 (67)	21 (39)	29 (44)
Yes broadly favouring standard treatment†	0 (0)	5 (9)	5 (8)
N/A	0 (0)	2 (4)	2 (3)
If statistically significant PRO difference exists, in which domain?
Symptoms only	4 (50)	8 (31)	12 (35)
PRO domains other than symptoms only (e.g. functional scales and/or global QoL)	3 (38)	6 (23)	9 (26)
Both domains (symptoms + domains other than symptoms)	1 (12)	12 (46)	13 (39)
Length of PRO assessment during RCT
Up to 6 months	7 (58)	20 (37)	27 (40)
Up to 1 year	3 (25)	8 (15)	11 (17)
More than 1 year	2 (17)	25 (46)	27 (41)
Unknown	0 (0)	1 (2)	1 (2)

^*Assessed if explicitly stated or if one or more authors were affiliated to a pharmaceutical company. This evaluation is based solely on information extracted from the paper

^†Often, multiple PRO domains (e.g. from multidimensional HRQOL questionnaires) are analysed at the same time in longitudinal PRO–RCTs; to illustrate, difference in such domains might favour the experimental treatment arm at a given time point and then favouring the control treatment arm at a different time point over the course of the study. So, the term “broadly” was inserted to account for this possible discrepancy

Effects of experimental intervention on clinical and patient-reported outcomes

Half of the included studies (n = 33; 50 %) showed a difference (in any direction) in the primary endpoint between the standard treatment and the experimental treatment groups, and seven studies (11 %) reported overall survival (OS) benefit favouring the experimental treatment. Forty-two of the 49 studies that reported OS did not find difference between treatment groups. However, 20 (48 %) of those 42 did show a statistical difference in PROs between groups. Overall, a difference in PROs between treatment arms was found in 34 studies (52 %), with the majority of these studies (n = 29/34; 85 %) describing PROs that favoured the experimental treatment (Table 1).

Quality of reporting of PROs and assessment of bias

Table 2 documents the extent to which the included studies adequately reported PROs. Overall, only 20 studies (30 %) stated the method of PRO administration, and most studies did not describe the rationale for choosing the PRO instrument used (n = 51, 77 %). Of the 12 studies where PROs were the primary outcome, three (25 %) failed to show evidence of using validated PRO instruments, and none of these 12 RCTs provided information on windows for valid PRO responses. In contrast, the validity and reliability of the PRO instrument used were reported in 70 % of studies (n = 46) using PRO as secondary endpoint. We note that in case of studies using multiple PRO measures, these were still rated as “using PRO validated measures” if at least one of the PRO measures used was validated. In addition, most studies (n = 59; 89 %) gave details of the time points at which PROs were collected and just over half the total number of studies (n = 37; 56 %) described the extent of missing data.

Table 2.

Level of Patient-Reported Outcomes (PRO) reporting by type of endpoint (PRO primary versus secondary endpoint of the trial)

	PRO endpoint n (%)		Total n (%)
	Primary	Secondary
Title and abstract
The PRO should be identified as an outcome in the abstract
No	0 (0)	11 (20)	11 (17)
Yes	12 (100)	43 (80)	55 (83)
The title of the paper should be explicit as to the RCT including a PRO
No	8 (67)	–	8 (67)
Yes	4 (33)	–	4 (33)
Introduction, background and objectives
The PRO hypothesis should be stated and should specify the relevant PRO domain if applicable
No	7 (58)	38 (70)	45 (68)
Yes	5 (42)	13 (24)	18 (27)
N/A (if explorative)	0 (0)	3 (6)	3 (5)
The introduction should contain a summary of PRO research that is relevant to the RCT
No	7 (58)	–	7 (58)
Yes	5 (42)	–	5 (42)
Additional details regarding the hypothesis should be provided, including the rationale for the selected domain(s), the expected direction(s) of change and the time points for assessment
No	9 (75)	–	9 (75)
Yes	3 (25)	–	3 (25)
Methods
Outcomes
The mode of administration of the PRO tool and the methods of collecting data should be described
No	6 (50)	40 (74)	46 (70)
Yes	6 (50)	14 (26)	20 (30)
Electronic mode of PRO administration*
No	6 (50)	14 (26)	20 (30)
N/A	6 (50)	40 (74)	46 (70)
The rationale for choice of the PRO instrument used should be provided
No	10 (83)	41 (76)	51 (77)
Yes	2 (17)	13 (24)	15 (23)
Evidence of PRO instrument validity and reliability should be provided or cited
No†	3 (25)	17 (31)	20 (30)
Yes	9 (75)	37 (69)	46 (70)
The intended PRO data collection schedule should be provided
No	1 (8)	6 (11)	7 (11)
Yes	11 (92)	48 (89)	59 (89)
PROs should be identified in the trial protocol post hoc analyses should be identified
No	10 (83)	46 (85)	56 (85)
Yes	2 (17)	8 (15)	10 (15)
Methods
The status of PRO as either a primary or secondary outcome should be stated
No	0 (0)	8 (15)	8 (12)
Yes	9 (75)	40 (74)	49 (74)
Unclear	3 (25)	6 (11)	9 (14)
A citation for the original development of the PRO instrument should be provided
No†	5 (42)	–	5 (42)
Yes	7 (58)	–	7 (58)
Windows for valid PRO responses should be specified and justified as being appropriate for the clinical context
No	12 (100)	–	12 (100)
Sample size
There should be a power sample size calculation relevant to the PRO based on a clinical rationale
No	4 (33)	–	4 (33)
Yes	8 (67)	–	8 (67)
Statistical methods
There should be evidence of appropriate statistical analysis and tests of statistical significance for each PRO hypothesis tested
No	0 (0)	1 (2)	1 (2)
Yes	5 (42)	12 (22)	17 (26)
N/A (If PRO hypotheses were not stated)	7 (58)	41 (76)	48 (72)
Statistical approaches for dealing with missing data should be explicitly stated, and the extent of missing data should be stated‡
No	6 (50)	23 (43)	29 (44)
Yes	6 (50)	31 (57)	37 (56)
The manner in which multiple comparisons have been addressed should be provided
No	7 (58)	–	7 (58)
Yes	5 (42)	–	5 (42)
Results
Participant flow
A flow diagram or a description of the allocation of participants and those lost to follow-up should be provided for PROs specifically
No	7 (58)	37 (69)	44 (67)
Yes	5 (42)	17 (31)	22 (33)
The reasons for missing data should be explained
No	10 (83)	44 (81)	54 (82)
Yes	2 (17)	10 (19)	12 (18)
Baseline data
The study patients characteristics should be described including baseline PRO scores
No	8 (67)	33 (61)	41 (62)
Yes	4 (33)	21 (39)	25 (38)
Outcomes and estimation
Are PRO outcomes also reported in a graphical format?*
No	4 (33)	32 (59)	36 (55)
Yes	8 (67)	22 (41)	30 (45)
The analysis of PRO data should account for survival differences between treatment groups if relevant
N/A (if not relevant)	12 (100)	–	12 (100)
Results should be reported for all PRO domains(if multidimensional)and items identified by the reference instrument
No	6 (50)	–	6 (50)
Yes	6 (50)	–	6 (50)
The proportion of patients achieving pre-defined responder definitions should be provided where relevant
N/A (if not relevant)	12 (100)	–	12 (100)
Discussion
Limitations
The limitations of the PRO components of the trial should be explicitly discussed
No	8 (67)	27 (50)	35 (53)
Yes	4 (33)	27 (50)	31 (47)
Generalisability
Generalisability issues uniquely related to the PRO results should be discussed
No	4 (33)	30 (56)	34 (52)
Yes	8 (67)	24 (44)	32 (48)
Interpretation
Are PRO interpreted? (not only re-stated)*
No	1 (8)	19 (35)	20 (30)
Yes	11 (92)	35 (65)	46 (70)
The clinical significance of the PRO findings should be discussed
No	11 (92)	36 (67)	47 (71)
Yes	1 (8)	18 (33)	19 (29)
Methodology used to assess clinical significance (in case this was addressed)*
Anchor based	0 (0)	5 (28)	5 (26)
Distribution based	1 (100)	10 (55)	11 (58)
Both	0 (0)	3 (17)	3 (16)
The PRO results should be discussed in the context of the other clinical trial outcomes
No	3 (25)	21 (39)	24 (36)
Yes	9 (75)	33 (61)	42 (64)
Other information
Protocol
A copy of the instrument should be included if it has not been published previously (It could be found in the article appendix or in the online version
No	5 (42)	–	5 (42)
Yes	7 (58)	–	7 (58)

Rating of items was independent of location of the information within the manuscript

N/A not applicable

– Indicates items that are not applicable as these are recommended to be reported only when PRO is a primary endpoint

^*The following items have not been included in the ISOQOL PRO reporting standards but have been added in this table for a more comprehensive evaluation of methodological aspects

^†We evaluated as “no” if all PRO measures used in the RCT were not validated

^‡This was rated as “yes” if at least extent of missing data was documented

A summary of the risk of bias for each RCT, categorised by high- or low-quality PRO reporting quality, is shown in Fig. 2. There was no apparent evidence of an association between quality PRO reporting and risk of bias.

Fig. 2.

Risk bar chart showing proportion of studies with low, unclear or high risk of bias across all RCTs (n = 66) by quality of PRO reporting

Assessment of changes in PRO reporting quality over time

PRO reporting improved over time from the period 1980–2003 to the period 2004–2012. Details of PRO instrument administration were only documented in 10 % of studies published up to 2003 whilst this percentage increased to 30 % for included RCTs published between 2004 and 2012. Similarly, clinical significance of PRO findings was found in 13 % of RCTs published up to 2003 but increased to 29 % in the present study (Fig. 3).

Fig. 3.

Descriptive comparison of level of reporting on selected key PRO issues in RCTs of colorectal describe both clinical and patient-reported outcomes between 1980 and 2003 (yellow bars) (Efficace et al. 2003) and 2004 and 2012 (blue bars)

Analysis of outcomes reporting, availability of robust data to inform decision-making and summary of studies with high-quality PRO reporting

Thirteen of the 66 studies identified were “high quality” in terms of PRO reporting meeting the pre-defined criteria (Supplementary table 1), of which two (Stephens et al. 2010; Stucky et al. 2011) had consistently low risk of bias in all domains and provide influential information that will likely support clinical decision-making. Five trials were conducted in patients with advanced/metastatic disease, six in patients with loco-regional disease and two reported the outcomes of patients with CRC of any disease stage.

In patients with advanced disease, two studies (Douillard et al. 2010; Peeters et al. 2010) described the effect of adding panitumumab to a chemotherapy regime. This regime was infusional fluorouracil, leucovorin and oxaliplatin (FOLFOX4; N = 1183) (Douillard et al. 2010) and fluorouracil, leucovorin, and irinotecan (FOLFIRI; N = 1186) (Peeters et al. 2010). In both trials, patients in the group receiving panitumumab had longer progression-free survival (1–2 months), but no difference in PROs was noted. A further study described the effects of panitumumab in patients with chemotherapy refractory metastatic colorectal cancer receiving best supportive care. Those patients receiving panitumumab reported improved quality of life if they had normal K-ras genetics but median progression-free survival irrespective of K-ras status was unaltered (Odom et al. 2011). One study, also in patients with chemotherapy refractory metastatic colorectal cancer receiving best supportive care, explored the effect of adding cetuximab. Cetuximab therapy was associated with improved overall survival and improved physical function and global health status, particularly in those with wild-type K-ras gene expression (Jonker et al. 2007). The final RCT was a non-inferiority design, comparing two different modes of administration of adjuvant chemotherapy for CRC; it demonstrated that 5-fluorouracil and capceitabine were non-inferior to capceitabine and oxaliplatin when objective response rate was assessed and quality of life did not differ between the two trial groups (Ducreux et al. 2011).

Six studies described PRO and clinical outcomes in patients with loco-regional disease, of which one RCT reported the effects of different chemotherapy treatments after surgery (Lembersky et al. 2006), two studies compared open with laparoscopic colorectal surgery (Janson et al. 2007; Stucky et al. 2011), two studies described different adjuvant/neo-adjuvant radio-chemotherapy regimens for CRC (Stephens et al. 2010; Tournier-Rangeard et al. 2008) and one study the administration of granulocyte colony-stimulating factor (G-CSF) or placebo to patients after surgery for CRC (Bauhofer et al. 2007).

Two of the RCTs in patients with loco-regional disease are of particular interest as they had both high-quality PRO reporting and also consistently low risk of bias (Stephens et al. 2010; Stucky et al. 2011). One described the outcomes of short-course pre-operative radiotherapy versus surgery and selective post-operative chemotherapy on QoL in patients with rectal cancer (Stephens et al. 2010) using the SF-36 and the EORTC QLQ-C38 questionnaire. A 61 % reduction in the relative risk of local recurrence with preoperative radiotherapy (an absolute difference in recurrence risk at 3 years of 6.2 %) was described, and no differences in sexual problems observed between groups. Although a good quality trial, it was noted that the limitations of the PRO instruments employed were not discussed and recommendations regarding the generalisability of these results to the wider rectal cancer population were not made. The other high-quality trial identified evaluated QoL in patients having open versus laparoscopic colonic surgery for the treatment of colorectal cancer (Stucky et al. 2011). This study showed that recurrence of cancer and overall survival was the same in both groups, the duration of post-operative analgesia and length hospital stay was shorter in those undergoing laparoscopic surgery and at 18 months post-surgery, but patients who had undergone laparoscopic-assisted colectomy had significantly greater improvements in global QoL, measures of daily living and health compared to the open surgery group. The largest trial (Lembersky et al. 2006) with high-quality reporting recruited patients with loco-regional CRC and compared adjuvant leucovorin combined with either oral uracil and tegafur or intravenous fluorouracil. Both overall survival and disease-free survival were similar in each group; in addition, no differences in either FACT-C overall scores or subscales were identified. However, a statistically significant but clinically small difference in scores for the SF-36 questionnaire was identified.

Two studies recruited patients with any disease stage. One study compared laparoscopic versus open surgery for colorectal tumours. Clinical outcomes in each study group were the same, but men having laparoscopic rectal surgery described worse sexual and erectile functions (Jayne et al. 2005). The other study recruiting patients with all disease stages compared early surgery for malignant descending colon obstruction against stent placement followed by late surgery (van Hooft et al. 2011). There were no differences in the clinical outcomes or global quality of life.

Discussion

This study identified 66 RCTs enrolling some 36,000 patients from 2004 to 2012 reporting clinical and PROs in colorectal cancer of which thirteen studies (20 %) were considered to have high-quality reporting of key methodological features of PRO design. Reporting of PRO outcomes is rare in general (only 66 studies over 10 years), and high-quality reporting methods are uncommon even among studies measuring PROs. These findings suggest that high-quality RCTs which describe clinical and- patient-reported outcomes are uncommon and evidence to support patient-centred decision-making about the optimal treatment for colorectal cancer is lacking.

The methodological limitations identified by this work reviewing colorectal cancer trials have also been noted by studies describing PROs after treatment of upper gastrointestinal and breast cancer, lung malignancy and in reviews describing treatment of multiple of cancer types (Claassens et al. 2011; Macefield et al. 2013; Ward et al. 2012; Whistance et al. 2012; Williamson et al. 2012). These findings of poor methodological quality have also been described in more recent work, highlighting PRO reporting in RCTs of prostate cancer treatment (Efficace et al. 2013).

This study also noted that a large number of different instruments were used to assess PROs, although the majority 60 % (n = 39) of studies used EORTC or FACT questionnaires, significant heterogeneity still existed making study comparison difficult and impairing meta-analysis. The issue of heterogeneity in outcome reporting has been highlighted in a number of reviews describing outcomes after breast reconstruction(Ward et al. 2012), bariatric surgery (Coulman et al. 2013), treatments for oesophageal cancer (Main et al. 2014) and colorectal cancer (Whistance et al. 2013). Consensus on the most relevant outcomes for patients and clinicians is necessary, and core outcome sets (COS) may provide a potential solution to heterogeneity of outcome reporting. A COS an agreed minimum set of outcomes should be measured and reported in all clinical trials of a specific condition, and these sets are suitable for both clinical audit and research in addition to randomised trials (Williamson et al. 2012). They promote uniformity in outcome reporting and facilitate systematic review and meta-analysis. The development of core outcome sets is promoted by Core Outcome Measures in Effectiveness Trials (COMET; http://www.comet-initiative.org/), which supports the application of agreed standardised core outcome sets, and recent work has highlighted methods that select and integrate PROs into core outcome sets (Macefield et al. 2014).

This review also identified studies with poor design, which also introduces the risk of bias, and it is recommended that recent SPIRIT guidance be used (Chan et al. 2013) when preparing the trial protocols to avoid poor design. However, it must be noted that the current SPIRIT statement does not specifically incorporate guidance regarding the design of trials that include PROs, and it is further recommended that guidelines for PRO use should be incorporated into revisions of this statement. In addition, trials should be reported according to CONSORT guidance (Moher et al. 2010) including its extension defining standards PRO reporting within RCTs (Calvert et al. 2013).

This review showed that the important improvements in key PRO methodological aspects have occurred over time. For example, whilst clinical significance of PRO results was only addressed in 13 % of studies published up to 2003, this has increased to almost a third in the current review. This suggests that whilst the current level of reporting is still suboptimal, improvements are occurring and should continue, a finding consistent with a recent review of PRO use in prostate cancer RCTs (Efficace et al. 2013). However, limitations persist in a number of areas of PRO reporting where current compliance to recommendations is poor. Specifically, future studies should pay particular attention to ensure RCTs define a schedule for PRO data collection, indicate an acceptable time window for questionnaire completion at each point in the study, document the reasons for missing data and particularly highlight the clinical significance of PRO data.

In contrast to RCTs in prostate cancer that report PROs (Efficace et al. 2013), the proportion of CRC trials with low risk of bias did not appear to be associated with the quality of PRO reporting. Approximately half the studies in CRC treatment with good quality PRO were at high risk or unclear of bias, whilst 10–30 % of prostate cancer trials with high-quality PRO reporting were associated with high risk or unclear of bias (Efficace et al. 2013) across most domains. The exceptions were reporting bias (selective reporting) where the twice as many RCTs with good quality PRO reporting (36 %) were at low risk of bias when compared to studies with lower-quality PRO reporting, and selection bias (allocation concealment) where low risk of bias was more common (51 %) in studies with low-quality PRO reporting. The association between high-quality reporting of PROs and a lower risk of selection bias may reflect improvements in trials practice and increased awareness of the issues relevant to trial design; however, the reasons for the association between lower risk of selection bias and lower-quality PRO reporting are unclear. A further challenge facing open-label studies, particularly those in surgery, is the risk of performance bias because of the difficulty in blinding participants to the intervention they received. Only a third of colorectal cancer RCTs identified in this review were at low risk of performance bias; however, the effect of performance bias on clinical and particularly patient-reported outcomes remains unclear and requires further evaluation.

This study has some limitations. Literature searches are imperfect and do risk omission of studies, and studies in languages other than in English were excluded, although current evidence suggests that this limitation does not significantly alter the conclusions of other systematic reviews (Juni et al. 2002; Moher et al. 2000). Additionally, the definition of high-quality reporting used was exclusively methodological and a strict approach when defining criteria necessary for a study to be classified as having high-quality PRO methodology was adopted. Although this resulted in fewer trials being classified as having high-quality reporting, it was considered critical that high-quality PRO methodology and reporting are promoted to ensure that trial results are robust and inform clinical practice. Strengths of this study included the rigorous and extensive searching of relevant databases, the use of a standardised approach to data validation which maintained data quality, the adoption of the highest quality criteria for assessing PROs and the use of face-to-face meetings when disagreement between reviewers occurred to reconcile differences until consensus on the extracted data was achieved.

To conclude, this review shows that only a fifth (13 studies; 20 %) of RCTs describing treatments for colorectal cancer report PROs with high quality and are likely to robustly inform clinicians regarding the impact of treatments for CRC on PROs. Most studies do not provide sufficient information for clinicians or policy-makers to appraise the results, and whilst the reporting of certain specific criteria (e.g. clinical significance of the PRO findings) has improved with time, overall quality remains poor and risk of bias is frequently high. In addition, current research practice means that the time, effort and expertise of both patients and researchers are wasted and research funds are not optimally used.

Therefore, we recommend that researchers and research funding bodies ensure strict adherence to SPIRIT guidance (Chan et al. 2013) when preparing study protocols, ensure trial design minimises the risk of bias and that the recommendations of the CONSORT guidance (Moher et al. 2010) including the PRO extension (Calvert et al. 2013) are applied by trialists and journal editors when reporting colorectal cancer trials.