Reporting Quality of Endoscopic Colorectal Cancer Screening Randomized Controlled Trials: Adherence to Recommendations and Interventions

Abstract

Background and Aims

In 2013, the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) steering group published recommendations to standardize reporting quality in randomized controlled trials (RCTs). We aim to assess adherence to SPIRIT reporting guidelines in RCTs on endoscopic colorectal cancer (CRC) screening and participants' adherence to trial protocols.

Methods

We searched databases for RCTs evaluating flexible sigmoidoscopy or colonoscopy for CRC screening published in English language through September 2023. Each eligible study was evaluated using the 8 core SPIRIT statement areas, totaling 51 points. Each item received 1 point if it met the criteria and 0 points if it did not. Adherence to SPIRIT items was calculated, and participant adherence to RCT protocols was assessed as the proportion of participants screened compared to those invited.

Results

Five RCTs, including 4 on flexible sigmoidoscopy and 1 on colonoscopy, were analyzed. Adherence to SPIRIT guidance ranged from 82.4% to 92.2%. The most missed recommendation was item 2b (trial registrations), scored 0 across all studies. Additionally, item 32 (informed consent materials) scored 20%, and items 17a & b (blinding) scored 40% each. In total, 587,572 participants were randomized across the 5 RCTs. Of these, 37% (200,610) underwent CRC screening, with 69.8% (139,983/200,610) adhering to the protocol. The Nordic-European Initiative on Colorectal Cancer (NordICC) trial, employing a unique invitation method, had a lower adherence rate of 42%. Excluding this trial would raise the adherence rate to 74.3% (128,050/172,390).

Conclusion

The published CRC screening trials have acceptable adherence to the SPIRIT reporting guidelines. However, reporting appended consent form materials and disclosing all WHO trial registration data can be improved.

Introduction

Randomized controlled trials (RCTs) are prospective studies designed to assess the effectiveness of a new intervention or treatment in terms of health-related outcomes. Randomization, a key component of RCTs, helps minimize bias and establishes a cause-and-effect relationship between the intervention and the desired outcome.¹ RCT protocols play a critical role in ensuring proper reporting and conduct of these trials. These protocols contain vital information such as study objectives, methodology, financial considerations, conflicts of interest, participant remuneration, ethical considerations, and post-trial provisions.² A comprehensive RCT protocol serves as a reference point for comparing publications and ensures accurate and thorough reporting of RCTs. Access to study protocols is vital across all study phases, from subject enrollment to the approval stage.³ Well-crafted protocols improve research quality, enhance research completeness, and increase research transparency.⁴

Inconsistencies among RCTs encompass various aspects such as sample size calculations, statistical analysis methods, and allocation concealment.^5,6 Insufficient reporting can compromise the reliability of scientific research and can increase the risk of misinterpreting study findings.⁴ The consequences of incomplete protocol reporting extend to study participants, investigators, reviewers, and sponsors. When crucial elements are missing from the study protocols, it can significantly impact the results' validity and lead to low-quality studies.

This led to a collaborative effort among researchers worldwide in 2007 and resulted in the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) initiative. This initiative aimed to enhance the quality of trial protocols. The final declaration, published in January 2013, presented a comprehensive 33-item checklist organized into sections covering administrative information, methodology, ethics, and appendices. Accompanying the checklist is an Explanatory and Elaboration Paper.^7,8 The SPIRIT protocol aligns with the ethical principles outlined in the 2008 Declaration of Helsinki and provides valuable guidance for investigators and study reviewers. Additionally, it serves as the foundation for study registration. By offering a standardized structure for clinical trials, the SPIRIT protocol facilitates more accurate and thorough reporting, ultimately improving the quality and outcomes of trials while minimizing bias.^4,8

Flexible sigmoidoscopy and colonoscopy are the 2 recommended endoscopic methods for colorectal cancer (CRC) screening.⁹ These recommendations are supported by evidence from observational studies and RCTs.10, 11, 12, 13, 14 However, the extent to which participants in the intervention arm of these RCTs actually receive the recommended interventions can vary. Adhering to the RCT protocol guidelines is crucial to maintaining high standards throughout the research process. Participant adherence with the RCT protocol enables readers to gain valuable insights into the study results and prevents misinterpretation. In this study, we sought to assess adherence to SPIRIT standardized reporting guidelines in RCTs of endoscopic screening of CRC. Also, we aim to evaluate the adherence of participants to RCTs protocols pertaining to endoscopic screening of CRC.

Methods

This study was conducted according to the Preferred Reporting Items for Systematic reviews and Meta-Analyses statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions¹⁵ (Supplementary Materials 1 and 2).

Literature Review

A thorough literature review was conducted from the inception of the databases to September 5, 2022, using PubMed, Embase, Scopus, and Cochrane. The search aimed to identify all RCTs that evaluated flexible sigmoidoscopy or colonoscopy for CRC screening. Various combinations of keywords such as “randomized controlled trials,” “RCT,” “colorectal cancer screening,” “colonoscopy,” “flexible sigmoidoscopy,” and “CRC screening” were utilized in the search process. Additionally, the reference lists of included studies and previous meta-analyses were hand-searched (backward snowballing) to identify any relevant articles that may have been missed during the initial search. The search was not limited by region or publication type to ensure comprehensive results, although it was restricted to English-language publications. As this study did not involve human subjects, patient consent and institutional review board approval were not required. The literature search was performed independently by 2 authors (KA, FJ) in consultation with an experienced medical librarian.

Eligibility Criteria

The inclusion criteria for this study were as follows: (i) RCTs specifically focused on CRC screening and (ii) CRC screening interventions involving flexible sigmoidoscopy or colonoscopy. On the other hand, studies were excluded if they met any of the following criteria: 1) insufficient information provided regarding the outcomes of interest, 2) study types such as case studies, cohorts, editorials, opinions, letters to the editor, book chapters, animal studies, or meta-analyses, and 3) long-term studies related to the included RCTs.

Data Extraction

Initially, a search was conducted in the selected databases to identify studies that focused on human subjects and evaluated flexible sigmoidoscopy or colonoscopy for CRC screening. The screening process involved the independent assessment of titles and abstracts by 2 investigators, followed by selecting studies that met the predefined inclusion criteria. Subsequently, a thorough evaluation was performed on all selected studies. Data from these studies were extracted and organized into a standardized table for further analysis. To ensure consistency and reliability, 2 researchers (KA, FJ) independently assessed the entire content of each article using predetermined selection criteria and scoring methods. The extracted data included the first author, publication year, country of study, enrollment period, study design, age group, CRC screening method, CRC incidence, CRC mortality, and the number of participants in both arm groups. To accurately report adherence to the SPIRIT guidelines, the study protocol and supplementary materials were carefully reviewed.

Data Synthesis and Statistical Analysis

We utilized the data obtained from the eligible RCTs, their corresponding protocols, and supplementary materials to investigate the adherence to the SPIRIT statement guidelines published in 2013, as well as the participant's adherence to the trial protocols. Statistical analyses were performed using Microsoft Excel, with the data organized and tabulated in spreadsheet format. Adherence to the guidelines and protocol was assessed using proportions (percentages). Each eligible study was evaluated based on the 8 core areas outlined in the SPIRIT statement, encompassing 51 points. One point was assigned to each item that met the specified criteria, while 0 points were assigned if the item did not fulfill the requirements. The proportion of adherence to the individual SPIRIT items was calculated. Two investigators (KA, SA) independently assessed each RCT, with any discrepancies resolved through discussion or consultation with a third investigator (FJ or OH). Furthermore, each study was examined to determine the participant's adherence to the proposed RCT protocols. Adherence to the protocol was defined as the proportion of participants who underwent screening out of the total number of individuals invited to participate in the screening.

Outcomes

Primary outcome was the adherence to SPIRIT standardized reporting guidelines in RCTs. Secondary outcome was the adherence of participants to RCT protocols.

Risk of Bias Assessment

The methodological quality of the included studies was evaluated using the Jadad scale for reporting RCTs.¹⁶ Two investigators (KA and SA) independently conducted the assessment, and any discrepancies were resolved by a third author (FJ). RCTs were assigned points (with a maximum score of 8) based on the modified Jadad scale. In this modified scoring system, studies were deemed of high quality if they achieved a total Jadad score of ≥3 when blinding was possible. For study designs where blinding was not feasible, a score of ≥2 was considered indicative of high quality. Since the total number of RCTs in our analysis was less than 10, we did not generate funnel plot tests as per the Cochrane guidelines. This is because with a small number of studies, the tests may not provide meaningful results to differentiate between chance and genuine asymmetry.¹⁷

Results

Results of Search and Characteristics of Included RCTs

Overall, 14,597 articles were initially identified by our search strategy. Seven thousand five hundred sixty-seven articles were removed after removal of duplicates, and 7030 underwent screening (Figure 1: Preferred Reporting Items for Systematic reviews and Meta-Analyses Flow chart). Finally, total of 5 RCTs were included in this analysis 10, 11, 12, 13, 14 (Figure 1). These RCTs assessed the effectiveness of CRC screening using either flexible sigmoidoscopy or colonoscopy. The trials were conducted in various countries, including Italy, United States, Norway, and some were multicountry trials encompassing England, Wales, Scotland, Poland, Sweden, and the Netherlands. The age range of participants in the included RCTs was between 55 and 64 years, with the exception of the study by Schoen et al,¹² which had an age range of 55–74 years. The primary outcomes of interest in these RCTs were CRC incidence, CRC mortality, or both. Additional details regarding the baseline characteristics of the included RCTs can be found in Table 1.

Figure 1.

Preferred Reporting Items for Systematic reviews and Meta-Analyses flow chart for search criteria.

Table 1.

Baseline Characteristics of the Included Studies

SPIRIT domain/RCT	Bretthauer 2022 (NordICC)	Atkin 2017 (UKFSST)	Holmes 2014 (NORCCAP)	Schoen 2012 (PLCO)	Segnan 2011 (SCORE)
Country	Poland, Norway, Sweden, and Netherlands	England, Wales, and Scotland	Norway	United States	Italy
Study period	2009–2014	1994–1999	1998–2011	1993–2001	1995–1999
Age group, y	55–64	55–64	50–64	55–74	55–64
Intervention	Colonoscopy	Flexible sigmoidoscopy	Flexible sigmoidoscopy or combination of flexible sigmoidoscopy and FOBT	Flexible sigmoidoscopy	Flexible sigmoidoscopy
Total people randomized	84,585	170,432	100,210	154,900	236,568
Total people invited to screening	28,220	57,237	20,572	77,445	17,136
Adherent (underwent screening), n (%)	11,843 (42%)	40,621 (71%)	12,955 (63%)	64,653 (83.5%)	9911 (57.8%)
Nonadherent (did not undergo screening), n (%)	16,377 (58%)	16,616 (29%)	7617 (37%)	12,792 (16.5%)	7225 (42.2%)
Control (nonscreening) group	56,365	11,2939	78,220	77,455	17,136
Primary outcome	Risks of colorectal cancer and related death	Incidence of colorectal cancer, including prevalent cases detected at screening, and mortality from colorectal cancer	Colorectal cancer incidence and mortality	Death from colorectal cancer	CRC incidence and CRC-specific mortality
Final RCT conclusion	The risk of colorectal cancer at 10 y was lower among participants who were invited to undergo screening colonoscopy than among those who were assigned to no screening.	Flexible sigmoidoscopy is a safe and practical test and, when offered only once between ages 55 and 64 y, confers a substantial and long-lasting benefit.	In Norway, once-only flexible sigmoidoscopy screening or flexible sigmoidoscopy and FOBT reduced colorectal cancer incidence and mortality on a population level compared with no screening. Screening was effective both in the 50- to 54-y and the 55- to 64-y age groups.	Screening with flexible sigmoidoscopy was associated with a significant decrease in colorectal-cancer incidence (in both the distal and proximal colon) and mortality (distal colon only).	A single flexible sigmoidoscopy screening between ages 55 and 64 y was associated with a substantial reduction of CRC incidence and mortality.

CRC, colorectal cancer; FOBT, fecal occult blood testing; NORCCAP, Norwegian Colorectal Cancer Prevention; NordICC,Nordic-European Initiative on Colorectal Cancer; PLCO, Prostate, Lung, Colorectal, and Ovarian; RCT, randomized controlled trial; SCORE, Screening for Colorectal; SPIRIT, Standard Protocol Items: Recommendations for Interventional Trials; UKFSST, United Kingdom Flexible Sigmoidoscopy Screening Trial.

Adherence to SPIRIT Statement

When considering the 8 main domains of the SPIRIT guidance, the domain with the lowest average adherence was “Appendices” with 60%. This was followed by the “Methods: Monitoring” domain with 75% adherence and the “Methods: Assignments of interventions” domain with 76% adherence. The highest adherence was observed in the “Introduction” domain, where all trials achieved 100% adherence. Other domains, including “Ethics and dissemination,” “Methods: Participants, interventions, and outcomes,” “Methods: Data collection, management, and analysis,” and “Administrative information” had adherence ranging from 80% to 98% (Table 2).

Table 2.

Trials Adherence to the SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) Statement

SPIRIT domain	Bretthauer 2022 (NordICC)	Atkin 2017 (UKFSST)	Holmes 2014 (NORCCAP)	Schoen 2012 (PLCO)	Segnan 2011 (SCORE)	Total %
Administrative information: (total 9), n (%)	6 (66.7%)	8 (88.9%)	8 (88.9%)	7 (77.8%)	7 (77.8%)	80%
Introduction: (total 4), n (%)	4 (100%)	4 (100%)	4 (100%)	4 (100%)	4 (100%)	100%
Methods:participants, interventions, and outcomes: (total 10), n (%)	10 (100%)	10 (100%)	10 (100%)	9 (90%)	10 (100%)	98%
Methods: Assignment of interventions: (total 5), n (%)	3 (60%)	4 (80%)	3 (60%)	5 (100%)	4 (80%)	76%
Methods: Data collection, management, and analysis: (total 6), n (%)	6 (100%)	6 (100%)	4 (67%)	6 (100%)	6 (100%)	93.3%
Methods: Monitoring: (total 4), n (%)	4 (100%)	2 (50%)	2 (50%)	4 (100%)	3 (75%)	75%
Ethics and dissemination: (total 11), n (%)	9 (81.8%)	8 (72.7%)	10 (90.9%)	11 (100%)	11 (100%)	89.1%
Appendices: (total 2), n (%)	1 (50%)	1 (50%)	1 (50%)	1 (50%)	1 (50%)	60%
Total (51)	43 (84.3%)	43 (84.3%)	42 (82.4%)	47 (92.2%)	46 (90.2%)	(86.6%)

NORCCAP, Norwegian Colorectal Cancer Prevention; NordICC,Nordic-European Initiative on Colorectal Cancer; PLCO, Prostate, Lung, Colorectal, and Ovarian; SCORE, Screening for Colorectal; SPIRIT, Standard Protocol Items: Recommendations for Interventional Trials; UKFSST, United Kingdom Flexible Sigmoidoscopy Screening Trial.

The most missed recommendation item was item 2b (trial registrations: All items from the World Health Organization Trial Registration Data Set) scoring 0 across all studies. This was followed by item 32 (informed consent materials: Model consent form and other related documentation given to participants and authorized surrogates) scoring 20%, and items 17a &b (blinding) with 40% each.

Individual trials' results

Bretthauer (Nordic-European Initiative on Colorectal Cancer (NordICC)) trial

The overall adherence of the NordICC trial¹³ to SPIRIT guidelines was 84.3%, which was similar to United Kingdom Flexible Sigmoidoscopy Screening (UKFSST) trial. The average adherence was lowest in “Appendices” (50%) domain, followed by the Methods: Assignments of interventions” (60%), and “Administrative information” (66.7%) domain. On the other hand, the NordICC trial achieved 100% in 3 domains; “Introduction”, “Methods: Participants, interventions, and outcomes” and, “Methods: Data collection, management, and analysis” (Table 2).

Atkin (UKFSST) trial

The UKFSST trial¹⁰ demonstrated an overall adherence rate of 84.3% to the SPIRIT guidelines. The average adherence was lowest in “Appendices” and “Methods: Monitoring” domains (50% each). Similar to NordICC, the UKFSST trial achieved 100% in 3 domains; “Introduction”, “Methods: Participants, interventions, and outcomes” and, “Methods: Data collection, management, and analysis” (Table 2).

Holmes (Norwegian Colorectal Cancer Prevention (NORCCAP)) trial

The NORCCAP trial¹⁴ exhibited an overall adherence rate of 82.4% to the SPIRIT guidelines, which was the lowest among all the trials. The average adherence was particularly low in the “Appendices” and “Methods: Monitoring” domains, with both domains scoring 50%. The “Methods: Assignments of interventions” domain achieved an adherence rate of 60%, while the “Methods: Data collection, management, and analysis” domain had an adherence rate of 67%. However, the NORCCAP trial demonstrated relatively higher adherence in the other domains (Table 2).

Schoen (Prostate, Lung, Colorectal, and Ovarian (PLCO)) trial

The PLCO trial¹² showed an overall adherence rate of 92.2% to the SPIRIT guidelines, which was the highest among all the trials. Notably, the PLCO trial achieved an adherence rate of 100% in 5 domains: “Introduction,” “Methods: Assignments of interventions,” “Methods: Data collection, management, and analysis,” “Methods: Monitoring,” and “Ethics and dissemination.” However, similar to other trials, the adherence to the “Appendices” domain was 50%. Overall, the PLCO trial demonstrated a strong commitment to adhering to the SPIRIT guidelines (Table 2).

Segnan (Screening for Colorectal (SCORE)) trial

The overall adherence of SCORE trial¹¹ to SPIRIT guidelines was 90.2%, ranking as the second highest among all the trials. The SCORE trial achieved 100% adherence to SPIRIT guidelines in 4 domains; “Introduction”, “Methods: Participants, interventions, and outcomes” “Methods: Data collection, management, and analysis,” and “Ethics and dissemination”. However, similar to other trials, the adherence to the “Appendices” domain was 50% (Table 2).

Participants' Adherence to Protocol

In terms of adherence to the study protocol, a total of 587,572 participants were initially randomized across all 5 RCTs. Among these participants, 200,610 (37%) were assigned to undergo CRC screening, while 342,115 (63%) were assigned to the no screening group. Notably, out of the participants assigned to CRC screening, 139,983/200,610 (69.8%) demonstrated adherence to the study protocol. It is worth mentioning that the Bretthauer et al¹³ study, known as the NordICC trial, employed a unique intervention approach. In this trial, invitations for CRC screening were sent to participants after randomization, resulting in an adherence rate of 42% (Table 1). Exclusion of the NordICC trial would increase the adherence rate to 74.3% (128,050/172,390).

Quality and Publication Bias Assessment

The 5 RCTs were considered of high quality using the modified Jadad scale (Table A1).

Discussion

RCTs provide the highest level of evidence for the effectiveness of medical interventions. The undisputed merits of CRC screening studies help formulate the society and government-sponsored guidelines for CRC screening. The SPIRIT statement was launched to standardize the reporting quality and improve study protocols' transparency.^18,19 To our knowledge, this is the first study to examine adherence reporting from endoscopic CRC screening studies in relation to the SPIRIT statement.

Our study observed an overall high adherence (86.6%) to the SPIRIT reporting guidelines among the published endoscopic CRC screening studies, ranging from 84.3% to 92.2%. These findings are consistent with a recent cross-sectional study by Lohner et al,²⁰ which examined 292 RCTs in Switzerland, Germany, and Canada. They reported an overall adherence rate of 75% to the SPIRIT checklist, compared to our study's result of 86.6%. This suggests that CRC screening studies are generally well-designed and well-documented, which in turn makes the results more acceptable for implementation into clinical practice. In contrast, a study by Tan et al⁴ assessed the reporting quality of 300 unspecified RCTs before and after the introduction of the SPIRIT statement in 2013. They found that adequate reporting was present in only 47.9% of the RCTs before the SPIRIT statement, which increased to 56.7% after its publication. This highlights the importance of having a standardized framework for reporting in RCTs. Several factors can contribute to the quality of reporting, including the involvement of multiple centers, longer protocols, and the presence of publicly reported journal compliance guidelines.⁴ These factors are commonly observed in CRC screening studies, which may explain the higher adherence to reporting standards in this field. It is worth noting that 2 of these trials were conducted before the establishment of the SPIRIT statement in 2013, and they reported overall adherence rates of 90.2% and 92.2%. In contrast, 3 trials were conducted after the establishment of SPIRIT, with overall adherence rates of 82.4%, 84.3%, and 84.3%. Although the limited number of studies do not allow for evaluating the impact of SPIRIT statements in improving the quality of RCT reporting pertaining to endoscopic CRC screening, these findings do highlight the importance of striving for better adherence in future trials.

The adequate reporting of the methods domains in the included trials is reassuring (ranging from 75% to 98%) as it ensures that the protocols contain the necessary information for critical evaluation and interpretation of the study. The ethical aspects of the trials were also adequately reported, with an average adherence rate of 89.1% in the ethics domain. This ensures that high ethical conduct is maintained throughout the research process, safeguarding the rights and well-being of the participants.¹⁸ However, the appendices domain showed the lowest adherence rate, achieving 50% in each trial. This is primarily due to under-reporting of items related to “consent or assent: ancillary studies and authorship eligibility guidelines and any intended use of professional writers”. Specifically, item 32, which pertains to informed consent materials such as model consent forms, had a low adherence rate of 20% among the included studies. This finding aligns with a study by Eleftheriadi et al,¹⁸ which reported that only 3% of item 32 was adequately reported. The insufficient reporting in this domain may be attributed to the assumption that a detailed consent process described in item 26 of the SPIRIT statement is sufficient, and there is no need to include consent forms in the ancillary documentation. Additionally, language barriers may contribute to the lack of inclusion of consent forms, as they might be written in the native language of the participants, which may not align with journal policies requiring “English-only” materials. However, providing a model consent form is crucial to ensure that relevant information is provided in sufficient detail and at the appropriate literacy level for the target population.

Although the overall adherence to the SPIRIT statement was acceptable, our study identified 4 individual checklist items that were inadequately reported. A study by Yang et al²¹ evaluated the reporting quality of study protocols in anesthesia using the SPIRIT statement and found a higher number of insufficiently reported checklist items compared to our study (18 vs 4). Similar to our findings, they also highlighted item 2b (trial registrations: All items from the World Health Organization Trial Registration Data Set) as one of the most missed recommendation items. To improve the reporting of item 2b, Qureshi et al²² suggest including a separate table that lists all elements from the WHO Trial Registration Data Set or indicating within the protocol that it contains all the required items. This can help ensure comprehensive reporting of trial registration information, which is crucial for maintaining the validity of evidence-based practice and the availability of reliable data. The prevalence of registered trials has increased significantly over the past decade.¹⁹ Including complete trial registration information enhances transparency, facilitates critical appraisal of study designs, and promotes the reproducibility of research findings. By adhering to the recommended reporting guidelines, researchers can contribute to the quality and trustworthiness of the evidence base in their respective fields.

Our study is subject to certain limitations that should be acknowledged. Firstly, we only included RCTs reported in English language, which may have introduced language bias and limited the inclusion of relevant studies published in other languages. Secondly, the absence of data collectors blinded to the RCT protocols' release date introduces the possibility of investigator bias. However, we implemented strict protocols and guidelines for reporting each item to minimize potential bias and maintain consistency in data collection. Lastly, the relatively small number of included trials in our study may restrict the generalizability of our findings. Although we made efforts to include all relevant RCTs meeting our criteria, a larger number of studies would have provided more robust results and enhanced the external validity of our study. Furthermore, the limited quantity of trials presents a challenge in terms of comparing the adherence of these trials to SPIRIT guidelines before and after the implementation of the SPIRIT guidelines.

Conclusion

In conclusion, our study demonstrates that published endoscopic CRC screening trials generally exhibit acceptable adherence to the SPIRIT reporting guidelines. This information is important because in light of recent changes in CRC screening age and variation in some societal guidelines, our study shows that overall the RCTs for endoscopic screening for CRC are well-conducted and this reinforces their findings and the need for CRC screening. However, our study has also highlighted some areas for improvement in RCT reporting for future trials. By addressing these areas of improvement, researchers and clinicians can enhance the credibility and validity of CRC screening studies, ultimately leading to better-informed decision-making and improved patient care.