Impact of feedback on adenoma detection rate: a systematic review and meta-analysis

Abstract

Background

Adenoma detection rate (ADR) is one of the most important quality indicators of colonoscopy. Monitoring endoscopists and providing feedback has shown to improve ADR. We performed a systematic review of the literature and meta-analysis to determine the effect of any form of feedback on ADR.

Methods

A literature search for comparative studies that employed any form of feedback to assess the impact on ADR before and after the feedback was done on MEDLINE, EMBASE, and Cochrane Database. The primary outcome of interest was ADR. Secondary outcomes included polyp detection rate, advanced adenoma detection rate, sessile serrated adenoma detection rate, withdrawal time, and cecal intubation rate. Cochrane Revman 5.3 software was used for statistical analysis.

Results

A total of 12 studies met the inclusion criteria for the analysis of primary outcomes. There were 78,355 subjects (45.42% male) with a mean age of 59.52 years. There was a significant improvement in ADR after any form of feedback compared to no feedback: 36.18% vs. 26.75%; pooled odds ratio 1.51, 95% confidence interval 1.37-1.66; P<0.001. There was a substantial heterogeneity (I²=82%). ADR improved in both active or passive feedback, irrespective of whether endoscopists knew about being monitored for their performance or not.

Conclusions

Monitoring and providing feedback to endoscopists in any form leads to improvement in ADR. Feedback is an easy and effective way of improving the ADR of endoscopists, especially in those not achieving the recommended benchmarks.

Introduction

Colorectal cancer (CRC) is the third most common malignancy and the second most common cause of cancer-related deaths in the United States and worldwide [1]. The American Cancer Society guidelines recommend screening of average-risk individuals for CRC starting at the age of 45 years. However, as per 2018 estimates, only 68.8% of individuals aged between 50 and 75 years were up to date with CRC screening in the United States [2]. Commonly used CRC screening methods are colonoscopy, guaiac-based fecal occult blood test (gFOBT), fecal immunohistochemical test, and multitarget stool DNA tests with fecal immunochemical testing [3-5]. Colonoscopy remains the most sensitive test for CRC screening offering treatment for precancerous lesions in the same session [6]. Although the role of screening colonoscopy has been well established in the prevention of CRC, the quality of colonoscopy has an impact on the effectiveness of such intervention. Previous studies have shown that the quality of colonoscopy varies significantly among the endoscopists [7]. Hence, multiple pre-procedure, intra-procedure, and post-procedure benchmarks have been proposed as quality indicators for screening colonoscopy [8,9]. Adenoma detection rate (ADR) is one of the most important benchmarks for assessing the quality of screening colonoscopy. According to the 2015 American Society for Gastrointestinal Endoscopy guidelines, ADR of more than 30% for men and more than 20% for women is recommended for screening colonoscopies in patients older than 50 years with an average risk of CRC [10].

ADR is affected by multiple factors, including study population (e.g., screening cohort, gFOBT positive, or population with a family history of colon cancer or surveillance population), quality of bowel prep, withdrawal times, inspection techniques, use of distal attachment devices, educational interventions, and training [11]. Longer withdrawal time and meticulous inspection techniques are associated with higher ADR [12,13]. There has been a strong emphasis on improving ADR of the endoscopist as studies have shown that low ADR is associated with a higher risk of post-colonoscopy colon cancer, and improving ADR can reduce this risk [14,15].

Various interventions have been evaluated to improve the quality of colonoscopy, especially ADR. These include: feedback to endoscopists; colonoscopy skill improvement training; and public reporting of benchmark indices. Multiple previous studies have shown that feedback in any form, including monitoring ADR of the endoscopists and providing report cards or active intervention with endoscopy quality improvement training, could lead to an improvement in their ADR [16-25]. Studies have reported feedback’s relationship to ADR variably. Also, different forms of feedback and their impact have not clearly defined or systematically reviewed before this. We have performed a systematic review and meta-analysis to examine the impact of feedback to endoscopists on the ADR and other parameters.

Materials and methods

Meta-analysis was performed according to the Preferred Reporting Items for Systematic Review and Meta-Analysis (PRISMA) statement [26] (Supplementary Table 1 ^{(2.2MB, pdf)} ).

Definitions

Currently, there is no standardized definition of ‘feedback’ for endoscopists to improve the quality of colonoscopy from any of the gastroenterology societies. Therefore, based on our literature review, we defined ‘feedback’ as an intervention in the form of report cards, instructions, presentations, or hands-on endoscopy quality improvement training programs for endoscopists to improve the quality of colonoscopy (e.g., ADR, polyp detection rate (PDR), advanced adenoma detection rate (AADR), withdrawal time (WT), or cecal intubation rate (CIR). We further classified feedback into 2 categories: ‘active feedback’, where the endoscopists receive hands-on training and/or individualized instruction or classroom presentations on techniques to improve the ADR; and ‘passive feedback’, where the endoscopists may receive their performance cards or public reporting of quality metrics, however, no further individualized instructions or classroom presentations or hands-on training sessions are provided.

Search strategy

An electronic search was performed in MEDLINE, Embase, and Cochrane database from inception to January 2020 for comparative studies that assessed feedback (in the form of scorecards, report cards, public reporting of colonoscopy quality metrics, quality improvement projects) to measure the effect of feedback on colonoscopy quality metrics before and after the intervention. The search for studies of relevance was performed using the “AND/or” combination of the following keywords “adenoma detection rate (ADR),” “feedback,” “colonoscopy,” “polyp detection rate (PDR).” References for major trials and review articles were also manually searched.

Study selection criteria

We included the studies that met the following inclusion criteria: studies that provided data regarding ADR before and after any form of feedback to the endoscopists, and the primary outcome of interests was ADR before and after the feedback in the same group of endoscopists. Since this study is designed to examine and quantify the change in ADR before and after the ‘feedback,’ we included only the study groups for primary outcome analysis.

Secondary outcomes included PDR, AADR, sessile serrated adenoma detection rate (SSADR), CIR, and WT. We excluded review articles, non-human studies, articles in abstract form only, case reports/series, and editorials. Studies where there was no ‘before’ and ‘after feedback’ comparison or any form of feedback was not used, were excluded as well.

Data extraction

Two investigators (VN and UB) independently reviewed the studies and imported the data into a standardized form. Whenever there was a lack of consensus on eligibility [27], the senior author (AR) reviewed the study independently and then made the final decision regarding the data point. Data extracted were the author, year of publication, study location, type of study, method of feedback, number of endoscopists, duration of follow up, patient demographics, the total number of subjects before and after feedback, and details on ADR, PDR, AADR, SSADR, CIR, and WT in the intervention and the control groups.

Quality assessment

The quality of each included study was assessed by 2 authors independently using the Newcastle-Ottawa scale [28]. The quality of each study included in the meta-analysis was assessed based on the selection of study groups, comparability of study groups, and assessment of outcomes.

Outcomes

The primary outcome was pooled ADR in included studies before and after the intervention (i.e., feedback). Sensitivity analysis was performed for the studies where only the passive form of feedback was used including report cards to inform endoscopists of their performance, public reporting of their quality metrics or annual evaluation of ADR and separately for studies where there was an active form of feedback was given like endoscopy quality improvement program, presentations, tips on how to improve the ADR. This was performed to stratify the impact of passive feedback from active feedback and assess their comparative effect on ADR. We also examined ADR improvement in only prospective studies to understand and derive the effect of longitudinal follow up. Finally, we also performed a sensitivity analysis to unmask any impact of the Hawthorne effect among endoscopists. Studies where endoscopist was aware of being monitored (before any form of feedback) were analyzed separately from studies where endoscopists were not aware of being monitored to examine the impact of vigilance on their ADR pre- and post-feedback.

However, it is imperative that we examine the effect of feedback in randomized controlled trials (RCTs) to establish whether the ‘feedback’ has any impact at all on ADR by comparing actual control and study groups. Therefore, we compared ADR in the control group and the study group from RCTs.

Secondary outcomes were pooled detection rate of PDR, AADR, and SSADR to examine the impact of feedback on these metrics. The impact of change on WT was also evaluated.

Statistical analysis

Pooled odds ratio (ORs) with 95% confidence interval (CIs) for the outcomes of interest were synthesized by meta-analysis using the DerSimonian-Laird random-effects model. Heterogeneity between the included studies was estimated using the inconsistency index (I²). Heterogeneity of 25%, 50%, and 75% was classified as low, moderate, and substantial heterogeneity, respectively. Publication bias was assessed using a funnel plot. The statistical analyses were performed using RevMan software (Review Manager Version 5.3; The Nordic Cochrane Centre, Copenhagen, Denmark, The Cochrane Collaboration 2015).

Results

Study characteristics

A total of 6785 records were identified, of which 199 articles were identified after the exclusion of duplicates and screening of articles based on title and abstract. A total of 30 full-length manuscripts were reviewed, and we found 12 studies that met our inclusion criteria for the primary outcome. Three other studies that did not report ADR, but reported data on secondary outcomes were included in the pooled estimates of secondary outcomes (total n=15) [29-31]. There were 4 RCTs [18,25,27,30] and 11 observational studies in total [7,16,17,19-24,29,31].

The primary goal of this study was to examine and quantify the change in ADR as a result of feedback among the endoscopists. Therefore, we included pre- and post-feedback ADRs from observational studies, and only the study groups from RCTs for the purpose of estimating the effect of feedback and the change in ADR before and after the intervention. Some of the studies included in the meta-analysis reported post-feedback ADR results at multiple intervals; Nielsen et al provided data on pre-feedback, early post-feedback ADR (phase 2), and late post-feedback ADR (phase 3) [22]. Similarly, Sey et al reported data at baseline, at 1 year after feedback and at 2 years after the feedback [17]. Whenever post-feedback results were reported at more than 1 interval, we considered pre-feedback (baseline) ADR and the ADR at the end of the study (late phase or phase 3) for statistical analysis. Study designs, patient characteristics, and individual study outcomes of the included studies are listed in Table 1. The flow diagram depicting our literature search strategy and study selection process is shown in Fig. 1.

Table 1.

Characteristics and demographics of eligible studies

Figure 1.

Flow chart of selecting eligible studies for the meta-analysis

A total of 12 studies were included for analysis of ADR, with a total of 78,355 colonoscopies performed by more than 217 endoscopists. There were 3 RCTs and 9 observational studies. The mean age of the study population was 59.52 years (reported in 9 studies), with 45.42% male. There were 49,607 subjects who underwent colonoscopy during the pre-feedback phase and 28,748 subjects during the post feedback period. Seven studies included subjects who underwent screening colonoscopy only [7,16,20,21,23,24,27] and 4 studies that included both screening colonoscopies and surveillance colonoscopies [18,19,22,25]. One study included subjects who underwent colonoscopy due to positive gFOBT and family history of CRC [17].

Among the 12 studies included in the meta-analysis, 6 studies used active feedback in the form of quality improvement training, presentations, or by providing tips on how to improve ADR. The remaining 6 studies used passive feedback in the form of either report cards only (5 studies) or by public reporting of benchmark indices (1 study). In a RCT by Kaminski et al, both the control group and the study group received report cards, but the study group also received active intervention in the form of hands-on training. Only the study group was considered for primary outcome analysis (before and after feedback of ADR), and the control group was included in a subgroup analysis with no active intervention [27].

Quality of the included studies was assessed using the Newcastle-Ottawa scale for cohort studies, as shown in Supplementary Table 2 ^{(2.2MB, pdf)} . Among the 15 studies included in the meta-analysis, 6 studies scored 6, 5 studies scored 7, and 4 studies scored 8. Supplementary Table 2 ^{(2.2MB, pdf)} provides details on each parameter of the Newcastle-Ottawa scale and scores for each parameter from each study as well as a summary score for cohort studies.

ADR

First, we examined if feedback can improve ADR in the study groups compared to the control groups from RCTs. Analysis of 3 RCTs showed that the feedback improved ADR significantly (OR 1.26, 95%CI 1.08-1.46; P=0.004), suggesting feedback is effective in achieving improvement in the ADR. Fig. 2 shows a forest plot for pooled estimates comparing the control group and study groups in the RCTs. Further, we compared ADR before and after the feedback in the same group of endoscopists from all included studies.

Figure 2.

Forest plot for pooled analysis of randomized controlled trials: control vs. study groups

CI, confidence interval

Pooled ADR was found to be higher when endoscopists performing colonoscopies received any form of feedback compared to none (36.18% vs. 26.75%; P<0.001). ADR increased significantly after the feedback, with pooled OR of 1.51 (95%CI 1.37-1.66; P<0.001). There was substantial heterogeneity in included studies (I²=82%). Fig. 3A shows the forest plot of studies that employed feedback in any form.

Figure 3.

(A) Forest plot for pooled analysis of adenoma detection rate of all studies; pre- vs. post- feedback analysis. (B) Forest plot for pooled analysis of adenoma detection rate from randomized controlled trials and observational studies. (C) Forest plot for pooled analysis adenoma detection rate of studies based on the type of population

CI, confidence interval

We compared a subgroup analysis of RCTs and observational studies to examine the difference, if any, in the pooled estimates between the 2. The pooled odds ratios of 3 RCTs and 9 observational studies were 1.58 (n=26,778; 95%CI 1.49-1.66; P<0.001) and 1.47 (n=48,321; 95%CI 1.27-1.69; P<0.001), respectively. The heterogeneity among the RCTs was ‘zero,’ whereas the heterogeneity among the observational studies was significant at 84%. However, the OR remained statistically significant in both the groups and suggesting no significant difference in the improvement in ADR between RCTs and observational studies. This also supports our decision to combine both observational studies and study groups from RCTs before and after feedback together for the primary outcome analysis. The forest plot analysis is shown in Fig. 3B.

The improvement in the ADR can be affected by the type of population undergoing colonoscopy. The prevalence of adenomas is likely to be higher among the surveillance group compared to the screening colonoscopy cohort. We compared the improvement in ADR among the studies that included screening colonoscopy cohort only and the studies that included both screening cohort and surveillance cohorts. A subgroup analysis showed that the improvement in the ADR was significant in both the groups (OR 1.43, 95%CI 1.25-1.64; P<0.001 vs. OR 1.67, 95%CI 1.51-1.84; P<0.001; Fig. 3C).

A subgroup analysis of the 7 studies that employed passive feedback in the form of report cards only or public reporting of ADR without training programs to improve the ADR showed that monitoring without active intervention alone could improve the ADR significantly (pooled OR 1.43, 95%CI 1.20-1.70; P<0.001). Supplementary Fig. 1 ^{(2.2MB, pdf)} illustrates pooled estimates of the studies that used report cards or public reporting only. When the post-feedback ADR rate of active feedback and passive feedback were compared, the pooled estimate suggested passive feedback being superior to active feedback (OR 2.65, 95%CI 2.55-2.77; P<0.001). Although post-feedback improvement in ADR of active feedback (mean difference 8.78, 95%CI 8.62-8.94; P<0.001) and passive feedback (mean difference 10.08, 95%CI 9.76-10.41; P<0.001) were comparable, the improvement with passive feedback was 1.3% higher than improvement in active feedback.

Subgroup analysis of studies that employed active interventions such as endoscopy quality improvement program, presentations, tips on how to improve the ADR, and other benchmark metrics also improved the ADR significantly (pooled OR 1.52, 95%CI 1.34-1.73; P<0.001). The results of the pooled estimates are shown in Supplementary Fig. 1 ^{(2.2MB, pdf)} . When the analysis was restricted to prospective studies alone, the feedback was shown to improve ADR significantly as well (n=7, pooled OR 1.52, 95%CI 1.35-1.70; P<0.001). Pooled estimates of the prospective studies are illustrated in Supplementary Fig. 2 ^{(2.2MB, pdf)} .

Furthermore, when the studies were separated into those where endoscopist was aware being monitored prior to any form of feedback (n=5, pooled OR 1.54, 95%CI 1.31-1.81; P<0.001) and those studies where they were not aware of being monitored (n=7, pooled OR 1.47, 95%CI 1.32-1.64; P<0.001), feedback was shown to improve their ADR irrespective of being aware of the monitoring. Pooled estimates of this subgroup analysis are shown in Supplementary Fig. 3 ^{(2.2MB, pdf)} .

Secondary outcomes

A total of 11 studies reported PDR, where the feedback group had 32,996 patients, and the control group had 19,132 patients. The PDR was significantly higher in the feedback group when compared to the control group (53.24% vs. 35.96%, pooled OR 1.46; 95%CI 1.25-1.71; P<0.001). Supplementary Fig. 4 ^{(2.2MB, pdf)} demonstrates the pooled analysis of the PDR.

Eight studies reported AADR involving 45,056 patients. The AADR improved from 7.1% in the control group to 10.1% in the feedback group (pooled OR 1.20, 95%CI 1.09-1.33; P=0.0004). Supplementary Fig. 5 ^{(2.2MB, pdf)} shows pooled estimates of 7 studies that reported AADR.

Only 2 studies reported SSADR, and no significant improvement in the SSADR was noted (pooled OR 1.34, 95%CI 0.65-2.77; P=0.43). Supplementary Fig. 6 ^{(2.2MB, pdf)} shows the forest plot of the included study and pooled estimates. Withdrawal time was reported by 4 studies included in this meta-analysis, and pooled estimates show that withdrawal time increased from 8.51 min in the control group to 10.19 min in the feedback group (P=0.214). There was no significant change in the cecal intubation rate after the feedback group when compared to the control group (85.6% vs. 86%; P=0.68).

Publication bias

We analyzed the publication bias of the included studies using the funnel plot. Fig. 4 shows the clustering of the effect size at the top of the funnel plot suggestive of no major publication bias in the included studies for the primary outcome estimate. None of the included studies was outside of the funnel plot limits.

Figure 4.

Funnel plot for bias analysis of studies included in the pooled estimation of adenoma detection rate

SE, standard error; OR, odds ratio

Discussion

This systematic review and meta-analysis of nearly 78,000 colonoscopies show that feedback in any form to endoscopist leads to a statistically significant improvement in ADR by almost 9%. Feedback also led to an improvement in AADR by 3%. A comparison of the control group and study groups from RCTs showed that ADR was significantly higher in the feedback group. Subgroup analysis of the studies showed improvement in the ADR with both passive and active forms of feedback. In addition, positive change in ADR with feedback was present irrespective of whether endoscopists were aware of being monitored. This clearly indicates the benefits of feedback. The findings on SSADR, WT, and cecal intubation were limited by a small number of studies reporting these outcomes. For the same reason, these outcomes could not be correlated with primary outcome measures.

It is also important to note that the improvement in ADR was significant across all formats of studies irrespective of the study designs. No significant difference in pooled estimates was observed when the studies were grouped as observational, RCTs, prospective randomized or nonrandomized, or retrospective studies. Similarly, the improvement in ADR was also observed when the analysis was restricted to studies that included population who underwent screening colonoscopy only as well as when both surveillance and screening cohorts were included. This is a clear indication that the improvement in ADR was not confounded by the population included in the studies.

There is scarce literature on the standard definitions of “feedback” to endoscopists on their performance. The types and methods of feedback have not been clearly defined or validated. It is also not clear whether passive feedback utilizing report cards alone is sufficient or active interventions such as quality improvement training, supervised colonoscopy sessions, and tips on how to improve ADR are required. A recently published meta-analysis evaluated the effect of training intervention on ADR. The study showed that training intervention improves the ADR; however, it was limited by a small number of studies included, and the study did not evaluate various forms of feedback [32]. Another currently in press meta-analysis by Bishay et al also found that ADR improves with feedback. However, the study missed 2 RCTs and was again limited by a 50% smaller number of colonoscopies. Moreover, the meta-analysis did not address whether there is any difference between various types of feedback [33]. In this meta-analysis, we attempted a comprehensive review of the available literature and evaluated the effect of feedback on ADR. Our study has a significantly large number of studies included and comprehensive in evaluating the effect of feedback based on the type of population, study design, type of feedback, and evaluated multiple secondary outcomes that are not reported in previous meta-analyses. This meta-analysis shows that structured feedback in any form, whether active or passive, has the potential to improve ADR. This, in turn, has implications for improving the quality of screening colonoscopy and the potential to reduce the incidence of post colonoscopy CRC.

The pooled estimates in this meta-analysis suggest that both passive feedback and active feedback improve the ADR. In this analysis, we did attempt to stratify feedback being monitoring of ADR (passive) vs. active intervention with the training of personnel to examine their relative potential. Although passive feedback resulted in higher ADR than active feedback (OR 2.65, 95%CI 2.55-2.77; P<0.001), we could not assess the factors that may be responsible for this difference. In a multicenter RCT [27], the effect of passive feedback and active feedback on ADR were compared. Endoscopists were randomized into the control group (received report cards only) and the study group (received a pre-training assessment, hands-on training, post-training assessment, and feedback). The study showed that both methods improved ADR significantly. However, the improvement in the active feedback group (6.5%) was significantly greater than the passive group (2.3%) compared to the pre-intervention phase. There are no other studies that have previously compared the effect of feedback in the form of report cards alone vs. active interventions. Therefore, future studies should compare various types of feedback to further crystallize our understanding of their effectiveness and benefits.

Previous studies have reported improvement in ADR with feedback varying from 2-23% [22,24]. Pooled estimates from our meta-analysis show that the average improvement in ADR is around 9%. This broad variation in the effect of feedback is likely due to the different methods and processes of feedback utilized by various studies. In this meta-analysis, we also determined that the improvement in quality metrics was not limited to ADR alone. Pooled estimates suggest that significant improvement was observed in PDR (9 studies) and AADR (7 studies) as well.

There are certain limitations of this study that should be acknowledged while reviewing pooled results. The definition of feedback is not clearly defined in the gastroenterology literature or by the gastroenterological societies. This limits our search criteria to be subjective and based on prior literature. We need to define different forms and grades of feedback to understand their impact and relative efficacy. Second, previous studies have applied different forms of feedback methods, including report cards, quality improvement training programs, supervision of endoscopists with low ADR, and classroom presentations. There was, therefore, substantial heterogeneity noted in this analysis. This is likely explained by the fact that all forms of feedback were taken together as a group, and even subgroup analysis involved broad categories of active or passive forms of feedback. The role of methodological heterogeneity and clinical heterogeneity cannot be explored due to lack of data and also due to not having more than a few studies of a particular type of feedback. Lastly, 3 studies included in the meta-analysis used trainees to assess the effect of feedback. It is expected that trainees will have a lower ADR to start with, and their ADR improves with training. Since colonoscopies were performed by trainees under the supervision of experienced endoscopists, we considered these studies to be valid for inclusion in the meta-analysis. When these 2 studies (Kahi et al and Rajasekhar et al) were excluded from the analysis primary outcome analysis, the change in the pooled estimates was negligible (pooled OR 1.58, 95%CI 1.47-1.69; P<0.001).

Some of the studies included in this systematic review reported that the improvement in ADR was substantially higher among the endoscopists who had the lowest ADR before feedback compared to endoscopists who had higher ADR to begin with [17]. This premise can be used to target the group of endoscopists that may benefit the most from feedback. When the feedback was provided anonymously, the performance improvement in ADR is postulated to be due to healthy competition among peers.

Given that in some studies, the endoscopists knew that they are participating in the research, the improvement in ADR could be due to the ‘Hawthorne effect”. However, our subgroup analysis above disputes this and shows the benefits of feedback regardless of the endoscopists being aware of monitoring or not. It is also not clear how long the improvement in ADR will last once the feedback is discontinued. Long term follow-up of the quality metrics is required to assess the longevity of the effect of feedback and to determine the frequency of feedback needed to sustain a high-quality colonoscopy.

Therefore, a consensus-based structured feedback strategy needs to be developed and validated through RCTs with sufficient follow up. Although this meta-analysis showed improvement in ADR due to feedback, it is not clear if feedback alone is enough to sustain the improvement. A combination of feedback reports, public reporting, and other incentives may be needed to sustain the improved benchmark indices. Based on our meta-analysis, feedback to endoscopists improves the quality of colonoscopy by increasing the ADR, PDR, and AADR. A well-defined and structured feedback mechanism should be implemented for improving this critical quality indicator of colonoscopy. Future studies are warranted to compare various forms of feedback methods to understand the benefits and limitations.

Summary Box.

What is already known:

• Adenoma detection rate (ADR) correlates with quality of colonoscopy

• Low adenoma rate is associated with increased risk of interval colon cancer

• Feedback to endoscopists improves ADR

What the new findings are:

• Meta-analysis shows feedback to endoscopists improves ADR

• Improvement in adenoma detection rate is independent of type of feedback

Biography

Bassett Medical Center, NY; Kansas City VA Medical Center, MO; University of Kansas Medical Center, KS; University of New Mexico, Albuquerque, NM, USA