Improved detection of adenomas and sessile serrated polyps is maintained with continuous audit of colonoscopy

Abstract

Background

The audit process may help improve performance indicators for colonoscopy quality but it is unclear whether this is sustained over several years.

Methods

44138 procedures for 28 endoscopists from 2004 to 2019 were analysed for polyp detection rate and withdrawal time. From 2012, 14 endoscopists were analysed with additional data on polyp histology and number of polyps removed.

Results

Polyp detection increased from 40.7% in 2004 to 62.2% in 2019; removal of polyps>1 cm remained constant (11%). Adenoma detection rate was 25.8% in 2012 and 28.3% in 2019. Sessile serrated polyp (SSP) detection rate increased from 4.5% to 14.7%; most of the increase was in the first 2 years of the histology part of the audit. There was a significant correlation of adenoma detection rate with mean number of adenomas (r=0.72, p=0.004) and a significant correlation of SSP detection with mean number of SSPs (r=0.85, p=0.0001).

Conclusion

The audit process appears to encourage a higher rate of polyp detection. This was due to increased detection of smaller polyps and increased detection of SSPs.

Summary box.

What is already known about this subject?

• Audit with appropriate feedback is an established method of improving quality in healthcare. The published data for colonoscopy audits have been relatively short-term and not always readily applicable to routine practice. Given the additional resource required, it is important to be able to prove a significant and sustained benefit from a continuous audit process. There is uncertainty about the benefit of adding data on histology of polyps and the number of polyps removed at each procedure.

What are the new findings?

• This study demonstrates a clinically important benefit from audit over the 16 years of the study. The data show that improved and more accurate feedback is provided if histology data are included. This study contributes to the debate regarding other metrics that may indicate performance, in particular the number of adenomas and sessile serrated polyps removed at each procedure.

How might it impact on clinical practice in the foreseeable future?

• This study should encourage the routine use of colonoscopy audit in both public and private units with a mix of diagnostic and surveillance procedures. Unit quality depends on the performance of all endoscopists; the audit process is particularly important to improve the performance of endoscopists with lower key performance indicators. Audit with histology is a key part of improving detection of sessile serrated polyps. The data show the inadequacy of relying on polypectomy rate as a marker of performance.

Background

A continuous audit process may lead to improving performance of colonoscopy because of regular feedback of key performance indicators (KPIs) and comparison with colleagues. A continuous audit over many years (as compared with a research project) requires that data collection is straightforward, not to time consuming and applicable to routine practice. The choice of data points needs to be considered carefully to gain maximum feedback with the least impact on routine practice.

The targets or ‘benchmarks’ can evolve over years and it is important that endoscopists are continually aware of the expectations and standards of their colleagues.^{1 2} Endoscopists who wish to compare their KPIs with colleagues or benchmark to international standards are confronted with potential differences in case-mix, such as gender, age and indications (diagnostic versus surveillance) that may affect adenoma detection rate (ADR) and other KPIs. ADR can be adjusted by assessing only those procedures where patients are aged >50 years (a common cut-off for screening). Adjusting for gender is most easily done by considering male and female results separately. Lower volume endoscopists may have inaccurate assessment of KPI; therefore, it is important to assess several years of data, particularly if subgroup analysis is used.³

Audit of individual endoscopists (with appropriate feedback) is often performed using polypectomy rates (PDR) as a surrogate marker of adenoma detection (or perhaps more commonly audit is not done at all).^4–6 The advantage of calculating PDR is that it is measured without the need for obtaining pathology. PDR may be susceptible to ‘gaming’ by endoscopists because of the removal of diminutive polyps that may not be adenomas. ADR is less vulnerable to manipulation because the specific pathology of the polyps must be confirmed. Guidelines for ADRs have suggested that finding more than 25% in a screening population aged 50 years and above is acceptable.⁷ This target of adenoma detection may now be outdated as postcolonoscopy colorectal cancer are lower with higher levels of adenoma detection.^{8 9} Appropriate targets for a typical busy private or public endoscopy unit will need to be estimated according to case-mix. Adding histology can be a difficult process for a continuous audit outside of research project. It is therefore important to evaluate how much extra information is obtained by adding histology to the audit. An overall increase in polyp detection rate is encouraging for an audit process but is likely to be of limited benefit unless histology data can confirm that significant polyps have been detected and removed.

There is also uncertainty about the need for collecting data on the number of adenoma removed. There is also debate on the appropriate metric for presenting this data. The denominator can be the total number of procedures or only procedures where adenomas were found. This latter option tests the idea that some colonoscopists are operating under a ‘one and done’ policy at times. The other proposed measure to detect ‘one and done’ is the ADR plus.^{10 11} This is the mean number of additional adenomas for procedures where more than one adenoma has been found.

Serrated polyps are considered to be precursor lesions that account for 15%–30% of colorectal cancers and they are over-represented as a cause of interval cancers.^{1 2} The detection rate and number of sessile serrated polyps (SSPs) have not been part of most audits. The skills required for the detection of SSPs may be different from the skills required for detection of tubular adenomas.

One of the earliest KPIs was withdrawal time with the benchmark of greater than 6 min suggested by earlier studies.¹² A previous study from our colonoscopy database of 67 570 procedures from 2000 to 2010 showed that there was a modest correlation of withdrawal time with polyp detection rate (r=0.42, p=0.03).¹³ Many publications have shown that encouraging longer withdrawal times (with feedback by an audit process) increases overall polyp detection and adenoma detection.^14–16 There are limited data on the correlation of withdrawal time with the detection of SSPs.¹⁷

Methods

A continuous audit at MercyAscot Endoscopy, Auckland, New Zealand was started in January 2004 and has continued up to March 2020. Data were collected by the endoscopy nursing staff immediately at the end of the procedure on insertion time, withdrawal time, polyp detection rate and the proportion of procedures with polyps estimated to be larger than 1 cm. Data on polyp histology were added to the audit in 2012 and data on number of polyps (both for adenomas and sessile polyps) were included from 2014. Data were collected for all procedures whether diagnostic or follow-up for polyps. There were no procedures that would be classified as screening and no procedures related to bowel cancer screening (eg, following FIT test). Data for the first 3 months of 2020 are combined into 2019 for analysis. There were 6-monthly meetings of the endoscopists with feedback on all the relevant KPIs. These reports were initially anonymised but after 2008, the endoscopists agreed to a completely open audit process. A detailed written summary was given to endoscopists (usually before the meeting).

The data from the histology reports were added several days later when reports were available. The number of polyps was added by the endoscopist based on the endoscopy and histology reports and this was checked again by the nursing staff who entered the data at a later date. The definition of SSP is based on the histology report only but the histopathologist was aware of the site of polyps at the time of reporting. International guidelines for reporting SSPs were in place before the start of the audit and it is unlikely that the working definition changed during the audit. The mean number of adenomas and SSPs is expressed per total number of procedures (APC and SSPPC). Adenoma per positive (APP) procedure is the mean number of adenomas for procedures where at least one adenoma has been detected. ADRplus was calculated as the mean number of additional adenomas found in procedures where one or more adenomas were detected.⁸ ADRplus is a measure of incremental gain after the first adenoma detected and is therefore independent of the ADR.

Withdrawal time is affected by the time taken for polypectomy and will increase as more polyps are detected and removed. Withdrawal time for procedures where no polyps are detected gives a more accurate indicator of the usual behaviour for a given endoscopist. The minimum number of procedures for each endoscopist to be included in this analysis was set at 100 for the polyp detection data from 2004 to 2020 and at more than 200 procedures per endoscopist for the polyp histology data.

The statistical analysis and graphs were performed using GraphPad Prism 8.

Results

Polyp detection rate and withdrawal time

Data were available for 44 138 procedures from 2004 to 2019 for 28 endoscopists with number of procedures more than 100 (range from 107 to 7628 procedures). The mean withdrawal time (for all procedures) was 10.3 min; this increased from 8.0 min in 2004 to 11.3 min in 2019. The mean withdrawal time where no polyps were detected was 7.3 min; this increased from 6.2 to 8.0 min from 2004 to 2019. Twenty-five endoscopists had a mean withdrawal time greater than 6 min; 10 had a withdrawal time more than 10 min. The polyp detection rate for 28 endoscopists was 53.5%; this increased from 40.7% in 2004 to 62.2% in 2019 (range 31.9%–83.0%, figure 1). The rate of detection of polyps greater than 1 cm was stable over 17 years, 11.2% in 2004 and 10.5% in 2019 (figure 1). Withdrawal time (where no polyps were detected) was significantly correlated with polyps detection rate; r=0.60, p<0.0007 (figure 2). There was no correlation of insertion time with polyp detection.

Figure 1.

The trends for polyp detection rate for all polyps and polyps greater than 1 cm for 2004–2019. The trends for insertion time and withdrawal time over the same time period are plotted for comparison.

Figure 2.

The correlation of withdrawal time with proportion of procedures where polyps were removed for 28 endoscopists from 2004 to 2019.

Polyp histology data from 2012

Data on polyp histology were available for 20 218 procedures for 14 endoscopists who had performed more than 200 procedures. The number of procedures per endoscopist ranged from 216 to 4010. The mean age was 61.7 years; 66.3 years for females and 59.9 years for males; 15 004 procedures were performed on patients aged 50 or more years.

Adenomas

The overall ADR was 30.4% (male 35.1%; female 26.2%) and ranged from 20.0% to 37.9%. This increased from 25.8% in 2012 to 28.3% in 2019 but was relatively stable at 28%–32% since 2013 (figure 3). For patients aged >50 years, the mean ADR was 35.2% (range 26.8%–43.0%). The mean ADR for females aged 50 years or more was 30.8% and the ADR for males was 40.1%. A total of 5387 procedures were considered to be first procedures for evaluation of new symptoms. The mean ADR for these procedures was 27.6%.

Figure 3.

The trends for polyp detection for all polyps, adenomas, SSPs, hyperplastic polyps and polyps removed but not confirmed by histology (no polyp) for 14 endoscopists from 2012 to 2019. SSP, sessile serrated polyp.

Sessile serrated polyps

The overall detection rate for SSPs (SSPDR) was 13.6% (female 14.4%, male 12.6%); 0.8% were traditional serrated adenoma and 0.14% fulfilled criteria for SSP syndrome. The range in SSPDR for the 14 endoscopists was between 7.2% and 17.3%. There was an increase in SSPDR from 4.5% in 2012 to 14.7% in 2019; most of the increase was from 2012 to 2015 (figure 3).

Hyperplastic polyps and no polyps detected

The percentage of colonoscopies with hyperplastic polyps detected and removed (HDR) was 26.3% (range 10.7%–41%). This was relatively stable over the period over the audit (23.8% in 2012 to 26.9% in 2019. The range for each endoscopist was between 10.7% and 37.8%. The category of ‘no polyps detected’ included inflammatory polyps, lymphoid aggregates and normal colonic tissue. Overall, there were 5.4% of procedures in this group (range 1.9%–10.8%). This rate was 3.6% in 2012 and 6.5% in 2019.

Table 1 shows the percentage of polyp findings (total polyps and polyp histology subgroups) ranked from the highest polyp detector (91.0%) to the lowest polyp detection rate (44.6%).

Table 1.

Individual data on polyp detection for each endoscopist

Histology	1	2	3	4	5	6	7	8	9	10	11	12	13	14
All polyps	90.8	74.1	64.2	62.5	59.6	59.1	57.3	56.0	54.4	52.4	52.1	51.9	47.6	44.6
Adenoma	37.9	34.3	35.2	35.8	27.8	29.3	28.2	28.5	30.9	29.4	29.2	25.3	20.0	26.6
ADR/PDR	0.42	0.46	0.55	0.57	0.47	0.50	0.49	0.51	0.57	0.56	0.56	0.49	0.42	0.60
SSP	17.3	13.9	13.3	13.4	10.7	12.5	15.5	10.0	7.8	10.3	15.4	12.1	7.2	11.3
Hyperplastic	37.8	40.7	22.4	23.0	30.0	26.9	29.4	27.1	24.4	20.8	16.9	21.6	24.9	10.7
No polyp	10.8	6.5	3.9	3.62	1.9	6.1	2.8	3.0	3.7	3.5	5.0	5.52	6.5	3.1

ADR, adenoma detection rate; PDR, polypectomy rate; SSP, sessile serrated polyp.

Number of adenomas and sessile serrated polyps (data from 2014)

There were 15 204 procedures with data on number of polyps (14 endoscopists).

The mean number of APC was 0.59 (range 0.31–0.79). The mean number of adenomas when at least one adenoma was detected (APP) was 1.93 (1.36–2.15). The mean ADRplus (mean additional number of adenomas detected in procedures with more than one adenoma) was 0.93 (range 0.36–1.15). The mean number of SSPs per total number of procedures (SSPPC) was 0.31 (range 0.16–0.42). The mean number of SSPs when at least one SSP was detected was 2.06 (range 1.4–2.39). The mean SSPplus was 1.07 (range 0.63–1.40). SSPs and adenomas were found together in 4.8% of procedures.

Correlations

There was a significant correlation between ADR, SSPDR and HDR with PDR (r=0.75, p=0.002; r=0.53, p=0.05 and r=0.82, p=0.0003 respectively). There was a significant correlation of ADR with mean number of adenomas (r=0.72, p=0.004; figure 4) and with ADRplus (r=0.95, p<0.0001) but not with APP (r=0.20, p=0.41). There was a significant correlation of SSPR with mean number of SSPs (r=0.85, p=0.0001; figure 5) and SSPDRplus but not with mean number of SSP in procedures with at least one SSP (r=0.22, p=0.45). There was a significant correlation between ADR and SSPDR (r=0.61, p=0.02, figure 6). Withdrawal time for 14 endoscopists from 2012 to 2019 (where no polyps were detected) was significantly correlated with PDR (r=0.57, p=0.03) and ADR (r=0.52, p=0.05) but not SSPDR, mean number of adenomas or mean number of SSPs (r=0.18, p=0.54). The proportion of PDR with adenoma (ADR) for each endoscopist ranged from 0.45 to 0.53 over the years 2012–2019 and ranged from 0.41 to 0.60 for each endoscopist (table 1). ADR was plotted against ADRplus to assess for ‘one and done’ as described by Wang et al¹⁰ (figure 7).

Figure 4.

The correlation of % adenomas with the mean number of adenomas for all procedures for 14 endoscopists.

Figure 5.

The correlation of % SSPs with mean number of SSPs for all procedures for 14 endoscopists. SSP, sessile serrated polyp.

Figure 6.

Correlation of % adenomas (ADR) with % SSPDR. ADR, adenoma detection rate; SSPDR, detection of SSPs; SSP, sessile serrated polyp.

Figure 7.

Correlation of % adenomas (ADR) with % SSPDR. ADR, adenoma detection rate; SSPDR, detection of SSPs; SSP, sessile serrated polyp.

Discussion

This study shows that the audit process (collection of KPI data and appropriate regular feedback) has resulted in improved detection and removal of polyps over the period from 2004 to 2020. This has been achieved mainly by the increased detection of small polyps (<1 cm) and because of increased recognition and removal of SSPs. The small increase in detection of adenomas is likely to be due to increased detection of small adenomas.

Polyp detection rate is influenced by the policy of some endoscopists who may choose to detect but not remove hyperplastic polyps particularly in the rectum. It is also not able to estimate how many polyps are removed but a polyp is not confirmed by histology. Table 1 shows that high polyp detectors have a high rate of ADR but also have high rates for hyperplastic polyps and also having no polyp confirmed by histology. It is uncertain whether this careful removal of all polyps and consequently high ADR may have gains in terms of a lower rate of interval cancer. The audit process could potentially increase the detection of diminutive polyps that would never lead to cancer (very small adenomas, hyperplastic polyps and lymphoid follicles).

Overall, the KPIs for this group of endoscopists were good but the variation suggests that some improvement is still possible, at least for some endoscopists. It is easy to compare results with other colleagues in the same institution who might have a similar case-mix but it is difficult to estimate an appropriate benchmark for a mixed population of diagnostic and follow-up procedures.⁴ The quality indicator for ADR has been agreed to be 25% for a screened population >50 years—male 30% and female 20%.⁴ The equivalent benchmark for mainly diagnostic procedures may be less depending on the average age.⁴ There was only 1 of 14 endoscopists who did not reach the benchmark of 25% for adenoma and 10% for SSPs.

This study has shown that KPIs improved over the audit period but many improvements seemed to reach a plateau. There was a significant improvement in PDR but this may now be stable at around 60%. The polyp detection rate is a simple measure and does correlate with ADR.^4–6 A PDR of 40% roughly correlates with an ADR of 25%.¹⁸ However, this simple measure (without the additional information from histology) does not reveal the full picture. Many publications have suggested conversion factors to give an estimated ADR.^{19 20} An earlier study suggested using a conversion factor of 0.68 for estimating ADR and 0.14 for SSPDR. Our data show a lower conversion factor (mean 0.51) that varied from 0.41 to 0.59 between endoscopists (table 1). The lower value for this study is likely to be due to increasing detection of SSPs. The addition of ADR and SSP ratio together gives a conversion factor of between 0.56 and 0.84 with a mean of 0.71. This study does not give any support to the reliable use of PDR as a surrogate marker of either ADR or SSPDR or the combination of both markers because of significant variability between endoscopists and the need for a locally validated conversion factor that could change over time.

The mean number of adenomas (APC) also closely correlates with ADR.^{21 22} Rex et al showed that an ADR of 30% equals an APC of 0.61 (0.55–0.66) and Kahi et al showed that for men an ADR of 25% equalled an APC of 0.46.^{5 23} A large French study shows a significant correlation between mean number of adenomas and ADR (r=0.84, p=0.01). This close correlation remained whether the data came from screening, surveillance or diagnostic procedures (r=0.91, p=0.0001). Our data also show this close correlation. The correlation is also present with flexible sigmoidoscopy examinations.²⁴ This leads to the question as to whether this measure is simply assessing the same skill as ADR. Some authors have advocated that this KPI should be the gold standard to measure the total polyp yield of colonoscopy.²⁵ There is significantly more work required to calculate the mean number of adenomas per procedure. At this stage, most endoscopists are unfamiliar with this metric and it is difficult to promote routine reporting of the mean number of adenomas unless further data showed some additional value.

The addition of data on SSP detection rate to the audit may have led to an increased detection rate for SSPs but these results have now reached a plateau (figure 2). This is a strong argument for introducing audit and perhaps also focused teaching sessions on SSP if SSP detection rates are low.²⁶ This study has shown a modest correlation of SSP detection with ADR similar to other studies; however, even for endoscopists with a high ADR, there can be a wide variation in SPPDR.²⁷ In our study, most endoscopists had an acceptable SSPDR (particularly more recent years) and the highest adenoma detectors also had high SSPDRs (table 1).

Adenomas per positive procedure (APP) is a difference measure as this was not correlated with ADR. This metric could assess the behaviour of ‘one and done’ where there is no incentive to look for another adenoma if the audit requirements are fulfilled by finding only one polyp. ADRplus as another measure that could potentially assess ‘one and done’ behaviour. Wang et al defined this metric as the mean additional number of adenomas for procedures where >1 adenoma was detected.¹⁰ The authors compared endoscopists from teaching and non-teaching hospitals and considered this was useful in addition to ADR. In another study of 25 324 screening colonoscopy (>50 years) involving 69 colonoscopists from 2009 to 2014, the authors found only five endoscopists with ADRs≥20% and low ADR-plus values. No endoscopist with an ADRs≥25% had low ADRplus values.¹¹ Overall, the data from this study and other studies would suggest that ‘one and done’ is not a common problem. ADRplus is a potentially confusing metric and variations on the method of calculation have led to different ranges and no certainty regarding what level might be considered an adequate benchmark.^{10 11}

One possible surrogate marker for the risk of postcolonoscopy colorectal cancer is the adenoma miss rate on tandem colonoscopy.^{28 29} In a meta-analysis of 43 publications and more than 15 000 tandem colonoscopies, the authors calculated miss rates of 26% for adenomas (95% CI 23% to 30%), 9% for advanced adenomas (95% CI 4% to 16%) and 27% for serrated polyps (95% CI 16% to 40%). The ADR, mean number of adenomas and APP were all independently associated with adenoma miss rate (AMR). An APP value greater than 1.8 was more effective in predicting AMR than an ADR value of at least 34%.²⁸

Withdrawal time is still a useful measure for an audit. This study shows that withdrawal time correlates with both PDR and ADR. Interestingly, there was no correlation with SSPDR suggesting that the detection of SSPs is not just a factor of time but probably also involves improved visual or pattern recognition. A study of 76 810 screening colonoscopies performed between 2004 and 2009 by 51 gastroenterologists showed that longer mean withdrawal times were associated with higher ADRs (3.6% per minute) and a decreased risk of interval colorectal cancer.³⁰

The important question is whether the audit process does really change performance? There may have been a general improvement in quality that would have occurred without the audit process. In our unit there have been regular upgrades of endoscopy equipment over the years with high definition Pentax equipment acquired in 2012 with iscan. There has been no use of end-of-scope devices such as caps. There were no major training days apart from the individual learning of endoscopists at international meetings and workshops.

Many other studies have confirmed the beneficial effect of colonoscopy audit although the duration of these studies has been relatively short. The introduction of a quarterly report card over 2 years from 2009 to 2011 lead to an ADR increasing from 44.7% to 53.9%. There were no significant changes in serrated polyp detection suggesting that specific learning tasks may be required.³¹ A Polish study showed that ADR can be improved (2004–2008) by annual feedback with comparison to quality benchmark indicators. The proportion of endoscopists with an ADR>25% increased from 8.1% in 2004 to 31.0% in 2008.³² A randomised single centre study of intervention with teaching and audit showed improvement in ADR from 36% to 47% whereas the control group was unchanged.³³ The mean number of adenomas per procedure also increased. The intervention group had two teaching sessions and private monthly feedback on their ADR, group averages and scores from other deidentified individuals. The whole group was followed up 5 months after the completion of the study and the improvement in the intervention group was maintained. A subsequent large multicentre trial of the same intervention showed a significant increase in ADR at the training sites (ADR of 31% at baseline and 42% after the intervention). However, ADR also increased at the control sites (from 36% to 39%); therefore, there was only limited evidence of a training effect.³⁴ A randomised trial from Dutch hospitals showed that two 45 min training sessions improved the detection rate for SSPs from 9.3% to 15.6%³⁵ A meta-analysis of 12 intervention studies (1 randomised controlled trial (RCT) and 11 non-RCTs involving 154 endoscopists and 33 184 colonoscopies) compared ADR prefeedback and postfeedback interventions.³⁶ The pooled ADR at baseline was 30.5% and 36% after intervention. There was a considerable degree of heterogeneity between the 12 included studies and not all studies showed improvement after intervention. This study is consistent with the conclusion from the meta-analysis that audit and feedback has a modest effect on improving ADR. There are minimal data on the trend for performance indicators if the audit is discontinued.³⁷

The only true test of intervention will be showing a reduction in postcolonoscopy rates of colorectal cancer. A recently reported large retrospective study has shown that after initiation of a quality improvement programme (regular feedback, educational meetings and benchmarking standards), the overall rate of interval colorectal cancer decreased from 0.15% to 0.08% (analysis limited to 10 endoscopists with more 5000 procedures).³⁸ These data support the routine measurement of ADR but there is also a strong argument for the inclusion of SSP detection rate. The audit process drives quality but there may be a plateau of KPI unless other specific training interventions are considered.