Public reporting of colonoscopy quality is associated with an increase in endoscopist adenoma detection rate

Abstract

Background

Colonoscopy is the predominant method for colorectal cancer screening in the US. Prior studies have documented variation across physicians in colonoscopy quality as measured by the adenoma detection rate (ADR). ADR is the primary quality measure of colonoscopy exams and an indicator of the likelihood of subsequent patient colorectal cancer. There is interest in mechanisms to improve ADR. In Central Illinois, a local employer and a quality improvement organization partnered to publically report physician colonoscopy quality.

Objective

To assess whether this initiative was associated with an improvement in ADR.

Design

This study compares ADR before and after public reporting at a private practice endoscopy center of 11 gastroenterologists in Peoria, Illinois who participated in the initiative. To generate ADR, colonoscopy and pathology reports from exams performed over four years at the endoscopy center were analyzed using previously validated natural language processing software.

Setting

Central Illinois Endoscopy Center

Results

The ADR for colonoscopy in the pre-public reporting era was 25.1%, and after public reporting was 36.4% (increase of 11.3%, p<0.001). Detection of advanced adenomas increased from 10.0% to 12.7% (p<0.001). Each physician’s ADR increased (range of 4.3% to 17.4%). Similar increases in ADR were observed when the analysis was restricted to screening colonoscopy.

Limitation

There was no concurrent control group to assess whether the increased ADR was due to a secular trend.

Conclusion

A public reporting initiative on colonoscopy quality was associated with a relative forty-five percent increase in ADR and a 25% increase in advanced adenoma detection. Public reporting may be a means to improve colonoscopy quality.

Introduction

Through the diagnosis and removal of premalignant colonic lesions, endoscopic screening significantly reduces colorectal cancer (CRC) incidence and mortality; however, the quality of endoscopic testing varies.^1–3 The adenoma detection rate (ADR), a key marker of colonoscopy quality, can vary 3-fold across endoscopists.^4–6 The lower a physician’s ADR, the greater the risk of his patients receiving a diagnosis of interval colon cancer, cancer detected after colonoscopy but before the next examination is due.^{5, 7}

Because of the direct link between ADR and subsequent cancer incidence, there is interest in interventions that can increase the detection rate of premalignant lesions.⁸ A recent systematic review evaluated interventions such as increasing endoscopic withdrawal time, enhanced segmental withdrawal, and provider feedback on improving quality. The review concluded that existing interventions have generally been ineffective or inconsistent in improving detection of premalignant lesions. For example, giving feedback to physicians on their polyp detection rates has had mixed impact.^{9, 10}

Although it is being used widely in health care, public reporting has not yet been studied as an intervention to improve colonoscopy quality.¹¹ In New York State, surgeon-specific mortality rates after cardiovascular surgery have been reported since 1997. Hospital quality is reported publicly on Medicare’s Hospital Compare website and Medicare is moving toward more widespread physician public reporting using its Physician Compare website.¹² Cardiology societies have begun studying the feasibility of public reporting using clinical data from the National Cardiovascular Database Registry.¹³ In the field of gastroenterology, public reporting could be based on national registries such as GI Quality Improvement Consortium (GIQuIC) or the National Endoscopic Database, which both include millions of colonoscopy procedures.^{14, 15} Given the growing use of public reporting in health care and interest in interventions to improve quality through ADR, the goal of this study was to assess whether public reporting was associated with improved colonoscopy quality.

Methods

Setting

Quality Quest for Health of Illinois (http://www.qualityquest.org) is a non-profit healthcare collaborative of providers, employers, and health plans that share the goal of improving quality of care in Central Illinois. One Quality Quest initiative focused on colonoscopy quality. A team of gastroenterologists, pathologists, and surgeons created a global colonoscopy quality measure that included the following evaluation criteria: (1) timing and interval of the colonoscopy; (2) serious procedural adverse events such as perforation, hospitalization, or bleeding requiring blood transfusion; report of the patient’s risk; (3) report of the quality of the bowel preparation; (4) completion of the procedure with appropriate photo documentation of landmarks; (5) complete information provided to the pathologist when specimens are sent for evaluation; (6) time spent in withdrawal or examining the colon; (7) appropriate recommendation for time of a follow-up colonoscopy. Adenoma detection rate was collected and reported to Quality Quest with the expectation that it would eventually be included in the public report. Surgeons and gastroenterologists in Peoria and Decatur, IL hand-collected the necessary data elements from their records and submitted the above data to establish a report of colonoscopy quality: http://www.qualityquest.org/quality-reports/colonoscopies/index.php. Physicians are listed by name in descending order of performance based on the overall colonoscopy quality index that combines the quality measures into a single number. The public reporting initiative began in August 2010. Twenty-eight endoscopists were included in the initiative whereas 9 others were excluded due to insufficient volume (<30 cases/year). Caterpillar, the major employer in the community, mandated endoscopist participation in the initiative as a condition for being included in the provider network for its employees. Among those participating were physicians at Central Illinois Endoscopy, a private practice of 11 gastroenterologists. The physicians are general gastroenterologists whose primary focus is clinical care. During the study period, the endoscopy center used high-definition endoscopic equipment.

Data

The data used in the public reporting initiative was not used in the analyses. Instead, in partnership with Central Illinois Endoscopy, all outpatient colonoscopy and pathology reports performed between July 2009 and May 2013 were independently analyzed by the research team. Colonoscopy was performed in a private endoscopy center. The practice uses an electronic health record (ProVation, Minneapolis, Minn), and the colonoscopy reports are a combination of structured reports and free text entered by the physicians. The electronic health record was implemented in July 2009.

Abstracting Necessary Data Elements from Reports

This study is a retrospective review of 20,040 colonoscopy and associated pathology reports. All reports were de-identified and transferred electronically to the Department of Biomedical Informatics at the University of Pittsburgh. Relevant data from the reports was abstracted using a previously developed Natural Language Processing (NLP) computer-based software application. Details of the development and testing of this tool are provided in more depth elsewhere.^{6, 16–18} In brief, NLP is a field of computer science through which programs can “read” language and pull out key variables. The accuracy of the NLP program was confirmed by comparing a set of 453 reports analyzed by the NLP program and manually abstracted by a group of physicians. Key variables identified included colonoscopy indication, quality of bowel preparation, removal of polyps, size of polyps, and presence of adenomas. Adenomas were identified in the pathology report and advanced adenomas were those with villous component, high-grade dysplasia, or were 10mm or greater in size. Using the Kappa statistic the agreement between physicians and the NLP program for the ADR measure was 0.72, a rate typically considered to indicate good agreement.¹⁸

Indication was broken down into 3 categories: (1) Screening (patients with no history of prior colonoscopy, inflammatory bowel disease or family history of colon cancer), (2) High-risk screening or surveillance (patients with history of prior polyps or family history of colon cancer), (3) Other (all other colonoscopies performed for symptoms such as GI bleeding or abdominal pain or history of inflammatory bowel disease). Up to 3 indications for the colonoscopy were captured. If the endoscopist reported the indication for colonoscopy as being both for screening and to address symptoms, it was defined as a screening colonoscopy.

Data Analysis

The primary aim was to compare colonoscopy quality in the period before and after the public reporting initiative of August 2010. Data in the 3 months before and the 3 months after public reporting (May 2010–October 2010) were excluded in the main analysis as this was a period of transition. In sensitivity analyses reported in the Appendix, this 6-month transition period was not excluded and results were found to be similar. ADR was calculated for all colonoscopy and separately for screening colonoscopy. Two of the 11 gastroenterologists joined the practice after the group’s involvement in the Quality Quest initiative. For the other 9 physicians, each physician’s pre- and post-period ADRs were compared. The data was analyzed using IBM SPSS (version 20.0, Armonk, NY). Univariate statistical analyses were performed to describe the study population. Student t tests and chi-square analyses were done to determine differences in continuous and categorical variables, respectively. The study protocol was reviewed and approved by the Institutional Review Boards of the University of Pittsburgh Medical Center and Harvard Medical School.

Results

A total of 20,040 colonoscopies were performed between July 2009 and May 2013, for an average of 426 reports per month. After the exclusion of a 6-month transition period, 17,526 reports were included in the main analysis.

Patient and procedure characteristics were compared between the pre- and post-public reporting period (Table 1). There was a greater percentage of patients aged 50 to 59 years within the post-public reporting period compared to the pre-public reporting period (37.6% vs. 32.6%, p<0.001 Screening colonoscopies were more common in the post-public reporting period (39.3% vs. 26.6%, p<0.001). A family history of colorectal cancer was reported in 8.5% of patients overall, with a higher rate in the post-public reporting cohort (8.7% vs 7.3%, p=0.023). The adequacy of colonoscopy preparation was also higher post-public reporting (96.3% vs 93.6%, p<0.001) with a lower rate of incomplete procedures (5.7% vs 6.8%, p=0.039).

Table 1.

Patient and procedure characteristics in pre and post reporting periods

	All Colonoscopies (n=17,526)	Colonoscopies before Public Reporting (July ‘09–April ‘10) (n=2,627)	Colonoscopies after Public Reporting (Nov ‘10–May ‘13) (n=14,899)	P value
Age of patient†	59.4 (SD 11.9)	60.2 (SD 12.1)	59.2 (SD 11.9)	<0.001
<40 years	1,007 (5.8%)	144 (5.6%)	863 (5.8%)	0.528
40–49 years	1,307 (7.5%)	209 (8.2%)	1,098 (7.4%)	0.292
50–59 years	6,388 (36.9%)	832 (32.6%)	5,556 (37.6%)	<0.001
60–69 years	5,223 (30.2%)	777 (30.5%)	4,446 (30.1%)	0.785
>70 years	3,398 (19.6%)	587 (23%)	3,811 (19%)	<0.001
Gender of patient *
Male	7,312 (46.9%)	1,191 (45.4%)	6,121 (47.2%)	0.104
Female	8,289 (53.1%)	1,431 (54.6%)	6,858 (52.8%)
Indication for colonoscopy
Screening	6,557 (37.4%)	699 (26.6%)	5,858 (39.3%)	<0.001
Surveillance	3,808 (27.4%)	838 (31.9%)	3,970 (26.6%)	<0.001
Other indications	6,161 (35.2%)	1,090 (41.5%)	5,071 (34.0%)	<0.001
Family history of colorectal cancer reported
Yes	1,487 (8.5%)	193 (7.3%)	1,294 (8.7%)	0.023
No	16,039 (91.5%)	2,434 (92.7%)	13,605 (91.3%)
Colonoscopy completed
Prep Adequate	16,814 (95.9%)	2,460 (93.6%)	14,354 (96.3%)	<0.001
Incomplete procedures	1,034 (5.9%)	178 (6.8%)	856 (5.7%)	0.039

^*1925 reports were missing the gender variable so calculation of gender frequencies were based on 15601 reports (2622 before PR and 12979 after PR)

^†203 reports were missing the age variable so mean age was calculated based on the remaining 17323 reports (2549 before PR and 14774 after PR)

The ADR for all colonoscopies was higher in the post-public reporting period compared to the pre-public reporting period (36.4% vs 25.1%, p<0.001) (Table 2). This difference was more pronounced with colonoscopies done for screening (37.4% vs 23.6%, p<0.001). ADR was higher in the post-public reporting period among both women (25.5% pre-period to 37.3% post period) and men (34.6% to 50.5%). The advanced adenoma detection rate and adenoma-to-polyp ratio were also significantly higher in the post-public reporting group compared with the pre-public reporting group for all colonoscopy and for screening colonoscopy (Table 2, Figure 1). All 9 gastroenterologists who practiced in both time periods had an increase in their ADRs post-public reporting (range of increase 4.3% to 17.4%) (Figure 2).

Table 2.

Adenoma detection rate by colonoscopy type in pre and post reporting periods

Quality measures	All Colonoscopies (n=17,526)	Colonoscopies before Public Reporting (July ‘09–April ‘10) (n=2,627)	Colonoscopies after Public Reporting (Nov ‘10–May ‘13) (n=14,899)
All Colonoscopies
Polyp removed during colonoscopy	9,299 (53.1%)	1,227 (46.7%)	8,072 (54.2%)
Adenoma removed during colonoscopy	6,084 (34.7%)	660 (25.1%)	5,424 (36.4%)
Advanced adenoma removed^	2,151 (12.3%)	264 (10.0%)	1,887 (12.7%)
Large adenoma removed	1,884 (10.7%)	226 (8.6%)	1,658 (11.1%)
Villous, dysplastic changes, or carcinoma	668 (3.8%)	95 (3.6%)	573 (3.8%)
Adenoma to polyp ratio	0.65	0.54	0.67
Limited to Screening Colonoscopies
Polyp removed during colonoscopy	3,751 (57.2%)	348 (49.8%)	3,403 (58.1%)
Adenoma removed during colonoscopy	2,353 (35.9%)	165 (23.6%)	2,188 (37.4%)
Advanced adenoma removed^	730 (11.1%)	45 (6.4%)	685 (11.7%)
Large adenoma removed	661 (10.1%)	42 (6.0%)	619 (10.6%)
Villous, dysplastic changes, or carcinoma	196 (3.0%)	14 (2.0%)	182 (3.1%)
Adenoma to polyp ratio	0.63	0.47	0.64
Other Quality Measures
Withdrawal time reported	2,954 (16.9%)	8 (0.3%)	2,946 (19.8%)
ASA classification of patient reported	11,239 (64.1%)	1 (0%)	11,238 (75.4%)
Informed consent documented	17,525 (100%)	2,627 (100%)	14,898 (100%)
Quality of colonoscopy prep reported	17,526 (100%)	2,627 (100%)	14,899 (100%)

^{^}Adenoma size ≥10mm or pathology consistent with villous component or dysplastic changes

Figure 1.

Polyp, adenoma and advanced adenoma detection rates in pre and post reporting periods

Figure 2.

ADR by physician in pre and post reporting periods

On other quality measures, a marked increase was noted in documentation of endoscope withdrawal time (19.8% vs. 0.3%, p<0.001) and patient ASA classification (75.4% vs. 0%, p<0.001) in the post-public reporting time period.

Discussion

Public reporting as a quality improvement intervention is new to gastroenterology but is used widely in other areas of healthcare. In August 2010, a local quality improvement organization in Central Illinois began publicly reporting the colonoscopy quality of individual physicians. To investigate the effect of this initiative on the quality of colonoscopy, we compared the ADR in a period before and after a single practice’s participation in a public reporting initiative. The ADR for all colonoscopies was 45% higher in the post-public reporting period (36.4% vs 25.1%) and advanced adenoma detection increased by almost twenty-five percent (12.7% vs 10%). The ADR for each gastroenterologist was higher after public reporting.

Three potential pathways by which public reporting can motivate health care providers to improve quality have been proposed: the selection pathway, change pathway, and reputation pathway.¹⁶ In the selection pathway, providers are motivated to improve quality because of concern that patients will use the information to select new providers, causing a decrease in their patient volume. In the change pathway, simply identifying deficiencies motivates providers to change, and it does not matter whether the feedback is offered privately or publicly. In the reputation pathway, providers are motivated by concerns about a diminished reputation among peers and negative public opinion.¹¹ Although these pathways are not mutually exclusive, prior work has hinted that the reputational pathway might be most important. One study compared quality improvement among hospitals receiving public versus private feedback and found public reporting had a larger effect on improving hospital performance.¹⁹ The implication is that the hospitals were motivated by concerns about patient choice or reputation. However, contrary to expectations, data suggest that patients do not use quality information to choose their providers,¹¹ including colonoscopy.²⁰ Concerns about reputation may explain why we found a larger increase in colonoscopy quality with public reporting compared to previous research that evaluated the impact of private feedback.²¹

Multiple factors likely contribute to the observed improvement in quality. The practice began a number of initiatives in response to public reporting. In an effort to improve bowel preparation, the center changed to a split dose preparation and set an internal goal that 90% of patients have good or excellent bowel preparation.^{22, 23} Internally within the practice, physicians received monthly feedback on their ADR, and other quality measures and performance were discussed at regular practice meetings.

Though public reporting is becoming more widespread, its impact on quality remains unclear. In a recent systematic review of public reporting in health care, the authors noted many studies either having a small impact or being ineffective in improving quality.¹⁷ One potential explanation for this mixed impact is that to motivate behavior change, physicians must accept the quality measure and trust the methodology.²⁴ In many of the prior studies on public reporting, the studies did not use clinically accepted measures.

Our study has several key limitations. We did not have a control group of gastroenterologists who did not participate in the public reporting initiative. Therefore we cannot assess whether the increases in ADR and advanced adenoma detection we observed were due to a generalized temporal trend or were due to the public reporting initiative. The timing of quality improvement we observed, however, correlates to the onset of public reporting. It is possible that differences in the patient populations pre- and post-public reporting account for some of the observed differences. Yet, when we limited our analyses to a more homogenous set of procedures, screening colonoscopy, the differences were even larger. Because a new electronic health record was introduced, we were only able to capture nine months of data before public reporting, a limited time period to characterize quality before public reporting. It is possible that the gastroenterologists responded to the public reporting initiative by changing their patterns of documentation and this change could explain the improvement we see in outcomes such as adequacy of bowl preparation. However, ADR was the major outcome and reporting bias is unlikely to play a role with this outcome as the presence of an adenoma was assessed by an outside pathologist. Our study included gastroenterologists within a single private practice and may not be generalizable to other practice settings in different regions. Also, the willingness of this practice to analyze and subject its data to scrutiny might signal an interest in quality improvement whereas other practices may not have responded similarly.

In summary, a public reporting initiative focused on colonoscopy quality was associated with a 45% relative increase in the ADR and a twenty-five percent increase in advanced adenoma detection. Public reporting may be a means to improve colonoscopy quality.

Glossary

ADR

adenoma detection rate

NLP

natural language processing

Appendix Material

Appendix Table 1.

Adenoma detection rate by colonoscopy type in pre and post reporting periods. Sensitivity analyses where colonoscopies from transition period were not excluded. This analysis was conducted to assess whether results were sensitive to exclusion of transition period.

Quality measures	All Colonoscopies (n=20,040)	Colonoscopies before Public Reporting (n=3,952)	Colonoscopies after Public Reporting (n=16,088)	P value
All Colonoscopies
Polyp removed during colonoscopy	10,548 (52.6%)	1,902 (48.1%)	8,646 (53.7%)	<0.001
Adenoma removed during colonoscopy	6,905 (34.5%)	1,092 (27.6%)	5,813 (36.1%)	<0.001
Advanced adenoma removed^	2,448 (12.2%)	415 (10.5%)	2,033 (12.6%)	<0.001
Large adenoma removed	2,143 (10.7%)	355 (9.0%)	1,788 (11.1%)	<0.001
Villous, dysplastic changes, or carcinoma	772 (3.9%)	149 (3.8%)	623 (3.9%)	0.765
Adenoma to polyp ratio	0.65	0.57	0.67	<0.001
Limited to Screening Colonoscopies
Polyp removed during colonoscopy	4,178 (57.0%)	568 (52.5%)	3,610 (57.7%)	0.001
Adenoma removed during colonoscopy	2,633 (35.9%)	306 (28.3%)	2,327 (37.2%)	<0.001
Advanced adenoma removed^	819 (11.2%)	89 (8.2%)	730 (11.7%)	0.001
Large adenoma removed	735 (10.0%)	77 (7.1%)	658 (10.5%)	0.001
Villous, dysplastic changes, or carcinoma	230 (3.1%)	30 (2.8%)	200 (3.2%)	0.459
Adenoma to polyp ratio	0.63	0.54	0.64	<0.001
Other Quality Measures
Withdrawal time reported	3,296 (16.4%)	62 (1.6%)	3,234 (20.1%)	<0.001
ASA classification of patient reported	11,253 (56.2%)	3 (0.1%)	11,250 (69.9%)	<0.001
Informed consent documented	20,037 (100%)	3,950 (99.9%)	16,087 (100%)	0.041
Quality of colonoscopy prep reported	20,040 (100%)	3,952 (100%)	16,088 (100%)	NS

^{^}Adenoma size ≥ 10mm or pathology consistent with villous component or dysplastic changes

Appendix Table 2.

Polypectomy rate stratified by size of polyp. This analysis was performed to assess whether increase in polyp detection and adenoma detection was driven primarily by physicians finding small polyps.

	All Colonoscopies (n=17,526)	Colonoscopies before Public Reporting (July ‘09–April ‘10) (n=2,627)	Colonoscopies after Public Reporting (Nov ‘10–May ‘13) (n=14,899)	P value
All polyp sizes	9,299 (53.1%)	1,227 (46.7%)	8,072 (54.2%)	<0.001
Polyp size 1–3 mm	1387 (7.9%)	175 (6.7%)	1212 (8.1%)	0.01
Polyp size 4–6 mm	3756 (21.4%)	425 (16.2%)	3331 (22.4%)	<0.001
Polyp size 7–9 mm	1395 (8.0%)	208 (7.9%)	1187 (8.0%)	0.932
Polyp size 10 mm or greater	1,905 (10.9%)	299 (11.4%)	1606 (10.8%)	<0.001
Size not reported	856 (4.9%)	120 (4.6%)	736 (4.9%)	0.415