Variation in Pathologist Classification of Colorectal Adenomas and Serrated Polyps

Abstract

Objectives

Endoscopist quality measures such as adenoma detection rate (ADR) and serrated polyp detection rates (SPDR) depend on pathologist classification of histology. While variation in pathologic interpretation is recognized, we add to the literature by quantifying the impact of pathologic variability on endoscopist performance.

Methods

We used natural language processing to abstract relevant data from colonoscopy and related pathology reports performed over two years at four clinical sites. We quantified each pathologist’s likelihood of classifying polyp specimens as adenomas or serrated polyps. We estimated the impact on endoscopists’ ADR and SPDR of sending their specimens to pathologists with higher or lower classification rates.

Results

We observed 85,526 colonoscopies performed by 119 endoscopists; 50,453 had a polyp specimen, which were analyzed by 48 pathologists. There was greater variation across pathologists in classification of serrated polyps than in classification of adenomas. We estimate the endoscopist’s average SPDR would be 0.5% if all their specimens were analyzed by the pathologist in our sample with the lowest classification rate, and 12.0% if all their specimens were analyzed by the pathologist with the highest classification rate. In contrast, the endoscopist’s average ADR would be 28.5% and 42.4% if their specimens were analyzed by the pathologist with lowest and highest classification rate, respectively.

Conclusions

There is significant variation in pathologic interpretation which more substantially affects endoscopist SPDR than ADR.

Introduction

Randomized trials have demonstrated that endoscopic screening reduces incidence and mortality from colorectal cancer.(1–4) Colonoscopy is the primary form of endoscopic screening; over 14 million colonoscopies are performed annually in the US.(5) Identifying and removing both adenomatous and serrated polyps during colonoscopy is important, as each are precursors of colorectal cancer.(6–10) Physicians vary in how often they detect pre-cancerous lesions(6, 11–14) and a lower physician adenoma detection rate (ADR) is associated with a higher subsequent rate of interval colorectal cancer.(10, 11, 15) Given this variation, specialty societies and experts have called for physicians to regularly monitor their performance on ADR (16, 17) and, increasingly, experts have called for monitoring serrated polyp detection rate (SPDR) as well.(18, 19)

The classification of polyps as adenomatous or serrated is done by a pathologist, so an endoscopist’s performance on ADR and SPDR is dependent on accurate and consistent pathologist classification. Previous studies have raised concern about variation in polyp classification across pathologists.(10, 12, 13, 20) For example, one study showed the same 20 images of adenomatous and serrated polyps to 168 pathologists and found substantial variability in classification.(21) Disagreement across pathologists may be particularly evident in classification of serrated polyps.(13, 14) One study compared classification of serrated polyps across 20 pathologists and found that inter-pathologist reliability in how polyps were classified was low.(20) A number of studies have reported a substantial proportion of polyps previously read as hyperplastic were reclassified as sessile serrated adenomas/polyps upon secondary review.(22–24)

While these studies highlight variation across pathologists primarily in test specimens, there has been limited prior research on variation of pathologists in real world practice and the impact of this variation on endoscopist quality performance. We used a large multi-site sample of colonoscopy and associated pathology reports to examine the variation in pathologist classification of conventional adenomas and serrated polyps, controlling for the endoscopist who performed the colonoscopy. We use these findings to illustrate how an endoscopist’s ADR and SPDR could be affected by variation in pathologic classification of polyps.

Methods

Sample of colonoscopies

Our dataset included colonoscopies between October 1, 2013 and September 30, 2015 at four clinical sites across the United States. The clinical sites were selected to vary in geographic location and structure of financial incentives related to colonoscopy performance. UPMC is a large mixed academic/non-academic health system based in Western Pennsylvania; The University of North Carolina (UNC) is an academic center; Central Illinois Endoscopy (CIE) is a private colonoscopy center in Peoria, Illinois; Kaiser Permanente Washington (KPW, formerly Group Health Cooperative) is a staff-model health maintenance organization in Washington State.

We excluded inpatient colonoscopies, those done on patients under age 40 and patients with inflammatory bowel disease, as these cases likely do not represent screening for colorectal cancer. We chose 40 years as the cutoff instead of 50 to ensure we captured colonoscopies that were performed for colorectal cancer screening among those with a family history. To ensure sufficient sample size for our analyses we excluded colonoscopies performed by low-volume endoscopists (<100 colonoscopies with a polyp removed over two years, n=3,068 excluded) or where the pathology specimen was assessed by a low-volume pathologist (<200 colonoscopies with a polyp removed and an associated pathology report over two years, n=509 excluded). We conducted sensitivity analyses using two alternative volume cutoffs to ensure that our results were robust to the chosen volume threshold (Appendix 1).

After these exclusion criteria, our sample included colonoscopy and pathology reports from 91 endoscopists and 32 pathologists from UPMC, 33 endoscopists and 5 pathologists from UNC, 11 endoscopists and 3 pathologists from CIE, and 17 endoscopists and 8 pathologists from KPW.

Analysis of colonoscopy and pathology reports

We used a natural language processing (NLP) computer program to collect the relevant data from the colonoscopy and associated pathology reports. NLP is a method in which a computer program “reads” free text, automating the manual abstracting otherwise required to extract needed data from colonoscopy and pathology reports. NLP abstraction has been used in a number of studies to assess colonoscopy quality.(25–27) Details about the NLP program used in the present study and its performance compared to human abstractors have been previously published.(25, 28, 29) The NLP program was highly accurate in adenoma or serrated polyp identification (accuracy > 95% for both outcomes).(29)

For each colonoscopy report, the NLP program identified the endoscopist and whether a polyp was removed. It also classified up to three indications for the colonoscopy. If a procedure had more than one indication, it was classified based on the following hierarchy: inflammatory bowel disease, screening without a family history of colorectal cancer, surveillance, screening with a family history of colorectal cancer, and diagnostic. We structured the hierarchy because, in our review of many colonoscopy reports, often incidental indications such as abdominal pain or diarrhea were added to colonoscopies that were primarily for screening purposes. Using this hierarchy, if the indications for a colonoscopy were screening and abdominal pain, it was classified as a screening colonoscopy. We then used patient and date identifiers to link each colonoscopy report to its corresponding pathology report. In the cases where there were multiple pathology reports corresponding to a single colonoscopy, we retained the pathology report with the latest date, as that was most likely to reflect the full results of the pathologic analysis.

For each associated pathology report, the NLP program identified which pathologist analyzed the specimens. For each specimen, the NLP program abstracted the location, whether there was an adenoma, and whether there was a polyp that was read as “serrated.” While we recognize that hyperplastic polyps are also in the family of serrated polyps according to WHO definitions,(30) our NLP program did not include hyperplastic polyps in the definition of serrated polyps. Current guidelines(31) are ambiguous on what endoscopists should do with proximal hyperplastic polyps, but do advocate for early follow-up after sessile serrated adenomas/polyps. Therefore, the distinction between hyperplastic polyps and sessile serrated adenomas/polyps is of critical clinical importance.(31) For the purposes of this analysis, we also did not distinguish between sessile serrated adenomas/polyps and traditional serrated adenomas because prior literature indicates that the majority of polyps read as serrated are sessile serrated adenomas/polyps. In addition, the terminology used in pathology reports across the sites to describe serrated polyps was inconsistent (see Appendix 2), so would not have allowed for more specific categorization of types of serrated adenomas.

We categorized location of the polyp into three segments: proximal colon (proximal to the splenic flexure, >50 cm from anal verge); distal colon (end of the sigmoid colon to the splenic flexure, 35-50 cm from anal verge); and rectosigmoid (sigmoid colon and rectum, ≤35 cm from anal verge). If an adenoma was identified in both a proximal and distal polyps from a given colonoscopy then it was included in both proximal and distal ADR. We chose to distinguish between polyps in the rectosigmoid colon vs. more proximal left colon (descending colon and splenic flexure) given that small rectosigmoid serrated polyps are considered to be lower risk.(32, 33) Furthermore, because hyperplastic polyps are far more prevalent than sessile serrated adenomas/polyps in the rectosigmoid colon,(34) we wanted to examine whether pathologists differed with respect to identifying serrated polyps in this location. As a sensitivity analysis, we considered an alternative location classification in which we separately classified serrated polyps found in the rectum. In this alternative, we divided the colon into three segments: proximal colon (proximal to the splenic flexure, >50 cm from anal verge); distal colon (end of the rectum to the splenic flexure, including the sigmoid colon, 15-50 cm from anal verge); and rectum (rectum only, ≤15 cm from anal verge).

Our unit of analysis was the colonoscopy (rather than specimen bottle) given this is the unit of analysis for endoscopist quality measurement. For simplicity we use the phrasing “number of reports analyzed” to describe the number of colonoscopy exams for which the pathologist analyzed the removed polyps, recognizing that this is a bit of a misnomer as the pathologist does not analyze the report, but rather individual pathologic specimens. All pathology reports in the sample were associated with colonoscopies in which at least one polyp was removed. In the cases where a patient received multiple colonoscopies, all such colonoscopies would be included in the sample.

Measures of pathologist classification

For each pathologist, we calculated an “observed classification rate” for adenomas and serrated polyps. Using adenomas as an example, the observed classification rate was calculated as the number of pathology reports in which the pathologist classified a polyp as an adenoma divided by the total number of pathology reports with a polyp the pathologist analyzed. For example, if a pathologist analyzed 1,000 pathology reports and classified 500 specimens as adenomas, their observed classification rate would be 50%.

In assessing variation across pathologists we also account for variation across endoscopists in identifying pathologic polyps. Some endoscopists are more likely to remove polyps that are adenomas or serrated polyps than others and this variation might impact the pathologists’ classification rate. For example, at the extreme, one endoscopist might remove many “polyps” that are normal tissue. A pathologist classifying those polyps would have a lower adenoma classification rate than a pathologist classifying polyps for an endoscopist who is more consistent in only removing pre-cancerous polyps.

We controlled for endoscopist variation in polyp removal in our calculation of each pathologist’s “expected classification rate.” This methodology builds upon methodology used to evaluate hospital and surgeon mortality and hospital readmissions,(35–38). Using adenomas as an example, a pathologist’s expected classification rate is the proportion of pathology reports analyzed in which we would expect there to be an adenoma, given the endoscopist who performed the colonoscopy. To calculate the expected classification rate, we fit a logistic regression model where the unit of analysis was each colonoscopy with a polyp removed; the outcome was a binary indicator for whether an adenoma was detected. The only predictor variables were endoscopist fixed effects (i.e., a binary yes/no variable for each of the 119 endoscopists) in our sample. The coefficients on each of these endoscopist fixed effects are an estimate of the underlying fraction of all polyps sent to any pathologist that are judged to be an adenoma.

We then use these endoscopist fixed effects to calculate a pathologist’s expected classification rate. The expected classification rate then represents, on average, the proportion of the pathology reports analyzed by the pathologist in which we would expect there to be an adenoma, given the endoscopists performing the colonoscopies. In Appendix 3 we demonstrate how much of the variation across pathologists is accounted for by this adjustment and provide some concrete examples of the expected and observed adenoma classification rates for pathologists. We repeated this procedure for serrated polyps as well as distal, proximal and rectosigmoid serrated polyps.

To compare classification of polyps across pathologists, we calculated standardized risk ratios – the ratio of the pathologist’s observed to expected classification rate for each outcome.(39) For example, if a pathologist’s standardized adenoma classification ratio is greater than one, she was more likely to classify a polyp as an adenoma than our data would predict based only on the endoscopists for whom she analyzed pathology reports.

We chose not to adjust for patient characteristics and colonoscopy indication because, while patient characteristics clearly impact the likelihood of identifying an adenoma during the colonoscopy, it is less clear that they impact whether a polyp is classified as an adenoma by a pathologist. We conducted a sensitivity analysis to assess whether adjusting for patient and colonoscopy factors would affect our measurement of pathologist classification rate. This analysis included patient age, gender and colonoscopy indication in generating the “expected classification rate” (Appendix 4) and found these patient characteristics had little impact on overall variation in pathologist classification of these outcomes.

Physician characteristics

To characterize the age and years of practice of the pathologists and endoscopists, we linked our dataset with data from Doximity. Doximity integrates information on physicians from the National Plan and Provider Enumeration System, National Provider Identifier Registry, and reports from self-registered members and collaborating hospitals and medical schools. This database has been used for other studies of the physician workforce.^35-37 Years of practice was measured between 2014 and the year of residency completion. We stratified pathologists into three tertiles of years of practice (<8 years, 8-17 years and 18-46 years). In consultation with physicians at each of the clinical sites, we identified whether each pathologist was specialized in gastroenterology. Pathologists were considered “GI pathologists” if they had either undergone formal subspecialty training in gastrointestinal pathology or if their work consisted mostly or entirely of analysis of gastrointestinal specimens.

Variation Analyses

We described the variation across pathologists’ standardized classification ratios using the coefficient of variation (COV), a measurement of variation calculated as the ratio of a variable’s standard deviation to its mean. Higher values of COV indicate greater variability or dispersion of data.

To put our findings in clinical context, we analyzed how an endoscopist’s serrated polyp detection rate (SPDR) and adenoma detection rate (ADR) would be different if all the polyps they removed were analyzed by the pathologist with the highest, lowest, and 25^th percentile or 75^th percentile standardized serrated polyp classification and adenoma classification, respectively (see Appendix 5 for details).

Results

There were 85,526 colonoscopies performed by 119 endoscopists in our sample, 50,453 of which had a polyp removed and an associated pathology report (Table 1). The colonoscopies were primarily conducted for screening (47%) or surveillance (32%). The patients were roughly equally divided by sex and 71% were between the ages of 50 and 69. An adenoma was detected in 35% of the colonoscopies and a serrated polyp was detected in 7%.

Table 1.

Characteristics of Pathology Reports (n=50,453)

	Colonoscopies		Pathology Reports
	n	%	n	%
Total	85,526	-	50,543	59.0
Site
UPMC	53,725	62.8	29,198	57.9
University of North Carolina	11,440	13.4	7,766	15.4
Central Illinois Endoscopy	11,117	13.0	6,896	13.7
Kaiser Permanente Washington	9,244	10.8	6,593	13.1
Patient Sex
Male	39,886	47.2	25,499	51.2
Female	44,546	52.8	24,283	48.8
Patient Age
Under 50	6,127	7.2	3,175	6.3
50-59	31,525	36.9	17,448	34.6
60-69	29,177	34.1	17,841	35.4
70+	18,622	27.8	11,953	23.7
Colonoscopy Indication
Screening	40,429	47.3	21,469	42.6
Surveillance	27,732	32.4	19,327	38.3
Diagnostic	15,536	18.2	8,755	17.4
Missing	1,829	2.1	902	1.8
Clinical outcomes
At least one adenoma detected	30,290	35.4	30,290	60.0
At least one serrated polyp detected	6,214	7.3	6,214	12.3
… in the distal colon	525	0.6	525	1.0
… in the rectosigmoid	743	0.9	743	1.5
… in the proximal colon	5,055	5.9	5,055	10.0

Notes:

(1) Patient sex and age categories may not sum to 100% due to missing data

There were 48 pathologists who analyzed the 50,453 pathology reports over the two-year period (Table 2). Over half (63%) of the pathologists were male. The median number of colonoscopy pathology reports analyzed by each pathologist in the two-year period was 997. The median number of years in practice for pathologists in the sample was 9. On average, the pathologists analyzed polyps removed by 17 different endoscopists (range 2 to 36), and endoscopists had their specimens analyzed by an average of 7 pathologists (range 2 to 18). Most pathologists were not specialized in gastrointestinal pathology (69%). Our sample has a similar gender balance to the national population of pathologists but has a smaller proportion of pathologists with 18 or more years of practice (Appendix 6).

Table 2.

Characteristics of Pathologists (n=48)

	n	%
Sex
Male	30	62.5
Female	18	37.5
Number of Pathology Reports Analyzed over 2-year Period
200-750	11	23.9
751-1000	13	28.3
1001-1250	13	28.3
1251-4434	9	19.6
Years in Practice
<8	16	33.3
8-17	17	35.4
18-46	15	31.3
Number of Endoscopists for whom the Pathologist Analyzed Specimens
<12	15	31.3
13-19	17	35.4
20-36	16	33.3
Pathologist with GI Experience
No	33	68.8
Yes	15	31.3

Notes:

(1) Years in practice is measured as the number of years since completion of residency, as of 2014

(2) The number of pathology reports is over a two-year period.

(3) GI experience was identified for each individual pathologist by local staff.

The 119 endoscopists in our sample were also mostly male (79%), and the median number of colonoscopies performed over the two-year period was 644 (Table 3). The average adenoma detection rate across the endoscopists was 36% and the average serrated polyp detection rate was 7%.

Table 3.

Characteristics of Endoscopists (n=119)

Endoscopist characteristics	n	%
Sex
Male	94	79.0
Female	25	21.0
Number of Colonoscopies Performed over 2-year Period
150-350	30	25.2
351-650	30	25.2
651-1150	30	25.2
1151-2654	29	24.4
Years in Practice
2-14	42	35.3
15-25	38	31.9
26-51	39	32.8
Endoscopist quality metrics	Average	Range
Adenoma detection rate	35.8%	(12.4%, 62.3%)
Serrated polyp detection rate	6.9%	(0.4%, 19.5%)

Notes:

(1) Years in practice is measured as the number of years since completion of residency, as of 2014

Variation in pathologist’s classification of polyps as adenomas or serrated polyps

We characterized variation across pathologists in their classification through the use of standardized classification ratios (see Appendix Table 7 for examples of calculations). There was more variation in pathologists’ standardized serrated polyp classification ratio (inter-quartile range (IQR) 0.81 to 1.12, COV 32.0) than standardized adenoma classification ratio (IQR 0.98 to 1.02, COV 3.7) (Figure 1). By location in the colon, there was less variation in pathologists’ standardized classification ratio for proximal serrated polyps (IQR 0.71 to 1.07, COV 30.9) compared to distal serrated polyps (IQR 0.40 to 1.14, COV 66.9) or rectosigmoid serrated polyps (IQR 0.34 to 1.46, COV 83.8) (Figure 2). This differential variation in serrated polyp classification by colonic location was consistent in a sensitivity analysis with alternative cutoffs for the colonic segments (Appendix 8).

Figure 1.

Pathologist variation in standardized adenoma classification ratio (left) and standardized serrated polyp classification ratio (right)

Notes:

(1) n=48 pathologists with 50,453 pathology reports.

(2) Path reports from physicians who performed fewer than 100 colonoscopies over the study period were excluded.

(3) Pathologists who analyze fewer than 200 pathology reports over the study period were excluded.

Figure 2.

Pathologist variation in standardized serrated polyp classification ratio by location of serrated polyp

Notes:

(1) n=48 pathologists with 50,453 pathology reports.

(2) Path reports from physicians who performed fewer than 100 colonoscopies over the study period were excluded.

(3) Pathologists who analyze fewer than 200 pathology reports over the study period were excluded.

(4) Proximal colon defined as proximal to the splenic flexure, >50 cm from anal verge; distal colon defined as end of the sigmoid colon to the splenic flexure, 35-50cm from anal verge; and rectosigmoid defined as sigmoid colon and rectum, ≤35cm from anal verge.

Impact of variation in pathologist classification rates on endoscopist quality metrics

In our sample, the average SPDR across the 119 endoscopists was 7%. We modeled the SPDR for all 119 endoscopists if all polyp specimens were analyzed by the pathologist with the lowest and highest classification rate, as well as the pathologist at the 25^th and 75^th percentiles of classification. If all polyp specimens were analyzed by the pathologist with the lowest versus the highest classification rate, the average SPDR would be 0.5% versus 12%, respectively. If analyzed by the pathologist at the 25^th and percentiles of classification rate, the average SPDR would be 5% and 9%, respectively. Figure 3 highlights the range of this impact across physicians.

Figure 3.

Translating Pathologist Serrated Polyp and Adenoma Classification Rates to Endoscopist Serrated Polyp and Adenoma Detection Rates for the 20 Highest-Volume Endoscopists

Notes: For each figure, the colored dot represents the endoscopist’s observed serrated polyp detection rate (red, left panel) or adenoma detection rate (blue, right panel). The black dot to the left of each colored dot represents what each endoscopist’s outcome would be if all of their colonoscopy specimens were analyzed by the pathologist with the lowest classification rate for that outcome in the sample (1% for serrated polyp classification, 48% for adenoma classification). The black dot to the right of each colored dot represent what each endoscopist’s outcome would be if all their colonoscopy specimens were analyzed by the pathologist with the highest classification rate for that outcome in the sample (20% for serrated polyp classification, 71% for adenoma classification). For additional details on how these values were calculated, please see Appendix 5.

Another way of framing these results is by the percent change in SPDR. An endoscopist’s SPDR would decrease by, on average, 91% (range 53% to 96%) if all the endoscopist’s specimens were analyzed by the pathologist with the lowest serrated polyp classification rate. SPDR would increase by, on average, 119% (range 5% to 1053%), if analyzed by the pathologist with the highest serrated polyp classification rate. If an endoscopist’s specimens were analyzed by the pathologist at the 25^th percentile of serrated polyp classification, their SPDR would decrease by 9%, on average, and if they were analyzed by the pathologist at the 75^th percentile of serrated polyp classification, their SPDR would increase by 62%, on average.

The average ADR among the 119 endoscopists in our sample was 36%. If all the specimens had been analyzed by the pathologist with the lowest adenoma classification rate, the average ADR across the 119 endoscopists would be 28% and if they had been analyzed by the pathologist with the highest adenoma classification rate, the average endoscopist ADR would be 42%. If we instead apply the pathologist’s classification rate at the 25^th and 75^th percentile, the average endoscopist ADR would be 32% and 38% respectively.

For an individual endoscopist, sending all of one’s specimens to the pathologist with the lowest adenoma classification rate could decrease ADR by, on average, 19% (range 10% to 28%). Sending all specimens to the pathologist with the highest adenoma classification rate, on the other hand, would increase ADR by, on average, 20% (range 8% to 44%). If an endoscopist’s specimens were analyzed by the pathologist at the 25^th percentile of adenoma classification, their ADR would decrease by 10%, on average, and if they were analyzed by the pathologist at the 75^th percentile of adenoma polyp classification, their ADR would increase by 9%, on average.

Discussion

We measured variation in pathologists’ adenoma and serrated polyp classification in a large, multi-site sample of colonoscopy exams. Consistent with prior work,(13, 14, 20, 21) we find substantial variation in classification of serrated polyps that exceeded the variation seen in pathologists’ classification of conventional adenomas. Variation in pathologist classification can have a substantial impact on endoscopist performance on quality measures. The impact on endoscopist SPDR was dramatic and implies that a substantial portion of the variation observed across endoscopists may be attributable to the pathologist that analyzes the specimens.

There are several explanations for why there is more variation between pathologists in classification of serrated polyps compared to classification of adenomas. First, serrated neoplasia is a relatively recent concept, and both the pathology diagnostic criteria and nomenclature of serrated polyps have evolved significantly over the past 2 decades.(19, 33, 40) This may lead to differences or disagreements between pathologists regarding pathology interpretation and terminology used. For example, some have advocated a minimum of 2 dilated crypt bases to diagnose a sessile serrated adenoma/polyp,(40) whereas others recommend only 1 dilated crypt base.(33) Second, in contrast to adenomatous lesions, specimen orientation and sectioning are more important with serrated polyps. Hyperplastic polyps and sessile serrated adenoma/polyps share the feature of crypt serration on histologic cross-section, but sessile serrated adenoma/polyps are characterized and diagnosed by the shape of colonic crypt bases (e.g. dilated or boot-shaped crypts(9)). Accordingly, issues such as tangential sectioning of polypectomy specimens can interfere with the ability of pathologists to accurately diagnose the lesion.(41) Additionally, effects of electrocautery or fragmentation of polyp specimens during polypectomy or retrieval can also result in pathologic cross-sections of serrated polyps that are difficult to interpret definitively as sessile serrated adenoma/polyps.(33)

It is possible that much of the variation between pathologists in serrated polyp classification come from differences in classifying a given specimen as a hyperplastic polyp versus a sessile serrated adenoma/polyp. According to the World Health Organization (WHO) criteria, a hyperplastic polyp is in the family of serrated polyps.(30) We were unable to measure the prevalence of hyperplastic polyps in our analysis because our NLP pipeline was not trained on that criterion. Therefore we cannot determine if pathologists with low serrated polyp classification rates have correspondingly higher hyperplastic polyp detection rates. However, the impact on clinical practice remains, as hyperplastic polyps are considered innocuous lesions, for which routine 10 year follow-up is recommended.(31) On the other hand, individuals with serrated polyps are advised to undergo surveillance colonoscopy similarly to subjects with conventional adenomas. The variation we observe in serrated polyp classification highlights the need for efforts within the pathology community to achieve greater consistency in pathologic classification.

Efforts to evaluate endoscopist performance, in particular for SPDR, should account for pathologist variation. Given the variation we observed, SPDR may currently not be well suited as a measure of endoscopic quality. Some guidelines for ADR assessment have argued that serrated polyps should be excluded from the calculation of ADR.(16) While we recognize that serrated adenomas are important precursors of colorectal cancer(7, 8) and the surveillance recommendations for sessile serrated adenomas/polyps are similar to those for conventional adenomas, our results suggest that the inconsistency in pathologist identification of serrated polyps may be the most compelling reason for excluding these lesions in the calculation of ADR.

Through there is less variation in classification of adenomas than serrated polyps, our analysis demonstrates that pathologist adenoma classification can still substantially impact an endoscopist’s ADR (increasing or decreasing ADR by roughly 20 percent). While identification of conventional adenomas is typically straightforward, it is possible that some pathologists in our sample are not obtaining deeper level sections of the tissue block or there could be disagreement on whether observed epithelial changes are reactive to an inflammatory processes or adenomatous changes. Inter-pathologist variation in adenoma classification may be clinically relevant to endoscopists near the threshold of acceptable detection rates who may be inaccurately penalized or rewarded in quality improvement programs based on the pathologist(s) interpreting their polypectomy specimens.

The strengths of this study are the large sample of pathologists and endoscopists from a range of health care systems and the translation of pathologist classification of serrated polyps and adenomas into implications for endoscopists. Our work also has some important limitations. First, though these findings are based on the largest sample of pathologists that has been published to date and the pathologists are similar in gender and years of practice to pathologists across the nation, it is unknown how representative our sample is of all pathologists. Second, we did not independently review pathology specimens to measure misclassification. The variation we observe in classification by pathologists is presumably driven by variation in pathologist assessment. However given this limitation, we do not know whether it is “better” or clinically more accurate to send specimens to a pathologist with a higher or lower classification rate. Future work should assess the relative accuracy of pathologists with high and low classification rate and the implications of pathologist variation on patients’ clinical outcomes. Finally, we focus is on pre-malignant serrated polyps, excluding hyperplastic polyps, and cannot distinguish between types of serrated polyps. However, our inability to distinguish between types of serrated polyps is less of a concern because prior research has highlighted that the vast majority of pre-malignant serrated polyps are sessile serrated adenomas/polyps.(42,43)

In summary, among a large, multi-site sample of U.S. pathologists, we find that, after accounting for endoscopist variation, there is substantial pathologic variation in interpretation of premalignant colorectal polyps, especially serrated polyps. This variation can have a substantial effect on endoscopists’ quality performance on SPDR and ADR.

Study Highlights.

1. WHAT IS CURRENT KNOWLEDGE

   • There is variation in pathologist classification of polyps removed during colonoscopy.
2. WHAT IS NEW HERE

   • Use of clinical data from a large multi-site sample of pathologists.
   • Application of a novel method to quantify how variation in pathologist classification impacts endoscopists’ quality scores.
   • Variation in pathologist classification can have a dramatic impact on endoscopist quality scores, especially serrated polyp detection rate.