Validation and Comparison of Tools for Selecting Individuals to Screen for Barrett’s Esophagus and Early Neoplasia

Abstract

Background & Aims:

Guidelines suggest endoscopic screening of individuals who are at increased risk for Barrett’s esophagus (BE) and esophageal adenocarcinoma. Tools based on clinical factors are available for identifying patients at risk, but only some have been validated. We aimed to compare and validate available tools.

Methods:

We performed a prospective study of 1241 patients, ages 40–79 y, presenting either for their first esophagogastroduodenoscopy (EGD) or their first endoscopic therapy of early neoplastic BE, from April 2015 through June 2018. We calculated risk scores for 6 previously published tools (the Gerson, Locke, Thrift, Michigan BE pREdiction Tools [M-BERET], Nord-Trøndelag Health Study [HUNT], and Kunzmann tools). We also investigated the accuracy of frequency and duration of gastroesophageal reflux disease (GERD), using data from a randomly selected 50% of patients undergoing their first EGD. We compared the ability of all these tools to discriminate patients with BE and early neoplasia from patients without BE, using findings from endoscopy as the reference standard.

Results:

BE was detected in 81/1152 patients during their first EGD (7.0%). GERD symptoms alone identified patients with BE with an area under the receiver operating characteristic curve (AUROC) of 0.579. All of the tools were more accurate in identifying patients with BE than the frequency and duration of GERD (AUROC for GERD, 0.579 vs range for other tools, 0.660–0.695), and predicted risk correlated well with observed risk (calibration). The AUROCs of the HUNT tool (0.796), the M-BERET (0.773), and the Kunzmann tool (0.763) were comparable in discriminating between patients with early neoplasia (n=94) vs no BE. Each tool was more accurate in discriminating BE with early neoplasia than GERD frequency and duration alone (AUROC, 0.667; P<.01).

Conclusions:

The HUNT, M-BERET, and Kunzmann tools identify patients with BE with AUROC values ranging from 0.665 to 0.695, and discriminate patients with early neoplasia from patients without BE with AUROC values ranging from 0.763 to 0.796. These tools are more accurate than frequency and duration of GERD in identifying individuals at risk for neoplastic BE.

Lay Summary

The HUNT tool, M-BERET, or Kunzmann tool should be used rather than gastroesophageal reflux disease symptoms alone for identifying individuals who should be offered screening for esophageal cancer.

INTRODUCTION

The incidence of esophageal adenocarcinoma has risen 6-fold over the last 4 decades.¹ Barrett’s esophagus (BE) is a change in the lining of the esophagus that is a precursor to the cancer. Symptoms of gastroesophageal reflux disease (GERD), namely heartburn and regurgitation, are major risk factors for both Barrett’s esophagus and esophageal adenocarcinoma.^{2, 3} Retrospective studies suggest that screening by esophagogastroduodenoscopy (EGD) is effective at reducing mortality associated with esophageal adenocarcinoma.^4–6 A number of specialty organizations have recommended endoscopic screening in patients with GERD symptoms to identify Barrett’s esophagus, and to perform surveillance in those with Barrett’s esophagus to identify neoplasia.^7–9 Over 2 million EGDs are performed annually in the U.S. in adults with GERD symptoms, at an estimated cost of over $2 billion.^10–12 Nonetheless, fewer than 15% of patients with esophageal adenocarcinoma have had an EGD prior to their diagnosis of the cancer.^{4, 13} Clearly, there is a need for a better personalized approach to selection of patients for endoscopic screening.

Specialty guidelines recommend screening GERD patients with multiple risk factors, but most do not state how many factors or what magnitude of each of those is required to trigger screening.^{7, 14–16} Other risk factors associated with Barrett’s esophagus include male sex, age, abdominal obesity, and tobacco use. To fill this gap, we previously developed the Michigan Barrett’s Esophagus pREdiction Tool (M-BERET) which incorporates GERD symptoms, age, waist-to-hip ratio, and pack-years of cigarette use to calculate a predicted probability of harboring Barrett’s esophagus.¹⁷ A number of other groups have also developed tools using clinical information for identifying patients at elevated risk for Barrett’s esophagus or esophageal adenocarcinoma,^18–21 and one study utilized concentrations of circulating cytokines and leptin.²² However, many of these tools have not been externally validated for identifying Barrett’s esophagus, and none have been validated for discriminating early neoplastic Barrett’s esophagus (such as high grade dysplasia) which is optimal for performing endoscopic therapy. Therefore, we aimed to validate these tools and directly compare them for predicting Barrett’s esophagus among patients undergoing their first EGD. We also aimed to compare them for discriminating patients with early neoplastic Barrett’s esophagus from those without Barrett’s esophagus.

METHODS

We conducted a cross-sectional study (“Validation and Extension of the Michigan Barrett’s Esophagus pREdiction Tool Study, M-BERET Study”) by enrolling individuals, aged 40–79, presenting for their first EGD and those presenting for their first endoscopic management of suspected early neoplastic Barrett’s esophagus at either the University of Michigan (UM) or the Ann Arbor Veterans Affairs Medical Center (AAVA) from April 2015 through June 2018.

Subject Recruitment

Patients were recruited consecutively within time constraints of the study team, and excluded if: they were younger than age 40 or greater than age 79, had a prior history of esophagectomy, known ascites, encephalopathy or esophageal varices, severe comorbidity limiting life expectancy to less than 5 years, inpatient status, known to be pregnant, prisoners, or if they were unable to read English or cooperate with the study. Because alarm features such as dysphagia are poor predictors of malignancy with positive predictive values < 1%, we did not exclude such patients from enrollment, but we did perform sensitivity analyses excluding them.^{23, 24} First EGD was defined as no prior EGD at all or no EGD since 1994, the year of the seminal publication describing short segment Barrett’s esophagus.²⁵ Early neoplasia was defined as either 1) history of esophageal adenocarcinoma suspected to be T1a based on prior endoscopy, histology, and radiology reports undergoing evaluation for their first endoscopic therapy, 2) history of Barrett’s esophagus with high grade dysplasia undergoing evaluation for endoscopic therapy, or 3) history of Barrett’s esophagus with low grade dysplasia on more than one endoscopy after escalation of therapy with proton pump inhibitors and confirmed by expert gastrointestinal pathologists at UM. Subjects undergoing their first EGD were recruited at the time of presentation to the endoscopy unit either at the UM University Hospital Medical Procedures Unit, the UM East Ann Arbor Medical Procedures Center (an ambulatory center located 5 miles from University Hospital), or the AAVA Endoscopy Suite (located 1 mile from University Hospital). Patients with early neoplasia were typically recruited at the time of their clinic visit, but sometimes when presenting to the endoscopy unit. Subjects were provided a $10 gift card for participating. The study was approved by the Institutional Review Boards of the University of Michigan (UM) and the Ann Arbor Veterans Affairs Medical Center (AAVA).

Anthropometrics, Blood Draw, Questionnaires, and Endoscopies

Prior to their EGD, while dressed in hospital gowns or pajamas, subjects had their weight, height, waist circumference, and hip circumference each measured in duplicate.^26–28 Subjects answered 4 questionnaires used for 3 of the prediction tools (described below),^17–19 questions eliciting self-description of race and ethnicity, as well as the validated Patient Assessment of Upper Gastrointestinal Symptom Severity Index (PAGI-SYM) questionnaire to characterize symptoms leading to the indication for the EGD.^{29, 30} Prior to the EGD, blood samples were drawn into serum separator tubes, allowed to stand at room temperature for 30–60 minutes, then refrigerated at 4°C for 60 minutes followed by centrifugation. Serum was stored in 1-mL aliquots at −80°C until the time of assay. During the EGD, the distal esophagus and gastroesophageal junction were inspected using narrow band imaging, and any suspected Barrett’s esophagus was classified in length using the Prague criteria.³¹ Biopsies or endoscopic resection was performed as per routine clinical practice. If dysplasia was suspected, specimens were reviewed by multiple expert gastrointestinal pathologists at UM to make a consensus diagnosis. Barrett’s esophagus was defined as suspected columnar mucosa proximal to the gastroesophageal junction, confirmed histologically as harboring specialized intestinal metaplasia.⁷ If Los Angeles Class C or D esophagitis was found,³² patients were instructed to repeat the endoscopy after a healing course of a proton pump inhibitor, and disease status was classified from the repeat endoscopy.

Prediction Tools

The Gerson tool was developed among patients undergoing EGD for clinical indication at Stanford University or the Palo Alto Veterans Affairs Medical Center and relies on responses to a 7-item questionnaire that queries severity of GERD and upper abdominal symptoms, plus race and ethnicity, which we elicited by self-descriptions.¹⁸ In the development study, the Gerson tool had an area under the receiver operating characteristics (AuROC) of 0.72 for discriminating Barrett’s esophagus, but it has not been validated.

The Locke tool was developed among patients undergoing EGD for clinical indication at the Mayo Clinic, and relies on responses to the 23-item Mayo Clinic Reflux Symptom Questionnaire (RSQ) that queries frequency and duration of GERD symptoms, dysphagia, acid-reducing medications, and a 17-item Psychosomatic Symptom Checklist.^{19, 33} In the development study, the Locke tool had an AuROC of 0.76 for discriminating Barrett’s esophagus, but it has not been validated.

The Thrift tool was developed from a population-based study in Australia comparing patients clinically diagnosed with Barrett’s esophagus to patients with erosive esophagitis.²⁰ The tool is limited to patients with GERD and relies on age, sex, cigarette smoking (current, former, or never), body mass index, education, and use of acid reducing medications. The AuROC was 0.70 in the development cohort compared to 0.61 in an external validation cohort in the original publication.

The M-BERET was developed with quasi-population based methodology among men aged 50–79 who were undergoing colonoscopy for colorectal cancer screening at either the UM or AAVA and were recruited to undergo EGD for research regardless of symptoms.¹⁷ The M-BERET relies on age, waist-to-hip circumference ratio, and an 8-item questionnaire querying typical frequency of heartburn or regurgitation without the use of acid reducing medications and a crude estimate of pack-years of cigarette smoking. The AuROC for discriminating Barrett’s esophagus in the development cohort was 0.72, and in external validation for discriminating Barrett’s esophagus in 4 unique cohorts, the AuROC ranged from 0.70 to 0.72.³⁴ Prior to initiating recruitment for the current validation study, we a priori extended the M-BERET by extrapolating the same linear β coefficient for age in the M-BERET to individuals 40–49 years old. In addition, we extended the M-BERET to women by updating a published meta-analysis of the association of sex with Barrett’s esophagus with results from the Clinical Outcomes Research Initiative National Endoscopic Database.^{35, 36} The summary random effects odds ratio (OR) of women versus men for Barrett’s esophagus was 0.386 (95% confidence interval [CI] = 0.282, 0.528), with β coefficient = −0.9519. We a priori extended the M-BERET to women by using that β coefficient.

The Nord-Trøndelag Health Study (HUNT) tool was developed from a prospective population-based cohort study in Nord-Trøndelag County, Norway.²¹ The tool relies on age, sex, GERD symptoms in the prior 12 months (yes or no), body mass index, and smoking (ever or never). The 10-fold internally cross-validated AuROC for 15 year risk of esophageal adenocarcinoma was 0.84. The HUNT tool has not been externally validated.

The Kunzmann tool was developed from a prospective cohort in the United Kingdom (UK Biobank).³⁷ It relies on age, sex, body mass index, smoking (current, former, never), and esophageal conditions (defined as self-report of GERD, Barrett’s esophagus, hiatal hernia, esophageal stricture, fundoplication, or acid reducing medications). The AuROC for 5 year risk of esophageal adenocarcinoma was 0.80, but it has not been externally validated. For the current study, we chose to limit the esophageal condition predictor to information about GERD symptoms to make the tool more generalizable for use in patients without prior EGD. In the current study, we utilized the response to questions about heartburn and regurgitation in the RSQ to classify the GERD predictor in the HUNT and Kunzmann tools.

The serum biomarker panel of leptin, interleukins (IL) IL6, IL8, IL10, and IL12p70 was developed from a case-control study at the Michael DeBakey Veterans Affairs Medical Center.²² The biomarker panel in combination with clinical factors of age, sex, race, waist-to-hip circumference ratio, Helicobacter pylori status, and GERD frequency and duration had an AuROC of 0.85 compared to a model of GERD symptoms alone of 0.74, but has not been externally validated.

Finally, to compare the available tools to typical clinical practice, we developed a tool for predicting Barrett’s esophagus based only on GERD frequency and duration (obtained from the RSQ questions regarding heartburn and regurgitation) in a randomly selected half of the first EGD subjects.

Assays

We assayed stored serum from all cases of Barrett’s esophagus with available serum (n = 164), and 340 randomly selected subjects without Barrett’s esophagus undergoing their first EGD. Serum was assayed in duplicate using 2-site enzyme-linked immunosorbent assays for the same biomarkers as in the prior publication,²² but using a different vendor (R&D systems, Minneapolis, MN).

Analysis

Data were manually entered into Microsoft Access (Microsoft, Bellevue, WA), then imported into SAS 9.4 (SAS Institute, Cary, NC). The tool based on GERD symptoms alone was developed in a random half of the first EGD subjects using logistic regression, choosing parameterization of heartburn frequency and duration, and regurgitation frequency and duration to fit the most parsimonious model with greatest AuROC for predicting Barrett’s esophagus among first EGD subjects. Scores for each of the 7 prediction tools were then calculated for all subjects. We first compared the 7 tools for predicting Barrett’s esophagus among the first EGD subjects. Because the Thrift tool is limited to discriminating among patients with GERD, we only compared it to other tools among patients with GERD symptoms in the prior year as assessed by the RSQ questionnaire. Because the Locke and Thrift tools use acid reducing medications as a predictor, and all of the early neoplastic Barrett’s esophagus patients in our current cohort were taking such medications in planning for endoscopic therapy, we did not assess those tools for discriminating early neoplasia. Tools were compared for discrimination using the AuROC and the optimal threshold for each tool was identified at the maximum of the Youden’s index (the point on the ROC curve where sensitivity + specificity − 1 is maximized). Among the first EGD subjects, the calibration of the tools were assessed using the Hosmer-Lemeshow test (p < 0.05 indicates poor fit), and the Net Reclassification Improvement (NRI) index for each tool was compared to the GERD model alone using ad hoc categories of < 5%, 5–10%, and > 10% prevalence of Barrett’s esophagus.^38–40 We also calculated the Integrated Discrimination Improvement (IDI), which is similar to the NRI but integrates over all possible thresholds.³⁹ Metrics of calibration and reclassification were not calculated for the early neoplasia analysis since the prevalence of early neoplasia was defined by the case-control design.

The serum biomarker score was calculated according to the prior publication: 1 point for each biomarker (leptin, IL6, IL8, IL12p70) that was in the highest quartile of the control group distribution and 1 point for IL10 in the lowest quartile.²² We used the distributions referenced by those authors,⁴¹ and found similar AuROCs when deriving the distributions from our random controls (data not shown). Among all the cases of Barrett’s esophagus and the randomly selected subjects without Barrett’s esophagus, we compared the AuROCs of the M-BERET to M-BERET plus serum biomarkers for discriminating Barrett’s esophagus among first EGD subjects, and also early neoplastic Barrett’s esophagus from no Barrett’s esophagus.

RESULTS

Patient Enrollment

1,678 patients undergoing their first EGD were recruited, and 265 were excluded (Figure 1). The most common reason for exclusion was a history of a prior upper endoscopy that had not been documented in the electronic medical record (256, 96.6%). Of the remaining 1,413 eligible subjects, 1,172 consented to the study (82.9%), of whom 20 were subsequently withdrawn, primarily due to aborted endoscopy due to medical reasons and no repeat attempt at endoscopy during the time frame of the study (n = 15). Of the 1,152 enrolled first EGD subjects available for analysis, 455 (39.5%) had their procedure at UM University Hospital, 312 (27.1%) at UM East Ann Arbor Medical Procedures Center, and 385 (33.4%) at the AAVA.

Figure 1. Flow Diagram of Recruitment and Classification of Subjects.

BE – Barrett’s esophagus

EGD – esophagogastroduodenoscopy

GERD was the most common indication identified by the endoscopist in 476 subjects (41.3%), followed by dysphagia in 276 (24.0%), epigastric pain in 152 (13.2%), and anemia in 151 (13.1%). The PAGI-SYM questionnaire was completed by 1,018 first endoscopy subjects, and GERD was among the most severe upper gastrointestinal symptoms in 321 (31.5%). Some other upper gastrointestinal symptoms were more commonly among the most severe, including bloating (n = 415, 40.8%) and epigastric discomfort (n = 342, 33.6%) (indications and most severe symptoms were not mutually exclusive). The endoscopist suspected Barrett’s esophagus in 199 subjects, among whom specialized intestinal metaplasia was confirmed in 81. A total of 12 subjects could not be classified for Barrett’s esophagus and were excluded from analysis: 5 with endoscopic suspicion of Barrett’s esophagus but not biopsied due to concern of coexisting varices, and 7 with Los Angeles Grade C or D esophagitis who never underwent repeat endoscopy to exclude Barrett’s esophagus. Most first EGD patients with Barrett’s esophagus had no neoplasia, but 6 were indefinite for dysplasia, 2 had low grade dysplasia (one of whom subsequently had high grade dysplasia on endoscopic resection, and the other confirmed to have low grade dysplasia on a subsequent EGD with biopsies after escalation of acid reducing therapy), 3 had high grade dysplasia, and 1 had adenocarcinoma subsequently found to be invading at least into the submucosa. Most patients with Barrett’s esophagus had short segment (median C0M1; interquartile range C0M1, C0M2; 81.7% < 3 cm). There were 3 additional subjects with bulky esophageal masses found to be adenocarcinoma but without endoscopically visible Barrett’s esophagus and not amenable to endoscopic therapy.

For early neoplasia subjects, 102 were recruited with none being excluded, and 94 consented to the study (92.2%) (Figure 1). 5 subjects were subsequently withdrawn; 3 due to the subject changing their mind, one for being previously enrolled at the AAVA prior to presenting to UM for further endoscopic therapy, and one due to being enrolled prior to confirmation of low grade dysplasia which subsequently was found to revert to non-dysplastic Barrett’s esophagus. Of the 89 early neoplasia subjects available for analysis, 84 (94.4%) had their procedure at UM University Hospital, none at UM East Ann Arbor Medical Procedures Center, and 5 (5.6%) at the AAVA. Sixty (67.4%) had their most recent endoscopy for surveillance of known Barrett’s esophagus, 22 (24.7%) for diagnostic indications (11 blood loss anemia, 5 abdominal pain, 4 dysphagia, 2 GERD symptoms despite proton pump inhibitors [PPI]), and 7 (7.9%) for screening (5 for typical GERD symptoms either responsive to PPI or never treated, 1 familial adenomatous polyposis, and 1 pre-operative for bariatric surgery). Those undergoing surveillance had a median of 2 prior endoscopies (interquartile range: 2, 4), and were first diagnosed with Barrett’s esophagus a median of 5.68 years prior (interquartile range: 2.98, 11.22).

Early neoplasia patients had longer Barrett’s esophagus than the first upper endoscopy patients found to have Barrett’s esophagus (median C1M3; interquartile range C0M2, C4M6; 46.3% < 3 cm). The most severe histological findings prior to enrollment included low grade dysplasia in 3 subjects (3.4%), high grade dysplasia in 67 (75.3%), and adenocarcinoma suspected to be T1a in 19 (21.3%). The most severe histological findings at the time of the index procedure was no neoplasia in 8 (9.0%), low grade dysplasia in 8 (9.0%), high grade dysplasia in 25 (28.1%), T1a adenocarcinoma in 31 (34.8%), and adenocarcinoma ≥ T1b in 11 (12.4%); one patient with pretreatment high grade dysplasia did not undergo any tissue sampling at the time of the index endoscopy but instead underwent radiofrequency ablation. For the analyses validating prediction of early neoplasia, we included these 89 subjects plus the 5 from among the first EGD subjects found to have Barrett’s esophagus with early neoplasia.

Characteristics of the enrolled subjects are described in Table 1. Subjects with Barrett’s esophagus or early neoplasia were older with more abdominal obesity, and were more frequently non-Hispanic whites, male, smokers, GERD sufferers, and used PPIs.

Table 1.

Descriptive Characteristics of Subjects

	First Upper Endoscopy Subjects			Early Neoplasia Subjects (n = 89)
	All (n = 1,152)	No Barrett’s Esophagus (n = 1,059)	Barrett’s Esophagus (n = 81)
Age (years)*	58.3 (9.9)	58.1 (9.9)	59.7 (10.2)	65.6 (8.2)
Male sex (%)§	674 (58.5%)	596 (56.3%)	69 (85.2%)	69 (77.5%)
Body mass index (kg/m2)*	30.2 (7.2)	30.2 (7.3)	29.9 (6.1)	31.4 (7.2)
Waist circumference (cm)*	105.5 (16.4)	105.3 (16.4)	106.8 (15.8)	110.1 (17.1)
Waist-to-hip ratio*	0.970 (0.076)	0.967 (0.076)	0.999 (0.073)	1.009 (0.070)
GERD ≥ weekly off meds (%)§	627 (55.2%)	561 (53.8%)	59 (72.8%)	70 (78.7%)
PPI use currently (%)§	574 (50.6%)	516 (49.5%)	54 (66.7%)	89 (100.0%)
Current smoker (%)§	203 (17.9%)	183 (17.6%)	16 (19.8%)	13 (14.6%)
Former smoker (%)§	412 (36.3%)	370 (35.5%)	41 (50.6%)	54 (60.7%)
Non-Hispanic white (%)§	894 (85.1%)	817 (84.8%)	66 (88.0%)	86 (96.6%)

Data expressed as mean (standard deviation)* or number (proportion)^§. Proportions are among subjects with available data.

GERD: gastroesophageal reflux disease, PPI: proton pump inhibitor

Prediction of Barrett’s Esophagus among First Upper Endoscopy Subjects

For predicting the finding of Barrett’s esophagus among subjects undergoing their first EGD, the tool based on GERD frequency and duration alone had only a modest area under the receiver operating curve (AuROC) of 0.579 (Figure 2a, Table 2). All 5 tools (Gerson, HUNT Study, Kunzmann, Locke, M-BERET) were statistically significantly better than GERD at predicting Barrett’s esophagus (AuROCs range: 0.660, 0.695, p < 0.05 for each comparison), but none of the 5 was superior than the other. All of the tools had similar goodness of fit (Hosmer-Lemeshow p > 0.05 for each) except the GERD model had poor calibration (Hosmer-Lemeshow p < 0.001). At its optimal threshold, the GERD score had a Youden’s index of 0.146. All of the models had greater Youden’s indices, Net Reclassification Improvement, and Integrated Discrimination Indices than GERD alone (Table 2), with M-BERET having the greatest (0.351, 43%, and 0.032, respectively). All of the models had substantial improvements in classification of controls compared to GERD alone (range: 28%, 46%), but the Kunzmann model also had 19% fewer cases of Barrett’s esophagus correctly classified than GERD alone.

Figure 2. Receiver Operating Characteristics Curves for (2a) Discriminating Barrett’s Esophagus among First EGD Subjects and (2b) Early Neoplastic Barrett’s Esophagus from No Barrett’s Esophagus.

Area under the receiver operating characteristics curves in parentheses. Note that Locke relies on current PPI use and early neoplastic Barrett’s esophagus patients were prescribed PPI in anticipation of endoscopic therapy, so Locke was not evaluated in the analysis for early neoplastic Barrett’s esophagus.

Table 2.

Test Characteristics for Identifying Barrett’s Esophagus among First EGD Subjects

Tool	AuROC (95% CI)	Hosmer-Lemeshow p-value	Youden’s Index	Sensitivity*	Specificity*	NRI vs. GERD	Cases Correctly Reclassified vs. GERD	Controls Correctly Reclassified vs. GERD	IDI vs. GERD
GERD	0.579 (0.505, 0.653)	< 0.001	0.146	56.7%	57.9%	n/a	n/a	n/a	n/a
Gerson	0.692 (0.630, 0.753)b	0.620	0.335	68.7%	64.9%	33%b	−6%	39%c	0.018b
HUNT	0.665 (0.595, 0.735)a	0.666	0.287	70.1%	58.6%	28%b	0%	28%c	0.015b
Kunzmann	0.667 (0.605, 0.730)a	0.083	0.299	73.1%	56.8%	23%b	−19%b	42%c	0.015b
Locke	0.660 (0.589, 0.731)b	0.909	0.267	62.7%	64.0%	28%b	−14%	42%c	0.024b
M-BERET	0.695 (0.625, 0.766)b	0.080	0.351	56.7%	78.3%	43%b	−3%	46%c	0.032b

AuROC: area under the receiver operator characteristic curve; CI: confidence interval; NRI: net reclassification improvement compared to GERD using thresholds of 5% and 10% to define 3 categories of risk; IDI: integrated discrimination improvement compared to GERD

^*Sensitivity and specificity at the point on the ROC curve where the Youden’s index is maximized.

^ap < 0.05 vs. GERD,

^bp < 0.01 vs. GERD,

^cp < 0.001 vs. GERD

Discrimination of Early Neoplasia from No Barrett’s Esophagus

For discriminating early neoplasia from no Barrett’s esophagus (Figure 2b, Table 3), the AuROCs of the HUNT Study tool (0.796), M-BERET (0.773), and Kunzmann tool (0.763) were comparable to each other, and each was more accurate than GERD symptoms alone (0.667) and the Gerson tool (0.680). The HUNT Study tool had the greatest Youden’s index (0.477). However, the tools only had very modest accuracy in discriminating early neoplasia from non-dysplastic Barrett’s esophagus (AuROC range: 0.491, 0.633) (Supplemental Figure 1).

Table 3.

Test Characteristics for Discriminating Early Neoplastic Barrett’s Esophagus from Subjects without Barrett’s Esophagus

Tool	AuROC (95% CI)	Youden’s Index	Sensitivity*	Specificity*
GERD	0.667 (0.616, 0.719)	0.356	81.5%	54.1%
Gerson	0.680 (0.622, 0.737)	0.336	68.5%	65.1%
HUNT	0.796 (0.751, 0.842)b,c	0.477	64.1%	83.6%
Kunzmann	0.763 (0.717, 0.809)a,c	0.396	68.5%	71.1%
M-BERET	0.773 (0.728, 0.818)a,c	0.425	70.7%	71.9%

AuROC: area under the receiver operating characteristics curve; CI: confidence interval

^*Sensitivity and specificity at the point on the ROC curve where the Youden’s index is maximized.

^ap < 0.01 vs. GERD,

^bp < 0.001 vs. GERD,

^cp < 0.001 vs. Gerson

Stratified Analyses

Among men undergoing their first EGD, M-BERET and Locke better discriminated patients with Barrett’s esophagus than GERD symptoms alone (Table 4). Among women, M-BERET and Gerson had numerically greater AuROC than GERD symptoms alone, but with imprecise estimates due to the few number of women with Barrett’s esophagus. Each of the models had better discrimination for Barrett’s esophagus among individuals with at least weekly GERD symptoms than relying on GERD duration and frequency alone. Among those without weekly GERD symptoms, the AuROCs for each model was also numerically greater than GERD frequency and duration alone, but had imprecise estimates given the few number of cases of Barrett’s esophagus among those without weekly GERD symptoms.

Table 4.

Area Under the Receiver Operating Characteristics Curve (and 95% Confidence Interval) for Discriminating Barrett’s Esophagus among Strata of First EGD Subjects

Tool	Men	Women	At Least Weekly GERD	No GERD or Less than Weekly
GERD	0.624 (0.544, 0.705)	0.555 (0.368, 0.743)	0.526 (0.430, 0.622)	0.516 (0.403, 0.628)
Locke	0.657 (0.582, 0.732)a	0.534 (0.367, 0.701)	0.636 (0.545, 0.727)b	0.607 (0.469, 0.746)
Gerson	0.617 (0.542, 0.691)	0.592 (0.476, 0.708)	0.673 (0.599, 0.746)b	0.683 (0.564, 0.803)
HUNT	0.630 (0.555, 0.706)	0.506 (0.311, 0.702)	0.666 (0.581, 0.750)b	0.585 (0.445, 0.725)
M-BERET	0.652 (0.572, 0.732)a	0.587 (0.414, 0.759)	0.681 (0.597, 0.764)b	0.638 (0.492, 0.785)
Kunzmann	0.571 (0.498, 0.645)	0.505 (0.322, 0.689)	0.675 (0.601, 0.748)b	0.649 (0.519, 0.779)
Thrift	n/a	n/a	0.675 (0.598, 0.751)b	n/a

^ap < 0.05 vs. Kunzmann,

^bp < 0.05 vs. GERD

For discriminating between early neoplastic Barrett’s esophagus and no Barrett’s esophagus (Table 5), among men, the HUNT tool was the most accurate, (AuROC = 0.771) and the Gerson tool the least (AuROC = 0.643), but among women, the M-BERET and Kunzmann tools discriminated the best (AuROCs = 0.820 and 0.816, respectively). Among those with at least weekly GERD symptoms, HUNT, M-BERET, and Kunzmann were each more accurate than GERD symptoms alone or the Gerson tool. Among those without weekly GERD symptoms, the HUNT and Kunzmann tools were the most accurate.

Table 5.

Area Under the Receiver Operating Characteristics Curve (and 95% Confidence Interval) for Discriminating Early Neoplastic Barrett’s Esophagus from Subjects without Barrett’s Esophagus within Strata

Tool	Men	Women	At Least Weekly GERD	No GERD or Less than Weekly
GERD	0.690 (0.630, 0.750)	0.651 (0.549, 0.754)	0.548 (0.483, 0.614)	0.750 (0.639, 0.861)
Gerson	0.643 (0.579, 0.707)	0.541 (0.406, 0.676)	0.656 (0.591, 0.722)a	0.667 (0.541, 0.792)
HUNT	0.771 (0.719, 0.824)a,b	0.757 (0.649, 0.865)b	0.752 (0.695, 0.809)a,b	0.818 (0.711, 0.925)b
M-BERET	0.739 (0.683, 0.795)b	0.820 (0.727, 0.913)a,b	0.753 (0.699, 0.807)a,b	0.736 (0.637, 0.835)
Kunzmann	0.690 (0.630, 0.750)b	0.816 (0.749, 0.884)a,b	0.751 (0.697, 0.805)a,b	0.787 (0.687, 0.887)b

^ap < 0.05 vs. GERD,

^bp < 0.05 vs. Gerson

Sensitivity Analyses

We conducted a number of sensitivity analyses to assess the robustness of the results. Excluding patients with an indication of dysphagia or anemia resulted in similar AuROCs for both prediction of Barrett’s esophagus among first upper endoscopy patients and discriminating early neoplastic Barrett’s esophagus from no Barrett’s esophagus (Supplemental Figures 2a and 2b). Since guidelines do not generally recommend screening at ages younger than 50, we also conducted analyses restricted to participants at least 50 years of age, finding similar AuROCs (Supplemental Figures 3a and 3b). The primary analysis defined early neoplasia based on the pathology interpretation from the endoscopy that led to the referral. We conducted a sensitivity analysis defining early neoplasia based on the pathology interpretation from index endoscopy done after consenting to the study (typically endoscopic mucosal resection), demonstrating similar AuROC for discriminating early neoplasia from no Barrett’s esophagus (Supplemental Figure 4).

Serum Biomarkers

Among the first EGD subjects with Barrett’s esophagus and randomly selected controls without Barrett’s esophagus, the addition of the serum biomarkers did not substantially alter the AuROC compared to the M-BERET alone (0.700 vs. 0.704, p = 0.45). Likewise, the addition of the serum biomarkers did not improve discrimination for early neoplastic Barrett’s esophagus from no Barrett’s esophagus (AuROC 0.776 for M-BERET plus biomarkers vs. 0.766 for M-BERET alone, p = 0.43).

DISCUSSION

We compared and validated available tools for identifying Barrett’s esophagus and early neoplastic Barrett’s esophagus. We found that each of the available tools discriminated Barrett’s esophagus significantly better than relying on GERD symptoms alone. For early neoplastic Barrett’s esophagus, which is arguably a more clinically relevant outcome, the M-BERET, HUNT, and Kunzmann tools all discriminated better than GERD alone or the Gerson tool, with reasonably good AuROCs (0.773, 0.796, and 0.763, respectively). In analyses stratified by sex or GERD symptoms, these 3 tools also performed well. Unfortunately, we did not find that the addition of information from levels of circulating cytokines and adipokines improved the discrimination beyond the clinical tools. While the M-BERET, HUNT, and Kunzmann tools are not accurate enough to rely on for diagnosing Barrett’s esophagus or early neoplasia, providers ought to rely on them rather than GERD symptoms alone for selecting patients to offer endoscopic screening (or non-endoscopic screening methods if those are validated in the future). In particular, the Net Reclassification Improvement demonstrated that up to 46% of individuals without Barrett’s esophagus would be more accurately classified for their risk by use of these tools than by relying on GERD symptoms alone, which could decrease the over-utilization of very low value screening EGDs in those patients. Simultaneously, these tools can be useful to identify patients at risk for Barrett’s esophagus or associated neoplasia even in those without frequent GERD symptoms, who account for approximately one half of individuals destined to develop esophageal adenocarcinoma and who would not otherwise typically undergo screening EGD.⁴²

Most of the tools were more accurate for discriminating early neoplastic Barrett’s esophagus than the largely non-dysplastic Barrett’s esophagus identified among those undergoing their first upper endoscopy. This could be due to the fact that most risk factors for Barrett’s esophagus have stronger magnitude of effects for the risk of esophageal adenocarcinoma. For instance, men are approximately 2.5-fold more likely to have Barrett’s esophagus than women, but six-fold more likely to develop esophageal adenocarcinoma.^{36, 43} Likewise, weekly GERD symptoms has an odds ratio of approximately 5 for esophageal adenocarcinoma, but no association with short segment Barrett’s esophagus, which made up the majority of our patients with non-dysplastic Barrett’s esophagus.^{2, 3}

Since we found the M-BERET, HUNT, and Kunzmann tools to have similar test characteristics, the choice of which tool to use could be guided by practical considerations such as the ease of use. A disadvantage of the M-BERET is it requires measuring waist and hip circumferences, which are not typically performed in clinical practice, and a crude calculation of pack-years of cigarette smoking. In contrast, the HUNT and Kunzmann tools rely on BMI and classify tobacco history as ever/never (HUNT) or current/former/never (Kunzmann), data which are easily and quickly obtainable from medical charts and patient interview. Online tools for calculating the scores are available for M-BERET (http://mberet.umms.med.umich.edu/) and HUNT (https://sites.google.com/view/oacrisk), and Kunzmann, et al published a nomogram for calculating the risk.³⁷

Our study has important limitations. For instance, most of the participants were white non-Hispanics, who are at the greatest risk for Barrett’s esophagus and EAC, and the M-BERET, HUNT, and Kunzmann tools do not incorporate race and ethnicity in their models. The participants were largely referred for endoscopy for clinical indications rather than true screening. The great majority of first EGD patients found to have Barrett’s esophagus had short segment Barrett’s esophagus, which has lower risk of progressing to cancer and is less likely to be associated with GERD than long segment Barrett’s esophagus. And the results for discriminating early neoplastic patients may have been biased by selection effects leading to their initial diagnosis of neoplasia; in particular, 79% had GERD symptoms, whereas only approximately 50% of esophageal adenocarcinoma patients in population-based studies have substantial GERD symptoms.⁴² In addition, the questionnaire responses of the early neoplastic patients may have been biased by recall effects for GERD symptoms. The tools should be additionally validated for outcomes of cancer in population-based studies. In addition, studies are needed to identify the ideal threshold predicted risk in these tools at which screening should be offered, which depends not only on the accuracy of the tools, but the costs and risks of screening and subsequent endoscopic treatment of early neoplasia.

Our study also had substantial strengths. We prospectively recruited the cohorts with the specific purpose of administering the multiple questionnaires and directly compared the available tools to each other, most of which had never been previously externally validated. We found that a few of the tools had substantially worse AuROC for discriminating Barrett’s esophagus than in their initial publication (e.g., Gerson 0.62 vs. 0.72 and Locke 0.66 vs. 0.76) demonstrating the importance of conducting studies to validate prediction tools. In contrast, we found that the M-BERET and Thrift tools had fairly similar AuROC for discriminating Barrett’s esophagus compared to the initial publication (0.70 vs. 0.72 and 0.68 vs. 0.70, respectively) which may have been due to those tools relying on risk factors that had already been demonstrated to be associated with Barrett’s esophagus and the use of a quasi-population or population-based designs. In addition, while each of tools were developed to predict either Barrett’s esophagus (Gerson, Locke, M-BERET, Thrift) or esophageal adenocarcinoma (HUNT and Kunzmann), we validated them for the first time for use in identifying early neoplastic Barrett’s esophagus which is the most clinically relevant outcome since it is the point at which endoscopic therapy is both possible and recommended. Finally, the results were robust to a number of sensitivity analyses, including excluding younger individuals or those with alarm features, and which endoscopy early neoplasia was defined by.

In summary, we compared multiple tools for identifying Barrett’s esophagus and early neoplastic Barrett’s esophagus, finding that the M-BERET, HUNT, and Kunzmann tools are the most accurate, particularly for identifying early neoplastic Barrett’s esophagus. The tools should be further validated for predicting esophageal adenocarcinoma and the ideal threshold predicted risk that should trigger screening needs to be identified.

What You Need to Know.

Background and Context:

Guidelines suggest endoscopic screening of individuals who are at elevated risk for Barrett’s esophagus and esophageal adenocarcinoma. We compared the accuracy of tools for predicting Barrett’s esophagus and discriminating early neoplasia.

New Findings:

All of the tools were more accurate than GERD symptoms alone for predicting Barrett’s esophagus. For discriminating early neoplasia, the HUNT tool, M-BERET, and Kunzmann tool were particularly accurate.

Limitations:

Most participants were non-Hispanic whites and were referred for endoscopy for clinical indications.

Impact:

The HUNT, M-BERET, or Kunzmann tools should be used rather than GERD for identifying individuals at risk for neoplastic Barrett’s esophagus.

Abbreviations:

AAVA

Ann Arbor Veterans Affairs Medical Center

AuROC

area under the receiver operating characteristics

BE

Barrett’s esophagus

BMI

body mass index

CI

confidence interval

EAC

esophageal adenocarcinoma

EGD

esophagogastroduodenoscopy

GERD

gastroesophageal reflux disease

HUNT

Nord-Trøndelag Health Study

IDI

integrated discrimination improvement

IL

interleukin

M-BERET

Michigan Barrett’s Esophagus pREdiction Tool

NRI

net reclassification improvement

OR

odds ratio

PAGI-SYM

Patient Assessment of Upper Gastrointestinal Symptom Severity Index

PPI

proton pump inhibitor

RSQ

Reflux Symptom Questionnaire

UM

University of Michigan