DEVELOPMENT AND VALIDATION OF A NATIONAL US ACHALASIA COHORT: THE VETERANS AFFAIRS ACHALASIA COHORT (VA-AC)

Abstract

BACKGROUND & AIMS:

Achalasia is an esophageal motility disorder associated with significant morbidity, yet achalasia-associated risk factors and outcomes are not well characterized. Our aim was to establish a national cohort of individuals with achalasia, utilizing Veterans Health Administration (VHA) data.

METHODS:

We iteratively developed combinations of International Classification of Diseases (ICD) and Current Procedural Terminology (CPT) code algorithms to validate an approach for identifying achalasia cases. We assessed algorithm accuracy for achalasia diagnosis through manual chart review of candidate achalasia cases and candidate non-achalasia controls. The prespecified endpoint chosen to establish algorithm performance success was achieving a one-sided 95% confidence lower bound for a positive predictive value (PPV) >85% for a random sample of 100 candidate achalasia cases. Once adequate performance was validated, we queried national VA data to establish and characterize a cohort of individuals diagnosed with achalasia between 1999–2020.

RESULTS:

Three rounds of algorithm modification and validation were conducted to achieve the prespecified performance endpoint. In the final round, a combination of 3 or more ICD codes for achalasia in the subject’s lifetime and a CPT code for esophageal manometry, achieved an observed 94% PPV (one-sided 95% confidence lower bound of 88.5%) for identifying achalasia. Applying the algorithm to national VA data identified a cohort of 2,100 individuals with achalasia – median age 65 years and 93% male.

CONCLUSIONS:

Using a rigorous validation approach, we established a national cohort of 2,100 individuals with achalasia within the VHA, one of the largest established to date. This cohort can be utilized to study risk factors for achalasia and outcomes over time.

INTRODUCTION

Achalasia is an esophageal motility disorder associated with significant morbidity.¹ While considered a rare disease, recent studies suggest a rising incidence and prevalence of achalasia,^2–4 in part explained by improved ability to diagnose the disorder, which has significantly evolved over the past two decades.^1–4 Despite the important healthcare burden associated with the rising incidence of achalasia, epidemiology of achalasia has not been well characterized.

Gaps in available knowledge of the epidemiology of achalasia stem from two critical issues which have limited the ability to ascertain meaningful epidemiologic observations: prospective studies have had limited power due to sample size constraints,^5–7 and larger retrospective studies utilizing administrative claims [e.g. International Classification of Diseases (ICD)] codes have not consistently utilized validated approaches for ascertaining achalasia cases.^2,8–11

The lack of large, well-characterized and validated cohorts of patients with achalasia has impeded the ability to identify risk factors for achalasia and understand important treatment outcomes such as quality of life, esophageal cancer, and need for esophagectomy. Therefore, the aim of this study was to establish a first-of-its-kind validated cohort of individuals with achalasia in the United States utilizing Veterans Health Administration (VHA) data.

METHODS

Study Design Overview

We used a systematic, stepwise approach to validate an algorithm of administrative claims codes for identifying individuals with a diagnosis of achalasia cared for by the VHA from 10/1999–12/2020 (Figure 1). First, we created a study basecohort within the total VHA database from which algorithms could be applied, and algorithm positive and negative individuals could be queried. We iteratively applied algorithms, consisting of ICD and/or Current Procedural Terminology (CPT) code combinations, attempting to identify individuals with a diagnosis of achalasia (hereafter, referred to as “true achalasia cases,” see Glossary of Terms¹²). Validity¹³ of the algorithm to identify true achalasia cases was assessed using manual chart review. For each algorithm, a random sample of 100 candidate achalasia cases (algorithm positive individuals) and 100 candidate non-achalasia controls (algorithm negative individuals) were reviewed. After completion of manual chart review, validation performance was assessed by measuring positive predictive value (PPV).¹³ The prespecified primary endpoint chosen to establish valid algorithm performance for identifying a diagnosis of achalasia was achieving a one-sided 95% confidence lower bound for PPV of >85%. If valid algorithm performance was not achieved, the algorithm was adjusted, and the process was repeated. Once optimal algorithm performance was achieved, we applied the algorithm to the entire VA database (including our study base) and identified all cases of achalasia within the VHA between 10/1999–12/2020, thus creating a validated achalasia cohort (hereafter referred to as the VA Achalasia Cohort (VA-AC)).

Figure 1.

Flow diagram of the validation process. GERD, Gastroesophageal reflux disease.

Abbreviations: GERD = gastroesophageal reflux disease; ICD = International Classification of Diseases; VHA = Veterans Health Administration

Data Sources

The VHA is one of the largest integrated health care providers in the US.¹⁴ Since 1999, all VHA facilities have used a universal electronic medical record, which allows clinical data sharing. Data from the millions of clinical encounters through the VHA are collected into a Corporate Data Warehouse and can be used for clinical research.

Study Base and Algorithm Validation Approach

We first established a study base of subjects within the larger VHA database, from which Veterans could be queried and reviewed as candidate achalasia cases and candidate non-achalasia controls (Figure 1). To select individuals with a higher pre-test probability of achalasia, we included individuals in our study base with at least one lifetime diagnosis of either dysphagia or GERD, the most common presenting symptoms of achalasia.^1–4 Creating this study base was essential to review candidate non-achalasia controls, reducing reviewer bias and evaluating for possible false negative controls.

After establishing our study base, we sequentially applied algorithms (Table 1) to identify candidate achalasia cases. Following our previously described approach for algorithm validation,¹⁵ an a priori goal for PPV was defined, and set to a goal of achieving a one-sided 95% confidence lower bound for PPV of >85%. Using previously established calculations, we determined that we would need to sample at least 100 candidate achalasia cases to establish a one-sided 95% confidence lower bound for PPV >85% (this calculation is associated with an overall observed PPV equal to 90%).¹⁵

Table 1.

Validation Algorithms and Performance Measurements

Algorithms	Description of Algorithm ICD and/or CPT Claims Processing Codes	True Positive Cases	Sample Size	PPV (Lower Bound)+	Subject Total from National VHA Database that Satisfies the Algorithm Criteria
Algorithm 1	A single ICD9 or ICD10 code for achalasia in the subject’s lifetime	45	100	45.0% (36.5%)	16,349
Algorithm 2	Two or more ICD9 and/or ICD10 code for achalasia in the subject’s lifetime AND a CPT code for esophageal manometry	87	100	87.0% (80.1%)	2,453
Algorithm 3	Three or more ICD9 and/or ICD10 codes for achalasia in the subject’s lifetime AND a CPT code for esophageal manometry	94	100	94.0% (88.5%)	2,100

^*ICD = International Classification of Diseases; CPT = Current Procedural Terminology; PPV = positive predictive value; VHA = Veterans Health Administration

⁺95% one-sided confidence lower bound based on exact binomial test

After applying an algorithm, we selected a random sample of 100 subjects who were “algorithm positive” (i.e. candidate achalasia cases) to evaluate for true achalasia cases and a random sample of 100 subjects who were “algorithm negative” (i.e. candidate non-achalasia controls) to evaluate for non-achalasia controls using a random number generator. The reference standard for confirming a true achalasia case vs non-achalasia control was manual chart review of free text progress and consultant reports, esophagogastroduodenoscopy (EGD) procedure reports, and esophageal manometry tracings and reports; not all of these reports were available for every subject. After chart review, algorithm performance was assessed using PPV among candidate achalasia cases. If optimal algorithm performance was not achieved (e.g., failure to achieve a PPV associated with a one-sided 95% confidence lower bound of >85%), the algorithm was adjusted, and the process was repeated by reviewing a new random sample. Once optimal algorithm performance was achieved, we applied the validated algorithm to the entire VHA database to identify all cases of achalasia between 10/1999–12/2020 and created a summary of demographic characteristics for the achalasia cohort.

True Achalasia Case Definition and Chart Review Process

Esophageal high-resolution manometry (HRM) is the reference standard for achalasia diagnosis.¹ Barium esophagram (BE) is a useful supportive test in the evaluation of achalasia, with reasonable specificity (88.0%) though suboptimal sensitivity (78.3%) for achalasia when compared to HRM as a reference standard.¹⁶ EGD is a diagnostic test recommended in all suspected cases of achalasia to evaluate for other etiologies of dysphagia and/or obstruction (e.g. malignancy or stricture).¹⁷ Endoscopic features suggestive of achalasia include esophageal dilation, tortuosity, a tight LES with a puckered-appearing gastroesophageal junction, and retained saliva and/or food products in the esophagus. While these features may help support the diagnosis, these features alone are not specific for the diagnosis of achalasia.^1,16

For our study, we used a stepwise approach for defining a true achalasia case starting with application of the most rigorous definition. We defined a manometric true achalasia case as an individual with:

• 1A manometry report or documented manometric data consistent with achalasia per the Chicago Classification v4.0:¹ 100% absent peristalsis (defined as all swallows with either failed peristalsis and/or premature contractions) and an abnormal median IRP.

  AND

• 2An EGD report with documented evaluation of the gastroesophageal junction and gastric cardia, without evident causes of pseudoachalasia (e.g. malignancy, stricture).

We defined a clinically established true achalasia case as an individual with:

• 3A subspecialty progress note or consultant note (limited to gastroenterology and surgery) documenting a diagnosis of achalasia in reference to a prior manometry study (in this instance the available manometry data was incomplete in the procedure reports).

  OR

• 4A subspecialty progress note or consultant note documenting a diagnosis of achalasia based on BE and supported by characteristic features on EGD.

  OR

• 5A subspecialty progress note or consultant note documenting a diagnosis of achalasia with reference to definitive treatment of achalasia, including surgical Heller myotomy, pneumatic dilation, or peroral endoscopic myotomy (POEM).

A subject meeting either the manometric or clinical criteria for achalasia diagnosis on chart review was considered a true achalasia case and utilized as such in the algorithm performance analyses.

Chart review of randomly selected candidate achalasia cases was conducted using a systematic approach. With each queried candidate achalasia case, an ICD diagnosis date for achalasia was determined based on the index ICD code for achalasia. During chart review, we searched for manometry reports within a year (both before and after) of the index ICD code date. If a manometry report was found, the manometry data and tracings were reviewed. If the manometry report was not identified, then subspecialist progress and/or consultant notes were reviewed for reference to manometry data or to a diagnosis of achalasia based on BE, EGD, or prior treatment of achalasia. All progress and consultant notes, BE reports, and EGD reports were reviewed two years before and after the index ICD code date. If the individual met the criteria for a true achalasia diagnosis as above, then they were counted as a true positive case. If the individual did not meet the criteria, then they were counted as a false positive case (Figure 1).

Non-Achalasia Control Definition and Chart Review Process

Candidate non-achalasia controls were queried from the study base as those who were “algorithm negative.” A sample of 100 subjects were randomly selected from the remaining individuals in the study base after the algorithm query was applied, and manual chart review was conducted to assess for achalasia case status.

Chart review was conducted for randomly selected candidate non-achalasia controls using a systematic approach. Since these subjects did not have an anchoring index ICD code date to base a review around, chart review began with a review of old and active medical problems for a diagnosis of achalasia. If there was no reference to achalasia, all consultant notes performed by gastroenterology and/or surgery were reviewed for any documentation of achalasia. If again there was no evidence of a diagnosis of achalasia, then EGD procedures were reviewed for evidence of definitive therapy for achalasia. Additionally, the five most recent primary care notes were reviewed for documentation of achalasia. If the individual had no documentation of achalasia based on the criteria in the above section, then he/she was counted as a true non-achalasia control (i.e. true negative control). If there was documentation of achalasia that met our manometric or clinical criteria above, then then they were counted as a false negative control (Figure 1).

Creating the Validated Achalasia Cohort

Once an algorithm met our a priori one-sided 95% confidence lower bound for PPV of >85%, the valid algorithm was then applied to the entire VHA database to query all possible individuals with achalasia. Demographic and comorbidity variables were gathered for this final validated achalasia cohort. Information regarding age, sex, race/ethnicity, body mass index (BMI), smoking status, aspirin exposure, and prior diagnosis of diabetes mellitus, candida esophagitis, and Barrett’s esophagus was collected at time of achalasia diagnosis.

Statistical Analysis

Structured Query Language (SQL) was used to query Veterans for each validation algorithm. PPV was calculated for all applied algorithms with one-sided 95% exact binomial confidence lower bounds. Summary statistics including median, 25^th and 75^th quartiles, and frequencies were used to characterize the patient population and describe the distribution of Veterans who met criteria for our validated achalasia algorithm. All statistical analyses were computed using Statistical Analysis Software and R.

RESULTS

Study Base

Our study base included 4,117,227 unique Veterans with at least one lifetime ICD code for dysphagia or GERD. Validation algorithms (Table 1) of administrative claims codes for identifying individuals with a diagnosis of achalasia were applied to this study base.

Establishing a Validated Algorithm for Identifying a Diagnosis of Achalasia

The first algorithm (algorithm 1) applied to the study base queried any individual with at least one lifetime ICD code encounter for achalasia. This resulted in 14,700 candidate achalasia cases and 4,102,527 candidate non-achalasia controls. PPV for algorithm 1 to identify true achalasia cases was 45% (one-sided 95% confidence lower bound of 36.5%). All candidate non-achalasia controls reviewed were true negative controls. As the lower bound for PPV was less than the pre-specified goal (>85%), the algorithm was adjusted, and the second algorithm was then applied.

Applied algorithm 2 queried any individual in the study base with two or more lifetime ICD code encounters for achalasia and a CPT code encounter for esophageal manometry, resulting in 2,415 candidate achalasia cases and 4,114,812 candidate non-achalasia controls. PPV for algorithm 2 to identify true achalasia cases was 87% (one-sided 95% confidence lower bound of 80.1%). All candidate non-achalasia controls reviewed were true negative controls. As the lower bound for PPV was still less than the pre-specified goal, the algorithm was adjusted again, and a third algorithm was developed and applied.

Applied algorithm 3 queried any individual in the study base with three or more lifetime ICD code encounters for achalasia and a CPT code encounter for esophageal manometry, resulting in 2,049 candidate achalasia cases and 4,115,178 candidate non-achalasia controls. PPV for algorithm 3 to identify true achalasia cases was 94% (one-sided 95% confidence lower bound of 88.5%). All candidate non-achalasia controls reviewed were true negative controls. As the lower bound for PPV in algorithm 3 met the pre-specified goal, algorithm 3 was maintained as the valid algorithm and the validation process completed.

National Cohort of Individuals with Achalasia

Applying the validated algorithm (algorithm 3) to the entire national VHA database, we identified a cohort of 2,100 total individuals with achalasia from 10/1999–12/2020. 1,552 Veterans with achalasia (74%) were diagnosed from 2009 onward (Figure 2), and there was a wide regional representation of patients with achalasia over the study period (Figure 3). Median age at achalasia diagnosis was 65 years (IQR 52–72) (Table 2), with 719 cases (34%) diagnosed in the sixth decade of life. 1,949 Veterans (93%) were male and 151 (7%) were female. Ethnically, 91% were non-Hispanic. Regarding race, 14 (0.7%) were Asian, 420 (20%) were Black, 9 (0.4%) were Other race, 16 (0.8%) were Pacific Islander, and 1,504 (72%) were White. Median BMI was 28.3 kg/m^² (IQR 24.9–32.3 kg/m^²), and 35% were diabetic. Smoking exposure was present in 1,177 (56%) and aspirin exposure was present in 814 (38.8%). Prior to achalasia diagnosis, candida esophagitis was present in 78 (3.7%) individuals and 95 (4.5%) had Barrett’s esophagus.

Figure 2.

VA-AC achalasia diagnoses by year from 1999 to 2020.

Figure 3.

VA-AC achalasia diagnoses by VA sites across the U.S. from 1999 to 2020.

Table 2.

VA-AC Baseline Characteristics from 10/1999 to 12/2020

Variable	Achalasia Subjects (n = 2,100)
Age, median (IQR)	65 (57–72)
Sex
Male, n (%)	1949 (92.8)
Female, n (%)	151 (7.2)
Ethnicity
Hispanic, n (%)	142 (6.8)
Non-Hispanic, n (%)	1908 (90.9)
Unknown, n (%)	50 (2.4)
Race
Asian, n (%)	14 (0.7)
Black, n (%)	420 (20.0)
Other, n (%)	9 (0.4)
Pacific Islander, n (%)	16 (0.8)
White, n (%)	1504 (71.6)
Unknown, n (%)	137 (6.5)
BMI, median (IQR)	28.3 (24.9–32.3)
Smoking Status
Current, n (%)	646 (30.8)
Former, n (%)	531 (25.3)
Never, n (%)	722 (34.4)
Unknown, n (%)	201 (9.6)
Aspirin Exposure
Yes, n (%)	814 (38.8)
Diabetes
Yes, n (%)	735 (35.0)
Candida Esophagitis
Yes, n (%)	78 (3.7)
Barrett’s Esophagus
Yes, n (%)	95 (4.5)

IQR = interquartile range; BMI = body mass index

DISCUSSION

We applied a rigorous, stepwise approach to validate an algorithm using administrative claims codes for identifying achalasia diagnoses, and then applied this validated algorithm within the VHA database to create a large, nationally representative cohort of 2,100 individuals with achalasia (termed the VA-AC). Despite an increasing prevalence of achalasia, well-characterized large cohorts of patients with achalasia have not been available, limiting impactful epidemiologic and outcomes studies of achalasia. Prior epidemiologic observations could be subject to bias due to small sample sizes in prospective studies and lack of validated approaches in retrospective studies using administrative claims codes for achalasia case ascertainment (Table 3 includes prior epidemiology studies of achalasia).^{2–11,18–23}

Table 3.

Prior Epidemiologic Studies of Achalasia, Sorted by Terminal Study Year

Study	Location	Years Studied	Study Design	Case Definition Criteria*	Cases Reviewed or Claims Codes Validated	Number Achalasia Patients Reported	Incidence rates (per 100,000/year)	Prevalence rate (per 100,000)
Mayberry et al	Cardiff, Wales	1926–1977	Retrospective	(1) EGD and BE or (2) Manometry	Yes	48	0.4	Not Reported
Mayberry et al	Nottingham, England	1966–1983	Retrospective	(1) EGD and BE or (2) Manometry	Yes	53	0.51	8
Arber et al	Israel	1973–1983	Prospective	2 of 3 studies: EGD, BE, Manometry	Yes	162	0.8–1.1	7.9–12.6
Howard et al	Edinburgh, Scotland	1986–1991	Prospective	Manometry	Yes	36	0.81	Not Reported
Ho et al	Singapore	1989–1996	Prospective	Manometry	Yes	49	0.3	1.8
Birgisson et al	Iceland	1952–2002	Retrospective	(1) Symptoms of achalasia and (2) exclusion of other esophageal disorders and either (3) BE or (4) Manometry	Yes	62	0.55	8.7
Farrukh et al	Leicester, England	1986–2005	Retrospective	(1) EGD and BE or (2) Manometry	Yes	14	0.89	Not Reported
Gennaro et al	Italy	2001–2005	Retrospective	Claims based – single ICD code for achalasia	No	565	1.59	Not Reported
Sadowski et al	Alberta, Canada	1995–2008	Retrospective	Claims based – single ICD code for achalasia + specialist note + CCP treatment code for balloon dilation or Heller myotomy	Yes (no specific criteria)	463	1.63	10.82
Enestvedt et al	United States	2000–2008	Retrospective	Claims based – EGD performed with ICD code for achalasia associated	No	896	Not Reported	Not Reported
Kim et al	Korea	2007–2011	Retrospective	Claims based – single ICD code for achalasia	No	3,105	0.39	6.29
Duffield et al	Australia	2004–2013	Retrospective	Manometry and EGD	Yes (gold standard – manometry)	350	2.3–2.8	Not Reported
Samo et al	Chicago, United States	2004–2014	Retrospective	Claims based – single ICD code for achalasia and free text search for EGJOO	Yes (gold standard – manometry)	379	0.77–1.35	4.68–14.42
Harvey et al	England	2006–2016	Retrospective	Claims based – single ICD code for achalasia	Hospital Episode Statistics (HES) – Yes; The Health Improvement Network (THIN) - No	HES – 10,509; THIN - 711	HES – 1.99; THIN – 1.53	THIN – 27.1
Sato et al	Japan	2005–2017	Retrospective	Claims based – single ICD code for achalasia	No	385	0.81–1.37	7
Gaber et al	United States	MarketScan 2001–2018, Medicare 2007–2015	Retrospective	Claims based – single inpatient or outpatient ICD code for achalasia	No	MarketScan – 2900 Medicare – 4907	MarketScan – 10.5 Medicare – 26.0	MarketScan – 18 Medicare – 162.1

EGD = esophagogastroduodenoscopy; BE = barium esophagram; ICD = International Classification of Diseases; EGJOO = esophagogastric junction outflow obstruction

^*Diagnostic studies or claims-based code criteria used in the study to identify achalasia cases

This first-of-its-kind cohort of individuals with achalasia was developed utilizing a systematic validation process, a critical feature of a reliable dataset. Prior studies have repeatedly highlighted shortcomings of reliance on a single ICD billing code for achalasia. In a population-based, retrospective study of individuals with achalasia in Iceland from 1952–2002, Birgisson et al. identified a 41% PPV with an ICD code for achalasia (530.0 and/or K22.0) based on procedure results (BE and manometry) and symptoms consistent with achalasia.²⁰ Similarly, in a population-based study of achalasia in Canada from 1996–2007, Sadowski et al. identified up to 50% of patients with an ICD code (530.0) for achalasia did not truly have a clinical diagnosis.³ Similar to these studies, we also found that a single ICD code for achalasia (530.0 or K22.0) had low PPV for a true achalasia diagnosis, with a PPV of just 45%. These findings support that querying achalasia cases using only an ICD code for achalasia is insufficient for identifying true achalasia cases.

Suboptimal performance of ICD codes for identifying achalasia may limit the impact and interpretability of prior epidemiologic studies which have utilized ICD coding alone, without validation, in the study design.^2,8–11 Authors of these studies acknowledge limitations such that “manometric data were not reviewed,” and “pseudoachalasia” or “other esophageal motility disorders such as diffuse esophageal spasm” could not be excluded. ^2,8–11 For example, our study would have included 14,700 “achalasia cases” using an ICD code alone within the VHA database; however approximately half or more of these candidate cases likely did not have a true diagnosis of achalasia. A recent study using US MarketScan and Medicare data selected individuals with “at least 1 claim with an ICD diagnosis code” for achalasia and reported a “strikingly higher incidence and prevalence” of achalasia than previously described in the literature.² While the detection of achalasia has certainly increased over time,⁴ incidence and prevalence rates relying on ICD coding alone for the definition of an achalasia case likely overestimate the true incidence and prevalence of the disease.

We identified a simple combination of ICD and CPT codes that achieved high PPV for identifying true achalasia cases. Our validated algorithm was likely successful for two main reasons. One, we incorporated a claims code (CPT) requirement for esophageal manometry, the reference standard for achalasia diagnosis. With the evolution of the Chicago Classifications,¹ manometry can distinguish achalasia from other esophageal motility disorders which may present with similar clinical presentations and nonspecific endoscopic evaluation. Secondly, we increased the number of ICD encounter codes which likely eliminated miscoding and improved selection for achalasia, serving as a proxy for ongoing clinical follow-up for achalasia. Algorithm 2 resulted in a marked improvement in PPV from our first Algorithm 1, with a PPV of 87%. Increasing the requirement to a minimum of three ICD code encounters for achalasia (Algorithm 3) resulted in a PPV of 94% with a 95% confidence lower bound of 88.5%, satisfying our a priori PPV goal of >85%.

Our validated cohort will allow us and other investigators to address important gaps in knowledge pertaining to achalasia epidemiology. For example, limited data are available on racial and ethnic differences in achalasia.^10,11,24 Future planned studies utilizing this cohort include evaluating trends in incidence of achalasia stratified by race, as well as identifying health disparities, if present, in outcomes and care. Additionally, we anticipate identifying risk factors for esophageal cancer in achalasia, developing a risk prediction tool for cancer which may further generate hypotheses about pathogenesis and possible surveillance strategies. Other important epidemiologic projects include, but are not limited to, identifying factors associated with treatment outcomes to better delineate treatment paths for specific populations, and generating hypotheses about the root-pathogenesis/mechanisms for achalasia development.

There are several notable strengths of our study. We used a rigorous, previously established systematic approach to algorithm application and chart review.¹⁵ During chart review, 73 of 94 achalasia subjects (78%) had sufficient manometric data to confirm achalasia diagnosis based on manometric criteria alone (Supplemental Table). Additionally, measurement approaches for all demographic information used in our cohort analyses have been previously validated by other investigators.^24–26 To our knowledge this is the largest validated US cohort of achalasia patients described to date. There are important limitations which may be considered in interpreting this report. First, it is unclear if our validated algorithm is applicable to other healthcare systems, offering an opportunity for future work with non-VHA data. Additionally, our validated cohort is representative of a VHA patient population and may limit generalizability to the entire US population. The vast majority were male (93%) and White (72%), with a median age of achalasia diagnosis of 65 years, ranging from 19 to 100 years. Although our cohort has a significantly higher male predominance, which may limit the generalizability of our cohort at this time, we anticipate that the absolute number of females with achalasia will continue to increase over time with the changing landscape of the US Armed Forces.^27,28 Additionally, it is important to note that prior epidemiologic studies have shown conflicting data regarding sex and age differences in achalasia.^{2–11,19–24} Another limitation is that manometry reports and tracings were, at times, unavailable as a comprehensive study report, and, as such, achalasia sub-types as well as opiate-related achalasia types could not be adequately ascertained for the entire cohort. Since achalasia sub-type cannot be queried using administrative claims codes, it is not feasible to obtain data regarding achalasia sub-types without extensive chart review for all achalasia cases. Additionally, index ICD code may not reflect the true date of achalasia diagnosis, and we cannot determine who may have been diagnosed with achalasia prior to the CDW creation in 1999. Since CPT codes for esophageal manometry reports were required in our final algorithm, our cohort may only comprise patients seen at VA centers who have the capacity to perform esophageal manometry. It is important to note however, that there was a wide distribution of subjects in our cohort spanning 38 states across the US. Another limitation of our study is some individuals with achalasia may have been missed by our algorithm, as our approach was designed to optimize positive predictive value, rather than sensitivity for detecting all achalasia cases. The aim of this project was to create a cohort of individuals with a high confidence diagnosis of achalasia, one which we could confidently use to explore epidemiologic gaps in knowledge regarding achalasia and examine associations between achalasia and our measured outcome(s). As such, our algorithm may not be fully sensitive for identifying all individuals with achalasia within the VA population, but we postulate our algorithm does result in an achalasia cohort where the diagnosis of achalasia is a high confidence diagnosis. Lastly, given the large number of algorithm negative candidates queried for each applied algorithm, we could not confidently evaluate other performance measures such as negative predictive value, sensitivity, and specificity for our algorithms.

CONCLUSIONS

Using a rigorous approach to develop and validate an algorithm for identifying individuals with achalasia, we established a population-based cohort of 2,100 individuals with achalasia within the VHA, one of the largest cohorts established to date. Our final validated algorithm for achalasia queried individuals with at least three ICD encounter codes for achalasia and a CPT code for esophageal manometry. This algorithm resulted in a 94% observed PPV with a one-sided 95% confidence lower bound of 88.5% for detecting true achalasia cases. We also found that using an ICD code alone for identifying individuals with achalasia had poor PPV of just 45%, suggesting that this approach should not be used as a strategy for epidemiologic studies examining achalasia risk factors and outcomes. Overall, we hope that this validation scheme will help set a benchmark standard for epidemiologic research in achalasia and other rare esophageal motility disorders. We aim to use our validated achalasia cohort to study risk factors for achalasia and outcomes over time, helping fill existing gaps in achalasia research.

What you need to know:

BACKGROUND:

Achalasia is associated with significant morbidity; however, evaluation of associated risk factors and outcomes is limited due to a lack of large, validated cohorts for study.

FINDINGS:

Using a rigorous, stepwise approach we developed a validated algorithm for identifying true achalasia cases, with a positive predictive value of 94%. Applying the algorithm to national Veterans Health Administration data identified a large, nationally representative cohort of 2,100 individuals with achalasia.

IMPLICATIONS FOR PATIENT CARE:

This validated VA Achalasia Cohort (termed the VA-AC) will allow further study development to address important gaps in knowledge pertaining to achalasia epidemiology.

Abbreviations:

BE

barium esophagram

BMI

body mass index

CDW

Corporate Data Warehouse

CPT

Current Procedural Terminology

EGD

esophagogastroduodenoscopy

GERD

gastroesophageal reflux disease

HRM

high-resolution manometry

ICD

International Classification of Diseases

IRP

integrated relaxation pressure

LES

lower esophageal sphincter

POEM

peroral endoscopic myotomy

PPV

positive predictive value

SQL

Structured Query Language

VA-AC

Veterans Affairs Achalasia Cohort

VHA

Veterans Health Administration

Glossary of Terms

Algorithm

A combination of administrative claims codes used to identify individuals with achalasia

True achalasia case

An individual meeting either the manometric or clinical study criteria for an achalasia case diagnosis

Non-achalasia control

An individual not meeting the study criteria for an achalasia case diagnosis

Validity

The degree in which a measurement represents the phenomena it is intended to measure

Valid algorithm performance

A priori goal for an algorithm to identify a diagnosis of achalasia – defined as a one-sided 95% confidence lower bound for PPV >85%

Positive predictive value (PPV)

The proportion of individuals positive for the algorithm criteria who have a confirmed diagnosis of achalasia based on manual chart review (positive achalasia cases among individuals positive for the algorithm)