Developing A Novel Algorithm to Identify Incident and Prevalent Dementia in Medicare Claims. The ARIC Study

Abstract

There is an urgent need to improve dementia ascertainment robustness in real-world studies assessing drug effects on dementia risk. We developed algorithms to dementia identification algorithms using Medicare claims (inpatient/outpatient/prescription) from 3,318 Visit 5 (2011–2013) and 1,828 Visit 6 (2016–2017) participants of the Atherosclerosis Risk in Communities (ARIC) Study, validated against ARIC’s rigorous syndromic dementia classification. Algorithm performance was compared to existing algorithms (Jain, Bynum, Lee). We further evaluated algorithms effectiveness in a 20% random Medicare sample aged ≥70 who initiating liraglutide or dipeptidyl peptidase 4 inhibitors (DPP4i) to assess 3-year adjusted risk difference (aRD) for dementia.

Our incident dementia algorithm required two dementia diagnostic codes within 1-year, or one dementia code plus a new dementia prescription within 90-days. It achieved a positive predictive value (PPV) of 69.2%, specificity of 99.0%, and sensitivity of 34.6% (population prevalence: 8.8%), comparable to extant algorithms (PPV 58.7~68.6%; sensitivity 25.5~40.4%). Prevalent dementia algorithm (without requiring incident diagnoses/prescriptions) demonstrated similar performance. In the Medicare sample, dementia risk ranged from 3.0%–12.5%, aRD comparing liraglutide to DPP4i varied −1.2% to −3.6%, with our algorithm closely matching the Bynum algorithm. Algorithm selection significantly impacts treatment effect estimates, highlighting its importance in in pharmacoepidemiologic research.

BACKGROUND:

Alzheimer’s disease (AD) and related dementias (ADRD) currently affect 50 million people worldwide¹. Only two disease modifying agents—lecanemab² and donanemab-azbt³ (approved in 2023 and 2024, respectively) and four symptom-relief drugs (donepezil, rivastigmine, galantamine, and memantine)—are available⁴. Given the modest benefits of these agents and the slow pace of new therapeutic development, drug repurposing (DR)—identifying novel indications for approved medications—offers a faster, lower-cost alternative^5,6. Pharmacoepidemiologic analyses using real-world data⁷, such as insurance claims from the Centers for Medicare and Medicaid Services (CMS), provide cost-effective, generalizable methods to evaluate DR candidates^8–12. However, assessing drug effectiveness within real-world settings relies on the accurate ascertainment of ADRD status,¹³ which has proven difficult from administrative claims.

Existing dementia identification algorithms from CMS Medicare claims^14–21 have been based on ICD-9 diagnostic codes only²², used a small validation set^19,22, or lack rigorous internal and external validation¹³. Pharmacoepidemiologic studies evaluating drug effectiveness in dementia prevention employ a high positive predictive value (PPV) algorithm to minimize false positives^24,25, erring on the side of excluding individuals without dementia. Thus, we developed dementia classification algorithms to identify incident and prevalent dementia in CMS Medicare claims in comparison with dementia gold standard classification based on an extensive battery of neurocognitive tests conducted at repeat clinic visits in the Atherosclerosis Risk in Communities (ARIC) Study²⁶. Using a 20% Medicare sample, we conducted an empirical study comparing the performance of the developed algorithm and selected existing dementia identification algorithms in the identification of dementia risk.

METHODS:

Study population

Details describing the ARIC cohort have been published elsewhere²³. Briefly, 15,792 men and women 45–64 years of age were recruited through population-based sampling from 4 geographically distinct U.S. communities of Washington County, MD, Jackson, MS, Forsyth County, NC, and selected suburbs of Minneapolis, MN²³. ARIC participants have been followed from the baseline clinic-based examination (1987–1989) through repeat clinic visits, active surveillance of hospitalizations, and annual (semi-annual since 2012) telephone follow-up interviews. Each visit (V)’s research protocol received approval from the Institutional Review Boards of the ARIC Study field centers, and informed consent was obtained from all participants.

Dementia status ascertainment in ARIC cohort

An in-person battery of neuropsychological tests was conducted in a quiet room by trained examiners using standardized protocols at V2 (1990–1993), V4 (1996–1998), V5 (2011–2013), and subsequent visits. During V2 and V4, the neurocognitive test battery included Word Fluency Test²⁷, Digit Symbol Substitution²⁸, and Delayed Word Recall tests²⁹. Starting from V5, the battery incorporated Mini-Mental State Examination, Digit Span Backwards²⁸, Boston Naming³⁰, Animal Naming²⁷, Wechsler Memory Scale-III, Trail Making Tests A and B³¹, Incidental Learning³², and Logical Memory tests²⁸. An algorithmic cognitive diagnosis based on the test information was performed³³, which is in line with the formulation of dementia in the National Institute on Aging–Alzheimer’s Association workgroups^34,35 and Diagnostic and Statistical Manual of Mental Disorders (5th Edition)^36–38 and could be altered by expert committee review. We used the robust ARIC dementia criterion standard³⁸, defined as the expert committee diagnosis based on the in-person cognitive evaluation or an algorithmic diagnosis. Any positive dementia classification at V5 was carried forward.

Linkage with CMS Medicare claims

Data for ARIC cohort participants were linked with the CMS Medicare claims for the years 1991–2018 using a finder file that included participants’ social security numbers, gender, and date of birth³⁹. We focused on hospitalizations and outpatient events occurring during the years 2011–2018. Information concerning ARIC study participant enrollment in fee-for-service (FFS) Medicare was obtained from monthly indicators of enrollment in Part A and B. Continuous enrollment periods were established to indicate uninterrupted CMS Medicare FFS coverage, defined as enrollment in CMS Medicare Part A and B and lack of enrollment in a Medicare Advantage (HMO) plan.

In the identification of dementia status from ARIC participants’ Medicare claims, we used a broad range of claims, including those obtained from Inpatient, Outpatient, Home Health Agency, Skilled Nursing Facilities (SNF)⁴⁰, and Carrier^. Files⁴¹. For inclusion in the study, ARIC participants were required to have non-missing dementia classification, and to be continuously enrolled in Medicare Parts A and B (fee-for-service, FFS Medicare) for ≥ 12 months preceding the date of V5 and V6, respectively (Figure 1), and provided consent for their data to be used in research. For algorithms involving dementia treatment, we additionally required participants to have ≥ 12 months of continuous enrollment in Medicare Part D prior to the ARIC visit.

Figure 1. Study design for developing the dementia identificaiton algoirthm.

Development of incident and prevalent dementia algorithms

A person-level file, aggregating all dementia diagnostic codes (identified by International Classification of Diseases (ICD), Ninth Revision, Clinical Modification (CM) (i.e., ICD-9-CM) or ICD-10-CM codes, Tables S1 & S2), was compiled for each cohort member based on claims for inpatient and outpatient encounters occurring in all-available or 1-year time window before ARIC V5 and before ARIC V6 (2016–2017), the observation period. The dementia codes preceding V5 and V6 revealed that only a small proportion of participants had inpatient dementia diagnostic codes, and among those, only a small proportion were in the primary or secondary position, suggesting dementia as an acute cause for hospitalizations (Appendix S1). This finding is consistent with our comprehensive literature review^14–21, which suggested that most dementia diagnoses occur in the outpatient setting.

Thus, without restricting to specific sources or positions of the diagnostic codes, we developed 11 algorithm candidates using a random 60% sample of participants with dementia classification at V6 as the training set and tested their performance in the remaining 40% of the data. The algorithm candidates accounted for factors such as code definition (inclusion of non-specific ICD-9 codes), encounter type (inpatient/outpatient), temporality/frequency of diagnosis, SNF stay, and dementia treatment (Table 1; Appendix S2).

Table 1.

Claims-based algorithm candidates to identify incident and prevalent dementia^a.

Algorithm	Components included							Definition of candidate algorithm
	1st Dx	2nd Dx	non-specific codes	Rx	Temporality of Rx	180-day washout of Rx	14-day of SNF
1	Yes	No	Yes	No	NA	NA	No	requiring 1 dementia Dx code except those 4 non-specific ICD-9 codes (290.40, 290.41, 290.11, 290.3).
2	Yes	Yes	No	No	NA	NA	No	requiring 2 dementia Dx codes for ≥ 1 day apart within a 1-year period.
3	Yes	Yes	No	No	NA	NA	Yes	requiring 2 dementia Dx codes for ≥ 1 day apart within a 1-year period; OR a dementia code preceding or preceded by a cumulative total of ≥ 14 days of SNF stay within a 1-year period.
4	Yes	Yes	Yes	No	NA	NA	No	a dementia Dx code was followed by a 2nd dementia code for ≥ 1 day apart within a 1-year period, ≥ 1 of the codes are not those 4 non-specific ICD-9 codes.
5	Yes	Yes	Yes	No	NA	NA	Yes	a dementia Dx code was followed by a 2nd dementia code for ≥ 1 day apart within a 1-year period, ≥ 1 of the codes are not those 4 non-specific ICD-9 codes; OR a dementia code preceding or preceded by a cumulative total of ≥ 14 days of SNF stay within a 1-year period.
6	Yes	No	Yes	No	NA	NA	No	requiring 1 inpatient dementia Dx code except those 4 non-specific ICD-9 codes;
7	Yes	Yes	No	Yes	Yes	Yes	No	2 dementia Dx codes ≥ 1 day apart within a 1-year period; OR a single dementia Dx followed by a new Rx for dementia treatment within the next 90 days, provided there was no such Rx in the 180 days preceding the Dx*.
8	Yes	Yes	No	Yes	No	Yes	No	a dementia Dx code must be followed by either another dementia code ≥ 1 day apart but within a 1-year period, OR the initial dementia code must be preceded or followed by a new Rx for dementia treatment within 90 days. Additionally, there must be no Rx for dementia treatment in the 180 days prior to the new Rx*.
9	Yes	Yes	No	Yes	No	No	No	a dementia Dx code was followed either by a 2nd dementia Dx code for ≥ 1 day apart within a 1-year period, OR the initial dementia code must be preceded or followed by a Rx for dementia treatment within 90 days.
10	Yes	Yes	Yes	Yes	Yes	Yes	No	a dementia Dx code was followed either by a 2nd dementia Dx code for ≥ 1 day apart within a 1-year period (≥ 1 the codes are not those 4 non-specific codes if ICD-9 codes are involved), OR by a new Rx of dementia treatment in the next 90 days (not preceded by such Rx in 180 days prior to diagnosis).
11	Yes	Yes	No	Yes	Yes	Yes	No	a dementia Dx code was followed by a 2nd dementia Dx code for ≥ 1 day apart, ≥ 1 of the codes are not those 4 non-specific codes; OR requiring 1 inpatient dementia Dx code except those 4 non-specific codes for ≥ 1 day apart within a 1-year period; OR a dementia Dx code followed by a new Rx of dementia treatment in the next 90 days and not preceded by such Rx in 180 days prior to Dx.

Abbreviations: Dx, diagnosis; Rx, prescription; 2^nd, second; SNF, skilled nursing facility; NA, not applicable. Performance of the 11 algorithm candidates is shown in Appendix S2.

^aDiagnostic codes are presented in Tables S1 and S2.

In the development of incident dementia algorithms involving dementia treatment, we limited analyses to 1,108 participants with continuous Medicare Part D coverage between visits V5 and V6, out of the 1,670 participants with dementia status ascertainment at both V5 and V6 and dementia-free at V5 (Figure 2). The sample size was further reduced to 881 when requiring a 180-day washout period for new dementia treatment prescription. Similarly, in the development of prevalent dementia algorithms involving dementia treatment, we limited analyses to 1212 participants with ≥ 1-year continuous Medicare Part D prior V6 (out of the 1,828 participants with dementia status at V6).

Figure 2. Flow chart of eligibility for analyses comparing dementia diagnosis in ARIC cohorts and in Medicare claims at V6 (2016–2017).

Rx, prescription. Samples for incident dementia are shown in the boxes on the right. The 3-year lookback and 4-year lookback windows (Italic fonts) are required for the Jain algorithm and its relaxed version, respectively.

From the listed 11 algorithm candidates, our final choices for identifying incident and prevalent dementia are described in Table 2 (algorithms 8 and 9), because they demonstrated: 1) the highest PPV in the ICD-10 era (although differences were not statistically significant), 2) flexible requirements (e.g., without requiring SNF stay), and 3) potential for enhanced PPV by including prescriptions for dementia treatment²¹.

Table 2.

Selected Algorithms from the claims-based algorithm candidates.

Incident dementia: a dementia diagnosis code must be followed by either another dementia code at least one day apart but within a 12-month period, OR the initial dementia code must be preceded or followed by a new prescription for dementia treatment within 90 days. Additionally, there must be no prescription for dementia treatment in the 180 days prior to the new prescriptiona. Prevalent dementia: a dementia diagnosis code must be followed by either another dementia code at least one day apart but within a 12-month period, OR the initial dementia code must be preceded or followed by a prescription for dementia treatment within 90 days.b

^aThis incident dementia identification algorithm (candidate algorithm 8) requires patients to have continuous Medicare Part D enrollment between V5 and V6. In pharmacoepidemiologic studies, this criterion could be adapted by requiring a minimum length of continuous Medicare Part D enrolment, e.g. 1-year.

^bThis prevalent dementia identification algorithm (candidate algorithm 9) requires patients to have ≥ 1-year continuous Medicare Part D enrollment prior to V6.

An incident dementia case (Table 2) was defined by the presence of a dementia diagnostic code followed by either 1) another dementia diagnostic code at least one day apart but within a 12-month period, OR 2) a new dementia treatment prescription either preceded or followed by a dementia diagnostic code within 90 days. Dementia treatments considered include four symptom-relief drugs only (as newer FDA-approved disease modifying agents are not available in the 2016–2017 claims data).

Note that incident dementia cases were identified from claims for inpatient and ambulatory care visits occurring between V5 and V6 among V6 participants free of dementia at the baseline V5, using all available follow-up time between V5 and V6 (Figure 2). That window of observation included the year 2015 when healthcare systems in the United States adopted the ICD-10 coding criteria (10/1/2015). In primary analyses, to capture the entire period of observation between V5 and V6, we used both ICD-9 and ICD-10 diagnostic codes. In secondary analyses, to capture only the ICD-10 diagnoses, we limited the look-back period to any time after 10/1/2015.

For prevalent dementia identification (Table 2), we used the incident dementia algorithm except relaxing the requirement for a new prescription by omitting the 90-day washout period. We assessed prevalent dementia using Medicare claims from participants with dementia classification before V6 and V5 (Figure S1), respectively.

Existing algorithms for dementia identification

Performance of the new algorithms against the gold standard ARIC classification was compared to the performance of 3 representative extant algorithms, selected for their distinct methodologies (Table 3). The Jain¹⁶ algorithm requires either ≥ 1 dementia code in ≥ 2 different years, or ≥ 1 dementia code and a 6-month nursing home stay in a 3-year window¹⁴; the Jain-relaxed algorithm expands this window to 4 years. The Bynum²⁰ algorithm requires either ≥ 1 dementia claim from Inpatient, Home Health, or Hospice files, or ≥ 2 dementia claims in the Carrier or Outpatient files that are ≥ 7 days apart; the Bynum-relaxed version added scenarios allowing dementia identification through related evaluation procedures or depression diagnosis. The Lee¹⁹ algorithm identifies dementia from diagnosis coded in primary/secondary positions in any inpatient, SNF, home health agency, hospital facility claims, and Carrier claim files; the Lee-relaxed algorithm allows dementia diagnostic codes in all positions.

Table 3.

Three existing claims-based dementia identification algorithms and their relaxed versions.

Jain 19,a	involves in a 3-year lookback window either (1) ≥ 1 claim with a code for ADRD in ≥ 2 different years, or (2) ≥ 1 ADRD claim and a total of ≥ 6-month nursing home stay.
Bynum 25,b	requires either (1) ≥ 1 dementia claim from MedPAR, Home Health, or Hospice files, or (2) ≥ 2 dementia claims in the Carrier or outpatient files that are ≥ 7 days apart. The inclusion criteria are continuous enrollment in Medicare FFS Part A & B for ≥ 6 months before and following their Health and Retirement Study interview or until death.
Lee 24,b	requires any dementia diagnostic codes in any inpatient, skilled nursing facility, home health agency, hospital outpatient, and Carrier claim files as one of the primary or secondary diagnosis codes. The inclusion criteria are enrollment in Part A & B FFS plan for ≥1 month.
Jain-relaxed	requires ≥ 1 dementia claims in 1 year AND (≥ 1 dementia claim in another year in the 4-year period OR nursing home stay ≥ 6 months in the 4-year period). The inclusion criteria are enrollment in Part A & B FFS plan for ≥1 month.
Bynum-relaxed	Bynum algorithm OR (3) ≥1 dementia claim AND ≥ 1 depression claims OR (4) dementia evaluation procedure AND ≥ 1 depression claims OR (5) dementia evaluation procedure AND ≥ 1 dementia claims. The inclusion criteria are 1-year continuously enrollment in Medicare FFS Part A & B for ≥ 6 months before and following their HRS interview or until death*.
Lee-relaxed	any codes in any inpatient, skilled nursing facility, home health agency, hospital outpatient, and carrier claim files. And the inclusion criteria are enrollment in Part A & B FFS plan for ≥1 month.

^aWhen implementing the Jain algorithm, we require a participant to have ≥ 3-year lookback window, thus the sample size for Jain algorithm is smaller than that for others.

^bOur Medicare-ARIC linkage include only the MedPAR (inpatient), Outpatient, Carrier, and Part D (medication) claims. Home Health and Hospice claims are not available for this cohort for the period examined in this study.

To assess comparative performance across all algorithms, we required study participants to have ≥ 1 year (≥ 3 and ≥ 4 years for Jain and Jain-relaxed algorithms, respectively) of continuous enrollment in Medicare Parts A and B, plus a ≥ 1-year enrollment in Part D prior to V5 and continuous Part D coverage between V5 to V6, for treatment-based algorithms. ICD-10 code sets for Lee et al. were obtained via forward/backward General Equivalence Mappings from their original ICD-9 codes.⁴²

Statistical analysis

We described the distribution of demographic, comorbidities, and cognitive function among V6 participants with dementia classification who had ≥ 1-year of continuous Medicare Parts A/B enrollment. Main analyses evaluated the performance (sensitivity, specificity, PPV, and negative predictive value (NPV)) of the incident dementia algorithms compared to the ARIC gold standard. Performance of the prevalent dementia algorithms was assessed at V5 and V6. Sensitivity analyses included: 1) examine algorithm performance using only 1-year only look-back period; 2) comparing algorithms based on ICD-9/ICD-10 codes vs ICD-10 codes alone; and 3) assessing relative performance of the selected vs. existing algorithms using the same inclusion criteria established for our algorithm. Algorithm performance was further stratified by age groups (66–75, 76–80, 81–100 at V5; 71–80, 81–100 at V6), sex, and race (White vs Black). Analyses were conducted using SAS 9.4 (SAS Inc., Cary, NC).

Empirical example

To demonstrate the application of our algorithm and existing algorithms in a real-world study, we implemented an active-comparator, new-user cohort design^7,43 identifying initiators of type 2 diabetes drug classes: glucagon-like peptide-1 receptor agonists (including liraglutide, a repurposing candidate for ADRD^44,45) and dipeptidyl peptidase 4 inhibitors (DPP4i⁴⁶), within a 20% random sample of fee-for-service US Medicare beneficiaries aged 70+ with continuous parts A, B, and D coverage from 2010 to 2019 (Figure 3). Eligible beneficiaries were required to have a second prescription for the same drug class, continuous enrollment in parts A and B, and no ADRD diagnosis in the 3 years¹⁶ prior to drug initiation and were followed from 180-day (latency period) post the second prescription to a maximum of 3 years (to increase the probability of identifying dementia¹⁶). Dementia risk was assessed using our incident dementia algorithm, alongside Jain, Bynum, and Lee algorithms. For our algorithm, we additionally censored beneficiaries at the time their Part D coverage ended. Propensity scores (PS) were estimated to balance measured confounders across the cohorts using inverse probability of treatment weighting (IPTW). We calculated crude dementia rate and 3-year risk difference (RD), and Kappa coefficients for algorithm agreement.

Figure 3. Study design for empirical study.

For our algorithm, we additionally censored beneficiaries at the time their Part D coverage ended.

RESULTS

Participant characteristics

Among the 6,515 participants who completed V5, 3,318 (mean age 75.8 (SD 5.2) years, 76.8% White, 59.8% female) met eligibility criteria for prevalent dementia analysis at V5 (non-missing dementia ascertainment, Medicare FFS enrollment for ≥ 1 year prior to the visit, and research consent; Figure S1). Of those, 190 (5.7%) were classified as having dementia, with 98 (51.6%) over age 80 years (Table 4). Participants with dementia had higher cardiometabolic disease burden and frailty compared to those without dementia.

Table 4.

Baseline characteristics for participants with dementia classification and at least 1-year of continuous enrollment in FFS Medicare Parts A and B at V5 (n=3,318) and V6 (n=1,828). The ARIC Study cohort.

Characteristic	V5				V6
	No prevalent dementia N=3,128	Prevalent dementia N=190	Total N=3,318	Missing # (%)	No prevalent dementia N=1,668	Prevalent dementia N=160	Total N=1,828	Missing # (%)
Age, years	75.8 (5.22)	79.9 (5.24)	76.1 (5.30)	0 (0.0)	79.6 (4.72)	82.9 (5.35)	79.9 (4.87)	0 (0.0)
Age group				0 (0.0)				0 (0.0)
66−75	1,644 (52.5)	49 (25.8)	1,693 (51.0)		372 (22.3)	18 (11.3)	390 (21.3)
76−80	828 (26.5)	43 (22.6)	871 (26.3)		667 (40.0)	41 (25.6)	708 (38.7)
81−85	524 (16.8)	64 (33.7)	588 (17.7)		407 (24.4)	46 (28.8)	453 (24.8)
86+	132 (4.2)	34 (17.9)	166 (5.0)		222 (13.3)	55 (34.4)	277 (15.2)
Sex, male	1,253 (40.1)	82 (43.2)	1,335 (40.2)	0 (0.0)	649 (40.7)	68 (49.6)	717 (41.4)	97 (5.3)
High school graduate or higher	2,694 (86.1)	124 (65.3)	2,818 (84.9)	4 (0.1)	1,481 (88.8)	110 (68.8)	1,591 (87.0)	0 (0.0)
Race/Ethnicity				0 (0.0)				97 (5.3)
White	2,425 (77.5)	123 (64.7)	2,548 (76.8)		1,227 (77.0)	102 (74.5)	1,329 (76.8)
Black	694 (22.2)	65 (34.2)	759 (22.9)		364 (22.8)	34 (24.8)	398 (23.0)
Other	9 (0.3)	2 (1.1)	11 (0.3)		3 (1.8)	1 (0.6)	4 (0.2)
Hypertension	2,341 (75.7)	133 (78.2)	2,474 (75.8)	55 (1.7)	1,311 (79.8)	125 (79.1)	1,436 (79.7)	27 (1.5)
Diabetes	1,038 (34.2)	76 (48.4)	1,114 (34.9)	126 (3.8)	565 (35.4)	59 (45.7)	624 (36.2)	104 (5.7)
CESD scale	3.2(2.98)	4.4(3.95)	3.2(3.05)	57 (1.7)	2.8(2.89)	4.0(3.61)	2.9(2.95)	158 (8.6)
CHD	456 (14.9)	48 (25.8)	504 (15.5)	67 (2.0)	254 (15.6)	33 (21.4)	287 (16.1)	41 (2.2)
Heart failure	499 (16.0)	58 (30.5)	557 (16.8)	0 (0.0)	321 (19.2)	45 (28.1)	366 (20.0)	0 (0.0)
Stroke	110 (3.5)	34 (17.9)	144 (4.3)	5 (0.2)	73 (4.4)	26 (16.3)	99 (5.4)	4 (0.2)
Cancer	109 (3.5)	5 (2.6)	114 (3.4)	0 (0.0)	57 (3.4%)	57 (3.4%)	57 (3.4%)	57 (3.4)
Frailty				334 (10.1)				191 (10.4)
Frail	191 (6.6)	25 (22.7)	216 (7.2)		129 (8.4)	16 (15.1)	145 (8.9)
Prefrail	1,356 (47.2)	75 (68.2)	1,431 (48.0)		869 (56.8)	75 (70.8)	944 (57.7)
Robust	1,327 (46.2)	10 (9.1)	1,337 (44.8)		533 (34.8)	15 (14.2)	548 (33.5)
Global cognitive factor score	−0.1(0.87)	−1.6(0.54)	−0.1(0.88)	1,443 (43.5)	−0.0(0.83)	−1.5(0.54)	−0.1(0.90)	20 (1.1)

Abbreviations: CESD, Center for Epidemiologic Studies Depression. CHD, coronary heart disease.

Of the 3,997 ARIC participants who completed V6, 1,828 (mean age of 80 (SD 4.9) years; 23% Black; 58.6% female) met eligibility criteria for prevalent dementia analysis at V6 (non-missing V6 dementia ascertainment, Medicare FFS enrollment for ≥ 1 year, provided research consent; Figure 2, Table 4). Among those, 160 (8.8%) had dementia, with 101 (63.1%) over age 80 years. Additionally, 1,679 participants who were dementia-free at V5 and had dementia status assessed at both visits, were eligible for incident dementia analysis at V6. Of those, 94 (5.6%) were classified as having incident dementia by V6. Baseline characteristics were similarly distributed between the full ARIC V6 population and the final analytic samples for prevalent and incident dementia (Table S3).

Performance of selected algorithms

At V6, the incident dementia algorithm had a PPV of 69.2% (95% CI 51.5%, 87.0%) and sensitivity of 34.6% (95% CI 21.7%, 47.5%) (Table 5). Exclusive use of ICD-10 dementia diagnostic codes resulted in sensitivity of 31.8 % (95% CI 18.1%, 45.6%) and PPV of 70.0% (95% CI 49.9%, 90.1%).

Table 5.

Performance of claims-based algorithms for Identifying Incident Dementia at V6 in the full data.

Lookback window	ICD code era	Algorithm	Required minimum months of enrollment in Part A & B FFS plan	TP	FP	FN	TN	Totala	Sensitivity	Specificity	Positive Predictive Value	Negative Predictive Value
All-available lookback window	Both ICD-9 and ICD-10 codes	This study incident dementia b	12	NTSR	NTSR	NTSR	NTSR	881	34.6 (21.7–47.5)	99.0 (98.4–99.7)	69.2 (51.5–87.0)	96.0 (94.7–97.3)
		Jain c	36	24	11	70	1574	1679	25.5 (16.7–34.3)	99.3 (98.9–99.7)	68.6 (53.2–84.0)	95.7 (94.8–96.7)
		Bynum d	12	37	26	57	1559	1679	39.4 (29.5–49.2)	98.4 (97.7–99.0)	58.7 (46.6–70.9)	96.5 (95.6–97.4)
		Lee e	12	38	25	56	1560	1679	40.4 (30.5–50.3)	98.4 (97.8–99.0)	60.3 (48.2–72.4)	96.5 (95.6–97.4)
		Jain-relaxed f	48	24	11	70	1574	1679	25.5 (16.7–34.3)	99.3 (98.9–99.7)	68.6 (53.2–84.0)	95.7 (94.8–96.7)
		Bynum-relaxed g	12	40	56	54	1529	1679	42.6 (32.6–52.5)	96.5 (95.6–97.4)	41.7 (31.8–51.5)	96.6 (95.7–97.5)
		Lee-relaxed h	12	45	46	49	1539	1679	47.9 (37.8–58.0)	97.1 (96.3–97.9)	49.5 (39.2–59.7)	96.9 (96.1–97.8)
All-available lookback window in ICD-10 era	ICD-10 i	This study incident dementia b	12	NTSR	NTSR	NTSR	NTSR	723	31.8 (18.1–45.6)	99.1 (98.4–99.8)	70.0 (49.9–90.1)	95.7 (94.2–97.2)
		Bynum d	12	25	17	55	1268	1365	31.3 (21.1–41.4)	98.7 (98.1–99.3)	59.5 (44.7–74.4)	95.8 (94.8–96.9)
		Lee e	12	30	14	50	1271	1365	37.5 (26.9–48.1)	98.9 (98.3–99.5)	68.2 (54.4–81.9)	96.2 (95.2–97.2)
		Bynum-relaxed g	12	30	40	50	1245	1365	37.5 (26.9–48.1)	96.9 (95.9–97.8)	42.9 (31.3–54.5)	96.1 (95.1–97.2)
		Lee-relaxed h	12	33	26	47	1259	1365	41.3 (30.5–52.0)	98.0 (97.2–98.7)	55.9 (43.3–68.6)	96.4 (95.4–97.4)
1-year lookback window (Sensitivity Analysis)	Both ICD-9 and ICD-10 codes	This study incident dementia b	12	NTSR	NTSR	NTSR	NTSR	881	25.0 (13.2–36.8)	99.5 (99.0–100.0)	76.5 (56.3–96.6)	95.5 (94.1–96.9)
		Bynum d	12	26	12	68	1573	1679	27.7 (18.6–36.7)	99.2 (98.8–99.7)	68.4 (53.6–83.2)	95.9 (94.9–96.8)
		Lee e	12	NTSR	NTSR	NTSR	NTSR	1679	31.9 (22.5–41.3)	99.4 (99.0–99.8)	75.0 (61.6–88.4)	96.1 (95.2–97.0)
		Bynum-relaxed g	12	30	15	64	1570	1679	31.9 (22.5–41.3)	99.1 (98.6–99.5)	66.7 (52.9–80.4)	96.1 (95.1–97.0)
		Lee-relaxed h	12	34	21	60	1564	1679	36.2 (26.5–45.9)	98.7 (98.1–99.2)	61.8 (49.0–74.7)	96.3 (95.4–97.2)
1-year lookback window (Sensitivity Analysis)	ICD-10 i	This study incident dementia b	12	NTSR	NTSR	NTSR	NTSR	723	27.3 (14.1–40.4)	99.6 (99.1–100.0)	80.0 (59.8–100.0)	95.5 (94.0–97.0)
		Bynum d	12	22	11	58	1274	1365	27.5 (17.7–37.3)	99.1 (98.6–99.6)	66.7 (50.6–82.8)	95.6 (94.5–96.7)
		Lee e	12	NTSR	NTSR	NTSR	NTSR	1365	33.8 (23.4–44.1)	99.3 (98.8–99.8)	75.0 (60.9–89.1)	96.0 (95.0–97.1)
		Bynum-relaxed g	12	26	14	54	1271	1365	32.5 (22.2–42.8)	98.9 (98.3–99.5)	65.0 (50.2–79.8)	95.9 (94.9–97.0)
		Lee-relaxed h	12	30	19	50	1266	1365	37.5 (26.9–48.1)	98.5 (97.9–99.2)	61.2 (47.6–74.9)	96.2 (95.2–97.2)

^aTotal sample size varies as different algorithms have different inclusion criteria, see below. The Numbers Too Small to Report (NTSR) designation signifies that the count is less than 11, as stipulated by the Center for Medicare and Medicaid Services rules and data use agreement. Additionally, other related cells are blocked to prevent the calculation of these numbers.

^bPrimary-incident algorithm, the definition is shown in Table 2.

^cThe inclusion criteria is enrollment in Part A & B FFS plan for ≥ 36 months. Skilled Nursing Facility (SNF) data in Medicare claims was used as a proxy for nursing home data, which are unavailable.

^dThe inclusion criteria is continuous enrollment in Part A & B FFS plans for ≥12 months.

^eThe inclusion criteria is continuous enrollment in Part A & B FFS plans for ≥12 months.

^fThe inclusion criteria is continuous enrollment in Part A & B FFS plans for ≥ 48 months.

^gThe inclusion criteria is continuous enrollment in Part A & B FFS plans for ≥12 months.

^hThe inclusion criteria is enrollment in Part A & B FFS plans for ≥12 month.

ⁱUsing ICD-10 codes only at V6 (i.e., among who attended V6 after October 1, 2016, and had a 1-year lookback window so that only ICD-10 codes were used to ascertain Medicare dementia diagnostic codes)

The prevalent dementia algorithm at V6 achieved a PPV of 75.4% (95% CI: 64.9%, 85.9%), specificity of 98.5% (95% CI: 97.8%, 99.3%), NPV of 94.6% (95% CI: 93.3%, 95.9%), and sensitivity of 44.1% (95% CI: 34.9%, 53.4%) (Table S4). In analyses restricted to ICD-10 dementia diagnostic codes, the sensitivity decreased to 39.1% (95% CI, 29.2%, 49.1%) and PPV increased to 83.7% (95% CI, 72.7%, 94.8%), as compared to using both ICD-9 and ICD-10 codes. Performance at of the V5 prevalent dementia algorithm based exclusively on the ICD-9 dementia diagnostic codes was comparable to the V6 results (Table S5).

Comparative performance of different algorithms

Our incident dementia algorithm had ~10% or higher PPV and similar sensitivity, specificity, NPV compared to the existing algorithms, with the exception of the Jain algorithm, which had comparable performance (Table 5). Our prevalent dementia algorithm, derived at both Visit 5 and V6, showed a 4–25% higher PPV with comparable sensitivity, specificity, and NPV compared to the Jain, Lee and Bynum algorithms (Tables S4 & S5).

Sensitivity and Subgroup analyses

Limitation of the lookback window to 1-year increased the PPV by ~8% and decreased the sensitivity by ~10%, and had minimal effects on specificity and NPV, in comparison with performance of the incident dementia algorithm when using the all-available lookback period (Table 5). When we restricted analyses to ICD-10 dementia diagnostic codes within the 1-year lookback period, the sensitivity decreased to 27.3% [95% CI 14.1%, 40.4%] vs. 34.6% [95% CI 21.7%, 47.5%] for the full lookback and the PPV increased to 80.0% [95% CI 59.8%, 100.0%] in comparison with the PPV of 69.2% [95% CI 51.5%, 87.0%] for the full lookback. Comparable trends were observed for the prevalent dementia algorithm (Table S5) or when using the same sample size defined by our dementia algorithm (Table S6 & S7).

In age-stratified analysis, both incident (Table S8) and prevalent (Tables S9 & S10) dementia algorithms had higher sensitivity in participants aged ≥81 years compared to those younger than 80. Although confidence intervals are wide, algorithm performance was generally worse among Black participants, and PPV was generally lower among female participants—except for prevalent dementia at V5.

Empirical example

In the liraglutide vs DPP4i new-user cohort, the Bynum algorithm identified the greatest number of incident dementia cases (risk: 7.2% vs 12.5%; rate: 26.1 vs 47.4 per 1000 person-years), followed by our algorithm (risk: 6.2% vs 11.1%; rate: 22.0 vs 42.0 per 1000 person-year), Lee algorithm (risk: 4.4% vs 7.8%; rate: 15.3 vs 28.8), and Jain algorithm (risk: 3.0% vs 5.8%; rate: 11.8 vs 24.7) (Table 4). Our algorithm was comparable to the Bynum algorithm in both the crude (−4.9% [95% CI: −5.6% to −4.3%] vs. −5.3% [95% CI: −5.9% to −4.6%]) and adjusted (−3.4% [−4.1% to −2.7%] vs. −2.2% [−2.7% to −1.6%]) risk difference, with a Kappa coefficient of 0.813 (Table S11).

DISCUSSION

We developed and validated Medicare claims-based incident and prevalent dementia identification algorithms using ARIC cohort’s rigorous neurocognitive assessments as the “gold standard”. Our incident dementia algorithm achieved a PPV of 69.2% (in a population with 8.8% dementia prevalence.), specificity of 99.0%, and a sensitivity of 34.6% using an all-available lookback window. Reducing the lookback window to one-year increased PPV by approximately 10% but decreased sensitivity by a similar margin. Our prevalent dementia algorithm showed a similar performance. Our algorithms did not significantly outperform existing algorithms tested against the ARIC gold standard. The advantage of the prevalent and incident dementia algorithms developed in this study is their flexibility in the use of inpatient or outpatient claims without additional reliance on nursing home data or restrictive diagnostic code positions.

Given the inherent tradeoff between sensitivity and specificity in algorithm development, we prioritized high PPV and specificity suitable for pharmacoepidemiologic studies evaluating drug effectiveness²⁴. The high specificity of our incident dementia algorithm minimizes false positives, critical for accurately estimating treatment effect on incident dementia. The PPV of our prevalent dementia algorithm reliably identifies cohorts with existing dementia, which is essential for assessing drug effectiveness on dementia progression.

Although our dementia algorithm achieved a specificity close to 1, the algorithm’s modest sensitivity suggests that many true cases were missed. For pharmacoepidemiologic studies assessing comparative effectiveness/safety on incident dementia, where high specificity is crucial for classifying outcomes²⁴, this modest sensitivity might have limited impact on the assessment of incident dementia using relative risk measures, provided that: 1) specificity is perfect²⁴; 2) highly sensitive exclusion criteria (e.g., any dementia diagnoses) are applied to exclude prevalent dementia, assuming nondifferential misclassification across drug groups; 3) there is nondifferential misclassification for incident dementia outcome between drug groups (noting this is a strong assumption and even minor violations can introduce bias²⁴); and 4) other biases (such as selection bias^7,47, time-related bias⁴⁸, and covariates mismeasurement⁴⁹) are sufficiently controlled. However, modest sensitivity will impact assessments using absolute risk measures. For studies assessing comparative effectiveness on slowing dementia progression, proxies for dementia progression, such as dementia-related hospital admission, emergency department visits⁵⁰, or nursing home admission⁵¹, could be utilized when cognitive function scores are unavailable⁵². In these scenarios, low sensitivity may similarly have limited impact, as dementia cohorts can be reliably identified among initiators of FDA-approved disease-modifying treatments (whose dementia status is confirmed) or initiators of drug repurposing candidates identified using the algorithm with high PPV (assuming sample size is sufficient). Nevertheless, low sensitivity could significantly influence other pharmacoepidemiologic research contexts, such as studies using dementia as a covariate or research specifically aimed at evaluating dementia incidence or prevalence⁵³.

The worse performance of our algorithm for incident dementia identification among Black participants, as compared to White participants, likely reflects missed or delayed dementia diagnoses or inadequately documented dementia in Medicare claims data⁵⁴. The low PPVs observed in women align with findings from an autopsy study indicating that women had significantly higher odds of a diagnosis of AD at equivalent pathological levels as compared to men⁵⁵.

Differences in the performance of our multiple examine algorithms, including those developed in this study, were sufficient to impact risk difference estimates in an empirical analysis. Our algorithm and the Bynum algorithm demonstrated strong agreement, identifying more cases and yielding larger crude risk differences compared to the Jain and Lee algorithms. The Bynum algorithm identified more cases than our algorithm in both the GLP1RA and DPP4i groups, resulting in a similar treatment effect estimate. The stricter diagnostic requirements of the Lee (at the primary or secondary position) and Jain algorithms (diagnosis appeared in different years or additionally requiring information from nursing home stays) likely led to an underestimation of incident cases, thus reducing observed risk differences.

Machine learning (ML)-based models integrating administrative claims with electronic health records (EHR) have shown for dementia classification outside of rigorous observational studies, benefiting from ML’s capability to leverage extensive clinical data. Several recent studies employing ML reported high specificity (86.4% to 98.6%) in dementia identification, however without achieving sensitivities greater than 32%^56,57. A limitation of existing dementia ML models is the lack of a gold standard based on neurocognitive assessments, potentially compromising accuracy. Note, relying solely on EHR data—even from a large, integrated healthcare system—without incorporating claims data from providers outside the system misses outcomes and has been shown to result in low sensitivity^58,59.

Our study has the following strengths. First, our validation method provides a rigorous ascertainment of dementia status—based on comprehensive neurocognitive assessments²³ in the ARIC study, thus significantly enhancing the reliability of algorithm’s performance evaluation. Second, our study offers both an independent validation to existing dementia classification algorithms and an empirical pharmacologic study to assess their comparative performance. Third, in our analyses we distinguish incident dementia from prevalent dementia classification, thus providing a valuable tool for pharmacoepidemiologic studies assessing drug effects on dementia incidence and dementia progression. Lastly, our algorithms covered both the ICD-9 and ICD-10 era, addressing a critical gap in existing algorithms.

Several limitations should be noted. First, adjudicated dementia classification was only available for ARIC participants who attended clinic visits, and restricting analyses to dementia-free participants of V5 with classification at V6 reduced our sample to about half, potentially introducing bias from informative missingness. Second, despite a larger sample size compared to previous studies, the modest number of dementia cases results in imprecise algorithm performance estimates. Additionally, requiring prescription claims (Medicare Part D) for the incidence dementia algorithm reduced our sample size, this reduction thus limiting generalizability (despite similar baseline characteristics between the full V6 population and the analytic samples). Third, due to the prodromal nature of dementia, cases identified by our incident dementia algorithm may be at moderate to late stages in disease progression⁶⁰. Fourth, the algorithm’s modest sensitivity impacts studies aimed at assessing dementia prevalence or incidence but may be less problematic in comparative pharmacoepidemiologic studies assessing dementia outcomes under certain assumptions (e.g., perfect specificity, sufficient confounding control, and nondifferential misclassification of dementia across drug comparison groups) as discussed earlier. Fifth, we validated our algorithms against the ARIC “gold standard” of robust dementia classification, which remains internal to the ARIC cohort. However, we lack an external validation in a different population. Thus, algorithm performance may be specific to Medicare beneficiaries in the ARIC cohort (restricted to Black and White participants with ≥1 year of Medicare Part A & B enrollment) and may not generalize to other populations or healthcare systems⁶¹. A substantial proportion of individuals with dementia are dually eligible for both Medicare and Medicaid. To enhance case identification—particularly for those residing in and outside of nursing homes—future studies should consider incorporating Medicaid claims into dementia detection algorithms⁶². Sixth, the study did not differentiate dementia subtypes, limiting our analysis to overall dementia classification. Lastly, although the empirical example demonstrates the impact of dementia identification algorithms on treatment effect estimates, the true treatment effect size remains unknown. Therefore, sensitivity analyses using various algorithms should be conducted in pharmacoepidemiologic studies assessing drug effectiveness on dementia.

CONCLUSION

Our Medicare claims-based dementia identification algorithms developed and validated the ARIC cohort show a modest sensitivity but high PPV and specificity, performing comparably to existing methods. Developed algorithms offer valuable tools for improving the robustness of identifying dementia outcomes in pharmacoepidemiologic studies, thus addressing an important methodological need in the field.

Table 6.

Risk differences of incident dementia outcome for Medicare beneficiaries aged 70+ initiating liraglutide vs DPP4i by intention-to-treat analysis over a maximum 3-year follow-up period^a,b.

Dementia identification algorithm	Cohort	N	# event	Risk %	Rate (#/1000 person-year)	Crude Risk Difference %	IPTW-weighted Risk Difference %	Kappa value
This study incident dementiac	Liraglutide	7,036	437	6.2	22.0 (20.0 to 24.4)	−4.9 (−5.6 to −4.3)	−3.4 (−4.1 to −2.7)	reference
	DPP4i	64,760	7,223	11.1	42.0 (41.0 to 43.1)
Jaind	Liraglutide	7,036	212	3.0	11.8 (10.4 to 13.6)	−2.8 (−3.3 to −2.4)	−1.2 (−1.7 to −0.7)	0.578
	DPP4i	64,760	3,782	5.8	24.7 (23.9 to 25.5)
Bynumd	Liraglutide	7,036	508	7.2	26.1 (23.8 to 28.6)	−5.3 (−5.9 to −4.6)	−3.6 (−4.3 to −2.9)	0.813
	DPP4i	64,760	8,080	12.5	47.4 (46.3 to 48.5)
Leed	Liraglutide	7,036	312	4.4	15.3 (13.6 to 17.2)	−3.3 (−3.9 to −2.8)	−2.2 (−2.7 to −1.6)	0.633
	DPP4i	64,760	5,037	7.8	28.8 (28.0 to 29.7)

Abbreviations: N, number; IPTW, inverse probability treatment weight.

^aPatients were followed up from their second prescription until the end of Medicare enrollment, death, or December 31, 2019, whichever came first. All patients were required to enter the cohort no later than December 31, 2016 so that we could potentially follow patients for a maximum of 3 years (the study ended on Dec. 31, 2019).

^bDeath was treated as a competing risk by setting the risk for dementia after death to 0 for patients who died.

^cPatients were censored as they lost part A or B or D of Medicare coverage during follow-up when using our incident dementia outcome.

^dPatients were censored as they lost part A or B of Medicare coverage during follow-up when using other algorithms.

Data Availability Statement:

The data use agreement (DUA) from Centers for Medicare & Medicaid Services (CMS) does not allow sharing Medicare data. The SAS codes for our novel algorithms are available on https://github.com/tianshengwang/IdentifyADRD.