Abstract
Introduction
Few studies have examined memory decline among patients with type 2 diabetes using different oral hypoglycemic drugs.
Methods
Participants with normal cognition (NC) or Alzheimer's disease (AD) dementia using a hypoglycemic medication (2005 to 2019) were identified from the National Alzheimer's Coordinating Center database. Delayed memory was assessed using the Wechsler Memory Scale Revised–Logical Memory test. Associations between oral drug classes and memory over time were examined using mixed‐effects models with inverse probability treatment weights.
Results
In NC (n = 1192), metformin use was associated with better memory performance over time, whereas in AD (n = 807), dipeptidyl peptidase‐4 (DPP4) inhibitor use was associated with a slower rate of memory decline. Interaction effects suggested greater benefit associated with DPP4 inhibitor use among APOE ε4 carriers.
Discussion
Associations between different oral hypoglycemic drugs and memory change were not consistent between cognitively normal elderly and those with AD dementia. APOE ε4 genotype modified some relationships.
Keywords: Alzheimer's disease (AD), apolipoprotein E (APOE), diabetes, dipeptidyl peptidase‐4 inhibitor (DPP4 inhibitor), memory, metformin, sulfonylurea, thiazolidinedione
1. INTRODUCTION
Type 2 diabetes (T2DM) has been shown to increase the risk of Alzheimer's disease (AD). 1 The basis for this is not fully understood, although accelerated accumulation of misfolded amyloid beta (Aβ) and hyperphosphorylated tau (p‐tau) proteins, 2 and brain insulin resistance have been proposed. 3 Widespread changes in cortical glucose metabolism are seen in AD, and it has been suggested that oral hypoglycemic (anti‐diabetic) agents may be of benefit. 3
The majority of previous observational studies have investigated associations between a specific drug class and dementia risk in people with T2DM who were cognitively normal at baseline. 4 Fewer studies have considered relationships between different drug classes and memory changes in cognitively normal people with T2DM, 5 , 6 or among people with T2DM who also have a diagnosis of AD dementia. Metformin has been associated with a lower dementia risk; 4 however, its effects on memory have been inconsistent. 5 , 6 Relatively few clinical studies have assessed associations between memory and dipeptidyl peptidase‐4 (DPP4) inhibitor use, 7 , 8 although sitagliptin, a DPP4 inhibitor, was associated with improved general cognition in people with T2DM and AD, 9 and was shown to slow Aβ accumulation in transgenic mouse models of AD. 10
Currently, little guidance is available to help to choose one antidiabetic drug class over another considering their effects on cognition. 11 We aim to determine associations between memory change over time and the use of oral hypoglycemic drug classes in cognitively normal elderly and in people with dementia due to AD. We hypothesize that different classes of oral hypoglycemic medications might exhibit different associations with memory over time, and that these associations may be specific to groups of cognitively normal elderly people or to those with AD. We also explored these relationships in people with amnestic mild cognitive impairment (aMCI) as a secondary population of interest. Because the apolipoprotein E (APOE) ε4 gene variant increases AD risk, 12 we examined how the relationships between drug use and memory decline might be modified by APOE genotype. This may help take a step toward personalized dementia prevention and treatment optimization in the context of T2DM.
2. METHODS
2.1. Data source
The National Alzheimer's Coordinating Center (NACC) was established in 1999, and the NACC database consists of data from ≈39 Alzheimer's disease centers (ADCs) funded by National Institute on Aging (NIA) across the United States. Data are structured in a standardized manner across different ADCs to form a Uniform Data Set (UDS) as described previously. 13 , 14 The UDS was implemented in September 2005, and data are still collected continuously. The ADCs enroll subjects by clinician referral, self‐referral by patients or family members, by active recruitment through community organizations and volunteers; therefore, the NACC database can be regarded as a referral‐based or volunteer case series. Written informed consent was obtained from all participants and co‐participants (usually a close friend or family member).
2.2. Subject selection
Participants using an antidiabetic medication were included in the study. Participants who met NIA‐Alzheimer's Association (NIA‐AA) or National Institute of Neurological and Communicative Disorders and Stroke/Alzheimer's Disease and Related Disorders Association (NINCDS/ADRDA) criteria for AD 15 , 16 were included in the AD dementia group. A group of aMCI (single‐domain or multiple‐domain) participants according to Petersen's criteria 17 were also identified and explored. Participants who did not meet either AD dementia or aMCI criteria were included in the control group. Subjects with vascular dementia (NINDS/AIREN [National Institute of Neurological Disorders and Stroke/Association Internationale pour la Recherche et l'Enseignement en Neurosciences] criteria 18 ), cancer, epilepsy, traumatic brain injury, Parkinson's disease, type 1 diabetes, diabetes insipidus, latent autoimmune diabetes, gestational diabetes, or Lewy body dementia were excluded from all groups.
2.3. Drug exposures and memory outcomes
Medication use within 2 weeks of each visit was identified from a structured medication inventory. Participants, or co‐participants if appropriate, were asked to bring to the study visit or report all prescription medications used currently or within the past 2 weeks, and the form was completed by trained ADC staff physicians. The drug classes of interest included metformin, sulfonylureas, thiazolidinediones, and DPP4 inhibitors. Less frequent hypoglycemic medications in the database, including sodium‐glucose transport protein 2 (SGLT2) inhibitors, acarbose, meglitinide, and miglitol, were classified as the use of an “other” oral hypoglycemic medication, which was included as a covariate in the analysis. Use of an injectable incretin mimetic and use of insulin were extracted and included as covariates.
The primary outcome of interest was delayed recall assessed using the Wechsler Memory Scale Revised–Logical Memory test IIA (score range from 0 to 25; better scores indicate better episodic memory performance) 19 because delayed recall was the most sensitive and specific domain available related to AD. Performance on immediate recall was also considered a secondary outcome (Logical Memory test IA). Because the two tests were expected to differ in their psychometric properties (eg, floor effects in AD), they were considered separately. The delayed recall trials occurred after a 20‐minute delay.
2.4. Statistical analysis
All analyses were conducted using R 3.5.1., and all figures were created using ggplot2 package. 20 Analyses were conducted separately in people with normal cognition and AD dementia. Associations in the aMCI group were also explored. To determine the associations between each drug class and memory over time, we considered drug × time interactions in mixed‐effects regression models. Random‐slopes linear mixed‐effects regressions were used in people with normal cognition and aMCI (lme4 package 21 ). Standardized coefficients (β) were used to express the effect size of the associations. When there are excess zeros in memory scores, mixed‐effects zero‐inflated Poisson or quasi‐Poisson regressions were used to handle potential floor effects (glmmTMB package 22 ). A Poisson regression was used when the data could not fit a quasi‐Poisson regression owing to no overdispersion. Effect sizes from Poisson or quasi‐Poisson regressions were expressed as rate ratios (RR), which indicate the fold‐change in memory score over time among people using the drug class related to people not using that drug class. All effect size estimates were adjusted for covariates including age, sex, education, baseline Mini‐Mental State Examination (MMSE) score (a measure of general cognition), comorbid depression, hypercholesterolemia and hypertension, concomitant use of the other oral and injectable (insulin and incretins) hypoglycemic medication classes, and concomitant use of AD medications.
Confounding by drug indication was addressed by inverse probability of treatment weighting (ipw package 23 ). Specifically, marginal structural models with time‐varying confounders and drug exposure were used to generate treatment probability weights (unstabilized). The models also consider previous drug exposure when weights are generated. 23 American T2DM management guidelines 24 were used to select the following weighting factors: comorbid cardiovascular disease, history of stroke, body mass index, and vitamin B12 deficiency based on their clear prior influence on the likelihood of the drug exposure. For the association between each oral drug class and memory over time, a separate regression model was performed with inverse probability of treatment weights specific to the drug class of interest.
The presence of APOE ε3/ε4 or ε4/ε4 genotype (classified as APOE ε4 carriers in this study) was tested as a potential modifier of the association between each oral drug class and memory over time using a drug × APOEε4 × time interaction term. Conditional associations between oral drug classes and memory over time among APOE ε4 carriers and non‐carriers were computed.
RESEARCH IN CONTEXT
Systematic review: The authors evaluated clinical studies investigating the effects of oral hypoglycemic drugs on memory in people with type 2 diabetes (T2DM) who had normal cognition or dementia due to Alzheimer's disease (AD). Limited studies investigated longitudinal effects in AD, and there was conflicting evidence regarding the benefits of metformin.
Interpretation: Metformin use was associated with better memory in cognitively normal people with T2DM, whereas dipeptidyl peptidase‐4 (DPP4) inhibitor use was associated with slower memory decline in AD. Relationships between memory and DPP4 use in normal cognition, and thiazolidinedione use in AD, depended on apolipoprotein E (APOE) ε4 carrier status, an important implication for personalized medicine.
Future directions: Further studies examining cognitive effects of DPP4 inhibitors in people with T2DM, and if these effects generalize to people without T2DM, are warranted. It may be important to consider APOE ε4 carrier status when prescribing and studying oral hypoglycemic drugs.
3. RESULTS
3.1. Subject characteristics
Of 42,022 subjects (147,565 UDS visits) conducted between September 2005 and November 2019, a total of 3830 subjects (9873 visits) had available memory score and involved use of drug(s) for diabetes. Details of the subject selection process are shown in Figure S1 in supporting information. In the analysis, 1192 cognitively normal participants (3166 visits, 73% with ≥2 visits, duration of follow‐up 3.4 ± 3.3 years), 671 participants with aMCI (1144 visits, 52% with ≥2 visits, duration of follow‐up 1.5 ± 2.2 years), and 807 participants with AD dementia (1493 visits, 60% with ≥2 visits, duration of follow‐up 1.9 ± 2.2 years) were included. Baseline characteristics are shown in Table 1. In each group, 65% to 66% were using a single diabetes medication at baseline, and 33% to 34% were using two or more medications. Baseline characteristics by baseline hypoglycemic medication use, and descriptive statistics for propensity score weights, are shown in Tables S1–S4 in supporting information. Sulfonylurea users had poorer memory scores in the cognitively normal group at baseline, and thiazolidinedione users had superior memory scores among participants with AD at baseline (Table S5 in supporting information).
TABLE 1.
Normal cognition (n = 1192) | Amnestic MCI (n = 671) | AD dementia (n = 807) | |
---|---|---|---|
Baseline demographics | |||
Age (y) | 72.25 (8.28) | 74.37 (8.2) | 76.11 (7.93) |
Female | 721 (60%) | 284 (42%) | 389 (48%) |
Body mass index (kg/m2) | 30.72 (5.9) | 29.77 (5.88) | 28.54 (5.68) |
Education (y) | 14.73 (3.4) | 14.26 (3.73) | 13.50 (3.99) |
Dementia‐related measures at baseline | |||
MMSE | 28.5 (1.72) | 26.78 (2.57) | 21.25 (5.43) |
APOE ε4 carrier | 294 (25%) | 206 (31%) | 391 (48%) |
AD medication use | 0 (0%) | 124 (18%) | 519 (64%) |
Comorbidities at baseline | |||
Hypercholesterolemia | 942 (79%) | 525 (78%) | 651 (81%) |
Hypertension | 937 (79%) | 537 (80%) | 640 (79%) |
Depression | 150 (13%) | 176 (26%) | 197 (24%) |
Stroke history | 14 (1%) | 15 (2%) | 14 (2%) |
Vitamin B12 deficiency | 68 (6%) | 47 (7%) | 81 (10%) |
Cardiovascular disease | 120 (10%) | 74 (11%) | 100 (12%) |
Oral hypoglycemic medications at baseline | |||
Metformin | 824 (69%) | 448 (67%) | 473 (59%) |
Sulfonylurea | 430 (36%) | 233 (35%) | 314 (39%) |
Thiazolidinedione | 155 (13%) | 101 (15%) | 126 (16%) |
DPP4 inhibitors | 55 (5%) | 41 (6%) | 58 (7%) |
Other oral drugs | 22 (2%) | 19 (3%) | 17 (2%) |
Injectable hypoglycemic medications at baseline | |||
Insulin | 187 (16%) | 129 (19%) | 158 (20%) |
Incretin mimetics | 16 (1%) | 6 (1%) | 13 (2%) |
Continuous variables and categorical variables were reported in observed/unweighted mean (SD) and counts (proportion), respectively.
Abbreviations: AD, Alzheimer's disease; APOE, apolipoprotein E; DPP4, dipeptidyl peptidase‐4; MCI, mild cognitive impairment; MMSE, Mini‐Mental State Examination.
3.2. Relationships between oral hypoglycemic drug classes and longitudinal memory change
Among cognitively normal people being treated for T2DM, metformin users showed better performance on immediate (β [95% confidence interval] = 0.069 [0.011, 0.12], P = .0202) and delayed (β = 0.089 [0.032, 0.146], P = .0024) memory over time compared to non–metformin users (Figure 1), whereas the use of a sulfonylurea, thiazolidinedione, or DPP4 inhibitor, showed no significant associations over time (Table 2). For comparison, the metformin estimate was comparable to the independent effect of APOE ε4 carrier status on delayed memory over time (β = –0.052 [–0.096, –0.008], P = .0217).
TABLE 2.
Normal cognition | Amnestic MCI | AD dementia | |||||||||
---|---|---|---|---|---|---|---|---|---|---|---|
Beta [95% CI] | t | df | P‐value | Beta [95% CI] | t | df | P‐value | RR [95% CI] | z | P‐value | |
Immediate memory | |||||||||||
Met × time | 0.069 [0.011, 0.128] | 2.33 | 751.92 | .0202 | 0.002 [−0.101, 0.106] | 0.04 | 229.72 | .9646 | 0.964 [0.900, 1.032] | −1.05 a | .2920 |
SU × time | 0.02 [−0.024, 0.063] | 0.89 | 637.22 | .3748 | 0.069 [−0.01, 0.148] | 1.72 | 135.43 | .0871 | 1.059 [0.998, 1.125] | 1.89 a | .0584 |
TZD × time | 0.015 [−0.021, 0.052] | 0.82 | 463.86 | .4115 | −0.065 [−0.136, 0.006] | −1.81 | 256.65 | .0720 | 0.889 [0.815, 0.970] | −2.66 a | .0078 |
Gliptin × time | −0.008 [−0.051, 0.035] | −0.36 | 955.98 | .7197 | 0.033 [−0.025, 0.092] | 1.11 | 197.91 | .2678 | 1.067 [0.983, 1.159] | 1.55 a | .1220 |
Delayed memory | |||||||||||
Met × time | 0.089 [0.032, 0.146] | 3.04 | 915.19 | .0024 | 0.028 [−0.063, 0.119] | 0.60 | 252.71 | .5480 | 0.925 [0.823, 1.040] | −1.31 a | .1904 |
SU × time | 0.019 [−0.024, 0.062] | 0.85 | 729.56 | .3934 | −0.006 [−0.081, 0.068] | −0.17 | 270.24 | .8679 | 1.042 [0.941, 1.154] | 0.79 a | .4269 |
TZD × time | 0.024 [−0.013, 0.061] | 1.26 | 735.56 | .2090 | 0.004 [−0.062, 0.069] | 0.11 | 311.41 | .9113 | 0.874 [0.740, 1.031] | −1.60 a | .1106 |
Gliptin × time | 0.005 [−0.037, 0.047] | 0.23 | 1309.16 | .8172 | 0.002 [−0.053, 0.057] | 0.08 | 397.84 | .9386 | 1.220 [1.060, 1.403] | 2.78 a | .0055 |
Notes: Inverse probability treatment weighting was used to adjust for confounding by indication, including comorbid cardiovascular disease, stroke history, body mass index, and vitamin B12 deficiency. The effects of each oral drug class were determined by separate regression models with inverse probability treatment weights toward the tested drug.
All estimates were controlled for age, sex, education, APOE ε4 carrier status, baseline MMSE score, comorbid depression, hypercholesterolemia, and hypertension, concurrent use of other hypoglycemic drug classes, and concurrent use of medications for AD.
Abbreviations: AD, Alzheimer's disease; Beta, standardized coefficient; CI, confidence interval; Gliptin, dipeptidyl peptidase‐4 inhibitors; MCI, mild cognitive impairment; Met, metformin; RR, rate ratio; SU, sulfonylurea; TZD, thiazolidinedione.
Mixed‐effects zero‐inflated quasi‐Poisson regression (random‐intercepts) was used.
In AD dementia, DPP4 inhibitor use was associated with slower decline in delayed memory (RR [95% confidence interval] = 1.22 [1.06, 1.40], P = .0055; Table 2; Figure 2), and thiazolidinedione use was associated with faster decline in immediate memory (RR = 0.89 [0.82, 0.97], P = .0078; Table 2; Figure 3), but no associations were observed between memory changes over time and metformin or sulfonylurea use.
None of the four oral drug classes showed significant associations with differential memory changes over time in people with aMCI (Table 2).
In post‐hoc models including thyroid disease, smoking, and benzodiazepine use as additional covariates, the results did not change (Table S6 in supporting information). Toinvestigatepossible bias due to attrition, additional post hoc models incorporated unstabilized inverse probability of censoring weights based on time‐varying MMSE score, and the results did not change (Table S7 in supporting information).
3.3. Interactions between oral hypoglycemic drug use and APOE ε4 carrier status in cognitively normal individuals
Conditional associations are reported in Table 3, and all interactions between APOE ε4 carrier status and certain oral hypoglycemic drug exposures over time are shown in Table S8 in supporting information.
TABLE 3.
Normal cognition | Amnestic MCI | AD dementia | |||||||||
---|---|---|---|---|---|---|---|---|---|---|---|
Beta [95% CI] | t | df | P‐value | Beta [95% CI] | t | df | P‐value | RR [95% CI] | Z | P‐value | |
Immediate memory in APOE ε4 carriers | |||||||||||
Met × time | 0.053 [−0.044, 0.151] | 1.07 | 720.54 | .2855 | 0.013 [−0.14, 0.166] | 0.17 | 409.87 | .8633 | 0.940 [0.866, 1.021] | −1.47 a | .1409 |
SU × time | 0.014 [−0.055, 0.084] | 0.41 | 671.70 | .6851 | 0.123 [−0.001, 0.246] | 1.95 | 74.00 | .0548 | 1.043 [0.963, 1.131] | 1.04 a | .2998 |
TZD × time | 0.022 [−0.038, 0.081] | 0.71 | 727.61 | .4761 | −0.052 [−0.144, 0.04] | −1.12 | 87.18 | .2668 | 1.021 [0.924, 1.128] | 0.40 a , b | .6855 |
Gliptin × time | 0.002 [−0.057, 0.062] | 0.08 | 1067.55 | .9352 | 0.059 [−0.005, 0.124] | 1.81 | 92.86 | .0729 | 1.135 [1.030, 1.251] | 2.55 a | .0107 |
Immediate memory in APOE ε4 non‐carriers | |||||||||||
Met × time | 0.058 [0.006, 0.111] | 2.17 | 782.85 | .0307 | −0.029 [−0.113, 0.056] | −0.67 | 491.81 | .5061 | 1.023 [0.945, 1.106] | 0.56 a | .5770 |
SU × time | −0.016 [−0.057, 0.026] | −0.74 | 684.48 | .4578 | 0.029 [−0.051, 0.109] | 0.72 | 145.69 | .4751 | 1.042 [0.971, 1.119] | 1.14 a | .2544 |
TZD × time | 0.002 [−0.034, 0.037] | 0.10 | 618.77 | .9213 | 0.027 [−0.079, 0.134] | 0.51 | 116.05 | .6132 | 0.813 [0.714, 0.925] | −3.14 a , b | .0017 |
Gliptin × time | −0.018 [−0.051, 0.015] | −1.06 | 562.89 | .2917 | 0.037 [−0.035, 0.109] | 1.01 | 108.50 | .3165 | 1.037 [0.930, 1.155] | 0.65 a | .5160 |
Delayed memory in APOE ε4 carriers | |||||||||||
Met × time | 0.043 [−0.053, 0.139] | 0.88 | 884.99 | .3788 | 0.051 [−0.085, 0.186] | 0.73 | 435.36 | .4641 | 0.838 [0.734, 0.956] | −2.63 a | .0086 |
SU × time | 0.018 [−0.05, 0.086] | 0.52 | 782.99 | .6026 | 0.042 [−0.077, 0.162] | 0.69 | 177.41 | .4887 | 1.077 [0.937, 1.239] | 1.05 a | .2942 |
TZD × time | 0.029 [−0.03, 0.087] | 0.96 | 890.46 | .3364 | 0.01 [−0.076, 0.096] | 0.23 | 129.39 | .8211 | 1.000 [0.819, 1.222] | 0.00 a | .9990 |
Gliptin × time | 0.058 [0.000, 0.117] | 1.95 c | 1172.29 | .0511 | 0.015 [−0.046, 0.075] | 0.48 | 326.22 | .6298 | 1.189 [1.012, 1.397] | 2.11 a | .0352 |
Delayed memory in APOE ε4 non‐carriers | |||||||||||
Met × time | 0.086 [0.035, 0.138] | 3.27 | 935.47 | .0011 | 0.004 [−0.074, 0.082] | 0.11 | 508.85 | .9156 | 0.994 [0.861, 1.148] | −0.08 a | .9344 |
SU × time | −0.018 [−0.058, 0.023] | −0.86 | 788.86 | .3888 | −0.023 [−0.099, 0.052] | −0.60 | 255.49 | .5471 | 0.979 [0.868, 1.105] | −0.34 a | .7312 |
TZD × time | −0.002 [−0.037, 0.033] | −0.12 | 1042.39 | .9071 | 0.046 [−0.047, 0.138] | 0.97 | 134.91 | .3355 | 0.762 [0.610, 0.952] | −2.40 a | .0166 |
Gliptin × time | −0.017 [−0.05, 0.016] | −1.01 c | 841.61 | .3151 | 0.024 [−0.047, 0.094] | 0.66 | 185.60 | .5115 | 1.245 [1.016, 1.527] | 2.11 a | .0350 |
Abbreviations: AD, Alzheimer's disease; APOE, apolipoprotein E; Beta, standardized coefficient; Gliptin, dipeptidyl peptidase‐4 inhibitors; MCI, mild cognitive impairment; Met, metformin; RR, rate ratio (numbers >1 imply better performance over time, numbers <1 imply poorer performance over time); SU, sulfonylurea; TZD, thiazolidinedione.
Mixed‐effects zero‐inflated quasi‐Poisson regression (random‐intercepts) was used.
Interaction between thiazolidinedione use and APOE ε4 carrier status significant (z = 2.86, P = .0042).
Interaction between DPP4 inhibitor use and APOE ε4 carrier status significant (t = 2.19, P = .0290).
In cognitively normal people, a significant interaction between APOE genotype and DPP4 inhibitor use was observed over time (interaction: β = 0.038 [0.0039, 0.072], P = .0290) such that ε4 carriers exhibited significantly slower decline in delayed memory than non‐carriers (Figure S2 in supporting information); a trend toward a positive relationship with delayed memory was seen in APOE ε4 carriers (β = 0.058 [0.000, 0.117], P = .0511) that was not seen in non‐carriers (β = –0.017 [–0.050, 0.016], P = .3151). Among APOE ε4 non‐carriers, metformin use was associated with better immediate and delayed memory over time (β = 0.058 [0.006, 0.11], P = .0307; β = 0.086 [0.035, 0.14], P = .0011; Figure S3 in supporting information), but these associations were not observed among APOE ε4 carriers.
3.4. Interactions between oral hypoglycemic drug use and APOE ε4 carrier status in individuals with AD dementia
In people with AD dementia, APOE ε4 carrier status interacted with thiazolidinedione use (interaction: RR = 1.26 [1.07, 1.47], P = .0042), such that ε4 carriers exhibited less decline in immediate memory than non‐carriers (Figure S4 in supporting information). In non–ε4 carriers with AD, thiazolidinedione use (RR = 0.813 [0.714, 0.925], P = .0017) was associated with poorer immediate memory performance over time, whereas thiazolidinedione use was not significantly associated with immediate memory over time in APOE ε4 carriers.
Metformin use was associated with a faster rate of delayed memory decline specifically among APOE ε4 carriers with AD dementia (RR = 0.84 [0.73, 0.96], P = .0086; Figure S5 in supporting information).
4. DISCUSSION
Among people being treated for T2DM, different oral hypoglycemic medication classes were associated with different rates of memory change over time. These relationships were not consistent between people with and without AD clinical symptoms, and some relationships were modified by APOE ε4 carrier status.
In cognitively normal elderly, metformin use was associated with better memory performance over time, in agreement with a previous meta‐analytic report that metformin was associated with a decreased hazard of incident dementia. 4 Although two longitudinal studies previously identified no significant relationship between metformin use and memory performance over time in cognitively normal people, 5 , 6 the present study differs from those studies by including and accounting for patients using combination therapies, and by offering a sample size with adequate power to detect a relatively small effect size. Although small, this effect size was comparable to that of the APOE ε4 allele in this study sample, suggesting that it may be clinically meaningful. In diabetes animal models, metformin administration mitigated neuroinflammation, reduced neuron loss, and improved memory, 25 consistent with the observed associations observed here in cognitively normal people with T2DM.
In AD dementia, the use of a DPP4 inhibitor was associated with a slower rate of memory decline, in agreement with a previous report that sitagliptin was associated with an improvement in MMSE scores over 6 months in people with T2DM and AD. 9 Increasing evidence suggests that DPP4 inhibitors might slow the accumulation of AD pathology. DPP4 is responsible for glucagon‐like peptide‐1 (GLP1) breakdown, and blocking DPP4 increases circulating GLP1 levels in humans. 26 In animal and cell models, DPP4 inhibitors have been shown to increase hippocampal GLP1 levels, reduce hippocampal Aβ and p‐tau levels, alleviate Aβ plaque formation, and suppress neuroinflammation. 10 , 27 , 28 , 29 , 30 The DDP4 inhibitor linagliptin also increased cerebral blood flow in AD mouse models. 31 Generally, the DPP4 inhibitors have been hypothesized to exert beneficial effects on cognition that are related to AD pathological mechanisms. This is consistent with the observed relationship with slower memory decline in patients with AD, and with their significantly larger associations with memory in cognitively normal APOE ε4 carriers, who may have accelerated early AD pathology. 12 The DPP4 inhibitors can affect systems unrelated to GLP1, 26 and therefore further studies would be needed to understand the mechanism(s) of action relevant to memory, and to determine whether the GLP1 receptor agonists (incretins) would be equally beneficial. 32 We did not assess relationships between insulin use and memory because insulin is likely to reflect longer duration of diabetes, resulting in a considerably different propensity to receive insulin therapy leading to confounding by indication. Instead, insulin use was controlled for in the analyses.
A significant association between thiazolidinedione use and faster memory decline was identified in AD dementia, consistent with a previous report from the ACCORD‐MIND (Action to Control Cardiovascular Risk in Diabetes‐Memory in Diabetes) study, 33 and an overall lack of cognitive benefit in clinical trials for AD. 34 , 35 , 36 In AD animal models, pioglitazone had no effects on cognition or cerebral glucose use, 37 , 38 suggesting a possible basis for the observed lack of clinical cognitive benefit. In the present study, thiazolidinedione use was associated with a greater rate of decline in delayed memory particularly in ε4 non‐carriers. This contradicts some 34 but not all 35 , 36 of the results from randomized trials of thiazolidinedione versus placebo for AD that stratified analyses by APOE ε4 carrier status; however, those trials excluded patients with T2DM. Further pharmacoepidemiological studies would be required to confirm the present findings, in the setting of AD with comorbid T2DM.
Although there was no relationship between metformin and memory decline in AD, metformin use was associated with a greater decline in delayed memory specifically among APOE ε4 carriers. Similarly, in cognitively normal ε4 carriers, metformin use was not associated with delayed memory performance over time, but it was associated with less decline over time among ε4 non‐carriers. The findings might explain inconsistencies in the cognitive benefits of metformin seen previously in the literature. 39 In APOE ε4 transgenic and APOE gene deficient mice, 40 , 41 metformin failed to activate the adenosine monophosphate kinase pathway, worsened spatial memory, and exacerbated neurodegeneration. The majority of animal or cell studies suggest that metformin can increase levels of Aβ precursor protein and β‐secretase, 42 , 43 , 44 , 45 or increase tau hyperphosphorylation. 40 , 41 We hypothesize that the interactions between metformin and the amyloid cascade in those with AD might outweigh the neuroprotective effects seen in cognitively normal people. This is supported by the benefits of metformin seen in cognitive normal people particularly among those not carrying an APOE ε4 allele that accelerates AD pathology. Metformin showed no relationship with memory in aMCI, consistent with clinical trial data, 46 suggesting altogether that in the presence of cognitive impairment due to AD, metformin might no longer be neuroprotective.
The APOE genotype also significantly modified the relationship between DPP4 inhibitor use and memory over time in cognitively normal elderly, such that DPP4 inhibitor use was associated with greater benefit in ε4 carriers. The mechanistic basis for this finding is unknown, although DPP4 inhibitors reduced inflammation in APOE gene deficient mice. 47 The abovementioned benefits of DPP4 inhibitors against AD pathology in animal and cell models suggest that their effects may be more readily apparent in ε4 carriers because they are at greater risk of accumulating AD pathology. In those with AD, the APOE genotype was not a significant effect modifier for DPP4 inhibitors, indicating a consistent association with slower decline in memory in AD regardless of APOE carrier status. The data in toto suggest that DPP4 inhibitors may be beneficial specifically in the context of AD pathogenesis, including that accelerated by the APOE ε4 allele in people who are not yet symptomatic.
A notable strength of the study is the use of inverse probability weighting to address confounding by indication; however, the NACC database did not collect variables that might have improved the inverse probability weighting procedure, such as HbA1c, duration of diabetes, socioeconomic status, and the presence of kidney disease, which might have resulted in residual confounding. As a second major limitation, drug exposure history prior to entry into the database was not available, which might have introduced bias; however, the use of a marginal structural model controlled for time‐dependent effects based on the drugs used at each available observation. The study relied on clinical AD criteria due to insufficient biomarker or postmordem data to confirm AD status. It cannot speak to possible mechanisms (eg, atrophy, cerebrovascular changes, concentrations of p‐tau and Aβ) underlying the observed relationships, due to limited neuroimaging and cerebrospinal fluid biomarker data; these and other potential molecular mechanisms should be explored further. Similarly, the study cannot address the question of whether the observed relationships were mediated by glycemic control or insulin sensitivity due to the lack of available HbA1c and fasting glucose/insulin data, which is also a major limitation of the study and a major source of potential bias. In addition, the sample size was insufficient for subgroup analyses of specific drugs within a class, some of which may have differential effects; for example, studies have suggested possible benefits of rosiglitazone 48 but not pioglitazone 49 in people with diabetes and MCI. The sample was also insufficient for analysis of SGLT2 inhibitors and GLP1 agonists, but their relationships with memory relative to other diabetes drug classes will be of great interest in future investigations given recent findings of reduced dementia incidence associated with these drug classes in addition to DPP4 inhibitors and metformin. 32 Previous studies suggested that DPP4 inhibitors may have beneficial effects on cognition in patients with diabetes and MCI; 7 , 8 however, significant associations were not identified here in aMCI, possibly due to a relatively small and heterogeneous sample. The present results cannot be generalized to people without T2DM. Episodic memory performance may be affected by practice effects, 50 other comorbidities, physical and social activities, and concomitant medications that could not be controlled for. Loss to follow‐up due to decline in cognitive function could potentially lead to bias; however, weighting models by inverse probability of censorship did not change the results. Medication adherence could not be ascertained from the dataset. As a strength of the modelling approach, zero‐inflated models were used to minimize bias resulting from excess zeros and potential floor effects in memory scores.
5. CONCLUSION
This study offers observational evidence suggesting that certain oral hypoglycemic drug classes may be preferred in people with T2DM who are at risk for or with diagnosed AD. Metformin use was associated with better memory performance over time in cognitively normal people, while in people with AD dementia, DPP4 inhibitor use was associated with slower rates of memory decline, and thiazolidinedione use was associated with a faster rate of decline. APOE ε4 carrier status may predict greater benefit of DPP4 inhibitors in cognitively normal individuals, and less benefit of metformin in people with AD. The results have implications for personalized prevention and treatment of AD among people with T2DM, and for planning trials in AD with comorbid T2DM.
CONFLICTS OF INTEREST
CW, MO, YW, NZA, JDE, PY, BRS, MKK, NH, KLL, BJM, JSR, SEB, and WS declare no conflicts of interest.
Supporting information
ACKNOWLEDGMENTS
The NACC database is funded by NIA/NIH Grant U01 AG016976. NACC data are contributed by the NIA‐funded ADCs: P30 AG019610 (PI Eric Reiman, MD), P30 AG013846 (PI Neil Kowall, MD), P30 AG062428‐01 (PI James Leverenz, MD), P50 AG008702 (PI Scott Small, MD), P50 AG025688 (PI Allan Levey, MD, PhD), P50 AG047266 (PI Todd Golde, MD, PhD), P30 AG010133 (PI Andrew Saykin, PsyD), P50 AG005146 (PI Marilyn Albert, PhD), P30 AG062421‐01 (PI Bradley Hyman, MD, PhD), P30 AG062422‐01 (PI Ronald Petersen, MD, PhD), P50 AG005138 (PI Mary Sano, PhD), P30 AG008051 (PI Thomas Wisniewski, MD), P30 AG013854 (PI Robert Vassar, PhD), P30 AG008017 (PI Jeffrey Kaye, MD), P30 AG010161 (PI David Bennett, MD), P50 AG047366 (PI Victor Henderson, MD, MS), P30 AG010129 (PI Charles DeCarli, MD), P50 AG016573 (PI Frank LaFerla, PhD), P30 AG062429‐01(PI James Brewer, MD, PhD), P50 AG023501 (PI Bruce Miller, MD), P30 AG035982 (PI Russell Swerdlow, MD), P30 AG028383 (PI Linda Van Eldik, PhD), P30 AG053760 (PI Henry Paulson, MD, PhD), P30 AG010124 (PI John Trojanowski, MD, PhD), P50 AG005133 (PI Oscar Lopez, MD), P50 AG005142 (PI Helena Chui, MD), P30 AG012300 (PI Roger Rosenberg, MD), P30 AG049638 (PI Suzanne Craft, PhD), P50 AG005136 (PI Thomas Grabowski, MD), P30 AG062715‐01 (PI Sanjay Asthana, MD, FRCP), P50 AG005681 (PI John Morris, MD), P50 AG047270 (PI Stephen Strittmatter, MD, PhD). We gratefully acknowledge funding from the CIHR PJT‐159711 (PI Walter Swardfager, PhD), NSERC RGPIN‐2017‐06962 (PI Walter Swardfager, PhD), The Alzheimer's Association (US) and Brain Canada AARG501466 (PI Walter Swardfager, PhD), The Michael J. Fox Foundation, The Alzheimer's Association (US), Weston Brain Institute, and Alzheimer's Research UK BAND3 (PI Walter Swardfager, PhD).
We gratefully acknowledge funding from the CIHR PJT‐159711 (PI Walter Swardfager, PhD), NSERC RGPIN‐2017‐06962 (PI Walter Swardfager, PhD), The Alzheimer's Association (US) and Brain Canada AARG501466 (PI Walter Swardfager, PhD), The Michael J. Fox Foundation, The Alzheimer's Association (US), Weston Brain Institute, and Alzheimer's Research UK BAND3 (PI Walter Swardfager, PhD).
MKK is supported by a Mid‐Career Investigator Award from the Heart and Stroke Foundation of Canada, and holds the Lillian Love Chair in Women's Health from the University Health Network/University of Toronto.
Wu C‐Y, Ouk M, Wong YY, et al. Relationships between memory decline and the use of metformin or DPP4 inhibitors in people with type 2 diabetes with normal cognition or Alzheimer's disease, and the role ApoE carrier status. Alzheimer's Dement. 2020;16:1663–1673. 10.1002/alz.12161
REFERENCES
- 1. Profenno LA, Porsteinsson AP, Faraone SV. Meta‐Analysis of Alzheimers disease risk with obesity, diabetes, and related disorders. Biol Psychiatry. 2010;67:505‐512. [DOI] [PubMed] [Google Scholar]
- 2. Miklossy J, Qing H, Radenovic A, et al. Beta amyloid and hyperphosphorylated tau deposits in the pancreas in type 2 diabetes. Neurobiol Aging. 2010;31:1503‐1515. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Kuehn BM. In Alzheimer research, glucose metabolism moves to center stage. JAMA. 2020;323:297. [DOI] [PubMed] [Google Scholar]
- 4. Campbell JM, Stephenson MD, Courten BD, Chapman I, Bellman SM, Aromataris E. Metformin use associated with reduced risk of dementia in patients with diabetes: a systematic review and meta‐analysis. J Alzheimers Dis. 2018;65:1225‐1236. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Wennberg AM, Hagen CE, Edwards K, et al. Association of antidiabetic medication use, cognitive decline, and risk of cognitive impairment in older people with type 2 diabetes: results from the population‐based Mayo Clinic Study of Aging. Int J Geriatr Psychiatry. 2018;33:1114‐1120. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Weinstein G, Davis‐Plourde KL, Conner S, et al. Association of metformin, sulfonylurea and insulin use with brain structure and function and risk of dementia and Alzheimer's disease: pooled analysis from 5 cohorts. PLoS One. 2019;14(2):e0212293. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Rizzo MR, Barbieri M, Boccardi V, Angellotti E, Marfella R, Paolisso G. Dipeptidyl peptidase‐4 inhibitors have protective effect on cognitive impairment in aged diabetic patients with mild cognitive impairment. J Gerontol A Biol Sci Med Sci. 2014;69:1122‐1131. [DOI] [PubMed] [Google Scholar]
- 8. Borzì AM, Condorelli G, Biondi A, et al. Effects of vildagliptin, a DPP‐4 inhibitor, in elderly diabetic patients with mild cognitive impairment. Arch Gerontol Geriatr. 2019;84:103896. [DOI] [PubMed] [Google Scholar]
- 9. Isik AT, Soysal P, Yay A, Usarel C. The effects of sitagliptin, a DPP‐4 inhibitor, on cognitive functions in elderly diabetic patients with or without Alzheimer's disease. Diabetes Res Clin Pract. 2017;123:192‐198. [DOI] [PubMed] [Google Scholar]
- 10. D'Amico M, Filippo CD, Marfella R, et al. Long‐term inhibition of dipeptidyl peptidase‐4 in Alzheimer's prone mice. Exp Gerontol. 2010;45:202‐207. [DOI] [PubMed] [Google Scholar]
- 11. Evans JG, Sastre AA. Effect of the treatment of type II diabetes mellitus on the development of cognitive impairment and dementia. Cochrane Database Syst Rev. 2003(1), CD003804 10.1002/14651858.cd003804. [DOI] [Google Scholar]
- 12. Liu C‐C, Kanekiyo T, Xu H, Bu G. Apolipoprotein E and Alzheimer disease: risk, mechanisms and therapy. Nat Rev Neurol. 2013;9:106‐118. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Besser L, Kukull W, Knopman DS, et al. Version 3 of the National Alzheimer's Coordinating Center's Uniform Data Set. Alzheimer Dis Assoc Disord. 2018;32(4):351‐358. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Morris JC, Weintraub S, Chui HC, et al. The Uniform Data Set (UDS): clinical and Cognitive Variables and Descriptive Data From Alzheimer Disease Centers. Alzheimer Dis Assoc Disord. 2006;20:210‐216. [DOI] [PubMed] [Google Scholar]
- 15. Mckhann GM, Knopman DS, Chertkow H, et al. The diagnosis of dementia due to Alzheimers disease: recommendations from the National Institute on Aging‐Alzheimers Association workgroups on diagnostic guidelines for Alzheimers disease. Alzheimers Dement. 2011;7:263‐269. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Mckhann G, Drachman D, Folstein M, Katzman R, Price D, Stadlan EM. Clinical diagnosis of Alzheimers disease: report of the NINCDS‐ADRDA Work Group* under the auspices of Department of Health and Human Services Task Force on Alzheimers Disease. Neurology. 1984;34:939‐944. [DOI] [PubMed] [Google Scholar]
- 17. Petersen RC. Mild cognitive impairment as a diagnostic entity. J Intern Med. 2004;256:183‐194. [DOI] [PubMed] [Google Scholar]
- 18. Roman GC, Tatemichi TK, Erkinjuntti T, et al. Vascular dementia: diagnostic criteria for research studies: report of the NINDS‐AIREN International Workshop. Neurology. 1993;43:250‐260. [DOI] [PubMed] [Google Scholar]
- 19. Chelune GJ, Bornstein RA, Prifitera A. The Wechsler Memory Scale—revised. Adv Psychol Assess. 1990) 7 65‐99. 10.1007/978-1-4613-0555-2_3. [DOI] [Google Scholar]
- 20. Gómez‐Rubio V. ggplot2 ‐ Elegant Graphics for data analysis (2nd edition). J Stat Softw. 2017) 77 . 10.18637/jss.v077.b02. [DOI] [Google Scholar]
- 21. Bates D, Mächler M, Bolker B, Walker S. Fitting linear mixed‐effects models using lme4. J Stat Softw. 2015;67 1–48. 10.18637/jss.v067.i01. [DOI] [Google Scholar]
- 22. Molliee B, Kristensen K, Koenjvan B, et al. glmmTMB Balances speed and flexibility among packages for zero‐inflated generalized linear mixed modeling. R J. 2017;9:378. [Google Scholar]
- 23. van del Wal WM, Geskus RB. ipw: an R package for inverse probability weighting. J Stat Softw. 2011;43 1–23. 10.18637/jss.v043.i13. [DOI] [Google Scholar]
- 24. 9. Pharmacologic approaches to glycemic treatment: standards of medical care in diabetes—2019. Diabetes Care. 2018;42(suppl 1):S90‐S102. [DOI] [PubMed] [Google Scholar]
- 25. Oliveira WH, Nunes AK, França MER, et al. Effects of metformin on inflammation and short‐term memory in streptozotocin‐induced diabetic mice. Brain Res. 2016;1644:149‐160. [DOI] [PubMed] [Google Scholar]
- 26. Mulvihill EE, Drucker DJ. Pharmacology, physiology, and mechanisms of action of dipeptidyl peptidase‐4 inhibitors. Endocr Rev. 2014;35:992‐1019. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27. Angelopoulou E, Piperi C. DPP‐4 inhibitors: a promising therapeutic approach against Alzheimer's disease. Ann Transl Med. 2018;6(12):255. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28. Kosaraju J, Gali CC, Khatwal RB, et al. Saxagliptin: a dipeptidyl peptidase‐4 inhibitor ameliorates streptozotocin induced Alzheimer's disease. Neuropharmacology. 2013;72:291‐300. [DOI] [PubMed] [Google Scholar]
- 29. Kosaraju J, Murthy V, Khatwal RB, et al. Vildagliptin: an anti‐diabetes agent ameliorates cognitive deficits and pathology observed in streptozotocin‐induced Alzheimers disease. J Pharm Pharmacol. 2013;65:1773‐1784. [DOI] [PubMed] [Google Scholar]
- 30. Kosaraju J, Holsinger RMD, Guo L, Tam KY. Linagliptin, a dipeptidyl peptidase‐4 inhibitor, mitigates cognitive deficits and pathology in the 3xTg‐AD mouse model of Alzheimer's disease. Mol Neurobiol. 2016;54:6074‐6084. [DOI] [PubMed] [Google Scholar]
- 31. Nakaoku Y, Saito S, Yamamoto Y, Maki T, Takahashi R, Ihara M. The dipeptidyl peptidase‐4 inhibitor linagliptin ameliorates high‐fat induced cognitive decline in tauopathy model mice. Int J Mol Sci. 2019;20:2539. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32. Wium‐Andersen IK, Osler M, Jørgensen MB, Rungby J, Wium‐Andersen MK. Antidiabetic medication and risk of dementia in patients with type 2 diabetes: a nested case–control study. Eur J Endocrinol. 2019;181:499‐507. [DOI] [PubMed] [Google Scholar]
- 33. Seaquist ER, Miller ME, Fonseca V, et al. Effect of thiazolidinediones and insulin on cognitive outcomes in ACCORD‐MIND. J Diabetes Complications. 2013;27:485‐491. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34. Risner ME, Saunders AM, Altman JFB, et al. Efficacy of rosiglitazone in a genetically defined population with mild‐to‐moderate Alzheimers disease. Pharmacogenomics J. 2006;6:246‐254. [DOI] [PubMed] [Google Scholar]
- 35. Harrington C, Sawchak S, Chiang C, et al. Rosiglitazone does not improve cognition or global function when used as adjunctive therapy to AChE inhibitors in mild‐to‐moderate Alzheimer's disease: two phase 3 studies. Curr Alzheimer Res. 2011;8:592‐606. [DOI] [PubMed] [Google Scholar]
- 36. Gold M, Alderton C, Zvartau‐Hind M, et al. Rosiglitazone monotherapy in mild‐to‐moderate Alzheimer's disease: results from a randomized, double‐blind, placebo‐controlled phase III study. Dement Geriatr Cogn Disord. 2010;30:131‐146. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37. Masciopinto F, Pietro ND, Corona C, et al. Effects of long‐term treatment with pioglitazone on cognition and glucose metabolism of PS1‐KI, 3xTg‐AD, and wild‐type mice. Cell Death Dis. 2012;3(12):e448. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38. Galea E, Feinstein DL, Lacombe P. Pioglitazone does not increase cerebral glucose utilisation in a murine model of Alzheimer's disease and decreases it in wild‐type mice. Diabetologia. 2006;49:2153‐2161. [DOI] [PubMed] [Google Scholar]
- 39. Markowicz‐Piasecka M, Sikora J, Szydłowska A, Skupień A, Mikiciuk‐Olasik E, Huttunen KM. Metformin – a future therapy for neurodegenerative diseases. Pharm Res. 2017;34:2614‐2627. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40. Zhang J, Lin Y, Dai X, Fang W, Wu X, Chen X. Metformin treatment improves the spatial memory of aged mice in an APOE genotype–dependent manner. FASEB J. 2019;33:7748‐7757. [DOI] [PubMed] [Google Scholar]
- 41. Kuhla A, Brichmann E, Rühlmann C, Thiele R, Meuth L, Vollmar B. Metformin therapy aggravates neurodegenerative processes in ApoE–/– Mice. J Alzheimers Dis. 2019;68:1415‐1427. [DOI] [PubMed] [Google Scholar]
- 42. Son SM, Shin H‐J, Byun J, et al. metformin facilitates amyloid‐β generation by β‐ and γ‐secretases via autophagy activation. J Alzheimers Dis. 2016;51:1197‐1208. [DOI] [PubMed] [Google Scholar]
- 43. Picone P, Nuzzo D, Caruana L, et al. Metformin increases APP expression and processing via oxidative stress, mitochondrial dysfunction and NF‐κB activation: use of insulin to attenuate metformins effect. Biochim Biophys Acta. 2015;1853:1046‐1059. [DOI] [PubMed] [Google Scholar]
- 44. Picone P, Vilasi S, Librizzi F, et al. Biological and biophysics aspects of metformin‐induced effects: cortex mitochondrial dysfunction and promotion of toxic amyloid pre‐fibrillar aggregates. Aging. 2016;8:1718‐1734 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45. Li J, Deng J, Sheng W, Zuo Z. Metformin attenuates Alzheimers disease‐like neuropathology in obese, leptin‐resistant mice. Pharmacol Biochem Behav. 2012;101:564‐574. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46. Luchsinger JA, Perez T, Chang H, et al. Metformin in amnestic mild cognitive impairment: results of a pilot randomized placebo controlled clinical trial. J Alzheimers Dis. 2016;51:501‐514. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47. Avogaro A, Fadini GP. the effects of dipeptidyl peptidase‐4 inhibition on microvascular diabetes complications. Diabetes Care. 2014;37:2884‐2894. [DOI] [PubMed] [Google Scholar]
- 48. Abbatecola AM, Lattanzio F, Molinari AM, et al. Rosiglitazone and cognitive stability in older individuals with type 2 diabetes and mild cognitive impairment. Diabetes Care. 2010;33:1706‐1711. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49. Hildreth KL, Pelt REV, Moreau KL, et al. Effects of pioglitazone or exercise in older adults with mild cognitive impairment and insulin resistance: a pilot study. Dement Geriatr Cogn Dis Extra. 2015;5:51‐63. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 50. Benedict RH, Zgaljardic DJ. practice effects during repeated administrations of memory tests with and without alternate forms. J Clin Exp Neuropsychol. 1998;20:339‐352. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.