Abstract
Introduction
The Alzheimer’s Disease Composite Score (ADCOMS) is a tool developed to detect clinical progression and measure treatment effect in patients in early stages of Alzheimer’s disease (AD). The psychometric properties of the ADCOMS have been established; however, the threshold for clinical meaningfulness has yet to be identified.
Methods
Anchor-based, distribution-based, and ROC curve analyses were used to estimate clinically meaningful thresholds for change in ADCOMS for patients with mild cognitive impairment (MCI) and AD dementia. This study included data from three sources: the Alzheimer’s Disease Neuroimaging Initiative (ADNI), the National Alzheimer’s Coordinating Center (NACC), and a legacy dataset that included data from four sources: the placebo group from three MCI trials and an earlier data cut from ADNI. Results were stratified by disease severity (MCI vs. dementia) and APOE ε4 carrier status.
Results
A total of 5355 participants were included in the analysis. The ADCOMS was able to detect change for MCI and dementia patients who experienced a meaningful decline in cognition (as defined by the Clinical Dementia Rating Scale Sum of Boxes [CDR-SOB]) between baseline and month 12. The following ADCOMS cut-offs were proposed: 0.05 for MCI and 0.10 for dementia.
Conclusions
The ADCOMS was previously established as a valid and reliable tool for use in clinical trials for MCI due to AD and dementia populations. By defining thresholds for clinically meaningful change of ADCOMS, this work is an important step in interpreting clinical findings and estimates of treatment effects in early stage AD trials.
Supplementary Information
The online version contains supplementary material available at 10.1007/s40120-022-00352-w.
Keywords: Alzheimer’s disease, Clinically meaningful change, Composite measure, Dementia, Mild cognitive impairment
Key Summary Points
Why carry out this study? |
To assess treatment effects in clinical trials for mild cognitive impairment (MCI) due to Alzheimer’s disease (AD), the FDA recommended the use of a composite measure. Following this recommendation, Eisai developed and validated the Alzheimer’s Disease Composite Score (ADCOMS). |
Although the psychometric properties of the ADCOMS have been established, the threshold for clinical meaningfulness has yet to be identified. The objective of this study was to identify that threshold. |
What was learned from the study? |
Based on a triangulation evaluating the range of estimates for meaningful change, the following cut-off points for meaningful change were proposed: 0.05 for MCI and 0.10 for dementia. |
By establishing the thresholds for clinically meaningful change, researchers can more confidentially use the ADCOMS as an outcome measure for clinical trials in MCI due to AD and for dementia. |
Introduction
Alzheimer’s disease (AD) is the most common type of major neurocognitive disorder, in which the death of brain cells leads to cognitive decline and memory loss [1]. The underlying pathology of AD begins years before the appearance of clinical symptoms. Research is ongoing to better understand the genetic, molecular, and clinical mechanisms of the disease in order to characterize early disease and prevent its progression to AD [2, 3]. Mild cognitive impairment (MCI) is an early stage impairment that precedes AD. Patients with MCI due to AD experience limited cognitive symptoms, but these impairments are mild by definition, and patients are able to function fairly normally [4, 5]. Even though early deficits in cognition may be subtle, it is important to identify and treat them to hopefully delay or prevent more severe neurodegeneration [6, 7].
Prior to the Food and Drug Administration’s (FDA) approval of aducanumab in 2021 [8], no new treatments had been approved by the FDA since 2003. Within that period of time, more than 200 investigational programs failed or were abandoned because of several factors, including treating patients after AD had already progressed, inappropriate treatment dosage, endpoints that lacked sensitivity to change, a lack of understanding of the pathophysiology of AD, and trial design issues [9]. Because early disease has limited and variable symptomatology, it is critical that the measurement of clinical outcomes in trials for AD treatments is sensitive to the specific symptoms and possible treatment effects in the early stages of disease. In the FDA’s 2013 guidance for developing treatments for early stage AD, they suggested that a composite measure may be the most appropriate for use in a clinical trial for MCI due to AD to measure clinical outcomes [10].
Following the FDA’s recommendations, Eisai developed the Alzheimer’s Disease Composite Score (ADCOMS), which combines components of widely used clinical scales that have been shown to be the most responsive to treatment effects [11]. The tool was developed for use in early AD (i.e., MCI and mild AD dementia) clinical trials. The ADCOMS tool was initially developed and validated using data from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) [12] and the placebo group from three clinical trials in patients with MCI [11]. To select the items for inclusion in the ADCOMS tool, a linear decline model was developed to depict disease progression over 12 months using all the items from available clinical scales in these datasets. Items with the highest combined sensitivity to change over time were selected for the final tool that includes four items from the Alzheimer’s disease Scale-Cognitive Subscale (ADAS-Cog) [13], two from the Mini Mental State Examination (MMSE) [14], and all six items from the Clinical Dementia Rating Scale Sum of Boxes (CDR-SOB) [15].
Like the FDA, the EU/US Clinical Trials in Alzheimer’s disease (CTAD) task force also recommends a composite measure for pre-dementia trials and has pointed to the ADCOMS tool as an appropriate measure that incorporates both cognitive and functional elements [16]. The ADCOMS tool has demonstrated improved sensitivity to clinical decline over other individual scores like the ADAS-Cog, CDR-SOB, and MMSE [17], and in a prodromal AD randomized trial, it was able to identify cognitive and functional benefits, suggesting that it is sensitive to treatment changes [18]. Although these psychometric properties of the ADCOMS have been established, the threshold for ascertainment of the clinical meaningfulness of disease progression using the ADCOMS has yet to be identified. Therefore, the objective of this study was to evaluate the clinical meaningfulness threshold of the ADCOMS among patients with MCI due to AD and mild AD. A secondary objective of this work was to compare the threshold between APOE ε4 allele carriers and non-carriers. The APOE ε4 allele increases the risk of AD and is associated with earlier age of dementia onset [19].
Methods
Materials
The following three data sources were used to conduct this study.
Alzheimer’s Disease Neuroimaging Initiative (ADNI) data Data used in the preparation of this article were obtained from the ADNI database (adni.loni.usc.edu). ADNI was launched in 2003 as a public-private partnership, led by Principal Investigator Michael W. Weiner, MD. The primary goal of the ADNI has been to test whether serial magnetic resonance imaging (MRI), positron emission tomography (PET), other biological markers, and clinical and neuropsychological assessment can be combined to measure and predict the progression of MCI and early AD. For up-to-date information, see www.adni-info.org. ADNI is a global, prospective, observational cohort study intended to develop clinical, imaging, genetic, and biochemical biomarkers for early detection and monitoring of AD [12]. The ADNI datasets used for this analysis (ADASSCORES, CDR, MMSE and BLCHANGE) were downloaded in December 2020 and included the following ADNI phases: ADNI1, ADNI2, and ADNI GO. The data selected for Visit 1 included the data at screening and baseline or month ‘0;’ the data selected for Visit 2 included data at month 12.
National Alzheimer’s Coordinating Center (NACC) data The NACC is a prospective, longitudinal study, which maintains a large database of standardized clinical neuropathological research data collected from the NIA-funded Alzheimer’s Disease Centers (ADCs) (funded by the National Institute on Aging [U24 AG072122]). The clinical variables contained in the NACC Uniform Data sets version 3.0 (UDS 3.0) were downloaded in December 2020 (N = 38,137). The following visits were selected: days from the initial visit (NACCFDYS) were either 0 for Visit 1 or between 335 or 395 for Visit 2, and the visit number (NACCVNUM) was limited to Visits 1 and Visit 2.
Combined Legacy Dataset The third data source is a combined legacy dataset, the one which was originally used to develop the ADCOMS [11]. It contains pooled data from four sources: (1) a data collection wave from the observational study, ADNI (ADNI-1; ADNI-MCI, downloaded on 20 May 2010); (2) the placebo group from the Alzheimer’s Disease Cooperative Study (ADCS), a randomized, double-blind, placebo-controlled trial to evaluate the safety and efficacy of vitamin E and donepezil HCL (Aricept) to delay clinical progression from MCI to AD (ADCS-MCI); (3) the placebo group of a 1-year, multicenter, randomized, double-blind, placebo-controlled evaluation of the efficacy and safety of donepezil hydrochloride in patients with MCI (E2020-A001-412 NCT00293176 or the MCI-412) study; (4) the placebo group from the Hippocampus Study: comparative effect of donepezil 10 mg/day and placebo on clinical and radiological markers (E2020-A001-412 NCT00293176 or the MCI-415). The sample size for the legacy dataset includes the following: n = 405 from ADNI, n = 264 from ADCS-MCI, n = 388 from MCI-412, and n = 103 from MCI-415. The two visits for this dataset were baseline and month 12. All procedures performed in these studies involving human participants were in accordance with the ethical standards of the institutional review board and with the 1964 Helsinki Declaration and its later amendments, Permissions were obtained to access and use the data from the ADNI and NACC databases. Participants who were categorized as having clinical impairment at baseline, but normal levels of impairment at the 12-month visit, were excluded from the analyses (N = 7 for ADNI; N = 122 for NACC).
To score the ADCOMS, variables from the MMSE, CDR-SOB, and ADAS-Cog are required. The ADAS-Cog was not collected in the NACC. Therefore, the ADCOMS was imputed for the NACC at baseline and month 12 using the following procedure: ADCOMS was first calculated using the ADNI dataset with the MMSE, CDR-SOB, and ADAS-Cog. Then, ADCOMS was predicted in the ADNI using only the items available in the NACC (MMSE items 1 and 7, CDR-SOB items). The regression weights obtained from that model were then applied to predict the ADCOMS in the NACC data.
The overall study analysis population included the subjects with non-missing ADCOMS at baseline and the 12-month visit. Thus, after selecting participants with non-missing ADCOMS at Visit 1 and Visit 2, the final sample size for the study population was n = 3675 for NACC, n = 714 for ADNI, and n = 966 for the legacy dataset. It should be noted that two of the datasets may contain overlapping patients (ADNI dataset N = 714 and the ADNI component of the legacy dataset N = 358). The legacy ADNI data were downloaded in 2010 using ADNI-1 and ADNI-MCI, while the ADNI data we downloaded were downloaded in 2020, using ADNI-1, ADNI-2, and ADNI-GO. Based on the size of the two samples, they could at most overlap by 50%.
Methods
We used multiple methods to derive the threshold for clinical meaningfulness (anchor-based method, ROC analysis, and distribution-based method). The anchor-based method used the CDR-SOB as the anchor with sensitivity analyses conducted using MMSE and ADAS-Cog as alternative anchors. The anchor-based method used CDR-SOB change ≥ 0.5 as a meaningful decline cut-off for the MCI group and CDR-SOB change ≥ 1 for the dementia group. These cut-offs were determined based on findings from the Andrews et al. (2019) study, where the minimum threshold for clinically meaningful decline was defined as 0.5 on the CDR-SOB [20]. Between visits in the Andrews et al. study, patients with MCI had an average increase of 0.98 on the CDR-SOB, and patients with dementia had an average increase of 1.63 on the CDR-SOB. Because here we are looking at the minimum values to define decline, and this required an integer value that could be measured within each patient, 0.5 and 1.0 were selected as the cut-offs for MCI and dementia stages, respectively. The mean changes in ADCOMS based on the CDR-SOB thresholds for meaningful change were evaluated using t-tests for both MCI and dementia.
The ROC method compared the model-based predictions for different ADCOMS cut-offs using the CDR-SOB pre-defined groups (≥ 0.5 for the MCI group and ≥ 1 for the dementia group). The ADCOMS cut-offs were selected using the highest values of Youden’s index (which is a measure of balance between the levels of sensitivity and specificity).
Distribution-based methods to evaluate meaningful change were also used, including three statistics: the ¼ standard deviation (SD), ½ SD at baseline, and the standard error of measurement (SEM). The SEM was computed by multiplying the SD of the measure by the square root of one minus its reliability (in this analysis, the reliability coefficient used was the intra-class correlation coefficient [ICC]). The ICC was computed on a stable population, defined using the following criteria: ‘No change’ response for impairment/worsening in non-memory cognitive domains on neuropsychological testing in the legacy dataset, no subject-reported decline in memory at both the baseline and month 12 visits for the NACC, and a ‘No’ [change] response on the clinician global impression of change (CGIC) at month 12 for the ADNI.
The anchor-based, distribution-based, and ROC methods described above were used for triangulation to derive preliminary thresholds for clinically meaningful change on the ADCOMS for MCI and dementia stages. In addition, we considered APOE ε4 status within MCI and dementia groups to determine whether carrier vs. non-carrier status would alter the thresholds for meaningful change. Probability density function (PDF) plots and cumulative density function (CDF) plots by CDR-SOB groups were used to provide supportive evidence for the appropriateness of the proposed thresholds.
Results
There were 5355 participants in the overall analysis sample who had ADCOMS (actual or imputed) at baseline and month 12. Sociodemographic and clinical characteristics are summarized in Table 1.
Table 1.
Combined legacy dataset (N = 966) |
NACC (N = 3675) |
ADNI (N = 714) |
|
---|---|---|---|
Age | |||
N | 966 | 3675 | 714 |
Mean (SD) | 72.56 (8.44) | 70.92 (9.86) | 75.34 (6.71) |
Median [range] | 74.0 [45.00–90.00] | 71.0 [22.00–100.0] | 75.6 [55.10–90.90] |
Missing, n (%) | 0 (0.0%) | 0 (0.0%) | 0 (0.0%) |
Sex (n, %) | |||
Male | 566 (58.6%) | 1687 (45.9%) | 416 (58.3%) |
Female | 400 (41.4%) | 1988 (54.1%) | 298 (41.7%) |
Missing | 0 (0.0%) | 0 (0.0%) | 0 (0.0%) |
Education (years) | |||
N | 312 | 3667 | 714 |
Mean (SD) | 14.75 (3.09) | 15.41 (3.31) | 15.66 (2.96) |
Median [range] | 15.0 [5.00–35.00] | 16.0 [0.00–30.00] | 16.0 [6.00–20.00] |
Missing, n (%) | 654 (67.7%) | 8 (0.2%) | 0 (0.0%) |
Global CDR | |||
N | 208 | 3675 | 714 |
Mean (SD) | 0.50 (0.00) | 0.44 (0.51) | 0.41 (0.30) |
Median [range] | 0.5 [0.50–0.50] | 0.5 [0.00–3.00] | 0.5 [0.00–1.00] |
Missing, n (%) | 758 (78.5%) | 0 (0.0%) | 0 (0.0%) |
CDR-SOB | |||
N | 966 | 3675 | 714 |
Mean (SD) | 1.64 (0.89) | 2.17 (2.97) | 1.72 (1.76) |
Median [range] | 1.5 [0.50–5.00] | 1.0 [0.00–18.00] | 1.5 [0.00–9.00] |
Missing, n (%) | 0 (0.0%) | 0 (0.0%) | 0 (0.0%) |
MMSE | |||
N | 966 | 3674 | 714 |
Mean (SD) | 27.14 (1.93) | 26.09 (4.67) | 26.85 (2.58) |
Median [range] | 27.0 [18.00–30.00] | 28.0 [0.00–30.00] | 27.0 [20.00–30.00] |
Missing, n (%) | 0 (0.0%) | 1 (0.0%) | 0 (0.0%) |
ADAS-Cog | |||
N | 963 | – | 714 |
Mean (SD) | 11.02 (4.54) | – | 11.43 (6.27) |
Median [range] | 10.7 [0.83–27.67] | – | 10.7 [0.00–42.67] |
Missing, n (%) | 3 (0.3%) | – | 0 (0.0%) |
ADCOMS | |||
N | 966 | 3675 | 714 |
Mean (SD) | 0.22 (0.10) | 0.29 (0.34) | 0.25 (0.21) |
Median [range] | 0.2 [0.03–0.58] | 0.1 [0.04–2.01] | 0.2 [0.00–1.17] |
Missing, n (%) | 0 (0.0%) | 0 (0.0%) | 0 (0.0%) |
Clinician diagnosis of cognitive status, n (%) | |||
Normal cognition | 0 (0.0%) | 1619 (44.1%) | 206 (28.9%) |
Impaired-not-MCI | 0 (0.0%) | 117 (3.2%) | 0 (0.0%) |
MCI | 966 (100.0%) | 686 (18.7%) | 349 (48.9%) |
Dementia | 0 (0.0%) | 1253 (34.1%) | 159 (22.3%) |
APOE | |||
APOE ε4 carrier | 406 (42.0%) | 1419 (38.6%) | 352 (49.3%) |
APOE ε4 non-carrier | 401 (16.5%) | 1952 (53.1%) | 362 (50.7%) |
Missing | 159 (16.5%) | 304 (8.3%) | 0 (0.0%) |
ADCOMS Alzheimer’s Disease Composite Score, ADAS-Cog Alzheimer’s Disease Scale-Cognitive Subscale, ADNI Alzheimer’s Disease Neuroimaging Initiative, CDR-SOB Clinical Dementia Rating Scale Sum of Boxes, MCI mild cognitive impairment, MMSE Mini Mental State Examination, NACC National Alzheimer’s Coordinating Center, SD standard deviation
The average ADCOMSs at baseline were 0.22 for the combined legacy dataset (range 0.03–0.58), 0.29 for NACC (range 0.04–2.01), and 0.25 for ADNI (range 0.00–1.17). The average ADCOMSs at baseline across the datasets were 0.22 for MCI patients and 0.64 for dementia patients (Table 2). There were minor floor effects limited to the normal cognition group (0.4%) and no ceiling effects. The sample consisted mainly of patients with MCI and dementia, although there were some in the NACC and ADNI who were considered to have normal cognition for their age. The combined legacy dataset did not include any dementia patients. The NACC data had fewer MCI patients than dementia patients, while the ADNI set had more MCI patients (MCI: combined legacy dataset N = 966; NACC N = 686; ADNI N = 249; dementia: combined legacy dataset N = 0; NACC N = 1253; ADNI N = 159).
Table 2.
Group | n | Mean | SD | Observed range (min–max) |
Floor n(%) |
Ceiling n(%) |
---|---|---|---|---|---|---|
Normal cognition | ||||||
All | 1825 | 0.06 | 0.04 | 0.00–0.48 | 7 (0.4%) | 0 (0.0%) |
APOE ε4 carrier | 542 | 0.06 | 0.04 | 0.00–0.34 | 3 (0.6%) | 0 (0.0%) |
APOE ε4 non-carrier | 1198 | 0.05 | 0.03 | 0.00–0.48 | 4 (0.3%) | 0 (0.0%) |
Impaired (not-MCI) | ||||||
All | 117 | 0.15 | 0.09 | 0.04–0.48 | 0 (0.0%) | 0 (0.0%) |
APOE ε4 carrier | 38 | 0.17 | 0.09 | 0.04–0.38 | 0 (0.0%) | 0 (0.0%) |
APOE ε4 non-carrier | 67 | 0.14 | 0.08 | 0.04–0.48 | 0 (0.0%) | 0 (0.0%) |
MCI | ||||||
All | 2001 | 0.22 | 0.11 | 0.03–0.75 | 0 (0.0%) | 0 (0.0%) |
APOE ε4 carrier | 885 | 0.24 | 0.11 | 0.04–0.75 | 0 (0.0%) | 0 (0.0%) |
APOE ε4 non-carrier | 890 | 0.21 | 0.11 | 0.03–0.71 | 0 (0.0%) | 0 (0.0%) |
Dementia | ||||||
All | 1412 | 0.64 | 0.34 | 0.04–2.01 | 0 (0.0%) | 0 (0.0%) |
APOE ε4 carrier | 712 | 0.63 | 0.30 | 0.09–2.01 | 0 (0.0%) | 0 (0.0%) |
APOE ε4 non-carrier | 560 | 0.62 | 0.33 | 0.04–2.01 | 0 (0.0%) | 0 (0.0%) |
ADCOMS Alzheimer’s Disease Composite Score, MCI mild cognitive impairment, SD standard deviation
APOE ε4 carrier status is missing for n = 463 participants
Validity
The convergent and divergent validities of the ADCOMS were examined by evaluating its correlation with the CDR-SOB, MMSE, ADAS-Cog, and Geriatric Depression Scale (GDS) (Table 3). The results demonstrated that the correlation of the ADCOMS with corresponding neurocognitive assessments was strong (r > 0.50), both at baseline and for the change from baseline to 12 months, with a few exceptions: MMSE in the MCI group at baseline, MMSE in the MCI group change over time, and the ADAS-Cog in both MCI and dementia groups' change over time. The correlation with the CDR-SOB was the strongest, suggesting that the CDR-SOB would make the best anchor for evaluation of the clinical meaningfulness of the ADCOMS. The correlation with the GDS, a measure of depression, was weak and non-significant, providing further confirmation that the ADCOMS is a measure of cognition, not depression.
Table 3.
Measure | ADCOMS, baseline | ADCOMS, change from baseline to month 12 | ||||||
---|---|---|---|---|---|---|---|---|
MCI | Dementia | MCI | Dementia | |||||
N | ra | N | ra | N | ra | N | ra | |
CDR-SOB | 2001 | 0.88**** | 1412 | 0.95**** | 2001 | 0.88**** | 1412 | 0.91**** |
MMSE | 2001 | – 0.39**** | 1411 | – 0.70**** | 2001 | – 0.44**** | 1411 | – 0.51**** |
ADAS-Cog | 1312 | 0.50**** | 159 | 0.62**** | 1312 | 0.40**** | 159 | 0.45**** |
GDS | 1016 | 0.02 | 1355 | 0.01 | 1010 | 0.08** | 1272 | 0.05 |
ADCOMS Alzheimer’s Disease Composite Score, ADAS-Cog Alzheimer’s Disease Scale-Cognitive Subscale, CDR-SOB Clinical Dementia Rating Scale Sum of Boxes, GDS Geriatric Depression Scale, MCI mild cognitive impairment, MMSE Mini Mental State Examination
*p < 0.05; **p < 0.01;***p < 0.001; ****p < 0.0001
aSpearman rank-order correlations
Establishing Clinical Meaningfulness of the ADCOMS
The ability of the ADCOMS to detect change was evaluated as the FDA recommended, namely by comparing changes in ADCOMS to changes in CDR-SOB, MMSE, and ADAS-Cog, which indicated that the patient’s cognition had declined [21]. In Table 4, the data for the CDR-SOB are presented (only the CDR-SOB results are shown), which show that the ADCOMS demonstrated statistically significant changes in MCI groups, defined by a ≥ 0.5 change on the CDR-SOB in all three data sets, p < 0.0001. The ADCOMS was also able to detect change between dementia patients (based on a ≥ 1.0 change on the CDR-SOB). Across datasets, the mean difference between MCI patients with a meaningful decline and no meaningful decline (as defined by scores on the CDR-SOB) ranged from 0.17 to 0.20 (Table 4). The mean difference between dementia patients with a meaningful decline and no meaningful decline ranged from 0.26 to 0.33. Similar results were observed when subjects were stratified by APOE ε4 carrier status, with the ADCOMS able to detect change within both groups. However, change from baseline in the APOE ε4 carriers tended to be numerically larger among MCI patients (MCI patients: Table 5; dementia patients: Table 6). This trend of larger changes in APOE ε4 was not consistently maintained for dementia patients.
Table 4.
ADCOMS change scorea | Total N | Change in CDR-SOBb | Mean difference (95% CI) |
t-test | p-valuec | |
---|---|---|---|---|---|---|
No meaningful decline n Mean (SD) |
Meaningful decline n Mean (SD) |
|||||
MCI | ||||||
Legacy data | 966 |
522 – 0.022 (0.066) |
444 0.158 (0.133) |
– 0.180 (– 0.194; – 0.167) | – 26.01 | < 0.0001 |
NACC | 686 |
332 – 0.031 (0.073) |
354 0.169 (0.137) |
– 0.200 (– 0.216; – 0.184) | – 24.05 | < 0.0001 |
ADNI | 349 |
141 – 0.016 (0.064) |
208 0.151 (0.116) |
– 0.167 (– 0.186; – 0.147) | – 17.12 | < 0.0001 |
Dementia | ||||||
NACC | 1253 |
538 – 0.012 (0.118) |
715 0.318 (0.227) |
– 0.330 (– 0.349; – 0.310) | – 33.32 | < 0.0001 |
ADNI | 159 |
65 0.036 (0.102) |
94 0.300 (0.223) |
– 0.264 (– 0.316; – 0.212) | – 10.04 | < 0.0001 |
ADCOMS Alzheimer’s Disease Composite Score, ADNI Alzheimer’s Disease Neuroimaging Initiative, CDR-SOB Clinical Dementia Rating Scale Sum of Boxes, CI confidence interval, MCI mild cognitive impairment, NACC National Alzheimer’s Coordinating Center, SD standard deviation
aChange score = month 12 score – baseline score
bMeaningful decline: CDR-SOB change ≥ 0.5 for MCI; ≥ 1 for dementia
ct-test
Table 5.
ADCOMS change scorea | Total N | Changes in CDR-SOBb | Mean Difference (95% CI) |
t-test | p-valuec | |
---|---|---|---|---|---|---|
No meaningful decline n Mean (SD) |
Meaningful decline n Mean (SD) |
|||||
APOE ε4 carrier | ||||||
Pooled data | 406 |
268 0.012 (0.079) |
138 0.234 (0.138) |
– 0.222 (– 0.247; – 0.197) | – 17.43 | < 0.0001 |
NACC | 289 |
181 – 0.007 (0.080) |
108 0.236 (0.135) |
– 0.243 (– 0.271; – 0.215) | – 16.96 | < 0.0001 |
ADNI | 190 |
114 0.026 (0.075) |
76 0.223 (0.111) |
– 0.197 (– 0.226; – 0.168) | – 13.61 | < 0.0001 |
APOE ε4 non-carrier | ||||||
Pooled data | 401 |
306 – 0.015 (0.072) |
95 0.182 (0.124) |
– 0.198 (– 0.224; – 0.171) | – 14.80 | < 0.0001 |
NACC | 330 |
228 – 0.009 (0.079) |
102 0.205 (0.126) |
– 0.214 (– 0.241; – 0.188) | – 15.88 | < 0.0001 |
ADNI | 159 |
102 – 0.005 (0.069) |
57 0.173 (0.103) |
– 0.177 (– 0.208; – 0.147) | – 11.65 | < 0.0001 |
ADCOMS Alzheimer’s Disease Composite Score, ADNI Alzheimer’s Disease Neuroimaging Initiative, CDR-SOB Clinical Dementia Rating Scale Sum of Boxes, CI confidence interval, MCI mild cognitive impairment, NACC National Alzheimer’s Coordinating Center, SD standard deviation
aChange score = month 12 score – baseline score
bMeaningful decline: CDR-SOB change ≥ 0.5 from baseline to month 12
ct-test
Table 6.
ADCOMS change scorea | Total N | Changes in CDR-SOBb | Mean Difference (95% CI) |
t-test | p-valuec | |
---|---|---|---|---|---|---|
No meaningful decline n Mean (SD) |
Meaningful decline n Mean (SD) |
|||||
APOE ε4 carrier | ||||||
NACC | 607 |
259 – 0.007 (0.113) |
348 0.301 (0.215) |
– 0.308 (– 0.334; – 0.281) | – 22.79 | < 0.0001 |
ADNI | 105 |
44 0.033 (0.097) |
61 0.309 (0.239) |
– 0.275 (– 0.342; – 0.208) | – 8.11 | < 0.0001 |
APOE ε4 non-carrier | ||||||
NACC | 506 |
224 – 0.019 (0.123) |
282 0.325 (0.234) |
– 0.344 (– 0.376; – 0.312) | – 21.25 | < 0.0001 |
ADNI | 54 |
21 0.041 (0.115) |
33 0.284 (0.193) |
– 0.243 (– 0.327; – 0.158) | – 5.78 | < 0.0001 |
ADCOMS Alzheimer’s Disease Composite Score, ADNI Alzheimer’s Disease Neuroimaging Initiative, CDR-SOB Clinical Dementia Rating Scale Sum of Boxes, CI confidence interval, NACC National Alzheimer’s Coordinating Center, SD standard deviation
aChange score = month 12 score – baseline score
bMeaningful decline: CDR-SOB change ≥ 1.0 from baseline to month 12
ct-test
To further evaluate thresholds for meaningful change, we evaluated sensitivity, specificity, and Youden’s index from an ROC curve analysis. For MCI patients, the threshold values with the best Youden’s indices from the ROC curve analyses were 0.05 for the combined legacy dataset, 0.02–0.03 for NACC, and 0.05 for ADNI. For dementia patients, the values with the best Youden’s indices were 0.11 for NACC and 0.13 for ADNI.
We also conducted the ROC analysis based on APOE ε4 status in patients with MCI and dementia. For MCI patients with APOE ε4 status, the best Youden’s indices from the ROC curve analyses were 0.09, 0.06, and 0.05 for the combined legacy dataset, 0.02–0.03 for NACC, and 0.08 for ADNI. For MCI patients without the APOE ε4 biomarker, the best Youden’s indices were 0.03–0.05 for the combined legacy dataset, 0.02–0.03 for NACC, and 0.04 for ADNI. For dementia patients with the APOE ε4 biomarker, the values with the best Youden’s index were 0.11 for NACC and ADNI, while for dementia patients without the biomarker, the best values were 0.06 and 0.11 for NACC and 0.13 for ADNI. [These data are included in Supplemental Table 1.]
Table 7 presents all of the above outcomes as well as several distribution-based statistics and the median change in ADCOMS to aid in the process of selecting the best threshold. The resulting thresholds were evaluated in CDF (Figs. 1 and 2) and PDF plots (Supplemental Figures). The dashed vertical lines in Figs. 1 and 2 reflect the proposed thresholds of 0.05 and 0.10 for MCI and dementia, respectively.
Table 7.
MCI | Dementia | |
---|---|---|
Anchor-based Approach (CDR-SOB) | ||
ROC analysis | 0.02–0.05 | 0.11–0.15 |
Mean differencea | 0.17–0.20 | 0.26–0.33 |
Median changeb | 0.14 | 0.26 |
Distribution-based approach | ||
0.25*SD | 0.02–0.03 | 0.05–0.09 |
0.5*SD | 0.05–0.06 | 0.09–0.17 |
SEM | 0.04–0.05 | 0.08–0.10 |
The range of values presented for MCI is from the three data sources (ADNI, NACC, pooled data). The range of values presented for dementia is from the two data sources (ADNI, NACC)
ADCOMS Alzheimer’s Disease Composite Score, ADNI Alzheimer’s Disease Neuroimaging Initiative, CDR-SOB Clinical Dementia Rating Scale Sum of Boxes, MCI mild cognitive impairment, NACC National Alzheimer’s Coordinating Center, SD standard deviation, ROC receiver-operating characteristic, SEM standard error of measurement
aMean difference between change in ADCOMS scores (from baseline to month 12) for patients with no meaningful decline vs. meaningful decline at month 12 (defined by CDR-SOB ≥ 0.5 for MCI; CDR-SOB ≥ 1 for dementia)
bMedian change in ADCOMS score among those who had meaningful decline on the CDR-SOB (defined by CDR-SOB ≥ 0.5 for MCI; CDR-SOB ≥ 1 for dementia)
The SEM for MCI ranged from 0.04 to 0.05 across the datasets (Table 7), and it was in a similar range for calculations based on 0.25 SD and 0.5 SD. For dementia patients, the SEM ranged from 0.08 to 0.10 across the datasets.
The median change in the ADCOMS among those who had meaningful decline was 0.14 for MCI patients and 0.26 for dementia patients.
Based on a triangulation evaluating the range of estimates for meaningful change, the following cut-off points for meaningful change were proposed: 0.05 for MCI and 0.10 for dementia. To determine this threshold, the ROC curve analysis was prioritized. However, the distribution-based estimates also provided ranges consistent with the thresholds selected. The mean differences and median values were de-emphasized. The median values reflect the median change in those in the decline and no-decline groups and therefore represent the mid-point of changes, not the threshold that is associated with meaningful change. Accordingly, they were given less weight in the triangulation. The same principle applies to the mean differences. The results of the APOE ε4 subgroup analyses provided roughly similar results, suggesting that the same thresholds for meaningful change can be used in these subgroups.
PDF and CDF plots were produced to supplement the triangulation result and evaluate how well the proposed thresholds performed in separating those with cognitive decline from those without decline (again stratified by meaningful change on the CDR-SOB) for the combined datasets.
Both the PDF and CDF plots show separation between the curves based on the proposed minimal clinically important differences between the decline and no decline groups for MCI (0.05) and dementia (0.10) (PDF plots in supplemental materials) (Fig. 1 for MCI; Fig. 2 for dementia).
Discussion
This article builds upon previous research which has established that the ADCOMS is a valid and reliable tool. A major strength is that the overall study sample was derived from three different sources, including the ADNI, NACC, and a legacy dataset made up of data from four studies (the placebo groups from three MCI studies as well as an earlier cut from the ADNI database, which also contained MCI patients). Across these disparate sources of information, using a large sample of patients who ranged in severity from MCI to dementia, the results were very consistent.
The ADCOMS was specifically developed to improve sensitivity to treatment effects by choosing items from well-known measures that were more sensitive to change than other components of the measures. In fact, the ADCOMS has previously demonstrated improved sensitivity to clinical decline when compared to traditional measures for AD, such as the ADAS-Cog [11]. The ADCOMS demonstrated sensitivity to both clinical decline and treatment effect in MCI populations. However, until now, thresholds for clinical meaningfulness were not evaluated. By defining thresholds for clinically meaningful change on the ADCOMS tool, this work is an important step in allowing researchers to interpret clinical findings and estimates of treatment effects in early stage AD trials.
Based on the triangulation evaluating the range of anchor-based and distribution-based estimates and the scores for participants at the relevant timepoints, the following thresholds for minimal clinically meaningful changes on the ADCOMS have been proposed: MCI: 0.05; dementia: 0.10.
Despite the large sample size and use of well-established and published databases, there are several important limitations to our study. One is that we used retrospective databases that were designed for other purposes than instrument validation. Accordingly, we were limited to the measures that were already included in the databases. It might have been beneficial to have other anchors to use for evaluation of meaningful change or convergent and divergent validity. Although we did also evaluate several clinician global ratings and concluded that they were not useful for the current analysis, it might have been useful to have been able to compare the ADCOMS results to other cognitive assessments such as the Montreal Cognitive Assessment (MOCA) [22]. Furthermore, since the ADCOMS was derived from the measures we used to validate the scale and establish meaningful thresholds for change, the relationships between the underlying concepts might have been overestimated, thus leading to lower estimates for the thresholds of meaningful change. Future research on meaningful change using the ADCOMS should include additional anchors to ensure broader validity.
A final limitation is that the NACC dataset does not include the ADAS-Cog tool, so the ADCOMS had to be imputed as described above in the study methods. However, the model that was used to impute the ADCOMS in the NACC data fit the data from the ADNI and combined legacy databases very well.
Conclusion
This paper builds upon previous research which has established that the ADCOMS is a valid and reliable tool. Establishing thresholds for clinically meaningful change is a critical step in the psychometric validation process that ensures researchers are able to interpret clinical findings. This work on establishing thresholds for clinically meaningful change on the ADCOMS will enable researchers to more readily use the ADCOMS as an outcome measure for clinical trials in both MCI due to AD and dementia.
Supplementary Information
Below is the link to the electronic supplementary material.
Acknowledgements
ADNI had no additional role in study design and data analysis, decision to publish, or preparation of the manuscript.
Funding
Sponsorship for this study was funded by Eisai Inc. including the fee for the journal’s Rapid Service Fee. Source(s) of Support: ADNI: Data collection and sharing for this project were funded by the Alzheimer's Disease Neuroimaging Initiative (ADNI) (National Institutes of Health Grant U01 AG024904) and DOD ADNI (Department of Defense award number W81XWH-12–2-0012). ADNI is funded by the National Institute on Aging, the National Institute of Biomedical Imaging and Bioengineering, and through generous contributions from the following: AbbVie, Alzheimer’s Association; Alzheimer’s Drug Discovery Foundation; Araclon Biotech; BioClinica, Inc.; Biogen; Bristol-Myers Squibb Company; CereSpir, Inc.; Cogstate; Eisai Inc.; Elan Pharmaceuticals, Inc.; Eli Lilly and Company; EuroImmun; F. Hoffmann-La Roche Ltd and its affiliated company Genentech, Inc.; Fujirebio; GE Healthcare; IXICO Ltd.; Janssen Alzheimer Immunotherapy Research & Development, LLC.; Johnson & Johnson Pharmaceutical Research & Development LLC.; Lumosity; Lundbeck; Merck & Co., Inc.; Meso Scale Diagnostics, LLC.; NeuroRx Research; Neurotrack Technologies; Novartis Pharmaceuticals Corporation; Pfizer Inc.; Piramal Imaging; Servier; Takeda Pharmaceutical Company; and Transition Therapeutics. The Canadian Institutes of Health Research is providing funds to support ADNI clinical sites in Canada. Private sector contributions are facilitated by the Foundation for the National Institutes of Health (www.fnih.org). The grantee organization is the Northern California Institute for Research and Education, and the study is coordinated by the Alzheimer’s Therapeutic Research Institute at the University of Southern California. ADNI data are disseminated by the Laboratory for Neuro Imaging at the University of Southern California. Data used in preparation of this article were obtained from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database (adni.loni.usc.edu). As such, the investigators within the ADNI contributed to the design and implementation of ADNI and/or provided data but did not participate in analysis or writing of this report. A complete listing of ADNI investigators can be found at: http://adni.loni.usc.edu/wp-content/uploads/how_to_apply/ADNI_Acknowledgement_List.pdf. NACC: The NACC database is funded by NIA/NIH Grant U24 AG072122. NACC data are contributed by the NIA-funded ADCs: P50 AG005131 (PI James Brewer, MD, PhD), P50 AG005133 (PI Oscar Lopez, MD), P50 AG005134 (PI Bradley Hyman, MD, PhD), P50 AG005136 (PI Thomas Grabowski, MD), P50 AG005138 (PI Mary Sano, PhD), P50 AG005142 (PI Helena Chui, MD), P50 AG005146 (PI Marilyn Albert, PhD), P50 AG005681 (PI John Morris, MD), P30 AG008017 (PI Jeffrey Kaye, MD), P30 AG008051 (PI Thomas Wisniewski, MD), P50 AG008702 (PI Scott Small, MD), P30 AG010124 (PI John Trojanowski, MD, PhD), P30 AG010129 (PI Charles DeCarli, MD), P30 AG010133 (PI Andrew Saykin, PsyD), P30 AG010161 (PI David Bennett, MD), P30 AG012300 (PI Roger Rosenberg, MD), P30 AG013846 (PI Neil Kowall, MD), P30 AG013854 (PI Robert Vassar, PhD), P50 AG016573 (PI Frank LaFerla, PhD), P50 AG016574 (PI Ronald Petersen, MD, PhD), P30 AG019610 (PI Eric Reiman, MD), P50 AG023501 (PI Bruce Miller, MD), P50 AG025688 (PI Allan Levey, MD, PhD), P30 AG028383 (PI Linda Van Eldik, PhD), P50 AG033514 (PI Sanjay Asthana, MD, FRCP), P30 AG035982 (PI Russell Swerdlow, MD), P50 AG047266 (PI Todd Golde, MD, PhD), P50 AG047270 (PI Stephen Strittmatter, MD, PhD), P50 AG047366 (PI Victor Henderson, MD, MS), P30 AG049638 (PI Suzanne Craft, PhD), P30 AG053760 (PI Henry Paulson, MD, PhD), P30 AG066546 (PI Sudha Seshadri, MD), P20 AG068024 (PI Erik Roberson, MD, PhD), P20 AG068053 (PI Marwan Sabbagh, MD), P20 AG068077 (PI Gary Rosenberg, MD), P20 AG068082 (PI Angela Jefferson, PhD), P30 AG072958 (PI Heather Whitson, MD), P30 AG072959 (PI James Leverenz, MD).
Additional Assistance
Danielle Rodriguez, an employee of Evidera and paid consultant to Eisai, was helpful at the outset of the study in retrieving the data used in the analysis from ADNI and NACC and in developing the statistical analysis plan.
Authorship
All named authors meet the International Committee of Medical Journal Editors (ICMJE) criteria for authorship for this article, take responsibility for the integrity of the work as a whole, and have given their approval for this version to be published.
Author Contributions
Amir Abbas Tahami Monfared: Study concept and design, Interpretation of results, Drafting of manuscript. Quanwu Zhang: Interpretation of results, Drafting of manuscript. William R. Lenderking: Study concept and design, Development of statistical analysis plan, Interpretation of results, Drafting of Manuscript. Yulia Savva: Development of statistical analysis plan and Statistical analysis, Interpretation of results, Drafting of manuscript. Mary Kate Ladd: Interpretation of results, Drafting of manuscript.
Disclosures
Amir Abbas Tahami Monfared and Quanwu Zhang are current employees of Eisai. William R. Lenderking, Yulia Savva and Mary Kate Ladd are employees of Evidera and were paid consultants to Eisai for this work and in connection with the development of this manuscript.
Compliance with Ethical Guidelines
All procedures performed in these studies involving human participants were in accordance with the ethical standards of the institutional review board and with the 1964 Helsinki Declaration and its later amendments. Informed consent was obtained from all individual participants involved in the studies. Permissions were obtained to access and use the data from the ADNI and NACC databases.
Data Availability
The ADNI (http://adni.loni.usc.edu/) and NACC (https://naccdata.org/) datasets analyzed during the current study are available online. The clinical trial data analyzed during this study are available from the corresponding author on reasonable request.
Footnotes
The members of the Alzheimer’s Disease Neuroimaging Initiative are mentioned in the acknowledgements section.
Data used in preparation of this article were obtained from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database (adni.loni.usc.edu). As such, the investigators within the ADNI contributed to the design and implementation of ADNI and/or provided data but did not participate in analysis or writing of this report. A complete listing of ADNI investigators can be found at: http://adni.loni.usc.edu/wpcontent/uploads/how_to_apply/ADNI_Acknowledgement_List.pdf.
Contributor Information
Amir Abbas Tahami Monfared, Email: Amir_Tahami@eisai.com.
for the Alzheimer’s Disease Neuroimaging Initiative:
James Brewer, Oscar Lopez, Bradley Hyman, Thomas Grabowski, Mary Sano, Helena Chui, Marilyn Albert, John Morris, Jeffrey Kaye, Thomas Wisniewski, Scott Small, John Trojanowski, Charles DeCarli, Andrew Saykin, David Bennett, Roger Rosenberg, Neil Kowall, Robert Vassar, Frank LaFerla, Ronald Petersen, Eric Reiman, Bruce Miller, Allan Levey, Linda Eldik, Sanjay Asthana, Russell Swerdlow, Todd Golde, Stephen Strittmatter, Victor Henderson, Suzanne Craft, Henry Paulson, Sudha Seshadri, Erik Roberson, Marwan Sabbagh, Gary Rosenberg, Angela Jefferson, Heather Whitson, and James Leveren
References
- 1.Weller J, Budson A. Current understanding of Alzheimer's disease diagnosis and treatment. F1000Res. 2018;7:1161. doi: 10.12688/f1000research.14506.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Dubois B, Hampel H, Feldman HH, Scheltens P, Aisen P, Andrieu S, et al. Preclinical Alzheimer's disease: definition, natural history, and diagnostic criteria. Alzheimers Dement. 2016;12(3):292–323. doi: 10.1016/j.jalz.2016.02.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Petersen RC. Clinical practice. Mild cognitive impairment. N Engl J Med. 2011;364(23):2227–2234. doi: 10.1056/NEJMcp0910237. [DOI] [PubMed] [Google Scholar]
- 4.Albert MS, DeKosky ST, Dickson D, Dubois B, Feldman HH, Fox N, et al. The diagnosis of mild cognitive impairment due to Alzheimer's disease: recommendations from the National Institute on Aging-Alzheimer's Association workgroups on diagnostic guidelines for Alzheimer's disease. Alzheimers Dement. 2011;7(3):270–279. doi: 10.1016/j.jalz.2011.03.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Gauthier S, Reisberg B, Zaudig M, Petersen RC, Ritchie K, Broich K, et al. Mild cognitive impairment. Lancet. 2006;367(9518):1262–1270. doi: 10.1016/S0140-6736(06)68542-5. [DOI] [PubMed] [Google Scholar]
- 6.Jongsiriyanyong S, Limpawattana P. Mild cognitive impairment in clinical practice: a review article. Am J Alzheimers Dis Other Demen. 2018;33(8):500–507. doi: 10.1177/1533317518791401. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Petersen RC. Mild cognitive impairment. Continuum (Minneap Minn). 2016;22(2 Dementia):404–418. doi: 10.1212/CON.0000000000000313. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Food and Drug Administration. FDA’s Decision to Approve New Treatment for Alzheimer’s Disease. Available at: https://www.fda.gov/drugs/news-events-human-drugs/fdas-decision-approve-new-treatment-alzheimers-disease. Published 2021. Accessed 1 Nov 2021.
- 9.Gauthier S, Albert M, Fox N, Goedert M, Kivipelto M, Mestre-Ferrandiz J, et al. Why has therapy development for dementia failed in the last two decades? Alzheimers Dement. 2016;12(1):60–64. doi: 10.1016/j.jalz.2015.12.003. [DOI] [PubMed] [Google Scholar]
- 10.Food and Drug Administration . Guidance for industry—Alzheimer's disease: developing drugs for the treatment of early stage disease. Washington, DC: Food and Drug Administration; 2013. [Google Scholar]
- 11.Wang J, Logovinsky V, Hendrix SB, Stanworth SH, Perdomo C, Xu L, et al. ADCOMS: a composite clinical outcome for prodromal Alzheimer's disease trials. J Neurol Neurosurg Psychiatry. 2016;87(9):993–999. doi: 10.1136/jnnp-2015-312383. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Alzheimer's Disease Neuroimaging Initiative (ADNI). http://adni.loni.usc.edu. Published 2017. Accessed 1 Nov 2021.
- 13.Mohs RC, Knopman D, Petersen RC, Ferris SH, Ernesto C, Grundman M, et al. Development of cognitive instruments for use in clinical trials of antidementia drugs: additions to the Alzheimer's Disease Assessment Scale that broaden its scope. The Alzheimer's Disease Cooperative Study. Alzheimer Dis Assoc Disord. 1997;11(Suppl 2):S13–21. doi: 10.1097/00002093-199700112-00003. [DOI] [PubMed] [Google Scholar]
- 14.Folstein MF, Folstein SE, McHugh PR. "Mini-mental state". A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res. 1975;12(3):189–198. doi: 10.1016/0022-3956(75)90026-6. [DOI] [PubMed] [Google Scholar]
- 15.Hughes CP, Berg L, Danziger WL, Coben LA, Martin RL. A new clinical scale for the staging of dementia. Br J Psychiatry. 1982;140:566–572. doi: 10.1192/bjp.140.6.566. [DOI] [PubMed] [Google Scholar]
- 16.Vellas B, Bateman R, Blennow K, Frisoni G, Johnson K, Katz R, et al. Endpoints for pre-dementia AD trials: a report from the EU/US/CTAD Task Force. J Prev Alzheimers Dis. 2015;2(2):128–135. doi: 10.14283/jpad.2015.55. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Evans S, McRae-McKee K, Wong MM, Hadjichrysanthou C, De Wolf F, Anderson R. The importance of endpoint selection: How effective does a drug need to be for success in a clinical trial of a possible Alzheimer's disease treatment? Eur J Epidemiol. 2018;33(7):635–644. doi: 10.1007/s10654-018-0381-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Hendrix SB, Soininen H, van Hees AMJ, Ellison N, Visser PJ, Solomon A, et al. Alzheimer's Disease Composite Score: a post-hoc analysis using data from the LipiDiDiet trial in prodromal Alzheimer's Disease. J Prev Alzheimers Dis. 2019;6(4):232–236. doi: 10.14283/jpad.2019.33. [DOI] [PubMed] [Google Scholar]
- 19.Monsell SE, Kukull WA, Roher AE, Maarouf CL, Serrano G, Beach TG, et al. Characterizing apolipoprotein E epsilon4 carriers and noncarriers with the clinical diagnosis of mild to moderate Alzheimer Dementia and minimal beta-amyloid peptide plaques. JAMA Neurol. 2015;72(10):1124–1131. doi: 10.1001/jamaneurol.2015.1721. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Andrews JS, Desai U, Kirson NY, Zichlin ML, Ball DE, Matthews BR. Disease severity and minimal clinically important differences in clinical outcome assessments for Alzheimer's disease clinical trials. Alzheimers Dement (N Y) 2019;5:354–363. doi: 10.1016/j.trci.2019.06.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Food and Drug Administration. Patient-focused drug development guidance public workshop: methods to identify what is important to patients & select, develop or modify fit-for-purpose clinical outcomes assessments. October 15–16, 2018.
- 22.Nasreddine ZS, Phillips NA, Bedirian V, Charbonneau S, Whitehead V, Collin I, et al. The Montreal Cognitive Assessment, MoCA: a brief screening tool for mild cognitive impairment. J Am Geriatr Soc. 2005;53(4):695–699. doi: 10.1111/j.1532-5415.2005.53221.x. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Availability Statement
The ADNI (http://adni.loni.usc.edu/) and NACC (https://naccdata.org/) datasets analyzed during the current study are available online. The clinical trial data analyzed during this study are available from the corresponding author on reasonable request.