Abstract
Introduction
The Alzheimer's Disease Cooperative Study - Clinical Global Impression of Change (ADCS-CGIC) was modified for use in mild cognitive impairment (MCI) trials and tested in the ADCS MCI randomized clinical trial of donepezil, vitamin E and placebo. We assessed feasibility for its use by determining whether or not: (1) it distinguished a medication effect at 6- and 12- months, (2) baseline demographic or clinical characteristics predicted change, (3) there was an association between MCI-CGIC and change in other clinical measures in order to evaluate external or concurrent validity.
Methods
We used a generalized estimating equations approach for ordinal outcome data to test the effects of treatment, baseline characteristics and change in clinical measures on the MCI-CGIC over 12 months, and ordinal logistic regression to assess the association between MCI-CGIC and change in clinical measures at 6 months and 12 months.
Results
On the MCI-CGIC overall, 12.9% and 10.6% were rated as having improved, and 31.6% and 39.8% as having worsened over 6- and 12-months, respectively. The MCI-CGIC did not distinguish the donepezil or vitamin E groups from placebo at 6 and 12 months treatment. Variables at screening or baseline that were associated with worse CGIC scores over 6 and 12 months included white race, greater years of education, worse depression, dementia severity rating, cognitive, and daily activities scores, and lower memory domain scores on a neuropsychological battery. Rate of worsening on the MCI-CGIC over 12 months was associated with change on the AD Assessment Scale-cognitive (ADAS-cog) and on executive function. Worsening at 6 months and 12 months, separately, were associated with the corresponding change in ADAS-cog, ADL, BDI, MMSE, CDR-sb, memory, and executive function.
Conclusions
Change detected by the MCI-CGIC was associated with baseline clinical severity and with change in clinical ratings over 6 and 12 months, supporting the validity of a CGIC approach in MCI. The effect size of the donepezil-placebo difference was similar to that of other outcomes at 12 months. About 40% of MCI patients were judged worse and about 11% improved, consistent with clinical experience and other ratings.
Keywords: Mild cognitive impairment, Alzheimer's disease, dementia, global impression of change, rating scales, donepezil, vitamin E, clinical trials
INTRODUCTION
Clinical Global Impression of Change (CGIC) ratings are commonly used as tests of clinical meaningfulness in Alzheimer's disease (AD) trials1. Overall global change or severity scales ratings, or alternatively, activities of daily living scales, are required co-primary outcomes in regulatory or registration trials for drugs for dementia2. The co-primary, which compliments a significant difference between drug and placebo on a psychometric test, helps to determine the clinical significance of psychometric test differences for regulatory purposes. The Alzheimer's Disease Cooperative Study-Clinician's Global Impression of Change rating (ADCS-CGIC) has been used commonly in 6 month long clinical trials3. It is the most frequently used example of a Clinician's Interview-Based Impression of Change with caregiver's input (i.e., a “CIBIC+”) requiring a direct interview with the patient without reference to other clinical information or test scores but with input from a caregiver2.
The Clinical Dementia Rating (CDR)4 has been used in longer-term trials as a co-primary outcome. It is a severity scale that assesses the current state of a patient without reference to a prior state and does not rate change over time. By comparison a CGIC requires a clinician's specific judgment as to whether or not a patient has changed since the beginning of a trial. A rationale for the use of severity scales in longer-term trials is that clinicians may not be able to assess this change because they may not remember or be able to recreate from notes or records the patient's baseline state and medical records at baseline may not be sufficient.
A modified version of the ADCS-CGIC was used in the ADCS-sponsored Memory Impairment Study (MIS) of patients with mild cognitive impairment (MCI)5. The ADCS MCI-CGIC was designed to provide a means to assess global change in an MCI clinical trial by providing a semi-structured format to allow clinicians to gather necessary clinical information from both the subject and informant in order to allow for an overall impression of clinical change. It differs from the ADCS-CGIC used in AD trials mainly in its shorter length and its use of probes tailored for aspects of cognition, daily function, and behavior most relevant to MCI subjects.
The goals of this investigation were to assess the feasibility of the use of this modified ADCS-CGIC in MCI clinical trials by assessing whether or not the CGIC distinguished a medication effect, whether or not baseline demographic or clinical characteristics predicted change, and whether or not there was an association between MCI-CGIC change and change in other clinical measures, to assess external or concurrent validity.
METHODS
The Memory Impairment Study (MIS)
The MCI-CGIC was used in the MIS to assess 6 month and 12 month outcomes. Briefly 790 patients with MCI were randomized to donepezil, vitamin E, or placebo, and 769 completed baseline assessments and were followed for up to three years5. The primary outcome of the trial was time to the development of possible or probable AD.
Secondary measures included the Mini-Mental State Examination (MMSE)6, the Alzheimer's Disease Assessment Scale cognitive subscale (ADAS-cog)7,8, a neuropsychological test battery, the CDR sum of boxes (CDR-sb), the ADCS MCI Activities of Daily Living Inventory (ADCS MCI-ADL)9,10; the Global Deterioration Scale (GDS)11, the Cleveland AD Quality of Life scale (CADQoL)12, Beck Depression Inventory (BDI), and the MCI-CGIC. The neuropsychological test battery (NTB) included the New York University (NYU) paragraph-recall test, Symbol Digit Modalities Test, category-fluency test, a number-cancellation test, Boston Naming Test, digits-backward test, clock-drawing test, and a maze-tracing task5.
Cognitive-domain scores for memory (the ADAS-cog immediate and delayed word-recall scores and the NYU immediate and delayed paragraph-recall scores), executive function (the digits-backward test, Symbol Digit Modalities Test, and number-cancellation test), language (the Boston Naming Test and category-fluency test), and visuospatial skills (the clock-drawing test) were calculated in addition to an overall composite cognitive-function score. These domain and composite scores were calculated as the weighted sum of the individual standardized test scores (standardized by dividing each score by the standard deviation of the baseline scores). Weights were calculated as the reciprocal of the sum of the correlation coefficients between the tests in each domain at baseline. (Maze tracing was not used in the domain score calculations).
The measures were administered at 6-month intervals over three years except that the ADAS-cog and NTB were administered at 3 months as well; and the CADQoL was administered at 3 months and then at 12-month intervals from baseline. The MCI-CGIC was administered at 6 and 12 months only because of concern that it might be unreliable over longer intervals. Results of the MIS outcomes on the clinical ratings scales at months 6 and 12 were non-significant on the ADCS-ADLs and the ADAS-cog. The study was conducted according to Good Clinical Practice guidelines, the Declaration of Helsinki, and the US Code of Federal Regulations title 21 Part 50 (Protection of Human Subjects) and title 21 Part 56 (Institutional Review Boards). Written informed consent was obtained from all participants and study partners.
There were significant effects in favor of donepezil compared to placebo on the MMSE at 6 and 12 months, on the CDR-sb at 12 months, but not at 6 months, the GDS at 6 months but not 12 months, a modified version of the ADAS at 6 and 12 months, and the overall NTB score at 6 and 12 months. Significant effects for vitamin E were observed on the NTB overall score at 6 months (none of these corrected for multiple comparisons)5.
The MCI-CGIC
The MCI-CGIC consists of worksheets used to record baseline information to serve as a reference for future ratings; worksheets designed to assist the clinician in gathering information at follow up visits to assess change from baseline; and a CGIC form for the clinician to rate his/her impression of change from baseline. The CGIC rating is made on a 7-point Likert-type scale where change from baseline is rated as marked improvement (1), moderate improvement (2), minimal improvement (3), no change (4), minimal worsening (5), moderate worsening (6), marked worsening (7).
At baseline, the clinician interviews the subject and informant about baseline status for later reference. At baseline only, clinical information about the subject may be used, including medical history, physical and neurological examination, and other ratings done at screening. A column headed “Area” on the forms identifies various areas that a clinician might consider while evaluating a subject for potential clinical change. A “Probes” column provides sample items that a clinician might find useful in assessing an area, and are intended only as guides for collecting relevant information. A third column provides space for notes. There are separate spaces for notes taken from the subject and informant interviews.
At interval assessments, the subject is interviewed first, followed by the informant. After completing the worksheets, the clinician records his/her clinical impression of change from baseline on the 7-item scale, referring only to his/her own baseline information when making the assessment, without consulting other study personnel and blinded to possible medication or adverse effects.
Hypotheses
We assessed three main hypotheses regarding the MCI-CGIC:
There is no association between treatment with donepezil, vitamin E or placebo and rate of change in the MCI-CGIC;
there is no association between rate of change in MCI-CGIC over 12 months and patient demographic characteristics and baseline scores on the secondary outcomes of the trial, including age, gender, ethnicity, ApoE4 genotype, ADAS-cog, MMSE, ADCS-ADL inventory, BDI, CADQoL, CDR sum of the boxes, and the four domains of the memory, executive function, language, or visuospatial domains of the neuropsychological battery;
There is no association between rate of change in MCI-CGIC over 12 months and corresponding change in secondary outcomes listed previously.
Statistical Analysis
To assess the rate of change in MCI-CGIC over 12 months and its association with each of the covariates of interest, the longitudinal analysis was done using a generalized estimating equations (GEE) approach, which accounts for within-subject correlation. Because very few change scores were at the extreme ratings of marked worsening, moderate worsening, marked improvement, or moderate improvement, we evaluated the rate of change of MCI-CGIC using two models. For the first model (GEE model 1), the response variable was the MCI-CGIC score at 6 and 12 months treated as an ordinal variable with 3 levels (i.e., improved, no change, or worse). The second model (GEE model 2) treated the MCI-CGIC as a dichotomous dependent variable (no change/improved and worse).
The GEE method is suitable for the longitudinal analysis of both binary and ordinal outcomes. These models estimate odds ratios that indicate the relationship between the response variable and the covariates. GEE models for binary data (assuming a logistic function) and ordinal data (using a proportional odds model) were used for the two models. The proportional odds model is very similar to the GEE model for binary data, with the difference being that a covariate effect leads to an increase in the likelihood of the patient being in any subsequent higher MCI-CGIC category. The interaction effect in each of the models (β6) describes the log-odds of moving from any cumulatively lower (improved) category into a cumulatively higher (worse) category.
To determine if the change in MCI-CGIC over 12 months differed between patients randomized to donepezil, vitamin E, or placebo, a GEE model for ordinal data was fit to the data to account for the clustering due to the repeated observations within a patient. The dependent variable in the model was the MCI-CGIC scores over time (6 months, 12 months) and the independent variables included a treatment factor (placebo, vitamin E and donepezil), a time factor (6 and 12 months), and the treatment by time interaction. Hochberg's method was used to adjust for multiple comparisons13 if the overall F test for the interaction effect was significant.
To determine whether baseline measures predict change in the MCI-CGIC over a year, an ordinal GEE model was fit. The dependent variable in the model was the MCI-CGIC scores over 12 months and the independent variables included change in secondary measures (as a time varying covariate), a time factor (6 and 12 months), and the secondary measure by time interaction.
The third analysis examined the validity of the MCI-CGIC by comparing it to the change in each of the secondary measures. To determine this, a GEE model was fit to evaluate if the rate of change in clinical predictors were associated with the rate of change in MCI-CGIC.
In addition to GEE model 1 and GEE model 2 as described above, we also fit an ordinal logistic model to determine the association of MCI-CGIC and change in secondary measures at 6 months (logistic 6-month) and 12 months (logistic 12-month LOCF) separately. Missing data at 12 months was imputed using last observation carried forward (LOCF).
All models were adjusted for three pre-specified covariates, age, ApoE4 status and screening MMSE, similar to the model used in the primary MCI report5 along with any additional observed confounders. No adjustments for multiple comparisons were made given the exploratory nature of the hypotheses. Possible co-linearity between baseline MMSE and other baseline predictors were assessed using Spearman correlation coefficients.
RESULTS
Of the 769 patients randomized to treatment, there were 651 MCI-CGIC ratings available at 6 months and 593 at 12 months.
Association between treatment and change on the MCI-CGIC
The distributions of MCI-CGIC change scores overall and by treatment at 6 and 12 months are displayed in Table 1. Overall the mean MCI-CGIC scores at 6 months were 4.10, 4.25, and 4.21, and at 12 months they were 4.25, 4.39 and 4.38 for donepezil, vitamin E and placebo, respectively.
Table 1.
MCI-CGIC ratings at 6 and 12 months by treatment assignment
| 6 Months | 12 Months | |||||
|---|---|---|---|---|---|---|
| CGIC | Donepezil, n (%) | Vitamin E, n (%) | Placebo, n (%) | Donepezil, n (%) | Vitamin E, n (%) | Placebo, n (%) |
| 1 | 2 (1.0) | 1 (0.4) | 2 (0.9) | 3 (1.7) | 1 (0.5) | 0 (0.0) |
| 2 | 7 (3.5) | 1 (0.4) | 1 (0.4) | 0 (0.0) | 3 (1.5) | 3 (1.5) |
| 3 | 29 (14.6) | 18 (8.2) | 23 (9.9) | 21 (11.6) | 12 (5.8) | 20 (9.7) |
| 4 | 99 (49.8) | 127 (57.7) | 135 (58.2) | 91 (50.3) | 104 (50.5) | 99 (48.1) |
| 5 | 55 (27.6) | 67 (30.4) | 63 (27.2) | 57 (31.5) | 70 (34.0) | 65 (31.6) |
| 6 | 7 (3.5) | 6 (2.7) | 7 (3.0) | 9 (4.97) | 16 (7.8) | 18 (8.7) |
| 7 | 0 (0.0) | 0 (0.0) | 1 (0.4) | 0 (0.0) | 0 (0.0) | 1 (0.5) |
| Mean (SD) | 4.10 (0.89) | 4.25 (0.70) | 4.21 (0.76) | 4.25 (0.85) | 4.39 (0.81) | 4.38 (0.85) |
Footnotes: There was no statistically significant effect between treatment groups.
There was no significant association between treatment and rate of change on the MCI-CGIC over 12 months (p=0.99, GEE Model 1 and GEE Model 2). Results were similar whether the 3-category or 2-category model was used (Table 2, Figure 1).
Table 2.
Analysis summary: (1) association between rate of change in the MCI-CGIC and donepezil or vitamin E treatment compared to placebo and; (2) association between change in MCI-CGIC over 12 months and patient demographic and baseline clinical characteristics. (Results are reported as odds ratios (95% confidence intervals) and with the corresponding Wald test p value)
| Model | Estimate | LCL | UCL | P | |
|---|---|---|---|---|---|
| Hypothesis 1, drug v. placebo | |||||
| Donepezil | GEE 11 | 1.09 | 0.66 | 1.81 | 0.942 |
| Vitamin E | GEE 1 | 1.02 | 0.73 | 1.42 | 0.942 |
| Donepezil | GEE 22 | 0.85 | 0.52 | 1.38 | 0.744 |
| Vitamin E | GEE 2 | 0.93 | 0.59 | 1.46 | 0.744 |
| Hypothesis 2 predictors | |||||
| Ethnicity | GEE 1 | 2.30 | 1.30 | 4.05 | 0.004 |
| GEE 2 | 2.48 | 1.21 | 5.10 | 0.013 | |
| Education | GEE 1 | 1.04 | 0.98 | 1.10 | 0.19 |
| GEE 2 | 1.08 | 1.03 | 1.14 | 0.002 | |
| Baseline ADL | GEE 1 | 0.93 | 0.89 | 0.97 | 0.001 |
| GEE 2 | 0.95 | 0.92 | 0.98 | 0.004 | |
| Screening ADASc | GEE 1 | 1.07 | 1.04 | 1.11 | 0.000 |
| GEE 2 | 1.08 | 1.04 | 1.12 | 0.000 | |
| Screening MMSE | GEE 1 | 0.93 | 0.84 | 1.03 | 0.16 |
| GEE 2 | 0.87 | 0.80 | 0.95 | 0.002 | |
| Baseline BDI | GEE 1 | 1.03 | 0.99 | 1.07 | 0.12 |
| GEE 2 | 1.05 | 1.02 | 1.08 | 0.002 | |
| Screening CDR Sum of Boxes | GEE 1 | 1.04 | 0.82 | 1.31 | 0.76 |
| GEE 2 | 1.36 | 1.11 | 1.67 | 0.004 | |
| Baseline Memory | GEE 1 | 0.67 | 0.51 | 0.87 | 0.003 |
| GEE 2 | 0.46 | 0.36 | 0.60 | 0.000 |
Results from the 3 Category Proportional Odds GEE Model to examine the effect of the covariate on the CGIC based on the analysis of change patterns over 12 months adjusted for the other covariates in the model. The model considers a three category CGIC as the response (see “Methods” section) and adjusted for age, APOE4 and baseline MMSE. Here, the association is measured by an adjusted odds ratio (aOR) with the corresponding lower (LCL) and upper (UCL) 95% confidence limit. An aOR > 1.00 indicates a greater odds of having higher CGIC scores (worsening) than lower scores and and an aOR< 1.00 indicates a greater odds of having lower CGIC (improvement) than higher scores.
Results from the 2 Category Proportional Odds GEE Model to examine the effect of the covariate on the CGIC based on the analysis of change patterns over 12 months adjusted for the other covariates in the model. This model considers a two category CGIC as the response (see “Methods” section) and adjusted for age, APOE4 and baseline MMSE.
Figure 1.
MCI-CGIC ratings at 6 months (A) and 12 months (B)
Because there might be differences in CGIC change based on baseline cognitive severity, the sample was split at the median ADAScog point into a higher scoring and lower scoring group. Although treatment was not associated with MCI-CGIC over time, there was a qualitative change in the estimates depending on screening ADAScog group (≥10.33 vs. <10.33). Donepezil-treated patients who were more cognitively impaired at baseline (aORdonepezil=1.70, p=0.34) were relatively more likely to have higher (worse) 12-month CGIC scores than those who were less impaired at baseline (aORdonepezil=0.65, p=0.30) when compared to placebo treatment. A similar pattern was observed for the vitamin E group (aORvitamin E=1.30, p=0.52, for the high group; aORvitamin E=0.70, p=0.30, for the low group, P-values Hochberg corrected).
Baseline Predictors of MCI-CGIC change (Table 2)
Several baseline variables predicted MCI-CGIC change over time. In both models (GEE Model 1 and GEE Model 2), being Caucasian (aOR=2.30; 95% CI=1.30, 4.05; p=0.004), having lower (worse) baseline ADL (aOR=0.93; 95% CI=0.89, 0.97; p=0.001), higher (worse) baseline ADAScog score (aOR=1.07; 95% CI=1.04, 1.11; p<0.001) and lower (worse) baseline memory domain score (aOR=0.67; 95% CI=0.51, 0.87; p=0.003) were predictors of worse ratings on the MCI-CGIC. In addition, more education, a lower screening MMSE score, a higher baseline BDI score, and a higher screening CDR-sb score were associated with worse ratings on the MCI-CGIC in GEE Model 2. Age, presence of ApoE4 genotype, gender, executive domain at baseline, language domain at baseline, visuospatial domain at baseline and CADQoL at screening or baseline were not associated with MCI-CGIC over time.
Association between the MCI-CGIC and change in secondary outcomes (Table 3)
Table 3.
Analysis summary: association between rate of change in MCI-CGIC over 12 months and changes in clinical outcomes. (Results are reported as odds ratios (95% confidence intervals) with the corresponding Wald test p value)
| Scale | Model | OR | LCL | UCL | P |
|---|---|---|---|---|---|
| ADAS-cog | GEE 11 | 1.09 | 1.03 | 1.15 | 0.005 |
| GEE 22 | 1.06 | 1.01 | 1.12 | 0.027 | |
| Logistic 6-month3 | 1.04 | 1.00 | 1.09 | 0.030 | |
| Logistic 12-month LOCF4 | 1.12 | 1.08 | 1.17 | 0.000 | |
| ADL | GEE 11 | 0.98 | 0.91 | 1.05 | 0.550 |
| GEE 22 | 0.94 | 0.88 | 0.99 | 0.023 | |
| Logistic 6-month3 | 0.96 | 0.92 | 0.99 | 0.020 | |
| Logistic 12-month LOCF4 | 0.91 | 0.88 | 0.94 | 0.000 | |
| BDI | GEE 11 | 0.99 | 0.94 | 1.04 | 0.685 |
| GEE 22 | 0.99 | 0.95 | 1.04 | 0.726 | |
| Logistic 6-month3 | 1.08 | 1.05 | 1.12 | 0.000 | |
| Logistic 12-month LOCF4 | 1.07 | 1.04 | 1.11 | 0.000 | |
| MMSE | GEE 11 | 1.00 | 0.89 | 1.13 | 0.992 |
| GEE 22 | 0.93 | 0.84 | 1.02 | 0.137 | |
| Logistic 6-month3 | 0.84 | 0.79 | 0.91 | 0.000 | |
| Logistic 12-month LOCF4 | 0.78 | 0.72 | 0.84 | 0.000 | |
| CADQoL5 | GEE 11 | -- | -- | -- | -- |
| GEE 22 | -- | -- | -- | -- | |
| Logistic 6-month3 | -- | -- | -- | -- | |
| Logistic 12-month LOCF4 | 0.99 | 0.96 | 1.03 | 0.700 | |
| CDR | GEE 11 | 1.03 | 0.75 | 1.44 | 0.839 |
| GEE 22 | 0.95 | 0.72 | 1.24 | 0.686 | |
| Logistic 6-month3 | 1.96 | 1.59 | 2.42 | 0.000 | |
| Logistic 12-month LOCF4 | 1.85 | 1.59 | 2.15 | 0.000 | |
| Memory | GEE 11 | 0.79 | 0.49 | 1.26 | 0.314 |
| GEE 22 | 0.78 | 0.51 | 1.18 | 0.238 | |
| Logistic 6-month3 | 0.64 | 0.47 | 0.86 | 0.003 | |
| Logistic 12-month LOCF4 | 0.56 | 0.43 | 0.74 | 0.000 | |
| Executive Function | GEE 11 | 0.79 | 0.48 | 1.29 | 0.337 |
| GEE 22 | 0.55 | 0.32 | 0.95 | 0.032 | |
| Logistic 6-month3 | 0.54 | 0.37 | 0.78 | 0.001 | |
| Logistic 12-month LOCF4 | 0.41 | 0.29 | 0.59 | 0.000 | |
| Language | GEE 11 | 0.37 | 0.16 | 0.87 | 0.023 |
| GEE 22 | 0.60 | 0.25 | 1.42 | 0.241 | |
| Logistic 6-month3 | 0.70 | 0.38 | 1.30 | 0.260 | |
| Logistic 12-month LOCF4 | 0.35 | 0.19 | 0.64 | 0.001 | |
| Visuospatial | GEE 11 | 0.80 | 0.41 | 1.56 | 0.520 |
| GEE 22 | 1.02 | 0.59 | 1.77 | 0.946 | |
| Logistic 6-month3 | 0.63 | 0.40 | 0.97 | 0.040 | |
| Logistic 12-month LOCF4 | 0.72 | 0.47 | 1.12 | 0.140 |
Results from the 3 Category Proportional Odds GEE Model to examine the effect of the covariate on the CGIC based on the analysis of change patterns over 12 months adjusted for the other covariates in the model. This model considers a 3-category CGIC as the response (see “Methods” section) and adjusted for age, APOE4 and baseline MMSE. Here, the association is measured by an adjusted odds ratio (aOR) with the corresponding lower (LCL) and upper (UCL) 95% confidence limit. An aOR > 1.00 indicates a greater odds of having higher CGIC scores (worsening) than lower scores and an aOR < 1.00 indicates a greater odds of having lower CGIC (improvement) than higher scores.
Results from a 2 Category Proportional Odds GEE Model to examine the effect of the covariate on the CGIC based on the analysis of change patterns over 12 months adjusted for the other covariates in the model. This model considers a two category CGIC as the response (see “Methods” section) and adjusted for age, APOE4 and baseline MMSE.
Results from a 3 Category Proportional Odds Ordinal logistic model to assess the effect of covariates on CGIC at 6 months. This model considers a 3-category CGIC as the response (see Methods section) and the model is adjusted for age, APOE4 and baseline MMSE. Here, the association is measured by an adjusted odds ratio (aOR) with the corresponding lower (LCL) and upper (UCL) 95% confidence limit. An aOR > 1.00 indicates a greater odds of having higher CGIC scores (worsening) than lower scores and an aOR < 1.00 indicates a greater odds of having lower CGIC (improvement) than higher scores.
Results from a 3 Category Proportional Odds Ordinal logistic model to assess the effect of covariates on CGIC at 12 months. This model considers a three category CGIC as the response (see “Methods” section) and the model is adjusted for age, APOE4 and baseline MMSE. Imputation is done using the last observation carried forward (LOCF). Here, the association is measured by an adjusted odds ratio (aOR) with the corresponding lower (LCL) and upper (UCL) 95% confidence limit. An aOR > 1.00 indicates a greater odds of having higher CGIC scores (worsening) than lower scores and an aOR < 1.00 indicates a greater odds of having lower CGIC (improvement) than higher scores.
The CADQoL was administered at 3 months and then at 12-month intervals from baseline.
ADAS-cog
A positive change (worsening) in ADAScog was associated with an increase in the odds of a higher (worsening) score on the CGIC in GEE Model 1 (aOR=1.09; 95% CI=1.03, 1.15, p<0.01) and GEE Model 2 (aOR=1.06; 95% CI=1.01, 1.12, p=0.03). A positive change in ADAScog was also associated with an increase in the odds of a higher score on the CGIC at both 6 months (aOR = 1.04, p = 0.03) and 12 months (aOR = 1.12, p < 0.01), separately. The coefficients for the LOCF analysis had the same directions as those of the coefficients in the above analysis. The odds ratios were mostly similar and did not significantly change the conclusions of the above analysis.
Mini-Mental State Examination
Change in MMSE was not associated with change in the MCI-CGIC over a year. The coefficients for the LOCF analysis had the same directions as those of the coefficients in the above analysis, with similar odds ratios.
NTB domains
Change in memory domain score (i.e., ADAScog immediate and delayed word-recall, NYU immediate and delayed paragraph-recall) and visuospatial domain (clock-drawing test) were not associated with the rate of change of CGIC in one year.
Change in executive function domain score (i.e., digits-backward, Symbol Digit Modalities, number-cancellation) was not associated with the CGIC in GEE Model 1 (aOR=0.79; 95% CI=0.48, 1.29, p=0.34), but was associated with a decrease in the odds of a higher CGIC score in GEE Model 2 (aOR=0.55; 95% CI=0.32, 0.95, p=0.03).
Language domain score (Boston Naming, category fluency) was associated with a decrease in the odds of a higher (worsening) score on the CGIC in GEE Model 1 (aOR=0.37; 95% CI=0.16, 0.87, p=0.02), but was not associated in GEE Model 2 (aOR=0.60; 95% CI=0.25, 1.42, p=0.24). The coefficients for the LOCF analysis had the same directions as those of the coefficients in the above analysis, with similar odds ratios.
CDR Sum of the Boxes
Change in CDR-sb was not associated with change in the CGIC over time. By ordinal logistic regression of the 3-category model at 6 and 12 months, adjusting for age, ApoE4 genotype, and screening MMSE, an increase in CDR-sb score (worsening) was associated with an increase in the odds of a higher score on the CGIC at both 6 months (aOR = 1.96, p < 0.01) and at 12 months (aOR = 1.85, p < 0.01), separately.
Activities of daily living
Positive change in the ADCS-ADL (improvement) was associated with an increase in the odds of a lower score on the CGIC in GEE Model 2 (p = 0.02), but was not significantly associated with the CGIC in GEE Model 1 (p = 0.55). However, the coefficients for these analyses and the LOCF analysis had the same directions as those of the coefficients in the above analysis and with similar odds ratios.
Beck Depression Inventory
Change in BDI was not associated with change in the MCI-CGIC over time. The coefficients for the LOCF analysis were similar to those of the coefficients in the above analysis.
Quality of Life
Change in QOL total score was not associated with the MCI-CGIC at 12 months (aOR = 0.99, 95% CI=0.96, 1.03, p = 0.70).
DISCUSSION AND CONCLUSIONS
The difference in magnitude of the MCI-CGIC between donepezil or vitamin E treatment compared to placebo was similar to other secondary clinical outcomes - both significant and not - in a trial in which the medications did not show overall advantages compared to placebo. Based on the CGIC ratings, there was modest worsening in MCI patients over 6 and 12 months. At 12 months approximately 11% were rated improved, 50% not changed, and 40% worsened. The mean donepezil-placebo difference in CGIC of -0.13 standard deviation units (Cohen's d) at 6 months and -0.15 at 12 months was half the mean treatment-placebo differences reported in trials of cholinesterase inhibitors with mild to moderate AD patients, was similar in magnitude to that of other outcomes ratings in the MIS trial5 (Table 4), and of the same magnitude as in another MCI trial19. Nevertheless, the lack of significant outcomes compared to placebo limits the ability to compare these effect sizes.
Table 4.
Treatment effect sizes at 6 and 12 months: Treatment vs. Placebo
| Cohen's d | |||
|---|---|---|---|
| Outcome | Treatment | 6 months | 12 months |
| CGIC | Donepezil | -0.13 | -0.15 |
| Vitamin E | 0.05 | 0.01 | |
| ADAS-cog | Donepezil | -0.13 | -0.11 |
| Vitamin E | -0.01 | 0.07 | |
| MMSE | Donepezil | 0.21 | 0.21 |
| Vitamin E | -0.08 | 0.11 | |
| ADL | Donepezil | 0.21 | 0.01 |
| Vitamin E | 0.16 | 0.07 | |
| CDR-sb | Donepezil | -0.12 | -0.13 |
| Vitamin E | 0.04 | 0.09 | |
Note: negative values on CGIC, ADAS-cog, and CDR-sb indicate differences in favor of drug
Screening or baseline characteristics of patients, white ethnicity, higher education, and generally worse functioning on cognitive, ADLs, and depression measures, predicted worse CGIC at 6 and 12 months These predictors of worse CGIC have been identified previously as predictors of conversion to AD or more rapid cognitive decline 14,15,16. In effect, the closer a participant was to AD the greater the likelihood for a change rating. The MCI-CGIC also predicted changes in clinical ratings scales at 6 and 12 months, thus providing evidence for its external or concurrent validity. In particular, increases in CDR-sb scores that reflect clinical progression of disease severity without reference to baseline functioning were associated with increasing CGIC ratings, providing additional evidence for the validity of this CGIC rating approach in MCI.
The CGIC is measured in ordinal categories and can be considered ordered from improved to no change to worse. Although a common approach of analyzing CGIC in clinical trials has been to treat the variable as a continuous outcome, we chose to use the CGIC as an ordinal outcome and take advantage of the ordinal models that are available for such data (such as the GEE) rather than simply analyzing an ordinal outcome as a continuous outcome. One advantage is that the logistic link function specification takes into account the ceiling and floor effects of the dependent variable, the CGIC, while the linear models do not17. This is particularly important when the dependent variable is skewed, or when different covariate groups are compared which have widely varying skewness of the dependent variables18.
We used several different regression models to evaluate the effect of the covariates on the CGIC at the 6 and 12 months assessments. The GEE modeling framework was used to assess the impact of treatment (hypothesis 1), baseline covariates (hypothesis 2) and change in clinical measures (hypothesis 3) on the CGIC at 6 and 12 months based on the analyses of change patterns over these time intervals. Fixed-effects logistic regression models were used to test the significance of the change in clinical measures at a particular time point (6 or 12 months) with the CGIC at that time. The results from the two modeling approaches had clinically similar effects, although the statistical significance differed. There are two major reasons for this: the generalization of the fixed-effects logistic model (binary or ordinal) for longitudinal data is the mixed-effects logistic regression model (binary or ordinal) and not a marginal model (GEE); and the marginal model and the mixed-effects regression model are different in the scale and interpretation of the regression coefficients. We decided to use the GEE model because the scientific interest in this study was the estimation and inference of the regression parameters and not of the variance-covariance structure of the longitudinal data. In addition, the time points in the study were fixed with little variability in terms of the timing of the measurements over time. The ease of interpretation of the odds ratio and the robustness of the model to variance-covariance misclassification makes the GEE an appropriate model for these particular analyses.
Moreover, the scientific questions that the two models answer are different. The GEE approach models the longitudinal experience of the study patients and provides information on change over a period of time, while the fixed-effects model (which models cross-sectional data) does not. Additionally, for hypothesis 3, not only does the CGIC change over time, but the values of the predictors (clinical variables) also change over time. The GEE analyses describes the dynamic relationship between the two variables in time, while the fixed-effects LR analyses describes the association between the two variables at one point in time (i.e., 6 or 12 months).
In a 6-month, randomized, placebo-controlled trial donepezil-treated MCI patients had better scores on the ADCS CGIC-MCI than did placebo patients but the differences were not significant at endpoint19. Although mean difference data was not provided, at six-months 32.6% of donepezil patients compared to 24.3% of placebo patients showed at least minimal improvement, and 51.7% vs. 60.4%, respectively, showed no change. By comparison, in the present trial at 6 months, 19.1% vs. 11.2% showed at least minimal improvement and 49.8% vs. 58.2% showed no change (Table 1, Figure 1) yielding virtually identical rate differences. The authors of the six month trial suggest that the lack of treatment differences might be due to the impact of MCI primarily on a single domain (memory) within the CGIC. Further, they suggest that CGIC ratings are influenced by minimal impairments in other areas at baseline along with minimal changes in the primary memory impairments associated with MCI over six months. The results of the present analyses, using a larger sample size, are similar to the lack of donepezil-placebo statistically significant differences on the CGIC-MCI reported19 but suggest further that there are various influences on a CGIC rating in MCI in addition to memory20. Thus, this study provides the first systematic evidence that CGIC ratings, at least in MCI patients, are based on more than assessments of memory.
Yet, these analyses also suggest that the CGIC, along with the other measures above, may be of more limited sensitivity and use for the more mildly cognitively-impaired MCI patients in clinical trials unless the memory domains are expanded. For the MCI amnestic subtype classification, an expanded memory domain would be appropriate. On the other hand, the lack of change in the MCI-CGIC and other outcomes may be reflecting that donepezil and vitamin E do not work in MCI patients over 6 to 12 month treatment durations.
Acknowledgments
Supported by NIH grants U01 AG 10483, Alzheimer's Disease Cooperative Study, P50 AG 05142, the University of Southern California Alzheimer's Disease Research Center (ADRC); P30 AG28383, the University of Kentucky Alzheimer's Disease Center; NIH grant R01 AG019241, University of Kentucky PREADViSE trial; P50 AG 10124 The University of Pennsylvania Alzheimer's Disease Center; P50 AG 05681 and P01 AG 03991, the Washington University Alzheimer's Disease Research Center; P30 AG 08051, the New York University School of Medicine Alzheimer's Disease Center.
Disclosures: Drs. Schneider, Raman, Reisberg, and Mr. Insel report that they have no relationships to disclose. Dr. Schmitt reports past clinical trial grant support from Pfizer. Dr. Doody reports involvement in past clinical trials for Eisai/Pfizer, and payments for unrelated projects or lectures and support from the George and Cynthia Mitchell research fund. Dr. Clark reports involvement in past clinical trials sponsored by Eisai/Pfizer and was the recipient of a one-year educational grant from Eisai. Dr. Morris reports that neither he nor his family own stock or have equity interest (outside of mutual funds or other externally directed accounts) in any pharmaceutical or biotechnology company; that from July 2006 to the present he has participated or is currently participating in clinical trials of antidementia drugs sponsored by Elan, Eli Lilly and Company, and Wyeth; and from January 2006 to the present has served as a consultant or has received speaking honoraria from Bristol-Myers Squibb, Elan, GE Healthcare, Genworth, Janssen-Cilag, Merck, Myriad, Neurochem, Neuroptix, and Schering-Plough. Dr. Petersen….. Dr. Ferris reports involvement in and consulting related to past clinical trials and studies for Eisai/Pfizer.
REFERENCES
- 1.Schneider LS, Clark C, Doody R, Ferris SH, Morris J, Raman R, Reisberg B, Schmitt FA. ADCS Prevention Instrument Project: ADCS-clinician's global impression of change scales (ADCS-CGIC), self-rated and study partner-rated versions. Alzheimer's Disease and Associated Disorders. 2006;20(4):124–138. doi: 10.1097/01.wad.0000213878.47924.44. check this. S3. [DOI] [PubMed] [Google Scholar]
- 2.Leber P. What is the evidence that a dementia treatment works? Criteria used by drug regulatory authorities. In: Qizilbash N, et al., editors. Evidence-Based Dementia Practice. Blackwells; Oxford, UK: 2003. pp. 376–387. [Google Scholar]
- 3.Schneider LS, Olin JT, Doody RS, Clark CM, Morris JC, Reisberg B, Schmitt FA, Grundman M, Thomas RG, Ferris SH, the Alzheimer's Disease Cooperative Study Validity and reliability of the Alzheimer's Disease Cooperative Study-Clinical Global Impression of Change. Alzheimer's Disease and Associated Disorders. 1997;11(suppl 2):S22–S32. doi: 10.1097/00002093-199700112-00004. [DOI] [PubMed] [Google Scholar]
- 4.Morris JC. The Clinical Dementia Rating (CDR): current version and scoring rules. Neurology. 1993;43:2412–2414. doi: 10.1212/wnl.43.11.2412-a. [DOI] [PubMed] [Google Scholar]
- 5.Petersen RC, Thomas RG, Grundman M, Bennett D, Doody R, et al. Vitamin E and donepezil for the treatment of mild cognitive impairment. New England Journal of Medicine. 2005;352(23):2379–2388. doi: 10.1056/NEJMoa050151. [DOI] [PubMed] [Google Scholar]
- 6.Folstein MF, Folstein SE, McHugh PR. “Mini-mental state”. A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res. 1975;12:189–198. doi: 10.1016/0022-3956(75)90026-6. [DOI] [PubMed] [Google Scholar]
- 7.Mohs RC. The Alzheimer's Disease Assessment Scale. Int Psychogeriatr. 1996;8:195–203. doi: 10.1017/s1041610296002578. [DOI] [PubMed] [Google Scholar]
- 8.Mohs RC, Knopman D, Petersen RC, Ferris SH, Ernesto C, Grundman M, et al. Development of cognitive instruments for use in clinical trials of antidementia drugs: additions to the Alzheimer's Disease Assessment Scale that broaden its scope. The Alzheimer's Disease Cooperative Study. Alzheimer Disease & Associated Disorders. 1997;11(Suppl 2):S13–21. [PubMed] [Google Scholar]
- 9.Galasko D, Bennett D, Sano M, et al. An inventory to assess activities of daily living for clinical trials in Alzheimer's disease. The Alzheimer's Disease Cooperative Study. Alzheimer Dis Assoc Disord. 1997;11:33S–39S. [PubMed] [Google Scholar]
- 10.Galasko D, Bennett DA, Sano M, et al. ADCS prevention instrument project: assessment of instrumental activities of daily living for community-dwelling elderly individuals in dementia prevention clinical trials. Alzheimer's Disease and Associated Disorders. 2006;20(S3):S152–S159. doi: 10.1097/01.wad.0000213873.25053.2b. [DOI] [PubMed] [Google Scholar]
- 11.Reisberg B, Ferris SH, de Leon MJ, Crook T. The Global Deterioration Scale for assessment of primary degenerative dementia. Am J Psychiatry. 1982;139:1136–9. doi: 10.1176/ajp.139.9.1136. [DOI] [PubMed] [Google Scholar]
- 12.Patterson MB, Whitehouse PJ, Edland SD, et al. ADCS Prevention Instrument Project: quality of life assessment (QOL) Alzheimer Disease & Associated Disorders. 2006;20:S179–90. doi: 10.1097/01.wad.0000213874.25053.e5. [DOI] [PubMed] [Google Scholar]
- 13.Hochberg Y. A sharper Bonferroni procedure for multiple tests of significance. Biometrika. 1988;75:800–802. [Google Scholar]
- 14.Stern Y, Albert S, Tang MX, Tsai WY. Rate of memory decline in AD is related to education and occupation: cognitive reserve? Neurology. 1999;53(9):1942–7. doi: 10.1212/wnl.53.9.1942. [DOI] [PubMed] [Google Scholar]
- 15.Grundman M, Petersen RC, Ferris SH, et al. Mild cognitive impairment can be distinguished from Alzheimer disease and normal aging for clinical trials. Archives of Neurology. 2004;61:59–66. doi: 10.1001/archneur.61.1.59. [DOI] [PubMed] [Google Scholar]
- 16.Fleisher AS, Sowell BB, Taylor C, Gamst AC, Petersen RC, Thal LJ, Alzheimer's Disease Cooperative Study Clinical predictors of progression to Alzheimer disease in amnestic mild cognitive impairment. Neurology. 2007;68:1588–1595. doi: 10.1212/01.wnl.0000258542.58725.4c. [DOI] [PubMed] [Google Scholar]
- 17.Hedeker D, Gibbons R. Longitudinal Data Analysis. John Wiley & Sons, Inc.; Hoboken: 2006. [Google Scholar]
- 18.Winship C, Mare RD. Regression models with ordinal variables. American Sociological Review. 1984;49:512–525. [Google Scholar]
- 19.Salloway S, Ferris S, Kluger A, et al. Efficacy of donepezil in mild cognitive impairment: a randomized placebo-controlled trial. Neurology. 2004;63:651–657. doi: 10.1212/01.wnl.0000134664.80320.92. [DOI] [PubMed] [Google Scholar]
- 20.Olin JT, Schneider LS, Doody RS, Clark CM, Ferris SH, Morris JC, Reisberg B, Schmitt FA. Clinical evaluation of global change in Alzheimer's disease: identifying consensus. Journal of Geriatric Psychiatry and Neurology. 1997;9:176–180. doi: 10.1177/089198879600900404. [DOI] [PubMed] [Google Scholar]