SIST-M-IR activities of daily living items that best discriminate clinically normal elderly from those with mild cognitive impairment

Abstract

Background

Activities of daily living (ADL) impairment is a hallmark of Alzheimer's disease (AD) dementia, but impairment in instrumental ADL (IADL) has been reported in mild cognitive impairment (MCI). The Structured Interview and Scoring Tool-Massachusetts Alzheimer's Disease Research Center (MADRC)-Informant Report (SIST-M-IR) includes 60 graded items that assist in scoring the Clinical Dementia Rating; it assesses the spectrum of cognitive and ADL changes relevant to early AD. Of the 60 SIST-M-IR items, 41 address IADL; we aimed to determine which of these best discriminate individuals with MCI from clinically normal (CN) elderly.

Methods

We assessed 447 subjects participating in the MADRC longitudinal cohort (289 CN, 158 MCI). We performed logistic regression analyses predicting the probability of CN vs. MCI diagnosis using the SIST-M-IR items. Analyses were adjusted for demographic characteristics.

Results

We found that 4 SIST-M-IR items best discriminated between CN and MCI subjects (MCI performing worse than CN): “participating in games that involve retrieving words” (p=0.0001), “navigating to unfamiliar areas” (p=0.001), “performing mental tasks involved in a former primary job” (p=0.002), and “fixing things or finishing projects” (p=0.002).

Conclusions

Our results point to the earliest functional changes seen in elderly at risk for AD, which could be captured by a few simple questions. Honing the sensitivity of clinical assessment tools will help clinicians differentiate those individuals with normal aging from those who are developing cognitive impairment.

Introduction

Identifying those at elevated risk for Alzheimer's disease (AD) before a diagnosis of dementia is made is important for implementing earlier treatments effectively [1,2]. Although the development of memory and other cognitive impairments in AD is often a slow and insidious process, studies have shown that mild impairment in memory and everyday functioning can already be detected at the prodromal stage of AD—amnestic mild cognitive impairment (MCI) [3-8]. Focusing on the subtle distinctions between the population with the earliest clinical manifestations of AD and the population of clinically normal (CN) elderly is of vital importance in establishing a set of criteria that would better predict the likelihood of an MCI diagnosis, and eventually lead to earlier detection of preclinical AD [9].

One of the key manifestations of advanced AD dementia is having difficulty with basic activities of daily living (ADL) or self-care tasks, consisting of activities such as grooming, bathing, dressing, and eating. During the earlier stages of AD, such as mild dementia and MCI, individuals exhibit difficulty in instrumental activities of daily living (IADL). These activities are more complex and affect an individual's ability to live independently in the community. IADL include activities such as remembering appointments or engagements, driving and navigating, performing household chores, preparing meals, participating in hobbies, and managing the finances.

For individuals with even mild cognitive deficits, reliable informant reports given by family members, close friends, or caregivers are a valuable method for determining the individual's level of functioning in the community. The Structured Interview and Scoring Tool—Massachusetts Alzheimer's Disease Research Center (MADRC) Informant Report (SIST-M-IR) [10] is a reliable and sensitive assessment tool used to rate individual items of the Clinical Dementia Rating (CDR) scale [11] in greater detail, particularly along the spectrum of early cognitive change, yielding valuable information regarding individuals' memory, orientation, judgment, and abilities to live and function independently, with many of the items focused on IADL.

The objective of this study was to determine which of the SIST-M-IR IADL-related items best distinguish individuals with MCI from CN elderly. Elderly individuals may observe many changes in everyday life, which may or may not be suggestive of a current diagnosis of MCI or future progression to MCI. Therefore, identifying the SIST-M-IR items that are most sensitive to changes consistent with early AD can further facilitate screening of individuals who are at risk by primary care and other clinicians. Those individuals could then be more thoroughly evaluated in order to confirm their diagnosis and ultimately be treated with disease-modifying agents and other interventions when such treatments become widely available for clinical use.

Materials and Methods

Participants

The study participants were taken from the MADRC clinical core longitudinal research cohort. The overall objective of the MADRC is to support research investigating the use of multimodal biomarkers and cognitive evaluations for diagnosing, monitoring, and ultimately treating AD and related dementias. The research conducted within the MADRC stresses the importance of refining methods for early detection of AD at its incipient stages such as MCI and preclinical AD. The MADRC is one of 29 National Institute on Aging (NIA) national Alzheimer's Disease Centers (ADCs), with each research center collecting ongoing longitudinal data and assessments from their participants. Participants consist of older adults with and without cognitive impairment; their diagnoses range from CN elderly, to MCI, to dementia. MADRC participants are evaluated on an annual basis and recruited from the community or clinically from the Massachusetts General Hospital Memory Disorders Unit. Each evaluation follows the ADC Uniform Data Set (UDS) [12] protocol and includes a standard battery of neuropsychological assessments, Mini-Mental State Examination (MMSE) [13], CDR, Functional Activities Questionnaire (FAQ) [14], Neuropsychiatric Inventory brief questionnaire form (NPI-Q) [15], medical history, and neurological examination. At the MADRC, the UDS neuropsychological battery is supplemented by a memory word-list (either the Free and Cued Selective Reminding Test [16] or the California Verbal Learning Test [17]) and by phonemic verbal fluency (Controlled Oral Word Association (F-A-S) [18]).

The SIST-M-IR and Weintraub ADL [19] are also added. The SIST-M-IR, FAQ, Weintraub ADL, and NPI-Q are completed by an informant who knows the participant well and are then reviewed by an experienced clinician, who elicits additional questions from the informant. The same clinician rates the CDR. The neuropsychological tests (including the UDS battery, supplementary tests, and MMSE) are administered by well-trained research assistants. These assessments are rated and scored separately. Then, the clinician reviews all of the above information in order to assign a diagnosis, which is later reviewed by a consensus panel.

The UDS was implemented in the MADRC annual assessment in 2005, and The SIST-M-IR was added in January 2008. For the current analyses, the dataset included 447 participants with baseline SIST-M-IR data: 289 participants were CN elderly and 158 participants had MCI. Participants ranged in age from 45 to 96 years old (inclusive) and were in general good health and medically stable. All participants had study partners that completed the SIST-M-IR questionnaire, providing information about the participants' daily functioning.

Participant diagnosis was made by an experienced clinician and followed up by a consensus diagnosis as previously described [20]. Diagnoses were based on the following information: clinical history obtained during the research visit (and when available, in a small minority of participants, notes from a clinical evaluation), UDS and supplemental neuropsychological testing results, MMSE, CDR, FAQ, Weintraub ADL, and NPI-Q. Responses on the 60 SIST-M-IR items were used to inform CDR rating. Strict cut-offs for neuropsychological tests were not employed to determine diagnosis. CN participants performed normally on cognitive neuropsychological testing in all domains and the majority (92%) had a CDR Global score of 0. MCI participants consisted of amnestic MCI single domain or amnestic MCI multiple domain, had subjective memory concerns (either by self or informant report), objective memory impairment on neuropsychological testing, essentially intact IADL (determined by a clinician without a strict cut-off; the clinician has access to the CDR, FAQ, SIST-M-IR, and Weintraub ADL), and did not meet criteria for dementia.

The Institutional Review Board of the Massachusetts General Hospital, Boston, approved this study. Written informed consent was obtained from all participants and study partners prior to participation in the research study.

Clinical Assessments

The Structured Interview and Scoring Tool—MADRC Informant Report (SIST-M-IR) consists of 60 items examining each of the 6 CDR domains (Memory, Orientation, Judgment and Problem Solving, Community Affairs, Home and Hobbies, and Personal Care), which assess daily functioning and cognitive changes across the spectrum of mild impairment [10]. All study partners were given the questionnaire to complete at the research visit and rated how much the participant had changed (or stayed the same) over the past 5 to 10 years. Each item is rated from 0 to 2 (0 = none/rarely, 1 = sometimes, 2 = often). Of note, in prior publications, the items were rated as 0, 0.5, and 1 [10,21]; for the purposes of the current analyses, scores of 0.5 were converted to 1 and scores of 1 were converted to 2. The score range refers to the participants' level of difficulty in performing the corresponding task with higher scores indicating greater difficulty. Out of the 60 SIST-M-IR items, the 41 items judged by a clinician (GAM) to be related to IADL gathered from the first 5 domains (excluding Personal Care) were included in the current study.

Statistical Analyses

All analyses and graphs for this study were performed using SAS Version 9.3 and JMP Version Pro 10 statistical and graphical software.

Preliminary Multiple Test Correction and Data Reduction Screening Tests

In order to determine which of the 41 IADL-related SIST-M-IR items best discriminate between CN and MCI diagnoses, preliminary data reduction analyses were performed. Forty-one t-tests of diagnosis mean differences were run initially on the SIST-M-IR items using a nonparametric resampling stepdown permutation test (using the SAS Multtest Procedure) to adjust the p values for the multiple tests. This method provides more powerful tests than a Bonferroni or Sidak correction would because these latter techniques assume tests are independent, whereas the SIST-M-IR item inter-correlations clearly indicate otherwise (see Results). The resampling method adjusts on the basis of observed correlations. However, for comparison, we also adjusted p values with ordinary Bonferroni, stepdown Sidak, and false discovery rate (FDR) methods.

We then ran a backward elimination discriminant analysis of the 41 SIST-M-IR items as initially simultaneous discriminators of MCI versus CN in order to further adjust for the inter-correlations between SIST-M-IR items. The SIST-M-IR items have only a few discrete values and therefore violate the normality assumption of the significance tests for the discriminant analysis, but we used this method merely as a preliminary data reduction technique to be followed subsequently by formal analysis. The pattern of missing values across the 41 SIST-M-IR items would have somewhat reduced the sample size for this analysis if listwise deletion were employed. Therefore, we decided to pre-estimate the pairwise deleted inter-correlation matrix of the 41 SIST-M-IR items and input that matrix into the analysis.

Primary Analyses

The 10 SIST-M-IR items surviving the preliminary data reduction above were entered as simultaneous predictors along with covariates associated with diagnosis (see Table 1), including baseline age, sex, years of education, and the American National Adult Reading Test intelligence quotient (AMNART IQ) [22] (an estimate of premorbid intelligence, which served as our proxy of cognitive reserve), in a backward elimination (p<0.05 cut-off) logistic regression model predicting the probability of being assigned a diagnosis of MCI versus CN. We decided to use logistic regression rather than discriminant analysis because in a logistic regression, the SIST-M-IR items are predictors, not dependent variables, and are not required to be normally distributed as they would be as dependent variables in a discriminant analysis.

Table 1.

Baseline demographics and characteristics of subjects.

Group	All	CN	MCI
n	447	289	158
Age*	73.8 ±9.5	71.4 ±9.7	78.3 ±7.5
Sex (% male)*	34.9%	27.3%	48.7%
Education	16.1 ±2.5	16.3 ±2.3	15.8 ±2.8
AMNART IQ**	122.2 ±8.2	123.1 ±7.5	120.5 ±9.3
MMSE*	28.6 ±1.9	29.2 ±1.1	27.6 ±2.4
CDR sum of boxes*	0.8 ±1.1	0.1 ±0.3	2.0 ±1.1

AMNART IQ (American National Adult Reading Test intelligence quotient), CDR (Clinical Dementia Rating), CN (clinically normal), MCI (mild cognitive impairment), MMSE (Mini-Mental State Examination). All values (except n and sex) represent mean ± standard deviation.

^*p<0.0001 for CN vs. MCI.

^**p<0.005 for CN vs. MCI.

Results

Table 1 provides baseline demographics and clinical characteristics for CN and MCI subjects. Significant differences were present between diagnostic groups in age, sex, AMNART IQ, MMSE scores, and CDR sum of boxes scores. SIST-M-IR item inter-correlations ranged from r=-0.02 to r=0.72 with most correlations ranging from r=0.2 to r=0.5.

Preliminary Data Reduction

The mean for the MCI group was significantly higher than that for the CN group on all 41 SIST-M-IR IADL related items, according to the unadjusted t-test p values and using any of the p value adjustment methods (resampling stepdown permutation, stepdown Sidak, Bonferroni, FDR), except for 2 items under the Bonferroni correction which is known to be conservatively biased. Therefore, in a univariate descriptive sense, there appeared to be a real difference between the MCI and CN groups with the MCI group having the higher mean, on probably every one of the 41 SIST-M-IR items. Thus, any significant SIST-M-IR differences between MCI and CN groups reported below are not likely to be chance effects related to multiple testing.

The backward elimination discriminant analysis, revealed 10 SIST-M-IR items as being simultaneous significant discriminators of MCI versus CN, adjusting each for the others. Table 2 shows the means and standard deviations for the scores for each of these 10 SIST-M-IR items. Due to muliticollinearity, after covarying for other SIST-M-IR items, 3 of the 10 items (“performing household tasks independently”, “participating in activities that involve patterns or following diagrams and instructions”, and “remembering game rules or keeping track of score”) showed discriminant coefficients in the counter-intuitive direction of predicting a higher adjusted CN mean than MCI mean, holding all else constant.

Table 2.

10 SIST-M-IR surviving initial data reduction.

SIST-M-IR Items	CN	MCI
“Relying more on others to remember appointments”	0.05 ±0.21	0.42 ±0.62
“Having difficulty with time relationships”	0.09 ±0.29	0.46 ±0.55
“Navigating to unfamiliar neighborhoods”*	0.06 ±0.26	0.40 ±0.59
“Performing mental tasks involved in a former primary job”*	0.05 ±0.23	0.46 ±0.60
“Forgetting items to buy without a list”	0.02 ±0.12	0.26 ±0.47
“Performing household tasks independently”	0.03 ±0.17	0.11 ±0.34
“Fixing things or finishing projects”*	0.07 ±.27	0.50 ±0.63
“Participating in activities that involve patterns or following diagrams and instructions”	0.02 ±0.15	0.14 ±0.40
“Remembering game rules or keeping track of score”	0.02 ±0.13	0.32 ±0.60
“Participating in games that involve retrieving words”*	0.03 ±0.16	0.35 ±0.57

CN (clinically normal), MCI (mild cognitive impairment), SIST-M-IR (Structured Interview and Scoring Tool—Massachusetts Alzheimer's Disease Research Center—Informant Report). The score range (0 = none/rarely, 1 = sometimes, 2 = often) refers to the level of difficulty in performing each item (higher scores indicate greater difficulty). All values represent mean ± standard deviation.

^*SIST-M-IR items that were found to significantly discriminate between a diagnosis of CN and MCI in the multivariate logistic regression model.

Primary Analyses

The 10 SIST-M-IR items surviving the preliminary data reduction above were then entered as simultaneous predictors along with the covariates of baseline age, sex, years of education, and AMNART IQ in a backward elimination logistic regression predicting the probability of being assigned a diagnosis of MCI versus CN at baseline. The model yielded 4 significant SIST-M-IR items, for each of which a higher score predicted greater probability of being assigned a diagnosis of MCI as compared to CN: “participating in games that involve retrieving words” (p=0.0001; Odds Ratio (OR) per unit=11.36, 95% Confidence Interval (CI)=3.66, 44.49), “navigating to unfamiliar neighborhoods” (p=0.001; OR=4.75, CI=1.90, 12.93), “performing mental tasks involved in a former job” (p=0.002; OR=5.35, CI=1.95, 16.17), and “fixing things or finishing projects” (p=0.002; OR=5.14, CI=1.85, 15.17). Older age also predicted higher probability of MCI for this sample (p=0.007; OR for decade of age=2.12, CI=1.27, 3.77) and was controlled for. The model as a whole was highly significant (p<0.0001).

Figure 1 illustrates the optimal linear combination of logistic regression predictors (the 4 SIST-M-IR items above and age) that determine the probability of being assigned a diagnosis of MCI versus CN. Additionally, the specificity and sensitivity of these 4 items was high, with the area under the receiver operating characteristic (ROC) curve equal to 0.88 (See Figure 2).

Figure 1.

The optimal linear combination of logistic regression predictors that show the probability of being assigned a diagnosis of MCI versus CN is illustrated. Age and 4 of the SIST-M-IR items were retained in the final model. The y-axis signifies the probability of MCI with 1 = MCI and 0 = CN. The circles represent actual subjects whereas the sigmoid curve is the predicted probability of being assigned a diagnosis of MCI as opposed to CN. CN (clinically normal), MCI (mild cognitive impairment), SIST-M-IR (Structured Interview and Scoring Tool—Massachusetts Alzheimer's Disease Research Center—Informant Report).

Figure 2.

The ROC curve corresponding to the logistic regression model indicates the specificity and sensitivity of the optimal linear combination of 4 SIST-M-IR items (and the covariate of age) retained in the final model discriminating MCI from CN. CN (clinically normal), MCI (mild cognitive impairment), ROC (receiver operating characteristic), SIST-M-IR (Structured Interview and Scoring Tool—Massachusetts Alzheimer's Disease Research Center—Informant Report).

Discussion

A thorough assessment of a well-characterized sample of CN elderly and individuals with MCI demonstrated that a handful of important questions targeting IADL addressed to an informant who knows the individual well can effectively distinguish between individuals assigned to the two diagnostic groups at baseline after adjusting for demographic characteristics.

The 4 questions that significantly discriminated between those assigned to a diagnosis of CN versus MCI were: “participating in games that involve retrieving words”, “navigating to unfamiliar neighborhoods”, “performing mental tasks involved in a former job”, and “fixing things or finishing projects”. Participants assigned a diagnosis of MCI were described as performing worse on these activities when compared to CN elderly. We have previously shown that a computer-based algorithm used to generate global CDR scores from the SIST-M-IR can successfully classify individuals whom clinicians diagnose as MCI versus CN [21]. Accordingly, in the current study, participants diagnosed as MCI performed worse than those diagnosed as CN on all 41 IADL-related items of the SIST-M-IR when looking at each item without adjusting for the others. However, the 4 high level activities mentioned above, tend to draw on multiple cognitive domains for effective performance and were the most sensitive of all the activities in the multivariate model. These 4 SIST-M-IR items draw on executive function, language, memory, and visuospatial processing indicative of their complexity. Multiple prior studies have indicated that executive dysfunction, as well as more global cognitive impairment are involved in IADL impairment [4,7,23]. In fact, the types of activities represented by these items are often recommended by clinicians as examples of cognitively stimulating behaviors that can be used to sustain cognitive health and ward off the development of early AD.

Similarly to the global score generated by the SIST-M-IR, several subjective IADL scales have successfully differentiated between CN elderly and MCI in large well characterized samples resembling the sample of the current study. Those scales include the CDR [4,5], the FAQ [4,5], the ADL-Prevention Instrument (ADL-PI) [24], and the Everyday Cognition [25]. Two performance-based IADL instruments have also been shown to successfully distinguish between CN elderly and MCI: the University of California, San Diego Performance-Based Skills Assessment [26] and the Financial Capacity Instrument [27]. All of these tests are clearly sensitive to early changes in AD. However, some of the tests are quite time consuming and require trained personnel to administer. Therefore, if a few brief questions, such as the 4 SIST-M-IR items identified in our analyses, can yield equivalent sensitivity, screening for early AD can be made widely available in the primary care setting.

A study utilizing the ADL-PI has shown that poor performance on this scale successfully predicts future cognitive decline in CN elderly [24]. We have also previously demonstrated that a global CDR score generated by the SIST-M-IR significantly predicts progression from CN and MCI to AD dementia after a mean follow-up period of 7 years [21]. In the sample of the current study, the follow-up period was only 3 years and accordingly only 7.4% of CN participants progressed to MCI. Therefore, we did not have enough incident cases of MCI to detect a significant association with the SIST-M-IR items. Future studies with a longer follow-up period will be able to better assess this relationship.

The type of item analyses employed in the current study in which we used large well-characterized cohorts to identify individual clinical items that best distinguish between CN and MCI subjects and best predict progression from CN to MCI could serve as a good model for future studies focusing on other assessment tools of subjective IADL and subjective cognitive concerns. This in turn might help identify valuable questions for screening purposes in the primary care setting, as well as help identify questions that could be combined into a single more sensitive subjective questionnaire for the assessment of early AD.

The current study had several limitations. Our well-characterized, healthy sample of non-demented individuals was highly educated and intelligent with a mean of 16 years of education and a mean AMNART IQ of 122. This sample is typical of academic-based studies and multicenter clinical trial samples of early AD. However, it is not representative of the general population. Therefore, the SIST-M-IR items identified here will need to be evaluated further in population-based studies in order to strengthen the argument for their utility as a screening tool in primary care and other clinical and research settings. On the other hand, these items might be useful in combination with other items derived from similar analyses in preclinical and prodromal AD clinical trials, which consist of similar samples, as sensitive IADL outcome measures. IADL impairment in individuals across the early AD spectrum, including CN, MCI, and mild AD dementia, has been associated with multiple AD biomarkers, including measures of atrophy, hypometabolism, cortical amyloid, and cerebrospinal fluid amyloid-beta 1-42, total tau, and phospho-tau [28-33]. The current study did not include AD biomarkers in the classification of diagnoses (the classification was based solely on clinical information). Therefore, we did not know what portion of CN elderly had preclinical AD and what portion of MCI individuals had prodromal AD. Future studies that include AD biomarkers will be able to better determine the utility of the SIST-M-IR items in early AD by providing better diagnostic classification of individuals as well as assessing associations of the SIST-M-IR items with biomarkers.

Conclusions

The results of the current study identified which IADL-related SIST-M-IR items were most informative in differentiating those assigned a diagnosis of MCI versus CN, thus helping to identify the earliest functional changes in elderly at risk for AD. Honing the sensitivity of clinical assessment tools will help clinicians differentiate those individuals with normal aging from those who are developing cognitive impairment. Moreover, from a practical point of view, providing primary care physicians with a short list of important questions targeting high level everyday activities may prove very helpful in screening for early AD. By identifying individuals at early stages of cognitive and functional decline, efforts towards treatment and ultimately prevention of AD will prove more effective.