The Ecological Validity of The Uniform Data Set 3.0 Neuropsychological Battery in Individuals with Mild Cognitive Impairment and Dementia

Abstract

Objective:

Ecological validity refers to the ability of neuropsychological measures to predict real world performance. Questions remain as to the ecological validity of commonly used measures, particularly regarding their relationships to global versus specific activities of daily living among those with neurodegenerative disease. We explored these issues through the lens of the Uniform Data Set 3.0 Neuropsychological battery (UDS3NB) in individuals with mild cognitive impairment and dementia.

Method:

UDS3NB and informant rated Functional Activities Questionnaire scales were evaluated from 2,253 individuals with mild cognitive impairment and dementia. Ordinal regression equations were used to explore the relationships of demographic and cognitive variables with overall and specific instrumental activities of daily living.

Results:

Delayed recall for visual and verbal material, and performance on trail making tests were consistent predictors of global and specific functions. Specific skills (i.e. naming or figure copy) showed differential relationships with specific activities, while phonemic fluency was not related to any particular activity.

Conclusions:

Measures in the UDS3NB predicted activities of daily living in individuals with MCI and dementia, providing initial support for the ecological validity of these tests. Specifically, measures that tap core deficits of AD, such as delayed recall, and sequencing/shifting, are consistent predictors of performance in daily tasks.

Ecological Validity and Neuropsychological Assessment in Older Adults with Cognitive Disorders: An Overview

Neuropsychological assessments among older adults are commonly undertaken with the goal of determining the extent to which cognitive problems are present and how such difficulties may impact activities of daily living (ADLs, both instrumental (iADLS) and basic physical self-maintenance tasks). Such evaluations play an important role in the diagnosis of conditions, like mild cognitive impairment (MCI; Albert et al., 2011) and dementia (McKhann et al., 2011). The study of how particular neuropsychological constructs and measures relate to and predict difficulties with real world function is termed the veridicality aspect of ecological validity (Franzen & Wilhelm, 1996).

Four primary meta-analyses in recent years summarize an expanding literature on the veridicality of neuropsychological assessment in older adult samples with regard to ADLs (McAlister, Schmitter-Edgecombe, & Lamb, 2016; Lindbergh, Dishman, & Miller, 2016; Overdorp, Kessels, Claasen & Oosterman 2016; Royall et al 2007). These studies suggest that cognitive measures explain on average 23% of the variance in day-to-day functioning in those with MCI (McAlister, Schmitter-Edgecombe, & Lamb 2016), though the amount of variance explained in broader clinical studies, some of which included patients with dementia, ranged from 0–78% (Royall et al, 2007). Moreover, a greater number of impaired cognitive domains in older adults is associated with greater dysfunction in iADLs (Lindbergh, Dishman, & Miller, 2016). The cognitive domain most associated with ADL impairments was executive functioning, though measures of episodic memory also explained significant variance in ADL scores (McAlister, Schmitter-Edgecombe, & Lamb 2016).

Against this empirical backdrop, several broad questions remain in the older adult cognitive disorder literature, two of which are considered in this manuscript. First, does the association of neuropsychological measures with iADLs vary as a function of the specific daily task being considered? Second, how do recently developed cognitive batteries relate to iADL functions in these populations?

Global vs. Activity Specific Veridicality

Studies of MCI and dementia frequently focus on ADL ability at a broad or global functional level. This trend reflects the fact that significant difficulties or dependence in ADL function serve as the dividing line between a diagnosis of dementia and MCI (Albert et al, 2011). However, as noted by Royall and colleagues, “The cognitive correlates of specific functional capacities are widely held to differ, both from each other and from general functional capacities, although this premise has seldom been tested empirically” (Royall et al, 2007, p. 252).

In the years since Royall’s observation, research has begun to investigate these specific relationships. For example, in a study evaluating financial management abilities in older adults with MCI and Alzheimer’s disease dementia (Sherod et al, 2009), arithmetic ability emerged as a significant predictive variable in both groups, explaining 55% and 46% of the variance in financial functioning, respectively. Of cognitive measures, Trail Making Test Part B (TMTB) emerged as a predictor in both groups, while episodic memory only explained significant variance in control and Alzheimer’s disease (AD) groups. In contrast, in a systematic review of driving abilities and cognitive assessment in AD and MCI groups (Hird, Egeto, Fischer, Naglie & Schweizer, 2016), TMTB and mazes performance emerged as the best predictors of driving performance, though tasks assessing other aspects of executive functioning, attention, visual functioning, and global cognition also contributed to the variance explained. In terms of medication management, executive and working memory skills, rather than episodic memory performances, emerged as the strongest predictors of medication adherence (Insel, Morrow, Brewer, & Figueredo, 2006). To date though, few studies have simultaneously considered the relationships of test scores from a comprehensive neuropsychological battery with several different iADLs, likely due to restrictions in sample sizes and limits on the number of neuropsychological tests employed.

Veridicality and the Uniform Data Set 3.0 Neuropsychological Battery

As Marcotte and colleagues (2010) noted, it is critical to evaluate optimum metrics for predicting real world performance in broad diagnostic groups to provide further evidence of the veridicality of neuropsychological assessment. Indeed, researchers and practitioners are increasingly called upon to use neuropsychological measures as a means of understanding or predicting real world outcomes, establishing diagnoses, guiding rehabilitative efforts, and indexing change in clinical trials (Marcotte et al., 2010). Thus, establishing that neuropsychological measures are valid for such purposes is critical, as noted in the guidelines for the adoption of psychological measures (American Educational Research Association, American Psychological Association, National Council on Measurement in Education, Joint Committee on Standards for Educational & Psychological Testing, 2014).

To this end, the field of aging research increasingly attempts to harmonize cognitive assessments across centers and populations, and the National Alzheimer’s Coordinating Center (NACC) has emerged as one of the largest multi-center collaborations in addressing cognitive disorders in aging. In 2015, the NACC released the Uniform Data Set 3.0 Neuropsychological Battery (UDS3NB; Besser et al., 2018; Weintraub et al., 2018), a revised version of the long-utilized NACC neuropsychological battery (Weintraub et al., 2009). The UDS3NB is composed of a core set of measures administered at Alzheimer’s Disease Research Centers across the country with wide a spectrum of patients. To aid in the clinical and research utilization of the core measures from this battery, normative and psychometric tools are freely available to help clinicians understand results from these data on the level of the individual (Devora, Beevers, Kiselica, & Benge, 2019; Kiselica, Kaser, Webber, Small, & Benge, 2020; Kiselica, Webber, & Benge, 2020a; Kiselica, Webber, & Benge, 2020b; Shirk et al., 2011; Weintraub et al., 2018).

The NACC database has proven to be fruitful source of data to explore questions regarding the everyday functioning of individuals with various forms of cognitive impairment and dementia. For example, prior studies with NACC samples have revealed subtle differences in ADL ability in subtypes of MCI (Hanfelt et al, 2011; Mis, Devlin, Drabick, & Giovannetti, 2019; Nowrangi, Rosenberg & Leoutsakos, 2016), ADL features that separate MCI from dementia (Teng et al, 2010), and variations in ADL functioning among both subgroups of Alzheimer’s disease (Mez et al, 2013) and fronto-temporal dementia (Moheb, Mendez, Kremen, & Teng, 2017). However, to date, the NACC database has not been leveraged to explore the veridicality of the UDS3NB.

The Current Study

In light of the above limitations noted in the literature, the current study sought to 1) evaluate the variability in the degree of association between UDS3NB neuropsychological measures and global vs. specific iADL functioning in individuals with MCI and dementia, and 2) provide evidence of the ecological validity of the UDS3NB.

Materials and Methods

Participants and Inclusion/Exclusion Criteria

Data from the current study were derived from the NACC central data repository, which consists of de-identified data from consented subjects at each Alzheimer’s Disease Research Center. Procedures at individual sites were conducted in compliance with appropriate laws, guidelines, and institutional review. Data from 35 Alzheimer’s Disease Centers were included in the initial data pull. The initial data request included all individuals who attempted the UDS3NB on their initial visit from March 2015 until the quarterly data freeze for our February 2018 data request. Individuals were included even if they were missing elements of the UDS3NB.

For the current study, only individuals with clinical diagnoses of MCI or dementia made by study staff after comprehensive neurological and neuropsychological evaluation were included (see Besser et al., 2018 for an overview and more details of the Uniform Data Set 3.0 for evaluation schema).

Instruments

Details of the UDS3NB are described in detail in other publications (Besser et al., 2018; Weintraub et al., 2018) and are not reiterated here in detail. Briefly, the battery consists of a measure of immediate and delayed recall of a short story, known as the Craft Story (Craft et al., 1996); a figure copy and recall task, the Benson Figure (Possin, Laluz, Alcantar, Miller, & Kramer, 2011); number span forwards and backwards trials; a confrontation naming task, the Multilingual Naming Test (Gollan, Weissberger, Runnqvist, Montoya, & Cera, 2012; Ivanova, Salmon, & Gollan, 2013); semantic (animals and vegetables) and phonemic (F- and L-words) verbal fluency tasks; and rapid visuomotor search and sequencing (Trail Making Test parts A and B; Partington & Leiter, 1949).

The Functional Activities Questionnaire (FAQ; Pfeffer et al., 1982) was used as the index of iADL functioning. The FAQ is a 10-item collateral rated scale consisting of a broad range of instrumental daily activities (item labels included in Table 3) focusing on complex, as opposed to basic, functions. Examples include traveling outside the neighborhood and organizing taxes and other records. The FAQ as employed by the NACC is available at https://www.alz.washington.edu/NONMEMBER/NACCFormsAndDoc.html In the UDS, each item is rated on an ordinal scale of 0 (no difficulty) to 3 (cannot do) the task. Responses that were missing, were never done by the respondent, or which the informant did not know were coded as missing for the sake of the current analyses. The median score of non-missing FAQ ratings was used as a global index of iADL functioning. Additionally, individual item scores were considered separately as ordinal variables.

Table 3:

Median Ratings for Functional Activity Questionnaire Total Score and Items (N = 2253)

Variable	Median	Interquartile Range
Total Scale	0.5	2
Bills	1	3
Taxes/Paperwork	2	3
Shopping	0	2
Games	0	1
Stove/Beverage Prep	0	1
Meal Preparation	0	2
Tracking Current Events	0	1
Paying Attention	0	1
Remembering Dates	1	2
Traveling	1	3

Statistical Analyses

Initial analyses of the dataset revealed missing data in the primary dependent variable (FAQ) and cognitive predictors, particularly TMTB. FAQ items were rarely missing because a questionnaire was actually unavailable (n = 34; 1.5%) or because the informant was not knowledgeable of the patient’s ability to perform the task (missing ranged from n = 6 (.2%) for the shopping item to a maximum of 64 (2.8%) for the games item). More concerning from a missing data standpoint are the responses of “not applicable/never did” which was endorsed for 10.8% of respondents for bill pay, 14.7% for taxes, 8.7% for game play, and 9.4% for meal preparation. While at a global scale level these items are typically discarded for calculation of median for applicable items only (Teng et al, 2010), the current study also considered individual item responses. Our clinical experience is that individuals who are rated as having never done a particular instrumental activity have varied reasons for such a rating. For example, an individual may have had lower premorbid global functioning or come from a socio-cultural background where a particular activity was not expected or supported. While multiple imputation is a recommended procedure for addressing missing data in many cases, in situations where the data missing may be dependent upon the value of the variable itself this approach is suspect (Enders, 2010). Our concern about imputing the missing value for this variable was that we do not have adequate access to all information to conclude that the item was missing randomly. Thus, we chose to use pairwise deletion of cases with missing FAQ data for individual prediction models and to not impute the missing data.

In a similar vein, TMTB variables were missing due to “cognitive or behavioral problems” in a relatively high proportion of cases (n = 446, 19.8% of the total sample). As prior work and clinical experience suggests such individuals may be unable to complete the measure due to a number of reasons such as low frustration tolerance, distractibility, executive dysfunction, or even the underlying disease process itself (Kiselica & Benge, 2019), we considered an inability to complete the measure as a potentially clinically meaningful variable in and of itself and not missing entirely at random (Enders, 2010). Thus, for the current study, valid median FAQ variables were used as dependent variables for the overall analysis. Cases with missing data were deleted pairwise (i.e., if data were missing on both the predictor and on the dependent variable [FAQ]), except in the case of TMTB. For this variable, secondary analyses were conducted using inability to complete the measure as a predictor in its own right.

A series of ordinal regressions were employed, using total median FAQ score and individual items as dependent variables. All statistical analyses were conducted in SPSS version 27 (IBM Corp., 2020). Predictors included demographic factors (age, education, sex, and ethnicity—coded as white/non-white given small sample sizes for individual racial groups), as well as raw score variables from the cognitive variables in the UDS3NB. Raw scores were used as they tend to be the more robust predictors of daily functioning (Barrash et al, 2010; Silverberg & Millis, 2009). In addition, neuropsychiatric symptom severity as assessed by the Neuropsychiatric Inventory Questionnaire (NPI-Q; Kauffer et al 2000) and self-reported depression (Geriatric Depression Scale 15 Item Short form (GDS-15; Sheikh & Yesavage, 1986) were included as control variables.

Assumptions for ordinal regression were assessed as follows. An initial multiple regression was conducted and tolerance and Variance Inflation Factors (VIF) were analyzed to check the assumptions of multi-collinearity. Given the large sample size, separate binomial logistic regressions using cumulative dichotomous median FAQ ratings were utilized to evaluate the assumption of proportional odds.

Given that the primary analyses involved 11 models with 4 demographic parameters, 2 neuropsychiatric variables, and 11 cognitive variables to have parameters estimated with p values (187 primary p values), there was an increased risk for generating false positive errors. We therefore used a False Discovery Rate (FDR; Benjamini and Hochenberg, 1995; McDonald, 2014) to minimize the risk of false discoveries at 5%.

As noted above, nearly a quarter of the sample was missing data on TMTB due to a cognitive or behavioral problem. Therefore, we ran a secondary set of analysis entering the TMTB variable as a dichotomous variable (those who completed the TMTB vs. those who could not complete due to a cognitive or behavioral problem; others coded as missing).

Results

Sample Description

Clinical and demographic data for the current sample is presented in Table 1. The sample was split roughly equally between those with MCI and dementia, with Alzheimer’s being the most commonly identified presumed etiology for impairment. The sample was predominantly white, not of Hispanic origin, and well educated. Eleven percent of the sample reported Black or African-American race, 2% Asian, and less than 1% reported native American or Hawaiian/Other Pacific Islander racial identification; given the relatively small numbers of non-white racial identification, race was collapsed into a dichotomous variable for subsequent analysis in the regression equations. Cognitive variables are described in Table 2, while Table 3 describes the FAQ variables.

Table 1:

Demographic and Clinical Data for the Total Sample (N=2253)

Variable	M	SD	%	N
Age	70.3	9.4
Education- Years	16.9	10.3
Race- White			85.2	1919
Hispanic Origin			94.7	2134
Hispanic Origin
sex- Female			50.6	1139
NPI-Q Severity	5.7	4.8
Rating Sum
GDS-15 Sum	2.7	2.8
Diagnosis-MCI			45.4	1023
Amnestic Single Domain			13.5	305
Amnestic Multiple Domain			23.2	522
Non-Amnestic MCI Single Domain			5.4	122
Non-Amnestic MCI-Multiple Domain			3.3	74
Diagnosis- Dementia			54.6	1230
Presumed Etiology/Syndrome *
Alzheimer’s			70.2	1582
Lewy Body			2.9	65
Frontotemporal Dementia			7.5	168
Primary Progressive Aphasia			6.7	151
Posterior Cortical Atrophy			1.7	39
Cerebrovascular Disease			9.1	206

^*Note: Not all individuals met criteria for a clear etiology/syndrome at the time of their visit. Presumed etiology represents the primary contributing diagnosis as judged by the rating clinician/team.; NPI-Q = Neuropsychiatric Inventory Questionnaire; GDS-15 = Geriatric Depression Scale 15 Item Version;

Table 2:

Descriptive Data for Neuropsychological Variables (N=2253)

Variable	M	SD
Benson Figure Copy	13.80	3.79
Benson Figure
Delayed Recall	5.63	4.57
Animal Fluency	13.44	6.09
Vegetable Fluency	8.62	4.52
Phonemic Fluency	19.99	9.70
Craft Story Immediate Recall	11.56	7.35
Digit Span Forward	5.91	1.43
Digit Span Backwards	3.80	1.45
Craft Story Delayed Recall	7.64	7.26
Multilingual Naming Test	25.40	6.61
Trail Making Test Part A (seconds)	56.28	35.56
Trail Making Test Part B (seconds)	157.65	88.38

The cognitive variable descriptive data, and percent missing values, are presented in Table 2.

Predicting ADL Performance with UDS3NB Scores

In initial modeling using all UDS3NB scores to predict average FAQ score, inclusion of both immediate and delayed recall of the Craft story introduced mild problems with multi-collinearity. This finding was unsurprising, given the high correlation between these scores. Given the particular theoretical importance of delayed recall in a predominantly Alzheimer’s disease sample (McKhann et al., 2011), the decision was made to omit immediate recall from subsequent models. Once this change was made, regression analyses no longer indicated concern for collinearity. The assumption of proportional odds was met, as assessed by a full likelihood ratio test comparing the fit of the proportional odds location model to a model with varying location parameters, χ2(85) = 53.895, p = .997.

The model statistically significantly predicted FAQ score over and above the intercept-only model, (χ2(17) = 409.997, p < .001; Pseudo R² values: Cox and Snell = .320, Nagelkerke =.339, McFadden=.134). Model parameters for individual variables are presented in Table 4.

Table 4.

Model Parameters for Predicting Median Functional Activities Questionnaire Rating Score from Cognitive and Demographic Factors

			95% Wald Confidence Interval					95% Wald Confidence Interval for Exp(B)
Variable	B	SE B	Lower	Upper	p	Significant after FDR	Exp(B)	Lower	Upper
Sex	0.19	0.14	−0.08	0.46	0.17		1.21	0.92	1.58
Race	0.71	0.20	0.32	1.10	0.00	*	2.04	1.38	3.01
Education	0.01	0.02	−0.03	0.06	0.64		1.01	0.97	1.06
Age	0.00	0.01	−0.01	0.02	0.75		1.00	0.99	1.02
NPI-Q Severity	0.19	0.02	0.16	0.22	0.00	*	1.20	1.17	1.24
GDS Total	−0.06	0.02	−0.11	−0.02	0.01	*	0.94	0.90	0.98
Benson Figure Copy	0.02	0.03	−0.03	0.08	0.45		1.02	0.97	1.08
Benson Figure Delayed Recall	−0.07	0.02	−0.11	−0.04	0.00	*	0.93	0.90	0.96
Animal Fluency	0.00	0.02	−0.04	0.03	0.82		1.00	0.97	1.03
Vegetable Fluency	−0.05	0.02	−0.09	−0.01	0.03		0.95	0.91	1.00
Phonemic Fluency	0.01	0.01	−0.01	0.02	0.58		1.01	0.99	1.02
Digit Span Forward	0.10	0.05	0.00	0.21	0.06		1.11	1.00	1.23
Digit Span Backwards	−0.02	0.06	−0.14	0.10	0.71		0.98	0.87	1.10
Craft Story Delayed Recall	−0.06	0.01	−0.09	−0.04	0.00	*	0.94	0.92	0.96
Multilingual Naming Test	−0.01	0.01	−0.03	0.02	0.71		1.00	0.97	1.02
Trail Making Test Part A (seconds)	0.01	0.00	0.01	0.02	0.00	*	1.01	1.01	1.02
Trail Making Test Part B (seconds)	0.00	0.00	0.00	0.00	0.00	*	1.00	1.00	1.00

Predicting Individual iADLs.

Analytic steps described above were repeated for each of the 10 ADL items. Full parameter estimates (similar to Table 4) for each item are reported in supplemental tables 1–10. Omnibus tests suggested each model predicted the specific iADL better than intercept only models. We summarize the results for individual predictors across these analyses in Table 5. This table illustrates which demographic and cognitive pooled parameters were considered statistically significant predictors after adjusting for the potential for false discoveries.

Table 5:

Summary of Significant Predictors of Individual Instrumental Activities of Daily Living

Predictor	Total Scale	Bills	Taxes/ Paperwork	Shopping	Games	Stove/ Beverage Prep	Meal Preparation	Tracking Current Events	Paying Attention	Remembering Dates	Traveling
Sex										*
Race	*	*	*	*	*			*	*	*	*
Education						*
Age											*
NPI-Q Severity	*	*	*	*	*	*	*	*	*	*	*
GDS Total	*						*	*		*
Benson Figure Copy								*
Benson Figure Delayed Recall	*	*	*	*				*	*	*	*
Animal Fluency
Vegetable Fluency		*	*	*			*
Phonemic Fluency
Digit Span Forward							*
Digit Span Backwards
Craft Story Delayed Recall	*	*	*	*	*		*	*	*	*	*
Multilingual Naming Test									*
Trail Making Test Part A (seconds)	*	*		*	*	*			*		*
Trail Making Test Part B (seconds)	*	*	*								*

Analysis of Completion vs. Non-Completion of Trail Making Test Part B

Next, the TMTB variable was recoded into those that could complete the TMTB task within the allotted time frame (300 seconds, n=1699) and those who could not. The reason for non-completion was listed as due to a cognitive or behavioral problem (n = 446). This variable was then substituted for the continuous TMTB variable in the equations above. Omnibus tests remained statistically significant for all models using this recoded variable (all p <.0001). In this model, TMTB dichotomized performances were statistically significant predictors of global and all individual FAQ items (all p <.05, not corrected for false discovery given the secondary nature of this analysis). Exponentiated beta weights can be interpreted as odds ratio. In this analysis, being able to complete the TMTB was associated with lower ratings of dependence for all ADL’s (all OR <1). The converse, that inability to complete the TMTB was a statistically significant predictor of greater ADL dependence also follows.

Discussion

Associations of Demographic Factors and Neuropsychological Measures with iADL Functioning in MCI and Dementia

In terms of global FAQ ratings, our findings again highlight the importance of considering non-cognitive factors such, as demographic and neuropsychiatric symptomatology when evaluating an individual’s functionality in any given task. In terms of overall FAQ rating, age and reported race were significant demographic predictors of iADL performance at the global and for most specific factors. This finding is similar to results from prior work from Garrett et al. (2013) and suggests the critical need to consider sociodemographic factors and increase the representation of individuals from diverse populations in large scale data collection to better understand the relationship between cognition and aging. In contrast, gender and education were not significant predictors of overall iADL functioning and only rarely related to specific functions, which contrasts to prior work (Brigola et al, 2019). This difference in results may reflect the relatively high education level of our sample and the limitations of the FAQ at capturing more nuanced daily activities that may differ between the genders. Neuropsychiatric factors again are demonstrated to be powerful predictors of day to day functioning, as established in prior studies as well (Delgado et al, 2019). Thus, our results again highlight the fact that iADL prognosis needs to consider more than cognition alone.

With this caveat in mind, the most consistent cognitive predictors of overall iADL function included visual and verbal delayed free recall and rapid visuomotor search and sequencing as indexed by Trail Making Tests A&B. Individuals with presumed Alzheimer’s disease tend to demonstrate problems with memory and executive functions (Karantzoulis & Galvin, 2011; Saxton, Smith, & Boller, 1991). Given that our current sample was predominantly composed of individuals with MCI or dementia presumed to be due to Alzheimer’s disease, the finding that episodic memory and executive type tasks were most strongly associated with iADL performance makes intuitive sense. It remains to be seen whether findings might differ in samples primarily composed of individuals with other etiologies (e.g., Lewy Body Disease) where presenting cognitive symptomatology differs. Additionally, it was beyond the scope of this paper to examine differences in the relationship between cognition and activities of daily living for MCI vs. dementia or among MCI subtypes. Past research indicates that there may be varying relationships between cognitive and functional performance, depending on diagnosis (Bangen et al., 2010; Marshall, Rentz, Frey, Locascio, Johnson, & Sperling, 2011).

A special note, however, is made of the fact that nearly a fifth of the sample had missing data on TMTB due to a cognitive or behavioral problem that precluded finishing the task within the 300 second time limit. As has previously been demonstrated in TBI (Pastorek, Hannay & Constant, 2004), the testability of an individual seems to have important explanatory power in its own right. Moreover, capacity to complete TMTB may help distinguish among individuals with different dementia etiologies (Kiselica & Benge, 2019). For clinicians and researchers, this finding again emphasizes the importance of careful consideration of why data may be missing in a sample or individual before drawing conclusions from the results. Future work could use categorical descriptions (able to complete or not complete) on TMTB to further address this question. That being said, the relatively strong relationship between TMTA&B and real-world functioning replicates a trend from prior meta-analyses (McAlister, Schmitter-Edgecombe, & Lamb, 2016; Lindbergh, Dishman, & Miller, 2016; Overdorp, Kessels, Claasen & Oosterman 2016; Royall et al 2007). While these measures can be critiqued for their lack of face validity (appearing like a common day to day tasks; Franzen & Wilhelm, 1996), our study again affirms that trail making tasks are robust cognitive correlates of a range of day to day functions.

Global vs. Specific iADLs

Delayed recall and rapid visuomotor search and sequencing (TMTA) were predictors of nearly all individual FAQ items. As others have argued, many iADLs implicitly require intact memory to be performed (Perneczky et al., 2006), such that the consistent relationship of memory scores to FAQ item scores is unsurprising. The result for the trailmaking test could also be expected, as slow processing speed is thought to have general negative impacts on cognitive and functional performance with age, in line with the processing speed theory of aging (Salthouse, 1996).

Subtle differences also emerged in the pattern of predictors for specific iADLs. For example, confrontation naming on the Multilingual Naming Test (MINT) was a significant predictor of only the item assessing paying attention to media (books/shows). It may be that prominent semantic loss makes this activity more difficult. For example, one could imagine that reading a newspaper or keeping up with a television show grows more difficult if one cannot understand or produce the words being spoken or recall the names of items being discussed. In a similar vein, figure copy which requires both top down and bottom up visuospatial processing and mental control (Possin et al., 2011), was more related to tracking current events. It may be that this measure captures organizational processes, rather than visuospatial abilities, which would explain the observed relationship. That being said, detailed experimental designs may better elucidate the cognitive mechanisms underlying performance in the iADLs assessed in this study.

The current findings may help to inform treatment efforts that attempt to improve performance in particular iADLs. For instance, identifying individuals at risk of a particular day-to-day ADL difficulty from their cognitive pattern and guiding them to focused interventions might improve outcomes. As an example, there has been discussion of using GPS technology to overcome deficits in visuospatial skills among those with dementia (Liu, Cruz, Ruptash, Barnard, & Juzwishin, 2017). Therefore, better understanding of how lab-based measures relate to real world functions may open the door to a precision medicine approach for cognitive rehabilitation.

We again emphasize that iADLs are driven by a complex interplay of sociodemographic, neuropsychiatric, and environmental, as well as cognitive factors (Marcotte et al., 2010. Thus, the observed relationships for specific daily functions are imperfect given the potential for additional, unmeasured explanatory variables to be considered. Furthermore, many of the iADL tasks as measured by the FAQ (e.g., “shopping alone for clothes, household necessities, or groceries”) are multi-component tasks with numerous but varying cognitive determinants. Thus, including measures of other domains (i.e., arithmetic ability; more detailed visuospatial functioning assessments; social cognition) might have produced a different pattern of associations, as has been suggested in prior studies (Sherod et al, 2009; Hird, Egeto, Fischer, Naglie & Schweizer, 2016; Insel, Morrow, Brewer, & Figueredo, 2006). Regardless, a battery that assesses core neuropsychological domains like delayed recall and sequencing/shifting in an MCI/dementia sample, appears to do a reasonable job predicting daily functioning across individual tasks.

The Ecological Validity of The UDS3NB

In addition to highlighting relationships of cognitive functions with specific iADL outcomes, the current study provides support for the UDS3NB as an ecologically valid set of measures of daily functioning in individuals with MCI and dementia. The purpose of the UDS3NB battery is to serve as a core set of measures for evaluating and tracking cognitive status in older adults with cognitive impairment, making these results are encouraging. Our current and prior (Kiselica & Benge, 2019) findings also suggest that it is important to consider missing data and reasons for missingness in relation to ADLs. Furthermore, it will be important to study other aspects of the battery in relation to functional outcomes, including factor scores, derived and discrepancy scores, and change scores (Devora, Beevers, Kiselica, & Benge, 2019; Kiselica, Kaser, Webber, Small, & Benge, 2020; Kiselica, Webber, & Benge, 2020a).

Many tasks in the UDSNB are commonly used by clinical neuropsychologists (i.e. trail making, animal fluency, number span), and the current results suggest that the royalty free, and brief battery is ecologically valid for use in MCI and dementia populations. Especially for the new and relatively understudied measures in the battery (i.e. Craft Story Memory and Benson figure copy and delay) these current results are encouraging for their continued use and development.

Limitations and Future Directions

The current sample was predominantly white, well-educated, and disproportionately composed of individuals with presumed Alzheimer’s disease diagnoses. Thus, the generalizability of the current results to a more diverse sample are unclear. For example, in samples with lower overall educational attainment, demographic factors might further attenuate the relationship between cognitive scores and daily activities. In a similar vein, if these analyses were run in groups composed of individuals with other primary cognitive problems (i.e. posterior cortical atrophy with its disproportionate visual impairments or primary progressive aphasia with its prominent language disturbances), measures in these different domains may prove more strongly predictive of real-world functioning (Moheb, Mendez, Kremen, & Teng, 2017). Such work should be the focus of future studies. It should also be noted that the current study included individuals with both MCI and dementia; thus, the iADL difficulties in this sample existed as a spectrum from none in some individuals with MCI to profound and total in dementia. In clinical practice, MCI and dementia are often considered as distinct entities. Modeling these groups separately may lead to different results and would likely need to include adjustment for significant skewness of the outcome measures, given greater within-group homogeneity. Finally, other demographic and health variables that could impact ADL measures were not considered in the current analyses.

The UDS3NB does not sample all cognitive constructs. For example, it lacks measures of prospective memory, which may be an even more important predictor of day to day functioning and errors (Tam & Schmitter-Edgecombe, 2013). In a similar vein, visual measures, such as the Useful Field of View test (Ball & Owsley, 1993) have been shown to significantly predict driving performance in dementia populations, over and above common neuropsychological measures (Myers et al, 2000). Thus, while the current results are reinforcing of the veridicality of common neuropsychological measures, continued exploration of other measures, domains, and clinically observed performance on lab-based ADL tasks are needed.

A final limitation regards the criterion variable, caregiver reported ADL difficulties. There are well documented limitations to caregiver reported ADLs (Gold, 2012), including potential for bias, insufficient periods of observation, and caregiver distress. Future studies employing a host of direct observational techniques in familiar environments are necessary to clarify how well the current findings generalize to different methods of measuring the criterion variable.

Conclusion

The current results suggest that core measures in the UDS3NB are reasonably correlated with day to day difficulties in this aging populations. In particular, delayed recall and processing speed/executive functioning tasks appear to be of relevance for predicting performance in activities of daily living. However, more work is needed to expand the assessment arsenal and tie cognitive test results to real world functioning, particularly in diverse samples.

Table 6:

Parameter estimates for the variable dichotomous completion vs. non-completion of TMTB in predicting instrumental activity performances

			95% Wald Confidence Interval				95% Wald Confidence Interval for Exp(B)
Variable	B	SE B	Lower	Upper	p	Exp(B)	Lower	Upper
Median FAQ Rating	−0.68	0.17	−1.01	−0.34	0.00	0.51	0.36	0.71
Bill Pay	−0.87	0.21	−1.28	−0.47	0.00	0.42	0.28	0.63
Taxes	−1.03	0.22	−1.46	−0.61	0.00	0.36	0.23	0.55
Shopping	−0.44	0.18	−0.79	−0.09	0.01	0.65	0.46	0.92
Games	−0.16	0.19	−0.54	0.22	0.40	0.85	0.58	1.24
Stove	−0.45	0.20	−0.84	−0.06	0.02	0.64	0.43	0.94
Meal Planning	−0.60	0.19	−0.97	−0.23	0.00	0.55	0.38	0.79
Events	−0.55	0.18	−0.90	−0.20	0.00	0.58	0.41	0.82
Paying Attention to Media	−0.44	0.18	−0.79	−0.08	0.02	0.65	0.45	0.92
Remembering dates	−0.84	0.18	−1.19	−0.49	0.00	0.43	0.30	0.61
Traveling	−0.71	0.18	−1.07	−0.36	0.00	0.49	0.35	0.70