Abstract
Background
Recent findings indicate that impairments in functional performance do occur among individuals diagnosed with mild cognitive impairment (MCI). Most assessment strategies for everyday competence are associated with challenges with reliability, are typically in paper and pencil format, or require in-person administration by a trained professional.
Objective
This paper reports on a novel technology-based assessment battery of everyday competence that includes ecologically valid simulations of daily activities important to independence.
Methods
The sample included 85 non-cognitively impaired older adults aged 65+ and 62 older adults diagnosed with amnestic MCI (aMCI). Participants completed standard measures of cognitive abilities and the computerized battery of everyday tasks, which included simulations of a doctor’s visit; and medication and financial management tasks.
Results
The older adults with aMCI performed significantly poorer on all three tasks in the everyday task battery. Performance on these measures were also moderately correlated with standard measures of cognitive abilities and showed good test-retest reliability.
Conclusions
The results show that it is feasible to use a technology-based assessment battery of everyday tasks with both non-cognitively impaired older adults and older adults with MCI. The use of this type of battery can overcome many of the logistic constraints associated with current functional assessment protocols.
Keywords: Assessment, cognition, everyday competence, mild cognitive impairment, technology
INTRODUCTION
The number of adults aged 65 and older has increased to about 45 million people, and is projected to more than double by 2060. The numbers of those aged 85 or more is projected to triple from approximately 6 million in 2013 to nearly 15 million in 2040 [1]. Population aging, especially the increase in the “oldest old,” has resulted in a tremendous interest in understanding age-related changes in cognition and the implications for everyday functioning. As evidenced by the recent Institute of Medicine (IOM) Report, Cognitive Aging: Progress in Understanding and Opportunities for Aging [2], cognitive aging is an important public health issue. The genesis of this concern relates to individual and societal fears that normative age-related changes in cognition (e.g., declines in abilities such as working memory) and other types of cognitive changes such as mild cognitive impairment (MCI) will impede a person’s ability to perform activities essential to independent living, increase the need for care and support of older people, and diminish the quality of life for both older adults and families [2]. Although some controversy remains surrounding the construct of MCI, the prevalence of MCI is increasing globally [3].
Living independently requires the ability to carry out basic Activities of Daily Living (ADLs) such as bathing and eating; as well as Instrumental Activities of Daily Living (IADLS) such as managing finances; medications and other health-related tasks (e.g., following a doctor’s orders); finding one’s way; and maintaining a household. Mynatt and Rogers [4] discussed the importance of performance on enhanced activities of daily living (EADLs), which refers to the ability to adapt to changes in the environment and one’s routine and learn new ways of doing things (e.g., scheduling a doctor’s appointment using a patient portal). EADL types of activities are becoming increasingly important to independence in today’s technology driven environment.
A person’s ability to perform activities essential to daily living is generally referred to as everyday competence [5, 6]. We recognize that everyday competence is complex and involves an interaction among an individual’s physical, cognitive, emotional, and social functioning [5] and importantly, that behaviors occur within a physical and social environment. Our focus is on everyday cognitive competence. The literature [e.g., 7–12] clearly indicates that cognitive abilities are important to the performance of daily activities, especially those that are complex or involve new learning such as learning to use new technologies. In fact, studies have shown that declines in cognitive abilities place older adults at greater risk than younger adults for experiencing decrements in making financial decisions [7, 8]; medication management [9]; driving performance [10]; performance on technology-based tasks [11, 12]; and using patient portals of electronic medical records [13].
Based on the recent consensus definition [14], MCI is generally recognized as a pathological condition with varied forms (e.g., amnestic, multiple-domain), distinct from dementia, where there is some level of cognitive impairment and subjective memory complaints but performance on functional activities remain largely intact. However, although the results of outcome studies have been variable, depending on the assessment strategies and the selection of participants, recent findings indicate that MCI does result in impairments in functional performance. For example, Burton and colleagues [15] found that individuals with MCI performed more poorly than cognitively intact older adults on IADLs. Similarly, Aretouli and Brandt [16] found that individuals with MCI had more difficulties performing everyday activities than older adults without cognitive impairments. Developing strategies to enhance the everyday competence of older adults (both with and without cognitive impairments), and their ability to perform activities essential to daily living requires understanding the types of challenges they experience when performing these activities.
It is generally known that traditional measures of cognitive abilities are not adequate in terms of predicting a person’s ability to perform everyday tasks. Most assessment strategies for everyday competence are typically in paper and pencil format, are self-report, involve an informant (e.g., relative or clinician), or require in-person administration by a trained professional. For example in the Burton study [15], participants completed self-report measures of IADL performance [17] and the Everyday Problem Solving Test [18], a paper and pencil measure of IADL activities. Performance was also assessed by informants’ ratings. Ratings of participants’ performance were also obtained from relative informants. In the Aretouli and Brandt study [16], two informant-reported ratings of daily functioning were used, The Activities of Daily Living-Prevention Instrument [19] and The Informant Questionnaire on Cognitive Decline in the Elderly [20]. While these measures provide insight into a person’s performance abilities, there are documented limitations with these types of measures such as rater bias, difficulty in identifying informants, and additional costs related to the the inclusion of informants [21]. To overcome these shortcomings, performance-based measures such as the Revised Observed Tasks of Daily Living (OTDL-R) [22] which includes nine tasks, representing medication use, telephone use, and financial management, have been developed. Performance on the OTDL-R is related to performance on paper and pencil measures of everyday abilities and measures of cognitive function. Similarly, the Naturalistic Action Test (NAT) [23] is a standardized performance-based measure that evaluates ADL performance across several tasks: preparing toast and coffee, wrapping a gift, and packing a lunch box and a school bag. The NAT was developed for use with brain injured and stroke patients; however, Giovannetti and colleagues [24, 25] have used the NAT to understand performance deficits in patients with mild dementia and to guide interventions to remediate these deficits. While performance-based measures, such as the OTDL-R and the NAT are more ecologically valid, they require administration by trained observers and may not capture the complexity of current everyday activities or activities that are necessarily relevant with respect to older adults and everyday functioning.
This study evaluated the psychometric properties of three major subtests of a new computerized functional capacity assessment battery in older adults with aMCI, the UM Computer-Based Functional Assessment Battery (UMCFAB) (developed by Czaja and colleagues) which assesses performance on a variety of everyday activities including: ATM Banking/Financial Management, Prescriptions Refill via Telephone/Voice Menu System, and a Doctor’s visit (understanding medication and follow-up visit instructions). The UMCFAB has previously shown good-moderate test-test reliabilities, concurrent and discriminative abilities among older adults with persistent mental illness and cognitively normal elders [26]. In the current study, we examined: criterion validity by comparing performance on this battery relative to in-person standardized assessments of cognition and sensitivity to group differences between individuals with aMCI and non-cognitively impaired older adults. We also report test-retest reliability for sample of aMCI patients. Test-retest reliabilities for non-impaired older adults are reported elsewhere (see Czaja et al. [26]).
METHODS
Sample
The sample was drawn from two NIH funded conducted at the University of Miami Miller School of Medicine (UMMSM) studies and included 147 participants (41 males and 106 females). All participants provided informed consent, and the studies were monitored by the UMMSM Institutional Review Board. Both studies had the same inclusion/exclusion criteria, used similar assessment batteries, and recruited participants from the same community. The first study was a longitudinal NIH-funded investigation of the relationship between novel markers of cognitive function, functional status, and biomarkers in cognitively impaired and non-impaired elders. The second, a minority supplement to a NIH-funded Center Grant on aging and technology, examined cognitive and functional status among older adults with aMCI and without cognitive impairment. All participants were recruited from the South Florida community through advertisement and affiliations with clinics. Participants were required to be independent in their ADLs, and not meet DSM-5 criteria for a Major Neurocognitive Disorder. Participants were also excluded if they had a diagnosis of active major depression or any other major neuropsychiatric disorder such as schizophrenia or bipolar disorder.
In both studies, individuals were administered the Clinical Dementia Rating Scale (CDR) [27] and the Hopkins Verbal Learning Test-Revised (HVLT-R) [28] and assigned to diagnostic categories by a clinician or post-doctoral psychology fellows, under the supervision of a clinician. All of the fellows had formal training administering the CDR.
The criteria for cognitively normal (CN) subjects (n = 85) were: a) no evidence by extensive clinical evaluation or history of memory or other cognitive impairment; b) Global CDR of 0; c) HVLT-R immediate and delayed recall scored within normal limits relative to age and education related norms, which was typically less than 1.0 SD below normative values for this measure.
The criteria for MCI (n = 62) were: a) subjective memory complaints by the participant and/or or collateral informant; b) evidence by clinical evaluation or history of memory or other cognitive decline; c) Global Clinical Dementia Scale of 0.5; d) HVLT-R immediate or delayed recall scores of 1.5 SD or below normal limits relative to age and education related norms.
Measures
UM computer-based functional assessment battery
The functional tasks developed for the project included a: money management (Automatic Teller Machine, ATM); an interactive telephone menu/prescription refill task; and a doctors task where the participant had to interpret and remember medication information and instructions for a follow-up visit provided by a physician. The ATM task was a replication of a current operational ATM system. The prescription refill task was a simulation of a local chain pharmacy. The tasks are available in English and Spanish, in multi-media format and include graphic representations, video, text, and concurrent speech instructions. They are delivered in person on a touch screen or mouse format in a PC environment. Sample screens for the ATM task are presented in Fig. 1. Each task consisted of subtasks with sequential demands. For the ATM task, participants were sequentially instructed to check the balance in their savings account, transfer money between their checking and savings accounts, and to withdraw cash from their savings account. For the Prescription task, they had to call the pharmacy (using a telephone keypad on the screen), refill two different prescriptions (the pill bottles appeared on the screen), and request a pickup time and date. The Doctor’s task involved participants watching a video of an actor playing the role of a physician providing medical information, information about required medications, and appointment information. The participants viewed two different vignettes. The first vignette involves medication management where participants are advised about a medication regime for five fictitious medications and then required to correctly dispense pills into a weekly medication organizer and answer queries related to dosage and timing of medications and special instructions (e.g., food). The second vignette focuses on scheduling a follow-up appointment. Following delivery of instructions by the physician, participants respond to a set of queries about preparing for the next visit.
Fig. 1.
Sample screens for the ATM task.
Performance metrics for all three tasks include a rate measure (total correct/time), which reflected a measure of task efficiency. Performance metrics also included errors, total correct and task completion time for each of the tasks.
Prior to implementation, in each study the tasks were piloted tested with CN and MCI older adults using standard usability protocols developed by Czaja and colleagues for older adults and technology-based tasks [40].
General protocol
Following a telephone screening, eligible and interested individuals were scheduled for an assessment and participation at the UM Center on Aging. Participants completed an assessment of demographic and health variables, technology experience and component cognitive abilities, including Trail Making Test A and B, Digit Symbol Substitution Test, and the HVLT-R Test. All participants, irrespective of prior computer experience, were provided with basic computer training using protocols developed by Czaja and colleagues to ensure they would comfortably use the technology required for the functional tasks. This training included training on the components of computer hardware (e.g., display screen, keyboard) and basic activities such as the use of a touchscreen and scrolling. Participants practiced these basic operations during the training. They were given additional instruction if it appeared that they could not complete the practice tasks or if they asked for further help. The computer-based functional tasks were then administered (described below) and delivered in person on a touch-screen computer, in English or Spanish, according to the primary language and preference of the participant. A sample of the aMCI participants repeated the functional tasks at 4–6 weeks following the initial assessments.
Cognitive measures
The relationship between the UMCFAB and makers of component cognitive abilities was examined using had the following cognitive measures:
Hopkins verbal learning task revised [28]
The HVLT-R is a brief verbal learning and memory test with three learning trials and four recall trails. Trial four is a delayed recall trial. The following scores are derived: the number of words recalled on each of the four trials, total recall (the sum of the number of words recalled on trials 1–3), learning (the difference between the number of words re- called on trial 1 and the higher of trials 2 or 3), and percent retained (the number of words recalled at trial 4 compared to the higher of trials 2 or 3).
Category fluency test [29]
This measure taps verbal fluency and the ability to quickly retrieve exemplars of a specific category (animals, fruits, and vegetables) for one minute. The total task takes three minutes and the score is the total of all correct items generated from the three categories.
Block design test [30]
The Block Design Test is a speeded test of visuospatial skills and non-verbal abstract reasoning abilities.
Trail making test parts A and B [31]
Trails A taps simple visual scanning abilities and psychomotor speed while Trails B taps more complex visual scanning and mental flexibility and requires an individual to alternate between numbers and letters and shift cognitive sets.
Statistical analyses
Diagnostic groups were compared on the cognitive and functional measures using a series of one-way-analyses of variance (ANOVA) models. Chi-square analyses with Yates correction for discontinuity was performed on categorical data. Test-retest reliabilities were obtained using paired t-test and Pearson Product Moment Correlation Coefficients (PPM). PPM analyses were also employed to examine the association between cognitive and functional measures. Criteria for statistical significance was p ≤ 0.05.
RESULTS
Sample demographics
Table 1 depicts the sample, which included 62 patients diagnosed with aMCI and 85 non-cognitively impaired older adult participants. On average, the aMCI participants (M = 75.9 y; SD = 7.7) were older than the non-impaired adults (M = 72.0 y; SD = 8.3; p = 007). Although, the non-impaired adult participants included more females the difference between the groups was not significant and there were no difference between the groups in level of educational attainment.
Table 1.
Demographic information for elderly participants with aMCI and elderly cognitive normal participants
| Cognitively Normal (n = 85) | aMCI (n = 62) | F-Value or χ2 | p-value | |
|---|---|---|---|---|
| Age (Range = 59–91 y) | 71.98 (SD = 8.3) | 75.85 (SD = 7.7) | 7.66 | p < 0.007 |
| Education (Range = 2–20 y) | 14.45 (SD = 3.2) | 14.63 (SD = 4.0) | 0.09 | p = 0.77 |
| Gender (% Female) | 77% | 67% | 1.08 | p = 0.30 |
| Primary Language English | 91.5% | 85.5% | 0.68 | p = 0.41 |
Discriminative validity
As indicated in Table 2, the aMCI participants scored lower on correct responses for the ATM, Doctors Task and Prescription Refill Task with (p-values < 0.001). In addition, the aMCI participants made more errors and were slower to perform on the ATM task (p-values < 0.001). Finally, the aMCI participants made more errors on the Doctor’s task (p < 0.001) and were also slower in completing its required elements (p < 0.03). With regards to task efficiency, as Table 2 indicates the aMCI participants scored lower on all of the rate measures relative to the non-impaired group indicting diminished task efficiency.
Table 2.
Comparison between elderly participants with aMCI and elderly cognitive normal participant on functional measures
| Cognitively Normal (n = 82)* | aMCI (n = 56)** | F-Value | p-value | |
|---|---|---|---|---|
| ATM Correct Score | 9.30 (SD = 1.3) | 7.73 (SD = 2.1) | 28.76 | p < 0.001 |
| ATM Error Score | 3.57 (SD = 2.9) | 6.16 (SD = 3.4) | 18.52 | p < 0.001 |
| ATM Time | 287.09 (SD = 134.5) | 457.94 (SD = 282.3) | 21.56 | p < 0.001 |
| ATM Rate | 0.039 (SD = 0.02) | 0.025 (SD = 0.02) | 22.79 | p < 0.001 |
| Doctors Task Correct | 14.72 (SD = 4.3) | 11.32 (SD = 4.4) | 20.63 | p < 0.001 |
| Doctors Task Error | 7.84 (SD = 4.2) | 11.94 (SD = 4.6) | 27.60 | p < 0.001 |
| Doctors Task Time | 980.84 (SD = 266.6) | 1100.80 (SD = 320.6) | 5.71 | p < 0.02 |
| Doctors Task Rate | 0.016 (SD = 0.006) | 0.011 (SD = 0.005) | 33.91 | p < 0.001 |
| Prescription Refill Correct Score | 14.14 (SD = 2.2) | 12.22 (SD = 4.0) | 10.55 | p < 0.003 |
| Prescription Refill Error Score | 1.42 (SD = 2.0) | 1.16 (SD = 1.5) | 0.54 | p = 0.46 |
| Prescription Refill Time | 206.94 (SD = 70.6) | 207.87 (SD = 81.7) | 0.004 | p = 0.95 |
| Prescription Rate | 0.074 (SD = 0.02) | 0.064 (SD = 0.02) | 5.16 | p < 0.03 |
Due to technical problems, we were unable to extract the task data for three of the CN older adults and 6 of those with aMCI.
Test-retest reliabilities
We conducted test-retest reliabilities for different the different subtests of the UMCFAB for the aMCI participants (see Table 3). Time to Complete the ATM task, time of completion for the Doctors Task and time to complete the Prescription Task all had high test-retest reliabilities at p < 0.001 ranging from r = 0.61 to r = 0.80. Further, there were statistically significant test-retest reliabilities for number of correct responses and number of error responses ranging from r = 0.52 to r = 0.79 for the ATM and Doctor’s task. Finally, as shown in the table, there were statistically significant test-retest reliabilities for the rate measures (a measure of task efficiency) ranging from 0.59 to 0.92.
Table 3.
Test-retest reliabilities for older participants with aMCI
| Time 1 | Time 2 | Test-Retest Paired t test |
Test-Retest r | |
|---|---|---|---|---|
| ATM Correct | 7.5 (SD = 1.8) | 7.8 (SD = 1.9) | −0.79 | 0.52* |
| ATM Errors | 6.7 (SD = 2.8) | 5.5(SD = 3.1) | 2.40* | 0.79*** |
| ATM Time | 384.6 (SD = 292.3) | 443.2 (SD = 455.9) | −0.102 | 0.89*** |
| ATM Rate | 0.026 (0.02) | 0.028 (SD = 0.02) | −1.02 | 0.92*** |
| Doctors Task Correct | 3.44 (SD = 4.2) | 3.28 (SD = 4.2) | 0.12 | 0.54** |
| Doctors Task Errors | 9.4 (SD = 4.5) | 9.0 (SD = 4.1) | 0.61 | 0.64*** |
| Doctors Task Time | 1201.1 (SD = 282.0) | 1139.0 (SD = 261.7) | 1.28 | 0.61*** |
| Doctors Task Rate | 0.012 (SD = 0.005) | 0.012 (SD = 0.005) | −0.50 | 0.69*** |
| Prescription Correct | 12.8 (SD = 3.5) | 13.6 (SD = 3.5) | −1.31 | 0.20 |
| Prescription Errors | 0.7 (SD = 0.8) | 0.3 (SD = 0.5) | 1.30 | 0.09 |
| Prescription Time | 182.9 (SD = 63.8) | 164.8 (SD = 35.4) | 0.94 | 0.80** |
| Prescription Task Rate | 0.074 (SD = 0.02) | 0.083 (SD = 0.02) | −1.52 | 0.59* |
Number of aMCI subjects for test-retest are as follows: ATM (n = 17); Doctors Task (n = 24); Prescription Task (n = 12).
p < 0.05;
p ≤ 0.01;
p ≤ 0.001.
Concurrent validity
As depicted in Table 4, there was complete neuropsychological data for 34 aMCI participants who had the ATM test, 35 aMCI participants who received the Doctor’s Task and 28 aMCI participants who received the Prescription Refill Task. As shown in the table, better scores on Trails A were highly associated with correct responses on the ATM task and Prescription Refill Task, as well as decreased times to complete the Prescription Refill Task. Correct Score on the Doctors Task was related to higher scores on Block Design. We also examined these relationships for the efficiency scores by (Total Correct/total time). For ATM rate, there were high negative associations with Trails A time (p = 0.003) and Trails B Time (p = 0.009). Prescription Refill rate was also had strong negative associations with Trails A time (p = 0.001) and Trails B Time (p = 0.001). The rate measure on the Doctor’s task was positively associated with Block Design Performance (p = 0.002) and negatively correlated with Trails B Performance Time (p < 0.02). Scores on the HVLT-R and Category fluency did not relate to any of the functional task indices, indicting that functional abilities were more related to measures involving simple and complex visual scanning, psychomotor speed and executive dysfunction.
Table 4.
Correlations between cognitive and functional variables among participants with aMCI
| HVLT-R Total |
HVLT-R Delayed |
Trails A | Trails B | Category Fluency |
Block Design |
|
|---|---|---|---|---|---|---|
| ATM Correct Score | 0.19 | 0.22 | −0.59*** | −0.52*** | 0.24 | 0.05 |
| ATM Time | −0.05 | −0.20 | 0.31 | 0.33 | 0.01 | −0.24 |
| ATM Rate | 0.18 | 0.18 | −0.53** | −0.48** | 0.14 | 0.38 |
| Doctors Score Correct | 0.02 | 0.19 | −0.22 | −0.39* | 0.24 | 0.54*** |
| Doctors Score Time | −0.05 | 0.13 | 0.16 | 0.19 | −0.05 | −0.16 |
| Doctors Task Rate | 0.10 | 0.21 | −0.25 | −0.42* | 0.16 | 0.52** |
| Prescription Refill Score Correct | −0.02 | −0.09 | −0.47* | −0.32 | −0.03 | 0.11 |
| Prescription Refill Time | −0.22 | −0.15 | 0.43* | 0.47** | −0.23 | −0.31 |
| Prescription Refill Rate | 0.06 | 0.09 | −0.64*** | −0.58*** | 0.08 | 0.33 |
p < 0.05;
p ≤ 0.01;
p ≤ 0.001.
Functional impairment in aMCI participants
We also determined the percentage of aMCI participants with functional impairment based scores at or below the 5th percentile for cognitively normal participants on the UMCFAB depicted in Table 5. Forty percent of aMCI participants had difficulties with the ATM task while 26% and 13% of aMCI participants had difficulties with the Doctor’s task and the Prescription Refill Task, respectively.
Table 5.
Percentage impairment in aMCI participants as a function of meeting criteria for the lower 5th percentile relative to cognitively normal elderly
| Impaired at 5th Percentile |
|
|---|---|
| ATM Correct (7 or less) | 40% |
| ATM Errors (8 or more) | 34% |
| Doctors Correct (6 or less) | 14% |
| Doctors Errors (16 or more) | 26% |
| Prescription Correct (8 or less) | 13% |
| Prescription Errors (6 or more) | 2% |
DISCUSSION
The current investigation represents the first study to investigate the use of of a computerized performance-based functional assessment battery in older adults with aMCI. The subtasks represented in the battery were actual representations of real world systems and thus assess a person’s ability to actually perform these tasks. We choose these subtasks for inclusion in the study as the ability to perform financial and medication management tasks is critical to independence.
Overall, we found that performance on our battery (The ATM, Doctors Task, and Prescription Refill Task of the UMCFAB) successfully discriminated between aMCI and cognitively normal older adults. Moreover, UMCFAB tasks that were studied demonstrated good test-retest reliabilities and adequate concurrent validity with cognitive measures in those with aMCI. In addition, administration of this type of battery was acceptable with this population. All of the participants were able to use the software to perform the tasks.
The best measures on the UMCFAB battery were those of efficiency calculated as a rate measure by the formula Rate = (Correct Responses/Total Time). We found that better scores on Trails A and B, tests of simple and more complex visual scanning and psychomotor speed were most highly related to both rate on the ATM and Prescription Refill tasks. In contrast, a higher rate of efficiency on the Doctor’s Task was related to better Trails B and Block Design Scores. While these are both speeded tests, Block Design also taps visuospatial skills and non-verbal reasoning abilities. Interestingly, none of the memory or language measures were related to performance on the CFAB functional measures.
This expands upon the work of Czaja et al. [26] that indicates that the UMCFAB has good to moderate test-retest reliabilities and good concurrent validity among cognitively normal older adults and older adults with persistent mental illness such as schizophrenia.
Although MCI has been traditionally defined as the lack of functional deficits that interfere with everyday life, this is often a clinical judgment in which the practitioner does not have the ability to to directly observe higher order skills such as that assessed by the UMCFAB. Although MCI diagnostic criteria exclude substantial deficits [32] that presumably affect independent daily living; research has identified that mild but significant functional deficits are present in individuals diagnosed with MCI across a number of domains [33–35]. Previous research has suggested that deficits in remembering appointments and medications, and organizing tax records or other business affairs, appear to be present in approximately 40 to 55% of individuals with aMCI [36]. This is despite the fact that these individuals are deemed functionally independent by examining clinicians. Clearly, direct assessment of functional ability is essential for both early diagnosis of MCI and longitudinally tracking the progression of disease [37].
There are various ways to assess functional abilities. Commonly used measures of functional abilities typically rely on inventories completed by patients and their informants or paper and pencil approaches. Although there is value in these approaches, it is important to note that self-reports and even the reports of collateral informants in AD and other neurodegenerative diseases may be biased [38]. There are instances when a collateral informant has had limited opportunity to observe the patient performing the activities that are being assessed, has a cognitive or psychiatric disability themselves, or is governed by unrecognized biases that can lead to inaccurate evalautions. Thus, their report may be of little value in providing useful functional information.
Paper and pencil measures of functional abilities such as the Performance-Based Skills Assessment [39], the Everyday Problem Solving Test [18] are also commonly used and have associated strengths and weaknesses. While these types of measures are convenient and relatively easy to administer, they require individuals to perform tasks in a role-play format that does not capture the complexities that exist in the everyday world. Further, these measures do not reflect how many of the tasks assessed by the measure are currently performed in today’s technology oriented world [40]. Another method of assessing functional capacity is direct observation of the patient performing behaviors in a clinical setting using highly trained and reliable observers. This assessment method also yields important insight into a person’s capacity but generally requires trained professionals, which can be costly and logistically problematic. In addition observing behaviors in a clinical setting does not capture the complexity of the real world or tap actual real-world functional behaviors that may be aided by compensatory strategies and rely contextual cues within the environment.
For these reasons, actual computer-based simulations of real-world functional tasks comprising the UMCFAB such as using a similar ATM machine used by banks, a Prescription Refill task that is analogous to telephone automated systems increasingly employed by pharmacies across the country and an actual real-world simulation of a Doctor’s task, which requires an individual to comprehend physician instructions, makes the UMCFAB battery a valuable mechanism for identifying and tracking real-world functional deficits over time. In addition to the excellent discriminative ability shown in the current aMCI and cognitively normal older adult samples, recent research has also demonstrated that the UMCFAB is highly sensitive to deficits in older adults with severe mental illness such as schizophrenia [26]. Of note, our findings that 40% of older adults with aMCI had difficulties with commonly encountered tasks such as an ATM machine and over a quarter of individuals had difficulties comprehend instructions on the Doctor’s Task indicate that subtle performance-based cognitive deficits can be detected utilizing technology-based direct functional assessment. The UMCFAB battery has also been translated into Spanish and can also be easily expanded to include other tasks and translated into other languages.
In addition, the UMCFAB employs a sophisticated computer interface with simulations of actual real world activities, is well tolerated by older adults and does not necessitate a skilled examiner. Since the battery records responses in real time (in milliseconds) and is completely computer-based, these functional measures can be delivered and automatically scored over the Internet remotely. Thus this battery can be easily administered to older adults who live in remote areas and do not have access or transportation to remote tertiary medical centers. Further, by employing virtual reality and video technology, the UMCFAB tasks have been developed that realistically simulate everyday functions. This is particularly relevant given older adults are increasingly required to interface with technology such as financial transactions with an ATM machine, online bill paying, using increasing automated prescription refill services at pharmacies, and engaging in Internet-based information-seeking and purchasing of goods and services.
In summary, the UMCFAB computer-based functional tasks were well tolerated by older adults with and without aMCI and demonstrated excellent discriminative ability. These results and previous studies of older adults with chronic mental health conditions (see Czaja et al. [26]) indicate that performance-based computer simulations of real-world tasks may be important tools with respect to assessing functional performance. This is especially true in cases where there is not a reliable informant who has an opportunity to observe functional performance or may have biased estimates of performance or the older adult is unable to access a clinical setting. Overall, the results of this study confirm the potential importance of performance-based computerized functional assessment and these measures are worthy of further research.
There are some limitations associated with the current investigation. In this study, we had relatively modest numbers of participants, especially those who were non-impaired, who were followed cross-sectionally. We plan on following these and additional participants over a longer time period to determine the extent to which biological markers (such as MRI) and novel and traditional cognitive measures can track functional decline using UMCFAB measures. We are also studying the properties of UMCFAB tasks in different ethnic and cultural language groups. The battery included in this study was also limited to three subtasks. We are expanding the array of UMCFAB tasks available to include a way finding task and an online banking task.
Acknowledgments
This research was supported by the National Institutes of Health/National Institute on Aging (P01AG17211-0252, Sara J. Czaja, PI; 1 R01 AG047649-01A, David A. Loewenstein, PI). We would like to thank Sankaran Nair and Shih Hua Fu for their technical assistance with this project and Chin Chin Lee for her help with participant recruitment.
Footnotes
Authors’ disclosures available online (http://j-alz.com/manuscript-disclosures/16-1183r2).
References
- 1.U.S Department of Health and Human Services. Administration on Aging: A Profile of Older Americans. Washington, DC: 2014. [Google Scholar]
- 2.Blazer DG, Yaffe K, Liverman CT. Cognitive Aging: Progress in Understanding and Opportunities for Action. National Academies Press; Washington, DC: 2015. [PubMed] [Google Scholar]
- 3.Centers for Disease Control and Prevention. Self-reported increased confusion or memory loss and associated functional difficulties among adults ≥60 years – 21 states. MMWR. 2011;62:345–350. [PubMed] [Google Scholar]
- 4.Mynatt E, Rogers W. Developing technology to support the functional independence of older adults. Ageing Int. 2001;27:24–41. [Google Scholar]
- 5.Willis SL. Cognition and everyday competence. In: Schaie KW, Lawton MP, editors. Annual Review of Gerontology and Geriatrics. Vol. 11. Springer; New York: 1991. pp. 80–109. [Google Scholar]
- 6.Diehl M. Everyday competence in later life: Current status and future directions. Gerontologist. 1998;38:422–433. doi: 10.1093/geront/38.4.422. [DOI] [PubMed] [Google Scholar]
- 7.Agarwal S, Driscoll J, Gabaix X, Laibson D. The age of reason: Financial decisions over the life cycle and implications for regulations. Brookings Institution; Washington, DC: 2009. [Google Scholar]
- 8.Samanez-Larkin GG. Financial decision making and the aging brain. APS Obs. 2013;26:30–33. [PMC free article] [PubMed] [Google Scholar]
- 9.Stilley CS, Bender C, Dunbar-Jacob M, Sereika J, Ryan CM. The impact of cognitive function on medication management: Three studies. Health Psychol. 2010;29:50–55. doi: 10.1037/a0016940. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Ball K, Owsley C, Stalvey B, Roenker DL, Sloane ME, Graves M. Driving avoidance and functional impairment in older drivers. Accid Anal Prev. 1998;30:313–322. doi: 10.1016/s0001-4575(97)00102-4. [DOI] [PubMed] [Google Scholar]
- 11.Czaja SJ, Sharit J, Lee CC, Nair SN, Hernandez MA, Arana N, Fu SH. Factors influencing use of an e-health website in a community sample of older adults. J Am Med Inform Assoc. 2013;20:277–284. doi: 10.1136/amiajnl-2012-000876. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Sharit J, Hernandez MA, Czaja SJ, Pirolli P. Investigating the roles of knowledge and cognitive abilities in older adult information seeking on the web. ACM Trans Comput Hum Interact. 2008;15:50–55. doi: 10.1145/1352782.1352785. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Taha J, Czaja SJ, Sharit J, Morrow DG. Factors affecting usage of a personal health record (phr) to manage health. Psychol Aging. 2013;28:1124–1139. doi: 10.1037/a0033911. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Albert MS, Dekosky ST, Dickson D, Dubois B, Feldman HH, Fox NC. The diagnosis of mild cognitive impairment due to Alzheimer’s disease: Recommendations from the National Institute on Aging and Alzheimer’s Association workgroup. Alzheimers Dement. 2011;5:270–279. doi: 10.1016/j.jalz.2011.03.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Burton CL, Strauss E, Bunce D, Hunter MA, Hultsch DF. Functional abilities in older adults with mild cognitive impairment. Gerontology. 2009;55:570–581. doi: 10.1159/000228918. [DOI] [PubMed] [Google Scholar]
- 16.Aretouli E, Brandt J. Everyday functioning in mild cognitive impairment and its relationship with executive cognition. Int J Geriatr Psychiatry. 2010;25:224–233. doi: 10.1002/gps.2325. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Lawton M, Brody E. Assessment of older people: Self maintaining and instrumental activities of daily living. Gerontologist. 1969;9:179–186. [PubMed] [Google Scholar]
- 18.Willis SL, Marsiske M. Manual for the Everyday Problems Test. The Pennsylvania State University; University Park: 1993. [Google Scholar]
- 19.Galasko D, Bennett DA, Sano M, Marson D, Kaye J, Edland SD. Alzheimer’s Disease Cooperative Study ADCS Prevention Instrument Project: Assessment of instrumental activities of daily living for community-dwelling elderly individuals in dementia prevention clinical trials. Alzheimer Dis Assoc Disord. 2006;20:S152–S169. doi: 10.1097/01.wad.0000213873.25053.2b. [DOI] [PubMed] [Google Scholar]
- 20.Jorm AF, Jacomb PA. The Informant Questionnaire on Cognitive Decline in the Elderly (IQCODE): Socio-demographic correlates, reliability, validity and some norms. Psychol Med. 1989;19:1015–1022. doi: 10.1017/s0033291700005742. [DOI] [PubMed] [Google Scholar]
- 21.Gitlin L, Czaja SJ. Behavioral Intervention Research: Designing, Evaluating and Implementing. Springer Publishing Company; New York: 2015. [Google Scholar]
- 22.Diehl M, Marsiske M, Horgas AL, Rosenberg A, Saczynski JS, Willis SL. The revised observed tasks of daily living: A performance based assessment of everyday problem solving in older adults. J Appl Gerontol. 2005;24:211–230. doi: 10.1177/0733464804273772. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Schwartz MF, Segal ME, Veramonti T, Ferraro M, Buxbaum LJ. The Naturalistic Action Test: A standardised assessment for everyday-action impairment. Neuropsychol Rehabil. 2002;12:311–339. [Google Scholar]
- 24.Giovannetti T, Libon DJ, Buxbaum L, Schwartz MF. Naturalistic action impairments in dementia. Neuopsychologia. 2002;40:1220–1232. doi: 10.1016/s0028-3932(01)00229-9. [DOI] [PubMed] [Google Scholar]
- 25.Giovannett T, Bettcher BM, Libon DJ, Brennan L, Sestito N, Kessler RK. Environmental adaptations improve everyday action performance in Alzheimer’s Disease: Empirical support from performance-based assessment. Neropsychology. 2007;21:448–457. doi: 10.1037/0894-4105.21.4.448. [DOI] [PubMed] [Google Scholar]
- 26.Czaja SJ, Loewenstein DA, Lee CC, Fu SH, Harvey PD. Assessing functional performance using computer-based simulation of everyday activities. Schizophr Res. 2016 doi: 10.1016/j.schres.2016.11.014. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Morris JC. The Clinical Dementia Rating (CDR): Current version and scoring rules. Neurology. 1993;43:2412–2414. doi: 10.1212/wnl.43.11.2412-a. [DOI] [PubMed] [Google Scholar]
- 28.Benedict RHB, Schretlen D, Groninger L, Brandt J. Hopkins Verbal Learning Test-Revised: Normative data and analysis of inter-form and test–retest reliability. Clin Neuropsychol. 1998;12:43–55. [Google Scholar]
- 29.Monsch AU, Bondi MW, Butters N, Salmon D, Katzman, Thal LJ. Comparisons of verbal fluency tasks in the detection of dementia of the Alzheimer type. Arch Neurol. 1992;49:1253–1258. doi: 10.1001/archneur.1992.00530360051017. [DOI] [PubMed] [Google Scholar]
- 30.Wechsler D. Wechsler Adult Intelligence Scale–Fourth Edition. Pearson, San Antonio: 2008. [Google Scholar]
- 31.Reitan RM. Validity of the trail-making test as an indicator of organic brain damage. Perception Motor Skills. 1958;8:271–276. [Google Scholar]
- 32.Knopman DS, Boeve BF, Petersen RC. Essentials of the proper diagnoses of mild cognitive impairment, dementia, and major subtypes of dementia. Mayo Clin Proc. 2003;78:1290–1308. doi: 10.4065/78.10.1290. [DOI] [PubMed] [Google Scholar]
- 33.Farias ST, Mungas D, Reed BR, Harvey D, Cahn-Weiner D, Decarli C. MCI is associated with deficits in everyday functioning. Alzheimer Dis Assoc Disord. 2006;20:217–223. doi: 10.1097/01.wad.0000213849.51495.d9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Reppermund S, Brodaty H, Crawford JD, Kochan NA, Draper B, Slavin MJ, Trollor JN, Sachdev PS. Impairment in instrumental activities of daily living with high cognitive demand is an early marker of mild cognitive impairment: The Sydney memory and ageing study. Psychol Med. 2013;43:2437–2445. doi: 10.1017/S003329171200308X. [DOI] [PubMed] [Google Scholar]
- 35.Weston A, Barton C, Lesselyong J, Yaffe K. Functional deficits among patients with mild cognitive impairment. Alzheimers Dement. 2011;7:611–614. doi: 10.1016/j.jalz.2010.12.011. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Brown PJ, Devanand DP, Liu X, Caccappolo E Alzheimer’s Disease Neuroimaging Initiative. Functional impairment in elderly patients with mild cognitive impairment and mild Alzheimer disease. Arch Gen Psychiatry. 2011;68:617–626. doi: 10.1001/archgenpsychiatry.2011.57. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Rockwood K. The measuring, meaning and importance of activities of daily living (ADLs) as an outcome. Int Psychogeriatr. 2007;19:467–482. doi: 10.1017/S1041610207004966. [DOI] [PubMed] [Google Scholar]
- 38.Loewenstein DA, Argüelles S, Bravo M, Freeman RQ, Argüelles T, Acevedo A, Eisdorfer C. Caregivers’ judgments of the functional abilities of the Alzheimer’s disease patient: A comparison of proxy reports and objective measures. J Gerontol B Psychol Sci Soc Sci. 2001;56:78–84. doi: 10.1093/geronb/56.2.p78. [DOI] [PubMed] [Google Scholar]
- 39.Patterson RC, Goldman S, McKibbin CL. UCSD performance-based skills assessment: Development of a new measure of everyday functioning for severely mentally ill older adults. Schizophr Bull. 2001;27:235–245. doi: 10.1093/oxfordjournals.schbul.a006870. [DOI] [PubMed] [Google Scholar]
- 40.Czaja SJ, Sharit J. Practically relevant research: Capturing real world tasks, environments, and outcomes. Gerontologist. 2003;43:9–18. doi: 10.1093/geront/43.suppl_1.9. [DOI] [PubMed] [Google Scholar]