Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2022 Apr 1.
Published in final edited form as: Am J Geriatr Psychiatry. 2020 Sep 7;29(4):395–404. doi: 10.1016/j.jagp.2020.08.019

The Computerized Functional Skills Assessment and Training Program: Sensitivity to Global Cognitive Impairment, Correlations with Cognitive Abilities, and Factor Structure

Philip D Harvey 1,2,3, Daniela Bolivar Forero 1, Lauren B Ahern 1, Lize Tiberica 3,4, Peter Kallestrup 3, Sara J Czaja 1,3,5
PMCID: PMC7936985  NIHMSID: NIHMS1628521  PMID: 32980252

Abstract

Objectives

We evaluated a novel computer-based functional skills assessment and training (CFSAT) program, which includes ecologically valid simulations of six everyday technology-related tasks. In this report, we describe the psychometric properties of the assessment in terms of sensitivity to impairment, factor structure and correlations with cognitive performance.

Design.

Cross-sectional baseline assessment prior to a treatment study.

Participants

Non-cognitively impaired (NC) older adults (n=62) and cognitively impaired older adults (n = 55), that ranged in age from 60 -86 years (M= 73.12), was primarily female (90%), and ethnically diverse (21% Hispanic, 52% African American). Participants were divided at baseline on the basis of MOCA scores and cognitive complaints.

Measurements.

The Brief Assessment of Cognition (BAC), app version, was used to measure cognitive performance and completion times on the 6 subtasks of the CFAST constituted the functional capacity measures.

Results

Performance on the CFSAT and BAC discriminated the two cognitive status groups. All of the cognitive domains on the BAC correlated significantly with all 6 CFSAT subtasks (all p<.01). Factor analyses suggested that the CFSAT and the BAC loaded on separate factors and regression analyses indicated that executive functioning and processing speed had the largest independent association with performance on the CFSAT.

Conclusions.

The CFSAT is sensitive to functional impairments seen in cognitively impaired older adults. Cognitive performance and CFSAT scores were related but nonredundant. Thus, the CFSAT appears to identify functional deficits that could be targeted with skills training interventions, likely augmented by pharmacological or computerized cognitive training interventions.

Introduction

Functional capacity measures are preferable to other strategies to assess abilities that underlie everyday functioning. Advantages of these measures include greater ecological validity than traditional neuropsychological tests, no reliance on self-report, and no requirement for informants [1]. Many previous functional capacity measures, such as the Everyday Problems Test [2] or the UCSD Performance-Based Skills Assessment [2] were paper and pencil simulations (e.g., maintaining a checkbook). However, as more everyday tasks are performed with technology (e.g., on-line banking and shopping), the ability to use technology for everyday tasks is increasingly important.

For example, retirement compensation is no longer provided by check, thus recipients require a bank account. Money management activities is commonly performed with automated teller machines (ATMs) and on-line banking. Many new devices, including computers, phones, and appliances do not come with a user’s manual. Many service companies such as utilities and transportation use a voice menu with multiple prompts. Previous research has suggested that many healthy older people have challenges interacting with these menus [4]. Another very relevant area of technology development is in health care, where applications extend across activities from scheduling appointments to refilling prescriptions. Scheduling and confirming healthcare appointments typically require using a voice menu to reach a human. Also, many medical groups and solo practitioners deliver prescriptions electronically to pharmacies, who generally prefer to confirm their receipt with electronic mail or text messages.

The challenges with performance of everyday activities using technology, common in healthy older people, are amplified in people with cognitive impairments. In previous studies, older non-impaired adults [5], individuals with mild cognitive impairment [MCI; 6], individuals with subjective cognitive complaints, [7] and older [8] and younger [9] participants with severe mental illness all manifested performance impairments on computerized assessments of functional skill, with evidence of correlation with cognitive performance measures providing evidence of the construct validity of these batteries.

A major concern with the development of functional capacity assessments is whether they are simply alternate versions of cognitive tests and are redundant with assessments of cognitive abilities. In schizophrenia, for example, it has been reported that certain functional capacity measures are so closely correlated with cognitive performance that the two domains are statistically unifactorial [10,11]. An additional concern is the nature of the correlation between cognition and functional performance. If only one cognitive domain is associated with performance on a functional capacity measure, it would be more efficient to use the cognitive test alone to predict functioning since many of these tests can be performed in as little as 90 seconds.

To address the above issues and the changing nature of everyday activities, our research team developed a computerized functional skills assessment and training program (CFSAT). This paper reports on the measurement properties of the assessment component of the program using that same sample of aging adults with and without cognitive impairments. These participants were assessed at baseline with six computer simulations of everyday tasks and a neuropsychological assessment of 6 different domains of cognition, ranging from verbal learning to reasoning and problem solving. The participants subsequently trained for up to 24 hours with training versions of the tasks, with those results being reported separately [12].

Our goals for the current analyses were multi-level. The first goal was to confirm the sensitivity of the functional skills assessment to pre-identified cognitive impairments. The second was to determine the convergence between cognition and functional skills. As multiple studies have shown a close relationship between cognition and functional capacity, this is critical construct validity information. Third, we were interested in determining whether the cognitive and functional capacity assessments, performed with technology and measuring multiple ability domains, were distinct. This is an important question that addresses the nonredundancy of the two domains. This question was addressed through with factor analyses testing different models of the structure of the cognition and functional skills assessments. Fourth, we were also interested in the factor structure of the functional skills measure on its own, because the overall assessment contains 6 apparently different functional domains. Previous studies of computerized functional skills measures have suggested a unifactorial structure despite assessing multiple skills [13]. Finally, we examined which of the 6 cognitive domains were correlated with functional capacity across the 6 tasks and a composite score. This analysis was aimed at determining whether a single cognitive domain explained a substantial proportion of the variance in the functional skills performance. Also, findings of correlation with multiple domains (e.g., verbal memory, working memory) could provide important information about which subgroups of older individuals might experience substantial challenges with functional skills tasks.

Methods

Study Design

A randomized treatment trial was conducted at three community centers in South Florida: City of Coral Gables Adult Activity Center, Village of Key Biscayne Community Centers and Charles Hadley Park Community Senior Center. The Coral Gables and Key Biscayne sites serve largely White/Non-Hispanic and Hispanic older adults and the population at Charles Hadley serves largely Black/African American older adults. Following a screening for basic eligibility and a baseline cognitive assessment, participants performed the baseline afunctional assessments described in this paper. The Institutional Review Board at the University of Miami Miller School of Medicine approved the protocol and all participants signed an informed consent form. Participants unable to provide consent on their own were not enrolled.

Participants

The participants were independently residing adults, aged 60 or older, with the ability to be examined in English, and having at least 20/60 vision with or without correction, able read a computer screen. The Montreal Cognitive Assessment (MOCA,14) was used to assess cognitive status. Participants were queried as to the current experience of memory problems. Patients were divided into normal cognition (NC) participants and those cognitive impairment CI. NC participants had a MOCA of ≥ 26 (adjusted for education to a cut-off of 24 for participants with low education [15] and no history of memory impairments or reports of bothersome memory complaints. CI participants had a MOCA ≥16 and ≤ 24-26 and reported a subjective history of memory complaints. Participants were compensated $30.00 for the baseline assessment.

Participants were excluded if they were deaf, or had a severe motor impairment (e.g., arthritis) that would negatively impact their ability to use a computer keyboard or mouse.

CFSAT Program

As described in Czaja et al. [12] the CFSAT program includes computer-based simulations of six everyday living tasks: using an ATM, a ticket kiosk, online banking, online shopping and prescription refill, telephone prescription refill, and medication management (comprehending instructions medication labels and organizing medication for a day and a week). The ATM, shopping, ticket kiosk, and online banking tasks are simulations of common systems and the telephone refill task is based on the voice menu of a local pharmacy chain. The tasks are presented in a multi-media format with graphic representations, text, and voice for the telephone prescription refill task. Each task consists of has multiple subtasks with progressively more challenging demands. For example, for the telephone prescription refill task, participants use a telephone keypad on the screen to call, refill prescriptions for bottles appearing on the screen, and request a pick-up time and date, etc. The banking task required entering login information, checking balances, setting up accounts, paying bills, and related tasks. Real time data was collected on completion time, errors, total correct and an efficiency measure, with time measured only while the participant was actively performing the tasks. The CFSAT program was delivered in person, on a touch screen or mouse format. Once the program was launched the participants proceeded through the tasks. After 4 or more errors on a subtask (e.g. repeatedly selected the wrong account) the program skipped ahead to the next subtask. Error feedback was delivered by repetition of the original instructions in a pop-up window. See supplementary Figure 1 for a visual depiction of the tasks.

Brief Assessment of Cognition: App version.

The Brief assessment of Cognition (BAC; 16; 17) is an assessment that measures critical dopmains of cognition known to be related to everyday functioning. The BAC App [18] delivers the same assessments as the paper BAC, but with computerized delivery for ease of administration and standardization. The cognitive domains assessed in the BAC app include:

Verbal Memory:

Subjects hear a list of 15 words to remember. Words are presented by the App in at a standard rate, eliminating the effects of rater variability.

Outcome measure:

Total number of words recalled across 5 learning trials.

Digit Sequencing:

Subjects are presented with randomly ordered auditory clusters of numbers (e.g. 936), which increase in length across trials from 2 digits to 8. Participants report the numbers in order, from lowest to highest, with 3 trials per length. The task terminates when all trials at a specific length are failed.

Outcome measure:

Number of trials with all items in the correct order.

Token Motor Task:

Subjects are presented with a virtual bowl and a supply of virtual tokens and swipe a token from each side of the tablet with each hand simultaneously and release them into the center container for 60 seconds.

Outcome measure:

Number of tokens correctly placed into the container 100.

Verbal Fluency.

Subjects are given 60 seconds to generate as many words as possible within the category “animals”. During Letter Fluency, subjects are asked to generate as many words as possible beginning with the letters F and S.

Outcome measures:

Total scores from both tasks are combined to produce the Verbal Fluency

Symbol Coding:

Subjects match numbers to non-meaningful symbols with the use of a key. Numbers are entered on the digital keypad and appear in the location below the corresponding symbol. Following instructions and practice, subjects are given 90 seconds to complete as many items as possible.

Outcome measure:

Number items completed correctly within 90 seconds.

Tower of London:

Subjects are shown two images presented on opposite sides of the screen. Each image shows a different configuration of 3 colored balls arranged on 3 pegs. The subject is required to accurately determine the total number of moves required to make the arrangement of balls identical to that of the opposing picture, while employing the standard rules employed in tower tests (balls are moved one at a time and balls on top of other balls must be moved first).

Outcome measure:

Number of correct responses.

Assessment Procedure.

All tests administered within the BAC App are completed under the supervision of a trained assessor. A female narrator operated by the App presents the instructions and the assessor can initiate repetition of instructions through the touch screen.

Protocol

A trained assessor screened interested participants for cognitive status with the MOCA and a short questionnaire on socio-demographic information and memory complaints. Participants who were eligible and consented to participate completed the baseline cognitive assessment using the BAC app. Participants were provided with an overview of the CFSAT program, which included a basic review of computer operations using protocols developed by Czaja and colleagues [19]. The assessor then launched the CFSAT program and participants completed the tasks in the following order: ticket kiosk, ATM, medication management, telephone prescription refill, online shopping & online prescription refill, and online banking. The baseline assessment was performed in groups of up to six concurrent participants and the assessment was proctored to answer questions from participants.

Data Analyses.

As noted above, task performance measures included task completion time and multiple other variables. The focus in these analyses was on time to completion, summed across subtasks within each domain, as this has proven to be the most informative measure for previous computerized functional capacity assessments [9]

Sensitivity to between groups differences was examined with t-tests, using Cohen’s d as an index of effect size. Pearson Correlations were computed between the 6 CFSAT tasks and the 6 BAC App domains. A principal component analysis (PCA) followed by rotated exploratory factor analysis (EFA) strategy was used to examine the structure of each of the assessments (CFSAT and BAC App) and determine if each assessment was uni-factorial or had a more complex factor structure. For each assessment, single factor and multifactorial solutions were computed. After determining the factor structures of CFSAT and BAC App independently, additional solutions were computed to test whether the CFSAT and BAC App constituted a single latent trait. Analyses were conducted across the entire sample, combining the CN and CI participants in order to capture a wide range of scores and to enhance to sample size to permit meaningful factor analyses.

Goodness of fit for the one-factor and two-factor solutions in the overall sample was quantified using maximum likelihood methods. As a two-factor model separating the CFSAT and BAC App is nested within the unifactorial model, the chi-square difference test (Likelihood ratio test) was used to examine the improvement in fit of the two-factor model compared to the unifactorial model [20]. When using these procedures small Chi-square values represent a better fit, so larger values reflect poorer fitting models. Since there are different tests for normal versus non-normal distributions, we tested the normality of the variables individually in the entire sample and used those results to inform our choice of univariate and multivariate tests.

In a final set of analyses, regression analyses were used to examine which of the BAC app domains was correlated with the composite on the CFSAT. This analysis was aimed at determining whether a limited set of cognitive measures explained the majority of the variance in performance on the CFSAT. First, the general relationship between the BAC app and CFSAT was evaluated with a simultaneous entry analysis, which was repeated with a stepwise procedure to identify the strongest predictors of CFSAT performance across the cognitive domains.

Results

Table 1 presents the sample demographic information, as a function of baseline cognitive status. One hundred and twenty-one participants fully completed the CFSAT baseline assessment on the CFSAT, however four participants were missing one or more scores on the BAC, resulting in 117 cases with no missing information. As can be seen in the table, the sample was racially and ethnically diverse and the cognitive status subgroups were similar in age, but differed in educational attainment. Table 2 presents the means and standard deviations for performance on the CFSAT tasks and the BAC app subtests. All variables were individually tested for normality in the entire sample. All of the BAC App domains had skewness and kurtosis statistics between −0.7 and +0.6. The CFAS manifested positive skewness, with higher scores being more common (Mean=1.40), accompanied by kurtosis statics indicating a similar shift toward poorer performance (M=1.56). When we examined the distributions of the NC group alone, none of the values for either statistic were greater than 0,6, suggesting the shift in the overall sample is due to poor performance of the CI sample, as we expected. As shown in the table, the NC subgroup performed significantly better than the CI subgroup on each of the CFSAT tasks and each of the BAC app subtests. Effect sizes for the differences (d) ranged from 0.67 to 1.02 for the CFSAT and from 0.49 to 1.18 for the BAC. Table 3 presents the correlations between the CFSAT tasks and the BAC app subtests. All correlations were significant at p<.01 or less.

Table 1.

Sample Demographic Information

Non- Cognitively
Impaired
(N = 62)
Cognitively
Impaired
(N = 55)
t p
Age (M, SD) 73.19 (6.39) 75.15 (6.39) −1.60 .11
Gender (N, %)
 Male 8(13) 3(5)
 Female 54(87) 39(95)
Years of Education Mean (SD) 15.67(2.41) 13.83 (2.73) 4.48 <.001
Race and Ethnicity (N, %)
 Hispanic 19(31) 8 (15)
 Non-Hispanic White 19(31) 6 (10)
 Non-Hispanic Black 21(33) 40(73)
 Asian 3 (5) 1(2)
MOCA (M, SD) 27.23 (2.05) 20.73 (3.05) 14.55 <.001
BAC Composite (M, SD) 0.53(0.92) −.60 (0.70) 7.44 <.001

Note. MOCA: Montreal Cognitive Assessment; BAC: Brief Assessment of Cognition

Table 2.

Performance on Six Computerized Functional Skills Tasks and Six Domains of Cognitive Functioning

Normal Cognition Cognitively Impaired
(n=62) (n=55)
M SD M SD t (116) P d
CFSAT
Ticket Task 773.46 323.34 1162.59 655.25 4.16 <.001 0.72
ATM Task 885.78 381.28 1467.85 858.43 4.86 <.001 0.82
Medication 840.40 312.16 1302.91 659.38 4.96 <.001 0.83
Management
Telephone Refill 619.48 225.57 895.41 524.99 3.79 <.001 0.67
Internet Banking 1076.25 448.25 1738.70 775.20 5.76 <.001 0.96
Ugreens Website 1004.65 452.14 1731.17 768.89 6.33 <.001 1.02
BAC App
Verbal Learning 38.70 9.99 25.91 7.35 7.77 <.001 1.18
Digit Sequencing 18.40 4.58 13.35 4.69 5.88 <.001 0.96
Token Motor 52.97 23.28 37.11 18.62 3.79 <.001 0.72
Verbal Fluency 51.87 15.81 36.20 12.38 5.90 <.001 1.00
Symbol Coding 26.49 9.60 20.59 9.91 3.26 <.001 0.49
Tower of London 14.10 4.32 9.98 5.11 4.72 <.001 0.80

Note. T-tests have 116 degrees of freedom.

CFSAT: Computerized Functional Skills Assessment and Training Program

BAC: Brief Assessment of Cognition.

Table 3.

Intercorrelations of Baseline Performance on the CFSAT Tasks and the BAC App Domains (Combined Sample, n=117)

Word List
Learning
Digit
Sequencing
Token
Motor Test
Verbal
Fluency
Symbol
Coding
Tower
of London
Ticket Purchase Task −.34** −34** −.39** −.32** −.42** −.44**
ATM Task −.37** −.41** −.38** −.33** −.49** −.46**
Medication Management −.46** −.47** −.42* −.38** −.44** −.50**
Telephone Refill −.36** −.28* −.27* −.24* −.40** −.39**
Internet Banking −.44** −.42** −.43** −.33** −.52** −.49**
U Greens Website −.37** −.40** −.34** −.41** −.37** −.45**

Note.

*

p<.01

**

p<.001; Pearson product Moment Correlations

CFSAT: Computerized Functional Skills Assessment and Training Program

BAC: Brief Assessment of Cognition

Individual measures:

The unrotated PCA suggested that both the CFSAT and the BAC App had a unifactorial, single component structure. Rotated EFA using maximum likelihood strategies with an oblique (oblimin) rotation was used to see if there were additional factors. For both solutions, the results were the same. Neither of the two-factor models produced a second factor with an eigenvalue above 0.5. On the CFSAT, none of the 6 tasks loaded higher on the second factor than the first and the same was true for all of the cognitive domains measured by the BAC App. The factor loadings for the single factor model for the CFSAT and the BAC App are presented in Table 4.

Table 4.

Individual Task Factor Loadings for CFSAT Tasks and BAC App Cognitive Domains:

CFSAT Loading
Ticket Task .92
ATM Task .93
Medication Management .93
Telephone Refill .86
Internet Banking .92
Ugreens Website .65
Eigenvalue 4.58
Variance Accounted for 76%
BAC App
Verbal Learning .81
Digit Sequencing .80
Token Motor .77
Verbal Fluency .77
Symbol Coding .72
Tower of London .80
Eigenvalue 3.64
Variance Accounted for 61%

Note. CFSAT: Computerized Functional Skills Assessment and Training Program

BAC: Brief Assessment of Cognition

Testing Combined Models:

The same strategy, a rotated EFA with maximum likelihood strategies, was used to test for unifactoriality for the CFSAT and BAC app. The factor analysis was repeated twice, first setting the program to extract one factor and then to extract two factors. The single factor and two-factor solutions for the analyses are presented in Table 5, clearly supporting a two-factor solution. The one factor-solution had a X2 goodness of fit index of 218.07 (54 DF), p<.001. The two-factor solution had a markedly better X2 goodness of fit index of 42.72 (43 DF), p=.48, suggesting that the model fit the data. The difference in the model fits was statistically significant, X2(11) =175.35, p<.001. Every CFSAT task loaded more strongly on factor 1 than factor 2 and every BAC app cognitive domain loaded more strongly on factor 2 than factor 1. The variance accounted for in the 2-factor model was 70%, as compared to 56% in the one-factor model.

Table 5.

Factor Models Examining one and two Factor Structures for the CFSAT and BAC App

CFSAT One Factor Model Two Factor Model
Component Loading Component Loading 1 Component Loading 2
Ticket Task .94 .94 −.11
ATM Task .98 .93 −.07
Medication Management .93 .83 −.08
Telephone Refill .81 .85 −.06
Internet Banking .92 .80 −.18
Ugreens Website .56 .41 −.31
BAC App
Verbal Learning −.48 −.49 .62
Digit Sequencing −.52 −.53 .60
Token Motor −.51 −.33 .53
Verbal Fluency −.42 −.43 .47
Symbol Coding −.55 −.39 .56
Tower of London −.54 −.49 .55
Eigenvalue 6.78 6.78 1.67
Variance Accounted for 56% 56% 14%

Cognitive Predictors of CFSAT Performance.

A single principal component derived from the factor analyses presented in table 4 was used to aggregate performance on the CFSAT tasks. A multiple regression analysis with simultaneous entry was used to examine the overall relationship between cognitive domains and CFSAT. The analysis was significant, F (6,110) = 10.41, p<.001, and there was 36% total variance shared between the CFSAT and BAC app. The analysis was repeated with a forward entry stepwise model, entering all 6 domains from the BAC app. This analysis was also significant, F (2,114) =28.79, p<.001. Two of the subtests entered the regression model, Tower of London, accounted for 27% of the variance, t(115)=3.76, p=<.001, followed by Symbol Coding, which accounted for an additional 7% of the variance, t(114)=3.43, p=.001.

Discussion

The results of this study suggest that the CFSAT program is correlated with, but separable from, measures of cognitive performance across multiple domains. Further, each of the CFSAT tasks were correlated with all of the cognitive abilities assessed by the BACS. A single-factor structure was discovered for the CFSAT, while previous findings of a unifactorial structure for the BAC app was also confirmed. Thus, the CFSAT measures abilities that goes beyond that measured by a neuropsychological assessment, suggesting that it is measuring cognitively relevant abilities in the domains of performance of technology-related functional skills.

The single factor solution for the CFSAT has some implications. First is that the subtests are tapping a general ability domain, albeit one that is associated with different cognitive abilities. Performance on the CFSAT is associated with more than a single cognitive ability, meaning that its meets previously described criteria for a meaningful functional capacity measure [21]. This also means that if investigators chose to use an abbreviated version of the CFSAT, individual subtests that were selected would be expected to be related to general technology-related functional skills construct.

The CFSAT was also found to be sensitive to differences in cognitive status such that individuals that were cognitively impaired performed worse on all six tasks. The effect sizes for CFSAT and BAC APP differences between the non-impaired and impaired samples were quite similar. Although the normal cognition sample performed better, the time to completion data suggested incomplete mastery of the tasks for both groups. This implies a need for training on these tasks even among non-impaired aging.

Our results converge with other [22-25] who have shown that individuals with a cognitive impairment, such as MCI, have difficulty with tasks assessing the performance of a wide range of everyday activities. The impact of cognitive impairment on everyday functional skills appears to increase with the severity, with greater impairments in patients with dementia compared to MCI [26-27]. In this study, participants selected for reduced MOCA scores showed wide ranging differences in performance on functional skills tasks compared to those with higher scores. The relative sensitivity of the CFSAT and BAC seemed generally similar. Also, there was variation in the differences between the NC and impaired samples, as would be expected because of variance in performance of everyday tasks in the real world. For example, the CI sample was most challenged by a verbal learning task and an Internet-based online banking task.

The correlation between individual cognitive domains and the composite measure of the CFSAT is consistent with previous research on cognition and functional skills. Previous studies have shown that cognitive abilities such as working memory, reasoning, processing speed, and attention are important for performance of a wide range of activities such as such internet-based information searching [28], medication management [29], and driving safely in later life [30]. Importantly our findings, similar to others [e.g., 31,32], demonstrate that neuropsychological tests are not sufficient to predict performance on everyday activities, underscoring the need for measures such as the CFSAT.

There are some limitations in this study. Because of limited resources, we did not have the opportunity to perform a full diagnostic assessment to formally confirmand MCI diagnosis. The sample sizes are unequal and the majority of the sample was female. We chose not to perform separate factor analyses in the cognitive performance subgroups because of the small number of participants relative to the number of variables, although our sample of 117 participants is adequate for the factor analyses involving 6 variables per assessment. Although all races and ethnicities in the local region are represented, the distribution of cognitive impairment is not consistent across racial/ethnic groups.

In conclusion, computerized simulations of everyday living skills, a measure of functional capacity, were sensitive to global cognitive impairments. These simulations were separable from but related to cognitive performance. Cognitive domains previously identified as important for the performance of everyday skills were found to be correlated with performance on the CFSAT tasks. The single factor solution for the CFAST tasks suggests that abbreviated assessments could be developed while still measuring the general contract of cognitively relevant, technology-related functional capacity.

Supplementary Material

1

Highlights.

1). What is the primary question addressed by this study?

This paper presents a study of the sensitivity to global impairment, construct validity, and factor structure of a computerized functional skills assessment and training program targeting technology-related functional skill performance in order adults.

2). What is the main finding of this study?

The assessment program was sensitive to global cognitive impairment measured with the MOCA, correlated, but not overlapping, with a neuropsychological assessment, and manifested an interpretable factor structure that was separable from that seen in a standard cognitive assessment battery.

3). What is the meaning of the finding?

The results of this study suggest that this program may be a valid way to identify technology-related functional skills deficits and target them for treatment with skills training programs, as both cognitively normal and impaired participants were able to complete the assessment while being challenged by the tasks contained in it.

Acknowledgements:

This research was supported by NIA grants 1 R21 AG041740-01 (Czaja and Harvey), and 1 R43 AG057238-01 (Kallestrup), as well as by a grant from the Wallace Coulter Innovation Foundation. Mr. Kallestrup is CEO of iFunction, Inc, and Drs. Harvey and Czaja are co-Chief Scientific directors and own equity in iFunction. Ms. Tiberica is a former employee of iFunction. Ms. Ahern and Ms. Forero have no biomedical conflicts of interest.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

  • 1.Moore DJ, Palmer BW, Patterson TL, Jeste DV. A review of performance-based measures of functional living skills. J Psychiatr Res. 2007;41(l-2):97–118. [DOI] [PubMed] [Google Scholar]
  • 2.Willis SL, Marsiske M Manual for the Everyday Problems Test. The Pennsylvania State University; University Park, PA, USA: 1997 [Google Scholar]
  • 3.Patterson TL, Goldman S, McKibbin CL, et al. UCSD Performance-Based Skills Assessment: development of a new measure of everyday functioning for severely mentally ill adults. Schizophr Bull. 2001;27(2):235–245. [DOI] [PubMed] [Google Scholar]
  • 4.Sharit J, Czaja SI, Nair S, Lee CC The effects of age and environmental support in using telephone voice menu systems. Hum Fact 2003, 45, 234–251. [DOI] [PubMed] [Google Scholar]
  • 5.Atkins AS, Stroescu I, Spagnola NB, et al. Assessment of Age-Related Differences in Functional Capacity Using the Virtual Reality Functional Capacity Assessment Tool (VRFCAT). J Prev Alzheimers Dis. 2015;2(2): 121–127. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Czaja SJ, Loewenstein DA, Sabbag SA, et al. A Novel Method for Direct Assessment of Everyday Competence Among Older Adults. J Alzheimers Dis. 2017;57(4): 1229–1238. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Atkins AS, Khan A, Ulshen D, et al. Assessment of Instrumental Activities of Daily Living in Older Adults with Subjective Cognitive Decline Using the Virtual Reality Functional Capacity Assessment Tool (VRFCAT). J Prev Alzheimers Dis. 2018;5(4):216–234. [DOI] [PubMed] [Google Scholar]
  • 8.Czaja SJ, Loewenstein DA, Lee CC, et al. Assessing functional performance using computer-based simulations of everyday activities. Schizophr Res. 2017;183:130–136. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Keefe RSE, Davis VG, Atkins AS, et al. Validation of a Computerized test of Functional Capacity. Schizophr Res. 2016;175(l-3):90–96. doi: 10.1016/j.schres.2016.03.038 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Harvey PD, Raykov T, Twamley EW, et al. Factor structure of neurocognition and functional capacity in schizophrenia: a multidimensional examination of temporal stability. J Int Neuropsychol Soc; 2013. 19(6):656–663. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Harvey PD, Aslan M, Du M, et al. Factor structure of cognition and functional capacity in two studies of schizophrenia and bipolar disorder: Implications for genomic studies. Neuropsychology 2006;30(l):28–39. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Czaja SJ, Kallestrup P, Harvey PD. Evaluation of a Novel Technology-Based Program Designed to Assess and Train Everyday Skills. Submitted for publication [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Harvey PD, Horan W, Atkins A, et al. Structure of Cognitive Performance and Functional Capacity in Schizophrenia: Evidence for Differences Across Functional Capacity Measures. Schizophr Res, in press [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Nasreddine ZS, Phillips NA, Bédirian V, et al. , TheMontreal Cognitive Assessment, MoCA: a brief screening tool for mild cognitive impairment. J Am Geriatr Soc, 2005, 53(4), 695–699. [DOI] [PubMed] [Google Scholar]
  • 15.Sink KM, Craft S, Smith SC, et al. ,. Montreal Cognitive Assessment and Modified Mini Mental State Examination in African Americans. J Aging Res, 2015, 872018. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Keefe RS, Goldberg TE, Harvey PD, et al. , The Brief Assessment of Cognition in Schizophrenia: reliability, sensitivity, and comparison with a standard neurocognitive battery. Schizophr Res 2004, 68(2–3), 283–297. [DOI] [PubMed] [Google Scholar]
  • 17.Keefe RS, Harvey PD, Goldberg TE, et al. ,. Norms and standardization of the Brief Assessment of Cognition in Schizophrenia (BACS). Schizophr Res 2008, 102(1–3), 108–115. [DOI] [PubMed] [Google Scholar]
  • 18.Atkins AS, Tseng T, Vaughan A, et al. Validation of the tablet-administered Brief Assessment of Cognition (BAC App). Schizophr Res. 2017;181:100–106. [DOI] [PubMed] [Google Scholar]
  • 19.Czaja SI, Boot WR, Charness N, et al. , Improving Social Support for Older Adults Through Technology: Findings From the PRISM Randomized Controlled Trial. Gerontologist 2018, 55(3), 467–477. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Raykov T, and Marcolides G 2011. Introduction to Psychometric Theory. London, Taylor and Francis. [Google Scholar]
  • 21.Green MF, Schooler NR, Kern RS, et al. Evaluation of functionally meaningful measures for clinical trials of cognition enhancement in schizophrenia. Am J Psychiatiy. 2011;168(4):400–407. [DOI] [PubMed] [Google Scholar]
  • 22.Burton CL, Strauss E, Bunce D,et al. . Functional abilities in older adults with mild cognitive impairment. Gerontology 2009, 55(5), 570–581. [DOI] [PubMed] [Google Scholar]
  • 23.Farias ST, Mungas D, Reed BR, H et al. , MCI is associated with deficits in everyday functioning. Alzheimer Dis Assoc Disord 2006, 20(4), 217–223. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Gomar JI, Harvey PD, Bobes-Bascaran MT, et al. , Development and cross-validation of the UPSA short form for the performance-based functional assessment of patients with mild cognitive impairment and Alzheimer disease. Am J Geriatric Psychiatiy 2011:, 79(11), 915–922. [DOI] [PubMed] [Google Scholar]
  • 25.Hughes TF, Chang CC, Bilt JV, et al. , Mild cognitive deficits and everyday functioning among older adults in the community: the Monongahela-Youghiogheny Healthy Aging Team study. Am J Geriatr Psychiatry 2012, 20(10), 836–844. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Goldberg TE, Koppel J, Keehlisen L, et al. , Performance-based measures of everyday function in mild cognitive impairment. Am J Psychiatry 2010,167(7), 845–853. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Goldberg TE, Harvey PD, Devanand DP et al. , Development of an UPSA Short Form for Use in Longitudinal Studies in the Early Alzheimer's Disease Spectrum. JPrev Alzheimers Dis. 2020;7(3): 179–183. [DOI] [PubMed] [Google Scholar]
  • 28.Czaja SI, Sharit J, Hernandez MA, et al. , Variability among older adults in Internet health information-seeking performance. Gerontechnology 2010, 9(1), 46–55. [Google Scholar]
  • 29.Stilley CS, Bender CM, Dunbar-Jacob J, et al. , The impact of cognitive function on medication management: three studies. Health Psychol 2010, 29(1), 50–55. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Edwards JD, Ross LA, Ackerman ML, et al. , . Longitudinal Predictors of Driving Cessation Among Older Adults From the ACTIVE Clinical Trial. The Journals of Gerontology: Series B, 200863(1), P6–P12. [DOI] [PubMed] [Google Scholar]
  • 31.Diehl M, Willis S, Schaie KW Everyday problem solving in older adults: observational assessment and cognitive correlates. Psychol Aging 1995, 10, 478–491. [DOI] [PubMed] [Google Scholar]
  • 32.Schmitter-Edgecombe M, Parsey C Cook DJ Cognitive correlates of functional performance in older adults: Comparison of self-report, direct observation, and performance-based measures. J. Int Neuropsychol Soc 2011, 17 (5), 853–864. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

1

RESOURCES