A Computerized Functional Skills Assessment and Training Program Targeting Technology Based Everyday Functional Skills

Abstract

Today, many functional skills are technology-based, so development of a technology-based training program has broad importance. Here we present a computerized functional skills training program that was paired in half of the participants with a commercially available cognitive training (CCT) program.

Non-impaired older individuals (NC) aged 60+ (n=45) and similarly aged individuals with mild cognitive impairment (MCI; n=50) were randomized to receive 12 weeks of twice-weekly computerized functional skills training (CFST) or 12 weeks of twice-weekly sessions split between CCT and CFST. Skills trained were use of an ATM; internet banking; ticket kiosk; telephone and internet prescription refill; medication management; and internet shopping. As with previous functional capacity assessments, we focus on completion time for each simulation.

51 participants completed the training program, either by mastering all 6 tasks (34) or completing 12 weeks of training. 44 more participants completed 4 or more training sessions so they were also analyzed for improvement up to their last training session. Completion time for all 6 tests significantly improved from the baseline assessment to the final training session in both groups of participants (all p<0.001 with an average improvement in task completion time of 45%). Further, there was no differential improvement in MCI and NC in the 6 tests from baseline to end of training (all t<1.66, all p>0.12). Finally, combined CCT plus CFST did not differ from CSFT alone on any of the percent-change score measures (all t<1.64, all p>0.11).

Both NC and MCI groups evidenced substantial improvements in performance. CCT supplementation led to similar functional gains with half as many training sessions. The NC participants proceeded through the training fairly rapidly even without CCT supplementation; MCI participants required more training but learned equivalently. These findings suggest that even in cases with memory impairments, functional skills can be efficiently learned with training.

SUMMARY:

This training protocol uses computerized training to teach technology-related everyday functional skills. These skills include financial skills, travel and transit, as well as medication management.

INTRODUCTION:

Many contemporary functional tasks are performed using technology. This includes banking and other financial management tasks, travel and transit tasks, and healthcare management. The challenges of using technology daily are amplified in older people whose lifetime exposure to technology may be more limited. Technology-based tasks can also be cognitively demanding. Older people and people with different challenges, such as severe mental illness^1-2 or cognitive deficits, may not have the financial or cognitive resources to use challenging technologies. These individuals have difficulty negotiating today’s digitized world, which is a threat to their independence.

Our previous research has shown that many healthy older people also have problems in performing everyday functional tasks efficiently³. Further, people with mild cognitive impairment have proportionately greater challenges performing these tasks⁴. Our research has shown that component cognitive abilities in healthy aging⁴, severe mental illness⁵ and MCI populations, is consistently correlated with the ability to perform veridical simulations of everyday functional tasks. Thus, cognitive ability is a rate limiter for people’s ability to initially perform, and learn, functional tasks using technology. This problem is exacerbated by the fact that many of these tasks, such as using an ATM, are thought of as “walk up tasks” or tasks that are intuitive and do not require training.

Currently, training in the performance of technology-relevant everyday functional is not systematically delivered. New technology, such as iPhones, tablets, and computers, are commonly delivered without any instructions for their use. Instructions for using websites are not commonly available. For example, the New York Metropolitan Transit Authority (MTA), website has no instructions on using their ticket kiosks other than instructions on how to refill a Metrocard.

The incremental deficits in cognitive performance can be partially addressed with computerized cognitive training (CCT) for some populations. Recent data have suggested that both healthy older people⁶ and people with MCI⁷ can make cognitive gains with CCT on the cognitive abilities targeted by the training. Thus, it would be expected that performance of functional tasks would also be facilitated with CCT efforts. However, the widely reported success of CCT in both older adult and MCI populations has not been accompanied by spontaneous improvements in the ability to perform novel functional skills. While previously acquired skills, such as driving⁸, can be facilitated through CCT, there is no evidence across populations that CCT alone is enough to lead to acquisition of novel functional skills.

CCT has also been shown to have protective effects against development of dementia, at least with a limited set of training routines. For instance, the ACTIVE trial showed that computerized speed training was associated with persistent improvements in cognitive performance that could be detected at a 10-year follow-up⁹. A subsequent follow-up study also reported a 30% decrease in dementia associated with all causes at the 10-year period as well¹⁰. As a result, since certain types of CCT appear to result in cognitive benefits among older people, the combination of CCT and computerized functional skills training should result in improvements in functional skills.

Thus, the current program involved the development of a set of ecologically valid functional skills tasks that are typically performed using some sort of technology, either internet, telephone, or in-person on a device such as an ATM. The tasks are presented in Table 1 and were chosen as important to living independently. Within the program, these tasks are performed in fixed difficulty, non-training formats first. Each task has multiple different graduated sub-tasks, which vary in difficulty of functional demands. After completion of the 6 fixed difficulty tasks, all participants are then trained on the Computerized Functional Skills Training (CFST) training simulations. These simulations provide direct performance feedback to the participant. Sample feedback is presented in Table 2. There is no involvement of a human trainer, nor does a human provide feedback. When a participant makes an error on a subtask, graduated feedback is provided across that increases in corrective information. For example, in the ATM task, if an individual makes an initial error inputting their pin number, basic corrective information is provided; if they make the same error a second time more corrective information is provided.

Table 1:

Computerized Functional Skills Training (CFST) Tasks

ATM Banking

Internet Banking

Ticket Purchase with Kiosk

Medication Management and Daily Organizer

Interactive Telephone Voice Menu for Prescription Refill

Internet Prescription Refill and On-Line Shopping

Table 2:

Error Feedback

Error 1 Repeat Instruction:	Your Pin is 1234. Please enter your PIN
Error 2. Guidance:	Your PIN is 1234. Please use the keypad to enter your PIN
Error 3 Direction:	Your PIN is 1234. Enter 1, followed by 2, followed by 3, followed by 4
Error 4 Demonstration	The four keys are lit up in sequence and the participants is instructed to touch them when they light up.

After 4 errors, the task proceeds to the next training step. However, when the participant returns to train later, this step is re-trained until it is passed. Each of the training modules is considered complete and the participant graduates after performing the entire task two times without any errors.

The study includes two research participant groups: (1) cognitively normal (CN) healthy older and (2) medically healthy older people with mild cognitive impairment (MCI). CN was defined as a Montreal Cognitive Assessment (MOCA)¹¹ score of 26 or more and no cognitive complaints. MCI was defined with a systematic assessment that included the MOCA, assessments of subjective complaints, and assessment with a structured neuropsychological assessments. Participants were excluded if their cognitive performance reflected impairment greater than MCI. Training was conducted on Windows computers although the software can be deployed in iOS as well. Training was proctored in a ratio of approximately 6 trainees per proctor.

The goal of the study is to determine 1) if CFST is efficacious in healthy older people, defined by improvements in performance of computer-based functional skills; (2) the relative efficacy of functional skills training for those with MCI as compared to those who are non-impaired; and (3) whether the provision of CCT enhances the CFST and if there is a differential effect for the MCI as compared to the NC.

PROTOCOL:

The study is a randomized trial, where half of the research participants (stratified by cognitive status) are randomized 1 to 1 to receive computerized cognitive training (CCT) on the Double-Decision task from the Posit Science Brain HQ training software combined with the CFST and the others are trained on the CFST alone. This research was reviewed and approved by the University of Miami Institutional Review Board and all participants provided signed informed consent.

1. Preparation

1.1. Obtain written informed consent

1.2. Screen the participant with the Montreal Cognitive Assessment (MOCA)¹¹.

1.2.1. Have the participant connect numbers and letters in sequence

1.2.2. Show the Participant 3 pictures of animals and have them name them

1.2.3. Read 3 verbal learning words and have participant recall them

1.3. Perform a baseline cognitive assessment using the tablet version of the Brief Assessment of Cognition (Figure 1)¹². Administer all of the subtests on the tablet app. The subtests are Verbal learning and Memory, Digit Sequencing, Token Motor Task, Symbol Coding, Verbal Fluency Examinations, and Tower of London.

Figure 1:

Representative stimuli from the BAC App showing the tower of London and Symbol coding subtests.

1.3.1 Present the overall instructions for the assessment on the app.

1.3.2 Present the instructions for the Symbol Coding Task.

1.3.3 Have the participant do practice coding.

1.3.4 Visualize the participant doing the coding task for 15 seconds.

1.3.5 Present the instructions for the Tower of London Task.

1.3.6 Have the participant do the practice items.

1.3.7. Visualize the participant solving the first test item.

1.4. Assess the participant with 6 different functional tasks (Figure 2) in a fixed difficulty format.

Figure 2:

The 6 CFST training tasks. These tests include Ticket Purchase, ATM and Internet Banking, Phone and Internet Prescription Refill and Shopping, and Medication Management

NOTE: During the fixed difficulty assessment, there is no training provided, and if a participant makes an error, the instructions are simply repeated. When a participant completes a task, they proceed to the next one. When all of the tasks are completed, training begins. The representative task demands of each task are presented below.

2. Task demands for fixed difficulty and training simulations

2.1. Launch Ticket Purchase Task.

2.1.1. Select purchase a new ticket.

2.1.2. Select Purchase a single ride ticket.

2.1.3. Check the balance on the transit card.

2.1.4. Add $60.00 to the transit card.

2.2. Launch the telephone refill task.

2.2.1. Dial the number for the pharmacy.

2.2.2. Enter the prescription number.

2.2.3. Select the time to pick up the medication.

2.3. Launch the ATM banking task.

2.3.1. Enter the PIN to start the session.

2.3.2. Check the balance in the checking account.

2.3.3. Withdraw $180.00 from the checking account.

2.4. Launch the medication label comprehension task.

2.4.1. Choose the correct time of day to take the medication.

2.4.2. Correctly identify how many pills to take each day.

2.4.3. Launch the medication organizer subtask.

2.4.4. Pack up a day’s worth of medications.

2.5. Launch the internet banking task .

2.5.1. Enter the User ID and password.

2.5.2. Check the balance in the checking account.

2.5.3. Make a transfer of $15.00 from savings to checking.

2.6. Launch the internet prescription refill and online shopping task.

2.6.1. Enter the userID and password.

2.6.2. Verify your identity by selecting the correct city (Miami), Street (Micapony) and Car color (Blue)

2.6.3. Select Prinivil and no generic equivalent.

2.6.4. Select Correct date and time for pickup.

3. Training after the fixed difficulty assessment

3.1. CFST alone condition

3.1.1. Explain the training task to the participant, stating that all 6 tasks will be trained

3.1.2. Launch the ATM banking task.

3.1.3. Enter PIN.

3.1.4. Select transaction ( check balance in checking).

3.1.5. Visualize the process of program-delivered feedback for errors.

3.1.6. Present 4 errors in sequence to demonstrate the feedback process.

3.2. CCT + CFST Condition

NOTE: The CFST training is the same as CFST alone but is for 30 minutes only.

Brain HQ DoubleDecision is the computerized cognitive training task in the combined condition.

Figure 3 depicts the Double-Decision Task.

Figure 3:

Brain HQ Double Decision Task Stimulus.

3.2.1. Explain the process of Training on Computerized cognitive training as well as skills training

3.2.2. Launch Brain HQ double Decision Task

3.2.3. Perform the practice items

3.2.4. Perform the actual test items

3.2.5. Visualize this process for about 30 seconds

4. Post-training assessment

NOTE: After graduation from all six training tasks or completion of 24 sessions, participants complete a post-test.

This next section should be presented as a verbal description of the post training assessments

4.1. Participants will perform different versions of all six trained tasks.

4.1.1. They will be told that the tasks are the same, but the content demands are different.

4.2. Participants will perform the BAC App assessment again.

REPRESENTATIVE RESULTS:

The patient flow is shown in Table 3. The first subject was screened on July 14, 2018. Screened participants included 78 females and 76 males, of whom 53 (33%) were Latino and 52 (33%) are of African American descent. Mean levels of educational attainment were similar across ethnic groups and averaged 15 years. However, there were cases who were training who had less than 8 years of education. 131 cases completed the eligibility assessment, with 16 cases excluded because of possible dementia and 4 cases excluded because of motor or vision problems. Of those 121 randomized 46% (n = 56) were HC and 54% (n = 65) were diagnosed at MCI and 50% in each group was randomized to CFST training alone. 51 cases completed training with 34 of them graduating, which translates into perfect performance twice in sequence on 30 individual subtasks. 44 cases are still training and 15 cases are waiting to train. The drop-out rate after any training was 9%. The mean MOCA score was 28.38 (SD=1.70) for the NC group and 22.68 (SD=3.02) for the MCI group.

Table 3:

The CONSORT Diagram for the clinical trial

Screened and signed consent	154
Ineligible	20
Withdrew before baseline	4
Baseline not completed	9
Completed baseline assessment	121
Completed training	51	42%
Still training	44	36%
Waiting to train	15	12%
Dropped out after training	11	9%

Figure 4 presents the training results for completers and Figure 5 presents the results for all participants who trained. These data are presented in terms of time to completion, although several other dependent variables are collected. Paired t-tests found that completion time for all 6 tests significantly improved from the baseline assessment to the final training assessment in completers (all t>8.16, all p <.001). Further, none of the simulations improved differentially in the NC and MCI samples, as indexed by percentage of improvement from baseline to end of training, all t<1.66, all p>.12. Finally, combined CCT plus CFSAT did not differ from CSFAT alone on any of the %-change score measures: All t<1.44, all p>.16. CCT supplementation led to similar CFSAT gains with half as many CFSAT training sessions and this improvement was consistent in the MCI and NC groups in both conditions. All t-tests for the six tests, across the MCI vs. NC samples and the skills training only vs. combined assessments are in Table 4. Both groups evidenced substantial improvements in performance on alternative versions of the assessment tasks. Importantly, NC participants who received skills training alone required an average of only 6 trainings per module to perfect their performance. The participants randomized to CCT & CFSAT trained an average number of 11 days on CCT, achieving an average of 50 levels per participant.

Figure 4:

Training gains from baseline in individuals who completed training. This is depicted in terms of time to completion at the first assessment, the final training session, and the alternate form of the simulations, across the 6 training tasks.

Figure 5.

Percentage Gain Per training Session: Completers and Still Training

Table 4.

T-Test results comparing Performance Across Participant Groups and Training Strategies

ATM Banking				t	p
	HC VS MCI			0.98	0.33
	Skills only vs. Combined Training			0.86	0.4
Medication Management
	HC VS MCI			0.57	0.57
	Skills only vs Combined Training			0.91	0.37
On-Line Banking
	NC vs. MCI			1.66	0.12
	Skills only vs Combined Training			0.56	0.96
Prescription Refill
	NC vs. MCI			0.21	0.84
	Skills only vs. Combined Training			1.44	0.16
Ticket Purchase Task
	NC vs. MCI			1.25	0.22
	Skills only vs. Combined Training			0.25	0.81
Internet Prescription Refill and Shopping
	NC vs. MCI			1.55	0.19
	Skills Only vs. Combined Training			0.16	0.87

DISCUSSION:

CSFT training led to substantial and rapid treatment gains with as few as 6 training sessions, with results applicable to both NC and MCI participants. Both participant groups evidenced substantial improvements in task performance. CCT supplementation led to similar CFST gains with half as many CFST training sessions. Importantly, NC participants who received skills training alone required an average of only 6 sessions per task (out of a possible 24) to perfect their performance. In summary: 1) both groups of participants demonstrated improvements in performance across all tasks; 2) the HC participants proceeded through the training fairly rapidly even without CCT supplementation; and 3) MCI participants required more training sessions but learned equivalently. These findings replicate our previous findings with older patients with schizophrenia and a separate sample of healthy controls.

Of primary importance is the improvement in training associated with computerized functional skills training in participants with MCI. These cases had substantial impairments in their episodic memory. However, they were still able to make substantial gains, proportionately equivalent to people with NC, across 6 different training simulations. Previous studies have shown dissociation of procedural and verbal memory learning in MCI and amnestic conditions^13,14. Thus, this study shows that functional skills can be learned fairly rapidly and efficiently, with few dropouts.

Supplementation of computerized skills training with CCT increased the efficiency of the skills training considerably, with gains doubled per unit training session compared to skills training alone. Thus, a combined intervention with CCT and CFST in individuals with MCI would likely have multiple benefits. First, dementia prevention may be facilitated by CCT. Skills training may also lead to increased independence or delay progressive functional changes in MCI. As a result, the potential benefits of combined training seem considerable and a major topic for further study with this protocol

Later studies will focus on real-world functional gains. Demonstration of such real-world gains would solidify the benefits of this training protocol. Implementing the protocol was quite efficient and participants reported high levels of satisfaction with their gains. For example, 98% or more of the participants stated that they “definitely” would be more able to do each of the 6 tasks in the real world.