Digital technologies for the assessment of cognition: a clinical review

Abstract

Dementia is the most widespread form of neurodegenerative disorder and is associated with an immense societal and personal cost. Prevalence of this disorder is projected to triple worldwide by 2050 leading to an urgent need to make advances in the efficiency of both its care and therapy research. Digital technologies are a rapidly advancing field that provide a previously unavailable opportunity to alleviate challenges faced by clinicians and researchers working in this area. This clinical review aimed to summarise currently available evidence on digital technologies that can be used to monitor cognition. We identified a range of pervasive digital systems, such as smartphones, smartwatches and smart homes, to assess and assist elderly demented, prodromal and preclinical populations. Generally, the studies reported good level of agreement between the digital measures and the constructs they aimed to measure. However, most of the systems are still only in the initial stages of development with limited data on acceptability in patients. Although it is clear that the use of digital technology to monitor and support the cognitive domains affected by dementia is a promising area of development, additional research validating the efficacy, utility and cost-effectiveness of these systems in patient populations is needed.

Introduction

In 2015, Alzheimer’s Disease International reported that an estimated 46.8 million people were living with dementia worldwide, with an associated cost of US$818 billion.¹ With the percentage of the world population aged over 60 years predicted to double by 2050,¹ the societal, economic and personal burden of this ‘dementia epidemic’² is a major concern for the sustainability of healthcare.³ It has been argued that addressing preventable risk factors for dementia, such as sedentary lifestyle and poor diet, could reduce the disease burden.³ To achieve this, we must understand which risk factors to target, and whether a critical intervention period exists.

Omnipresent digital technologies, such as smartwatches and smartphones, could help address these two issues, through their ability to obtain a wealth of ecologically valid, longitudinal information on health, behaviour and cognitive function. This rich feedback provides new opportunities to identify potentially modifiable risk factors, detect early changes in behaviour indicative of disease and track disease trajectories over time.⁴

As well as monitoring health and cognition, digital technologies that provide adaptive assistance are now emerging due to advances in machine learning. The application of domotics—the integration of technology into residential structures⁵—to dementia care has resulted in the development of adaptive ‘smart homes’ that assist patients with activities of daily living (ADLs) affected by their cognitive deficits.⁶ With the majority of dementia care costs falling on unpaid carers and long-term institutional social care,⁷ technological developments promoting patient independence and quality of life will be crucial to reducing the growing burden of neurodegenerative diseases.

Beyond providing assistance, digital technology is being explored as a potential method of non-pharmacological intervention. Cognitive training smartphone applications, which aim to strengthen various cognitive domains⁸ (defined by the National Institute of Health as attention, perception, declarative memory, language, cognitive control and working memory),⁹ are particularly prevalent. There is already evidence that ‘structured programmes of cognitively demanding computer tasks’ can be beneficial for healthy brain ageing.¹⁰ The adoption of these programs into downloadable smartphone applications addresses several former limitations including: integration into daily life, implementation burden and distribution scope.¹¹

This review aimed to provide a summary of current developments in digital technologies for cognitive monitoring, assistance and training in elderly demented, prodromal and preclinical populations. We focused our search on three technologies based on their immediate relevance to cognition, dementia and healthcare: mobile (smartphone and tablet) applications, wearable technology and smart home systems.

Methods

We searched PubMed, ScienceDirect and Google Scholar records (last search 25 October 2017) using the terms: ‘mobile technology’ or ‘wearable technology’ or ‘smartphone’ or ‘smartwatch’ or ‘smarthome’ or ‘domotics’ and ‘dementia’ or ‘cognition’ or ‘elderly’ or ‘Alzheimer’s’ or ‘health’. Articles in English were screened, and studies selected that evaluated digital technologies to monitor or assist cognitive function in older adults. We included technologies that deduced cognitive function through ADL performance. Articles piloting relevant technology in younger adults were also included when intent to apply the technology to older populations at risk of, or living with, dementia was demonstrated. Additional studies referenced by articles identified in the original search were also included. Due to the heterogeneity of digital technology and outcomes reviewed, a comprehensive systematic review was beyond the scope of this paper.

Presentation

Our review identified 24 articles detailing digital technologies with the capacity to analyse cognitive function in older adults at various stages of a dementing disease.

Mobile applications

Seven identified articles presented cognitive-focused smartphone/tablet applications. The apps’ objectives could be generally divided into three categories: cognitive monitoring, assistance and training.

Cognitive monitoring

Three apps were designed to monitor cognitive function in older adults. iVitality¹² and Color-Shape Test (CST)¹³ aim to provide reliable means to assess cognition in ‘at-risk’ populations for dementia; DelApp¹⁴ focuses on identifying delirium in hospital inpatients. iVitality employs five digitally adapted versions of standard cognitive tests (Stroop, Reaction Time, Trail Making, N-Back and Memory-Word tests).¹² The tasks were piloted on 151 individuals with familial risk of dementia (mean age 57 years) over six months,¹² with moderate correlation for digital Trail Making and Stroop against their lab-based counterparts (correlation coefficients range 0.4–0.6) and adherence ranging from 48% to 67%. The developers concluded that smartphone-based cognitive testing is feasible in cognitively normal individuals aged 50+ years, with acceptable levels of correlation with gold standard, lab-based testing.¹²

CST is a smartphone-optimised webpage developed to measure cognitive processing speed in the elderly,¹³ through participants’ learning and recalling shape colours. In a feasibility study of 57 cognitively healthy older adults CST performance correlated with performance on standard measures of global cognition (Mini Mental State Examination), processing speed and attention (digit span and trail making tests), but not tests of executive function (verbal fluency) or episodic memory (logical memory test).¹³ As only 18 participants had prior possession of a smartphone, the authors concluded the task’s usability in older adults was not dictated by smartphone familiarity.

DelApp is a computerised version of the Edinburgh Delirium Test Box (EDTB); an assessment of visual sustained attention in general hospital inpatients.¹⁴ No statistical difference was found between performance on the DelApp and the standard EDTB in 20 elderly cognitively healthy inpatients. DelApp also reliably differentiated patients with delirium from patients with dementia (despite comparable cognitive performances) and from those with no cognitive impairment in 156 elderly inpatients (area under the curve = 0.96 with 98% sensitivity and 93% specificity).¹⁴ The conclusion was that DelApp provides an accessible and reliable means of monitoring delirium emergence in inpatient populations.

Cognitive assistance

The technology Adoption and Usage Tool (TAUT),¹⁵ was designed to provide cognitive assistance for individuals with episodic memory deficits through an adaptive ADL reminder system. By assimilating information from user inputs and context-aware sensors, the app adapts reminder delivery to improve integration with the user’s lifestyle over time. TAUT is currently is being tested in an elderly, cognitively impaired population,¹⁶ following a pilot in healthy younger adults where 73% of reminders were acknowledged within 12.38 seconds.¹⁵

Cognitive training

Four apps were designed as primary and secondary dementia prevention methods with varied cognitive targets. Two apps, SMART and Fit Brains,¹⁷ use multidomain programmes of gamified cognitive tasks to improve cognitive reserve in preclinical older adults. SMART specifically targets attention and memory (working and declarative), while Fit Brains generally covers most of the NIH cognitive domains.¹⁷ In a randomised control trial, the efficacy of the apps for improving working and declarative memory in 53 older adults (mean age 59 years) with subjective memory complaints was investigated comparatively over eight weeks.¹⁷ Statistically significant improvements in overall and auditory verbal working memory scores on the memory diagnostic system (MDS, a computerised neuropsychological battery)¹⁸ were demonstrated for SMART, but not Fit Brains. The authors concluded that the greater focus of the SMART program’s tasks led to the working memory improvements on the MDS. However, this did not translate to participants’ self-reported memory contentment, which only improved post-test in the Fit Brains group.¹⁷

Two additional cognitive training apps are currently in design. The modified Attention Training Application (ATA)¹⁹ implements an adaptive working memory and attention (dual-n-back) task over  two weeks to reduce executive deficits in people with mild cognitive impairment (MCI). The task was piloted on 12 patients with MCI and healthy older adults (mean age 79 years), who on average rated the ATA as 60% interesting and 72.5% easy to use.¹⁹ Suggested modifications are being implemented for future feasibility testing.

Healthebrain⁸ employs a three-week square-stepping exercise (SSE) to improve visuospatial memory in preclinical and MCI older adults. The user learns and reproduces patterns displayed on a smartphone screen by walking, holding the device parallel to the floor.⁸ Non-computerised SSE has been observed to improve global cognitive functioning, especially attention and cognitive control, in older adults.²⁰ On piloting the app with 19 healthy or MCI older adults (mean age 68 years), 60% of the participants reported the app easy to use, or comparable to the lab-based SSE task.⁸ Future work is needed to establish the validity of an app-based SSE program as a cognitive intervention.

Wearables

Ten studies identified reported wearable technologies (smartwatches, accelerometers, cameras and glasses) for elderly preclinical and demented populations, with objectives that could be divided into cognitive monitoring and assistance.

Cognitive monitoring

Smartwatches

Four articles presented smartwatches that assess physical and, by proxy cognitive, function in patients with dementia. WanderRep²¹ is a smartwatch-based reporting tool for caregivers of wandering persons with dementia. The smartwatches’ ability to record time, location, temperature and activity level is used to create a personalised profile of wandering risk. By modelling patient behaviour, irregular and dangerous wandering can be detected, and caregivers alerted.²¹ The authors piloted the smartwatch with one care home based patient with dementia, and five professional caregivers who determined potentially dangerous wandering events. The system reported high sensitivity and specificity to detect dangerous events (78% and 89%, respectively) and thus supported smartwatch use in supporting independent living.²¹

Three identified systems (Max,²² u-Healthcare,²³ Basis B1²⁴) use smartwatch-derived measures to create activity profiles of patients with dementia. Both Max and u-Healthcare rely on location and step data to infer activity; Max employs a Bluetooth sensor system to obtain room-specific data, while u-Healthcare uses GPS, accelerometer and ambient light sensors to profile physical activity inside and outside the home. Max was piloted in the homes of 13 healthy controls from the Dementia Care Ecosystem²⁵ over 39 months.²² Reported room detection accuracy was 91%, and distinct user behaviour patterns could be detected. u-Healthcare was trialled in eight participants with an average reported step detection accuracy of 94.7%.²³

Basis B1 monitors patient activity using broader biological measures (optical blood flow, body temperature and galvanic skin response) captured by a smartwatch, combined with medical history.²⁴ It was piloted with one patient with dementia alongside their existing home based care and was reported not to cause discomfort or anxiety, and provided the caregiver with new information on the patient’s night disturbances, sleep and physical activity.²⁴ The authors suggest that smartwatch monitoring systems could be complementary tools for existing care practices by monitoring cognitive health and behaviour when caregivers are unavailable, and patient self-report is unreliable.²⁴

Smartwatch technology has also been developed for preclinical populations. The wrist wearable unit (WWU)²⁶ monitors home based physical activity levels of preclinical older adults longitudinally using measures of step count, acceleration and heart rate. Routine user activity, and subsequent deviations, are reported to healthcare professionals via an online platform. WWU was piloted in groups of 2–20 healthy adults.²⁶ WWU-derived activity levels correlated well with users’ subjectively reported activity, WWU-calculated heart rate fell within ± four bpm of pulse oximeter measures and device worn/unworn status was correctly identified to one minute accuracy.²⁶ The authors concluded that WWU could help to reliably determine preclinical function, from which changes indicative of physical and cognitive decline could be ascertained.

Accelerometers

Accelerometers, portable electromechanical sensors, offer a more established and lower cost activity monitor than smartwatches. One study used accelerometer data to monitor older adults’ physical activity, intending to infer cognitive status.²⁷ A waist-worn triaxial accelerometer was used by 274 community-dwelling older adults over 22 months. Light physical activity (measured by the accelerometer and defined using established cut-offs²⁸) was independently associated with lower scores (ie, better cognitive function) on the AD8—an eight-item informant interview probing memory, orientation, judgement and ADLs²⁹ at follow-up.²⁷ The study suggested that promotion of higher levels of objectively measured light physical activity could help protect cognitive function in older adults.

Wearable cameras

One study used a custom-made wearable camera system, worn by caregivers, to monitor patients’ dementia-related behaviour.³⁰ The system was piloted with 18 patients with dementia and their caregivers over two weeks or for one 3–5-day period. Three hundred and forty-one hours of usable video was collected, containing 248 salient events (dementia-related behaviour or caregiving interactions).³⁰ Further development may lead to this technology being used to provide validation of caregiver observations, and accessible, unbiased monitoring of changes in patients’ behaviour, cognition and needs over time.³⁰

Cognitive assistance

Smartwatches

One study investigated commercially available smartwatch technology to provide ADL assistance for patients with dementia,³¹ implementing smartwatch apps, and a paired smartphone, to assist scheduling, navigation, orientation to time and communication, as well as monitor activity levels. The system was tested by five memory clinic patients and their spouses in a controlled lab setting. Initial feedback suggested only the scheduling, orientation and communication functions were usable (90%–100% success rate completing tasks using these functionalities, compared with 0% on the navigation and emergency help tasks).³¹ Results from a follow-up home pilot are pending.

Wearable camera

SenseCam³² is a wearable camera system supporting autobiographical memory consolidation and retrieval in cognitively impaired people. SenseCam captures pictures every 30 s, or in response to specific triggers (for example movement) which the patient subsequently reviews.³² During a two-week testing phase, the proportion of events of patients with MCI correctly recalled increased significantly from 38% at baseline (no review) to 68% at day 13; while a diary review method showed no significant change in the patient’s recall at day 13 (30%).³² This recall improvement for events reviewed using SenseCamwas sustained at six months’ follow-up. The patient also reported an increase in self-esteem and confidence.

Smart glasses

One article presented a head-mounted display system to assist patients with mild-to-moderate dementia with navigation inside and outside the home,³³ consisting of a pair of smart glasses with implanted light emiting diodes (LEDs) and various sensors (including accelerometers and a global positioning system tracker), which communicate with a remote android unit via a Bluetooth headset. Caregivers can use the remote unit to monitor the patient’s location and send navigational cues through the glasses’ LEDs. Acceptability of visual navigational cues was demonstrated in feasibility testing in a group of patients with dementia, with cue usability significantly influenced by LED positioning and dementia severity.³³

Smart homes

Extensive research exists into applying smart home technology to dementia populations, particularly focusing on providing daily assistance. While not designed to directly monitor cognition, by observing changes in patients’ ADLs, we here report seven smart home systems which have potential to infer and monitor cognitive function.

Cognitive monitoring

Smart home monitoring and assistance typically uses a three-stage iterative process. Technologies in the infrastructure of the building—for example magnetic contact sensors, passive infrared motion sensors and pressure mats³⁴—monitor the environment. Machine learning principles then conceptualise the data into patterns of behaviour, and subsequent deviations.^{35 36} Finally, a decision-maker system reacts to behavioural deviations and provides real-time feedback to patients and caregivers.³⁴ One study found that acceptance of smart home monitoring, and alert technology by patients and their families was predicated on perceived enhancements to the safety and independence of the patient, and to delay institutionalisation.³⁷

Three identified studies assessed the accuracy of smart homes to determine participants’ performance while completing ADLs. The Machine-to-Machine (M2M)/Internet of Things (IoT) platform smart home showed ADL detection accuracy to be 80%–100% for most activities, including wandering detection and forgetting to take a shower.³⁸ The DemaWare2 system reported an average of 82% precision for recognising activities performed in a lab environment and 75% for ADLs performed in a residential smart home.³⁵ The third found a significant correlation (r=0.54) between the scores of ADL performance assigned by clinicians observing patient behaviour, and the performance scores assigned by the smart home.³⁶ However, for all, the smart home’s accuracy for determining the user’s performances varied depending on the type of activity being assessed; all three studies found smart home predictions of watching television, including remembering to turn off the TV, were less accurate than other ADLs. One explanation given was that quicker activities, involving fewer sensor interactions, were harder for the smart home to accurately identify, so subsequent deviations were less likely to be detected.³⁶ Sensor type may also make a difference, for example, detection of ‘forgetting to turn off the TV’ was reliant on the sound exceeding the sensor’s threshold.³⁸

A collection of studies by the Oregon Center for Aging and Technology (ORATECH) group have assessed whether by monitoring a number of ADLs, a smart home could detect cognitive function and decline. The ORCATECH group collected continuous, daily data from 480 homes since 2007, using an unobtrusive activity monitoring smart home (comprising motion, contact and pressure sensors, computer and phone monitoring, medication trackers and wireless scales).³⁹ One study reported no differences in daily recorded computer activity (based on mouse movements) at baseline between groups of cognitively healthy and MCI participants. However, at follow-up two years and three years later, a significant decrease in number of days with computer use, mean daily use and an increase in day-to-day variability was found in the MCI group only.⁴⁰ The authors concluded that computer usage is likely to be sensitive to cognitive change due to its reliance on multiple cognitive domains. The same investigators monitored medication adherence using a device with sensors built into the medication box signalling open and close events, demonstrating that the participants who performed worse on the Alzheimer’s Disease Assessment Scale-cognitive subtest (ADAS-Cog) had significantly poorer medication adherence than the better performing group.⁴¹

The capability of two smart home systems to discriminate cognitive states and dementia status has also been examined. The DemaWare2 system³⁵ monitored data from 27 Alzheimer’s disease, 38 MCI and 33 cognitively healthy participants as they performed ADLs in a lab-based smart home. For the ADLs ‘making a phone call’ and ‘paying a bank bill remotely’, the system could distinguish between the three participant groups with up to 84% accuracy.³⁵ Similarly, the second study evaluated the ability of a machine learning method to classify the cognitive health status of 263 participants (196 cognitively healthy, 51 patients with MCI and 16 patients with dementia) from eight ADLs performed in a lab-based smart home.³⁶ Reasonable accuracy in differentiating between cognitively healthy and dementia participants was reported when combining data from all eight ADLs, and on some individual ADLs (sweeping, cooking and dressing), but not for differentiating between cognitively healthy and MCI participants. By including only ADLs that demonstrated good prediction accuracy in isolation, the classification performance of the combinatorial model was improved.³⁶ These studies demonstrate that, while still in pilot stages, monitoring ADL performance by smart homes is feasible and can identify cognitive decline over time and infer cognitive states and dementia status.

Discussion

Through this review we attempted to highlight the breadth of digital technology currently available for the assessment of cognitive function in elderly demented and preclinical populations. We identified technologies that allow direct monitoring of cognition (eg, smartphone apps) and those that monitor broader indices of activity and function that could be used to deduce cognitive ability. This is a rapidly developing area, with the number of dementia-focused digital technologies doubling approximately every five years.⁴ The timely adoption of such technology in clinical practice is a challenge that requires effective communication between developers and clinicians about the availability of such solutions.⁴ Overcoming this barrier could provide some of the most promising opportunities to reduce the burden of dementia.

The main value that currently available digital technology can offer, in terms of cognitive monitoring, is the capacity to provide more ecologically valid, high-granularity data. This type of data could be crucial in the development of much needed pharmacological dementia treatments. Despite considerable investment, no disease-modifying treatments are currently available, with several high-profile failures,³ which may have been due to testing compounds too late in the disease process. Longitudinal, high-frequency measurements by digital technologies could be used to detect subtle cognitive changes in at-risk populations, allowing targeted interventions earlier in the illness.

Longitudinal monitoring of behaviour and physical health could also be of immediate benefit clinically. Systems providing information on potentially fluctuating neurobehavioural symptoms, not displayed at clinic visits, may assist clinicians in making earlier diagnoses from fewer visits. This could therefore reduce the time and cost burden for both clinicians and patients. Similarly, technological monitoring will enable clinicians to objectively track behavioural and cognitive changes more closely over time,³⁴ reducing the need to rely on subjective accounts. Finally, such systems can provide feedback directly to at-risk participants allowing for behavioural interventions targeting modifiable risk factors.

For existing patients with dementia, the development of increasingly adaptive, assistive systems can help preserve independence levels and quality of life for as long as possible.⁴² Maintaining independence is a crucial goal for the sustainability of dementia care, as current population predictions show the caregiver-to-patient ratio is expected to reduce dramatically by 2050⁴. Furthermore, the trend of older adults moving away from cities into rural areas post-retirement poses a significant challenge to healthcare delivery. Therefore the development of systems that provide support and prevent emergency situations^{24 34} when carers are unavailable, or allow carers to remotely monitor and assist multiple patients,³⁴ will be increasingly important in the future.⁴

Despite the potential advantages of deploying digital technology to both dementia research and care, it carries ethical implications. Consent and capacity, is particularly relevant given the potential security and privacy threats associated with digital technology.⁴³ Patient confidentiality may be violated by an intrusion on, or revelation of, something private; such violations are considered to include use of video or audio technology³⁵ where content is fedback to carers. While data collected should be secure and encrypted to maintain confidentiality, there are inherent security risks when collecting and transferring personal data via a network.⁴³ Privacy and security are major considerations in user adoption of digital technology, with higher acceptance reported for non-invasive, reliable equipment.⁴⁰

A further ethical concern is socioeconomic barriers that may prevent utilisation of digital technology for healthcare across society.⁴ While few of the reviewed studies reported any costs of the technologies implemented, it is unlikely to be trivial. Beyond the initial hardware cost, further hidden costs associated with technology usage, update and protection (for example, internet access and insurance policies) may cause a socioeconomic divide. Such finances need to be managed, and with cognitive decline often affecting ADLs, such as financial organisation,^{35 36} the burden associated with introducing smart technology may be considerable and infeasible. Nevertheless, digital monitoring of patients with cognitive decline may provide a lower-cost alternative to full-time care.³⁴ These arguments highlight the need for research investigating the cost-benefit ratio of technologies and variations between demographic groups.

Although out of this review’s scope, it is worth mentioning there are many new digital technologies on the horizon which could be adopted for dementia care. One such development, broadly referred to as ‘bodyNET’,⁴⁴ involves a network of sensors and smart devices worn as part of clothing, on the skin, or even implanted into the body. This technology is under early development but should allow for a passive data monitoring that can be used alongside the principles of assistive technology.⁴⁴ Other technologies are being specifically developed to target dementia-related deficits. One example is PARO:⁴⁵ a socially assistive robot seal pup, which has been shown to increase social interactions between patients with dementia in a group therapy session. Another system, BikeAround,⁴⁶ employs a stationary bike paired with a virtual reality projection of Google Street View to allow patients with dementia to ‘visit’ personally significant locations and, in doing this, tap into their autobiographical memories.

In conclusion, developments in digital technology for the monitoring and assistance of older adults with and without a dementing disease is a rapidly growing area of interest. This technology has the potential to greatly improve the efficiency of dementia drug development, as well as optimise the provision of dementia care in settings of increasing demand. Many of the developments appear promising in their initial pilot stages, however further research is needed to validate the measures and assess long-term outcomes of users.