Intelligent Assistive Technology Applications to Dementia Care: Current Capabilities, Limitations, and Future Challenges

Abstract

The number of older Americans afflicted by Alzheimer disease and related dementias will triple to 13 million persons by 2050, thus greatly increasing healthcare needs. An approach to this emerging crisis is the development and deployment of intelligent assistive technologies that compensate for the specific physical and cognitive deficits of older adults with dementia, and thereby also reduce caregiver burden. The authors conducted an extensive search of the computer science, engineering, and medical databases to review intelligent cognitive devices, physiologic and environmental sensors, and advanced integrated sensor networks that may find future applications in dementia care. Review of the extant literature reveals an overwhelming focus on the physical disability of younger persons with typically nonprogressive anoxic and traumatic brain injuries, with few clinical studies specifically involving persons with dementia. A discussion of the specific capabilities, strengths, and limitations of each technology is followed by an overview of research methodological challenges that must be addressed to achieve measurable progress to meet the healthcare needs of an aging America.

The graying of the world population poses formidable socioeconomic challenges to the provision of acute and long-term healthcare. Approximately 28 million persons worldwide suffer from dementia and account for 156 billion dollars in direct care costs annually.¹ In the United States alone, the population aged 65 and older will double in size to 72 million within the next 25 years, with the segment over age 85 years growing at the fastest rate.² Alzheimer disease (AD) is the leading cause of dementia in this elderly cohort and afflicts nearly 4.5 million Americans.³ If current population trends continue, an estimated 13 million Americans will have the disease by 2050.³ Currently, the national direct and indirect costs of caring for AD exceed $100 billion a year, with long-term institutional care accounting for majority of the expenditure.⁴ More recently, a study commissioned by the Alzheimer's Association predicts that the total Medicare spending on AD will triple to $189 billion by 2015,⁵ and around mid-century, the Medicare and Medicaid combined cost of caring for AD will exceed $1 trillion annually.⁵

In light of these sobering demographic shifts and socioeconomic forecasts, there is a burgeoning interest in the early detection and management of AD, as well as technological innovations that may promote “aging-in-place.” A host of cognitive aids, environmental sensors, video and audio technologies, and advanced integrated sensor systems are under development to monitor the health, safety, and well-being of cognitively and/or functionally impaired persons.^6,7 Progressive maturation of these innovations has the potential to improve the quality-of-care and quality-of-life of the elderly and their family caregivers, and delay or obviate the need for institutional care (i.e., nursing homes).⁸ It has been estimated that a 1-month delay in nursing home placement of all Americans older than 65 years would reduce healthcare expenditures by $1.2 billion annually.⁹ Remote mobile health monitoring, however nascent its present capacities, is predicted to be the next major wave in the reform of healthcare delivery systems, and honors the oft-stated preference of the elderly to remain independent for as long as possible, even in the face of increasing disability.¹⁰ Moreover, proactive involvement of the elderly in the design and implementation of these intelligent assistive technologies will maximize the likelihood of their acceptance of the technology.¹¹

Although a range of assistive technologies to compensate for physical and cognitive impairments have emerged in recent decades, the scientific literature has remained largely confined to the engineering and computer science domains with few exceptions.^6,7,10 Moreover, the physical and rehabilitation medicine literature to date has primarily addressed the use of assistive devices for physical disability, typically in younger adults, and not the cognitive, functional, and behavioral sequelae of dementia in particular. Thus, the major goal of this manuscript is to systematically review for clinicians and clinical researchers alike the current availability, capabilities, and developmental stage of technologies that with continued research may find applications in dementia care. A glossary of technological terms and concepts is provided in the Appendix to assist the reading of the manuscript.

METHODOLOGY

The authors (AJB, VA, and JF) searched MEDLINE (1965–present), PsycINFO (1966–present), Embase (1967–present), and the Cochrane Database of Systematic Reviews (1988–present) for relevant key terms with the assistance of a medical librarian. In addition, the following engineering and computer science databases were searched: INSPEC (1898–present), Web of Science (1945–present), and the AccessScience Encyclopedia of Science and Technology. The Google Internet search engine was used to retrieve descriptions of commercially available products as well as to identify technologies that were not cited in the scientific databases. Our primary goal was not to provide the most comprehensive or exhaustive survey of currently available assistive technologies, but rather to offer the reader an appreciation of the range of emerging devices that may find a role in dementia care. The database search terms (alone and in combination) included “technology,” “assistive technologies or devices,” “sensors,” “wearable sensors,” “environmental sensors,” “video or audio technologies,” “biosensors,” “ubiquitous healthcare technologies,” “pervasive healthcare technologies,” “smart homes,” “artificial intelligence,” and “elderly,” “dementia,” “disability,” or “Alzheimer disease.” The search yielded approximately 250 articles, book chapters, and conference proceedings. We identified additional relevant citations by reviewing the reference lists of our original sources. The authors (AJB, VA, and JF) reviewed the abstracts of the retrieved sources and excluded duplicate citations as well as assistive devices or sensors that did not have direct implications for one or more functional impairments associated with dementia (memory loss, executive dysfunction, aphasia, agnosia, apraxia, visuospatial impairment, behavioral, activity, and mood disturbances). The aforementioned step was critical since a majority of the assistive technologies were developed for persons with typically nonprogressive traumatic, anoxic, or focal vascular brain injuries, and a determination had to be made as to which ones seemed to be potentially modifiable for the deficits of those with progressive neurodegenerative dementias. We also specifically excluded telecare approaches that have been amply reviewed elsewhere.¹² Moreover, cognitive aids that lack the ability to determine the user's current activity context and necessity or appropriateness of an intervention, such as simple reminder systems were also eliminated. This report thus focuses on intelligent cognitive aids, physiological sensors, simple, and multimodal environmental sensors and advanced integrated systems that may have particular relevance for dementia care.

RESULTS

Our review of the clinical, engineering, and computer science literature databases identified 58 total technologies, both basic and advanced, with potential applications to dementia care. Table 1 summarizes the type, capabilities, developmental stage, target population, and current strengths and limitations of each technology. Most importantly, it also notes the presence or absence of clinical trials data for these devices. When a sensor or system serves more than one purpose (i.e., cognitive as well as functional assistance), we categorized it according to the developer's primary intent. Of the total, we identified 11 cognitive orthotics,^13–23 15 environmental sensors,²⁴ 10 physiological sensors,^25–34 and 22 advanced integrated sensor systems.^35–56 To date, the greatest attention has been paid to the development of memory aids (six studies).^13–18 Few devices (six: Memory Glasses, Visually Enhanced Recipe Application (VERA), Cook's Collage, Intelligent Mobility Platform, Opportunity Knocks, and Activity Compass)^13,19–23 address any of the other progressive cognitive and functional impairments associated with dementia such as aphasia, agnosia, apraxia, visuospatial, or executive dysfunction. Moreover, clinical studies specifically involving dementia subjects have been published for only three systems (Cognitive Orthosis for Activities in the Home [COACH], CareWatch, and CareMedia)^44–46 although reports summarizing case studies or prototype evaluations are available for an additional 21 systems.^{13–21,30–35,47,48,50–52,54}

TABLE 1.

Cognitive Orthotics, Environmental and Advanced Integrated Sensors With Potential Applications to Dementia Care

Cognitive, Functional, or Physiological Deficit	Product Name	Description and Approach	Level of Maturity	Target Population	Clinical Studies with Dementia Subjects	Pros and Cons
Cognitive–prospective memory aid	Memory Glasses13	A wearable context-aware memory aid and reminder system, in the form of eyeglasses, which may also be effective in managing anomia and certain forms of agnosias	Research prototype	Normal aging, amnesia, MCI, mild AD or other dementia, prosopagnosia	No	Prototype evaluations have been conducted, but only with healthy users
Cognitive–prospective memory aid	MemoClip14	A memory aid in the form of a badge that is clipped to clothing that associates task information with time, location and context	Research prototype	Normal aging, amnesia, mild AD, or other dementia	No	Prototype evaluations have been conducted, but only with healthy users
Cognitive–prospective memory aid	Friedman1	A prospective memory aid consisting of a wearable microcomputer using radio and ultrasound to communicate with the user's environment. It determines the user's location and provides task-related information	Research prototype	Normal aging, MCI, mild AD, or other dementia	No	Prototype evaluations have been conducted, but only with healthy users. It provides voice prompts only as needed, thus decreasing the user's dependence on the system
Cognitive–prospective memory aid	PEAT (Planning and Execution Assistant and Trainer)16	An automatic planning software that is customizable for individual perceptual and cognitive deficits. The system operates on a PDA (personal digital assistant) or mobile phone, and provides personalized cueing to guide the user through multi-step procedures and ADLs using digital pictures and voice recordings. It is flexible (tasks and times) and adaptive	Commercial product	As above	No	PEAT is available in preconfigured or non-configured versions, and can be customized to particular human needs and output devices
Cognitive–prospective memory aid	ISAAC17	A wearable fully individualized cognitive aid that organizes and delivers individualized prompts, procedural and personal information. It delivers prompts in synthesized speech audio, text, checklists, or graphic format	Commercial product	Normal aging, MCI, mild AD or other dementia, anoxic or traumatic brain injury	No (only case studies of users with anoxic or traumatic brain injury have been published)	ISAAC is easily adaptable, and suitable for individuals with poor vision or hearing impairment. The device requires user training however that may be difficult for some persons with cognitive impairment
Cognitive–prospective memory aid	AutoMinder18	A cognitive orthotic capable of modeling a subject's daily plans, tracking their execution, and determining whether and when to provide reminder(s)	Research prototype	Normal aging, MCI, mild to moderate AD, or other dementia	No	AutoMinder is an early prototype that remains to be evaluated with its stated target population
Cognitive–aphasia	VERA (Visually Enhanced Recipe Application)19	An interface using text and sound options that allow primarily visual cooking instructions to be individually customized for aphasic users	Research prototype	Normal aging, aphasia	No	It is not clear whether one interface modality (text or sound) is superior to the other for providing cooking reminders
Cognitive–aphasia	Cook's Collage20	A video-based reminder system that guides a user through a multistep cooking task by displaying the previous six steps taken on a monitor	Research prototype	Normal aging, aphasia	No	A longitudinal study with an anticipated sample of size of 20 participants (10 older, 10 younger) is currently in progress
Cognitive–navigational tool	IMP (Intelligent Mobility Platform)21 Indoor navigation system	A walker-based device designed to navigate lost or confused users. It uses a laser beam range-finder, a handheld computer with a touchscreen interface, and navigation software	Research protoytpe	Normal aging, MCI, mild to moderate AD or other dementia	No	IMP is only suitable for persons who require an ambulatory device such as a walker
Cognitive–navigational tool	Opportunity Knocks22 Navigation system for outdoors	A cell phone embedded device using GPS chip and Bluetooth that learns the subject's standard routes in the community, and alerts the subject of a navigational error by making a knocking sound and re-routes the lost individual	Research prototype	As above	No	The device may need substantial training before it effectively knows a user's route
Cognitive–navigational tool	Activity Compass23	A Palm pilot-based GPS system that learns a subject's routine travel behavior in order to predict likely destinations, and re-routing a lost individual	Research prototype	As above	No	Activity Compass incorporates explicit feedback from the user in terms of what aspects of the route are followed or ignored. The system however is large and bulky as a handheld device
Environmental24	Motion detector	Senses movement and posture, and uses ultrasound to sense distance	Commercial product			Effective in combination with other environmental sensors to track activity patterns and deviations from personal norms
Environmental	Light sensor	Detects light	Commercial product			As above
Environmental	Temperature/heat sensor	Detects and calibrates temperature	Commercial product			As above
Environmental	Force sensor	Monitors rapid change in pressure	Commercial product			As above
Environmental	Pressure sensor	Monitors presence or absence of pressure	Commercial product			As above
Environmental	Contact sensor	Registers contact with doors, appliances, furniture	Commercial product			As above
Environmental	Video camera	Records moving image and sound	Commercial product			As above
Environmental	Proximity detector	Senses proximity	Commercial product			As above
Environmental	Door and window sensor	Senses whether a door or window has been opened	Commercial product			As above
Environmental	Leak and spill detector	Detects presence of liquid spills	Commercial product			As above
Environmental	Glass break detectors	Detects if glass has been shattered	Commercial product			As above
Environmental	Vibration/sound detectors	Detects vibration against a surface	Commercial product			As above
Environmental	Electrical usage sensors	Detects usage of electricity and change in pattern	Commercial product			As above
Environmental	Water/sewer usage sensor	Detects flow through pipes and change in pattern of flow	Commercial product			As above
Environmental	Voice activation sensors	Voice commands that permit interactions with doors, security systems, lights, blinds, appliances	Commercial product			As above
Physiological/functional	Bedwetting alarms25	Pad monitors the presence of moisture	Commercial product	Any persons with bedwetting, irrespective of cognitive impairment		Multiple different types of bedwetting alarms are commercially available
Physiological/functional	GlucoMON26	Real-time monitoring of blood glucose	Commercial product	Any person whose blood glucose levels need to be closely monitored		GlucoMon works with existing blood glucose meters
Physiological/functional	VTT gait monitor27	Monitors gait and recognizes changes in gait pattern	Research prototype	MCI, mild to moderate AD, or other dementia	No	The device will require substantial training to effectively recognize deviations from baseline gait patterns
Physiological/functional	Displacement sensor28	Monitors gait	Research prototype	As above	No	As above
Physiological/functional	Accelerometers29	Track and record one, two, and three-dimensional motion	Commercial product	As above	No	Effective in combination with other environmental sensors to track activity patterns and deviations from personal norms
Physiological/functional	Fall detector30	An impact and posture/orientation detector that works in conjunction with existing home alarm systems	Commercial product	As above	No	As above
Physiological/functional	SIMBAD Project (Smart Inactivity Monitor Using Array-Based Detectors)31	A wall-mounted inactivity and fall detector consisting of low-cost, array-based passive infrared sensors	Research prototype	As above	No	SIMBAD does not require the subject to wear or activate the device. It accurately detected falls in only 35.7% of controlled laboratory tests
Physiological/functional	University of Virginia floor vibration-based fall detector32	A piezoelectric sensor coupled to the floor surface to evaluate the floor's vibration patterns	Research prototype	As above	No	Falls were simulated using anthropomorphic dummies, and demonstrated 100% true positives and 0% false alarms. The system does not require the subject to wear or activate the device
Physiological/functional	Fall detector33	A fall detection system that is based on computer processing of images collected via a ceiling-mounted digital camera	Research prototype	As above	No
Physiological/functional	Vibrating gel insoles34	Viscoelastic silicone gel insoles that are embedded with vibrating elements to improve balance	Research prototype	As above	Yes; 15 young (mean age 23 ± 2 years) and 12 older (mean: age 73 ± 3 years) participated in a clinical trial; all 8 sway parameters improved in older users	The lightweight sensor is easy to embed in shoes
Advanced integrated sensor set	BodyMedia35	A wearable device in the form of an upper arm band that collects metabolic and physiological information (calories, steps taken, sleep and wake cycle)	Commercial product	Any individual who requires monitoring of metabolic and physiological parameters	No	The armband may feel uncomfortable or intrusive for some older users
Advanced integrated sensor set	WearNET36	A multi-sensor activity and context monitoring system	Research prototype	MCI, mild to moderate AD or other dementia	No
Advanced integrated sensor set	SenSay (Sensing and Saying)37	A multi-sensor activity and context monitoring system in the form of a wrist-worn device	Research prototype	As above	No	The wrist device is large, and some training time may be needed to implement the system with new users
Advanced integrated sensor set	Multi-accelerometer based systems38	A system designed to monitor activity and context	Research prototype	As above	No
Advanced integrated sensor set	Garmin Forerunner39	A wrist worn GPS and monitor that tracks heart rate, speed, distance, and calories burned	Commercial product	Any person requiring monitoring of metabolic or physiological parameters	No	The Garmin Forerunner was developed primarily for athletes. The data may not provide sufficient granularity for some research applications
Advanced integrated sensor set	Medical mood ring40	A wearable device in the form of a ring that monitors temperature, heart rate, and blood oxygen level. It contains a wireless link that can transmit vital signs to a cell phone or computer to allow a caregiver to determine remotely whether a person needs assistance	Research prototype	As above	No
Advanced integrated sensor set	Tadiran's MDkeeper41	A device that monitors pulse, 1-lead EKG and blood oxygen level. The data are transmitted, either in real-time or periodically, to a remote medical center for further analysis and care	Commercial product	As above	No	The proprietary software is pre-programmed and cannot be modified
Advanced integrated sensor set	Ciclosport Alpin 542	A device that monitors heart rate with altimeter functions	Commercial product	As above	No	The device was developed for athletes, and the data may not provide sufficient granularity for some research applications
Advanced integrated sensor set	Boardbug43	A wrist worn device that monitors behavior through sound, temperature, location	Commercial product	Any person whose activities need to be monitored irrespective of cognitive impairment	No
Advanced integrated sensor set	CareWatch44	A multi-sensor caregiver alarm system that detects whether a cognitively impaired person is in bed, moving within the house, or opening a door	Research prototype	MCI, mild to moderate AD or other dementia	Yes	Published data are available from a clinical trial with dementia users
Advanced integrated sensor set	CareMedia45	An automated video and sensor analysis system that monitors activity, behaviors, and social interactions continuously in real-time	Research prototype	Mild to severe AD or other dementia	Yes	A feasibility study and a follow-up comprehensive study have been conducted on a locked dementia unit
Advanced integrated sensor set	COACH (Cognitive Orthosis for Assisting Activities in the Home)46	A computer vision based system that associates hand positions with specific handwashing steps to provide cueing strategies	Research prototype	MCI, mild to moderate AD or other dementia	Yes	10 Moderately to severely demented users performed 25% more steps correctly without the assistance of a caregiver
Advanced integrated sensor set	Radiofrequency (RF) transmitter based home monitoring system47	The home of a person at risk for dementia is instrumented with various motion detection devices, and a small wireless network consisting of 3 RF receivers. The person at risk for dementia and his/her family caregiver must wear an RF transmitting wrist watch to monitor the at risk person's motion within the home	Research prototype	Persons at risk for dementia	No	The feasibility of the research prototype has only been demonstrated in a single home using a 3-week longitudinal record of RF transmission data as part of a larger study of persons at risk for dementia
Advanced integrated sensor set	PROACT (Proactive Activity Toolkit)48	PROACT is a computer based approach involving the use of RF technology that attempts to automatically recognize and record ADLs and the quality of their performance. The user of this system must wear a prototype glove that is capable of detecting and recording RF signals from RF tags that are unobtrusively placed on objects. Data collected by the glove are then analyzed to infer the ADLs that are executed	Research prototype	MCI, mild to moderate AD or other dementia	No	14 subjects (age range 25 to 63 years; mean 39) performed 14 ADLs over a 6 week period in a research home. PROACT correctly detected that an ADL occurred 88% of the time. Of these, it correctly recognized the specific ADL activity in 73% of cases
Advanced integrated sensor set	Sensvest49	An intelligent garment that monitors heart rate, temperature, movement	Research prototype	Any person whose physiological parameters need to be monitored	No
Advanced integrated sensor set	VTAM T-shirt (Vetement de Tele-Assistance Medicale)50	A Biosensor and bioactuator based garment that monitors EKG, respiratory rate and temperature, and also detect a fall	Research prototype	As above	No	A prototype system has been tested with only 3 users whose demographic and clinical characteristics were not reported
Advanced integrated sensor set	European Commission SmartFabrics Project (WEALTHY garment, MyHeart, MERMOTH, OFSETH)51a	An intelligent garment in the form of a vest that monitors EKG, respiration, EMG, and physical activity	Research prototype	As above	No	Potential users may not wish to wear a comprehensive garment. Wealthy garment has been tested with 5 cardiac and 7 healthy adult users (no subject over age 64 years)
Advanced integrated sensor set	MagIC vest (Maglietta Interattiva Computerizzata)52	An intelligent garment that monitors EKG, respiratory frequency, and assesses motion	Research prototype	As above	No	A prototype system has been tested with 9 healthy and 14 cardiac inpatient users (no further description of the demographic or clinical characteristics of the cohort reported)
Advanced integrated sensor set	SmartShirt garment53	An intelligent garment that monitors heart rate, EKG, respiration, temperature, and a host of other vital functions	Research prototype	As above	No	The garment has been launched for field trials in 2006
Advanced integrated sensor set	LifeShirt54	An intelligent garment that measures respiration, posture, and activity	Research prototype	As above	No
Advanced integrated sensor set	Intel Multisensor board55	A pedometer, accelerometer, and GPS sensor combined into a mobile package worn on the belt	Research protoytype	Any person whose activities need to be monitored.	No	The device may be too bulky for some users, and additional software will need to be developed to make data useful
Advanced integrated sensor set	Microsoft Sensecam56	A neck worn sensor package, including a digital camera that takes images passively, without user intervention, and sensors that note change in light, body heat, and a three axis accelerometer	Research prototype	As above	No	Low resolution images may not be sufficient for memory support. Case reports with older persons with limbic encephalitis and mild to moderate AD suggest improved recall of autobiographical events

^aFull names for the acronyms not provided by the system developers.

AD: Alzheimer disease; ADLs: activities of daily living; GPS: Global Position Sensor; MCI: Mild Cognitive Impairment; RF: radiofrequency.

Cognitive Aids

Prospective Memory Aids

In contrast to the myriad of commercially available simple task and time-based reminder systems, the aids reviewed here are context-aware, and use artificial intelligence to determine whether and when an appropriate reminder or procedural guidance is necessary for task execution. They are programmed either to improve performance of multiple different tasks throughout a routine day (Memory Glasses, MemoClip, Friedman)^13–15 or a sequence of steps in either single or multiple tasks (Planning and execution assistant and training, ISAAC, AutoMinder, Friedman).^15–18 They also possess customizable multimodal display and prompt delivery systems that leverage the individual's remaining cognitive and sensory capacities (text, audio, visual graphics, etc.). Although these context-aware, adaptive prospective memory aids are a step in the right direction, only planning and execution assistant and training and ISAAC are presently available as commercial products,^16,17 and clinical trials specifically with dementia subjects are lacking for all of them. Moreover, considerable research will be necessary to make such devices capable of handling deviations from programmed routines and contextual uncertainties.

An Example of a Prospective Memory Aid: Memory Glasses

Memory Glasses is a context-aware memory aid that is embedded in glasses.¹³ The goal of the system is to deliver reminders to the wearer in a timely, situation-appropriate way, without requiring intervention on the part of the wearer beyond the initial request to be reminded. This system is different from passive reminder systems, such as a standard Personal Digital Assistant, which cannot know the user's activity context. Memory glasses leverages a variety of computer perception techniques, based in part on captured visual images, which permit context awareness. The accuracy of context-awareness when delivering a prompt is important since distraction at the wrong time (e.g., crossing the street or driving a car) could be life-threatening. Although the computational underpinnings of this system are beyond the scope of this article, Memory Glasses has the ability to be adaptable to user preferences, sensory and cognitive limitations, and the user's primary tasks. In addition to Mild Cognitive Impairment, mild Alzheimer disease, and other dementias, this device is expected to be helpful in cueing memory for names (anomia) and faces (prosopagnosia). The evidence base for its efficacy is limited at present since clinical trials with demented older adults are lacking.

An Example of a Retrospective Memory Aid: Microsoft's SenseCam

SenseCam is a wearable digital camera that is designed to take photographs passively while it is being worn.⁵⁴ It is fitted with a wide-angle (fish-eye) lens that maximizes its field-of-view, thus ensuring that nearly everything in the wearer's view is captured by the camera. SenseCam records events as they happen, requires little effort on part of the user, and permits subsequent viewing of image sequences that is thought to lead to consolidation of autobiographical memory. A small number of cases (3–4 total) of limbic encephalitis, amnesia, and mild to moderate AD have suggested efficacy in increasing recall of greater proportion of autobiographical details for 3 (in AD) to 10 months (limbic encephalitis) after the event. As with the Memory Glasses, however, methodologically rigorous clinical trials have yet to be conducted.

Aphasia and Agnosia

Two cognitive aids have been developed to assist aphasic users in the execution of cooking tasks.^19,20 VERA, which provides individually tailored visual cooking instructions, has been tested with four aphasic subjects (age range: 29–73 years).¹⁹ The most severely aphasic subjects performed best with the system, whereas those who were less impaired performed best with text-based instructions or equally well with both. In contrast, Cook's Collage is a video-based reminder system that displays the previous six steps completed in the cooking task to reorient the individual to the remainder of the activity.²⁰ A clinical trial involving 20 subjects is currently in progress. As is apparent, intelligent assistive technologies to compensate for cognitive deficits other than prospective memory dysfunction are sorely lacking, and the existing ones such as VERA and Cook's Collage address only a specific task (i.e., cooking) within a much broader domain of dysfunction (aphasia). In addition to these two devices, Memory Glasses are anticipated to assist individuals with anomia by prompting them with appropriate words. It may also assist in certain forms of visual agnosias such as prosopagnosia (difficulty recognizing faces).¹³

Visuospatial Dysfunction

Three navigational tools have been developed that may assist mildly to moderately cognitively impaired subjects who are disoriented. The Intelligent Mobility Platform is a walker-based device that uses a laser beam range-finder, a handheld computer with a touch-screen interface, and a navigation software to orient a person in the proper direction using a red arrow.²¹ In contrast to this indoor navigation system, Opportunity Knocks is a cell phone-embedded device using Global Position Sensor chip and Bluetooth that learns the individual's standard routes in the community.²² It alerts the person of a navigational error by making a knocking sound and subsequently recalculating the proper route. Activity Compass is another Global Position Sensor-based system that accomplishes much the same as Opportunity Knocks.²³ None of these systems is commercially available, and all await rigorous clinical testing in applicable populations.

Physiological Sensors

Vital Signs and Metabolic Parameters

The monitoring of vital signs and basic metabolic parameters has advanced significantly from devices that measure a single parameter (e.g., temperature) cross-sectionally without the capacity for remote communication with those that record multiple parameters simultaneously (e.g., pulse, blood pressure, oxygen saturation, blood glucose), monitor continuously in real-time, and transmit digital reports to family and professional caregivers (Medical Mood Ring, Tadiran's MDkeeper).^40,41 In addition, fabrics with embedded biosensors have now been developed that permit continuous remote physiologic monitoring of multiple vital functions.^49–54 These “smart garments” are capable of alerting family and professional caregivers of aberrations from a prior baseline and incident medical conditions that may otherwise escape detection until complications are evident and unavoidable. Although prototype system evaluations have been conducted with small samples of nondemented subjects, a large-scale field trial has only recently been launched for one of these biotextile-based systems (SmartShirt).⁵³ Specific reconfigurations of these garments will likely be necessary to ensure the compliance and comfort of persons with dementia. These include light weight, capacity to embed biosensors in inconspicuous everyday clothing, resistance of these biosensor fibers to physiologic functions such as sweating, ease of laundering, and lack of discomfort or potential for allergic skin reactions. Moreover, the transmission of the clinical data to family and professional caregivers must be unobtrusive, even when significant deviations from the person's baseline are being detected (i.e., the garment must not trigger loud alarms).

Fall Detectors

Although both manual and automated simple alarm systems exist to alert caregivers of a fall, the automated ones must be worn at all times, and the manual ones require the user to activate the response system in the event of a fall. Two recent approaches attempt to overcome these limitations by using passive unobtrusive sensors.^31,32 The Smart Inactivity Monitor using Array-Based Detectors Project deploys wall-mounted low-cost, array-based passive infrared sensors to detect inactivity and falls.³¹ Although it does not require the individual to wear or activate a device, falls were accurately detected in only 37.5% of controlled laboratory experiments. In contrast, the University of Virginia is developing a piezosensor-based system that records floor vibration patterns.³² Laboratory experiments involving anthropomorphic dummies accurately detected falls in 100% of cases with no false alarms. Neither system has been deployed in real world residential settings with dementia subjects.

Example of a Computer Vision-Based Fall-Detection System

Recently, investigators have attempted to use video image processing technologies as an approach to fall detection.^33,57 Lee and Mihailidis³³ conducted a pilot study using 21 volunteers between the ages of 20 and 40 years to evaluate the efficacy and performance of a fall-detection system that uses a ceiling-mounted video camera. Trials were conducted in a mock-up bedroom setting, with a bed, a chair, and other typical bedroom furnishings. The subjects were asked to assume a series of postures, namely walking/standing, sitting/lying down in an inactive zone, stooping, lying down in a “stretched” position, and lying down in a “tucked” position. These five scenarios were repeated three times by each subject in a random order. These test positions totaled 126 fall-simulated tasks and 189 non-fall–simulated tasks. The true positive rate for fall detection was 77% with a false positive rate of 5%. The system has several major technical limitations including the fact that it can only track one person at a time, and often assumes that the tracked person and his or her mobility assistive device are one object. The investigators are grappling with these challenges and are now linking this fall-detection system with a community emergency response team, and experimenting with artificial intelligence techniques to determine what level of assistance a fallen person may require in various scenarios.³³

One additional study merits mention. Priplata et al.³⁴ measured eight postural sway parameters in 15 young and 12 elderly (mean age: 73 ± 3 years) participants who were asked to stand on viscoelastic silicone gel insoles that were embedded with three vibrating elements called tactors. All eight sway parameters improved for the elderly participants and, in fact, they outperformed younger subjects in several specific balance measurements.

Environmental Sensors

A wide variety of low-cost sensors are commercially available to measure a single factor (e.g., water usage) or multiple environmental contextual factors (e.g., appliance hazard sensors).²⁴ In the study of Biswas et al.,⁵⁸ the prototype agitation monitoring system for persons with dementia illustrates the ability of multiple simple environmental sensors working in concert with one another to tackle a clinical problem. The investigators instrumented a laboratory with acoustic, pressure and ultrasound sensors to detect movements of a single experimental subject. The intensity and duration of the movements were rated according to the total body movements and up and down movements sub-scales of the scale to observe agitation in persons with dementia of the Alzheimer type.⁵⁹ The ultrasonic and pressure sensors alone detected agitation in 59% and 73% of instances, respectively. The agitation recognition rate improved with multimodal sensor fusion with Bayesian inference to 94%. Although the clinical finding is inconsequential because it is from a highly structured, controlled, and predictable environment with only one nondemented subject, this prototype system demonstrates the application of multiple inexpensive sensors to tackle a specific clinical problem.

Advanced Integrated Sensor Systems

A wide range of advanced integrated sensor systems already exists and many are rapidly emerging. By fusing data from a network of heterogeneous sensors and applying artificial intelligence, these systems not only improve activity and behavioral recognition above and beyond the capacity of unimodal sensors, but also advance the level of sophistication of the supervision, guidance, and feedback provided to their users. We discuss three systems that have already been subjected to clinical studies with representative dementia subjects and two emerging research prototypes that will soon be ready for such studies.

CareWatch

Unattended home exits, particularly at night, pose considerable dangers to persons with dementia, and are burdensome and detrimental to the health of their family caregivers. CareWatch consists of a security system control panel, wireless receiver, and motion, door opening and bed occupancy sensors to alert the caregiver of both emergency and nonurgent situations through customizable text or voice alarms.⁴⁴ In addition to prototype system evaluations that have already been conducted, the investigators are currently conducting a randomized trial with dementia subjects by randomly assigning 27 homes to CareWatch and 28 homes to the control condition (usual care). Although final results have not yet been published, CareWatch has operated for >200 months of combined system time without any major failures. Moreover, there have been no unattended exits during the night in homes instrumented with CareWatch, although comparison data from control homes have not been provided. The investigators plan to measure the family caregiver's sleep, daytime fatigue, mood, burden, and depression after 12 months of system use.

COACH

This system represents a prototype system to guide persons with moderate to severe dementia through a handwashing task that has been shown to be stressful for family caregivers.⁴⁵ Using a video camera, hand-tracking bracelets, and machine learning algorithms, COACH monitors the progress of the handwashing activity, determines the context, and provides prerecorded verbal prompts if and when it detects a problem in task execution (e.g., forgetting to wash hands after using the soap). In a clinical trial of 10 subjects with moderate to severe dementia, COACH increased by 25% the number of handwashing steps that were correctly completed without caregiver assistance.⁴⁵ The investigators are currently refining COACH with a new color vision hand and object tracking system that obviates the need to wear tracking bracelets, and machine learning algorithms that are capable of handling uncertainty more robustly. Moreover, a visual prompting capability is being tested in comparison with the previously developed verbal feedback methods.

CareMedia: Video and Sensor Analysis for Geriatric Care

CareMedia leverages fundamental advances in video image processing to track and analyze the activities and behaviors of nursing home dementia unit residents.⁴⁶ Specifically, the project attempts to capture in real-time, continuously video/audio data that were processed to identify normative behavior, and aberrant low frequency, high impact behaviors such as falls, physical, and verbal aggression that often escape detection by noncontinuous recording methods. A feasibility study involving four ceiling-mounted video cameras and microphones in the nonprivate spaces of a locked dementia unit captured seven bouts of physical aggression (three un-witnessed by staff) and six elopements (none witnessed by staff) among eight residents over 80 hours of observation.⁶⁰ The investigators further observed that >75% of all social interactions in the shared spaces of this unit occurred during meal times. Subsequently, a more comprehensive study was conducted using 23 ceiling-mounted video cameras and microphones that recorded 15 consenting subjects with severe dementia 24 hours a day for 25 consecutive days, amounting to 13,800 camera-hours of video data. To date, trained human coders have identified 185 bouts of physical aggression in nearly 500 hours of meal time (lunches and dinners) videos (unpublished data). The investigators will use the human annotated video data to train machine learning algorithms to semiautomatically, and eventually automatically, identify activities of interest such as falls and aggression. Moreover, this voluminous dataset will provide training materials to develop data mining techniques and privacy protection mechanisms that will be absolutely critical to the clinical implementation of such a system.

Wearable Radiofrequency Transmitters

As part of a larger study, Almudevar et al.⁴⁷ recently instrumented the home of a person at risk for dementia with radiofrequency (RF) transmitters to reliably reconstruct his or her movement trajectory within the home. In addition to instrumenting the home with various motion detection devices and a small wireless network consisting of three RF receivers, the person at risk for dementia and his or her caregiver wore an RF transmitting wrist watch to record the movement trajectory. Analysis of a 3-week longitudinal record of RF data suggested that the proposed methodology reliably reconstructed the movement trajectory of the home occupants. Regions of high occupancy within the home were identified, and transitions between the regions were accurately tracked. Although the pilot findings are encouraging, further replication is necessary in the homes of a large cohort of subjects at risk for dementia, and with diverse home (architectural) designs that may pose challenges to the instrumentation and capture of RF data.

Proactive Activity Toolkit

Like Almudevar et al.,⁴⁷ the Proactive Activity Toolkit (PROACT) investigators also used RF technology to determine the performance characteristics of PROACT in automatically recognizing the activity of daily living (ADL) that was being performed and the quality of its execution.⁴⁸ Objects in a research home were instrumented with 108 RF identification tags, and the RF signals transmitted by these tags were recorded and analyzed by a prototype glove worn by the study participants. Over a 6-week period, 14 participants aged 25–63 years (mean: 39), randomly performed 12 of 14 prespecified ADL tasks (e.g., oral hygiene, toileting, washing, safe use of appliances). PROACT correctly inferred that an activity occurred 88% of the time; moreover, it correctly identified the specific ADL in 73% of cases. The failure to correctly identify an ADL was highest for activities closest to metals or water since both of these substances absorb radio waves that most RF identification tags use. Nonetheless, this is the first time that a pervasive computing approach has correctly inferred 9 of 14 ADLs in an experimental setting. Along with those of the aforementioned advanced integrated sensor systems, these findings provide strong impetus for continued research with intelligent assistive technologies that offer the promise of stabilizing or reducing dementia-associated functional disability.

DISCUSSION

This review has identified a range of commercially available and emerging assistive technologies that with further interdisciplinary research and modifications may have potential applications to dementia care. The considerable paucity of clinical trials specifically involving older persons with dementia, however, limits their current applications to this population. Their continued limitations notwithstanding, emerging products such as COACH demonstrate slow but steady progress in a field that began some decades ago with nonintelligent, primarily unifunctional devices to one that now incorporates heterogeneous sensor networks and artificial intelligence to sense and produce sophisticated predictive models of human activity and behavior. Clearly, much remains to be done by way of designing technologies and environments that are intelligent, context-aware, unobtrusive, passive (i.e., require minimal user initiation and maintenance), portable, inexpensive, in compliance with privacy regulations, and acceptable to their end users. Moreover, ultimate success must be measured not simply by functional improvement within limited specific domains but by personally meaningful impact on the user's global quality-of-life.⁸

A striking seminal observation of this review is the fact that most assistive technologies were developed principally for younger persons with typically nonprogressive traumatic or anoxic brain injuries, thus raising concerns about their generalizability to the progressive deficits associated with neurodegenerative dementias. We were able to identify only three clinical studies involving dementia subjects specifically, and these relied on small samples and varied in methodological rigor.^44–46 Future research with assistive technologies must proactively incorporate older adults in determining their specific needs, device design and privacy preferences, and in outcomes evaluations. Although some reports of user acceptance and preferences exist, the subjects of these investigations were uniformly cognitively intact, and often were questioned about their adaptation to the technology only after its implementation.^61,62 Curiously, although many older adults express receptivity toward assistive technologies in hypothetical scenarios, they tend to minimize personal need when such devices are offered to them.⁶¹

An approach that will be critical both in new product development as well as modification of existing ones is the concept of User Centered Design.⁶³ User Centered Design is a product development philosophy and multistage problem-solving process that aims to take into account user needs, preferences, and values upfront to optimize user acceptance of the end product. The process begins with an analysis of user characteristics (demographic, cognitive, physical, and perceptual), his or her physical and social environment, and tasks that are the target of the proposed technological intervention. Next, user goals that would be necessary for successful implementation of the product are identified (i.e., esthetics, ergonomic design, cost). After a detailed user and task analysis, conceptual designs and prototypes are developed and subjected to iterative modifications within an experimental setting before deployment in real-world settings with actual users. Although this approach requires greater initial research investment, it ultimately reduces product development time and cost while simultaneously improving user acceptance and satisfaction.⁶⁴ A detailed description of both the product design process and proactive involvement of users with dementia and their family and professional caregivers are an urgent research and development priority.

Beyond the complexity of designing devices that are commensurate with an individual's needs and preferences, critical research gaps also exist in the engineering and computer science domains. These include a) efficient collection and storage of voluminous real-time continuous data from multimodal sources (basic and advanced sensors, audio and video technologies, etc.) that lend themselves to user-friendly access and analysis; b) development of automated data reduction and mining techniques to point to clinically meaningful events and deviations from a prior baseline (i.e., finding the needle in a haystack); c) integration, analysis, and correlation of the data with clinical parameters; d) further advancements in wireless network technologies that are capable of transmitting real-time multimodal digital data confidentially to caregivers; and finally e) development of increasingly sophisticated computational and statistical techniques to model human activities and behaviors.⁶⁵

Ubiquitous health monitoring technologies also raise serious ethical considerations. The very systems that are designed to promote independence not uncommonly require varying degrees of privacy impingements to collect the data during both the developmental phase and the routine use. The CareMedia project is representative of a pervasive direct observational approach that collects personally identifiable data (facial and body images, voice, etc.) that precisely for that reason renders it ideally situated to proactively assist care providers in detecting physical and emotional changes from the individual's prior baseline.⁶⁶ Successful implementation of assistive technologies for research and clinical purposes will require not only an analysis of stakeholder-specific ethical considerations but also setting specific issues (i.e., personal home versus nursing home).^65,66 The Alzheimer's Association Working Group on Technologies has recently proposed guidelines for technology research and development for in-home persons with dementia that may serve as a starting point for this vitally important dialogue.⁶⁷ Likewise, the United States Department of Health and Human Services has published a report, “Barriers to Implementing Technology in Residential Long-Term Care Settings,” that offers researchers valuable guidance pertaining to unmet technology needs and approaches that may facilitate their successful and ethical implementation.⁶⁸

In conclusion, while some progress has been made in developing intelligent assistive devices that may find applications in dementia care, daunting computational and ethical challenges remain. The dialogue that has ensued over the last decade regarding the role of assistive technologies in modern healthcare is reflected in the rapidly increasing numbers of publications, and conferences and workshops that are specifically devoted to this topic. Moreover, there is a proliferation of research “smart homes,” academic and industry-based laboratories focused on computational models of human activity, behavior and cognition, and the formation of the Alzheimer's Association Working Group on Technologies. Commercial products that result from these endeavors will be arriving en masse in the coming years, and it will be important for clinical researchers to critically evaluate their performance characteristics and utility in “real world” settings. Academic interest in this area has also led to increased attention to developing national funding mechanisms that would specifically consider interdisciplinary technology research applications so that these do not fall through the cracks of existing agencies.⁶⁸ In the final analysis, however, evidence-based clinical and cost-effectiveness data will be critical to reimbursement by third-party payors if widespread acceptance and use of technological resources is to become a reality.

APPENDIX: GLOSSARY OF TECHNOLOGICAL TERMS AND CONCEPTS

Advanced Integrated Sensor System

A system that uses information from different types of sensors (e.g., motion, temperature, and pressure detectors) separately or in combination to record the day-to-day life activities of individuals.

Artificial Intelligence

Computing technologies that are designed to sense, understand, and respond to an activity or environment in a way that a human would.

Biosensors

Sensors that provide continuous, real-time monitoring of vital signs, and other physiological functions.

Cognitive Orthotic

A device that assists with cognitive tasks such as memory (often used synonymously with cognitive aid).

Context-Aware

Computer capacity for determining the environmental and situational context for the task at hand before prompting a reminder or delivering an intervention (e.g., transmitting a prompt to a driver only if it detects deviations from the navigational path).

Data Mining

Computing techniques that process voluminous amounts of data from disparate sources (e.g., environmental sensors, audio, video) to identify events of interest such as falls. These techniques permit scientists to find the proverbial “needle in a haystack.”

Intelligent Assistive Technologies

Technologies that sense and respond to user needs are adaptable to changing situations and compensate either for physical or cognitive deficits.

Smart Homes

Homes with a range of built-in sensors (e.g., temperature, pressure, fall detector) that monitor an individual's daily functioning and provide prompts for task completion as needed.

Ubiquitous or Pervasive Healthcare Technologies

Computational devices placed in personal homes, motor vehicles, public spaces, and buildings to provide real-time monitoring of an individual's day-to-day activities (e.g., fully instrumented residential neighborhoods that are now being developed for persons with disabilities).