BESI: Behavioral and Environmental Sensing and Intervention for Dementia Caregiver Empowerment—Phases 1 and 2

Abstract

Background and Objectives:

Caregiver burden associated with dementia-related agitation is one of the commonest reasons a community-dwelling person with dementia (PWD) transitions to a care facility. Behavioral and Environmental Sensing and Intervention for Dementia Caregiver Empowerment (BESI) is a system of body-worn and in-home sensors developed to provide continuous, noninvasive agitation assessment and environmental context monitoring to detect early signs of agitation and its environmental triggers.

Research Design and Methods:

This mixed methods, remote ethnographic study is explored in a 3-phase, multiyear plan. In Phase 1, we developed and refined the BESI system and completed usability studies. Validation of the system and the development of dyad-specific models of the relationship between agitation and the environment occurred in Phase 2.

Results:

Phases 1 and 2 results facilitated targeted changes in BESI, thus improving its overall usability for the final phase of the study, when real-time notifications and interventions will be implemented.

Conclusion:

Our results show a valid relationship between the presence of dementia related agitation and environmental factors and that persons with dementia and their caregivers prefer a home-based monitoring system like BESI.

Background and Objectives

Dementia is a brain syndrome characterized by decline in 1 or more (Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition) ¹ or 2 or more (International Statistical Classification of Diseases and Related Health Problems, 11th Revision) ² cognitive domains causing significant impairment with independence in everyday activities. Dementia due to Alzheimer’s disease is the commonest cause and accounts for 60% to 80% of all cases of dementia. ³ There are 5.8 million people in the United States with dementia due to Alzheimer’s disease. ³ In 2018, more than 16 million Americans provided unpaid care for person with dementia (PWD), of which the vast majority were family members and friends. They provided an estimated 18.5 billion hours of unpaid care worth US$233.9 billion. ³

Agitation in dementia has been defined as “inappropriate verbal, vocal, or motor activity that is not explained by needs or confusion per se” ⁴ and is “consistent with emotional distress.” ⁵ Caring for a PWD requires differing sets of skills with increasing expertise being needed as the disease progresses. ⁶ Agitation can be physically nonaggressive, aggressive, or verbally agitated behavior. ⁷ Agitation occurs in up to 90% of PWD ^8,9 and is one of the principal factors prompting institutionalization of community-dwelling PWD. ¹⁰ Caregivers of PWD report high levels of emotional, physical, and financial stress ¹¹ and are susceptible to disease and health complications. ¹² Caregivers experience increased burden due to the challenges and stresses of caring for PWD. Forty percent (40%) of caregivers suffer from depression, ¹³ with rates of depression increasing as the severity of dementia worsens. ¹⁴

Behavioral and Environmental Sensing and Intervention for Dementia Caregiver Empowerment (BESI) was developed to facilitate early detection and prevention of agitation, reduce caregiver burden, improve caregiver self-efficacy, and extend aging in place. Technology for health applications in community-dwelling older adult populations generally focuses on home and health monitoring using various types of sensors, for example, body-worn motion, ¹⁵ acoustic, physiological (eg, heart rate), and passive infrared (PIR) motion detectors, ^{16 -18} video cameras, ^19,20 pressure pads, and smart-assisted living systems with multiple sensing modalities. ²¹

The Alarm-net project ¹⁶ estimated activities of daily living using many PIR sensors per living space. CareWatch ¹⁸ was designed to assist caregivers of PWD by tracking PWD location and monitored them at night to improve caregiver sleep using a commercial security system and several sensors. Dem@care ²⁰ targets timely assessment and promotion of self-independence of PWD with extensive use of cameras in the living space and instrumentation of household objects. The BehaviorScope project ²¹ estimated activity patterns by detecting indoor location using cameras, PIRs, and door/window contact sensors. These systems are often intrusive and come with substantial concerns about privacy, extensive use of “human-in-the-loop” at the monitoring end, and limited environmental sensing modalities and scope of applications. Hence, they do not sufficiently meet the requirements for the BESI project.

The BESI was designed as an unobtrusive, privacy-protecting, real-time platform with minimal burden on the human-in-the-loop (Figure 1). It is a cyber-socio-physical system comprised of:

Figure 1.

Graphical representation of Behavioral and Environmental Sensing and Intervention for Dementia Caregiver Empowerment (BESI).

1. body-worn inertial sensors and in-home environmental (acoustic, light, temperature, humidity, barometric pressure) and motion sensors;

2. smartwatch and tablet applications for caregivers to log PWD activities, agitation events, and caregiver data;

3. data analysis techniques to detect and assess agitation and environmental context from sensors;

4. models for the relationship between agitation and the environment that are trained for each PWD/caregiver dyad based on b and c; and

5. Automated real-time notifications to the caregiver based on c and d.

As a cyber-socio-physical system, it combines computing, networking, and social elements with physical systems to create a system that has technical capabilities to enhance quality of life. ²² The cyber-physical elements include the environmental and wearable sensors, while the social elements include the caregiver observations logged on the tablet application, the PWD, and the living environment.

The PWD-worn inertial sensor (eg, TEMPO, ¹⁵ Shimmer, ²³ Pebble ²⁴ ) uses a built-in accelerometer to collect data on physical activity and agitation patterns. Environmental sensors are housed in relay stations placed in different sections of the home based on the living space model. ²⁵ This model conceptualizes the home as a constellation of 3 overlapping spheres—the physical/architectural space, the social and cultural space, and the technological space.

In a previous study, ¹⁵ we demonstrated the capability of wearable technology to assess physical agitation based on PWD movements and validated TEMPO-based agitation assessments against the observation-based Cohen-Mansfield Agitation Inventory (CMAI) ⁷ and Aggressive Behavior Scale. ²⁶ This feasibility study was a pilot for what would become the BESI system and was completed in a long-term care facility with nursing home patients and professional caregivers.

We used the Progressively Lowered Stress Threshold model ^27,28 as a conceptual model for agitation in dementia. It posits that stress build-up over time leads to outbursts and that changing the environment can help reduce stress. Caregiver and patient-centered choices of interventions have been most effective. ^28,29

Aims

Phase 1

• Confirm that BESI can accurately collect the needed data.

• Develop a tablet-based application to document PWD activities and agitation episodes that are linked to the system using time stamps.

Phase 2

• Validate BESI’s capability to assess agitation and environmental events in situ.

• Develop a model based on the dyad-specific relationship between agitation and the environment in preparation for real-time interventions in phase 3.

Research Design and Methods

This is a mixed method, prospective study using descriptive, qualitative, and quantitative measures of caregiver-PWD dyads as it incorporates the collection, analysis, and mixing of qualitative and quantitative data within a single study. ³⁰ A hybrid remote ethnographic method (REM) ^31,32 is used to implement the approach. We refer to “hybrid” because the method was not comprised solely of remote data collection (tablet device, wearable device, sensors) but also involved self-report data collection in participants’ homes, such as face-to-face data collection in the form of interviews; demographic, medical, and social information; and psychiatric questionnaires used for clinical assessment.

Phase 1 (System Verification)

Phase 1 was conducted in controlled settings (laboratory and homes of 2 healthy volunteers) and study settings (homes of 2 study dyads). The clinical team visited dementia support groups and obtained verbal consent from participants. We provided participants with a broad outline of the study. Information on caregiver-friendly terminology used to describe agitation was elicited from participants and subsequently integrated into the tablet application. The usability of the application was then tested in dementia support groups using the Systems Usability Scale ³³ and additional related questionnaires. ³⁴

Home visits were made to interested dyads to describe the study in detail and obtain and document informed consent, demographics, and medical information. An assessment battery ^7,35,36 (Table 1) was completed and the BESI system installed at a second home visit. The assessment battery includes measures of agitation, ⁷ cognition, ^35,36 neuropsychiatric symptoms, ³⁷ dementia staging, ³⁸ depression, ^39,40 quality of life, ⁴¹ sleep, ⁴² functional assessment, ^43,44 burden, ⁴⁵ and self-efficacy, ⁴⁶ for one or both members of the dyad.

Table 1.

Assessment Battery.

Assessment	Domain Measured	Validity/Reliability
Cohen-Mansfield Agitation Inventory–Community (CMAI-C) 5	Agitation	α = .92
Modified Mini-Mental Status Examination (3MS) 35	Cognition	α = .91
Clinical Dementia Rating (CDR) Scale 36	Cognition	α = .89
Neuropsychiatric Index–Brief (NPI-Q) 37	Neuropsychiatric symptoms and psychopathology in dementia	α ≥ .79
Functional Assessment Staging in Alzheimer’s Disease (FAST) 38	Dementia staging Numerical values assigned to subscores (1-16)	Rater consistency – .86 Rater agreement – .87
Cornell Scale for Depression in Dementia 39	PWD depression >12 probable depression	α = .84
Center for Epidemiologic Studies Depression Scale Caregivers (CES-D) 40	Caregiver depression	α = .88
Alzheimer’s Quality of Life Scale (QoL-AD) 41	Quality of life	α = .88-.89
Pittsburg Sleep Quality Index (PSQI) 42	Sleep quality	α = .83
Barthel Index 43,44	Functional capacity (0-100)	kw: .93-.95
Zarit Burden Scale–Short Form 45	Caregiver burden	α = .88
Revised Scale for Caregiving Self-Efficacy (RSSE) 46	Caregiver self-efficacy  -Obtaining respite  -Disruptive behaviors  -Upsetting thoughts	α > .8

Abbreviations: kw, Kruskall-Wallis H test; PWD, person with dementia.

Phase 2 (System Effectiveness)

In Phase 2, we sought validation of BESI’s effectiveness in assessing agitation and environmental events and developed models based on dyad-specific relationships between these 2 variables. BESI was deployed for 30 days in each home. Caregiver data from the application were reviewed weekly to correlate information provided by BESI and the caregiver. Caregivers were contacted to verify or clarify information as needed.

Recruitment

In phase 1, 2 dyads with past or current participation in dementia support groups were recruited. In phase 2, anticipating that the recruitment process would become more difficult with increasing length of deployment, we expanded recruitment to include community events, health fairs, wellness centers, and physicians’ offices and partnered with community advocacy groups. This was done to better address the issues of gender, racial, and geographic diversity in our sample. Participants enrolled in prior phases were eligible to continue to participate in subsequent phases if they continued to meet the inclusion criteria.

Inclusion Criteria

The caregiver-PWD dyad is the focus of this study. Person with dementia regardless of etiology or disease severity aged 50 years and older and living at home with a stable caregiver(s) for at least 2 months. Caregivers may be spouses/significant others, children, other relatives, or full-time paid caregivers. Each dyad serves as their own control.

Exclusion Criteria

Dyads are excluded if the PWD had been hospitalized in the previous 2 months, lived in a long-term care facility or in multiple homes, had no primary caregiver, or if there were any concerns about the caregiver’s capacity to provide consent.

Ethics

Participants deemed not to have the capacity to consent ⁴⁷ are included in this study using the legally authorized representative process. All participants provided assent which was obtained verbally and documented in the research record. This study was approved by the Carilion Clinic Institutional Review Board #1820, Roanoke, Virginia.

Sample Size

In phase 1, we recruited 2 dyads enrolled for 7 days each. In phase 2, 10 dyads were enrolled for 30 days each.

Clinical Data

Clinical data from demographic forms and assessment battery were obtained for each dyad. Data from the assessment batteries were collated and helped quantify and substantiate behaviors and their impact on the PWD and caregiver separately and as a unit.

Tablet Application Data

A tablet application was used to collect caregiver-reported agitation episodes and severity level; PWD physical, behavioral, and social activities; and the emotional states of the dyad using graphical user interface. This allows caregivers to provide additional information that cannot be gathered from the sensors (eg, mood of caregiver, observations on the PWD). This information is essential for understanding the complete spectrum of emotional states and degree of agitation. Icons were used to display information (real-time objects) and developed for the older adult population and thus be easily interpreted by any user group. Icons and text colors were chosen to maintain enough contrast ratio enabling them to stand out on screen, grabbing the user’s attention, a necessary step for human visual information processing. Different colors were used for each icon, as users might group objects with similar properties. Semantically near icons were used to help older adults to offset decreases in the perceptual capabilities. ⁴⁸ In phase 2, we simplified the tablet design and visual presentation by offering key choice selections only. Data were uploaded to a secure web-based server to protect user privacy.

Wearable Devices Data

In phase 1, we used Shimmer3 ²³ for the PWD and the Pebble ²⁴ for the caregivers. The Pebble ²⁴ was used for both the PWD and caregivers in phase 2. They feature a programmable 3-axis accelerometer along with Bluetooth connectivity. The inertial motion data from the PWD are programmed to sample its accelerometer at 50 Hz and sends that data continuously to any connected relay station. The PWD Pebble ²⁴ runs an application that displays a standard watch face on the foreground and handles accidental button presses while collecting data in the background. The caregiver Pebble ²⁴ runs an event marker application that time stamps agitation episodes. ⁴⁹

Environmental Sensor Data

We collected environmental data on room temperature (kelvin), light level (lux), relative humidity (%RH), air pressure (Pa), and audio noise level (dB). Microphones were programmed to filter out identifiable voice frequency components and only stored low-frequency audio noise. Doorway sensors (dual PIR directional motion detecting sensor) determined location by collecting in-home movement. These data were obtained using off-the-shelf sensors.

Relay and Base Station Data

Each relay station houses the in-home environmental sensors. The relay stations have an embedded computer board and both Bluetooth and Wi-Fi connectivity. No cameras were used. Relay stations communicate with the base station via Wi-Fi on a local network to synchronize time and to send status updates periodically. The base station facilitates remote monitoring and system management.

Data Analysis

Data analysis involved mixed data integration, data mining, and in situ modeling using factor and path analysis techniques. Analyses using SAS 9.4 were conducted in the preliminary stages to understand the univariate characteristics of the data, most of which did not demonstrate patterns of normal distribution. ^50,51 The act of transforming mixed data formats leads to several problems with the reliability and validity of results. ⁵² Consequently, preliminary analysis methods were exploratory in phase 1 and more confirmatory in phase 2. We used categorization for preliminary qualitative data analysis.

We applied data characterization methods to identify the extent to which the data could be analyzed with methods associated with causal paths and clustering. This involved assessment of the distributions, assessment of variance or dispersion, and exploration of communality. The distributions skew to the right (positive) with kurtotic features away from the tail and required distribution free or nonparametric modeling methods. Dispersion and reliability were measured using the coefficient of variation. We explored communality as a preliminary feature through correlations and eigenvalues based on correlations and determinant matrices.

In phase 2, we developed a model that would describe specific relationships between agitation and environmental factors using statistical and computational methods. Teager energy was extracted from the raw accelerometer data and correlated with personalized patterns of physical agitation and preagitation activities. Each environmental sensor in each relay station generated a separate data stream. Low-frequency audio noise data representing room-level sounds were fused to extract vocal activity, localization, in-home activity/context information, social density, and interactions especially during an agitation event. All the streams were filtered and imputed separately to minimize noise effects and to achieve synchronization. These preprocessed signals were then put into the data modeling pipeline to extract features and build models of correlation between environmental parameters and agitation events.

Statistical Model to Validate BESI Capabilities

The focus of this analysis was at the n-of-1 level where each dyad was analyzed as a single case study rather than aggregating across dyads. This was used to validate the extent to which ambient variables aligned with the causal paths associated with agitation. SAS 9.4 software was used to implement an exploratory factor analysis (EFA) to identify the latent factors.

Model Development Based on Dyad-Specific Relationships

In the preliminary classification design, a supervised network classifier was used to distinguish between the phases of recorded agitation episodes. Nominal decreases in error are a major source for the network to reach a virtual standstill. To counteract this issue, the resilient back-propagation network (Rprop) was chosen due to its convergence speed, accuracy, and robustness with respect to training parameters (Table 2). ⁵³

Table 2.

Comparison of Neural Networks (NN).

Concept	Traditional BPNN	Rprop NN
Determining ΔWeight	Uses magnitude of the gradient	Uses (only) sign of the gradient
Value of ΔWeight	Uses single learning rate for all weights/biases	Maintains separate weight deltas for each weight/bias (each weight/bias adapts during training)
Training speed	Slow	Fast

Abbreviation: BPNN, back propagation neural network.

The preliminary goal was to determine the environmental factor or factors that would best distinguish agitation from nonagitation. Therefore, each environmental data type was analyzed separately. We used the neural network to classify input patterns as either agitation or nonagitation data. For each environmental data type, the mean, maximum, and standard deviation (SD) values of each 1-minute segment of raw data were calculated and stored. In addition, acknowledging time-of-day differences in agitation, the hour of each 1-minute time segment was used as the fourth and final feature. The caregiver record of time of agitation was used as ground truth. An agitation episode may persist from a few minutes to over an hour. We used the recorded agitation minute along with 3 minutes which precede and follow it to classify the middle agitation phase, for a total of 7 minutes. Six minutes prior to and subsequent to the described phase are used as preagitation and postagitation, respectively. Thus, a total of 19 minutes of data were used to characterize an agitation episode. For each minute of the 19-minute duration of agitation, the mean, maximum, SD, and time of day (hour of agitation) were used as inputs to the classification module. The neural network was then trained using all the valid agitation minutes, along with an equal amount of nonagitation data. We did not use endogenous and exogenous parameters to analyze these data.

Not every reported agitation was used in our analysis. For example, agitations which occur between 8:01 pm and 5:59 am were excluded. Likewise, when 2 recorded agitations occurred within 20 minutes of one another, only the first agitation was selected. Lastly, agitations recorded during the first and last day of recording was excluded due to system verification during setup and disband.

Each 19-minute time segment was combined to form the agitation data set, which, along with a same-sized randomly selected nonagitation data set, was passed on to the neural network for binary classification. Due to missing data, zeroes, and negative values, some agitation and nonagitation data were excluded from further analysis.

In our preliminary analysis, the number of hidden layer neurons was established as 10 for all dyads. Likewise, the number neurons in the input layer was 4, representing the 4 features. The output layer consisted of a 2-value binary set, namely [1 0] or [0 1]. Weights were initialized to small random values, the learning rate was initialized at 0.4, and the momentum set to 0.95. The transfer function used for the hidden layer is sigmoidal. For the output layer, the softmax transfer function was used. Finally, the data were divided into 70% training, 15% testing, and 15% cross-validation. Cross validation is used to prevent overprocessing of the classification network. Mean square error was used as the performance criteria. The neural network was tested for each type of environmental data.

Power Analysis

Much of the data were combined to reflect temporal and spatial behaviors of dyads using weighted averaging, relative frequencies, covariance values, and medians. Given this complexity and the large data set, we obtained from each dyad sample size estimations using traditional power equations, and central limit theorem assumptions would not be adequate and could introduce both type I and type II error risk. No power analysis was required for phase 1 as it was a trial phase. Power was assessed in phase 2 using goodness-of-fit index (GFI) and Bentler comparative fit index (CFI).

Results

Demographic Data

Dyads from both phases had difficult behaviors and agitation outbursts, and thus, all 12 dyads were analyzed together. The PWD ages ranged from 68 to 92 years (x = 79.67; SD = 7.50). Person with dementia were 50% male/female. Educational level was high (6 >12 years; 2 >16 years; 1 >18 years). Despite attempts to reach diverse participants, all were Caucasian. Ten were married and 2 widowed. Family history of dementia was varied. Nine lived in single-family homes and 3 resided in apartments, with 2.13 being the average number of persons in the home. The dyads spent approximately 18 to 21 hours together daily. Caregiver ages ranged from 43 to 86 years (x = 71.83; SD = 12.18). Caregivers were 67% female. Educational level was also high for caregivers (10 >12 years; 2 >16 years).

Person With Dementia Assessment Battery Data

The CMAI-C ⁵ describes the specific behaviors of agitation and is rated by the caregiver as to frequency of behaviors and of items that score ≥2, that is, the behavior occurred (Table 3). Average frequency of behaviors was 62 (SD: 16.16), and average number of agitation behaviors was 11.25 (SD: 4.79). The Modified Mini-Mental Status Examination, ³⁵ a brief assessment of cognition, showed average scores of 55.5 (range: 23-82). The Clinical Dementia Rating Scale ³⁶ stages dementia with 0 indicating no cognitive impairment, 0.5 mild cognitive impairment, and ≥1 dementia. The PWD scores range from 0.9 to 2.16. Ten were within the dementia category and 2 in the mild cognitive impairment category. The Neuropsychiatric Index–Brief ³⁷ severity ranged from 3 to 20, and caregiver distress ranged from 0 to 27. The Functional Assessment Staging of Dementia ³⁸ stages cognition from normal (1 and 2) to severe (7). Five scored 4 (mild), two 4.5 (mild to moderate), three 5 (moderate), and one each scored 6a and 6b (moderately severe). The Cornell Scale for Depression in Dementia ³⁹ rates depression in PWD and only 1 PWD was rated >12, that is, probable depression. Quality of life ⁴¹ scores ranged from 22 to 51, with higher numbers indicating better quality of life. The Pittsburg Sleep Quality Index ⁴² assesses sleep quality (worse quality of sleep with higher scores). Sleep difficulties were not severe in this study. The Barthel Index ^43,44 assesses PWD needs in activities of daily living, and higher scores show more independence (0-100). One participant was severely dependent, 7 moderately dependent, and 4 minimally dependent.

Table 3.

PWD Assessment Battery Results.

Assessment	Mean	Max Score	SD	Range
CMAI-C 5 —agitation frequency	62.00	203	16.16	35-92
CMAI-C 5 —number of behaviors (≥2)	11.25	29	4.79	2-19
3MS 35	55.5	100	18.14	23-82
CDR 36	1.50	3	0.37	0.9-2.16
NPI-Q 37 –symptom severity	11.33	36	5.53	3-20
NPI-Q 37 –caregiver distress	12.33	60	8.94	0-27
FAST 38	4.75	7	0.94	4-7
Cornell Scale for Depression 39 –PWD	3.25	38	3.55	0-13
QoL-AD 41 –PWD	38.33	52	7.9	22-51
QoL-AD 41 –caregiver rated	32.00	52	4.65	25-38
PSQI 42 –PWD rated	2.25	21	1.86	0-6
PSQI 42 –caregiver rated	5.50	21	2.91	2-11
Barthel Index 43,44	83.75	100	11.70	60-100

Abbreviations: CDR, Clinical Dementia Rating; CMAI-C, Cohen-Mansfield Agitation Inventory–Community; FAST, Functional Assessment Staging in Alzheimer’s Disease; 3MS, Modified Mini-Mental Status Examination; NPI-Q, Neuropsychiatric Index–Brief; PSQI, Pittsburg Sleep Quality Index; PWD, person with dementia; QoL-AD, Alzheimer’s Quality of Life Scale; SD, standard deviation.

Caregiver Assessment Battery Data

Depression in caregivers was assessed using the Center for Epidemiologic Studies Depression Scale–caregivers (Table 4). ⁴⁰ Higher scores indicate more symptomatology. Depression was not severe in these caregivers. Using the Zarit Burden Scale, ⁴⁵ 2 rated little or no burden, 9 indicated mild to moderate burden, and 1 rated moderate to severe. The Revised Scale for Caregiving Self-Efficacy ⁴⁶ demonstrated a wide range of confidence in caregivers.

Table 4.

Caregiver Assessment Battery Results.

Assessment	Mean	Max Score	SD	Range
Cornell Scale for Depression 39 – caregiver	10.33	38	6.34	4-21
CES-D 40	12.58	60	9.69	0-33
Zarit Burden Scale–short form 45	18.42	48	6.73	8-35
RSSE 46 –obtaining respite	67.92	100	26.45	24-100
RSSE 46 –disruptive behaviors	68.46	100	17.22	46-100
RSSE 46 –upsetting thoughts	77.84	100	18.19	28-95

Abbreviations: CES-D, Center for Epidemiologic Studies Depression Scale Caregivers; SD, standard deviation.

Tablet Application Data

Evolution of caregiver mood with agitation was observed through the day (Figure 2). On day 1, the caregiver reported mostly favorable states (“okay,” “busy”) and reported 3 low-level severity agitation episodes (rated 3, 2, 2; max 10). On day 3, the caregiver started out in a favorable state (“hopeful”), but with the occurrence of 2 major agitation episodes (both rated 8/10), this changed to “busy” and eventually “exhausted.”

Figure 2.

Caregiver emotion with agitation events.

In phase 2, the average number of submitted surveys decreased from weeks 1 to 4 (Figure 3)—agitation (8.2-6.3), daily (15.6-11), and activity (23.2-18.6). The most common agitation levels reported by caregivers, in descending order of frequency, were 3, 4, 5, and 1 (range: 2-7).

Figure 3.

Changes in number of agitation surveys, daily surveys, and activity surveys submitted over 4 weeks on day 6.

The most frequent agitation types reported, in descending order, were verbal aggression, withdrawal, communication trouble, and frustration (Figure 4). The most reported caregiver emotions were categorized as neutral—“okay” and “hanging in there” followed by “busy” and “calm.” The next most frequent emotions were categorized as negative—“frustrated,” “exhausted,” and “discouraged.”

Figure 4.

Survey data of frequency versus behavior during agitation episodes during phase 2.

Wearable Devices Data

Applying threshold-based detector of physical agitation enabled estimates of agitation. ⁵⁴ Figure 5 shows a time window across agitation episodes on 1 day from a weeklong deployment and a temporal relationship between agitation and Teager energy magnitudes. This algorithm is designed to have lower type II errors and high sensitivity as it misclassified one high-energy activity period as physical agitation (false positive) but did not miss any physical agitation episodes. In phase 2, we observed a wide variation in the number of marked agitation events, that is, button presses (Figure 6), throughout each deployment with the overall average number of marked events decreasing between weeks 1 and 4 (4.67-3.11).

Figure 5.

Accelerometer data, Teager energy, and detected agitation event on day 6.

Figure 6.

Changes in the number of button presses submitted over 4 weeks on day 6.

Environmental Sensors Data

Figures 7 and 8 show luminescence and temperature patterns and low-frequency audio noise levels, respectively, in 2 different rooms (bedroom and living room) for 5 consecutive days. The agitation episodes are marked on both figures and demonstrate the variability in environmental parameters across rooms temporally and during agitation episodes. In phase 2, one of the matrices used to measure system effectiveness is the availability of collected environmental data. This is represented as a percentage of the obtained data in each deployment. On average, we collected 98% data for each phase 2 deployment.

Figure 7.

Luminescence and temperature data across 2 rooms for 5 days.

Figure 8.

Low-frequency audio noise data across 2 rooms for 5 days.

Audio Processing Data

In phase 1, the audio sensor system did not yield enough information for more complex audio processing. As the deployments progressed in phase 2, we observed that the audio data could have a potential in detecting the early stages of verbal agitation and an upgraded audio data collection system was implemented. The new system extracted useful audio features, for example, pitch, volume, voice quality, and discarded the audio stream for privacy protection. To validate all audio features, in-lab experiments were conducted with 2 students imitating conversations with silences, normal conversations, and conversations with verbal agitation. The classifier result using these features yielded 98.8% accuracy in differentiating between verbal agitation and nonverbal agitation episodes. By comparing these audio data with the deployment tablet survey on verbal agitation episode reports in one deployment, the classifier produced 76.7% accuracy in differentiating the PWD verbal agitation events from normal conversation situations.

Statistical Model Data

Temperature, pressure, and time showed strong correlations with agitation in phase 1 and thus were retained for further analyses in phase 2. The n-of-1 analysis served to validate BESI capability by helping to identify, for each dyad, the ambient factors that were correlated with variance in a latent factor, for example, for each data set for 4 agitation episodes in 1 dyad, the 30 minutes surrounding the timestamped episode produced 924 analyzed data points. The varimax rotation method was used in the EFA.

Temperature, pressure, and time demonstrated high factor loadings of 0.77, 0.93, and 0.96, respectively, showing strong Spearman rho correlations with Teager energy scores. These values ranged from r _s = 0.70 to 0.85 (P =.05, df = 75). The fit values were moderate with values relatively near but below the threshold of 0.90 (GFI: 0.89 and Bentler CFI: 0.85).

Tables 5 –10 show the accuracy results of distinguishing between agitation and nonagitation using environmental data for 6 dyads in phase 2. The number of agitations varied per dyad. Due to excluded minutes of data described earlier, an inconsistency in the number of minutes of agitation was also realized. Precision is calculated as the number of agitations correctly classified divided by the total number of minutes that should be classified as agitation. Recall is calculated as the total number of agitations correctly classified divided by the total number of minutes that were classified as agitation. Along with recall and precision, F scores, which leverage recall and precision, were also calculated for both the training and testing data sets. To further clarify the nearly perfect accuracies, only one communal relay station was used in this preliminary report—the one that centered the living room area. For these 6 dyads, most of the agitations occurred in this area of the home. Also, there was a high cutoff epoch of 2000, which means as long as the validation data did not indicate overtraining, the data continued to train until one of the other stopping criterion was met.

Table 5.

Agitation Classification Accuracies for Dyad A.

Total Number of Agitations (minutes): 36 (677)	Training			Testing
	Recall (%)	Precision (%)	F Score (%)	Recall (%)	Precision (%)	F Score (%)
Audio (dB)	81.8	87.0	84.31	83.0	86.2	84.57
Light (lux)	82.5	91.9	86.95	81.2	91.7	86.12
Temperature (°K)	81.3	94.8	87.51	81.5	94.1	87.34
Humidity (%)	82.6	92.4	87.25	80.2	92.4	85.85
Air pressure (Pa)	82.2	90.7	86.23	82.2	90.9	86.33

Table 6.

Agitation Classification Accuracies for Dyad B.

Total Number of Agitations (minutes): 21 (392)	Training			Testing
	Recall (%)	Precision (%)	F Score (%)	Recall (%)	Precision (%)	F Score (%)
Audio (dB)	84.0	96.2	89.66	81.6	95.1	87.9
Light (lux)	83.6	98.1	90.25	82.3	97.0	89.0
Temperature (°K)	83.3	98.2	90.15	82.1	97.7	89.2
Humidity (%)	85.8	97.3	91.16	83.5	97.7	90.0
Air pressure (Pa)	84.2	96.5	89.91	82.8	94.1	88.1

Table 7.

Agitation Classification Accuracies for Dyad C.

Total Number of Agitations (minutes): 9 (152)	Training			Testing
	Recall (%)	Precision (%)	F Score (%)	Recall (%)	Precision (%)	F Score (%)
Audio (dB)	94.2	100.0	97.03	94.1	100.0	96.98
Light (lux)	93.6	99.9	96.62	92.7	100.0	96.21
Temperature (°K)	94.7	100.0	97.26	93.0	100.0	96.34
Humidity (%)	92.1	100.0	95.89	91.3	100.0	95.46
Air pressure (Pa)	92.4	100.0	96.05	92.6	100.0	96.14

Table 8.

Agitation Classification Accuracies for Dyad D.

Total Number of Agitations (minutes): 27 (496)	Training			Testing
	Recall (%)	Precision (%)	F Score (%)	Recall (%)	Precision (%)	F Score (%)
Audio (dB)	93.6	100.0	96.69	93.5	100.0	96.66
Light (lux)	93.6	99.0	96.22	93.7	99.5	96.51
Temperature (°K)	93.7	99.9	96.71	93.0	99.9	96.34
Humidity (%)	93.8	99.9	96.76	94.2	99.9	97.01
Air pressure (Pa)	93.0	99.9	96.32	93.7	99.9	96.73

Table 9.

Agitation Classification Accuracies for Dyad E.

Total Number of Agitations (minutes): 10 (181)	Training			Testing
	Recall (%)	Precision (%)	F Score (%)	Recall (%)	Precision (%)	F Score (%)
Audio (dB)	93.8	100	96.79	93.3	100	96.55
Light (lux)	94.4	100	97.12	94.3	100	97.07
Temperature (°K)	93.9	99.6	96.65	94.5	99.27	96.94
Humidity (%)	94.5	99.8	97.09	95.1	100	97.50
Air pressure (Pa)	94.3	99.8	96.95	94.2	99.49	96.78

Table 10.

Agitation Classification Accuracies for Dyad F.

Total Number of Agitations (minutes): 79 (1475)	Training			Testing
	Recall (%)	Precision (%)	F Score (%)	Recall (%)	Precision (%)	F Score (%)
Audio (dB)	87.1	96.6	91.62	87.8	96.5	91.97
Light (lux)	91	98	94.53	90.1	98.1	94.21
Temperature (°K)	87.6	98.3	92.46	87.9	98.2	92.81
Humidity (%)	87.5	98	92.46	87.3	98.3	92.47
Air pressure (Pa)	87.7	97.8	92.45	87.9	97.8	92.59

These results signal a valid relationship between the presence of PWD agitation and environmental factors within the homes of the dyads, with most F scores percentages ranging within the 80s and 90s, reaching 100% for some data types.

Discussion and Implications

New methods of managing and analyzing diverse sensor streams are valuable in an increasingly instrumented world. This study uses REMs, providing information from a quasi-naturalistic observation of a cyber-human system designed to support caregivers of PWD. Our results, based on data from 12 dyads enrolled in phases 1 and 2 of the BESI study, showed high potential for early agitation detection and consequent intervention and facilitated targeted changes in BESI, thus improving its overall usability as the study moved from phases 1and 2 to phase 3.

The tablet application was updated between phases 1 and 2 to make it easier to use for caregivers. ³⁴ The visual design was simplified by offering key choice selections, thereby minimizing scrolling confusion in participants. The process of installing BESI was also streamlined to minimize disruption in the home setting (eg, obtaining a layout of the home at the initial home visit to facilitate sensor placement during the second home visit, keeping aesthetics in mind). Working closely with caregivers promoted the creation of clear communication lines with the clinical team based on the caregivers preferred method, for example, e-mail, texts. We also set up a predetermined midpoint check-in with caregivers, either in person or by phone, based on the dyad needs.

Our results lead to the understanding that the dyads prefer a home-based monitoring system like BESI. Identification of factors that participants had difficulties with led to the individualization of the word lists on the tablet and increased efficiency in the clinical assessment and education on the wearable device and tablet use, leading to shorter deployment visits and increased comfort with the new technology. These participant-oriented design decisions influenced the team in understanding how caregivers and PWD dyads interact with technology. Caregiver emotions were reported in mostly neutral terms followed by negative ones. This trend remained consistent among most of the dyads throughout phase 2, thereby providing us a general idea about the caregivers’ overall emotional state.

We collected 98% of the environmental sensor data for each phase 2 deployment, inferring that the system is effective in collecting the required data. Although the BESI system does not resolve the agitation recognition dilemma, the current system is robust and scalable to cover a variety of floor plans and an arbitrary number of rooms for any real-home deployment.

Comparing audio data with deployment tablet data on verbal agitation reports demonstrated clear differentiation in PWD verbal agitation events from normal conversation situations. Our preliminary result also shows that the triggers and signs of dementia agitation can be dynamic and PWD-specific. Thus, it is difficult to create one model that can accurately predict/detect agitation for every PWD. Dyads and their accompanying data arose from varying contexts including the layout of the residence, individual differences in interactions with wearable devices, and error variances introduced by uniqueness of ambient factors in the home as measured by sensors. We used the environmental sensor data in our prediction model to predict potential upcoming agitation episodes. The prediction model uses the agitation labels which the caregiver provided through the tablet. Even though we received enough agitation labels to build the prediction model, we recognized that they may not be precise.

Prior to running the neural networks, we accounted for missing data, zeros, inaccurate data values, and so on. Although we were not trying to predict the “exact time” using our model, the inexact timestamp of the agitation could limit our model’s ability. Nevertheless, the model should still inform us of the upcoming agitation episode with an acceptable accuracy.

Overall, the dissimilarities in accuracy results can be used to corroborate the selection of optimal feature sets for further processing. Based on these models, the ambient factors of humidity, light, and temperature showed promising results as factors strongly associated with agitation. The undetermined precision in agitation detection limits real-time intervention recommendations for phase 3. Recruitment of caregivers and PWDs in the home setting has been challenging despite support from community stakeholders, particularly in recruiting racially diverse dyads. Internet access and speed also proved to be a limitation in rural settings.

Assessments informed levels of cognitive impairment, functional abilities, and offered insight into factors potentially impacting these, such as sleep and depression.

The primary aim of the BESI study is to detect agitation early enough that we can suggest changes whether interpersonal, communication, or environmental. Phase 3 will identify agitation in the early stages and provide just-in-time intervention recommendations to the caregiver with the goal of preventing behavioral outbursts. The goal of reducing caregiver burden and eventually reducing risk of institutionalization may prove difficult to assess, even with the use of multiple validated assessment tools and developing technologies. However, this capability is in much demand especially as we learned of caregiver desire to “help” even with noted lack of previous experience.

The BESI study hopes to emphasize the role of interprofessional collaborations as essential in the development of technologies that address critical societal needs. Given the rising older adult population and increasing health-care costs, ⁵⁵ there is an acute need to find a compassionate yet cost-effective solution for dementia care. Community-based care that promotes aging-in-place while alleviating caregiver burden must be a goal. Context matters, and as technologies increase in complexity, designers and developers will need to rely on ethnographic methods. System deployment in situ, even at the prototype level, provides meaningful data to support user requirements.

The outcome of this project presents an opportunity for PWD and caregivers to reduce the likelihood of moving out of familiar and nurturing ecosystems. Future studies include determining the optimal feature set to be used in distinguishing agitation episodes from nonagitation episodes, incorporating caregiver mood in the analysis, and providing just-in-time notifications to alert caregivers to potential agitation and offering behavioral intervention recommendations or suggestions based on unique dyad assessment information, for example, hobbies, music preferences, activity levels, mood.

Follow-up studies will evaluate if BESI provides enough caregiver support to effectively resolve most day-to-day challenges and lessen the need for clinician involvement, potentially reducing health-care burden and associated costs. Ultimately, this work contributes to our understanding of customizing interventions, integrating technology, and prioritizing the lived context to provide support to caregivers challenged by complex disorders such as dementia.