Using Passive Sensing to Estimate Relative Energy Expenditure for Eldercare Monitoring

Abstract

This paper describes ongoing work in analyzing sensor data logged in the homes of seniors. An estimation of relative energy expenditure is computed using motion density from passive infrared motion sensors mounted in the environment. We introduce a new algorithm for detecting visitors in the home using motion sensor data and a set of fuzzy rules. The visitor algorithm, as well as a previous algorithm for identifying time-away-from-home (TAFH), are used to filter the logged motion sensor data. Thus, the energy expenditure estimate uses data collected only when the resident is home alone. Case studies are included from TigerPlace, an Aging in Place community, to illustrate how the relative energy expenditure estimate can be used to track health conditions over time.

I. Introduction

By the year 2030, the elderly population will double [1]. Technology that can help seniors “age in place” has been highlighted in recent years, spurred by an aging population. In response to this trend, many researchers have been investigating new approaches in caring for the elderly. One example of this research focus at the University of Missouri has resulted in TigerPlace (TP), an aging in place community for seniors. Technologies to support independent living for older adults have been available for several years, such as [3][4][5][6][7][17]. Live-in laboratory smart homes with sensors and actuators have also been established such as the Aware Home at Georgia Tech [8] and MIT's PlaceLab [9].

One focus of our research is the creation of intelligent software that uses sensors to uncover patterns of activity helpful to caregivers [2]. Sensor networks have been installed in 36 apartments in TP. Data collection has been ongoing for over 2 years in many apartments. This longevity has facilitated the investigation of sensor data and algorithms for recognizing changes in health conditions. The goal is to capture patterns representing physical and cognitive health conditions and then recognize when these patterns begin to deviate from the norm.

In previous work [10], we focused on the visualization of motion activity density and time away from home (TAFH) in the form of a density map that can be used to monitor the life style patterns of older adults. The work was extended in [11] with a dis-similarity measure based on texture features for comparing density maps and automatically determining changes in activity patterns.

In a motion density map, different colors represent different levels of density in the motion sensor data, as shown in the scale in Figure 1. The density d is computed as the number of all motion sensor hits s during an hour divided by time at home during that hour, t. The density is defined as $d=\frac{s}{t}$. The motion sensors generate events every 7 seconds if continuous motion is detected. In lab tests, we have captured one-sensor density values ranging from 14 hits/hour for near motionless sitting to over 300 hits/hour for pacing. In Figure 1, the X-axis shows hours in a day; the Y-axis shows days in a month. Black represents TAFH. The density levels range from 50 (gray) to 150 (yellow) to 300 (green) to 550 (blue).

Figure 1.

An example of a motion density map

In this paper, a relative energy expenditure measure based on motion density is proposed for evaluating energy expenditure changes of elders. Although the activity density map and similarity measure are useful as a visualization and for automatically identifying changes, these changes are computed as distance values without a direction of being better or worse. An estimate of energy expenditure can provide a measure of health, with the assumption that a decline in energy expenditure correlates to deteriorating health conditions. In this paper, we propose estimating the relative energy expenditure using motion density, TAFH and visitor information. The visitor information is extracted using a fuzzy inference system as described in Sec. II. In Sec. III, we present the energy expenditure algorithm; case studies from TigerPlace residents are included in Sec. IV. We conclude in Sec. V.

II. Algorithm to Estimate Visitor Time

Because we are interested in capturing activity of the elderly resident, it is important that we filter out the times when one or more visitors are present. Visitors with high activity levels have an especially large impact on the motion sensor density, which can add noise to the energy expenditure estimate of the resident.

To estimate visitor times, we consider motion sensors that represent different locations. An apartment structure and configuration of location motion sensors determine a sequence of motion sensor firings when people move about the apartment. Non-overlapping motion sensors cannot fire at the same time if there is only one person in the apartment. For example, Figure 2 displays a room plan for one of the TigerPlace apartments; there are six location motion sensors marked from one to six in this apartment. Bathroom sensor 1 and kitchen sensor 5 cannot fire in the same second if there is only one person in the apartment. To make this feature more sensitive, we can assume that no one can walk from bathroom sensor 1 to kitchen sensor 5 in less than 1 second. Using this strategy, sensor firings that are caused by multiple people can be distinguished. Each data point is flagged with a visitor marker which represents whether this data point is caused by multiple people. Next, we evaluate whether there are visitors between the visitor markers. For this purpose, two features are extracted and used to classify the data into actual and false visitor events using a set of fuzzy rules [12]. First, the activity density tends to increase significantly if there is an active visitor. Second, the time length between visitor markers is typically not very long. Based on these two features, a fuzzy inference system was developed to evaluate possible visitor events.

Figure 2.

The floor plan of an apartment showing sensor placement

A. Choice of Membership Functions

Based on the characteristics of the features described above, we use trapezoidal-shaped membership functions [13]. The trapezoidal curve is a function of a vector, x, and depends on four scalar parameters a, b, c, and d, as given by equation 1.

$$f(x,a,b,c,d)=\text{max}\left(\text{min}\left(\frac{x-a}{b-a},1,\frac{d-x}{d-c}\right),0\right)$$ (1)

The fuzzy logic system of discriminating visitors has two inputs, duration between visitor markers, and the density ratio r which is the ratio of density d between visitor markers to average density m during this period of time as shown in equation 2.

$$r=\frac{d}{m}$$ (2)

Figure 3 and Figure 4 show the membership functions for the linguistic variables used. For membership functions of duration between visitor markers (Figure 3), the X-axis represents time in minutes, and the Y-axis represents degree of membership function. For membership functions of the density ratio (Figure 4), the X-axis represents the ratio, and Y-axis represents degree of membership function.

Figure 3.

Membership functions for the duration between visitor markers

Figure 4.

Membership functions of the density ratio

B. Fuzzy Rules and Inferencing

Table I displays the rules of the fuzzy system which discriminate visitors. Duration and Ratio represent the two features described above, which are duration (in minutes) between two visitor markers and the density ratio. “Confidence of visitor” which changes from poor to excellent represents the fuzzy output membership functions from 0 to 1. The rules were developed empirically, and a Sugeno [14] inference system is used.

TABLE I.

Fuzzy rules for visitor discrimination

Rule	Duration	Ratio	Confidence of visitor
			Linguistic	Membership
1	Short	Low	Good	0.67
2	Short	Medium	Excellent	1
3	Short	High	Excellent	1
4	Medium	Low	Poor	0.33
5	Medium	Medium	Good	0.67
6	Medium	High	Excellent	1
7	Long	Low	Poor	0.33
8	Long	Medium	Poor	0.33
9	Long	High	Good	0.67

C. Validation

For validating the visitor recognition algorithm, two validation data sets have been used. The first data set is from a test apartment at TigerPlace. Researchers conducted experiments for two weeks. A log file was written by researchers to record their activity. The log file was used as ground truth to validate the algorithm of discriminating visitors.

Sixteen visitor events were recorded; 14 were found by the algorithm with 1 false positive. The false positive was caused by a cleaning person. The cleaning person came into the apartment several minutes after a visitor left. The time between the visitor and the cleaning person (several minutes) was detected as a visitor event, since time length between the two visitor markers is very short and the density during this period of time happened to be higher than normal (due to the cleaning activity).

In the 16 visitor events, 12 of them had a high activity visitor; the algorithm found all of these 12 events. Four visitor events had a low activity visitor; the algorithm found only 2 of these. This validation result indicates that the algorithm for discriminating visitors works very well for high activity visitors. This result is adequate for our energy expenditure estimate, because low activity visitors do not significantly affect the motion sensor density.

The data collection by researchers in the test apartment is not suitable for collecting sensor data with a specified condition over long periods of time (e.g., several months). To address this problem, a sensor network simulator was developed [15]; the output of the simulator was validated against the output of the physical sensor network to ensure realistic data simulation. The second validation data set was collected using this simulator; 35 visitor events were recorded, and 34 were found by the algorithm. The accuracy rate is 97% with no false positives. This validation result also indicates the algorithm for discriminating visitor events works well.

The visitor identification algorithm has also been applied to logged sensor data from TigerPlace apartments. Figure 5 provides an example in which the upper figure is the density map of a resident in May, 2007, and the bottom figure is the visitor map of the same month. The X-axis represents hours in a day, and the Y-axis represents days in a month in both maps. In the visitor map, the white areas represent visitor events. From Figure 5, we can see that the visitor areas correspond to high activity density areas. On the 3rd, 10th, 17th, 24th and 31th from 11 AM to 12 PM there are visitor events regularly; these events are consistent with the cleaning schedule every Thursday.

Figure 5.

An example of a motion density map (top) and the corresponding visitor events (bottom). White areas represent the visitor events identified with the visitor recognition algorithm.

III. Energy Expenditure Estimate

A. Energy Expenditure Calculation

Energy expenditure refers to the amount of energy that a person uses to do daily activities. A low energy expenditure (and especially a declining energy expenditure) can be an indication of poor health. In this section, we describe an estimation of relative energy expenditure using motion sensor data to capture general activity level. To estimate energy expenditure of an elderly resident, we exclude the visitor time and compute the estimate based on the time spent alone at home.

The algorithm for estimating relative energy expenditure is given below:

1. Calculate the times for visitor events using the visitor recognition algorithm in Sec. II.

2. Calculate the time away from home [10].

3. Filter the visitor times and the time away from home from the motion sensor data.

4. Calculate the total number of sensor hits, S, for each day after filtering.

5. Compute the total time, T, the resident was either away-from-home or had visitors for each day (in hours).

6. Calculate the total number of sensor hits, E, each day using equation 3.

   $$E=\frac{S}{24-T}$$ (3)

To make the energy expenditure estimate more consistent, the density of long periods of time away from home (over 12 hours) is substituted by the average density of the resident. Here, we compute the energy expenditure estimate for a 7-day period.

B. Testing Results

The energy expenditure estimate has been applied to logged sensor data from TigerPlace apartments. The next section includes two case studies showing weekly estimates over time with corresponding health profiles. Figure 6 is an example. The X-axis represents the week number, and the Y-axis represents the average motion events per hour by week which estimates relative energy expenditure. From this figure we can see that the energy expenditure of this resident declined over time, which was consistent with the resident's health history. We are investigating the use of relative energy expenditure estimates as one parameter in monitoring health and changing activity level of elders. In the case of declining energy expenditure, a clinician would investigate further and offer an intervention appropriate for the resident's condition.

Figure 6.

Case study #1: weekly relative energy expenditure estimates for 03/01/2007~12/01/2007. The area highlighted in green shows the corresponding month for Figure 7. The area highlighted in red shows the corresponding month for Figure 8.

IV. Case Studies

Two case studies from TigerPlace residents are described in this section [16]. These case studies provide an illustration of the relative energy expenditure estimates over time and are used to test the algorithm proposed in Sec. III. A brief health history is included for each case study.

A. Case Study #1

Although she was never diagnosed, it is suspected that the resident in case study #1 had dementia, in part due to an irregular pattern shown in her motion density maps. Figure 6 shows the weekly energy expenditure estimate, which declines over time. At the beginning of March 2007, the energy expenditure estimate is around 140. In the end of November, the energy expenditure estimate is about 90.

The motion density maps of July and December 2007 are displayed in Figure 7 and Figure 8 to show the changes in activity patterns and decrease in overall activity level. In Figure 7, the upper figure is the density map for July 2007, and the bottom figure is the visitor map of the same month. From Figure 7, we can see that the visitor areas correspond to high activity density areas in the density map. In Figure 8, the upper figure is the density map for September 2007, and the bottom figure is the visitor map of the same month. The visitor areas also correspond to high activity density areas in the density map.

Figure 7.

Activity density map (top) and visitor map (bottom) for case #1 July 2007

Figure 8.

Activity density map (top) and visitor map (bottom) for case #1 September 2007

In Figure 6, the density map of July (Figure 7) corresponds to the bars highlighted in green (weeks 17~20), and the largest energy expenditure of this month is around 140; the density map of September (Figure 8) corresponds to the bars highlighted in red (weeks 25~28), and the largest energy expenditure of this month is around 100. The energy expenditure of July is more than that of September. Comparing the density map of July (Figure 7) and the density map of September (Figure 8), several observations can be made. In Figure 7, there are many green and yellow areas in the density map of July, and the color tone of this map is deep yellow to deep green which corresponds to a density of 150 to 300. In the density map of September (Figure 8), both green and yellow areas reduce, and the color tone of this map is grey to deep yellow which correspond to a density of 50 to 200. From the color changes of the density maps, it is clear that the activity level of this resident decreases dramatically. This conclusion is consistent with the result of the relative energy expenditure analysis. Second, the black areas which represent the away-from-home events decrease. The black areas in the density map of September (Figure 8) are much less than that of July (Figure 7). This means that the resident went out much less in September (Figure 8). This usually indicates a decreasing activity level, which is consistent with the energy expenditure analysis.

B. Case Study #2

Figure 9 shows the weekly relative energy expenditure estimate for case study #2. This resident was diagnosed with depression. The energy expenditure also showed a declining trend over time. Compared to the previous resident, the energy expenditure of this resident was much lower.

Figure 9.

Case study #2: weekly relative energy expenditure estimates for 12/01/2008~03/01/2010. The area highlighted in green shows the corresponding month for Figure 10. The area highlighted in red shows the corresponding month for Figure 11.

In Figure 9, the density map of December 2008 (Figure 10) corresponds to the bars highlighted in green (weeks 1~4) and the largest energy expenditure of this month is around 50; the density map of February 2010 (Figure 11) corresponds to the bars highlighted in red (weeks 61~64), and the largest energy expenditure of this month is around 32. The energy expenditure of December 2008 (Figure 10) is much more than that of February 2010 (Figure 11). The density maps of December, 2008 and February, 2010 are displayed in Figure 10 and Figure 11 and illustrate a decreasing activity level. Comparing the two density maps, several observations can be made. In Figure 10, the color tone of the density map of December 2008 is grey to light yellow which corresponds to a density of 20 to 100. In the density map of February 2010 (Figure 11), the color tone fades to grey which corresponds to a density of 10 to 50. From the color changes of the density maps, it is clear that the activity level of this resident decreases dramatically. This conclusion is consistent with the result of the energy expenditure analysis. Second, the black areas which represent the away-from-home events decrease. Initially, there is little activity in the apartment but much time away from home. Later, there is much less time away from home, and the away-from-home time indicates inconsistent meals in the dining room. This also shows the decreasing activity level of this resident, which is consistent with the energy expenditure analysis.

Figure 10.

Activity density map (top) and visitor map (bottom) for case #2 December 2008

Figure 11.

Activity density map (top) and visitor map (bottom) for case #2 Feburary 2010

V. Conclusion

In this paper, we have proposed an algorithm for estimating visitor events from passive infrared motion data and have included results from validation experiments. The visitor algorithm is then used to estimate the relative energy expenditure by computing the motion sensor density for the times in which the resident is home alone. Two case studies from TigerPlace residents are included to show energy expenditure profiles and corresponding health histories. The case studies illustrate declining energy expenditure estimates that correspond to health problems.

The validation of the visitor recognition algorithm is significant, because the visitor events have a crucial impact on the interpretation of the sensor data. However, there are limitations on the passive motion sensors, and the visitor recognition algorithm can only reliably detect visitors with a higher activity level. Motion sensors cannot provide personal identification. Thus, the classification made by the system will contain a degree of ambiguity to identify who performed the activity, and it is also a challenge to identify the number of persons in all cases. Also, the motion sensor fires every 7 seconds if there is motion nearby, and useful information can be lost because of the 7 second resolution. On the other hand, the motion sensors used in this project are inexpensive, readily available, and easy to deploy.

This paper includes our preliminary work in investigating the relative energy expenditure estimate as one parameter in identifying early signs of illness and functional decline. Our aim is to provide an automated system for detecting changes in patterns and activity level that will aid caregivers in the monitoring process. An alert system is currently being tested for this purpose.

Our future research goals will expand on feature extraction and automated reasoning at different time scales using the logged sensor data, focusing especially on early detection of changes in patterns. The major goal of our extended research team is to introduce advanced sensors, novel signal/image processing, and high level reasoning to enhance the independence and safety of older people while maintaining privacy and minimizing interference. Keeping track of the day-to-day physical, cognitive, and social capabilities plays a major role in this effort.