SVM to Detect the Presence of Visitors in a Smart Home Environment

Abstract

With the rising age of the population, there is increased need to help elderly maintain their independence. Smart homes, employing passive sensor networks and pervasive computing techniques, enable the unobtrusive assessment of activities and behaviors of the elderly which can be useful for health state assessment and intervention. Due to the multiple health benefits associated with socializing, accurately tracking whether an individual has visitors to their home is one of the more important aspects of elders’ behaviors that could be assessed with smart home technology. With this goal, we have developed a preliminary SVM model to identify periods where untagged visitors are present in the home. Using the dwell time, number of sensor firings, and number of transitions between major living spaces (living room, dining room, kitchen and bathroom) as features in the model, and self report from two subjects as ground truth, we were able to accurately detect the presence of visitors in the home with a sensitivity and specificity of 0.90 and 0.89 for subject 1, and of 0.67 and 0.78 for subject 2, respectively. These preliminary data demonstrate the feasibility of detecting visitors with in-home sensor data, but highlight the need for more advanced modeling techniques so the model performs well for all subjects and all types of visitors.

I. Introduction

The emerging problem of supporting and caring for an aging population has received a great deal of attention in recent years. The high cost of institutionalization (i.e. nursing homes) and the reduced quality of life in such environments has led to a search for options that allow seniors to maintain their independence. One approach to helping seniors remain in their homes is ambient sensing environments, in which technologies placed in the home provide continuous data about the health status of the residents [1–3]. This “smart home” approach is most effective when it is unobtrusive, i.e., the seniors are not required to do anything outside of their normal daily activities, and thus unobtrusive technologies that are integrated into the individual’s home play an important role in this approach. Unfortunately, identification of activities using such passive technologies is much more challenging if there are multiple people in the environment [4], since the IR passive sensors cannot differentiate who is moving through the space. Thus, being able to identify when multiple people are present is an important part of interpreting and disambiguating in-home data.

The ability to identify when visitors are in the home is particularly important for assessing socialization in seniors. Socially isolated individuals generally exhibit higher blood pressure [5], higher all cause mortality [6], and are at increased risk of developing cognitive decline [7]. Since a decrease in visitors to the home often heralds an increase in the isolation of an individual, detecting visitors to the home can allow for early detection of changes in socialization levels, enabling earlier intervention and support.

A number of approaches have been proposed to manage the multi-person identification problem. More complex sensors such as video cameras have the advantage of allowing identification of different individuals moving through the home, but most seniors consider the use of video monitoring as a violation of their privacy. Body worn tags (RFID, UWB, WiFi etc.) can provide good information about the relative location of multiple individuals in the home [8, 9]. In an elderly population, however, this approach is impractical especially for long periods of time as they may forget to wear the sensors or take them off when they become uncomfortable. Alternative approaches not requiring body-worn tags include algorithmic techniques that attempt to disambiguate passively acquired sensor data based on statistical properties. For example, we recently used Gaussian Mixture Models to separate the walking speeds of individuals moving through a sensor line in two-person homes [10]. While this approach was effective for its specific purpose (measuring walking speed), it was not more widely applicable to identifying when visitors were in the home. Crandall and colleagues used graph and rule-based algorithms as well as Bayesian Updating Graphs to estimate the number of individuals present [11]. While effective, this technique required a very high density of sensors, which is not practical for large-scale monitoring. Thus, techniques to identify visitors to the home in passively monitoring environments are still needed.

The main focus of this paper is to develop a method to detect the presence of visitors in the home environment using only the data provided by wireless motion sensors in each room. A support vector machine (SVM) was developed to distinguish these events, using several key features from the sensor data as the input and subject diary entries detailing when visitors were present in the home as the ground truth. The quality of the model is estimated using thousand fold repeated random sub-sampling.

II. Methods

A. Data Collection and Pre-processing

The data used in this study came from the homes of ORCATECH Living Lab subjects. The ORCATECH Living Lab is a group of 31 seniors who have agreed to participate in ongoing research about the role of technology in maintaining independence. A core set of technologies is continually maintained in their homes, including pyro-electric motion sensors (MS16A, x10.com) in each room and contact sensors (DA10A, x10.com) on the doors to the home. Only the motion sensor data were used in this study. For a period of 6 weeks, thirteen subjects completed diary entries twice a day—once at around noon and once around 11:00pm—regarding the specific times visitors were present in their homes. The diaries were incomplete for six participants (for example, the participant would report that they had a visitor but not record the times of the visit), and five participants lived in multi-person homes where there was little time with only one person in the home. Two participants lived alone and had complete diary entries; those data were used for this analysis. Incidentally, these data suggest the difficulty in using self-report as the source of socialization data [12].

Over the course of the six week period, subject 1 recorded 31.75 hours where visitors were present and subject 2 recorded 89.5 hours where visitors were present. These self-report data were used as the ground truth in model development. During this time, sensor data from the motion sensors in these subjects’ homes were also collected. All data from any days where any sensor in the house was not functioning properly were excluded. This resulted in the exclusion of two days of data from subject 1 and zero days of data from subject 2. A total 4.75 hours of visitor data were therefore excluded for subject 1.

To create the features used in the model development, the sensor data from each home was divided into 15 minute epochs. Four different sets of features were then calculated for each epoch.

The first feature set we incorporated in the model is the total time the subject spent in each room or the dwell time per room. This feature set was chosen because visitor events like dinner guests or a game night would increase the dwell time in the dining room or living room while the visitor is present as well as decrease the dwell time in whatever room of the house the subject normally resides in while alone. Dwell time was calculated by assuming that no sensor firings were missing, and therefore that consecutive firings of the same sensor represented continuous movement in the same room. The dwell times then form a series of triplets {R, t, d} where R is the room ID, t is the start of the dwell time, and d is the duration of the dwell time in the room. The duration d is calculated as the difference between the time of the first firing in a room A and the time of the first subsequent firing in a different room B. The start time t is the time of the first sensor firing in the consecutive sequence of sensor firings within room A. From this we calculated the total dwell time for each room, for each 15 minute epoch.

The second feature set used in the model is the total number of sensor firings in each room. The underlying hypothesis is that as the number of people present in the home increases, the activity level in the home will also increase, thus increasing the number of sensor firings.

The third feature set used in the model is the number of room transitions for each room couplet. In a house with n rooms, there are n(n−1)/2 different room couplets, and the number of transitions was computed for each of these couplets. Transition profiles are expected to change in two ways when visitors are present in the home. First, anytime a visitor is present in the home and in a different room than the resident, the number of transitions between those two rooms is likely to increase. On the other hand, anytime the visitor and resident are in the house in the same room, the number of room transitions is expected to decrease or remain the same while the number of sensor firings and dwell time for that room will likely increase.

The fourth feature set corresponds to the time of day. Because human activity patterns follow a circadian rhythm, there will be differences in normal activity patterns as a function of the time of day. It is therefore important to include the time of day as a feature in the model design. The time of day was coded as a dummy variable with ‘1’ assigned to the four epochs starting at midnight, 12:15am, 12:30am, and 12:45am, and linearly increasing each hour; thus ‘24’ was assigned to the four epochs starting at 11:00pm, 11:15pm, 11:30pm and 11:45pm.

B. Model Development

SVM was used to classify the data. SVM [13] is a well established machine learning technique to classify data which has been used previously in smart home environments to detect abnormal behavioral patterns [14]. In general, SVM is used to determine non-linear boundaries for classification. The theory underlying SVM is based on the notion of mapping the raw data into a high-dimensional space where it can be categorized by a hyperplane decision boundary. When projected back into the original data space, the hyperplane is mapped into a nonlinear surface. The mapping can be performed directly using kernel-based transformation. Because the decision boundary for distinguishing visitor epochs from non-visitor epochs is likely to be non-linear, as illustrated in fig. 1, we used SVM with a Gaussian Radial Basis Function kernel with σ = 1 to map the data into a higher dimensional space to perform the classification.

Fig. 1.

An example of total dwell time in the living room versus the total number of sensor firings in the dining room for subject 1 from 5:00–7:00pm for both the case where a visitor was reported in the home and the case where no visitors were reported during this time. The non-linear decision boundary between these events is also shown.

In order to limit the complexity of the model, we reduced the feature set to include only those features corresponding to rooms that are frequently used when visitors are present. That is, only the sensor features from the kitchen, dining room, living room, main bathroom and the transitions between these rooms were used in the final model.

Initially, the SVM model was trained using all epochs during all 24 hours of the day to train and test the model. However, because the features included in the model may look considerably different during the night (because the subject is likely sleeping), this could lead to biased results. We therefore also tested the model using only those epochs that occur during the daytime hours—that is, 7am to 11pm—to eliminate potential bias associated with the night hours.

Because there were so many more epochs where visitors were not present (3732 for subject 1 when night hours are excluded) as compared to epochs were visitors were present (108 for subject 1), we trained the model on ¾ of the visitor epochs. When nighttime epochs were excluded, we trained on 15% more non-visitor epochs than visitor epochs and 50% more non-visitor epochs than visitor epochs when night hours were included. We tested on all the remaining data. For all model fits, we used 1000 fold repeated random sub-sampling to determine the mean out-of-sample performance and 95% confidence intervals. We report on those results below.

III. Results and Discussion

Table I presents the results of the SVM model for classifying epochs where visitors are and are not present in the home for each subject and both models. As can be seen, excluding nighttime epochs from the model does not significantly affect the sensitivity of the analysis. However, the specificity decreases by about 0.07 when nighttime epochs are excluded for both subjects. This decrease in the specificity when nighttime epochs are excluded is to be expected as the sequences of sensor firings that are likely to occur during the night (e.g. bedroom, bathroom) are very different from those that occur during the day (e.g. kitchen, living room). However, even when the nighttime epochs are excluded, the model still performs reasonably well with sensitivity and specificity of 0.89 and 0.80 for subject 1, and 0.67 and 0.69 for subject 2, respectively.

Table I.

Sensitivity and specificity of the SVM model for visitor detection for subject 1 for all epochs in a 24-hour period, daytime epochs only and 95% confidence intervals (CI) for 1000 random splits of the data into model fitting and classification sets.

	Subject 1		Subject 2
	Sensitivity	Specificity	Sensitivity	Specificity
All 24-hour Epochs	0.902	0.861	0.672	0.782
95% CI	(0.778, 1.00)	(0.813, 0.893)	(0.589, 0.756)	(0.752, 0.810)
Daytime Epochs	0.888	0.796	0.670	0.693
95% CI	(0.741, 1.00)	(0.743, 0.850)	(0.584, 0.753)	(0.657, 0.728)

It is also important to note that the model performs considerably worse in detecting epochs where visitors are present for subject 2 than for subject 1. The average sensitivity for subject 1 is 0.90 as compared to 0.67 for subject 2. There are several possible causes for this difference in sensitivity. First, the ground truth in this model is limited to the self-report of the subjects. It is well known that self report, especially in the aging population, is prone to inaccuracies [12]. Forgetting to report visitors in the home would decrease the specificity of the model, while inaccurately reporting the time that visitors arrive and leave would decrease the sensitivity. It is therefore possible that subject 2 did not accurately record the precise times that visitors arrived and left the home, resulting in a lower sensitivity of the model.

The home layout and placement of sensors in these two homes is also considerably different. Subject 1 has an open kitchen that feeds into the dining room and living room. In this layout, it is easy for an individual in the kitchen to interact with individuals in the living and dining rooms. As a result, the number of transitions between these three rooms can increase dramatically when visitors are present. Further, the sensors are well placed so subjects can be detected in any room of the house. Because there are so few dead zones in the house, changes in behavioral patterns due to the presence of a visitor can be detected more easily, resulting in a higher sensitivity of the SVM model in detecting visitors.

In contrast, the sensor layout appears to have problems for subject 2. While there is a sensor in the dining room, its purpose is not to detect individuals in the dining room, but rather to detect individuals leaving the home. The field of view of the sensor is pointed mostly at the front door. As a result, it seems to have high error rate in accurately detecting the presence of an individual in or passing through the dining room. This would dramatically affect the performance of this model in detecting visitors as it is dependent on the transition profiles, number of sensor firings, and dwell times in these rooms.

These differences in home layouts and in the positioning of the sensors highlight the importance of sensor placement in a smart home. In configuring a smart home, it is critically important to place sensors appropriately to accurately detect the movement of the subject throughout the entire home. Placing a sensor with an occluded or limited field of view is likely to limit the ability to detect behavioral patterns or changes in behavioral patterns in the home environment. As Crandall’s study showed [15], a large number of sensors can greatly improve the recognition of subject movements. However, it remains an open question how the number and placement of sensors can be optimized to minimize the intrusion into the senior’s home while still allowing for good recognition of visitors.

While the current approach for detecting visitors in the home performs well it still has several limitations. First, the visitors recorded in this study were only daytime visitors—neither subject reported overnight guests. However, the presence of an overnight guest in the home would be incredibly important to identify from both a socialization perspective and to effectively model multiple individuals in the home. The model therefore needs to be generalized for the case where visitors remain in the home overnight.

Further, the model only detects the presence or absence of any number of visitors in the home, but cannot quantify how many visitors are present. Several of the diary entries report varying numbers of visitors present in the home, and accurately detecting differences between large groups of visitors and more personal encounters with small groups is important in assessing socialization practices.

Finally, this model was only tested on two subjects and two home layouts, and therefore needs to be validated on more subjects. Because the behavioral patterns associated with the presence of visitors in the home will change for different people, the generalizability of the model must be tested for multiple individuals and multiple home layouts. This is especially important as the performance of the model varied considerably between the two subjects tested.

IV. Conclusions

Because of the multiple health impacts of socialization in the growing aging population, continually and unobtrusively monitoring the socialization practices of elderly individuals is increasingly important. Key to this goal is the detection of visitors present in the home. Using the dwell time, number of firings, number of transitions, and hour of the day as features in a support vector machine, we were able to distinguish periods where visitors are present in the home with 90% sensitivity and 86% specificity for subject 1, while the sensitivity and specificity were lower at 67% and 78% for subject 2, respectively.

In the future, we plan to improve this result by incorporating the door sensors that detect the opening and closing of the front door into the model. Because visitors must enter and exit the home through the doorway, modeling changes in the presence or absence of a visitor in the home around door firings may increase performance of the model and allow for visitor detection at a greater granularity.