Implementation study of wearable sensors for activity recognition systems

Abstract

This Letter investigates and reports on a number of activity recognition methods for a wearable sensor system. The authors apply three methods for data transmission, namely ‘stream-based’, ‘feature-based’ and ‘threshold-based’ scenarios to study the accuracy against energy efficiency of transmission and processing power that affects the mote's battery lifetime. They also report on the impact of variation of sampling frequency and data transmission rate on energy consumption of motes for each method. This study leads us to propose a cross-layer optimisation of an activity recognition system for provisioning acceptable levels of accuracy and energy efficiency.

1. Introduction

In recent years, we have witnessed a rapid surge in assisted living technologies because of a rapidly ageing society. The ageing population, the increasing cost of formal healthcare, the caregiver burden and the importance that individuals can continue living independently, all motivate the development of a new ubiquitous health monitoring system.

One of the important services that can be offered by such a system is remotely assessing the physical and cognitive well-being of the people by monitoring the activities of daily living. These activities can be used to characterise human behaviours, and with continuous monitoring, anomalies such as falls can be detected early and required actions can be taken quickly.

However, fall detection is a major challenge in the public healthcare domain, especially for the elderly, and reliable surveillance is a necessity to mitigate the effects of falls. The technology and products related to fall detection have always been in high demand within the security and the healthcare industries. An effective fall detection system is required to provide urgent support and to significantly reduce the medical care costs associated with falls.

A low-cost unobtrusive solution for an all-day and any-place activity monitoring system can be a wearable activity recognition system. For a wearable activity recognition system, multiple wearable sensor motes are used to form a self-managing wireless body area network (WBAN). Sensed data is continuously transmitted to a base station mote. This data is then used to recognise different activities through various processing techniques ranging from simple rule-based approaches to complex machine-learning-based approaches. However, there are challenges that arise and need to be addressed when developing mobile activity recognition systems. Among these challenges, detection accuracy and network lifetime are the most important. Although any errors in an activity recognition, such as fall detection, can cause delayed help and endanger the health of users. A short network lifetime, which for wearable device should be at least one week [1], can result in patient dissatisfaction and rejection of systems. Activity recognition systems can be implemented using ambient sensors or via wearable sensors. In the former case, human activity recognition is achieved either by information collected from a number of sensors attached to various objects (such as a bed, door, stove or video/audio equipment) or by processing video captures of the user's living space. In the latter case, activity recognition can be fulfilled by inferring information that is gathered from a set of wearable sensors that are attached to the human body. In this Letter we conduct a series of real life tests to observe the feasibility of using WBANs for monitoring various daily activity patterns based on 802.14.5 wireless standard.

The body of this Letter is organised as follows: Section 2 presents the related work and technologies for activity recognition systems, Section 3 discusses the methods that were used for the evaluation and Section 4 demonstrates our results and reports the trade-off for the accuracy and the energy consumption of three applied techniques. Finally, Section 5 concludes the Letter along with proposing a mechanism to improve the performance of the pattern recognition technique, which will be implemented and investigated in our future work.

2. Related work

Incorporating ambient sensors, specifically video cameras, for activity recognition has been extensively studied [2–5]. The downside of such techniques is that processing video data is very costly and an external infrastructure is required, for example, infrared cameras, which may be influenced, by environmental conditions such as background light or heat. Moreover, such techniques can give rise to privacy issues. The operation of these systems is limited to use where the system is deployed and cannot be used to monitor a user outside the operation site.

The aforementioned limitations motivate the use of wearable sensors that can be employed to provide inexpensive and mobile activity recognition systems. These systems can be trained to recognise only specific predefined activities or movements to overcome the privacy issue.

The wearable sensors can measure body movements using accelerometers or gyroscope, in addition to physiological signals such as heart rate and glucose concentration, or environment variables such as temperature, light and humidity.

Activity recognition systems cover a number of interesting application areas such as sport assistance [6], entertainment [7] and industrial processes [8]. A notable example in these areas, which is materialised in end-user products, is the Kinect game [9] console developed by Microsoft. It relies on the recognition of gestures or even full body movements to allow users to interact with the game. In [10–14], the authors present systems for fall detection as well as health threats by monitoring the physiological signals.

In the context of mobile activity recognition for healthcare applications, detection accuracy and wearable sensors' lifetime are two major and mutual goals. Extensive research has focused on these two criteria separately in the context of wireless sensor networks and wearable computing [15]. However, these two goals demand for sensory processing and transmission energy consumption optimisation techniques that generally result in detection accuracy reduction [16]. Among research that targets trade-off between detection accuracy and energy consumption, each study focused on only one particular challenge. Yan et al. [17] proposed an activity-based strategy for continuous activity recognition with focus on both the sampling frequency and the classification features that are adapted in real time. Their results shows that in an ideal condition activity-based strategy can achieve an energy savings of 50%.

Zappi et al. [18] focused on selecting the minimum set of sensors according to their contributions to classification accuracy. The proposed dynamic sensor selection method showed the trade-off between the classification accuracy and the battery lifetime by minimising the number of motes necessary to obtain a given classification accuracy for activity recognition. This method was tested by recognising manipulative activities of assembly-line workers in a car production environment.

French et al. [19] focused on the impact of sampling frequency. They evaluated different selective sampling strategies. The strategies they tested were a baseline uniform sampling strategy, one that samples over the distribution of duration times of activities, and one that samples based on the probability of a transition occurring.

In this work we aim to investigate the impact of various parameters in detection accuracy and wearable sensors lifetime. Parameters such as transmission rate, sampling frequency and nodal processing can impact significantly the overall system's detection accuracy and energy consumption in a selected wireless protocol.

Several wireless technologies are proposed for wearable sensor network. Among these technologies, most of the currently existing projects of mobile activity recognition systems employ IEEE 802.15.4 standard as the wireless communication technology. Hence we also employ this technology in our investigation and set this standard for our proposed solution which can be also applied to other standards. We summarise the specification of the wireless technologies applicable for wearable activity recognition systems, list and compare them in Table 1 with respect to the performance parameters.

Table 1.

Specifications of wireless technologies for wireless wearable sensors

Standard	BW	Power	Stack Size	Stronghold
Bluetooth	1 Mbps	40 mA TX, standby 0.2 mA	∼100 + KB	interoperability, cable replacement
Bluetooth 4.0 (LE)	∼300 kbps	10 mA TX, standby 0.024 mA	250 KB	low cost, ability to run for years on standard coin-cell batteries
802.15.4	250 Kbps	30 mA TX, standby 356 mA	34 KB/14 KB	long battery life, low cost
802.15.6	75.9 Kbps (narrow band) up to 15.6 Mbps ultra wide band	0.1–1 mA TX	N/A	low cost, high reliability, ultra-low power, short-range wireless communication in or around the human body
802.11ah	0.72–8.67 Mbps for 2 MHz channel	5–10 mW TX	N/A	optimised for extended communication range, power efficiency, scalable operation

3. Evaluation methods

We apply three methods for data transmission, namely, ‘stream-based’, ‘feature-based’ and ‘threshold-based mechanisms’ to study the accuracy against energy efficiency of transmission and processing power which affects the mote's battery lifetime. We also investigate the impact of variation of two parameters, that is, sampling frequency and data transmission rate on energy consumption of motes for each of the three discussed methods. For each experiment, a predetermined sequence of five postures is given to four case studies to follow for 25 min. We run the experiment for the three methods at four different sampling frequency rates of 1, 5, 10 and 16 Hz as our used hardware spec provide.

In ‘stream-based’ mechanism each free range mote senses accelerometer data and encapsulates the information in a packet with 48 bytes of payload size and sends it without any extra processing to the base station mote. In ‘feature-based’ mechanism to evaluate the impact of in-node processing on energy consumption, each free range mote extracts a set of features over a predefined time window and encapsulates them in a packet with 264 bytes of payload size and sends it to the base station mote. In ‘threshold-based’ mechanism, to evaluate the impact of communication on energy consumption, each free range mote sends the raw sensed data to the sink node if its difference with the last transmitted data is greater than a predefined threshold value. Similarly, each mote encapsulates raw sensed data in a packet with 48 bytes of payload size to send it to the base station mote.

In addition to the aforementioned data transmission methods, we also employ the method proposed by Ghasemzadeh et al. [15] to evaluate the impact of feature selection on detection accuracy and mote energy consumption. In this work the authors investigated the impact of eliminating irrelevant and redundant features on detection accuracy and energy consumption. In the first step the relevance of each feature and classification tasks, and also relevance between each pair of features, are computed based on a symmetric uncertainty parameter. Then, in the second step, features that have a value below an acceptable threshold value (λ_R) to the classification tasks, which is an application design parameter, are eliminated from the features set. Then, in the third step and by analysis of remaining features, among strongly correlated features which have a relevance value greater than a redundancy threshold value (λ_D), features that have higher computation energy are eliminated from the set features. Hence, with a larger value of λ_D, fewer features are considered as strongly correlated and as a result the number of optimal features will be increased.

In the following Section we will describe our experimental set-up and discuss the result of employing these methods of data transmission and feature selection.

4. Experimental analysis

In this Section, we present our testbed set-up and the goal of our experiments. The WBAN-based activity recognition monitoring scenarios are deployed in the form of a testbed implementation in our laboratory by wireless sensor motes. We conducted a series of tests to analyse the impact of a number of parameters on the accuracy of pattern recognition and evaluated the trade-off with the power consumption rate in wireless wearable sensor motes.

4.1. WBAN testbed – hardware and software set-up

The experiments in our WBAN laboratory were performed applying the testbed implementation using Sun SPOT motes [21] (see Fig. 1) manufactured by Oracle (formerly Sun Microsystems). We selected the number of motes and their placement based on the suggested configuration in [20] to investigate the impact of transmission rate, sampling frequency and nodal processing.

Figure 1.

Sun SPOT nodes' placement

The sun SPOT motes are categorised into two type of motes, namely free range motes and base station motes. Each free range mote contains a processor, radio, sensor board and a battery, and the base station motes contain the processor and radio only. Each mote uses a 32 bit ARM9 micro-processor running the Squawk VM and an IEEE 802.15.4 compliant radio. While there are four built-in sensing units available on free range motes, we only collected the data sensed by the accelerometers as relevant information for our experiment and switch off the other sensory motes to reduce the energy consumption in our experiments. The architecture of a mote that can play the roles of both free range and base station motes is presented in Fig. 2. Each free range mote can be embedded in the environment and perform sensing and communication tasks, and each base station mote can communicate with free range motes through wireless links. The base station mote can be connected to an external server like a PDA, smartphone or laptop through a USB interface for supporting the user interface. Hardware specifications of the motes are given in Table 2.

Figure 2.

Sun SPOT node architecture model

Table 2.

Specification of Sun SPOT mote

Processor	180 MHz, 32 bit, ARM920T
Power usage			deep sleep mode: 33 μA
shallow sleep: 24 ∼ 46 mA
active: 80 ∼ 104 mA
RF chip	TI CC2420
RF power	−25 ∼ 0 dBm
Power supply	3.7 V rechargeable 750 mAh lithium-ion battery
RF current draw			receive mode: 18.8 mA
transmit mode: 8.5 ∼ 17.4 mA
sleep mode: 1.0 μA
Sensors	accelerometer (MMA7455L chip), temperature, light, humidity

As can be inferred from this table, a full battery charge of the motes, in best cases, can operate continuously for about 9 h in active mode, about 31 h in shallow sleep mode and about two and a half years in deep sleep mode. Although using deep sleep mode is extremely desirable, switching from this mode to active mode takes about 3.55 s and this downtime can hinder the recognition of activity significantly, which cannot be acceptable for medical applications, hence we do not apply this mode in our experiments. Using a combination of active mode and shallow sleep mode, a full battery charge of the motes can last for about 7–14 h of experiments depending on the variable settings.

In our testbed, we placed two sensor motes on the body of the person under study; one mote on his chest and the other one on his right thigh, as illustrated in Fig 1. These motes communicate with a base station mote within the free range motes’ transmission range. This mimics a wearable sensor network. We selected this set-up based on the suggested configuration in [20] to maintain a certain level of accuracy. The wearable sensor network is used for collecting accelerometer data for out-of-body and offline post-processing. To have a minimum transmission power which is more suitable for intra-WBAN communication and also to have a signal strength greater than −80 dB over a distance of 5 m, our motes radio transmission power is set based on the result of experiments carried out in [22]. Hence, we set our motes radio transmit power level at level 7 where the transmit radio power is equal to −15 dBm and consumes 9.9 mA of the current.

In these experiments we attempt to identify five body postures, namely, STAND, SIT, LIE DOWN, WALK and RUN, which represent both intensive and non-intensive activity scenarios. We perform a set of controlled experiments in which a person under study is given pre-determined sequences of the five aforementioned postures to follow. Each of the five postures are detected using a set of features that are listed in Table 3.

Table 3.

Features extracted from each mote data

Feature	Description
Amp	amplitude of signal segment
Med	median of signal segment
Mean	mean value of signal segment
Max	maximum amplitude of signal segment
P2P	peak to peak amplitude
Var	variance of signal segment
Std	standard deviation
Cov	covariance of two signal
RMS	root mean square power
S2E	start to end value

To evaluate the detection accuracy, we temporally correlate postures with the sequence provided to the person under study and employ the J48 decision tree of Weka toolkit [23] as a classifier.

The free range application extends the Java ME MIDlet class and runs on top of a number of libraries. These libraries provide an environment where a free range application can access a variety of sensors and communication components. The host application is a regular Java SE program which uses various SPOT libraries that provide access to the SPOT device, basic I/O and the low-level MAC radio protocol such as USB connection to a PC or socket connections to other host applications.

4.2. Experimental analysis of accuracy against energy consumption

As discussed before, in our experiments we focus on two main challenges, that is, detection accuracy and power consumption. For wearable WSN applications, the accuracy of the state of the person under study is important. Furthermore, to keep and increase the level of accuracy, more frequent sensory information needs to be collected and processed which results in higher power consumption. This in turn results in the need to charge the batteries of the motes in a shorter period of time and causes inconveniency for the system user.

To evaluate the impact of sleep mode, we use shallow sleep mode in the implementation of all three mechanisms. We set motes to shallow sleep mode in between any two consecutive sampling times and after the required computations.

In the following Sections, first the impact of various parameters on energy consumption is analysed. This is followed by the discussion of the results of detection accuracy analysis for varying parameters. Finally, packet delivery ratio (PDR) for different packet sizes and sampling frequencies and its impact on detection accuracy are investigated.

4.2.1. Energy consumption

As the frequency of sampling increases the accuracy of the activity recognition will increase. However, the energy consumption tends to increase which will result in a shorter lifetime of the motes.

In the first step of energy consumption analysis we experienced the impact of sampling frequencies on the energy consumption for the three aforementioned mechanisms. We select the ‘stream-based’ mechanism to study this effect.

The energy consumption of ‘stream-based’ mechanism at different sampling frequency rates meets the expectation, as shown in Fig. 3. The difference between energy consumption at 16 and 1 Hz sampling frequency is significant by about 28% less on the energy consumption by reducing the sampling frequency rate in this mechanism. The amount of energy saving is about 31 and 32% for ‘feature-based’ and ‘threshold-based’ mechanisms, respectively.

Figure 3.

Energy consumption of stream-based mechanism at different sampling frequencies

Although our observations show the impact of the sampling frequency on the energy saving in the range of 30%, we further analyse the discussed methods to improve the energy efficiency by looking at the impact of in-node processing against the transmission rate.

We select a sampling rate of 16 Hz which yields the maximum power consumption in Fig. 3 for our further evaluations. Fig. 4 shows the comparison of average energy consumption of different mechanisms with a sampling frequency rate of 16 Hz. In this sampling frequency we observe that the number of transmitted packets in the ‘threshold-based’ mechanism becomes about one-fourth of this number in the ‘stream-based’ mechanism. However, to the contrary of our expectations despite this reduction in data transmission, the overall energy consumption of ‘threshold-based’ mechanism becomes 15% greater than ‘stream-based’ mechanism. We can justify this to be because of the processing energy consumption for comparison operations. Since in our implementation the mote's radio unit is always on, we cannot benefit from a lesser number of transmissions and the impact of the processing energy is no longer negligible in this case.

Figure 4.

Energy consumption of experiment of the three mechanisms at 16 Hz sampling frequency

The results also yield an energy consumption increase of about 15% in ‘feature-based’ mechanism in comparison with ‘stream-based’ mechanism because of the energy consumption for the feature set extraction. This result indicates that the nodal processing has a greater impact on energy consumption in comparison with transmission rate when motes are always on.

To look closer at the impact of nodal processing on energy consumption, the per bit energy consumption of different mechanisms is depicted in Fig. 5. As illustrated in this figure, operations of ‘threshold-based’ mechanism and feature set extraction of ‘feature-based’ mechanism can result in an increase of about 4.5 and 3.5 times higher per bit energy consumption, respectively, at 16 Hz sampling frequency rate in comparison with ‘stream-based’ mechanism. This also confirms our justification on the impact of the nodal processing in comparison with the transmission rate.

Figure 5.

Comparison of per bit energy consumption for different mechanisms

Moreover, increasing the sampling frequency rate of motes will result in a reduced energy consumption required for computation and transmission of each bit of sensed data. This is because of the fact that in our implementation radio interface is always on and there is a slight difference between energy consumption in active and shallow sleep modes of motes as shown in Table 1. Besides, as shown in this figure, at best the energy consumption is about 2000 times more than the Zigbee standard (119 nJ/bit); hence this standard is not a suitable technology for activity recognition where the required accuracy does not permit the nodes to go into deep sleep mode.

4.2.2. Detection accuracy

To evaluate the detection accuracy against varying parameters, such as sampling frequencies and number of features, and its relation with energy consumption in this step, the collected data from each mote are post-processed in offline mode and a dataset of 30 features (10 features for accelerometer data in each direction) is constructed. Fig. 6 illustrates the detection accuracy using information from a single mote or with combined information from both motes for ‘stream-based’ mechanism.

Figure 6.

Comparison of detection accuracy for features extracted from information of a single mote and combined information of both motes

As can be seen from Fig. 6 the mote which is placed on the right thigh, because of its more appropriate location, gives a better accuracy than the mote which is placed on the chest. This mote also gives a better accuracy than the combined features from both motes for 5 and 16 Hz sampling frequency rates. The results also show a trend to better detection accuracy with increasing sampling frequency except for 16 Hz sampling frequency rate. This is because of the high data loss as a result of a larger packet size. While the variation of the detection accuracy is less than 1% for different sampling frequency rates, the impact is important for healthcare applications.

4.2.3. Packet delivery ratio (PDR)

As shown in Fig. 7, in ‘feature-based’ mechanism because of a larger packet size and limited set number of retransmissions, the PDR decreases significantly in comparison with ‘stream-based’ mechanism. Moreover with increasing sampling frequency and as a result of increasing data transmission rate, the PDR decreases significantly so that, in the ‘stream-based’ mechanism with 16 Hz sampling frequency and with data transmission rate equal to 6 Kbit/s, the PDR becomes lower than 80% which is inappropriate for healthcare applications.

Figure 7.

Comparison of PDR between stream-based and feature-based mechanisms

The high packet loss especially for the chest mote which is about 24% at 16 Hz causes decreased detection accuracy.

4.2.4. Feature selection evaluation

To evaluate the impact of this mechanism in our experiments, we use a dataset of combined features from both motes in ‘stream-based’ mechanism at 16 Hz sampling frequency as the initial dataset. Fig. 8 shows the number of relevant features and detection accuracy as a function of the relevance threshold (λ_R) for our experiment. Clearly, the number of relevant features decreases as the λ_R increases. As illustrated in this Figure, selecting a large number of features does not have an effective relevance with classification tasks. Hence their elimination from classification features does not have a significant impact on detection accuracy. For instance, by changing the λ_R from 0.1 to 0.2, the number of relevant features decreases from 50 to 25, while the detection accuracy only varies slightly (about ± 0.06%).

Figure 8.

Number of relevant features and accuracy against λ_R

To evaluate the impact of eliminating redundant features to detection accuracy and energy consumption we choose 30 relevant features to make a dataset for our study. The number of optimal features and related classification accuracy against λ_D for this dataset are shown in Fig. 9.

Figure 9.

Number of optimal features and accuracy against λ_D

As stated earlier, by increasing the λ_D, the number of optimal features increases. For example, increasing the λ_D value from 0.15 to 0.3 leads to an increase in the number of optimal features from about 3 to 8. An interesting observation in this Figure is that with only five features we can have an accuracy of about 99.16%, which differs slightly from the best possible detection accuracy for a complete feature set (i.e. 99.18%).

Fig. 10 shows the energy that is needed for feature extraction over the duration of the experiment (i.e. 25 min) at different sampling frequency rates and over a time window that contains five samples.

Figure 10.

Energy consumption for feature extraction for different sampling frequency against λ_D

5. Conclusions and future works

In this Letter, we carried out a series of experiments to study the impact of various parameters on accuracy against energy efficiency of a wearable activity recognition system. Results of these experiments indicate greater impact for sampling frequency, nodal processing transmission rate, in sequence.‏

Different applications of activity recognition systems may have different requirements in terms of accuracy levels and tolerable delay. Sampling frequency, transmission rate and the method of nodal processing, can each significantly impact in satisfying these requirements. Furthermore, for a given application accuracy level and tolerable delay, detection of different activities can also be done with different combinations of these parameters. For instance, for a given accuracy level, detection of a running state may require a higher sampling frequency and transmission rate and more classification features in comparison with detecting a laying down state.

A solution for satisfying these application requirements in an energy-efficient manner can be a cross-layer design, in which the related parameters are being adaptively configured to meet the application requirements. In future work, we plan to develop a cross-layer-based activity recognition system and investigate the performance of this solution to make a trade-off between application requirements and energy efficiency.

7. Funding and declaration of interests

This research was in part supported by a grant from IPM (grant no. CS1393-4-48). Conflict of interest: none declared.