Computational Methods for Estimating Energy Expenditure in Human Physical Activities

Abstract

Accurate and reliable methods for assessing human physical activity energy expenditure (PAEE) are informative and essential for understanding individual behaviors and quantifying the impact of physical activity (PA) on disease, PA surveillance and for examining determinants of PA in different populations. This paper reviews recent advances in the estimation of PAEE, in three interrelated areas: 1) types of sensors worn by human subjects, 2) features extracted from the measured sensor signals, and 3) modeling techniques to estimate the PAEE using these features. The review illustrates three directions in the PAEE studies, and provides recommendations for future research, with the aim to produce valid, reliable, and accurate assessment of PAEE from wearable sensors.

1. Introduction

Physical activity (PA) is defined as body movement generated by skeletal muscle action that increases energy expenditure above resting levels (46). Physical activity dose is a function of activity type, intensity and duration which determines the actual energy expenditure (PAEE) of the activity being performed. Engaging in physical activities on a regular basis by means of walking, jogging, sports or muscle-strengthening activities, is effective for improving health, fitness and quality of life, preventing and lowering the risk of cardiovascular disease, weight control, diabetes, obesity, early death, certain cancers, and stroke (64). PA observational studies that provide evidence of the dose-response between PA and health, improve our understanding of the link between PA and health. Healthy People 2020 has listed physical activity as one of the targeted behaviors to improve health among youth and adults (65). Thus, accurate monitoring and assessment of PA is critical and of significant interest to the research community.

Physical activity energy expenditure (PAEE) has been assessed by different lab-based methods including direct (66) or indirect (19) calorimeters to quantify the intensity of activities. Such methods are accurate in assessing the PAEE, but are generally costly and performed in lab settings where sophisticated lab equipment is required. In lab studies using indirect calorimetry as the criterion measure, steady-state values are used to develop prediction models from wearable sensors. The criterion energy expenditure for a given activity is usually expressed as the metabolic equivalent (METs) using either measured RMR or 3.5 ml.kg/min as the denominator of the MET calculation. It remains unclear which is the best method for expression of METs for purposes of activity monitor calibration (38). In free living studies where activities are not fixed and can vary from moment to moment, steady-state is usually not attained and the use of indirect calorimetry is not an appropriate criterion measure. In this case, direct observation may be preferred where energy expenditure of these more random types of activity is estimated from the Compendium of Physical Activities (1).

Activity monitors are now capable of storing a large amount of raw acceleration data, allowing examination of detailed features from the monitor sensor output. Using machine learning techniques to process these data can detect activity types and patterns of physical activity as well as intensity and energy expenditure. Combining physiological measures (e.g., respiration, skin temperature and heart rate) with the motion output from accelerometer sensors may further enhance the PA assessment (35, 43). Recently, such a multi-sensing approach has emerged as a technique for accurate and reliable PA assessment under free-living environments (25, 31, 32, 35). This paper reviews the various types of body-worn sensors and different computational techniques that have been applied to the PA assessment based on recent publications. Specifically, this review focuses on the estimation of PAEE. Methods for recognizing PA patterns have been reviewed elsewhere (51).

Estimating PAEE typically involves three steps, as shown in Figure 1. Human body motion and physiological variables (other than demographic or anthropometric variables) are first captured by the sensor system, which consists of either single or multiple body-worn sensors. Different features in the time, frequency, or other domains are extracted from the raw sensor measurements characterizing the original signals for reducing the computational complexity and redundancy. It is common practice that signal output by sensors are first pre-processed (e.g., filtering, artifact removal), before relevant “features” are extracted. Discussion on signal pre-processing is outside the scope of this review, and can be referred to in Preece et al. (51). These extracted features are then used as inputs to data fusion algorithms for estimating/modeling the PAEE. The modeling methods reported in past studies are of varying complexity, ranging from simple linear regressions to advanced machine learning techniques (6), which extract (often hidden) relationships and correlations between physical activity data and the PAEE. Examples of such machine learning methods include hidden Markov models, quadratic discriminant analysis, artificial neural network and support vector machines. These methods are discussed in detail later in the paper. Regardless of the specific techniques used, PAEE estimation has generally been performed in a supervised manner, for which an estimation model is first developed (or trained), based on features extracted from the sensor signals. The criterion energy expenditure is measured by respiratory gas exchange device when subjects are performing physical activities. The trained model is then used to estimate PAEE by using the data that were not included initially in the training set.

Figure 1.

Typical flow chart for PAEE estimation.

The rest of this review paper is structured as follows. In section 2, body-worn sensors used for estimating PAEE are first introduced; in section 3, feature generation, including feature extraction and selection, are presented; computational techniques are described in section 4; and conclusions and future directions are described in section 5.

2. Body-Worn Sensors

2.1 Accelerometers and Foot Pressure Sensors

Table 1 shows a list of the body-worn sensors that have been used to estimate of PAEE. Among them, the accelerometer is one of the most commonly used wearable sensors for estimating the PAEE, due to its capability of detecting the quantity and intensity of human body movements. While simple in structure, the accelerometer can effectively detect both static and dynamic accelerations caused by changes in posture, body motion, or transition in motion patterns. Specifically, static acceleration responds to gravity, and has been used to measure tilt angles or posture changes (23, 67). Dynamic acceleration captures the intensity of body movements, and thus has been extensively applied to estimate PAEE. A majority of the studies have been based on single uniaxial accelerometer that is usually placed on the hip (6, 8, 15, 17, 22, 26, 27, 28, 34, 36, 37, 40, 47, 50, 52, 56, 57), and details using such single accelerometer-based activity monitors for estimating PAEE are presented in previous reviews (14, 24, 45, 32). It is noted that activities involving mostly upper-body motion would result in inaccurate estimation of the PAEE when only using a hip accelerometer. Placing additional accelerometers at various locations on the body improves the estimation of PAEE (3, 43, 52, 69, 70). A five-accelerometer device (IDEEA) with accelerometers located on the chest, frontal area of each thigh, and bottom of each foot achieved a 56% higher prediction accuracy for estimating energy expenditure than a single hip-mounted accelerometer device (ActiGraph) (52). Favorable results for estimating energy expenditure were also reported for the IDEEA in comparison to other multi-accelerometer activity monitors, e.g., the IDEEA improved the accuracy of estimating energy cost by 26.4% compared to the ActiReg, which consists of two accelerometers (3). Noteworthy are the PAEE estimation results reported recently by combining single or dual accelerometers with advanced machine learning techniques, which have achieved comparable or even more favorable estimation accuracy compared to the IDEEA. For example, Staudenmayer et al. (57) demonstrated a comparable estimation performance with a mean square error (MSE) of 0.56 METs compared to the MSE of 0.45 METs by the IDEEA (52) by processing a single-uniaxial-accelerometer signal with an artificial neural network (ANN), and Rothney et al. (52) achieved an MSE of 0.25 METs by applying ANN to a biaxial accelerometer. More details on machine learning techniques for PAEE estimation will be discussed in later sections.

Table 1.

Sensors used for estimating PAEE.

Types	Locations	References
Accelerometer(s) alone
Single	Hip	(6, 8, 15, 17, 22, 26, 27, 28, 34, 36, 37, 40, 47, 50, 52, 56, 57)
Multiple	Hip, chest, thigh (left and right), bottom of foot (left and right), wrist	(3, 43, 52, 69, 70)
Combination of pressure sensors and accelerometer(s)	Bottom of each foot: heel, metatarsal bones and great toe	(53, 60)
Combinations of physiological sensors and accelerometer(s)
Heart rate (ECG)	Chest,	(10, 15, 16, 18, 21, 38, 41, 58, 59, 68)
Respiration	Rib cage, abdomen	(32, 43)
Heat flux	Upper arm	(2, 4, 12, 25, 31, 35)
Galvanic skin response	Upper arm
Skin temperature	Upper arm

The performance of PAEE estimation mainly depends on two independent factors: the number/type of sensors and the choice of the analytic technique used to estimate PAEE. When combining advanced machine learning techniques with fewer sensors, e.g., applying ANN to a single accelerometer (52, 57), comparable estimation accuracy of PAEE can be achieved compared to devices employing more accelerometers that used simpler and less powerful analytic techniques. On the other hand, if the estimation technique remains the same, devices with more sensors generally demonstrate a better estimation performance than those with fewer sensors (43). Despite the estimation accuracy by using multiple sensors, the disadvantage of a multi-sensor system, is that it increases subject burden, data processing and product cost. These factors limit the utility of multi-sensor devices in free-living conditions.

Another type of sensor used for estimating PAEE is a pressure device, which is typically placed on the bottom of the feet to detect the contact pressure during human movement (54, 61). For example, Sazonova et al. (54) developed a footwear-based device integrating one accelerometer and five force-sensitive resistors positioned at the heel, metatarsal bones, and the great toe. By using both the accelerometer and pressure sensor signals in the model, better PAEE prediction with a RMSE of 0.69 METs was achieved than that using only the accelerometer signal (RMSE of 0.77 METs).

2.2 Physiological Sensors

For different types of activities that produce similar acceleration profiles but have different energy expenditure, accelerometry-based activity monitors cannot provide accurate estimates of PAEE (43). For example, walking at a certain speed may result in acceleration outputs similar to that of walking at the same speed while carrying a load or going uphill, although the energy expenditure is different. Under such circumstances, sensors that measure human physiological responses, such as heart-rate (10, 15, 16, 18, 21, 38, 41, 58, 59, 68), respiration (32, 43), or an armband (2, 4, 12, 25, 31, 35) consisting of heat flux, galvanic skin response, and skin temperature sensors, have been studied to determine if these additional sensors improve the estimation of the energy expenditure. For example, combining the measurement of heart-rate with acceleration estimates the PAEE more precisely (R² = 0.81, RMSE = 0.64 METs) than the PAEE estimate using a single hip accelerometer (R² = 0.41, RMSE =1.22 METs) (59). Similarly, by combining one piezoelectric respiratory sensor, which secured to the abdomen for detecting human breathing frequency and minute ventilation during activities, with two accelerometers at the hip and wrist to estimate the corresponding PAEE for 50 subjects, we reported an RMSE as low as 0.42 METs, which is 22% less than using the hip accelerometer alone (32, 43). These results have shown that the combination of physiological sensors and accelerometer improves the accuracy of PAEE estimation. However, using only physiological sensors are not recommended, as a past study (43) has shown a less accurate estimation of the PAEE by using the physiological sensors alone (RMSE = 0.72 METs) than combining them with accelerometers (RMSE = 0.42 METs). Thus, compared to the single-accelerometer approach, using multiple sensors, either multiple accelerometers or integrating physiological or pressure sensors, may improve the PAEE estimation.

3. Feature Generation

The raw signals from body-worn sensors during physical activities are often sampled over periods of time and contain a considerable number of data points. Although a study (9) used raw data points directly for estimating the PAEE, features are usually extracted from the original sensor signals and used as inputs to more advanced post-processing computational algorithms for PAEE estimation. Figure 2 shows an example of a hip accelerometer signal and the extracted features during treadmill activities at various speeds (e.g. 1.1 km/h, 3.1 km/h, 5.0 km/h, 7.1 km/h, 8.0 km/h, 10.0 km/h, 12.1 km/h, and 14.0 km/h). The features are the inter-quartile range (IQR) values extracted from the original accelerometer signals for each activity (42). Figure 2 illustrates that the IQR values increase linearly with the speed of locomotion, and despite the individual variability, the IQR values may be used to represent the original signal as inputs to the PAEE estimation algorithm. Using the features as inputs enables substantial data reduction as compared to using the original raw signals. For example, for signals sampled at 30 Hz, data volume will be reduced by 300 times if features are extracted every 10 seconds. As a result, the computational load of the algorithm is significantly reduced.

Figure 2.

An example of features extracted from a hip accelerometer signal measured during treadmill activities at various speeds.

In the following section, commonly used features for estimating the PAEE are summarized. These include both time-domain and frequency-domain features, as well as other parameters (e.g., height, weight, BMI, body composition) of the person wearing the sensors. A list of the features that have been studied in past studies is shown in Table 2, together with their respective mathematical formulae.

Table 2.

Features generated for PAEE estimation.

Features		References
Time domain
1	Data points of raw sensor signals: {si, i = 1,…,N} s: raw sensor signal, N: number of the sensor data points within a certain period of time.	(9)
2	Accelerometer counts (integral/sum of the signals over a period of time): ${\sum }_{i=1}^N{s}_i$.	(13, 17, 44, 50, 56)
3	Mean: $\overline{s}=\frac{1}{N}{\sum }_{i=1}^N{s}_i$	(6, 8, 32, 42, 43)
4	Standard deviation: ${\sigma }_s=\sqrt{\frac{1}{N}{\sum }_{i=1}^N\left(s_i-\frac{1}{N}{\sum }_{i=1}^N{s}_i\right)}$	(6, 8, 32, 43, 54)
5	Coefficients of variation (CV): σs/|s̄|, σs: standard deviation, s̄: mean.	(17, 52, 54)
6	Peak-to-peak amplitude	(6, 8)
7	Percentiles (10th, 25th, 50th, 75th, 90th): $s_j^{\prime }+N\left(p-p_j\right)\left(s_{j+1}^{\prime }-s_j^{\prime }\right)/100$ $\left\{s_i^{\prime },i=1,\dots ,N\right\}$ : sorted sensor data in increasing order, pj = 100(j−0.5)/ N : percent rank of jth data point, $s_j^{\prime }$ : jth sorted data value, p: pth percentile, find j such that pj+1 ≤ p ≤ pj.	(33, 43, 57)
8	Interquartile range: difference between the 75th and 25th percentiles.	(42, 52)
9	Correlation between accelerometer axes: $\frac{{\sum }_{i=1}^N\left(s_i-\overline{s}\right)\left(v_i-\overline{v}\right)}{\sqrt{{\sum }_{i=1}^N{\left(s_i-\overline{s}\right)}^2{\sum }_{i=1}^N{\left(v_i-\overline{v}\right)}^2}}\overline{s},\overline{v}$ : mean of signal s and v.	(6, 43)
10	Autocorrelation: $\frac{{\sum }_{i=1}^{N-k}\left(s_i-\overline{s}\right)\left(s_{i+k}-\overline{s}\right)}{{\sum }_{i=1}^N{\left(s_i-\overline{s}\right)}^2}$, s̄ : mean, k: lag k autocorrelation	(57)
11	Skewness: $\frac{\frac{1}{N}{\sum }_{i=1}^N{\left(s_i-\overline{s}\right)}^3}{{\left(\frac{1}{N}{\sum }_{i=1}^N{\left(s_i-\overline{s}\right)}^2\right)}^{3/2}}$, measure of asymmetry of the signal probability distribution.	(52)
12	Kurtosis: $\frac{\frac{1}{N}{\sum }_{i=1}^N{\left(s_i-\overline{s}\right)}^4}{{\left(\frac{1}{N}{\sum }_{i=1}^N{\left(s_i-\overline{s}\right)}^2\right)}^3}-3$, degree of the peakedness of the signal probability distribution.	(52)
13	Signal power: $\left\{s_i^2,i=1,\dots ,N\right\}$ or sum of the signal power ${\sum }_{i=1}^N{s}_i^2$.	(52)
14	Log-energy: ${\sum }_{i=1}^N\log \left(s_i^2\right)$	(41)
15	Peak intensity: number of the signal peak appearances within a certain period of time.	(52)
16	Zero crossings: number of times the signal crosses its median.	(54)
Frequency domain
17	Dominant frequency	(6, 8, 43)
18	Amplitude of dominant frequency	(6)
19	Entropy: − Σk p (fk) log p(fk) fk: kth frequency component of the signal, p(fk): probability mass function of the fk.	(6, 43)
Demographic and anthropometric features
20	Age, height, weight, BMI, gender, genetics, occupation, education	(13, 20, 28, 52, 53, 54, 68)
21	Body composition: fat mass, fat-free mass	(13)

3.1 Time-domain Features

Time-domain features are directly extracted from the time series of the original, raw sensor signals. A typical procedure for extracting time-domain features is to first divide a sensor signal into consecutive small windows. Then, time-domain features, as listed in Table 2, are computed over each window. The size of the signal window ranges from 0.25 to 60 seconds (31, 68), but the determination of the window size usually depends on the specific application and the impact to the estimation accuracy. For example, the window size can be fixed using a sliding window, or varied using an event- or activity-defined window. Compared to the event- or activity-defined windowing technique which requires a pre-processing step to recognize the specific event or activity and increases the computational complexity, the sliding window is more suited to real-time applications due to its simpler implementation (51). A more detailed review of the windowing techniques is discussed in Preece et al. (51).

As statistical measures, time-domain features provide insight into the following aspects of the measured signal:

1. general description of the raw signal, such as the mean (6, 8, 32, 42, 43) and standard deviation (6, 8, 32, 43, 54);

2. representation of the signal strength, such as accelerometer counts (13, 17, 44, 50, 56), signal power (52), log energy (41) and peak-to-peak amplitude (6, 8);

3. distributions of the signal, such as 10th, 25th, 50th (median), 75th, 90th percentiles (32, 43, 57) and interquartile range (42, 52);

4. measures of the signal probability distribution, such as kurtosis (52), skewness (52) and coefficient of variation (CV) (17, 52, 54);

5. and others, such as peak intensity (52), zero crossings (54), autocorrelation (57), and cross-correlation (6, 43).

Specifically, peak intensity is the number of the signal peak appearances within a signal window, and zero crossings is the number of the times that the signal crosses its median. Staudenmayer et al. used lag one autocorrelation of the accelerometer counts to measure the signal temporal dynamics (57), and others have used cross-correlation to quantify the similarity between signals from different axes of one accelerometer (6) or from accelerometers placed at different body locations (43) to measure the coordination or variation between different body segments during activities.

3.2 Frequency-domain Features

Frequency-domain features are extracted from the coefficients obtained by performing spectral analysis, usually Fast Fourier Transform (FFT), on the original sensor signals. The values of the coefficients represent the amplitudes of the corresponding frequency components. In particular, the coefficient with the largest amplitude corresponds to the dominant frequency of the signal, which is an indicator of the activity intensity level. Both the dominant frequency (7, 8, 43) and its amplitude (7) have been commonly chosen as the frequency-domain features for PAEE estimation. Similar to time-domain features, measures of the probability distribution of the FFT coefficients have also been used for estimating the PAEE. Bonomi et al. used spectral entropy from the accelerometer signals as one of the features for the estimation of activity intensity (7). The spectral entropy measures the frequency concentration, and can be used as an indicator of the activity patterns (43). For example, low spectral entropy corresponds to high frequency concentration which usually represents simple or uniform activity patterns such as treadmill locomotion. In contrast, high spectral entropy corresponds to low frequency concentration, which indicates complex or non-uniform activity such as team sports, tennis, or self-paced walking.

3.3 Demographic and Anthropometric Features

In addition to the features computed directly from the sensor signals, subject demographic and anthropometric characteristics (13, 20, 28, 52, 53, 54, 68), such as age, height, weight, body-mass index (BMI), body composition (e.g., fat mass, fat-free mass), gender, have also been considered for PAEE estimation. As an example, Heil combined age, gender, height and weight, with accelerometer counts to estimate the activity energy expenditure (28). Carter et al. integrated age, gender, height, body mass, fat-free mass and fat mass with accelerometer data into the model to predict free-living energy expenditure (13). However, the impact of these anthropometric and demographic variables on PAEE estimation varies and in different studies. For example, Carter et al. (13) found that the body mass, age and gender were not significant predictors for the estimation of total energy expenditure (TEE) in young adults, and explained 0% of the estimation variance. Height, on the other hand, was found to be an important predictor of TEE, and explained 46% of the estimation variance. Plasqui et al. (48) also found that height is important for estimating TEE in adults, and combining age and body mass together with height explained 64% of the variation in TEE estimation. Similar to (13), they also found that gender is not significant for TEE estimation. Discrepancy also exists, as Plasqui et al. (48) found age and body mass contributes the estimation of TEE in adults and Carter et al. (13) concluded the opposite. Noteworthy is that Carter et al. (13) found that the fat-free mass instead of the body mass is significant and explained 48% of the variance of TEE estimation. Therefore, the effect of demographic and anthropometric features requires further investigation and data validation to improve our understanding of whether and how these parameters may affect the PAEE estimation accuracy. Moreover, measurement errors related to features such as percent fat or the race and gender differences in percent fat, although not discussed (13), may lead to inaccurate estimation of TEE, thus deserve further research.

4. Modeling of PA Energy Expenditure

Despite the different features discussed in previous section, it is noted (43) that the presence of irrelevant or redundant features may worsen the estimation performance. Often an additional step of removing irrelevant or redundant features and selecting “good” features from the overall feature set is integrated into the estimation process and is important for establishing an accurate model for PAEE estimation. Such a step is, however, not in the scope of this review. With the features extracted from the sensor signals, modeling methods are needed to use these inputs to estimate the PAEE associated with the activities. A list of recent modeling methods for PAEE estimation is shown in Table 3.

Table 3.

Modeling methods for estimating PAEE.

Methods	Theory	References
Standard regression	$y={\beta }_0+{\sum }_{i=1}^p{\beta }_i{x}_i+{\sum }_{i=1}^p{\sum }_{j=1}^p{\beta }_{\mathit{ij}}{x}_i{x}_j+\epsilon$ y: estimated PAEE (MET, kcal, etc.), x1, …, xp: features, xixj’s: polynomial representations (interactions) between features, β’s: regression coefficients.	(5, 18, 52, 53, 57, 68)
Log-linear regression	$\log \left(y\right)=k·\log \left(\mathrm{mass}\right)+{\beta }_0+{\sum }_{i=1}^p{\beta }_i{x}_i+\epsilon$ y: estimated PAEE (MET, kcal, etc.), x1, …, xp: features, mass: body mass, k and β’s: regression coefficients.	(13)
Hidden Markov model	Activity classified as a = arg max (Pr(x ∣ Aj)) Aj: jth activity, j=1,…n, Pr(x∣Aj): probability of the features x that come from activity Aj.	(48, 50)
Quadratic discriminant analysis	Probability of the signals that come from activity a, a ∈ {1, 2,…,n}, a total of n activities, $\mathrm{Pr}\left(A=a\mid X=x_a\right)=\frac{{\left|{\sum }_a\right|}^{-1/2}{e}^{-\frac{1}{2}{\left(x_a-{\mu }_a\right)}^T{\left|{\sum }_a\right|}^{-1}\left(x_a-{\mu }_a\right)}}{{\sum }_{j=1}^n{\left|{\sum }_j\right|}^{-1/2}{e}^{-\frac{1}{2}{\left(x_j-{\mu }_j\right)}^T{\left|{\sum }_j\right|}^{-1}\left(x_j-{\mu }_j\right)}}$ Σa : covariance matrix for activity a, x: features extracted from sensor signals, μ: mean of the features.	(50)
Artificial neural network	$y={\beta }_0+{\sum }_{h=1}^H\left[{\beta }_{1,h}\varphi \left({\beta }_{2,h}+{\sum }_{i=1}^p{\beta }_{k+2,h}{x}_i\right)\right]+\epsilon$ y: estimated PAEE (MET, kcal, etc.), x1, …, xp: features extracted from sensor signals, β’s: coefficients to be determined during training, H: the size of the hidden layers, ϕ(z) = 1/(1 + e−z) : activation function.	(52, 57)
Support vector machines	$y=w^T\varphi \left(x\right)+b={\sum }_{i=1}^N{\beta }_{i}K\left(x_i,x\right)+b$ y: estimated PAEE (MET, kcal, etc.), w: weighing factors, φ: transformation function, x: features extracted from sensor signals, βi: coefficients to be determined during training, K(xi, x) = ϕ(xi)T ϕ(x) : kernel function.	(32, 43, 53)
Hybrid approach/piecewise regression	$y=\{\begin{array}{ll}{f}_1\left(x_1,x_2,\dots ,x_p\right)& \mathrm{if}\mathit{Criterion}\mathit{1}\cdots \\ f_k\left(x_1,x_2,\dots ,x_p\right)& \mathrm{if}\mathit{Criterion\; k}\\ f_{k+1}\left(x_1,x_2,\dots ,x_p\right)& \mathrm{otherwise}\end{array}$ y: estimated PAEE (MET, kcal, etc.), f1, …, fp: regression models, x1, …, xp: features.	(11, 17, 21, 43, 53, 54)

4.1 Linear Regressions

Regression analysis is a commonly used technique for quantifying the relationship between one or more independent variables with the dependent variable. Models based on linear regression or multiple linear regression have been extensively used for the estimation of total energy expenditure (TEE), due to advantages such as simplicity, low computational demand, and early adoption by the research community. Traditionally, TEE has been modeled using linear regression with the accelerometer counts extracted from a body-worn, single-axis accelerometer. Despite the reasonably good correlation between the counts and the TEE as illustrated in past studies (24, 45), the estimation accuracy of accelerometer count-based linear regression has been shown to contain systematic errors and varies with the type of activities. For example, it has been shown that, with a linear regression model, the TEE of walking and light activities could be overestimated, while the TEE of moderate-intensity lifestyle activities may be underestimated (5). Such a limitation is presumably due to the insufficient information provided by the counts, which were calculated from the single-axis accelerometer and the simplicity of the linear model that considers the accelerometer counts as the sole variable. Efforts have been made for improving the estimation accuracy by increasing the model complexity, by either extracting more features such as percentiles, kurtosis, skewness from the single-axis accelerometer signal (52, 57), or including physiological (18) or anthropometric features (53, 54) such as height, weight, age. In some studies, polynomial representations or pair-wise interactions between the features as variables (68) in the model have also been used. In addition, variants of standard linear regression models have been proposed for estimating PAEE. For example, Carter et al. (13) considered the TEE to be proportional to body size as an allometric model (55), and therefore a log-linear model was used in their study and the logarithm of TEE was linearly related with the accelerometer counts, the percent fat and the logarithm of the total body mass.

4.2 Nonlinear Regressions

Despite the advantages and reasonable results of using linear approaches for PAEE estimation, questions have been raised as to whether the linear regression is an appropriate approach to model PAEE with body-worn sensor signals (14, 24). Studies (17, 62) have indicated nonlinear relationship between the PAEE and corresponding signals for certain types of activities. For example, Crouter et al. (17) has shown a non-linear, exponential and polynomial (third order) relationship, for two different groups of activities, walking/running and lifestyle/leisure, respectively. Treuth et al. (62) also presented a third-order polynomial regression model for estimating the PAEE. In addition to the “nonlinear” regression approaches discussed above, a few other methods that have recently gained attention for PAEE estimation and are reviewed below.

The Hidden Markov Model (HMM) and Quadratic Discriminant Analysis (QDA) are two commonly used pattern recognition techniques. Although the outcome variable from these computational models is activity type, PAEE can be inferred by combining the techniques with existing the Compendium of Physical Activities PAEE estimates (1). Specifically, the HMM is a probabilistic pattern recognition technique that models the transitions between activities as a discrete stochastic process. The activities, represented as states during such a process, are hidden or unknown, but can be determined by the observable parameters, which are the features extracted from the body-worn sensor signals. Specifically, a separate HMM model can be trained for each activity type (50). Observations/features can then be tested by each HMM model, and the specific activity type of the tested observations or features are identified from the HMM model that yields the largest probability. In comparison, Quadratic Discriminant Analysis (QDA) is a simple machine learning technique that establishes a quadratic surface, which is a pre-defined model that separates activities based on the training feature set. Similar to HMM, QDA classifies the activity type of the observation or feature set according to its probability estimated by the pre-trained model. The specific activity type is determined by the largest probability (50). Both HMM and QDA have been used for estimating the PAEE through a two-stage process. Activity types were first recognized by either HMM or QDA, once trained from a sample dataset (49, 50). The PAEE was then estimated as the corresponding MET level based on the Compendium of Physical Activities (1). For example, Pober et al. (50) employed such a two-stage approach by using HMM to identify the activity type of each observation, and then estimated the PAEE from the Compendium of Physical Activities. Such an approach did not provide the actual energy expenditure values of each activity, but rather an estimate of the time the subjects spent at each PAEE level was used. As a result, the PAEE estimation accuracy by such an approach would rely on and be limited to the values listed in the Compendium of Physical Activities.

Artificial neural network (ANN) and support vector machine (SVM) are two machine learning and nonparametric regression techniques that estimate the PAEE. Different from HMM and QDA, which operate in the original feature/input space, ANN and SVM first transform the original features into a different and usually higher feature space/domain, where the relationship between the features and PAEE is modeled. Such a model built in a higher feature space is less computationally demanding for determining activity types for each new data point added than the model built using the original features, which is often a complex, nonlinear, and implicit function. Specifically, ANN transforms the original features into one or more hidden layers or artificial neurons using activation functions such as the “softmax” function. The SVM converts the original features which usually are in a lower dimensional space to a higher dimensional space by using kernel functions such as a Gaussian radial basis function. Various studies have successfully applied ANN (52, 57) and SVM (33, 43, 53) for estimating PAEE, and promising results have been reported. For example, by applying an ANN model to signals from a biaxial accelerometer worn at hip, Rothney et al. (52) predicted METs of daily activities including deskwork, self-paced walking, jogging, biking, etc., with a mean squared error of 0.25 (equivalent to RMSE of 0.50 METs). Similar results have been reported in other studies, (e.g., in [57]), where the ANN was applied to signals measured by a uniaxial accelerometer in 48 subjects for estimating PAEE (RMSE = 0.75 METs). A PAEE estimation model was developed based on the SVM to achieve an estimation of PAEE with the RMSE of 0.42 METs (43). These advanced machine learning methods are superior compared to other regression methods. For example, Staudenmayer et al. (57) reported a 60% reduction of the PAEE estimation error (RMSE) by ANN over the traditional linear regression method, and a 40% reduction over a two-regression model (17).

4.3 Hybrid Approaches

It has been recognized that the accuracy of the PAEE regression models significantly depends on the specific types of activities used to develop the prediction models, and thus the estimation accuracy of certain types of activities may be better than others (5). As a result, no single regression model is able to estimate the PAEE for a wide range of activities of different intensity levels (14, 17). Therefore, branched regression models and hybrid approaches combining different computational techniques have been developed to enhance the PAEE estimation performance. Such a hybrid approach first separates the signal features into different activity groups based on either certain threshold criteria (11, 17, 21) or activity classification results (43, 53, 54). A regression model or a machine learning method is then used for estimating PAEE after being trained for each activity group. For example, Crouter et al. (17) used the coefficient of variation of accelerometer counts to separate walking/running (CV ≤ 10) and lifestyle/leisure time (CV > 10) activity types. Counts from each activity type were then directed to the appropriate regression model for estimating PAEE. Better PAEE estimation with an average percent error of 1.9% with a mean bias of 0.1 METs (95% confidence interval: -1.1~1.5 METs), was reported using the branched regression model in comparison to using a single linear regression model (60) (average percent error of 4.3%, bias of -0.4 METs). Their results indicate that using this branched regression model for different groups of activities of similar intensity improves the overall PAEE estimation accuracy by 56%. Sazonova et al. (54) applied SVM to first classify activities into one of the four groups (sit, stand, walk and cycle), and then used a separate linear model for each activity group to estimate the PAEE. Similarly, we investigated the utility of SVM to classify activities of similar intensity into four groups (sedentary, household, moderate locomotion, and vigorous sports). Instead of using a linear model, we used the support vector regression – the regression version of SVM, to estimate the PAEE. Result showed that the hybrid approach recognized the four activity groups 89.3% of the time and estimated the PAEE energy expenditure with a RMSE of 0.42 METs (43).

5. Conclusions and New Directions

This paper has presented an overview of recent advances in the estimation of PAEE from body-worn sensors. Specifically, different types of body-worn sensors, features extracted from the sensor signals, and computational methodologies that use the features as inputs have been reviewed. Past studies have primarily applied linear regression models to the accelerometer counts extracted from uniaxial accelerometer activity monitors to estimate the PAEE, due to merits such as relative low cost, low subject burden, long operating life, and simple implementation. Although such an approach is promising (24, 45), the lack of an accurate estimation of PAEE is one major factor that limits its broader impact (14). Seen from the various recent studies, the state of the art of research in PAEE estimation appears to be shifting in the following three directions:

1. Sensors. New designs and use of multiple accelerometers at different body locations. This is shown in the combined use of physiological sensors such as heart rate monitors and respiratory sensors, with one or more accelerometers, for PAEE estimation. The increased subject burden and cost will need to be examined for viable applications.

2. Features. Calibration studies that include additional variables other than the accelerometer counts in the estimation model, such as features extracted from the time and frequency domains of the sensor signals. These features represent the statistical distribution and temporal dynamics of the signals, and produce better estimates of the PAEE than using accelerometer counts alone.

3. Methodology. Advanced machine learning techniques or fusion of different techniques for better differentiation of activities with different intensities. These advanced computational methods should be flexible across subject groups and types of activities, and enable interactions among all input features. Consequently, they should provide better estimation accuracy of PAEE than the traditional regression methods.

Research activities in these three areas have the potential to further advance the state of science and engineering related to the assessment of energy expenditure, and produce a valid, reliable, and accurate estimation of PAEE. Moreover, the current techniques were mostly developed and validated in lab-controlled conditions. Although in some studies, the data were collected in free-living settings, the validation of the developed model was performed in the same way as in lab settings, where the models are generally cross-validated within the whole dataset instead of being evaluated on independent datasets. How to apply the lab-developed estimation model to data collected in real free-living scenario, and how to address various issues related to the validation of the model on free-living data, such as detecting activity transitions (for branched models, where different models are applied to different activity types), selecting an appropriate window size (which is related to the resolution of the detection of activity transitions), collecting labeled training data (the ground truth), etc., certainly deserve significant efforts and investigation in future studies.