Determining Physical Activity Characteristics from Wristband Data for Use in Automated Insulin Delivery Systems

Abstract

Algorithms that can determine the type of physical activity (PA) and quantify the intensity can allow precision medicine approaches, such as automated insulin delivery systems that modulate insulin administration in response to PA. In this work, data from a multi-sensor wristband is used to design classifiers to distinguish among five different physical states (PS) (resting, activities of daily living, running, biking, and resistance training), and to develop models to estimate the energy expenditure (EE) of the PA for diabetes therapy. The data collected are filtered, features are extracted from the reconciled signals, and the extracted features are used by machine learning algorithms, including deep-learning techniques, to obtain accurate PS classification and EE estimation. The various machine learning techniques have different success rates ranging from 75.7% to 94.8% in classifying the five different PS. The deep neural network model with long short-term memory has 94.8% classification accuracy. We achieved 0.5 MET (Metabolic Equivalent of Task) root-mean-square error for EE estimation accuracy, relative to indirect calorimetry with randomly selected testing data (10% of collected data). We also demonstrate a 5% improvement in PS classification accuracy and a 0.34 MET decrease in the mean absolute error when using multi-sensor approach relative to using only accelerometer data.

Graphical Abstract

I. Introduction

INCORPORATING physical activity (PA) in daily life provides several health benefits, such as reduced cardiovascular risk factors and improved well-being. PA also promotes metabolic health and glycemic control by increasing the effect of insulin on glucose uptake. Blood glucose responses to PA in people with Type 1 diabetes (T1D) are highly variable, and the imbalance in glucose availability and utilization during PA can increase the risk of dangerously low blood glucose levels (hypoglycemia) [1]–[3]. Therefore, people with T1D must make insulin dose adjustments to respond to the diverse modalities, intensities, and durations of PA [1]–[3]. Automated insulin dosing systems must also be informed of PA to adjust the computations of the proper insulin dose. PA increases the blood glucose consumption rate and excess insulin can further escalate blood glucose utilization, which can increase the risk of hypoglycemia during or several hours after PA in people with T1D [1]–[4]. Informing the insulin dose calculations of the PA characteristics is possible through the use of physiological data collected in real time from wearable devices [5]–[14]. The physiological variables measured by the sensors in wearbable devices must be processed to obtain valuable information for the medical dosing calculations [10]–[14]. Knowledge of the type, intensity, and duration of physical activities can be obtained by detecting and interpreting the physical state (PS) and estimating the energy expenditure (EE), leading to the preventive and therapeutic benefits for people with chronic diseases [3], [15]–[24].

Many consumer products provide good enough accuracy for tracking various daily activities and ascertaining summary health statistics [10]–[14], [25]–[30]. Several studies monitor PA using mobile phones and sensors embedded in clothing, and some studies compare the effects of the wearable sensor location on the accuracy of PA tracking [31]–[38]. PA information in medical applications such as insulin dose decisions requires stringent levels of accuracy and continuous use in free-living conditions with real-time data streaming [1], [3], [39]–[46]. Several studies report EE estimation exclusively from accelerometer sensors [47], [48], and some commercial products also provide EE estimates [5], [49]. A few studies use chestbands, armbands, and experimental devices to estimate the EE [50]–[57]. Despite the proliferation of numerous devices, wrist-worn PA trackers are preferred in applications where requirements include the convenience of the device and the continuous use in free-living and diverse ambulatory conditions encountered in daily life [20]–[24], [58].

In this study, data from the Empatica E4 [6] wristband device comprising of several sensors (Table I) are used with a sensor fusion approach and machine learning (ML) techniques to determine the PS of the subject and estimate the EE during the ongoing PA. We demonstrate that employing several wristband sensors (skin conductance and temperature, accelerometers, and blood volume pulse) and integrating the sensor data can improve the accuracy of the PS classification and the EE estimation compared to the use of a single sensor (for instance, accelerometers), resulting in more reliable PA information for therapeutic decision making [59]–[64]. Compared to the use of a single sensor (accelerometers), we show significant improvements in the PA classification (the accuracy increases by approximately 5%) and EE estimates (the mean absolute percentage deviation decreases by approximately 10%) when multiple sensors are integrated and used simultaneously for tracking PA. The PS considered in this work (sedentary state (SS), bicycling (BK), resistance training (RT), running (RN), and activities of daily life (DA)) are classified with an accuracy of 94.8%. The Empatica E4 device measures skin temperature (ST), 3-dimensional accelerometer (ACC), galvanic skin response (GSR), and blood volume pulse (BVP) based on photoplethysmography (PPG) from which heart rate (HR) is derived [6]. Features are extracted from these biosignals and the feature variables are used in ML algorithms to determine the PS and EE. The EE estimation accuracy of the proposed algorithms is compared to data from indirect calorimetry method using the COSMED Omnia K5 metabolic system [65], and the results demonstrate a low mean absolute error of 0.25 MET relative to the comparison indirect calorimetry method [65]. Over 430 h of experiments are conducted involving 25 subjects participating in the five PS considered for classification, and the accuracies of the various ML techniques are assessed.

TABLE I.

Biosensors of Empatica E4, Biosignals and Example Indicators

Sensor	Biosignal	Example Indicator
Infrared Thermopile	ST	Increased with resistance exercise [60], fall during biking exercise [59]
PPG	BVP	Alteration during bike exercise, noise (motion artifact) increase during any kind of physical activity
3-D ACC	ACC	Intensity of running, relatively stable during sedentary state and bike
PPG	HR	Increased during running, bike and resistance exercise, stable during sedentary state
GSR	GSR	Capturing sweating rate, intensity of daily activities, bike or running

II. Procedure

A. Data Collection

Wristband data are collected during five different PS (SS, BK, RT, RN, and DA). A total of 25 subjects (12 male/13 female) participated in a subset of the physical activities during the experiments. The subject demographic information (mean ± standard deviation) are: age 24.88 ± 3.15 yr, weight 72.42 ± 14.15 kg, height 1.70 ± 0.09 m, body mass index 25.00 ± 4.97 kg/m². Some subjects repeated certain physical activities, resulting in a number of experiments for each of the 25 subjects involved in the study (Table II). All experiment protocols were approved by the Institutional Review Board.

TABLE II.

Breakdown of the PS and EE Experiments for Data Collection

PS Experiments with Empatica E4
PA	Number Experiments	Number Subjects	Total Time (min)
SS	96	7	16758
DA	43	7	2783
RT	24	8	1016
RN	108	21	3741
BK	56	16	1602
EE Experiments with Empatica E4 and COSMED K5
PA	Number Experiments	Number Subjects	Total Time (min)
SS	10	4	2432
DA	6	4	673
RT	9	2	296
RN	30	7	645
BK	43	10	791

SS experiments are conducted while subjects are laying down or sitting and resting, and include activities like watching a movie, surfing the internet, reading a book, attending class lectures, working on a computer, and attending meetings. Seven subjects participated in 96 experiments for the SS data collection (Table II). The duration of the SS experiments is relatively longer than other PA because of the lack of fatigue and protracted comfort of the subjects during resting conditions relative to the other PS. Data for DA are collected from 7 subjects in 43 experiments including walking in the city, walking on the treadmill (1.7 mph – 3.5 mph), washing dishes, house cleaning, teaching a class, and shopping [66]–[71]. RN experiments are conducted mostly on the treadmill and includes a few outdoor runs. RN data include 180 experiments conducted by 21 different subjects with different speeds in the range of 3.5 – 8.0 mph [66]–[71]. The thresholds for walking and running were selected to correspond with the criteria used in existing literature studies [66]–[71]. BK experiments are primarily conducted with a stationary bike except for a few outdoors biking activities, with a total of 56 BK experiments conducted involving 16 different subjects at variable intensities in the range of 50 – 150 W.

RT activities include weightlifting, dumbbell chest press, leg extension, leg curl, lat pull down, dumbbell shoulder press, and seated row (rowing machine workout). Approximately 17 h of RT data are collected from 8 subjects. All data are recorded with labeled PS information. Overall, over 430 h of data from 25 subjects are collected for common types of daily PA from the large subject group [3], [39], [72]–[79].

In addition to the wristband data (Empatica E4) [6], indirect calorimetry data are collected with the COSMED Omnia K5 metabolic system during the 98 EE data collection experiments (Table II) to develop the EE estimation model [65]. The duration of data collected with the K5 metabolic system and E4 wristband is shorter than data collected using the E4 wristband only due to concerns for comfort of the subjects. Overall 80 h of experiments are conducted with the K5 metabolic system.

B. Pre-Processing

The Empatica E4 is a research grade device used in several studies to monitor subjects throughout daily life for events necessitating medical decisions or interventions [80]–[89]. The ACC, BVP, GSR, ST, and HR biosignals are measured by the Empatica E4 at 64, 32, 4, 4 and 1 samples per second, respectively [6]. Although more frequent sampling and raw data collection of the measurements can provide better tracking of PA in shorter time scales, the objective of this work is to use the PA information in insulin dosing decisions made every one to five minutes. Therefore, we use the Empatica E4 measurements and integrate them to determine the average PS and EE over each one-minute interval.

Different filters are used to eliminate noise from the collected data. A median filter and a Savitzky-Golay filter are chosen for ACC, HR, ST, and GSR data filtering [61]. The BVP signal is corrupted by motion artifacts and environmental noise, thus a cascaded signal processing technique is used, including band-pass filter (Butterworth), nonlinear recursive least squares for adaptive noise cancellation, and wavelet decomposition [90], [91]. The use of the sequential signal processing techniques reduced the correlation between the raw BVP signal and the denoised BVP signal from 0.258 to 0.008, demonstrating that the cleaned and processed BVP data are uncorrelated with the accelerometer measurements (Fig. 1).

Fig. 1.

Raw and Processed BVP Signals

C. Feature Extraction

The filtered biosignals are used to extract features at each one-minute interval. A comprehensive literature review is conducted to determine the appropriate feature extraction methods, including statistical parameters (standard deviation, median, max, kurtosis, skewness, etc.), mathematical functions (mean, differentiation, arccosine, etc.) and data-specific calculations (zero-cross of ACC readings, total-energy response of GSR, maximum amplitude of the very low frequency response of BVP, etc.) [61], [92]–[99]. Furthermore, we generated new features as the ratio of extracted features, which can be informative to distinguish the PS. For example, during BK the ACC readings (average magnitude of ACC) are relatively stable, yet the GSR readings (average magnitude of GSR) increase because of the higher sweating rate. Therefore, the ratio of the average magnitudes of ACC to GSR can be informative for discriminating the BK exercise from the other PS. To our knowledge, this has not been reported previously. The features pool includes 866 primary features extracted from 3D ACC, raw BVP, processed denoised BVP [90], HR, ST, and GSR readings, with 225, 148, 148, 98, 111, and 136 primary features, respectively. The same features are extracted from both raw and processed BVP readings because the raw BVP signal includes motion artifacts that can aid in discriminating certain PS [90]. In addition, the preprocessed signal represents HR variability independent of the motion artifacts, thus both the raw and preprocessed HR signals can provide informative features. An additional 1, 350 features are obtained from the ratios of the primary features, bringing the total features to 2, 216. A list of selected features that are informative for PS classification and EE estimation is provided in Table III.

TABLE III.

List of Select Informative Features

Physical State Classification
Feature Name	Time Delay (min)	p-value
Norm of 4th detail coefficient of Daubechies wavelet decomposition of ACC X-axis	0	≤ 0.01
Total energy response of BVP	1	≤ 0.01
Range for Magnitudes of ACC	0	≤ 0.01
Kurtosis of GSR	3	≤ 0.01
Standard Deviation of ACC / Std of GSR	0	≤ 0.01
Median of HR / Median of GSR	0	≤ 0.01
Std of ACC / Mean of ACC	0	≤ 0.01
Skewness of BVP	1	≤ 0.01
Minimum of HR	0	≤ 0.01
Mean of ST	0	≤ 0.01
p-values calculated using a two-sided t-test at α = 1% significance level
EE Estimation (Features with COSMED EE Measurements)
Feature Name	Time Delay (min)	Correlation
Std of magnitude of ACC (X-Y-Z)	0	0.56
Mean of logarithm of magnitude of ACC (X-Y-Z)	0	0.55
Maximum of ACC Z-axis	0	0.55
Zero cross of BVP	0	0.49
Minimum of high frequency of ST (0.15–0.4 Hz)	0	0.49
Root-mean-square energy of GSR	4	0.48
Maximum of power spectral density estimate of HR	0	0.48
1st Quartile of HR	0	0.48
Norm of 1st detail coefficient of Daubechies wavelet decomposition of GSR	0	0.48
Summation of absolute differentiation of GSR	12	0.34

During the SS experiments, a relatively larger dataset is collected. To avoid the negative impact of imbalances in the class sizes, the feature variables data for the other PS are upsampled by generating synthetic samples using the adaptive synthetic sampling approach (ADASYN) [100]. Finally, 102, 950 min of data are obtained; 16, 758, 20, 747, 23, 886, 18, 712, and 22, 847 mins of data for the SS, DA, RT, BK, and RN states, respectively.

Physiological time delay exists for some of the measurements during exercise, such as the delayed increase of the GSR relative the the onset of PA (Fig. 2). We consider the inherent time delay in the measurements within the modeling paradigms by incorporating time-delayed versions of the extracted features in the dataset, which enables consideration of timed delays of up to 15 min for the measurements. Therefore, any time delay of up to 15 min in the physiological measurements is captured by the models.

Fig. 2.

Example of the Time Delay in GSR Measurements Relative to the Start of Exercise as ACC and EE Rise in Advance of the GSR Readings

The data collected are divided into three mutually exclusive sets for training (80% of data), validating (10% of data), and testing (10% of data) our models and algorithms.

D. Dimension Reduction

The same pool of feature variables is used for PS classification and EE estimation. Most of the extracted features are highly correlated with each other (such as median of HR and mean of HR). Highly correlated features do not provide additional information for ML algorithms, and their use may cause bias to the same attributes, leading to poor estimation and classification accuracy. Principal component analysis (PCA) is used to reduce the dimensionality of the extracted features [101]. We determine all possible combinations of PS classes and compute the p-values for each feature variable between the PS classes. Therefore, we evaluate whether the feature variables are statistically different across pairs of classes, for all possible combinations of PS. The feature variables are retained for model development when the p-values from a two-sided t-test depict a significant difference across PS classes at a significance level of α = 1%. We use the more stringent significance level criteria relative to α = 5% to ensure a sufficient difference in the feature variables across the PS. Selected features (1730 features) are used in the PCA dimension reduction [102]. The number of retained principal components (PC) is optimized by varying the variance explained from 80 to 99% of the total variance in the feature variable space. Based on the classification accuracy, the first 175 PCs are retained, capturing over 95% of the variation in the overall features. The retained PCs are then normalized to range from 0 to 1 [103].

Partial least squares (PLS) regression is used to reduce the features to a smaller set of latent variables (LV) that are correlated with the EE [101]. The variance explained by the retained LVs is varied from 80 to 99%, and the number of LVs to be retained are determined based on the cross-validation of EE estimation accuracy. The first 75 LVs are retained, explaining over 97% of the total variation in all the extracted feature variables. The latent variables are normalized to range between 0 and 1.

E. Machine Learning Algorithms

The normalized PCs/LVs (inputs) and the corresponding numerically labeled PS information recorded (outputs) are used with supervised ML algorithms to classify the PS. The ML algorithms used include k-nearest neighbors (k-NN), support vector machines (SVM), naive Bayes (NB), decision tree (DT), neural networks (NN), linear discrimination (LD), ensemble learning (EL), and deep learning with long short-term memory (LSTM-DL). The models of the normalized LVs (inputs) and the corresponding gold-standard (from K5 metabolic system data) EE values reported in MET (Metabolic Equivalent of Task) (outputs) are trained and validated with several regression models including support vector regression (SVR), k-NN, linear regression (LR), DT, NN, Gaussian process regression (GPR), EL and LSTM-DL.

1). k-nearest neighbors:

We used hyperparameter optimization to determine distance [104], which is found as the cosine distance metric to compare relative distances between the feature variables of the testing data with the feature variables of the training data. Based on optimization results, the 10 neighbors with the shortest sorted distances are identified as being representative of each class. We assign the class label to the new data according to the most common class within 10 closest training samples.

2). Support Vector Machine/Regression:

The SVM have the advantage of easy adaptation to a feature-based classification approach for feature variables with nonlinear relations [105]. SVM functions are defined by determining a separating hyperplane in the high-dimensional feature space that best distinguishes two classes at the training stage. The separating hyperplane determined is used on-line with testing data to assign a class to the test data sample, based on the feature variable inputs [105]. We used SVM/SVR both to determine PS and estimate EE [105].

3). Decision Tree:

DT generates a predictive model that can be used as classification or regression model [106]. DT algorithms are sensitive to variations in data and relatively unstable [106].

4). Naive Bayes:

NB models are based on statistical information and compute the variance and mean of the selected subset of feature variables for a specific number of clusters [107]. We compute the Gaussian and kernel probability density for each feature, and we compute posterior probability values for the clusters [107]. The computed posterior values are compared to determine the classification.

5). Neural Network:

NN can capture nonlinear relations with multiple hidden layers [108]. Overfitting and predicting good initial parameters (number of layers, number of neurons, activation function) are the major challenges, which can be addressed by optimization. We used ordinary two-layer NN (15 hidden neurons in each layer) by utilizing the selection criteria (Eq. (1)) [109] for both PS classification and EE estimation

$$NHN=\frac{Ns}{\gamma \left(Ni+No\right)}$$ (1)

where Ni is the number of inputs, No is the number of outputs, Ns is the number of samples, and γ is an arbitrary scaling factor usually in the range 2–10 [109].

6). Gaussian Process Regression:

GPR is a stochastic statistical technique that seeks a multivariate normal distribution within random variables from the feature space domain. GPR measures the similarity between points (the kernel function) to predict the value from training data [110]. We used GPR to estimate EE.

7). Ensemble Learning:

EL uses trained multiple learners, in this case multiple decision trees, to achieve better accuracy than the individual models can achieve alone. Different methods are used to establish an ensemble learning model including boosting and bagging method [111]. Hyperparameters optimization helps to select the best method with the optimum number of learning cycle, learning rate and minimum leaf size. We developed an ensemble learning model to both classify PS and estimate EE [111].

8). Linear Discrimination/Regression:

LR uses features to create a linear model [112]. LD finds a linear combination of features that characterizes or separates two or more classes of objects or events. It is unable to capture nonlinearity. Physiological responses may contain some nonlinearities that cannot be handled with a linear model. We developed linear models to compare their classification and estimation performance with the other ML techniques [112].

9). Long-Short Term Memory and Deep Learning:

Deep learning (DL) is a combination of different layers including fully connected, long short-term memory (LSTM), softmax, regression, Rectified linear unit (ReLU), and dropout layers [113]. We evaluated various network structures to determine the most accurate deep NN for PS classification and EE estimation. Fig. 3 illustrates our proposed DL structure for PS classification and EE estimation. In addition, we used the L2 regularization term to reduce the risk of overfitting (value: 0.05).

Fig. 3.

Flow Chart of Proposed Deep Learning Structure

LSTM is a recurrent NN (RNN) architecture used in DL [114], [115]. Long-term dependencies are a significant problem which could not be solved with ordinary RNN. A modification of the ordinary RNN structure enables handling this problem with the solution of vanishing gradient problem. It commonly contains input gate (i), cell (C_t), update gate, forget gate (f), output gate (o), hidden state (h_t) and memory cell (g) (Fig. 4) [114], [115]. Update gate is a combination of the output of f, i, and g. LSTM enables evaluation of past and current information without long-term dependencies. In addition to current information, previously extracted features of biosignals have been considered in the LSTM to capture physiological alterations more rapidly. Deeper LSTM (two LSTM layers) is used with 30 hidden neurons for classification. We used a single LSTM layer with 10 hidden neurons for EE estimation. The numbers of hidden neurons (NHN) are selected based on Eq. (1).

Fig. 4.

Structure of LSTM Model

Dropout layer drops out hidden and visible units of network randomly, commonly used for regularization to reduce the risk of overfitting [113]. The softmax function can be used as a logistic regression model for the multi-classification task [113]. It transforms the numerical probability outputs of the network into the discrete class labels. It is commonly used as a last layer of a DNN. The fully connected layer is an ordinary hidden layer, which is generally used after LSTM layer. It has five hidden neurons (equal to the number of PS classified) for PS classification, and a single hidden neuron for EE estimation. ReLU layer performs a threshold operation to each element of the input, where any value less than zero is set to zero. It is commonly used as an activation function to improve the learning ability of the network [113]. Regression layer is commonly the last layer of estimation-based deep NN designs. It converts all predicted probabilities to actual expected output estimation value [113].

10). Hyperparameter Optimization:

The various hyperparameters for the different ML techniques are optimized with Bayesian optimization (expected improvement acquisition function) to achieve the best classification accuracy [116]. Table IV lists the hyperparameters for each ML algorithm. A large number iterations (100 iterations) are performed for each ML algorithm using 4-fold cross-validation techniques.

TABLE IV.

List of Hyperparameters Optimized for the ML Algorithms

ML Algorithm	Hyperparameter(s)
SVM/SVR	Kernel Scale, C parameter
DT	Min. Leaf Size
k-NN	Num. of Neighbours, Distance Function
LD	Delta, Gamma
NB	Distribution Name, Width
EL	Method, Learning Rate, Minimum Leaf Size

III. Results

The results are evaluated for the PS classification and the EE estimation algorithms using the testing data set and by employing various ML metrics.

A. PS Classification

We evaluated different ML algorithms (Fig. 5). LSTM-DL achieved a 94.8% accuracy with the test dataset, which was better than the other algorithms as the next-best algorithm (EL) achieved a classification accuracy of 93.2%. LSTM has the ability of bridging long time-lags between inputs. The LD and NB algorithms have lower classification performances below 80% with the same dataset and inputs.

Fig. 5.

Accuracy of Different Machine Learning Algorithms (Testing Data Set (10%)) (EL: Ensemble Learning, LSTM-DL: Long-Short Term Memory-Deep Learning, k-NN: k Nearest Neighbours, LD: Linear Discrimination, SVM: Support Vector Machine, DT: Decision Tree)

Precision and F-score are important common ML metrics. Table V reports the values of the metrics for different ML algorithms used for all PS. The receiving operating characteristic (ROC) curve provides a good illustration of recall, precision, and F-score with a single plot presenting the performances of the classification models (Fig. 6).

TABLE V.

Recall, Precision and F-Score (SS: Sedentary State, DA: Daily Activity, RT: Resistance Training, RN: Running, BK: Biking)

Recall
Algorithm	SS	DA	RT	RN	BK	Mean
LSTM DL	0.875	0.954	0.934	0.985	0.991	0.947
k-NN	0.918	0.865	0.896	0.976	0.985	0.928
SVM	0.820	0.805	0.843	0.908	0.890	0.853
LR	0.824	0.666	0.822	0.922	0.851	0.817
NB	0.930	0.592	0.840	0.711	0.822	0.779
DT	0.898	0.837	0.908	0.896	0.889	0.886
EL	0.973	0.901	0.934	0.960	0.907	0.935
Precision
Algorithm	SS	DA	RT	RN	BK	Mean
LSTM DL	0.985	0.886	0.962	0.935	0.974	0.948
k-NN	0.968	0.936	0.985	0.837	0.890	0.923
SVM	0.906	0.767	0.884	0.864	0.852	0.855
LR	0.852	0.796	0.833	0.758	0.797	0.807
NB	0.836	0.792	0.703	0.755	0.727	0.763
DT	0.903	0.848	0.914	0.884	0.879	0.886
EL	0.956	0.924	0.957	0.876	0.943	0.931
F-score
Algorithm	SS	DA	RT	RN	BK	Mean
LSTM DL	0.927	0.919	0.948	0.959	0.982	0.947
k-NN	0.942	0.899	0.938	0.901	0.935	0.923
SVM	0.861	0.785	0.863	0.886	0.871	0.853
LR	0.837	0.726	0.827	0.832	0.823	0.809
NB	0.881	0.678	0.766	0.732	0.771	0.766
DT	0.901	0.843	0.911	0.890	0.884	0.886
EL	0.964	0.912	0.946	0.916	0.925	0.933

Fig. 6.

ROC with LSTM Algorithm

We also compare the improvement in PS classification accuracy obtained by the proposed sensor fusion approach with an single sensor approach employing only ACC data. We generate features from the ACC data and train models with the features to generate algorithms relying exclusively on ACC data. Using ACC data only results in an accuracy of approximately 90%, which is lower than the almost 95% accuracy obtained through the sensor funsion approach integrating the five different biosignal simultaneously (Table VI).

TABLE VI.

Comparison of PS Classification and EE Estimation Results of Our Proposed Method with Other Similar Studies (RF: Random Forest; LMT: Logistic Model Tree; Inertial sensor includes 3D sensing unit: accelerometers, magnetometers, and gyroscopes; All acceloremeter sensors were 3-D)

Comparison of PS Classification Results
Year	Reference	Used Sensors	Accuracy(%)	Algorithm	Sensor Place	Device
2016	[117]	ACC	81	NN	Wrist	Wrist-mounted 3-D Accelerometers
2015	[118]	ACC,Gyroscope	85	LMT	Neck	MinimaxX S4
2016	[119]	Inertial Sensor	97	DT	Shoes	Physilog Gait Up
2015	[120]	Inertial Sensor	95	k-NN	Wrist, Ankle, Trunk	ReSense
2020	[35]	ACC	95	RF	Hip	ActiGraph
2020	[38]	ACC	93	DL	—	Smart Phone
2020	[37]	ACC	89	SVM	Hip and Thigh	GT3X+ and GT9X
2020	[36]	ACC, BVP, HR	92	RF	Hip and Wrist	Microsoft Band, ADXL335
—	Proposed Method	ACC, HR, ST, GSR, BVP	95	LSTM-DL	Wrist	Empatica E4
—	Proposed Method	ACC (Using only ACC)	90	LSTM-DL	Wrist	Empatica E4
Comparison of EE Estimation Results
Year	Reference	Used Sensors	MAE (MET)	MAPE (%)	RMSE (MET)	Device
2015	[50]	ACC, HR, NB	0.52	23.2	—	Zephyr, Shimmer2, BodyMedia Fit
2014	[51]	ACC, ECG	0.70	25.7	—	ADXL330 and five ECG Necklace
2015	[52]	ACC, HR, ST	0.87	32.8	—	BodyMedia FIT
2015	[53]	ACC, HR, RR	0.61	27.5	—	BodyMedia Fit, Zephyr, Shimmer
2015	[54]	ACC	0.31	—	—	LIS3L02AQ3 Sensor Data
2014	[55]	ACC, HR	—	—	1.0	ActiGraph GT3X+, Polar RS400
2020	[56]	HR	—	—	0.71	Suunto Smart Belt
2020	[57]	HR	—	16.8	—	Misfit Shine 2
—	Proposed Method	ACC, HR, ST, GSR, BVP	0.25	12.1	0.51	Empatica E4
—	Proposed Method	ACC (Using only ACC)	0.59	22.0	0.79	Empatica E4

B. EE Estimation

The root-mean square error (RMSE), mean absolute percentage error (MAPE), and mean absolute error (MAE) are calculated for the estimated EE compared to the indirect calorimeter measurements for the different ML algorithms (Table VII) for the testing dataset (10%). LSTM-DL performed better than the other algorithms with an MAE of 0.25 MET compared to 0.26 MET for the GPR, which has the best accuracy among the other algorithms.

TABLE VII.

Comparison of EE Estimation with Different Machine Learning Algorithms and Metrics

Algorithm	RMSE (MET)	MAPE (%)	MAE (MET)
LSTM DL	0.509	12.086	0.254
k-NN	0.520	14.763	0.335
SVM	0.537	13.174	0.264
LR	0.533	12.954	0.262
NB	0.610	15.603	0.464
GPR	0.534	12.880	0.261
EL	0.608	14.029	0.437
DT	1.539	25.157	0.961

We also evaluate the accuracy of the EE estimated when using the proposed sensor fusion approach against the case where a single sensor measurement is used, in this case only ACC data. We again generate features from the ACC data only for comparison, and evaluate it against the situation where features from all measured variables are integrated for EE estimation. The estimation error in terms of RMSE increased from 0.51 to 0.79 MET when only ACC data is used, or in MAPE from 12.1 to 22.0%, and in MAE from 0.25 to 0.59 MET (Table VI). Therefore, the use of multiple sensors and a sensor fusion approach results in better EE estimation accuracy in comparison to a single sensor approach.

IV. Discussion of the Results

A. PS Classification

The confusion matrix for LSTM-DL (Fig. 7) illustrates all misclassifications for each PS. The SS class is the most accurately classified PS because accelerometer readings are mostly stable and SS is easily distinguished from other states even when relying solely on the ACC data. A few BK activities are misclassified as SS (0.2% of the BK class samples are misclassified, a total of 21 mins out of 2225 mins) because during both BK and SS classes, the ACC readings are relatively stable (holding the handlebar of the bike). The SS activities are the most misclassified state (87.5% accuracy) because some low-intensity DA have similar biosignals readings and characteristics as the SS class. The GSR and ACC measurements have higher standard deviations during RT relative to other activities because the measurement values for GSR and ACC decrease substantially during the resting period between sets of repetitions. For example, the standard deviation of GSR across a RT experiment was found to be 0.99 μS compared to a maximum of 0.60 μS for other activities. The variability in the standard deviation of GSR and ACC is not the only indicator of RT class. The mean of ACC and GSR readings are also relatively lower during RT compared to RN or higher-intensity DA. For example, the mean of ACC and GSR across RT experiments were found to be 65.5 g and 0.79 μS compared to an average of 63.9 g and 0.06 μS for SS activities, respectively.

Fig. 7.

Confusion Matrix for LSTM-DL based Classification Result

Overall, the proposed method achieved 94.8% accuracy (Fig. 7) with the data collected from 25 participants and 430 h of experiments with five different PS. The results compare favorably against similar recent studies, with reported accuracies ranging between 95 to 98% using sensors in smart garments or shoes, and with novel devices integrating accelerometers, gyroscopes, and barometric pressure sensors [3], [35]–[38], [55], [67], [117]–[121].

A limitation of the proposed work is the number of subjects involved in the study and the classification of only five PA. The accuracy of the proposed methods can be improved by collecting more data from numerous subjects. The discrepancies in the number of subjects due to the PA preferences of individuals can lead to unbalanced class sizes and biased results, which we tried to mitigate by using upsampling techniques. Future work will augment the data with new subjects, particularly for exercises where few subjects performed the activity.

B. EE Estimation

Independent of the training, testing, and validation experiments, a new experiment is conducted for circuit training of approximately 110 min to evaluate the EE estimation accuracy. The experiment includes stationary bike (20 min), resting (10 min), rowing machine workout (20 min), resting (10 min), treadmill running (20 min), resting (10 min) and activities of daily living (20 min). The treadmill speed fluctuates between 3.5 and 6.0 mph at 0% incline. Stationary bike intensity is in the range of 50 – 110 W. Activities of daily living are emulated with treadmill walking at low speed in the range of 1.5–3.5 mph with 0% incline. Resistance exercise is done with a rowing machine workout in the range of 50 – 75 W. During resting periods, the subject sat in a chair and avoided any physical activities. Indirect calorimeter measurements and estimated EE values (LSTM-DL) shown in Fig. 8 illustrate that the EE estimation algorithms can track EE accurately over the range of physical activities.

Fig. 8.

EE Estimation Results Using an Independent Testing Dataset

Overall, the EE is estimated with approximately 0.5 MET RMSE over the 8 h testing dataset including different types of medium-intensity exercises. A separate experiment (approximately 2 h (110 min)) is conducted to verify our model, which yielded similar findings as the testing dataset (Fig. 8). Our method can achieve competitive results relative to values reported in recent literature focusing on EE estimation (Table VI) [50]–[57], [122]. The MAE of 0.25 MET achieved by our method is competitive with the literature reported values. Moreover, our results are based on a convenient and practical wristband physical activity tracker, thus suitable for use in free-living ambulatory settings.

C. Extensions of Proposed Method

Our method is based on a convenient and practical wristband device, similar to popular consumer wearable wristband activity trackers [10]–[14], [25]–[28]. This makes it an appealing solution for monitoring the PA type and intensity in medical applications [3], [20]–[24]. The high accuracy of the results from a wristband device means the approach can be useful for clinical decision making in managing chronic diseases such as diabetes and obesity [15]–[19]. The PS and EE information generated from the refined and processed signals can be used to inform automated insulin delivery systems to automatically modulate the insulin dosing based on the type and intensity of the PA [1], [3], [39]–[46]. Specifically, if an increase in the EE is detected, then the insulin doses can be decreased or suspended to avoid excess insulin delivery that may lead to an adverse low glucose values. Predictions of future glucose trajectory profiles are frequently used in managing Type 1 diabetes, and the models used for making the glucose predictions can be augmented with additional supplementary inputs like EE values to increase the accuracy of the glucose forecasts. The automated insulin dosing systems rely on control algorithms to determine the insulin dose amount, and using the PS and EE estimates to modify the control parameters by making the control algorithm suggest less insulin when EE is elevated can be instrumental in improving the lives of people using automated insulin delivery systems.

V. Conclusion

An approach to detect the PS and estimate the EE through physiological signals reported by a single wristband device with multiple sensors is presented. The proposed method achieved a 95% accuracy for PS classification and a RMSE of 0.51 MET for EE estimation. We also demonstrated that there is a significant improvement in PS classification and EE estimation using multiple sensors simultaneously in a sensor fusion approach as opposed to the use of a single sensor for monitoring PA. The accuracy of the PS classification and EE estimation results by using the sensors in a convenient wristband activity tracker will enable precision medicine by considering physical activity type and intensity in computing insulin dosing decisions.