Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2019 May 1.
Published in final edited form as: IEEE Sens J. 2018 Mar 8;18(9):3752–3758. doi: 10.1109/JSEN.2018.2813996

Accelerometer-Based Detection of Food Intake in Free-living Individuals

Muhammad Farooq 1, Edward Sazonov 1
PMCID: PMC6197813  NIHMSID: NIHMS959184  PMID: 30364677

Abstract

The goal of this pilot study is to evaluate the feasibility of using a 3-axis accelerometer attached to the frame of eyeglasses for automatic detection of food intake. A 3D acceleration sensor was attached to the temple of the regular eyeglasses. Ten participants wore the device in two visits (first, laboratory; second, free-living) on different days, reporting the food intake episodes using a pushbutton. Hold-one-out procedure was used to test the algorithm for food intake detection. The accelerometer signal was split into epochs of varying durations (3s, 5s, 10s 15s, 20s, 25s, and 30s); 152 time and frequency domain features were computed for each epoch. A two-stage procedure was used for finding the best feature set suitable for classification. The first stage used minimum Redundancy and Maximum Relevance (mRMR) to get the 30 top-ranked features and the second stage used forward feature selection along with a kNN classifier to get the optimum feature set for each hold-one-out set. The best average F1-score combined from laboratory and free-living experiments was 87.9 +/− 13.8% (Mean±Standard Deviation) for 20s epochs; and 84.7 +/− 7.95% for the shortest epoch of 3s. The results suggest that accelerometer may provide a compelling alternative to other sensor modalities, as the proposed sensor does not require direct attachment to the body and, therefore, significantly improves user comfort and social acceptability of the food intake monitoring system.

Index Terms: Wearable sensors, Eating recognition, Eyeglasses, Accelerometer, Food intake detection, Chewing, Dietary intake, Monitoring

I. Introduction

RECENT years have seen a significant increase in the body-worn sensors proposed for objective monitoring of food intake. These sensors are developed as a possible remedy to the limitations faced by the current techniques relying on self-reporting of dietary intake, which suffer from underestimation and impose a burden on the user [1]. Many of body-worn sensors for monitoring of food intake rely on food intake detection through detection of hand-to-mouth gestures [2], chewing [3], [4] and swallowing [5], [6]. Use of inertial sensors for hand to mouth gesture recognition was proposed in [7]. For monitoring of chewing or mastication, acoustic [8], proximity [9], and strain sensors [4], [10] have been proposed in the literature. For monitoring of swallowing, similar acoustic [11], strain [6], and bio-impedance measurement [5] sensors have been proposed. The results obtained with these body-worn devices are promising in terms of automatic detection of food intake. However, one common limitation of these devices is the need for their attachment to the body at different regions of the head e.g. on the larynx [5], [6]; in ear [8], [9]; or on the temporalis muscle [4], [10], [12]. Proper attachment of the sensors to the skin is required for better signal quality and, therefore, affects the performance of these devices. In addition, some of the head- and neck- worn sensors may create discomfort to the users and raise concerns about the social acceptability, which makes user compliance and the long-term usage of these devices questionable.

One solution is to incorporate the sensors into items of daily usage and daily wear in such a way that their impact on daily living is minimized while their performance is not compromised. One such possibility is to integrate sensors into eyeglasses. The eyeglasses are prevalent in most of the modern societies, for example, about 64% of the US population use eyeglasses on a daily basis [13]. Therefore, integrating sensors in eyeglasses is a viable and promising choice for monitoring of food intake. Researchers have explored eyeglasses with different types of sensors for objective and automatic detection of mastication and food intake. These approaches can be divided into two groups; one that relies on commercially-available glasses such as Google Glass and the other which are specifically designed to house food intake sensors such as 3D printed glasses. In [14], Raman et al., utilized the inertial sensors present on the Google Glass to capture head movements associated with chewing. They also integrated other sensors to capture audio and hand movements associated with dietary intake and have shown to achieve an accuracy of 82.7% for food intake detection in a controlled laboratory environment. Google glass is a specialized hardware with a display and cannot be considered as an everyday item. In [15], Zhang et al. presented 3D printed eyeglasses with attached EMG electrodes to detect chewing events. For chewing cycle detection, they reported precision and recall above 90% for in-lab study and above 77% for free-living part of their study. EMG electrodes require constant contact with the skin, and their placement varies with the size and type of the eyeglasses used. Chung et al. in [16], used load cells embedded in the glasses to detect chewing by monitoring the contraction and relaxation of the temporalis muscle. An adhesive piezoelectric strain sensor [4], [10] attached to the regular eyeglasses detected chewing events with a rate of 96.28% in both controlled laboratory and unrestricted free-living testing [10]. However, in this case, the sensor had to be adhered to the temporalis muscle, which limits its long-term usage. An accelerometer mounted on the temple of the glasses [17] was used to differentiate between a limited number of tasks in controlled setting: eating different foods, drinking from a cup and sitting idle in a chair. Due to the limited number of activities and lack of motion from the activities of daily living, the full potential of using accelerometers on the eyeglasses was not explored. In addition, most of the published studies relying on eyeglass sensors for detection of food intake were limited to the controlled laboratory conditions and their performance was not evaluated in unconstrained free-living environment.

Many of the eyeglass-based sensors for food intake detection require direct contact of the sensors with skin and are attached using medical adhesive (e.g. EMG or strain sensors). This limits the usability of the devices and might cause discomfort to the user. These approaches are also sensitive to the placement of the sensors that require careful placement on a specific location such as temporalis muscle. The proposed approach addresses these issues with the following contributions: (i) The use of a 3D accelerometer attached to the frame of regular glasses avoids direct sensor attachment to the body and issues associated with incorrect sensor placement and poor body contact. Such attachment also ensures that the device can potentially be removed and reattached to a wide variety of regular glasses and no expertise are required to wear the glasses as might be needed for other sensor attachments (ii) The food intake recognition methodology was tested both under controlled laboratory conditions and in the free-living individuals. Evaluation of the device in the wild is critical to study the practical usability of the device for real-life situations. (iii) The performance of the proposed method was evaluated at several time resolutions in models that do not require subject dependent calibration.

II. Methods and Material

A. Data Collection Protocol

Ten participants (8 males and 2 females) volunteered for this study. The recruited population had an average age of 29.03+/−12.20 years (mean +/− STD) and average body mass index (BMI) of 27.87 +/− 5.51 kg/m2. Participants were included if they did not report any difficulties with chewing. Participants were not paid for their participation. The study was approved by the Institutional Review Board at the University of Alabama. Participants signed an informed consent before the experiments.

Participants came for two visits on different days. The first visit consisted of a controlled laboratory experiment and the second visit consisted of a laboratory portion followed by unconstrained free-living testing. During the first visit each participant had to perform several tasks in the following order. First, there was a 5-minutes rest period (using phone or computer) which was followed by a small eating episode where a slice of cheese pizza was consumed. Eating was followed by a 5-minute period where the participants talked to the investigator. The last activity performed was walking on the treadmill for 5 minutes at a speed of 3 miles per hour (mph). Research suggest that depending on the age range, the normal walking speed is in the range of 2.8 to 3.37 mph [18] and therefore, a walking speed of 3 mph was chosen. Throughout the experiment, there were no restrictions on the body or head movements of the participants. They were also allowed to talk during the meal.

Eight of the ten participants came for the second visit. The remaining two participants chose not to continue with the study. The second visit had two parts, where, in the first part participants were required to perform several movements that could potentially result in motion artifacts in the sensor signals. These movements included up down, left and right head motions, upper body (trunk) movements, raising hands and transitioning between sitting and standing. These movements were performed 5 times each, and this laboratory session took about 15 minutes in total. The laboratory part was followed by a period of unrestricted free-living where participants were asked to follow their daily routine and have at least one meal that was usually their lunch at the place of their choice such as cafeteria etc. Total duration of the free-living data collected from eight participants was about 23 hours, or approximately 3 hours per person. Participants were required to keep a log of their eating episodes only. Out of 23 hours of free-living data about 3 hours were spent on eating. Since we were mostly interested in the ability of the system to detect food intake, users were not required to keep record of their other activities. During the free-living part, there were no restrictions where the participants obtained their food, the type of foods or manner in which the foods were consumed. Food items included pizza, pasta, sandwiches, fried rice, salads, apples, oranges, nuts and beverages such as water, soda and orange juice. The participants were asked to report all eating events (both solids and liquids).

B. Sensor System and Annotation

The wearable sensor system used in this study consisted of a small sensor module which housed a low-power 3D accelerometer (ADXL335 from Analog Devices, Norwood, MA, USA) and a Bluetooth module (RN-42). The sensor module was connected to the right temple of non-corrective eyeglasses by heat-shrink tube (Fig. 1).

Fig. 1.

Fig. 1

The sensor module with the Accelerometer connected to the right temple of the eyeglasses (the heat shrink tube attachment is not shown).

Data from the accelerometer was sampled at 100 Hz by a microprocessor (MSP430F2418, Texas Instruments, Dallas, TX, USA). Collected data from the accelerometer were wirelessly transmitted to an Android smartphone via the Bluetooth module. The data were processed offline for algorithm development and evaluation. Participants used a pushbutton (Fig. 1) to mark consumption of solid and liquid foods. For solid foods, participants were asked to press the button at the moment when the food was placed in the mouth (a bite), and hold the button until the last swallow related to a given bite. For liquids, they were instructed to press the button from when they brought the liquid to their mouth (a sip from the container or straw) until last swallow. Pushbutton signals were used as a reference for the development of signal processing and pattern recognition algorithms.

The working principle of the proposed sensor relies on the detection of temporalis muscle activity during chewing and other facial activities. During the chewing cycle, the lower jawbone (mandible) has up-down and side to side grinding movements which results in the contraction and relaxation of the temporalis epidermis [19]. This work used the oscillatory movements of the temporalis muscle during chewing captured by the accelerometer for detection of chewing events. The temporalis muscle also participates in the sucking, or drawing the liquids into the mouth by creating a negative pressure in the oral cavity. The accelerometer used in this work captured the acceleration in three-dimensional space across three axes i.e. Accx(t), Accy(t) and Accz(t) axes. Net acceleration AccNet(t) was computed from the accelerometer signals:

ACCNet(t)=ACCx(t)2+ACCy(t)2+ACCx(t)2 (1)

Fig. 2 shows the accelerometer signals and corresponding pushbutton signals during eating episodes. Fig. 2 (a) shows a segment of the experiment that contains both food intake and walking (physical activity). Walking has higher signal variations along all three axes compared to chewing. Fig. 2 (b) shows a zoomed version of the eating segments. Periodic patterns are visible in all three axes related to chewing. There are signal transitions along y-axis right before chewing segments. These transitions correspond to head movements (posture changes) associated with bites. Further analysis is needed to determine if this information can be used for detecting bites.

Fig. 2.

Fig. 2

(a) The first two rows show a segment of an experiment which involved eating and walking. Signals from 3-axes of the accelerometer are shown on the first row. Second row shows the pushbutton signal used by the participants for annotation of chewing bouts. Here ‘1’ indicates chewing and ‘−1’ indicates no chewing. (b) First row shows a closer look of the chewing segments along with the corresponding pushbutton signals (fourth row).

C. Feature Extraction

A high-pass filter with a cutoff frequency of 0.1 Hz was used to remove the DC component from the signal, and the signals were normalized to compensate for inter-subject variations [3]. For feature computation, signals were divided into non-overlapping fixed time segments/windows called decision epochs. Epoch duration determined the time resolution of the chewing detection. Our previous studies have used the epoch duration of the 30s for piezoelectric strain sensor placed on the jaw [3] and 3s for piezoelectric strain sensor placed on the temporalis muscle [4]. In [20], we have shown that an epoch size <5s is desirable to preserve information about the meal microstructure (eating bouts). To determine the best epoch duration for detection of chewing using the accelerometer signals, this work explored seven durations i.e. 3s, 5s, 10s, 15s, 20s, 25s, and 30s. The pushbutton signals were used for assignment of labels to each epoch. If more than half of an epoch belonged to food intake, the epoch i was assigned a label Ci = ‘1’ (food intake), otherwise the label Ci=‘−1’ (no food intake).

For each of the 3 axes of accelerometer and the net acceleration signal, a set of 38 features were computed for ith epoch. The corresponding feature sets were represented by fi,x, fi,y, fi,z and fi,Net for Accx(t), Accy(t), Accz(t) and AccNet (t), respectively. The feature vectors contained a combination of time and frequency domain features. Time domain features consisted of 1) the number of zero crossings per epoch, 2) the number of peaks per epoch, 3) mean, median, the standard deviation of the epoch data. Frequency-domain features consisted of features related to the frequency spectrum such as entropy and the standard deviation of the spectrum and the peak frequency of the spectrum. For frequency domain features, the signals were filtered in three different frequency bands which have been found to correspond to different activities i.e. 1.25 – 2.5 Hz contains information about chewing, 25–100 Hz contains information about physical activity such as walking and 100 – 300Hz contains information about speech [21], [22]. Details of these features are given in [3]. The final feature vector for the ith epoch was formed by concatenating the corresponding feature vectors i.e. fi = {fi,x, fi,y, fi,z, fi,Net }, which resulted in a vector with 152 features. The computed features along with the labels for each epoch were used for training of different classification models to detect food intake.

D. Feature Selection and Classification

To reduce redundancy in the computed features and avoid overfitting, a two-stage feature selection procedure was used. In the first stage the computed features were ranked based on their mutual information (relevancy and redundancy measurements) using the minimum Redundancy and Maximum Relevance (mRMR) [23]. The mRMR selection can be used for both continuous and discrete datasets. The second stage applied Forward Feature Selection (FFS) to the top-ranked 30 features selected by mRMR. The combination of mRMR and/FFS is a common way of practical utilization of mRMR in feature selection [23]. For classification, a k-nearest neighbor (kNN) classifier with k = 10 (found to give best results in initial experimentation) was used. The distance metric used for kNN was Euclidean distance. Separate classification models were trained, one for each epoch size.

Feature selection and classification were performed using a leave-one-out cross-validation procedure. During the 10-fold cross validation, features from 9 participants (training set) were first ranked using mRMR. Next, the subset of top 30 features was further reduced by FFS, applied in a 5-fold cross validation performed on the training data, where average classification accuracy was used as the selection criterion. The final set of features was used to train a classifier that was tested on the participant excluded from the training set (the 10th participant). This ensured that the test data is not used in the feature selection. For the test participant, the accuracy was evaluated separately on laboratory and free-living data. The cross-validation procedure was repeated 10 times such that data from each participant was used for testing once. The F1-score (weighted average of precision and recall) was used as the measure of classification accuracy:

F1=2PrecisionRecall/(Precision+Recall), (2)
Precision=TP/(TP+FP), (3)
Recall=TP/(TP+FN), (4)

where TP, FP, and FN denote true positives, false positives, and false negatives, respectively. Reported results are the average values for across test sets (10 test sets for laboratory part and 8 sets for free-living part). By choosing the F1 measure for evaluation of the classification models, the true negatives (non-food intake epochs) were not considered. Duration of food intake is relatively short (a few percent) part of daily life and inclusion of true negatives in the accuracy metrics would artificially inflate the performance of the classification models.

III. Results

Since feature selection was performed separately for each fold of the dataset and for a given epoch size, a different number of features were selected for each fold. Overall, the minimum number of features selected for a fold was 3 whereas the maximum number of selected features was 12. For each epoch size, some of the features repeated more than once during the 10-fold cross validation procedure. Table I shows the selected features with a frequency of 3 or more during feature selection, for each epoch size. Tables II and III show the F1-score along with the precision and recall of the kNN classifiers for different epoch size for laboratory and free-living datasets, respectively. Table IV show the combined results. The best combined result of 87.9 +/− 13.8% was obtained for 20s epoch.

TABLE I.

SELECTED FEATURES BASED ON THE FFS PROCEDURE FOR DIFFERENT EPOCH DURATIONS. FIRST COLUMN SHOWS THE DIFFERENT FEATURES WHICH WERE SELECTED AT LEAST THREE TIMES. THE NUMBER REPRESENTS THE EPOCH DURATIONS FOR WHICH A FEATURE WAS SELECTED.

X-axis Y-axis Z-axis Net-Acceleration
Number of Zero Crossings (ZC) 25
Mean time between ZC 10,15, 3,15,20,25,30
Number of Peaks (NP) 3,5,20 3,5,10,15,20,25,30 3,5,10,15,20,25,30
Range of amplitudes 3,5,10,15,20,25
Mean time between Peaks 3,5,10,15,25 3,5,10,15,25 3,5,10,15
ZC/NP 3,5,10,15,20,25 10,20,25,30 3,10,15,30
Slope sign changes 3,20,30 3,5,10,15,20,30
Spectrum energy (talking frequency band) 20,25,30
Abs(entropy_spectrum_chew)/Abs(entropy_spectrum_walk) 3,5,10,15,25 3,5, 3,
Walking energy/talking_energy (frequency bands) 3,5,10,20 3,5,10,15,20,25,30
Ppectrum_energy (chewing frequency band) 15,20,25,30
Entropy 3

TABLE II.

PRECISION, RECALL (SENSITIVITY), AND F1-SCORE FOR DIFFERENT EPOCHS, FOR LABORATORY PART. ALL VALUES ARE IN PERCENT. EPOCH SIZES ARE IN SECONDS.

Epoch (sec) Precision (%) Recall (%) F1-score (%)
3 90.3 +/− 4.9 92 +/− 7.2 90.9 +/− 4.4
5 90.1 +/− 7.8 93.6 +/− 6.7 91.3 +/− 5.4
10 90.2+/− 11.5 93.9 +/− 7.8 91.5 +/− 5.8
15 84.6 +/− 13.4 87 +/− 20.5 83.6+/− 16
20 88.6 +/− 16.9 94.8 +/− 9.0 90.1+/− 11.8
25 84.1+/− 23.5 90.6 +/− 10.6 83.8 +/− 18.9
30 83.9 +/− 16.8 98.3+/− 0.1 89.5 +/− 10.9

TABLE III.

PRECISION, RECALL (SENSITIVITY), AND F1-SCORE FOR DIFFERENT EPOCHS FOR FREE-LIVING EXPERIMENTS. ALL-VALUES ARE IN PERCENT.

Epoch (sec) Precision (%) Recall (%) F1-score (%)
3 83.9+/− 11.2 75.3 +/− 13.5 78.6 +/− 10.5
5 85.1 +/− 10.8 77.1 +/− 15.1 80.0 +/− 10.5
10 86.5 +/− 10.2 76.5 +/− 12.7 80.0 +/− 9.2
15 91.6+/− 6.4 75.7 +/− 29.4 79.4+/− 21.4
20 88.6 +/− 8.5 85.4 +/− 19.4 85.8+/− 11.7
25 86.9 +/− 10.2 80.2 +/− 14.2 81.8+/− 8.7
30 84.7 +/− 7.6 88.2 +/− 12 84.9 +/− 6.0

TABLE IV.

PRECISION, RECALL (SENSITIVITY), AND F1-SCORE FOR DIFFERENT EPOCHS FOR COMBINED (LABORATORY AND FREE-LIVING) RESULTS. ALL VALUES ARE IN PERCENT.

Epoch (sec) Precision (%) Recall (%) F1-score (%)
3 87.1+/− 9.3 83.7 +/− 10.9 84.7 +/− 7.95
5 87.6 +/− 10.9 85.6 +/− 10.3 85.8 +/− 7.5
10 88.4 +/− 9.9 85.2 +/− 25.0 85.7+/− 18.7
15 88.1 +/− 12.7 81.4+/− 14.2 81.5+/− 11.8
20 88.6 +/− 16.9 90.1 +/− 12.4 87.9 +/− 13.8
25 85.5 +/− 12.2 85.4 +/− 6.0 82.8 +/− 8.5
30 84.3 +/− 12.2 93.3 +/− 6.0 87.2 +/− 8.5

IV. Discussion

The objective of this pilot work was to propose and evaluate the ability of a single 3-axis accelerometer attached to the temple of the glasses to detect food intake in free-living individuals. This work used a heat-shrink tube to connect the sensor to the temple of regular eyeglasses without the need of special 3D printed frames to house the electronics. Connecting the sensor to regular eyeglasses without the need for special hardware (3D printed frames or Google Glass) is a viable option since about 64% of the US population uses eyeglasses. Such a system can help in improving the comfort of the user while using the device as well as potentially improve the user compliance. The sensor module presented here was based on older technology (Bluetooth 2.0). The size and form-factor of the device can be dramatically miniaturized with use of modern Bluetooth LE platform.

This work explored different epoch durations for detection of food intake. Selecting proper epoch duration is important because the epoch duration defines the time resolution of the food intake recognition and, in turn, meal microstructure [20]. For example, smaller epoch will provide better time resolution and can be helpful in detection of short eating episodes such as snacking. Longer epochs can provide better accuracy by using more data but can result in lower time resolution and inaccurate representation of the meal microstructure.

The feature selection procedure resulted in different number of features for different folds of 10-fold cross validation. There were several features common among the various folds of the selection process. Features such as number of peaks, average time between peaks, average time difference between zero crossings and slope sign changes etc. are related to the periodicity of the signal. Other selected features are associated with the spectral contents of the signals such as the spectral energy of different frequency bands of different activities (chewing, walking and talking) and entropy.

A general trend for both the laboratory and free-living results was that the performance of the classifier increased with the increase in epoch duration (decrease in time resolution) up to a certain epoch size (10s for laboratory (F1-score: 91.5+/−5.8%) and 20s for free-living data (F1-score: 85.8+/−11.7%)). For combined data (laboratory and free-living), there is an increasing trend until 20s epoch size (average F1-score: 87.9 +/− 13.8%). Considering the range of chewing frequency (0.94 to 2.17 Hz), the epoch durations of 10s and 20s will ensure the presence of multiple chewing events. Recent wearable systems presented in the literature have reported food intake detection accuracies in the ranges of 80% to 99.4% in controlled laboratory studies and 89% to 96% in unrestricted free-living conditions, using a wide variety of sensors for monitoring of bites, chew and swallowing. The system presented here has a comparable accuracy with a much simpler and user-friendly sensor. The presented sensor may be suitable to study dietary intake patterns and extract information about meal microstructure, such as meal duration, number of eating bouts, etc.

The sensor was tested in both controlled laboratory setting as well as in an unrestricted free-living. The presented methodology for food intake detection was robust to account for inter-person variations. Models were trained using leave-one-out cross-validation, which ensured that participant (subject) specific calibration of the models was not required and that the models can be generalized to larger populations.

One limitation of this study is that the intake of liquids was considered together with intake of solids, as most meals are consumed mixed. This was done to ensure that the user eating behavior is not changed or restricted in any way. Although previous research suggests that there are characteristic jaw movements during consumption of liquids similar to those of chewing [22], however, further research is needed in an attempt to differentiate solid and liquid intake with the proposed approach. A single push-button for ground truth was used for both solid and liquids and, therefore, it was not possible to differentiate between solid and liquid intake events in the free-living part of the study.

Also, the ability of the device to detect food intake when the participants were physically active (such as eating while walking) was not explicitly tested. There are other approaches that can detect eating even if the user is physically active such as snacking on the move. However, that approach required sensor placed directly on the temporalis muscle [4]. Further studies will explore the long-term use of the device and will focus on issues related to user comfort and compliance.

Another limitation of this pilot study was the small sample size of 10 participants. Although, the results presented in this pilot study are promising, further studies will be conducted to replicate these results in a larger population and for longer durations. User compliance with wearing of eyeglasses for longer term monitoring needs to be tested in future studies. The use of pushbutton to provide accurate ground truth data could potentially limit consumption of certain foods that may require use of both hands. However, it is not required for actual use of the proposed device in free-living and, thus, is not a limitation of the proposed approach in general. Future research will also explore the possibility of including a camera in the device. In this case, the sensor will be used for detection of eating episodes, and the camera will be triggered based on the sensor signals to take images of the food being consumed. Computer vision techniques such as deep learning methods could potentially be used for recognition of the type of food consumed.

An added potential advantage of this device is its potential ability to recognize physical activity being performed by the participants because of the use of an accelerometer as shown in [4]. Accelerometers are a popular choice to differentiate among activities such as sitting, standing, walking, going upstairs and downstairs [24]. Thus, by using this approach, there is a possibility to use a single sensor for monitoring of both dietary intake (energy intake) and physical activity patterns (energy expenditure), and this will be the topic of further research.

V. Conclusion

This work presented a novel approach for automatic and objective detection of food intake using a single 3-axis accelerometer sensor. The accelerometer was connected to the temple of the glasses and monitored the periodic movements of the eyeglass frame caused by the contraction and relaxation of the temporalis muscle during eating. This work explored different epoch durations for determining the best time resolution. Overall, best average F1-score of 87.9% was achieved for 20s whereas for shortest epoch size of 3s, the average F1-score achieved was 84.7%. These results show that the proposed approach can provide accuracy comparable to other devices presented in literature without the need of using sensors that require constant contact with the skin.

Acknowledgments

Research reported in this publication was supported by the National Institute of Diabetes and Digestive and Kidney Diseases (grants number: R01DK100796). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Biographies

graphic file with name nihms959184b1.gif

Muhammad Farooq received Master and PhD in Electrical Engineering from University of Alabama, Tuscaloosa, AL, in 2013 and 2016, respectively. He received bachelor’s in Electrical Engineering from University of Engineering and Technology, Peshawar, Pakistan. Currently, he is working as Senior Machine Learning Engineer at Phyn. Prior to this, he worked as Postdoctoral Research Fellow in the Department of Electrical and Computer Engineering at the University of Alabama. He has research interests in the development of wearable systems, sensor networks and machine learning algorithms for preventive, diagnostic, and assistive health technology with a special focus on physical activity and dietary intake monitoring.

graphic file with name nihms959184b2.gif

Edward Sazonov (IEEE M’02, SM’11) received the Diploma of Systems Engineer from Khabarovsk State University of Technology, Russia, in 1993 and the Ph.D. degree in Computer Engineering from West Virginia University, Morgantown, WV, in 2002. Currently he is a Professor in the Department of Electrical and Computer Engineering at the University of Alabama, Tuscaloosa, AL and the head of the Computer Laboratory of Ambient and Wearable Systems (http://claws.eng.ua.edu). His research interests span wireless, ambient and wearable devices, and methods of biomedical signal processing and pattern recognition. Devices developed in his laboratory include a wearable sensor for objective detection and characterization of food intake; a highly accurate physical activity and gait monitor integrated into a shoe insole; a wearable sensor system for monitoring of cigarette smoking; and others. His research has been supported by the National Science Foundation, National Institutes of Health, National Academies of Science, as well as by state agencies and private industry and foundations.

References

  • 1.Schoeller DA, Bandini LG, Dietz WH. Inaccuracies in self-reported intake identified by comparison with the doubly labelled water method. Can J Physiol Pharmacol. 1990 Jul;68(7):941–949. doi: 10.1139/y90-143. [DOI] [PubMed] [Google Scholar]
  • 2.Dong Y, Scisco J, Wilson M, Muth E, Hoover A. Detecting periods of eating during free-living by tracking wrist motion. IEEE J Biomed Health Inform. 2014 Jul;18(4):1253–1260. doi: 10.1109/JBHI.2013.2282471. [DOI] [PubMed] [Google Scholar]
  • 3.Fontana JM, Farooq M, Sazonov E. Automatic Ingestion Monitor: A Novel Wearable Device for Monitoring of Ingestive Behavior. IEEE Trans Biomed Eng. 2014 Jun;61(6):1772–1779. doi: 10.1109/TBME.2014.2306773. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Farooq M, Sazonov E. A Novel Wearable Device for Food Intake and Physical Activity Recognition. Sensors, vol. 2016 Jul;16(7):1067. doi: 10.3390/s16071067. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Farooq M, Fontana JM, Sazonov E. A novel approach for food intake detection using electroglottography. Physiol Meas. 2014 May;35(5):739. doi: 10.1088/0967-3334/35/5/739. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Kalantarian H, Alshurafa N, Le T, Sarrafzadeh M. Monitoring eating habits using a piezoelectric sensor-based necklace. Comput Biol Med. 2015 Mar;58:46–55. doi: 10.1016/j.compbiomed.2015.01.005. [DOI] [PubMed] [Google Scholar]
  • 7.Jasper PW, James MT, Hoover AW, Muth ER. Effects of Bite Count Feedback from a Wearable Device and Goal Setting on Consumption in Young Adults. J Acad Nutr Diet. 2016 Jun;0(0) doi: 10.1016/j.jand.2016.05.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Passler S, Fischer W-J. Food Intake Monitoring: Automated Chew Event Detection in Chewing Sounds. IEEE J Biomed Health Inform. 2014 Jan;18(1):278–289. doi: 10.1109/JBHI.2013.2268663. [DOI] [PubMed] [Google Scholar]
  • 9.Bedri A, Verlekar A, Thomaz E, Avva V, Starner T. Detecting Mastication: A Wearable Approach. Proceedings of the 2015 ACM on International Conference on Multimodal Interaction, New York, NY, USA. 2015:247–250. [Google Scholar]
  • 10.Farooq M, Sazonov E. Segmentation and Characterization of Chewing Bouts by Monitoring Temporalis Muscle Using Smart Glasses with Piezoelectric Sensor. IEEE J Biomed Health Inform. 2016;PP(99):1–1. doi: 10.1109/JBHI.2016.2640142. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Makeyev O, Lopez-Meyer P, Schuckers S, Besio W, Sazonov E. Automatic food intake detection based on swallowing sounds. Biomed Signal Process Control. doi: 10.1016/j.bspc.2012.03.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Wang S, Zhou G, Hu L, Chen Z, Chen Y. CARE: Chewing Activity Recognition Using Noninvasive Single Axis Accelerometer. Adjunct Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2015 ACM International Symposium on Wearable Computers, New York, NY, USA. 2015:109–112. [Google Scholar]
  • 13.Zhang R, Bernhart S, Amft O. Diet eyeglasses: Recognising food chewing using EMG and smart eyeglasses. 2016 IEEE 13th International Conference on Wearable and Implantable Body Sensor Networks (BSN) 2016:7–12. [Google Scholar]
  • 14.Rahman SA, Merck C, Huang Y, Kleinberg S. Unintrusive Eating Recognition Using Google Glass. Proceedings of the 9th International Conference on Pervasive Computing Technologies for Healthcare, ICST, Brussels, Belgium, Belgium. 2015:108–111. [Google Scholar]
  • 15.Zhang R, Amft O. Monitoring chewing and eating in free-living using smart eyeglasses. IEEE J Biomed Health Inform. 2017 Apr; doi: 10.1109/JBHI.2017.2698523. [DOI] [PubMed] [Google Scholar]
  • 16.Chung J, Chung J, Oh W, Yoo Y, Lee WG, Bang H. A glasses-type wearable device for monitoring the patterns of food intake and facial activity. Sci Rep. 2017 Jan;7:41690. doi: 10.1038/srep41690. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Biallas M, Andrushevich A, Kistler R, Klapproth A, Czuszynski K, Bujnowski A. Feasibility Study for Food Intake Tasks Recognition Based on Smart Glasses. J Med Imaging Health Inform. 2015 Dec;5(8):1688–1694. [Google Scholar]
  • 18.Lee I-M, Hsieh C-C, Paffenbarger RS., Jr Exercise intensity and longevity in men: The Harvard Alumni Health Study. J Am Med Assoc. 1995;273(15):1179–1184. [PubMed] [Google Scholar]
  • 19.Elsevier: Gray’s Anatomy, 41st Edition: Standring. [Online]. Available: https://elsevier.ca/product.jsp?isbn=9780702052309. [Accessed: 02-Feb-2017].
  • 20.Doulah A, et al. Meal Microstructure Characterization from Sensor-Based Food Intake Detection. Front Nutr. 2017;4 doi: 10.3389/fnut.2017.00031. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Fontana JM, Sazonov ES. A robust classification scheme for detection of food intake through non-invasive monitoring of chewing. 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) 2012:4891–4894. doi: 10.1109/EMBC.2012.6347090. [DOI] [PubMed] [Google Scholar]
  • 22.Sazonov E, Fontana JM. A Sensor System for Automatic Detection of Food Intake Through Non-Invasive Monitoring of Chewing. IEEE Sens J. 2012 May;12(5):1340–1348. doi: 10.1109/JSEN.2011.2172411. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Peng H, Long F, Ding C. Feature selection based on mutual information: criteria of max-dependency, max-relevance, and min-redundancy. IEEE Trans Pattern Anal Mach Intell. 2005;27(8):1226–1238. doi: 10.1109/TPAMI.2005.159. [DOI] [PubMed] [Google Scholar]
  • 24.Karantonis DM, Narayanan MR, Mathie M, Lovell NH, Celler BG. Implementation of a real-time human movement classifier using a triaxial accelerometer for ambulatory monitoring. IEEE Trans Inf Technol Biomed. 2006 Jan;10(1):156–167. doi: 10.1109/titb.2005.856864. [DOI] [PubMed] [Google Scholar]

RESOURCES