Energy intake estimation from counts of chews and swallows

Abstract

Current, validated methods for dietary assessment rely on self-report, which tends to be inaccurate, time-consuming, and burdensome. The objective of this work was to demonstrate the suitability of estimating energy intake using individually-calibrated models based on Counts of Chews and Swallows (CCS models). In a laboratory setting, subjects consumed three identical meals (training meals) and a fourth meal with different content (validation meal). Energy intake was estimated by four different methods: weighed food records (gold standard), diet diaries, photographic food records, and CCS models. Counts of chews and swallows were measured using wearable sensors and video analysis. Results for the training meals demonstrated that CCS models presented the lowest reporting bias and a lower error as compared to diet diaries. For the validation meal, CCS models showed reporting errors that were not different from the diary or the photographic method. The increase in error for the validation meal may be attributed to differences in the physical properties of foods consumed during training and validation meals. However, this may be potentially compensated for by including correction factors into the models. This study suggests that estimation of energy intake from CCS may offer a promising alternative to overcome limitations of self-report.

Introduction

The study of ingestive behavior in humans is important to identify and analyze specific patterns of food intake associated with chronic diseases, such as obesity and type 2 diabetes (Bellisle, 2009). It is critical to ensure that dietary intake of free-living subjects be measured accurately and objectively. Doubly-labeled water (Schoeller & van Santen, 1982; Schoeller & Webb, 1984) is the most precise method for measuring energy intake over a long period of time, but cannot identify individual eating episodes. Food frequency questionnaires, food records, and 24-hour dietary recalls rely on subjects’ self-report of their daily dietary intake and prone to self-report errors (Thompson & Subar, 2008). Errors in self-reported intake occur when subjects incorrectly report portion sizes and/or foods consumed (Beasley, Riley, & Jean-Mary, 2005) or change their eating behavior when asked to record intake (Goris & Westerterp, 1999; Goris, Meijer, Kester, & Westerterp, 2001).

Previous studies have explored the use of self-report in combination with technology to improve the accuracy of estimating energy intake (Ngo et al., 2009; Thompson, Subar, Loria, Reedy, & Baranowski, 2010). Audio reports (van Horn et al., 1990), photographic food records (Martin et al., 2014), personal digital assistants (McClung et al., 2009) and smart cards (Lambert et al., 2005) are some of the methods investigated. While these methods are faster and less burdensome than pen and paper recording, they still rely on subjects having to take some action to report intake. Most of these tools do not reduce underreporting of dietary intake (Ann Yon, Johnson, Harvey-Berino, & Gold, 2006; McClung et al., 2009) and further development is necessary to improve validity and reliability (Ngo et al., 2009). Consequently, it is vital to develop innovative methods to measure energy intake of free-living subjects objectively, unobtrusively, and accurately.

Automatic methods for objective dietary assessment based on wearable sensors have been explored as a potential solution to replace self-reported intake (Päβler, Wolff, & Fischer, 2012; Sazonov et al., 2008; Scisco, Muth, Dong, & Hoover, 2011; Sun et al., 2010). Food intake detection through recognition of chewing and swallowing instances differentiates food intake from other activities of daily living, such as talking, yawning, laughing, spontaneous swallows (saliva), head motion, etc. and does not require user input (J. M. Fontana, Farooq, & Sazonov, 2014; Makeyev, Lopez-Meyer, Schuckers, Besio, & Sazonov, 2012; Sazonov & Fontana, 2012). An earlier study (Sazonov et al., 2009) used the information extracted from the temporal sequence of chews and swallows to estimate the mass of food consumed but only considering a highly restricted selection of foods. The insight obtained from that preliminary study on mass intake estimation was used to design a new study focused on energy estimation, which is described in this article.

The objective of this work was to demonstrate the suitability of using individualized models based on Counts of Chews and Swallows (CCS) to objectively estimate energy intake. The CCS models were obtained from a laboratory study where chews and swallows were monitored using wearable sensors and video observation. The performance of the CCS models was compared against weighed food records, diet diaries, and photographic food records.

Subjects and Methods

Subjects

30 healthy subjects (15 females and 15 males) with a mean (±SD) age of 29 ± 12 y (range: 19–58 y) and a mean (±SD) body mass index (BMI, in kg/m²) of 27.9 ± 5.5 (range: 20.5–41.7) were recruited to participate in this study. The study was approved by an Institutional Review Board and all subjects read and signed an informed consent form before participating. Subjects with temporo-mandibular joint (TMJ) disease, dysphagia or other difficulties for chewing and/or swallowing were excluded from the study.

Study design

Four different methods were used in this study to estimate the total amount of food ingested by subjects at meal time: weighed food records completed by study staff; diet diaries completed by subjects; photographic food records taken by study staff; and mathematical models based on CCS. Data from weighed records, diet diaries, and photographic records were entered into the food analysis program, Nutrient Data System for Research (NDS-R; University of Minnesota, Minneapolis, MN) to derive total energy intake. Data entry was performed by a single, trained operator at the Colorado Clinical and Translational Sciences Institute’s (CCTSI) Nutrition Core. Total energy intake for the CCS model was estimated by combining mass estimations and caloric densities of each food eaten. Individual caloric densities were extracted from the nutritional analysis performed by NDS-R.

Comparisons were made between CCS models, diet diaries, and photographic records with respect to weighed records which were used as the gold standard. The difference was expressed as the absolute value of the percent of error (hereafter reporting error).

Protocol

Each subject consumed 4 full meals in 4 different visits at the laboratory. Visits occurred approximately 1–4 weeks apart, at exactly the same time of the day, but not necessarily on the same day of the week with the expectation that any potential difference in the eating behavior between weekend and weekdays (Haines, Hama, Guilkey, & Popkin, 2003) should be accurately captured. Approximately one third of the total subjects were scheduled for breakfast, one third for lunch, and one third for dinner time to cover the variety of foods typical for these meals.

Each subject was asked to choose two different meal selections according to their own preferences from the menu offered by one of the Clarkson University food courts. Precise nutritional information of each food selection of the menu was readily available. A typical meal selected by subjects contained 1 to 3 different food types and 1 or 2 different drinks.

Table 1). The first meal selection was served in three of the visits (exact same food types and drinks) and used as training meals for CCS mathematical model development. The second meal selection was randomly served either in the third or the fourth visit and considered the validation meal.

Table 1.

Description of type and frequency of the foods served during the experiments (30 participants, four meals each). Portion sizes were standard sizes of foods sold in the food court

Food Item	Number of times served	Food Item	Number of times served
Apple	13	Muffin	7
Bacon	7	Oatmeal	8
Bagel	3	Orange	7
Banana	20	Pancakes	1
Breadstick	3	Pasta	12
Brownie	3	Pickle	1
Burger	3	Pizza	16
Carrots	2	Potatoes	5
Cereal with milk	17	Salad	16
Chicken nuggets	5	Sandwich	22
Chips	12	Sausage	6
Chocolate Milk	6	Soda	12
Cinnamon Roll	6	Spinach	2
Coffee	9	Squash	1
Cookie	33	Stir fry	6
Corn	4	Meatballs	1
Eggs	10	Tea	13
French fries	6	Toast	8
Hot Chocolate	16	Turkey breast	4
Juice	22	Waffle	2
Meatloaf	1	Water	54
Milk	9	Yogurt	19
Mixed veggies	2

The subjects’ eating behavior was recorded with wearable sensors for monitoring of swallowing and jaw motion (Table 2 and Figure 1) and a video monitoring system (J. Fontana, Lopez-Meyer, & Sazonov, 2011). Each visit followed the procedure shown in Figure 2. All subjects had an unlimited time to consume the meal in the amount they desired. The stored sensor data and video were then used by a human rater to manually annotate the foods being consumed, bites, chewing sequences and boundaries of every spontaneous (saliva) and food swallow using custom-designed software (Sazonov et al., 2008). The swallowing instances and the number of chews in a chewing sequence were counted and annotated. The annotations were used to build the energy intake models described in this manuscript.

Table 2.

Description of the instrumentation module used to monitor the ingestive behavior of participants during each visit.

MODALITY	SENSOR	SENSOR DESCRIPTION	SENSOR LOCATION
Swallowing	Throat microphone (IASUS NT)	The microphone allowed the detection of swallowing sounds.	Over the laryngopharynx (Figure 1, left). Fastened to the neck using a neoprene collar.
Chewing	Piezoelectric strain sensor (LDT0-028K, Meas-Spec Inc.)	The strain sensor allowed monitoring the jaw motion during chewing.	Immediately below the earlobe (Figure 1, right). Attached to the skin using medical adhesive (Hollister 7730).
Videotape	Digital camera (PS3 Eye, Sony Corporation)	A PS3 Eye camera was used to videotape the participants throughout the experiment.	Position of the camera was adjusted to obtain an acceptable view of the subject under study.

Figure 1.

Wearable sensors used to monitor swallowing and chewing activities. The figure on the left hand side shows the swallowing sensor, a throat microphone that captured sounds at the larynx level. The figure on the right hand side shows the chewing sensor, a piezoelectric strain sensor that captured changes in the skin curvature produced during motion of the jaw bone.

Figure 2.

Flow diagram describing the protocol followed to collect food intake data at each visit.

Energy intake measurements

Weighed food records

A trained member of the study staff documented the food choices and weighed all foods and beverages individually before and after each meal.

Diet diary

During the first visit, subjects were trained in completing a diet diary by a member of the study staff. Provided instructions included verbal and written information on estimation and recording of foods and portion sizes as well as “a portion estimation aid” sheet (CCTSI Nutrition Core, 2014a, 2014b). The training was only provided on the first visit so that subsequent records were obtained under conditions most similar to free living.

At the end of each meal, subjects were instructed to record each item they consumed in a blank diary, indicating the type of food, amount consumed, and preparation style. No help was given to the subjects during this stage; however, the journal was reviewed to ensure that it was completed appropriately (i.e. all food listed had a portion size and description assigned). Subjects were not prompted to add any food or drink they had forgotten to list.

Photographic food records

Pre- and post-meal photographs were taken using a digital camera by a trained member of the study staff. The serving plate occupied the entire field of view and photographs were taken at a 45 degree angle so that the depth of foods could be estimated (Higgins et al., 2009). Pictures of each food item were taken before the meal and after subjects finished eating and then used by a trained CCTSI Nutritionist to estimate portion sizes and calculate total energy intake.

Counts of Chews and Swallows

Estimation of the energy consumed during each meal was performed using individually-calibrated CCS models. Figure 3 illustrates the procedure for training and validation of the mass estimation models. During model training, the following individually-calibrated parameters were computed for each subject based on the total number of chews and swallows observed in the training meals:

• -Average mass per chew of solid foods (MPChew)
• -Average mass per swallow of solid foods (MPSwS)
• -Average mass per swallow of liquids (MPSwL).

Figure 3.

Procedure for training and validation of the mass estimation models. Counts of chews and swallows from meal selection 1 were used to train the models. Same information from meal selection 2 was used to validate the models.

These parameters were used to create models to predict the mass of consumed solids (M_S) and liquids (M_L) using the methods of (Sazonov et al., 2009). A detailed description of the prediction models can be found in the Appendix. During model validation, the models created in the training stage were used to estimate the mass ingested in the validation meal, thus testing ability of the method to generalize energy intake predictions to foods and beverages different from those in the training meal.

The energy content of each food item was assessed by multiplying the mass estimated from the count of chews and swallows by caloric density which was extracted from the nutritional analysis by NDS-R. The total energy consumed in a meal was computed as a sum of the energy content of all food items in a meal (see Appendix for details). The same methodology was used to estimate the total energy intake for both training and validation meals.

Two additional models, the first relying only on counts of chews and the second relying only on counts of swallows, were created in a similar manner and evaluated to determine whether or not chews or swallows can be used independently to estimate energy intake.

Statistics

Sample size estimation was based on the results obtained in (Sazonov et al., 2009), which achieved 92% accuracy of mass estimation for solid food and 84% for liquids. Sample size estimation was performed using a Wilcoxon-Mann-Whitney two-sample rank-sum test to detect statistical significance in between-method reporting error. It was assumed that the probability that the reporting error in one group was higher or lower than another group is 0.756, corresponding to the effect size of approximately one common standard deviation of reporting error. Based on these assumptions, we estimated that a sample size of 20 subjects would ensure 80% power at 5% significance. In anticipation of subject dropout and possible failures of measurement equipment leading to exclusion of subjects, a sample size of 30 subjects was selected for this study.

The reporting error in estimated energy intake (EI) in a meal was calculated for each method with respect to weighed records (EI_WR) as follows:

$$\text{Reporting error}=|\frac{EI-{EI}_{WR}}{{EI}_{WR}}\times 100\%|.$$

Means and SDs of the reporting errors were calculated for subjects who completed the 4 visits and had weighed food record, diet diary, photographic food record, and CCS data available for all meals. For the training meals, the average reporting error was evaluated using the 3-fold cross-validation technique (Kohavi, 1995). Two of the training meals were used to train a CCS model and the third training meal was used to evaluate accuracy. This process was repeated 3 times, with each meal used once to evaluate the model. The results were averaged to produce a single estimation of the reporting error. For the validation meal, the CCS model was trained on the data from the three training meals and evaluated on the validation meal.

Reporting errors for CCS models, diet diaries, and photographic records were compared using a one-sided Wilcoxon-Mann-Whitney two-sample rank-sum test, which is preferred over Student’s t-test due to the potential presence of outliers in the reporting errors. Additionally, Bland-Altman analysis (Bland & Altman, 1999) accounting for the clustering effect of repeated measures was performed to assess the accuracy of the estimation methods for the training meals.

To evaluate the impact of physical properties of different foods on estimates of energy intake, means and SDs of MPChew and MPSw_S values were calculated for 7 representative food items and compared using two-sample t-tests.

In all cases, statistical differences were considered significant at p < 0.05. R software (Version 2.15.1, The R Foundation for Statistical Computing, Vienna, Austria) was used to perform Wilcoxon-Mann-Whitney tests, whereas Matlab (R2011b, MathWorks Inc, Natick, MA) was used to perform all t-test analyses. SAS 9.3 software (SAS Institute Inc, Cary, NC, USA) was used for Bland-Altman analysis.

Results

Two subjects were excluded from the 30 subjects participating in the study, because of missing sensor data caused by equipment failure. The remaining 28 subjects had a mean (±SD) age of 29 ± 12 y (range: 19–58 y) and a mean (±SD) BMI of 28.0 ± 5.6 (range: 20.5–41.7).

Reporting errors for all methodologies are presented in Table 3. For the training meals, the average reporting error of the CCS model (based on both chews and swallows) was significantly lower than that of diet diary (p < 0.01), but not significantly different than the average error of photographic records (p = 0.16). The Bland-Altman analysis indicated that the bias in energy estimation was positive (overestimated) for the photographic method, but negative (underestimated) for diet diary and the CCS model (Table 4). Additionally, energy intake estimation of the CCS model had the lowest bias and the narrowest limit of agreement with the gold standard, weighed food method (Figure 4). For the validation meal, the reporting error of the CCS model was not significantly different from either the diet diary or photographic food records.

Table 3.

Reporting errors (in %) for energy intake estimation for training and validation meals relative to energy intake assessed from the weighed records¹ (N=28). (Mean values and standard deviations)

Energy estimation methods	ENERGY
	Training meals		Validation meal
	Mean	SD	Mean	SD
CCS Models
Chews only	19.42	10.14	30.42	23.08
Swallows only	18.76a	10.35	34.27	31.86
Chews and Swallows	15.83a	9.41	32.23	24.84
Diet diary	27.86	29.67	25.69	21.90
Photographic food records	19.95	11.45	21.11	15.55

¹Differences between each CCS models, diet diary, and photographic food records were tested by using onesided Wilcoxon-Mann-Whitney two-sample rank-sum test.

^aSignificantly lower than diet diary, p < 0.05.

Table 4.

Results of the Bland-Altman analysis for the energy estimation (in kCal) for the training meals (N=28).

	Bias	SD of individual difference	Lower LOA	Higher LOA
Weighed vs. Diet diary	−60	367.1	−779.54	659.57
Weighed vs. Photographic records	83.6	230.6	−368.31	535.45
Weighed vs. CCS models	−8.6	186.2	−373.52	356.28

LOA: Limits of Agreement.

Figure 4.

Bland-Altman plots for the training meals. They address the accuracy of the dietary assessment methods evaluated in this study with respect to the weighed food records. CCS models (bottom) presented the narrowest limits of agreement when compared to diet diary (top) and photographic food records (middle).

For the training meals, the CCS model based only on counts of swallows showed average reporting error significantly lower than the diet diary (p = 0.04), but not significantly different from photographic food records (p = 0.42). For the validation meal, no significant differences were found between the CCS model and either diet diary or photographic records.

The CCS model based only on counts of chews showed reporting errors that were not significantly different than the diet diary and photographic methods for both the training and validation meals.

Differences in the MPChew and in the MPSw_S values between 7 representative food items are shown in Table 5. The results indicate the presence of significant differences in the MPChew values between all food types (p < 0.0001 in all cases) and significant differences in the MPSw_S values between most food types (except pizza-pasta and pizza-cookies pairs), which may be attributed to differences in the food densities.

Table 5.

Mass per chew (MPChew in g/chew) and mass per swallows of solid (MPSw_S in g/swallow) for most representative food items selected in this study.¹ Consumption of dry and crispy foods (i.e. potato chips and cookies) required significantly more chewing than soft and watery foods (i.e. pasta and yogurt). (Mean values and standard deviations)

	MPChew		MPSwS
Variable	Mean	SD	Mean	SD
Chips	0.09a	0.05	1.14	0.57
Cookies	0.40a	0.18	5.68c	3.99
Salad	0.32a	0.21	3.98	2.76
Pizza	0.51a	0.38	6.11b,c	3.88
Sub	0.63a	0.22	8.24	3.68
Pasta	1.41a	0.96	6.44b	2.33
Yogurt	2.63a	2.53	7.13	3.42

¹Two samples t-test was performed to analyze differences in each parameter among different foods.

^aSignificant differences were found among all food items (p < 0.0001 in all cases).

^b,cNo significant differences were found for food items sharing the same letter. The remaining combinations of food items showed significant differences in MPSw (p < 0.05).

Discussion

Estimation of energy intake from the individually-calibrated mathematical models relying on counts of chews and swallows may offer a promising alternative to overcome the limitations of self-report. In this study, a novel CCS method was implemented that included no self-report from the subject.

Results of cross-validation on the training meals show that CCS models were able to capture individual responses to consumption of identical meals, estimating the energy consumed in a meal significantly better than diaries and with a lower bias than both diet diary and photographic food record methods (Table 4). A further comparison of these methods by the Bland-Altman plot analysis demonstrated that the CCS was the best method for energy intake estimation (Figure 4). These results concur with the results from our previous study (Sazonov et al., 2009). However, the current study covered a much wider variety of foods (45 vs. 5) and assumed no restriction in the way the food was consumed, thus presenting a more realistic scenario of food intake.

Evaluation of CCS models on the validation meal demonstrated satisfactory performance for most of the subjects, although the prediction errors were not significantly different from either the diet diary or the photographic food record methods. One explanation is that in the present study diet diary records appeared to be highly accurate, with one third of subjects achieving errors lower than 10%, whereas the expected range of error is 35%-50% (Lichtman et al., 1992; Suchanek, Poledne, & Hubacek, 2011). Therefore, it appears that under the conditions used in this study, the accuracy of the diet diary method was high and that CCS models matched this high level of accuracy. In addition, unlike the diet diary, where reporting error increases with the duration of the recording period (A. H. C. Goris, Meijer, & Westerterp, 2001), it could reasonably be expected that CCS models would not show major changes over long periods of time as they do not rely on subject participation and motivation. The lower reporting bias observed over the three training meals with the CCS model supports this assumption.

The increase in reporting error observed for the validation meal suggests that CCS model performance is affected by the differences in physical properties of the food items (i.e. hardness, moisture, density, tackiness, etc.) consumed during the training and the validation meals (Table 5). These differences negatively affected the mass estimation of the solids and liquids consumed, and, in turn, impacted the accuracy of energy intake estimates. A potential solution to improve the performance of the models may be to adjust the MPChew and/or MPSw parameters according to the consumed food. The ratios of these parameters for different food types can be used as a correction factor for energy estimation. As an example, the average prediction error for a subject consuming salad (training meal) and pasta (validation meal) was reduced from 29.4% to 11.9% when applying such a correction factor. Another possible approach could be to compute separate MPChew and MPSw parameters for groups of foods with similar physical properties.

In this study, three training meals were used to build the mathematical models. The rationale for selecting three meals was based on several previous studies that established that at least three days of recording are necessary to obtain a realistic estimate of spontaneous energy intake in free-living subjects (Nelson, Black, Morris, & Cole, 1989). Future studies will be focused on extending the evaluation of model performance to free-living conditions.

One of the main benefits of the proposed methodology is the potential to implement the sensor system as a minimal burden wearable device. The sensor burden evaluated by a survey at the completion of the study indicated that chewing and swallowing sensors did not significantly affect the way subjects consumed their meals (J. M. Fontana & Sazonov, 2013) suggesting that the recording burden can be significantly attenuated.

Although a substantial amount of resources (equipment and personnel) was required to perform the experiments of this study, the demand for resources will be substantially reduced when the wearable sensor system is implemented as a self-administered device with support for automatic food photography. Integrating a miniature camera into the wearable device (Liu et al., 2012) and triggering the camera from the jaw motion or swallowing sensors will allow to capture images of the food being consumed and use these images for estimation of caloric density. The development of such wearable device is currently underway (J. M. Fontana et al., 2014).

The insight earned and the limitations encountered in this study will be taken into consideration in designing new studies. In particular, the ability of the wearable sensors to obtain clear chewing and swallowing counts under free-living circumstances needs to be evaluated. Moreover, the changes in the eating behavior of free living individuals being monitored by the sensors (observation effect) need to be quantified.

Conclusions

The results of this study indicate that models for estimation of energy intake based on the counts of chews and swallows may present an appealing alternative to self-report. Further technology development and human studies are needed to evaluate applicability of the method to free-living individuals and accuracy of energy intake measurement under free living conditions.

Highlights.

• Energy intake estimates were obtained for three identical meals and a different meal

• Suitability of models based on Counts of Chews and Swallows (CCS) was evaluated

• Compared to traditional methods, CCS models showed the lowest bias for repeated meals

• Food properties may affect the performance of CCS models

• Energy intake estimation by CCS does not require user input

Appendix

The following models were used to estimate the amount of mass and energy ingested by individuals based of the counts of chews and swallows.

The total mass ingested (M_T) by one subject during the entire meal was calculated as:

$$M_T=M_S+M_L$$ (1)

where M_S is the mass of solid food ingested and M_L is the mass of liquids ingested in the meal.

Mass of solids (MS) was predicted as:

$$\begin{array}{ll}{M}_s\hfill & =\text{Mass from count of swallows + Mass from counts of chews}\hfill \\ M_s\hfill & =w_s\cdot {MPSW}_S\cdot N_{sw}^s+w_c\cdot (MPChew\cdot c_f)\cdot N_{\mathit{\text{chew}}}\hfill \end{array}$$ (2)

where:

• w_s ∈ {0,0.5,1} = weight parameter for mass prediction using number of swallows;

• w_c ∈ {0,0.5,1} = weight parameter for mass prediction using number of chews;

• MPSw_S = subject's average mass per swallow of solid food;

• MPChew = subject's average mass per chew;

• $N_{sw}^s$ = total number of swallows for solid food intake;

• N_chew = total number of chews;

• c_f = correction factor;

The parameters w_s and w_c assigned a weight to the mass prediction using counts of swallows and counts of chews respectively. The sum w_s + w_c must be equal to 1 in all cases. If mass estimation was based only on counts of swallows, then w_s = 1 and w_c = 0. If mass estimation was based only on counts of chews, then w_s = 0 and w_c = 1.

For estimating the consumed mass in the validation meal, a correction factor c_f was used to modify the parameters the MPChew based on the number of chews per swallow (CPSw) observed in the training stage. The c_f was calculated as:

$$c_f=\frac{{CPSw}_{\mathit{\text{training}}}}{{CPSw}_{\mathit{\text{validation}}}}$$

where CPSw_training was the subject's average chews per swallow rate calculated from the training meals and CPSw_validation was the average number of chews per swallow for each food item in the validation meal.

Mass of liquids (M_L) was predicted as:

$$M_L={MPSw}_L\times N_{sw}^L$$ (3)

where:

• MPSw_L = subject's average mass per swallow of liquid

• $N_{sw}^L$ = total number of swallows for liquid intake;

Finally, the total energy intake (EI) was calculated as:

$$EI={\sum }_i^N{m}_{T_i}\cdot {CD}_i$$ (4)

where m_T is the consumed mass for the distinct food type i as calculated using equation (2) or (3) and CD_i is the caloric density associated to the same food type i. N is the total number of distinct foods types consumed in the meal.