Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2015 Jan 1.
Published in final edited form as: J Hum Nutr Diet. 2013 Jul 15;27(0 1):72–81. doi: 10.1111/jhn.12014

Measuring food intake with digital photography

Corby K Martin 1, Theresa Nicklas 2, Bahadir Gunturk 3, John B Correa 1, H Raymond Allen 1, Catherine Champagne 1
PMCID: PMC4138603  NIHMSID: NIHMS614677  PMID: 23848588

Abstract

The Digital Photography of Foods Method accurately estimates the food intake of adults and children in cafeterias. When using this method, imags of food selection and leftovers are quickly captured in the cafeteria. These images are later compared to images of “standard” portions of food using a computer application. The amount of food selected and discarded is estimated based upon this comparison, and the application automatically calculates energy and nutrient intake. Herein, we describe this method, as well as a related method called the Remote Food Photography Method (RFPM), which relies on Smartphones to estimate food intake in near real-time in free-living conditions. When using the RFPM, participants capture images of food selection and leftovers using a Smartphone and these images are wirelessly transmitted in near real-time to a server for analysis. Because data are transferred and analyzed in near real-time, the RFPM provides a platform for participants to quickly receive feedback about their food intake behavior and to receive dietary recommendations to achieve weight loss and health promotion goals. The reliability and validity of measuring food intake with the RFPM in adults and children will also be reviewed. The body of research reviewed herein demonstrates that digital imaging accurately estimates food intake in many environments and it has many advantages over other methods, including reduced participant burden, elimination of the need for participants to estimate portion size, and incorporation of computer automation to improve the accuracy, efficiency, and the cost-effectiveness of the method.

Keywords: dietary assessment digital photography, energy intake, free-living, food intake, food photography

Introduction

Accurately measuring food intake in cafeteria and free-living conditions poses methodological and analytical challenges. Commonly used methods include self-reported food intake (e.g., food records, 24-hour dietary recall, and food frequency questionnaires), though these methods are associated with limitations that have been previously outlined (Goris et al., 2000, Livingstone et al., 2004, Zegman, 1984). The availability and miniaturization of digital cameras and wireless communication devices (e.g., Smartphones) has led to the development of methods to quantify food intake that uses images of food selection and plate waste (Martin et al., 2012, Martin et al., 2009a, Nicklas et al., in press, Six et al., 2010). The purpose of this report is to describe recently validated methods of food intake estimation that utilize digital images and to outline how these methods have been adapted to estimate food intake in a number of environments, including free-living conditions, Head Start settings, and children’s homes. The studies described herein were approved by the respective Institutional Review Boards of the institutions where the research was conducted (the Pennington Biomedical Research Center or Baylor College of Medicine). Informed consent was obtained from all study volunteers prior to the initiation of any procedures.

The Digital Photography of Foods Method: Measurement of food intake in cafeteria settings

Description of the Digital Photography of Foods Method

The Digital Photography of Foods Method allows researchers and clinicians to quickly and unobtrusively estimate food intake in cafeteria settings (Williamson et al., 2004, Williamson et al., 2003, Williamson et al., 2002). When using this method, digital video cameras in the dining location are used to capture images of peoples’ food selection, as well as any food that remains after the meal (leftovers). Images of precisely weighed standard portions of the foods served are also collected, and these food images are linked to the Food and Nutrient Database for Dietary Studies 3.0 (2008), or a custom recipe, which allows energy and nutrient intake to be calculated. At a later date in the laboratory, trained raters use a custom-built computer program called the Food Photography Application© to simultaneously display images of the person’s food selection, leftovers, and the standard portion for each food consumed. The rater estimates the percent of the standard portion that the person selected, as well as the percent of the standard portion that remains on the plate (leftovers). The Food Photography Application© then automatically calculates the energy and nutrient content of food selection and leftovers (food intake is calculated as the difference between selection and leftovers). The Food Photography Application© calculates energy and nutrient content from reference values in the Food and Nutrient Database for Dietary Studies 3.0 (2008), the manufacturer’s nutrient information, or a custom recipe (the Food Photography Application© utilizes updated nutrient databases as they are released). Figure 1 includes images of a participant’s food selection (Image A), leftovers (Image B), and the standard portion image (Image C). Data collection is unobtrusive and data analysis is efficient, allowing for the collection and analysis of large volumes of data.

Figure 1.

Figure 1

Open in a new tab

Food intake is estimated with the Food Photography Application© by comparing images of participants’ food selection (A) and leftovers (B) to an image containing a known amount of food (C) to estimate portion size. The application then automatically calculates the energy and nutrient content of food selection, leftovers, and food intake.

Validity of the Digital Photography of Foods Method

The Digital Photography of Foods Method has been used in a number of settings and its reliability and validity (accuracy) has been established in adults (Williamson et al., 2004, Williamson et al., 2003, Williamson et al., 2002) and children (Martin et al., 2007, Nicklas et al., in press). Portion size estimates for food intake correlate highly with weighed portion sizes (r = 0.92, p < 0.0001) (Williamson et al., 2003) and the mean difference between directly weighed total food intake and digital imaging estimates is 5.2 ± 0.95 g (mean ± SEM), with no systematic bias over levels of food intake (Williamson et al., 2003). Among categories of foods, the mean error from digital imaging varies; for example, digital imaging overestimates beverage intake by 7.6 ± 3.07 g (mean ± SEM) or 4.3%, though digital imaging overestimates condiment intake by 4.9 ± 1.63 g (mean ± SEM) or 16.6% (Williamson et al., 2003). Agreement is consistently high among the raters who evaluate the images. The intra-class correlation coefficient for energy intake was 0.92 (95% confidence intervals or CI: 0.79, 0.97, p < .0001) in Martin et al. (2009a) and 0.93 in Martin et al. (2007), indicating that multiple raters can use the method to estimate food intake and that estimates do not vary meaningfully between and among raters who evaluate images. This is important when the method is used to estimate the food intake of large study samples when multiple raters are necessary. In previous studies, raters included Registered Dietitians, research associates with college degrees, and student workers who were attending college. Hence, it appears that individuals with at least some college experience can be trained to accurately estimate food intake using the method.

The Digital Photography of Foods Method has proven effective at estimating the food intake of large numbers of people sampled from a variety of geographical areas. For example, the method has been used to estimate the food intake of: 1) Soldiers (N =139) during basic combat training (Williamson et al., 2002), 2) elementary school children (N = 670) in a 2-year study (Williamson et al., 2007), and 3) children (N = 2060) from more than 30 schools across the state of Louisiana in a 3-year study (Williamson et al., 2008). In the Williamson et al. 2007 2008 studies, food intake was estimated at lunch for 3 days on 3 separate occasions. The semi-automated Food Photography Application© streamlined data management and analysis, making the analysis of large numbers of food images efficient and feasible. Additionally, the method provides rich data on food selection, leftovers, and food intake. The utility of these data has been demonstrated in a study that examined the effects of second servings on food intake (Martin et al., 2007) and another study comparing children’s food selection and food intake to national dietary guidelines (Martin et al., 2010, Nicklas et al., submitted for publication-a, Nicklas et al., submitted for publication-b).

The Digital Photography of Foods Method has also been adapted and used to estimate the food intake of preschool-age children enrolled in Head Start using digital cameras. Specifically, the method was used to collect food intake data at lunch for preschool children (N = 796) on 3 separate occasions across 16 Head Start centers (Nicklas et al., submitted for publication-a). Similar procedures were used by study personnel in the homes of 231 preschool children to capture food intake data during dinner meals on two separate occasions (Nicklas et al., submitted for publication-b). In these settings, the method was found to be valid through comparison of the estimated gram weights of foods from the method to directly weighed foods (Nicklas et al., in press). The average correlation between estimated weights and actual weights was 0.96 (p < 0.0001). The average mean difference in weight between the estimated and actual amounts was 10.6 grams (mean ± SD for weighed food = 255.4 ± 29.9 g vs. 244.8 ± 28.9 g for estimates from digital images) and the difference in energy was 25 kilocalories. Dependent t-tests indicated that the estimated weights using digital images did not differ significantly from the actual weight and Bland and Altman (Bland and Altman, 1986) analysis indicted that no systematic or magnitude bias was present. The estimated mean weight of all foods was 5% less the actual mean weight. Across food categories, mean error ranged from −9.7% for entrees (mean ± SD for weighed food = 80.6 ± 19.6 g vs. 72.8 ± 16.8 g for estimates from digital images) to 6.2% for vegetables (mean ± SD for weighed food = 48.2 ± 17.3 g vs. 51.2 ± 21.2 g for estimates from digital images). The standard deviations were not consistently larger or smaller for either directly weighed foods or the estimates obtained from digital images.. These projects with a program such as Head Start further illustrate the adaptability of the method for use with different populations and environments, as well as the ability of researchers and clinicians to utilize the method independently or in collaboration with groups who have previous experience with the method.

The Remote Food Photography Method: Measurement of food intake in free-living conditions

Description of the Remote Food Photography Method

The Remote Food Photography Method (RFPM) estimates the food intake of people in free-living conditions by asking them to capture images of food selection and leftovers with a cell or Smartphone (Martin et al., 2012, Martin et al., 2009a). Figure 2. Participants are trained to label images of foods that are not easily recognizable with a brief description (e.g., fried chicken). The images are then sent in near real-time through a wireless network to the Food Photography Application© where they are analyzed to estimate food intake using methods that have been previously described (Martin et al., 2012, Martin et al., 2009a, Williamson et al., 2004, Williamson et al., 2003). Similar to analysis of food images with the Digital Photography of Foods Method, images from participants’ phones are compared to images of foods with a known portion size to estimate food intake. In the case of the RFPM, images of foods with known portion sizes are stored in a searchable database called the “archive.” The archive consists of several thousand images of standard portions of foods and it provides the following information that is needed to estimate food intake based on participants’ food images: 1) a standard portion image for portion size estimation, and 2) a link to the nutrient information for each food, which is obtained from either the Food and Nutrient Database for Dietary Studies 3.0 (2008), the manufacturer’s nutrient information, or a custom recipe.

Figure 2.

Figure 2

Open in a new tab

When using the RFPM, participants use a Smartphone to capture images of their food selection, leftovers, and a reference card. These images are then immediately sent to a server for analysis. Reprinted from Martin et al. (2012).

The RFPM can be used easily by people to capture food intake information over long periods of time; therefore, semi-automated procedures were developed that rely on both human operators and computer automation to expedite data analysis. This semi-automated approach is adaptable, which allows new technology to be integrated into the procedures and the Food Photography Application© as the Food Photography Application© is further developed and updated. The Food Photography Application© remains the central location for data storage, management, and analysis; therefore, the semi-automated approach can be applied to data collected in both cafeteria settings with the Digital Photography of Foods Method and in free-living conditions with the RFPM. Descriptions of semi-automated techniques are described in more detail in (Gunturk, 2011, Martin et al., 2009b, Zhang and Gunturk, 2008a, Zhang and Gunturk, 2008b, Zhang and Gunturk, 2009). Briefly, when food images are captured a black and white reference card, which is the size of a credit card, is included in the image (Figure 2). This card allows the computer application to correct and standardize images for color and perspective, allowing participants to eat food from any size plate. Computer imaging algorithms can also be employed to automatically identify foods and estimate portion size (Martin et al., 2009b). A Smartphone-based application (i.e., an “app”) has been developed that facilitates the automatic identification of foods via bar code scanning and PLU (Price Look Up) codes. The app also includes a voice message feature that allows participants to quickly use the Smartphone to record a description of the food. The app packages these data (e.g., the food image and the identity of the food from its barcode, PLU number, or the participant’s voice message) and transmits the data package to the Food Photography Application© for estimation of food selection and leftovers, with food intake being calculated as the difference between these two values, as described earlier.

The RFPM is an innovative approach to food intake assessment, but it is susceptible to certain weaknesses, including people forgetting to capture food images or losing the Smartphone. To address these concerns, ecological momentary assessment (EMA) methods (Stone, 1994) were integrated into the RFPM to reduce the occurrence of missing data and to foster data integrity. EMA methods utilize communication technology to measure behavioral phenomenon (e.g., tobacco use, binge eating) in real-time in natural settings, which results in ecologically valid data (Stone, 1994). The RFPM incorporated EMA methods by sending prompts (emails, text messages) to peoples’ Smartphones to remind them to capture food images. These prompts are scheduled for automatic delivery through the Food Photography Application© based on individual participant’s meal times, and the timing and frequency of the prompts can be modified. An example of a prompt is “Did you eat or drink anything today and forget to take a picture?”, and participants must reply to each prompt by answering “yes” or “no” with the push of a few buttons or they can type a detailed response. The Food Photography Application© tracks the delivery and response to each prompt and sends reports to the study team summarizing these data, as well as the number of food images received. The study team can easily detect missing data since data transfer occurs in near real-time. Figure 3 includes an illustration of the data collection and transfer process. If a participant is not able to capture images of his/her foods, a back-up method (e.g., food record) is used.

Figure 3.

Figure 3

Open in a new tab

The RFPM uses Ecological Momentary Assessment (EMA) methods to improve data quality and minimize missing data. Prompts are automatically sent to participants’ Smartphones to remind them to capture images of their foods and to send these images to the research staff (the images are received by, and managed in, a computer program called the Food Photography Application©). The Food Photography Application© also stores responses to the prompts, and it sends automated reports to the research team and they can quickly identify when data acquisition problems occur. Reprinted from Martin et al. (2012).

The RFPM also has inherent limitations that are a function of the available technology that is used to improve the efficiency of data collection and analysis. For example, although we developed computer imaging algorithms to automatically identify foods and estimate portion size (Martin et al., 2009b), these functions are not yet sufficiently advanced to allow for fully automated data analysis. Rather, the process remains semi-automated and a human operator oversees the data collection, management, and analysis process. As new technology is released, however, it can be readily incorporated into the RFPM procedures and a good example of this is use of bar code scanning and PLU (Price Look Up) codes to automatically identify foods.

The validity of the Remote Food Photography Method for measuring the food intake of adults

The reliability and validity of the RFPM was first examined in a sample of adults (N = 50) who used the method over 3 days in both laboratory and free-living conditions (Martin et al., 2009a). During this study, energy intake estimated with the RFPM was compared to the energy intake from directly weighed foods and beverages. Weighed intake was obtained in free-living conditions by providing a cooler to participants that contained pre-weighed foods for their evening meals. They returned the cooler the following morning and the post-weight was obtained. The RFPM produced reliable energy intake estimates over three days in both laboratory (intraclass correlation coefficient or ICC = 0.62, p < 0.0001) and free-living (ICC = 0.68, p < 0.0001) conditions, and these values were consistent with natural variation in energy intake. Weighed energy intake correlated highly with estimates from the RFPM in both laboratory (r = 0.93, p < 0.0001) and free-living (r = 0.95, p < 0.0001) conditions. In the laboratory at lunch meal, the RFPM non-significantly (p = .08) underestimated mean (± SEM) energy intake by 36 ± 19 kcal (mean ± SEM energy intake from weighed foods = 655 ± 50 kcal vs. 619 ± 53 kcals from digital image estimates), which corresponds to mean underestimation of 5.5%. In free-living conditions compared to doubly labeled water, the RFPM significantly (p = .02) underestimated energy intake by 97 ± 38 kcal (mean ± SEM energy intake from weighed foods = 1461 ± 105 kcal/day vs. 1364 ± 115 kcal/day from digital image estimates), which corresponds to mean underestimation of 6.6%. Although the RFPM significantly underestimated energy intake in free-living conditions, the underestimate was 97 kcal/day, which is relatively small compared to the error associated with other methods. Additionally, Bland and Altman analyses (Bland and Altman, 1986) indicated that the error from the RFPM did not differ over energy intake levels, and regression analyses indicated that error did not vary by body weight or body mass index (BMI) (Martin et al., 2009a).

Two additional studies that are reported in (Martin et al., 2012) examined the reliability and validity of the RFPM at measuring the energy intake of adults over six days in free-living conditions and provided a much more stringent test of the method. In the first study, the rigor of EMA methods on the RFPM’s validity was examined. Participants were assigned to one of two EMA groups. The first group received Standard EMA Prompts (n = 24), which consisted of 2 to 3 prompts/day that were sent to the Smartphone around generic meal times (e.g., noon). If data acquisition problems occurred, participants were contacted within 1 to 2 days by study personnel. In the Customized EMA Prompt group (n = 16), participants received 3 to 4 prompts/day that were delivered at individual participants’ meal times, and study personnel contacted participants quickly (i.e., within 24 hours) if data acquisition problems occurred. All participants used the RFPM for six full days in their natural environment and energy intake was simultaneously measured with the doubly labeled water method. When compared to the doubly labeled water method, the RFPM significantly underestimated energy intake when Standard (mean ± SD = −895 ± 770 kcal/day, p < .0001), but not Customized EMA Prompts (mean ± SD = −270 ± 748 kcal/day, p = .22) were used. Error (energy intake estimated with the RFPM minus that from doubly labeled water) was significantly smaller with Customized vs. Standard Prompts and did not vary over levels of energy intake in either group. The error of −270 kcal/day in the Customized group reflects mean participant error of −8.8 ± 29.8% (Martin et al., 2012).

In the second study, a larger sample of adults (N = 50) used the RFPM for six days in their natural environment while their energy intake was also measured with the doubly labeled water method (Martin et al., 2012). All participants received Customized EMA prompts, as these procedures were incorporated into the RFPM after the first study. Compared to doubly labeled water the RFPM did not significantly over- or underestimate energy intake. Mean (± SD) participant error was −152 ± 694 kcal/day (p = 0.16) or −3.7 ± 28.7%, with energy intake from the doubly labeled water method being 2,360 ± 626 kcal/day (mean ± SD) and estimated energy intake from the RFPM being 2,208 ± 665 kcal/day. During this study, participants also used the RFPM during two laboratory-based buffet meals. These test meals provided a test of the RFPM’s ability to accurately estimate nutrient intake, which cannot be obtained from the doubly labeled water method. During the laboratory-based buffet meals, the RFPM’s error was only −4 ± 73 kcal (mean ± SD) and the error for the three macronutrients and 5 other nutrients was non-significant, though the RFPM significantly (p < .01) overestimated vitamin A and cholesterol (Figure 4). Bland and Altman (Bland and Altman, 1986) analyses were conducted for each nutrient and vitamin and none were significant. Importantly, the RFPM was not associated with undereating when used in free-living conditions, and in all comparisons Bland and Altman analyses indicated that the RFPM’s error did not differ significantly over levels of energy intake. Figure 5 illustrates the Bland and Altman for the comparison of the RFPM to energy intake measured with the doubly labeled water method in 50 adults where mean participant error was −152 ± 694 kcal/day, with no differences in error over levels of energy intake. Regression analyses also found that the RFPM’s error did not differ by age (Adj. R2 = 0.03, p = 0.29), or body weight (Adj. R2 = 0.03, p = 0.12) (Martin et al., 2012).

Figure 4.

Figure 4

Open in a new tab

Validity of the RFPM at measuring the energy intake of adults in free-living conditions and the energy and nutrient intake of adults during laboratory-based buffet meals. The gold standard comparison measures were: the doubly labeled water method for free-living conditions, and directly weighed foods during the buffet meals. Mean participant error is displayed and asterisks indicate that the RFPM estimate differed significantly from the gold standard (alpha = .05 for the comparison of free-living data and alpha = .01 for all other comparisons).

Figure 5.

Figure 5

Open in a new tab

Bland and Altman analysis comparing energy intake (EI) estimated with the Remote Food Photography Method (RFPM) to the gold standard-EI measured with doubly labeled water (DLW). The RFPM’s error was similar across levels of EI. Reprinted from Martin et al. (2012).

The validity of the Remote Food Photography Method at measuring the food intake of children

The validity of the RFPM at measuring the food intake of preschool-age children was evaluated during two collaborative pilot projects with Baylor College of Medicine and the Pennington Biomedical Research Center. In addition to testing the RFPM’s validity, these studies demonstrated that the RFPM can be used in multi-site trials, during which data collection and analysis is managed from a single site. Data were collected in the Houston, Texas area and the images were transferred wirelessly and in near real-time to the Food Photography Application© in Baton Rouge, Louisiana where they were analyzed to estimate food intake. This effort was facilitated by the development of a web-based interface that allows EMA prompt management to occur remotely.

During the first collaborative pilot validity study, 12 mothers in the Houston area were trained to use a Smartphone to capture images of foods consumed by their Head Start child in the home over a 24-hour period. Study personnel also used a Smartphone to capture images of the foods consumed by the same children at Head Start (HS) on the same day. Another team of study personnel weighed food selection and leftovers during the same 24-hour period at HS and children’s homes. Total 24-hour food intake estimated with the RFPM was compared to directly weighed food intake. As illustrated in Figure 6, the RFPM significantly overestimated food intake by a mean of 13%, though this mean difference was very small (9.9 g). Examination of the data identified foods that were difficult to analyze in this pilot study, and elimination of these foods decreased the percent error by 45% to ~7%. The problematic foods included canned fruit and vegetables with liquid, condiments, mixed meals, layered foods, beverages, and foods with inedible portions. We have since adapted our procedures to better collect and analyze data from these problematic foods. Also, during this pilot study, there was a significant correlation of 0.95 between the mean g weight of foods from the RFPM and mean actual g weight. The mean amount of energy served to the preschool children (1,434 kcal) is similar to the amount of energy consumed by children 2 to 5 years from the 2007-2008 NHANES data, whereas the actual amount of energy consumed by preschool children in the pilot study was 840 ± 297 (mean ± SD) kcal (weighed intake) to 976 ± 338 kcal (RFPM) (Table 1). These data suggest that the parental recalls of young children’s intake in NHANES reflect what was served to the children and not what was actually consumed; thus, this assumption needs to be tested in future studies. These preliminary data are consistent with a previous study (Fisher et al., 2008) where parental 24-hour recalls significantly overestimated young children’s energy intake. Preliminary pilot data on the mean energy and macronutrient intake between the RFPM and weighed intake was very similar (Table 1).

Figure 6.

Figure 6

Open in a new tab

Validity of the RFPM at measuring the food intake (g) of preschool-age children over 24-hours in the children’s natural environments, including their home and a Head Start center.

Table 1.

Mean Energy and Macronutrient Intake: Pilot Data Comparing RFPM with Weighed Food Intake

RFPM* mean SD CV% Weighed Intake Mean SD CV%
Energy (kcal) 976.4 338.3 34.7 Energy (kcal) 840.4 296.9 35.3
Protein (kcal) 161.9 64.9 40.1 Protein (kcal) 138.2 53.2 38.5
Carbohydrate
(kcal)
526.8
190.2
36.1		 Carbohydrate
(kcal)
451.1
173.5
38.5
Fat (kcal) 296.6 135.9 45.8 Fat (kcal) 258.4 119.7 46.3
Open in a new tab
*

12 HS Children

There was no significant difference between study personnel and mothers in the mean amount of food consumed in the home based on the analysis of food images (the difference was 48 g) for the preschool children. There also was no significant difference between the weighed intake of foods consumed in the home and the amount estimated from the mothers’ food images (difference was 32 g). There was a 35 g difference in total food intake consumed in the home between weighed intake and intake generated from the RFPM.

During the second collaborative pilot and feasibility study another set of 12 mothers used a Smartphone to capture images of foods consumed by their HS child in the home over a 24-hour period. During the same period, study personnel captured images of foods consumed in the HS center. Similar to the first pilot study, data were transmitted to, and analyzed at, the Pennington Center. After study completion, interviews were conducted with each mother to assess feasibility of the RFPM. There were no significant differences between mean 24-hour intake of these 12 children and the 12 children who participated in the pilot validity study. Results from the interviews with the mothers showed that none of the participants found the Smartphone difficult to utilize, with more than 80% reporting that the study was easy to very easy to complete. 92% reported they would not change anything about the procedures, the way in which they were trained to use the Smartphone, or the EMA prompts. All participants reported that it would not be difficult to capture food images with a Smartphone when eating out. 100% of the mothers reported they would definitely participate in the study again.

The two pilot studies demonstrated that the RFPM can be used to estimate the food intake of preschool-age children in HS and children’s homes. Further, the pilot studies demonstrated that the RFPM is adaptable to different environments and participants, and that data can be collected and transferred to a central server for analysis from remote locations.

Discussion

Methods that rely on digital images have been found to reliably and accurately estimate food intake. In cafeteria settings, the Digital Photography of Foods Method accurately estimates the food intake of adults (Williamson et al., 2004, Williamson et al., 2003, Williamson et al., 2002) and children (Martin et al., 2007, Nicklas et al., in press). Digital photography was also found to be a feasible method for collecting intake data on lunch and dinner meals consumed by HS preschool children (Nicklas et al., in press), though there is currently no tests of the method against the doubly labeled water in free-living conditions . In people’s natural environments, the Remote Food Photography Method (RFPM) accurately estimates the food intake of adults (Martin et al., 2012, Martin et al., 2009a) and the results from two pilot studies reported herein indicate that the RFPM accurately estimates the food intake of preschool-age children in HS and home environments. Importantly, data from the RFPM’s validity studies indicate that the method accurately estimates food intake at both the group and individual level, as indicated by the Bland and Altman analysis depicted in Figure 5. Finally, these methods provide objective data on food selection, leftovers, and food intake. Therefore, rich data are obtained about the energy and nutrient content of the foods selected, discarded, and consumed. Such data can be used to evaluate the effect of an intervention on multiple facets of eating behavior.

Approaches that rely on digital images have many strengths, including reduced participant burden and elimination of the need for participants to estimate portion size. These factors could contribute to user-satisfaction with these methods. For example, satisfaction with the RFPM was favorable among adults who used the method (Martin et al., 2012, Martin et al., 2009a) and among mothers who used the method to estimate the food intake of their children. The RFPM was designed to minimize participant burden and new technology is being developed to further reduce participant burden and improve efficiency and affordability. For example, a Smartphone-based “app” has been developed that improves the user experience and incorporates bar code scanning and Price Look Up codes to automatically identify foods. Lastly, the RFPM is similar in cost or less costly than self-report methods to estimate food intake, namely, pen-and-paper food records and 24-h recall.

The utility of data from methods that rely on digital images has been clearly established. For example, the effect of interventions or second servings on the food intake of elementary school children in school cafeterias has been demonstrated in a number of studies that relied on the Digital Photography of Foods Method (Martin et al., 2007, Williamson et al., 2012, Williamson et al., 2007) and the method has been used to quantify the food intake behavior of preschool-age children in HS and their homes (Nicklas et al., submitted for publication-b). Further, this method has been used to objectively determine if food provision and intake in school cafeterias (Martin et al., 2010) and HS Centers (Nicklas et al., submitted for publication-a) meets national guidelines, and this information can be used to inform policy decisions. The RFPM data has also proven useful, particularly since Smartphones are used to collect and transfer data, and data can be analyzed in near real-time. These features allow clinicians to quickly review patients’ data and to send data-driven feedback that can be delivered directly to patients’ Smartphone. Hence, the RFPM provides a foundation for patients to quickly receive feedback about their eating behavior, as well as to receive recommendations to modify their food intake to meet a goal, such as losing weight or eating a healthy diet. Importantly, almost half of Americans (46%) already own a Smartphone (Smith, Accessed April 18, 2012.), indicating that many people already own the data collection device. Further, the RFPM Smartphone app that was recently developed can be downloaded onto participants’ own iPhone or Android smartphone, which we have found participants prefer compared to being loaned a smartphone during the data collection process since they need to only carry one device.

Digital imaging methods are not without limitations, however. As noted earlier, people can forget to capture food images, lose the Smartphone, or experience technical problems that could impede collecting RFPM data. Participants are trained to use a back-up method, which is typically a food record, if they are not able to capture images of their foods, and a back-up method is utilized on 8.9% of days and makes up 9.7% of total energy intake estimates (Martin et al., 2012). This information highlights that the RFPM is an imperfect tool and that flexibility in its use, such as training participants to use a back-up method, can facilitate the collection of complete data. An additional limitation is the fact that current computer imaging algorithms are not sufficiently advanced to correctly identify foods and accurately estimate the amount of food in the images with 100% accuracy. Hence, the data analytic procedures remain semi-automated and a human operator supervises the process. The incorporation of bar code scanning and PLU (Price Look Up) codes to automatically identify foods has significantly improved the efficiently and user experience in identifying foods, however. Finally, certain types of foods, such as condiments, can be difficult to analyze with digital images, though these types of foods are typically difficult to estimate using other methods, as well.

In summary, methods that utilize digital images have proven effective at quantifying food intake in cafeteria settings and people’s natural environment. Approaches that rely on digital images have many advantages, including reduced participant burden and the ability to quantify food intake at the group and individual level. Work continues to further automate digital imaging approaches.

Acknowledgments

Sources of Funding:

This work was partially supported by National Institutes of Health grants R21 AG032231 and K23 DK068052 (PI: C. Martin). In addition, this work was partially supported by United States Department of Defense grant W81XWH-05-2-0082 and a NORC Center Grant # 1P30 DK072476 entitled “Nutritional Programming: Environmental and Molecular Interactions”, sponsored by the NIDDK. The contents of this publication do not necessarily reflect the views or policies of the USDA, nor does mention of trade names, commercial products, or organizations imply endorsement from the U.S. government. This research project was also partially supported by the USDA – Agricultural Research Service through specific cooperative agreement 58-6250-6-003. The archive of standard portion images was made possible through collaboration with Thomas Baranowski, Baylor College of Medicine, whose research is supported by grants and contracts with the National Cancer Institute.

Footnotes

The work reported in this report was undertaken at the Pennington Biomedical Research Center, 6400 Perkins Road, Baton Rouge, LA 70808; Baylor College of Medicine, 1100 Bates Avenue, Houston, TX 77030; and Louisiana State University, Department of Electrical and Computer Engineering, Baton Rouge, LA 70803.

Conflicts of Interest:

The authors report no Conflicts of Interest

References

  1. BLAND JM, ALTMAN DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986;1:307–10. [PubMed] [Google Scholar]
  2. FISHER JO, BUTTE NF, MENDOZA PM, WILSON TA, HODGES EA, REIDY KC, DEMING D. Overestimation of infant and toddler energy intake by 24-h recall compared with weighed food records. Am J Clin Nutr. 2008;88:407–15. doi: 10.1093/ajcn/88.2.407. [DOI] [PubMed] [Google Scholar]
  3. GORIS AH, WESTERTERP-PLANTENGA MS, WESTERTERP KR. Undereating and underrecording of habitual food intake in obese men: selective underreporting of fat intake. Am J Clin Nutr. 2000;71:130–4. doi: 10.1093/ajcn/71.1.130. [DOI] [PubMed] [Google Scholar]
  4. GUNTURK BK. Fast bilateral filter with arbitrary range and domain kernels. IEEE Transactions on Image Processing. 2011;20:2690–2696. doi: 10.1109/TIP.2011.2126585. [DOI] [PubMed] [Google Scholar]
  5. LIVINGSTONE MB, ROBSON PJ, WALLACE JM. Issues in dietary intake assessment of children and adolescents. Br J Nutr. 2004;92(Suppl 2):S213–22. doi: 10.1079/bjn20041169. [DOI] [PubMed] [Google Scholar]
  6. MARTIN CK, CORREA JB, HAN H, ALLEN HR, ROOD J, CHAMPAGNE CM, GUNTURK BK, BRAY GA. Validity of the Remote Food Photography Method (RFPM) for estimating energy and nutrient intake in near real-time. Obesity. 2012;20:891–899. doi: 10.1038/oby.2011.344. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. MARTIN CK, HAN H, COULON SM, ALLEN HR, CHAMPAGNE CM, ANTON SD. A novel method to remotely measure food intake of free-living individuals in real time: the remote food photography method. The British journal of nutrition. 2009a;101:446–56. doi: 10.1017/S0007114508027438. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. MARTIN CK, KAYA S, GUNTURK BK. Quantification of food intake using food image analysis. IEEE Engineering in Medicine and Biology Magazine. 2009b;1:6869–6872. doi: 10.1109/IEMBS.2009.5333123. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. MARTIN CK, NEWTON RL, JR., ANTON SD, ALLEN HR, ALFONSO A, HAN H, STEWART T, SOTHERN M, WILLIAMSON DA. Measurement of children's food intake with digital photography and the effects of second servings upon food intake. Eating behaviors. 2007;8:148–56. doi: 10.1016/j.eatbeh.2006.03.003. [DOI] [PubMed] [Google Scholar]
  10. MARTIN CK, THOMSON JL, LEBLANC MM, STEWART TM, NEWTON RL, JR., HAN H, SAMPLE A, CHAMPAGNE CM, WILLIAMSON DA. Children in school cafeterias select foods containing more saturated fat and energy than the Institute of Medicine recommendations. The Journal of Nutrition. 2010;140:1653–60. doi: 10.3945/jn.109.119131. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. NICKLAS TA, O'NEIL C, STUFF J, LIU Y, FISHER J. Differential serving size selection and food consumed by preschool children in Head Start. submitted for publication-a. [Google Scholar]
  12. NICKLAS TA, O'NEIL C, STUFF J, LIU Y, HUGHES S. Characterizing dinner meals served and consumed by low-income preschool children. doi: 10.1089/chi.2011.0114. submitted for publication-b. [DOI] [PubMed] [Google Scholar]
  13. NICKLAS TA, O'NEIL CE, STUFF J, GOODELL LS, LIU Y, MARTIN CK. Validity and feasibility of a digital diet estimation method for use with preschool children: A pilot study. Journal of Nutrition Education and Behavior. doi: 10.1016/j.jneb.2011.12.001. in press. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. SIX BL, SCHAP TE, ZHU FM, MARIAPPAN A, BOSCH M, DELP EJ, EBERT DS, KERR DA, BOUSHEY CJ. Evidence-based development of a mobile telephone food record. J Am Diet Assoc. 2010;110:74–9. doi: 10.1016/j.jada.2009.10.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. SMITH A. 46% of American adults are smartphone owners. Pew Internet & American Life Project. 2012 Apr 18; Accessed. [Google Scholar]
  16. STONE AA, SHIFFMAN S. Ecological Momentary Assessment (EMA) in behavioral medicine. Ann Behav Med. 1994;16 [Google Scholar]
  17. USDA Food and Nutrient Database for Dietary Studies, 3.0. Agricultural Research Service, Food Surveys Research Group; Beltsville, MD: 2008. [Google Scholar]
  18. WILLIAMSON DA, ALLEN HR, MARTIN PD, ALFONSO A, GERALD B, HUNT A. Digital photography: a new method for estimating food intake in cafeteria settings. Eat Weight Disord. 2004;9:24–8. doi: 10.1007/BF03325041. [DOI] [PubMed] [Google Scholar]
  19. WILLIAMSON DA, ALLEN HR, MARTIN PD, ALFONSO AJ, GERALD B, HUNT A. Comparison of digital photography to weighed and visual estimation of portion sizes. J Am Diet Assoc. 2003;103:1139–45. doi: 10.1016/s0002-8223(03)00974-x. [DOI] [PubMed] [Google Scholar]
  20. WILLIAMSON DA, CHAMPAGNE CM, HARSHA D, HAN H, MARTIN CK, NEWTON R, JR., STEWART TM, RYAN DH. Louisiana (LA) Health: design and methods for a childhood obesity prevention program in rural schools. Contemp Clin Trials. 2008;29:783–95. doi: 10.1016/j.cct.2008.03.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. WILLIAMSON DA, CHAMPAGNE CM, HARSHA D, HAN H, MARTIN CK, NEWTON RL, SOTHERN M, STEWART TM, WEBBER LS, RYAN D. Effect of an environmental school-based obesity prevention program on changes in body fat and body weight: A randomized trial. Obesity. 2012 doi: 10.1038/oby.2012.60. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. WILLIAMSON DA, COPELAND AL, ANTON SD, CHAMPAGNE C, HAN H, LEWIS L, MARTIN C, NEWTON RL, JR., SOTHERN M, STEWART T, RYAN D. Wise Mind project: a school-based environmental approach for preventing weight gain in children. Obesity. 2007;15:906–17. doi: 10.1038/oby.2007.597. [DOI] [PubMed] [Google Scholar]
  23. WILLIAMSON DA, MARTIN PD, ALLEN HR, MOST MM, ALFONSO AJ, THOMAS V, RYAN DH. Changes in food intake and body weight associated with basic combat training. Mil Med. 2002;167:248–53. [PubMed] [Google Scholar]
  24. ZEGMAN MA. Errors in food recording and calorie estimation: Clinical and theoretical implications for obesity. Addict Behav. 1984;9:347–50. doi: 10.1016/0306-4603(84)90033-9. [DOI] [PubMed] [Google Scholar]
  25. ZHANG M, GUNTURK BK. Multiresolution bilateral filtering for image denoising. IEEE Trans Image Process. 2008a;17:2324–33. doi: 10.1109/TIP.2008.2006658. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. ZHANG M, GUNTURK BK. A new image denoising framework based on bilateral filter. Proceedings, SPIE Int. Conf., Visual Communications and Image Processing. 2008b;6822:68221B–8. doi: 10.1117/12.768101. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. ZHANG M, GUNTURK BK. Compression artifact reduction with adaptive bilateral filtering. Visual Communications and Image Processing. 2009;7257:72571A–11. [Google Scholar]

RESOURCES