A novel method to remotely measure food intake of free-living people in real-time

Abstract

The aim of this study was to report the first reliability and validity tests of the Remote Food Photography Method (RFPM), which consists of camera-enabled cell phones with data transfer capability. Participants take and transmit photographs of food selection and plate waste to researchers/clinicians for analysis. Following two pilot studies, adult participants (N=52, 20≤BMI≤35) were randomly assigned to the dine-in or take-out group. Energy intake (EI) was measured for three days. The dine-in group ate lunch and dinner in the laboratory. The take-out group ate lunch in the laboratory and dinner in free-living conditions (participants received a cooler with pre-weighed food that they returned the following morning). Energy intake was measured with the RFPM and by directly weighing foods. The RFPM was tested in laboratory and free-living conditions. Reliability was tested over three days and validity was tested by comparing directly weighed EI to EI estimated with the RFPM using Bland-Altman analysis. The RFPM produced reliable EI estimates over three days in laboratory (r=.62, p<.0001) and free-living (r=.68, p<.0001) conditions. Weighed EI correlated highly with EI estimated with the RFPM in laboratory and free-living conditions (r’s>.93, p<.0001). In two laboratory-based validity tests, the RFPM underestimated EI by -4.7% (p=.046) and -5.5% (p=.076). In free-living conditions, the RFPM underestimated EI by -6.6% (p=.017). Bias did not differ by body weight or age. The RFPM is a promising new method for accurately measuring the EI of free-living people. Error associated with the method is small compared to self-report methods.

Introduction

The gold standard for measuring food or energy intake (EI) in free-living humans is the doubly labeled water (DLW) method. Doubly labeled water provides an accurate measure of total daily energy expenditure (TEE) and, during a period of energy balance, TEE is equal to EI ^{1, 2}. When a large energy deficit is present during the DLW period, however, it is difficult to obtain an accurate (valid) estimate of an individual’s short-term EI using DLW, even if change in energy stores is considered ³. This limitation is noteworthy, since researchers and clinicians frequently require an estimate of EI during diets or periods of calorie restriction. Additional limitations of the DLW method include: 1) cost, 2) availability, and 3) its inability to provide important information about the type and micro-and macronutrient composition of foods ingested. Nevertheless, seemingly few valid and reliable alternatives for estimating EI are available.

Self-report methods are frequently used to collect EI data, including 24-hour food recall and pen-and-paper food records. When estimating EI with 24-hour food recall, a trained individual interviews the participant about his/her food and beverage consumption over the previous 24-hours. This method relies on the ability of the participant to accurately recall the types and amounts of foods consumed during the previous 24-hours, and it assumes that these foods are representative of habitual EI. Consequently, this method is subject to error ^4-6. For example, EI estimated with 24-hour recall in New Zealand’s National Nutrition Survey resulted in significant underestimation of EI, particularly among women and obese participants ⁵, and 24-hour recall with multiple pass methodology resulted in significant underestimation of EI in a sample of African-American women diagnosed with type II diabetes ⁶. Efforts to improve the accuracy of 24-hour recall have been disappointing. For example, financial incentives were not found to improve diet recall ⁷.

Food records are another self-report method that is frequently used to estimate food intake. When using food records, participants estimate the portion sizes of foods that they eat and record this portion size and a description of the food on the food record, which is typically pen-and-paper. The accuracy of this method has been questioned ^{8, 9}. Tightly controlled studies that compared self-reported EI to EI measured with DLW indicate that people significantly underreport their EI when using food records ^{10, 11}. Moreover, overweight and obese people tend to underreport EI to a greater degree than lean people ¹⁰, and people tend to selectively underreport fat intake ¹². Different methods have been used to try to improve the accuracy of food records, but the validity of these methods remains questionable. For example, an individualized approach to food records that considered variability in EI and estimated energy requirements resulted in underestimates of EI of 27% or more ¹³, and a method that included visual aids to improve portion size estimates underestimated EI by 14% with marked variability ¹⁴.

Software applications housed on hand-held devices such as Personal Digital Assistants (PDAs) have been developed to reduce the burden of keeping a food record and to improve their accuracy. Nevertheless, food records kept on PDAs appear no more accurate than 24-hour recall ¹⁵ or traditional food records ¹⁶ and do not improve the accuracy of self-report EI ¹⁷. Importantly, Beasley et al. ¹⁵ noted that the largest source of error in estimated EI resulted from participants’ poor estimation of portion size. People have difficulty estimating portion size even after training and these errors negatively affect EI estimates ¹⁸. Our laboratory found that participants more accurately estimated portion size after extensive training, though a large degree of variability in portion size estimates remained ¹⁹. Together, these findings indicate that methods are needed that do not rely on the participant to estimate portion sizes.

Methods to measure EI that do not rely on the participant to estimate portion size are limited. Direct observation of food selection and plate waste by trained observers results in accurate EI estimates ^{20, 21}, but this method is not applicable to free-living conditions. The digital photography of foods method ^{22, 23}, however, could be modified for free-living conditions. When using this method, the plate of foods selected by an individual is photographed and the individual’s plate waste is photographed following the meal. Reference or standard portions of known quantities of the foods are also photographed. In the laboratory, trained registered dietitians (RDs) use these photographs to estimate the portion size of food selection and plate waste by comparing these photographs to the standard portion photographs. These estimates are entered into a computer application that calculates the grams, kilojoules (kJ) or kilocalories (kcal), and macro- and micro-nutrients of food selection, plate waste, and food/energy intake based on a United States Department of Agriculture (USDA) database ²⁴. The digital photography method has been found to be highly reliable and accurate when used to measure the EI of adults ^{22, 23} and children ²⁵ in cafeteria settings. Portion size estimates from digital photography correlate highly with weighed portion sizes (r’s > .90, p’s < .0001) and mean differences between directly weighing foods and digital photography are minimal (< 6 g) ²³. Agreement is high among trained RDs who estimate portion sizes using digital photography, e.g., intraclass correlation coefficients (ICCs) are consistently > .90 ²⁵.

To our knowledge, only a few studies have used digital photographs to estimate EI in free-living conditions. A research group in Japan used a PDA with a digital camera to collect EI data in free-living conditions. The method results in EI estimates comparable to weighed food records. Nevertheless, these studies relied on samples of young lean college women who were majoring in food and nutrition, and the criterion (food records) are prone to error ^{26, 27}. When used in a more diverse sample of males and females, the method resulted in EI estimates that were significantly lower than estimates from weighed records, and underreporting in males was associated with obesity ²⁸. Nevertheless, these important studies demonstrate the likely utility of such technology.

The purpose of this paper is to describe the development of the Remote Food Photography Method (RFPM) and to report the first reliability and validity tests of this method. The RFPM relies on the validated digital photography method ^{22, 23} to analyze food photographs and estimate EI, but the photographs are collected in free-living conditions with a camera-enabled cell phone and are sent to the researchers in near real-time via a cellular network. Two pilot studies and the main study, which includes tests of reliability and validity, are described. It was hypothesized that initial reliability and validity data would suggest that the RFPM is a viable method for measuring EI in free-living conditions. The three studies reported herein were conducted at the Pennington Biomedical Research Center (PBRC), Baton Rouge, LA, USA. All applicable institutional and government regulations concerning the ethical use of human volunteers were followed. All participants provided written informed consent and the research was approved by the Institutional Review Board of the PBRC.

Pilot Study 1

The purpose of Pilot Study 1 was to overcome the primary limitation of adapting the digital photography method for free-living humans, which is the need for a photograph of a weighed standard portion from the dining location. When the digital photography method is utilized in the cafeteria, a photograph is taken of a standard portion of weighed food. In the laboratory, trained RDs estimate EI by comparing the photographs of food selection and plate waste to the standard portion photograph. When photographs are collected in free-living conditions, it is unfeasible for participants to weigh and take a photograph of a standard portion of each food that they eat.

Procedures

To overcome this limitation, we created a large database of standard portion photographs from previous studies and we took new standard portion photographs of foods that are frequently consumed (e.g., cereal) but were underrepresented in the database. This database is called the “archive” and it includes over 2100 photographs of different foods, and over 250 photos of different portion sizes of the same foods. Each food is linked to energy and nutrient information from the USDA database (USDA 2000). In rare cases when a food item is not represented in the USDA database, data from the manufacturer is utilized (e.g., food label, website).

During Pilot Study 1, we simulated food selection and plate waste in the laboratory by taking photographs of foods with a PDA (Palm Pilot, Zire™ 72; Palm Inc., Sunnyvale, CA, USA) equipped with a digital camera. We took photographs of 31 different foods that were grouped together to simulate 16 meals (e.g., hamburger, ketchup, French fries, soda). Each simulated meal consisted of a mean of 3.88 foods; hence, each food was represented twice with two different portion sizes. For each simulated meal, the amount of food selected and plate waste was determined randomly. Food selection ranged from 35% to 800% of the standard portion, and plate waste ranged from 0% to 90% of the amount of food selected. The amount of food in each photograph was carefully weighed and recorded.

Data Analytic Plan

To test if use of a standard portion photograph from the archive facilitated reliable and valid/accurate estimates of EI, trained RDs estimated food selection and plate waste using two methods in the following order: 1) without photographs of a standard portion, and 2) with photographs of a standard portion from the archive. For each of the two methods, we compared the RDs’ estimates of food selection to the actual amount of food that was selected with dependent t-tests. Similarly, plate waste and EI from each method was compared to weighed plate waste and EI. To test inter-rater reliability, two RDs estimated EI for all of the foods (100% over-sample) and ICCs were calculated. We predicted that the RDs EI estimates would not differ significantly from EI measured by directly weighing foods when a standard portion photograph from the archive was used, but the estimates would differ significantly when no photograph from the archive was used.

Results

Inter-rater agreement between the two trained RDs who estimated EI was high with and without the archive (Table 1); therefore, the RDs’ estimates were averaged. Estimated food selection and EI differed significantly from weighed food selection and EI when the archive was not utilized, but this difference was not significant when the archive was utilized (Table 1). Estimated plate waste did not differ from weighed plate waste for either method. Estimated EI only differed from weighed EI by -8.2% on average when the archive was utilized.

Table 1.

Comparison of food selection, plate waste, and EI estimated with the RFPM to weighed EI when a standard portion photograph from the archive was and was not utilized during the estimation procedure (Pilot Study 1). Intraclass correlation coefficients (ICCs) with 95% confident intervals represent agreement between the two RDs who estimated EI (100% over-sample)

		ICC (95% CI)	Estimated kJ (kcal)		Weighed kJ (kcal)		Difference (%) of the means	t	df	P
			Mean	SEM	Mean	SEM
No Standard Portion Photo was Used	Food Selection	.93*** (.81-.98)	502 (120)	59 (14)	548 (131)	59 (14)	-8.4%	-2.21	61	.01
	Plate Waste	.94*** (.84-.98)	142 (34)	25 (6)	146 (35)	21 (5)	-2.9%	-0.35	61	.73
	Energy Intake	.91*** (.76-.97)	360 (86)	46 (11)	406 (97)	50 (12)	-11.3%	-2.30	61	.03
Standard Portion Photo was Used	Food Selection	.95*** (.86-.98)	523 (125)	59 (14)	548 (131)	59 (14)	-4.6%	-1.12	61	.27
	Plate Waste	.92*** (.81-.97)	151 (36)	25 (6)	146 (35)	21 (5)	+2.9%	0.51	61	.61
	Energy Intake	.92*** (.79-.97)	372 (89)	46 (11)	406 (97)	50 (12)	-8.2%	-1.58	61	.12

^***p<.0001.

Discussion

The first pilot study demonstrated that trained RDs can reliably and accurately estimate EI with standard portion photographs from the archive. Estimated food selection, plate waste, and EI differed by -8.2% on average from directly weighed food when the archive was used.

Pilot Study 2

Participants and Procedures

Pilot Study 2 was a feasibility study to determine if free-living people could use the PDAs to take pictures of their food selection and plate waste. Male and female adults (age 18 to 54 years) with body mass index (BMI; kg/m²) between 19 and 35 (inclusive) were enrolled. Participants were asked to take pictures of their food selection, including calorie containing beverages, and plate waste for four consecutive days and to label the pictures with a brief description of the foods. When participants returned the PDAs to the researchers, they were asked about their satisfaction with the method and factors that hindered their ability to take photographs of their foods. Trained RDs estimated EI using the digital photography method. During this process, they recorded problems with the pictures that limited their ability to estimate EI from the photographs. No data analyses were planned for this pilot and feasibility study, which served the purpose of identifying barriers to collecting digital photography data in free-living conditions and analyzing the data.

Results

Participant characteristics for Pilot Study 2 are depicted in Table 2. The sample was predominantly Caucasian (67%) and female (79%). Pilot Study 2 demonstrated that free-living people can use PDAs to take pictures of their food selection and plate waste and also identified factors that inhibit application of the method to free-living conditions. Moreover, the study identified ways to modify procedures to overcome these factors.

Table 2.

Baseline characteristics and sex and race distribution (n, %) of study participants

	Age (years)		Height (cm)		Weight (kg)		BMI (kg/m2)
	Mean	SEM	Mean	SEM	Mean	SEM	Mean	SEM
Pilot Study 1	-	-	-	-	-	-	-	-
Pilot Study 2	33.9	1.6	166.8	1.7	70.7	1.9	25.5	0.7
Main Study	32.4	1.5	170.6	1.4	77.2	1.9	26.5	0.5

	Sex		Race
	Male n (%)	Female n (%)	Caucasian n (%)	African American n (%)	Other n (%)
Pilot Study 1	-	-	-	-	-
Pilot Study 2	9 (21%)	33 (79%)	28 (67%)	12 (28%)	2 (5%)
Main Study	23 (46%)	27 (54%)	35 (70%)	15 (30%)	0 (0%)

Note. Pilot Study 2, n = 42; Main Study, n = 50.

First, participants occasionally forgot to take photographs of their food selection and plate waste. To address this problem, ecological momentary assessment (EMA) ²⁹ technology was adopted and alarms were installed on the Palm Pilots™ to remind participants to take photographs of their foods. Second, the researchers could not review the data until the participants returned the PDA at the end of the four-day study. Therefore, systematic mistakes in taking the pictures could not be corrected, and these mistakes negatively affected data quality for the entire study. To address this problem, future research relied on cellular phones equipped with digital cameras and data transfer capabilities that allowed photographs to be sent to the researchers in near real-time. Third, many photographs were difficult to analyze due to poor lighting; consequently, the new data collection devices (cell phones) include a flash and computer software will be utilized to digitally enhance photographs. Fourth, participants occasionally provided poor descriptions of foods in pictures due to the time required to type the description. To correct this problem in future research, participants will be instructed to quickly record voice (or text) messages with the cell phone to describe the foods. Finally, participants required experiential training with the data collection devices to take good photographs; therefore, we created a 20 to 30 minute training paradigm.

Discussion

The results of the second pilot study provided important information about factors that inhibit data collection in free-living conditions, and the study identified ways to overcome these limitations. The results of this study were instrumental in guiding development of the RFPM and addressing factors that affected data quality.

Main Study

The aim of the main study was to finalize the development and procedures of the Remote Food Photography Method (RFPM) and to obtain reliability and validity data from laboratory and free-living conditions.

Measures

Remote Food Photography Method (RFPM)

The procedures for the RFPM were finalized based on the results of the two pilot studies and consisted of the following: 1) Participants were trained to use the cell phones to take pictures of their food selection and plate waste, label the pictures, and send these pictures to the researchers over the cellular network. 2) Participants received and were asked to respond to four to six automated prompts that reminded them to take photographs and to send the photographs to the researchers. These prompts utilized ecological momentary assessment (EMA) ²⁹ principles and consisted of emails and text messages. 3) Participants were provided with a telescoping pen to standardize the distance of the camera from the food and were instructed to take photographs at a 45 degree angle. 4) Participants were instructed to record their EI using traditional pen-and-paper methods or a voice recording on the phone in case of technology failure (we also interviewed participants at the end of the data collection period to name foods that were difficult to identify or were poorly described). Participants were trained on these procedures during a 20 to 30 minute individual experiential training session during which they showed mastery of the procedures prior to being dismissed. The phones used during the research were Motorola i860 phones (Motorola, Inc., Schaumburg, IL, USA), which were equipped with 1.3-mega pixel cameras and 4X digital zoom. Data analysis/estimation of EI follows the validated digital photography of foods method ^{22, 23} and is outlined in the Introduction of this paper.

Participants completed a form before data collection that asked them to rate their comfort level with using a cell phone (1 = not at all comfortable, to 5 = extremely comfortable). After data collection, participants completed a form that asked about their satisfaction with the RFPM, and their satisfaction with sending the photographs to the researchers (1 = extremely dissatisfied to 6 = extremely satisfied). Participants also rated their perceived ease of using the RFPM (1 = very difficult to 6 = extremely easy), and if they would rather use the RFPM or pen-and-paper records to record intake. Collection of these data began after we started the trial; therefore, data were only available on a subset of the study sample.

Weighed EI

Energy intake measured by directly weighing foods served as the gold standard criterion for EI and was used to test the validity of the RFPM over three days. Energy intake was measured under two conditions. First, participants ate meals in the Eating Laboratory of the PBRC. Second, a group of participants were provided with pre-weighed food in a cooler and were asked to eat this food in their natural environment for dinner. The participants returned the cooler the following morning and the remaining food was weighed to calculate EI. Participants used the RFPM in the laboratory and their natural environment.

Participants

Fifty-two participants were randomized and two participants failed to complete the study. Inclusion criteria were: healthy male or female age 18 to 54 years with BMI 20 to 35 (inclusive). Exclusion criteria were: use of medications that influenced eating behavior, diagnosis of chronic illness, tobacco use, aversion to study foods, and, for females, irregular menstrual cycle and pregnancy. The inclusion and exclusion criteria were lenient to recruit a diverse sample that is representative of people who participate in research or seek weight loss treatment, increasing the generalizeability of the study.

Random Assignment and Procedures

All participants completed training in the RFPM. Participants were randomly assigned (1:1 ratio) to the dine-in or take-out group. On each test day, all participants consumed a standard breakfast (Table 3). In the dine-in condition, participants completed three days of food intake testing, during which lunch and dinner were eaten in the laboratory. In the take-out condition, participants completed three days of food intake testing, but they only ate lunch in the laboratory. Dinner was eaten out of the cooler in free-living conditions. Participants used the RFPM during all lunch and dinner meals; therefore, two EI measures were collected: 1) weighed EI, and 2) EI estimated with the RFPM.

Table 3.

Description (energy and macronutrient content) of the foods served during breakfast, lunch, and dinner during the main study

	Serving Size (g)	Energy kJ (kcal)	Fat (g)	Protein (g)	Carb. (g)
Breakfast
Nutri-Grain Bar® (1 bar)	37.0	586 (140)	3	1	26
Skim milk (1 carton)	230.8	377 (90)	0	8.1	11.4
Sun-Maid® Raisins (1 box)	43.3	540 (129)	0	1.3	34
Totals	311.1	1502 (359)	3	10.4	71.4
Lunch*
Ham Sandwiches (3 hoagies)	708.0	4858 (1161)	36.8	81.4	123.2
Roast Beef Sandwiches (3 hoagies)	708.0	5243 (1253)	36.1	105.5	120.4
Turkey Sandwiches (3 hoagies)	708.0	4653 (1112)	28.3	89.2	120.4
Ruffles® Reduced Fat Potato Chips	122.8	2544 (608)	26.0	8.7	82.4
Famous Amos® Chocolate Chip Cookies	224.0	4845 (1158)	54.0	15.5	154.6
Totals	1054.8	12042 to 12631 (2878 to 3019)	108.3 to 116.8	105.6 to 129.7	357.4 to 360.2
Dinner
Spaghetti Noodles with Prego	300 g noodles 265 g sauce	3021 (722)	12.4	19.1	131.1
Spaghetti Noodles with Alfredo	300 g noodles 265 g sauce	3858 (922)	45.5	19	97.7
Arezzo ® Meatballs (10 meatballs)	480	2021 (483)	42	16.6	8.4
Pepperidge Farm ® Texas Toast with Cheese (4 slices)	192	3012 (720)	44	16	68
Dole® Tropical Fruit Salad (1 container)	732	2008 (480)	0	0	126
Pepperidge® Farm Iced Brownie (4 brownies)	296	5021 (1200)	48.4	14.4	189.2
Totals	2830	15920 (3805)	179.9	66.1	522.7

Note.

^*Participants selected and were served one of the three sandwich choices for lunch.

In the dine-in and take-out conditions, the same types and amounts of foods were provided (Table 3) and test days were 2 to 7 days apart. Excess food was provided to ensure plate waste and provide a conservative test of the RFPM since the presence of plate waste creates more opportunity for error and is not always present in peoples’ natural environment, i.e., many people eat all of their portions.

Data Analytic Plan

The reliability of the RFPM was examined by calculating ICCs for EI measured with the RFPM and directly weighing foods over the three days of testing in both free-living and laboratory conditions. Agreement among the three RDs who scored the photographs was examined with ICC’s (27% of meals were over-sampled). For the following analyses, energy intake data were averaged over the three days of testing.

Three series of analyses were conducted to test the validity of the RFPM. First, EI estimated with the RFPM was compared to weighed EI measured in the Eating Laboratory. These analyses relied on data from the dine-in group (their lunch and dinner data were combined). A second laboratory-based test of the RFPM relied on the take-out group’s lunch data. Third, the take-out group’s dinner data were analyzed to test the validity of the RFPM in free-living conditions. The second and third series of analyses allowed examination of the accuracy of the RFPM in laboratory vs. free-living conditions within the same sample of participants. For each series of analyses, Pearson correlation coefficients were calculated between the RFPM and weigh-back method, and the Bland and Altman technique ³⁰ was used to determine if: 1) the methods differ significantly from each other, and 2) the accuracy of the RFPM varies with the amount of food eaten (EI).

The validity of the RFPM was also examined as a function of body mass or weight (kg), age, and sex. Error was calculated by determining the percent difference between EI estimated with the RFPM and directly weighed foods. Error was regressed against body weight with and without gender in the model to test for significance of slopes. A similar analysis was conducted with age. These analyses were conducted for each of the three samples outlined in the previous paragraph, namely, 1) the combined lunch and dinner data from the dine-in group, 2) the take-out group’s lunch data, which were collected in the laboratory, and 3) the take-out group’s dinner data that were collected free-living conditions.

Results

Participant characteristics for the main study are depicted in Table 2. The sample consisted of 54% females and 70% Caucasians.

Ecological Momentary Assessment (EMA) and Missing Data

A total of 157 EMA prompts were sent and all were received, though 37 (23.6%) were delivered at the incorrect time (prompts were sent via an email server that delayed delivery of some messages). Participants responded to 118 (98.3%) of the 120 prompts that were delivered on time.

Each group, dine-in and take-out, consumed 150 meals that were measured with the RFPM and by directly weighing foods. For the dine-in group, all 150 meals were consumed in the Eating Laboratory. When using the weigh-back method, data from 3 meals were not analyzed due to technical error or protocol violations, including eating only certain ingredients from the sandwiches, which made it impossible to obtain weighed intake. Four meals were not analyzed due to poor picture quality or missing pictures.

The take-out group consumed 150 meals (75 meals in the laboratory and 75 meals in free-living conditions). In the laboratory, data from 7 meals were not analyzed due to: 1 meal had missing pictures, 3 meals had poor quality pictures, and 3 meals involved protocol violations. In free-living conditions, data from 11 meals were not analyzed due to: 5 meals had poor quality pictures, and 6 meals had protocol violations. Protocol violations included not eating the food provided by the researchers due to eating other foods.

Reliability of RFPM

Agreement was high among the three RDs who used photographs from the RFPM to estimate EI. The ICC’s (95% confidence intervals are in parentheses) for food selection, plate waste, and EI were .99 (.99-.99), .91 (.87-.94), and .88 (.81-.91), respectively. The ICCs for intake (kJ) of fat, carbohydrate, and protein were .92 (.88-.94), .85 (.77-.89), and .85 (.79-.90), respectively. These values are consistent with inter-rater agreement when the digital photography method is used in cafeteria settings ^{22, 23, 25}.

Energy intake estimated with the RFPM was reliable over the three days of testing. Based on the dine-in group’s combined lunch and dinner data, the RFPM produced reliable estimates of EI in laboratory conditions (ICC = .62, p < .0001). Moreover, data from the dinner meal of the take-out group indicated that the RFPM produced reliable estimates of EI in free living conditions (ICC = .68, p < .0001). These values are consistent with ICCs calculated from directly weighing food intake in the laboratory over a number of days ³¹. In the present study, the ICCs for food intake measured in the laboratory were .68 and .70 for lunch and dinner, respectively (p-values < .0001).

Validity of RFPM

Participants took and sent photographs of good quality to the researchers. Figure 1 contains representative pictures from the study.

Figure 1.

Representative pictures of food selection and plate waste taken by a participant and sent to the researchers via the wireless network.

The first series of analyses tested the validity of the RFPM in laboratory conditions and included the dine-in group’s combined lunch and dinner data. Energy intake estimated with the RFPM correlated highly with weighed EI, though the RFPM significantly underestimated EI (-368 kJ or -88 kcal, -4.7%; Table 4). Bland and Altman analysis indicated that this bias was consistent over different levels of EI (Figure 2).

Table 4.

Summary of analyses to test the validity of EI estimated with the RFPM compared to directly weighed foods in laboratory and free-living conditions. Energy intake (kJ) estimated with the RFPM was compared to weighed EI with Pearson correlations and dependent t-tests.

	Location of Data Collection	EI, kJ (kcal), estimated by RFPM		EI, kJ (kcal), measured by directly weighed food						Delta kJ (kcal)	Delta kJ (kcal)	Percent difference of the means
		Mean	SEM	Mean	SEM	Pear. Corr.	t	df	p	Mean	SEM
EI (Lunch and Dinner, Dine-In Group)	Lab.	7448 (1780)	469 (112)	7816 (1868)	435 (104)	.93***	-2.10	24	.046	-368 (-88)	174 (42)	-4.7%
EI (Lunch, Take-Out Group)	Lab.	2590 (619)	222 (53)	2741 (655)	209 (50)	.93***	-1.86	22	.076	-151 (-36)	81 (19)	-5.5%
EI (Dinner, Take-Out Group)	Free-living	5707 (1364)	481 (115)	6113 (1461)	439 (105)	.95***	-2.57	23	.017	-406 (-97)	159 (38)	-6.6%

^***p<.0001. Pear. Corr. = Pearson Correlation Coefficients.

Figure 2.

Bland and Altman analysis comparing the RFPM to weighed EI in the laboratory. Lunch and dinner data from the dine-in group was included in these analyses. The RFPM bias in estimating EI was consistent over different levels of EI, R²= .03 (adjusted R² = -.01), p = .39.

The second series of analyses tested the validity of the RFPM in laboratory conditions by analyzing the take-out group’s lunch data. Energy intake estimated with the RFPM correlated highly with weighed EI, and the RFPM non-significantly underestimated EI (-151 kJ or -36 kcal, -5.5%; Table 4). Bland and Altman analysis indicated that this bias was consistent over different levels of EI (Figure 3).

Figure 3.

Bland and Altman analysis comparing the RFPM to weighed EI in laboratory conditions by analyzing lunch data from the take-out group. The RFPM bias in estimating EI was consistent over different levels of EI, R²= .03 (adjusted R² = -.02), p = .45.

The third series of analyses tested the validity of the RFPM in free-living conditions by analyzing the take-out group’s dinner data. Energy intake estimated with the RFPM correlated highly with weighed EI, and the RFPM significantly underestimated EI (-406 kJ or -97 kcal, -6.6%; Table 4). Bland and Altman analysis indicated that this bias was consistent over different levels of EI (Figure 4).

Figure 4.

Bland and Altman analysis comparing the RFPM to weighed EI in free-living conditions by analyzing the dinner data from the take-out group. The RFPM bias in estimating EI was consistent over different levels of EI, R²= .08 (adjusted R² = .04), p = .18.

Validity of the RFPM as a function of body mass (kg), age, and sex.

Regression analysis indicated that there was no significant association between body weight (kg) and the RFPM’s error (percent difference between EI estimated with the RFPM and weighed EI) with (Adj. R², -0.07 to 0.08; p-values > .45) or without (Adj. R², -0.04 to 0.01; p-values > .30) sex in the models. Age also was not associated with error (Adj. R², -0.07 to 0.08; p-values > .65). Error did not differ significantly between men and women in the take-out group at lunch, t(21)=-0.73, p = .47, or dinner, t(22)=-0.75, p = .46. Error was significantly larger for women (-9.2%) compared to men (-0.7%) in the dine-in group, t(23)=-2.08, p = .05.

Satisfaction Ratings

Thirty-five participants’ had data on their comfort level (1 = not at all comfortable, to 5 = extremely comfortable) of using cell phones before data collection. Thirty (85.7%) participants rated their comfort level as 4 or higher. After data collection, 47 participants’ had satisfaction data (1 = extremely dissatisfied to 6 = extremely satisfied), and 37 (78.8%) participates rated their satisfaction with the RFPM as 5 or higher and 40 (85%) participants rated their satisfaction with sending photographs to the researchers as 5 or higher. Lastly, 44 (93.6%) participants rated the ease of use of the RFPM as 5 or higher, and almost all participants (44, 93.6%) indicted they would rather use the RFPM rather than pen-and-paper to record food intake.

Discussion

The RFPM was found to produce reliable and valid estimates of EI in both laboratory and free-living conditions. The degree of error for the RFPM was very small (-4.7% to -6.6%) when compared to error of 37% or more associated with self-report methods ^10-12, and the error was similar in both laboratory and free-living conditions. Moreover, error associated with the RFPM was not found to vary by the amount of energy/food eaten or body weight, which is a significant liability of self-report methods ⁵. Error also did not differ by age, and only one of three comparisons found that error differed by sex (this comparison relied on data from the laboratory). The majority of participants rated their satisfaction with the RFPM and with sending pictures to the researcher favorably and almost all (93.6%) participants indicated that they would rather use the RFPM compared to pen-and-paper records to record food intake. Thus, the RFPM appears to be a promising new method for estimating group and individual EI.

Implications of the Findings

The RFPM has important implications for both research and clinical practice. Obtaining accurate estimates of food and macronutrient intake from free-living individuals is important for researchers who investigate the nutritional adequacy of the diet and interventions to modify food intake and body weight. Additionally, the RFPM allows people to receive virtually immediate feedback about their energy and micro- and macronutrient intake from professionals in a remote location. People can obtain clinical and dietary recommendations based on food intake data that is reviewed in near real-time, and they do not need to go to a clinic to receive this feedback.

The RFPM is a promising method to estimate EI with innovative technology. Nevertheless, our experiences during these studies remind us of the complexities of collecting EI data from free-living people and the barriers that negatively affect data quality, as described in the following paragraphs. Therefore, the RFPM continues to be modified to address these barriers, and these modifications include the following.

First, the RFPM relies on the ability of participants to remember to take photographs of their food selection and plate waste. The use of EMA methodology provided automated prompts reminding participants to take photographs, but missing data is unavoidable. To reduce missing data, it is clear that secondary methods are needed to capture EI data when the participant forgets to take photographs or when photographing food is not possible. In current and future studies, we include explicit instructions for participants to record food intake with pen-and-paper or to use the phone to make voice recordings describing the food and portion size. Additionally, we monitor incoming data and contact participants immediately if their data is of poor quality or if photographs are missing. Participants have responded positively to these procedures, since the procedures indicate that the researchers are invested in data collection and that participants are accountable.

Second, participants’ food descriptions do not always convey important details about the food item, such as the method of preparation. Therefore, when participants return the cell phone to the researchers, they answer questions about foods that are difficult to identify and that have poor descriptions. During this meeting, the participant also provides the researchers with pen-and-paper records or voice recordings that were used to record EI in the case of technology failure.

Third, a considerable amount of personnel time is required to schedule EMA prompts, monitor incoming data, etc. Therefore, the methodology is being further modified and computer applications are being developed to: 1) automatically deliver EMA prompts and modify the prompts based on the participants’ adherence, 2) monitor incoming data and alert the researcher and participant when data appear to be missing, e.g., when an odd number of photographs are received, 3) digitally enhance the quality and lighting of photographs, and 4) automatically estimate the amount of food represented in pictures using computer imaging algorithms. These aims are part of ongoing research and development to facilitate data collection in free-living conditions where participants eat ad libitum and to minimize under-recording or participants failing to take and send photographs. Also, the computer imaging techniques being developed will determine the angle and distance of the camera phone from the food, which will facilitate accurate estimates of EI even when foods are eaten off of plates that very in size. To accomplish this aim, participants will include a standard size object in the photograph, e.g., a dollar bill or six inch ruler. An additional aim of the ongoing research is to determine if participants under-eat when they use the RFPM. Under-eating is known to occur when food intake is monitored using other methods, e.g., Goris et al. ¹².

Strengths and Limitations

The study has important strengths, including a diverse study sample and excellent retention (only two participants failed to complete the protocol). Additionally, participants considered under-reporters were not excluded from the data analysis, which is common in other studies. Lastly, the RFPM was tested in controlled laboratory and free-living conditions in the same sample (the take-out group), which allows direct comparisons of the accuracy between laboratory and free-living conditions.

The study has important limitations that also must be considered. First, participants in the take-out group of the main study collected data during free-living conditions, though they ate food from a cooler provided by the researchers. Consequently, the variety of food was limited and not necessarily representative of their habitual daily intake. Although these conditions were contrived, they were necessary since other criterion measures of EI during free-living conditions, e.g., doubly labeled water, were not feasible for this study. Second, EI was only measured for three days.

Conclusions

The studies reported herein describe the development and initial support for the reliability and validity of the RFPM, as well as high user satisfaction ratings. Ongoing research is refining the methodology and testing the RFPM in free-living conditions against EI measured with doubly labeled water. Our findings suggest that the RFPM is a promising new method for estimating EI in free-living conditions.