Validity of the Remote Food Photography Method (RFPM) for estimating energy and nutrient intake in near real-time

Abstract

Two studies are reported; a pilot study to demonstrate feasibility followed by a larger validity study. Study 1’s objective was to test the effect of two ecological momentary assessment (EMA) approaches that varied in intensity on the validity/accuracy of estimating energy intake with the Remote Food Photography Method (RFPM) over six days in free-living conditions. When using the RFPM, Smartphones are used to capture images of food selection and plate waste and to send the images to a server for food intake estimation. Consistent with EMA, prompts are sent to the Smartphones reminding participants to capture food images. During Study 1, energy intake estimated with the RFPM and the gold standard, doubly labeled water (DLW), were compared. Participants were assigned to receive Standard EMA Prompts (n=24) or Customized Prompts (n=16) (the latter received more reminders delivered at personalized meal times). The RFPM differed significantly from DLW at estimating energy intake when Standard (mean±SD = −895±770 kcal/day, p<.0001), but not Customized Prompts (−270±748 kcal/day, p=.22) were used. Error (energy intake from the RFPM minus that from DLW) was significantly smaller with Customized vs. Standard Prompts. The objectives of Study 2 included testing the RFPM’s ability to accurately estimate energy intake in free-living adults (N=50) over six days, and energy and nutrient intake in laboratory-based meals. The RFPM did not differ significantly from DLW at estimating free-living energy intake (−152±694 kcal/day, p=0.16). During laboratory-based meals, estimating energy and macronutrient intake with the RFPM did not differ significantly compared to directly weighed intake.

The study of food intake behavior and weight change, as well as the delivery of nutrition and weight loss interventions, has been hindered by the lack of accurate methods to quantify energy intake (EI) and nutrient intake in free-living conditions. New methods have been developed, however, that rely on camera-enabled cell and Smartphones to collect energy and nutrient intake data, including a method developed by our group called the Remote Food Photography Method (RFPM)(1). Herein, the results from two studies are reported that examined the reliability and validity (accuracy) of the RFPM at estimating energy and nutrient intake. During Study 1 (a pilot study), we tested the effect of two ecological momentary assessment (EMA) approaches, which varied in intensity, on the validity of estimating EI with the RFPM over six days in free-living conditions (EMA is a method to promote data quality and completeness, as described in the Introduction section for Study 1). The RFPM’s procedures were then refined based on the results of Study 1. During Study 2, the reliability and validity of the RFPM at estimating energy and nutrient intake in free-living and laboratory conditions in a larger sample was tested. The effect of estimating energy intake with the RFPM on eating behavior (undereating and overeating) and user satisfaction was also examined during Study 2.

STUDY 1

Introduction

The RFPM was developed to measure the energy and nutrient intake of people while they live in their natural environment. When using the RFPM, participants rely on camera-enabled cell or Smartphones to capture images of their food selection and plate waste (Figure 1). The food images are then sent to a server in near real-time via the wireless network and are stored in a custom-built computer program called the Food Photography Application^©. Based on previously described and validated methods (1–3), participant’s energy and nutrient intake is then estimated by comparing their food images to images of foods with a known portion size. These images are contained in a searchable “archive”, which provides: 1) a standard portion image for portion size estimation, and 2) a match to the Food and Nutrient Database for Dietary Studies 3.0 (4), the producer’s nutrient information, or a custom recipe for energy and nutrient calculation. Image analysis is performed with a semi-automated process that relies on human raters and computer automation. Descriptions of the semi-automated techniques can be found in (5–8) and the black and white reference card in Figure 1 is used to correct/standardize images for color and perspective, which facilitates analysis of the food images by both human raters and computer applications.

Figure 1.

When using the RFPM, participants use a Smartphone to capture images of their food selection, plate waste, and a reference card. These images are then immediately sent to a server for analysis.

Ecological momentary assessment (EMA) methods(9) are used to minimize missing data and promote data quality when using the RFPM. EMA relies on communication and other technologies(e.g., palm top computers, pagers, cell phones) to measure behaviors in “real-time” in natural settings (9). Hence, EMA methods maximize ecological validity and eliminate bias associated with retrospective recall (9). EMA methods have successfully been used to quantify behaviors such as binge eating (10)and alcohol consumption (11) in participants’ natural environments. EMA methods were incorporated into the RFPM by sending prompts (emails, text messages) to participants’ Smartphones reminding them to capture images of food selection and plate waste. Sample prompts include: “Can you remember to take before and after pictures of your lunch and send them to us?”, “Did you eat or drink anything today and forget to take a picture?”. Participants are instructed to reply to each prompt with a yes or no response, or they can type a more detailed message. Missing data are easy to detect because food images are sent in near real-time and can be reviewed continuously, and participants’ responses to prompts are tracked to identify when problems occur with data acquisition. Figure 2 illustrates the stream of communication and data transfer between the research team and the participants. When food images are not captured, participants use a back-up method, such as a food record or a verbal recall conducted via the Smartphone, to capture food intake data.

Figure 2.

The RFPM uses EMA methods to improve data quality and minimize missing data. Prompts are automatically sent to participants’ Smartphones to remind them to capture images of their foods and to send these images to the research staff (the images are received by, and managed in, a computer program called the Food Photography Application^©). The Food Photography Application^© also stores responses to the prompts, and it sends automated reports to the research team and they can quickly identify when data acquisition problems occur.

During previous research, the RFPM was found to underestimate EI by only 6.6% when used at home over three days (1). During that study, participants ate pre-weighed food from a cooler and they returned the cooler the following day, which allowed the remaining food to be directly measured. Although feasibility was demonstrated, the study did not provide a stringent test of the method since the RFPM was not used over many days in truly free-living conditions. Additionally, it could not be determined if the rigor of the EMA methods affected the RFPM’s validity.

The aim of Study 1, reported herein, was to test the effect of two EMA approaches, which varied in intensity, on the validity of estimating EI with the RFPM compared to the gold standard, doubly labeled water (DLW), over six days in free-living conditions. A secondary aim was to quantify how frequently a back-up method was needed when food images were not captured. It was hypothesized that more frequent prompts delivered around personalized meal times would result in more accurate estimates of EI. The results of this pilot study were used to refine the RFPM’s procedures.

Methods

Overview

During a two-week doubly labeled water period, participants used the RFPM to estimate EI during one week, while wearing accelerometers to measure physical activity during the other week (the accelerometry data will be included in another report). The order in which participants used the RFPM and wore the accelerometers was randomized and balanced across participants.

Ethics

Study 1 was approved by the Institutional Review Board (IRB) of the Pennington Biomedical Research Center (PBRC). Participants provided written informed consent prior to enrollment, and data collection was conducted according to guidelines for the ethical treatment of humans and good clinical practice.

Participants and Group Assignment

Forty overweight and obese (25 ≤ BMI ≤ 40 kg/m²) adult (18 ≤ years ≤ 65) participants were recruited for this study, which included a test of the RFPM’s validity and evaluation of the association between baseline activity levels and weight change over one year (the activity level and weight change data will be included in another report). Overweight and obese adults were recruited since they were the focus of the weight change aim of the study, and it is recognized that their participation in the RFPM pilot study (Study 1) resulted in a sample with no healthy weight individuals. Exclusion criteria were use of weight loss medications; diagnoses of diabetes, cardiovascular disease or cancer; and weight instability, defined as >0.5 kg weight change based on regressed daily body weights over seven consecutive days.

For six consecutive days, participants utilized the RFPM, which is described in the Introduction and in (1). The EMA methods differed between the Standard Prompt group and the Customized Prompt group, as follows.

1. Standard Prompts (n=24). Participants received 2 to 3 prompts per day that were sent directly to the Smartphone around generic meal times (e.g., 8:00 AM, noon). Participants were contacted by study staff within 1 to 2 days if data acquisition problems were detected.

2. Customized Prompts (n = 16). Participants received 3 to 4 prompts per day and were contacted within 24 hours if data acquisition problems were detected. The prompts were customized and delivered around individual participant’s meal times.

The first 24 participants were assigned to the Standard Prompt group and the last 16 participants were assigned to the Customized Prompt group. Non-random assignment to conditions was necessary since the ability to deliver customized prompts was not available at the beginning of the study (these procedures are computer-assisted). The Customized Prompt group received more prompts that were delivered around participant’s meal times, and Study 1 was not designed to determine if the number or timing of the prompts was most effective at improving data quality. Participants did not receive compensation or incentives for compliance with EMA prompts, and compliance data (the percent of prompts to which participants responded) is reported herein. The limitations of Study 1, a pilot study, are reviewed in the Discussion section.

Measures

Body weight

Body weight was recorded at screening to confirm eligibility and on Days 0, 7, and 14 of the doubly labeled water period.

Doubly labeled water (DLW) to measure energy intake

DLW is the gold standard for measuring EI in free-living humans and was used as the criterion measure, to which EI estimated with the RFPM was compared. On day 0, participants provided two urine samples before being dosed with DLW (1.425 g of 10% enriched H₂¹⁸O and 0.075 g of 99.9% ²H₂O per kg of body weight). Participants provided additional urine samples 4.5 and 6 h after dosing, with a sample at 3 h being discarded. On days 7 and 14, participants provided two more urine samples. Each sample was analyzed for ¹⁸O and ²H abundance by isotope ratio mass spectrometry using automated devices for deuterium (H/Device, Finnigan) and ¹⁸O (GasBench, Finnigan) (12). The isotopic enrichments of the post-dose urines compared with the pre-dose samples were used to calculate elimination rates (k_d and k_O) using linear regression. CO₂ production was calculated using the equations of Schoeller et al. (13), later modified by Racette et al. (14). Total energy expenditure was calculated by multiplying rCO₂ by the energy equivalent of CO₂ based on the estimated food quotient of the diet at each time point (0.86).

Energy intake was calculated for each week of the DLW period by adjusting total daily energy expenditure for change in energy stores following the methods outlined by Schulz et al. (15). Briefly, EI (kcal/day) = total daily energy expenditure(kcal/day) + change in energy stores (kcal in grams of body fat + kcal in grams of fat free mass change). It was assumed that 2/3 of change in body weight was metabolic and 1/3 was water, and that ¾ of the change in metabolic weight was fat mass and ¼ was fat-free mass (16). For weight loss, we assume 9 kcal/g of fat mass and 1 kcal/g of fat free mass. For weight gain, we assume 13.2 kcal/g of fat mass and 2.2 kcal/g of fat free mass(17–19).

Energy intake measured with the RFPM

The RFPM, described in the Introduction and in (1), was used to measure EI for one week during the DLW period. Six complete days of data were collected and analyzed since the DLW procedures resulted in partial data on days 0, 7, and 14. Energy intake was averaged over the six days to allow a comparison to the criterion measure (EI from DLW). Food images were analyzed by human raters using previously described methods (1).

Data analysis

All statistical analyses were conducted using SAS version 9.1 (Cary, NC) and PASW version 18 (Somers, NY). Alpha was set at .05 and all tests of significance were two-sided.

The validity of the RFPM was examined by calculating the RFPM’s error when compared to EI measured with DLW (EI estimated with the RFPM minus EI measured with DLW). The unit of analysis was kcal/day (mean percent error among participants is also shown, but was not evaluated with inferential statistics). Bland & Altman analysis (20) was then used to evaluate the validity (accuracy) of EI estimated with the RFPM compared to EI measured with DLW for both the Standard and Customized Prompt groups. Specifically, this analysis determined: 1) if EI estimated with the RFPM differed significantly from EI measured with DLW, and 2) if the accuracy of the RFPM varied as a function of the amount of energy consumed. Lastly, the error associated with the RFPM was compared between the two prompt groups using an independent samples t-test.

To examine the secondary aim of the study, the percent of days on which a back-up method was used and the percent of EI estimates derived from back-up methods were quantified. Additionally, compliance with the EMA prompts was calculated as the percent of EMA prompts to which participants responded, and compliance was compared between the prompt groups with an independent samples t-test (data were available on 28 of 35 participants for this analysis).

Results

Characteristics of the study sample

The characteristics of the study sample are provided in Table 1. The mean ± SD age and BMI of the sample were 43.0 ± 14.3 years and 31.9 ± 3.7 kg/m², respectively. Three participants did not complete the study and two Customized Prompt group participants’ DLW data could not be analyzed due to elevated ND/NO ratios, resulting in a final sample of 35 participants.

Table 1.

Characteristics of the study sample.

	Age (y)	Height (cm)	Weight (kg)	BMI (kg/m2)	Gender		Race
					Male	Female	Caucasian	African American
Study 1
Total Sample (N = 40)	43.0 ± 14.3	165.3 ± 6.9	87.5 ± 13.8	31.9 ± 3.7	9 (22.5)	31 (77.5)	30 (75.0)	10 (25.0)
Standard Prompts (n = 24)	42.5 ± 15.5	167.2 ± 6.8	87.5 ± 14.5	31.1 ± 3.7	7 (29.2)	17 (70.8)	18 (75.0)	6 (25.0)
Customized Prompts (n = 16)	43.7 ± 12.9	162.4 ± 6.3	87.6 ± 13.3	33.1 ± 3.4	2 (12.5)	14 (87.5)	12 (75.0)	4 (25.0)
Study 2 (N = 50)	41.0 ± 12.8	164.1 ± 8.1	84.1 ± 16.6	31.2 ± 5.1	6 (12.0)	44 (88.0)	31 (62.0)	19 (38.0)

Notes. Gender and race are reported as number (percent). All other values represent mean ± SD.

Accuracy of the RFPM between the Standard and Customized Prompt Groups

Standard Prompt

When Standard Prompts were used, EI estimated with the RFPM significantly differed from EI measured with DLW, t(21) = −5.45, p< .0001. The RFPM underestimated EI by a mean ± SD of 895 ± 770 kcal/day or 34.3 ± 28.2% (Table 2). This level of bias did not vary as a function of the amount of energy ingested, Adj. R² = −0.03, p = .55 (Figure 3, Panel A).

Table 2.

Comparison of energy and nutrient intake estimates from the Remote Food Photography Method (RFPM) and two gold standards: 1) energy intake measured by doubly labeled water in free-living conditions, and 2) energy and nutrient intake measured in laboratory-based buffet meals.

	EI measured with the RFPM	EI measured with the gold standard	Difference	p-value	Mean participant error (%)
Study 1a
Standard Prompts (n = 22)	1570 ± 636	2465 ± 557	−895 ± 770	<.0001	−34.3 ± 28.2
Customized Prompts (n = 13)	1907 ± 536	2177 ± 491	−270 ± 748	0.22	−8.8 ± 29.8
Study 2
Free-living (n = 42)a	2208 ± 665	2360 ± 626	−152 ± 694	0.16	−3.7 ± 28.7
Buffet Meals (n = 49)
Energy (kcal)	583 ± 190	587 ± 209	−4 ± 73	0.67	1.2 ± 15.1
Fat (kcal)	143 ± 57	136 ± 60	7 ± 19	0.02	8.7 ± 19.8
Carbohydrate (kcal)	350 ± 124	367 ± 136	−17 ± 55	0.04	−3.2 ± 18.3
Protein (kcal)	98 ± 27	92 ± 29	6 ± 17	0.03	10.0 ± 23.9
Vitamin A (mcg)	91 ± 36	78 ± 34	13 ± 17	<.0001	22.9 ± 31.9
Vitamin C (mg)	7 ± 3	6 ± 2	1 ± 2	0.02	17.3 ± 38.4
Calcium (mg)	246 ± 73	236 ± 73	10 ± 43	0.11	6.9 ± 20.7
Iron (mg)	5 ± 2	5 ± 2	0 ± 1	0.23	3.3 ± 14.6
Sodium (mg)	1315 ± 353	1331 ± 431	−15 ± 237	0.65	2.6 ± 20.8
Cholesterol (mg)	48 ± 15	39 ± 14	9 ± 11	<.0001	30.1 ± 41.2
Fiber (g)	4 ± 1	4 ± 1	0 ± 0	0.95	0.9 ± 13.1

Notes: Food intake values reflect mean ± SD. P-values in bold text were statistically significant (alpha was set at 0.01 for the buffet meal comparisons from the main study and all other comparisons relied on alpha equal to 0.05).

^aDoubly labeled water was used as the gold standard in Study 1 and for free-living comparisons in Study 2. Energy intake in these comparisons reflects kcal/day.

Figure 3.

Bland and Altman analysis comparing energy intake (EI) estimated with the Remote Food Photography Method (RFPM) to the gold standard - EI measured with doubly labeled water (DLW). The RFPM’s error was similar across levels of EI in the Standard Prompt (Panel A) and Customized Prompt (Panel B) groups from Study 1, as well as the sample from Study 2 (Panel C).

Customized Prompt

When Customized Prompts were utilized, EI estimated with the RFPM did not significantly differ from EI measured with DLW, t(12) = −1.30, p = .22. The RFPM underestimated EI by a mean ± SD of 270 ± 748 kcal/day or 8.8 ± 29.8% (Table 2). Bias did not vary as a function of the amount of energy ingested, Adj. R² = −0.08, p= .78, (Figure 3, Panel B).

Comparison of Standard to Customized Prompts

When Customized Prompts were utilized, the error associated with the RFPM was significantly smaller than when Standard Prompts were utilized, t(33) = −2.35, p< .05. The difference in the RFPM’s error between the Standard and Customized Prompt groups was 625 ± 762 kcal/day (mean ± SD).

Use of back-up methods

A back-up method for estimating energy intake was utilized on 8.9% of days during data collection. Energy intake estimates from back-up methods constituted 9.7% of total EI estimates. In almost all cases, participants used food records when they could not capture images of their foods.

EMA Compliance

Mean ± SD EMA compliance for the Standard and Customized Prompt groups was 95.7 ± 16.5% and 99.2 ± 2.2%, respectively. Compliance did not differ significantly between groups, t(26) = −0.60, p = .55.

Discussion

The results indicate that more rigorous EMA methods, which include more prompts delivered around customized meal times and quicker feedback when data problems occur, improves the validity of the RFPM. The accuracy of the RFPM is noteworthy. Over 6 days in free-living conditions, this method underestimated EI by only 270 kcal/day or 8.8% when Customized Prompts were utilized. Further, when both Standard and Customized Prompts were used, the error associated with the RFPM was consistent over levels of EI. EMA methods have been used to quantify binge eating (10), dieting lapses (21), and alcohol consumption (11) in participants’ natural environments, and our results indicate that not all EMA procedures equally promote data quality. It appears that the EMA methods should be customized based on the endpoints being measured and the environment in which data are collected.

In the case of technology failure or the participant not having access to the Smartphone, a back-up method such as a food record or food recall was used to capture data. Although these methods are not perfect, it is important to recognize that situations will occur when participants are less likely to use a Smartphone to capture food images. In such situations, it is imperative to obtain an alternative food intake estimate. During this study, a back-up method was used on less than 10% of assessment days, but failure to capture these data could negatively influence the validity of energy and nutrient intake estimates. This study also indicates the importance of collecting EI data with a comprehensive and flexible approach. The RPFM provides that flexibility and is adaptable to different situations and patient characteristics.

Study 1 has many strengths, including a test of the RFPM over six days in truly free-living conditions against the gold standard, DLW. Limitations were also present, however. First, the study sample was small, particularly in the Customized Prompt group. The small sample size negatively influenced statistical power, and the failure to find a significant difference between EI measured with DLW and the RFPM could be due to limited statistical power. Additionally, participants were not randomly assigned to the Standard and Customized Prompt groups and both the number and timing of EMA prompts were modified between groups. Despite these limitations, the results of this pilot study indicate that: the RFPM is a feasible and promising approach to assess energy intake, the RFPM is worthy of further study, and that more rigorous EMA methods promote data quality and accuracy.

STUDY 2

Introduction

The RFPM was refined based on the results of Study 1. Specifically, the RFPM relies on customized EMA methods and prompts are sent around participants’ individual meal times. Further, the number of EMA prompts can be increased or decreased depending on the integrity and completeness of participants’ data. Participants with missing food images or poor quality images receive more prompts and participants with good quality data receive fewer prompts.

When customized EMA methods were used during Study 1, the RFPM underestimated energy intake by 8.8%, which represents an improvement over self-report methods (food records and 24-hour recall). Self-report methods also suffer from other limitations, including increased error among participants with larger body mass (22), selective underestimation of dietary fat (23), and undereating during the monitoring period (23, 24), yet Study 1 was not designed to determine if the RFPM also suffered from these limitations. Therefore, four aims were examined during Study 2. First, the reliability and validity of the RFPM at estimating EI were tested against EI measured with DLW over six days in free-living conditions in a larger and more diverse sample. Second, the validity of estimating energy and nutrient intake, including dietary fat intake, with the RFPM in two laboratory-based buffet meals was evaluated. Third, the effect of measuring energy intake with the RFPM on potential changes in energy intake, either reducing it (undereating)or increasing it (overeating) was quantified. Fourth, participants’ satisfaction with the RFPM was examined with a questionnaire. It was hypothesized that energy and nutrient intake estimates from the RFPM would not differ significantly from the criterion measures, and that the RFPM’s bias would not differ over levels of energy and nutrient intake.

Methods

Overview

Participants used the RFPM during two laboratory-based meals during Study 2. Subsequently, during a two-week doubly labeled water period, participants used the RFPM to estimate EI for one week, and they wore accelerometers to measure physical activity during the other week (the accelerometry data will be included in another report). The order in which participants used the RFPM and wore the activity monitors was randomized and balanced across participants.

Ethics

Study 2 was approved by the IRB of the PBRC. Participants provided written informed consent prior to enrollment.

Participants

Fifty adult(18 ≤ years ≤ 65) lean (n=6, BMI 18.5–24.9), overweight (n=10, BMI 25–29.9), and obese (n=34, BMI≥30) participants without a diagnosis of diabetes, cardiovascular disease, or cancer were recruited. Exclusion criteria included medications that influence body weight and weight instability, defined as >0.5 kg weight change based on regressed daily body weights over seven consecutive days.

Measures

Body weight

Body weight was recorded in the clinic at screening to confirm eligibility and on Days 0, 7, and 14 of the doubly labeled water period. During the DLW period, participants also recorded daily body weight at home using a provided scale.

Doubly labeled water (DLW)

DLW was one EI criterion measure for this study, against which EI estimates from the RFPM were compared. The DLW procedures were identical to Study 1, with one exception. When calculating EI by adjusting total daily energy expenditure for change in energy stores, change in body weight during each week of the DLW period was quantified by regressed daily body weights, which participants collected at home.¹

Laboratory-based buffet meals

Directly measured energy and nutrient intake from two buffet meals served as additional criterion measures, to which estimates from the RFPM were compared. These two lunches were scheduled 2 days apart. The meals consisted of sandwiches, pretzels, cookies, fruit cocktail and a choice of one beverage (regular or diet soda, or sweetened or unsweetened tea). During these meals, participants used the RFPM without assistance from the staff and their energy and nutrient intake was also quantified by covertly weighing food provision and plate waste. The endpoints obtained from these meals were: energy, fat (kcal), carbohydrate (kcal), protein (kcal), vitamin A (mcg), vitamin C (mg), calcium (mg), iron (mg), sodium (mg), cholesterol (mg), and fiber (g).

Energy intake measured with the RFPM

The RFPM, described in the Introduction for Study 1 and in (1), was used to measure EI during the DLW period and the laboratory-based meals. Food images were analyzed by human raters as previously described (1), though the food images were scaled during analysis using the reference card and computer algorithms. Based on results of Study 1, Customized EMA methods were used (incentives were not provided for compliance with EMA prompts). After participants were trained to use the RFPM, they were allowed to use the method for a 1 to 3 day run-in period before being dosed with DLW.

Undereating and overeating

Undereating and overeating were quantified as the change in energy stores (kcal/day) during each week of the DLW period, i.e., when participants were and were not using the RFPM. Negative values represent undereating. Change in energy stores was calculated based on regressed daily body weights using the coefficients for change in body mass outlined in the Doubly Labeled Water section for Study 1.

User-satisfaction

User-satisfaction was assessed with a six-point scale anchored from 1 to 6, with 6 being the most favorable rating. The ratings included: overall satisfaction with the RFPM, satisfaction with sending food images via the Smartphone, satisfaction with the RFPM training, usefulness of the run-in, ease of use of the RFPM, and satisfaction with the EMA methods.

Data Analytic Plan

All statistical analyses were conducted using SAS version 9.1 (Cary, NC) and PASW version 18 (Somers, NY). Unless otherwise specified, alpha was set at .05 and all tests of significance were two-sided.

The first aim was to examine the reliability and validity of the RFPM for estimating EI compared to DLW. Reliability was quantified over six days by calculating intraclass correlation coefficients (ICCs) for daily EI estimated with the RFPM. Similar to Study 1, validity was examined by calculating the RFPM’s error when compared to EI measured with DLW and conducting Bland & Altman analysis (20). The unit of analysis was kcal, though mean percent error among participants is also shown.

The second aim was to test the validity of estimating energy and nutrient intake with the RFPM compared to two laboratory-based buffet meals. Eleven endpoints were evaluated from the buffet meals; therefore, alpha was adjusted to .01. Dependent t-tests were conducted to determine if participants’ food intake differed from the first to the second buffet meal and could therefore not be averaged. Energy and nutrient intake estimated with the RFPM was then compared to directly measured energy and nutrient intake using Bland and Altman analysis, similar to those used in Study 1 (i.e., mean differences were evaluated, and error was examined as a function of the amount of food ingested). Lastly, regression analysis was used to determine if the RFPM’s error varied over levels of baseline body mass (weight in kg and BMI) and age (years).

The third aim was to determine if use of the RFPM resulted in overeating or undereating. A PROC MIXED procedure was conducted to test if significant undereating or overeating occurred during the week when participants used the RFPM while controlling for a potential order effect (this model also determined if a significant order effect was present).

The fourth aim was to examine participants’ satisfaction with the RFPM. User-satisfaction was examined with a six-point scale and was quantified by calculating the percent of participants who rated their satisfaction as five or higher (i.e., who rated their satisfaction with the two most favorable ratings).²

Results

Attrition

Fifty participants were recruited and drop-outs were not replaced. During the two-week study, eight participants failed to complete the protocol. A concerted effort was made to analyze as much data as possible from all participants. Therefore, the sample size varied for analysis of data from the laboratory-based meals, which were collected first, and free-living data, as outlined here.

Laboratory-based Meals

Forty-nine (98%) participants completed the first buffet meal, while 44 (88%) completed the second meal. Data solely from the first meal was used for participants who failed to complete the second test meal; hence, 49 participants’ data were analyzed.

Free-Living Energy Intake Estimates

Free-living EI data were analyzed from the 42 participants who completed the protocol and provided urine samples for DLW analysis. Of these 42 participants, one participant did not use the RFPM for one day due to work obligations. Another participant did not use the RFPM for two days due to travel. Hence, these participants’ energy intake estimates were averaged over four and five days, rather than six days.

Characteristics of the study sample

Demographics for the study sample (88% female, 62% Caucasian) are reported in Table 1. The mean ± SD age and BMI of the sample was 41.0 ± 12.8 years and 31.2 ± 5.1 kg/m², respectively.

Free-living Energy Intake Estimates

Over six days, daily EI estimated with the RFPM was reliable, with an intraclass correlation coefficient of 0.74. Energy intake measured with the RFPM did not differ significantly compared to EI measured with DLW, t(41) = −1.42, p = 0.16. The RFPM underestimated EI by 152 ± 694 kcal/day or 3.7% (Table 2). Bland and Altman analysis indicated that this level of bias (error) did not differ significantly over levels of EI (Adj. R² = −0.02, p = 0.67). Figure 3, Panel C.

Laboratory-based Buffet Meals

Energy intake measured during the first and second buffet meals was very similar (mean ± SD, 592 ± 196 kcal vs. 602 ± 242 kcal)and did not differ significantly, t(43) = −0.42, p = 0.68. This was also true for the additional 10 nutrients that were evaluated (data not shown); therefore, participants’ food intake data from the two buffet meals were averaged.

During the buffet meal, EI measured with the RFPM and by directly weighing foods did not differ significantly (p = .67). The RFPM underestimated EI by only 4 ± 73kcalor1.2 ± 15.1%(mean ± SD). Table 2. No significant differences were detected between measuring macronutrient intake (kcal from protein, fat, or carbohydrate) and intake of vitamin C, calcium, iron, sodium, or fiber with the RFPM compared to directly weighing foods. Significant differences were detected between measuring vitamin A and cholesterol with the RFPM compared to directly weighing foods (p < .01), with the RFPM overestimating intake of these nutrients. Table 2. The RFPM’s bias did not differ significantly over levels of the 11variables evaluated (data not shown).

Regression analysis indicated that the RFPM’s error did not differ over levels of participants’ body weight, Adj. R² = 0.03, p = 0.12; BMI, Adj. R² = −0.02, p = 0.61; or age, Adj. R² = 0.03, p = 0.29.

Under and Overeating

The order in which participants used the RFPM did not significantly influence the magnitude of undereating or overeating, F(1, 38) = 0.52, p = .48. During the week when the RFPM was used, no significant undereating or overeating was present [LS mean ± SEM = −122 ± 99 kcal/day; t(39) = −1.22, p = .23].

User Satisfaction

Participants rated their satisfaction with the RFPM on a six-point scale, with 6 being the most favorable rating. The majority of participants rated their overall satisfaction with the RFPM (82% of participants) and satisfaction with sending food images via the Smartphone (85% of participants) as 5 or higher. In addition, 93% and 96% of participants rated their satisfaction with the RFPM training and the usefulness of the run-in, respectively, as 5 or higher. Finally, 93% and 89% of participants rated the ease of use of the RFPM and satisfaction with the EMA methods, respectively, as 5 or higher.

Discussion

The results from Study2 support the reliability and validity of the RFPM for measuring the energy and nutrient intake of adults. The RFPM underestimated EI in free-living conditions by only 3.7%, which is a clear improvement over self-report methods that underestimate EI by 37% or more (22, 23, 25). Much of the error associated with self-report methods is secondary to participants’ inaccurate estimation of portion size (26), and we found in an earlier study that, even after extensive training, participants’ portion size estimates improved only modestly (27). The RFPM does not rely on participants to estimate portion size, which is likely one reason for its accuracy. Study 2 also demonstrated that the error associated with the RFPM was consistent over levels of energy and nutrient intake. This is an additional improvement over self-report methods that have larger underestimates among individuals with higher body mass (22) and for dietary fat(23). Finally, unlike food records (23, 24), the RFPM was not associated with undereating.

The finding that the RFPM was not associated with undereating is important since one purpose of assessing EI in free-living conditions is to obtain a sample of habitual EI. If participants react to being monitored and alter their EI level, then the utility of the method is diminished. The results of Study 2 indicate that the RFPM can be used to obtain an accurate estimate of habitual EI and that participants do not change their EI as a result of the using the RFPM. During certain interventions that do not require a measurement of habitual EI, a reduction in EI can be advantageous. For example, if people reduce EI and lose weight while they self-report their food intake during a weight loss trial, the method could be viewed as beneficial, even though habitual EI cannot be accurately determined. Finally, it is recognized that determining the effect of any assessment method, including the RFPM, on undereating involves simultaneously quantifying EI with another procedure. Doubly labeled water was used in this study and although it is not associated with significant burden (participants must provide urine samples on Days 0, 7, and 14), it is possible that the presence of this procedure affected habitual EI. Nonetheless, any effect of the DLW method on EI would be constant across the two weeks of assessment, which would eliminate its effect on EI from our undereating parameter. The RFPM relies on cellular networks and EMA methods to collect energy and nutrient data in near real-time, and these near real-time data have significant advantages and clinical utility. First, RFPM data can be surveyed by personnel or computer software to detect quickly when data are missing or when problems with data quality occur. The participant can then be contacted via the data collection device, a Smartphone, to resolve the problem. This is an advantage over other methods, such as food records, where problems with data quality cannot be identified until the participant returns the record. Food records are frequently kept for many days or weeks; therefore, missing or incomplete data are identified for dates that occurred 7 to 14 days earlier. In these cases, it is very difficult for participants to recall specific food items and portion sizes, and recall bias negatively affects the accuracy of their report. Smartphones and similar technology are being used more frequently to collect energy and nutrient intake data in a number of settings(28–31) and our results suggest that EMA methods will likely promote data quality, compliance, and the validity of energy and nutrient intake estimates when food photography is used. Further, our results can be applied to other disciplines where EMA methods are used to measured behaviors such as binge eating, alcohol consumption, smoking, or exercise. Our results indicate that personalization of the number and timing of prompts improves data quality, and these findings can be used to improve EMA data in other settings.

Another advantage of near real-time energy and nutrient intake data is the ability to quickly provide patients with feedback about their behavior while they live in their natural environment. This is important since behavior change is fostered by receiving specific feedback based on objective data that is temporally contiguous to the target behavior (feedback that is delayed or unspecific is less effective at inducing behavior change) (32). This advantage led to the RFPM being incorporated into SmartLoss, a Smartphone-based weight loss application. When using SmartLoss, people receive feedback about their weight change and energy and nutrient intake in near real-time, in addition to suggestions for healthier alternatives to certain foods. They also receive information about adherence to their individualized diet. The RFPM can also be used in clinic-based weight management or health improvement interventions, where services are provided on a fee-for-service basis or as part of a clinical research program. Indeed, it is foreseeable that technologies such as the RFPM might be used in lieu of more traditional self-report methods.

The RFPM and also can be used in epidemiological research, where data can be collected and analyzed remotely on a large scale. Transfer of food images and EMA prompts occur via wireless networks; therefore, data can collected anywhere that has a cell signal, and many participants prefer to use their own phone during data collection (even if the cell signal is not reliable, food image data are stored on the phone and can be transmitted once a reliable cell signal is obtained). Further, we developed a web-based portal for the Food Photography Application^© that allows the scheduling of EMA prompts, tracking of EMA responses, and management of RFPM data to occur via the internet from remote locations. We have successfully used these procedures in a pilot study with a group of investigators located in a different state. We have worked with many groups to provide the RFPM as a service, and we are exploring the ability to make the technology more widely available on a fee-for-service basis and as a Smartphone “app”.

The results of Study 2 indicate that satisfaction with the RFPM is very favorable, and these results support our earlier findings (1). The favorable user-satisfaction ratings likely reflect efforts to minimize participant burden. Participants are not required to estimate portion size or use pen-and-paper to record what they eat, except when a back-up method is needed. Other researchers also have found that people prefer food photography over self-report methods (33). It is possible that satisfaction with the RFPM will increase further as additional technology (e.g., bar code scanning, voice-to-text applications) is incorporated into the method. Participants also were satisfied with the training they received and the run-in period. It appears that the run-in can be helpful, as participants can use the RFPM for a day or more and the study team can answer questions that they have during this time. We found that some participants had questions or required brief feedback during the run-in, but they required little assistance thereafter and the quality and completeness of their data remained exceptional.

The RFPM might be viewed as an expensive alternative to self-report methods, however, we have found that the cost of the RFPM is similar, and sometimes more affordable, than self-report methods. A definitive cost-effectiveness trial is planned, though the RFPM appears to save cost by reducing the amount of time required to estimate energy and nutrient intake and through the use of computer automation to streamline data management and analysis.

The strengths of Study 2 include comparison of the RFPM to DLW over six days, a test of the ability of the RFPM to accurately measure nutrient intake, and objective evaluation of undereating during RFPM use. A potential limitation is the failure to follow participants for more than one week. It is possible that use of the RFPM will diminish over many weeks, though the EMA methods allow data acquisition to be tracked objectively and in real-time.

In summary, the RFPM accurately estimates energy and nutrient intake and its accuracy is not influenced by the amount of food ingested or the individual’s body mass or age. Further, the RFPM is not associated with undereating and data are collected in near real-time, which provides a number of advantages including the ability to provide participants with timely feedback about their behavior. Importantly, this feedback is delivered directly to the participants’ Smartphone and they receive it as long as they are in a location with cellular service. This timely feedback that is based on objective data promises to more effectively help people change their behavior and improve their health. Lastly, participants rated their satisfaction with the RFPM method very favorably. The validity of the RFPM at assessing the food intake of children and adolescents is currently being examined.