Table 1.

Overview of image-assisted methods to measure food intake and studies validating these methods.

Method	Methodology	Review / Analysis	Study Setting	Sample Size	Outcome	Reference Method	Reliability / Validity
Digital Photography of Foods Method (DPFM) [13,16,18,23]	Images of food selection and plate waste are captured with digital (video) cameras.	Human raters compare food images to images of weighed standard portions	Laboratory [13]	60 test meals of 10 different portion sizes	Portion size	Weighed foods	Significant correlation with weighed foods of 0.92. Mean error in portion size was +5.2 g (SE 0.95) or 4.7% relative to weighed foods.
			School cafeteria; 5 consecutive days of school lunches [16]	43 school children	EI	Weighed foods	ICC for total EI was 0.93. Convergent validity was supported by significant correlation between food intake and adiposity (r=0.45) and discriminant validity was supported by non-significant correlation between food intake and depressed mood (r=0.1).
			One laboratory-based test meal [18]	22 preschool children	EI	Weighed foods	Significant correlation of DPFM with weighed foods of 0.96 and mean error in total intake of −4% compared to weighed foods.
			School cafeteria; 7 days of school lunches and dinners [23]	239 school children	EI	Weighed foods	Mean error in total intake of DPFM of 3 g (SD 20) or 1% compared to weighed foods.
Digital Photography + Recall (DP+R) [28]	Images of food selection and plate waste of cafeteria meals including notes to identify ambiguous foods and measuring cups/spoons to guide portion size estimation. Dietary recall to document any foods or beverages consumed outside the cafeteria.	Human raters compare food images to images of weighed standard portions and perform multi-pass dietary recall	Cafeteria and free-living conditions over 7 days	91 adults with overweight/ obesity	EI	DLW	The mean EI estimated by DP + R was not significantly different from DLW, overestimating DLW by 264 kJ (SD 3138; 63 kcal [SD 750]) or 6.8% (SD 28) per day. No proportional bias variation as a function of the level of EI (r=−0.13, p=0.21).
Remote Food Photography Method (RFPM) [12,25,30–33,39]	Images of food selection and plate waste (including a reference card) are captured via smartphone app and sent to laboratory for analysis.	Human raters compare food images to images of weighed standard portions	Free-living conditions (6 days) and 2 laboratory-based buffet meals [12]	50 adults	EI	DLW (free-living) and weighed foods (laboratory)	In free-living conditions, RFPM underestimated total EI by 636 kJ (SD 2904; 152 kcal [SD 694]) or 3.7% (SD 28.7) per day (p=0.16); ICC for daily EI was 0.74.
							In the laboratory, underestimation for total EI was 17 kJ (SD 305; 4 kcal [SD 73]) or 1.2% (SD 62.8) and error for macronutrients was not significantly different from weighed foods.
			Pre-packed lunch (consumed in laboratory) and dinner meals (consumed in laboratory or at home) over 3 days [25]	52 adults	EI	Weighed foods	RFPM underestimated EI by 4.7%−5.5% (laboratory) and by 6.6% in free-living conditions. ICCs for EI were significant for laboratory (r=0.62; p<0.01) and free-living conditions (r=0.68, p<0.01).
			Laboratory; 12-hour period [30]	54 preschool children	EI	Weighed foods	RFPM significantly overestimated total EI by 314 kJ (SD 452; 75 kcal [SD 108]) or 7.5% (SD 10.0). The MPE for the macronutrient intakes ranged from 2.9% (fat) to 11.7% (protein), with high variability around the mean.
			Free-living conditions over 7 days [31]	39 preschool children	EI	DLW	RFPM underestimated total daily EI by a mean 929 kJ (SD 1146; 222 kcal [SD 274]) or 15.6%.
			Laboratory; 2 visits 5–10 days apart [32]	53 adults	EI	Weighed foods	RFPM underestimated EI of 2, 4, and 6 fl oz servings of infant formula by 6.7 kJ (SD 1.7; 1.6 kcal [SD 0.4]), 20.1 kJ (SD 2.5; 4.8 kcal [SD 0.6]), and 25.9 kJ (SD 4.2; 6.2 kcal [SD 1.0]), and overestimated intake by 0.4 kJ (SD 5.0; 0.1 kcal [SD 1.2]) kcals in 8 fl oz servings, but was equivalent to weighed intake within 7.5% for all servings.
			Laboratory [33]	7 bottles for each serving size (1, 2, 3, and 4-scoop) containing 5, 10, and 15% more and less formula than recommended	Serving size	Weighed foods	RFPM underestimated servings (1–4 scoops) of powdered instant formula by a mean 0.05 g (90% CI −0.49, 0.40) compared to directly weighed servings, with the MPE ranging from 0.32% to 1.58%. Estimates for all serving sizes were within 5% equivalence bounds.
			Free-living conditions over 6 days at 2 time points (early vs late pregnancy) [39]	23 pregnant women with obesity	EI	DLW	RFPM captured 64.4% (early pregnancy) and 62.2% (late pregnancy) of DLW-measured total daily EI and was not equivalent to DLW within 20% equivalence bounds. The underestimation was significantly associated with low reporting of snacks (R2=0.4).
Food Record App (FRapp) [43]	Images of food selection and plate waste including fiducial marker, captured with smartphone app. Additional options to capture food intake are speech-to-text conversions, capturing food label/nutrition facts images, selecting from recently recorded foods.	Human raters analyze recordings (images of food, labels or text recordings) of eating events	Free-living conditions over 3 days	18 adolescents	N/A1	N/A1	N/A1
Nutricam Dietary Assessment Method (NuDAM) [44]	Images of food selection (with fiducial marker) combined with a voice recording describing the foods, leftovers, location, and meal occasion, and a brief follow-up phone call the next day.	Trained professionals analyze food images, voice recording, and phone calls	Free-living conditions over 3 days	10 adults, diagnosed with T2DM	EI	DLW	NuDAM underestimated total daily EI by 24% compared to DLW.
Multiple-pass 24-hour dietary recall + SenseCam (MP24+SC) [45]	SC (worn around the neck on a lanyard) captures images of eating events every 20 seconds, triggered to turn on by its sensors. Images of eating events are combined with MP24.	Review of food images and MP24 with participant to allow modification of self-report; estimation of EI by trained dietitian	Free-living conditions over 3 non-consecutive days	40 adults (20 men, 20 women)	EI	DLW	MP24 + SC underestimated EI by 9% in men and by 7% in women compared to DLW. The addition of SC reduced the error in EI by approximately 50% compared to MP24 alone.
Micro-camera [47]	Micro-camera is worn on the ear and captures audiovisual recordings during meal times. Recordings are combined with food diary entries.	Human raters analyze food images and food diaries	Free-living conditions over 2 days	6 adults	EI	DLW	Compared to DLW, daily EI was underestimated by 3912 kJ (SE 1996; 935 kcal [SE 477]) or 34% by food diaries alone and by 3507 kJ (SE 2170; 838 kcal [SE 519]) or 30% when combined with the micro-camera. The difference between the two methods was significant (p=0.02).
mobile Food Record (mFR) [50–52]	Food images are captured with the mFR app and sent to a server for analysis. After review by the user, volume and nutrient content are estimated by the app.	Automatic portion size estimation based on statistical pattern recognition techniques of the image	Free-living conditions over a 24-hour period [50,51]	15 adolescents	Portion size	Weighed foods	Mean error in automated weight estimates using mFR compared to known weights ranged from a 38% underestimation to a 26% overestimation, with 75% of all analyzed food being within 7% of the true value.
			Free-living conditions over 7.5 days [52]	45 adults	EI	DLW	mFR EI correlated significantly (r=0.58) with DLW-measured daily EI and underestimated EI by 12% (SD 11) for men and 10% (SD 10) for women compared to DLW, with no systematic bias with increasing EI.
GoCARB [58]	Food images are captured with a smartphone from two different angles including a reference card.	Automatic segmetation and recognition of food items and reconstruction of their 3D shape	Cafeteria	19 adults, 114 test meals	Carbohydrate content, food recognition	Weighed foods	The mean absolute estimation error of GoCARB compared to precisely weighed carbohydrate content was 26.9% (SD 18.9). Automatic food recognition was correct for 85.1% or all food items.
FoodCam [59]	The user captures a picture of the food and draws boxes around it to initiate the analysis process. The system populates possible food items and the user selects the best fit.	Automatic food recognition and portion size estimation	Laboratory	N/A2	N/A2	N/A2	N/A2
Snap-n-Eat [60]	The user captures a picture of the food and the system automatically estimates energy and nutrient content.	Automatic portion size estimation by image segmentation	Laboratory	2,000 food images for 15 food categories	Food classification	N/A	85% accuracy when classifying 2000 images of food items of 15 different categories.
eButton [61]	Food images are captured automatically by a chest-worn camera every 2–4 seconds. Human rater selects 3D models from software’s library, overlaying the food, and volume of food is then estimated by the software.	Semi-automatic analysis of food images	Laboratory	7 adults capturing 100 pictures of foods	Portion size	Seed displacement method	The mean relative error across all food samples was −2.8% (SD 20.4) and the error for 85 out of 100 foods was between −30% and 30% compared to seed displacement.

¹Feasibility study only, to date no validation of the method.

²Usability study only, to date no validation of the method.

Abbreviations: CI, confidence interval; DLW, doubly labeled water; EI, energy intake; ICC, intra-class correlation coefficient; kJ, kilojoule MPE, mean percent error; SD, standard deviation; SE, standard error; T2DM, type 2 diabetes mellitus.