Abstract
The science and tools of measuring energy intake and output in humans have rapidly advanced in the last decade. Engineered devices such as wearables and sensors, software applications, and Web-based tools are now ubiquitous in both research and consumer environments. The assessment of energy expenditure in particular has progressed from reliance on self-report instruments to advanced technologies requiring collaboration across multiple disciplines, from optics to accelerometry. In contrast, assessing energy intake still heavily relies on self-report mechanisms. Although these tools have improved, moving from paper-based to online reporting, considerable room for refinement remains in existing tools, and great opportunities exist for novel, transformational tools, including those using spectroscopy and chemo-sensing. This report reviews the state of the science, and the opportunities and challenges in existing and emerging technologies, from the perspectives of 3 key stakeholders: researchers, users, and developers. Each stakeholder approaches these tools with unique requirements: researchers are concerned with validity, accuracy, data detail and abundance, and ethical use; users with ease of use and privacy; and developers with high adherence and utilization, intellectual property, licensing rights, and monetization. Cross-cutting concerns include frequent updating and integration of the food and nutrient databases on which assessments rely, improving accessibility and reducing disparities in use, and maintaining reliable technical assistance. These contextual challenges are discussed in terms of opportunities and further steps in the direction of personalized health.
Keywords: dietary assessment, food apps, wearable device, physical activity, mobile health, image recognition, image-based dietary records
Statement of Significance: This article is the first to discuss the status and challenges of current and emerging technology tools designed to measure individual food intake, eating behavior, and physical activity through the perspectives of 3 stakeholders: researchers, users, and developers. The objective of this work is to bring together experts to address interdisciplinary and cross-cutting issues with the shared mission of improving the measurement of energy intake and expenditure.
Introduction
The collective and cross-disciplinary contributions of scientists, engineers, software developers, and experts from multiple technical domains are beginning to arrive at what even a few decades ago was just a dream: personalized health. The fields of personalized nutrition and physical activity have broadly kept pace with other health disciplines in this regard, contributing to deeper understanding of complex, multitiered relations between food, eating behaviors, metabolic regulation, and energy balance. The future of personalized health and the next generation of nutrition and physical activity guidance rely heavily on what we can learn about individual behavior, which requires accurate assessment of these behaviors.
This article discusses the status of and ongoing challenges for current and emerging technology tools designed to measure individual food intake, eating behavior, and physical activity through the perspectives of 3 stakeholders: researchers, users, and developers. These tools have multiple applications including monitoring outcomes in interventions that strive to alter dietary intake (1) or physical activity (2), and have the potential to transform energy metabolism research and improve health outcomes. With growing interest in determinants that influence individual variability in health outcomes, such as genetic, behavioral, and psychological differences, these tools can enable self-monitoring, allow for detailed research analysis, and provide an avenue for personalized professional recommendations. Previous review articles have summarized the current state of tools for assessing dietary intake (3–6), eating behavior (7, 8), and physical activity (6, 9, 10).
Broadly, current tools tend to be either active (requiring user input) or passive (not requiring user input). Examples include engineered devices such as wearables and sensors, mobile phone applications (apps), and Web-based tools. One promising area of emerging tools is sensor technology that aims to enable more accurate and objective measurement of dietary intake and eating behavior than self-report. These sensor-based tools generally fall into 3 categories: wearable sensors, camera-based devices, and weight scale–based devices. Wearable sensors include devices with sensors on the head or neck to detect chewing or swallowing (11–16), wrist-based inertial sensors to detect hand-to-mouth gestures as a proxy for bites (12, 17, 18), and others (19–21). Camera-based methods (21–25) use food images to recognize consumed food and estimate energy intake. Weight-scale devices are used in dining locations to continuously weigh consumed food (26–28), although eating behaviors can only be captured at the location of the instrument (29).
Multimodal sensing technology has advanced steadily, with the development of devices that have improved estimates of physical activity, energy expenditure, and sleep, and provide important contextual information. For example, for tracking activity, a multimodal sensing device may include traditional actigraphy and ≥1 of the following: multiple accelerometers (30), gyroscopes (31), magnetometers (31), inclinometers (32, 33), Global Positioning System (GPS) (34, 35), photovoltaic sensors (36–38), heart-rate sensors (39), wireless proximity sensors (40), galvanic skin sensors (41), and user-friendly screen displays (42). However, few if any devices on the market contain all these features, due in part to manufacturing costs, battery demands, and size limitations. Going forward, advancements will likely involve improving existing features and combining them into a single device (43), much like a commercially available smartwatch (44). Each of these tools, and others discussed in this article, present challenges and opportunities for stakeholders (Table 1). Underpinning most new or emerging tools are questions of user burden, validity, and privacy.
TABLE 1.
Researcher challenges | User challenges | Developer challenges | |
---|---|---|---|
Dietary intake and eating behavior |
|
|
|
Energy expenditure and physical activity |
|
|
|
Cross-stakeholder challenges |
|
Before proceeding to specific challenges, we submit the following underlying premise: that stakeholders share the goal of accurately measuring intake and expenditure by 1) maximizing the capture of objective data, and/or 2) minimizing error in the capture of subjective data. For emerging tools, this generally means moving toward technology that can capture data as freely as possible from user input. Further, any new tools should reduce or minimize the burden on users and researchers (45). For researchers, tools should maximize the amount and completeness of data collected, include a reliable system of data storage and retrieval (46), and, when possible, have automated, standardized, and harmonized data coding that uses shared terminology and definitions (45). For users, tools should be simple and intuitive, provide privacy controls (47, 48), and require minimal instruction (49, 50) and time to complete assessments (46). For developers, particularly where monetization opportunities exist, satisfying the demands of researchers and users should ensure use by both groups remains high and continuous. Finally, sustained user adherence is a desirable goal for all stakeholders.
Current Status of Knowledge
State of technology tools: assessing energy intake compared with expenditure
Current physical activity tools are considerably more advanced than dietary intake tools. Although both intake and expenditure methodologies previously relied heavily on subjective data, technology for measuring expenditure has successfully integrated expertise across wide-ranging fields (e.g., optics, electromechanical engineering, inferential statistics) and has advanced in nearly all necessary technical and nontechnical domains, from complex algorithms that can differentiate between psychological or physical stressors (51), to the aesthetic elegance of wearable devices. Meanwhile, intake methodologies still overwhelmingly rely on digital adaptations of paper-based instruments of self-reported intake, including diaries, records, and image-based approaches.
Assessing intake may be more complex than assessing expenditure because intake is a question of measuring not just behavior, but also endless heterogeneous origins, preparations, and combinations of foods. Further, even if people were able to perfectly describe the foods they ate, they would not be able to report their nutritional qualities. There are significant opportunities for advancing the development of intake technologies that, similarly to expenditure-measuring technologies, make use of a wide range of scientific fields to better capture both food intake and eating behavior.
Researcher perspectives
Many challenges and opportunities exist for researchers to address both input-focused and output-focused needs. Here, we define input-focused needs as those related to the quality of incoming data (e.g., accuracy of user food intake reports, completeness of food and nutrient databases), whereas output-focused needs include generating research-ready data that are use-compatible across multiple platforms and use commonly agreed-upon terminologies, researcher and/or user dashboards, and other output, such as automated health messaging.
Input-focused needs
Upgrade self-reported dietary intake assessments
Conventional methods of dietary assessment are interviewer-administered 24-h dietary recalls, FFQs, and dietary records, all of which are self-report (52) and subject to error via limitations of human memory, social desirability bias (52, 53), and reactivity to self-monitoring [i.e., altered energy intake on reporting days (54)]. Development of modern dietary assessment tools has focused on digital adaptations of these conventional methods [e.g., online 24-h dietary records (55), online FFQs (56)] and food-logging apps (57), which are already in widespread use. Although these tools will continue to experience self-report limitations, there is room for other improvements, especially with respect to accuracy of portion size estimation and amount of user burden.
Enhance portion size estimation
A participant's ability to estimate and remember portion sizes of consumed foods has been a large source of error in dietary assessment (58, 59) and thus is a target for improvement. Some new, common self-report methods (e.g., food-logging apps) use reference images of portion sizes to assist users with estimation (3, 50, 60). Flexibility in entering portion size is another consideration. Some software allows users to choose portion sizes from a predefined list, or enter them manually, and choose between different measurement units, such as standardized portions or household measures (3, 50, 61). Additional software improvements would allow for inclusion of dimensions and packaged food amounts (3, 50, 61) and automatic conversion of variably reported portion sizes into standard metric units for research purposes, meeting both user and researcher needs. Ideally, data will be harmonized for use across different platforms, necessarily preceded by the development of common data terminologies, to also allow for accurate comparisons of data points, such as nutrient calculations.
Promising emerging approaches use images of consumed foods, such as image-assisted dietary recall [in which images are used to assist a research participant (62, 63)], image-based dietary record [in which images document eating occasions (63, 64)], and automated image analysis (65). Advantages of such methods are reduced reliance on participant memory and direct visual documentation of eating occasions (66). In particular, image-assisted 24-h dietary records have been shown to reduce underreporting (62).
Food images can be analyzed with manual, semiautomatic, and automatic approaches (5). Manual image analysis has the most potential for immediate application in research; however, approaches with higher levels of automation require further development. As with image-assisted methods, accurate analysis requires high-quality images (67).
Simplify and maximize food lists
A great deal of user burden in self-report software and apps derives from lists of food items. Presented lists depend on the quality of the underlying food databases and affect the accuracy of user entry and output data (68, 69). Determining the optimal length of the food list has been a challenge (61, 68). Although extensive and highly detailed lists may benefit researchers (70–72), for users, scrolling through long lists can be burdensome (68). Even so, concise food lists also may be problematic (61), even if they produce only small differences in total nutrient intake compared with extensive lists (69, 73), because users may feel frustrated when precise food items cannot be found. There is limited research on how users find the “best match” when an exact match is missing (68). Although barcode entry eases user burden, manufacturer data on which researchers subsequently rely may be incomplete. Hence, researchers must compare the benefits and limitations of different databases, as well as their effects on user-entry behavior, and the specificity of resulting data.
Validate dietary assessment tools
Several reviews have examined the validity [i.e., acceptable levels of accuracy, precision, and reliability (52)] of technology tools for measuring dietary intake (3, 45, 60, 61, 66, 74–76). A recent review (3) of technology-based tools for research, surveillance, or consumer use identified interviewer-administered 24-h dietary records, weighed portions, biomarker data, and direct observation of eating occasions as common reference/validation measures. Although most of the reviewed comparison studies showed acceptable levels of agreement between the technology tool and the traditional self-report method (within ∼60 kcal), it was observed that use of validation biomarkers was lacking (3). Such comparisons can provide valuable information, but researchers should be cautious of possible correlated errors and seek validation studies that use objective measures such as doubly labeled water (DLW) or direct observation.
Improve energy expenditure assessment
As with dietary assessment, self-report via diaries or questionnaires was the most common method for measuring physical activity in research (77–79). Although such methods are inexpensive and convenient, they have poor reliability and validity compared with DLW (10). Like intake data, self-reported physical activity is affected by question misinterpretation, recall bias, and social desirability (10, 78, 79). Floor effects have been observed with unstructured or spontaneous activities (e.g., housework, gardening), resulting in failure to capture low-intensity activities (77, 79). Overestimation is another common issue (77, 78).
As noted, the use of physical activity devices has become increasingly common by consumers (80, 81), and in epidemiological (82) and intervention (79) studies. Common activity trackers include pedometers, accelerometers, and heart-rate monitors. Despite their widespread use, these devices are still somewhat limited in capturing physical activities that vary in intensity and displacement (i.e., stationary compared with mobile). Pedometers can measure only walking activity in step counts (83). Accelerometers have limited sensitivity with detecting light-intensity activities and nonambulatory activities such as cycling and weightlifting (9). The perceived relation between heart rate and energy expenditure has been the premise of using heart-rate monitors, but they have poor correlation at low and high intensities (10, 83). Beyond physical activity, energy expenditure can already be measured by direct calorimetry using existing, innovative, portable tools, such as the Personal Calorie Monitor (84). However, the ongoing challenge is to develop devices or analytic methods that can assess all types of physical activities and energy expenditure, as well as associated physiological phenomena (e.g., body temperature, perspiration, heart rate) (85). The field is already moving toward integration. Recent studies show it is possible to distinguish, using a wristband device, between simultaneous psychological and physical stressors (51). Another recent device undergoing validation is a commercial wristband containing a photoplethysmogram, accelerometer, thermometer, capacitive touch sensor, and gyroscope (86, 87).
Additional significant upgrades to existing devices and tools would also address user- and/or population-based differences in activity, which can vary by sex, age, ability, health status, and other characteristics (10). New tools should include feedback and data output that reflect user characteristics such as age, sex, body composition, fitness, and perceived exertion. In addition, given that most validation studies have been done in laboratory settings, key environmental characteristics that influence perceived exertion such as elevation, temperature, and humidity (88–90) would ideally be captured by newer devices and software, and integrated into expenditure estimation algorithms.
Standardize validation studies of activity trackers
Thus far, validation studies of activity trackers have exhibited heterogeneity in study design and activity calculations, posing challenges to comparisons. Variable aspects of study design include definition of “valid” days that are suitable for analysis [e.g., 10 h of wear time (91)], device placement [e.g., hip compared with wrist (9, 92)], and context [laboratory compared with free-living (9)]. As noted, many validation studies are conducted in the laboratory. However, pattern recognition models based on laboratory data have limited validity in free-living settings (9, 93).
In addition to study design, the devices themselves exhibit heterogeneity in sensitivity, sampling frequency, noise-separating filters, and other aspects of data capture (91). Algorithms for obtaining desired output such as steps, energy expenditure, and distance use different underlying calculations, which are further obscured by their proprietary nature and restricted sharing (9, 92). In data analysis, there is little consensus on best practices for data processing, algorithms (94), and data interpretation [e.g., the “cut-point conundrum” (95)]. Given these variables of study design and calculations, standardizing data output and validation methods is logistically difficult, and will likely require significant and ongoing collaboration between researchers and developers.
Consider reactivity in self-monitoring
Reactivity in self-monitoring—the conscious or unconscious changes in behavior as a reaction to the act of self-monitoring (54)—is a recognized phenomenon in both intake and expenditure research. For example, wearing an activity monitor may cause a participant to exercise more than usual (96), or using a food app may shift participant eating behavior away from complex dishes to mitigate the burden of logging foods (53, 68, 97). To date, few studies have examined how technology tools induce this reactivity (66). From a researcher perspective, it is beneficial to have control over the feedback or health messages a user receives from a program. The frequent desire of researchers to minimize reactivity to self-monitoring is often in direct contrast to user preferences to access and use their own health data.
Output-focused needs
Simplify dashboards and enhance communication with users
Online 24-h dietary records, online FFQs, and food-logging apps should have a customizable dashboard for research participant management tasks such as registering new participants, updating contact information, viewing lists of usernames, and exporting files (49, 70). Such improvements need not be limited to dietary data. Integrating both real-time intake and expenditure data in a live dashboard is aspirational, and would provide researchers (and users, if appropriate) with opportunities to detect and address missing data due to technical issues or participant noncompliance (98).
Immediate communication with participants would be beneficial as well. In particular, Ecological Momentary Assessment (EMA) prompts have been shown to be successful methods of user engagement (98). EMA involves real-time measurements of behaviors and experiences of research participants in their natural settings (99). Advantages of EMA-based communication with participants include the ability to provide feedback on image or input quality and address and edit implausible or incomplete entries (45).
Enhance user and researcher output of dietary intake
Researchers also must specify the output desired from technology tools, including transformations of raw intake data. Some tools, such as the Automated Self-Administered 24-hour (ASA24) Dietary Assessment Tool, already perform automated calculations of food and nutrient intake, including food group and supplement data (61). Researchers have an important role in determining the accuracy of calculations, decisions that should not rest with developers alone (68). Updated tools should improve the accuracy of nutrient intake calculations derived from recipe functions that prompt users to enter ingredients and preparation methods (100), and include foods, food groups, food patterns, and supplement data. Further, these should be equipped to export data in multiple file formats for both users (if desired) and researchers (46). Cross-platform compatibility—or the ability to readily harmonize data across different platforms—to accurately compare the accuracy and validity of multiple inputs, and to integrate outputs, would be an ideal outcome in current and future software/platform iterations. As mentioned, such harmonization requires the development of common data terminology as well as essential metrics that can be easily translated for a variety of end-users (e.g., researchers, clinicians, users).
Incorporate novel analysis techniques for activity tracker data
As noted, many activity trackers have built-in proprietary algorithms for measuring activity counts and translating them to minutes of activity or energy expenditure. Researchers have more recently focused on machine learning to analyze activity counts, as well as raw acceleration data (94, 101). Machine-learning algorithms create a predictive model by associating patterns of raw data based on known reference activities (102), thereby addressing concerns of physical activity as a nonlinear action and heterogeneity of developer-defined activity counts. Identifying the most relevant method of machine learning for a given application is a key consideration, and may include random forest (103), artificial neural network (104), and support vector machine (105, 106) approaches, among others. Distinctions between free-living and laboratory-based activities (94, 107) and consideration of on-body location (12) will be able to further refine estimates of expenditure.
User perspectives
Streamline the user interface of dietary assessments
Potential users of digital dietary assessments include consumers, research participants, and patients. Accordingly, developing new tools should be an iterative process that involves usability testing and improvements based on user feedback (108), which has often emphasized the importance of aesthetics, simplicity, intuitiveness, and practicality (70, 108–110). Notably, users have expressed preferences for a clean layout with no pop-ups (70) and a flat interface with a single screen for multiple recall activities such as selecting food items, recording times of meals, and specifying portion sizes (108, 111). Some users prefer a predefined list of meals or template that gives structure to the recall (108). As users make entries on the main screen, a side navigation panel with a dynamic list of entered items and options to edit them has been shown to be helpful (108). Graphics and images, such as examples of portion sizes, also could improve aesthetics, ease of use, and data validity (109).
Increase the convenience and decrease the burden of dietary assessment
Any new technology tool should be convenient and minimally disruptive to the user's lifestyle (112, 113). In research, investigator preferences for detailed, accurate data often conflict with user needs for convenient reporting methods (70). Users have noted difficulties with logging food intake in various situations such as commuting, at the workplace, and in social gatherings (113), and perceive the recording process as time-consuming and burdensome (113, 114). Hence, tools could have the option to customize the level of detail for dietary assessment (115), or different tools could accommodate specific needs of users (and, ostensibly, researchers).
As mentioned, tools could provide multiple options for data entry, such as image capture, text, selection from databases, and barcode scanning (63), and should be adaptable to different devices including smartphones and others (116), thereby catering to user preferences. Moreover, tools should allow users to either make entries during eating occasions or make all entries in 1 sitting, similar to a recall, although this flexibility may be problematic in research settings (109). Regardless of the data entry method, users should be able to edit entries at any time and review them before final submission (108, 109). Ultimately, features that make tools flexible and convenient help users adhere to long-term reporting of dietary intake.
Improve wearability, comfort, and acceptability
Comfort and acceptability are important considerations for wearable devices. The ideal wearable intake or expenditure device is portable, lightweight, unobtrusive, and aesthetically pleasing (117). Examples of current intake wearables include cameras worn around the neck (62, 118), a microcamera attached to the ear (119), a badge-like miniature camera (65), and a head-mounted camera (120). Users reported discomfort with using an ear-worn microcamera (119) or neck-worn camera (62, 96) and a preference for small, inconspicuous designs (96).
Another important consideration is creating a device that can be easily worn in the correct orientation such that users’ body shapes and postures do not affect data capture and quality (62, 121). Device placement is critical for activity trackers as well; the hip is the most widely used target owing to its proximity to the center of mass and ability to capture most movements. However, many people remove devices before sleeping or showering, resulting in poor compliance, and belts can move and twist throughout the day (92). Innovative “smart clothing” (122, 123)—although eminently wearable—suffers from similar limitations. Device placement on the nondominant wrist has garnered great interest because of its potential to increase compliance and total wear time (92), but wrist-worn trackers may fail to accurately capture energy expenditure of nonambulatory arm movements (44) and may not function properly in populations that use assistive devices (124). Hence, developing a device that is accurate, functional, and acceptable for daily continuous wear by diverse users is an ongoing challenge.
User comfort with devices in public and social settings is another important consideration, especially for image-capture tools. Notably, users have expressed feeling embarrassed or self-conscious taking images or videos of their meals in front of other people (113, 114), and wearable cameras often attract unwanted attention (125). Hence, when designing studies, investigators should weigh the benefits and limitations of attention-drawing tools (e.g., wearable cameras) compared with more discreet ones (e.g., apps).
Improve accessibility and reduce disparity
Smartphone ownership is growing rapidly worldwide, but growth has been largely restricted to younger and better-educated populations, especially in emerging economies (126). Similarly, users of health apps and health-related wearables tend to be younger, more highly educated, and more affluent than nonusers, indicating possible disparities in access to these tools (127, 128). Disparities render these tools inaccessible to older adults, individuals with lower socioeconomic status, and other populations that may have low digital or eHealth literacy [defined as “the ability to seek, find, understand, and appraise health information from electronic sources and apply the knowledge gained to addressing or solving a health problem” (129)]. It falls to developers and entities such as public institutions, nonprofit organizations, and research bodies to facilitate universal access to these tools (128, 130). A promising approach is to develop affordable tools appropriate for a wide range of reading and eHealth literacy levels (128, 131). In dietary assessment, this may entail using images of food items and portion sizes, developing educational material intended to expand nutrition knowledge, and providing assistance with interpreting results. Tools should be available in multiple languages and connected to food databases that are suited to ethnic dietary patterns. These efforts would promote equitable access and potentially support public health efforts.
A major area for improvement is accessibility for older adults. Aging is associated with changes in vision, hearing, motor function, and cognition, and many older adults have limited digital literacy (116, 132). Given these challenges, adoption rates of health apps is low among smartphone owners age 65 y and older, and downloaded health apps are shortly abandoned (133). To encourage wider adoption and long-term use, tool features should include adjustable text size and color contrast between the background, text, and images (111, 133). Buttons should be large enough for easy operation (133), and text and symbols that accompany each icon should be unambiguous and/or explicitly indicate their function, eliminating user guesswork (111). Tools should avoid using symbols that may be unfamiliar to older users with limited technology experience (111). Further, navigation structure should be consistent and simple (133), and each recording task should minimize the number of steps toward completion (111). Feedback should be available in different modes (e.g., audio, vibrotactile, visual) (133), and the tool should generate messages and warnings to prevent errors due to unintended actions (133). Overall, where possible, tool development should follow the principles of universal design (134).
Build motivation to encourage long-term use
User burnout, especially for recording dietary intake, is a challenge commonly observed in research settings (68, 110, 135). Tools should minimize user recording fatigue and make the experience enjoyable, incentivizing users to regularly maintain their records. An important motivation is the opportunity to set personal goals and monitor progress (115). Whereas researchers may seek to prevent reactivity to self-monitoring and restrict display metrics for specific hypotheses, users often prefer to see quantification of their health data and behaviors, and to identify opportunities for improvement (127). The process of self-quantifying behavior can boost an individual's confidence and self-efficacy (115, 127), which can be powerful motivation to continue using the tool long-term. Thus, adaptions of full quantification approaches to meet researcher needs may instead include reporting to users abbreviated measures such as adherence to a chosen dietary pattern (e.g., ketogenic or paleo diets), intake of certain nutrients (e.g., calcium, folate), or the balance of recorded dietary intake (e.g., healthy, neutral, unhealthy) (115). Tools may display health behaviors as visually appealing graphs or organized metrics, or in comparison with previous behaviors, personal goals, or peers (115).
Tools should also be interactive and engage users as much as possible. For example, when an app detects a lapse in dietary recording using EMA or similar approaches, it should remind and encourage users to make regular entries (76, 115, 135). Gamification could augment the entertainment value of tools (110, 131), and rewards such as coupons and discounts could be effective incentives (115). Further, a social network where users can share their results, discuss their concerns, and exchange advice could promote camaraderie (110, 127, 131) and motivate users to continue recording dietary intake for sustained periods.
Developer perspectives
Capitalize on opportunities to improve existing tools
Developers should explore technology-enhanced features that further streamline the process of recording dietary intake. Multiple modes of user entry such as text entry, database browsing, voice recording, speech-to-text, and image capture (63) can decrease user burden. Allowing the user to save favorite foods, view lists of recent items, and copy entries also saves time (46).
Innovative technology including data-driven approaches, augmented reality, and portable systems can further enhance tool features. An online 24-h dietary record or food-logging app with a data-driven algorithm might make suggestions based on user intake history (46) and prompt forgotten items (136). Applications of augmented reality, such as a ruler function embedded in a smartphone camera, would be helpful for estimating portion size (137).
Integrate food, nutrient, and food-image databases
Developers should focus efforts on maintaining continuous access of apps/software to high-quality, regularly updated food composition databases (138), including public data sets [e.g., the USDA's FoodData Central (139) and the European Food Safety Authority (EFSA) Comprehensive European Food Consumption Database (140)], licensed databases produced by research- or consumer-oriented companies, and nutrition fact labels provided by manufacturers (68). Any comprehensive database would include the most recent data on supplements, branded products, restaurant dishes, nonlabeled food items, culture-specific foods, food groups, food patterns, and product reformulations (68, 141). With new products on the market every year, updating databases remains a challenge (68, 71). The USDA Global Branded Food Products Database, a component of FoodData Central, is one such database that currently incorporates industry-provided nutrient data on labeled food items (139). Any efforts are necessarily ongoing, and should consolidate multiple sources of data, maintain a complete and comprehensive database, and standardize data coding of food intake.
Image-based methods of assessment, discussed below, require large and diverse food-image databases (142, 143). Currently, most image data sets are tailored for specific studies or types of food (142), and no publicly available, general food-image database yet exists. Some initiatives have compiled food images online (144, 145), but photos often vary in lighting, angle, and other characteristics, and may not include food volume or nutritional information (142). Going forward, an organized food-image database expanding on existing food and nutrient databases will be crucial if image-based intake assessment methodologies are to advance beyond their current nascent state.
Improve image-based methods of assessment
Active and passive image capture
As an emerging set of methods, dietary assessment using images requires further technical refinement.
Both active and passive image capture approaches have challenges with obtaining analysis-ready, high-quality images (118, 146). The ideal methods require minimal user instruction and have high tolerance for user error. However, with active capture, a primary challenge is user burden. Users must follow specific and often demanding steps for high-quality image capture (142), e.g., place food on a brightly colored dish (147) or a container with a specific shape (24), separate food items (148), take pictures at a 45–60° angle (63), and place in the frame fiducial markers (63) of known color and dimension (5, 63, 143). In the realm of cutting-edge technology, virtual reality could eliminate the need for some of these steps, including using fiducial markers (137).
Passive image capture also presents technical and privacy challenges. This approach involves a wearable device that is in continuous operation and takes images at an adjustable rate, such as the badge-like eButton (65) or neck-worn SenseCam (62, 118). Passive capture devices can result in images of suboptimal quality especially under poorly lit conditions (62, 118), tend to require considerable amounts of power (65, 117), and have limited memory capacity (117, 121). Improved devices should thus facilitate passive capture of images under a variety of environmental conditions and more efficiently use battery power and memory. Fortunately, single-unit devices with multimodal gating mechanisms (e.g., including inertial and acoustic sensors to detect chewing sounds) hold promise for preserving battery life, maintaining privacy (117), and avoiding unnecessary data collection (119, 149).
Automatic image analysis
Once captured, images can be analyzed using manual, semiautomatic, or automatic approaches (5). In a manual approach, nutritionists calculate nutritional content from an image using the user descriptions of ingredients and portion sizes, food analysis software, and food databases (150–152). However, manual approaches require extensive user and staff training, time, and resources (63). Automatic approaches use software and classification models to segment, recognize, and calculate volumes of food, thereby reducing user input (153, 154). This strategy currently faces issues with generality, because food databases in automatic approaches are often limited in terms of the number and types of food items (63). Further, the segmentation and recognition phases rely on high-quality images where all food items are clearly visible (63). As an alternative to fully automatic approaches, semiautomatic approaches use classification software that relies on cues provided by users or researchers, such as manually identifying foods or segmenting items (98, 155). However, as with fully manual approaches, the required human input in even semiautomatic approaches may be too burdensome for practical or long-term use.
Segmentation, food recognition, and portion size estimation
After retrieval of necessary images, image analysis consists of 3 main phases: segmentation of food regions and items, extraction and recognition of food properties, and estimation of portion size (5, 117, 143). Segmentation generally uses algorithms that rely on graph-based, color, or spatial representations of the images; algorithmic techniques such as region-growing and edge (100) or circle detection (156) are often used (142, 143). Accuracy decreases as the number of unique foods increases (142), and further decreases if foods are similar in color, contour, or other characteristics (143). The selection of an appropriate segmentation algorithm depends on the types of foods, characteristics of the images, automation level, and amount and type of user input.
Compared with segmentation, food recognition is more complex. The main strategies for recognition are traditional classifiers and deep learning techniques (143). Traditional classifiers extract specific visual features from the images, such as shapes, texture, and pixel color. This approach requires the researcher to manually identify the important features of the image during development (143). This information is then organized and fed into models such as support vector machines (24, 157), Bag of Features (157, 158), and K-Nearest Neighbors (24). However, these machine-learning techniques are poor at recognizing mixed foods or foods with similar appearances (143), which may have different nutritional content (5). As alternatives to traditional classifiers, deep learning techniques could eliminate the need for user or researcher input after training/development (143, 159, 160), and have performed significantly better than traditional techniques (143, 160). However, this is an emerging approach and requires further refinement.
The final step in the analysis process is portion size estimation. In fully automated image analyses, deriving a 3-dimensional quantity from a single 2-dimensional image is a challenge (142, 143). Attempts to measure volume include generating 3-dimensional shape models based on the food type (161, 162), and using multiple pictures or short videos to reconstruct the food item (25, 162). Although these techniques appear promising, they require large amounts of processing power and time (143, 147).
Create new tools
As noted, there is a dearth of cutting-edge technology tools to assess intake, especially relative to expenditure technologies, with most being digital adaptations of paper-based methodologies. Branches of optics, thermo-sensing, and other technologies are not exclusive to expenditure assessment tools and are currently underutilized for assessing dietary intake. There have been inconsistent advancements in these tools, but several promising ones include portable, handheld near-infrared (NIR) analysis sensors (163) and smart utensils with light spectrophotometers (164) that analyze the nutrient composition of foods. NIR is a long-standing technology in food testing and an established method for quantifying macronutrients in many types of food and agricultural products, notably for food adulteration (165, 166). NIR could move toward wide consumer use, but first it must be miniaturized and a database must be compiled of nutrient profiles for foods against which calibration training must occur (160). It is easy to imagine a future in which handheld food analysis tools integrating chemo-sensing, spectroscopy, optics, etc., become as common as wrist-worn activity trackers.
Ensure technical assistance is available
Although intake and expenditure assessment tools should be as intuitive as possible and require minimal user training, technical assistance for users will likely always be necessary. Developers should consider tutorials and help guides to accompany apps and devices, tailored to the computer literacy of the target audience (110, 116). Effective assistance is crucial for increasing user comfort with technology and willingness to continue user engagement as consumers or research participants (116, 167).
Ethical and Legal Considerations
Scientific interest in recording free-living individual behavior has led to rapid growth of digital health research (130) and federally funded studies on pervasive technologies (168). The ability to collect unprecedented amounts of continuous, real-time personal data has contributed to growing ethical and legal concerns (169), recently culminating in 2018 policies such as the European Union's General Data Protection Regulation (170) and the California Consumer Privacy Act (171). User privacy is a concern in research with pervasive technologies (48, 169) and, hence, technologies should comply with ethical guidelines. Researchers have found the current regulatory infrastructure and ethical guidelines to be insufficient (169), and updating them to reflect ongoing technological progress will be challenging (48). Further, standards of data security and privacy largely differ among various stakeholders such as technology companies, engineers, and scientists working with human subjects (48). Variable familiarity with novel technology or privacy risk management could also lead to variability in institutional review board (IRB) reviews of research protocols (48) and under- or overprotection of participants (47). Guidance on app development (e.g., on compliance with the Health Insurance Portability and Accountability Act) from entities such as the US Department of Health and Human Services may be an important resource (172) for responsible development of and research with new digital health tools.
Research ethics of pervasive technologies
Pertinent aspects of research ethics surrounding pervasive technologies include informed consent, participant privacy, bystander rights, and data management (48, 169). Researchers have speculated on the existence of the “privacy paradox,” where users express privacy concerns while consenting to broad terms of service and wide sharing of personal information on hundreds of apps and websites (173). This purported discrepancy between stated concerns and actual behavior may suggest users’ insufficient understanding of how their data are collected and their inability to protect their own interests (173), suggesting that obtaining meaningful informed consent may be difficult. Hence, the informed consent process should convey information, especially the potential risks of data breach and loss of privacy, in a way that is appropriate for the participant's technological literacy and knowledge about data usage (48).
As for participant privacy, sensitive data such as GPS coordinates and images should be unlinked from personally identifiable information and protected health information (174). Other strategies include providing the user with more control over data collection, such as the option to remove the recording device, a privacy or on-and-off switch (65, 121), and the opportunity to privately review and delete sensitive images (174, 175). Whereas the privacy of the research subject is prioritized, the status of bystander rights under regulations is ambiguous, especially regarding privacy in specific circumstances (e.g., home, workplace, public park) and the participant's responsibility to disclose use of a recording device (48). To prevent possible violations of privacy, past study protocols have instructed participants to confer with family and cohabitants before the start of a study and provided them with a procedure for responding to individuals who did not want to be recorded (174). As technology enhances the granularity of recorded data on free-living behavior, violation of participant and bystander privacy is a growing concern.
Finally, data management has its own set of challenges, and poor practices could increase the risk of data breach (48). Researchers should submit detailed protocols for maximizing data security, and IRBs should consult experts for best practices on technology, data security, and law (48, 174).
Emerging initiatives for ethical practices
There are recommended practices for obtaining informed consent, protecting participant privacy, respecting bystander rights, and maximizing data security. However, there are risks of harm to participants that remain unknown (176). Some initiatives have aimed to help researchers and IRB members navigate this uncertainty. One approach is directly asking research participants about their experiences with pervasive technologies, the extent to which the informed consent process reflected actual experiences, and their perceptions of data confidentiality (125). Another noteworthy initiative is Connected and Open Research Ethics (CORE), an interdisciplinary online community that connects researchers, ethicists, IRB affiliates, and other stakeholders of digital health research (47). CORE features a library and forum for posting questions and sharing resources such as examples of IRB protocols and informed consent forms (47). Such interdisciplinary resource-sharing efforts will promote awareness of the risks of digital health research and, ultimately, responsible and ethical practices.
Conclusions and Directions
Consumer preferences continue to drive developer enhancements to technologies designed to capture health-related data. Opportunities and challenges for researchers and developers abound. Many emerging tools rely on underlying research into technologies unrelated to consumer health behavior, such as artificial intelligence and machine learning, GPS, optics, accelerometry, or image recognition. Adapting these innovations for assessing dietary intake and energy expenditure requires ongoing collaboration between researchers and developers in the context of user acceptability.
Ever closer to personalized health
Knowledge of accurate dietary intake and energy expenditure is expected to provide insight into the etiology of illness and inform tailored preventive and treatment interventions (177, 178). The accelerated adoption of telehealth approaches due to the COVID-19 pandemic (179, 180) will make ongoing adoption of emerging behavior technologies even more likely in clinical practice. Such tools are already beginning to be implemented by practitioners to support personalized health recommendations (181–183).
New dietary and activity assessment tools provide opportunities for real-time monitoring and guidance. For example, providers may select nutrients or food groups of clinical interest; identify and recommend the optimal amount of exercise based on a patient's age, fitness, and health status (9, 184); and choose to display specific metrics to their patients (181). Messaging features allow health care providers to give immediate feedback or answer patient questions, as well as serve as a vehicle for brief counseling sessions, which, for example, have been shown to increase physical activity in patients (177, 185). In addition, cloud-based systems allow multiple providers to access data and coordinate care (49, 68, 76, 186). However, for practitioners to meaningfully use complex and voluminous nutritional and activity data in clinical practice, they will need efficient, targeted, and clinically effective algorithms. It is not reasonable to expect that small or even large clinical practices or hospital systems will develop their own such algorithms for use with their patient populations; these will need to be generated by researchers in conjunction with developers, with the clinical guidance of expert providers.
Collaboration among stakeholders
Developing a technology tool requires interdisciplinary collaboration and effective communication between developers and other stakeholders, be they researchers or end-users. Given their different training backgrounds, and involvement at different stages of a tool's development and application, collaborators must work toward achieving at least a baseline understanding of their respective needs, limitations, and operations. For example, most developers are trained in engineering, mathematics, and/or computational sciences, and thus researchers must gain a basic understanding of a developer's vocabulary to ensure an effective cross-discipline collaboration. Conversely, because researchers work with human subjects, developers must have some understanding of research ethics involving human subjects (187). If a commercial product is used in a study, its terms of service and privacy policy may conflict with human research protections (48). Researchers are also required to support tool development with scientific evidence, such as theories of behavior change (115) or accurate calculation of nutrient intake (68).
Researchers are similarly encouraged to understand the workflow of typical device or software development processes and challenges. Researchers, whether involved in product development or validity studies, should be prepared to navigate complex legal areas, especially intellectual property and proprietary issues (187). In particular, studies on the validity of consumer activity trackers have encountered difficulties comparing algorithms and evaluating ongoing updates to software and hardware (177, 188). This becomes particularly important in long-term research studies, which should carefully plan for technology updates, product discontinuations, etc. Going forward, with the common goal of developing valid tools, researchers and developers may have to find the delicate balance between protecting ownership rights and establishing a framework for sharing open-source code.
Finally, among the many opportunities may be some obvious ones. For example, given the many devices that can now readily detect various activity types and related physiological phenomena, it would be a natural next step to assess whether these devices may be informative with respect to assessing intake and eating behavior. That is, can these ostensible “activity trackers” also be used to assess hunger by heart rate variability, or macronutrient content of a meal given postprandial body temperature? Collaboration opportunities not just between stakeholders, but between the intake and expenditure sides of the energy balance equation, are evident.
Summary
All emerging technologies require improvements in accessibility, acceptability, and availability. In addition, as technologies become ever-more pervasive, increasing attention must be paid to ethics and responsible use. Current tools in expenditure assessment have successfully integrated diverse scientific domains to accurately capture activity and other physiological phenomena with minimal to no user input. Opportunities for improvement remain, especially with regard to capturing dietary intake, despite improvements rendered from digital adaptations of older methodologies. Although considerable advancements are occurring in image-based assessment approaches, there remains a pressing need for transformational technologies—perhaps still to be discovered—that move the field definitively beyond self-report (189) and integrate advances across the domains of chemo-sensing, spectroscopy, and many others. Such innovations will likely require “out of the box” creativity and engineering from researchers and developers; this is the present and future challenge.
ACKNOWLEDGEMENTS
We thank Adela Hruby for editorial work. The authors’ responsibilities were as follows—SKD: developed and drafted the manuscript with content contributions from AJM, ES, CBW, SD, and RPS; and all authors: participated in reviewing and editing and read and approved the final manuscript.
Notes
Editorial work for the manuscript was supported by the Institute for the Advancement of Food and Nutrition Sciences (IAFNS; Washington, DC). IAFNS is a nonprofit science organization that pools funding from industry and advances science through in-kind and financial contributions from the public and private sectors. The opinions expressed herein are those of the authors.
Author disclosures: CHK is a senior science fellow consultant at the IAFNS. All other authors report no conflicts of interest.
Perspective articles allow authors to take a position on a topic of current major importance or controversy in the field of nutrition. As such, these articles could include statements based on author opinions or point of view. Opinions expressed in Perspective articles are those of the author and are not attributable to the funder(s) or the sponsor(s) or the publisher, Editor, or Editorial Board of Advances in Nutrition. Individuals with different positions on the topic of a Perspective are invited to submit their comments in the form of a Perspectives article or in a Letter to the Editor.
Abbreviations used: CORE, Connected and Open Research Ethics; DLW, doubly labeled water; EMA, Ecological Momentary Assessment; GPS, Global Positioning System; IRB, institutional review board; NIR, near-infrared.
Contributor Information
Sai Krupa Das, Jean Mayer USDA Human Nutrition Research Center on Aging at Tufts University, Boston, MA, USA; Friedman School of Nutrition Science and Policy, Tufts University, Boston, MA, USA.
Akari J Miki, Jean Mayer USDA Human Nutrition Research Center on Aging at Tufts University, Boston, MA, USA.
Caroline M Blanchard, Jean Mayer USDA Human Nutrition Research Center on Aging at Tufts University, Boston, MA, USA.
Edward Sazonov, Department of Electrical and Computer Engineering, University of Alabama, Tuscaloosa, AL, USA.
Cheryl H Gilhooly, Jean Mayer USDA Human Nutrition Research Center on Aging at Tufts University, Boston, MA, USA; Friedman School of Nutrition Science and Policy, Tufts University, Boston, MA, USA.
Sujit Dey, Department of Electrical and Computer Engineering, University of California, San Diego, La Jolla, CA, USA.
Colton B Wolk, Jean Mayer USDA Human Nutrition Research Center on Aging at Tufts University, Boston, MA, USA.
Chor San H Khoo, Institute for the Advancement of Food and Nutrition Sciences, Washington, DC, USA.
James O Hill, Department of Nutrition Sciences, School of Health Professions, University of Alabama at Birmingham, Birmingham, AL, USA; Nutrition Obesity Research Center, University of Alabama at Birmingham, Birmingham, AL, USA.
Robin P Shook, Center for Children's Healthy Lifestyles & Nutrition, Children's Mercy Kansas City, Kansas City, MO, USA; School of Medicine, University of Missouri-Kansas City, Kansas City, MO, USA.
References
- 1. Villinger K, Wahl DR, Boeing H, Schupp HT, Renner B. The effectiveness of app-based mobile interventions on nutrition behaviours and nutrition-related health outcomes: a systematic review and meta-analysis. Obes Rev. 2019;20(10):1465–84. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Kirk MA, Amiri M, Pirbaglou M, Ritvo P. Wearable technology and physical activity behavior change in adults with chronic cardiometabolic disease: a systematic review and meta-analysis. Am J Health Promot. 2019;33(5):778–91. [DOI] [PubMed] [Google Scholar]
- 3. Eldridge AL, Piernas C, Illner A-K, Gibney MJ, Gurinović MA, De Vries JHM, Cade JE. Evaluation of new technology-based tools for dietary intake assessment—an ILSI Europe Dietary Intake and Exposure Task Force evaluation. Nutrients. 2019;11:55. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Vu T, Lin F, Alshurafa N, Xu W. Wearable food intake monitoring technologies: a comprehensive review. Computers. 2017;6(1):4. [Google Scholar]
- 5. Doulah A, Mccrory MA, Higgins JA, Sazonov E. A systematic review of technology-driven methodologies for estimation of energy intake. IEEE Access. 2019;7:49653–68. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. McClung HL, Ptomey LT, Shook RP, Aggarwal A, Gorczyca AM, Sazonov ES, Becofsky K, Weiss R, Das SK. Dietary intake and physical activity assessment: current tools, techniques, and technologies for use in adult populations. Am J Prev Med. 2018;55(4):e93–e104. [DOI] [PubMed] [Google Scholar]
- 7. Kankanhalli A, Shin J, Oh H. Mobile-based interventions for dietary behavior change and health outcomes: scoping review. JMIR Mhealth Uhealth. 2019;7(1):e11312. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Bell BM, Alam R, Alshurafa N, Thomaz E, Mondol AS, de la Haye K, Stankovic JA, Lach J, Spruijt-Metz D. Automatic, wearable-based, in-field eating detection approaches for public health research: a scoping review. NPJ Digit Med. 2020;3(1):38. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Ainsworth B, Cahalin L, Buman M, Ross R. The current state of physical activity assessment tools. Prog Cardiovasc Dis. 2015;57(4):387–95. [DOI] [PubMed] [Google Scholar]
- 10. Sylvia LG, Bernstein EE, Hubbard JL, Keating L, Anderson EJ. Practical guide to measuring physical activity. J Acad Nutr Diet. 2014;114(2):199–208. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Sazonov E, Schuckers S, Lopez-Meyer P, Makeyev O, Sazonova N, Melanson EL, Neuman M. Non-invasive monitoring of chewing and swallowing for objective quantification of ingestive behavior. Physiol Meas. 2008;29(5):525–41. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Amft O, Troster G. On-body sensing solutions for automatic dietary monitoring. IEEE Pervasive Comput. 2009;8(2):62–70. [Google Scholar]
- 13. Sazonov ES, Makeyev O, Schuckers S, Lopez-Meyer P, Melanson EL, Neuman MR. Automatic detection of swallowing events by acoustical means for applications of monitoring of ingestive behavior. IEEE Trans Biomed Eng. 2010;57(3):626–33. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Päßler S, Fischer W-J. Food intake activity detection using a wearable microphone system. In: 7th International Conference on Intelligent Environments (IE); 25–28 July, 2011; Nottingham, United Kingdom. Amsterdam (Netherlands): IOS Press; 2011. p. 298–301. [Google Scholar]
- 15. Päßler S, Fischer W-J. Food intake monitoring: automated chew event detection in chewing sounds. IEEE J Biomed Health Inform. 2014;18(1):278–89. [DOI] [PubMed] [Google Scholar]
- 16. Fontana JM, Farooq M, Sazonov E. Automatic ingestion monitor: a novel wearable device for monitoring of ingestive behavior. IEEE Trans Biomed Eng. 2014;61(6):1772–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Dong Y, Scisco J, Wilson M, Muth E, Hoover A. Detecting periods of eating during free-living by tracking wrist motion. IEEE J Biomed Health Inform. 2014;18(4):1253–60. [DOI] [PubMed] [Google Scholar]
- 18. Salley JN, Hoover AW, Wilson ML, Muth ER. Comparison between human and bite-based methods of estimating caloric intake. J Acad Nutr Diet. 2016;116(10):1568–77. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Rahman T, Adams AT, Zhang M, Cherry E, Zhou B, Peng H, Choudhury T. BodyBeat: a mobile system for sensing non-speech body sounds. In: MobiSys ’14: Proceedings of the 12th Annual International Conference on Mobile Systems, Applications, and Services; 16–19 June, 2014; Bretton Woods, NH. New York: Association for Computing Machinery; 2014. p. 2–13. [Google Scholar]
- 20. Mirtchouk M, Merck C, Kleinberg S. Automated estimation of food type and amount consumed from body-worn audio and motion sensors. In: Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing; 12–16 September, 2016; Heidelberg, Germany. New York: Association for Computing Machinery; 2016. p. 451–62. [Google Scholar]
- 21. Doulah A, Ghosh T, Hossain D, Imtiaz MH, Sazonov E. “Automatic ingestion monitor version 2” – a novel wearable device for automatic food intake detection and passive capture of food images. IEEE J Biomed Health Inform. 2021;25(2):568–76. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Martin CK, Kaya S, Gunturk BK. Quantification of food intake using food image analysis. In: Annual International Conference of the IEEE Engineering in Medicine and Biology Society; 2–6 September, 2009; Minneapolis, MN. Manhattan (NY): IEEE; 2009. p. 6869–72. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23. Pouladzadeh P, Shirmohammadi S, Al-Maghrabi R. Measuring calorie and nutrition from food image. IEEE Trans Instrum Meas. 2014;63(8):1947–56. [Google Scholar]
- 24. Zhu F, Bosch M, Khanna N, Boushey CJ, Delp EJ. Multiple hypotheses image segmentation and classification with application to dietary assessment. IEEE J Biomed Health Inform. 2015;19(1):377–88. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25. Kong F, Tan J. DietCam: automatic dietary assessment with mobile camera phones. Pervasive Mob Comput. 2012;8(1):147–63. [Google Scholar]
- 26. Kissileff HR, Klingsberg G, Van Itallie TB. Universal eating monitor for continuous recording of solid or liquid consumption in man. Am J Physiol. 1980;238(1):R14–22. [DOI] [PubMed] [Google Scholar]
- 27. Chang K-h, Liu S-y, Chu H-h, Hsu JY-j, Chen C, Lin T-y, Chen C-y, Huang P. The diet-aware dining table: observing dietary behaviors over a tabletop surface. In: Fishkin KP, Schiele B, Nixon P, Quigley A, editors. Pervasive computing. Berlin and Heidelberg (Germany): Springer Berlin Heidelberg; 2006. p. 366–82. [Google Scholar]
- 28. Papapanagiotou V, Diou C, Langlet B, Ioakimidis I, Delopoulos A. Automated extraction of food intake indicators from continuous meal weight measurements. In: Ortuño F, Rojas I, editors. Bioinformatics and biomedical engineering. Cham (Switzerland): Springer International Publishing; 2015. p. 35–46. [Google Scholar]
- 29. Papapanagiotou V, Diou C, Ioakimidis I, Sodersten P, Delopoulos A. Automatic analysis of food intake and meal microstructure based on continuous weight measurements. IEEE J Biomed Health Inform. 2019;23(2):893–902. [DOI] [PubMed] [Google Scholar]
- 30. Kelly LA, McMillan DG, Anderson A, Fippinger M, Fillerup G, Rider J. Validity of actigraphs uniaxial and triaxial accelerometers for assessment of physical activity in adults in laboratory conditions. BMC Med Phys. 2013;13(1):5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31. Hibbing PR, Lamunion SR, Kaplan AS, Crouter SE. Estimating energy expenditure with Actigraph GT9X inertial measurement unit. Med Sci Sports Exerc. 2018;50(5):1093–102. [DOI] [PubMed] [Google Scholar]
- 32. Bassett DR, John D, Conger SA, Rider BC, Passmore RM, Clark JM. Detection of lying down, sitting, standing, and stepping using two activPAL monitors. Med Sci Sports Exerc. 2014;46(10):2025–9. [DOI] [PubMed] [Google Scholar]
- 33. An H-S, Kim Y, Lee J-M. Accuracy of inclinometer functions of the activPAL and Actigraph GT3X+: a focus on physical activity. Gait Posture. 2017;51:174–80. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34. Wieters KM, Kim J-H, Lee C. Assessment of wearable global positioning system units for physical activity research. J Phys Act Health. 2012;9(7):913–23. [DOI] [PubMed] [Google Scholar]
- 35. Carlson JA, Schipperijn J, Kerr J, Saelens BE, Natarajan L, Frank LD, Glanz K, Conway TL, Chapman JE, Cain KL et al. Locations of physical activity as assessed by GPS in young adolescents. Pediatrics. 2016;137(1):e20152430. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36. Albrecht BM, Stalling I, Recke C, Bammann K. Accelerometer-assessed outdoor physical activity is associated with meteorological conditions among older adults: cross-sectional results from the OUTDOOR ACTIVE study. PLoS One. 2020;15(1):e0228053. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37. Flynn JI, Coe DP, Larsen CA, Rider BC, Conger SA, Bassett DR Jr. Detecting indoor and outdoor environments using the Actigraph GT3X+ light sensor in children. Med Sci Sports Exerc. 2014;46(1):201–6. [DOI] [PubMed] [Google Scholar]
- 38. Joyce DS, Zele AJ, Feigl B, Adhikari P. The accuracy of artificial and natural light measurements by actigraphs. J Sleep Res. 2020;29(5):e12963. [DOI] [PubMed] [Google Scholar]
- 39. Brage S, Westgate K, Franks PW, Stegle O, Wright A, Ekelund U, Wareham NJ. Estimation of free-living energy expenditure by heart rate and movement sensing: a doubly-labelled water study. PLoS One. 2015;10(9):e0137206. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40. Kuzik N, Carson V. Accelerometer Bluetooth proximity validation in parents and early years children. Meas Phys Educ Exerc Sci. 2018;22(4):287–93. [Google Scholar]
- 41. Correa JB, Apolzan JW, Shepard DN, Heil DP, Rood JC, Martin CK. Evaluation of the ability of three physical activity monitors to predict weight change and estimate energy expenditure. Appl Physiol Nutr Metab. 2016;41(7):758–66. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42. Hurter L, Fairclough SJ, Knowles ZR, Porcellato LA, Cooper-Ryan AM, Boddy LM. Establishing raw acceleration thresholds to classify sedentary and stationary behaviour in children. Children (Basel). 2018;5(12):172. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43. Bagot KS, Matthews SA, Mason M, Squeglia LM, Fowler J, Gray K, Herting M, May A, Colrain I, Godino J et al. Current, future and potential use of mobile and wearable technologies and social media data in the ABCD study to increase understanding of contributors to child health. Dev Cogn Neurosci. 2018;32:121–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44. Wright SP, Hall Brown, TS, Collier SR, Sandberg K. How consumer physical activity monitors could transform human physiology research. Am J Physiol Regul Integr Comp Physiol. 2017;312(3):R358–67. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45. Sharp DB, Allman-Farinelli M. Feasibility and validity of mobile phones to assess dietary intake. Nutrition. 2014;30(11–12):1257–66. [DOI] [PubMed] [Google Scholar]
- 46. Rusin M, Arsand E, Hartvigsen G. Functionalities and input methods for recording food intake: a systematic review. Int J Med Inf. 2013;82(8):653–64. [DOI] [PubMed] [Google Scholar]
- 47. Torous J, Nebeker C. Navigating ethics in the digital age: introducing Connected and Open Research Ethics (CORE), a tool for researchers and institutional review boards. J Med Internet Res. 2017;19(2):e38. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 48. Nebeker C, Harlow J, Espinoza Giacinto R, Orozco-Linares R, Bloss CS, Weibel N. Ethical and regulatory challenges of research using pervasive sensing and other emerging technologies: IRB perspectives. AJOB Empir Bioeth. 2017;8(4):266–76. [DOI] [PubMed] [Google Scholar]
- 49. Arens-Volland AG, Spassova L, Bohn T. Promising approaches of computer-supported dietary assessment and management—current research status and available applications. Int J Med Inf. 2015;84(12):997–1008. [DOI] [PubMed] [Google Scholar]
- 50. Cade JE. Measuring diet in the 21st century: use of new technologies. Proc Nutr Soc. 2017;76(3):276–82. [DOI] [PubMed] [Google Scholar]
- 51. Sevil M, Rashid M, Hajizadeh I, Askari MR, Hobbs N, Brandt R, Park M, Quinn L, Cinar A. Discrimination of simultaneous psychological and physical stressors using wristband biosignals. Comput Methods Programs Biomed. 2021;199:105898. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 52. Kirkpatrick SI, Baranowski T, Subar AF, Tooze JA, Frongillo EA. Best practices for conducting and interpreting studies to validate self-report dietary assessment methods. J Acad Nutr Diet. 2019;119(11):1801–16. [DOI] [PubMed] [Google Scholar]
- 53. Subar AF, Freedman LS, Tooze JA, Kirkpatrick SI, Boushey C, Neuhouser ML, Thompson FE, Potischman N, Guenther PM, Tarasuk V et al. Addressing current criticism regarding the value of self-report dietary data. J Nutr. 2015;145(12):2639–45. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 54. Nelson RO, Hayes SC. Theoretical explanations for reactivity in self-monitoring. Behav Modif. 1981;5(1):3–14. [Google Scholar]
- 55. Subar AF, Kirkpatrick SI, Mittl B, Zimmerman TP, Thompson FE, Bingley C, Willis G, Islam NG, Baranowski T, McNutt S et al. The Automated Self-Administered 24-hour dietary recall (ASA24): a resource for researchers, clinicians, and educators from the National Cancer Institute. J Acad Nutr Diet. 2012;112(8):1134–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 56. Fallaize R, Forster H, Macready AL, Walsh MC, Mathers JC, Brennan L, Gibney ER, Gibney MJ, Lovegrove JA. Online dietary intake estimation: reproducibility and validity of the Food4Me food frequency questionnaire against a 4-day weighed food record. J Med Internet Res. 2014;16(8):e190. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 57. Lemacks JL, Adams K, Lovetere A. Dietary intake reporting accuracy of the Bridge2U mobile application food log compared to control meal and dietary recall methods. Nutrients. 2019;11(1):199. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 58. Cade JE, Warthon-Medina M, Albar S, Alwan NA, Ness A, Roe M, Wark PA, Greathead K, Burley VJ, Finglas P et al. DIET@NET: best practice guidelines for dietary assessment in health research. BMC Med. 2017;15(1):202. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 59. Timon CM, Cooper SE, Barker ME, Astell AJ, Adlam T, Hwang F, Williams EA. A comparison of food portion size estimation by older adults, young adults and nutritionists. J Nutr Health Aging. 2018;22:230–6. [DOI] [PubMed] [Google Scholar]
- 60. Timon CM, van den Barg R, Blain RJ, Kehoe L, Evans K, Walton J, Flynn A, Gibney ER. A review of the design and validation of web- and computer-based 24-h dietary recall tools. Nutr Res Rev. 2016;29(2):268–80. [DOI] [PubMed] [Google Scholar]
- 61. Conrad J, Koch SAJ, Nöthlings U. New approaches in assessing food intake in epidemiology. Curr Opin Clin Nutr Metab Care. 2018;21(5):343–51. [DOI] [PubMed] [Google Scholar]
- 62. Gemming L, Rush E, Maddison R, Doherty A, Gant N, Utter J, Ni Mhurchu C. Wearable cameras can reduce dietary under-reporting: doubly labelled water validation of a camera-assisted 24 h recall. Br J Nutr. 2015;113(2):284–91. [DOI] [PubMed] [Google Scholar]
- 63. Boushey CJ, Spoden M, Zhu FM, Delp EJ, Kerr DA. New mobile methods for dietary assessment: review of image-assisted and image-based dietary assessment methods. Proc Nutr Soc. 2017;76(3):283–94. [DOI] [PubMed] [Google Scholar]
- 64. Boushey CJ, Spoden M, Delp EJ, Zhu F, Bosch M, Ahmad Z, Shvetsov YB, DeLany JP, Kerr DA. Reported energy intake accuracy compared to doubly labeled water and usability of the mobile food record among community dwelling adults. Nutrients. 2017;9(3):312. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 65. Sun M, Burke LE, Mao Z-H, Chen Y, Chen H-C, Bai Y, Li Y, Li C, Jia W. eButton: a wearable computer for health monitoring and personal assistance. In: Proceedings of the 51st Annual Design Automation Conference; 1–5 June, 2014; San Francisco, CA. New York: Association for Computing Machinery; 2014. p. 1–6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 66. Archundia Herrera MC, Chan CB. Narrative review of new methods for assessing food and energy intake. Nutrients. 2018;10(8):1064. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 67. Howes E, Boushey CJ, Kerr DA, Tomayko EJ, Cluskey M. Image-based dietary assessment ability of dietetics students and interns. Nutrients. 2017;9(2):114. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 68. Gilhooly CH. Are calorie counting apps ready to replace traditional dietary assessment methods?. Nutr Today. 2017;52(1):10–8. [Google Scholar]
- 69. Blanchard CM, Chin MK, Gilhooly CH, Barger K, Matuszek G, Miki AJ, Côté RG, Eldridge AL, Green H, Mainardi F et al. Evaluation of PIQNIQ, a novel mobile application for capturing dietary intake. J Nutr. 2021;151(5):1347–56. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 70. Carter MC, Albar SA, Morris MA, Mulla UZ, Hancock N, Evans CE, Alwan NA, Greenwood DC, Hardie LJ, Frost GS et al. Development of a UK online 24-h dietary assessment tool: myfood24. Nutrients. 2015;7(6):4016–32. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 71. Carter MC, Hancock N, Albar SA, Brown H, Greenwood DC, Hardie LJ, Frost GS, Wark PA, Cade JE. Development of a new branded UK food composition database for an online dietary assessment tool. Nutrients. 2016;8(8):480. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 72. Thompson FE, Subar AF. Dietary assessment methodology. In: Coulston AM, Boushey CJ, Ferruzzi MG, Delahanty LM, editors. Nutrition in the prevention and treatment of disease. 4th ed. Cambridge (MA): Academic Press; 2017. p. 5–48. [Google Scholar]
- 73. Evans K, Hennessy Á, Walton J, Timon C, Gibney E, Flynn A. Development and evaluation of a concise food list for use in a web-based 24-h dietary recall tool. J Nutr Sci. 2017;6:e46. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 74. Illner AK, Freisling H, Boeing H, Huybrechts I, Crispim SP, Slimani N. Review and evaluation of innovative technologies for measuring diet in nutritional epidemiology. Int J Epidemiol. 2012;41(4):1187–203. [DOI] [PubMed] [Google Scholar]
- 75. Conrad J, Nöthlings U. Innovative approaches to estimate individual usual dietary intake in large-scale epidemiological studies. Proc Nutr Soc. 2017;76(3):213–9. [DOI] [PubMed] [Google Scholar]
- 76. Allman-Farinelli M, Gemming L. Technology interventions to manage food intake: where are we now?. Curr Diab Rep. 2017;17(11):103. [DOI] [PubMed] [Google Scholar]
- 77. Shephard RJ, Aoyagi Y. Measurement of human energy expenditure, with particular reference to field studies: an historical perspective. Eur J Appl Physiol. 2012;112(8):2785–815. [DOI] [PubMed] [Google Scholar]
- 78. Steene-Johannessen J, Anderssen SA, van der Ploeg HP, Hendriksen IJM, Donnelly AE, Brage S, Ekelund U. Are self-report measures able to define individuals as physically active or inactive?. Med Sci Sports Exerc. 2016;48(2):235–44. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 79. Silfee VJ, Haughton CF, Jake-Schoffman DE, Lopez-Cepero A, May CN, Sreedhara M, Rosal MC, Lemon SC. Objective measurement of physical activity outcomes in lifestyle interventions among adults: a systematic review. Prev Med Rep. 2018;11:74–80. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 80. Gartner . Gartner forecasts global spending on wearable devices to total $81.5 billion in 2021. Stamford (CT): Gartner; 2021. [Google Scholar]
- 81. Grand View Research . Wearable technology market size, share & trends analysis report by product (wrist-wear, eye-wear & head-wear, foot-wear, neck-wear, body-wear), by application, by region, and segment forecasts, 2020–2027. San Francisco (CA): Grand View Research; 2020. [Google Scholar]
- 82. Rowlands AV, Mirkes EM, Yates T, Clemes S, Davies M, Khunti K, Edwardson CL. Accelerometer-assessed physical activity in epidemiology: are monitors equivalent?. Med Sci Sports Exerc. 2018;50(2):257–65. [DOI] [PubMed] [Google Scholar]
- 83. Ndahimana D, Kim E-K. Measurement methods for physical activity and energy expenditure: a review. Clin Nutr Res. 2017;6(2):68–80. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 84. Lyden K, Swibas T, Catenacci V, Guo R, Szuminsky N, Melanson EL. Estimating energy expenditure using heat flux measured at a single body site. Med Sci Sports Exerc. 2014;46(11):2159–67. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 85. O'Driscoll R, Turicchi J, Beaulieu K, Scott S, Matu J, Deighton K, Finlayson G, Stubbs J. How well do activity monitors estimate energy expenditure? A systematic review and meta-analysis of the validity of current technologies. Br J Sports Med. 2020;54(6):332–40. [DOI] [PubMed] [Google Scholar]
- 86. Miller DJ, Capodilupo JV, Lastella M, Sargent C, Roach GD, Lee VH, Capodilupo ER. Analyzing changes in respiratory rate to predict the risk of COVID-19 infection. PLoS One. 2020;15(12):e0243693. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 87. Berryhill S, Morton CJ, Dean A, Berryhill A, Provencio-Dean N, Patel SI, Estep L, Combs D, Mashaqi S, Gerald LB et al. Effect of wearables on sleep in healthy individuals: a randomized crossover trial and validation study. J Clin Sleep Med. 2020;16(5):775–83. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 88. Sánchez R, Villena M. Comparative evaluation of wearable devices for measuring elevation gain in mountain physical activities. Proc Inst Mech Eng P J Sport Eng Technol. 2020;234(4):312–9. [Google Scholar]
- 89. Levine A, Buono MJ. Rating of perceived exertion increases synergistically during prolonged exercise in a combined heat and hypoxic environment. J Therm Biol. 2019;84:99–102. [DOI] [PubMed] [Google Scholar]
- 90. Maughan RJ, Otani H, Watson P. Influence of relative humidity on prolonged exercise capacity in a warm environment. Eur J Appl Physiol. 2012;112(6):2313–21. [DOI] [PubMed] [Google Scholar]
- 91. Plasqui G, Bonomi AG, Westerterp KR. Daily physical activity assessment with accelerometers: new insights and validation studies. Obes Rev. 2013;14(6):451–62. [DOI] [PubMed] [Google Scholar]
- 92. Schrack JA, Cooper R, Koster A, Shiroma EJ, Murabito JM, Rejeski WJ, Ferrucci L, Harris TB. Assessing daily physical activity in older adults: unraveling the complexity of monitors, measures, and methods. J Gerontol A Biol Sci Med Sci. 2016;71(8):1039–48. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 93. Sardinha LB, Júdice PB. Usefulness of motion sensors to estimate energy expenditure in children and adults: a narrative review of studies using DLW. Eur J Clin Nutr. 2017;71(3):331–9. [DOI] [PubMed] [Google Scholar]
- 94. Farrahi V, Niemelä M, Kangas M, Korpelainen R, Jämsä T. Calibration and validation of accelerometer-based activity monitors: a systematic review of machine-learning approaches. Gait Posture. 2019;68:285–99. [DOI] [PubMed] [Google Scholar]
- 95. Trost SG. State of the art reviews: measurement of physical activity in children and adolescents. Am J Lifestyle Med. 2007;1(4):299–314. [Google Scholar]
- 96. Arab L, Estrin D, Kim DH, Burke J, Goldman J. Feasibility testing of an automated image-capture method to aid dietary recall. Eur J Clin Nutr. 2011;65(10):1156–62. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 97. Thompson FE, Subar AF, Loria CM, Reedy JL, Baranowski T. Need for technological innovation in dietary assessment. J Am Diet Assoc. 2010;110(1):48–51. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 98. Martin CK, Correa JB, Han H, Allen HR, Rood JC, Champagne CM, Gunturk BK, Bray GA. Validity of the Remote Food Photography Method (RFPM) for estimating energy and nutrient intake in near real-time. Obesity. 2012;20(4):891–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 99. Shiffman S, Stone AA, Hufford MR. Ecological momentary assessment. Annu Rev Clin Psychol. 2008;4(1):1–32. [DOI] [PubMed] [Google Scholar]
- 100. Zhang MM. Identifying the cuisine of a plate of food. Technical Report CSE 190. La Jolla (CA): University of California at San Diego; 2011. [Google Scholar]
- 101. Troiano RP, McClain JJ, Brychta RJ, Chen KY. Evolution of accelerometer methods for physical activity research. Br J Sports Med. 2014;48(13):1019–23. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 102. Narayanan A, Desai F, Stewart T, Duncan S, Mackay L. Application of raw accelerometer data and machine-learning techniques to characterize human movement behavior: a systematic scoping review. J Phys Act Health. 2020;17(3):360–83. [DOI] [PubMed] [Google Scholar]
- 103. Pavey TG, Gilson ND, Gomersall SR, Clark B, Trost SG. Field evaluation of a random forest activity classifier for wrist-worn accelerometer data. J Sci Med Sport. 2017;20(1):75–80. [DOI] [PubMed] [Google Scholar]
- 104. Trost SG, Wong W-K, Pfeiffer KA, Zheng Y. Artificial neural networks to predict activity type and energy expenditure in youth. Med Sci Sports Exerc. 2012;44(9):1801–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 105. Wang Z, Wu D, Chen J, Ghoneim A, Hossain MA. A triaxial accelerometer-based human activity recognition via EEMD-based features and game-theory-based feature selection. IEEE Sens J. 2016;16(9):3198–207. [Google Scholar]
- 106. Mimouna A, Khalifa AB, Ben Amara NE. Human action recognition using triaxial accelerometer data: selective approach. In: 15th International Multi-Conference on Systems, Signals & Devices (SSD); 19–22 March, 2018; Hammamet, Tunisia. Manhattan (NY): IEEE; 2018. p. 491–6. [Google Scholar]
- 107. Keadle SK, Lyden KA, Strath SJ, Staudenmayer JW, Freedson PS. A framework to evaluate devices that assess physical behavior. Exerc Sport Sci Rev. 2019;47(4):206–14. [DOI] [PubMed] [Google Scholar]
- 108. Simpson E, Bradley J, Poliakov I, Jackson D, Olivier P, Adamson AJ, Foster E. Iterative development of an online dietary recall tool: INTAKE24. Nutrients. 2017;9(2):118. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 109. Bucher Della Torre S, Carrard I, Farina E, Danuser B, Kruseman M. Development and evaluation of e-CA, an electronic mobile-based food record. Nutrients. 2017;9(1):76. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 110. Chen YS, Wong JE, Ayob AF, Othman NE, Poh BK. Can Malaysian young adults report dietary intake using a food diary mobile application? A pilot study on acceptability and compliance. Nutrients. 2017;9(1):62. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 111. Watkins I, Kules B, Yuan X, Xie B. Heuristic evaluation of healthy eating apps for older adults. J Consum Health Internet. 2014;18(2):105–27. [Google Scholar]
- 112. Kelly P, Marshall SJ, Badland H, Kerr J, Oliver M, Doherty AR, Foster C. An ethical framework for automated, wearable cameras in health behavior research. Am J Prev Med. 2013;44(3):314–9. [DOI] [PubMed] [Google Scholar]
- 113. Kerr DA, Dhaliwal SS, Pollard CM, Norman R, Wright JL, Harray AJ, Shoneye CL, Solah VA, Hunt WJ, Zhu F et al. BMI is associated with the willingness to record diet with a mobile food record among adults participating in dietary interventions. Nutrients. 2017;9(3):244. [Google Scholar]
- 114. Zang J, Song J, Wang Z, Yao C, Ma J, Huang C, Zhu Z, Smith LP, Du S, Hua J et al. Acceptability and feasibility of smartphone-assisted 24 h recalls in the Chinese population. Public Health Nutr. 2015;18(18):3272–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 115. Zahry NR, Cheng Y, Peng W. Content analysis of diet-related mobile apps: a self-regulation perspective. Health Commun. 2016;31(10):1301–10. [DOI] [PubMed] [Google Scholar]
- 116. Kirkpatrick SI, Gilsing AM, Hobin E, Solbak NM, Wallace A, Haines J, Mayhew AJ, Orr SK, Raina P, Robson PJ et al. Lessons from studies to evaluate an online 24-hour recall for use with children and adults in Canada. Nutrients. 2017;9(2):100. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 117. Prioleau T, Moore E, Ghovanloo M. Unobtrusive and wearable systems for automatic dietary monitoring. IEEE Trans Biomed Eng. 2017;64(9):2075–89. [DOI] [PubMed] [Google Scholar]
- 118. O'Loughlin G, Cullen SJ, McGoldrick A, O'Connor S, Blain R, O'Malley S, Warrington GD. Using a wearable camera to increase the accuracy of dietary analysis. Am J Prev Med. 2013;44(3):297–301. [DOI] [PubMed] [Google Scholar]
- 119. Pettitt C, Liu J, Kwasnicki RM, Yang G-Z, Preston T, Frost G. A pilot study to determine whether using a lightweight, wearable micro-camera improves dietary assessment accuracy and offers information on macronutrients and eating rate. Br J Nutr. 2016;115(1):160–7. [DOI] [PubMed] [Google Scholar]
- 120. Schiboni G, Wasner F, Amft O. A privacy-preserving wearable camera setup for dietary event spotting in free-living. In: 2018 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops); 19–23 March, 2018; Athens, Greece. Manhattan (NY): IEEE; 2018. p. 872–7. [Google Scholar]
- 121. Gemming L, Utter J, Ni Mhurchu C. Image-assisted dietary assessment: a systematic review of the evidence. J Acad Nutr Diet. 2015;115(1):64–77. [DOI] [PubMed] [Google Scholar]
- 122. Muhammad Sayem AS, Hon Teay S, Shahariar H, Fink PL, Albarbar A. Review on smart electro-clothing systems (SeCSs). Sensors. 2020;20(3):587. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 123. E&T Editorial Staff . Sensors woven into clothing could monitor vital signs. [Internet]. London: The Institution of Engineering and Technology; 2020. [Cited 2021 Jul 8]. Available from: https://eandt.theiet.org/content/articles/2020/04/sensors-woven-into-clothing-could-monitor-vital-signs/. [Google Scholar]
- 124. Walker RK, Hickey AM, Freedson PS. Advantages and limitations of wearable activity trackers: considerations for patients and clinicians. Clin J Oncol Nurs. 2016;20(6):606–10. [DOI] [PubMed] [Google Scholar]
- 125. Nebeker C, Lagare T, Takemoto M, Lewars B, Crist K, Bloss CS, Kerr J. Engaging research participants to inform the ethical conduct of mobile imaging, pervasive sensing, and location tracking research. Transl Behav Med. 2016;6(4):577–86. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 126. Taylor K, Silver L. Smartphone ownership is growing rapidly around the world, but not always equally. Washington (DC): Pew Research Center; 2019. [Google Scholar]
- 127. Régnier F, Chauvel L. Digital inequalities in the use of self-tracking diet and fitness apps: interview study on the influence of social, economic, and cultural factors. JMIR Mhealth Uhealth. 2018;6(4):e101. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 128. Bol N, Helberger N, Weert JCM. Differences in mobile health app use: a source of new digital inequalities?. Inform Soc. 2018;34(3):183–93. [Google Scholar]
- 129. Norman CD, Skinner HA. eHealth literacy: essential skills for consumer health in a networked world. J Med Internet Res. 2006;8(2):e9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 130. Müller AM, Maher CA, Vandelanotte C, Hingle M, Middelweerd A, Lopez ML, DeSmet A, Short CE, Nathan N, Hutchesson MJ et al. Physical activity, sedentary behavior, and diet-related eHealth and mHealth research: bibliometric analysis. J Med Internet Res. 2018;20(4):e122. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 131. Lee HE, Cho J. What motivates users to continue using diet and fitness apps? Application of the uses and gratifications approach. Health Commun. 2017;32(12):1445–53. [DOI] [PubMed] [Google Scholar]
- 132. Takemoto M, Manini TM, Rosenberg DE, Lazar A, Zlatar ZZ, Das SK, Kerr J. Diet and activity assessments and interventions using technology in older adults. Am J Prev Med. 2018;55(4):e105–15. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 133. Harrington CN, Ruzic L, Sanford JA. Universally accessible mHealth apps for older adults: towards increasing adoption and sustained engagement. In: Antona M, Stephanidis C, editors. Universal access in human–computer interaction. Human and technological environments: 11th International Conference, UAHCI 2017, Held as Part of HCI International 2017, Vancouver, BC, Canada, July 9–14, 2017, Proceedings, Part III. Cham (Switzerland): Springer International Publishing; 2017. p. 3–12. [Google Scholar]
- 134. Steinfeld E, Maisel J. Universal design: creating inclusive environments. Hoboken (NJ): Wiley; 2012. [Google Scholar]
- 135. Gilmore LA, Duhé AF, Frost EA, Redman LM. The technology boom: a new era in obesity management. J Diabetes Sci Technol. 2014;8(3):596–608. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 136. Osadchiy T, Poliakov I, Olivier P, Rowland M, Foster E. Validation of a recommender system for prompting omitted foods in online dietary assessment surveys. In: PervasiveHealth ’19: proceedings of the 13th EAI International Conference on Pervasive Computing Technologies for Healthcare; 20–23 May 2019; Trento, Italy. New York: Association for Computing Machinery; 2019. p. 208–15. [Google Scholar]
- 137. Yang Y, Jia W, Bucher T, Zhang H, Sun M. Image-based food portion size estimation using a smartphone without a fiducial marker. Public Health Nutr. 2019;22(7):1180–92. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 138. Charrondiere UR, Rittenschober D, Nowak V, Stadlmayr B, Wijesinha-Bettoni R, Haytowitz D. Improving food composition data quality: three new FAO/INFOODS guidelines on conversions, data evaluation and food matching. Food Chem. 2016;193:75–81. [DOI] [PubMed] [Google Scholar]
- 139. USDA, Agricultural Research Service (ARS) . FoodData Central. [Internet]. Beltsville (MD): USDA ARS; 2019. [Cited 2021 Feb 20]. Available from: fdc.nal.usda.gov. [Google Scholar]
- 140. European Food Safety Authority (EFSA) . The EFSA Comprehensive European Food Consumption Database. [Internet]. Parma (Italy): EFSA; 2020. [Cited 2021 Apr 1]. Available from: https://www.efsa.europa.eu/en/food-consumption/comprehensive-database. [Google Scholar]
- 141. Steele R. An overview of the state of the art of automated capture of dietary intake information. Crit Rev Food Sci Nutr. 2015;55(13):1929–38. [DOI] [PubMed] [Google Scholar]
- 142. Hassannejad H, Matrella G, Ciampolini P, De Munari I, Mordonini M, Cagnoni S. Automatic diet monitoring: a review of computer vision and wearable sensor-based methods. Int J Food Sci Nutr. 2017;68(6):656–70. [DOI] [PubMed] [Google Scholar]
- 143. Subhi MA, Ali SH, Mohammed MA. Vision-based approaches for automatic food recognition and dietary assessment: a survey. IEEE Access. 2019;7:35370–81. [Google Scholar]
- 144. Bossard L, Guillaumin M, Van Gool L. Food-101 – mining discriminative components with random forests. In: Fleet D, Pajdla T, Schiele B, Tuytelaars T, editors. Computer vision – ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6–12, 2014, Proceedings, Part VI. Cham (Switzerland): Springer International Publishing; 2014. p. 446–61. [Google Scholar]
- 145. Pouladzadeh P, Yassine A, Shirmohammadi S. FooDD: food detection dataset for calorie measurement using food images. In: Murino V, Puppo E, Sona D, Cristani M, Sansone C, editors. New trends in image analysis and processing – ICIAP 2015 workshops. Cham (Switzerland): Springer International Publishing; 2015. p. 441–8. [Google Scholar]
- 146. Rollo ME, Ash S, Lyons-Wall P, Russell A. Trial of a mobile phone method for recording dietary intake in adults with type 2 diabetes: evaluation and implications for future applications. J Telemed Telecare. 2011;17(6):318–23. [DOI] [PubMed] [Google Scholar]
- 147. Zhu F, Bosch M, Woo I, Kim S, Boushey CJ, Ebert DS, Delp EJ. The use of mobile devices in aiding dietary assessment and evaluation. IEEE J Sel Top Signal Process. 2010;4(4):756–66. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 148. Dahl Lassen A, Poulsen S, Ernst L, Kaae Andersen K, Biltoft-Jensen A, Tetens I. Evaluation of a digital method to assess evening meal intake in a free-living adult population. Food Nutr Res. 2010;54:5311. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 149. Liu J, Johns E, Atallah L, Pettitt C, Lo B, Frost G, Yang G-Z. An intelligent food-intake monitoring system using wearable sensors. In: Proceedings of the 2012 Ninth International Conference on Wearable and Implantable Body Sensor Networks (BSN); 9–12 May, 2012; London, United Kingdom. Washington (DC): IEEE Computer Society; 2012. p. 154–60. [Google Scholar]
- 150. Ptomey LT, Willis EA, Honas JJ, Mayo MS, Washburn RA, Herrmann SD, Sullivan DK, Donnelly JE. Validity of energy intake estimated by digital photography plus recall in overweight and obese young adults. J Acad Nutr Diet. 2015;115(9):1392–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 151. McClung HL, Champagne CM, Allen HR, McGraw SM, Young AJ, Montain SJ, Crombie AP. Digital food photography technology improves efficiency and feasibility of dietary intake assessments in large populations eating ad libitum in collective dining facilities. Appetite. 2017;116:389–94. [DOI] [PubMed] [Google Scholar]
- 152. Prinz N, Bohn B, Kern A, Püngel D, Pollatos O, Holl RW. Feasibility and relative validity of a digital photo-based dietary assessment: results from the Nutris-Phone study. Public Health Nutr. 2019;22(7):1160–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 153. Akpro Hippocrate EA, Suwa H, Arakawa Y, Yasumoto K. Food weight estimation using smartphone and cutlery. In: Proceedings of the First Workshop on IoT-enabled Healthcare and Wellness Technologies and Systems; 30 June, 2016; Singapore. New York: Association for Computing Machinery; 2016. p. 9–14. [Google Scholar]
- 154. Jia W, Li Y, Qu R, Baranowski T, Burke LE, Zhang H, Bai Y, Mancino JM, Xu G, Mao Z-H et al. Automatic food detection in egocentric images using artificial intelligence technology. Public Health Nutr. 2019;22(7):1168–79. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 155. Fang S, Liu C, Tahboub K, Zhu F, Delp EJ, Boushey CJ. cTADA: the design of a crowdsourcing tool for online food image identification and segmentation. In: 2018 IEEE Southwest Symposium on Image Analysis and Interpretation (SSIAI); 8–10 April, 2018; Las Vegas, NV. Manhattan (NY): IEEE; 2018. p. 25–8. [Google Scholar]
- 156. Matsuda Y, Yanai K. Multiple-food recognition considering co-occurrence employing manifold ranking. In: Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012); 11–15 November, 2012; Tsukuba, Japan. Manhattan (NY): IEEE; 2012. p. 2017–20. [Google Scholar]
- 157. Hoashi H, Joutou T, Yanai K. Image recognition of 85 food categories by feature fusion. In: Proceedings of the 2010 IEEE International Symposium on Multimedia; 13–15 December, 2010; Taichung, Taiwan. Washington (DC): IEEE Computer Society; 2010. p. 296–301. [Google Scholar]
- 158. Kawano Y, Yanai K. Real-time mobile food recognition system. In: Proceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops; 23–28 June, 2013; Portland, OR. Washington (DC): IEEE Computer Society; 2013. p. 1–7. [Google Scholar]
- 159. Christodoulidis S, Anthimopoulos M, Mougiakakou S. Food recognition for dietary assessment using deep convolutional neural networks. In: Murino V, Puppo E, Sona D, Cristani M, Sansone C, editors. New trends in image analysis and processing—ICIAP 2015 Workshops. Cham (Switzerland): Springer International Publishing; 2015. p. 458–65. [Google Scholar]
- 160. Pouladzadeh P, Shirmohammadi S, Yassine A. You are what you eat: so measure what you eat!. IEEE Instrum Meas Mag. 2016;19(1):9–15. [Google Scholar]
- 161. Xu C, He Y, Khannan N, Parra A, Boushey C, Delp E. Image-based food volume estimation. In: Proceedings of the 5th International Workshop on Multimedia for Cooking & Eating Activities; 21 October, 2013; Barcelona, Spain. New York: Association for Computing Machinery; 2013. p. 75–80. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 162. Hassannejad H, Matrella G, Ciampolini P, Munari I, Mordonini M, Cagnoni S. A new approach to image-based estimation of food volume. Algorithms. 2017;10(2):66. [Google Scholar]
- 163. Thong YJ, Nguyen T, Zhang Q, Karunanithi M, Yu L. Predicting food nutrition facts using pocket-size near-infrared sensor. In: 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC); 11–15 July, 2017; Jeju, Republic of Korea. Manhattan (NY): IEEE; 2017. p. 742–5. [DOI] [PubMed] [Google Scholar]
- 164. Huang Q, Yang Z, Zhang Q. Smart-U: smart utensils know what you eat. In: IEEE INFOCOM: IEEE Conference on Computer Communications; 15–19 April, 2018; Honolulu, HI. Manhattan (NY): IEEE; 2018. p. 1439–47. [Google Scholar]
- 165. Calibre Control . Why is NIR an effective food testing technique? [Internet]. Warrington (United Kingdom): Calibre Control International; 2019. [Cited 2021 Apr 1]. Available from: https://www.calibrecontrol.com/news-blog/2019/10/28/why-is-nir-an-effective-food-testing-technique. [Google Scholar]
- 166. Trimble S. Food-processing relies on near-infrared spectroscopy. [Internet]. Camas (WA): Felix Instruments; 2020. [Cited 2021 Apr 1]. Available from: https://felixinstruments.com/blog/food-processing-relies-on-near-infrared-spectroscopy/. [Google Scholar]
- 167. Ward HA, McLellan H, Udeh-Momoh C, Giannakopoulou P, Robb C, Wark PA, Middleton L. Use of online dietary recalls among older UK adults: a feasibility study of an online dietary assessment tool. Nutrients. 2019;11(7):1451. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 168. Dunseath S, Weibel N, Bloss CS, Nebeker C. NIH support of mobile, imaging, pervasive sensing, social media and location tracking (MISST) research: laying the foundation to examine research ethics in the digital age. NPJ Digit Med. 2018;1(1):20171. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 169. Klurfeld DM, Hekler EB, Nebeker C, Patrick K, Khoo CSH. Technology innovations in dietary intake and physical activity assessment: challenges and recommendations for future directions. Am J Prev Med. 2018;55(4):e117–22. [DOI] [PubMed] [Google Scholar]
- 170. Haug CJ. Turning the tables — the new European General Data Protection Regulation. N Engl J Med. 2018;379(3):207–9. [DOI] [PubMed] [Google Scholar]
- 171. McGraw D, Mandl KD. Privacy protections to encourage use of health-relevant digital data in a learning health system. NPJ Digit Med. 2021;4(1):2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 172. Office for Civil Rights, US Department of Health and Human Services . Resources for mobile health apps developers. [Internet]. Washington (DC): Office for Civil Rights; 2020. [Cited 2021 Jun 24]. Available from: https://www.hhs.gov/hipaa/for-professionals/special-topics/health-apps/index.html. [Google Scholar]
- 173. Bashir M, Hayes C, Lambert AD, Kesan JP. Online privacy and informed consent: the dilemma of information asymmetry. Proc Assoc Inf Sci Technol. 2015;52(1):1–10. [Google Scholar]
- 174. Nebeker C, Linares-Orozco R, Crist K. A multi-case study of research using mobile imaging, sensing and tracking technologies to objectively measure behavior: ethical issues and insights to guide responsible research practice. J Res Admin. 2015;46(1):118–37. [Google Scholar]
- 175. Thomaz E, Parnami A, Essa I, Abowd GD. Feasibility of identifying eating moments from first-person images leveraging human computation. Proceedings of the 4th International SenseCam & Pervasive Imaging Conference; 18–19 November, 2013; San Diego, CA. New York: Association for Computing Machinery; 2013. p. 26–33. [Google Scholar]
- 176. Nebeker C, Bartlett Ellis RJ, Torous J. Development of a decision-making checklist tool to support technology selection in digital health research. Transl Behav Med. 2020;10(4):1004–15. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 177. Lobelo F, Rohm Young D, Sallis R, Garber MD, Billinger SA, Duperly J, Hutber A, Pate RR, Thomas RJ, Widlansky ME et al. Routine assessment and promotion of physical activity in healthcare settings: a scientific statement from the American Heart Association. Circulation. 2018;137(18):e495–522. [DOI] [PubMed] [Google Scholar]
- 178. Chen J, Gemming L, Hanning R, Allman-Farinelli M. Smartphone apps and the nutrition care process: current perspectives and future considerations. Patient Educ Couns. 2018;101(4):750–7. [DOI] [PubMed] [Google Scholar]
- 179. Koonin LM, Hoots B, Tsang CA, Leroy Z, Farris K, Jolly T, Antall P, McCabe B, Zelis CBR, Tong I et al. Trends in the use of telehealth during the emergence of the COVID-19 pandemic — United States, January–March 2020. MMWR Morb Mortal Wkly Rep. 2020;69(43):1595–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 180. Wosik J, Fudim M, Cameron B, Gellad ZF, Cho A, Phinney D, Curtis S, Roman M, Poon EG, Ferranti J et al. Telehealth transformation: COVID-19 and the rise of virtual care. J Am Med Inform Assoc. 2020;27(6):957–62. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 181. Forster H, Walsh MC, Gibney MJ, Brennan L, Gibney ER. Personalised nutrition: the role of new dietary assessment methods. Proc Nutr Soc. 2016;75(1):96–105. [DOI] [PubMed] [Google Scholar]
- 182. Michel M, Burbidge A. Nutrition in the digital age—how digital tools can help to solve the personalized nutrition conundrum. Trends Food Sci Technol. 2019;90:194–200. [Google Scholar]
- 183. Sauceda A, Frederico C, Pellechia K, Starin D. Results of the Academy of Nutrition and Dietetics’ consumer health informatics work group's 2015 member app technology survey. J Acad Nutr Diet. 2016;116(8):1336–8. [DOI] [PubMed] [Google Scholar]
- 184. Zubin Maslov P, Schulman A, Lavie CJ, Narula J. Personalized exercise dose prescription. Eur Heart J. 2018;39(25):2346–55. [DOI] [PubMed] [Google Scholar]
- 185. Crump C, Sundquist K, Sundquist J, Winkleby MA. Exercise is medicine: primary care counseling on aerobic fitness and muscle strengthening. J Am Board Fam Med. 2019;32(1):103–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 186. Spanakis EG, Santana S, Tsiknakis M, Marias K, Sakkalis V, Teixeira A, Janssen JH, de Jong H, Tziraki C. Technology-based innovations to foster personalized healthy lifestyles and well-being: a targeted review. J Med Internet Res. 2016;18(6):e128. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 187. Buday R, Tapia R, Maze GR. Technology-driven dietary assessment: a software developer's perspective. J Hum Nutr Diet. 2014;27:10–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 188. Lobelo F, Kelli HM, Tejedor SC, Pratt M, McConnell MV, Martin SS, Welk GJ. The Wild Wild West: a framework to integrate mHealth software applications and wearables to support physical activity assessment, counseling and interventions for cardiovascular disease risk reduction. Prog Cardiovasc Dis. 2016;58(6):584–94. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 189. Ershow AG, Ortega A, Timothy Baldwin J, Hill JO. Engineering approaches to energy balance and obesity: opportunities for novel collaborations and research: report of a joint National Science Foundation and National Institutes of Health workshop. J Diabetes Sci Technol. 2007;1(1):95–105. [DOI] [PMC free article] [PubMed] [Google Scholar]