Workshop 1: Use of technology in dietary assessment

A breakout session focusing on the use of technology in dietary assessment involved attendees posing a series of questions and making several observations in relation to an earlier presentation on the ‘Use of technology in children’s dietary assessment’.

1. The first issue discussed related back to a slide showing the process of image segmentation and estimation of food volume. Using computer software, image calibration and image segmentation allow for the volume of food on a person’s plate to be calculated. However, testing has shown that this procedure is not always accurate. Shadows created by a food item sitting on a plate make the object appear larger than it really is.

It is hoped that, in time and with further development of the software, the volume of food eaten by individuals will improve dietary assessment studies. Weight, as opposed to volume, is used to estimate the edible portion of a food item with the portion size of an apple, for example, coded as small, medium or large, and either peeled or cored. Estimation of food volume should help avoid the approximations presently involved in determining daily energy and nutrient intakes and provide a more accurate quantification of the edible portion of a food.

2. Mobile hand-held devices have evolved to function as computers, and many now comprise a camera. Potentially, this makes them ideal tools to use in dietary assessment surveys. Results from pilot studies have shown good compliance rates among participants who were asked to take images of their food using hand-held devices (that is, camera-PDA devices). When asked to use a disposable camera, adolescents willingly took more pictures of the food they were eating than was requested.

Successful development of hand-held devices should enable study participants to create a record of their food consumption over the day. The images will be stored in the hand-held devices before being sent to a central computer. Importantly, it should be possible to match the images of foods with their nutrient composition using any of the available nutrient databases.

Complications and technology developments

Testing and pilot studies have highlighted several complications in using hand-held devices to take images of food. Problems that have been encountered thus far include: composite meals, opaque drink containers, the manner in which food is often layered and mixed on people’s plates, the cultural diversity of food and non-food items mistaken for foods.

Composite meals

During the breakout session, there was interest in how the hand-held device would be able to deal with composite dishes and quantify the amount of each ingredient eaten if the meal had been made following a recipe.

Prompts on the user interface will ask the respondent to provide as much information as possible about the food they eat. For example, if an individual eats a slice of toast, a prompt will be needed asking whether butter was spread on the toast. Similarly, if oil has been used in the cooking process, a prompt will be needed. However, although the main goal in the development of this technology is to improve dietary assessment, making it more accurate and reliable, a subsidiary of the development of the technology is to reduce respondent burden and improve cooperation.

There are, however, many challenging tasks ahead. For example, the dimensions of a screen on camera-PDA devices are often small, limiting the size of the drop-down menus that will provide the prompts. The interface has to be user friendly so that all sectors of the society involved in dietary surveys can use it.

It is possible that composite meals such as ready meals and those eaten from containers could be dealt with using the UPC symbols on the food containers. These symbols identify the product in the container, but as of yet the codes are not linked to a nutrient database. Getting to this stage is a major project by itself, and even if this point is reached, quantifying the amount of food that might be left over is a further complication.

Drink containers

One of the most notable challenges faced in the development of a mobile food record has been how to deal with opaque drink containers.

Measuring the volume of liquid in a container is not easy. One possibility is to have the respondent make a mark on the outside of the container he is drinking from before taking an image to show how much liquid is present in the holder. As noted above, however, one of the primary objectives in the development of this technology is to reduce respondent burden. An individual is unlikely to have a marker pen in hand if he is having a meal in a restaurant.

A second alternative to dealing with opaque drink containers is to transfer the beverage to a clear container. Again, this does not lessen the hindrance to the study participant and may not be possible in a restaurant setting. Furthermore, some drinks are not conducive to being transferred from one container to another, and there is the possibility of spillages.

These complications could be overcome by having the mobile device provide a prompt when an image of an opaque drink container is taken. The individual would be directed to make a mark on the screen of the PDA showing the level to which the container is filled.

Cultural diversity of food

The wide range of foods now available in the marketplace dictates that an electronic database be created that links to the nutrient composition of all available foods. The creation of a database that is up to date, reflects what is available for consumption and provides data on specific food constituents of interest are ongoing challenges. The images of food taken by people will need to be matched with those in the digital database.

Some countries (for example, the USA) already have existing databases providing nutrient information on many different foods that are available. Other countries have yet to develop such a system. Even where a comprehensive database is in place, challenges constantly present themselves. New products are continually entering the marketplace, while the profile of foods is changing with new crops being grown and with changing farming practices.

Meal presentation

People like to mix different food items on their plates. Any device to assess what was eaten needs to be able to detect where different foods are overlapping and how much of a particular food is present on a plate. During development, free-living individuals may have to separate their foods before taking an image. However, the ultimate aim of using technology is to capture habitual food intake and not cause a change in behaviour.

Food identification

A problem has also been encountered during testing devices when the software has identified an object as a food item when in fact it has been a foreign object. For example, a rubber ball might be mistaken for an apple. The implications that this could have on the results of a diet survey are clear, and yet some individuals might take images of foods they do not eat and thus provide an inaccurate account of their daily food intake.

A further obstacle is that an image taken of a food does not account for the fact that it may be a low-fat or reduced-sugar form of the food. It is likely that the respondent will have to provide some additional information in this situation, such as where the dairy products are being consumed. The same is true with meat. It is unlikely that the software will be able to identify the type of meat that is being eaten. As a consequence, some user input will often be required.

Capturing images

The use of an appropriate fiducial marker to assist in the estimation of food volume is also being tested. The fact that subjects will have to carry the markers around with them needs to be considered. Finding an appropriate object to position alongside the food before an image is taken, as well as determining what sort of objects people are willing to carry around with them (and remember to take), still requires further research.

Furthermore, study participants will require guidance on how to use their camera-PDAs effectively and take images that are of good quality. The angle at which the device is held is important. Shadows can result if images are not taken at the correct angle. Lighting and the colour of the crockery can also have an impact on the quality of the image. Working to resolve problems such as these are all part of the development process in using technology to assist dietary assessment.

Technology issues

The use of technology in dietary assessment is a pioneering field of research. As technology develops, it might be possible to overcome some of the complications that have been encountered. The project requires teams and sub-committees from a number of different disciplines working to resolve the issues.

Questions still remain regarding which operating system to use and whether it would be beneficial to use a mobile phone provider.

Ideally, the aim is to compare the results of energy intake from dietary assessment studies using technology with the gold standard Doubly Labelled Water technique.

Summary

Where we are

In summary, the breakout session highlighted the potential that exists for technology to improve the accuracy of dietary assessment studies. Existing methods, although being well established and having some benefits, also have limitations. Furthermore, filling in diet diaries, food-frequency questionnaires and carrying out 24-h recalls can be burdensome and dull.

It is hoped that using a mobile hand-held device such as a mobile phone will improve the accuracy of food records by providing a more exact description of the amount and type of food eaten. It is also expected to improve subject compliance. Pilot studies have shown promising results, with participants often taking pictures of the food they were eating for more days than was required.

It is hoped that the device will produce data that are superior to current information from study participants about food habits, and that people of all ages and backgrounds will be able to use the technology. Initially, adolescents and those <50 years of age are likely to be the target population. The technology might not be as appropriate for the elderly or for those with particular learning difficulties.

Challenges

A number of challenges that have to be overcome before hand-held devices can be used in diet surveys were also discussed during the breakout session.

For example, how this method is going to deal with people eating extra portions of food. Pilot studies are needed to ascertain whether the camera can be used to capture the habits of people when they eat, for example, helping themselves from plates and dishes positioned in the centre of the table. It is important that individuals do not change the way they eat or what they eat.

Keeping respondent burden to a minimum is also a challenge. Having too many prompts asking for details such as whether milk is added to a cup of tea or coffee, or whether a food item contains low fat begins to undermine one of the objectives of developing these devices.

Furthermore, prompts given on the interface of the device have to be clear and user friendly. The extent of effort individuals are willing to make in recording and in what they can tolerate in the instructions needs to be evaluated.

Advances

Teamwork and collaboration with groups that might not normally be involved in working on nutrition studies is a characteristic of the ongoing work that is being carried out. Food is not usually considered in terms of volume, and a great deal of time has been spent on conversion factors to weights. Working with groups from engineering, computer and information technology sectors, a method that automates the process of calculating the volume of different foods is being developed.