Combining Ground-Truthing and Technology to Improve Accuracy in Establishing Children's Food Purchasing Behaviors

Abstract

Developing nutrition-focused environmental interventions for youth requires accurate assessment of where they purchase food. We have developed an innovative, technology-based method to improve the accuracy of food source recall among children using a tablet PC and ground-truthing methodologies. As part of the B'more Healthy Communties for Kids study, we mapped and digitally photographed every food source within a half-mile radius of 14 Baltimore City recreation centers. This food source database was then used with children from the surrounding neighborhoods to search for and identify the food sources they frequent. This novel integration of traditional data collection and technology enables researchers to gather highly accurate information on food source usage among children in Baltimore City. Funding is provided by the NICHD U-54 Grant #1U54HD070725-02.

Background

Challenges in Children's Food Purchasing Behaviors

There is an increasing body of evidence indicating that an individual's food environment may aid in determining his or her likelihood of developing obesity and other diet-related chronic diseases.^1,2,3 Low-income neighborhoods tend to have fewer fresh produce options and more fast foods and calorically dense packaged goods^4,5, thus influencing what types of foods are purchased by the residents of these neighborhoods. In Baltimore City, children spend approximately $4 each day and $1 per visit on trips to corner stores and carryouts - purchasing unhealthy snack foods and sugar-sweetened beverages³ - but little is known about where exactly they are making these purchases. For these children, corner stores and carryouts represent significant hubs for their food purchasing and consumption behaviors. For those unfamiliar with these unique structures of the urban environment, corner stores are small, local shops that sell snack foods, milk, bread and sodas. Carryouts are similar to large fast food chains, but tend to be smaller, primarily family-owned operations that sell prepared foods, such as fried chicken wings, lake trout (a Baltimore specialty), or Chinese foods.

A key methodological challenge lies, therefore, in developing a method for allowing children to accurately recall where their food purchases are made.^6,7 Although there is considerable data on children's perspective of their food choices⁸ and what they are eating^9,10, there remains a gap in the literature when it comes to documenting, with heightened accuracy, their food purchasing behaviors. This information is essential as a first step in tailoring interventions aimed at a child's food environment, such as determining the popular neighborhood stores where educational signage and/or fresh produce and healthy packaged goods ought to be placed to provide maximum impact.

What Is Ground-Truthing?

Ground-truthing is the method of physically verifying various qualities of neighborhoods. It has often been characterized as both a painstaking and costly process.^11,12,13 Regardless, ground-truthing is considered the gold standard for measuring and evaluating a food environment, thus providing incontrovertible evidence of what is and isn't present in a particular neighborhood at a specific point in time. Given this information, the question then arises: how can we incorporate new technological advances to maximize the advantages of this method, minimize costs, and, in so doing, accurately measure children's food purchasing behaviors?

Introduction to the Baltimore Food Source Software Development

The Baltimore Food Source Software (BFSS) program sought to address both of these challenges - by using the ground-truthing methodology and the development of a tailored software - in order to create a visual database of food sources in Baltimore City that would aid children in identifying exactly where they purchase their food.

The research questions that were addressed are as follows:

• 1)How can ground-truthing and tablet personal computer (PC) technology be used to create a comprehensive visual database of the food environment in a given neighborhood?
• 2)Will the accuracy of food sources recalled by children improve with the use of the BFSS and its visual database?

Methods

This work was conducted as part of the formative research for the B'More Healthy Communities for Kids (BHCK) study, a multilevel intervention trial that seeks to improve the food environment in low-income neighborhoods in Baltimore City. The intervention works in neighborhoods immediately surrounding specific recreation centers and was approved by the Johns Hopkins Bloomberg School of Public Health Institutional Review Board (IRB #00004203).

The process of developing and testing the BFSS program was conducted in 4 phases.

The goal was to create a visual database that would facilitate a child in recalling where they typically purchase food; through the use of photographs and a user-friendly interface.

Phase 1 Ground-Truthing to Develop a Database of Food Sources

In order to effectively map the food environment, Google maps (www.maps.google.com) were used to determine the main roads that lay approximately a half mile around each of the fourteen target recreation centers – the “zones” previously set as part of the larger BHCK intervention. After establishing these boundaries, a strategy was created for walking the blocks within each area in order to maximize efficient coverage. Whenever a food source (carryout, corner store, convenience store, restaurant, grocery store, market or bar) was found on a block, one member of the data collection pair would capture on a Lenovo A1-07 tablet PC the necessary identifiable information – the awning color, street address, cross street, store name, GPS location, photograph and a listing of food items advertised on the exterior of the source. These particular characteristics were recorded because formative research with neighborhood children identified them as the most memorable features of a food source. Whenever possible, a photograph of the front of the food source (to aid in clear identification) and the GPS coordinates for the source were taken directly across the street.

Thoroughness of canvassing was double-checked by ensuring that each of the boundary roads was reached and mapped. For the duration of the ground-truthing process, only one tablet was used. Most zones were covered over the course of a single day, which typically took between 5 and 6 hours to complete. In some instances, it was difficult to determine whether a food source was still operational, as many carry-outs are open only in the evenings or on weekends. All sources that were believed to be operational were mapped. If the street number was missing from the store front, the store address was searched for and updated using Google maps. If the Google search yielded no results, the address was surmised based on the numbering pattern of the street and the nearest available street number.

Phase 2: Cross-Checking and Mapping the Food Source Data Using ArcMap GIS

Once the ground-truthing process was completed, the data for that area's food sources were uploaded to an online storage system and converted into a GIS-friendly shape file. Using version 10 of ArcMap GIS¹⁴, the shape file was then further converted to individual data points – each representing a single food source - and mapped onto a 1989 North American Mercator projection of the United States, overlaid with the 2010 census tract information for Baltimore City.

Phase 3: Tablet Software Development

We created the Baltimore Food Source Software (BFSS) program using the data derived from the ground-truthing process. The goal was to create a simple method for searching the food source database and pulling up photos that had been taken of each source, thereby making it easy for the children to describe and visually identify where they commonly purchase food. The BFSS has search fields for each category that was originally entered during the ground-truthing process: store name, food source type, address, cross street, awning colors, and the foods advertised externally. The software also uses the GPS data for each food source and the coordinates of the identified recreation centers to enable the food sources to be listed in terms of distance from the recreation center where the child was recruited. This listing is accompanied by a photo of the food source, which can be expanded to full screen so that the child can closely examine the photo to be sure that it matches the store described. Once the correct food source is identified, that food source is “selected”, and the selection is recorded in the software, enabling totals to be calculated at a later time of the most popular food sources around each recreation center.

Phase 4: Piloting the Baltimore Food Source Software

Piloting of the BFSS program was conducted with a convenience sample of 12 children, ages 10 -14, from 3 of the 14 recreation center zones. These children were identified by a group of BHCK data collectors and had given their assent for participation, in addition to the consent given by their primary caretaker. Both the assent and consent forms had been previously approved by the Johns Hopkins Bloomberg School of Public Health Institutional Review Board (IRB #00004203). The children were interviewed by a trained data collector at the recreation centers and asked to identify the food sources that they most commonly visit with this prompt:

“What are the places that you visit most often when you have your own money to spend on food?”

Once a child had a food source in mind, the source was either named or described and, if there was insufficient information to match the food source without use of the software (i.e: an exact address or the correct name of the source), then the descriptors the child used were searched for and the food source was verified using the features of the BFSS program. After obtaining the descriptors and identifying each of the five food sources, the children were then asked:

• 1)What do you like most about the tablet program?
• 2)What do you like least about the tablet program?
• 3)Is there anything about the tablet program that should be changed?
• 4)Would this interview have been different if we did not have the tablet?
• 5)If yes to Question 4, how?

Each child was given a $20 gift card to Walmart for their participation in this piloting process. The pilot interviews each took approximately 30 minutes.

The accuracy of child recall during piloting was calculated in two different ways. First, accuracy of recall with assistance from the BFSS program was determined by dividing the total number of unique food sources named by the children (TUFS) by the number of food sources that were correctly found by the BFSS program or “correct sources” (CS). This fraction was then multiplied by 100 to create a percentage. Thus, we can represent the number of food sources correctly identified by the BFSS program in this way: CS/TUFS * 100 = % TUFS correctly found by the BFSS program.

The second method of calculating accuracy of recall was intended to measure presumed accuracy of food source recall had the children not been provided access to the BFSS program. This value was determined by dividing the total number of unique food sources (TUFS) described by the children with those that were found via a widely available internet search engine, such as Google maps or Bing, (IFS or “internet food sources”).

Thus, we can represent the number of food sources correctly identified using only child description and publicly available internet search methods in this way: IFS/TUFS * 100 = % TUFS correctly found using only child recall and the results of a public internet search.

These search engines are commonly used as a secondary validation measure in determining neighborhood qualities, with varying success.^15,16,17 However, as our study aims to go a step further by pinpointing the specific food sources that children in each neighborhood frequent, it was essential for us to see if our tailored BFSS program truly aided in boosting the number of food sources that could be described and then correctly identified by children.

When developing the calculations for the IFS numerator, it was found that only when a child gave either the exact name of a food source or the exact address of a food source, was there enough information to successfully find that food source through public internet search engines. Other descriptors, such as awning color, recreation center zone and type of food sold, were only useful as search terms within the BFSS program. Additionally, many of the sources found through public internet search engines did not have a clear corresponding photograph, while the food sources described and searched for in the BFSS program presented children with large, clear photographs that swiftly aided in positive identification.

Results

Food Source Mapping

The process of ground-truthing and mapping using the tablet and ArcMap GIS took approximately 120 hours and yielded 333 separate food sources, with an average of 22 food sources within each recreation center zone.

Accuracy of the Baltimore Food Source Software

The pilot of the BFSS program involved 12 children from three different recreation center zones. Six interviewees were 11 years, two were 10 years, two 13 years, and the remaining age groups (12 and 14 years) each had one interviewee. Sixty food (60) sources altogether were described by the children. However, our goal was to determine the total number of unique food sources described by the children. The intention of this was to eliminate undue inflation of the BFSS program's efficacy by counting as “correctly identified” the same food source across interviews. In order to create a more accurate representation of the BFSS program's efficacy, repetitions of the same food source by different children were discounted, leading to a total of thirty-eight (38) unique food sources. Of the 38 unique sources, thirty-two (32) were correctly found using the BFSS program (CS/TUFS * 100), resulting in an accuracy rate of 84%. This means that the BFSS program correctly identified 84% of the food sources described by the children. Of the six food sources that were not successfully identified by the BFSS program, one lay within the half-mile recreation center zone, but had become operational only after the ground-truthing process had ended, one was named and found by Google maps, but lay outside the recreation center “zone”, three were named, but unable to be found through any search means (BFSS, Bing or Google maps), and one simply had insufficient information with which to conduct a search of any kind.

Of the 38 unique sources identified, 12 were called by their exact name and thus able to be successfully found using either Bing or Google maps. This indicates that these 12 food sources might have been found without use of our tailored BFSS program and through reliance solely on child recall and publicly available search engines. Thus, the presumed accuracy of pinpointing food sources frequently visited by the children had the tablet and BFSS program not been utilized (IFS/TFS * 100) was estimated at 31%

Children's Perceptions of the Baltimore Food Source Software

The children who were interviewed for the pilot of the BFSS program all had similar comments regarding their experience of the BFSS program and the tablet in general. The children all seemed to enjoy the software's search feature and their ability to look at photos of the food sources. The children also felt that the interviews would have proceeded differently had the software not been used–with all twelve explaining that the ability to see pictures helped them to correctly identify the food sources that they were describing.

Discussion

This is the first paper to examine the intersection of conventional methodologies (such as ground-truthing) with tailored technological innovations in order to gather accurate self-reported data on children's food purchasing behaviors. Since children are a population that often struggles with memory and recall¹⁸, the 84% accuracy rate of food source identification, aided by the BFSS program, is a notable accomplishment.

In terms of the pilot interviews conducted for the BFSS program, a major strength lay in giving the interviewed children an opportunity to provide feedback on their experience of the tablet. These critiques will be useful as the database expands and the software is revised. Additionally, the pictorial nature of the search results enabled children to recall and identify food sources that they frequented but had entirely forgotten about - only recalling the food sources after seeing them during a separate search. This further supports the suggestion that the use of the BFSS program might capture many more frequented food sources than through the child's unaided recall abilities and a traditional search engine mechanism.

Within the larger scheme of the BHCK intervention, the BFSS program has been uploaded onto eight different tablets and will soon be used by BHCK data collectors during their interviews with over 700 low-income children in Baltimore City. Thus, the BFSS program is a major asset in tailoring BHCK's intervention. The results found by the BFSS program will allow researchers to identify specific stores that are most commonly frequented by the children and to place healthy foods and produce in those locations; thereby providing maximum exposure for the targeted age range.

Both the process of ground-truthing and the development and pilot testing of the BFSS program yielded important insights that expand the potential utility of technology in studies aiming to gather information on vulnerable populations. Ground-truthing was indeed arduous, but the data gathered on the types and array of food sources within each recreation center zone was consistent with previous literature on the Baltimore City food environment^19,20. In addition, the high accuracy of the BFSS program in finding vaguely described food sources is encouraging for future research endeavors which seeks to gather specialized information from groups who are often challenged by recall inaccuracy.

Limitations

A notable limitation of this study's ground-truthing process was that only one team went out in the field. It would be preferable to have two teams – one to go out and collect the main food source data and one to retrace the first team's footsteps to ensure that no food sources were missed. Additionally, the timing of the ground-truthing and the absence of regular updates to the database meant that food sources opened after the ground-truthing process was completed were not captured. Ideally, ground-truthing should be repeated every 4 - 6 months to ensure an up-to-date database.

There were also some limitations that accompanied the piloting of the BFSS program. Firstly, pilot testing was not conducted with children from all of the recreation centers and therefore provides a potentially incomplete view of the tablet's accuracy. Ideally, the tablet would be tested with more children (>25) in all of the recreation centers to improve generalizeability. Additionally, any stores frequented by children that lay outside of the half-mile zone are not in the database and, as such, cannot be identified using the BFSS program.

Conclusions

The tailored, visual approach of the BFSS program was largely accurate in capturing the neighborhood food sources frequented by Baltimore City children. The encouraging results of this approach indicate that future implications for the usage of a tablet or tablet-related software in specialized interventions are ripe for innovation.

Decades of studies have shown that spatial, episodic and executive control memory continually improve with age, but that these domains of memory are the most impaired in pre-pubescent children.^21,22 Studies have also been undertaken which seek to develop new interview and visualization techniques which aim to compensate for such recall deficiencies in children.^23,24 As the need for improving the urban food environment for children grows in impact and urgency, tailored software programs such as BFSS that greatly increase the accuracy of child recall could prove useful. The ground-truthing methodology, overall development, and piloting of the BFSS program also adds to the current body of literature addressing the specialized research tools needed for navigating the limitations of this population.

Looking past the utility of the BFSS program, tailored tablet software could be an excellent method for obtaining visual identification of other important food-related information, such as the specific foods children purchase once inside corner stores or the places in neighborhoods where children congregate to eat. It could also be used to expand upon^25,26 or develop new child-friendly interactive nutrition education tools. Overall, the usage of ground-truthing and the results from developing and piloting the Baltimore Food Source Software program appears to successfully aid researchers in determining where low-income children in Baltimore City get their food.

Figure 1.

Process of ground-truthing, mapping, and the Baltimore Food Source Software development and testing

Figure 2.

Food source type by percentage of total food sources identified during groundtruthing (n = 333).

Figure 3.

Food Sources and Recreation Center Zones Mapped for the BHCK Study*

Table 1.

Accuracy of the Baltimore Food System Software during Piloting with Baltimore City Children

	Number of Food Sources Correctly Identified	Number of Food Sources Named	Accuracy
With Tablet PC – Total Food Sources1	49	60	82%
With Tablet PC – Unique Food Sources2	32	38	84%
Without Tablet PC (Child Recall Alone)	12	38	31%

¹Refers to every food source described in the twelve interviews with children

²Excludes repetition of food sources across the twelve interviews