Field validation of food outlet databases: The Latino food environment in North Carolina

Abstract

Objective

Obtaining valid, reliable measures of food environments that serve Latino communities is important for understanding barriers to healthy eating in this at-risk population.

Design

The primary aim of the study was to examine agreement between retail food outlet data from two commercial databases, Nielsen TDLinx (TDLinx) for food stores and Dun & Bradstreet (D&B) for food stores and restaurants, relative to field observations of food stores and restaurants in 31 census tracts in Durham County, NC. We also examined differences by proportion of Hispanic population (</≥23.4% Hispanic population) in the census tract and for outlets classified in the field as ‘Latino’ on the basis of signage and use of Spanish language.

Setting

174 food stores and 337 restaurants in Durham County, North Carolina

Results

We found that overall sensitivity of food store listings in TDLinx was higher (64%) than listings in D&B (55%). Twenty-five food stores were characterized by auditors as Latino food stores, with 20% identified in TDLinx, 52% in D&B, and 56% in both sources. Overall sensitivity of restaurants (68%) was higher than sensitivity of Latino restaurants (38%) listed in D&B. Sensitivity did not differ substantially by Hispanic composition of neighborhoods.

Conclusion

Our findings suggest that while TDLinx and D&B commercial data sources perform well for total food stores, they perform less well in identifying small and independent food outlets, including many Latino food stores and restaurants.

Introduction

Evidence suggests that the local food environment has implications for diet and physical activity behaviors (1,6), but a lack of accurate environmental data remains problematic for much of this research (7,10). Most studies have relied on two secondary commercial data sources, Dun & Bradstreet (D&B) and InfoUSA, to characterize the retail food environment (9,14). Results from field validation studies demonstrate moderate levels of agreement between these data sources and ground-level observations (6, 9,11, 13, 15), suggesting that these data are best used in combination when characterizing the retail food environment (12, 16,18).

However, no studies have assessed the validity of Nielsen TDLinx (TDLinx), a commercial database known for its rigorous data collection and research-based outlet type classification (12, 20, 21). Unlike other commercial databases that update listings on a quarterly basis (e.g., D&B), TDLinx updates its listings on a monthly basis (19), providing an advantage in areas with rapid food outlet turnover.

Few studies have evaluated the accuracy of commercial listings to characterize the food environment in areas with high proportion of Latino residents, a fast-growing segment of the U.S. population with high risk for diet-related chronic diseases (22). While Latinos, particularly less acculturated Latinos, tend to shop at tiendas (23, 24), it is unknown how well represented tiendas and other small specialty stores are in commercial data sources. Obtaining valid, reliable measures of food environments in Latino communities is important for understanding barriers to healthy eating in this at-risk population.

Our primary aim was to examine agreement between retail food outlet data from two commercial databases, TDLinx (food stores only) and the D&B Duns Market Identifiers File (food stores and restaurants), relative to a field-based census of food stores and restaurants in 31 census tracts in Durham County, NC of varying Hispanic population composition. We also tested whether agreement differed by Hispanic composition of the census tract and by field-based classification of ‘Latino’ stores.

Methods

Geographic area

Direct field observations were conducted in 31 of the 60 census tracts in Durham County, NC, an area experiencing rapid population growth, and increase in Hispanic population (25). Census tracts were selected to obtain a balanced representation of neighborhoods with predominately Hispanic, Black, and White populations. Census tracts with the highest proportions each for non-Hispanic white (n=10), non-Hispanic black (n=10), and Hispanic (n=10) were visited. Given its population and food outlet density, we also included the census tract containing the Central Business District (CBD). The observed tracts represented 49.9% of the Durham County population.

Data sources

We obtained data for Durham County from two commercial databases for 2012: Nielsen TDLinx (referenced May, 2012) (Nielsen, New York, NY) (20) and the D&B Duns Market Identifiers File (referenced July, 2012) (Dun & Bradstreet, Inc., Short Hills NJ) (26). TDLinx uses official industry-standard definitions for food store categories when available or uses their own rigorously developed definitions supported by trade associations (e.g., Food Marketing Institute) and trade publications (e.g., Progressive Grocer), classified with a standard trade channel and sub-channel code (Supplemental Table 1). D&B uses 8-digit US Census standard industry classification (SIC) codes to categorize food outlets. TDLinx only captures food stores with $1 million in sales, while D&B does not have a criterion for sales volume and collects both food store and restaurant data.

Field census

We developed an iPad data collection program adapted from a web-based Counter Tobacco Audit Tool (27), that was preloaded with harmonized categories of food stores (from TDLinx and D&B) and restaurants (D&B); categories not found in the TDLinx and D&B databases for Durham County were classified as “Other” (Supplemental Table 1). Between July and August 2012 (4 weeks), two teams of two trained data collectors each conducted a driving census of all food stores and restaurants in 31 census tracts, recording and classifying all food outlets and collecting latitude and longitude of the locations using the iPad data collection tool.

The pairs of field data collectors (one driver, one data collector) drove all roads and streets in each census tract except private, unpaved or residential roads. All food outlets open for business and selling publicly accessible food were included and the following data were collected: name, address, latitude/longitude, currently open/closed, outlet type, and whether it was a primarily Latino outlet. Conjoined outlets (e.g., KFC/Taco Bell) were separately classified as two outlets. All field censuses took place between 9-5pm, with data collection from the car except in the CBD where due to store density, data were collected on foot.

Outlets were classified as a food store, restaurant, or both using categories and type sub-categories (Supplemental Table 1) based on characteristics observed from the outside and at the entrance of each establishment. Size of the facility, items sold, type of service provided, and posted menus (restaurants only) guided the selection of outlet type. Stores and restaurants were classified as Latino/non-Latino on the basis of store name and language of signage on windows and doors (English, mostly Spanish, both languages equally) (28).

Reliability analysis

Inter-rater reliability for identifying food outlets was conducted in the census tract that contained the CBD and a second census tract containing the largest number of food outlets. The observed proportion of agreement for both census visits in each tract (i.e., number of agreements divided by the total observations) was calculated for food stores, restaurants, and total food outlets.

Statistical analysis

Sensitivity (proportion of outlets observed on the ground that were listed in the commercial databases) was calculated to assess the level of agreement between field census and secondary data sources of food stores (TDLinx and D&B) and restaurants (D&B), with the field census considered as the “gold standard”. Food outlets from the field census and commercial databases were matched based on food outlet name and address. Sensitivity was calculated by Latino/non-Latino classification, and by Hispanic composition of the census tract (defined as ≥23.4% Hispanic population (upper quartile of distribution)). Food outlets present in TDLinx or D&B and absent from the field census were investigated using the databases’ latitude and longitude coordinates and ArcGIS and Google Earth

Results

Inter-rater reliability was 91% for all food outlets in one census tract and 79% in census tract containing the CBD, a tract with relatively high number of outlets. The data collectors identified 174 food stores on the ground across the 31 census tracts (Table 1). One hundred eleven (64%) and 95 (55%) of these food stores were listed in TDLinx and D&B, respectively. For TDLinx and D&B combined, 131 (75%) food stores observed on the ground were listed in either source. For TDLinx, sensitivity was highest for convenience stores (76%), whereas agreement in D&B was highest for grocery stores and supermarkets (65%); levels of agreement in TDLinx and D&B were lowest for small specialty stores (6% and 29%, respectively).

Table 1.

Agreement statistics for Nielsen TDLinx and D&B, by food store and restaurant type

	Sensitivity *
Food outlet category †	TDLinx	D&B	TDLinx &/or D&B
ALL FOOD STORES (n=174) ‡	111§ (64%)	95‖ (55%)	131 (75%)
Convenience stores
Conventional convenience store (n=75)	65 (87%)	43 (60%)	65 (87%)
Gas station/kiosk (n=21)	10 (48%)	7 (33%)	11 (52%)
Other (n=3)	0 (0%)	2 (67%)	2 (67%)
Total (n=99)	75 (76%)	52 (53%)	79 (80%)
Grocery and supermarket¶
Conventional supermarket (n=21)	20 (95%)	17 (81%)	20 (95%)
Supercenter (n=4)	3 (75%)	1 (25%)	3 (75%)
Superette (n=24)	10 (42%)	14 (58%)	16 (67%)
Natural/gourmet foods (n=2)	0 (0%)	1 (50%)	1 (50%)
Total (n=52)	33 (63%)	34 (65%)	41 (79%)
Small specialty food stores
Retail bread/baked goods (n=6)	0 (0%)	4 (67%)	4 (67%)
Dairy products stores (n=2)	0 (0%)	0 (0%)	0 (0%)
Fruit and vegetable markets (n=2)	0 (0%)	1 (50%)	1 (50%)
Meat and fish markets (n=2)	1 (50%)	1 (50%)	2 (100%)
Other (n=6)	0 (0%)	1 (17%)	1 (17%)
Total (n=18)	1 (6%)	7 (29%)	8 (44%)
Wholesale club (n=1)	1 (100%)	0 (0%)	1 (100%)
Other food stores (n=3)	0 (0%)	1 (33%)	1 (33%)
ALL RESTAURANTS (n=337)	n/a	228** (68%)	n/a
Counter service restaurants
Cafe/donut/coffee/bagel restaurant/ice cream parlor (n=29)	n/a	13 (45%)	n/a
Deli (n= 16)	n/a	9 (56%)	n/a
Lunchroom/Cafeteria (n=2)	n/a	0 (0%)	n/a
Snack and non-alcoholic beverage bars (n=1)	n/a	0 (0%)	n/a
Other (n= 56)	n/a	40 (71%)	n/a
Total (n=104)	n/a	62 (57%)	n/a
Fast food restaurants
Fast food chain (n=79)	n/a	64 (81%)	n/a
Fast food non-chain (n=14)	n/a	10 (71%)	n/a
Total (n=93)	n/a	74 (80%)	n/a
Sit-down restaurants††
Fine Dining (n=14)	n/a	12 (86%)	n/a
Full service & Family Restaurants (n=114)	n/a	78 (68%)	n/a
Other (n=3)	n/a	0 (0%)	n/a
Total (n= 133)	n/a	91 (68%)	n/a
Other restaurants (n=7)	n/a	1 (14%)	n/a

^*Match defined as a food store or restaurant observed in the field and listed in a secondary data source

^†Food outlets that were classified as both a food store and a restaurant by the field auditors (n=10) were included in both the food store and restaurant counts, regardless of their classification in secondary data sources

^‡One food store was not given a category during the field audit

^§Three matches were categorized as “both food store and restaurant” by the field team

^‖Eight matches were categorized as “both food store and restaurant” by the field team

^¶One food store in the grocery and supermarket category was not given a sub-category during the field audit

^**Eight matches were categorized as “both food store and restaurant” by the field team

^††Two restaurants were not given a store sub-category during the field audit

The field data collectors identified 337 restaurants (Table 1). Among these, 228 (68%) were listed in D&B. A moderately high number of counter service restaurants and sit-down restaurants were missing from D&B (40% and 32%, respectively).

Twenty-five food stores were characterized by data collectors as Latino food stores, with 20% identified in TDLinx, 52% in D&B, and 56% in either D&B or TDLinx (Table 2). The data collectors identified 26 Latino restaurants, 38% of which were listed in D&B. Agreement between the databases and the field census of food stores and restaurants did not differ substantially by Hispanic composition of census tracts (Table 2).

Table 2.

Agreement statistics for Latino food stores and restaurants, and for all food stores and restaurants by Hispanic composition of census tract, Nielsen TDLinx and D&B

	Sensitivity*
Food outlet type†	TDLinx	D&B	TDLinx &/or D&B
Food stores‡
Latino stores (all census tracts)
Grocery and Supermarket (n=9)	1 (11%)	4 (44%)	4 (44%)
Convenience stores (n=6)	2 (33%)	4 (67%)	4 (67%)
Small specialty food stores (n=8)	1 (13%)	4 (50%)	5 (63%)
Other (n=1)	0 (0%)	0 (0%)	0 (0%)
Total Latino food stores (n=25)	5 (20%)	13 (52%)	14 (56%)
All food stores by Hispanic/non-Hispanic census tracts
Hispanic§ census tracts (total food stores, n=73)	51 (70%)	40 (55%)	60 (82%)
Non-Hispanic census tracts (total food stores, n=101)	71 (70%)	52 (51%)	74 (73%)
Restaurants
Latino restaurants (all census tracts)
Counter service, not fast food (n=12)	n/a	4 (33%)	n/a
Fast food (n=1)	n/a	0 (0%)	n/a
Sit-down restaurants (n=13)	n/a	6 (46%)	n/a
Total Latino restaurants (n=26)	n/a	10 (38%)	n/a
All restaurants by Hispanic/non-Hispanic census tracts
Hispanic census tracts§ (total restaurants, n=130)	n/a	86 (66%)	n/a
Non-Hispanic census tracts (total restaurants, n=207)	n/a	114 (55%)	n/a

^*Match defined as a food store observed in the field and listed in a secondary data source

^†Food outlets that were classified as both a food store and a restaurant by the field auditors (n=10) were included in both the food store and restaurant counts, regardless of their classification in secondary data sources

^‡One food store was not given a store sub-category during the field audit

^§Hispanic census tract defined as ≥23.4% Hispanic population (upper quartile)

Discussion

Studies investigating associations between neighborhood food environments and health outcomes commonly use commercial data sources to characterize the food environment. These data are usually less expensive and more time-efficient than direct field observations, albeit of lesser quality and validity. Secondary data sources often underestimate total food outlets, resulting in inaccuracies that may bias study findings (10). Furthermore, the quality and validity of these data may differ by racial/ethnic composition of the population (7, 9, 11, 13). While others have investigated validity of food stores in rural (29, 30) and Native American communities (9), there has been little research in Latino communities and by Latino food outlets, despite the fact that Latinos are at high risk for diet-related chronic diseases.

No research to date has investigated the validity of TDLinx, a comprehensive and time-varying database of retail food stores. We found that overall agreement between field census and TDLinx data in Durham, NC was higher than that for D&B, suggesting that TDLinx may be more useful for characterizing total food stores. Additionally, we found that combining both secondary data sources improved overall accuracy by 12% (75% for both databases minus 63% for TDLinx alone). On the other hand, the comparatively low levels of agreement in TDLinx and D&B for small specialty stores (6% and 29%, respectively), suggests that smaller stores were poorly identified by both databases. Our reliability assessment in the CBD indicated agreement for 37 of 47 food stores and restaurants. We speculate reasons for relatively poor reliability included: stores were closed (n=2); lack of signage or poor signage (n=5); and human error (n=3), potentially due to high density of stores in the CBD (n=47).

In our study, the accuracy of food outlet listings in both databases did not differ considerably between Hispanic and non-Hispanic census tracts. TDLinx captured only 20% of Latino stores (compared to 68% of overall stores), while D&B performed better, capturing 58% of Latino stores (compared to 52% of overall stores). However, Latino-specific accuracy was much poorer than in the total sample. Furthermore, the added-value of using both databases for this purpose was minimal (56%), suggesting that both secondary data sources may be inadequate for characterizing local Latino food stores. However, it is possible that such food stores in Durham County, NC, an area with a new and growing Latino population (25), may not have yet become part of these commercial food listings.

A potential limitation of TDLinx is that the database only captures food outlets with ≥$1 million in sales. Latino food stores, such as tiendas and bodegas, tend to be smaller than non-Latino food stores (31) and thus more likely to be missed in the commercial databases. Latino food stores captured in D&B and absent from TDLinx had sales volumes in the hundreds of thousands, and thus did not meet the sales volume criterion of TDLinx. Tiendas and bodegas are an important food resource in Latino communities and immigrant neighborhoods, but it is unclear to what extent these stores are supportive of healthy eating.

Although data collectors were extensively trained before collecting data, the field team may have under-counted food outlets. These data were obtained approximately a month prior to data collection, during which food outlets may have opened, closed, or moved, resulting in additional variation in food outlet counts. These results may also not be generalizable for other areas with different neighborhood characteristics (e.g., communities with more long-standing Latino communities or a higher percentage of Hispanic residents). Nevertheless, ours is the first study to assess the validity of a novel commercial database, TDLinx, in Latino and non-Latino food outlets. In addition, we compare findings using D&B, a more commonly used database, which had relatively similar sensitivity compared to other studies (32).

Because of the comparatively higher agreement between TDLinx and the field census for total food stores, our study provides support for using TDLinx, alone or combined, with other commercial databases such as D&B to characterize neighborhood food stores. However, both secondary data sources poorly identified small and independent food stores, with D&B performing slightly better for Latino food stores. Investigators should be cautious of using these data to characterize neighborhoods with small and ethnic food stores, and consider supplementing secondary data sources with primary data collection if resources are available.

Abbreviations

TDLinx

Nielsen TDLinx

D&B

Dun & Bradstreet