Measuring Availability of Healthy Foods: Agreement Between Directly Measured and Self-reported Data

Abstract

A major challenge in studies of the impact of the local food environment is the accuracy of measures of healthy food access. The authors assessed agreement between self-reported and directly measured availability of healthful choices within neighborhood food stores and examined the validity of reported availability using directly measured availability as a “gold standard.” Reported availability was measured via a phone survey of 1,170 adults in Baltimore, Maryland, in 2004. Directly measured availability was assessed in 226 food stores in 2006 using a modified Nutrition Environment Measures Survey in Stores (NEMS-S). Whites, college-educated individuals, and higher income households (≥$50,000) had significantly higher reported and directly measured availability than did blacks, those with less education, and lower income households. Persons in areas with above average directly measured availability reported above average availability 70%–80% of the time (sensitivity = 79.6% for all stores within 1 mile (1.6 km) of participants’ homes and 69.6% for the store with the highest availability within 1 mile). Those with below average directly measured availability reported low availability only half the time. With revisions to improve specificity, self-reported measures can be reasonable indicators of healthy food availability and provide feasible proxy measures of directly assessed availability.

There is growing interest in understanding whether availability, affordability, and quality of healthier foods are equitably distributed across areas and how these 3 dimensions of access shape dietary behaviors related to a range of chronic diseases (1–3). Neighborhood healthy food availability has been shown to be cross-sectionally associated with residents’ dietary patterns and behaviors (1, 4, 5) and with chronic disease-related outcomes such as obesity, hypertension, and diabetes (1, 6, 7). Healthy food is also typically less available in minority, rural, and poor neighborhoods, although not always consistently (1–3). These results suggest that differential access to healthy foods could be an important contributor to socioeconomic and racial/ethnic diet-related health disparities.

A major challenge in studies of the impact of the local food environment is the measurement of access. Improved measures are necessary to assess whether changes in food environments relate to changes in diets. Measurement approaches to characterize the environment include counting the number of supermarkets (8–10), directly measuring store inventories using validated store assessment tools (11, 12), and surveying residents of neighborhoods (5). Of the 3 measures, direct assessment of stores is likely the closest to a “gold standard” and the most resource intensive to implement. Counting supermarkets is logistically more feasible but relies on the potentially incorrect assumption that only supermarkets offer healthy foods and that the availability and quality of foods offered by supermarkets are invariant across neighborhoods (12–14). In contrast, survey measures are often easy to implement and can be used to obtain information from residents on various aspects related to healthy food access. For these reasons, assessments of the extent to which survey measures are reasonable proxies for directly measured healthy food availability remain an important epidemiologic and public health research question.

Although explored with respect to the physical activity environment (15–19), few food environment studies have evaluated agreement between survey and other measures of availability. The few that have done so indicate that food environment measures are associated but not necessarily identical (20–22). To date, only 1 study (22) has compared survey measures of food access with direct measures of what is available in stores. However, the measures used were designed specifically for an urban population disproportionately affected by food insecurity.

The primary objective of these analyses was to assess the agreement between self-reported availability of healthful choices within neighborhood food stores and systematic direct measures of availability in a general population sample by using validated and standardized instruments. The secondary objective was to examine the validity of self-reported availability by using directly measured availability as a gold standard. In addition, we examined whether associations differed by demographic characteristics.

MATERIALS AND METHODS

Survey-reported availability of healthy foods

Reported availability of healthful choices within neighborhood food stores was measured through a self-administered survey of 1,746 adults in 208 census tracts in Baltimore, Maryland. The survey was conducted between January and August of 2004 and was a random digit dialing sample (23). The final response rate for the survey was 46.5%, and the sample was approximately representative of the areas from which it was drawn (23).

Survey participants were asked to think of their neighborhood as the area within a 20-minute walk or a mile (1.6 km) from their home and to indicate the extent to which they agreed with the following statements regarding the availability of healthy foods in their neighborhood of residence:

1. A large selection of fruits and vegetables is available in my neighborhood.

2. The fresh fruits and vegetables in my neighborhood are of high quality.

3. A large selection of low-fat products is available in my neighborhood.

Assessments of healthy food availability within 1 mile of residents’ homes were previously found to be associated with better dietary quality (4, 5). Questions were graded on a 5-point Likert scale (0 = strongly agree to 4 = strongly disagree). Cronbach’s alpha and test-retest reliability for the 3 items were high (α = 0.78; ρ = 0.69, 95% confidence interval: 0.57, 0.77) (23). Responses were summed to obtain an overall score to measure reported neighborhood availability of healthy foods. Scores were standardized to a 100-point scale, with higher scores indicating greater reported availability of healthy foods (original scale ranged from 3 to 15, with a mean and standard deviation of 9.9 and 3.1, respectively).

Directly measured availability of healthy foods

An initial list of 365 food store establishments in 159 census tracts in Baltimore, Maryland, was purchased from InfoUSA, Inc. (Papillion, Nebraska), in 2004. These spatially contiguous tracts were part of a larger study, the Multiethnic Study of Atherosclerosis (MESA) Neighborhood Study (12), and contained the majority (71%) of the study sample. The area encompassed 112 census tracts in Baltimore City and 47 in Baltimore County (12). Food stores included supermarkets, grocery stores, and convenience stores (9).

Because of inaccuracies in commercial directories (24), improvements were made by comparing the list with 2006 local phone books and 2006 local health department food license records and by driving through the main thoroughfares of all study neighborhoods between May and July of 2006. Ten new stores verified as open for business in 2004 per the store owner were added (8 supermarkets and 2 grocery stores). Of the 375 stores in the improved list, 86 (23%) had closed for business permanently at the time of data collection (3 supermarkets, 14 convenience stores, and 69 grocery stores), and 42 (11%) were liquor stores and also food places and warehouses that did not sell directly to the public. The majority of the closed stores were located in downtown minority and poor neighborhoods where many other retail stores were present. In addition, 21 food store managers (9%) of corner stores located in predominantly black and lower-income city neighborhoods refused to be part of the study, leaving 226 stores.

The availability of healthier foods in the 226 stores was directly assessed in 2006 by using an adapted version of the Nutrition Environment Measures Survey in Stores (NEMS-S) (12). The original instrument showed high interrater reliability (kappas of 0.84–1.00) and test-retest reliability (kappas of 0.73–1.00) (11). Quality was assessed in the original instrument but not in these analyses because of the limited variability in quality observed for fruits and vegetables (11). Trained research assistants visited the 226 stores to assess the availability of 8 food groups: nonfat/low-fat milk, fruits, vegetables, low-fat meat, frozen foods, low-sodium foods, 100% whole wheat bread, and low-sugar cereals. A Healthy Food Availability Index (HFAI) score was calculated for each store according to standardized protocol (12) by using only 5 of the 8 original food groups (nonfat/low-fat milk, fruits, vegetables, low-fat meat, and frozen foods) to be consistent with the self-reported measure. Information on the original distribution of HFAI scores by store type and neighborhood characteristics is provided elsewhere (12). HFAI scores were standardized to a 100-point scale, with a higher score indicating greater availability of healthy foods; the original scale ranged from 0 to 19, with a mean (standard deviation) of 4.6 (5.7). HFAI scores of all stores within a 1-mile Euclidean buffer around each participant’s home were averaged to obtain a summary measure of the average directly measured availability of healthful choices within neighborhood food stores. The score of the store with the highest HFAI score within 1 mile of each participant’s home was also used as an alternate estimate of neighborhood directly measured availability.

Analyses

The analytical sample was constructed by combining the survey data and the direct measure data on the basis of the geographic area where the food stores were assessed. Only survey participants who lived within the 159 contiguous tracts (n = 1,264) where all stores were directly measured for availability were included in these analyses. Of those 1,264 participants, 52 participants were excluded because information was not available for all 3 survey questions. An additional 42 individuals were excluded because of missing demographic information, leaving 1,170 participants available for analysis.

Self-reported availability was used as a continuous variable in analyses. Directly measured availability was used as both a continuous variable and a categorical variable of 5 groups. If no stores were available to rate within 1 mile of the participant’s home, directly measured availability was assigned a zero score in analyses with directly measured availability as a continuous variable and classified as no stores for categorical analyses. When stores were available to rate, scores were divided into categories on the basis of quartiles derived from the sample distribution.

In descriptive analyses, we examined the distribution of self-reported and directly measured availability by participant’s self-reported race/ethnicity (non-Hispanic white, non-Hispanic black, and other), educational attainment (less than high school, high school graduate, some college, college graduate, postbaccalaureate degree), and annual household income ($0–$11,999, $12,000–$34,999, $35,000–$49,999, and ≥$50,000). Differences across demographic subgroups were tested by using chi-square tests for proportions and analysis of variance for means.

To assess agreement between self-reported and directly measured store healthy food availability, we calculated intraclass correlation coefficients using mixed models and estimated means of reported availability across categories of directly measured availability. To assess the validity of the self-reported measures against the proxy gold standard of directly measured store food availability, we calculated the sensitivity and specificity of the self-reported measures by dichotomizing each measure at the mean. Variation in sensitivity and specificity was also examined by demographic subgroups. Because sensitivity and specificity can vary greatly by how measures are dichotomized, a receiver operating characteristic (ROC) curve was plotted to show how the selection of different cutoff points for the self-reported availability measure affected calculated sensitivity and specificity. Only the directly measured availability for all stores within 1 mile was used in these analyses. The ROC curve was created by estimating sensitivity and specificity for the self-reported measures using 10 different threshold values based on distributional deciles. Directly measured availability was dichotomized at the mean for all thresholds.

RESULTS

The majority of survey respondents were female and had at least some college education (66% and 67%, respectively) (Table 1). Forty percent of participants were non-Hispanic white, and 53% were non-Hispanic black. A third of the study sample had a household income over $50,000. Those excluded from analyses because they did not live within the 159 contiguous tracts were more likely to be white, to have a postbaccalaureate degree, and to earn $50,000 or more than those within the study area (data not shown; P < 0.01).

Table 1.

Reported and Directly Measured Availability^a of Healthful Choices in Food Stores Within 1 Mile^b of Participants’ Homes by Selected Demographic Characteristics, Baltimore City and County, Maryland, 2004–2006

			Reported Availability		Directly Measured Availability
	No. of Adult Participants	%	Mean (SD)	P Valuec	No Stores Within 1 Mile		All Storesd		Highest Stored
					%	P Valuec	Mean (SD)	P Valuec	Mean (SD)	P Valuec
Total	1,170	100	57.6 (25.6)		6.4		34.3 (25.7)		76.9 (24.0)
Sex				0.6957		0.8467		0.6369		0.1172
Male	402	34	57.2 (24.7)		6.2		34.8 (25.7)		78.5 (23.3)
Female	768	66	57.8 (26.1)		6.5		34.1 (25.8)		76.1 (24.3)
Race/ethnicity				<0.0001		0.5501		<0.0001		<0.0001
White, non-Hispanic	467	40	64.0 (23.4)		7.3		44.6 (27.0)		84.5 (22.2)
Black, non-Hispanic	617	53	52.7 (26.3)		5.7		26.3 (21.2)		71.1 (23.6)
Other	86	7	57.5 (24.8)		7.0		37.6 (28.9)		77.9 (25.1)
Educational level				0.0528		0.0454		<0.0001		0.0001
Less than high school	158	14	54.7 (24.5)		2.5		25.3 (20.0)		73.7 (23.9)
High school degree	234	20	55.3 (25.4)		4.3		30.6 (23.6)		74.9 (24.4)
Some college	358	31	57.2 (26.9)		7.3		32.9 (23.9)		74.3 (24.4)
College degree	245	21	59.0 (25.0)		7.4		39.0 (27.4)		82.1 (22.2)
Postbaccalaureate degree	175	15	61.8 (24.7)		9.7		44.8 (30.0)		80.9 (23.9)
Annual household income				<0.0001		<0.0001		<0.0001		0.0022
$0–$11,999	137	12	49.4 (26.7)		2.9		22.0 (15.9)		74.5 (21.4)
$12,000–$34,999	330	28	55.9 (25.2)		2.1		29.0 (21.0)		73.8 (24.7)
$35,000–$49,999	182	16	54.5 (25.7)		6.6		35.6 (25.4)		76.5 (25.3)
$50,000 or more	394	33	62.6 (24.9)		10.9		42.1 (28.8)		80.9 (23.2)
Missinge	127	11	59.6 (24.7)		7.1		38.1 (28.8)		76.8 (24.1)

Abbreviation: SD, standard deviation.

^aDirectly measured availability is measured by the Healthy Food Availability Index score. Reported availability is measured by summing 3 survey questions on the selection and quality of fruits and vegetables and the selection of low-fat products. Each measure is standardized to a 100-point scale, with higher scores indicating greater availability of healthy foods.

^bOne mile = 1.6 km.

^cP value for differences between subgroups using chi-square tests for proportions and analysis of variance for means.

^dAmong persons with at least 1 store within 1 mile of their home, directly measured availability of healthful choices in all food stores within 1 mile and in the store with the highest availability of healthful choices within 1 mile.

^eA total of 127 participants who did not report income were included in analyses.

On average, participants rated the self-reported availability of healthy foods within 1 mile from their home as 58 of 100 (median, 58; interquartile range, 42–75) (Table 1). Self-reported availability varied significantly by race/ethnicity and income. White participants and those with annual household incomes over $50,000 rated their environment significantly better than did black participants and those with incomes less than $12,000 (scores of 64 and 63 vs. 53 and 49, respectively). Eleven percent of persons with income over $50,000 lived in areas where no stores were within 1 mile compared with 2%–7% of those with lower incomes. The average directly measured availability of all stores within 1 mile was 34.3 (median, 22.2; interquartile range, 14.8–51.4). The average directly measured availability when using only the store with the highest availability within 1 mile was 76.9 (median, 89.6; interquartile range, 63.5–94.8). For 81% of the participants, the store with the highest availability within 1 mile was a supermarket. Similar to the patterns observed by using reported availability, non-Hispanic whites, those with higher education, and those with higher incomes lived in areas with significantly higher directly measured availability of healthy foods when measured by all stores within a mile and when only the highest availability store was used.

Reported availability increased with directly measured availability from 46.4 in the lowest quartile to 69.6 in the highest quartile for all stores and from 48.1 to 65.0 for the highest store (Table 2). This same general pattern of higher reported availability with higher directly measured availability was observed within most demographic subgroups for both direct measures, with the exceptions of those whose race/ethnicity was categorized as other and those with the highest educational group, both of which had limited sample sizes. Overall, persons with no stores within a mile reported availability ratings similar to those observed for persons in the third quartile of reported availability (Table 2), although there was some variation in stratified analyses. For example, persons with incomes between $12,000 and $50,000 with no stores reported availability comparable to that of the lowest 2 quartiles, but these analyses were limited by small sample size. The intraclass correlation coefficient for the reported and directly measured availability as continuous variables (with persons with no stores within 1 mile assigned zero directly measured availability) was 0.35 for all stores and 0.14 for the highest store (data not shown). The intraclass correlation coefficient was similar for analyses limited to persons with stores within a mile (0.34 and 0.21, respectively; data not shown).

Table 2.

Mean Reported Availability^a of Healthful Choices in Food Stores Within 1 Mile^b of Participants’ Homes by Quartiles of Directly Measured Availability^a by Selected Demographic Characteristics, Baltimore City and County, Maryland, 2004–2006

		Quartiles of Directly Measured Availabilityc
	No. of Adult Participants	All Stores						Highest Store
		No Stores	First	Second	Third	Fourth	Ptrendd	First	Second	Third	Fourth	Ptrendd
Total	1,170	61.3	46.4	51.7	61.5	69.6	<0.0001	48.1	51.0	64.7	65.0	<0.0001
Sex
Male	402	58.3	48.3	53.1	58.3	67.3	<0.0001	51.4	50.5	58.2	63.8	0.0002
Female	768	62.8	45.5	50.9	63.1	70.9	<0.0001	46.8	51.3	67.8	65.7	<0.0001
Race/ethnicity
White, non-Hispanic	467	63.7	55.0	58.4	60.9	71.1	<0.0001	59.3	54.7	64.9	67.0	0.0064
Black, non-Hispanic	617	58.6	45.0	46.2	62.6	67.7	<0.0001	45.3	48.9	65.4	62.0	<0.0001
Other	86	63.9	51.9	52.6	57.9	64.0	0.2689	48.6	59.5	58.3	59.7	0.4622
Educational level
Less than high school	158	58.3	49.0	53.0	59.6	66.7	0.0047	50.2	53.3	63.9	58.5	0.0889
High school degree	234	65.0	43.8	48.2	66.8	66.3	<0.0001	44.7	49.6	64.8	65.2	<0.0001
Some college	358	67.6	42.2	48.9	62.9	73.2	<0.0001	44.6	49.9	66.7	67.9	<0.0001
College degree	245	57.9	51.0	53.4	56.0	71.0	0.0002	50.7	46.4	67.8	64.5	0.0004
Postbaccalaureate degree	175	53.9	61.8	57.5	61.9	66.9	0.0382	61.0	62.2	56.4	65.6	0.1188
Annual household income
$0–$11,999	137	72.9	39.6	46.9	58.0	74.3	0.0003	42.1	45.6	70.2	57.8	0.0389
$12,000–$34,999	330	45.2	46.9	50.7	64.4	67.2	<0.0001	45.8	50.3	65.3	66.8	<0.0001
$35,000–$49,999	182	51.4	38.5	46.9	63.0	67.8	<0.0001	41.9	50.4	66.4	60.8	0.0002
$50,000 or more	394	65.1	53.6	57.8	61.5	69.6	0.0061	56.5	57.9	62.2	66.2	0.0568
Missinge	127	63.9	52.3	52.4	50.8	73.5	0.0035	54.9	49.5	65.6	66.9	0.0371

^aDirectly measured availability is measured by the Healthy Food Availability Index score. Reported availability is measured by summing 3 survey questions on the selection and quality of fruits and vegetables and the selection of low-fat products. Each measure is standardized to a 100-point scale, with higher scores indicating greater availability of healthy foods.

^bOne mile = 1.6 km.

^cAmong persons with at least 1 store within 1 mile of their home, directly measured availability of healthful choices in all food stores within 1 mile and in the store with the highest availability of healthful choices within 1 mile.

^dP value for trend using linear regression.

^eA total of 127 participants who did not report income were included in analyses.

The sensitivity and specificity of self-reported measures of availability compared with direct measures (assigning a zero to those with no stores) dichotomizing both at the mean were 79.6% and 46.8%, respectively, for all stores and 69.6% and 50.9%, respectively, for the highest score store (Table 3). In other words, people who lived in areas of above average directly measured availability reported availability in their areas as above average 70%–80% of the time. However, persons living in areas with below average availability reported availability below average only half of the time. Sensitivity was consistently lower for males and non-Hispanic blacks, compared with females and other racial/ethnic groups regardless of which directly measured availability measure was used. Specificity was also notably lower for whites. The patterns were more variable by education and income. The ROC curve in Figure 1 shows sensitivity and specificity by using various alternative cutpoints for reported availability using the directly measured availability of all stores within a mile as the standard. Dichotomizing reported availability at the median instead of the mean increases sensitivity at the cost of lower specificity.

Table 3.

Sensitivity and Specificity of Reported Availability of Healthful Choices in Food Stores Within 1 Mile^a of Participants’ Homes Using Directly Measured Availability as a Proxy “Gold Standard” by Selected Demographics, Baltimore City and County, Maryland, 2004–2006^b

	No.	All Stores		Highest Store
		Sensitivity, %	Specificity, %	Sensitivity, %	Specificity, %
Total	1,170	79.6	46.8	69.6	50.9
Sex
Male	402	73.4	46.8	65.9	47.6
Female	768	82.9	50.4	71.5	52.5
Race/ethnicity
White, non-Hispanic	467	81.1	38.4	72.8	32.6
Black, non-Hispanic	617	77.5	54.9	65.1	56.0
Other	86	79.4	46.2	71.2	51.9
Educational level
Less than high school	158	78.4	46.3	65.6	48.5
High school degree	234	80.8	50.6	69.0	54.3
Some college	358	84.2	55.7	72.1	55.3
College degree	245	79.0	47.1	69.5	49.3
Postbaccalaureate degree	175	72.4	37.5	69.1	36.5
Annual household income
$0–$11,999	137	80.0	57.3	55.0	61.4
$12,000–$34,999	330	82.4	50.7	74.6	56.7
$35,000–$49,999	182	82.2	59.6	63.9	55.6
$50,000 or more	394	75.7	38.0	72.2	39.7
Missing	127	83.0	48.6	72.0	46.2

^aOne mile = 1.6 km.

^bSensitivity and specificity were calculated by dichotomizing self-reported and directly measured availability at their means.

Figure 1.

Receiver operating curve for self-reported availability versus directly measured availability, Baltimore City and County, Maryland, 2004–2006. ▪, estimate based on dichotomizing the self-reported availability of all stores at the mean of self-reported availability; ▴, estimate based on dichotomizing the self-reported availability of all stores at the median of self-reported availability; •, estimates based on dichotomizing the self-reported availability of all stores at the distributional deciles of reported availability. Directly measured availability was dichotomized at the mean for all thresholds of self-reported availability. The area under the curve = 0.658.

DISCUSSION

Minority race, lower education, and lower income participants generally reported lower availability of healthier foods and lived in areas with lower directly measured availability than did white, more highly educated, and higher income participants, respectively. Agreement between both measures as assessed by using intraclass correlation coefficients and continuous measures was weak (intraclass correlation coefficient = 0.35 for the average of all stores). However, there was clear evidence that self-reported availability significantly increased with directly measured availability across most demographic subgroups. Persons living in areas with above average directly measured availability reported above average availability 70%–80% of the time. Conversely, only half of the respondents living in areas with low availability reported availability below average. Using the average of all stores within a mile to approximate neighborhood availability of healthy foods gives equal weight to the various neighborhood stores. Alternatively, using the store with the highest availability assumes that the store that offers the greatest variety of healthy foods is the best approximation of the neighborhood’s availability. Findings were similar regardless of which measure was used.

Similar to other analyses of the association between survey measures and other measures of the food environment (20, 21), reported availability and directly measured availability were associated but not identical. One previous study found that participants living in areas of low supermarket density rated their perceived availability of healthy foods 17% lower than those living in areas with the highest densities of supermarkets (21). Another found that perceptions of food price and availability were significantly associated with food-purchasing patterns, whereas audits of the stores measuring the actual availability and price of identical foods were not (20). When trying to quantify opportunities for communities to access healthier foods, directly measuring what is available in stores is thought to be one of the more valid and reliable indicators of availability. While survey measures are far more feasible to collect for studies spanning large geographic areas, they are necessarily affected by a host of individual factors and, therefore, have their own sources of measurement error. One of the only studies that compared perceptions with store audits found that there was no evidence to suggest that participants’ perceptions differed from the reality of their local food environments in terms of the availability of fresh fruits and vegetables, in particular, and healthy foods, in general, within an urban community context with high levels of household food insecurity (22). In that study, perceptions were measured via 4 items and were developed from qualitative feedback from an urban, food-insecure population. Perceptions were compared with audits of stores derived from an inventory tool that recorded the layout and flow of the store, types of foods sold in the store, and whether or not the store sold alcohol or tobacco products. Understanding the relation among different measures of the local food environment in a larger population sample like ours is important to the interpretation of studies that use them and to the development of more valid and reliable measurement instruments.

Although our results showed that the sensitivity of reported availability questions is relatively high, a person who lived in an area with below average directly measured availability reported low availability of healthy foods only half of the time. Low specificity may result from participants’ overestimating 1 mile and reporting on a larger neighborhood and thus more stores. This is consistent with the finding that some people who have no stores within a mile still report good availability of healthy foods and with the finding of lower specificity for higher income and more educated participants who are more likely to have cars. In another study, 95% of people reported doing their primary food shopping at a supermarket, and 47% reported shopping primarily within a mile of home (5). However, only 31% lived within 1 mile of a supermarket (5). A limitation of our approach is that we used Euclidean distances rather than network distances, although it is unlikely that respondents are using precise network distances when they report regarding availability within a mile. Clearly, misspecification of distances in survey responses likely introduced important measurement error in the survey measures and reduced agreement between both types of measures. Alternatively, those living in areas with poor directly measured availability of healthy foods may report on availability where they usually shop rather than the availability of healthy foods near their home as the questions specify. This explanation is also consistent with the fact that persons who had no stores within a mile reported generally high availability despite the absence of stores near their home. Revising the questions or adding a preface to reemphasize that each of the questions specifically refers to stores near the home may help to improve specificity. A third possible explanation for the low specificity of the self-reported questions is that NEMS-S might set higher or different standards for healthy food availability than survey respondents themselves do, resulting in those who live in areas with low availability appearing to overestimate availability. Finally, those who report good availability but have poor observed availability may live in areas where stores closed between 2004 and 2006. Almost a quarter of the stores that InfoUSA identified as present in the study area in 2004 could not be audited because they were closed at the time of data collection in 2006. A limitation of these analyses is that we could not categorically rule out which of these potential explanation(s) is the cause of the low specificity of self-reported questions. Additional research on the agreement between both types of measures is therefore needed. Existing evidence also suggests that, when feasible, the use of both self-reported and direct observation measures may be desirable because they may tap into slightly different constructs (5).

These analyses are subject to several other limitations. No one tool has been established as the benchmark to quantify availability of healthy foods so that no true gold standard exists. However, we are aware of no better instrument currently in existence to act as a proxy gold standard than the NEMS-S tool (11) that we used. An additional limitation is the fact that there was a small percentage of store owners who refused to allow raters to enter into their store (9%). On average, these types of stores have been shown to have poor availability (12) and are common in predominantly black and low income areas (9). Another limitation is that, while the survey sample is representative of the Maryland study area (23), it is unclear how applicable the findings of these analyses may be to other areas of the country. Finally, other barriers aside from availability may also hamper access to healthy foods, such as store hours of operation and price, which were not assessed here. Asking separate questions about other potential barriers could help to improve characterization of the local food environment.

An ideal measure would allow us to quickly, accurately, and reliably assess the availability, affordability, and quality of healthier foods in areas. However, existing tools fall short of this ideal in assessing one or more of these constructs. Self-report measures are not perfect but good at identifying areas with above average availability of healthy foods and may offer additional insights into what foods are available in areas (5, 21). With revisions to improve specificity, the subset of questions used to measure reported availability may be a reasonable indicator of what healthy foods are available and may be viable proxies for direct measures giving researchers more options for measuring the food environment.

Glossary

Abbreviations

HFAI

Healthy Food Availability Index

NEMS-S

Nutrition Environment Measures Survey in Stores

ROC

receiver operating characteristic