Comparing Food Security Before and During the COVID-19 Pandemic: Considerations When Choosing Measures

Food insecurity is a public health issue in the United States. Over the past 25 years, national prevalence of food insecurity—encompassing both low and very low food security—has been consistently over 10%, and very low food security alone has persisted at over 3%.¹ The onset of the COVID-19 pandemic in March 2020 and ensuing economic crisis and disruption of food systems² have only served to exacerbate its prevalence.³ In response, the research community has made laudable efforts to track the pandemic’s influence on the prevalence of food insecurity across the country. However, for the sake of expediency, many food-security measures used were inconsistent with validated, prepandemic measures.4, 5, 6, 6 This incompatibility will likely lead to estimation errors, which in turn will produce inaccurate conclusions resulting in potentially detrimental policy and resource-allocation decisions. This commentary aims to reinforce the necessity of consistent and careful use of validated measures when making comparisons between pre- and postpandemic data to arrive at accurate conclusions and recommendations.

Why Does Using Consistent, Validated Measures Matter for Public Health Policy and Program Implementation?

Without a doubt, struggles to obtain and afford food during the pandemic have increased dramatically.² To make proper decisions about solutions, policy makers at every level need to be accurately informed about food-insecurity prevalence in the population. Health care and service providers and community leaders designing rapid response programs to get food resources to patients, clients, neighbors, and other community members also need accurate food-insecurity prevalence estimates. State and federal agencies and public assistance program managers respond to food-insecurity prevalence by increasing staff and infrastructure to manage an increased demand in program participation. Measuring the success of their efforts depends, at least in part, on food-security prevalence estimates from accurate data, enabling comparisons before, during, and—whenever it may be—after the pandemic. Inaccurate data will consequently result in misleading conclusions about who was hardest hit, who needs ongoing help, or whether programs are actually making a difference and may even lead to harmful decisions about program funding or resource allocation and deployment. In the worst case, such a scenario could compound the suffering already sustained in many communities, if, for example, policy makers prematurely decide to roll back or terminate COVID-related programs and benefits before those at the lower end of the income spectrum and most at risk of food insecurity have had a chance to recover.⁶

A Note on Validation of Food-Security Measures

The Household Food Security Survey Module (HFSSM) measures food security and insecurity at the household level. Households composed only of adults respond to 10 questions and those with children under age 18 years respond to an additional 8 questions.^{7 , 8} Households are classified as having high, marginal, low, or very low food security over the 12 months prior to the survey according to the number of affirmative answers.

The HFSSM was developed and validated with specific Item Response Theory psychometrics that confirmed its validity and reliability. This household-level scale, with a recall period of 12 months, is considered the most reliable scale and has been used since its development as the reference measure in the United States and as the basis for other countries to develop and validate their own national surveys. A shorter 6-item adult version,⁹ a 9-item version for children ages 12+ years,¹⁰ and a 2-item screen¹¹ were validated against the HFSSM for use in community and clinical settings where the full module is impractical. In addition, a 30-day reference period was validated for these subscales.^{12 , 13} To develop and validate a shorter scale or screen, researchers can use several distinct methodologies to assess its validity, reliability, accuracy, and effectiveness compared with the reference measure. Optimally, the validated measure carries over the metrics with which the reference measure was developed, such as the level of response and recall periods.

Collecting Food-Security Data During the COVID-19 Pandemic: Comparing Measures

On March 13, 2020, the US government declared a national emergency concerning COVID-19.¹⁴ To document rapid changes in food insecurity since the COVID-19 pandemic began, researchers initiated both governmental and nongovernmental studies using a variety of scales, questions, and time frames different from the original validated instruments. Some examples are (1) the US Census Bureau Household Pulse Survey (HHPS), which includes the food-sufficiency question from the HFSSM, using a recall period of 7 days⁴; (2) the Hamilton Project in association with the Middle Class Initiative Survey of Mothers with Young Children, which includes 2 questions from the HFSSM, using a time frame from the beginning of the pandemic¹⁵; and (3) the University of Southern California Center for Economic and Social Research’s Understanding Coronavirus in America tracking survey, which includes some questions from the HFSSM in modified form and a 7-day recall period.⁵

Every effort to measure food insecurity during the pandemic is a positive and laudable initiative. When developing a new survey, the decision to use one food-security measure or another is conditioned on the objective of the given research question, among other factors. In general, new food-security surveys are most helpful if resulting prevalence estimates can be compared with prepandemic food-insecurity prevalence. To do so, researchers should either compare results from the same set of validated questions or make sure adaptations are made appropriately to enable meaningful comparisons. To avoid misinterpretation of data trends, it is essential either to present food-insecurity prevalence during the pandemic only or to use the same set of validated questions to produce food-insecurity trends over time periods that bridge the start of the pandemic.

Example From the Field: The Household Pulse Survey

In addition to the HFSSM, the Current Population Survey also includes the single-item household food-sufficiency measure in its December supplement. Despite collecting data on food insufficiency, the US Department of Agriculture does not conduct analysis of data from this question on a regular basis as it does with the HFSSM results, though it recommends—when minimizing respondent burden is a high priority—use of the food-sufficiency question together with income to screen out potential respondents highly unlikely to be food insecure.⁶

The food-sufficiency question was included in the HHPS to collect data on food adequacy during the pandemic—before March 13—and in the 7 days prior to date of survey panels from April through October 2020.∗ Despite being a validated question, prevalence estimates based on this measure cannot be compared directly with prepandemic HFSSM estimates of food-security prevalence for several reasons.

First, food insufficiency and food insecurity measure different dimensions of food inadequacy. HFSSM food-security questions address 4 dimensions (anxiety or worry about the adequacy of food supply, quality of food intake, quantity of food intake, and social acceptability of the sources of food), whereas food sufficiency measures at most 2 dimensions (quantity and possibly quality of food intake).⁷

Second, although food sufficiency has 4 levels of response, which could potentially allow a category-by-category comparison with the 4-level HFSSM measure of food security (food secure, marginally food secure, low food security, and very low food security), the conceptual foundations of the measures are quite different. In addition, it is important to note that when dichotomizing households on the food-sufficiency question responses, it is necessary to use 2 categories: “food sufficiency,” when collapsing the 2 less severe answers (enough of the kinds of food we want to eat, and enough but not always the kinds of food we want to eat), and “food insufficiency,” when aggregating the 2 more severe answers (sometimes not enough to eat, and often not enough to eat).

Third, the HFSSM was validated at the household level, as was the food-sufficiency question. The dimension of individual-level food security was not included in the measurement framework for either the HFSSM or the food-sufficiency question (unless there is only 1 adult or 1 child in a household). For this reason, it is more acceptable to refer to individuals who live in food-secure or food-insecure households, instead of food-secure or food-insecure individuals, although this difference may seem subtle and is often not observed. This is even more important for children when not using the child-level scale. Thus, prevalence estimates utilizing the food-sufficiency question should use household-level weights, and not individual-level response weights.

Fourth, the time reference for the food-sufficiency question used in the December Current Population Survey is 12 months, as opposed to a 7-day recall period, or an indefinite “before March 13, 2020” period used in the HHPS. The HFSSM was validated with 12-month and 30-day recall periods, and these are the only recall periods that would allow comparability between pre- and during-pandemic prevalences. Using the Current Population Survey data, it is possible to observe that prevalence of food insecurity with the 30-day recall period is far below annual rates. In this sense, the prevalence observed with a recall period less than 12 months has a high probability of providing lower food-insecurity prevalence.⁷ Thus, other measures utilizing 30-day, 7-day, or nonspecific prepandemic dates are not directly comparable with the HFSSM prepandemic prevalence estimates based on a 12-month recall period.

Conclusion

The rapid response of the research community to ensure that food adequacy is monitored and addressed is commendable. However, if accurate comparisons are to be made between data from prior to the COVID-19 pandemic and data during the pandemic using the food-sufficiency question, or any other short scale or screen, researchers should compare results derived with the same measure during both periods or make adequate efforts to reconcile data differences and achieve consistency across periods. Comparisons using different measures will likely lead to estimation errors arising from variation in level of response (eg, individual versus household level) and/or recall period (eg, 12 months or 30 days versus 7 days, or not determined), or other inconsistencies, and have the harmful potential to lead to inaccurate conclusions and detrimental policy and resource-allocation decisions.

Biographies

A. Poblacion is a research scientist, Department of Pediatrics, Boston Medical Center, Boston, MA.

S. Ettinger de Cuba is an executive director, Department of Pediatrics, Boston University School of Medicine, Boston, MA.

J. T. Cook is a principal investigator, Department of Pediatrics, Boston University School of Medicine, Boston, MA.