The Healthy Communities Study Nutrition Assessments

Abstract

Multifaceted community interventions directed at improving food environments are emerging, but their impact on dietary change and obesity prevalence has not been documented adequately. The Healthy Communities Study (HCS) is seeking to identify characteristics and combinations of programs and policies that are associated with children’s diets and obesity-related outcomes in various types of communities across the U.S. The purpose of this paper is to describe the methods used in 2013–2015 in the HCS to assess dietary intake, school nutrition environments, and other nutrition-related behaviors. The conceptual framework of the HCS is based on the socioecological model and behaviors shown in previous studies to be related to obesity in children-guided selection of domains. Nine domains were identified as essential measures of nutrition in the HCS: (1) intake of selected foods and beverages; (2) food patterns and behaviors; (3) social support; (4) home environment; (5) school environment; (6) community environment; (7) breastfeeding history; (8) household food insecurity; and (9) dieting behaviors and body image. Children’s dietary intake was assessed using a dietary screener and up to two automated 24-hour recalls. Dietary-related behaviors were assessed by a survey administered to the parent, child, or both, depending on child age. School nutrition measures were obtained from a combination of school staff surveys and researcher observations. Information from these measures is expected to contribute to a better understanding of “what is working” to improve the dietary behaviors that are likely to prevent obesity and improve health in children.

Introduction

Recent and unprecedented increases in the prevalence of childhood obesity and the associated health and economic costs¹ have led to urgent calls for preventive action. A CDC expert panel in 2009 proposed more than two dozen community strategies to improve the availability of healthful foods and beverages.² In 2012, the IOM issued recommendations for a systems approach involving changes in five sectors of society to ensure that healthful foods and beverages are easily accessible in all places where people “live, work, play and learn.”³ Schools were identified as a focal point for intervention.

Dietary behaviors recommended in community interventions are those likely to be associated with population trends in obesity-related outcomes, having in most cases moderate to strong research evidence linking them to risk of excessive weight gain and obesity. Such behaviors include high intakes of sugar-sweetened beverages, fast food, and other energy-dense, nutrient-poor foods; low intakes of whole grains, fruits, and vegetables; skipping breakfast; and eating while watching TV.^4,5

Multifaceted community interventions directed at improving food environments are emerging, but their impact on dietary intake and obesity has not been studied adequately. Among the few interventions that have been evaluated, several have shown some success in changing dietary behavior and slowing weight gain in children.^6–11 The Healthy Communities Study (HCS) is examining associations between characteristics of community programs and policies (CPPs) and diet and physical activity behaviors and obesity-related outcomes among elementary and middle school children in diverse communities in the U.S.

The purpose of this paper is to describe the methods used to assess dietary intake and other nutrition-related behaviors in children and to examine the nutrition environment in homes, communities, and selected schools in HCS study communities. Information from these measures is expected to contribute to a better understanding of how best to improve dietary behaviors through CPPs, with the ultimate goal of preventing obesity and improving child health.

Methods

Overview

A conceptual framework of potential impacts of CPPs informed the choice of nutrition variables for measurement at the individual and school levels. The framework is based on theories of change from the socioecological model, which posits that individual behaviors are influenced by both proximal factors, such as family and peers, and more distal factors, such as school, community, social, and economic environments.¹² Children’s BMI and waist circumference are measures of “longer-term” outcomes, whereas dietary behaviors and intakes likely precede and mediate reduction in obesity-related outcomes, and serve as “medium-term” outcomes (Figure 1). Increased access to healthful foods in the home, school, and community environments serve as “short-term” conditions that favor individual behavior change and change in obesity-related outcomes at the community level.^13,14 Other influences on individual behaviors and obesity-related outcomes may be modified by community programs, or particular factors may mediate or modify the effects of programs on measured outcomes, such as social support from family and peers for healthy eating¹⁵ and history of breastfeeding.^16–18 Because of concerns about unintended and harmful consequences of obesity prevention programs on children, potential adverse outcomes also should be included in evaluations.¹⁹

Figure 1.

Logic model for relationship of community programs and policies with household nutrition-related measures and child weight outcomes.¹

¹Up arrow signifies ‘more’; down arrows signifies ‘less’: ‘difference’ signifies more or less depending on measure.

Based on this conceptual framework, nine domains were identified for measurement of nutrition in the HCS (Appendix Table 1 describes the rationale):

1. food and beverage intake;

2. food patterns and behaviors;

3. social support;

4. home environment;

5. school environment (reported and objectively assessed);

6. community environment;

7. breastfeeding history;

8. household food insecurity; and

9. dieting behaviors and body image.

These domains and the measurement methods and items for each were selected based on reviews of the literature relevant to behavior change and obesity-related outcomes, and were refined in consultation with an HCS nutrition expert subcommittee (composed of approximately 20 nutrition and dietary assessment experts) and the HCS Observational Study Monitoring Board. Preference was given to measures with published validity and reliability, previous use in national or large community-based surveys of children and schools, and ease of administration.

Because the HCS protocol included a number of assessments during each household visit, the number of nutrition-related items was limited to those that could be completed in approximately 25–30 minutes. The entire sample of HCS households received a standard protocol of assessments, including questions on usual dietary intake; a randomly selected subset of approximately 10% of households completed an enhanced protocol, which consisted of additional assessments and a second home visit. The enhanced protocol included two self-administered, computer-based, 24-hour dietary recalls for validation and calibration of survey estimates of dietary intake. Onsite observations of the school nutrition environment were made in selected elementary and middle schools in each community, and questionnaires were completed by school staff.

The study was initially approved in 2011 and annually reviewed through 2015 by the Battelle Memorial Institute IRB. Parents provided written informed consent for their child’s participation. A full description of the human subjects protections is included in John et al.²⁰ All HCS data were collected in 2013–2015.

Household Nutrition Measures

The standard protocol for households included a “dietary screener” for assessment of dietary intake, questions about dietary patterns and family meals, and questions about other behaviors associated with obesity-related outcomes. Further details on the survey questions for each domain, their sources, and available information about their validity and reliability are provided in the Appendix (Appendix Table 1).

Household nutrition data were collected by Field Data Collectors (FDCs) who resided in or near the study communities. The FDCs administered the nutrition questions on the household survey, giving a scripted neutral introduction and clear instructions to the parent/adult caregiver and child regarding who was to respond and how to seek clarification from parents when the child was the primary respondent. The FDC read each question aloud to the primary respondent and entered the response in a pre-programmed electronic tablet. The primary respondent was determined by the child’s age as follows: parent/adult proxy for children aged 4–8 years, with child assistance; children aged 9–11 years, with assistance from the parent/proxy; and children aged ≥12 years, with input from parent/proxy only if needed.

The Dietary Screener Questionnaire (DSQ) was developed by the National Cancer Institute (NCI) and was used to collect dietary intake data on all HCS study children (interviewer-administered version).²¹ This 26-item food frequency questionnaire was included in the National Health and Nutrition Examination Survey (NHANES) for people aged 2–65 years in 2009–2010. HCS participants (or proxies for young children) were asked to report their intakes of selected foods consumed as meals or snacks at home, school, or anyplace else over the past 30 days in number of times per day, week, or month. Items included fruits and vegetables, dairy, sugar-sweetened beverages, other energy-dense foods of minimal nutritional value (e.g., fried potatoes, chocolate/candy, donuts/sweet rolls, cookies/cakes/pies, ice cream/frozen desserts, chips/crackers), and whole grains. To be consistent with the study’s focus on obesity, DSQ items related to meat intake were excluded and a question was added on frequency of consuming chips and crackers, an energy-dense group of items commonly consumed by children. Publicly available NCI-generated scoring algorithms were used by the research team to convert respondent frequencies of intake to estimated quantities of select food groups and nutrients, based on age- and gender-specific 24-hour dietary recall portion size data from NHANES.²² Outcomes included quantitative estimates of amounts consumed daily for fruits/vegetables/legumes with and without fried potatoes, dairy, total added sugar, sugar from sugar-sweetened beverages, whole grains, dietary fiber, frequency of consumption of energy-dense foods of minimal nutritional value, and usual intake of lower-fat milk (≤1%).

For the DSQ, built-in range checks and scripted probes were included for each question when responses were out of the usual range. The DSQ was administered early in the interview because of its primary importance as an outcome measure.

Questions were included on the household survey for food patterns (skipping breakfast, eating while watching TV, frequency of eating at a fast food restaurant, and frequency of family dinners), perceived social support for healthful eating (e.g., eating fruit and vegetables), availability of select foods in the home (e.g., fruit, dark green leafy vegetables, chips/crackers, low-fat/nonfat milk, sugar-sweetened beverages), school and community environments, breastfeeding initiation and duration, household food insecurity, and dieting behaviors and body image (e.g., perception of weight, weight-based teasing, and meal skipping for weight control) (Appendix Table 1).

The child version of the Automated Self-Administered 24-hour Recall (ASA24^™-Kids; 2012 and 2014 versions) web-based system was administered to a randomly selected subsample of approximately 10% of participants as part of the enhanced protocol in order to provide detailed information about food and nutrient intakes for validation and calibration of DSQ intakes. Similar intake variables will be compared between the ASA24 and DSQ, such as cups of fruits and vegetables and ounces of sugar-sweetened beverages. In addition, energy and select nutrients will be compared with DSQ variables; for example, calories from solid fats and added sugars will be compared with DSQ foods of minimal nutritional value.

The ASA24-Kids was developed by the NCI to simulate the interview structure, probes, and food and nutrient database used for conducting interviewer-administered 24-hour recalls in NHANES.²³ The automated interview involves an animated character who asks questions and gives instructions on completing various stages of the recall. Probes for portion sizes of foods reported are based on photographs of graduated serving sizes from which the respondent can select. A Spanish language version of the program was used when appropriate. The ASA24-Kids is a modified version of the adult version of the ASA24 with simplified questions, developed based on research with children.²⁴ The ASA24 has been shown to perform reasonably well compared to interviewer-administered recalls.^25–27

To ensure high response rates, children or their proxies completed the 20–30-minute ASA24-Kids in the presence of the FDC during home visits, rather than on their own after the household visit. Respondents completed the ASA24-Kids twice, a week apart, by logging onto the web-based program on an HCS tablet computer. Respondents were unaware prior to the interview that they would be completing 24-hour recalls, thus avoiding potential reactivity. The FDC’s role was to provide minimal assistance; they introduced and logged onto the web-based program, provided the computer to the primary respondent, answered respondent questions in a neutral manner, and avoided intervening in the self-completion process.

Each FDC received centralized in-person training and was certified on all standard and enhanced protocol nutrition measures by experienced researchers prior to beginning data collection in the field. Approximately 5 hours of training, online and in-person, was given for the DSQ and ASA24-Kids. Trainees viewed audio-recorded slide presentations online as well as video demonstrations of both tools, including appropriate procedures for handling “difficult” situations. To be certified, trainees must have demonstrated at least 80% competency, including use of neutral introduction, adherence to script and protocol, standardized responses to commonly asked questions, appropriate responses to various interview scenarios, and correct answers on quizzes. Field supervisors and senior research staff made quality assurance checks of FDCs on selected home visits. Data were reviewed monthly to identify percentages of missing data, irregularities in time for administration, extreme responses, and percentage of don’t know and refusal responses to the DSQ. FDCs who did not maintain 80% competency were re-trained and further monitored for quality performance.

School Nutrition Environment Measures

In each HCS community, up to two elementary and two middle schools were randomly selected for assessment of the nutrition environment. School measurements were intended to supplement the information on CPPs obtained through key informant interviews (see Fawcett and colleagues²⁸), and to obtain an objective description of the schools attended by study participants. This information will be useful for assessing the extent to which programs and policies have influenced the school nutrition environment, and the potential association with dietary intakes and BMI that may result from modification of the school nutrition environment. Three complementary instruments were designed to measure the school nutrition environment: the Lunch and Competitive Foods Observation Form (LCFO), the School Foodservice Questionnaire (SFSQ), and the nutrition-related aspects of the School Policies and Practices Questionnaire (SPPQ).

The LCFO was conducted by a team of five HCS researchers during school site visits and took approximately 25 minutes per school. Most items were observed and documented immediately prior to, or during, the lunch service. Competitive foods sold in vending machines and aspects of the facilities were observed at other times. The SFSQ was completed online by the foodservice director or designee at the selected schools. The SPPQ was completed online by a designated school staff member who was advised to consult with other school personnel to answer the questions accurately.

The LCFO was used to gather information about competitive foods (number of different venues and types of foods and beverages offered at any time during the school day), school meal foods, meal service (length of lunch period, time spent in line, staff interaction with students), and dining facilities (water availability, adequacy of dining areas). The SFSQ was used to collect information about school foodservice that cannot be observed readily, including meal program eligibility and student participation, school participation in selected state and federal nutrition programs, cooking methods, and self-reported implementation of the district’s food-related school wellness policies. The SPPQ was used to gather information about student enrollment and attendance, nutrition education (quantity and quality), and school wellness committee and coordinator (existence and function). Each of the instruments asks respondents to report on the length of time the above practices and policies have been in place. Questions were adapted from instruments developed by the authors and used in previous studies, including the School Nutrition Dietary Assessment Study III and the School Health Policies and Practices Study, and from the School Nutrition Association (University of California, Berkeley, Atkins Center for Weight and Health, unpublished observations, 2015).^29–35 Further details on the school nutrition measures are provided in the Appendix (Appendix Table 2).

Data collectors were centrally trained and certified prior to data collection by experienced researchers. Training included an interactive face-to-face component of approximately 2.5 hours, followed by supervised practice at a school. School visits were followed by debriefing with trainers. Staff members were certified during a second school visit at the end of the training, during which the trainee and the trainer completed the LCFO independently. Trainees had to achieve at least 80% congruence (calculated as proportion of items in agreement) with the trainer for certification. Those who failed were given an additional training and re-certification opportunity at another school. Those who failed on the second attempt were excluded from gathering data on the LCFO. Quality control assessments, similar to those used for certification, were conducted by a trainer, who attended at least two school site visits with each of the certified staff, once in the early weeks of data collection and another at a later date. Remediation and retesting processes were instituted in the event of inadequate field performance.

Discussion

The HCS is a unique study of a large sample of diverse communities with varying numbers and intensities of programs and policies aimed at improving children’s diet and physical activity behaviors and obesity-related outcomes. In selecting nutrition measures for the HCS, the investigators had to balance the need for valid and reliable measures with the need for ease of standardized administration by multiple field staff members. In addition, they sought measures that would yield nationally representative comparison data whenever possible.

The advantages of the DSQ over food frequency questionnaires are its focus on dietary behaviors linked to obesity risk; its feasibility and short time for administration by field interviewers without nutrition backgrounds; and the quantitative estimates available for selected foods and nutrients, from applying NCI scoring algorithms to converting frequencies to quantities. Likewise, as a tool for validation/calibration of the DSQ, the ASA24-Kids offers the unique advantage of enabling researchers to collect detailed dietary intake data using standardized procedures at a relatively low cost (without interviewer administration). Owing to multiple assessments and associated time constraints in HCS households, it was not feasible to utilize the ASA24-Kids with the entire HCS sample.

The school nutrition assessment combined direct observation and reported measures to obtain objective, valid measures of school food, with more-comprehensive reports by school staff on factors that may influence children’s food consumption at school. The representativeness of the observations of competitive foods is likely to be high, because there is little day-to-day variability. Foods available through school meals have more day-to-day variability, so one day of observation may not fully reflect the extent to which the school meets U.S. Department of Agriculture standards. The staff questionnaires include many factual items (e.g., school meal participation rate), which are less subject to reporting error. The more-subjective questions were obtained from established surveys when possible, or were selected based on their performance in previous research. The school policy, practice, and environmental measures were adapted from national and other large surveys and thus will yield comparable data on a range of variables thought to be important for supporting healthful food consumption at school.

With such a large team of data collectors, quality assurance and quality control posed a considerable challenge. Standardized training of field staff and quality assurance procedures were developed to ensure adherence to data collection protocols, and to monitor the validity of the collected data, as measured by congruence with comparison data (e.g., NHANES dietary intakes for DSQ and ASA24-Kids, trainer observations for school lunch and competitive foods observations).

Conclusions

The HCS provides an unprecedented opportunity to document the types and intensities of nutrition-related community interventions taking place across the country, and to assess their association with dietary behaviors and obesity-related outcomes in the intended beneficiaries—children. The recent documented slowing or leveling off of rates of obesity in the U.S.^36,37 and certain locations^38,39 suggests that some communities may have mounted sufficient efforts to produce the beneficial changes observed in child BMI. Findings from this comprehensive examination of nutrition domains in this large study will inform nutrition-related interventions for reducing childhood obesity, a critical public health issue.