Study Designs and Analytic Strategies for Environmental and Policy Research on Obesity, Physical Activity, and Diet

Abstract

Numerous authoritative reports have identified environmental and policy interventions as the most promising strategies for creating population-wide improvements in diet, physical activity, and obesity. Yet many methodologic challenges to conducting environmental and policy research must be overcome to enable this area of study to advance. A meeting titled “Study Designs and Analytic Strategies for Environmental and Policy Research on Obesity, Physical Activity, and Diet” was held April 8, 2008. Participants from diverse backgrounds identified priority gaps in knowledge and generated recommendations for promising methods to enhance environmental and policy research related to obesity. Final recommendations were based on a postmeeting participant survey.

Existing methods were identified that could be applied to advance the field, including prospective studies, evaluations of natural experiments, and economic studies. Training for investigators in the use of appropriate statistical methods for complex designs and interdisciplinary collaboration were recommended. Methodologic research priorities included the development of measures of policy and health impact assessments and the investigation of policy adoption and implementation. The results of this conference can be used to improve the quality and quantity of environmental and policy research as well as the translation to action to control obesity.

Introduction

Obesity is one of the most serious and prevalent health problems in the U.S. Two thirds of adults and one third of children and adolescents are either overweight or obese, with low-income and certain racial/ethnic populations disproportionately affected.^1,2 Although multiple genetic loci create predispositions to obesity, it is widely believed that alterations of environments have caused changes in dietary and physical activity behaviors, triggering the current epidemic.^3,4 Numerous health authorities have identified environmental and policy interventions as the most promising strategies for creating population-wide improvements in eating, physical activity, and obesity, including reports by the U.S. Surgeon General,⁴ WHO,⁵ the IOM,^6,7 the CDC,⁸ and the International Obesity Task Force.⁹

Evidence is growing rapidly that the attributes of built environments, including neighborhood design and lack of access to attractive parks, are associated with obesity¹⁰ and physical activity.^11,12 Food-environment attributes such as the lack of access to supermarkets and the concentration of fast-food restaurants also are related to excess energy intake and obesity.^13–16 However, there are special challenges to conducting environmental and policy research that could undermine the momentum in this area of study, compromise advancements in understanding obesity-related environmental and policy etiologic factors and interventions, and delay the development of evidence-based solutions to the obesity epidemic. Three methodologic concerns can be identified.

1. There are limited RCTs that intervene on environmental or policy factors. Investigators rarely are able to control environmental or policy factors, so RCTs are impossible in most situations. For example, zoning codes, grocery store–location decisions, and park renovations are outside the control of investigators. Thus, the vast majority of studies to date are cross-sectional, which limits the advancement of the science and its credibility for guiding policy change.

2. There are analytic challenges because most environmental and policy studies have nested or other complex data structures that presumably require multilevel or spatial statistics. Additionally, there is confounding between the characteristics of residents of neighborhoods and the characteristics of those environments, presenting a difficult selection problem in observational studies. There is a need to develop innovative statistical models tailored to the demands of environmental and policy studies.

3. Although the development of built environment and food environment measures is proceeding, the development of measures of policies has lagged. Few investigators in this field are familiar with the measurement of policy variables, so methods and perspectives need to be adapted from other fields. Advancing the field will require interdisciplinary teams of researchers and the building of a diverse network of researchers. Policy measurement could entail research that investigates the factors supporting the successful adaptation and implementation of a particular policy. Policy measurement could also involve understanding a policy's impact on changing the built and the food environments.

Purpose for the Conference

As an initial step in improving the methodology of environmental and policy research related to obesity, diet, and physical activity, a national meeting of experts was organized to develop and prioritize recommendations. The conference was responsive to the first theme of the strategic plan for NIH Obesity Research (obesityresearch.nih.gov): “Research toward preventing and treating obesity through lifestyle modification.” The conference contributed to improving methodology for studies consistent with the NIH Obesity Research goals:

Under this theme, the goals and strategies for achieving them encompass identifying modifiable behavioral and environmental factors that contribute to the development of obesity in children and adults, and designing and testing potential intervention strategies.

The conference targeted methodologies central to both etiologic and intervention research on environmental and policy issues. For purposes of this conference, the dietary behaviors most related to obesity were of interest, and physical activity could include the entire spectrum of types and intensities as well as sedentary behaviors.

The conference identified high-priority gaps in knowledge and considered the relevance of recommendations for contributing to understanding and eliminating disparities in obesity, diet, and physical activity. A high priority was placed on involving new and minority investigators in this conference, including those with personal experience in and insights about the low-income populations and communities of color disproportionately affected by obesity, to expand both this new and evolving field and these investigators' capacity to conduct important research.

Conference Objectives

The conference objectives were to:

• identify the highest-priority gaps in knowledge on environment and policy factors that influence obesity, diet, and physical activity;

• identify promising study designs to enhance the quality and rigor of environmental and policy research on obesity, diet, and physical activity;

• identify promising analytic strategies that can be applied to improve environmental and policy research on obesity, diet, and physical activity;

• identify measures of policies of particular relevance to obesity prevention that either can be used in research or need to be developed;

• identify other strategies that can advance the research field, such as commissioned papers, think-tank meetings, curriculum design, and training; and

• build the capacity of new investigators to make important contributions to environmental and policy research on obesity, diet, and physical activity.

Methods

The organizing committee (Table 1) developed the conference goals, format, methods, and participant invitation list. NIH staff served on the organizing committee and played important roles in planning the conference and facilitating the dissemination and use of the recommendations.

Table 1.

Conference organizing committee

Name	Affiliation
Tanya Agurs-Collins, PhD, RD	Program director/nutritionist, Health Promotion Research Branch, National Cancer Institute, Bethesda MD
Audie Atienza, PhD	Program director, Health Promotion Research Branch, National Cancer Institute, Bethesda MD
Carlos J. Crespo, DrPH, MS	Professor and director, School of Community Health, Portland State University, Portland OR
Lori Carter-Edwards, PhD	Duke University, Durham NC; representing AACORN
Terry Huang, PhD, MPH	Director of Pediatric Obesity and Metabolic Syndrome Research, National Institute of Child Health and Human Development, Bethesda MD
Shiriki Kumanyika, PhD	University of Pennsylvania, Philadelphia PA; representing AACORN
J. Michael Oakes, PhD	Professor, University of Minnesota, Minneapolis MN
C. Tracy Orleans, PhD	Senior scientist and distinguished fellow, Robert Wood Johnson Foundation, Princeton NJ
Charlotte A. Pratt, PhD, RD	Program director, National Heart, Lung, and Blood Institute, Division of Prevention and Population Studies, Bethesda MD
James F. Sallis, PhDa	Professor, San Diego State University, San Diego CA
Thomas L. Schmid, PhD	CDC, Physical Activity and Health Branch, Atlanta GA
Mary Story, PhDa	Professor, University of Minnesota. Minneapolis MN

^aCo-chair

AACORN, African American Collaborative Obesity Research Network, based at the University of Pennsylvania and directed by Vikki Lassiter, MS

The full report of the conference, including slide presentations, is available on websites (www.activelivingresearch.org and www.healthyeatingresearch.org), and the highlights are summarized here. The 1-day invitational meeting brought together research staff from multiple funding agencies and senior and junior investigators from diverse fields to make recommendations for improving the methodology of environmental and policy research based on gaps in the literature and public health needs.

Leading scientists made focused presentations with slides and brief written summaries of specific recommendations. Discussants provided an additional perspective on each topic. Initial presentations identified both the need for environmental and policy research and specific gaps in knowledge related to diet and physical activity. Later presentations proposed methodologies that could be used to improve research to fill the gaps. After each presentation, participants formed small groups to discuss targeted questions and generate recommendations. Each small group was facilitated by a presenter and assigned a recorder. The groups were instructed to recommend promising methods to improve the rigor and quality of environmental and policy research related to obesity prevention as well as to specify research priorities, including studies to improve methodology. Most small groups used a voting process to prioritize recommendations.

After the conference, recommendations were compiled across all small groups and edited to combine similar points. Then draft lists of recommendations were sent to presenters for their review. Following this step, all meeting participants were asked to complete an online survey to rank their highest priorities from each of nine lists in five topic areas, as shown in Table 2. Surveys were completed by 41 of 53 conference participants, for a 77% response rate.

Table 2.

Top-scoring recommendations for improving the methodology of environment and policy research related to obesity, diet, and physical activity^a

	Rating average (1=lowest; 5=highest)
TOPIC 1: Research status and gaps related to environment, policy, and physical activity
Top five promising methods that can be used in research now:
1. Natural experiments with good measures that are practical and cost effective	3.97
2. Surveillance systems with good measures of environmental variables	3.44
3. Define individual and environmental factors using mixed methods and other new models to study both simultaneously	3.08
4. Evaluation of policy process	2.93
5. Opportunity to study synergies of environmental factors with health outcomes and ecologic sustainability (e.g., carbon reduction and walking/bicycling)	2.78
Top five research priorities:
1. Prospective studies and evaluations of natural experiments to improve evidence of causality	4.14
2. Test multiple levels of social–ecologic model and evaluate interaction of variables across levels (e.g., environmental and social)	3.54
3. Develop appropriate measurement tools for target populations, particularly those at high risk for obesity (physical activity and correlates)	3.35
4. Understand how local communities can be mobilized to initiate policy change	3.25
5. Understand how local (and other) policies are implemented (e.g., how implementation varies across different locales or populations and what factors affect implementation)	3.05
Top two priorities for funding and communicating research:
1. Dedicated funding and study sections for multilevel, environmental, and policy studies	2.64
2. Speed up dissemination of results so that they can be translated more quickly to policy and practice changes (alternatives to journals, including blogs, working papers, websites)	2.57
TOPIC 2: Research status and gaps related to environment, policy, and diet
Top five promising methods that can be used in research now:
1. Policy change evaluations that assess (1) implementation, (2) enforcement, (3) community acceptance, and (4) impact over time on rates of obesity or obesogenic nutrition behaviors	3.78
2. Surveillance research to track changes in food-industry activities with the potential to influence nutrition behaviors (e.g., packaged portion sizes, reduced-calorie options) would allow researchers to (1) identify opportunities for natural experiments, (2) examine the influence of industry activities on nutrition behaviors and obesity, and (3) determine how industry activities shift in response to policy changes	3.08
3. Observational multilevel studies, including research designed to examine interactions between individuals and food environments (e.g., what individual factors increase susceptibility to obesogenic food environments)	2.89
4. Studies designed to examine (or quantify) the influence of multiple environmental domains and their interactions on rates of obesity (or obesogenic nutrition behaviors)	2.79
5. Cross-disciplinary and transdisciplinary collaborations that incorporate complementary methodologies (e.g., qualitative and quantitative approaches)	2.76
Top five research priorities:
1. Conduct research in minority and low-income populations, such as the evaluation of policies to reduce/eliminate disparities in access to food (e.g., tax incentives for stores in low-income neighborhoods)	3.86
2. Develop standardized measures of food environment and nutrition policies (for various types of environments and contexts) to improve the comparability of findings across studies)	3.67
3. Examine motivations for food choices, including tensions between internal and external (environmental) factors on behavior	2.94
4. Conduct research relating to home and family food environments	2.76
5. Conduct research guided by systems theory	2.60
Top two priorities for funding and communicating research:
1. Create special funding mechanisms for conducting time-sensitive natural experiments and collaborative research	1.73
2. Encourage cross-disciplinary collaborations and mentoring of junior scientists by those experienced in this area	1.57
TOPIC 3: Promising study designs for environmental and policy research and evaluation
Top five promising study designs:
1. Promote rapid evaluation of natural experiments	4.17
2. Develop policy surveillance measures and data-collection systems	3.84
3. Promote health impact assessment techniques	3.38
4. Support international research and comparisons to expand range of environments and policies assessed	2.95
5. Exploit and promote analysis of cohort/panel data	2.85
TOPIC 4: Developing measures of policy for obesity, diet, and physical activity
Top five priorities for developing measures of policy:
1. Measures to support surveillance of food- and activity-related policies	3.54
2. Measures of strength of policy, policy enforcement, policy implementation that can be compared across policy domains	3.50
3. Improved measures of valuation: What do different communities value? Valuation models in the health field	3.07
4. Develop HIA methods and funding mechanisms	3.00
5. Develop measures of community support for policy changes	3.00
TOPIC 5: Statistical approaches for environmental and policy research
Top five priorities for statistical approaches:
1. Develop an accessible compendium of metrics or indicators derived from available environmental measures	3.43
2. Conduct training workshops in designing and analyzing multilevel studies	3.14
3. Use both quantitative and qualitative methods, including case studies	3.13
4. Training in analytic techniques tailored for small samples	2.95
5. Consider variability of environmental and policy factors in designing studies: How much variability is necessary?	2.89

^aBased on rankings of conference participants (n=41)

Survey Results and Discussion

The top-ranked two to five recommendations for each list are provided in Table 2. Recommendations varied by topic area, but several themes repeatedly emerged as priorities.

Several useful research methods were identified that could advance the knowledge of obesity-related environmental and policy factors but were underutilized due to the limitations of funding mechanisms or a lack of familiarity among investigators. More emphasis on rigorous prospective investigations or quasi-experimental evaluations of natural experiments (i.e., environment or policy changes not controlled by the investigator) would advance this field, which has relied mainly on cross-sectional studies. Using existing measures to conduct surveillance of food and built environments and industry practices could advance both research and public health practice. There was strong consensus that additional cross-sectional, multilevel studies would be valuable to improve the understanding of interactions across environmental, social, and individual factors, and to examine how multilevel associations and the outcomes of interventions may vary by age and population subgroup. (A multilevel approach takes into account the individual, social, and environmental factors related to physical activity and eating and considers how each level influences behavior or interacts with the other levels.)

Research priorities were identified that had strong promise for informing approaches to reverse the obesity epidemic. Studies of policymaking and implementation, including methods of engaging communities in advocacy, were identified as high priority. Health impact assessment was ranked as a promising policy-assessment method in need of further development. Incorporating economic methods into diet and physical activity research also was recommended. To facilitate such research, improvements in policy measurement and the engagement of policy research experts were viewed as necessary. Some recommendations for speeding the communication and application of research may be outside the purview of research-funding agencies but could be considered by other scientific organizations and health policy groups. A repeated theme was the urgent need for research that could inform environmental and policy solutions for the low-income populations and communities of color at highest risk of obesity. Tailored measures, interactions between environmental and individual variables, policy-change processes, and community-engagement methods need to be examined for each high-risk subgroup. Mixed methods involving quantitative and qualitative data collection are promising for accelerating the research on high-risk populations.

Conference participants endorsed the need for supporting the development and use of common environmental measures and training in appropriate statistical strategies for environmental and policy research. In particular, training in the methods to analyze multilevel studies, the analysis of studies with small numbers of units, and the utilization of combinations of qualitative and quantitative data were deemed high priorities.

Because of the particular challenges of conducting environmental and policy research as well as the need for collaboration among diverse interdisciplinary groups, dedicated research-funding mechanisms were recommended. Advanced training in methods of environmental and policy research and support for interdisciplinary collaboration were highly ranked.

Conclusion

The continuing obesity epidemic is one of the most serious threats to health in the U.S. and across the globe.^1–5 Although public health authorities have identified environmental and policy changes as essential to controlling the obesity epidemic,^3–9 research to generate evidence-based solutions is hampered by methodologic challenges. The conference described here engaged a diverse group of investigators and funders to recommend strategies for advancing environmental and policy research related to obesity, diet, and physical activity. Through a systematic process, specific recommendations were identified for making better use of current methods, improving measures and methods for future studies, enhancing the capacity of investigators to conduct challenging environmental and policy studies, supporting collaboration among diverse disciplines, and accelerating the application of research to changes in policy and practice. To overcome the challenges, funding agencies were encouraged to take actions that would result in a greater priority being given to environmental and policy research and to additional training and support for investigators.

The results of this conference can be used to improve the quality and quantity of environmental and policy research as well as the translation to action to control obesity, which is consistent with the goals of major funding organizations, including the NIH^17,18 (www.nhlbi.nih.gov/meetings/workshops/child-obesity/index.htm); the CDC⁸; and the Robert Wood Johnson Foundation (www.activelivingresearch.org and www.healthyeatingresearch.org). The recommendations of this group of experts merit careful consideration by investigators and research-funding agencies.