Abstract
Objective
In order to address the social determinants of health, an increasing number of public health practitioners are implementing Health in All Policies initiatives aimed at increasing cross-sector collaboration and integrating health considerations into decisions made by “non-health” sectors. Despite the growth in practice nationally and internationally, evaluation of Health in All Policies is a relatively new field. In order to help inform evaluation of Health in All Policies initiatives in the United States, this study sought to develop a practice-grounded approach, including a logic model and a set of potential indicators, which could be used to describe and assess Health in All Policies activities, outputs, and outcomes.
Design
Methods included: a) a review of the literature on current Health in All Policies approaches, practices, and evaluations; and b) consultation with experts with substantive knowledge in implementing or evaluating Health in All Policies initiatives. Feedback from experts was obtained through individual (n=11) and group (n=14) consultation.
Results
The logic model depicts a range of potential inputs, activities, outputs, and outcomes of Health in All Policies initiatives; example indicators for each component of the logic model are provided. Case studies from California, Washington, and Nashville highlight emerging examples of Health in All Policies evaluation and the ways in which local context and goals inform evaluation efforts.
Conclusions
The tools presented in this article synthesize concepts present in the emerging literature on Health in All Policies implementation and evaluation. Practitioners and researchers can the tools to facilitate dialogue among stakeholders, clarify assumptions, identify how they will assess progress, and implement data-driven ways to improve their Health in All Policies work.
Keywords: Health in All Policies, social determinants of health, evaluation
Introduction
Health is determined by multiple factors outside the direct control of the health care and public health systems, such as access to educational, economic, and job opportunities; transportation and housing options; and neighborhood quality and safety.1–3 These “social determinants of health” – which shape the environments where people live, work, and play – affect health, functioning, and quality-of-life outcomes.1 To address the social determinants of health, public health systems have begun to expand their scope of practice to include partnerships with a broader range of sectors, using strategies such as Health in All Policies (HiAP). HiAP is “an approach to public policies across sectors that systematically takes into account the health implications of decisions, seeks synergies, and avoids harmful health impacts in order to improve population health and health equity.”4 HiAP has an explicit focus on long-term, systemic change and intersectoral engagement.5–7 HiAP initiatives are coordinated primarily by formal structures and mechanisms of governments, although they may include nongovernmental actors.5,8
There is substantial variation in the process, structure, and scope of HiAP initiatives; to support practitioners, guiding principles and strategies have been identified. Rudolph et al (2013)9 defined five key elements that drive a HiAP approach: promoting health equity and environmental sustainability; supporting intersectoral collaboration; benefiting multiple partners; engaging stakeholders; and creating structural or procedural change. The authors emphasize the importance of thinking beyond formal legislation when considering “policy,” to include institutional practices such as decision-making processes, allocation of resources, and priority-setting. In their review of HiAP in the United States, Gase et al (2013)10 identified seven commonly used strategies to facilitate HiAP implementation at the federal, state, and local levels: developing and structuring cross-sector relationships; incorporating health into decision-making processes; enhancing workforce capacity; coordinating funding and investments; integrating research, evaluation, and data systems; synchronizing communications and messaging; and implementing accountability structures.
Despite the growth in HiAP practice, the evaluation of HiAP initiatives is a relatively new field.11 There are many challenges to comprehensive evaluation efforts, including the complex nature of intersectoral policy development, variation in the implementation of HiAP initiatives, and the complexities inherent in attributing outcomes to HiAP work.11–13 The scant literature on HiAP evaluation often acknowledges two persisting gaps in knowledge: first, the scarcity of information on the processes of HiAP agenda setting, development, and implementation; and second, the lack of guidance for comprehensive approaches to evaluation that considers the links between the short- and long-term outcomes of HiAP initiatives.14–15
Recent work, primarily conducted in the international sphere, has addressed the first of these gaps by examining the process of HiAP development and implementation. Storm et al (2014)16 developed and tested a model which classified HiAP implementation into six stages, ranging from unrecognized to institutionalized. The authors defined key characteristics at the individual, organizational, and political levels for measuring HiAP growth; management strategies most relevant for moving HiAP initiatives to subsequent stages; and core “necessary conditions” for initiatives at each stage. Shankardass et al (2015)17 developed a theoretical conceptualization for the study of HiAP that emphasizes the importance of the context of initiation and implementation in influencing mechanisms of HiAP collaboration. The authors outline activities that lead to outcomes of acceptability (are sectors willing to collaborate); feasibility (do sectors have the capacity to collaborate); and, ultimately, sustainability (will collaboration last).
A second area of HiAP evaluation has focused on assessing the impact of specific tools or processes. As one component of a broader assessment, South Australia examined the impact of its health lens analysis process; findings suggest that health lens analyses have resulted in increased understanding by policy-makers, changes in policy direction, increases in policy-relevant research, and stronger partnerships.12,18 Researchers in the Netherlands examined the impact of a HiAP coaching program to study whether municipalities were able to make progress in intersectoral collaboration at the strategic, tactical, or operational levels; findings showed mixed results.19 Groups in Australia, New Zealand, and the United States have shown promising impacts of Health Impact Assessments (HIA) on decision-making processes and have described factors that can facilitate their success.20–22
The present study builds on previous work by identifying a potential approach and set of tools that practitioners and researchers can use to evaluate HiAP initiatives. Based on the Centers for Disease Control and Prevention (CDC) Framework for Program Evaluation in Public Health23 and other evaluation references,24 this study sought to develop: 1) a logic model to describe the sequence of events for bringing about change (i.e., a graphical representation of how the HiAP initiative is supposed to work) and 2) a set of potential indicators that could be used to define and measure progress of HiAP initiatives. Acknowledging the need for guidance relevant to the US context, three case studies were identified to illustrate emerging examples of HiAP evaluation nationally. This practice-grounded effort aims to provide a starting place for discussion about how to evaluate HiAP initiatives, rather than a definitive answer.
Methods
Between June 2015 and January 2016, the study authors used an iterative process of literature review and expert consultation to develop the evaluation tools and case studies.
Literature Review
Two reviews of the literature were conducted in June 2015 (preliminary review) and October 2015 (targeted, in-depth review) to understand a) current HiAP approaches and practices and b) evaluation frameworks or specific evaluations of HiAP initiatives. Due to the limited literature in the United States, the authors looked to international evaluation work, including efforts in Australia, Europe, and Canada. Queried databases included PubMed, Google Scholar, and JSTOR. Search terms included "Health in All Policies" AND (“evaluation” OR “assessment” OR “study” OR “framework”). In addition, abstracts from the 2015 National Health Impact Assessment Meeting and the 2015 Annual Meeting of the American Public Health Association as well as websites showcasing HiAP activities, including the CDC, the National Association of County and City Health Officials, and the National Network of Public Health Institutes were searched. While no literature was excluded because of its published date, recent sources were prioritized.
Expert Consultation
Experts – individuals with substantive knowledge in implementing or evaluating HiAP or HIA –provided feedback on draft products. The lead author facilitated two group-based input processes: 1) in June 2015, during a post-conference workshop of the National Health Impact Assessment Meeting (n=8 individuals who self-selected to participate in a group discussion on HiAP evaluation), and 2) in September 2015, during a HiAP community of practice call (n=6 practitioners who meet regularly to discuss HiAP implementation issues). Individual feedback was solicited via email from 11 additional experts, identified based on the professional networks of the study authors; all provided comments via email or a brief (30 minute) phone meeting. Experts were provided with a description of the project and a draft of the logic model and indicators and asked to “provide feedback on the scope and content of the tools, especially in light of your own HiAP efforts.”
Results
Evaluation Logic Model and Indicators
The logic model depicts a range of inputs, activities, outputs, and outcomes of HiAP initiatives (Figure 1). On the far left, general inputs (i.e., resources used to undertake HiAP work) include staff, funding, policies or mandates, data, and other resources. The next column lists eight potential activities for implementing HiAP. Seven of these activities are drawn from previously identified strategies for implementing HiAP in the United States.10 One additional activity, building support and awareness, was added to emphasize the importance of creating awareness and cultivating champions to implement a HiAP approach (e.g., through targeted communications strategies). Building support and awareness and developing cross-sector relationships are at the top of the model because they are often foundational to implementing other activities. In practice, the nature and scope of activities is likely to vary across jurisdictions depending on a range of contextual factors such as organizational culture and capacity, policy windows, and stakeholder and leadership buy-in.
Figure 1.
Logic Model Depicting the Potential Inputs, Activities, Outputs, and Outcomes of Health in All Policies Initiatives
In the third column, outputs (i.e., the tangible products of HiAP activities) include collaboration structures, engagement tools and processes, political support and capacity structures, plans and protocols, assessments, and accountability structures. Along with tracking activities, measuring outputs (examples of output indicators provided in Table 1) allows for process evaluation to identify differences between the way a HiAP initiative is intended to operate and the way it has been implemented.24 The outputs included in the model represent the links between HiAP implementation and its potential outcomes; they can help provide a better understanding of what has been created as a result of HiAP-focused activities, how well the initiative is working, and where improvements can be made.
Table 1.
Example Process and Outcome Indicators for Evaluating Health in All Policies Initiatives
| Outputs | Example Indicators |
| Collaboration structures |
|
| Engagement tools and processes |
|
| Political support and capacity structures |
|
| Plans and protocols |
|
| Assessments |
|
| Accountability structures |
|
| Short-Term Outcomes | Example Indicators |
| Strengthened partnerships between health, other sectors, and non- governmental stakeholders |
|
| Increased understanding, perceived value of, and commitment to advance HiAP across sectors |
|
| Increased consideration of health/equity in a defined policy, plan, or project across sectors |
|
| Increased capacity and systems for engaging in and implementing HiAP across sectors |
|
| Intermediate Outcomes | Example Indicators |
| Institutionalized policies, practices, and funding that support health and equity across sectors |
|
| Sustainable systems that support cross-sectoral collaboration |
|
Judging the impact of HiAP activities and associated outputs requires a focus on the next four columns: the short, intermediate, and long-term outcomes of HiAP implementation. There are four potential short-term outcomes: strengthened partnerships, increased understanding and commitment to HiAP, increased consideration of health and equity, and strengthened capacity and systems for engaging in HiAP. The two intermediate outcomes focus on sustainability and institutionalization of these practices, while the three long-term outcomes include improvements in social and physical environments (i.e., social determinants of health), more efficient and effective government, and improved population health and equity. Paired with assessments of formative and process measures, measuring outcome indicators (examples provided in Table 1) can help determine a HiAP initiative’s effectiveness in meeting its goals.
Each of the logic model’s core components will vary depending on the context and goals of the HiAP initiative under study. These components are also likely to vary over time. The process of developing and implementing a HiAP approach is not linear; feedback loops and interactions connect HiAP activities, outputs, outcomes, and contextual factors. As described in other conceptualizations of HiAP implementation,11,25 the processes used to build relationships, train staff, communicate with decision-makers, frame problems, analyze data, and develop policy alternatives, operate in a complex and iterative web. Early successes of one activity, for example, may lead to changes in the level of stakeholder buy-in, which may facilitate implementation of additional activities. As HiAP has an explicit focus on long-term implementation,5 the important role of time in advancing the implementation and institutionalization of HiAP should be considered.
Evaluation Design and Methods
The logic model and indicators can be applied using a variety of evaluation approaches, designs, and methods. Internationally, a number of HiAP evaluations have been driven by a critical action research approach12,19 – whereby emerging findings inform ongoing implementation – and a realistic evaluation approach12,17 – which emphasizes mechanisms and context (i.e., how and why something is working). Related approaches can also be considered, such as developmental evaluation and improvement science, which are suitable for situations of high complexity in their early stages of development and emphasize conducting small tests of change to support innovation.26–27
The evaluation design employed should be driven by the evaluation questions of greatest interest. Case studies have been commonly used to gain in-depth information about a concrete HiAP project.17–18 Such a design is especially useful in providing an understanding of contextual influences and diverse actor perspectives. Alternative designs, such as quasi-experimental designs, that include multiple measurement time points (e.g., time series) or data from a non-equivalent comparison group (e.g., a neighboring jurisdiction not implementing HiAP) may be useful in understanding the impacts of HiAP. Such designs improve on basic pre-test post-test designs by ruling out many of the threats to internal validity, such as history and selection.28 Efforts to track a set of output or outcome indicators over time can complement in-depth assessments of a specific HiAP process or project.
With regard to data collection methods, a variety of both quantitative and qualitative options are available. Choice of methods will depend on the evaluation questions of interest and practice constraints (e.g., staffing, timeline, funding). Mixed-methods approaches can offer a robust picture of the phenomenon under study by providing concrete numbers (e.g., of policies implemented) as well as the meaning, context, or reasons underlying such figures (e.g., ways in which the policies were changed). Key informant interviews or surveys could be conducted to assess stakeholder knowledge, perspectives, experiences, or behaviors. Social network analysis or assessments of coalition effectiveness can be used to examine the number and types of connections between individuals or organizations, identify how changes spread within/across organizations, and provide input on the quality of collaborative structures (e.g., trust, effectiveness). Document analysis could be conducted to assess the themes or indicators present in a variety of written materials, such as legislative policies, organizational protocols/policies, instructional/training documents, meeting minutes, or staff position descriptions.
Case Studies
Three case studies illustrate emerging efforts to evaluate HiAP initiatives in the United States.
California Health in All Policies Task Force
The California HiAP Task Force, formed in 2010 through a gubernatorial executive order, convenes 22 state agencies with the goal of promoting health, equity, and environmental sustainability by embedding health and equity into government decision-making. Key activities include building cross-sector relationships, developing shared language and goals, building the capacity of individuals and member agencies to understand how their work impacts health and equity, and aligning goals and actions across government. The California Department of Public Health, in partnership with the Public Health Institute, staffs the Task Force by conducting background research; providing policy analysis and public health expertise; and facilitating collaborative decision-making. The Task Force has many accomplishments, including multi-agency work plans that outline collaborative commitments for action.29 Developing the work plans through a consensus-based, stakeholder-driven process has strengthened cross-sector relationships and increased partners’ understanding of the opportunities and importance of embedding health and equity into their work.30 To date, work plan implementation has produced many outputs, including a state-level farm-to-fork office31 and criteria for incorporating equity and health into state grant-making guidelines.32
A plan for a formal evaluation to examine the impact and effectiveness of the Task Force is currently being developed. Key evaluation questions include: a) How has the California HiAP initiative promoted cross-sector collaboration?; b) To what extent has the Task Force influenced policies that can impact population health, well-being, and equity?; and c) What factors facilitate the Task Force in meeting its goals (e.g., leadership support, legislative or mandated structures, backbone staff)? Potential indicators include the level of investment in cross-sector collaboration, number and types of policies that support collaboration, number and quality of partnerships across sectors, number and types of shared health/equity goals, frequency of reference to health and equity in “non-health” government programs, and number and types of changes to policies, processes, and programs. The evaluation aims to identify activities that are most effective in achieving the goals of the initiative, which can inform future work of the Task Force and the larger HiAP movement. In addition, demonstrating the impacts of this approach may help secure additional resources to support HiAP initiatives.
Washington State Health Impact Review Authority
The Washington State Board of Health (Board), in collaboration with the Governor’s Interagency Council on Health Disparities (Council), conducts Health Impact Reviews (HIR) to support HiAP implementation within the state’s legislative process. A HIR uses published scientific literature, data and/or expert opinion to provide an objective, evidence-based analysis of a proposed legislative or budgetary change to determine its likely impacts on health and health disparities. In 2006, the Board and Council were given statutory authority (RCW 43.20.285) to conduct these reviews if requested by the Governor or a state legislator. By statute, these reviews must be completed within ten days of the request during legislative session.33 The short-term goals of the HIR work are to increase consideration of health and equity in specific state policy and budget decisions and understanding of the social determinants of health among policy-makers. In the intermediate-term, this tool has the potential to institutionalize the consideration of health and equity in state-level decision-making, even outside of the HIR process. Process data showing an increase in the number of requested HIRs (7 requests between 2007 and 2009, compared to 22 requests between 2013 and 2015) suggest the growing acceptance and potential value of such a tool. Additionally, Board staff have observed that the HIR process helps facilitate connections between legislators and community members and other experts who can then work directly with the bill sponsor to share their insights.
An in-depth evaluation of the HIR process will occur in 2016. The evaluation will examine request patterns, including which legislators are making requests, what type of bills/provisions are reviewed, and which legislators are not making requests and why. Because HIRs can only be initiated by request, this information will provide important insights into the program’s potential for growth and improvement. The evaluation will also examine how HIRs are used in the decision-making process (e.g., how policymakers thought about and proceeded on each bill). This information can help inform how best to structure ongoing HiAP work in the state, including how to refine the HIR process or products or implement potentially complementary HiAP activities (e.g., enhanced communications or training). The evaluation will include a content analysis of HIR documents and associated legislation; an online survey of legislators and/or staff; and semi-structured key informant interviews with legislators who requested a HIR, individuals at the Board, and other key partners.
Metro Nashville-Davidson County Health in All Policies Initiative
Nashville’s HiAP initiative was established in 2015 at the request of the Healthy Nashville Leadership Council and Nashville’s Mayor to promote the consideration of health in decision-making. The initiative, housed within the Metro Nashville Public Health Department, seeks to develop and implement social, economic, and environmental policies that promote health and health equity throughout the county. The initiative is implemented through a 3-tier workgroup structure: 1) a Leadership Roundtable of departmental directors, who identify strategies within their respective departments to incorporate health into decision-making; 2) a Health Coordinators Group of inter-departmental staff, who vet existing and forthcoming polices, programs, and plans to ensure they promote health and health equity; and 3) a Steering Committee of health department staff, who provide education through training, workshops, and technical assistance. The goals of the initiative are to enhance understanding of the social determinants of health and encourage cross-sector collaboration and coordination in addressing health-related issues. In the long-term, the initiative aims to develop a sustainable structure and process to improve population health and health equity
A process evaluation of the Metro Nashville-Davidson County HiAP initiative, currently under development, aims to examine the extent to which the initiative is effectively promoting the consideration of health within other Metro agencies’ policies. The evaluation will assess a variety of short-term outcomes, including the characteristics and functioning of the workgroups (e.g., sectors represented, level of participation, types of communication), knowledge of the social determinants of health among participants, and the ways in which participants are considering health in their day-to-day roles. The evaluation will include an online survey administered annually to all workgroup participants, followed (as needed) by in-depth interviews with a sub-set of workgroup participants to understand their experiences. Findings will be shared with workgroup participants to promote transparency, shared interpretation of results, and action planning. By tracking key indicators over time, the evaluation can help identify promising practices, highlight additional training and technical assistance needs, and inform strategies for improvement (i.e., how the process and structure may need to adapt). Findings may also be useful in securing staff time and funding for sustaining the initiative.
Discussion
The evaluation approach and tools presented in this article provide a starting place to describe and measure the range of potential inputs, activities, outputs, and outcomes of HiAP initiatives being implemented in the United States. By integrating concepts present in the emerging literature9–12 and expanding on ways to judge the quality and success of HiAP initiatives, the logic model and indicators can be used by practitioners and researchers to advance the evaluation of HiAP and address some of the challenges associated with this work.
The context-specific nature of HiAP means that there is no standardized approach to either its implementation or evaluation. What practitioners choose to evaluate should be guided by the goals of their program and the types of information needed to improve their efforts or demonstrate success. Because HiAP initiatives often operate within a broader context of health and equity-focused efforts (e.g., healthcare reform, other cross-governmental collaborations), it can be difficult to attribute the work of a HiAP initiative to the outcomes observed. Moreover, because of the political nature of HiAP, not all projects may be documented in public records. Clearly outlining HiAP activities, outputs, and outcomes through a logic model can help address these challenges. Practitioners can use the tools provided in this article to facilitate dialogue among stakeholders, clarify assumptions, identify how they will assess progress, and implement data-driven ways to improve their work.
Evaluation represents a key strategy to build and sustain HiAP practice, helping to facilitate a better understanding of the impacts of different HiAP approaches and key factors that drive success. The work presented in this article provides a starting place to answer many of the outstanding questions related to HiAP in the United States, such as: are there core HiAP activities that can be implemented across contexts?; what inputs are needed to develop a successful HiAP model?; what activities are associated with both intentional and unintentional successes?; and what is the role of community-based partners in driving activities and outcomes?
While the practice-grounded tools presented in this article are a step forward, there are limitations to this work. First, because HiAP is an emerging field in the United States, there were very few published examples of national evaluation efforts. While the study authors gathered input from a number of experts, their experiences may not represent the full scope of HiAP practice. Second, because the logic model and indicators were developed so that they can be applied to a range of HiAP initiatives, they are framed using broad, as opposed to initiative- or sector-specific, language. Practitioners and researchers will need to use the examples provided to determine if and how each component can be applied to their HiAP work. Finally, the logic model and indicators represent works in progress; applying these tools to additional HiAP initiatives will facilitate a better understanding of their applicability and utility.
Implications for Policy & Practice
To facilitate further growth and success in HiAP practice, additional evaluation of HiAP initiatives is needed to better understand how to structure and make the case for investments in cross-sector work. Practitioners and researchers can use the logic model and indicators presented in the article to clarify and measure the goals, activities, and outcomes of their HiAP work. By considering assessment and evaluation approaches at the outset of HiAP strategy development, practitioners can lay a foundation for success within their own processes and contribute to the growing body of practice-based evidence.
Supplementary Material
Acknowledgments
The authors thank all of the experts who provided input and feedback on the evaluation logic model and indicators.
Funding
This work was supported in part by a grant from the National Institutes of Health/National Center for Advancing Translational Science, University of California, Los Angeles Clinical Translational Science Institute [Gase, TL1TR000121].
Footnotes
Conflicts of Interest
The authors declare that they have no conflict of interest.
References
- 1.Healthy People. Office of Disease Prevention and Health Promotion. Washington, DC: U.S. Department of Health and Human Services; 2020. Retrieved from: http://www.healthypeople.gov/2020/topics-objectives/topic/social-determinants-health. [Google Scholar]
- 2.Braveman PA, Egerter SA, Williams DR. The Social Determinants of Health: Coming of Age. Annual Review of Public Health. 2011;32:381–398. doi: 10.1146/annurev-publhealth-031210-101218. [DOI] [PubMed] [Google Scholar]
- 3.Frieden TR. A Framework for Public Health Action: The Health Impact Pyramid. Am J Public Health. 2010;100(4):590–595. doi: 10.2105/AJPH.2009.185652. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.World Health Organization (WHO) 8th Global Conference on Health Promotion: Helsinki. 2013 Retrieved from: http://www.healthpromotion2013.org/health-promotion/health-in-all-policies.
- 5.Freiler A, Muntaner C, Shankardass K, Mah CL, Molnar A, Renahy E, O’Campo P. Glossary for the implementation of Health in All Policies (HiAP) J Epidemiol Community Health. 2013;67:1068–1072. doi: 10.1136/jech-2013-202731. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Ståhl T, Wismar M, Ollila E, Lahtinen E, Leppo K, editors. Health in All Policies: prospects and potentials. Helsinki: Ministry of Social Affairs and Health; 2006. [Google Scholar]
- 7.Kickbusch I, Buckett K, editors. Implementing Health in All Policies. Adelaide: Government of South Australia; 2010. [Google Scholar]
- 8.Government of South Australia: SA Health. Health in All Policies: the 10 principles. 2007 Retrieved from http://www.sahealth.sa.gov.au/wps/wcm/connect/public+content/sa+health+internet/health+reform/health+in+all+policies.
- 9.Rudolph L, Caplan J, Ben-Moshe K, Dillon L. Health in All Policies: A Guide for State and Local Governments. Washington, DC and Oakland, CA: American Public Health Association and Public Health Institute; 2013. [Google Scholar]
- 10.Gase LN, Pennotti R, Smith KD. “Health in All Policies”: Taking stock of emerging practices to incorporate health in decision making in the United States. J Public Health Management Practice. 2013;19(6):529–540. doi: 10.1097/PHH.0b013e3182980c6e. [DOI] [PubMed] [Google Scholar]
- 11.Bauman AE, King L, Nutbeam D. Rethinking the evaluation and measurement of Health in All Policies. Health Promotion Int. 2014;29(S1):i143–i151. doi: 10.1093/heapro/dau049. [DOI] [PubMed] [Google Scholar]
- 12.Baum F, Lawless A, Delany T, MacDougall C, Williams C, Broderick D, Wildgoose D, Harris E, McDermott D, Kickbusch I, Popay J, Marmot M. Evaluation of Health in All Policies: concept, theory and application. Health Promotion Int. 2014;29(S1):i130–i142. doi: 10.1093/heapro/dau032. [DOI] [PubMed] [Google Scholar]
- 13.Pinto AD, Molnar A, Shankardass K, O’Campo PJ, Bayoumi AM. Economic Considerations of Health in All Policies Initiatives: evidence from interviews with key informants in Sweden, Quebec and South Australia. BMC Public Health. 2015;15:171–180. doi: 10.1186/s12889-015-1350-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Khayatzadeh-Mahani A, Sedoghi Z, Mehrolhassani MH, Yazdi-Feyzabadi V. How Health in All Policies are developed and implemented in a developing country? A case study of a HiAP initiative in Iran. Health Promotion Int. 2015:1–13. doi: 10.1093/heapro/dav062. [DOI] [PubMed] [Google Scholar]
- 15.Ndumbe-Eyoh S, Moffatt H. Intersectoral action for health equity: a rapid systematic review. BMC Public Health. 2013;13(1056):1–13. doi: 10.1186/1471-2458-13-1056. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Storm I, Harting J, Stronks K, Schuit AJ. Measuring stages of Health in All Policies on a local level: the applicability of a maturity model. Health Policy. 2014;114:183–191. doi: 10.1016/j.healthpol.2013.05.006. [DOI] [PubMed] [Google Scholar]
- 17.Shankardass K, Renahy E, Muntaner C, O’Campo P. Strengthening the implementation of Health in All Policies: a methodology for realist explanatory case studies. Health Policy and Planning. 2015;30:462–473. doi: 10.1093/heapol/czu021. [DOI] [PubMed] [Google Scholar]
- 18.Lawless A, Williams C, Hurley C, Wildgoose D, Sawford A, Kickbusch I. Health in All Policies: evaluating the South Australian approach to intersectoral action for health. Canadian Journal of Public Health. 2012;103(S1):s15–s19. doi: 10.1007/BF03404454. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Steenbakkers M, Jansen M, Maarse H, de Vries N. Challenging Health in All Policies, an action research study in Dutch municipalities. Health Policy. 2012;105:288–295. doi: 10.1016/j.healthpol.2012.01.010. [DOI] [PubMed] [Google Scholar]
- 20.Bourcier E, Charbonneau D, Cahill C, Dannenberg AL. An Evaluation of Health Impact Assessments in the United States, 2011–2014. Prev Chronic Dis. 2015;12:140376. doi: 10.5888/pcd12.140376. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Haigh F, Baum F, Dannenberg AL, Harris MF, Harris-Roxas B, Keleher H, et al. The effectiveness of health impact assessment in influencing decision-making in Australia and New Zealand 2005–2009. BMC Public Health. 2013;13:1188. doi: 10.1186/1471-2458-13-1188. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Haigh F, Harris E, Harris-Roxas B, Baum F, Dannenberg AL, Harris MF, et al. What makes health impact assessments successful? Factors contributing to effectiveness in Australia and New Zealand. BMC Public Health. 2013;15(1):1009. doi: 10.1186/s12889-015-2319-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Centers for Disease Control and Prevention (CDC) Framework for program evaluation in public health. MMWR. 1999;48(No. RR-11) [PubMed] [Google Scholar]
- 24.Crosby RA, DiClemente RJ, Salazar LF. Research methods in health promotion. San Francisco, CA: Jossey-Bass; 2006. [Google Scholar]
- 25.de Leeuw E, Peters D. Nine questions to guide development and implementation of Health in All Policies. Health Promotion International. 2014 doi: 10.1093/heapro/dau034. [DOI] [PubMed] [Google Scholar]
- 26.Gamble J. A developmental evaluation primer. Montréal: The J.W. McConnell Family Foundation. 2008 Retrieved from: http://aea365.org/blog/michael-quinn-patton-on-developmental-evaluation-applying-complexity-concepts-to-enhance-innovation-and-use/#sthash.K8xdLZ7M.dpuf. [Google Scholar]
- 27.Kuo T, Gase LN, Inkelas M Population Health and Policy Workgroup. Dissemination, implementation, and improvement science research in population health: opportunities for public health and the CTSAs. Clinical and Translational Science. 2015 doi: 10.1111/cts.12313. [Epub ahead of print] [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Campbell DT, Stanley JC. Experimental and quasiexperimental designs for research. Boston: Houghton Mifflin Company; 1966. [Google Scholar]
- 29.Strategic Growth Council, Health in All Policies Task Force. Focus Areas and Action Plans. Retrieved from: https://www.sgc.ca.gov/s_abouthiaptaskforce.php.
- 30.Strategic Growth Council, Health in All Policies Task Force. Resources, Reports and Guiding Documents. Retrieved from: https://www.sgc.ca.gov/docs/HiAP_Charter_updated_1.12.15.pdf.
- 31.Strategic Growth Council, Health in All Policies Task Force. Farm to Fork Action Plan. Retrieved from: https://www.sgc.ca.gov/docs/Farm_to_Fork_Final_Implementation_Plan_HiAP.pdf.
- 32.Strategic Growth Council, Health in All Policies Task Force. Integrating health and equity into non-health grants case story. Pending public release. 2016 [Google Scholar]
- 33.Washington State Board of Health. Health Impact Reviews Information Sheet. 2015 Retrieved from: http://sboh.wa.gov/OurWork/HealthImpactReviews.
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.

