Implementation of the Health Equity Impact Assessment (HEIA) tool in a local public health setting: challenges, facilitators, and impacts

Abstract

Setting

The Simcoe Muskoka District Health Unit (SMDHU) is a local public health agency in Ontario, serving the County of Simcoe and District of Muskoka’s population of over 540,000 people in a mixed urban-rural environment. SMDHU has had a strategic focus on the determinants of health since 2007.

Intervention

The use of the Health Equity Impact Assessment (HEIA) tool was encouraged throughout SMDHU. An online survey was conducted to assess the implementation of mitigation strategies identified through the HEIAs; in particular, the facilitators and barriers/challenges, and the impacts at both the program level and the level of priority populations, themselves, were assessed.

Outcomes

Agency programs reported 40 HEIAs conducted between 2013 and 2017. While three of the 40 HEIAs were at an early stage, the remaining 37 were at an advanced stage of implementing mitigation strategies, or beyond. Reports from these 37 advanced-stage HEIAs indicated that 30 of them observed or measured impacts on their programs, services, and/or policies (i.e., program-level impacts), following the implementation of HEIA mitigation strategies. These impacts included improved collaboration with external partners, development of more accessible and inclusive communication, increased availability of information and population health data, increased access to services for priority populations, reduced financial barriers for accessing services, and increased staff training on health equity and priority populations. However, few respondents had measured or purposefully assessed impacts on target populations themselves (i.e., population-level impacts); therefore, such impacts are unknown. Key facilitators to the implementation of mitigation strategies were availability of staff, collaboration and support from other areas of the organization, sufficient time, collaboration with community partners, and the inclusion of HEIAs in program operational plans. Absence of these facilitators was identified as a barrier to implementation of mitigation strategies.

Implications for policy and practice

The use of HEIAs served as a catalyst for change in this local public health agency, leading to equity-oriented impacts on public health programming, planning, service delivery, and organizational policy. Support and capacity-building for measuring and evaluating impacts of mitigation strategies on intended priority populations themselves are recommended, to ensure more robust learning and improvement.

Setting

Simcoe Muskoka and priority populations

The Simcoe Muskoka District Health Unit (SMDHU) is a local public health agency in Ontario, serving the County of Simcoe and District of Muskoka’s population of over 540,000 people, in a mixed urban-rural environment. SMDHU provides a wide range of programs and services, in accordance with the Ontario Public Health Standards: Requirements for Programs, Services, and Accountability (Ministry of Health and Long-Term Care 2018b).

The Simcoe Muskoka population includes a number of priority populations at risk of negative health outcomes (SMDHU 2019).

Priority populations are those that are experiencing and/or are at increased risk of poor health outcomes due to the burden of disease and/or factors for disease; the determinants of health including the social determinants of health; and/or the intersection between them. They are identified using local, provincial and/or federal data sources; emerging trends and local context; community assessments; surveillance; epidemiological and other research studies. (MOHLTC 2018a)

Examples of priority populations that may be impacted by health inequities in Simcoe Muskoka include persons living with disabilities, francophone communities, Indigenous communities, people experiencing homelessness, LGBTQ2S people, people who live in rural and/or remote areas, people with low educational attainment, people living in low income, racialized populations, immigrant and refugee populations, and age groups that may be particularly vulnerable to the social determinants of health (i.e., children and seniors). There are also significant intersections between these groupings.

Since 2007, SMDHU has had a strategic focus on addressing the determinants of health that create health inequities in our population. A comprehensive health equity framework and set of recommendations were developed to address this. One of those recommendations included the consistent use of a Health Equity Impact Assessment (HEIA) tool throughout the agency.

Intervention

Health Equity Impact Assessment tool

A health equity tool is a document or resource that provides a set of steps, questions, or framework that lead to the goal of improving health equity by promoting the inclusion of health equity in policies and programs (Pauly et al. 2016). The tool identifies and targets populations that may require extra attention during program planning and implementation.

The Ontario HEIA tool was developed by the Ontario Ministry of Health and Long-Term Care (MOHLTC) to support improved health equity, reduction of avoidable health disparities between population groups, and improved targeting of health care investments (MOHLTC 2019). The HEIA tool comprises a workbook and template to help identify unintended potential impacts of a policy, program, or initiative on vulnerable, marginalized, or priority populations within general populations (MOHLTC 2012). The Ministry, in collaboration with Public Health Ontario, also created a HEIA Public Health Unit Supplement to assist local public health agencies to use the tool to meet the health equity requirements of the Ontario Public Health Standards (Public Health Ontario 2012). There are five steps in the HEIA process: (1) scoping of populations that may be impacted by the initiative; (2) potential impacts of the initiative; (3) mitigation of negative impacts (i.e., recommended changes to the initiative); (4) monitoring of the initiative; and (5) dissemination of results and recommendations.

HEIA is a decision support tool which walks users through the steps of identifying how a program, policy or similar initiative will impact population groups in different ways. HEIA surfaces unintended potential impacts. The end goal is to maximize positive impacts and reduce negative impacts that could potentially widen health disparities between population groups—in short, more equitable delivery of the program, service, policy, etc. (MOHLTC 2019)

The use of HEIAs has been widely promoted in Ontario as a tool for addressing health inequities throughout the health care sector through several organizations, including the MOHLTC and the Centre for Addiction and Mental Health (CAMH) (CAMH 2019). A research project was conducted in 2017 to assess the use of the HEIA tool in Ontario’s local public health agencies specifically (Thomson & Shankardass 2017). Based on responses from 29 of 36 health agencies, all reported an awareness of HEIA, and most (79.3%) had completed an equity assessment of some type, including 19 (66%) that had used HEIA specifically. The majority of health agencies found the tool “Useful” or “Very Useful” and outlined facilitators and barriers for HEIA use. Even with these results, evaluation of health equity-related activities within local public health agencies is still in its early stages; therefore, there is a need to prioritize and strengthen evaluation of health equity activities (Cohen et al. 2018).

In Manitoba, Canada, an Equity-Focused Health Impact Assessment (EFHIA) was conducted on a proposed provincial Teen Triple P (TTP) Positive Parenting Program, with the aim of achieving equity of access and outcomes for families of diverse backgrounds (Cohen et al. 2016). An evaluation of the EFHIA found that the recommendations from it were well received and raised equity-related issues that will be considered in future program planning decisions, and at the time of the evaluation were being further explored through a small TTP pilot.

Elsewhere, the use of HEIAs and related health equity tools has not been widely evaluated, particularly in the context of local public health practice. Evaluations to date have more often been process-oriented than outcome-oriented. Those that describe outcomes—including indicators of effectiveness/impact—have reported findings that are cautiously optimistic. For example, an evaluation of an EFHIA in Australia found that equity-oriented changes to development and implementation of a health promotion policy implementation plan occurred as a result of the EFHIA, though noted uncertainty about attributing changes solely to the EFHIA (Harris-Roxas et al. 2011). An evaluation of three additional EFHIA case studies, conducted on health sector plans in Australia, found that the EFHIAs all had some impact on decision-making about the plans, via enhanced understanding of equity and development of options to modify plans accordingly (Harris-Roxas et al. 2014). However, only one of the case studies had documentation that their plan was actually revised to incorporate the EFHIA recommendations.

As such, there remains a need to continue documenting the implementation experience and, in particular, the impacts of HEIAs and related tools—both on public health practice and on populations themselves. Health equity tools can be strengthened by sharing knowledge of how organizations respond to the results of evaluations of health equity processes (Harris-Roxas et al. 2014). This evaluation seeks to add to that body of literature through the experience of SMDHU.

Data collection: HEIA mitigation implementation survey

SMDHU began conducting HEIAs in 2013, in keeping with its strategic direction on the determinants of health. All programs within the agency were required to conduct a HEIA on a service, program, initiative, or organizational policy of their choice.

In 2017, SMDHU undertook an evaluation to determine the impacts on programs, services, policies, and/or priority populations as a result of implementing the HEIAs, and the facilitators and barriers to implementation. Results would inform the future use of the HEIA tool within the agency.

A survey questionnaire (Appendix 1) using a mixed quantitative and qualitative method was developed to collect data to understand the impacts of conducting the HEIAs and implementing the mitigation strategies. The survey questions were developed with the intent to supplement a gap in information about HEIA mitigation strategy impacts that was not queried during a previous assessment of HEIAs conducted by the agency in 2013. This previous assessment (completed in 2014) provided a summary of the numbers and types of HEIAs conducted and described the effect the HEIA process had on SMDHU’s practices.

An online survey method was used to collect the required information from a purposeful sample of agency management staff over a 10-week period in 2017. The survey collected information on HEIAs conducted at the agency over the 4-year period between 2013 and 2017 in order to answer the following questions:

1. What are the observed and/or measured changes to SMDHU’s services, policies, and/or programs as a result of implementing HEIA mitigation strategies? (program-level impacts)

2. What are the observed and/or measured impacts on the priority population(s) as a result of implementing HEIA mitigation strategies? (population-level impacts)

3. What are the challenges and/or facilitators faced by programs in implementing HEIA mitigation strategies?

In the analysis, frequencies and percentages of responses were calculated for each question. A textual and content analysis of open-ended responses was done by reviewing and analyzing texts for emerging themes and/or repeated occurrences that may suggest the observations were not isolated.

Limitations

Three limitations were identified in relation to the survey data collection method, data sources, and analysis. Where possible, the effects of these limitations were mitigated.

• Technical: the online survey platform was incompatible with some browsers, so respondents were advised to use the most compatible browser(s) identified.

• Recall bias: the survey asked current program managers about events that occurred up to 4 years previous. In addition, some managers may have been new to the role and may not have had direct knowledge of their program’s HEIA project, which occurred prior. To mitigate this, we asked program managers to involve the program staff with the most knowledge of the HEIAs in the completion of the survey.

• Social desirability bias: the survey was administered to program managers by SMDHU—their employer—and was not anonymous. To mitigate the potential for responses to tend towards being positive, managers were informed that the survey purpose was for the agency’s learning and to inform future use of the HEIA tool within the agency.

Outcomes

The survey received 40 responses, each representing a HEIA. All 23 programs of the agency reported at least one HEIA, with many programs reporting more than one. Therefore, a 100% response rate was achieved for programs (23/23), and it is likely that programs reported all their HEIAs (40/40), although there were no other means through which to confirm this. Three of the 40 HEIAs reported were at the scoping or potential impacts stage (step 1 or 2) and had not reached the implementation stage of their mitigation strategies and were therefore screened out of the survey. Over 40% of the HEIAs (17) had reached the final step of implementation (Step 5: Dissemination) (Fig. 1).

Fig. 1.

Stages of implementation of HEIA mitigation strategies (n = 40)

Facilitators to implementation of HEIA mitigation strategies

The most commonly identified factors that facilitated the implementation of HEIA mitigation strategies were the availability of staff resources, sufficient time, collaboration and support from other program areas in the agency, collaboration with community partners, understanding of the HEIA tool, HEIA being included in program mandate and/or operational plan, staff competency/skill, availability of required data/information, and support of their director for the HEIA process (Table 1).

Table 1.

Facilitators for implementation of HEIA mitigation strategies

Answer choices*	# of responses
Staff resources available	25
Collaboration and support from other areas of the organization (if it was required)	25
Sufficient time	22
Collaboration with community partners (if it was required)	18
In my program’s mandate or operational plan	18
Understanding of the HEIA tool: why use it and how to use it	17
Staff competency/skills	17
Required information/data were available	16
Director support	15
Budget resources available	14
Agency support	12
Staff training provided	11
Collaboration with the priority population(s) (if it was required)	11
Other (please specify)	4
None of the above	0
Total number of respondents for this question = 33

*Respondents could choose as many facilitators as applicable

Qualitatively, respondents reported that organizational and management support for the HEIA process was a facilitator, and when the HEIA was an agency priority they received the support needed for their HEIAs. Available and dedicated resources were a key factor reported by respondents. These resources included human resources, time, partnerships, and financial resources. Respondents reported that having sufficient time to plan and implement their HEIAs was also a facilitator, as well as the availability of data and information to inform the HEIA. Many reported that the training and support they received from within the agency facilitated the implementation of their HEIA strategies, and helped them understand the HEIA process.

Working with external partners, including municipalities, community partners, professional partners, and professional bodies, committees, and work groups, also helped the implementation of mitigation strategies when the strategies were cross-sectoral in nature. Some respondents said that when the scope of the mitigation strategies was within their control and decision-making authority, it was easier to implement the strategies; for example, being responsible for managing a part of the agency’s process or product gave the control, freedom, and flexibility to implement strategies.

Barriers and challenges to implementation of HEIA mitigation strategies

The most commonly identified barriers and challenges to starting and/or completing HEIA mitigation strategies were lack of time, lack of staff resources, limited/no collaboration with community partners, program area workload, and lack of budget resources. Some respondents said that they hoped to continue the implementation of their HEIA mitigation strategies once required resources became available. Others said that they realized they had identified more mitigation strategies than was feasible or realistic to implement.

Other reasons given for not starting or completing mitigation strategies were:

• Modifications to the initial identified strategies following a literature review, or expansion to include a broader population than the initial identified priority population

• Inability to get external agency commitment

• Competing program priorities and changes to the program focus

• Grant funding requirements leading to shift of focus from initially identified strategies

• Lack of resources to provide the identified services

• Internal processes within the agency, such as approval procedures and policies

• Still waiting for research reviews (to provide further direction to the mitigation strategies)

Impacts on programs, services, and priority populations

Of the 37 HEIAs that had identified mitigation strategies, 30 respondents had observed and/or measured impacts that they attributed to having implemented their HEIA mitigation strategies. All of these impacts were in regards to programs, services, or organizational policies (program-level impacts); none was clearly identified as impacts on priority populations themselves (population-level impacts).

As a result of implementing the mitigation strategies, program-level changes in support of health equity were observed, from the service providers’ perspectives. These changes included:

• Enhanced collaboration with external partners and increased advocacy for public policy changes

• Greater access to information for priority populations, through increased variety of communication strategies and improvements to resources and information, such as appropriate literacy levels, language of communications, and communication channels

• More inclusive and accessible services for priority populations, including increased accessibility for those with disabilities

• Removal of financial barriers to access programs

• Increased access to specific skills and knowledge training for staff and community partners, particularly more equity-focused staff training

• More internal collaboration among agency programs

• Changes in program planning processes to better incorporate equity considerations

• Use of more evidence-informed decision-making

Population-level impacts were not explicitly reported by respondents, as they had not been measured or documented through any quantitative or qualitative data collection. Impacts on priority populations were assumed in some cases; however, these populations had not been asked about how the mitigation strategies affected them or their health.

Implications

Discussion: key findings and implications for policy and practice

There were a number of findings from this evaluation that can help inform future HEIA use. Tyler et al. (2014) categorized their evaluation findings of facilitators and barriers of using health equity tools into three levels: system level, organizational level, and operational level. If we consider our evaluation findings in this way, we see clear similarities to Tyler et al. in our facilitators and barriers with regard to implementing mitigation strategies:

• System-level facilitators and barriers are the socio-political environment under which the public health system operates (Tyler et al., 2014). System-level facilitators common to Tyler et al. and our evaluation include a clear strategic mandate, or the mandated use of the health equity tools which makes allocated resources available for use to ensure strategic mandates are achieved. System-level barriers include lack of resources and competing priorities for attention and funding.

• Organizational-level facilitators common to Tyler et al. and our evaluation include the support from leaders and decision makers, management buy-in, and organizational readiness to implement the HEIA process. A lack of management support at agency level creates a barrier to using the tool.

• At an operational level, a detailed data collection strategy for information needed to complete an evaluation, skilled staff, and good working collaborations internally and with external partners facilitate the use of the tool and implementation of mitigation strategies. In both evaluations, a lack of data created a barrier to the HEIA process.

When considering impacts, our survey results showed that changes were made to agency programs, policies, or services as a result of identifying and implementing HEIA mitigation strategies in 30 of 37 advanced-stage HEIAs. This suggests that the use of the HEIA tool acted as an important catalyst for change to address program-level factors that may contribute to health inequities. The impacts on policies, programs, and services were demonstrated through multiple adjustments that had been made to these policies and services, and were also commonly measured by process and administrative outputs such as updating resources and/or a website, using simple frequency counts (e.g., the number of people who attended a service or program event), or anecdotal information. Some respondents reported on anticipated future impacts. However, none of the respondents reported developing indicators specifically to measure impacts as a result of their HEIAs, or to conducting a formal evaluation of their HEIA.

Overall, it appeared that it was easier for programs to identify and report on these changes to programs, policies, and services (program-level impacts) than it was to identify and report on any direct impacts to the priority populations themselves (population-level impacts), and that attributing potential population-level impacts solely or partly to the HEIA mitigation strategies was challenging. While it is plausible that any number of the program-level impacts may have led to population-level impacts on priority populations, few respondents had explicitly sought to document such impacts and, therefore, they remain unknown. For example, we do not know whether the priority populations found the service to be more accessible; whether their experience suggests inclusivity (non-judgemental experience); or whether it is a service they would value having in the future. Furthermore, we do not know whether the changes that occurred as a result of the mitigations had any direct impact on their health or addressed the specific factors that created health inequities, while also recognizing that there are many other factors contributing to these outcomes.

Lessons learned

Our evaluation of the HEIA tool in a local public health setting has demonstrated that HEIAs are a catalyst for equity-oriented change in public health programming, planning, service delivery, and organizational policy. The HEIAs were clearly found to lead to impact on programming change at the public health agency level, and their use for this purpose should continue. However, given that changes were not measured at the priority population level, we do not know whether such population-level impacts were achieved.

In order to address this knowledge gap, we may be able to make novel use of data we are already collecting, and there are additional measures that we could consider collecting—whether for the purposes of a stand-alone HEIA evaluation or as new routine data from clients who access our programs and services. As an example of the latter, starting in 2017, postal codes were collected at SMDHU as part of prenatal class registration, and this was combined with census and birth registry data to understand whether prenatal education provided by the health agency was reaching women of reproductive age and new mothers living in low-income neighbourhoods. In an example from outside SMDHU, since 2013, a range of demographic data have been routinely collected from clients of hospitals and community health centres within the Toronto Central Local Health Integration Network (LHIN), in various languages, using eight Standardized Demographic Questions (Appendix 2). These questions have been shown to have operational feasibility with almost all clients being amenable to answering them (Sinai Health Systems, 2017). There are also variations of the demographic questionnaire for children and youth. This type of data would allow us to identify and track which populations are using our services and whether or not we have achieved the intended impacts of health equity-oriented strategies.

Many of our survey respondents, on reflection, indicated that they had identified too many mitigation strategies and this created a challenge for their program when it came time to implementing the strategies. Adding a step of prioritizing strategies for feasibility based on available resources would help to keep the mitigation strategies achievable.

The results from the evaluation led to recommendations regarding the use of the HEIA tool at SMDHU in the future (SMDHU 2018), which included:

1) Embed HEIAs as an integral long-term approach for integrating health equity into public health agency operations and planning processes, especially for new programs and services and/or when significant changes/updates are being implemented to existing ones.

2) Provide supports to build staff capacity around prioritizing and evaluating HEIA mitigation strategies, and in particular how to measure impacts of mitigation strategies on priority populations.

Our experience adds to the limited body of primary evaluations on the use of health equity tools to date and, in particular, provides a novel focus on the impacts of one such tool. Additional evaluations and knowledge sharing around experiences using the variety of available health equity tools would be of value.

Appendix 1. Survey questions

1. What year was your HEIA conducted?

2. Please identify your program and the topic of the original HEIA conducted in 2013/14.

3. Using the HEIA tool as a guide, please indicate what step of the HEIA you are currently at.

4. What were the challenges in not being able to start or complete a full HEIA? Check all that apply.

5. Please tell us more about your experience with the challenge(s) identified above.

6. Do you intend on going any further with your HEIA at this time or in the future?

7. How many mitigation strategies were identified in your HEIA?

8. Of the identified mitigation strategy (ies), how many were fully implemented (i.e., the mitigation strategy was completed)?

9. What factors contributed to the full implementation of the mitigation strategy (ies)? Check all that apply.

10. Please tell us more about your experience with the factors identified above.

11. Of the identified mitigation strategies, how many were partially implemented (i.e., the mitigation strategy was started but is not yet fully completed)?

12. What are the reasons these mitigation strategies were not fully implemented? Check all that apply.

13. Please tell us more about your experience with the factors identified above.

14. Of the identified mitigation strategies, how many have not yet started?

15. What are the reasons for these mitigation strategies not being started? Check all that apply.

16. Please tell us more about your experience with the factors identified above.

17. Do you intend to go any further with these mitigation strategies in the future?

18. We are interested in learning about any changes or impacts that have occurred as a result of the implementation of the mitigation strategies to: your program, service, policy or initiative; and the priority population(s). If there was no change or impact noticed or observed, indicate why.

Appendix 2. Toronto Central LHIN’s 8 standardized demographic questions

Source – pages 21 to 23 of report at http://torontohealthequity.ca/wp-content/uploads/2013/02/Measuring-Health-Equity-Demographic-Data-Collection-Use-in-TC-LHIN-Hospitals-and-CHCs-2017.pdf

1. What language would you feel most comfortable speaking in with your health care provider? Check ONE only.

2. Were you born in Canada?

3. Which of the following best describes your racial or ethnic group? Check ONE only.

4. Do you have any of the following? Check ALL that apply.

   (options include chronic illness, developmental disability, drug/alcohol dependence, learning disability, mental illness, physical disability, sensory disability, none, other, prefer not to answer, do not know).

5. What is your gender? Check ONE only.

6. What is your sexual orientation? Check ONE only.

7. What was your total family income before taxes last year? Check ONE only.

8. How many people does this income support?

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.