GRADE Equity Guidelines 4: Guidance on how to assess and address health equity within the evidence to decision process

Abstract

Objective

The aim of this paper is to provide detailed guidance on how to incorporate health equity within the GRADE (Grading Recommendations Assessment and Development Evidence) evidence to decision process.

Study design and setting

We developed this guidance based on the GRADE evidence to decision (EtD) framework, iteratively reviewing and modifying draft documents, in person discussion of project group members and input from other GRADE members.

Results

Considering the impact on health equity may be required, both in general guidelines, and guidelines that focus on disadvantaged populations. We suggest two approaches to incorporate equity considerations: 1) assessing the potential impact of interventions on equity and; 2) incorporating equity considerations when judging or weighing each of the evidence to decision criteria. We provide guidance and include illustrative examples.

Conclusion

Guideline panels should consider the impact of recommendations on health equity with attention to remote and underserviced settings and disadvantaged populations. Guideline panels may wish to incorporate equity judgments across the evidence to decision framework.

This is the fourth and final paper in a series about considering equity in the GRADE guideline development process. This series is coming from the GRADE equity subgroup.

Background

Health care recommendations may alleviate or worsen health inequities [1–3]. For example, recommendations for testing and treatment for hepatitis C may worsen the health of disadvantaged populations with hepatitis C, relative to less disadvantaged populations, if recommendations do not consider how to ensure medical care and treatment for all hepatitis C positive populations [4].

In this series, health inequity is defined as avoidable differences in health that are considered unfair or unjust [5]. We refer to disadvantaged populations as those that are at risk for health inequities and we use the acronym PROGRESS-Plus¹ to refer to the factors associated with social disadvantage [6].

Recognizing the importance of health equity in guidelines, the GRADE working group established an Equity working group in 2013. This paper is the last in a four-part series of papers considering health equity in guideline development [7–9]. This paper focuses on how to consider and address health equity in the GRADE evidence to decision (EtD) process.

The GRADE EtD frameworks support explicit judgments about the main factors that influence the strength of recommendations, including impact on health equity [2,3]. These criteria include the priority of the problem, how substantial desirable and undesirable health effects are, certainty of the evidence of effects, how people value the main outcomes, whether desirable effects outweigh undesirable health effects, resource requirements and whether the net health benefits are worth the incremental cost, impact on equity, acceptability to stakeholders, and feasibility of the intervention [2,3]. The EtD framework can be used for different types of decisions, including clinical recommendations, tests, coverage decisions, and health system and public health recommendations [2,3]. There is a debate within the guideline community about the relevance of health equity for individual decisions, which we describe in paper 1 of this series [7]. GRADE guidance states that from a population perspective, equity, acceptability, and feasibility are important considerations, however the importance of these criteria is often limited from an individual patient perspective [2], but this needs further discussion.

EtD frameworks allow guideline panel to assess criteria and, if appropriate, make subgroup recommendations for specific settings and disadvantaged populations. In addition to the specific EtD criteria of “equity impact”, it may be desirable to assess health equity across other criteria in the EtD. For example, the Canadian Migrant Health Guideline Panel in considering [10] evidence for HIV testing in populations coming from HIV endemic countries, the panel sought evidence on acceptability (one of the EtD criteria) for refugee populations, and this led to the recommendation for inclusion of voluntary and anonymous testing options to improve access to testing. Therefore, by assessing health equity considerations within other EtD criteria, guideline panels may improve the comprehensiveness of judgments.

This paper argues that, in addition to assessing equity impact as a specific criterion, it may be useful to systematically consider health equity issues within each of the GRADE EtD criteria. The examples within this paper launch a discussion as to the best way to integrate equity considerations within the EtD process. We will show how assessing equity across the criteria can influence the strength of the recommendation.

Existing guidance

Over the past 10 years, authors have proposed questions to incorporate equity during the development of guidelines [1,11]. The WHO guideline manual includes guidance for panels addressing equity, gender and human rights [5,12]. The previous papers in the series address how to consider equity in formulating the research question [8] and rating the certainty of synthesized evidence [9]. Finally, the EtD framework publications provide the guidance that this paper builds upon [2,3,13].

Guidance on rating health equity in evidence to decision criteria

The GRADE EtD process includes three steps: 1) formulating the question; 2) making an assessment for each of the relevant criteria that determine a decision; and 3) drawing conclusions.

1. Formulating the Question

The GRADE EtD process begins with the formulation of review questions, with details about the PICO (population, interventions, comparisons and outcome) and other details relevant to the perspective and background. In the second paper of this series, Akl et al. outlines how health equity can be considered within panel formulation and development of equity-sensitive questions [8]. Analysis focusing on resource limited settings, specific populations (e.g. disadvantaged) can lead to different recommendations [2]. For example, an equity sensitive question focused on implementation issues in populations at high risk for hepatitis C (Box 1 shows that the high prevalence subgroups were identified at the question formulation stage) [14]. The question may highlight health equity in different ways, such as by focusing on a disadvantaged population or by assessing effects across subgroups that may be disadvantaged.

Box 1. Evidence to Decision (EtD) framework, question section.

Question: Should the general population in Canada be screened for HCV?

Population: Canadian general population

Intervention: screen, link to care, and treat as indicated

Comparison: do not screen

Main outcomes: morbidity and mortality

Setting: Canada, a high income country with overall low HCV prevalence

Perspective: population perspective (health system)

Subgroups: High HCV prevalence (e.g. certain migrant populations, and IV drug users)

Background New treatments emerging but not trials on screening and on patient important outcomes, and considerable concern for resource requirements

Adapted from Canadian Task Force on Preventive Health Care Screening for HCV [14]

2. Making an assessment of each relevant criterion

We provide guidance and relevant examples and assess and address health equity for each criterion of the EtD framework.

Is the problem a priority? 

Are the consequences of the problem serious, that is, severe or important in terms of the potential benefits or savings [2]. The more severe the consequences of the problem (for example, the more debilitating, life threatening or expensive), the more likely a strong recommendation may be made. Many diseases disproportionately affect low income populations (e.g. cardiovascular disease, infectious disease, motor vehicle accidents) [15]. In addition, disadvantaged populations may face limited access to appropriate treatment and care (e.g. cataracts in low income countries or tuberculosis in indigenous populations). In these cases, guideline panels might consider these conditions as high priority because they place value on redressing unfair burden carried by disadvantaged populations. Similarly, undernutrition and its correlation to poor educational consequences in low income neighborhoods, may mean school feeding programs may reduce health inequities and will be seen as a priority [16].

How substantial are the undesirable and desirable anticipated health effects? 

With larger desirable health effects, it is more likely that the intervention should be recommended, whereas larger undesirable effects (including burden and inconvenience) may lead to not recommending an intervention. Assessing both desirable and undesirable effects for specific disadvantaged populations may lead to different recommendations. When assessing the size and importance of effects, it is important to consider both relative and absolute effects. Even though relative effects are similar across populations, absolute effects may differ if baseline risks are substantially different for disadvantaged populations [17].

What is the overall certainty of the evidence of effects? 

The less certain the evidence is for the main desirable and undesirable effects, the less likely it is that a strong recommendation should be made for a population [3]. If the intervention is recommended and implemented, it is more likely that the intervention should be evaluated. GRADE recommends rating imprecision, indirectness and other certainty of evidence domains as part of an iterative process that is akin to, but not equal to, statistical accuracy [18].

Indirectness is an important criterion when considering disadvantaged populations, as described in a previous paper of this series [9]. For example, the available evidence may be limited for disadvantaged populations as these groups are often under-represented in clinical and population research. If there is strong rationale to expect the relative treatment effect to substantially vary between the general population and one or more disadvantaged groups, the panel could rate down the certainty of effect for indirectness. However, as described in the second paper of this series [8] this scenario is extremely rare. Panels should be cautious in rating down certainty of effects for indirectness unless there is a strong rationale, such as evidence of subgroup effects or effect modifiers, for why the evidence would not apply to the specific disadvantaged group. Similarly, it is possible (but generally less likely) that a greater body of evidence may exist for a disadvantaged population (e.g. as a consequence of more research having been conducted in response to a high burden in this population). Under such a situation, it is possible that greater certainty in the evidence may result in a different recommendation that is targeted at disadvantaged groups. However, potential for stigma or other adverse consequences that may arise from targeted recommendations should be considered [19].

Is there important uncertainty about or variability in how much people value the main outcomes? 

How people value important outcomes and the variability in how they value them (the extent that individuals with different views would make different decisions), can impact the strength of recommendations. Greater uncertainty about these values will decrease the likelihood of a strong recommendation [2]. Valuation of outcomes also depends on when the outcomes occur and on the values about future outcomes compared to short-term outcomes. Certain disadvantaged populations may value the main outcomes differently than the general population. Explicitly considering the views of disadvantaged groups [20] concerning the importance of outcomes helps panels to decide whether adapting recommendations for subgroups is warranted. For example, refugee populations from tuberculosis (TB) endemic countries show less concern for latent (asymptomatic) TB than the general population [21]. The limited concern for latent TB of certain patients should be factored into recommendations for the implementation of Mantoux testing. Evidence measuring views of outcomes in a range of populations, however, may be limited.

As another example, in the development of the Australian renal guidelines, discrete choice analyses identified that taking indigenous people living in rural communities valued the additional time at home over improved survival, and this led to a conditional recommendation to offer a delay in instituting renal dialysis for rural and remote Australian indigenous populations [22].

Do the desirable effects outweigh the undesirable health effects? 

Deciding whether the desirable effects outweigh the undesirable health effects depends on the assessments for the three previous criteria relating to the size and importance of effects, the certainty of the evidence for those effects and certainty and variability in valuation of outcomes for different populations. When relevant population subgroups need to be considered in guidelines, this may require making this assessment separately for each population subgroup of interest.

For instance, the current American College of Cardiology blood cholesterol guidelines use a cardiovascular disease risk calculation that includes race and gender as considerations for estimating the desirable effects from treatment, because the balance of desirable and undesirable health effects varies due to differences in baseline event rates across ethnicity and gender [23].

Resource use 

The EtD includes three criteria related to resource use: 1) size of resource requirements, 2) certainty of evidence about resource requirements and 3) whether the cost effectiveness of the intervention favours the intervention or the comparator? Judgments may consider other criteria including the certainty of evidence of desirable and undesirable effects, variability in valuation of outcomes and uncertainty about the costs [2].

When considering health equity in assessing resource use (costs), an intervention with lower resource requirements and lower effectiveness, compared to the next best alternative may sometimes be preferred in some settings to promote health equity. NICE, in the UK, formulated recommendations about options with lower effectiveness and lower resource requirements in situations where either there is uncertainty in how much less effective the intervention is, or there is a need to provide access to the intervention for disadvantaged groups [24]. In South Africa, for instance, in order to treat more people with end-stage kidney disease, dialysis guidelines recommend the re-use of dialyzers, albeit with close monitoring to ensure effective removal of waste products [25]. These guidelines also recommend that patients should be informed before starting dialysis, that re-use is being practiced. For considerations of costs and budget impact in this example, the resource use needs to consider resources required for monitoring [26].

Occasionally, if other criteria for strong recommendations are met, but the absolute treatment cost is high, this may be a reason to make a conditional recommendation.

When considering cost-effectiveness studies, there is a need to assess the certainty in the model parameters by conducting sensitivity analyses, applicability to the settings of interest and quality of the study. Also, while studies such as cost-effectiveness analyses provide useful information to inform decision-makers, these methods do not typically address the issues of budget impact or patient-level affordability which may be important, depending on the perspective of the recommendation.

Guideline development panels might identify economic evaluations which have explicitly considered health equity in one of four ways: 1) background reviews; 2) health equity impact assessment; 3) analyzing opportunity costs for equity considerations; and 4) equity weighting of health outcomes [27]. Equity weighting methods include: adding an equity weight or social welfare function (SWF) whereby society is prepared to sacrifice overall health benefits to promote a more equitable distribution of these benefits; statistical programming that quantifies the opportunity cost of equity; and multi-criteria decision analysis based on trade-offs between a range of criteria of which, cost may be one, i.e. through the process of discrete choice analysis [28]. Economic evaluations which have explicitly considered health equity may be helpful when rating equity-sensitive questions.

What would be the impact on health equity? 

The GRADE EtD framework original guidance [2] upon which this paper builds, recommends considering four questions to assess potential impact on health equity (Figure 1).

Figure 1. Detailed judgments about the impact on health equity.

The criterion of potential impact on health equity focuses on both relative and absolute effectiveness for the intervention, the importance of the problem, and identifying considerations to ensure health inequities are reduced or not increased [2]. However, data on specific disadvantaged groups and settings may not be available. In such cases, considering health equity across the previous criteria may help in answering the above questions. For example, judgments on new testing and treatment guidelines for hepatitis C may also benefit from health equity considerations within balance of desirable and undesirable effects, cost requirements, acceptability, and feasibility criteria [29].

By explicitly examining the potential impact on health equity, a panel may discover differential effects on disadvantaged populations (e.g. health equity in relation to specific characteristics: economic status, employment or occupation, education, place of residence, gender or ethnicity) [2]. These four questions may also be considered at other points during the evidence to decision process. For example, the question about differential effects may be considered in assessing the size of desirable and undesirable effects for disadvantaged populations, as described in the second paper of this series [8]. For example, the WHO Reproductive Health Group judged that guidelines recommending only a loop electrosurgical excision procedure (LEEP) for invasive cervical cancer might increase health inequities in settings without LEEP technology. Therefore, to prevent a negative effects on health equity, WHO suggested cryotherapy and LEEP where available [30]. Careful monitoring of effects should be a priority when more accessible options are considered.

An organization or panel might decide in advance to make explicit detailed judgments for one or more criteria (Figure 1). Alternatively, they may only make explicit detailed judgments when these judgments help resolve disagreements.

Is the intervention acceptable to key stakeholders, including patients? 

The less acceptable an intervention is to key stakeholders (including disadvantaged people), the less likely it is that it should be recommended, or if it is recommended, the more likely it is that an implementation strategy will be needed to address concerns about acceptability [2]. Acceptability of an intervention may differ for different populations due to four main reasons: 1) different distribution of benefits, harms and costs, 2) timing of outcomes (e.g. now or in the future), 3) different values about the relative importance of desirable and undesirable health effects, and 4) ethical considerations, such as patient autonomy and justice.

It is extremely important to collect input from key stakeholders from disadvantaged populations in considering acceptability since assumptions by panel members may be biased by their personal experience. Some examples of methods to collect information about acceptability include surveys of stakeholders (e.g. WHO guideline on men who have sex with men [31] or targeted literature reviews to collect information about acceptability, such as done in the Canadian Migrant Health Guidelines [10]).

Differences in acceptability between groups, if strong enough, could impact the strength of a recommendation for specific groups and should be reported and addressed during implementation. Key stakeholders of disadvantaged populations may include communities, health workers and interest groups [2]. For example, in the adaptation of the Canadian Reducing Pain with Vaccination Guidelines, the WHO SAGE panel conducted a survey of national immunization program leaders in low and middle income countries. This stakeholder survey found that one of the evidence based recommendations [32], using two practitioners to give simultaneous vaccines, could create health inequities in overcrowded and under-resourced clinics. The recommendation was therefore removed to reduce the risk of health inequities and improve uptake of the more acceptable recommendations.

Certain stakeholders may disapprove of the intervention culturally or traditionally, religiously or morally. For example, a panel developing public health recommendations for preventing sexually transmitted infections, including HIV, conducted a survey of men who have sex with men, with and without HIV, and transgender people [31]. The results of this work showed that marginalized and transgender people preferred community-based HIV testing over health facility testing. This variation in acceptability demanded additional implementation resources for community-based testing to prevent inequities.

Is the intervention feasible to implement? 

Feasibility is an assessment of how easy it is to carry out the intervention, put it into practice or policy, or stop an existing intervention. The less feasible an intervention, the less likely it should be recommended when considering the perspective of the health practitioner. Interventions with low feasibility (or high barriers to implementation) may lead to a weak or conditional recommendation [2].

Indeed, decision makers often cite feasibility and cost as critical barriers to implementation and may also consider sustainability of the intervention [2]. Panels may identify unique health equity concerns relating to access, barriers to implementation in certain settings, and program feasibility. Detailed judgments may include consideration of barriers to the sustainability of the option(s) [33]. These barriers include guideline factors, individual health professional factors, patient factors, professional interactions, incentives and resources, capacity for organizational change, and social, legal and political factors [34].

Low feasibility may leave panels hesitant to exclusively recommend an intervention that is effective and even cost-effective, but may be too resource intensive for a particular setting (e.g. low income country). For example, the WHO recommends that visual inspection of the cervix with acetic acid in resource limited settings be considered for cervical cancer screening as an alternative for the more expensive and technically challenging Papanicolaou tests [30].

3. Drawing conclusions

Drawing conclusions begins with the panel reviewing the judgments they have made for all of the criteria in their assessment and considering the implications of those judgments for the recommendation or decision. The importance of each EtD criterion for a recommendation can vary depending on the population and limitations in resources. To formulate a recommendation a panel must consider the trade-offs and health equity concerns of each judgment. Based on their assessment, the panel draws conclusions about the strength of recommendation or type of decision, such as a strong or weak (also called conditional, discretionary, or qualified) recommendation for or against an intervention or option. The conclusions also include relevant considerations about subgroups, implementation, monitoring and evaluation, and research priorities (see Table 2). Panels can reach these conclusions in different ways, including the use of informal or formal consensus processes or voting.

Table 2. Evidence to Decision frameworks: Conclusions.

Type of recommendation	Strong recommendation against the option	Conditional recommendation against the option	Conditional recommendation for either the option or the comparison	Conditional recommendation for the option	Strong recommendation for the option
	X	○	○	○	○
Recommendation	In the general population, the guideline panel recommends not screening for HCV
Justification	Very low quality evidence that desirable effects favor the intervention?
Subgroup considerations	Consider testing high prevalence Hepatitis C populations, given the emerging promising treatment modalities
Implementation considerations	Implementation considerations added to suggest considering testing high risk populations
Monitoring and evaluation	Monitoring needed to assess screening rates across populations
Research priorities	Need for research on patient important outcomes bridging testing and linkage to care and treatment

Drawn from Canadian Task Force on Preventive Health Care Screening for Hepatitis C. [14].

We propose that assessing health equity across EtD criteria will help to decide amongst three possible types of recommendations:

• 1)A general recommendation that can be applied across different populations and settings. Assessment of health equity across the criteria may increase the confidence of the panel that a general recommendation is warranted and that the intervention is applicable for disadvantaged populations and settings.
• 2)A general recommendation that can be accompanied with subgroup and implementation considerations, to promote health equity or mitigate worsening health inequities.
• 3)A separate recommendation for a specific disadvantaged population when evidence of meaningfully different effects for a specific setting or subgroup is identified.

Methodologic Challenges

One challenge in assessing potential impact on health equity across EtD criteria is that within a specific disadvantaged population, such as people living on low-income, there is heterogeneity in the experience of health inequity due to variation across other social determinants such as gender, employment and age. Thus, the assessment for one population may vary depending on the intersection of these different characteristics. Guideline panels may need to choose the most important disadvantaged populations for the problem or condition of interest, and recognize that there is no “one-size-fits-all” approach to mitigating health inequities.

The second challenge is that the EtD process requires a step-wise approach. We recognize that it may not be possible or desirable to assess equity for every criterion. For example, if an intervention is simply not acceptable for a disadvantaged population, then it is redundant to also assess feasibility and resource use considerations for this population. The acuity to assess and address the needs of disadvantaged groups may require additional research training and systematic processes. Assessing health equity across each criterion adds another level of complexity and may only be warranted for selected criteria and scenarios.

Research Agenda

Researchers could monitor the use of EtD frameworks to see if health equity impacts or considerations are being reported. When, if ever, do health equity concerns influence the direction or strength of recommendations? Researchers will need to ask: which criteria most commonly report gaps in health equity consideration reporting? Does assessing health equity across EtD criteria improve the impact of the guideline. What are the health equity methodological and data barriers facing guideline panels and review groups? Finally, how do panel members and stakeholders respond to health equity and implementation considerations or options? Comparison of different approaches to considering health equity in guideline development could advance the equity related methods.

In conclusion, health decisions should incorporate health equity considerations. The EtD frameworks can facilitate explicit and systematic judgement of potential effects on health equity. When relevant for disadvantaged populations, panel members should consider providing the corresponding specific subgroup information (and make independent judgments as well as the overall population judgments) within the specific criteria. Preventing harms, including unintended harms, may be especially important for disadvantaged populations who often have a higher risk of disease and fewer resources to address harms. Panels should judge and report the estimated impact of interventions or options on health equity and modify recommendations, or develop specific implementation strategies to prevent harms or reduce health inequities.

Key Points.

• Panels should consider the potential for an intervention to have unwanted effects on equity when moving from evidence to decision making.

• Incorporating health equity judgments into other evidence to decision criteria should be considered, both in general guidelines and in disadvantaged population specific guidelines.

• Panels may wish to point out these potential adverse effects such that decision makers could take mitigating actions in their implementation and monitoring of the recommendation.