Skip to main content
PMC home page
Elsevier Sponsored Documents logoLink to Elsevier Sponsored Documents
. 2017 Oct;90:76–83. doi: 10.1016/j.jclinepi.2017.01.015

GRADE equity guidelines 3: considering health equity in GRADE guideline development: rating the certainty of synthesized evidence

Vivian A Welch a,∗, Elie A Akl b,c, Kevin Pottie a,d, Mohammed T Ansari e, Matthias Briel f,g, Robin Christensen h, Antonio Dans i, Leonila Dans j, Javier Eslava-Schmalbach k, Gordon Guyatt l, Monica Hultcrantz m,n, Janet Jull o, Srinivasa Vittal Katikireddi p, Eddy Lang q, Elizabeth Matovinovic r, Joerg J Meerpohl s,t, Rachael L Morton u, Annhild Mosdol v, M Hassan Murad w, Jennifer Petkovic x, Holger Schünemann y,z, Ravi Sharaf aa, Bev Shea ab, Jasvinder A Singh ac,ad,ae, Ivan Solà af, Roger Stanev ag, Airton Stein ah, Lehana Thabaneii ai, Thomy Tonia aj, Mario Tristan ak, Sigurd Vitols al, Joseph Watine am, Peter Tugwell an
PMCID: PMC5680526  PMID: 28389397

Abstract

Objectives

The aim of this paper is to describe a conceptual framework for how to consider health equity in the Grading Recommendations Assessment and Development Evidence (GRADE) guideline development process.

Study Design and Setting

Consensus-based guidance developed by the GRADE working group members and other methodologists.

Results

We developed consensus-based guidance to help address health equity when rating the certainty of synthesized evidence (i.e., quality of evidence). When health inequity is determined to be a concern by stakeholders, we propose five methods for explicitly assessing health equity: (1) include health equity as an outcome; (2) consider patient-important outcomes relevant to health equity; (3) assess differences in the relative effect size of the treatment; (4) assess differences in baseline risk and the differing impacts on absolute effects; and (5) assess indirectness of evidence to disadvantaged populations and/or settings.

Conclusion

The most important priority for research on health inequity and guidelines is to identify and document examples where health equity has been considered explicitly in guidelines. Although there is a weak scientific evidence base for assessing health equity, this should not discourage the explicit consideration of how guidelines and recommendations affect the most vulnerable members of society.

Keywords: Health, equity, GRADE, Guidelines, Systematic review, Meta-analysis, Subgroup analysis, Applicability, Indirectness

What is new?

Key findings

  • •

    This paper provides consensus-based guidance for including health equity considerations in guideline development.

What this study adds to what was known?

  • •

    This paper adds an equity framework to the Grading Recommendations Assessment and Development Evidence (GRADE) guidance for rating the certainty of evidence in systematic reviews.

What is the implication and what should change now?

  • •

    Considering health equity in rating the certainty in synthesized evidence requires a priori elaboration of the disadvantaged populations and settings of interest, and methods to assess both relative and absolute effects for these populations.

  • •

    GRADE judgements about directness require transparent reporting of how judgements were made.

1. Background

Health inequities are differences in health that are not only unnecessary and avoidable but are also considered unfair and unjust [1]. As described in the introductory paper in this series, we use the acronym PROGRESS Plus (Place of residence, Race/ethnicity/culture/language, Occupation, Gender/sex, Religion, Education, Socioeconomic status, or Social capital + personal, relational and time-dependent characteristics) to identify individual and context-specific characteristics across which health inequities may occur [2].

Guideline panels need to decide early on whether they plan to develop equity-sensitive recommendations (as described in the introductory paper in this series). Using explicit prompts may be helpful in this process [3]. In principle, considering health equity is important for two main types of guidelines: (1) universal interventions where health inequity is a concern [4], [5], [6], [7]; and (2) targeted or dedicated interventions aimed at one or more disadvantaged populations that have experienced health inequities. An example of the latter is the Canadian immigrant health guidelines [8], developed to raise awareness of migrant health needs and improve access to effective preventive screening.

This paper provides guidance to address health equity when rating the certainty in synthesized evidence using the Grading Recommendations Assessment and Development Evidence (GRADE) approach. This paper is the third paper in a four-part series on health equity and GRADE, with the introduction [Welch et al.], overall process [Akl et al.], and evidence to decision methods [Pottie et al.].

2. Existing guidance

As discussed in the earlier two papers in this series, several authors have assessed how guidelines consider health inequity concerns [Welch et al. this series, Akl et al. this series]. None of these papers focus on rating the certainty of synthesized evidence (i.e., quality of evidence) using the GRADE approach.

3. GRADE certainty in synthesized evidence and health equity

The GRADE approach of presenting the evidence by outcome and the associated certainty (i.e., quality of evidence) involves the production of summary tables. These tables include evidence profiles (with details on the rating of certainty for each outcome) and summary of finding (SoF) tables that are intended for the public, patients, purchasers, payers, practitioners, product makers (e.g., manufacturers, industry), and policy makers [9].

Five methods can be used to assess health equity with the GRADE approach:

  • a)

    Include health equity as an outcome

  • b)

    Consider patient-important outcomes relevant to health equity

  • c)

    Assess differences in the magnitude of effect in relative terms between disadvantaged and more advantaged individuals or populations

  • d)

    Assess differences in baseline risk and hence the differing impacts on absolute effects for disadvantaged individuals or populations

  • e)

    Assess indirectness of evidence to disadvantaged populations and/or settings.

3.1. Consider including health equity as an outcome for the SoF tables

If health inequity is considered an important concern by relevant stakeholders, then health equity could be included as an outcome in the Population, Intervention, Comparison, Outcome questions, analytic framework, and SoF table. In doing so, guideline developers must recognize that health equity is primarily assessed with a subgroup analysis. The developers should also note that this may risk excluding other patient-important outcomes, if SoF tables are limited to only seven outcomes as recommended by GRADE. For example, the NICE guideline on maternal and child nutrition identified impact on health inequalities as one of its key priorities and framed its key question as: “What nutritional interventions are effective in improving the health of preconceptual, pregnant, and postpartum mothers and children (up to 5 years) and reducing nutrition-related health inequalities” [10]. By including health equity as an outcome in the SoF table, it is easier for guideline panels to find the information (or lack thereof) about health equity and consider it in their deliberations.

The direction and size of the effect on health equity is influenced by decisions such as the reference comparator group, use of relative or absolute measures, and whether the outcome is a desirable or undesirable event [11], [12]. For example, the choice of absolute or relative effects can change the conclusions about health inequalities. This is illustrated by gender disparity in stomach cancer mortality rates in the United States between 1930 and 2000 has decreased when looking at absolute differences (the rates for both men and women have declined). However, the relative risk for men compared to women has increased (increased disparity, male/female ratio) [13].

A lack of evidence about a critical health equity outcome should not be a reason to omit this from the SOF table. Indeed, this should be explicitly identified as an empty row, highlighting the need for further research to answer questions about health equity.

3.1.1. Example 1

The Community Guide Water fluoridation guideline [14] included “health disparities” as an outcome in the analytic framework and the SoF table because the Community Task Force placed a high value on reducing socioeconomic disparities in dental caries. Socioeconomic disparities were measured as the difference in absolute terms of a continuous outcome (caries). The evidence review found three studies that provided insufficient evidence about socioeconomic disparities to draw conclusions, highlighting a gap in the evidence base (Table 1).

Table 1.

Effect of Community Water Fluoridation on socioeconomic health inequities in caries [14]

Outcome Measure Evidence
Health equity as measured by socioeconomic disparities in caries % of caries reduction Inconsistent results on socioeconomic disparities (three studies)
dmft/DMFT No data on socioeconomic disparities
Open in a new tab

Abbreviation: DMFT/dmft, decayed, missing, or filled teeth.

Upper case refers to permanent teeth; lower case to primary teeth.

3.1.2. Example 2

“Equity impact” was the primary outcome of a systematic review on interventions to reduce smoking in adults [15]. Equity impact was assessed as the difference in the magnitude of a dichotomous outcome in absolute terms, defined as a difference in absolute effect on prevalence in lower socioeconomic status compared to higher socioeconomic status. This review showed that while increases in price or taxes reduced health inequities in smoking, mass media campaigns were more likely to worsen health inequities. This type of review provides evidence that could be used to include impact on health equity as an outcome of interventions.

3.2. Consider patient-important outcomes relevant to health equity

As described in the previous paper in our series [Akl et al. in this series], the evidence synthesis process should consider the relative importance of different outcomes, determined with input from stakeholders representing disadvantaged groups. The evidence base for these outcomes should then be assessed. Examples of patient importance and health equity were provided in the previous paper in this series such as the importance of inconvenience of a subcutaneous chelation pump for people with sickle cell disease [described in Akl et al. in this series].

3.3. Assess differences in the magnitude of effect in relative terms between disadvantaged and more advantaged individuals or populations

Average effects obscure differences between subpopulations—that is, subgroup effects may exist. Examining whether effects differ across socioeconomic status or other variables relating to health inequity requires investigating heterogeneity in the treatment effect—for example, using statistical approaches such as meta-regression or subgroup analysis. However, such results may not be available in the literature. There is evidence that systematic reviews underreport subgroup analyses from primary studies [16], [17]. Furthermore, many primary studies fail to assess possible subgroup effects related to disadvantaged populations.

Relative effects are usually similar across diverse populations and settings, and spurious subgroup effects are common [18]. Thus, if analysis suggests an apparent subgroup effect, it is important to assess the credibility of the apparent effect [19]. Sun and colleagues [20], [21] describe several criteria to help do this such as determining a priori which subgroup analysis to conduct, finding a low P-value associated with a statistical test for interaction, and providing results from within-study comparisons. Sun et al. also showed that subgroup analyses reported in the literature rarely meet these criteria. Evidence synthesis that involves subgroup analyses should therefore consider the full set of credibility issues, using an appropriate checklist, and avoid making conclusions based on chance findings (Table 2).

Table 2.

Checklist for assessing credibility of subgroup analyses [22]

  • Design
    • Is the subgroup variable a characteristic measured at baseline or after randomization?
    • Is the effect suggested by comparisons within rather than between studies?
    • Was the hypothesis specified a priori?
    • Was the direction of the subgroup effect specified a priori?
    • Was the subgroup effect one of a small number of hypothesized effects tested?
  • Analysis
    • Does the interaction test suggest a low likelihood that chance explains the apparent subgroup effect?
    • Is the significant subgroup effect independent?
  • Context
    • Is the size of the subgroup effect large?
    • Is the interaction consistent across studies?
    • Is the interaction consistent across closely related outcomes within the study?
    • Is there indirect evidence that supports the hypothesized interaction (biological rationale)?
Open in a new tab

If applying the criteria in Table 2 leads to a conclusion that the subgroup effect is credible, the guideline panel should provide different estimates of relative and absolute effect for the subgroups. The panel should then consider making different recommendations for patients in these subgroups or consider whether recommendations that apply to the overall population need to be adapted to enhance equity. When the credibility of subgroup effects is low, the guideline panel may suggest that further research is needed. Few subgroup analyses meet all of these criteria; however, when most criteria are met, decision making must consider the likely existence of subgroup effects.

3.3.1. Example: hypertension and ethnicity

The Eighth Joint National Committee guideline on management of hypertension recommends a calcium channel blocker or thiazide-type diuretic as initial therapy in the black hypertensive population (whereas an angiotensin-converting enzyme inhibitor, angiotensin receptor blocker, calcium channel blocker, or thiazide-type diuretic is recommended for others with hypertension) [23]. This recommendation was based on a prespecified subgroup analysis of the ALLHAT trial (n = 18,102 participants, 35% black [24]) that showed stroke was 51% (95% CI: 1.22, 1.86) greater for blacks treated with an ACE inhibitor first compared to those treated with a calcium channel blocker. The guideline panel rated this subgroup effect as moderate quality evidence. Had the panel not identified this subgroup effect, use of an ACE inhibitor as a first-line agent would have increased health disparities between black and white ethnic groups.

3.4. Assess differences in baseline risk and the differing impacts on absolute effects for disadvantaged individuals or populations

A higher baseline risk of adverse events in any population may lead to greater absolute harm from an intervention and conversely a higher baseline prevalence of the outcome of interest may lead to greater absolute benefit [25]. The SoF table should present the baseline risks and risk differences for each relevant population and provide supporting evidence. Because disadvantaged populations have a disproportionate burden of almost all health conditions, it is particularly important to consider the baseline risk for these populations. Baseline risk of adverse event rates or for the outcomes of interest for specific populations are best assessed using the most robust observational data on the actual population rather than from randomized trials. GRADE guidance regarding assessing certainty of estimates of risk from broad populations is available [26], [27].

3.4.1. Example 1: WHO guidelines on vitamin A supplementation in children 6–59 months

In 2011, WHO recommended vitamin A supplementation for children aged 6 months to 5 years in countries where vitamin A deficiency is a public health problem (strong recommendation) [28]. This was based on findings of a Cochrane review with a relative risk for all-cause mortality of 0.76 (95% CI: 0.69, 0.83). The baseline risk of all-cause mortality was estimated at 0/1,000 in low-risk populations and 90/1,000 in high-risk populations (with vitamin A deficiency), based on control group event rates in the trials. Thus, the absolute effects in terms of numbers of deaths prevented with vitamin A compared to the control group were 0/1,000 for low-risk and 22/1,000 for high-risk populations.

3.4.2. Example 2: national guide to a preventive health assessment for Aboriginal and Torres Strait Islander people

In Australia, a guideline panel sought to determine the optimal age at which to begin a series of preventive interventions in the Australian Aboriginal and Torres Strait Islander population. The panel recommended preventive interventions at an earlier age than the general population on the basis of higher prevalence of preventable diseases in Aboriginal and Torres Strait Islander populations. For example, type II diabetes is 3–4 times more common than in the general Australian population at all ages, leading to a recommendation for screening starting from age 18, instead of age 40 years for the general population [29].

3.5. Assess indirectness of evidence to disadvantaged populations

GRADE quality (or certainty) “reflects our confidence that the estimates of the effect are correct. In the context of recommendations, quality reflects our confidence that the effect estimates are adequate to support a particular recommendation. ‘Quality’ as used in GRADE means more than risk of bias and so may also be compromised by imprecision, inconsistency, indirectness of study results, and publication bias” [30]. Qualitative evidence may also be important when considering health equity. Certainty for qualitative evidence synthesis can be rated using the CerQUAL tool [31] in which the domain “relevance” is most closely aligned with directness.

Indirectness refers to the comparability between the population, the intervention, or the outcomes measured in research studies and those under consideration in a guideline or systematic review [32]. The GRADE approach evaluates the lack of directness as “indirectness.” Direct evidence may be lacking because some populations may not represent a large proportion of trial populations (e.g., migrants and refugees), and data are unlikely to be disaggregated for specific subgroups. Direct evidence may also be lacking because some populations are explicitly excluded from trials, such as pregnant women and people with multiple morbidities [33], [34], [35], [36]. Because multiple morbidities are more common in socioeconomically disadvantaged people [37], this may result in disproportionate exclusion of disadvantaged populations from trials. When direct evidence for the relevant disadvantaged population is not available, guideline developers will have to evaluate the indirectness of evidence obtained from other populations [38].

As a rule, certainty of the evidence should not be rated down for indirectness for population differences unless there are compelling reasons to anticipate differences in effect due to biology/physiology, sociocultural influences, or setting-specific resource issues that impact the effectiveness or harms of the intervention. In other words, one anticipates a different subgroup effect in either relative or absolute impact of treatment, though evidence is not available to make a formal assessment. (If it were, it should be formally assessed, as in Sections 3.3 and 3.4) Guideline panels need to consider that rating down for indirectness could in itself increase inequities if this leads to less use of an effective intervention by disadvantaged groups. In other words, lower certainty in effect estimates may lead to a weak recommendation and therefore under-use of a beneficial treatment. Rating down for indirectness should therefore be done cautiously because effective interventions are needed even more in some populations that are often excluded from trials, such as those with multiple morbidities.

3.5.1. Example 1: Canadian migrant guidelines not rated down for indirectness

The quality of the evidence was not rated down for indirectness in the Canadian migrant guideline addressing screening for latent TB; the panel considered the evidence not to be indirect for migrants. Although no migrants were included in studies of intervention effectiveness, the developers did not expect different relative effects [39].

3.5.2. Example 2: CDC guidelines for brief alcohol counseling for people with HCV infection rated down for indirectness

The Centers for Disease Control and Prevention recommended brief alcohol screening and counseling for all person with HCV infection, based on a systematic review of 22 randomized trials which found a reduction of alcohol consumption of 38.42% (95% CI: 30.91, 65.44) more than the control groups after 1 year. This evidence was rated down for indirectness by the guideline panel because none of the trials included persons with hepatitis C virus (HCV) infection [40].

4. Methodologic challenges

In developing this guidance, we identified a number of methodologic challenges. First, assessing effects on health equity is not a linear process. There may be a need to revisit the focus of the guideline during the evidence review process, including the consideration of important disadvantaged groups. NICE does this explicitly by revisiting their key questions regarding health equity throughout the process.

Second, there are often limitations in the underlying evidence base including poor reporting of sociodemographic characteristics [41], [42], under-reporting of subgroup analyses that are not statistically significant [21], [42], and use of multivariable models that may be overadjusted for effect mediators, and/or include unnecessary collinear variables [43]. Lack of evidence on whether the effects are consistent or different for disadvantaged populations makes it difficult to judge indirectness and rate certainty of evidence. When the evidence base is insufficient to assess effects on health equity, guideline panels need to make these limitations explicit and transparently report how they made judgments.

Third, epidemiologic evidence addressing baseline risk for specific disadvantaged groups may be difficult to obtain for the population or geographic region for which the recommendations are being developed. Health systems at local, regional, and national levels do not have consistent or reliable methods for reporting health status across all sociodemographic indicators of interest. Guideline panels should transparently report how they determined baseline risk estimates.

Fourth, assessing directness of evidence depends on the clinical and methodological expertise and judgment of SoF developers. The GRADE Guideline Development Tool includes an explicit checklist when producing SoFs to ask whether the evidence is direct across population, intervention, comparison, and outcome and document the decision for rating down, if performed.

5. Research agenda

The most important research priority in the field of health equity and guidelines is to systematically identify further examples of how guideline panels have assessed health equity considerations and incorporated these assessments into recommendations using transparent methods. For example, all WHO guidelines make their evidence to recommendation tables and SoFs publicly available for research such as this. These assessments could provide examples of whether, and how, the five issues (a–e) above have been considered for different situations, such as assessing the credibility of subgroup analyses and judging indirectness for disadvantaged populations.

In conclusion, the GRADE process provides a structured approach to assess effects on health equity. Health equity considerations warrant increased use of these methods in systematic reviews and guidelines. The findings of assessing health equity using these five steps in guideline development provides a basis for judging “impact on equity” which is part of the DECIDE framework, and details about this process are covered in the fourth paper of this series [Pottie et al.].

Acknowledgments

The authors gratefully acknowledge the help from Manosila Yoganathan and Shalini Suresh in preparing this manuscript for submission.

Footnotes

Funding: V.A.W. at the Bruyère Research Institute is supported by her Ontario Early Researcher Award. P.T. at the University of Ottawa is supported by his Canada Research Chair in Health Equity. The Musculoskeletal Statistics Unit at the Parker Institute (RC) is supported by grants from the Oak Foundation (OCAY-13-309). S.V.K. is funded by a NHS Research Scotland Senior Clinical Fellowship (SCAF/15/02), the Medical Research Council (MC_UU_12017/13 & MC_UU_12017/15) and the Scottish Government Chief Scientist Office (SPHSU13 & SPHSU15).

Conflict of interest: V.A.W. is a coconvener of the Campbell and Cochrane Equity Methods Group. R.L.M. holds an Australian NHMRC Early Career Fellowship #1054216. J.A.S. reports grants from Takeda and Savient, personal fees from Savient, Takeda, Regeneron, Merz, Bioiberica, Crealta and Allergan pharmaceuticals, WebMD, UBM LLC, and the American College of Rheumatology, grants from Horizon pharmaceuticals, outside the submitted work; and J.A.S. is a member of the executive of OMERACT, an organization that develops outcome measures in rheumatology and receives arms length funding from 36 companies; a member of the American College of Rheumatology's (ACR) Annual Meeting Planning Committee; Chair of the ACR Meet-the-Professor, Workshop and Study Group Subcommittee; and a member of the Veterans Affairs Rheumatology Field Advisory Committee. R.C. reports grants from the Oak Foundation. A.D. reports personal fees from Lectures on GRADE and clinical practice guideline development from different medical specialty organizations, outside the submitted work. H.S. reports that he is Co-Chair of the GRADE Working Group but does not report any direct financial conflicts of interest. P.T. reports other from Amgen, Astra Zeneca, Bristol-Myers Squibb, Celgene, EliLilly, Genentech/Roche, Genzyme/Sanofi, Horizon Pharma Inc, Merck, Novartis, Pfizer, PPD, Quintiles, Regeneron, Savient, Takeda Pharmaceutical, UCB Group, Vertex, Forest, Bioiberica, other from Astra Zeneca, personal fees from Bristol-Myers Squibb, Chelsea, UCB, other from Pfizer Canada, Hoffman La-Roche, Eli Lilly and Company, other from Elsevier, Little Brown, Wolters Kluwer Ltd, John Wiley & Sons Ltd, other from Abbott, Roche, Schering Plow/Merck, UCB, BMS, outside the submitted work; and I am an advisory committee member of the Canadian Reformulary Group Inc., a company that reviews the evidence for health insurance companies employer drug plans. All other authors have nothing to disclose.

References

  • 1.Whitehead M. The concepts and principles of equity and health. Int J Health Serv. 1992;22:429–445. doi: 10.2190/986L-LHQ6-2VTE-YRRN. [DOI] [PubMed] [Google Scholar]
  • 2.O'Neill J., Tabish H., Welch V., Petticrew M., Pottie K., Clarke M. Applying an equity lens to interventions: using PROGRESS ensures consideration of socially stratifying factors to illuminate inequities in health. J Clin Epidemiol. 2014;67:56–64. doi: 10.1016/j.jclinepi.2013.08.005. [DOI] [PubMed] [Google Scholar]
  • 3.Welch V., Akl E.A., Guyatt G., Pottie K., Eslava-Schmalbache J., Ansarif M.T. GRADE Equity: introduction: why health equity and GRADE guidelines? J Clin Epidemiol. 2016 [Google Scholar]
  • 4.National Institute for Health and Clinical Excellence . 2nd ed. National Institute for Health and Clinical Excellence; London, UK: 2012. Positively equal: a guide to addressing equality issues in developing NICE clinical guidelines. Available at http://www.nice.org.uk. Accessed June 1, 2017. [Google Scholar]
  • 5.Mitchell J., Trangle M., Degnan B., Gabert T., Haight B., Kessler D. Institute for Clinical Systems Improvement (ICSI); 2013. Depression, adult in primary care. Available at https://www.icsi.org/guidelines__more/catalog_guidelines_and_more/catalog_guidelines/catalog_behavioral_health_guidelines/depression/. Accessed June 1, 2017. [Google Scholar]
  • 6.Eslava-Schmalbach J. 2013. Colombia guidelines on complications of pregnancy and childbirth. [Google Scholar]
  • 7.Suspected cancer in primary care: guidelines for investigation, referral and reducing ethnic disparities. New Zealand Guidelines Grup; Wellington, NZ: 2009. Available at http://www.health.govt.nz/system/files/documents/publications/suspected-cancer-guideline-sep09.pdf. Accessed June 1, 2017. [Google Scholar]
  • 8.Pottie K., Greenaway C., Feightner J., Welch V., Swinkels H., Rashid M. Evidence-based clinical guidelines for immigrants and refugees. CMAJ. 2011;183:E824–E925. doi: 10.1503/cmaj.090313. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Concannon T.W., Meissner P., Grunbaum J.A., McElwee N., Guise J.M., Santa J. A new taxonomy for stakeholder engagement in patient-centered outcomes research. J Gen Intern Med. 2012;27:985–991. doi: 10.1007/s11606-012-2037-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Maternal and child nutrition . NICE; London, UK: 2014. NICE public health guidance 11. Available at https://www.nice.org.uk/guidance/ph11. Accessed June 1, 2017. [Google Scholar]
  • 11.Harper S., King N.B., Young M.E. Impact of selective evidence presentation on judgments of health inequality trends: an experimental study. PLoS One. 2013;8:e63362. doi: 10.1371/journal.pone.0063362. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Keppel K., Pamuk E., Lynch J., Carter-Pokras O., Kim I., Mays V. Methodological issues in measuring health disparities. Vital Health Stat 2. 2005:1–16. [PMC free article] [PubMed] [Google Scholar]
  • 13.Harper S., Lynch J. National Cancer Institute; Bethesda, MD: 2005. Methods for measuring cancer disparities: using data relevant to healthy people 2010 cancer-related objectives. Cancer surveillance monograph series, number 6. Available at http://seer.cancer.gov/archive/publications/disparities/measuring_disparities.pdf. Accessed June 1, 2017. [Google Scholar]
  • 14.Preventing dental caries: community water fluoridation. The community guide community preventive services task force. 2013. Atlanta. [Google Scholar]
  • 15.Brown T., Platt S., Amos A. Equity impact of population-level interventions and policies to reduce smoking in adults: a systematic review. Drug Alcohol Depend. 2014;138:7–16. doi: 10.1016/j.drugalcdep.2014.03.001. [DOI] [PubMed] [Google Scholar]
  • 16.Tugwell P., Maxwell L., Welch V., Kristjansson E., Petticrew M., Wells G. Is health equity considered in systematic reviews of the Cochrane Musculoskeletal Group? Arthritis Rheum. 2008;59:1603–1610. doi: 10.1002/art.24206. [DOI] [PubMed] [Google Scholar]
  • 17.Humphreys D.K., Ogilvie D. Synthesising evidence for equity impacts of population-based physical activity interventions: a pilot study. Int J Behav Nutr Phys Act. 2013;10:76. doi: 10.1186/1479-5868-10-76. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Schmid C.H., Lau J., McIntosh M.W., Cappelleri J.C. An empirical study of the effect of the control rate as a predictor of treatment efficacy in meta-analysis of clinical trials. Stat Med. 1998;17:1923–1942. doi: 10.1002/(sici)1097-0258(19980915)17:17<1923::aid-sim874>3.0.co;2-6. [DOI] [PubMed] [Google Scholar]
  • 19.Burke J.F., Sussman J.B., Kent D.M., Hayward R.A. Three simple rules to ensure reasonably credible subgroup analyses. BMJ. 2015;351:h5651. doi: 10.1136/bmj.h5651. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Sun X., Briel M., Busse J.W., You J.J., Akl E.A., Mejza F. Credibility of claims of subgroup effects in randomised controlled trials: systematic review. BMJ. 2012;344:e1553. doi: 10.1136/bmj.e1553. [DOI] [PubMed] [Google Scholar]
  • 21.Sun X., Briel M., Busse J.W., You J.J., Akl E.A., Mejza F. The influence of study characteristics on reporting of subgroup analyses in randomised controlled trials: systematic review. BMJ. 2011;342:d1569. doi: 10.1136/bmj.d1569. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Sun X., Briel M., Walter S.D., Guyatt G.H. Is a subgroup effect believable? Updating criteria to evaluate the credibility of subgroup analyses. BMJ. 2010;340:c117. doi: 10.1136/bmj.c117. [DOI] [PubMed] [Google Scholar]
  • 23.James P.A., Oparil S., Carter B.L., Cushman W.C., Dennison-Himmelfarb C., Handler J. 2014 evidence-based guideline for the management of high blood pressure in adults: report from the panel members appointed to the Eighth Joint National Committee (JNC 8) JAMA. 2014;311:507–520. doi: 10.1001/jama.2013.284427. [DOI] [PubMed] [Google Scholar]
  • 24.Leenen F.H., Nwachuku C.E., Black H.R., Cushman W.C., Davis B.R., Simpson L.M. Clinical events in high-risk hypertensive patients randomly assigned to calcium channel blocker versus angiotensin-converting enzyme inhibitor in the antihypertensive and lipid-lowering treatment to prevent heart attack trial. Hypertension. 2006;48:374–384. doi: 10.1161/01.HYP.0000231662.77359.de. [DOI] [PubMed] [Google Scholar]
  • 25.Manuel D.G., Ho T.H., Harper S., Anderson G.M., Lynch J., Rosella L.C. Modelling preventive effectiveness to estimate the equity tipping point: at what coverage can individual preventive interventions reduce socioeconomic disparities in diabetes risk? Chronic Dis Inj Can. 2014;34:94–102. [PubMed] [Google Scholar]
  • 26.Spencer F.A., Iorio A., You J., Murad M.H., Schunemann H.J., Vandvik P.O. Uncertainties in baseline risk estimates and confidence in treatment effects. BMJ. 2012;345:e7401. doi: 10.1136/bmj.e7401. [DOI] [PubMed] [Google Scholar]
  • 27.Iorio A., Spencer F.A., Falavigna M., Alba C., Lang E., Burnand B. Use of GRADE for assessment of evidence about prognosis: rating confidence in estimates of event rates in broad categories of patients. BMJ. 2015;350:h870. doi: 10.1136/bmj.h870. [DOI] [PubMed] [Google Scholar]
  • 28.World Health Organization . World Health Organization; Geneva, Switzerland: 2011. Guideline: vitamin A supplementation in infants and children 6-59 months of age.http://apps.who.int/iris/bitstream/10665/44664/1/9789241501767_eng.pdf?ua=1&ua=1 Available at. Accessed June 1, 2017. [Google Scholar]
  • 29.NACCHO/RACGP . 2nd ed. National Aboriginal Community Controlled Health Organisation (NACCHO) and The Royal Australian College of General Practitioners (RACGP); South Melbourne: 2012. National guide to a preventive health assessment for Aboriginal and Torres Strait Islander people. Available at http://www.racgp.org.au/download/documents/AHU/2ndednationalguide.pdf. Accessed June 1, 2017. [Google Scholar]
  • 30.Balshem H., Helfand M., Schunemann H.J., Oxman A.D., Kunz R., Brozek J. GRADE guidelines: 3. Rating the quality of evidence. J Clin Epidemiol. 2011;64:401–406. doi: 10.1016/j.jclinepi.2010.07.015. [DOI] [PubMed] [Google Scholar]
  • 31.Lewin S., Glenton C., Munthe-Kaas H., Carlsen B., Colvin C.J., Gulmezoglu M. Using qualitative evidence in decision making for health and social interventions: an approach to assess confidence in findings from qualitative evidence syntheses (GRADE-CERQual) PLoS Med. 2015;12:e1001895. doi: 10.1371/journal.pmed.1001895. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Guyatt G.H., Oxman A.D., Kunz R., Woodcock J., Brozek J., Helfand M. GRADE guidelines: 8. Rating the quality of evidence–indirectness. J Clin Epidemiol. 2011;64:1303–1310. doi: 10.1016/j.jclinepi.2011.04.014. [DOI] [PubMed] [Google Scholar]
  • 33.Bartlett C., Doyal L., Ebrahim S., Davey P., Bachmann M., Egger M. The causes and effects of socio-demographic exclusions from clinical trials. Health Technol Assess. 2005;9:1–152. doi: 10.3310/hta9380. [DOI] [PubMed] [Google Scholar]
  • 34.Braun J., Brandt J., Listing J., Zink A., Alten R., Golder W. Treatment of active ankylosing spondylitis with infliximab: a randomised controlled multicentre trial. Lancet. 2002;359:1187–1193. doi: 10.1016/s0140-6736(02)08215-6. [DOI] [PubMed] [Google Scholar]
  • 35.O'Dell J.R., Mikuls T.R., Taylor T.H., Ahluwalia V., Brophy M., Warren S.R. Therapies for active rheumatoid arthritis after methotrexate failure. N Engl J Med. 2013;369:307–318. doi: 10.1056/NEJMoa1303006. [DOI] [PubMed] [Google Scholar]
  • 36.Charytan D., Kuntz R.E. The exclusion of patients with chronic kidney disease from clinical trials in coronary artery disease. Kidney Int. 2006;70:2021–2030. doi: 10.1038/sj.ki.5001934. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.Barnett K., Mercer S.W., Norbury M., Watt G., Wyke S., Guthrie B. Epidemiology of multimorbidity and implications for health care, research, and medical education: a cross-sectional study. Lancet. 2012;380:37–43. doi: 10.1016/S0140-6736(12)60240-2. [DOI] [PubMed] [Google Scholar]
  • 38.Welch V., Brand K., Kristjansson E., Smylie J., Wells G., Tugwell P. Systematic reviews need to consider applicability to disadvantaged populations: inter-rater agreement for a health equity plausibility algorithm. BMC Med Res Methodol. 2012;12:187. doi: 10.1186/1471-2288-12-187. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Greenaway C., Sandoe A., Vissandjee B., Kitai I., Gruner D., Wobeser W. Tuberculosis: evidence review for newly arriving immigrants and refugees. CMAJ. 2011;183:E939–E951. doi: 10.1503/cmaj.090302. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40.Smith B.D., Morgan R.L., Beckett G.A., Falck-Ytter Y., Holtzman D., Teo C.G. Recommendations for the identification of chronic hepatitis C virus infection among persons born during 1945–1965. MMWR Recomm Rep. 2012;61:1–32. [PubMed] [Google Scholar]
  • 41.Furler J., Magin P., Pirotta M., van Driel M. Participant demographics reported in “Table 1” of randomised controlled trials: a case of “inverse evidence”? Int J Equity Health. 2012;11:14. doi: 10.1186/1475-9276-11-14. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Welch V., Tugwell P., Petticrew M., de Montigny J., Ueffing E., Kristjansson B. How effects on health equity are assessed in systematic reviews of interventions. Cochrane Database Syst Rev. 2010;(12):MR000028. doi: 10.1002/14651858.MR000028.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43.Morton R.L., Schlackow I., Mihaylova B., Staplin N.D., Gray A., Cass A. The impact of social disadvantage in moderate-to-severe chronic kidney disease: an equity-focused systematic review. Nephrol Dial Transplant. 2016;31:46–56. doi: 10.1093/ndt/gfu394. [DOI] [PubMed] [Google Scholar]

RESOURCES