Methodological guidance for incorporating equity when informing rapid-policy and guideline development

Abstract

Objectives

We provide guidance for considering equity in rapid reviews through examples of published COVID-19 rapid reviews.

Study Design and Setting

This guidance was developed based on a series of methodological meetings, review of internationally renowned guidance such as the Cochrane Handbook and the Preferred Reporting Items for Systematic Reviews and Meta-Analysis for equity-focused systematic reviews (PRISMA-Equity) guideline. We identified Exemplar rapid reviews by searching COVID-19 databases and requesting examples from our team.

Results

We proposed the following key steps: 1. involve relevant stakeholders with lived experience in the conduct and design of the review; 2. reflect on equity, inclusion and privilege in team values and composition; 3. develop research question to assess health inequities; 4. conduct searches in relevant disciplinary databases; 5. collect data and critically appraise recruitment, retention and attrition for populations experiencing inequities; 6. analyse evidence on equity; 7. evaluate the applicability of findings to populations experiencing inequities; and 8. adhere to reporting guidelines for communicating review findings. We illustrated these methods through rapid review examples.

Conclusion

Implementing this guidance could contribute to improving equity considerations in rapid reviews produced in public health emergencies, and help policymakers better understand the distributional impact of diseases on the population.

What is new?

Key findings

• •We provide guidance for incorporating equity in rapid reviews and illustrated their feasibility by providing examples of published rapid reviews considering equity in different stages of their development.

What this adds to what was known?

• •The dependence on rapid reviews for informing policy related to COVID-19 has highlighted gaps in research methods, including the consideration of health equity in rapid reviews.
• •We provide a stepwise approach that has been implemented successfully in COVID-19 rapid reviews.

What is the implication and what should change now?

• •We propose that equity be considered at the forefront of rapid reviews, starting from team values and composition.
• •Develop an evaluation and feasibility framework to assess the impact of this guidance.

1. Introduction

Many public health and policy responses to mitigate the spread of the Coronavirus (COVID-19) in 2020 and 2021 contributed to controlling the transmission of COVID-19 and the burden it places on nations’ health and public health systems. However, some of these interventions may have exacerbated pre-existing health inequities [[1], [2], [3], [4], [5]]. Low-wage workers and racialized communities have been disproportionately affected by the risk and severity of infection and restrictions of non-essential work activities [6,7]. Children experiencing economic vulnerability and food insecurity were likely harmed by school closures [8]. The reduced access to health services has heavily impacted people experiencing disabilities [9,10]. Even with the distribution of the COVID-19 vaccine, underserved and racialized communities have been hesitant to engage with health systems, stemming from a long history of neglect and mistreatment in health research and service delivery [11]. Considering health inequities when developing evidence may mitigate the inequitable accrual of harm and deprivation of interventions that could improve health outcomes.

Methodological guidance for incorporating equity in systematic reviews is available. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses–Equity extension (PRISMA-Equity) 2012 guidelines steer authors of systematic reviews to consider equity at all stages of the review [12,13]. The Cochrane Handbook for Systematic Reviews of Interventions includes a chapter for considering health equity in reviews [14]. Equity could be considered from incorporating an intersectionality lens in question formulation [15] to review processes, such as identifying patient-important outcomes.

Given the rapidly changing conditions of the COVID-19 pandemic, the need for timely, high-quality evidence has never been more apparent [16,17]. Rapid reviews, a form of systematic reviews but less time- and resource-intensive, considering health equity fill this need [18]. The need to engage a broad range of stakeholders to improve the relevance and quality of the research is increasingly acknowledged [19]. A stakeholder is defined as an “individual or group responsible for or affected by health- and healthcare-related decision”, including members of the public [20]. Greater involvement of stakeholders in evidence syntheses can support the inclusion of perspectives of populations experiencing inequities and social and organizational factors that may influence review findings, contributing to a more equitable evidence base [21]. However, there is no guidance on considering health equity through stakeholder engagement or in the process of rapid evidence synthesis.

In this paper, we provide guidance on incorporating equity throughout the rapid review process and provide examples from published COVID-19 rapid reviews to illustrate its application.

2. Methods

We convened an equity task force in the COVID-19 Evidence Network to support Decision-making (COVID-END) network following their principles, which include ensuring diversity, equity and inclusion, to focus on equity issues facing COVID-19 related synthesis [22]. COVID-END is an extensive network of people and organisations from different countries (high, low-and middle-income) engaged in identifying and using the best available evidence to better coordinate the COVID-19 pandemic response. The COVID-END secretariat invited two co-chairs to co-lead (with consideration for gender, geographical balance and career stage). Membership was open to all COVID-END network members. This led to a diverse team, with 70% of the authors identifying as women, 17% of the authors from low-middle-income countries and 40% were early career researchers. We recruited two citizens to the task group, following the same principles to seek diversity, with help from the COVID-END Network. Our team also identified with the following stakeholder groups: 33% as principal investigators, 22% as providers and 11% as policymakers. This paper was developed through iterative meetings of the equity task force and circulation of working drafts. The final draft was then circulated to the broader network for feedback.

2.1. Reviewing existing guidance on incorporating health equity in research

We identified the following resources from the expertise of members within the task force: the Strategy for Patient-Oriented Research (SPOR) Evidence Alliance guidance on intersectionality reflective exercise [23], SPOR Evidence Alliance work and budget plan [24], PRISMA-E guideline [12,25], the equity chapter in the Cochrane Handbook [14] and Sex and Gender Equity in Research (SAGER) guidelines [26]. The intersectionality exercise focuses on intersecting social factors and their interaction with compounding power structures (e.g., media, education system) and forms of discrimination (e.g., sexism) [27,28]. The PRISMA-E guideline recommends concepts which reviewers should consider and report when applying an equity lens in their review. The Cochrane Handbook equity chapter lists the following steps: question development, identification of evidence, appraisal of evidence, evidence synthesis and interpretation of findings. We developed our guidance according to the steps listed in the Cochrane Handbook equity chapter although ensuring that we satisfy reporting standards recommended in the PRISMA-E guideline. Accordingly, we used the PROGRESS-Plus framework which stands for Place of residence, Race or ethnicity, Occupation, Gender or sex, Religion, Education, Social capital, Socioeconomic status, personal characteristics associated with discrimination (e.g., disability), features of relationships (e.g., smoking parents, exclusion from school), and time-dependent relationships (e.g., leaving the hospital) to identify populations experiencing inequities [29].

2.2. Involvement of stakeholders in the development of this guidance

We sought to include stakeholders with different perspectives from the COVID-END network in the design of this guidance. These include patient partners, providers, journal editors and policymakers with expertise in evidence synthesis. Contributors participated in the development of the first draft of the guidance though weekly task force meetings. Subsequently, we used an iterative approach to revise our guidance involving experts in evidence synthesis methodology, health equity experts and policymakers.

2.3. Examples of COVID-19 rapid reviews that incorporated equity

We identified reviews focused on populations experiencing inequities to indicate how review questions can be developed for each factor of PROGRESS-Plus. We searched for Jour. These reviews were identified by searching the National Collaborating Center for Methods and Tools [30], COVID-END inventory [31], SPOR Evidence Alliance [32] and seeking suggestions from the team.

3. Results

We identified the following eight areas where equity can be incorporated in rapid reviews: 1) engaging relevant stakeholders in conducting, designing and interpreting the review, (2) reflecting on equity in team values and composition, (3) identifying population(s) experiencing inequities, (4) conducting searches in relevant inter-disciplinary databases, (5) collecting data for equity, (6) analysing evidence on equity, (7) evaluating the applicability of the findings to populations experiencing inequities or other settings (8) adhering to reporting guidelines for communicating review findings. An illustration of this guidance is shown in Fig. 1 and examples from rapid reviews are provided in Table 1 .

Fig. 1.

A flow chart for applying an equity lens to rapid evidence synthesis.

Table 1.

Examples of equity considerations in the process of rapid evidence synthesis

Steps to consider equity	Rationale	Example(s)
Stakeholder engagement	In the spirit of equity, inclusion and diversity, the research team should consider including representatives of populations that experience inequities and diverse experiences.	In a rapid review on the change in level of vaccine protection over time in COVID-19 vaccinated, there is selected information incorporated into the review provided by patient/citizen partners (2 people) with lived experience on the subject matter [33].
Question formulation	Identifying the priority population, defining where the inequity lies and the choosing the appropriate study designs to answer the question is important for evaluating impacts on health equity.	In the introduction: “As vaccines became available, large proportions of populations over age 12 have been vaccinated and some public health measures have been relaxed, leaving those under age 12 vulnerable to infection and severe illness” [34]. In the eligibility criteria, Guidelines and synthesis were prioritized as they generally take into account the available body evidence and could be applied broadly to subpopulations [34].
Identification of evidence	Evidence relating to populations experiencing health inequities draws not only on health, but social, cultural, and political factors. Thus, authors should consider a wide range of literature when searching for relevant studies.	“A gray literature search was also conducted, including: MedRxiv, Google, McMaster Health Forum (CoVID-END), and websites of international government organizations (e.g., Center for Disease Control and Prevention [CDC], World Health Organization [WHO])” [35]. A review evaluating risk factors for children searched for the population of interest in all possible fields (title, abstract, subject heading, etc.) [34].
Data collection and appraisal of evidence	Contextual factors and study process may influence outcomes as they relate to health equity, so authors should consider such factors and that could help interpret the findings of the study.	“A review assessing the mortality and length of stay outcomes with the use of tele-medicine-supported critical care medicine compared to traditional bedside critical care found that the degree of impact of tele-ICU adoption is linked to location (urban vs. rural) among other factors” [36]. Crawshaw al conducted a qualitative rapid review for aimed at assessing the level of vaccine acceptance in racialized populations. They evaluated the participants included in the qualitative primary studies to verify that the findings of the review apply [37].
Evidence synthesis	To assess the impact of health equity on outcomes, the authors should not only provide average results, but should report differences in effects across populations of interest.	“Unknown length of surgical delay highest source of anxiety - male were more likely to proceed in spite of COVID-19 risk, Only 7% stated that they would continue to delay due to fear of contracting COVID-19 in hospital” [38].
Interpretation of findings	Focusing on interpreting the evidence available for the previously identified priority populations as not all evidence is applicable to all groups of the population.	“Across studies exploring perceptions of different vaccines, safety was a primary concern both as a motivator for seeking vaccination (i.e., to protect oneself and others from illness) and as a reason to not seek vaccination (i.e., potential side effects) [for First Nations, Inuit and Métis peoples in Canada and Indigenous Peoples globally]. The confidence in this finding is low (GRADE-CERQual) however, it is possible that this finding is a reasonable representation of the phenomenon of interest” [39].

From our review of existing guidance, considering equity in rapid reviews requires attention at different stages of its development. We felt that reflecting on equity in the team composition (section 3.2), question formulation (section 3.3) and reporting sample characteristics (section 3.5) would be applicable and feasible for all rapid reviews. Subsequent steps need to be decided on based on resources, priorities, and whether equity is an objective of the review.

3.1. Involving relevant stakeholders in the conduct and design of the review

Focusing on equity reflects a concern for diversity, inclusion, and justice. Thus, the participation of those affected by inequities in the research process is paramount. We highlight key steps that could contribute to equity in the research team and the review development process. The selection of stakeholder groups depends on the nature and scope of the question, and the group representatives should ensure diversity in team expertise and lived experience [40,41].

Due to the expected quick turnaround time for reviews in the pandemic, best practices may need to be modified to suit the timelines, such as engaging individuals with experience or developing alternative approaches for training. One example is the 10-hour rapid review course through the SPOR Evidence Alliance; this course was co-designed and co-delivered by two experienced patient partners [42]. These strategies could be used to develop a cadre of stakeholders that could be drawn on for multiple reviews.

Stakeholders could also critique the study question to ensure it is relevant to policy and clinical practice. Often questions are defined by the commissioner with little room for changes, however stakeholders could identify further questions that could be addressed in the review. Stakeholders could also identify interdisciplinary libraries and gray literature sources, provide insights on participant characteristics, study design features or identify outcomes that may be relevant to addressing equity, provide their perspectives on the relevancy of key findings, and participate in appropriately disseminating the evidence (e.g., plain language summaries).

3.2. Reflecting on equity in team values and composition

Equity considerations commence from the stage of team formation; equity values should be formulated as part of the team values and culture. To ensure that a supportive environment is provided within the research team, research team members should discuss participating in at least one of the potential Equity, Diversity and Inclusion (EDI) training activities such as the SPOR Evidence Alliance's reflective EDI exercise [23], San'yas indigenous cultural safety training [43] and Equity training provided by the National Equity Project [44]. Taking this training together as a team can build trust and foster a safe space for meaningful discussion. Furthermore, team members should consider completing training that improves team capacity building and effective stakeholder engagement [45].

Including people with lived experience relevant to the review topic as part of the team strengthens the review process by incorporating context-specific understanding, based on experience and tacit understanding of an issue [46]. Doing so requires the research team to address how to support effective and meaningful engagement with those stakeholders, although also building in supports and recognition for those contributing their experience-based expertize. For example, the research team could consider compensating stakeholders–especially patients, caregivers or members of the public for their contributions [[47], [48], [49]] and refer to appropriate support in the event of increased patient stakeholders’ stress when discussing their lived experience.

3.3. Developing research question to assess health inequities

When equity is discussed at the stage of question formulation, the review authors could focus on a population experiencing inequities (the PROGRESS-Plus framework can aid in the identification process) or consider such populations as subgroups of interest [37,39,50]. Box 1 provides examples of rapid reviews focused on populations experiencing inequities. The review authors should supplement these decisions with an a priori definition of how the intervention is expected to influence health equity for the identified populations. The inclusion criteria of studies could be restricted to a specific context to account for the applicability of the findings. For example, “studies included in this review were restricted to those conducted in countries with welfare systems relevant to the Norwegian context” [62].

Box 1. Examples of COVID-19 rapid evidence synthesis questions focused on populations experiencing inequities across PROGRESS-Plus.

Place of residence: How do rural communities and health systems prepare for and respond to pandemics or disease outbreaks? [51]

Race or ethnicity: What is known about the impact of the COVID-19 pandemic on Indigenous communities in Canada? [52]

Occupation: What is known about health care worker intent to leave their occupation in the context of the COVID-19 pandemic? [53]

Gender or sex: What interventions and strategies can health systems use to sustain and improve health and wellbeing of women, children and adolescents during pandemics and epidemics? [54]

Religion: What is the excess burden of morbidity and mortality from COVID-19 experienced by members of the Muslim community? [55]

Education: Does education (among other factors) impact adherence to COVID-19 public health guidelines, including physical distancing, wearing face masks and hand hygiene? [56]

Socioeconomic status: What is known about the harms being experienced by community dwelling low-income populations from staying at home for long periods of time during current or past pandemics? [57]

Social capital: What is the risk of COVID-19 transmission associated with different (community) activities (e.g., dining, exercising) or settings (e.g., educational, hospitality)? [58]

Disability: What is the impact of COVID-19 on people with physical disabilities? [59]

Features of relationships: What is the effectiveness of COVID-19 vaccines in adults living in long-term care facilities or congregate care for older adults? [60]

Time-dependent relationships: What are the social and health and well-being harms of staying at home during current or past pandemic experienced by community dwelling populations living with low income? [61]

It is common for inequities to coexist across different dimensions and interact, causing multiplicative effects. This has also been shown for comorbidities for people experiencing disabilities, and they are frequently excluded from primary studies [63]. Glover et al. has demonstrated that these intersecting inequities may result in more severe adverse effects caused by COVID-19 policies [1]. Review authors may therefore, decide to investigate the effect of intersectionality on populations experiencing inequities.

Review authors should also choose the study designs according to their “fitness for purpose” and, if possible, provide a rationale for their choice [64].

3.4. Conducting searches in relevant disciplinary databases

Reviewers may need to consider searches in social databases or other inter-disciplinary databases from low- and middle-income countries to identify relevant evidence for socio-economic impacts on different populations, depending on the review's objective (interventions vs. barriers and facilitators, etc.). Local databases and governmental and non-governmental websites could be investigated as potential gray literature sources. Review authors should also ensure that search terms capturing equity-related content have been included within the search string. Authors should aim to adopt validated filters relevant to their topic when searching for studies that are equity relevant [[65], [66], [67], [68]]. If there are no validated filters, authors should be mindful that unvalidated equity filters could limit their searches and risk missing relevant evidence. For example, a filter that restricts to English language studies in the case of COVID-19.

3.5. Collecting data for equity

Rapid reviews with an equity lens need to plan the variables of interest for data collection across PROGRESS-Plus [29] or other dimensions associated with inequities. This step is necessary for evidence appraisal and analysis across dimensions of inequities.

The review authors should capture elements of study design to evaluate the nature of participant inclusion or exclusion as it may influence the applicability of the results for populations experiencing inequities [69,70]. Review authors should also assess if the chosen methodology and theories by the primary authors articulate possible pathways to addressing inequities [71,72].

Reviewers should collect data on sample characteristics such as context and population demographics that interact with other contextual elements and influence health inequities. Capturing information on retention and attrition across populations experiencing inequities is also essential, as they may affect the generalizability of the review findings. When possible, outcome data should be collected in both relative and absolute differences between groups.

3.6. Analysing evidence on equity

Analysis of equity includes critical appraisal and analyses to explore equity questions. Critical appraisal assesses study design factors like recruitment and attrition that influence health equity. The approach for appraisal of evidence depends on the type of evidence investigated. The review authors should consider checking for baseline imbalance across PROGRESS-Plus factors for quantitative evidence. When appraising qualitative evidence, the review authors should consider if the primary research authors designed the question to assess outcomes related to health equity (i.e., impact of intervention, acceptability) by evaluating if and how they included populations experiencing inequities.

Additional synthesis methods may be needed to address questions related to equity. Subgroup analyses are usually conducted. Other methods such as moderator analysis, meta-regression and sensitivity analysis may be more relevant, depending on the question and how the review authors decide to consider equity at the question conceptualization stage. All these analyses should be pre-planned, accompanied with a rationale linked to an analytical framework (i.e., logic model) [73] and adhere to reporting standards to ensure their credibility [[74], [75], [76]].

For qualitative evidence, consider the sources of the quotations and how they were analyzed [77,78]. These analyses should also be pre-planned and accompanied by theory-based rationales (e.g., a logic model) [79].

3.7. Evaluating the applicability of the findings to populations experiencing inequities or other settings

Influence on health equity should be interpreted from the findings of the review. The principles of interpretation include: (1) evaluating who was included in the studies and judging if they are representative of people with the condition; (2) if there were any differences in recruitment, retention, effects found, what are the potential impacts on policy and practice. Cochrane reviews require the use of the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach to formally evaluate the quality of the overall body of evidence [[80], [81], [82]]. GRADE quality of evidence includes assessment of directness to the population of interest, consistency across studies, imprecision of findings and risk of bias resulting from inherent design or conduct of studies and publication bias. This tool could link the confidence of the findings to the population of interest. However, as a rule of thumb, the certainty of evidence should not be rated down for indirectness unless there is compelling evidence for differences in effect due to variations across populations [83]. GRADE-CERQual could be used for qualitative evidence to evaluate the confidence in the findings to the population of interest.

3.8. Adhering to reporting guidelines for communicating review findings

Reporting guidelines improve the reporting of different study designs [84,85]. Adopting reporting guidelines such as the PRISMA-Equity [12], SAGER guidelines [26] and International Committee of Medical Journal Editors (ICMJE) [86] when constructing the review encourages the completeness of reporting of equity-relevant information. This information is vital for emphasizing equity in the review, leading to policymakers' improved judgment of applicability and integration in policies and programs.

4. Discussion

We identified areas where researchers could consider equity in rapid review development. This guidance could be used by groups and agencies responsible for rapid decision-making during emergencies to ensure that populations experiencing inequities are considered when informing policy and developing guideline recommendations.

Although there is evidence on how marginalization impacts poor and socially isolated groups’ health, their perspectives are often poorly reflected in available evidence bases [87]. Greater involvement of these stakeholders in reviews can support greater inclusion of social factors that may influence review findings [21,[88], [89], [90]]. Major funding institutes such as the Canadian Institute of Health Research (CIHR) and National Institutes of Health Research (NIHR), support the inclusion of patients, the public and other end-users in the research process [91]. However, despite major advances and recognition of the importance of patient and public involvement in clinical and policy decision-making, their level of involvement remains low, possibly due to the rapidly evolving nature of the COVID-19 pandemic [92]. Nevertheless, we argue that it is even more critical to engage stakeholders, particularly those most likely to experience disproportionate harm, and hope this guidance facilitates this process for future effective pandemic preparedness [93,94].

Rapid reviews need to have a translation plan that considers how to convey findings on equity to impact health systems and health outcomes [[95], [96], [97], [98], [99], [100]]. However, policymakers face several challenges when applying a health equity lens [[101], [102], [103]]. Engaging stakeholders throughout the process, including developing messages for relevant audiences, is consistent with an integrated knowledge translation (iKT) approach [104]. Policymakers need to consider balancing the goal of improving overall population health to reduce health inequities [105,106]. The iKT products may want to include evidence on overall health and distributional health outcomes, if possible, to inform decisions.

Our approach to developing this guidance has limitations. First, we developed this guidance through an iterative approach with weekly meetings among the author team and circulated the paper with the wider COVID-END group. Our team includes individuals with diverse backgrounds and different experiences. However, the proportion of black and indigenous color (BIPOC) individuals is unknown. Second, the guidance we drew upon was not systematically searched; instead, we depended on the expertize of COVID-END group members. COVID-END includes 57 partners from various organizations of evidence synthesis, technology assessment and guideline development communities. Thus, covering the full spectrum of contexts where evaluating the pandemic response is taking place. Third, we did not find an exemplar review that applied all the proposed steps in the review process so applying all the available guidance in a single review may disrupt the short time frame required by commissioners of rapid evidence syntheses.

5. Conclusion

The COVID-19 pandemic has highlighted the magnitude of health inequities existing across the globe. The dynamic nature of the pandemic calls for rapid and up-to-date evidence to inform policy and decision-making. We anticipate that researchers conducting rapid reviews in the COVID-19 pandemic and other public health emergencies will find the guidance we propose in this paper helpful in explicitly considering health equity in their development process. Meaningful and timely patient and public involvement appears more and more clearly as a necessity because it has been argued that ‘the insights they provide are the key to ethical decision making, which is the only sustainable solution to inequities’ [107].

CRediT authorship contribution statement

Omar Dewidar: Conceptualization, Methodology, Writing – original draft, Writing – review & editing, Visualization. Brenda Allen Kawala: Conceptualization, Methodology, Writing – original draft, Writing – review & editing, Visualization. Alba Antequera: Investigation, Writing – review & editing. Andrea C. Tricco: Conceptualization, Methodology, Writing – review & editing. David Tovey: Methodology, Writing – review & editing. Sharon Straus: Methodology, Writing – review & editing. Rebecca Glover: Methodology, Writing – review & editing. Janice Tufte: Methodology, Writing – review & editing. Olivia Magwood: Methodology, Writing – review & editing. Maureen Smith: Methodology, Writing – review & editing. Cheow Peng Ooi: Methodology, Writing – review & editing. Anna Dion: Methodology, Writing – review & editing. Mireille Goetghebeur: Conceptualization, Methodology, Writing – review & editing. Ludovic Reveiz: Methodology, Writing – review & editing. Stefano Negrini: Methodology, Writing – review & editing. Peter Tugwell: Methodology, Writing – review & editing. Jennifer Petkovic: Methodology, Writing – review & editing. Vivian Welch: Conceptualization, Methodology, Writing – review & editing, Supervision.