Skip to main content
NCBI home page
F1000Research logoLink to F1000Research
. 2022 Jun 6;11:615. [Version 1] doi: 10.12688/f1000research.122185.1

Reporting of health equity considerations in equity-relevant observational studies: Protocol for a systematic assessment

Omar Dewidar 1,a, Tamara Rader 2, Hugh Waddington 3, Stuart G Nicholls 4, Julian Little 5, Billie-Jo Hardy 6, Tanya Horsley 7, Taryn Young 8, Luis Gabriel Cuervo 9, Melissa K Sharp 10, Catherine Chamberlain 11,12, Beverley Shea 5, Peter Craig 13, Daeria O Lawson 14, Anita Rizvi 15, Charles Shey Wiysonge 16, Tamara Kredo 8,16, Miriam Nkangu Nguliefem 5, Elizabeth Ghogomu 1, Damian Francis 17, Elizabeth Kristjansson 4, Zulfiqar Bhutta 18,19, Alba Antequera Martin 20, G J Melendez-Torres 21, Tomas Pantoja 22, Xiaoqin Wang 23, Janet Jull 24, Janet Hatcher Roberts 25, Sarah Funnell 26,27, Howard White 28, Alison Krentel 1,5, Michael Johnson Mahande 29, Jacqueline Ramke 30,31, George A Wells 32, Jennifer Petkovic 1, Peter Tugwell 1,4,33, Kevin Pottie 34,35, Lawrence Mbuagbaw 14, Vivian Welch 1,5
PMCID: PMC12413612  PMID: 40919294

Abstract

Background: The mitigation of unfair and avoidable differences in health is an increasing global priority. Observational studies including cohort, cross-sectional and case-control studies tend to report social determinants of health which could inform evidence syntheses on health equity and social justice. However, the extent of reporting and analysis of equity in equity-relevant observational studies is unknown.

Methods: We define studies which report outcomes for populations at risk of experiencing inequities as “equity-relevant”. Using a random sampling technique we will identify 320 equity-relevant observational studies published between 1 January 2020 to 27 April 2022 by searching the MEDLINE database. We will stratify sampling by 1) studies in high-income countries (HIC) and low- and middle-income countries (LMIC) according to the World Bank classification, 2) studies focused on COVID and those which are not, 3) studies focused on populations at risk of experiencing inequities and those on general populations that stratify their analyses. We will use the PROGRESS framework which stands for place of residence, race or ethnicity, occupation, gender or sex, religion, education, socioeconomic status, social capital, to identify dimensions where inequities may exist. Using a previously developed data extraction form we will pilot-test on eligible studies and revise as applicable.

Conclusions: The proposed methodological assessment of reporting will allow us to systematically understand the current reporting and analysis practices for health equity in observational studies. The findings of this study will help inform the development of the equity extension for the STROBE (Strengthening the Reporting of Observational studies in Epidemiology) reporting guidelines.

Keywords: Observational studies, health equity, research reporting, research methodology, study design

Introduction

Less than a decade remains until the 2030 deadline set by the United Nations to meet the 17 Sustainable Development Goals 1 aimed at global economic, social, and environmental transformation. One of the 17 goals, and a cross-cutting theme across several others, 2 is health equity, defined as the absence of differences in health that are avoidable and unjust. 3 The achievement of health equity as a priority has been widely endorsed by several international organizations 4 , 5 and resonates with the moral imperative of social justice and recognition of human rights principles.

A major challenge is the lack of evidence that includes thoughtful, rigorous collection of health equity data to inform program development and policymaking. 6 In fact, global organizations such as the World Health Organization (WHO) and the Pan American Health Organization (PAHO) have called for the improvement of equity considerations in the reporting of research. 7 – 10 Thus, the integration of health equity in research is needed in both primary and secondary research. 11 – 14 In primary research, observational studies are particularly well-suited for investigating health equity as they often include populations at risk of experiencing inequities. 15 , 16 Observational studies are analytical or descriptive studies evaluating a research question without changing exposure to an intervention. 17 In the absence of a control group, they can be descriptive in nature but do not permit the investigation of causal associations. 18 Observational studies predominate in health-related research, 19 , 20 and are used to investigate numerous types of research questions, such as detecting rare or late adverse effects of treatments and to explore complex systems. 21 Furthermore, observational studies have highlighted the vast inequities in society during the COVID-19 pandemic, 22 and simultaneously, have had an important role in informing public health responses. 23 , 24

The well-known STROBE (Strengthening Reporting of Observational studies in Epidemiology) reporting guideline, which is intended to be used for the reporting of observational studies, is used by over 60% of authors of observational studies. 25 However, despite widespread use of the STROBE reporting guidelines, there is a persistent lack of integration and reporting of sex and gender (among other dimensions) in observational studies. 26 – 28 Comprehensive reporting in research is essential to assess its reliability, reproducibility and methodological rigour, and ensure that the research meets the needs of the potential users. 29 Using reporting guidelines contributes to meeting these goals by increasing the completeness and transparency of research papers. 30 – 33 The inadequate reporting may be, in part, due to lack of guidance on reporting equity items in such studies.

Complex health systems questions and addressing social determinants of health may require multi-stakeholder and intersectoral collaboration and engagement, for which equity perspectives can serve as a linchpin. 34 Observational studies also allow for addressing questions on health policy, health systems, and health services organization, delivery, prioritization, and implementation. 35 – 37 Different frameworks are available to describe factors associated with health equity. 38 , 39 The Campbell and Cochrane Equity Methods Group, for example, proposes using the PROGRESS-Plus framework. This acronym is widely used and stands for Place of residence, Race or ethnicity, Occupation, Gender or sex, Religion, Education, Socioeconomic status, and Social capital. 39 The “plus” accounts for additional characteristics that could result in inequitable health outcomes. Although the reporting on equity in observational research would increase the value of research outcomes through the recognition of how these dimensions impact interventions, the application and use of equity frameworks in reporting health research are not yet standard practice. Therefore, our objective is to systematically evaluate how equity-relevant observational studies describe characteristics of their samples, design features and analysis, and interpretation of their findings across PROGRESS-Plus; we also aim to use the findings of this study to inform the development of an equity reporting guideline.

Methods

Figure 1 outlines the overarching process to be followed in the conduct of this study; from identifying relevant studies at title and abstract stage to data extraction and analysis.

Figure 1. Study flow diagram.

Figure 1.

Open in a new tab

Eligibility criteria

We will include health equity-relevant observational studies from all settings, published between 1 January 2020, and 27 April 2022. We define health equity, health equity-relevant studies and observational studies in Table 1. We chose to focus on equity-relevant studies, since these are most likely to have methods of describing populations experiencing inequities, and are most likely to include methods to assess similarities or differences in effects for populations experiencing inequities.

Table 1. Definitions of terms used in the methodological assessment.

Term Definition and explanation
Health inequity Health inequity is defined as avoidable and unfair differences in health. 3
Health equity-relevant studies We define “health equity-relevant” as studies that focus on individuals or populations experiencing inequities or studies of mixed populations that analyzed at least one health outcome across one or more PROGRESS factor. 12 , 40 We define health outcomes according to the WHO definition, as “changes as in health status that result from the provision of health services”. 41 This includes studies focused on populations at risk of experiencing inequities such as people who are homeless and asylum seekers, as well as studies which present disaggregated results or evaluate outcomes across one or more PROGRESS factors. However, studies which control for PROGRESS in analyses without analyzing differences across the factors are not considered equity-relevant. For example, studies of breast cancer in women would not be considered health equity-relevant since women are not disadvantaged in opportunities for health as it relates to breast cancer care. However, if stratified analysis was done to compare mortality of women living in rural vs urban area (or other PROGRESS factors), the study would be considered equity-relevant. Similarly, a study of low-income or racialized women accessing breast cancer care would be considered health equity-relevant because they experience different opportunities in obtaining care.
Observational studies We define observational studies as analytical or descriptive studies evaluating a research question without changing exposure to an intervention. 17 , 18 Observational studies are classified in the STROBE reporting guideline into three types: 1) cohort: following an exposed population over time; 2) case-control: comparing exposures between people with a particular disease outcome (cases) and people without that outcome (controls); and 3) cross sectional: assessing all individuals in a sample at the same point in time. 21 Studies conducted using routinely collected data stored in administrative datasets can be classified into these three types of studies. 42
Open in a new tab

We will not include studies focused on Indigenous populations in this evaluation, as they will be assessed in a separate but complementary study led by Indigenous researchers as it will require a different sampling approach and methodological assessment. We decided not to use “Plus” criteria for eligibility since they require a greater deal of judgement regarding the contextual factors that may result in inequities. We will exclude case series and case reports since they do not include a reference population or comparison group enabling assessment of differences across PROGRESS factors.

Sampling strategy

Based on consultation with our interdisciplinary team of researchers and decision-makers, and patients and public, we will use a three-factor, randomized sampling strategy to balance: 1) studies in high-income countries (HIC) and low- and middle-income countries (LMIC) according to the World Bank classification, 2) studies focused on COVID-19 and those which are not, 3) studies focused on populations experiencing inequities and those that stratify their analyses. We chose to stratify by country because of the importance of the context and resources in LMIC settings which affect health inequities both among LMIC and between LMIC and HIC, 43 – 45 Given the evidence on exacerbation of inequities during the COVID-19 pandemic, we decided to ensure balance across studies related to COVID-19 and those that are not. 46

Although we chose only three factors for the sampling strategy, we will assess all aspects of PROGRESS-Plus in the included studies; thus other populations experiencing inequities will be included, such as people who are living with low income, homeless, migrants, asylum-seekers, and racialized individuals. Table 2 lists examples of studies that meet our eligibility criteria.

Table 2. Examples of health equity-relevant observational studies published after 2020.

LMIC HIC
COVID+ COVID- COVID+ COVID-
General population- i.e. mixed Epidemiological characteristics of patients with severe COVID-19 infection in Wuhan, China: evidence from a retrospective observational study 47 Effectiveness of seasonal malaria chemoprevention at scale in west and central Africa: an observational study 48 Patterns of use of secondary mental health services before and during COVID-19 lockdown: observational study 49 School Entry, Educational Attainment, and Quarter of Birth: A Cautionary Tale of a Local Average Treatment Effect 50
Focused on populations experiencing inequities Ethnic and regional variations in hospital mortality from COVID-19 in Brazil: a cross-sectional observational study 51 Demystifying the factors associated with rural–urban gaps in severe acute malnutrition among under-five children in low- and middle-income countries: a decomposition analysis 52 Hesitant or Not? The Association of Age, Gender, and Education with Potential Acceptance of a COVID-19 Vaccine: A Country-level Analysis 53
A case-control and cohort study to determine the relationship between ethnic background and severe COVID-19 54 		Health Care Utilization Before and After the “Muslim Ban” Executive Order Among People Born in Muslim-Majority Countries and Living in the US 55
Open in a new tab

Sample size

We will randomly select a sample of 320 equity-relevant observational studies, since assessing all potential eligible studies would be unmanageable for our review team. To ensure we assess a sample that provides comparable results, we conducted sample size calculation for binary outcomes (categorical questions for consideration of equity in the studies) with 95% confidence intervals of ± 6% for observed proportions of 50% ( i.e. assuming that half of the studies would report at least one PROGRESS characteristic). This sample size has also been used in similar studies. 56 – 59 We acknowledge that the study does not have sufficient power to allow a comparison of reporting across the three prioritized groups, so we will only report data descriptively. Eligible studies will be exported into Microsoft Excel and sorted using a random sequence using the built-in random-number generator. We will sample the studies as specified in Table 3.

Table 3. Sampling strategy for equity-relevant studies across the prioritized topics.

LMIC HIC Total
COVID+ COVID- COVID+ COVID-
General population 40 40 40 40 160
Focused on populations experiencing inequities 40 40 40 40 160
Total 80 80 80 80 320
Open in a new tab

COVID+: studies related to COVID-19, COVID-: studies not related to COVID-19. LMIC: Low-middle income countries, HIC: High Income countries.

Search strategy

We have designed a search strategy in collaboration with a librarian (TR) experienced in systematic reviews, using a method designed to optimize term selection. 60 A validated search filter to identify equity-focused studies was adapted for use in this study, 61 and additional terms to increase retrieval of papers about indigenous populations were added (lines 213-232). 62 , 63 These were developed in collaboration with our Indigenous Steering Committee. Studies related to Indigenous populations will be identified from these search terms and will be utilized to conduct an independent but highly complementary study focused on Indigenous populations.

We used a validated search filter for observational studies with two extra terms to retrieve cross-sectional studies. 64 We further added the terms “Instrumental variable”, “discontinuity design”, “interrupted time series”, “discontinuity design”, “matching”, “synthetic control” and “difference-in-difference” to our search strategy to ensure that we capture observational econometric studies that meet our eligibility criteria. Review articles and randomized controlled trials will be excluded from these results using validated study design filters for Ovid MEDLINE. 65 The draft search strategy is stored in the Open Science Framework (OSF) repository ( Extended data 67 ). We will test the search strategy with a set of 10 studies that we identify to be eligible.

We will restrict our search to the last two years because we are interested in describing current analysis and reporting of equity, not trend over time. This period also captures the girth of the literature published throughout the pandemic. We conducted a preliminary search of Ovid MEDLINE for papers published between 1 January 2020, to 17 April 2022 and obtained 16,263 citations. Due to the high volume of results retrieved, we decided that searching only one database would be feasible. In the conduct of this study, we expect an adequate number of observational studies in this period to meet our eligibility criteria.

Screening

Studies will be screened at title and abstract level by a single investigator using DistillerSR software. 66 One investigator will screen the full texts of studies, and OD will validate. Conflicts will be resolved through discussion with a third reviewer at weekly meetings. We will use the DistillerSR randomization function to randomly sample citations at a set interval of 20% of unreviewed citations until we reach the planned sample size.

At full-text screening, we will tag eligible studies according to our prioritized topics (Type of equity relevance, country income setting, COVID-19 research).

Data extraction

Data will be extracted independently by two extractors. We will develop and pretest a data extraction form using DistillerSR. We will conduct training on the data collection with data extractors using sets of three to five studies at a time, until there is 80% agreement across all items between extractors. Conflicts will be resolved by discussion between extractors and reaching consensus. We will develop a data dictionary to ensure consistent extraction across team members and refine during pre-testing.

We will capture study characteristics including the purpose of the study (focused on populations experiencing inequities or whether the study included general populations in which there may be populations experiencing inequities), title/abstract, background/rationale, study design, population characteristics, results, analyses, interpretation of applicability and discussion. We will collect details on the motivation behind studies for a separate evaluation of the way equity is defined in these studies and the motivation for studying equity-seeking populations. The complete list of items is provided in our OSF repository ( Extended data 67 ).

We will assemble the author contact details for a separate survey of authors (of the included studies) to ask about engagement with populations experiencing inequities, since we believe that engagement with populations experiencing inequities informs better reporting on health equity. The reason for conducting the survey is because we expect that some of the results related to equity would not be well reported in the articles.

Data analysis

We will descriptively present the reporting of each item for the observational studies in our sample by tabulating the data for each item to assess the frequency of reporting. We will conduct subgroup analysis on studies that considered health equity as a reason for conducting the study.

Dissemination

Our findings will be shared on pre-print servers and submitted for peer-review publication. In addition, we plan on sharing our findings on Twitter and at relevant conferences for wider dissemination.

Discussion

The primary objective of this study will be to describe the reporting and analysis of equity in equity-relevant observational studies. By focusing on studies that meet this criteria, we will document studies that have missed opportunities for providing information regarding populations at risk of experiencing inequities. Moreover, this exercise may help us identify examples of equity considerations in observational studies, according to our candidate items for the equity extension for STROBE reporting guideline, to illustrate how and to what extent they are presented in the literature.

Our study is not without limitations. First, our search of only MEDLINE would be expected to yield studies mostly from HIC and focused on clinical topics. However, we plan to mitigate this selection bias by purposefully stratifying sampling across country income setting and randomly selecting a sample from the eligible studies. Second, although a third of early COVID-19 literature has been shared as preprints, we will not include studies from pre-print servers as studies in peer-review journal articles tend to show substantial improvements in the quality of reporting compared to pre-prints. Therefore, it may not be appropriate to pool preprints with peer-reviewed published articles. Future work could aim to investigate the difference in reporting of health equity in articles published in pre-print servers compared to peer-reviewed articles.

The proposed list of extraction items is extensive and provides several gateways for future projects to fill gaps in understanding the integration and reporting of equity in research. Using the results of this methodological assessment, we plan to conduct a sociological discourse analysis of motivation for assessing health equity in equity-relevant observational studies. In accordance with the plan for this extension, we will conduct an independent study on Indigenous research, led by Indigenous researchers, to identify and describe well-reported studies using the CONSIDER checklist. Furthermore, the selective sampling across prioritized groups provides an opportunity to conduct more focused investigations, tailored to each respective topic ( i.e. LMIC, COVID-19 research).

Data availability

Underlying data

No data are associated with this protocol.

Extended data

Open Science Framework: STROBE-equity reporting guidelines, https://osf.io/cp3z2. 67

This project includes the following extended data:

  • •

    Search strategy.docx

  • •

    Extraction items mapped to the STROBE-Equity candidate items.docx

Data are available under the terms of the Creative Commons Attribution 4.0 International license (CC-BY 4.0).

Acknowledgements

We would like to acknowledge our partners and collaborators on the STROBE-Equity team.

Indigenous Research Steering Committee: Catherine Chamberlain (Australia), Sarah Funnell (Canada), Billie-Jo Hardy (Canada), Matire Harwood (Aotearoa New Zealand), Michelle Kennedy (Australia).

Patients and Public Steering Committee: Holly Ellingwood (Canada), Regina Greer-Smith (USA), Clara Juando-Prats (Canada), Janet Jull (Canada), Bev Shea (Canada), Janice Tufte (USA).

Knowledge Users Steering Committee: Marc Avey (Canada), Luis Gabriel Cuervo (USA), Tanya Horsley (Canada), Tamara Kredo (South Africa), Elizabeth Loder (USA), Melissa Sharp (Ireland), Laura Weeks (Canada), Howard White (UK).

Technical Oversight Committee: Elie A. Akl (Lebanon), Alba Antequera (Barcelona (Spain)), Zulfiqar A. Bhutta (Canada), Peter Craig (Scotland), Omar Dewidar (Canada), Alassane Dicko (Canada), Cindy Feng (Canada), Isabel Fortier (Canada), Damian Francis (USA), Elizabeth Ghogomu (Canada), Janet Hatcher-Roberts (Canada), Alison Krentel (Canada), Elizabeth Kristjansson (Canada), Laurenz Langer (South Africa), Daeria O. Lawson (Canada), Julian Little (Canada), Olivia Magwood (Canada), Michael Johnson Mahande (Tanzania), Lawrence Mbuagbaw (Canada), G.J. Melendez- Torres (UK), David Moher (Canada), Ainsley Moore (Canada), Loveline Lum Niba (Cameroon), Stuart G. Nicholls (Canada), Miriam Nkangu (Canada), Stephen G. Noorduyn (Canada), Ekwaro Obuku (Uganda), Patrick M. Okwen (Cameroon), Tomas Pantoja (Chile), Jennifer Petkovic (Canada), Mark Petticrew (UK), Kevin Pottie (Canada), Tamara Rader (Canada), Jacqueline Ramke (Aotearoa New Zealand), Alison Riddle (Canada), Larissa Shamseer (Canada), Hugh Sharma Waddington (UK), Peter Tanuseputro (Canada), Peter Tugwell (Canada), Erik Von Elm (Switzerland), Harry Wang (Canada), Xiaoqin Wang (Canada), George Wells (Canada), Charles S. Wiysonge (South Africa), Luke Wolfenden (Australia), Taryn Young (South Africa).

All the team members have consented to be listed in the manuscript.

Funding Statement

This work was supported by the Canadian Institutes of Health Research (CIHR), grant number 173269.

The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

[version 1; peer review: 1 approved

References

  • 1.Decade of Action - United Nations Sustainable Development: United Nations. Reference Source
  • 2. SDG 10: Health and reduced inequalities. World Health Organization Regional Office for Europe. Reference Source [Google Scholar]
  • 3. Whitehead M: The concepts and principles of equity and health. Int. J. Health Serv. 1992;22(3):429–445. 10.2190/986L-LHQ6-2VTE-YRRN [DOI] [PubMed] [Google Scholar]
  • 4. Braveman PA, Kumanyika S, Fielding J, et al. : Health disparities and health equity: the issue is justice. Am. J. Public Health. 2011;101 Suppl 1:S149–S155. 10.2105/AJPH.2010.300062 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5. Pottie K, Welch V, Morton R, et al. : GRADE equity guidelines 4: considering health equity in GRADE guideline development: evidence to decision process. J. Clin. Epidemiol. 2017;90:84–91. 10.1016/j.jclinepi.2017.08.001 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6. Welch V, Dewidar O, Tanjong Ghogomu E, et al. : How effects on health equity are assessed in systematic reviews of interventions. Cochrane Database Syst. Rev. 2022;2022(1):Mr000028. 10.1002/14651858.MR000028.pub3 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7. The WHO strategy on research for health. World Health Organization;2012. Reference Source [Google Scholar]
  • 8. Thirteenth General Programme of Work 2019-2023. World Health Organization. Reference Source [Google Scholar]
  • 9. Etienne C, Abbasi K, Cuervo LG: Research for health in the Americas. BMJ. 2018;362:k2944. 10.1136/bmj.k2944 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10. Promoting health equity, gender and ethnic equality, and human rights in COVID-19 responses: Key considerations. Pan American Health Organization;2020. Reference Source [Google Scholar]
  • 11. Brownson RC, Kumanyika SK, Kreuter MW, et al. : Implementation science should give higher priority to health equity. Implement. Sci. 2021;16(1):28. 10.1186/s13012-021-01097-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12. Welch VA, Norheim OF, Jull J, et al. : CONSORT-Equity 2017 extension and elaboration for better reporting of health equity in randomised trials. BMJ. 2017;359:j5085. 10.1136/bmj.j5085 [DOI] [PubMed] [Google Scholar]
  • 13. Welch V, Petticrew M, Tugwell P, et al. : PRISMA-Equity 2012 extension: reporting guidelines for systematic reviews with a focus on health equity. PLoS Med. 2012;9(10):e1001333. 10.1371/journal.pmed.1001333 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14. Welch VA, Akl EA, Guyatt G, et al. : GRADE equity guidelines 1: considering health equity in GRADE guideline development: introduction and rationale. J. Clin. Epidemiol. 2017;90:59–67. 10.1016/j.jclinepi.2017.01.014 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15. Marmot M, Bell R: The Sustainable Development Goals and Health Equity. Epidemiology. 2018;29(1):5–7. 10.1097/EDE.0000000000000773 [DOI] [PubMed] [Google Scholar]
  • 16. Antequera A, Lawson DO, Noorduyn SG, et al. : Improving Social Justice in COVID-19 Health Research: Interim Guidelines for Reporting Health Equity in Observational Studies. Int. J. Environ. Res. Public Health. 2021;18(17). 10.3390/ijerph18179357 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17. Reeves BC, Deeks JJ, Higgins JP, et al. : Chapter 24: Including non-randomized studies on intervention effects. Higgins JPT, Thomas J, Chandler J, et al., editors. Cochrane Handbook for Systematic Reviews of Interventions. Cochrane;2021. version 6.2 (updated February 2021). [Google Scholar]
  • 18. Grimes DA, Schulz KF: An overview of clinical research: the lay of the land. Lancet. 2002;359(9300):57–61. 10.1016/S0140-6736(02)07283-5 [DOI] [PubMed] [Google Scholar]
  • 19. Funai EF, Rosenbush EJ, Lee MJ, et al. : Distribution of study designs in four major US journals of obstetrics and gynecology. Gynecol. Obstet. Investig. 2001;51(1):8–11. 10.1159/000052882 [DOI] [PubMed] [Google Scholar]
  • 20. Scales CD, Norris RD, Peterson BL, et al. : Clinical research and statistical methods in the urology literature. J. Urol. 2005;174(4 Pt 1):1374–1379. 10.1097/01.ju.0000173640.91654.b5 [DOI] [PubMed] [Google Scholar]
  • 21. Elm E, Altman DG, Egger M, et al. : Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. BMJ. 2007;335(7624):806–808. 10.1136/bmj.39335.541782.AD [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22. Glover RE, Schalkwyk MCI, Akl EA, et al. : A framework for identifying and mitigating the equity harms of COVID-19 policy interventions. J. Clin. Epidemiol. 2020;128:35–48. 10.1016/j.jclinepi.2020.06.004 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23. Tenforde MW, Fisher KA, Patel MM: Identifying COVID-19 Risk Through Observational Studies to Inform Control Measures. JAMA. 2021;325(14):1464–1465. 10.1001/jama.2021.1995 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24. Guidance for implementing non pharmacological public health measures in populations in situations of vulnerability in the context of COVID-19. Pan American Health Organization;2020. Reference Source [Google Scholar]
  • 25. Sharp MK, Bertizzolo L, Rius R, et al. : Using the STROBE statement: survey findings emphasized the role of journals in enforcing reporting guidelines. J. Clin. Epidemiol. 2019;116:26–35. 10.1016/j.jclinepi.2019.07.019 [DOI] [PubMed] [Google Scholar]
  • 26. Dewidar O, Podinic I, Barbeau V, et al. : Integrating sex and gender in studies of cardiac resynchronization therapy: a systematic review. ESC Heart Fail. 2022;9(1):420–427. 10.1002/ehf2.13733 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27. Park H, Dewidar O, Tanjong-Ghogomu E, et al. : Reporting and analysis of Sex and Gender in Transitions of Care for Older Adults: A Methods Study. University of Ottawa. J. Med. 2022;11(2). 10.18192/uojm.v11i2.5945 [DOI] [Google Scholar]
  • 28. Jahn I, Börnhorst C, Günther F, et al. : Examples of sex/gender sensitivity in epidemiological research: results of an evaluation of original articles published in JECH 2006–2014. Health Res. Policy Syst. 2017;15(1):11. 10.1186/s12961-017-0174-z [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29. Simera I, Moher D, Hirst A, et al. : Transparent and accurate reporting increases reliability, utility, and impact of your research: reporting guidelines and the EQUATOR Network. BMC Med. 2010;8:24. 10.1186/1741-7015-8-24 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30. Jin Y, Sanger N, Shams I, et al. : Does the medical literature remain inadequately described despite having reporting guidelines for 21 years? - A systematic review of reviews: an update. J. Multidiscip. Healthc. 2018;11:495–510. 10.2147/JMDH.S155103 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31. Stevens A, Shamseer L, Weinstein E, et al. : Relation of completeness of reporting of health research to journals' endorsement of reporting guidelines: systematic review. BMJ. 2014;348:g3804. 10.1136/bmj.g3804 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32. Münter NH, Stevanovic A, Rossaint R, et al. : CONSORT item adherence in top ranked anaesthesiology journals in 2011: A retrospective analysis. Eur. J. Anaesthesiol. 2015;32(2):117–125. 10.1097/EJA.0000000000000176 [DOI] [PubMed] [Google Scholar]
  • 33. Turner L, Shamseer L, Altman DG, et al. : Does use of the CONSORT Statement impact the completeness of reporting of randomised controlled trials published in medical journals? A Cochrane review. Syst. Rev. 2012;1:60. 10.1186/2046-4053-1-60 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34. Chircop A, Bassett R, Taylor E: Evidence on how to practice intersectoral collaboration for health equity: a scoping review. Crit. Public Health. 2015;25(2):178–191. 10.1080/09581596.2014.887831 [DOI] [Google Scholar]
  • 35. Ciapponi A, Lewin S, Herrera CA, et al. : Delivery arrangements for health systems in low-income countries: an overview of systematic reviews. Cochrane Database Syst. Rev. 2017;2017. 10.1002/14651858.CD011083.pub2 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36. Pantoja T, Opiyo N, Lewin S, et al. : Implementation strategies for health systems in low-income countries: an overview of systematic reviews. Cochrane Database Syst. Rev. 2017;2017(9):Cd011086. 10.1002/14651858.CD011086.pub2 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37. Wiysonge CS, Paulsen E, Lewin S, et al. : Financial arrangements for health systems in low-income countries: an overview of systematic reviews. Cochrane Database Syst. Rev. 2017;2017(9):Cd011084. 10.1002/14651858.CD011084.pub2 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38. Gender-based analysis plus (GBA+). Government of Canada; Reference Source [Google Scholar]
  • 39. O'Neill J, Tabish H, Welch V, et al. : Applying an equity lens to interventions: using PROGRESS ensures consideration of socially stratifying factors to illuminate inequities in health. J. Clin. Epidemiol. 2014;67(1):56–64. 10.1016/j.jclinepi.2013.08.005 [DOI] [PubMed] [Google Scholar]
  • 40. Jull J, Whitehead M, Petticrew M, et al. : When is a randomised controlled trial health equity relevant? Development and validation of a conceptual framework. BMJ Open. 2017;7(9):e015815. 10.1136/bmjopen-2016-015815 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41. World Health Organization: European Observatory on Health Systems and Policies. Glossary. 2009. [Google Scholar]
  • 42. Benchimol EI, Smeeth L, Guttmann A, et al. : The REporting of studies Conducted using Observational Routinely-collected health Data (RECORD) statement. PLoS Med. 2015;12(10):e1001885. 10.1371/journal.pmed.1001885 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43. Orach CG: Health equity: challenges in low income countries. Afr. Health Sci. 2009;9 Suppl 2:S49–S51. [PMC free article] [PubMed] [Google Scholar]
  • 44. Freeman T, Gesesew HA, Bambra C, et al. : Why do some countries do better or worse in life expectancy relative to income? An analysis of Brazil, Ethiopia, and the United States of America. Int. J. Equity Health. 2020;19(1):202. 10.1186/s12939-020-01315-z [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 45. Kruk ME, Gage AD, Joseph NT, et al. : Mortality due to low-quality health systems in the universal health coverage era: a systematic analysis of amenable deaths in 137 countries. Lancet. 2018;392(10160):2203–2212. 10.1016/S0140-6736(18)31668-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 46. Bambra C, Riordan R, Ford J, et al. : The COVID-19 pandemic and health inequalities. J. Epidemiol. Community Health. 2020;74(11):964–968. 10.1136/jech-2020-214401 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 47. Wang F, Cao J, Yu Y, et al. : Epidemiological characteristics of patients with severe COVID-19 infection in Wuhan, China: evidence from a retrospective observational study. Int. J. Epidemiol. 2021;49(6):1940–1950. 10.1093/ije/dyaa180 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 48. Partnership A-S: Effectiveness of seasonal malaria chemoprevention at scale in west and central Africa: an observational study. Lancet. 2020;396(10265):1829–1840. 10.1016/S0140-6736(20)32227-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 49. Tromans S, Chester V, Harrison H, et al. : Patterns of use of secondary mental health services before and during COVID-19 lockdown: observational study. BJPsych Open. 2020;6(6):e117. 10.1192/bjo.2020.104 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 50. Barua R, Lang K: School Entry, Educational Attainment, and Quarter of Birth: A Cautionary Tale of a Local Average Treatment Effect. J. Hum. Cap. 2016;10(3):347–376. 10.1086/687599 [DOI] [Google Scholar]
  • 51. Baqui P, Bica I, Marra V, et al. : Ethnic and regional variations in hospital mortality from COVID-19 in Brazil: a cross-sectional observational study. Lancet Glob. Health. 2020;8(8):e1018–e1026. 10.1016/S2214-109X(20)30285-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 52. Fagbamigbe AF, Kandala NB, Uthman AO: Demystifying the factors associated with rural–urban gaps in severe acute malnutrition among under-five children in low- and middle-income countries: a decomposition analysis. Sci. Rep. 2020;10(1):11172. 10.1038/s41598-020-67570-w [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 53. Lazarus JV, Wyka K, Rauh L, et al. : Hesitant or Not? The Association of Age, Gender, and Education with Potential Acceptance of a COVID-19 Vaccine: A Country-level Analysis. J. Health Commun. 2020;25(10):799–807. 10.1080/10810730.2020.1868630 [DOI] [PubMed] [Google Scholar]
  • 54. Zakeri R, Bendayan R, Ashworth M, et al. : A case-control and cohort study to determine the relationship between ethnic background and severe COVID-19. EClinicalMedicine. 2020;28:100574. 10.1016/j.eclinm.2020.100574 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 55. Samuels EA, Orr L, White EB, et al. : Health Care Utilization Before and After the “Muslim Ban” Executive Order Among People Born in Muslim-Majority Countries and Living in the US. JAMA Netw. Open. 2021;4(7):e2118216-e. 10.1001/jamanetworkopen.2021.18216 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 56. Petkovic J, Jull J, Yoganathan M, et al. : Reporting of health equity considerations in cluster and individually randomized trials. Trials. 2020;21(1):308. 10.1186/s13063-020-4223-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 57. Page MJ, Shamseer L, Altman DG, et al. : Epidemiology and Reporting Characteristics of Systematic Reviews of Biomedical Research: A Cross-Sectional Study. PLoS Med. 2016;13(5):e1002028. 10.1371/journal.pmed.1002028 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 58. Taljaard M, McRae AD, Weijer C, et al. : Inadequate reporting of research ethics review and informed consent in cluster randomised trials: review of random sample of published trials. BMJ. 2011;342:d2496. 10.1136/bmj.d2496 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 59. Moher D, Tetzlaff J, Tricco AC, et al. : Epidemiology and reporting characteristics of systematic reviews. PLoS Med. 2007;4(3):e78. 10.1371/journal.pmed.0040078 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 60. Bramer WM, Jonge GB, Rethlefsen ML, et al. : A systematic approach to searching: an efficient and complete method to develop literature searches. J. Med. Libr. Assoc. 2018;106(4):531–541. 10.5195/jmla.2018.283 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 61. Prady SL, Uphoff EP, Power M, et al. : Development and validation of a search filter to identify equity-focused studies: reducing the number needed to screen. BMC Med. Res. Methodol. 2018;18(1):106. 10.1186/s12874-018-0567-x [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 62. Umaefulam V, Kleissen T, Barnabe C: The representation of Indigenous peoples in chronic disease clinical trials in Australia, Canada, New Zealand, and the United States. Clin. Trials. 2022;19(1):22–32. 10.1177/17407745211069153 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 63. Upadhyay N, Aparasu R, Rowan PJ, et al. : The association between geographic access to providers and the treatment quality of pediatric depression. J. Affect. Disord. 2019;253:162–170. 10.1016/j.jad.2019.04.091 [DOI] [PubMed] [Google Scholar]
  • 64. Study design search filters: BMJ Best Practice. Reference Source
  • 65. Search Filters for MEDLINE in Ovid Syntax and the PubMed translation. McMaster University. Reference Source [Google Scholar]
  • 66. DistillerSR: Version 2.35. Evidence Partners. 2022. Accessed May 25 2022. Reference Source
  • 67. Dewidar O, Welch V, Rizvi A: STROBE-Equity methods study. 2022, May 26. 10.17605/OSF.IO/CP3Z2 [DOI]
F1000Res. 2025 Sep 5. doi: 10.5256/f1000research.134140.r171061

Reviewer response for version 1

Mireya Martinez Garcia 1

I read with interest the paper entitled “Reporting of health equity considerations in equity-relevant observational studies: Protocol for a systematic assessment”, the idea is original and the study's findings will surely contribute to inform the development of an equity reporting guide.

  1. Introduction: The authors could mention some examples of the inequities highlighted by the observational studies during the COVID-19 pandemic, and their impact in informing public health responses.

    The authors mention that “Complex health systems questions and addressing social determinants of health may require multi-stakeholder and intersectoral collaboration and engagement, for which equity perspectives can serve as a linchpin” Could you be more specific and mention a few examples briefly?

    The authors mention that the “objective is to systematically evaluate how equity relevant observational studies describe characteristics of their samples, design features and analysis, and interpretation of their findings across PROGRESS-Plus” also in the “Figure 1. Study flow diagram” or in Table 2 are mentioned, but in the Methods/Eligibility criteria you decided not to use “Plus” criteria for eligibility since they require a greater deal of judgment regarding the contextual factors that may result in inequities … the authors could clarify it.

  2. Methods: In the Methods/Eligibility criteria section you mentioned “We will not include studies focused on Indigenous populations… it will require a different sampling approach and methodological assessment.” but in the Search strategy then you said “…and additional terms to increase retrieval of papers about indigenous populations were added”. The way of presenting the text is a bit confusing, although it is understood that a complementary study will be carried out in collaboration with the Indigenous Steering Committee. How did it affect the final count of the studies recovered by including those related to the indigenous population?

    Although the authors include the reference (Bramer WM, de Jonge GB, Rethlefsen ML, et al.), please briefly describe the method designed to optimize term selection mentioned in the Search strategy section.

    Why do the authors believe that using the MEDLINE database is sufficient for their search? beyond the high volume of results obtained in the preliminary search. What advantages or disadvantages does this recovery system have? 

    Please briefly describe the DistillerSR software and why that was your favorite for the screening step?

    The complete list of items regarding motivation behind studies evaluation is in an OSF repository. Could the authors give some examples or more details of them? to facilitate the reader a more fluent reading of the protocol and not have to consult supplementary material or the Extended data each time.

Thanks

Is the study design appropriate for the research question?

Yes

Is the rationale for, and objectives of, the study clearly described?

Yes

Are sufficient details of the methods provided to allow replication by others?

Yes

Are the datasets clearly presented in a useable and accessible format?

Yes

Reviewer Expertise:

Health Systems; Health Policy; Social Analysis of Healthcare; Socio-medical Research

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

F1000Res. 2023 Aug 17. doi: 10.5256/f1000research.134140.r183191

Reviewer response for version 1

Ihoghosa Iyamu 1

Thanks for the opportunity to review this study protocol. It focuses on an important topic of health equity analyses and reporting in scientifica articles and this is very important. There are however opportunities to provide additional detail that can improve the quality and replicability of the study.

My comments are outlined as major and minor comments below:

Major comments 

  1. There appears to be a struggle to balance feasibility with rigour and the idea of a comprehensive study. For example, while the authors note their intent to search only one database as a limitation, I wonder if it makes sense to have a more specific question that allows the team search more databases to ensure a less biased sampling frame for the study. The Cochrane group recommends searching Embase and Medline at least, in addition to other registries. https://training.cochrane.org/handbook/current/chapter-04

    If the questions are more specific, that might be achieved.

  2. More specific details are needed for the screening procedures. Are the authors screening all the 16,000 plus titles and abstracts returned in the search, is there some automation going into the screening process especially because the authors use DistillerSR? Further, some studies do stratified analyses of outcomes as sub-analyses but may not include this in the abstract. Are there chances that your current approach and framing of the questions may miss these sort of studies? What is the protocol to ensure that papers of interest are not inadvertently excluded due to the current strategy. On the random sampling, it would also be great to know if this is a sampling with replacement or not.

  3. The authors describe a follow-up survey as part of this study but only do so in passing. Questions about the survey include; What kinds of questions will you be asking? Is it a simple "did you engage with populations experiencing inequities?", How do you define engagement? What will be your threshold for saying engagement was done or not done? More information is required for clarity on this.

  4. One also wonders why only a 2 year window was chosen for the study. One wonders if the last 2 years appropriately capture scientific practices and protocols or if this period is greatly biased by COVID and not necessarily reflective of the spectrum of practices within the scientific community. For example, were engagements with historically marginalized communities not altered during the pandemic? How is that reflected in the studies included?

  5. One is not sure about the framework being adhered to in this study. For example, is there a PRISMA guideline or something similar that this largely follows?

Minor comments 

  1. The first instance of "equity-relevant" studies should be accompanied by a reference to the definition. This was only done later and could be confusing.

  2. What is the justification for 80% agreement? What would that amount to in terms of say a Cohen's Kappa or some other standardized form of agreement metrics?

  3. Authors state they will only use the PROGRESS framework, but in sections of the manuscript, they refer to PROGRESS-Plus

Is the study design appropriate for the research question?

Yes

Is the rationale for, and objectives of, the study clearly described?

Yes

Are sufficient details of the methods provided to allow replication by others?

Partly

Are the datasets clearly presented in a useable and accessible format?

Not applicable

Reviewer Expertise:

Health services research; Digital health equity; sexual health

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

F1000Res. 2022 Dec 16. doi: 10.5256/f1000research.134140.r151179

Reviewer response for version 1

Khic-Houy Prang 1

Thank you for the opportunity to read your study protocol which aims to develop a health equity reporting guideline for observational studies. I commend the authors on undertaking an ambitious review across diverse settings and population groups to identify health equity relevant items. Further clarifications about the health equity definition, inclusion and exclusion criteria of the review are required.

  1. I agree that health equity is important to capture in observational studies, however health equity is rather a difficult construct to measure that embody many social determinants and individual factors that influence access to care and ultimately health outcomes. I am unclear how you will differentiate between health equity studies and non-health equity studies. Most (if not all) observational studies will generally include a demographic table of their sample, as such based on your inclusion criteria then all studies would be considered a health equity-relevant study? I am not sure whether sub-group analysis will be enough to identify health equity studies as these studies may reflect health disparity? How will you differentiate between health equity and health disparity studies? Perhaps it might help to narrow the focus to intersectionality analysis?

  2. The definition of population at risk of experiencing inequities is broad and includes all type of population. It would be difficult to identify as this is likely to be dependent on the disease and the context, and the comparison group. Will all the LMICs studies automatically be considered a population at risk because of the setting? What about specific population groups in HICs? You gave some examples of at-risk groups such as migrants and asylum seekers. Will it be worth considering at the outset who these groups are and narrow the scope of the review?

  3. Can you clarify why you chose PROGRESS Plus over other health equity frameworks to guide the review?

  4. The inclusion criteria are observational studies and observational econometric studies. What about quasi-experimental and natural experiment studies given the focus on COVID studies?

  5. The proposed review includes a systematic search approach but does not include any quality assessment or risk of bias. Could you please clarify what type of review this is?

  6. Is there a chance that you may missed studies using only one search engine? What about the grey literature? Will limiting the time period to two years (2020-2022) skew your search to predominantly COVID studies given the prioritisation of COVID studies published during this period over non-COVID studies?

  7. Given that your research question is rather broad and does not focus on a specific population groups, interventions or health outcomes, the search strategy is rather complex. It seems to be based on the factors in the PROGRESS framework, I wonder if it would be useful to limit the search to observational study designs and heath equity-related terms, although noting that studies without using health equity-related terms will unlikely be picked up in the search. Given that the focus of the study is on health equity specifically, then perhaps it may be better to focus on such studies and describe what these studies have reported? Best practice for systematic and scoping reviews involves two investigators screening the full text articles.

  8. There are almost 40 authors listed in the paper. This seems excessive for a protocol paper, with most of the authors’ contribution limited to reviewing and editing. Do all members in each of the committees warrants authorship?

  9. Minor edits: A typo in figure 1 ‘experiencing inequities’. In tables 2 and 3, it would be helpful perhaps to not use ‘COVID +’ or ‘COVID –‘ as the headers as it appeared to denote COVID studies with positive or negative results, perhaps COVID-related studies and non-COVID related studies may be more suitable? Reference required on page 7 regarding the survey - ‘ […] that engagement with populations experiencing inequities inform better reporting on health equity’.

Is the study design appropriate for the research question?

Partly

Is the rationale for, and objectives of, the study clearly described?

Yes

Are sufficient details of the methods provided to allow replication by others?

Partly

Are the datasets clearly presented in a useable and accessible format?

Not applicable

Reviewer Expertise:

Health services and systems research, evaluation and implementation science, mixed-methods research

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

Associated Data

    This section collects any data citations, data availability statements, or supplementary materials included in this article.

    Data Availability Statement

    Underlying data

    No data are associated with this protocol.

    Extended data

    Open Science Framework: STROBE-equity reporting guidelines, https://osf.io/cp3z2. 67

    This project includes the following extended data:

    • •

      Search strategy.docx

    • •

      Extraction items mapped to the STROBE-Equity candidate items.docx

    Data are available under the terms of the Creative Commons Attribution 4.0 International license (CC-BY 4.0).

    Articles from F1000Research are provided here courtesy of F1000 Research Ltd

    RESOURCES