Lies, damned lies and epidemiology: why global health needs good epidemiological practice guidelines

Summary box.

• Epidemiology, like any other discipline, is liable to malpractice. Questionable research practices have no place in global health as they steer research in the wrong direction, misguide public policy and undermine society’s trust.

• Guidelines can address part of the problem by facilitating structured and transparent processes.

• Key features of global health epidemiology revolve around the transnational and interdisciplinary nature of global health, its focus on equity, large-scale use and sustainability.

• Guidelines for good epidemiological practice (GEP) in global health are not available, but a number of documents have laid the foundations for their development. Stakeholders involved in the commissioning, conduct, appraisal and publication of global health research should endorse a common set of GEP guidelines.

“Figures often beguile me, particularly when I have the arranging of them myself.”

Mark Twain, Chapters from my Autobiography (1906)

Introduction

Epidemiology is the cornerstone of global health. It shapes policy decisions and evidence-based practice by identifying disease risk factors and preventive healthcare targets. Most epidemiological findings are genuine and make an important contribution to global health, but some findings are obtained from ill-designed, poorly implemented, inappropriately analysed or selectively reported studies. These practices in the grey zone between deliberate misconduct (which includes fabrication, falsification and plagiarism) and ideal scientific behaviour are commonly referred to as ‘questionable research practices’.¹

The only systematic review available on scientific research misconduct to date² pooled 21 surveys mostly in biomedical, medical and clinical sciences from the UK, USA and Australia. The review suggested that 2%–14% of scientists may have fabricated or falsified data, with nearly three-quarters admitting other questionable research practices. The few data available from low-income and middle-income countries suggest that research misconduct is just as common in those countries.³

There is little research into the quality of global health research, but what it reveals is very sobering. A recent review of 121 randomised control trials performed in sub-Saharan Africa suggests that many studies suffer from inadequate methods and incomplete reporting.⁴ For example, a description of adequate randomisation sequence generation was present in only 62% of reports, intervention allocation concealment in 39% and adjustments for incomplete outcome data in 64%. Interventions were completely reported in only 60% of studies. The most worrying data come from two recent Nigerian studies, which suggest staggering levels of scientific misconduct: nearly 70% of interviewed researchers admitted to some form of personal scientific misconduct,⁵ while 96% believed that one or more forms of scientific misconduct had occurred in their workplace.⁶

Scientific misconduct and questionable practices have no place in global health as they steer research in the wrong direction, misguide public policy and undermine society’s trust. Studies can and should be conducted according to the highest attainable professional standards. While the reproducibility crisis is complex and multifaceted, guidelines can address part of the problem by facilitating structured and transparent processes. In this article we argue that global health would greatly benefit from adopting its own set of widely endorsed good epidemiological practice (GEP) guidelines.

Existing GEP guidelines

Over the past 15 years a number of documents aimed at improving epidemiological practice have been published. These can be divided into three categories: (1) ethical guidelines for research provided by international organisations⁷; (2) guidelines for GEP laid out by international⁸ and national epidemiological associations^9–11; and (3) academic guidelines for reporting epidemiological studies.¹²

Existing guidelines have helped to mainstream quality assurance in epidemiology. However, there are limits to their applicability to global health research: (1) international guidelines provide only general standards (which means their use as practical guidance is limited); (2) national guidelines lack international legitimacy; and (3) epidemiology encompasses a wide range of specialties (each with their own idiosyncrasies and corresponding implications for data collection, processing and use). In our opinion, the combination of these three factors justifies the development of internationally endorsed GEP guidelines specifically for global health.

GEP for global health

Koplan et al ¹³ describe global health as ‘an area for study, research, and practice that places a priority on improving health and achieving equity in health for all people worldwide’. They further state that ‘Global health emphasises transnational health issues, determinants, and solutions; involves many disciplines within and beyond the health sciences and promotes interdisciplinary collaboration; and is a synthesis of population based prevention with individual-level clinical care’. This definition is useful to reflect on how epidemiological practice in global health differs from applications in other health areas.

• Stakeholder involvement: The priority of global health is to improve health and achieve equity for all. This ambitious goal requires concerted efforts across a number of stakeholders, including national governments, technical agencies, donors, practitioners, civil society and so on. For research findings to translate into action, it is crucial that all relevant stakeholders are intimately involved in all phases of research. At all levels, findings need to provide answers to knowledge or action gaps and need to be translated into user-specific formats for effective dissemination. Epidemiologists involved in global health therefore need to ensure that they work closely with stakeholders throughout the entire duration of a study, including study design, data analysis and interpretation of findings.

• Empowering incountry researchers: The transnational nature of global health means that most research is conducted through international partnerships. As much as possible, incountry researchers should have a leading role in the planning and conduct of research in their country. This is key to ensure findings are contextualised, relevant and locally owned. However, it is important to recognise that research collaborations involving two research groups that contribute equally to funding and have equal scientific capacity are rare. Epidemiologists working in global health need to ensure that they respond to gaps in their partners’ epidemiological skills by facilitating appropriate training and mentoring.

• Ethical review: Most ethical issues specifically pertaining to global health epidemiology have been addressed in the literature on the ethics of implementation research (eg, obtaining consent for cluster randomised trials, contextual equipoise, lack of study controls, unanticipated outcomes).¹⁴ In addition, global health epidemiologists face a number of dilemmas in the ethical review processes. First, the blurred boundary between research and public health practice means that it is not always clear-cut whether a study needs ethical review. Second, transnational research implies that at least dual (if not multiple) review is required, at the site of the organisation leading the study (‘sponsor’) as well as locally.⁷ Difficulties can arise when ethical review is not possible at one site (for lack of local capacity or willingness to review a study conducted in a foreign country) or if reviews conflict with each other.

• Emergency epidemiology: Responses to epidemic emergencies (eg, recent Ebola and Zika outbreaks) are clearly global. In these situations, data are crucial to make informed decisions, but they are also hard to obtain and analyse. Epidemiologists in emergency settings usually work against extreme time pressure, raising a number of methodological and ethical issues. The 2015 West African Ebola epidemic, which pushed researchers beyond conventional models of drug development, provides a case in point.

• Equity and gender: With its focus on ‘achieving equity for all people’, global health acknowledges that social determinants such as gender, socioeconomic status, education, age, ethnicity or religion have an impact on health. Gender typically intersects with the other determinants of health. For example, poor women may be significantly more disadvantaged compared with better-off women, while better-off women may enjoy similar status compared with their male counterparts. It is crucial to reflect on the effect of gender when conducting household surveys, since gender, unlike other determinants, varies within a household. Men and women in the same household may provide different answers to the same questions; they may report differently in the presence or absence of their partner; and their answers may vary according to the interviewers’ gender. Sampling, data collection tools and field procedures need to take these types of issues into account to ensure valid data. Similarly, data analysis and interpretation of results should be gender-aware to meaningfully contribute to understanding equity in the study context.

• Mixed-methods research: Global health promotes interdisciplinary collaboration, including epidemiology alongside, for example, anthropology, sociology, political sciences, economics and mathematical modelling. While some of these methods rely on qualitative research methods, others are more quantitative in nature. Mixed-methods research combines the breadth and generalisability of quantitative methods with the depth and validity of qualitative research. It is a very powerful approach that can enable to both quantify the magnitude of changes and associations (‘what’), as well as to understand the mechanisms and contributing factors (‘how’ and ‘why’). But to maximise the success of a mixed-methods approach, study plans need to include regular moments of reflection with peers across all involved disciplines, throughout all study phases (especially in design, analysis and interpretation).

• Use of existing sources of data: Global health research is often concerned with large-scale use and sustainability. Secondary analysis of national data is therefore an important source of information, with data typically extracted from health records or nationally representative surveys. Epidemiologists need to consider two issues when using these data. First, conditions for use of data provided by programme managers and ministries should be fairly negotiated and clearly documented. Second, data quality needs to systematically be appraised for completeness, coherence and credibility.

• Public data sharing: The transnational nature of global health has an important implication on study completion. Funding bodies and publishers are increasingly encouraging public data sharing to maximise the return of investment on research, to increase transparency and accountability, to reduce the cost of duplicating data collection, and to promote potential new data uses. However, in international research collaborations, data sharing can turn into an unfair one-way process which provides valuable data for scientists with high analytical capacities in developed countries who may not have contributed to study design and data collection. Thus epidemiological data sharing in global health research needs to be considered within the frame of levelled partnerships, where training and infrastructure development in low-income and middle-income countries are assigned high priority.

In our experience, these features are not adequately reflected in existing GEP principles. We believe that they should be, for GEP guidelines to be applicable and useful to global health research. The recently developed KIT Royal Tropical Institute internal GEP guidelines are an example of how this can be done. They can be found here along with details of the AGREE II (Appraisal of Guidelines for Research and Evaluation) methodology we followed for their development.¹⁵

Conclusions

Widely endorsed guidelines for GEP in global health are not available. Yet a number of documents have laid the foundations for their development. We invite all stakeholders involved in the commissioning, conduct, appraisal and publication of global health research to make use of this existing body of work to develop and endorse a common set of GEP guidelines. While GEP guidelines will not by themselves guard global health from questionable research practices, they are certainly part of a concerted effort to ensure transparency.