Building the capacity of public health programmes to become data rich, information rich and action rich

Abstract

Good quality, timely data are the cornerstone of health systems, but in many countries these data are not used for evidence-informed decision making and/or for improving public health. The SORT IT (Structured Operational Research and Training Initiative) model has, over 8 years, trained health workers in low- and middle-income countries to use data to answer important public health questions by taking research projects through to completion and publication in national or international journals. The D2P (data to policy) training initiative is relatively new, and it teaches health workers how to apply ‘decision analysis’ and develop policy briefs for policy makers: this includes description of a problem and the available evidence, quantitative comparisons of policy options that take into account predicted health and economic impacts, and political and feasibility assessments. Policies adopted from evidence-based information generated through the SORT IT and D2P approaches can be evaluated to assess their impact, and the cycle repeated to identify and resolve new public health problems. Ministries of Health could benefit from this twin-training approach to make themselves ‘data rich, information rich and action rich’, and thereby use routinely collected data in a synergistic manner to improve public health policy making and health care delivery.

Good quality, timely data form the foundation of an effective health system.¹ In an ideal world, these data should be defined, collected, reviewed, cleaned, analysed, interpreted and utilised to support public health policy and strategic planning, programme implementation and advocacy.² Unfortunately, in most countries around the world the reality is rather different. Mounds of data are routinely collected from all levels of the health sector on activities such as daily out-patient attendances, antenatal care visits, in-patient discharges, births and deaths, disease-specific patient screening, diagnostic services, vaccinations, tobacco use—the list goes on. When gathered in the form of paper-based aggregate reports, these are usually submitted up the hierarchical ladder from health centres to district hospitals to provincial centres, and eventually come to rest in the Ministry of Health. Alternatively, the Ministry may receive information from national statistics offices or non-governmental survey organisations. Regardless of how the data are collected and transmitted, these are often stored away as reports in shelves or in databases at various levels of the health system, and are rarely used thereafter. The full potential to utilise these data for evidence-informed decision making and for improving public health is thus rarely achieved.

Operational research (OR) is a discipline that encourages the use of routine data to enhance the quality, coverage, effectiveness and efficiency of health systems, health services or disease control programmes.³ The capacity for undertaking this type of research and then using the research evidence to change policy and practice is poor in many low- and middle-income countries. The purpose of this perspective paper is to present two training approaches that in sequence can enable good OR to be conducted and then used for policy change.

In 2009, SORT IT (Structured Operational Research and Training Initiative) courses were developed with the purpose of teaching participants how to conduct and publish OR and consider how it may be used to influence policy and practice.⁴ To date, these courses have been used to train over 550 individuals from 87 predominantly low- and middle-income countries (LMICs). Briefly, the SORT IT courses consist of three 1-week modules over 9 months, with clearly defined milestones and outputs, and strong face-to-face and remote mentorship.⁴ In modules 1 and 2, participants develop ethically sound research protocols and electronic data collection tools, which are focused on programme- or health system-related challenges using already collected secondary data. The protocols are implemented in the field, and the data are then analysed and the scientific papers are written in module 3. After agreement by all co-authors, and within 4 weeks of module 3, these papers are submitted to peer-reviewed national or international open access journals, with the findings and policy implications disseminated to a national/international audience.⁴

During one of our visits to Fiji in 2011 to set up SORT IT courses in the South Pacific, the Minister of Health commented that his country was ‘data-rich, but information-poor’, which is probably true for most LMICs. He wanted to transform his Ministry into one that was both ‘data-rich and information-rich’. We explained to him that embracing the SORT IT model would enable his country to achieve this aim, and several of the published OR studies using routine data collected during the first SORT IT course conducted in Fiji did in fact identify problems such as access to cervical cancer services and correctable deficiencies in the management of pregnant women, which was valuable information for the Ministry of Health.^5,6

The SORT IT Model (Figure) works by training participants to use routinely collected programme data to investigate and find solutions to overcome constraints in health service delivery. The ‘rich’ data, i.e., voluminous and raw, is thus transformed into ‘rich’ information, i.e., analysed and distilled. To maintain quality control standards, and to ensure that the information is widely disseminated, SORT IT aims to publish the work in peer-reviewed international or national scientific journals and/or present the information at scientific conferences. However, for OR to achieve its purpose, the journey must not stop there.

FIGURE.

The operational research and policy development processes of SORT IT and D2P. D2P = data to policy; SORT IT = Structured Operational Research and Training Initiative.

The ultimate goal of OR is to use this information to foster evidence-informed decision making and influence policy and practice, be it at the local, national or international level, so that health care delivery and disease control programmes can be improved.⁷ However, decision or policy makers often do not have the skills or the technical background to understand the scientific language presented in a research article, and there may be limited national capacity to perform policy-relevant data analysis.⁸ Other important barriers that can prevent the translation of research findings into policy include poor packaging of research results, absence of personal contact, lack of timeliness or relevance of the research and poor communication.^8,9

The techniques of policy-relevant data analysis and evidence synthesis include elements such as decision analysis, health impact assessment and cost-effectiveness analysis.^10,11 Ministry of Health epidemiologists, monitoring and evaluation officers and programme and planning officers typically do not receive training in these methods, and the absence of this technical capacity is a major cause of the D2P gap, even when OR capacity exists.

While the ultimate goal of SORT IT is to use research findings to influence policy and practice, and SORT IT participants are enabled to do this, the D2P training course recently developed by Vital Strategies in collaboration with the US Centers for Disease Control and Prevention (CDC), as part of the Bloomberg Philanthropies Data for Health Initiative, is specifically designed to bridge the gap between data and policy at government level (Figure). The D2P course, which has been developed in line with the SORT IT principles of intensive mentorship and tangible outputs, trains participants in policy-relevant methods, stakeholder analysis, feasibility assessments and communication skills. The course is hosted and guided by Ministries of Health, who select the participants and determine policy questions. Participants include government epidemiology and surveillance officers who wish to develop advanced analytic skills in policy development and decision analysis, and policy and planning staff who want to learn quantitative skills to enhance their current approaches to developing policy recommendations. Participants ideally work in teams of two or three on a shared policy topic.

The outputs of the D2P course are 1) a policy brief, which includes a description of the problem under consideration, quantitative comparisons of various policy options, a political and feasibility assessment, and specific policy recommendations; and 2) communication materials, which include a PowerPoint presentation of the brief and an ‘elevator’ speech about the policy issue.

Just as the peer-reviewed publication is not the end of the SORT IT journey, the policy brief is not the end of the D2P course. The evidence and the policy brief must next be used to change minds of policy makers and influence better practice on the ground, which is the ‘action-rich’ part of the equation. A key step is effective dissemination, which includes audience-tailored messages, use of the media (especially social media), implementation circulars, rapid advice, updated guidelines, revised screening/diagnostic/treatment algorithms, new training materials or new monitoring tools. Another important component is policy advocacy—to internal Ministry of Health decision makers, other government agency stakeholders (often including the finance ministry and chief minister's office), legislators, affected private sector entities, advocates and the public.

Ultimately, the impact of these changes needs to be monitored by assessing whether programme outcomes have improved and whether morbidity or mortality have been reduced. Our group has several examples of how OR has been used to change policy and practice, improve programme performance and reduce morbidity and mortality in Africa and Asia.^12–14 If there has been change or, conversely, if there is no change or improvement, then we need to find out ‘how’ and ‘why’ this has happened, and this gives birth to the next cycle of OR (Figure).

It is essential that this journey from data collection to impact be properly monitored and assessed, following the sound adage, ‘what gets measured gets done’. Published studies through SORT IT would provide evidence to D2P. The D2P would then use both published evidence and unpublished data in a decision-analysis framework to prepare policy briefs. This approach of combining SORT IT and D2P thus maximises the potential opportunity to use existing data—both published and unpublished—for evidence-informed decision making in public health.

Ministers of Health in the future should demand that OR and decision analysis are used to make their Ministries ‘data-rich, information-rich and action-rich’. Only then will this mound of health data and the discipline of OR realise their full potential. In parallel, advocacy and educational efforts also need to be made to support Ministers of Health to understand the importance of this research evidence and ensure that it is collected and used effectively to improve their health services and health systems.