Promoting operational research through fellowships: a case study from the South-East Asia Union Office

Abstract

In 2009, the International Union Against Tuberculosis and Lung Disease (The Union) and Médecins Sans Frontières (MSF) jointly developed a new paradigm for operational research (OR) capacity building and started a new process of appointing and supporting OR fellows in the field. This case study describes 1) the appointment of two OR fellows in The Union South-East Asia Office (USEA), New Delhi, India; 2) how this led to the development of an OR unit in that organisation; 3) achievements over the 5-year period from June 2009 to June 2014; and 4) challenges and lessons learnt. In June 2009, the first OR fellow in India was appointed on a full-time basis and the second was appointed in February 2012—both had limited previous experience in OR. From 2009 to 2014, annual research output and capacity building initiatives rose exponentially, and included 1) facilitation at 61 OR training courses/modules; 2) publication of 96 papers, several of which had a lasting impact on national policy and practice; 3) providing technical assistance in promoting OR; 4) building the capacity of medical college professionals in data management; 5) support to programme staff for disseminating their research findings; 6) reviewing 28 scientific papers for national or international peer-reviewed journals; and 7) developing 45 scientific abstracts for presentation at national and international conferences. The reasons for this success are highlighted along with ongoing challenges. This experience from India provides good evidence for promoting similar models elsewhere.

From a public health perspective, operational research (OR) can be defined as research into strategies, interventions, tools or knowledge that can enhance the quality, coverage, effectiveness or performance of the health system or disease programme in which the research is being conducted.¹ It can be viewed as a spectrum of activities that encompasses reviews of data already collected in patient registers, treatment cards, patient files or electronic data sets, evaluations of operational practices or assessments of the implementation of new strategies and technologies.

OR is often observational in nature, and can involve descriptive or cross-sectional studies, retrospective or prospective cohort studies and sometimes case-control studies. Recent guidelines for the reporting of observational studies (the STROBE [Strengthening the Reporting of Observational Studies in Epidemiology] statement) provide a logical structured roadmap for this type of research, thereby improving its scientific credibility.² In addition, pragmatic, cluster-randomised trial designs and qualitative research methods are also increasingly being viewed as being within the ambit of OR. OR needs to be conducted within a sound ethics framework that includes the principles of informed consent and data confidentiality, and in all cases study results should be fed back to the local communities in an accessible and understandable manner.³

The key role of OR in improving health programme performance is well-recognised, and the subject is consequently being strongly promoted by donors and technical agencies. For example, The Global Fund Against AIDS, Tuberculosis and Malaria (The Global Fund, Geneva, Switzerland) recommends the allocation of up to 10% of country grants towards monitoring and evaluation, including OR. As this provision is not explicit—it is hidden under ‘monitoring and evaluation’—potential users are often unaware of it and do not make use of this excellent opportunity for programme strengthening. OR implementation is weak in many of the low- and middle-income countries that are most in need of it, and lack of OR capacity is one of the main reasons behind this.

THE UNION CENTRE FOR OPERATIONAL RESEARCH

OR has always been a priority for the International Union Against Tuberculosis and Lung Disease (The Union, Paris, France) due to the visionary leadership of great public health leaders such as Annik Rouillon, Karel Styblo and Donald Enarson, who served The Union for many years. The initial efforts of OR capacity building included the development of a guide,⁴ and a training programme that started in 1997 with the support of the United States Center for Disease Control and Prevention’s Division of TB Elimination.⁵ The vision at the time was to identify and train participants from different countries in quality-assured data capture and analysis and research protocol development and encourage them to conduct multicentric collaborative OR projects. There were several challenges, including high attrition among the participants and little output in terms of publications,^6,7 and the initiative could not be sustained, mainly due to lack of continued funding. Despite these limitations, the initiative sowed the first seeds of OR in the minds of a number of people who went on to become leaders of international organisations, and it marks The Union’s first steps in OR capacity building.

On 1 January 2009, The Union received financial support from Bloomberg Philanthropies to establish the Centre for Operational Research and Strategic Information. One of the objectives was the promotion of OR and was supported by two principal goals: 1) to establish a new paradigm for OR capacity building, and 2) to appoint and support OR fellows in the field.

Operational research capacity building

In March 2009, The Union and Médecins Sans Frontières (MSF) Luxembourg met in Paris to develop a new course for integrated OR and training that not only teaches the principles of the ‘what, why and how of OR’, but also incorporates the development, implementation and writing up of a research project as an integral part of the course.⁸ Training is combined with ‘doing’, akin to an apprenticeship. The success of a participant is judged on whether or not a research project has been designed and completed, with a paper submitted to a peer-reviewed journal within the stated time frame of the course. The two organisations also track whether these papers have been published, whether the research has had any impact on policy and practice,^9–11 and whether the skills imparted to the participants and their organisations are used after the course to further develop, implement, write up and publish other relevant studies.¹²

After running courses together for 3 years, The Union and MSF joined forces with the Special Programme for Research and Training in Tropical Diseases (TDR), which is based at the World Health Organization (WHO), Geneva, Switzerland. In July 2012, the three organisations developed a formal blueprint for training public health programme staff under the Structured Operational Research Training Initiative (SORT IT), which has been described elsewhere.¹³

Operational research fellows

The Union instituted a new cadre of staff known as the ‘OR fellow’, whose task is to strengthen OR capacity and implementation in the field (Table 1). Appointed using strict selection criteria, fellows work within disease control programmes or supportive non-governmental organisations in their countries. They work full-time or part-time for The Union and are given support and time to carry out relevant OR. Fellows must successfully complete one of the OR courses within their first year of appointment; and they are expected to initiate, complete and publish their own OR as well as drive country-based OR. They are on 1- or 2-year performance-based contracts, with one of the key milestones being the submission of two papers to peer-reviewed journals by the end of a 12-month period; failure to achieve this results in termination of the contract. Fellows start in junior fellowships, and after 2 years progress to senior fellowships. After 4 years, with at least eight papers submitted, fellows may be considered for PhD programmes that allow submission of published papers to count towards the final degree. Following this successful model, MSF also appointed OR fellows under the same conditions as in The Union.

TABLE 1.

Terms of reference for operational research fellows

PURPOSE OF THE CASE STUDY

The main purpose of the present manuscript is to describe 1) how the appointment of two OR fellows in the Union South-East Asia (USEA) office, New Delhi, India, led to the development of an OR unit in that organisation; 2) the achievements and their reasons over the 5-year period from June 2009 to June 2014; and 3) challenges and lessons learned.

DEVELOPING THE OPERATIONAL RESEARCH UNIT IN INDIA

OR as a strategic direction was always part of the vision of the leadership of the USEA Regional Office and the office of the Executive Director of The Union, Paris, France, and several efforts had been made since 2008 to procure funding for OR capacity building. In June 2009, the first OR fellow (SS) in India was appointed on a full-time basis. A decision had been made several months earlier between the regional director of USEA office and the Director of OR in Paris (ADH) to appoint an OR fellow in India. SS was working as a WHO-RNTCP (Revised National Tuberculosis Control Programme) Medical Consultant in India’s RNTCP, and was keen to learn about OR. The national programme manager of India’s RNTCP and the WHO Medical Officer for Tuberculosis displayed exemplary leadership by supporting this appointment, despite human resource constraints at the Central TB Division of Ministry of Health and Family Welfare of Government of India (New Delhi, India). SS submitted his curriculum vitae along with references, was interviewed and appointed, with the generic terms of reference underpinning the job (Table 1). His previous experience with OR was limited. He had published two papers in peer-reviewed journals, one in 2005 on dengue fever as third author and one in 2008 on initial default in tuberculosis (TB) patients in India in which he was the eleventh author.^14,15 He had also had one successful abstract submission for the Annual Conference of the Indian Academy of Preventive and Social Medicine in 2005. He had never acted as a peer reviewer for a scientific paper and had never been a facilitator on an OR training course. The financial support for his position was provided through the Bloomberg Philanthropies. This appointment marked the formal beginning of OR initiatives at the USEA office.

In September 2011, the Department for International Development (DFID), London, UK, also started to provide support to enhance OR capacity building at The Union. One of their areas of support was for a regional OR course for South-East Asia along with an additional OR fellow to support SS in India. At the time, there was a WHO-RNTCP Medical Consultant (AMVK) attending the OR capacity building course in Paris. He had proven his competence and enthusiasm for OR during the course, and following the same procedures as with SS, he was formally appointed in February 2012, with the same generic terms of reference underpinning his tasks. He too had had no previous experience with OR. He had published one paper as sole author on confidence intervals,¹⁶ and had presented a paper at the 22^nd Annual Conference of the Indian Society for Medical Statistics on the prevention of parent-to-child transmission of HIV/AIDS (human immunodeficiency virus/acquired immune-deficiency syndrome). He also had never acted as a peer reviewer for a scientific paper and had never been a facilitator on an OR training course.

At the time AMVK was undertaking his OR training, two senior staff officers of the USEA office underwent the same OR training in Paris. This helped in the creation of an enabling environment for OR in the USEA office. A research associate position was established and filled to support a tuberculosis knowledge, attitude and practice survey funded by the Global Fund in 2012. A small team was thus established in the office. These factors ensured that OR featured as one of the six strategic goals of the ‘Vision 2020’ internal document which details the plans, goals and strategic directions for the USEA office for the period 2012–2020.

ACHIEVEMENTS AND REASONS FOR SUCCESS

Successful completion of operational research courses

SS completed the first ever OR capacity building course held at The Union Headquarters in Paris with two OR projects on paediatric TB and recurrent TB in India, both of which were published.^17,18 AMVK similarly completed the second OR capacity building course in Paris with one project on antiretroviral therapy in HIV-infected TB patients in India.¹⁹

Facilitation and mentorship on further operational research courses

Both OR fellows went on to facilitate in modules of other OR courses, with SS starting his facilitation career in June 2010 and AMVK in February 2012. The details of the modules at which each fellow facilitated each year are shown in Table 2. As of June 2014, SS had facilitated at 21 modules and AMVK at 40.

TABLE 2.

OR courses and other training workshops at which the OR fellows facilitated, 2009–2014

In addition to the OR modules and courses, the OR fellows were invited as resource persons to teach at other capacity building workshops conducted by the WHO, the RNTCP, the National AIDS Control Programme (NACP) and medical colleges. In May 2012, AMVK and SS were invited by the WHO and the RNTCP to build data management capacity for about 65 WHO-RNTCP consultants working in India. Following this training, a data management module using the EpiData open access software (EpiData Association, Odense, Denmark) was developed by AMVK for the RNTCP to enable the structured capture of data collected during programme evaluations conducted across the country. To institutionalise this mechanism, the Central TB Division of the Ministry of Health of the Government of India issued official directives to train all data entry operators (more than 650) in using EpiData and how to use the specific data management module to capture data from the internal evaluations of the RNTCP. The information thus captured is now compiled nationally and used for programme reviews, decision making and research. Several of these WHO-RNTCP consultants, who were instrumental in DOTS expansion in India, became participants in OR courses and developed as OR trainers and leaders.²⁰

Published papers

The number and details of papers published each year by the two OR fellows are shown in Appendix 1 and Figure 1. It should be noted that several of these research projects were mentored by the two OR fellows together. The number of publications increased exponentially between 2009 and 2013, indicating the value of a critical mass of people involved full time in OR. Altogether, over the 5-year period, the two OR fellows were co-authors on 75 published research papers and 21 opinion and review papers, with a total of 96 published papers. Of these, 71 (74%) were published in journals offering immediate and free open access to their readership, with most of the remainder being published in journals offering free open access after a period of time. In the 6 years before the OR fellows joined the USEA office, there had been one paper published by the office.

FIGURE 1.

Annual number of publications co-authored by the staff of the USEA office and contribution of OR fellows, 2003–2013. OR = operational research: USEA = Union South-East Asia.

Impact of operational research on policy and practice

Several of the OR studies conducted or mentored by the OR fellows have contributed to changes in national policy and practice. Some examples of this impact are highlighted in Table 3.

TABLE 3.

Examples of OR projects co-ordinated and mentored by OR fellows that contributed to changes in policy and practice, 2009–2013

Technical assistance in promoting operational research

The two OR fellows became known as leaders in OR by partners and other stakeholders, and were invited to be a part of national-level technical committees that assist and recommend the national programme. SS participated as a member of the National Standing Committee on Operational Research for the RNTCP, while AMVK participated as a member of the National Technical Working Group on TB-HIV for the NACP and as a member of the state-level Operational Research Committee of Karnataka State, India. In addition, they have provided technical assistance to states in the country for conducting OR, analysing data and publishing papers. In addition, several research projects (including nationally representative knowledge, attitudes and practice surveys on TB and projects related to tobacco control) initiated by other departments of the USEA office were supported technically by the two OR fellows from design to analysis to publication.

Building capacity of medical college professionals in data management

A collaborative framework was created between the Karnataka State RNTCP, medical colleges and The Union in 2013 to build the capacity of public health professionals in medical colleges in conducting OR. One of the first initiatives of this collaboration was to engage with public health professionals in medical colleges and academic institutions in India and conduct a 2-day course on Epi-Data for the residents and faculty of public health departments of medical colleges in and around Bangalore. This workshop, funded by Eli Lilly (Indianapolis, IN, USA), was organised by the USEA office (led by AMVK) in collaboration with the Department of Community Medicine at the Bangalore Medical College and Research Institute (BMCRI, Bangalore, India) and the Karnataka State Tuberculosis Cell (Bangalore, India) of the RNTCP of the Government of India. A second initiative was replicated in Puducherry, India, in collaboration with the Jawaharlal Nehru Institute of Postgraduate Medical Education and Research (JIPMER), one of the premier medical institutions in India catering for the medical college faculty and scientists of the Indian Council of Medical Research from Puducherry. The feedback from participants suggests that they have begun to use EpiData in their research work and have included it in their teaching curriculum for postgraduate and undergraduate medical students. The sessions in these courses were video-captured, edited and have been posted on YouTube (http://www.youtube.com/watch?v=1SoxNpj-Ncw) to be used as self-learning resource materials by interested public health professionals worldwide. These online learning materials were also shared with the participants of the online virtual OR training course initiated by The Union in North America.

Support to programme staff for the dissemination of research findings

The OR unit at the USEA office has supported several programme staff officers who are working with the RNTCP and India’s National HIV Programme to draft and submit scientific abstracts to national and international conferences. Several abstracts have been accepted for oral or poster presentations. To facilitate dissemination of research findings, a national dissemination workshop was conducted in 2013 by the USEA office in collaboration with the RNTCP; this was attended by representatives of all stakeholders working for TB control in India. Such workshops provide a great opportunity for researchers to share their findings directly with RNTCP managers and advocate for policy change.

Scientific papers reviewed for national or international peer-reviewed journals

The number of papers that each OR fellow reviewed each year is shown in Figure 2. Although the OR fellows were invited to review many papers, they could not accept all the offers given their busy schedules. Altogether, 28 papers were reviewed.

FIGURE 2.

Annual number of papers reviewed by the OR fellows of The Union South-East Asia Office, 2009–2013. OR = operational research.

Scientific abstracts presented at national and international conferences

The number of abstracts accepted for presentation with OR fellows as co-authors at conferences each year is shown in Figure 3. Altogether, 45 conference abstracts were written.

FIGURE 3.

Annual number of conference abstracts co-authored by OR fellows of The Union South-East Asia Office, 2009–2013. OR = operational research.

Creation of a national pool of resource persons

In the initial courses co-ordinated by the OR fellows, support was provided by external facilitators. Once the fellows became independent facilitators, they started identifying outstanding participants in their OR courses and provided them with an opportunity to co-facilitate in future courses. This helped in grooming young, committed participants as future facilitators. As a result of this facilitator-grooming initiative, there is now a pool of independent, national facilitators in India who can be drawn into facilitating national and regional courses without the need for external (out-of-country) expertise. Several of these facilitators are now able to adapt and replicate the SORT IT model in their own networks. Notable efforts include those made by the Postgraduate Institute of Medical Education and Research, Chandigarh, India, to advance locally relevant research in tobacco control and that of the National Institute for Research in Tuberculosis in Madurai, India, to build capacity of medical college professionals in TB research. It augurs well for the future sustainability of this model of capacity building.

LESSONS AND CHALLENGES

This initiative has several important lessons and take-home messages. First, the fellows had both worked in national disease control programmes before taking up their OR fellowship positions, and understood the importance of identifying programmatically relevant OR questions and ensuring completion of the studies that were initiated.

Second, this experience, combined with their OR training, which focused on study completion and movement of research findings to policy and practice, has ensured that programme managers are engaged at all stages of the OR: protocol development, data collection, data analysis, data interpretation, paper writing and dissemination. Engagement with programme managers has had a cascading effect in promoting and developing an OR culture within national programmes.

Third, an enabling environment for OR has developed at the USEA office, with the regional director strongly supportive of this form of research. In addition to the OR fellows, three senior staff members of the office and one consultant from the Central TB Division of the Ministry of Health and Family Welfare, Government of India, were trained in OR at Paris, one of whom went on to become the national professional officer for TB control at the WHO India office. This has enabled strong links to be developed with key partners, the WHO Country Office and the Indian RNTCP.

Fourth, these initiatives have led to the creation of a national pool of resource persons capable of independently facilitating OR courses without the need for external international facilitators. The role of the WHO-RNTCP consultant network and the support of the WHO Country Office lead for TB in contributing to this resource pool deserve special mention.

Finally, there has been strong, ongoing and regular mentorship support from the Director of the Centre for Operational Research, Paris, which has included frequent visits to the USEA office. This has provided additional motivation and enthusiasm for the fellows to perform to their full potential.

Despite these lessons, there are ongoing challenges. First, even in a country such as India, which has ample resources of its own, external financial support is needed in the short and medium term to launch and sustain this initiative. Donor money is needed to support both OR fellows and capacity building courses until such time as the national government appreciates the benefits and value of such domestic support. There has, and continues to be, great support from several donor institutions such as Bloomberg Philanthropies, the Global Fund, the United States Aid for International Development (Washngton DC, USA) and the UK DFID for OR capacity building, without which these initiatives could not have been sustained. The OR fellows have not been successful in tapping into the OR funds earmarked within the RNTCP, as these are cumbersome and time-consuming to access. This needs re-thinking, as utilising national funds earmarked for OR may help in undertaking programmatically relevant and challenging OR studies that require prospective study designs.

Second, building OR capacity and conducting OR is an ongoing process, with success and sustainability depending on repeating the process over and over again, often due to staff turnover. It is vital for OR fellows to also identify promising individuals at OR courses and engage them rapidly as facilitators for future courses so that momentum can be maintained.

Third, it is vital that enthusiastic and driven OR fellows are supported in their career paths so that they can attend skill building courses, develop their own capacity and submit their work for Masters and PhD degrees. In this regard, adequate and appropriate remuneration is needed to retain such people within their institutions. The 2013 World Health Report explicitly acknowledges the need to strengthen research endeavour, not only in academic centres but also in public health programmes, close to the supply of and demand for health services.²¹ Important donors, such as DFID, recognise this need and are prepared to give the necessary support, provided that value for money is achieved.

CONCLUSION

This case study testifies to the huge research output that resulted from placing one, and then two, OR fellows in the USEA office. Of particular note, in the 6 years before the fellows joined there was one published paper from the office, while in the first 5 years since the fellows joined, the cumulative number of published papers reached nearly one hundred. Research publications are an objective way to measure research output, an indicator that the research has been properly designed, implemented, analysed and written up to a high enough standard to get through peer review and be published in a respectable international journal.²² Published papers are also a crucial way of disseminating research findings to a wide audience, especially if open-access journals are used as the media for publication.

Before joining as OR fellows, the two individuals had little engagement in OR. They learnt and subsequently taught their craft as they developed, with a number of factors facilitating this process, such as their own enthusiasm and passion for the subject, a supportive and enabling environment and strong mentorship from the leadership. Selecting the right individuals for the job is an essential determinant of success, but the performance-related contracts that The Union offers to all OR fellows allow a way out if a mistake is made in selection and individuals fail to deliver.

Currently, one OR fellow is pursuing a PhD training programme at McGill University, Montreal, QC, Canada, and during this time can only spend 4 months of the year in India. The USEA office therefore has one full-time resident OR fellow, and although this has not led to a decrease in the volume of publications in the first 6 months of 2014 (24 published papers by 30 June 2014), it nevertheless, puts undue strain on the one individual. In an enabling environment in a busy country office, we believe that there should be a minimum of two full-time OR fellows at any one time, and if resources permit this number should be increased to three. We are convinced that in this situation that the whole is greater than the sum of its parts, and that total research output will be maximised as good OR fellows will feed off and stimulate each other.

We believe that our experience in India provides good evidence for promoting similar models elsewhere. We need to select highly motivated individuals who, provided they are well supported, mentored and given performance-related contracts, can facilitate high-quality research that impacts not only locally but also at the national and international levels. Furthermore, it is essential to have a critical mass of OR fellows, with two at any one time being a minimum. We hope that public health institutions and donors will read and learn from this story.