Operational research capacity building in Asia: innovations, successes and challenges of a training course

Abstract

A structured training course on operational research (OR) based on the model created by the International Union Against Tuberculosis and Lung Disease and Médecins Sans Frontières was conducted in the South Asian region in 2012. Many innovations were introduced into the administration, structure and content of the course. Of 12 participants, 11 successfully completed all pre-defined milestones. Several challenges were identified. The main challenges included shortage of time, especially for data analysis and interpretation, and insufficient numbers of experienced facilitators. Appropriate modifications have been made to the structure and processes of the next course scheduled for 2013. We describe these modifications and the innovations, successes and challenges of this model of training.

Operational research (OR) is increasingly being recognised as a science that is important for reducing the gap between knowledge and implementation, optimising the performance of health programmes and achieving improved health outcomes.¹ Despite recognition of the necessity of OR by global health organisations, donor agencies and national health programmes, the amount of research in global health that is actually conducted and published in both resource-limited and industrialised countries remains limited.^2–4 This may partly be due to the limited capacity of health professionals to conduct and publish OR.

Drawing on lessons from several OR capacity-building initiatives in the past,^5–7 the model conceived by the International Union Against Tuberculosis and Lung Disease (The Union) and Médecins Sans Frontières (MSF) has been implemented since 2009, with excellent results—trained participants in low- and middle-income countries who have become independent facilitators and mentors in subsequent courses, presentations at international conferences, publications in peer-reviewed journals, and, more importantly, impact on national and local policy and practice.⁸ In 2012, we replicated this training model for health professionals in the South Asian region, with several innovative modifications drawn from lessons learnt from previous courses. In the spirit of applying OR thinking to OR capacity building, we describe the innovations, successes and challenges of the first South Asian OR course.⁹

METHODS

The Union/MSF training model has been described in detail elsewhere.^2,8 Briefly, it is an output-oriented mentorship programme with three 5-day modules spread over a period of 9–12 months: Module 1 is on ‘research protocol development’, Module 2 on ‘data entry and analysis’ and Module 3 on ‘scientific paper writing’. Participants who submitted a scientific manuscript for publication to a peer-reviewed journal within 4 weeks of completion of Module 3 were considered to have successfully completed the course.

RESULTS

In this first Asian course, held in Nepal in 2012, 12 participants, mostly health professionals (physicians, programme managers, paramedical workers and data analysts) working in health programmes from India, Nepal, Bhutan, Bangladesh, Pakistan, Sri Lanka, Indonesia, Timor Leste and Cambodia, selected through a competitive process, were required to attend the three modules and complete interim milestones linked to each module to remain in the course. The number of facilitators varied from module to module—six for Modules 1 and 3 and two for Module 2.

Eleven of the 12 participants successfully completed all the milestones and submitted 12 scientific manuscripts (one participant completed two projects) on topics ranging from tuberculosis, human immunodeficiency virus and tobacco control to health system financing for publication in international peer-reviewed journals. Of these, six papers were accepted for publication within 3 months of submission; how many more will be published and what their impact on policy and practice will be is being tracked. One participant met the milestones for Modules 1 and 2 successfully, but was not able to complete the project in time; the long delays obtaining local ethics approval demotivated the participant from continuing on the project.

Three junior facilitators from this course acted as independent facilitators, and two participants from this course participated as junior facilitators in the next course conducted in February 2013.

Several innovations introduced into the structure and process of the OR course are summarised in Table 1. We also faced several challenges during the course, which are listed in Table 2.

TABLE 1.

Innovations introduced into the Union-MSF operational research training model in South Asia, 2012

Innovation	Description	Advantage
Adjoining the first two modules	Modules 1 and 2 were conducted back-to-back, with a weekend break in between	This reduced overall costs by 25–30%
Course venue	Nepal was chosen as the course venue, as it provided visas on arrival for all participants and facilitators	This simplified travel logistics and saved time and inconvenience for participants and organisers
Introducing new sessions into the curriculum	While the core modules remained the same, some new sessions were introduced: a session on ‘how to back up data’ using a combination of open-access tools such as Dropbox (http://www.dropbox.com) and FreeCommander (http://www.freecommander.com) was taught in Module 2. A session on organising and managing references was taught in Module 3 using another open access tool, Mendeley (http://www.mendeley.com)	This met some of the expressed needs of the participants in the described areas
Organisation of course materials	In addition to a hard-back course folder, all participants were provided with a CD-ROM containing all the course materials organised like a website, which was then shared via the online file-sharing service Dropbox	This facilitated user-friendly access for participants and facilitators (often from different countries) to course material before and during the module
Introducing a milestone for Module 2	For the first time, participants were required to submit the following as outputs of Module 2: data collection plan, electronic data capture formats in EpiData (EpiData Association, Odense, Denmark, http://www.epidata.dk) and a plan for data analysis including dummy tables and figures	This helped in emphasising the value of quality assurance in data capture and analysis in operational research, one of the key objectives of the course
Peer support strategy in Module 2	The 12 participants were divided into six pairs and each member of the pair was to support the other during the course. Based on the experience in Module 1, a fast learner was paired with a slow learner.	This helped in coping with the shortage of facilitators
Mentoring junior facilitators	The facilitators were paired to provide mentorship to four participants in such a way that one senior facilitator* was paired with a junior facilitator; all the junior facilitators had been participants in previous courses	This helped to train the junior facilitator to become an independent trainer in the future
Using SurveyMonkey	SurveyMonkey (http://www.surveymonkey.com), an online survey entry and analysis software, was used for participant evaluation at the end of the course	This saved time in analysing evaluation and providing feedback

^*One who is experienced in both conducting and publishing papers and has excellent writing skills.

The Union = International Union Against Tuberculosis and Lung Disease; MSF = Médecins Sans Frontières.

TABLE 2.

Challenges faced in implementing the Union-MSF operational research training model in South Asia, 2012

• Burden of work for facilitators: having only two facilitators in Module 2 created a large workload. Similarly, having six facilitators in Module 1 and 3 was also insufficient, especially in Module 3, as each facilitator’s group had four participants. With each group writing four papers in 5 days, this number was too small

• Data analysis and interpretation: most participants complained that not enough time was given in class to this important subject

• Better organisation of course material: several participants requested that the course material be organised as a book instead of a course folder, to increase its shelf life

• Delays in local ethics approval: some participants had to wait for a long time (range 1–6 months) before obtaining ethics approval from their national/local ethics committees. Given the tight timelines of the course, this left them with very little time for the actual conduct of the project

• Scheduling of the sessions on literature search and managing references: several participants commented on the need to bring this session upstream to Module 1 rather than Module 3. This has now been done

• Publication in open-access journals for enhanced dissemination in resource-limited settings: this adds an additional burden to the course budget (range 500–1500 USD per article)

• Portal of call for applications: we received only 15 applications to our initial call for applications, which was sent through the national tuberculosis programme managers of countries in the region. We then informed WHO country focal points and contacts of non-governmental organisations, and extended the deadline for application by a week. The number of applications rose to 50 within a week. Drawing on these lessons, in 2013 we advertised on http://www.devnetjobs.org/ and received 128 applications

The Union = International Union Against Tuberculosis and Lung Disease; MSF = Médecins Sans Frontières; WHO = World Health Organization.

DISCUSSION

The success of the Union-MSF courses was replicated in Asia, with most participants completing their milestones. There were several innovations with this course. The positioning of Modules 1 and 2 back-to-back helped in two ways—one technical and the other logistical. First, the 2–3 month gap between Module 1 and Module 2 in the original Union/MSF model meant that some participants started data collection and capture before Module 2 with several errors. This often meant that they had to repeat their data capture efforts once they learnt more appropriate methods of data management in Module 2. Having Modules 1 and 2 back-to-back meant that the participants were equipped right from the start, not only with a clear study protocol, but also with the necessary quality-assured tools for data capture and analysis. Second, the positioning of the two modules also meant that only two trips to the course location were required instead of three, thus saving on resources and travel costs, with an overall reduction of 25–30%. However, the change posed challenges for female participants, especially those with young children, who had to be away from their families for 2 weeks.

The peer support strategy was widely appreciated and had multiple advantages, promoting the concept of peer support and learning, and providing opportunities for fast learners to undergo training as facilitators. Fast learners were often quick to catch on and were thus able to engage in the support and teaching of their peers. This took some of the pressure off the facilitators, and was one of the reasons why Module 2 could be managed with just two facilitators.

Organising course content in the form of a website and sharing it via Dropbox, a file-sharing service, promoted easy access for facilitators (often from different countries) prior to the module for their review and inputs. Its offline access feature is advantageous, obviating the need for continuous internet connectivity.¹⁰

There were also challenges. The main challenge was shortage of time, especially for the data analysis component and for facilitators experienced in conducting and publishing OR. To address these challenges, the following modifications were made to the course structure in 2013: Module 2 was extended by a day and Module 3 by 2 days, with the additional time to be used for teaching participants about data analysis and supporting individual projects with data analysis support. We also increased the number of facilitators to eight for each module—divided into four groups of two (one junior and one senior facilitator)—with each group mentoring three participants. From the feedback of the participants, Module 2 course materials will be provided as a spiral-bound book instead of a course folder and this new system evaluated. If participants find it more user-friendly than the folders, this will be adopted for the other modules as well. Paying up-front costs for open access publication (US$500–1500) had also been a problem; these needed to be covered in the course budget.

While this description is limited by the low numbers, we believe that the experience will be of value to people involved in or interested in conducting similar courses across the globe. Overall, the South Asian OR course successfully achieved its goals and provided several lessons for further optimisation. We hope that the participants inspired by this course will continue their work and become leaders in OR in their own networks and countries.¹¹