Does research make a difference to public health? Time for scientific journals to cross the Rubicon

The International Union Against Tuberculosis and Lung Disease (The Union), Médecins sans Frontières (MSF) and the Special Programme for Research and Training in Tropical Diseases (TDR) hosted at the World Health Organization, have joined forces under a programme called the Structured Operational Research and Training Initiative (SORT IT) to improve health care delivery and public health through operational research. This is done through integrated operational research and training programmes, in which participants from low- and middle-income countries are enrolled and taken through a research project from protocol development to implementation, data collection and analysis to the writing and submission of a paper to a peer-reviewed journal.¹ Between April 2009 and January 2014, we have undertaken 18 programmes (of which 14 are completed), enrolling a total of 212 participants.

We carefully monitor outputs from each SORT IT programme. Two important outputs, which are relatively easy to monitor, are the number and percentage of enrolled participants who complete the programme and the number of scientific papers that are published in peer-reviewed journals within 18 months of submission. However, an additional, vital output that needs to be monitored is whether any action has been taken in response to the research findings and the subsequent impact of research on policy and practice. Thus, one of our SORT IT targets is to record whether this has been documented in over 80% of projects within 18 months of study completion.

It is frequently stated that it takes 17 years for research evidence to reach clinical practice,² although for operational research with a focus on health service delivery we would expect a much shorter time lag. We have recently completed an assessment of the impact on policy and practice for our first eight programmes through a questionnaire survey. The amount of work that went into obtaining the data was considerable, with repeated e-mails sent to course alumni over a period of several months, and personal calls to chase up non-responders. Without the help of a dedicated person allocated to this task, it is unlikely we would have succeeded in our task.

We therefore propose an innovative role for a journal, namely that of tracking and publishing the impact of its articles. This fulfils a mandate of operational research, and if, for example, this was taken on by Public Health Action (PHA), it would place this journal at the forefront of the discipline. Papers in PHA are published quarterly. An automated e-mail could go out to the corresponding author of a paper published in the Journal 18 months after publication. The e-mail could have a website link that takes the author to a site in which he/she can answer whether the paper has had any influence on policy and practice (Yes/No). If the answer is no, then reasons can be mentioned such as the paper being a review (not applicable), a descriptive methodology or a specific local context. If the answer is yes, then reasons can be articulated for claiming that the research has had such an effect. Supporting material could be added. Authors could be given four weeks to respond and their collective responses then published in either the following quarterly issue of the Journal or annually, in a section entitled ‘Research to Policy and Practice’. The papers would be classified into No response; Not applicable; No effect on policy and practice; Positive effect on policy and practice. In this way, authors could communicate to a wide audience whether their research has made a difference, and this could be the start of a journal-based observatory that systematically tracks this important operational research outcome.

In the January 2014 issues of the Lancet (January 11–17 and 18–24), there was an excellent five-paper series on increasing the value and reducing the waste of research. An editorial summarised some of the key problems and made useful recommendations.³ A consistent theme in the series was that research is considered a waste if it does not impact on health care, but there were no practical suggestions or recommendations to monitor whether research has made a difference to the public's health and whether, in particular, it has contributed to the optimal use of available resources or an improvement in the effectiveness of health care or health system interventions.

We would like to advocate that PHA takes the lead in this valuable and exciting new venture, and we hope that other journals would then follow its example. In a December 2013 editorial, PHA was judged to be meeting its goals and targets and wanted to increasingly be seen as the home of operational research.⁴ We agree with this assessment and vision. However, we believe that the Journal must push the frontiers of operational research and become a vehicle for analysing whether what we do is truly providing health solutions for the poor. This means going beyond the published paper and becoming a forum for assessing whether the research has made a difference. It is a bold step for a journal to aspire to doing more than publish papers, but operational research is about more than just publishing – it is a step to better health care.

Operational research is also about ‘learning while doing’. The proposal we make is in its concept phase, and it may require a task force to work out the finer details of how to translate the idea into practice and how to define impact. The monitoring process is likely to evolve over time, but this could be the first step in a journey to formally and systematically evaluate whether the research is delivering the goods.⁵ Can PHA cross this Rubicon?