Investigating Time Lags and Attribution in the Translation of Cancer Research

Short abstract

RAND Europe and HERG were asked by UK medical research funders to investigate returns to public/charitable investment in cancer research. This article details the five case studies conducted as part of this work and observations emerging across them.

Abstract

In 2012, RAND Europe and the Health Economics Research Group (Brunel University) were commissioned by the Wellcome Trust, Cancer Research UK, the National Institute for Health Research and the Academy of Medical Science to conduct a study of the returns to the public/charitable investment in cancer-related research. This study built on previous work published in the 2008 “What's it worth?” report that estimated the economic returns to medical research in terms of spillover benefits and health gain. The 2008 study was extensively quoted and cited as a clear justification for the economic importance of medical research and appears to have played a role in achieving the protection of the medical science budget in the recent public expenditure cuts.

This cancer study used a similar approach to that used in the previous study, but with some methodological developments. One of the methodological developments was the inclusion of case studies to examine the validity and variability of the estimates on elapsed time between funding and health gains, and the amount of health gains that can be attributed to UK research. This study provides the full text of the five case studies conducted as well as some discussion of observations emerging across the case study set.

Previous work has shown that medical research offers a good return on investment.

Buxton et al. (2008) estimated the economic returns to medical research in the cardiovascular and mental health fields in terms of spillover benefits and health gain, finding that every pound invested in research led to a stream of benefits equivalent to around 39 pence each year in perpetuity. The work was extensively quoted and cited as a clear justification for the economic importance of medical research and appears to have played a role in achieving the protection of the medical science budget in the recent public expenditure cuts.

This work is part of our recent study which replicated that work in cancer research, with some methodological developments including case studies.

Buxton et al. (2008) recommended that the approach should be repeated in other clinical areas to both test and refine methods and to see whether rates of return appeared to be similar. We have now completed such a study in the field of cancer research, again finding that research is economically beneficial (Glover et al., 2014). To do this, we estimated: (1) public and charitable expenditure on cancer-related research in the UK; (2) the net monetary benefit (NMB), ie the health benefit valued in monetary terms minus the cost of delivering that benefit, for a prioritised list of interventions; (3) the proportion of NMB attributable to UK research; (4) the elapsed time between research funding and health gain; and (5) the internal rate of return from cancer-related research investments on health benefits. One of the main methodological developments in this new study was the inclusion of case studies to investigate two of these elements: the elapsed time between research funding and health gains, and the proportion of health gains that can be attributed to UK publicly funded research.

This study contains the five detailed case studies conducted and outlines observations emerging from them.

The case studies conducted were:

• The use of guaiac-based faecal occult blood test in bowel cancer screening

• Cancer service configuration

• Smoking reduction

• The use of tamoxifen in the treatment of breast cancer

• Total mesorectal excision (TME) in rectal cancer.

The aim of the case studies was to examine the estimates of the elapsed time between funding and health gains and the fraction of health gains that can be attributed to UK research.

These estimates form part of the model used to evaluate the economic returns and were calculated for the numerical estimation based on the analysis of citations on guidelines. The case studies were used to inform the sensitivity analyses conducted around those estimates. This study also includes some observations emerging across the case study set both on these two key issues, but also on wider issues relating to the translation of research into practice.

The case studies were selected to cover a variety of interventions.

Case studies were purposively selected in discussion with the Steering Group (made up of representatives of the funders) to cover a range of different types of intervention (ie pharmaceutical, screening, surgical, prevention and service organisation) and cancer sites with the highest burden of disease (i.e., lung, breast, bowel, prostate).

Trying to understand and estimate elapsed time between funding and health impact is difficult.

Estimates from guideline analysis uses the time difference between the date of publication of the guideline and the average date of publication of cited references and adds to this estimates for the time between awarding of funding and publication, and the time between recommendation and use. By contrast, the case studies describe the process of research translation from a qualitative point of view and as such the focus moves away from an average time lag figure for the total research spend and towards the overall elapsed time from the original research breakthrough to some measure of when the health gains might start accruing. This can provide a detailed picture of the body of research related to the intervention and the timescales involved, supporting the guidelines analysis, but means that the quantitative estimates are not directly comparable.

There is a range of factors which contribute to the elapsed time.

The case studies identify a range of factors influencing elapsed time, from the length of time needed to conduct randomised controlled trials (RCTs) in the field of bowel cancer screening, to the training needed for surgeons to implement TME. However, some themes do emerge across several case studies which are of interest.

Having networks in place can support rapid research translation.

Policy uptake after the publication of research evidence in particular appears to have been facilitated by the availability of networks in two of the case studies (bowel cancer screening and service configuration). Having the structures and networks in place means that the transfer of knowledge can happen more quickly once research evidence becomes available.

A “champion” can play an important role in the translation of research and in reducing elapsed time.

Several case studies (tamoxifen, TME and service configuration) illustrate the role of a “champion” in the translation of research and in reducing time lags, which has been observed previously (Wooding et al. 2011, 2013). In all cases, researchers played a wider role in helping to facilitate the translation of research in their particular areas, by engaging with policy and decision makers or facilitating training.

Analysing case studies allows us to consider the wider range of inputs to attribution more carefully than analysis of guidelines.

The analysis of guidelines to estimate the proportion of health gain from research in cancer that can be attributed to UK public research funding has some limitations. It assumes each publication cited has equal weight; neglects any work which is not cited on guidelines (eg early basic research); does not account for the fact that some of the evidence for a particular intervention might have been unnecessary; and only provides an estimate for part of the translation process with estimates for the time from funding to publication and from inclusion on guidelines to implementation needing to be supplemented. Analysing case studies addressed some of these issues by allowing us to consider the wider range of inputs more carefully.

Assigning relative importance to different pieces of research and different stages of the research process is challenging.

Although case studies offer some advantages, assigning a figure to attribution remains difficult since it is challenging to assign relative importance to different pieces of research evidence. For example, how important is the initial discovery of a drug relative to an important clinical trial of that drug, or a metaanalysis of evidence across a number of trials? Deciding the relative weighting between these different countries contributing to multi-country studies is also not necessarily straightforward.

Analysis of the case studies conducted suggests that the estimate of 17 per cent for the proportion of health gains attributable to UK publicly funded research obtained from the guidelines analysis may be a low estimate.

Three out of the five case studies appear to show that a higher proportion of the research contributing to the new intervention or approach was publicly funded and conducted in the UK, with the other two not providing any clear conclusions in either direction. However, this is a very small sample and as such no generalisable conclusions about the overall level of attribution across the research field can be drawn.

The role that research evidence plays varies between the case studies.

The role of evidence, and the importance of different types of evidence differed between case studies. For the service configuration and bowel cancer screening case studies, local research which could take into account the specific context was important, perhaps due to the nature of these interventions. In the case of bowel cancer screening, having the right kind of evidence available, through the economic evaluation “built into” one particular trial may have facilitated the speedy policy uptake. By contrast, in the smoking case study, the extent to which research evidence was an important driver of policy actions at all is not so clear cut, with campaigning and public opinion playing an important role in policy change.

A number of different actors can play an important role in the translation of research into practice.

The role of a “champion” was illustrated in two case studies as described above, but other important actors in research translation were identified. The TME case study illustrates the role of practitioners, with surgeons playing an important role in uptake, both in terms of their willingness to engage with the new approach, and the training needed for them to use the new approach. Policy makers can also play an important role, through their level of connection with relevant researchers as described above, but also potentially through their willingness to act early, potentially before evidence is conclusive, to facilitate rapid implementation as described in the bowel cancer screening study.

Estimating the economic impact of research is challenging.

As well as some of the issues in estimating elapsed time and attribution as described above, the service configuration case study illustrates an additional challenge in the estimation of the economic impact of research. Estimates of the health benefits of a particular intervention (in terms of quality-adjusted life year (QALY) gains) used in the economic model (in Buxton et al., 2008 and Glover et al., 2014) are largely extrapolated from trial data, mainly derived from UK relevant health technology assessments. This service configuration case study shows that the gains actually realised in the NHS may differ from those observed in trials – in this particular instance suggesting a situation where trial gains may underestimate what is now being achieved, though the reverse may be true in other cases.