It is not just about money

Abstract

Many scientists are concerned that academic research has become too conservative. It is not a lack of money but a confluence of factors that prevent support for innovative, long‐term research projects.

For many years, scientists have been concerned about and increasingly dismayed with the state of research funding and its stifling effect on innovation, despite periodic interventions and criticisms by senior researchers. Many scientists worry that major funding agencies in the USA and Europe have become more risk averse at the expense of bold innovative ideas and research projects with the potential for ground‐breaking science. Most of the frustration is directed at research assessment with its reliance on impact factor (IF) and high‐profile journals that has a direct effect on both funding and career progression. Then, there is the ever‐increasing bureaucracy for grant applications and at universities, sapping away time and energy from productive research and creative thinking. The overall effect is that researchers, particularly in the life sciences, increasingly embark on shorter‐term, safer projects with more immediate quantifiable returns instead of high‐risk, long‐term research. Although there has been some progress in some areas, there is a general sense among the broad research community, particularly in the biosciences, of banging heads against brick walls when trying to convince decision makers of the need for radical reform of how science is funded, managed, and communicated.

Peer review at the pillory

“I have been struggling for over 30 years to gain recognition of what is perhaps the major problem facing science”, said Don Braben, Honorary Professor in the Office of the Vice‐Provost for Research at University College London and well‐known for his advocacy of academic freedom and blue skies research. Braben brought his campaign to a head in 2014 with an open letter published in the UK's Daily Telegraph newspaper signed by more than 30 leading scientists, including four Nobel Laureates 1. The authors identified the peer review system for assessing grants and publication as a major obstacle to fundamental research and argued that “sustained open‐ended enquiries in controversial or unfashionable fields are virtually forbidden today and science is in serious danger of stagnating”.

… researchers, particularly in the life sciences, increasingly embark on shorter‐term, safer projects with short‐term quantifiable returns instead of high‐risk, long‐term research.

Some of the letter's signatories, including Braben himself, concede that the peer review system worked all right for standard research projects but tends to deter blue skies research and prevents unexpected turns and twists during projects. “Most, if not all, the funding agencies in the world have accepted the fact that the only way of judging the excellence of a research proposal is by using peer review to assess it”, Braben commented. “At the margin where great discoveries are made peer review fails. The great discoveries of the 20^th century, made by Planck, Einstein, Avery, Mitchell, etc., would not initially have survived consensus opinions. Young researchers are particularly disadvantaged”.

The problem, according to Peter Lawrence, developmental biologist at the University of Cambridge, UK, started around 20 years ago at Harvard Business School when it developed measures for the quantitative evaluation of projects. This is not a criticism of these statistical methods themselves, which Harvard still teaches today as a basis for solving business and project management problems, but their indiscriminate application to science. “Modern science, particularly biomedicine, is being damaged by attempts to measure the quantity and quality of research”, Lawrence wrote as early as 2007 in Current Biology 2. “Scientists are ranked according to these measures, a ranking that impacts on funding of grants, competition for posts and promotion. The measures seemed at first rather harmless but, like cuckoos in a nest, they have grown into monsters that threaten science itself. Already, they have produced an ‘audit society’ in which scientists aim, and indeed are forced, to put meeting the measures above trying to understand nature and disease”. Evaluation by IFs has already led to a scientist's “primary aim being downgraded from doing science to producing papers and contriving to get them into the ‘best’ journals they can”, Lawrence added. “Now there is a new trend: the idea is to rank scientists by the numbers of citations their papers receive. Consequently, I predict that citation‐fishing and citation‐bartering will become major pursuits” 2.

The great discoveries of the 20^th century, made by Planck, Einstein, Avery, Mitchell, etc., would not initially have survived consensus opinions.

Science versus bureaucracy

The problem is further compounded by the trend toward shorter‐term tenure, which is causing young scientists in particular to become preoccupied with obtaining future posts or even trading research for more secure administrative positions. Lawrence notes that in the USA, where the problem has become particularly acute, “non‐classroom costs have ballooned, administrative payrolls being a prime example. The number of employees hired by colleges and universities to manage or administer people, programs and regulations increased 50% faster than the number of instructors between 2001 and 2011” 3.

Modern science, particularly biomedicine, is being damaged by attempts to measure the quantity and quality of research.

This trend has been taking place for at least 40 years. According to data from the US National Center for Educational Statistics, public and private institutions in the USA spent US$20.7 billion on teaching during the 1980–1981 academic year, 41% of their total expenses, and US$13 billion on academic support, student services, and institutional support, which amounts to 23% of expenses. By the 2014–2015 school year, instructional costs were US$148 billion (29%), while schools’ administrative expenses had almost caught up with US$122.3 billion (24%) 4.

Bloated bureaucracy is not confined to US universities but happens across Europe as well. Figures published by the UK's Higher Education Statistics Agency indicated that central university administration costs have been rising well ahead of inflation, up 7% at £2.7 billion for the year 2016/2017 (https://www.hesa.ac.uk/news/07-03-2018/income-and-expenditure-he-providers-201617).

While spending for administration has been ballooning, the number of long‐term, tenured positions in academia has not kept up in lieu of limited tenure or contracts. Lawrence fears that this trend toward insecure tenure, along with increasing administration, undermines science by luring researchers away from doing science into administrative positions. “Many gifted young scientists, particularly women, have found these posts to be a wiser option than research”, he said. “These posts are usually secure and carry pensions and therefore take a large amount of money from the total scientific and teaching effort. They can be contrasted with the ephemeral and insecure support given to many teachers, researchers, and their younger dependents such as students and postdocs”.

Conservative review

In the meantime, funding agencies have become aware that their peer review system often works against risky blue skies research projects upon which scientific and subsequently technological progress ultimately depends. “When funds are plentiful it is easy for the whole system of peer review to support real blue skies risky projects, not poor‐quality ones but ones that if they are successful will lead to the big breakthroughs”, explained Richard Cogdell, Professor from the Institute of Molecular Cell & Systems Biology, University of Glasgow, UK. “However, when funds are limited the peer review system is very biased towards safe projects. These are sure to deliver but will always be incremental”.

Some of the research councils in the UK have now introduced sandpits to address this problem with some success. These are residential interactive workshops lasting 5 days involving 20–30 participants, where free thinking is encouraged to delve into the problems on the agenda and unearth innovative solutions. But sandpits are just a start and need follow‐up by properly funded research programs. “We really need to get out the message that funders should not be risk adverse”, Cogdell insisted. “Difficult science, blue skies science, often fails. But we need those failures to make big progress. The funders must be willing to accept this and not to penalize those researchers who are brave enough to take those risks and not to be afraid to fail sometimes. I would like to see the research councils have a significant fraction of their funding put aside for risky long‐term projects and to justify these properly to government”.

This argument is backed up by Richard Roberts, Chief Scientific Officer at New England Biolabs (NEB), a major supplier of enzymes in Ipswich, MA, USA. “Very few politicians understand that research is like venture capital”, he said. “One has to fund a lot in order to get the unexpected breakthroughs that lead to revolutionary discoveries. They seem not to understand that one cannot predict where truly innovative research will lead”. Roberts, who won the 1993 Nobel Prize in Physiology or Medicine with Phillip Allen Sharp for the discovery of introns in eukaryotic DNA, also agrees with another point Cogdell made, namely that the problem is particularly acute in the USA and could even threaten the country's lead in biomedical science. He suggested that the problem dates back to 2004, when the National Institutes of Health (NIH) budget hit a plateau that lasted until 2009 when US President Barack Obama's stimulus plan drastically increased the NIH budget (http://www.cbo.gov/sites/default/files/cbofiles/attachments/02-22-ARRA.pdf). “This encouraged many universities to hire more people than they probably should have done, in order to raise more overhead money”, Roberts added. “Although the budget slowly rose after 2015, the competition increased also. There was always more bureaucracy than there probably should have been, compared to say the UK, but I am told it is increasingly more time consuming to provide all of the non‐scientific material required by funding agencies”.

While spending for administration has been ballooning, the number of long‐term, tenured positions in academia has stalled or even been reduced in lieu of limited tenure or contracts.

But there is change ahead as some funding agencies not only take a more enlightened approach when deciding which projects to back but are actually dedicated to support blue skies research. One example is CIFAR (formerly Canadian Institute for Advanced Research), a Canadian charity now funding long‐term research on an international basis. With an annual budget of about CA$41 million (€28 million), CIFAR is a small player on the global stage but has successfully leveraged its funds by stimulating collaborations between sectors and countries, rather like the European Science Foundation (ESF) did until 2015 (https://en.wikipedia.org/wiki/European_Science_Foundation#Past_Activities).

“Our goal is to catalyze deep and sustained conversations around important questions”, said Alan Bernstein, CIFAR President and CEO. “Hence, each of our 13 global research programs is reviewed every 5 years and can continue indefinitely as long as we feel that real advances are being made. Because we are not a funder and because we expect the fellows in each program to meet at least twice a year, the questions must be compelling and interesting”. Bernstein is keen to avoid the term blue skies research in this context because it implies that it lacks direction or purpose; instead, the aim of CIFAR's funding is to be flexible and open‐minded in pursuit of understanding.

One point, sometimes overlooked, is that few universities now seem to support research from internal sources, except as a pump‐priming exercise designed to attract external funding.

Like Roberts, Bernstein ultimately holds politicians responsible for the increasing anxiety researchers face over funding. He dates the beginning of this trend back to the 1990s but reckons that it accelerated around a decade ago. “Most funders are under increasing pressure from government or donors to account for how the funds are spent and the outcomes of the research that is funded”, Bernstein explained. The decline in success rates for funding has also contributed to a waste of time and energy as scientists as they have to write more applications. “Success rates in grant competitions have decreased from an ‘ideal’ number of about 35% to, in North America, around 10%”, Bernstein said. “Inevitably, one of the consequences of this precipitous drop in success rates is the necessity to write more grant applications”.

Reforming research assessment

Another fundamental problem is a perceived need for accountability and performance assessment and a trend among academic institutions to see research as a direct profit center rather than source of intellectual capital. “One point, sometimes overlooked, is that few universities now seem to support research from internal sources, except as a pump‐priming exercise designed to attract external funding”, said John Allen from University College London and a longstanding critic of metrics‐driven performance evaluation in research. “The aim of research has become one of increasing income”. Along with an increasing bureaucracy, this has spurred research assessment to absurd heights where the IF and the title of journals a researcher has published in have become more important than the actual content of these publications.

This has spurred an initiative called DORA (San Francisco Declaration On Research Assessment; http://sfdora.org), set up in 2013 initially as a web page collecting signatures from scientists looking to reform the way research is assessed and grants are awarded. “Since that time, we have evolved into an active initiative campaigning for change”, commented Anna Hatch, DORA Program Director. “Community engagement is a big part of what DORA does. We organize online community interviews for members of the scholarly community to discuss some of the thornier challenges related to research assessment. DORA also organizes interactive workshops at conferences to critically evaluate research assessment practices”.

Hatch argued that DORA has already played a role in changing attitudes among some key funders. “For example, the Dutch Research Council has introduced a narrative CV format into some of its funding schemes to encourage a focus on research quality rather than quantity”, she said. The UK‐based Wellcome Trust has also been singled out by DORA as a positive example given its policy that expert reviewers take into account the diverse range of research outputs and focus on the content and quality of publications, rather than reputations. “As a signatory to DORA we have put in various processes to ensure that applicants are not assessed on whether they have published previously”, said Robert Kiley, Head of Open Research at the Wellcome Trust. As an example of such processes, Kiley cited modifications to grant application forms so that researchers can now include preprints, datasets, software, research materials and inventions, patents, and other commercial activities; previously, applicants were just asked to list their publications, as many other funders did and still do.

Kiley added that more fundamental changes have to be made to reform research assessment, for example, alternatives to the traditional research article for sharing information. He conceded though that any initiatives have to overcome an important hurdle, which is the enormous popularity of the IF as an assessment tool owing to its simplicity. Any research assessment however requires a much more nuanced understanding of the skills, qualities, and attitudes of researchers, which can never be captured or expressed meaningfully by a single metric.

Any research assessment however requires a much more nuanced understanding of the skills, qualities and attitudes of researchers, which can never be captured or expressed meaningfully by a single metric.

But it is not clear yet if the major funders of research, first and foremost the NIH and NSF in the USA, will embrace new ways of research assessment and peer review. Even more, there are concerns that the European Research Council (ERC), which was initially welcomed as a breath of fresh air upon its formation in 2007, might also become more conservative in assessing its applications. With a budget of €13 billion for the 7‐year 2014–2020 period, the ERC was set up to award grants on merit rather than on the basis of allocating EU funds equitably among member states, and the explicit aim of boosting the quality of European research.

It also raised hopes that Europe could overtake the USA over fundamental research, but this opportunity may have been missed, according to Howard Jacobs, who leads the Academy of Finland‐funded FinMIT Centre of Excellence at Tampere University. “In theory, Europe should be moving into new areas faster”, he said. “Impressionistically or anecdotally, ERC itself seems to have become a bit more risk averse than it was at the start, when wild and whacky projects were well supported, even if they had a high failure rate. In those early days, this didn't matter, because the approach was new, and the successes invariably outshone the failures. But now that ERC has a reputation to maintain, there seems to be a greater tendency to fund ‘safer science’ than before”.

Jacobs is not pessimistic overall but is aware of what the forces of change are up against in making research assessment as impartial and objective as possible. “It is, and will continue to be, a hard and lonely battle”, he commented. “Not enough scientists have heard about DORA and what it stands for, or against. […] In the end, far too many final recommendations are still guided by raw bibliometrics. We have a long way still to travel”.

Even if more funding bodies use much more nuanced systems for assessing grant applications, it would not alleviate the other problem of lack of tenured position that discourages many young scientists. The provision of more sandpits may help to further improve grant review but it must be accompanied by better job security and a more enlightened approach to research assessment. Ultimately, the administrative load and the reliance on simple bibliometric measures can only be reduced by taking funding out of the hands of bureaucrats and allowing scientists to allocate and manage budgets directly, commented Lawrence.

EMBO Reports (2020) 21: e50165