Maximizing the return on taxpayers' investments in fundamental biomedical research

Abstract

The National Institute of General Medical Sciences (NIGMS) at the U.S. National Institutes of Health has an annual budget of more than $2.3 billion. The institute uses these funds to support fundamental biomedical research and training at universities, medical schools, and other institutions across the country. My job as director of NIGMS is to work to maximize the scientific returns on the taxpayers' investments. I describe how we are optimizing our investment strategies and funding mechanisms, and how, in the process, we hope to create a more efficient and sustainable biomedical research enterprise.

People often ask me to explain my job as director of the National Institute of General Medical Sciences (NIGMS). This turns out not to be a very hard question to answer, because my primary role is easy to define: I work to maximize the return on taxpayers' investments in the fundamental biomedical research and training the institute funds. In my first year and a half at NIGMS, I have found this job description to be very useful as a central premise to guide decision making. In this article, I will describe how this guiding principle is helping us reshape our strategies for supporting fundamental biomedical research.

A diverse investment portfolio is a strong portfolio

A core principle of both financial investment and biology is that diversity leads to strength. In biology, hybrid vigor produces healthy organisms, because each parental genome contributes complementary strengths and balances the other's weaknesses. In investing, diversity spreads risk, maximizing the chances of finding some big winners while reducing the chances of collapse based on sudden declines in a few companies' fortunes.

These principles also apply to supporting fundamental biomedical research (Lauer, 2014). Diversity at all levels—from the kinds of science to the regions in which it is conducted to the backgrounds of the people conducting it—strengthens the institute's research portfolio and should lead to the best returns on the taxpayers' investments. It is impossible to know where or when the next big advances will arise, and history tells us that they frequently spring from unexpected sources. It is also impossible to know what threads of foundational knowledge will be woven together to produce a new breakthrough. Supporting a wide variety of lines of inquiry will improve the chances of important discoveries being made. This includes studying a diversity of organisms, because important and useful processes almost certainly remain to be discovered in areas of biology we have not yet explored.

Scientists' past or current experiences have a significant impact on the problems they choose to study and on the ideas they have for approaching these problems. Health burdens from specific diseases differ from state to state and population to population, and these variances can drive the kinds of questions researchers ask. In addition to variations in regional health burdens, other environmental factors—such as local plants and animals—can influence an investigator's research directions. For example, Baldomero Olivera, now a prominent researcher at the University of Utah, began studying deadly cone snail venom while he was working in the Philippines, where the animals are endemic and people routinely died of snail stings (Telis, 2014). Olivera's work on cone snail toxins has transformed neuroscience and has already led to one Food and Drug Administration–approved drug and several more in clinical trials.

We also need to consider the identification and development of scientific talent when planning how we invest in biomedical research. For example, if cutting-edge biomedical research were only being conducted in 25 states, it would mean that high school and college students in the other 25 states could get research experience only if they were willing and able to move. The loss of talent to science in the United States caused by this “research experience gap” would be severe.

Emphasis on investigator-initiated research

A key question is how to identify and fund the best science. In addition to the value of supporting a diverse research portfolio, it is important to recognize that the best ideas come from investigators themselves. Although there are times when management and top-down direction can help break through systemic barriers or open up bottlenecks—particularly when the development of new technologies is required—fundamental research works best when investigators are following their noses and setting their own directions.

During the National Institutes of Health (NIH) budget-doubling period (1998–2003), the fraction of the NIGMS portfolio dedicated to investigator-initiated research declined from 99 to 80% because of increasing use of programmatic initiatives (Figure 1); that is, funding targeted at specific scientific areas. With the budget doubling more than a decade behind us, it is time to return the institute's focus to investigator-initiated research, to ensure that new scientific territory is opened for exploration by adventurous investigators.

FIGURE 1:

The fraction of the NIGMS portfolio committed to investigator-initiated research has declined over the past two decades. The blue bars (left axis) show the funds NIGMS committed to funding opportunity announcements (FOAs) targeted at specific areas of research in each fiscal year shown. The red line with triangles shows the change in the percentage of the NIGMS portfolio dedicated to investigator-initiated research. The analysis does not include fellowship, career development, and training awards; programs transferred to NIGMS from the former National Center for Research Resources; and some other programs. Jim Deatherage (NIGMS) performed the data analysis.

The value of team science

A criticism of NIGMS' renewed emphasis on investigator-initiated research has been that it will disadvantage team science and collaboration. But the dichotomy is not between investigator-initiated research and team science; it is between investigator-initiated research and targeted or top-down research, in which funds are earmarked for specific scientific areas. Team science aimed at studying fundamental biomedical problems can—and usually should—be investigator-initiated.

Interdisciplinary team science is undoubtedly extremely important and will become increasingly so as we delve deeper into the complexities of biological systems. One way to support team science is for independently funded principal investigators (PIs) to form collaborations organically based on common interests and complementary skills, a method that, for fundamental research, seems likely to yield better results than collaboration borne of incentives such as targeted funding–opportunity announcements. NIH also has several mechanisms dedicated to supporting investigator-initiated team science, including multi-PI R01s and program project grants (P01s). The Centers of Biomedical Research Excellence within the NIGMS Institutional Development Award program also support team science, with a focus on developing the careers of junior investigators (NIGMS, 2015). Moving forward, we plan to try to understand what kinds of teams benefit the most from unified grant mechanisms such as multi-PI R01s and P01s.

Building the foundation for breakthroughs: the value of steady progress

Discovery is the currency of science. Press coverage, prizes, and renown all revolve around the concepts of discoveries and breakthroughs. Discoveries are usually portrayed as sudden events—eureka moments—in which a “paradigm shift” occurs. And yet most major advances in science actually happen when a series of small steps coalesce into an important new understanding.

For example, the discovery of restriction enzymes, a truly transformative advance that propelled biomedical research in the 1970s into the age of molecular biology and launched the biotechnology industry, began in the 1950s with the description of bacterial resistance to phage infection. Dozens of papers published in various journals led to the insights that brought the Nobel Prize to Warner Arber, Daniel Nathans, and Hamilton Smith in 1978 (Loenen et al., 2014). Discoveries arise from a complex web of knowledge, and without the network created by the steady progress of many researchers, they would not occur.

As discussed earlier, it is impossible to know in advance where in this web the next big breakthroughs will arise or which strands of knowledge will be required to make them. The original observation of CRISPR sequences, for example, was made at the end of a 1987 Journal of Bacteriology paper that was otherwise devoted to reporting the sequence of the gene for the Escherichia coli alkaline phosphatase isozyme–converting protein Iap (Ishino et al., 1987). Describing the mysterious repeat sequences, the paper ends with the sentence, “So far no sequence homologous to these has been found elsewhere in procaryotes, and the biological significance is not known.” Slowly, through years of careful characterization of the CRISPR pathway, our understanding of what this initial observation meant fueled the development of a novel technology that has dramatically improved our ability to replace genes in living cells, paving the way for advances in medicine and biotechnology. This and many other instances like it show how important it is for us to support as wide a web of research as possible.

The optimal lab size

In 1985, Bruce Alberts wrote a prescient commentary in which he laid out the inefficiencies created in the basic biomedical research enterprise when labs become too large (Alberts, 1985). There are a number of reasons for these inefficiencies, but they mostly come down to bandwidth. For example, the more people a PI has in his or her lab, the less time he or she can devote to training and supervising each one. In addition, the bigger a lab gets, the more time the PI must spend on the grant-writing and administrative tasks needed to keep the operation running and the less time he or she has for actually doing research. For these reasons, Alberts argued that the per capita output of a lab would generally diminish above a certain size and that NIH and other agencies should cap PI funding, thereby limiting lab size and optimizing scientific productivity and quality.

In 2010, NIGMS conducted an analysis of the productivity and scientific impact of the research the institute funds as a function of each NIGMS investigator's total direct-cost support from NIH (Figure 2). This study showed that, on average, these metrics increase only shallowly with funding above a moderate level (∼$250,000–300,000) and then actually decrease above ∼$750,000 (Berg, 2010a, b). Although a few investigators beat the averages and increase their productivity at a proportional or better rate as their funding increases, other well-funded investigators perform even worse than the averages. Similar results have recently been published for chemical and biological research supported by other agencies (Fortin and Currie, 2013; Gallo et al., 2014) and by the NIH (Berg, 2015; Danthi et al., 2015).

FIGURE 2:

A 2010 analysis of researchers funded by NIGMS showed that, on average, productivity did not scale proportionally with funding beyond a relatively moderate direct-cost threshold. (Top panel) Average number of publications associated with NIH grants of NIGMS-funded investigators as a function of their total NIH direct costs (red line with circles) and the average impact factor of the journals in which each set of investigators published (blue line with squares). (Bottom panel) The unbinned data used to generate the averages shown in the top panel. Jeremy Berg, Paul Sheehy, and Matt Eblen (NIGMS) performed the data analysis.

Thinking about these data from the perspective of an investor of taxpayer funds, one can do a simple back-of-the-envelope calculation to determine what the best investment strategy for fundamental research should be. Imagine that we need to choose between funding two applications. The first is from a talented established investigator and is for a new R01, which would be his third, bringing him from $400,000 to $600,000 in total direct funding. The second is from a promising new investigator with no other funding. It would bring her $200,000 in total direct support. In the first case, the additional $200,000 would buy taxpayers, on average, approximately one more paper during the funding period over what the established PI would have produced with $400,000 (Figure 2). In the second case, the new PI would produce five papers on average in the funding period if she were awarded the grant. Thus the choice seems obvious: taxpayers net four more papers by funding the new PI than by giving the established PI a third grant.

Of course, there are caveats to this analysis. For example, some biomedical research simply costs more money than the average, and this needs to be considered when grant budgets are set. And what about scientific merit? A difficulty here is that recent analyses have indicated that the peer-review process does not have sufficient resolution to accurately distinguish among the most promising applications, with at best modest correlations between score and productivity or impact for funded applications (Berg, 2011, 2013; Gallo et al., 2014; Kaltman et al., 2014; Danthi et al., 2015). So, for example, if the established PI's application scored in the fifth percentile, and the new PI's scored in the 15th percentile, could we really be confident that the former is likely to produce more important work than the latter? Along these same lines, as mentioned earlier, although we fund some PIs who beat the average productivity versus funding curve, we also fund some who are below it, again indicating that picking the high performers is not easy, at least with the current process. These considerations suggest that we should be very selective in allowing PIs to accumulate high funding levels and that, in general, funding more investigators at a moderate level rather than a few at a high level will yield the best payoffs.

Changing our funding metric

A question that at first glance may seem trivial but is, I believe, a significant one is whether our key metric for how we use the funds we invest in biomedical research should be the number of grants we award or the number of investigators we support.

Currently we focus on the number of grants we fund. For example, we report to the scientific community and Congress on the success rate for grant funding: how many grants we awarded in a year divided by the number of grant applications we received. In theory, focusing on grants tells us how many projects we are funding. However, because PIs can have more than one NIGMS research grant, this focus distances our funding decisions from the key question of how many investigators are in our portfolio. If we instead used the number of PIs we support as the key parameter to drive our funding and programmatic decisions, it would reduce the number of variables, allowing us to focus on the most important ones.

For example, once we decided on the optimal number of NIGMS-supported investigators, we could then set a mean and median direct-cost target per investigator based on our total research budget. We could also model and implement a rational funding distribution. Once we had established the total number of investigators who should be supported by the institute, we could determine how many new PIs should enter—and how many established PIs should exit—the system each year. We would also be better able to ensure the diversity and breadth of the institute's research portfolio.

Overall this seemingly simple shift in how we view our mission could be a useful catalyst for reequilibrating the biomedical research enterprise to maximize its effectiveness, efficiency, and sustainability. Promoting this shift is something we are working on at NIGMS, both as part of our new strategic plan and through a new funding mechanism, described in the following section.

A funding experiment: supporting research programs instead of specific projects

Reequilibrating the biomedical research enterprise to make it more efficient and sustainable will require major changes in every part of the system. Because we do not know a priori which changes will work best and because there is always a risk of unintended negative consequences, the soundest approach will be to experiment and to expand initiatives that succeed and abandon those that do not. This model requires us to define in advance the outcomes we hope to achieve and to collect the necessary data to measure them as the experiments progress.

One such experiment is the NIGMS Maximizing Investigators' Research Award (MIRA; Preusch, 2015). The MIRA program aims to transform how we support fundamental biomedical research, shifting away from the current paradigm of funding specific, predefined projects to one in which we focus on supporting the overall research program in each investigator's lab. The goals of the MIRA pilot are to 1) increase funding stability for investigators to enhance their ability to take on ambitious research and approach problems creatively; 2) increase flexibility for investigators to follow new directions as ideas and opportunities arise, which should help maximize the chances for breakthroughs; 3) improve the distribution of funding, allowing the institute to support an optimally broad and diverse portfolio of investigators; and 4) reduce the amount of time spent writing and reviewing grant applications, freeing up time to focus on research, training, and mentoring. Each of these goals aims to help maximize the scientific returns on taxpayers' investments.

Creating a bright future for biomedical research

There has recently been—appropriately—much discussion of the problems facing the biomedical research enterprise (e.g., Ioannidis, 2011; Bourne, 2013b, c; Alberts et al., 2014). Among the many challenges, perhaps the most worrying is that junior scientists are becoming increasingly discouraged about their career prospects (Bourne, 2013a; Polka and Krukenberg, 2014), a growing crisis that could leave a serious deficit in the nation's scientific capacity for years to come. But despite the many challenges we face, there are reasons to be optimistic about the future. Scientifically, our deepening and expanding knowledge is opening up incredible new frontiers in research, and developments in technology are allowing us to address questions that a decade ago seemed completely inaccessible. Although I have not discussed it here, renewed focus on improving scientific education, training, and career development seems likely to lead to an even more skilled, productive, and efficient workforce in the coming years. There is a growing recognition that we owe it to the nation and future generations to reoptimize the biomedical research enterprise (Lorsch, 2015), and momentum is building in many sectors to make the changes this reoptimization will require. For example, the NIH (Rockey, 2012, 2015; Maas, 2015), members of Congress (Harris and Young, 2014; Hearing on the FY 2016 National Institutes of Health Budget Request, 2015), professional societies (Federation of the American Societies for Experimental Biology, 2015), and academic leaders (Daniels, 2015) are focusing attention on the challenges facing junior scientists and are trying to develop strategies to help this critically important group. If these trends continue, and all of the stakeholders take on their share of the responsibility, I am confident we can develop a more efficient and sustainable biomedical research enterprise and, in the process, accelerate advances in human health and prosperity.

Abbreviations used:

MIRA

Maximizing Investigators' Research Award

NIGMS

National Institute of General Medical Sciences

NIH

National Institutes of Health

PI

principal investigator.