Reforming Science: Structural Reforms

Abstract

Science has a critical role to play in addressing humanity's most important challenges in the twenty-first century. However, the contemporary scientific enterprise has developed in ways that prevent it from reaching maximum effectiveness and detract from the appeal of a research career. To be effective, the methodological and culture reforms discussed in the accompanying essay must be accompanied by fundamental structural reforms that include a renewed vigorous societal investment in science and scientists.

EDITORIAL

“There are three basic flavors of incentive: economic, social and moral.”

—Steven D. Levitt and Stephen J. Dubner, Freakonomics (31)

Our premise is that science is not as healthy as it could be, nor as it needs to be to effectively address the challenges facing humanity in the 21st century. The current hypercompetitive environment has created an insecure working environment for scientists, fostered poor scientific practices, including frank misconduct, and created widespread disillusionment throughout the scientific community, from trainees to senior investigators. In the preceding commentary, we have discussed a number of methodological and cultural reforms to address some of the problems with contemporary science (8). However, we recognize that changes in scientific methods and culture will have a limited impact in the absence of fundamental structural reforms in the way that science is supported. This is because, as we have already discussed, many dysfunctional aspects of science are rational responses by scientists to incentives presented by the current system. True reform will require changing these incentives by addressing fundamental structural aspects of the scientific enterprise that create the constraints under which science is performed. Since most science is supported by public funding, any structural reform will inevitably involve changes in the way governmental financial support is provided to scientists. As any allocation of public funds is ultimately a result of the political process, any attempt to implement structural reforms will by necessity involve engagement with the political process and its agencies. Since countries differ in their political and scientific organization, structural reforms will differ from country to country. This essay focuses on biomedical science in the United States, the system with which the authors are most familiar, but we hope that readers will find many of the themes to be of universal relevance.

CURRENT STRUCTURAL PROBLEMS WITH BIOMEDICAL SCIENCE IN THE UNITED STATES

The primary problem of inadequate funding.

At the root of most of the problems with American science today is a lack of sufficient resources to support the current enterprise. Grant paylines that commonly exceeded 50% in the 1960s are now below 10% in many disciplines. Overall success rates of research proposals, including both renewal and new applications, submitted to the National Institutes of Health have fallen by more than 50% since 1965 (Fig. 1). As the ameliorating effects of the 2009 ARRA (American Recovery and Reinvestment Act) stimulus funding come to an end, the full impact of the deficient federal investment in science is only now being fully felt. Grant review panels are regularly forced to decide between competing highly meritorious projects. While some competition is inarguably good for science, excessive competition is demoralizing, destructive, and counterproductive. Funding agencies cannot continue to reject more than nine-tenths of grant applications without seriously damaging science. In the current climate, good ideas are going unsupported, opportunities are being squandered, and capable scientists are being lost. It may be tempting to demand an increased contribution of resources from researchers' institutions, but this is unlikely to be successful, at least in the near term, because institutional budgets have also taken a hit from depreciating investments and reduced revenues to state governments that find themselves unable to meet commitments made during better economic times. In fact, increasing evidence suggests that the indirect costs provided by federal grants are inadequate to meet the true institutional costs of doing research. A recent study from the University of Rochester found that the institution was required to contribute 40 cents for every grant dollar generated by its new faculty hires in basic science, even though the investigators were highly successful, obtaining an average of $800,000 (1999–2006 U.S. dollars) in grant revenues per faculty member per year (14). Present indirect cost rates thereby create a perverse calculus in which the greater an investigator's success in obtaining grants, the greater the resulting burden on the institution.

Fig 1.

Overall research grant success rates at the National Institutes of Health, 1965 to 2010 (9, 33).

An increasing emphasis on targeted research funding.

The present funding shortage has been exacerbated by a reduced emphasis on investigator-initiated projects (e.g., NIH R01-supported projects) in favor of targeted research and big science (6, 26). Investigator-initiated R01s have declined by 15% as a proportion of the NIH research budget since 1997-1998 (17), while targeted research grants have doubled over the same period of time (33). From these data we infer a worrisome trend in which a major funding agency is increasingly directing what scientists should work on rather than allowing scientists to make that decision for themselves. Moreover, political influences on funding can distort scientific priorities and neglect important areas of investigation. Targeted funding for currently recognized problems is politically more palatable than spending on what appear to be esoteric projects associated with basic research. While we recognize that public agencies that are mandated to improve the health of society must address immediate problems with targeted funding, it is important to realize that novel therapies and transformative discoveries are critically dependent on basic research. History has repeatedly shown that serendipity plays an important role in scientific progress, leading to penicillin, Teflon, Viagra, and PCR.

The Manhattan Project and the war on AIDS are often touted as models for successful focused research, but these may represent the exception rather than the rule. The Manhattan Project was successful in part because the field of physics had advanced sufficiently to make the creation of a nuclear fission-based bomb feasible, and also perhaps because the physicists were lucky that their prototypes worked the first time (37). In other words, the basic science on nuclear fission had already been done so that the challenges in making the bomb were largely in the technical and engineering realms. The war on AIDS was able to rapidly deliver effective antiviral agents because preceding decades of research on retroviruses and rational drug design had created a scaffolding on which to build a new program. It is noteworthy that the advances against AIDS relied on basic science investments made decades earlier at a time when there was little evidence that retroviruses were involved in human disease. Put another way, the curiosity of earlier scientists in studying viruses that caused tumors in animals combined with the prescient financial support of prior generations for basic research paid off with tremendous dividends once retroviruses became associated with human disease. Today, retroviruses are known to cause AIDS, hepatitis, and various tumors, and drugs that target retroviruses are making a tremendous difference in human health. In contrast, no effective AIDS vaccine has yet been developed, possibly because the immunology is not sufficiently well understood, and the war on cancer has produced some advances but many disappointments. Hence, the success of targeted research may be a function of how much basic science is known about a problem.

Leaky pipelines.

Although women receive nearly half of all doctoral degrees, they make up only about 17% of tenured science faculty in the United States (18). The “leaky pipeline” for women in science careers reflects in large part a disproportionate unwillingness of young female scientists to sacrifice everything outside of their careers for the sake of professional achievement. Women with Ph.D. degrees and young children are 35% less likely than men with young children, and 28% less likely than women without children, to obtain tenure (18). The pipeline is also leaky for underrepresented minorities in science (4, 12, 21), and recent evidence indicates that minority applicants are less likely to receive research funding from the NIH, even after accounting for education, country of origin, training, employer, prior research awards, and publications (38). The leaky pipeline represents an enormous systematic loss of talent and diversity to science.

Increasing administrative burden.

American scientists are increasingly burdened with regulations and administrative responsibilities that touch all aspects of research (20). Although some regulation is clearly necessary, there is a point at which the costs exceed the benefits. Other than publications and grant applications, much research-associated paperwork is designed to address concerns about animal welfare, patient safety, and the accountability of public funds (19). Although the scientific portion of an NIH research grant has been mercifully shortened to 12 pages, the administrative portion is typically much longer; program project grants or training grants may reach hundreds of pages. Most research projects require annual progress reports, and ARRA-supported research requires quarterly progress reports, even though in the research world, progress does not occur with sufficient regularity to justify a 3-month reporting interval. (Nevertheless, grateful recipients of ARRA funding have gladly complied with this request.)

With regard to vertebrate animal experimentation, society has wisely insisted that vertebrate animals be treated humanely in research. However, there are some contradictions in the way that society deals with animal issues. For example, a typical animal euthanasia protocol for mice requires an extremely detailed description of the methods to avoid pain and suffering, yet a stroll into any hardware store reveals several methods for killing mice with extreme pain and suffering, including asphyxiation and/or cervical dislocation by mouse trap, terminal exhaustion on a glue pad, and poison. The contrast between the societal approaches to murine death in the lab and the private home creates the paradox that regulations in place to ensure laboratory animal welfare can slow research intended to help society, while citizens do not require any protocols to exterminate vermin. Is the life of a mouse living in the walls of a private home less valuable than that of a mouse in a laboratory cage? (We hope the solution won't be to increase the regulatory requirements for exterminators!) Paperwork requirements for clinical research are even more onerous. The Infectious Diseases Society of America has recently called attention to the excessive regulatory burden resulting from the Health Insurance Portability and Accountability Act and the “mission creep” of Institutional Review Boards (22, 42). No wonder that a graduate student recently wrote, “When I grow up … I fear I will become an administrator … rather than an investigator” (24).

Grant peer review.

Review panels are able to accurately identify bad science but have a poor record of distinguishing highly innovative work or work that challenges existing dogma. Reviewers can be counted on to identify the top 20 to 30% of grant applications, but identifying the top 10% is impossible without a crystal ball or time machine. It is well documented that grant peer review is insufficiently precise to provide reliable rank ordering of applications (28). Moreover, the present review system places an excessive emphasis on potential conflicts of interest, leading in many cases to the disqualification of reviewers best able to provide a knowledgeable review, and insufficient emphasis on competence, vision, and relevant expertise.

STRUCTURAL REFORMS

Time for a renewed investment in science.

The political dialogue over research funding in the United States and many other countries has become a recurring discussion about shrinking budgets and competing priorities. Since 1963, the federal investment in research and development as a percentage of GDP has fallen steadily (17). If the major problems facing humanity are to be addressed, then this trend must be reversed. To place this in some perspective, the 2011 federal investment in applied and basic research was equivalent to only about one percent of the estimated cost of the wars in Iraq and Afghanistan (5, 10). A recent report indicates that U.S. research output is now being surpassed by Europe and Asia (1). The cold war may be over, but the threats facing the modern world are no less formidable. As an independent group of economists have recently concluded, the time is right for a major sustained government investment in infrastructure (2), and we submit that such an initiative must include the scientific enterprise. This will not only generate jobs in the short run, as a majority of grant expenditures are dedicated to personnel costs, but will lead to the discoveries that spawn new industries and create unimagined efficiencies in the long run. As the economists' report argues, “Labor costs will never be lower. Equipment costs will never be lower. The cost of capital will never be lower. Why wait?” (34). Scientific innovation is a valuable national resource that should not be squandered. Until increased resources are available, a diversion of more funding to untargeted funding mechanisms, such as investigator-initiated applications (R01s), could help to sustain the scientific enterprise during the present period of resource scarcity. Furthermore, devoting a portion of funding to institutions for salary support, rather than devoting all federal research support to individual projects, would provide greater stability to the system.

Balancing and renewing the scientific workforce.

If the fundamental structural problem with science today is an inadequacy in financial support of the current scientific workforce, then an obvious potential solution is to reduce the size of the workforce (27). However, while this might ameliorate competition for funding in the short run, this option would be tremendously shortsighted, as experts from across the political spectrum agree that more scientists, not fewer, are needed to address society's many challenges and generate the innovative discoveries that will resuscitate the global economy (15, 30, 43). A publication from the National Academies of Sciences called Rising Above the Gathering Storm makes a cogent argument to expand the pipeline of scientists, engineers, and mathematicians (11). The urgent need for more scientists documented in this report led to the National Math and Science Initiative (NMSI), a program designed to improve mathematics and science education and attract the best and brightest students to scientific careers. Notably, this has been a bipartisan initiative, as the NMSI was initiated during the administration of President George W. Bush and has been vigorously supported by the Obama administration, which added the “Educate to Innovate” program to enhance educational opportunities in science, technology, engineering, and mathematics.

However, it must be recognized that one of the greatest obstacles to recruiting the best and brightest students to scientific careers is the unhappiness of scientists working in the current environment. Anxiety over the future is at an all-time high, and there is concern that stopgap measures to set aside funds for new investigators have only intensified competition for funds among senior scientists (13). If the poor morale of active scientists is not addressed, all of the new initiatives will be for naught. It makes little sense to aggressively recruit bright young students to lifelong careers of struggle and uncertainty, especially when scientific training often requires a tremendous investment in time and resources. Few scientists would go as far as Jonathan Katz, a Washington University at St. Louis physicist who wrote an essay entitled “Don't become a scientist!” (29), but many mentors nevertheless make their reservations known to their trainees, even if inadvertently. Recruitment becomes much more straightforward if trainees can envisage a clear path to career success. An NIH working group has been formed to specifically address the future biomedical research workforce, and their report is anticipated in mid-2012 (32). We hope the report will agree with us that the scientific workforce should be expanded. We also hope that this report will help to clarify the optimal distribution of the scientific workforce, propose a means by which sustained support of that workforce can be achieved, and suggest measures to address the specific needs of female and underrepresented minority scientists (21, 25). Solving the leaky pipeline will require not only initial recruitment efforts and investment, but the establishment of sustained support mechanisms and the institution of more flexible and family-friendly policies (40). Less than 0.1% of the world's population is presently working as scientists or engineers (35), and only a fraction of this small percentage is involved in the generation of new knowledge. On this slender thread hangs society's future.

Recognizing the critical importance of basic research.

We have discussed the current emphasis on “translational” research in an earlier essay (17). While we acknowledge the importance of removing obstacles that impede the translation of basic discoveries into useful applications, we are concerned that an excessive focus on translation may eventually become a cautionary tale. Immediately following World War II, Vannevar Bush, the President's Science Policy Adviser, wrote, “It is wholly probable that progress in the treatment of … refractory diseases will be made as the result of fundamental discoveries in subjects unrelated to those diseases, and perhaps entirely unexpected by the investigator … (progress) results from discoveries in remote and unexpected fields … Government has provided over-all coordination and support; it has not dictated how the work should be done” (7). Elsewhere in the report, Bush astutely observed that “scientific progress on a broad front results from the free play of free intellects, working on subjects of their own choice, in the manner dictated by their curiosity” (7). In Lives of a Cell, Lewis Thomas wrote that “If I were a policy maker … (I would) give high priority to a lot more basic research” (39). Sadly, the wisdom that basic research provides the essential raw material for practical applications appears to have less appeal to current policymakers. The dramatic decline in the success rate of grant applications seeking support for basic research needs to be urgently addressed (3).

Restricting laboratory size.

The efficiency of laboratories in functioning, exchanging ideas, and producing new information must be a function of the lab size and composition. This raises the question: what is an optimal lab size? An interesting study by Jeremy Berg, the former head of the National Institute of General Medical Sciences, suggested diminishing returns once a laboratory has more than about $750,000 in direct costs (2006 U.S. dollars) (41). Other possible beneficial changes from a limitation on laboratory size would be the creation of more principal investigator positions and a renewed emphasis on investigator-initiated projects.

Regulatory and review reform.

An unfortunate aspect of regulations is that new requirements are added to old ones but the paperwork burden never seems to be reduced. The aforementioned abbreviation in NIH grant length is a noteworthy exception. An effort to similarly streamline the administrative sections of grant applications and reporting requirements would be welcome, perhaps as part of a comprehensive effort to limit the regulatory burden on scientists to measures that actually serve a meaningful purpose. In addition, the current mechanisms for grant peer review should be reexamined (16). Some scientists have advocated dramatically abbreviating the length of grant applications (36) and emphasizing the scientist rather than the project (23), but any changes will result in winners and losers and are bound to be controversial unless overall competition for funding is restored to a reasonable level.

A scientific study of science.

Despite the unquestioned success of science and the scientific method, it is remarkable how little we know about how to configure the scientific enterprise in an optimal manner to confront the problems facing humanity. For example, we do not know the answers to the following questions. How many scientists do you need to ensure a steady stream of innovation that will allow consistent economic growth? What is the optimal size of a research group? How well does peer review perform? What is the optimal time for scientific training? What is the relationship between length of scientific training and subsequent success in science? What is the optimal award time for a research grant to promote productivity without encouraging too much comfort and lassitude? Without answers to these questions, it is difficult to make the best choices as we struggle to restructure certain aspects of science. In fact, much of what we think we know in this realm is anecdotal and largely derived from individual experiences. However, each of these questions could be the subject of rigorous study, and the answers of such studies could provide information to inform future decision making.

CONCLUDING REMARKS

True reform will require addressing major structural aspects of the scientific enterprise—a reduction in personal pressures on scientists, perhaps by a greater institutional commitment of “hard” salary support, an emphasis on quality rather than quantity of publication, the fostering of a cooperative and collaborative culture, a reduced dependence on journal impact measures, and the development of more stable and sustainable sources of research funding. Additional structural changes will be required to enhance cooperation, allow risk taking, reduce funding pressures, and provide more flexible career pathways to prevent the ongoing loss of capable scientists along the pipeline (25). A society serious about confronting the real challenges of the future cannot afford to leave so many good scientists behind. The current global economic recession calls for intensive investment to renew the scientific infrastructure, which includes not only bricks, mortar, and equipment but human resources as well. Nations that recognize this opportunity will be the ones that rule the future.