How One Division at NHLBI Establishes Its Scientific Priorities

The scientific priorities of the National Heart, Lung, and Blood Institute are delineated in the NHLBI Strategic Plan, which was released in March 2007. The plan lays out a broad agenda for government-funded biomedical research and training in cardiovascular disease, lung, and blood diseases and sleep disorders; approximately 90% of the Institute’s research budget supports extramural researchers, through grants and contracts. One of the major roles of NHLBI program staff is to establish, implement, and evaluate specific priorities identified by the Strategic Plan.

There are two Divisions that oversee extramural cardiovascular research supported by the NHLBI -- the Division of Cardiovascular Diseases (DCVD) and the Division of Prevention and Population Sciences (DPPS). In a later column, we will describe the different roles of these Divisions in detail. Briefly, DCVD oversees basic, translational, and disease-oriented research, and DPPS oversees epidemiological research and trials that are specifically concerned with primary prevention or management of cardiovascular risk factors.

Extramural research supported by the NHLBI can be classified into five major types:

1. Investigator-initiated projects with total direct costs of no more than $500,000 per year (e.g., R01 grants): Investigators are free to submit proposals for review without any special pre-review consideration.

2. Investigator-initiated projects with total direct costs exceeding $500,000 in at least one year but not exceeding a cap, currently set at $1.515 million in any year: Investigators must first contact NHLBI staff for a pre-review in order to obtain permission to submit a formal application. The pre-review occurs on a Division level. (See http://www.nhlbi.nih.gov/funding/policies/500kweb.htm and http://www.nhlbi.nih.gov/funding/policies/grantcaps.htm for more details)

3. Investigator-initiated projects with total direct costs exceeding a cap of approximately $1.5 million in at least one year (see #2 above): Investigators must first contact NHLBI staff for pre-review in order to obtain permission to submit a formal application. This review is more extensive than category 2, with a formal review involving leadership throughout the entire Institute.

4. Institute-initiated grant programs: NHLBI program staff members develop formal Request for Applications (RFAs) for research grants that address issues felt to be of high priority by the Institute. These RFAs undergo a process of internal review, followed by external review by our Board of External Experts and Advisory Council.

5. Institute-initiated contracts: NHLBI program staff members develop a formal Request for Proposals (RFPs) for contract work deemed to be of high priority by the Institute. This typically is used for "big science projects," like multicenter clinical trials, major cohort studies, and making resources (such as genotyping) available to investigators. As with RFAs, these projects undergo a process of internal review, followed by external review by our Board of External Experts and Advisory Council.

In a later column, we will review the processes for applying for large grants in detail.

To help scientific staff in our Division (DPPS) articulate how they develop scientific priorities and make rational, transparent, and fair decisions, we polled them to identify factors they use to establish priorities. Through a process of collecting, culling, and combining factors, we identified a set of 16, shown in Table 1. We then asked staff to apply these factors by ranking their importance in prioritizing a set of sample research proposals for large-scale projects (categories 3 and 5 above). We later discussed the composite results and explored why individuals applied specific factors to specific projects.

Table 1.

The sixteen factors used in considering scientific priority, in alphabetical order

•	Feasibility
•	Innovation
•	Likelihood that entity outside of DPPS would support
•	Multidisciplinary nature of the research
•	Potential for impact on individual health
•	Potential for impact on national health care expenditures
•	Potential for impact on population health
•	Qualifications and track record of the proposer
•	Quality of proposed methods
•	Quality of writing and organization of the proposal
•	Relevance to a global (beyond US) agenda
•	Relevance to the DPPS mission
•	Relevance to the NHLBI Strategic Plan
•	Research need/lack of studies
•	Study cost
•	Uniqueness or timeliness of opportunity

The most important factor our staff identified was “impact on population health.” The next four most important factors were “feasibility,” “relevance to the DPPS mission,” the “need for or previous lack of studies on this topic,” and “quality of the proposed methods.” Some factors were not considered as important in the rankings because of overlap in the content with better-ranked factors. For example, since the DPPS mission is subsumed within the NHLBI Strategic Plan, the factor related to the Strategic Plan did not fare as well. “Study cost” ranked in the middle; the importance of cost might be expected to be subject to general budget pressures and has been a particular subject lately as the Division considers research on costly medical technology.

Several of these factors coincide with other lines of thought on research priority-setting. The Institute of Medicine outlined and endorsed six criteria that are used in priority-setting at NIH: 1) public health needs, 2) scientific quality of the research, 3) potential for scientific progress (the existence of promising pathways and qualified investigators), 4) portfolio diversification along the broad and expanding frontiers of research, and 5) adequate support of infrastructure (human capital, equipment and instrumentation, and facilities) (2). The first three criteria, in particular, echo the top-ranked DPPS factors. The World Health Organization budget is set, in large part (and at least in principle), based on evidence of burden of disease (3). Some of our staff’s thoughts are also consonant with another recently described construct for making research priority decisions within a context of insufficient evidence for clinical decision-making. Chalkidou and colleagues suggest three major questions: 1) Is there a net benefit?; 2) Is it worthwhile to collect additional evidence?; and 3) Should we wait to get this information? The first two questions overlap our identified factors of “impact of population health,” “relevance to the DPPS mission,” and “previous lack of studies on this topic” (4).

It is critical to point out that our work in establishing, implementing, and evaluating scientific priorities is distinct from peer review. Our staff place high value on peer review, particularly to determine scientific merit of specific proposals or ideas, and we do not try to supplant this process. Some of the factors that peer reviewers use to evaluate proposals overlap with the factors our staff use to consider over-arching priorities; perhaps the most relevant overlapping factor is “feasibility,” as this issue clearly directly ties into our responsibility as stewards of public monies. Nonetheless, in general, factors used to establish scientific priorities will differ based on the specific mission of the funding agency or group, and hence transcends the specific strengths or weaknesses of any one specific scientific idea or proposal.

It is also critical to point out that establishment, implementation, and evaluation of scientific priorities are not activities that NHLBI staff members partake in isolation. Just as the NHLBI Strategic Plan was conceived and developed in close consultation with the extramural scientific community, activities of program staff also occur in nearly constant communication with extramural scientists. There are, for example, formal mechanisms for this, such as NHLBI-sponsored working groups or workshops; in other cases, ongoing dialogue about scientific priorities occurs within other settings, such as scientific meetings. In presenting this summary of our own internal discussions about how best to consider scientific priorities, we hope to stimulate ongoing dialogue with the scientific community not only on specific fields of endeavor but also on how best we can work together to ensure that NHLBI taxpayer supported funds are best used in the public interest.