Strategic Transformation of Population Studies: Recommendations of the Working Group on Epidemiology and Population Sciences From the National Heart, Lung, and Blood Advisory Council and Board of External Experts

Abstract

In 2013, the National Heart, Lung, and Blood Institute assembled a working group on epidemiology and population sciences from its Advisory Council and Board of External Experts. The working group was charged with making recommendations to the National Heart, Lung, and Blood Advisory Council about how the National Heart, Lung, and Blood Institute could take advantage of new scientific opportunities and delineate future directions for the epidemiology of heart, lung, blood, and sleep diseases. Seven actionable recommendations were proposed for consideration. The themes included 1) defining the compelling scientific questions and challenges in population sciences and epidemiology of heart, lung, blood, and sleep diseases; 2) developing methods and training mechanisms to integrate “big data” science into the practice of epidemiology; 3) creating a cohort consortium and inventory of major studies to optimize the efficient use of data and specimens; and 4) fostering a more open, competitive approach to evaluating large-scale longitudinal epidemiology and population studies. By building on the track record of success of the heart, lung, blood, and sleep cohorts to leverage new data science opportunities and encourage broad research and training partnerships, these recommendations lay a strong foundation for the transformation of heart, lung, blood, and sleep epidemiology.

Editor's note: An invited commentary on this article appears on page 369, and the authors’ response appears on page 372.

Population studies have entered an exciting period in which advances in assay methods, imaging technologies, and electronic data have created new scientific opportunities. To take advantage of these and to develop relevant scientific priorities in the next decade, strategic planning is crucial, especially in times of resource constraints (1).

Within this context, the rapidly emerging field of data sciences and the advent of new digital data sources, which range from electronic devices used by individuals to databases within healthcare systems, greatly expand the opportunities for population science (2). Indeed, the new tools that have emerged can serve as novel platforms for epidemiologic research, including the use of digital tools and mobile health applications to ascertain exposure and the availability of electronic, administrative, and medical records data to sample populations and ascertain outcomes. In addition to new data sources, the evolution of work processes, the opportunities for collaborations, and digital communications offer new avenues for the population science community to incorporate these novel approaches into new or existing studies. Implementing the recommendations outlined in this report will put the epidemiology community in a position to address unmet needs and novel questions, to exploit new data sources creatively and efficiently, and to train the next generation of population scientists.

APPROACH AND METHODS USED TO GENERATE THIS REPORT

In September of 2013, the National Heart, Lung, and Blood Institute (NHLBI) assembled a working group on epidemiology and population sciences from its Advisory Council and Board of External Experts. The working group was charged with making recommendations to the National Heart, Lung, and Blood Advisory Council about how the NHLBI could take advantage of new scientific opportunities and delineate future directions for the study of epidemiology of heart, lung, blood, and sleep diseases. Working group members, whose backgrounds covered a broad spectrum of population, clinical, and basic science research experience, attended 19 webinars to review key aspects of NHLBI-funded cohort studies and to garner information on new methods for data acquisition relevant to the field.

Related activities elsewhere within the National Institutes of Health (NIH), such as in the National Cancer Institute, were also considered. The working group had conference calls and an in-person meeting hosted by the NHLBI on May 13–14, 2014, to generate these recommendations. The overarching objective was to identify actionable directions that would both benefit from immediate engagement and be consistent with the goals of the NHLBI and the NIH.

THEMES AND RECOMMENDATIONS

The themes, which emerged from the deliberations of the working group, included 1) defining the compelling scientific questions and challenges in population sciences and epidemiology of heart, lung, blood, and sleep diseases; 2) developing methods and training mechanisms to integrate “big data” science into the practice of epidemiology; 3) creating a cohort consortium and inventory of major studies to optimize the efficient use of data and specimens; and 4) fostering a more open, competitive approach to determining which epidemiology and population studies to support. Seven actionable recommendations are summarized in Table 1. The order does not indicate prioritization, and each actionable recommendation and its rationale are discussed below.

Table 1.

Summary of Recommendations

Recommendation	Description
Create a scientific forum on population sciences	The NHLBI should convene a scientific forum to anticipate the major scientific questions and methodological needs in epidemiology and population science over the next 10–20 years.
Launch electronic epidemiology, particularly in collaboration with other organizations and agencies	The NHLBI should actively engage in studies to establish the validity, reliability, and scalability of electronic tools for primary data collection. In doing so, the NHLBI should partner with other organizations and agencies.
Build the data-science workforce	The NHLBI should help establish an adequate workforce to conduct population sciences “of the future,” and one approach is to create multifaceted and complementary career development grants.
Develop a dynamic compendium of epidemiologic resources	Resources should be dedicated to creating a dynamic compendium of large epidemiologic resources, including cohort studies, clinical trials data sets, registries, biorepositories, and other relevant epidemiologic resources, to assist the research community in identifying and accessing key existing resources and to improve the return on the investment from these studies.
Integrate epidemiology and clinical trials	Where genuine efficiencies can be created, the NHLBI should encourage the integration of clinical trials and epidemiologic studies.
Create a cohort consortium	The NHLBI should create a cohort consortium to support large-scale collaborations and provide a coordinated, interdisciplinary approach to addressing scientific questions, achieving economies of scale, creating opportunities for collaboration, and accelerating the pace of research and the implementation of new methods.
Implement competitive external evaluation of cohorts	The NHLBI should implement a competitive peer review–based model for its portfolio of large epidemiologic and population studies.

Abbreviation: NHLBI, National Heart, Lung, Blood Institute.

Recommendation 1: The NHLBI should convene a scientific forum to anticipate the major scientific questions and methodological needs in epidemiology and population science over the next 10–20 years.

Population science addresses questions on an array of topics that range from exposures and etiology to prediction, disease distribution and surveillance, natural experiments, population biology, health services research, and interventions for disease prevention. Key public health questions and unmet needs should be identified and prioritized by engaging relevant scientific communities. The development of a robust, prioritized research agenda is especially important in times of financial constraints.

The NHLBI should convene a scientific forum through avenues such as blogs, workshops, and strategic planning groups with the specific goal of identifying the high-priority questions for population science. The NHLBI should also explore how this activity might be integrated into its ongoing Strategic Visioning work. Scientific communities to be engaged in the forum should include not only epidemiology scientists, clinical researchers, and basic scientists, but also industry experts, pharmaceutical companies, public health experts, members of the disease-affected communities, researchers from other NIH institutes and centers, members of professional societies, and other interested parties. The scientific questions and unmet needs identified through this process should help define the research agenda and drive the development of methods. Balanced engagement across the spectrum of population research, including but not limited to population surveillance, etiological epidemiology, disease prediction, genetic epidemiology, and outcomes research, should be discussed. Given the constrained resources, the emerging position of the Patient-Centered Outcomes Research Institute in health services research represents an important development. Potential collaborations and synergies with the NIH Health Care Systems Research Collaboratory and the Patient-Centered Outcomes Research Institute should be actively explored. NHLBI involvement should complement or supplement rather than duplicate or compete with activities of other funding agencies or organizations.

Recommendation 2: The NHLBI should actively engage in studies to establish the validity, reliability, and scalability of electronic tools for primary data collection. In doing so, the NHLBI should partner with other organizations and agencies.

The sources of health information are changing dramatically. Data are increasingly originating from diverse electronic sources stored in numerous new formats that generate very large data sets and evolve very rapidly. The NHLBI should respond to this major transformation of electronic information related to health and should support innovation to define the role of new data sources for population research. Validation of the data collected using all new electronic methods is scientifically required before full field deployment. Individual investigators, many of whom were supported by the NHLBI, have organically initiated steps to use novel digital tools for clinical and epidemiologic research. The NHLBI has already funded cost-effective studies that leverage professional society registries. These efforts should be encouraged, scaled up to address appropriate questions, and designed to remain current, and the sharing of knowledge and experience acquired by individual research teams should also be fostered. Because the various electronic tools vary across the country and often change rapidly over time, the reliability and validity of new data sources will require ongoing study and evaluation.

The NHLBI should actively promote investigator-initiated testing of innovative approaches for data collection and analysis. For instance, a strategically developed portfolio of mechanisms to encourage high-risk, high-payoff proof-of-concept studies can be rapidly evaluated and expanded/validated or discontinued. This line of research will provide opportunities for the NHLBI-supported epidemiologic studies to foster collaborations that leverage or are coordinated with these and similar efforts that occur elsewhere within NIH and within other sectors. With the advent of these new methods, ensuring the continuity and consistency of the data over time poses additional challenges. It is useful to distinguish between electronic data generated by mobile devices or personal monitors, such as Fitbit (Fitbit Inc., San Francisco, California), and the electronic medical record because the validation efforts for these tools will differ. Conducting research in health-care systems poses unique challenges because electronic medical records are designed for medical clinical care rather than clinical research. For example, data generated by medical encounters for specific problems or symptoms differ markedly from the data acquired by a standardized examination visit in a cohort study. The currently dominating electronic medical record systems (by Epic Systems Corporation (Verona, Wisconsin) and Cerner Corporation (Kansas City, Missouri)) and the proliferation of local add-ons to the main platforms have created extraordinary complexity, and the commercial interests might not align well with the research interests of public health scientists. Other large-scale data sources, not only the mobile health devices but also omics, repositories, and national pharmacy databases, pose their own methodological challenges that should be considered and addressed as well.

Recognizing the need to explore and exploit new data sources, the NIH has appointed a new Associate Director for Data Sciences and created the NIH Big Data to Knowledge initiative (3).

The NIH is also investing in improved ways to conduct research in health-care systems (e.g., the NIH Health Care Systems Research Collaboratory) and is working closely with Patient-Centered Outcomes Research Institute to develop large-scale research networks that are capable of conducting low-cost, high-impact observational and interventional studies. The proliferation of efforts in data sciences challenges the NHLBI to effectively leverage investments made by other institutes, offices, and agencies to maximize synergies and avoid duplication and redundancies. Indeed, these initiatives might be incompletely coordinated, and an organized approach to electronic epidemiology would benefit the scientific community. The NHLBI should support partnerships and programs to develop, validate, and share methods to use digital tools for clinical and epidemiologcy research. The NHLBI should partner with the NIH Office of the Associate Director for Data Sciences to develop strategies to address the use of electronic medical records for research through a more concerted, NIH-wide voice. Partnerships with health systems, and perhaps with insurers and other NIH institutes and centers, will be essential to the success of this process. Linkage of data from the Centers for Medicare and Medicaid Services to the NHLBI studies should be actively encouraged, and the NIH should address regulatory, administrative, and ethical barriers to use of this information for medical research. The NHLBI should also exploit opportunities to foster international collaborations with the large cohort studies and biobanks in Europe and elsewhere.

Recommendation 3: The NHLBI should help establish an adequate workforce to conduct population sciences “of the future,” and one approach is to create multifaceted and complementary career-development grants.

Ultimately, the future of population sciences and its ability to make scientifically sound and efficient use of digital tools will be largely defined and undertaken by young investigators. The training and career development of investigators to conduct population studies of the future will require robust mentorship from established population scientists and methodologists and extensive training in data sciences. In the setting of existing studies, mentoring is often best done by study investigators who are expertly conversant in the data set, its conduct, and the strengths and weaknesses of data elements and the proposed scientific study. To work in the new digital enterprise, young investigators will need additional training in information science, new analytic methods, and team science.

To prepare this new generation of investigators, the NHLBI should proactively partner with others working in this area, particularly the NIH Office of the Associate Director for Data Sciences, because training is one of the programmatic themes of this new office. Review panels for these career development applications should consist of established investigators with multidisciplinary representation. Additional considerations might include the following.

• For junior investigators (e.g., K08/K23 mechanisms): In addition to the standard requirements, key components to career-development awards should include the opportunity for education in areas directly relevant and applicable to electronic data, electronic epidemiology, medical informatics, use of electronic medical record systems, engineering and operations research applied to population sciences, and relevant analytical approaches.

• For mid-career investigators (e.g., K18 mechanism; 1–2 years of support for re-tooling): These awards would help mid-career investigators to redirect population sciences research, to learn and apply the new data sciences methods, and to accelerate or optimize their use.

• For senior investigators: The development of mentoring awards (e.g., similar to the National Cancer Institute's K05 mechanism) could help provide research development support for mentees.

• In addition to training and early career funding, the NHLBI should promote and recognize team science, including a focus on dissemination research.

Recommendation 4: Resources should be dedicated to creating a dynamic compendium of large epidemiologic resources, including cohort studies, clinical trial data sets, registries, biorepositories, and other relevant epidemiologic resources, to assist the research community in identifying and accessing key existing resources and to improve the return on the investment from these studies.

Investment in cohort studies by the NHLBI and other federal agencies has led to the accumulation of extensive high-quality phenotypic and genotypic data, biobanks of preserved specimens, and data sets of images of participants. These data and specimens constitute a national resource and can yield critical insights in addressing current and future questions in population science. Although major efforts have succeeded in making the cohort data available to a broad community of investigators and much of the relevant information on cohorts already exists on individual cohort websites, the research community would benefit from an inventory of the resources within and across cohorts—one that is synthesized, maintained, and expanded over time.

This process of creating such a resource could include several steps. First, a comprehensive survey of the entire research community supported by the NHLBI should be conducted in order to generate a complete inventory of all relevant epidemiologic resources, including cohort studies, clinical trials data sets, registries, and biorepositories. The survey should identify key characteristics necessary to design and construct the dynamic compendium (e.g., sample sizes, demographic characteristics, exposures, and outcomes, including variable definitions). The second step should consist of the design and development of the compendium and the user interface. A third step should include mechanisms to foster access to these resources and collaboration.

The architecture is envisioned to be analogous to PubMed, registration of clinical trials (ClinicalTrials.gov), or the University of California Santa Cruz Genome Browser. It should enable nimble growth, cross referencing, and user queries. The platform should also facilitate mentored access to data and specimens in the cohort studies, encourage efficiency in future studies, and minimize redundancy in data collection. The platform will need to incorporate timely information about rapidly enlarging data sets and new patient populations. Finally, the NHLBI should seek partnerships with national and international organizations, so that this international compendium can facilitate the creation of research consortia to address scientific questions that require large sample sizes and well-phenotyped populations.

Recommendation 5: Where genuine efficiencies can be created, the NHLBI should encourage the integration of clinical trials and epidemiologic studies.

There are several opportunities for integration of clinical trials and epidemiology studies.

• Clinical trial databases, especially those from pragmatic trials that enroll broadly representative populations, provide opportunities to conduct observational research.

• Cohort studies might be used in selected circumstances to recruit participants to clinical trials if the interventions are not inconsistent with the goal to describe natural history in cohorts.

• Finally, within health-care systems, ongoing clinical trials and observational assessments of cohorts might allow for the parallel evaluation of the implementation of preventive and therapeutic strategies.

The settings for the integration of clinical trials and epidemiologic studies can include population-based samples or health-care systems, and the closed systems of care are likely to be efficient settings for recruiting patients and following them in clinical trials and cohort studies. In hybrid designs, the proposed trials and cohort studies should not interfere with the aims of the other, and the hybrid designs should create genuine efficiencies. The enrollment and consent process should anticipate and appreciate the evolving nature of large studies. Hence, it is important that the integration of clinical trials with epidemiologic studies be scientifically justified and operationally practicable. The opportunities for the appropriate integration of clinical trials and cohort studies would be enhanced by the dynamic compendium (recommendation 4). The experiences of the NIH Health Care Systems Research Collaboratory and the Food and Drug Administration Mini-Sentinel (4) are likely to provide guidance on the opportunities and challenges of integrating clinical trials and observational studies.

Recommendation 6: The NHLBI should create a cohort consortium to support large-scale collaborations and provide a coordinated, interdisciplinary approach to addressing scientific questions, achieving economies of scale, creating opportunities for collaboration, and accelerating the pace of research and the implementation of new methods.

Population-based cohort studies provide unique opportunities with proven value, and they should retain an important place in the overall NHLBI portfolio, even as new methods are developed, validated, and used. The power and reach of current large cohorts should be preserved by infrastructure funding sufficient in amount and certainty to preserve their substantial and ongoing value. However, the current silo-like approach to funding and managing large-scale epidemiologic and population-based studies occasions inefficiencies and missed scientific opportunities. Although the individual cohort studies have unique characteristics and individual value, the working group recommends that the NHLBI create a cohort consortium to synthesize, in a virtual way, the existing studies into a coordinated management and scientific structure. The NHLBI cohort consortium should foster harmonization of existing data, encourage de novo data collection methods across cohorts, and at the same time, preserve not only the unique features of each contributing cohort but also the energy, enthusiasm, and creativity of its investigators. Among its agenda items, the cohort consortium should provide opportunities for creating large synthetic cohorts that expand the representation beyond any single study. The cohort consortium should also consider evaluation of methods to incorporate into existing cohort studies data from mobile, home-monitoring, and electronic medical records to supplement and widen the duration between examinations. An integrated and cutting-edge design, together with initiatives to develop novel methods, should facilitate and catalyze the use of big data emerging across the research portfolio, including population science, behavioral, outcomes, and genetics/genomics studies. The NHLBI cohort consortium should eventually link with cohorts funded by other NIH institutes and centers to build a national research resource available to the broad scientific community.

The NHLBI cohort consortium should form a steering committee responsible for overall policy, management, and scientific direction of the cohort consortium. The steering committee, which should include cohort investigators as well as other senior scientists, could offer insights to the NHLBI on the relative advantages of infrastructure support, enhancing resource utilization, and identifying economies of scale. This process could be used to reduce the marginal/incremental cost for new studies and encourage work across sites. Flexibility to shift emphasis to new imperatives would be enhanced. Maintenance of the cohort studies' stature and competitiveness in a rapidly evolving global arena of population science demands the development of this national research resource.

Recommendation 7: The NHLBI should implement a competitive peer review–based model for its portfolio of large epidemiologic and population studies.

The working group endorses the importance of both investigator-initiated research and the peer-review mechanism for evaluating research. Major new work and large-scale scientific investigations should be determined through the peer-review process, and investigator-initiated grants should be the predominant driver of the research agenda.

Historically, the NHLBI cohort studies have been funded as contracts that provide the institute with a high level of control over these large projects. At the start of a new cohort study, the initial contract proposals for field centers, laboratories, and a coordinating center are all selected on the basis of rigorous peer review. Once funded, these cohort studies undergo periodic contract renewals that are reviewed; however, the cohort nature of these studies—particularly the participants and the field centers—has made it difficult for other investigators or institutions to compete with existing centers for these investigations. Typically, the renewed contracts have supported additional clinic visits that include expensive hypothesis-driven examination components, such as measures of subclinical disease. For all the cohorts, these same clinic visits have also incorporated additional hypothesis-driven examination components funded by investigator-initiated, often NIH peer-reviewed applications. Over time, the cohort studies accumulate valuable data and specimens, and even though the vast majority of clinic visits for participants include a mix of contract-funded and investigator-initiated components, the large epidemiologic studies might at times have appeared to operate, in part, outside of the realm of peer review.

In an effort to increase hypothesis-driven, investigator-initiated research, the NHLBI recently revised its approach to funding examination components. At this time, contracts continue to fund infrastructure, which includes participant contacts, events data collection, and the management of data and stored biospecimens. If applicants obtain NIH funding for hypothesis-driven, investigator-initiated grants that include at least 1 major examination component, the contracts will also fund a skeleton clinic visit. In other words, each major expensive examination component will require submission as an investigator-initiated grant application and subsequent peer review. Failure to obtain NIH grant funding for an investigator-initiated examination component will also mean no NHLBI contract funding of clinic visits. One potential concern is the short duration of grants; for instance, a novel measure of subclinical disease that is simply a study of prevalence in a 4-year study might be more attractive to peer reviewers with a long-term horizon as an investigation that evaluates the measure's association with cardiovascular events. Additionally, synergies among examination components will not be apparent to reviewers who evaluate separate applications. Nonetheless, for the most part, because previous clinic visits had already involved both contract- and grant-funded examination components, this new approach to funding the clinic examinations of participants in cohort studies represents a difference of degree rather than kind. Funding for the cohort studies is in a period of transition. Other models of peer review for examination components are possible, and the steering committee of the cohort consortium (recommendation 6) should manage these transitions and coordinate changes.

Basic infrastructure, such as participant contacts and event data collection, serves as the foundation for hypothesis-driven applications for examination components or novel assays; yet, investigator-initiated grant applications to support infrastructure alone lack hypotheses and are therefore unlikely to fare well in regular study sections accustomed to reviewing hypothesis-driven research. One option is to continue to fund infrastructure through the contract mechanism. Another option is to develop a new competitive model for the review of infrastructure applications. The National Cancer Institute uses a U01 mechanism to support core infrastructure (PAR-14-160), with the result that a special emphasis panel reviews applications for support of infrastructure for cancer cohorts. The working group recommends a similar mechanism with a study section within the NHLBI to review the infrastructure component of NHLBI cohorts. In this way, not only the infrastructure component of the cohort studies but also other types of new infrastructure or study platforms could undergo fair and rigorous competitive review. The review criteria should emphasize innovation, the validation of emerging study designs, and measurement technologies to achieve efficiencies. This approach provides opportunities to preserve valuable research resources in the existing cohorts and, at the same time, to develop new forms of population studies. A process and timetable for the overall transition toward the competitive peer review–based model with a clear staging of the migration should be developed for all the cohorts, their examination components, and their infrastructure support.

Metrics of past performance and future promise should be integral parts of program evaluation, but the working group cautions the NHLBI to not rely on any single metric, such as number of publications or citations, and to focus on the future proposed science. The working group recommends that the NHLBI keep metrics that would facilitate a robust competitive process of evaluating epidemiologic and population studies, similar to what is done for all other studies within the NHLBI, so that the best infrastructure grants can get funded. For research purposes, appropriate data should be available to compare between groups of interest (e.g., earlier stage versus later stage investigators).

CONCLUDING REMARKS

The 7 broad recommendations presented herein are intended to put the epidemiology community in a position to address unmet needs and novel questions, to take advantage of new data sources creatively and efficiently, and to train the next generation of population scientists. Recognizing that more work is needed to address the opportunities and challenges inherent to each recommendation, the working group invites the needed continuing dialogue with the scientific communities and with key stakeholders.