The current state of funded NIH grants in implementation science in genomic medicine: A portfolio analysis

Abstract

Purpose

Implementation science offers methods to evaluate the translation of genomic medicine research into practice. The extent in which the NIH human genomics grant portfolio includes implementation science is unknown. This brief report’s objective is to describe recently funded implementation science studies in genomic medicine in the NIH grant portfolio, and identify remaining gaps.

Methods

We identified investigator-initiated NIH research grants on implementation science in genomic medicine (funding initiated 2012–2016). A codebook was adapted from the literature, three authors coded grants, and descriptive statistics were calculated for each code.

Results

42 grants fit the inclusion criteria (~1.75% of investigator-initiated genomics grants). The majority of included grants proposed qualitative and/or quantitative methods with cross sectional study designs, and described clinical settings and primarily white, non-Hispanic study populations. Most grants were in oncology and examined genetic testing for risk assessment. Finally, grants lacked the use of implementation science frameworks, and most examined uptake of genomic medicine and/or assessed patient centeredness.

Conclusion

We identified large gaps in implementation science studies in genomic medicine in the funded NIH portfolio over the past five years. To move the genomics field forward, investigator-initiated research grants should employ rigorous implementation science methods within diverse settings and populations.

INTRODUCTION

The rate of translation of genomic discoveries to benefit patient and population health has been slow compared with the rate of discovery.^1,2 As such, the majority of genomic research falls within the discovery and development phases (T0–T1), and only 2% of research falls within the translational phases (T2–4).³ Implementation science (IS) is a field of research that examines methods and strategies that are aimed to improve the translation of research discoveries to practice settings, making IS well suited to speed the rate of translation of genomic discoveries to benefit patient and population health.⁴

An increasing number of applications in genomics have been included in evidence based guidelines and are improving patient health.⁴ For those evidence-based genomics applications, implementation research can improve their translation into clinical and public health practice to improve health. For developing genomic applications, implementation should be considered across the research continuum; by planning for implementation early, the length of time between bench to bedside may be reduced once the evidence-base for the application has accrued.

In a recent literature review,⁵ we examined the extent to which translational genomic medicine research has incorporated IS methods. In the review, we identified several important gaps in the current literature, including a lack of rigorous IS methods (e.g., suboptimal use of IS conceptual frameworks), lack of attention to IS components such as capacity building and sustainability, low reporting of race and ethnicity as well as a lack of diversity in study populations and settings, and finally, most studies were descriptive and within the field of oncology. It remains unclear to what extent the NIH portfolio of funded grants will address some of these identified gaps in the literature. As such, the objective of this brief report was to examine the current NIH grant portfolio to: (1) ascertain whether recently funded NIH grants are bridging identified gaps from the literature, and (2) determine what gaps in the NIH portfolio persist.

METHODS

NIH extramural grants funded in fiscal years 2012–2016 were identified on September 30, 2016 through an internal NIH tool, Query, View, Report (QVR). QVR allows users to search, view, and retrieve detailed information about NIH applications and awards.

Search Process

We used QVR to create a search strategy of weighted terms, or “custom fingerprints,” to identify genomic medicine grants that included IS approaches. In our first step, we created three custom fingerprints, designed and performed by three authors (AK, MC, MR; Supplemental Table 1). These fingerprints were based on modified search terms from grants that were (1) submitted in three Implementing Genomics in Practice (IGNITE)⁶ funding announcements (n=729) (AK); (2) reviewed by the Dissemination and Implementation Research in Health (DIRH) study group and combined with a separate search using the Research, Condition, and Disease Category (RCDC) term ‘Human Genome’ (n=1557) (MC), and (3) reviewed by the DIRH study group with genomics terms, including: Human Genome, genomic medicine, personalized medicine, precision genomic medicine, genetic and genetic medicine (n=494) (MR). Grants ascertained from the searches were combined, de-duplicated, and reviewed for inclusion (by abstracts and specific aims) to include 508 grants (Table 1). In our second step, we created a custom fingerprint in QVR based on these 508 grants, and we identified an additional, relevant 427 grants. As a final step in the search process, we searched for all grants using the terms ‘Human Genome’ and ‘Human Subjects’ in QVR, and we identified an additional 51 relevant grants (total, n=986).

Table 1.

Inclusion and exclusion criteria for implementation science and genomic medicine studies in the NIH grant portfolio.

Inclusion Criteria	Notes
2012–2017 grant award initiated
F30, K01, K07, K08, K22, K23, K25, K99, R01, R03, R21, R33
Effectiveness Studies	Includes effectiveness studies examining clinical utility, costs, and health outcomes of testing
Comparative Effectiveness
Patient satisfaction with genetic services
Speed/timeliness of genetic services
Patient/provider/public awareness/knowledge/attitudes/perceptions/preferences about genetic services
Predictors of willingness to pay for genetic services	Includes multiple stakeholders
Providers' readiness to deliver genetic services
Uptake of genetic services	Includes barriers and facilitators for uptake of genetic testing/counseling
Strategies for recruitment into genomic research
Workforce
Exclusion Criteria
DP1, DP2, DP3, DP5, G13, F31, F32, F33, I21, I01, IK2, IS1 OT2, P01, P20, P30, P40, P41, P50, R13, R24, R25, RM1, R43, SC1, TL1, U mechanisms, continuations of grants awarded before 2012	Non-HHS Federal Awards, Other transactions, Program projects, Institutional grants or fellowship programs, Conferences, Resource-related research projects, Education projects, Small Business Innovation Research Grants (Phase I), Linked Training Award, Cooperative Agreement, Research Enhancement award
Content analysis of guidelines, policies, insurance criteria, literature
Predictive/Prognostic Model Validation or Evaluation
Efficacy study
Risk factor analysis
Case study
Article not written in English
Prevalence of mutations within a population
Discovery or mechanism of action
Conference abstract
Health services research among carriers or high risk groups
Measure development to assess psychosocial outcomes of mutation carriers
Psychosocial outcomes only regarding genetic services
Not a research study (no methods or results section)
Quality assessments/improvement

After restricting the set to select awarded, investigator-initiated, research and career development grants (R01, R03, R21, R33, K01, K07, K23, and K99; n=154), a review of each application’s research strategies was performed. Upon full review, additional grants were excluded based on inclusion and exclusion criteria⁷ (Table 1). Our final analytic sample included 42 grants that were coded.

Coding Methods

The initial codebook was adapted from a NIH portfolio review of IS funded studies, with additions and modifications made based upon our previous literature analysis of IS studies in the translational genomic medicine literature⁵. A subset of grants (n=9) were triple coded by all three authors (AK, MC, MR). Coding discrepancies were discussed and agreement was reached with additional comments and clarification added to the codebook to establish coding consistency. The remaining grants were divided for individual coding by the three authors. Questions about coding that occurred during this process were addressed and resolved by all three coders through consensus. To assess quality control across the single-coded grants, one author retrospectively coded a random sample of 20% of grants and found 92% agreement in coding. Additional quality control checks and review were performed on data when the single-coded data was merged into the final analytic file.

As a secondary analysis, we separately described cooperative agreements funded during the same period (2012–2016) with an adapted codebook. One author abstracted information about study design (i.e., study setting), genomics (i.e., disease area), and IS (i.e., use of IS frameworks and capacity indicators), and a second author check for consistency of coding and questions about inclusion were discussed until consensus was reached.

Analysis

Codes were summed, and descriptive statistics (i.e., proportions, means) were calculated.

RESULTS

We identified 42 genomic medicine grants that included elements of implementation science, representing approximately 1.75% of the investigator-initiated research grants in genomics (Table 2, Supplementary Table 2). Most included grants included T3 research (81%, n=36), while only 12% included T2 phase research, which represented “pre-implementation” focused evaluation research (i.e., research primarily focused on validity and utility, but in this case, also included some IS component). Only one grant included T4 research.

Table 2.

Descriptive statistics for included NIH grants on implementation science in genomic medicine (funding initiated 2012–2017).

Study Design Characteristics			Genomic Research Foci			Implementation Science Foci
	n	%		n	%		n	%
Translational Research Phase			Type of genetic test			Implementation Science
T2	5	11.9	Germline	31	73.8	Implementation	37	88.1
T3	36	81.0	Somatic	4	9.5	Dissemination	12	28.6
T4	1	2.4	NA	4	9.5	Adoption	2	4.8
			Cell-free DNA	3	7.1
Methods			Type of genetic test			Sustainability
Qualitative	35	83.3	NA/NR	16	38.1	Cost analysis	5	11.9
Quantitative	32	76.2	Single Gene	9	21.4	Capacity building	3	7.1
Cost	4	9.5	WGS	6	14.3	Maintenance	1	2.4
Other	3	7.1	Gene Panel	5	11.9
Comparative	3	7.1	WES	5	11.9
Simulation	1	2.4	Other	2	4.8
			Unknown	1	2.4
Study Design			Purpose of Genetic Test			IS Framework/Conceptual Model
Cross sectional	25	59.5	Oncology	19	45.2	For formative research (e.g. exploratory, qualitative research)	2	4.8
Cohort	9	21.4	Other	12	28.6	For intervention design	1	2.4
Randomized	9	21.4	Newborn Screening	3	7.1	Measured variables (e.g. primary or secondary outcomes)	1	2.4
Pre/Post	7	16.7	Prenatal	3	7.1
Other	3	7.1	General Clinical Seq.	2	4.8
Simulation	2	4.8	NA	2	4.8
Case Control	1	2.4
Unit of analysis						Collaborative Processes
Individual	35	83.3				None	27	64.3
Study Site	3	7.1				Patient engagement	5	11.9
Unclear	3	7.1				Stakeholder	5	11.9
						Designing for dissemination	2	4.8
						Team Science	2	4.8
						CBPR	1	2.4
Research Setting						IS Outcomes
Clinical	32	76.1				Patient-centeredness	22	52.4
Other	5	11.9				Uptake	14	33.3
Public Health	4	9.5				Feasibility	11	26.2
Both	1	2.4				Effectiveness	10	23.8
						Acceptability	6	14.3
						Costs	5	11.9
						Satisfaction	5	11.9
						Fidelity	3	7.1
						Equity	3	7.1
						Efficiency	2	4.8
						More than 3	2	4.8
						Other	1	2.4

Proportions are not mutually exclusive.

Grant Study Design Characteristics

Most grants proposed qualitative (n=35, 83.3%) and/or quantitative methods (n=40, 95.2%), including cost analyses (n=4), comparative effectiveness (n=3) and simulation modeling (n=1). Furthermore, study designs primarily included cross-sectional designs (n=25), but also included cohort (n=9), randomized controlled trials (n=9), pre-/post (n=7), and case-control (n=1) designs. Most awarded grants proposed research within the clinical setting (n=32, 76.2%) rather than public health settings (n=4, 9.5%), or other settings (e.g., online) (n=5, 11.9%). Proposed study samples primarily consisted of white (average proportion of whites=71.3%, median proportion of whites=75.6%), non-Hispanic (average proportion of Hispanics =81.2%, median proportion of Hispanics=88.9%) study populations.

Genomic Research Focus

Few grants focused on family history collection (11.9%, n=5); Instead, most described germline genetic testing (73.8%, n=31), with a minority of grants focusing on somatic (9.5%, n=4) or cell-free DNA testing (7.1%, n=3). More specifically, awarded grants included research on single gene tests (21.4%, n=9), whole genome sequencing (14.3%, n=6), whole exome sequencing (n=5, 11.9%) or gene panel testing (n=5, 11.9%). Half of the grants focused on the use of genomic medicine for risk assessment (n=21), and fewer included research aims related to diagnostic (n=11, 30%), therapeutic (n=7, 26.1%), preventive (n=4, 9.5%) or prognostic (n=1, 2.4%) testing. Most awarded grants included a focus on cancer screening or treatment (n=19, 45.2%) as opposed to other disease areas, such as newborn screening (n=3, 7.1%), prenatal testing (n=3, 7.1%), or other diseases/disorders (e.g., cardiovascular health, general pharmacogenomics, undiagnosed diseases, autism, Huntington’s disease, kidney disease, psychosis, hearing loss) (40.6%). Finally, many grants proposed to assess patient (54.8%) and provider (21.4%) attitudes, including assessment of barriers and facilitators to the implementation of genomic medicine (n=7, 16.7%).

Implementation Research Focus

Most IS in genomic medicine grants had aims related to implementation (n=37, 88.1%) rather than dissemination(s) (n=12, 28.6%) or adoption (n=2, 4.8%). Nine grants included sustainability indicators, such as costs (n=5, 11.9%), capacity building (n=3, 7.1%) or maintenance (n=1, 2.4%) measures. Only four grants used conceptual models from the IS field, with all using the Diffusion of Innovations model⁸. Half (n=2) used this model for formative research, one used the model for intervention design, and the other grant used the model for measurement.

While most grants did not explicitly include collaborative processes, two grants included designing for dissemination, five included patient engagement and five included stakeholder engagement, two grants included team science approaches, and one included community-based participatory research. Measured implementation and process outcomes included patient-centeredness (e.g., assessing patient barriers and facilitators) (n=22, 52.4%), uptake (n=14, 33.3%), feasibility (n=11, 26.2%), effectiveness (n=10, 23.8%), acceptability (n=6, 14.3%), costs (n=2 monetary, n=1 non-monetary, n=1 both, 9.5%), fidelity (n=3, 7.1%), equity (n=3, 7.1%), and efficiency (n=3 7.1%). Most studies included an individual unit of analysis (n=35, 83.3%), while three (7.1%) analyzed at the level of the study site (and unit of analysis was unclear in three grants).

In our secondary analysis, similar gaps were found among funded cooperative agreements (data not shown). For example, among the 39 cooperative agreements funded in IS in genomic medicine between 2012–2016, only 5% used IS frameworks, most studied oncology (n=10), most occurred in the clinical setting (n=34), and while slightly more U grants included measurement of capacity indicators (primarily by measuring costs), only approximately 30% included these indicators.

DISCUSSION

We found that the currently funded implementation research in genomic medicine includes primarily T3 implementation research in clinical settings that focuses on germline testing, risk assessment, and oncology. Like the published literature, only one grant included T4 research, suggesting a remaining gap in moving the field forward through all translational research phases.

The study designs proposed in these grant awards were typically cross-sectional, used an individual level unit of analysis, incorporated quantitative and qualitative methods, and occurred within clinical settings. Further, few studies included simulation, cost, and comparative effectiveness analyses. These characteristics largely reflect those found in the current literature. However, funded grants may partially close certain gaps. For example, the funded grant proposals incorporated qualitative methods into their study designs more than the current literature. The racial/ethnic diversity of populations in funded grants was similar to that reported in the current literature, being primarily white, non-Hispanic; however, unlike the published literature, which often lacked information about the racial/ethnic composition of their study populations, information about the racial composition of study populations was reported for all grants per reporting rules for human subjects research at the NIH. The majority of funded awards proposed the use of clinical settings, however this proportion was even larger among the funded grants, perhaps because many NIH grants are funded to academic institutions within clinical settings. Like the published literature, most grants include cross-sectional study designs, however more randomized control trials and pre/post studies were found among the funded grants than the current literature, suggesting that the body of research is beginning to shift from descriptive and exploratory studies to interventions within clinical and, to a lesser extent, public health settings.

Findings from this portfolio analysis were similar to the current IS literature in genomic medicine, which has primarily focused on germline testing to assess cancer risk. Further, the proportion of funded grants examining risk assessment and/or oncology was even greater than the current research literature, despite there being more variation in the type of genomic technology (i.e., germline, somatic, cell-free DNA) studied among the funded grants. This suggests a sustained gap in research examining applications of genomic medicine to disease areas outside of oncology and applications beyond risk assessment, such as prevention, prognosis, diagnosis or therapeutic settings (e.g., pharmacogenomics).

Finally, the majority of the literature did not include sustainability measures or incorporate IS conceptual frameworks, the same was true for funded grants, though rates of including sustainability measures and conceptual frameworks were higher in funded grants (e.g., 7.1% sustainability indicators in the literature vs. 20% in the funded grants), perhaps due to the presence of the DIRH study section which specifically reviews and awards IS grant applications on metrics including rigorous IS research methods⁹. Finally, like the literature, the most frequently measured IS and process outcomes were patient centeredness (often through the collection of barriers and facilitators) and uptake. Incorporating more rigorous IS methods and measures will allow practitioners and researchers to more effectively translate evidence based genomic discoveries to the benefit patient care.

While this portfolio analysis presents an overview of the currently funded IS in genomic medicine research, the analysis does have limitations. While our multipronged search was comprehensive, it is possible that we missed funded grants that include IS in translational genomic research. Of note, this review does not include grants whose initial funding began prior to 2012. As such, we did not include grants resulting from Clinical Sequencing Exploratory Research (CSER),¹⁰ which seeks to translate genomics into clinical practice. CSER2 will extend the efforts of CSER to promote IS studies in genomic medicine.¹¹ The IGNITE consortium was created to enhance the implementation of genomic medicine by supporting the development of methods for incorporating genomic information into clinical care.⁶ These consortiums are intended to fill gaps in the implementation of genomic medicine research, and their success should be evaluated in future studies. Such cooperative agreements were not included in the primary analysis, as we only examined independent research awards. We did not include these grants in the primary analysis because (1) our objective was to describe grants that investigators are submitting, rather than evaluating the success of funding announcements to award IS grants in genomic medicine, and (2) the scope of cooperative agreements differs from investigator-initiated grants; thus, a different codebook would need to be developed, and findings may not be directly comparable to investigator-initiated grants. Trends in gaps appeared to traverse the investigator-initiated grants and cooperative agreements. Finally, there are other agencies that have funded and can fund IS work, including AHRQ, PCORI, CDC, among others. This portfolio analysis does not provide a snapshot of IS research being funded outside of the NIH, however funding from these agencies is outside the scope of the paper. For these reasons, this analysis may underrepresent the currently funded IS studies in genomic medicine, however we would anticipate that the research gaps identified in this analysis would be similar.

Overall, this portfolio analysis demonstrates a continued need for research at the intersection of IS and genomic medicine. The NIH wide Dissemination and Implementation Research in Health funding announcement explicitly mentions genomics, and provides another vehicle for funding research at the intersection between genomic medicine and implementation science.¹² Moving forward, research that employs rigorous IS methods and measures and examines diverse genomic technologies will help move the translation of genomic medicine to improve patient care and ultimately population health.