The Prevalence of Dissemination and Implementation Research and Training Grants at National Cancer Institute–Designated Cancer Centers

Abstract

Background

Dissemination and implementation (D&I) research is a key factor in the uptake and use of evidence-based cancer control interventions. National Cancer Institute (NCI)–designated cancer centers are ideal settings in which to further D&I knowledge. The purpose of this study was to summarize the characteristics of NCI-funded D&I science grants in the nation’s cancer centers to understand the nature, extent, and opportunity for this key type of translational work.

Methods

We used the National Institutes of Health Research Portfolio Online Reporting Tool to identify active NCI-funded grants in D&I science at NCI clinical cancer centers (n = 13) and comprehensive cancer centers (n = 51) as well as their academic affiliates. Active projects were eligible for inclusion if they 1) were awarded directly to an NCI cancer center or an academic or research affiliate, and 2) identified D&I content in the abstract. Portfolio data were collected in February 2021.

Results

We identified 104 active NCI-funded D&I research or training grants across the 64 cancer centers; 57.8% of cancer centers had at least 1 NCI-funded D&I grant. Most awards (71.1%) were for research grants. Training grants constituted 29.1% of D&I-focused grants. Overall, 50.0% of grants (n = 52) concentrated on specific cancers. Almost two-thirds of grants (n = 68, 65.4%) had a stated health equity focus.

Conclusions

More than one-half of NCI-designated cancer centers have active funding in D&I science, reflecting a substantial investment by NCI. There remains considerable room for further development, which would further support NCI’s translational mission.

The National Cancer Institute (NCI) has been a leading source of funding for the creation of evidence-based programs, practices, policies, interventions, and strategies to improve cancer prevention and control (1). Beyond research funding, NCI offers opportunities for training and networking in dissemination and implementation (D&I) science in an effort to increase the awareness and use of D&I methods across the cancer control continuum (2). Increasingly, the focus of these efforts has been on effective and equitable delivery of evidence-based interventions to all who may benefit from them (3,4).

Neta et al. (1) described NCI’s funding portfolio through the National Institutes of Health Dissemination and Implementation Research in Health program announcements and highlighted the gaps and opportunities for future research to advance D&I in cancer control, providing valuable tools to support efforts to speed translation of evidence-based findings to practice and policy. However, as of yet there has been no comprehensive picture of the state of D&I research specifically at NCI-designated cancer centers. These cancer centers are central to developing a critical mass of D&I expertise because they provide a cancer research infrastructure across the United States, typically include expertise in cancer prevention and control and population science and serve a liaison function between researchers and the communities they serve (5). Further, as cancer centers increasingly develop their approaches to community engagement and outreach and strive for health equity, D&I expertise can play an important role in ensuring that these efforts are evidence based and adapted to local contexts and populations (6). The purpose of this study was to summarize the extent to which NCI cancer centers have active NCI-funded D&I research and training grants and examine the characteristics of those awards to identify strengths and highlight gaps in D&I focus areas.

Methods

Identification of NCI-Designated Cancer Centers

NCI-designated cancer centers comprise basic, clinical, and comprehensive cancer centers. Basic cancer centers focus on laboratory research. Clinical cancer centers have met NCI standards for cancer prevention, clinical services, or research, whereas comprehensive cancer centers meet NCI standards  in all 3 categories and demonstrate an added depth and breadth of research. NCI clinical cancer centers (n = 13) and comprehensive cancer centers (n = 51) were included in this analysis. A list of current NCI-designated cancer centers and their affiliated academic organizations was established by visiting the NCI website (https://cancercenters.cancer.gov/center/cclist) and reviewing each cancer center’s webpage. If needed, we conducted additional web searches to confirm affiliated academic organizations. To illustrate, a cancer center operating in a consortium may include the cancer center itself, a university medical school, a school of public health, and several academic hospitals, whereas other cancer centers may operate within 1 academic unit or act as a free-standing institution. From this point forward, we refer to all included centers as cancer centers.

Identification of D&I Grants

We used the NIH Research Portfolio Online reporting Tool (RePORTER; https://RePORTER.nih.gov/) to identify active NCI-funded grants in D&I science at cancer centers and their academic affiliates. We used the NIH definition of dissemination as “the targeted distribution of information and intervention materials to a specific public health or clinical practice audience” and implementation as “the use of strategies to adopt and integrate evidence-based health interventions and change practice patterns within specific settings,” following prior portfolio reviews (1,7,8). A reference librarian helped to develop a targeted search strategy. We conducted a test search with 19 initial search terms, then cross-checked the grants identified with a few known D&I abstracts to ensure they were appropriately retrieved by the search terms (7). After reviewing a subset of abstracts, the study team condensed the search terms to include: ((Implementation OR Translational OR Evidence-informed OR Evidence-based OR evidence) OR (“Hybrid Type-1” OR “Hybrid Type-2” OR “Hybrid Type-3”)) AND (Intervention OR Practice OR Science OR Research) AND (Diffusion OR Dissemination OR Implementation OR “Dissemination and Implementation”).

Our initial search included all actively funded grants regardless of organization or institution. Active projects were eligible to be included if they 1) were awarded directly to an NCI cancer center or an academic or research affiliate, and 2) had D&I content identified in the abstract. Active projects were excluded if they focused only on animal or cell-line research (ie, did not have direct application to human health). No exclusion restrictions were placed on funding type (eg, research grant, program project or center grant, training grant). The RePORTER search was conducted in February 2021 and yielded an initial 1423 active grants. After removing projects that were not awarded to a cancer center or affiliate, 950 projects remained. A further 194 projects were identified as duplicates and removed. A total 756 project abstracts were reviewed by the coding team; 136 were identified as D&I research. Projects with the same project identifier (eg, units within larger centers) were collapsed, resulting in a sample size of 104 abstracts retained for the final analysis (Figure  1).

Figure 1.

Flow diagram summarizing the analytic sample identification. ID = identifier (project number); NCI = National Cancer Institute; NIH = National Institutes of Health; RePORTER = NIH Research Portfolio Online reporting Tool.

Abstract Coding and Analysis

We developed a detailed codebook based on previous NIH grant portfolio reviews (1,7,8). To ensure consistency in coding, 2 coders reviewed an initial random sample of 50 abstracts and discussed discrepancies in code application; a third researcher reviewed coding consistency and calculated interim inter-rater reliability. Once adequate reliability was achieved, data extracted from RePORTER were divided between the 2 coders, weekly coding goals were established, and abstracts were randomly assigned for coding. Ten percent of all project abstracts were double-coded to assess agreement. Any discrepancies were brought to the larger research team for discussion and final inclusion decisions. The final inter-rater reliability (kappa) was .91. As planned, each abstract was also coded for types of cancer addressed, topic areas mentioned (eg, screening, survivorship, prevention behaviors), and aspects of health equity mentioned (eg, social, economic, or structural determinants of health; income; race or ethnicity; housing or homelessness; sexual or gender identity; disability; immigration; urban or rural).

Results

Distribution of D&I Grants Across Cancer Centers and by Funding Mechanism

We found a total of 104 D&I grants at 37 of the 64 cancer centers (see Table  1). Of 13 clinical cancer centers, 5 held at least 1 D&I grant (38.5%); of 51 comprehensive cancer centers, 32 held at least 1 D&I grant (62.7%). Slightly more than one-half of all cancer centers had at least 1 active NCI-funded D&I research or training grant (57.7%). However, only 33.4% of cancer centers had more than 1 D&I grant. Only a few cancer centers stood out with a substantial number of D&I grants; these notably had a varied portfolio with both investigator-initiated and collaborative research grants as well as training grants. Among the 37 clinical and comprehensive cancer centers that had D&I grants, 7 (18.9%) of the cancer centers held at least 1 D&I grant, and the remainder of the cancer centers had their D&I grants awarded through an affiliated institution.

Table 1.

Dissemination and implementation research and training grants at National Cancer Institute–Designated Cancer Centers

No. of dissemination and implementation grants	Frequencya (%)
0	27 (42.2)
1	15 (23.4)
2-3	13 (20.3)
4-6	7 (10.9)
≥9	2 (3.1)

^a Number of cancer centers.

Table  2 summarizes the mechanisms used to fund D&I studies. Of the 104 D&I grants, the vast majority (71.1%) were research focused (39.5). Of these, 55.4% were individual investigator–initiated mechanisms (R01, R37, R21, R03), whereas 44.6% were collaborative and multiproject mechanisms (eg, P01, P50). Training grants constituted 29.1% of D&I grants, and the vast majority (71.0%) were individual training awards (eg, K and F mechanisms). Nine of the D&I-focused training grants (30.0%) were institutional awards (T32 and R25). The D&I-focused training grants were concentrated in less than one-third of the cancer centers.

Table 2.

Frequency of dissemination and implementation grant mechanisms

Mechanism	Frequency (%)
Research mechanisms
R01	35 (33.7)
R37	3 (2.9)
R21	2 (1.9)
R03	1 (1.0)
P30	8 (7.7)
P50	7 (6.7)
P01	4 (3.8)
P20	1 (1.0)
UH3	4 (3.8)
U01	5 (4.8)
U54	2(1.9)
UM1	2 (1.9)
Training mechanisms
K07	8 (7.7)
K08	5 (4.8)
K01	3 (2.9)
K00	1 (1.0)
K23	1 (1.0)
K24	1 (1.0)
F31	1 (1.0)
F99	1 (1.0)
R25	1 (1.0)
T32	8 (7.7)
Total	104 (100.0)

Focus of D&I Grants

We conducted a planned review of the content areas to which D&I methods were applied in the studies that we identified. Fifty-two of the 104 of D&I grants (50.0%) identified 1 or more specific cancers of focus. The most frequently studied cancers were colorectal (17.3%), breast (16.3%), and cervical (12.5%). Those grants that did not include a specific cancer focused more generally on health behaviors, community outreach, or health information technology. Commonly studied topics were cancer screening (31.7%), survivorship (21.2%), and tobacco and vaping (18.3%) (Table  3).

Table 3.

Dissemination and implementation grant topic areas (not mutually exclusive)

Topic area of grant	Sum (%)
Cancer type
Colorectal	18 (17.3)
Breast	17 (16.3)
Cervical	13 (12.5)
Lung	9 (8.7)
Prostate	7 (6.7)
Head and neck	3 (2.9)
Anal	2 (1.9)
Endometrial	2 (1.9)
Leukemia	2 (1.9)
Multiple myeloma	2 (1.9)
Ovarian	2 (1.9)
Skin	2 (1.9)
Penile	2 (1.9)
Vulvar	2 (1.9)
Kidney	1 (1.0)
Lymphoma	1 (1.0)
Thyroid	1 (1.0)
Risk factor
Screening	33 (31.7)
Survivorship	22 (2.2)
Tobacco	19 (18.3)
HPV	13 (12.5)
Genetic factors	9 (8.7)
HIV	9 (8.7)
Physical activity	8 (7.7)
Diet	6 (5.8)
Obesity	4 (3.8)

^a HPV = human papillomavirus.

Among the D&I grants, 65.4% of grants (n = 68) had a stated health equity focus. Most of these grants included more than 1 health equity topic (65.1%), with 47.1% containing 3 or more health equity foci. The most common health equity topics noted were social, economic, or structural determinants of health (86.8%), race or ethnicity (45.6%), social needs (42.6%), socioeconomic status or income (36.8%), and rurality (27.9%). Equity topics discussed to a lesser degree included housing (2.9%), immigration status (2.9%), and sexual orientation or identity (1.5%). Disability was not included in any of the grants reviewed.

Discussion

The purpose of this study was to characterize the landscape of D&I research and training activity at NCI clinical and comprehensive cancer centers. Our ability to prevent, diagnose, and treat cancer has greatly improved over the past decades, and yet we know that without effective and sustained efforts to translate discoveries into practice across a range of settings, the benefits of evidence-based interventions will be limited, varied in reach, and likely to result in health inequity (9). Although prior reviews have focused on the nature of the NCI D&I funding portfolio without respect to setting, we focus on the nation’s cancer centers as a way to further understand the extent to which D&I research has penetrated these critical translational settings. Development of expertise in D&I science at cancer centers reflects a growing opportunity to enhance evidence translation to practice and policy. We found that approximately 58% of cancer centers have at least 1 funded D&I grant. This is encouraging and may reflect awareness among both cancer center investigators and leadership about the growing importance of D&I science in helping increase the pace and quality of evidence translation (4,9). The much higher percentage of comprehensive cancer centers with at least 1 funded D&I grant, compared with clinical cancer centers, may reflect the greater resource and facility availability in comprehensive cancer centers. However, only one-third of cancer centers had more than 1 current D&I grant, indicating that the majority of cancer centers have relatively limited funding in D&I; funding can also be viewed as tenuous when it is tied to 1 particular investigator. It should also be noted that the vast majority of D&I grants at cancer centers were awarded to affiliated institutions, which may further make the impact on the cancer centers more tenuous. However, we were not able to unpack the nature of the relationships between the cancer centers and their affiliates, and thus it is unclear the extent to which this is an issue for sustainability of D&I work that will directly affect the cancer centers.

Development of a critical mass of D&I expertise at individual cancer centers will be important given that this expertise has been identified as a cross-cutting area of study across the cancer control continuum, from etiology to survivorship (10). As previously described in a strategic planning effort by a Clinical Translational Science Award recipient, the development of D&I capacity in affiliated institutions requires strategic planning, strong leadership, time to allow for an incremental and iterative process, and intentional allocation of non-NIH funds (11). Without consideration of D&I science as a strategic priority of a cancer center, it will likely be a very slow route to developing the local expertise needed to ensure effective translation and sustainment of the cancer center’s emerging evidence base. Further, it is possible that there are specific characteristics of cancer centers that promote D&I science. This was beyond the scope of this article but is an area for future consideration.

Leppin et al. (12) offer an integrated framework for articulating the role of D&I science within and across all the translational research spectrum, which may be useful to cancer center leadership who wish to use coherent strategies for routinely and proactively accelerating research translation. Training grants may also offer a resource for enhancing D&I expertise; of note, only 9 cancer centers had an institutional training grant with a D&I focus.

We specifically focused on active NCI funding, because our interest lies in current D&I activity and capacity at cancer centers. This focus coincides with the selection of implementation science as 1 of 7 working group areas at the launch of the Cancer Moonshot accelerator in 2016, following the 21st Century Cures Act passed by Congress (10,13). In addition to increased grant funding opportunities, NCI’s focus on training in D&I science has likely had an important role to play in increasing D&I science at cancer centers. An evaluation of the NCI’s Training in Dissemination and Implementation Research in Health found that overall, trainees submitted more peer-reviewed NIH grants per person and had better funding outcomes than a comparison sample of unselected applicants whose application score was within 1 SD of the mean for all trainees’ scores for the same application year (8).

Almost one-half of studies in our sample focused on D&I related to 3 cancers: colorectal, breast, and cervical. Although this may at least in part reflect the availability of evidence-based interventions in cancer control, there does appear to be room to diversify the application of D&I methods to other cancer-related topics. Of note, there are a number of cancers for which there are only 1 or 2 D&I grants in the entire NCI portfolio. These topics may be good targets for expansion of D&I research, especially where there is robust evidence for both prevention and treatment interventions (eg, skin cancer). Although there are certainly many more grants in these areas that do not focus on D&I, a specific focus on implementation in a range of settings will be important to fulfilling their potential for improving cancer control outcomes. The required community outreach and engagement component of NCI’s cancer center core grant funding for clinical and comprehensive cancer centers may also benefit from consideration of the wide range of evidence translation efforts that could benefit from D&I methods (5).

There is rightly growing interest in considering health equity across the research enterprise, and there have been several recent calls to action to consider an explicit focus on equity in D&I research (14,15). Our finding that 65% of D&I abstracts mentioned topics related to equity is encouraging but does not ensure that the studies are actually targeting these factors. This finding also shows there are several underexamined equity areas, such as disability, sexual and gender identity, housing, and immigration status (6,16). Cancer centers are a vital place to address cancer inequities, as reflected in NCI’s charge to develop robust community outreach and engagement efforts, to engage with individuals and organizations in their geographic catchment areas, and to address local cancer needs (17).

Several limitations to this study should be noted. First, we limited our scope to NCI-designated clinical and comprehensive cancer centers with grants funded through NCI. Although some cancer center scientists may have D&I studies funded by other NIH institutes, our interest was specifically in cancer-focused D&I research, hence our focus on NCI-funded research. We recognize that this may have resulted in an undercount of cancer-focused D&I activities at a specific cancer center, but because NCI is a major funder of D&I research, we anticipate that the impact on our findings is limited. Some D&I research may also be conducted at nondesignated cancer centers, although we expect that NCI-designated clinical and comprehensive cancer centers are most likely to have the range of expertise needed for D&I research. Second, we only had access to grant abstracts. Although comprehensive, our search terms may not have captured all NCI grants with a D&I focus, depending on the extent to which relevant methods were mentioned in the abstract. Prior portfolio analyses on which our methods are drawn have been conducted by NCI staff that have access to internal NIH systems and more data. Thus, we were limited in the variables we could examine. We were not able to ascertain the specific funding dates of the included awards because in many cases the selected grants were competing renewals. In this circumstance, RePORTER lists the initial date of the parent award. However, all selected grants were active, and thus most were likely funded in the past 5 years. Third, we were only able to dichotomously categorize studies as having some element of D&I science and could not differentiate based on whether D&I was the primary focus. We further cannot conduct more detailed analysis on areas of focus within the grants because we are limited to the information in the abstracts. Fourth, we cannot conclude that cancer centers without D&I funding lack relevant expertise, because our data reflect a snapshot in time.

More than one-half of NCI-designated clinical and comprehensive cancer centers are currently engaged in NCI-funded D&I research. Topic areas tend to cluster around colorectal, breast, and cervical cancer and focus on cancer screening. We suggest there is room to increase the D&I focus at cancer centers in ways that will support NCI’s translational mission. There are opportunities for growth, including an assessment of study activities in terms of cancer prevention and control behaviors, phases of the cancer control continuum addressed, and the types of implementation efforts under study. In particular, cancer centers that do not have robust D&I research portfolios could encourage and support their investigators to build expertise by participating in one of the many D&I training programs now available and by joining the Consortium for Cancer Implementation Science, a network designed to build the cancer-focused D&I research community (2).

Funding

This work was supported by the National Cancer Institute (via a supplement to grant 5P30CA006516-55, and grants 2T32CA057711 and 1P50CA244433).

Notes

Role of the funder: Sponsors were not responsible for the design of the study; the collection, analysis, and interpretation of the data; the writing of the manuscript; and the decision to submit the manuscript for publication.

Disclosures: The authors have no conflicts of interest to disclose.

Author  contributions: Conceptualization (KE, AH, RL, NM, SR), Data Curation (AH, NM), Formal Analysis (AH, NM), Funding Acquisition (KE), Investigation (KE, AH, RL, NM, SR), Methodology (KE, AH, RL, NM, SR), Project administration (KE, AH, NM), Supervision (KE), Validation (NM), Writing—original draft (KE, AH, RL, NM, SR), Writing—review & editing (KE, AH, RL, NM, SR).

Data Availability

Study data are available upon request per email to the corresponding author.