Impact of the Oklahoma IDeA Network of Biomedical Research Excellence research support and mentoring program for early-stage faculty

Abstract

The Oklahoma IDeA Network of Biomedical Research Excellence (OK-INBRE) provides a formalized mentoring program and grant awards to new and early-stage faculty throughout Oklahoma. The OK-INBRE Research Project Investigator (RPI) award program has supported 30 faculty from both research-intensive universities and primarily undergraduate institutions (PUIs) over the past 15 yr. To examine the impact of this program, we assessed the career trajectory of OK-INBRE RPI awardees and compared their productivity with a control group of applicants who applied for but did not receive an RPI award. A mixed-methods approach was employed to assess longitudinal programmatic impact. Regression analyses were conducted to estimate the effect of an RPI award on faculty productivity, controlling for institutional affiliation. Key informant interviews were conducted to capture qualitative information about satisfaction and additional outcomes. OK-INBRE RPI awardees had a higher number in total and mean number of publications. In achieving extramural funding, RPI awardees were 12.5 times (P = 0.005) as likely to receive a grant award of any type and 4.5 times (P = 0.06) as likely to receive a subsequent federal grant as those in the control group. Many RPI awardees attributed their career success to OK-INBRE, but they also helped to identify barriers to advancement or productivity associated with their specific home institutions. The combined data indicate that OK-INBRE plays a significant role in launching new and early-stage investigators on a path toward independent research careers, which will in turn have a positive impact on the future of the biomedical research enterprise in Oklahoma.

NEW & NOTEWORTHY The Oklahoma IDeA Network of Biomedical Research Excellence (OK-INBRE) has been offering a formalized mentoring program and grant awards to new and early-stage faculty throughout Oklahoma for the past 15 yr. The program has been shown to play a significant role in launching participants on a path toward productive research careers, which will in turn be impactful on the biomedical research enterprise in Oklahoma.

INTRODUCTION

Financial support for research endeavors and structured mentorship programs are both crucial to the career development and advancement of early-stage investigators as they strive to gain national-level funding and research independence. Research funding is vital in helping these faculty generate the preliminary data needed to develop competitive grant applications and launch successful research programs funded by national agencies (1, 2). Mentorship that is structured, informal, or even from peer support, can foster confidence and job satisfaction, establish research-focused laboratory programs, enhance career advancement, including successful navigation of the promotion and tenure process, and strengthen institutions’ ability to retain early-stage faculty in academia (3–6). Mentoring programs also benefit faculty in several other processes, such as onboarding and enculturation; setting career goals and milestones; improving time management skills; and facilitating work-life balance (7). Challenges resulting from the ever-changing research environment, however, require a thoughtful and systematic approach in both mentorship programs and funding that provide research support for faculty to generate the preliminary data needed to help them publish manuscripts and pursue extramural funding. In addition to other activities that build biomedical infrastructure within Oklahoma, the Oklahoma IDeA Network of Biomedical Research Excellence (OK-INBRE) program provides mentoring to higher education institutions to help them develop and sustain innovations, advance research faculty and students, and enhance their overall research core capacity.

The OK-INBRE program provides research support to new and early-stage faculty, as well as research opportunities for both undergraduate and graduate students. The program goals are to 1) support the research career development of promising investigators through competitively reviewed and awarded grant mechanisms, 2) enhance the pipeline and diversity of undergraduate students entering graduate programs via experiential student training initiatives at primarily undergraduate institutions (PUIs), community colleges, and the precollege level, 3) provide research infrastructure that supports both the research of the OK-INBRE investigators and statewide biomedical and educational research efforts, and 4) build the knowledge base in science and technology among the state’s biomedical research workforce. Mechanisms aimed at achieving these priorities were instituted across campuses of research-intensive universities, PUIs, and four community colleges throughout the state. One of the largest initiatives supported by the OK-INBRE program is the Research Project Investigator (RPI) award. The RPI program provides significant research funding and mentorship to promising new investigators to help guide their career development as they work toward securing extramural grant funding and publishing high-impact manuscripts supporting their research projects.

OK-INBRE has continually tracked and evaluated the progress of faculty supported through the RPI program. The purpose of this study was to assess the impact and outcomes of RPI funding on faculty who were awarded RPIs in comparison to their counterparts who applied for RPI awards but were not funded. The overall sample size of investigators funded and mentored through the OK-INBRE RPI system is now at a level that can be statistically assessed for the overall effectiveness and outcomes of this important program with a mixed-methods approach. Various elements of the RPI program and faculty activities and career progress were examined, and the combined quantitative and qualitative data allowed us to fully assess the impact of the OK-INBRE RPI program. Our underlying hypothesis was that the OK-INBRE RPI funding support to faculty provides an advantage with regard to manuscript productivity and successful research grant funding and aids in the career development of awardees compared with nonawardees.

METHODS

OK-INBRE Research Program Structure

OK-INBRE administers the RPI program to faculty in multiple research-intensive campuses and PUIs from all geographic regions of Oklahoma (Table 1). The OK-INBRE leverages the RPI program to meet one of its aforementioned goals to increase research infrastructure (e.g., equipment and core facilities) and build biomedical research capacity (e.g., increase number of faculty funded by, and working in, biomedical research areas). The RPI program awards multidisciplinary developmental research projects to investigators that are 2.5 yr in duration with an opportunity to competitively renew for an additional 2.5 yr. The maximum allowable funding period is 5 yr.

Table 1.

Institutions within the INBRE network

Institution	Location	Enrollment (students)
Research intensive
University of Oklahoma Health Sciences Center*	Oklahoma City (U)	3,624
University of Oklahoma	Norman (U)	31,678
Oklahoma State University	Stillwater (R)	25,594
Oklahoma Medical Research Foundation	Oklahoma City (U)	NA
PUI
University of Central Oklahoma	Edmond (U)	16,000
Langston University†	Langston (R)	3,000
Northeastern State University	Tahlequah (R)	8,100
Southwestern Oklahoma State University	Weatherford (R)	5,500
Southeastern State University	Durant (R)	5,200
Cameron University	Lawton (R)	5,000

This table presents the list of institutions, both research intensive and primarily undergraduate institutions (PUIs), in the IDeA Network of Biomedical Research Excellence (INBRE) network, along with location, urban (U)/rural (R) status, and enrollment as a proxy for size. NA, not available. *OUHSC is made up of professional and graduate schools. †Historically black institution of higher education.

All RPI applications undergo peer review by a panel of senior investigators. Reviewers are selected from the research-intensive institutions who have achieved National Institutes of Health (NIH) funding and have served on NIH study review panels. The selection meetings are carried out in the manner and style of an NIH study panel meeting. All applications are scored based on the NIH scoring system of 1–9 for specific criteria on significance, innovation, investigator, approach, and environment. Applicants receive thorough written critiques from at least two reviewers. Awards are provided to faculty strictly based on the final scores of the study section panel.

Up to eight applications are selected for RPI funding each funding cycle, which includes the four top-scoring applications from research-intensive institutions and the four top-scoring applications from PUIs. After the initial funding period, the awardees must submit a competitive renewal application that includes a research progress report and critiques from all applications submitted during their RPI award period to national funding agencies. The goal for RPI awardees at research-intensive institutions is to become independently funded with an NIH R01 or equivalent. PUI investigators with RPI awards are expected to provide an enriching laboratory research environment for undergraduates to perform biomedical research and demonstrate productivity through manuscript publications and submit grant applications to regional and national foundations, professional organizations, or statewide agencies.

In addition to the $100,000 direct costs per year research support to RPI investigators, the OK-INBRE program provides awardees with a structured mentoring program. Mentoring support consists of a mentor in their field who would give guidance to the RPI awardee on all aspects of their research and career development. A larger mentoring team consisting of senior researchers representing the multidisciplinary biomedical research themes of the OK-INBRE program also assists in providing feedback on their research projects and productivity on a biannual basis. This two-pronged approach ensures that RPI faculty have multiple resources, receive feedback on their research, and are educated about and prepared for potential extramural funding opportunities through the guidance of experienced researchers.

Analytical Approach

This study employed a mixed-methods approach to fully capture the impact of OK-INBRE RPI funding, as described below. This study was approved by the OUHSC Institutional Review Board (IRB no.: 7434). All participants provided verbal informed consent to be included in the study to provide information regarding program improvement.

Quantitative analyses.

Data collection and sample.

Data were generated and analyzed from a total of 63 OK-INBRE RPI applicants, 30 of whom were awarded RPIs. Demographic and institution information were extracted from the applications. Data for publications and grant awards were obtained both through direct contact with all applicants and by examination of publicly available funding award databases. Grant award support for all RPI applicants was documented with surveys in addition to NIH RePORTER, Federal RePORTER, faculty webpages, grant program webpages, information contained in OK-INBRE grant applications, and progress reports from OK-INBRE-supported faculty. External grant support captured included federal, state, industry, foundation, and association or professional society funding. Search limits and inclusion criteria applied were 1) funding obtained during the time period that began at the first year of RPI funding or the first instance of RPI application for nonawardees and continued for 10 yr thereafter and 2) grant funding through October 2018 among those who have not reached 10 yr after the initial RPI application or award. Subprojects awarded to an RPI applicant from a grant award led by another principal investigator were excluded from the analyses. Publication productivity was assessed by PubMed searches with name and affiliation identifiers. Data capture on publications was assessed for a 10-yr period after the RPI was awarded or after the first application for an RPI for the control group of nonawardees. The combined grant award and publication data were merged, processed, and cleaned. Initial descriptive statistics were compiled, plotted, and examined for outliers.

Measures.

Variables constructed for analysis include continuous and dichotomous measures. As the variable of interest aimed to test the effect of the RPI award on outcomes of productivity, holding an RPI award was assigned the value of 1. Variables describing RPI awardees and nonawardees include sex (male = 0; female = 1), type of institution (research intensive = 1; PUI = 0), and number of years after RPI award or application. Variables quantifying productivity include 1) total number of publications, 2) mean number of publications (calculated by dividing the total number of publications by the number of years after RPI award or application), 3) total number of external grants of all types, and 4) a dichotomous variable of success in securing at least one external grant (yes = 1; no = 0).

Data analyses.

Descriptive statistics on the OK-INBRE RPIs were tabulated for demographic and institutional data. As the type of institution and the variations in sizes and capacity among offices of sponsored programs between research-intensive and PUI institutions may impact the capability of RPIs to secure external funding, RPI awardees and nonawardees were stratified by institutions (research intensive vs. PUI) and differences were estimated within each subcategory.

This study is hypothesis driven and has a small sample size. Hence, variables were selected and specified a priori. Although the significance level was set at 0.05, per convention, we considered a P value of 0.10 to be marginally statistically significant, based on the whole context of the study, including scientific merit, aims, methods, and limitations (8). Given the distribution of samples, as stratified by institutional affiliation, we first examined the difference in productivity, using the Wilcoxon rank-sum test. Because of the nonparametric distribution with a significant proportion of the sample posting zero values, the Wilcoxon rank-sum test assigns a rank for each RPI awardee or nonawardee based on their number of publications, where the higher the ranking, the higher the number of publications. The rankings were then summed for each group and the Wilcoxon test examined if medians of the rankings between the RPI awarded and control groups differed statistically. Multiple regression analyses were conducted to model the total number of publications as a function of RPI award, institutional affiliation, and sex, controlling for the number of years after award or application. The same model was applied to the outcome mean number of publications (9, 10).

Multiple logistic regression analysis estimating the likelihood of success in securing external grant funding was modeled as a function of RPI award, institutional affiliation, and sex, controlling for the number of years after award or application (10, 11). Odds ratios were calculated on the dichotomous variable indicating whether or not grant funding was secured (yes = 1; no = 0). All analyses were conducted with SAS v.9.4 (SAS Institute, Cary, NC).

Qualitative analysis.

Key informant interviews.

To understand the full impact of the OK-INBRE RPI program on faculty research productivity and career trajectory, all RPI awardees and nonawardees were invited to participate in the qualitative portion of this study as key informants. A follow-up e-mail/phone call was delivered if the first e-mail invitation did not yield a response. Once a positive response was received, the OK-INBRE evaluator scheduled and conducted the key informant interview via the telephone or in-person site visit. All interviews were audio-recorded and transcribed. All participants received a $50 check for their participation. Twenty-seven interviews were conducted, and saturation was achieved after 20.

Key informant questions.

The semistructured questions differed between the two groups of key informants, awardees and nonawardees. Both sets of questions captured professional information, such as current title and institutional affiliation, number of years at their institution, and the institution from which they applied. Among the RPI awardees and nonawardees, each was asked about the application process, overall experience with OK-INBRE, benefits and barriers of the program, career trajectory, and suggestions for program improvement.

Data analyses.

Transcripts of the key informant interviews were analyzed with standard content analytic techniques and grouped or clustered into major themes (12). Emerging themes were examined and discussed with the study team by comparing individual themes within and across interviews to achieve consensus. This is an iterative, constant-comparative process to identify overarching themes and patterns on mentoring, research, and career advancement, as well as feedback on the OK-INBRE program. The analysis was conducted by INBRE’s internal evaluator, and the results were discussed within the INBRE group to achieve consensus. Results of quantitative and qualitative inquiries were systematically triangulated to gain greater insights.

RESULTS

In the study sample of 63 RPI applicants, awardees who were included in this study were all funded between 2004 and 2016. The sample included 21 women (33%), and 29 (46%) were faculty at research-intensive institutions at the time of application. Additionally, 48% (n = 30) of the sample received an RPI award, where 44% (n = 28) received their RPI between 2011 and 2016 and 56% (n = 35) were awarded between 2004 and 2009. Among the 33 nonawardees, 36% (n = 12) applied for their awards between 2011 and 2016 and 64% (n = 21) applied between 2004 and 2009. Among the 30 RPI awardees, 20% (n = 6) received the award after their second application and 3% (n = 1) were successful with their third application.

Recognizing the possible differences in resources and infrastructure, a number of awards are reserved for PUI applicants. Table 2 shows the number of applicants from each institution and the percentage of those who received a RPI. Of the 34 applicants from the six PUI institutions, an overall average of 59% of the PUI applicants received an RPI. Of the 29 applicants from research-intensive institutions, the overall average for receiving an RPI award was 34%.

Table 2.

RPI awards by institution

Institution	RPI Awards
	Total Applicants	RPI, n	RPI, %
Research intensive
A	21	8	38
B	4	0	0
C	2	2	100
D	2	0	0
Total	29	10	34
PUI
A	10	5	50
B	9	7	78
C	8	4	50
D	3	2	67
E	3	1	33
F	1	1	100
Total	34	20	59

Total number of applications for Research Project Investigator (RPI) from each institution and number of RPI awards (%) awarded are listed. PUI, primarily undergraduate institution.

Quantitative Results

Productivity was examined in the context of published manuscripts and securing extramural funding. In terms of manuscript production, the median rankings based on either the total or mean number of publications for RPI awardees were significantly higher than those of nonawardees. For the total sample group (n = 63), 27% (n = 17) reported no publications whereas the majority (52%, n = 33) have 1–10 published manuscripts, with a median number of 6 (Table 3). As shown in Fig. 1, RPI awardees ranked higher in the total number of publications in comparison to nonawardees. Similarly, awardees ranked higher in mean number of publications compared with nonawardees. Regression results showed that those who received an RPI increased their productivity by 5.7 in number of publications with marginal significance (P = 0.06) and 0.9 in mean number of publications with statistical significance (P = 0.05). However, the most significant association with the number of publications is the affiliation with research-intensive institution, where awardees in research-intensive institutions increased the number of their publications by 10.21 (P = 0.001). The same factor is associated with the mean number of publications (Table 4).

Table 3.

Characteristics and career achievement of RPI faculty

	N	%
Demographics and affiliation (n = 63)
Sex
Female	21	33
Male	42	67
Research-intensive institution	29	46
Primarily undergraduate institution	34	54
Career achievement (n = 63)
RPI awardee	30	48
Number of total publications
Range (0–64)
0	17	27
1–10	33	52
11 or more	13	21
Received at least 1 grant of any type	26	41
No. of research grants
R mechanism
0	42	67
1	11	17
2	5	8
3 or more	5	8
All other grant types*
0	49	78
1	8	13
2	2	3
3 or more	4	6

Sample characteristics of all Research Project Investigator (RPI) award applicants and descriptive statistics of publication and grant award records. *Includes federal, industry, other government, and professional society.

Figure 1.

Nonparametric distribution of number of publications between Research Project Investigator (RPI) awardees and nonawardees. Because of the nonparametric distribution, the Wilcoxon rank-sum test assigns a rank or score for each RPI awardee (ranking distribution on left) or nonawardee (ranking distribution on right) based on the total number of publications, where the higher the ranking the higher the number of publications. The rankings were then summed for each group. The medians, represented by the diamond symbol in the boxplot, of the rankings between the 2 groups were statistically different, with the median rankings (e.g., greater total number of publications) for the RPI awardee group higher than that of the nonawardee group. The Z score and its significance indicate the statistical significance between the rankings of the 2 groups.

Table 4.

Effects of RPI award on publication productivity

	Parameter Estimates (SE)	P Value
Total no. of publications
Intercept	−4.48 (5.17)	0.39
RPI	5.68 (2.98)	0.06
Research-intensive institution	10.2 (2.97)	0.001
Sex	3.63 (3.12)	0.39
Years after award/application	0.45 (0.52)	0.25
Mean no. of publications
Intercept	0.16	0.84
RPI	0.91 (0.46)	0.05
Research-intensive institution	1.62 (0.45)	0.0007
Sex	0.68 (0.48)	0.45
Years after award/application	−0.06 (0.08)	0.16
Model fit
R 2	0.20
Adjusted R2	0.15

Findings from multiple regression analyses estimating the outcome variables total number of publications and mean number of publications as a function of Research Project Investigator (RPI) award, institutional type, controlling for sex and number of years since RPI award, are reported. RPI awards have a significant and positive impact on the total and mean number of publications, although affiliation with a research-intensive institution had an even greater impact.

With regard to successful external grant applications, for the total sample group (n = 63), 26 (41%) obtained funding of any kind. Among these, 11 obtained one grant award and 5 obtained two grant awards (range of number of grants = 1–7). Across different types of grant mechanisms, 21 of these secured an NIH R-type of award (e.g., R03, R21, R01) (Table 3). As shown in Fig. 2, at research-intensive institutions 90% of RPI awardees compared with 47% nonawardees were able to secure at least one grant. Similarly, among those at PUIs 35% of RPI awardees were able to obtain at least one grant of any kind, compared with only 7% of the nonawardees. Examining the effect that an OK-INBRE RPI award had on the total number of grants received, we observed that 20% of the RPI awardees had at least one successful grant application, compared with 15% of nonawardees. No nonawardee was able to obtain 4 or more grants, whereas 10% of RPI awardees reported 4–9 successful grant applications and 7% reported 10 or more (Fig. 3).

Figure 2.

Receiving a grant by institution and Research Project Investigator (RPI) status. Descriptive statistics were generated to tabulate % of the sample receiving a grant of any type by type of institution [research intensive vs. primarily undergraduate institution (PUI)] and RPI award. Among those in research-intensive institutions, 90% of RPI awardees received a grant of any type compared with 47% of nonawardees. This difference is statistically significant [Cochran–Mantel–Haenszel (CMH) statistics = 4.88, P = 0.027]. Among those at PUIs, 35% of RPI awardees received a grant of any type compared with 7% of nonawardees, and the observed difference is marginally statistically significant at P = 0.063 (CMH statistics = 3.45).

Figure 3.

The effect of Research Project Investigator (RPI) award on number of grants obtained. Among a subsample of those who had secured grant funding, Oklahoma IDeA Network of Biomedical Research Excellence (OK-INBRE) RPI awards demonstrated impact on the total number of grants received. Twenty percent of the RPI awardees had at least 1 successful grant application, whereas only 15% of faculty nonawardees received a subsequent award. No nonawardees were able to obtain 4 or more grants, whereas 10% of RPI awardees reported 4–9 and 7% reported 10 or more successful grant applications.

Odds ratios derived from multiple logistic regression analyses showed that RPI awardees were 4.46 times [95% confidence interval (CI): 0.94, 21.2, P = 0.06] as likely as nonawardees to receive a federal grant and 12.5 times (95% CI: 2.13, 73.6, P = 0.005) as likely to receive a grant of any type. The impact of a research-intensive institutional affiliation is even more remarkable in this regard, with an odds ratio of 16.4 (95% CI: 3.24, 83.49, P = 0.0007) and 20.8 (95% CI: 3.51, 123.4, P = 0.0008) for obtaining a federal grant or grant of any kind, respectively, compared with PUIs (Table 5).

Table 5.

Effects of OK-INBRE RPI awards on success in obtaining grant awards

	Odds Ratio (95% confidence interval)	P Value
Federal grants (R mechanism)
RPI	4.46 (0.94, 21.15)	0.06
Research-intensive institution (reference = no)	16.4 (3.24, 83.5)	0.0007
Sex (reference = male)	1.27 (0.34, 4.80)	0.72
Years after award/application	1.30 (0.99, 1.71)	0.06
All grant types*
RPI	12.5 (2.13, 73.9)	0.005
Research-intensive institution (reference = no)	20.8 (3.51, 123.3)	0.0008
Sex (reference = male)	1.26 (0.34, 4.71)	0.73
Years after award/application	1.27 (0.98, 1.65)	0.08

Test for global null hypothesis (beta = 0)	χ2	DF	P > χ2
Likelihood ratio	18.7	4	0.0009
Score	15.8	4	0.0033
Wald	11.6	4	0.0202

Findings from logistic regression analyses estimating the outcomes variables likelihood of securing a federal grant and that for all grant types as a function of Research Project Investigator (RPI) award, institutional type, controlling for sex and number of years since RPI award, are reported. RPI awardees and those affiliated with research intensive institutions were more likely to obtain a grant of any type (federal and all others) than nonawardees. OK-INBRE, Oklahoma IDeA Network of Biomedical Research Excellence. *Includes federal, industry, other government, and professional society.

Qualitative Results

Qualitative information generated with key informant interviews allowed us to better assess achievements of RPI awardees compared with nonawardees. These interviews also provided detailed information that helped to inform interpretation of the quantitative results. Four major themes emerged from the synthesis of interview transcripts, which were as follows: theme 1—all key informants indicated the RPI application process to be fair and helpful; theme 2—among awardees, perceived benefits of the RPI award differed by the type of institution with which awardees were affiliated (PUI recipients highlighted release time from teaching as a key benefit of the OK-INBRE RPI program, whereas faculty at research-intensive institutions identified mentoring and coaching in research as a key benefit of the RPI program); theme 3—RPI awardees attributed much of their career success to OK-INBRE; and theme 4—barriers to advancement or productivity were mostly attributed to factors associated with specific issues at their home institutions, but many expressed the need for OK-INBRE to identify or develop mechanisms that could help bridge funding gaps. Results of the four themes that emerged from the qualitative key informant interviews are outlined below.

Theme 1.

Regardless of award status, all key informants reported that the RPI application process is both fair and easy to navigate. They found reviewer critiques to be the most helpful and appreciated that this application process mirrors that of the NIH application, in both its overall operational process and grant review system. However, some RPI applicants suggested that it might help to improve the match between primary reviewers’ expertise and the proposed research topic when possible. At least two key informants indicated that they believed that their applications were scored more poorly than they had expected because their research topics were particularly specialized and that possibly the review panel did not have perfectly matching expertise. A suggestion was made to identify possible ad hoc reviewers whose expertise would align with applicant projects to help in the review process and provide more relevant feedback on applications with unique research topics that are not well covered among the established faculty reviewers in Oklahoma.

Theme 2.

Among all RPI recipients regardless of institutional affiliation, elements of the RPI program that had the largest impacts were mentoring and career development coaching, the opportunity to network, funding support, the ability to involve students, and research infrastructure improvement. RPI awardees cited the single most important benefit as the uninterrupted funding to focus on their research agenda. In particular, awardees from PUIs cited release time from the classroom as the most critical component of RPI awards, as they were able to use that time for research and build a research portfolio that they would not otherwise be able to accomplish with their typical teaching responsibilities. In addition, the opportunity to present their research and receive feedback for their work as well as networking were deemed of great value for awardees at PUIs. As one key informant noted, “What’s useful is to get to know people from other campuses; we were getting educated. Sometimes, you can bring back an idea from another campus and say, why don’t we do this? Both scientifically and as a faculty member trying to juggle things.” Mentorship made a significant difference for those in research-intensive institutions. Many of those RPI awardees indicated that having a mentor who has received NIH funding to guide them through developing their research programs and the application process facilitated their progress in becoming nationally funded and fully independent researchers. On the other hand, it was also noted that for PUI awardees the distance from their mentors, who tended to be based in the urban centers, posed some access restrictions.

Theme 3.

The benefits of RPI awards translated into an upward career trajectory. All RPI awardees have been promoted and received tenure on their expected timelines. Many credited the OK-INBRE RPI award as the most important “launch pad” for their careers because of the mentorship and protected time OK-INBRE provided. Those in PUIs pointed to RPI funding as a way to expose students to research and opportunities that they would otherwise not have. Protected time allowed PUI awardees to mentor students by supporting them financially to work in a laboratory, develop curriculum, write manuscripts, and prepare grant applications. PUI faculty indicated that through their RPI-supported work they mentored and created opportunities for students outside of the classroom, each reporting in the range of 40–70 students with whom they worked, since the receipt of their RPI award. Many of these students went on to pursue advanced degrees in sciences (13). PUI faculty also credited the RPI program for the sense of confidence in their capability to succeed in research, as the mission of PUIs does not necessarily focus on research endeavors. A key informant from a PUI credited the RPI program for opportunities, “Whether it’s research or administration, it’s having the ability to do research, even on a small scale. Even though I have moved on to be an administrator, INBRE helped me keeping up that confidence. I still have the ability to do research, write, and train. It opened up a lot of doors for me. Applause, applause for INBRE!” Another key informant independently echoed, “RPI’s guarantee [of funding] is substantial. It gives you the confidence that ‘I can do it.’” Many faculty members from research-intensive institutions also credit much of their success in securing NIH funding to their RPI awards and mentoring experience. One former RPI recipient who has obtained multiple NIH grants but has since moved on to another research-intensive institution for family reasons stated, “I really owe a debt of gratitude to the program. I had the ideal experience. It does work; it can work, and it worked for me.”

Theme 4.

Suggestions for improvement for the RPI program mostly pointed to barriers related to logistics that varied depending on the home institution of individual faculty. For example, a number of PUI-based faculty pointed to the difficulty with the administrative grant process and timely release of funds on their campuses so that they could pay for animal upkeep and equipment. Many PUI RPIs do not have access to additional bridge funding when the RPI award ends if they have been unable to secure additional external awards. They appreciated that there is some bridge funding available from OK-INBRE, but those current programs pose constraints at their institution’s administrative level that, at times, they could not overcome. They encouraged OK-INBRE to consider remedies to bridge periods of lapsed funding to better support career advancement for PUI faculty who had received RPI awards. However, one PUI-based faculty who had achieved success in securing a number of NIH grants based on his RPI award specified leveraging the “layered funding mechanisms” (e.g., mini-grants, equipment, and collaborative grants) provided by OK-INBRE to continuously build a research program. He would use the alternative funding mechanisms to provide continuity whenever he had a break between extramural funding agencies, which enabled him to continue data generation to lay the ground work for future grant applications.

DISCUSSION

The results of our study are consistent with the OK-INBRE RPI award mechanism having achieved its intended objectives to impact the research environment and research capacity in Oklahoma. The OK-INBRE RPI funding award program has provided opportunities and promoted success for researchers to obtain external funding support, produce publications, and become independent investigators. Awardees of the OK-INBRE RPI program, with funding support and structured mentorship, demonstrated significantly higher productivity and extramural funding compared with their nonawardee counterparts. Combined, the RPI awardees had a mean total of 9.4 publications compared with the control group, which had a mean total of 6 publications, a difference of almost 40%. Collectively, RPI awardees obtained a total of 28 grants (mean = 0.9), whereas nonawardees secured 15 (mean = 0.5). These findings apply to awardees at both research-intensive institutions and PUIs. Taking into account that at baseline between RPI awardees and nonawardees there was no significant difference in mean number of publications (both nonawardees and awardees = 0.79) and mean number of grants (awardees = 0.2 and nonawardees = 0.3), particularly among PUI applicants, the greater increase in both of these indicators among the RPI awardees compared with those of nonawardees demonstrated the impact of RPI support. These findings are also consistent with those previously reported in the literature. As an example, Farrokhyar et al. (2) found that research investments, even small seed funding, are “effective for scientific productivity and professional growth of novice investigators and production of research with new initiatives and innovations.” Additionally, much of the previous literature focuses on the role of mentorship in the achievements of new investigators. It was clear that mentoring relationships have a significant impact on the generation of future extramural awards by faculty, better focus faculty on their research careers, and help to increase retention not only at the home institution but also within academia and the research environment in general (3–5, 14–17). Moreover, these effects on the success of junior investigators persist over time, and this was also identified for the OK-INBRE RPI awardees. This is consistent with a report by Libby et al. (6) indicating that institutional investment in mentored research training for early-career faculty began to show benefits in terms of the number of grants awarded 1 yr after program participation and this positive trend maintained over time.

Our findings also show that types of resources were valued differentially and were highly dependent on the home institution of the OK-INBRE RPI project investigator. In their qualitative responses, whereas RPIs at research-intensive institutions cited the mentoring component of the OK-INBRE program to be a significant factor for their success, RPIs at PUIs credited the funding for teaching release time, which allowed them time to focus on their research. This difference between the perspective of RPI faculty at PUIs, located mostly in rural areas of Oklahoma, and that of faculty at a research-intensive institution, situated in an urban center, may in part be due to OK-INBRE mentors being centrally located in the state. The long distance, as identified through interviews with PUI faculty, created some barriers to RPIs at PUIs in having convenient access to their mentors. Given the obvious importance of mentorship that we found in our present study, which has also been noted by others in the literature (3, 4, 6, 15), the OK-INBRE program is currently exploring opportunities to strengthen the mentoring program for PUI faculty, such as customizing the mentored research training according to the setting in which the investigator conducts her/his work. OK-INBRE designates specific mentors to work with each funded RPI investigator, along with help from a larger mentoring committee and a guiding Scientific Theme Director for each of the mentoring programs. A potential redesign in this system would entail pairing the research-intensive mentor with a mentor counterpart at a PUI. In particular, OK-INBRE aims to continue building and sustaining the research and mentoring infrastructure at PUIs. These efforts will require a continued dialogue and close working relationship with leadership at each PUI to foster an environment conducive to supporting research for RPI awardees and nonawardees. An expanded peer mentoring program could further enhance the experience for RPI awardees at PUIs, as peer mentoring has been shown to increase productivity and self-efficacy (18) and has proven particularly effective for underrepresented subgroups such as women (19). Additionally, many in the OK-INBRE RPI program (funded and not funded) have indicated that workshops could be of great utility. In this regard, a series of formalized workshops will be designed and offered to enhance development of important skill sets for early-stage investigators (e.g., scientific writing, grant writing, laboratory management) and help RPI awardees navigate the promotion and tenure process (4). Such workshops will also allow OK-INBRE to introduce more opportunities for networking, which also may lead to increased peer mentoring, providing a “sounding board” for participants to maintain the momentum for funding and to learn how to better balance research and teaching demands. Finally, OK-INBRE will develop a plan to include current and prior RPIs on future OK-INBRE peer review study sections so that they may observe how grant applications are reviewed and critiqued. These combined strategies are simple to establish, low cost, and financially sustainable and could provide significant and enduring impact on the growth and success of faculty investigators in Oklahoma.

This study contributes several significant observations and adds to the current literature as it offers a comprehensive evaluation of a well-established, structured, multi-institutional training program with robust data from well-documented outcome metrics. The strength of this study lies in the rigor of its design. First, the mixed-methods approach enhanced the validity of results, where the qualitative themes complemented the interpretation of quantitative findings. Second, the availability of longitudinal data over a 15-yr period with a control group allowed us to detect and measure the magnitude of the OK-INBRE RPI program effect on productivity, grant awards, and career advancement. Third, the purposeful sampling of RPI awardees and nonawardees as key informants ensured that our results were “information rich” (20). Fourth, the triangulation of the data ensured internal validity, with iterative coding procedures that examined how the codes related to larger thematic findings of the study (21).

On the other hand, this study was also limited by data availability. First, the data lacked additional demographic variables other than sex and descriptions of specific institutional affiliation in many instances. The outcome variable measuring productivity only included those that have been reported by RPIs or derived from publicly available databases. It is possible that additional information such as nonindexed publications or institutional level grant awards exist that were not included in the present analysis. Nevertheless, neither of these factors would likely play a major role in career advancement since peer-reviewed publications and extramural grant awards are typically more important to tenure, promotion, and overall faculty advancement. Second, achievements of RPI awardees and nonawardees may not have been perfectly equivalent at baseline, engendering possible selection bias. To further validate the impact of the program, we conducted the analyses comparing the groups on sex, prior funding, and prior publications at baseline. Our findings showed that there was no significant difference between males and females. The sex variable was an independent variable in both the logistic and regression analysis examining the likelihood of obtaining a grant of any kind and number of publications. Moreover, there is a set of criteria on which the RPI applicants were evaluated. Their prior success in publishing and securing smaller grants were merely two of the many elements for evaluation. As a rule, the applicants would not be eligible if they had secured a federal R-mechanism grant before their application and award. As mentioned above, there was no significant difference in the number of publications and grants between awardees and nonawardees at baseline. Although it was likely that RPI awardees and nonawardees are not entirely comparable, the selection committee did its best to evaluate qualifications of each applicant as a package.

In the highly competitive environment of biomedicine and science, new and early-stage faculty investigators are facing increasing challenges in securing grant funding and obtaining resources to support their career development and research productivity (22). Freel et al. (15) and Alberts et al. (23) have demonstrated research funding challenges and their negative impact on early-stage investigators. Compared with investigators 30 yr ago, the success rate for NIH R01-equivalent grants in the mid-2010s had dropped nearly 15% and early-stage investigators in the same comparative time frame tended to be 4–6 yr older when they received their first independent NIH R series grant. This study has shown that OK-INBRE, with its emphasis on mentorship and research support funding for up to 5 yr, has significantly increased faculty development and moved them on the path to independent research careers. These combined outcomes from this comprehensive analysis of the OK-INBRE program indicate that an RPI award ultimately had a profound impact on the careers of faculty, the biomedical research enterprise, and overall health and economic well-being for future generations of Oklahomans.

GRANTS

This work was supported in part by National Institute of General Medical Sciences Grant P20GM103447 to D.R.A. and the Oklahoma Tobacco Settlement Endowment Trust awarded to the University of Oklahoma, Stephenson Cancer Center.

DISCLOSURES

No conflicts of interest, financial or otherwise, are declared by the authors.

AUTHOR CONTRIBUTIONS

A.F.C. conceived and designed research; A.F.C. analyzed data; A.F.C., D.H., and D.R.A. interpreted results of experiments; A.F.C. prepared figures; A.F.C. and D.H. drafted manuscript; A.F.C., D.H., and D.R.A. edited and revised manuscript; A.F.C., D.H., and D.R.A. approved final version of manuscript.