Development and Evaluation of Two Abbreviated Questionnaires for Mentoring and Research Self-Efficacy

Abstract

Objectives

To reduce respondent burden for future evaluations of the National Heart, Lung, and Blood Institute-supported Programs to Increase Diversity Among Individuals Engaged in Health-Related Research (PRIDE), a mentored-research education program, we sought to shorten the 33-item Ragins and McFarlin Mentor Role Instrument (RMMRI), measuring mentor-role appraisals, and the 69-item Clinical Research Appraisal Inventory (CRAI), measuring research self-efficacy.

Methods

Three nationally recruited, junior-faculty cohorts attended two, annual 2-3 week Summer Institutes (SI-1/SI-2: 2011/2012, 2012/2013, 2013/2014) at one of six PRIDE sites. Mentees completed the RMMRI two months after mentor assignment and the CRAI at baseline (pre-SI-1) and 6-month (mid-year) and 12-month (post-SI-2) follow-up. Publications data obtained from Scopus in October 2015 were verified with mentees’ curriculum vitae. The RMMRI and CRAI were shortened using an iterative process of principal-components analysis. The shortened measures were examined in association with each other (multiple linear regression) and with increase in publications (repeated-measures analysis of covariance).

Results

PRIDE enrolled 152 mentees (70% women; 60% Black, 35% Hispanic/Latino). Cronbach’s alphas for the new 9-item RMMRI, 19-item CRAI, and four CRAI-19 subscales were excellent. Controlling for baseline self-efficacy and cohort, RMMRI-9 scores were independently, positively associated with post-SI-2 scores on the CRAI-19 and three subscales (writing, study design/data analysis, and collaboration/grant preparation). Controlling for cohort, higher RMMRI-9 and post-SI-2 CRAI-19 scores were each associated with greater increase in publications.

Conclusions

The RMMRI-9 and CRAI-19 retained the excellent psychometric properties of the longer measures. Findings support use of the shortened measures in future evaluations of PRIDE.

Introduction

The National Institutes of Health (NIH) has called for increasing the diversity of the biomedical research workforce,1,2 which has inspired the development of diversity-focused educational and research-training programs at academic institutions across the United States.3-7 These programs aim specifically to promote the recruitment and retention of individuals from underrepresented groups in science, medicine, and biomedical-research careers (eg, women especially at senior levels, persons from racial/ethnic minority groups, disadvantaged backgrounds, or with disabilities). A diverse biomedical-research workforce is important, because a diversity of perspectives is expected to yield a broader spectrum of novel research questions for studying disease risk, pathogenesis and outcomes, response to treatments, and ways to reduce health disparities.7-10

A growing body of evidence indicates that mentors can promote junior faculty members’ professional development and positively support their perceived self-efficacy.11-14 Successful mentoring relationships are determined, in part, by mentees’ perceptions of their mentors’ roles,15 as mentees may have multiple mentors, each playing a different role.16 The Ragins and McFarlin Mentor Role Instrument (RMMRI),17 developed based on mentor role theory,16 focuses on psychosocial and career-advancement roles. But to our knowledge, this reliable measure has not been examined in association with improvements in research self-efficacy or publications.

In 2010, the National Heart, Lung, and Blood Institute (NHLBI) funded six diversity-focused, mentored-research education programs for early-career investigators engaged in heart, lung, blood, and sleep research. This initiative, called PRograms to Increase Diversity among Individuals Engaged in Health-related Research (PRIDE),4 offered research education and skills training during two annual summer institutes, with individual mentoring throughout the one-year training period. PRIDE mentees were matched with one or more experienced faculty mentors whose research interests and expertise were well-aligned with mentees’ needs. PRIDE mentors, including the primary research mentor, were from the mentee’s home institution, summer-institute site, or elsewhere. As part of the PRIDE evaluation, mentees were asked to complete two questionnaires—the 35-item RMMRI,17,18 a measure of mentor-role appraisals, and the 69-item Clinical Research Appraisal Inventory (CRAI),19 a measure of research self-efficacy. Self-efficacy is a person’s belief that he/she has the ability to complete actions to achieve specific behavioral goals.20 Social cognitive career theory21 contends that having social support and few social barriers to achieving career goals promotes self-efficacy and career goals.22 There remains a paucity of research about self-efficacy in relation to outcomes expectations and interventions for diverse groups,22-24 and no studies about self-efficacy and mentoring support for research productivity. Thus, we hypothesized that positive mentees’ appraisals of their PRIDE mentors’ supportive roles would be associated with an increase in mentees’ research self-efficacy (Hypothesis 1) and that greater research self-efficacy and positive mentor-role appraisals would be associated with an increase in publications after PRIDE training (Hypothesis 2).

Because the RMMRI and CRAI questionnaires are completed several times during and after PRIDE, we sought to develop shorter versions to reduce respondents’ burden for future use. This article reports on the item-reduction process for each questionnaire and on tests of our hypotheses using the shortened questionnaires.

Methods

Early-career investigators (with no prior R01 or equivalent research award) from underrepresented groups in biomedical research (based on race/ethnicity, disability or disadvantaged status) were eligible for PRIDE. Three cohorts (2011-2012, 2012-2013, and 2013-2014) for each of six PRIDE programs (at Columbia University, New York University, SUNY Downstate Medical Center, Augusta University, Washington University School of Medicine in St. Louis, and San Diego State University) were recruited nationally. The PRIDE Coordination Core at Washington University School of Medicine assists PRIDE programs with their competitive application review, applicant selection, data collection, and data management for evaluation.

Although each PRIDE program had a different heart, lung, blood, and sleep research focus, all six shared common training elements (ie, didactic and/or laboratory experiences specific to each program’s research emphasis; training in study design, data analysis, and responsible conduct of research; and mentoring for their research, grant writing, and career planning) and similar program structure (ie, attending two, onsite, 2-3 week Summer Institutes [SI-1 and SI-2], an onsite mid-year meeting, and an annual scientific meeting in Bethesda, Maryland promoting cross-program networking). Mentees worked closely with their mentor(s) during and after SI-1 and SI-2. The Institutional Review Boards at Washington University School of Medicine and at each PRIDE site approved the participation and data-collection activities for evaluation, and mentees provided written informed consent.

Measures

After consenting, mentees provided their curriculum vitae (CV) and completed questionnaires regarding demographics and educational backgrounds before SI-1 (baseline). Mentees are surveyed annually for 10 years to measure long-term outcomes. Publications data were obtained from Scopus in October 2015, at least one year after SI-2, and verified with mentees’ CVs.

We measured mentees’ research self-efficacy using the 69-item CRAI,19 at baseline (pre-SI-1), six months later, and again at 12-month follow-up (post-SI-2). Response options for the CRAI items ranged from No confidence (0) to Total confidence (10). Higher mean scores indicate greater confidence in one’s research self-efficacy.

We measured mentees’ appraisals of their PRIDE mentor’s roles using the RMMRI17,18 two months after the mentor was assigned (giving mentees and mentors time to interact). As described elsewhere,18 33 items were used to create an overall score and 11, 3-item subscale scores. Two single items measuring satisfaction with, and the effectiveness of, their PRIDE mentor are not discussed further. RMMRI response options ranged from Strongly Disagree (1) to Strongly Agree (7). Higher mean scores indicate more favorable appraisals of mentors’ roles. We added a “Not applicable” response option to this questionnaire for evaluation of the PRIDE mentors, because we believed some items would not be relevant to PRIDE mentors, who usually were not at the mentee’s home institution (eg, “Uses influence in the organization for my benefit”). The “Not applicable” option allowed us to determine which, if any, items did not reflect a PRIDE mentor’s role.

Analysis

We report frequencies (%) to describe the characteristics of the sample and each cohort and report means (SD) of the shortened RMMRI and CRAI measures. Tests were performed using IBM SPSS Statistics©, Version 24.0 (IBM Corporation, 1989, 2016). Two-sided P values <.05 were considered significant.

To reduce the number of items on the 33-item RMMRI and 69-item CRAI, we used an iterative process of exploratory principal components analysis (PCA) with varimax rotation for each measure.25 We used eigenvalues >1.000 as the criterion for determining the number of factors, retaining items that loaded ≥.600 on a single factor and dropping items that loaded ≥.400 on multiple factors.26 At each step, we also retained certain items with particular relevance to the mentoring and research-skills mission of PRIDE, regardless of factor loadings, as factor loadings can change after other items are dropped. We computed Cronbach’s alpha to measure the internal consistency of items that loaded on each factor and dropped items if alpha could be increased by eliminating those items. Mean scores for the shortened RMMRI and CRAI measures were computed if at least half the items on the scale were completed by the mentee. We used average measures intraclass correlations (ICC) to examine concordance between the original and shortened versions of the RMMRI and CRAI measures.

We examined whether the shortened CRAI was sensitive to change over time (pre-SI-1, six months, and post-SI-2) using repeated-measures analysis of variance. To test Hypothesis 1, we used multiple linear regression analysis to evaluate the associations between the shortened measures of mentor-role appraisals and post-SI-2 research self-efficacy, controlling for baseline research self-efficacy and cohort (1, 2, or 3) at step 1, then adding the mentor-role appraisals at step 2. To test Hypothesis 2, we examined the change in total number of publications from each mentee’s baseline (pre-SI-1) through October 2015 (at least one year after PRIDE) using repeated-measures analysis of covariance (RM-ANCOVA), by cohort (to control for length of time after PRIDE), to determine whether increase in publications varied by mentees’ post-SI-2 research self-efficacy and by their PRIDE mentor-role appraisals.

Results

Of 460 PRIDE-program applicants screened across all cohorts, 300 were eligible, and 254 were asked to submit a full application based on their prior academic background/training and research interests. Of these, 163 were invited to participate,158 mentees enrolled, and 152 from 100 medical schools/academic institutions in the United States and Puerto Rico completed the program. Baseline characteristics of PRIDE mentees are shown in Table 1.

Table 1. Baseline characteristics of PRIDE mentees.

Characteristics	Total, N = 152 (%)	Cohort 1, 2011-2012, n = 44 (%)	Cohort 2, 2012-2013, n = 47 (%)	Cohort 3, 2013-2014, n = 61 (%)
Sex
Male	46 (30.3)	15 (34.1)	14 (29.8)	17 (27.9)
Female	106 (69.7)	29 (65.9)	33 (70.2)	44 (72.1)
Race
African American	91 (59.9)	31 (70.5)	27 (57.4)	33 (54.1)
American Indian/Alaska Native	9 (5.9)	4 (9.1)	2 (4.3)	3 (4.9)
Pacific Islander	1 (.7)	0 (.0)	0 (.0)	1 (1.6)
Asian	3 (2.0)	1 (2.3)	2 (4.3)	0 (.0)
Multiple/non-White/no response	22 (14.5)	4 (9.1)	8 (17.0)	10 (16.4)
White	26 (17.1)	4 (9.1)	8 (17.0)	14 (23.0)
Hispanic/Latino ethnicity
Yes	53 (34.9)	9 (20.5)	21 (44.7)	23 (37.7)
No	99 (65.1)	35 (79.5)	26 (55.3)	38 (62.3)
Disability
Yes	2 (1.3)	1 (2.3)	0 (0)	1 (1.6)
No	150 (98.7)	43 (97.7)	47 (100.0)	60 (98.4)
Degree
Doctorate (eg, PhD)	90 (59.2)	24 (54.5)	28 (59.6)	38 (62.3)
Medical (eg, MD, DO)	56 (36.8)	17 (38.6)	18 (38.3)	21 (34.4)
MD/PhD	6 (4.0)	3 (6.8)	1 (2.1)	2 (3.3)

RMMRI and CRAI Item Reduction

In shortening the RMMRI, we first dropped 19 items to which >10% (range, 13.8-47.4%) of mentees responded “Not applicable.” Starting with 14 of the original 33 items,18 the PCA yielded a single-factor solution. We dropped five more items that seemed to be redundant. The final, 9-item factor (RMMRI-9) included items judged to be supportive-mentorship behaviors that PRIDE mentors could reasonably provide during and after program participation (Table 2). Cronbach’s alpha was .938, indicating very high internal consistency of these items.27 The RMMRI-9 and 33-item RMMRI scores were highly concordant (ICC=.952), supporting the criterion validity of the RMMRI-9. Mean (SD) RMMRI-9 scores were high at 6.0 (.9).

Table 2. Single-factor solution of the RMMRI-9 after principal components analysis, n=97.

My mentor…	RMMRI-9a (Cronbach’s alpha=.938)
1. Gives me advice on how to attain recognition	.744
2. Provides support and encouragement	.855
3. Serves as a role model	.845
4. Is someone I identify with	.796
5. Guides my personal development	.764
6. Serves as a sounding board	.784
7. Accepts me as a competent professional	.845
8. Thinks highly of me	.867
9. Sees me as being competent	.863

RMMRI, Ragins and McFarlin Mentor Role Instrument.

a. Total variance explained=67.1%.

To shorten the 69-item CRAI, the PCA yielded a 19-item, 4-factor solution: writing manuscripts, study design/data analysis, collaborations/grant preparation, and consent forms/process (Table 3). Although two items loaded >.400 on two factors, we retained each item on the factor on which it loaded most highly.25,27 Cronbach’s alpha coefficients for items on each factor and the overall 19-item CRAI (CRAI-19) were very high (range, .907-.960),27 and the CRAI-19 and 69-item CRAI scores were highly concordant (ICC=.994). Mid-range mean (SD) scores were reported on the baseline CRAI-19 (6.7 [1.7]) and each subscale: writing (7.3 [1.8]), study design/data analysis (6.4 [2.1]), collaborations/grant preparation (6.4 [1.8]), and consent forms/process (7.4 [2.5]).

Table 3. Four-factor solution of the CRAI-19 after principal components analysis^a, n=131.

	CRAI-19 Factorsb
	Factor 1c	Factor 2c	Factor 3c	Factor 4c
1. Choose an appropriate research design that will answer a set of research questions and/or test a set of hypotheses	.774b	.329	.268	.149
2. State the purpose, strengths and limitations of each study design	.760 b	.307	.317	.203
3. Determine an adequate number of subjects for your research project	.809 b	.346	.109	.083
4. Design the best data analysis strategy for your study	.880 b	.217	.201	.089
5. Evaluate the reliability and validity of a given measurement	.714 b	.313	.350	.259
6. Ensure data collection is reliable across trials, raters, or equipment	.707 b	.174	.311	.356
7. Avoid the violation of statistical assumptions	.816 b	.142	.352	.051
8. Prepare a research proposal suitable for submission in one’s research area	.373	.707 b	.290	.189
9. Locate appropriate forms for a grant application	.300	.667 b	.206	.328
10. Prepare a project budget for a grant application	.241	.654 b	.142	.430
11. Identify faculty collaborators from within and outside the discipline who can offer guidance to the project	.239	.776 b	.255	.118
12. Sustain effective collaborations	.207	.789 b	.318	.157
13. Terminate a collaboration that isn’t working	.206	.765 b	.164	.032
14. Write a literature review that critically synthesizes the literature relevant to your own research question	.251	.392	.746 b	.229
15. Write a methods section that conveys sufficient methodological detail to permit subsequent replication of your work by others	.411	.284	.758 b	.229
16. Write the results section of a research paper that clearly summarizes and describes the results, free of interpretative comments	.391	.225	.836 b	.132
17. Write a discussion section for a research paper that articulates the importance of your findings relative to other studies in the field	.338	.353	.805 b	.117
18. Apply the appropriate process for obtaining informed consent from research subjects	.175	.228	.164	.913 b
19. Write a human subjects consent form containing the appropriate elements	.179	.206	.187	.910 b

CRAI, Clinical Research Appraisal Inventory.

a. Cronbach’s alphas: CRAI-19, .960; Factor 1, writing, .955; Factor 2, Study design/data analysis, .907; Factor 3, Collaboration/grants, .948; Factor 4, Consent process, .953.

b. Denotes items included in computation of each subscale’s scores.

c. Percentage of variance explained: Factor 1=27.7%; Factor 2=22.1%; Factor 3=18.1%; Factor 4=12.9%; overall CRAI-19=80.9%.

Testing Hypotheses

The overall CRAI-19 and each of the four subscales increased over the first year (each P<.001), although the magnitude of change was small (Figure 1). To test Hypothesis 1, separate linear regression models examined whether the RMMRI-9 was independently associated with the post-SI-2 overall CRAI-19 and with each subscale, after controlling for cohort and the respective baseline CRAI measure (Table 4). Higher mentor-role appraisals on the RMMRI-9 were independently associated with greater post-SI-2 research self-efficacy on the overall CRAI-19 (P=.001) and on three CRAI-19 subscales—writing, study design/data analysis, and collaboration/grant preparation (each P≤.003).

Figure 1. Change in unadjusted mean scores on the overall CRAI-19 and each subscale from baseline (before the first PRIDE Summer Institute) to 12-month follow-up.

Table 4. Multiple linear regression models of the association between the RMMRI-9 and post-SI-2 CRAI-19 overall and each subscale, controlling for the respective baseline CRAI-19 measure and cohort, n=108^a.

	Adjusted R2	Standardized Beta	t	P
Model 1, CRAI-19 Overall
(Constant)			2.445	.016
CRAI-19 overall, baseline		.553	6.906	<.001
Cohort	.355	.066	.848	.398
RMMRI-9	.415	.261	3.406	.001
Model 2, CRAI-19 Writing subscale
(Constant)			2.268	.025
CRAI-19 subscale, baseline		.549	7.134	<.001
Cohort	.366	-.050	-.657	.513
RMMRI-9	.421	.249	3.307	.001
Model 3, CRAI-19 Study design/data analysis subscale
(Constant)			2.020	.046
CRAI-19 subscale, baseline		.460	5.267	<.001
Cohort	.252	.002	.027	.979
RMMRI-9	.311	.259	3.153	.002
Model 4, CRAI-19 Collaboration/grant preparation subscale
(Constant)			2.079	.040
CRAI-19 subscale, baseline		.599	7.843	<.001
Cohort	.405	.144	1.953	.054
RMMRI-9	.448	.226	3.029	.003
Model 5, CRAI-19 Consent process subscale
(Constant)			2.673	.009
CRAI-19 subscale, baseline		.738	11.291	<.001
Cohort	.553	.140	2.152	.034
RMMRI-9	.557	.090	1.386	.169

RMMRI, Ragins and McFarlin Mentor Role Instrument; SI-2, second Summer Institute; CRAI, Clinical Research Appraisal Inventory.

a. Adjusted R² values are for variables entered at each step (CRAI-19 measures and cohort entered at step 1; RMMRI entered at step 2). All other data shown in the table are from the regression model at step 2.

To test Hypothesis 2, RM-ANCOVAs examined whether change in number of publications after PRIDE varied by cohort, post-SI-2 research self-efficacy (overall CRAI-19 and each subscale, in separate models) and mentor-role appraisals (RMMRI-9). Increase in publications was similar across cohorts (Figure 2). We observed significant interactions between increase in publications and each research self-efficacy measure and between increase in publications and the RMMRI-9 in the overall-CRAI-19 and writing-subscale models (Table 5). Higher research self-efficacy and RMMRI-9 scores correlated with greater increase in publications after PRIDE.

Figure 2. Change in unadjusted mean number of publications from baseline (before the first PRIDE Summer Institute) to follow-up at least one year after completing the second Summer Institute, by cohort (N = 110).

Table 5. Repeated-measures analysis of variance of increase in publications after PRIDE, by cohort, with post-SI-2 CRAI-19 (overall and each subscale) and RMMRI-9, n=110.

	F	P	Partial eta squaredb
Model 1, CRAI-19 Overall
Change across all participants	.004	.948	.000
Change-by-cohort	.910	.406	.017
Change-by-post-SI-2 CRAI-19 overall	17.241	<.001	.141
Change-by-RMMRI-9	4.242	.042	.039
Model 2, CRAI-19 Writing subscale
Change across all participants	.006	.938	.000
Change-by-cohort	.389	.679	.007
Change-by-post-SI-2 CRAI-19 subscale	24.723	<.001	.191
Change-by-RMMRI-9	4.825	.030	.044
Model 3, CRAI-19 Study design/data analysis subscale
Change across all participants	.685	.410	.006
Change-by-cohort	.625	.537	.012
Change-by-post-SI-2 CRAI-19 subscale	11.004	.001	.095
Change-by-RMMRI-9	2.483	.118	.023
Model 4, CRAI-19 Collaboration/grant preparation subscale
Change across all participants	.834	.363	.008
Change-by-cohort	1.115	.332	.021
Change-by-post-SI-2 CRAI-19 subscale	5.781	.018	.052
Change-by-RMMRI-9	1.674	.199	.016
Model 5, CRAI-19 Consent process subscale
Change across all participants	.035	.850	.000
Change-by-cohort	1.581	.211	.029
Change-by-post-SI-2 CRAI-19 subscale	13.474	<.001	.114
Change-by-RMMRI-9	1.202	.275	.011

SI-2, second Summer Institute; CRAI, Clinical Research Appraisal Inventory; RMMRI, Ragins and McFarlin Mentor Role Instrument.

a. Interactions between change in publications and each variable are shown; main effects were not significant.

b. Partial eta squared (η²), a measure of effect size, is the variance explained by a given variable of the variance remaining after excluding the variance explained by other predictors: .01 (small), .09 (medium), .25 (large) effects.

Discussion

For future evaluations of PRIDE, we shortened longer versions of the RMMRI and CRAI in current use. The high internal consistency of items on the RMMRI-9, CRAI-19, and each CRAI-19 subscale is remarkable, especially for the CRAI-19 subscales, as such high Cronbach’s alphas, a benchmark for reliability of psychological measures,28 are often observed in questionnaires with larger numbers of items.27

The 33-item RMMRI17,18 was not developed specifically to evaluate mentoring roles to promote biomedical-research careers, and some items did not reflect roles that PRIDE mentors were expected to play, as suggested by >10% of mentees choosing “Not applicable” in response to 19 RMMRI items. PRIDE mentors are expected to promote their mentees’ knowledge and research skills in the mentor’s field of study, enhance their research self-efficacy, and nurture their abilities to prepare papers and competitive grant applications.4 PRIDE research mentors were not all from the mentees’ home institutions, as mentees were recruited nationally. While all mentors should play important roles for career advancement and psychosocial support, several items on the 33-item measure pertained directly to roles of a mentor at one’s home institution, were deemed “not applicable” by >10% of mentees, and therefore not included in the PCA. The RMMRI-9 items are all relevant to PRIDE’s focus on research-career development and supportive mentor roles.

PRIDE mentees completed the 69-item CRAI,19 a previously shortened version of a 92-item measure.29 We conducted our PCA using the CRAI-69 to develop a shorter, but comparable research self-efficacy measure that is relevant to specific goals of PRIDE. The CRAI-19 retained the excellent psychometric properties of the CRAI-69,19 and scores on the CRAI-19 and CRAI-69 were highly concordant and sensitive to change after training. Small changes in CRAI-69 scores at 1-year follow-up also were reported in a study of research-program trainees supported by one medical-school’s Clinical and Translational Science Award; the increase in CRAI-69 scores was larger for predoctoral and postdoctoral trainees than for program trainees receiving the Mentored Career Development Award from NIH (KL2),19 who, like PRIDE mentees, reported higher baseline scores. Mean research self-efficacy scores for both PRIDE and KL2 trainees were >6 at baseline, with small increases at 1-year follow-up,19 which is not surprising, as junior faculty in PRIDE and KL2 research-training programs would be expected to have more research experience at program entry than predoctoral and some postdoctoral trainees. Notably, US medical graduates with prior research experiences are more likely to obtain full-time academic-medicine faculty appointments.30,31 Thus, small changes in CRAI-19 scores after PRIDE would be expected.

In support of Hypothesis 1, higher mentor-role appraisals on the RMMRI-9 were independently associated with higher research self-efficacy on the post-SI-2 overall CRAI-19 and three subscales, controlling for cohort and the respective baseline CRAI-19 measure. In an evaluation of NIH Minority Research Training Programs, both the research experience itself and mentoring were reported by trainees to be important benefits of participating in these programs, regardless of their training level (undergraduate, graduate, postdoctoral, and junior faculty).3 Trainees especially valued mentoring to improve research skills and provide opportunities for scholarship, career development and personal growth.3

In support of Hypothesis 2, we demonstrated that the increase in number of publications after participating in PRIDE varied in association with post-SI-2 CRAI-19 and RMMRI-9 scores, which to our knowledge has not been reported using any versions of these measures. Higher scores on the overall CRAI-19 and each subscale were associated with greater increase in number of publications at least one year after SI-2, as were higher scores on the RMMRI-9 in those models that included the overall CRAI-19 and writing subscale. Given the excellent psychometric properties of the RMMRI-9 and CRAI-19 and their respective associations with increase in publications after PRIDE, we are using these shortened measures in our long-term evaluations of both PRIDE and PRIDE II cohorts (2014-2018).

Limitations

As an observational cohort study of a self-selected sample of motivated junior faculty working toward research independence, we lacked a randomly selected comparison group; thus, causal inferences cannot be made. The observed increase in publications could have resulted independently of the PRIDE program. The sample size was limited by the capacity of each PRIDE program to recruit and/or support mentees, and findings might not be generalizable to participants in other types of training programs. There also is potential for social desirability bias in self-reported data, which can compromise the validity of these assessments.33 Notably, the RMMRI-9 focuses on mentor roles that foster PRIDE-mentees’ pursuit of research careers; other potentially salient mentor roles to promote career development were not retained,17,18 and we lack knowledge of mentees’ other mentoring relationships.

Conclusions

Being socialized to self-identify as a biomedical researcher is important and necessary to encourage researchers from underrepresented groups to persist in biomedical-research careers.34 The NHLBI aims to promote socialization and integration of PRIDE mentees pursuing heart, lung, blood, and sleep basic, clinical and translational research careers. Our findings indicate positive associations between the RMMRI-9 and CRAI-19 and between increase in publications at least one year after PRIDE training and both RMMRI-9 and CRAI-19 measures. Moreover, the RMMRI-9 and CRAI-19 retained the excellent psychometric properties of the longer versions.18,19 Findings support use of the shortened measures in future evaluations of PRIDE and potentially other research-training programs, especially when mentors may not be at the mentee’s home institution. With planned 10-year follow-up of PRIDE mentees, we will evaluate the RMMRI-9 and CRAI-19 in relation to long-term productivity (ie, publications, grant submissions and awards) and mentees’ retention in biomedical-research careers (eg, faculty promotion), as is fitting given national priorities and efforts to increase the diversity of the biomedical-research workforce.1-3,9,32