Abstract
Background:
NIH funding to address basic reproductive health for common female conditions remains disproportionately low, in part due to low success rates of grant applications by obstetrician-gynecologists.
Objective:
To evaluate the scholarly productivity of individuals supported by the National Institutes of Health (NIH) Child Health and Human Development Women’s Reproductive Health Research (WRHR) K12 career development award (CDA), created to advance careers of obstetrician-gynecologist physician-scientists.
Study Design:
We performed a cohort study of individuals who completed at least 2 years of WRHR training by June 30, 2015 and had at least 5 years follow-up. Earliest training start date was December 1, 1998. Primary outcomes from public data sources (NIH RePORTER, PubMed, iCite) were: 1) number of total and R01 NIH grants as PI; 2) numbers of total, first and last author publications; and 3) median and highest publication impact factor measured by the relative citation ratio (RCR). Secondary outcomes from an email survey subcohort were: total number of research grants, federally funded grants, and number of NIH grants as co-investigator; institutional promotions and academic appointments, national and NIH leadership roles; career and mentorship satisfaction. Outcomes were recorded at 5, 10, and 15 years post-graduation and aggregate anonymized data divided into 3 groups using WRHR completion dates: June 30 of 2005, 2010 and 2015. Temporal trends were assessed. Results were stratified by gender and by institutions funded for 5-year cycles (1–2 vs 3–4). Analyses used Fisher exact or Pearson chi-square tests, and Mantel-Haenszel tests of trend.
Results:
The distribution of the cohort (N=178) by graduation completion date was: on or before June 30, 2005 [57(32%)]; July 1 2005-June 30, 2010 [60(34%)] and July 1, 2010-June 30, 2015 [61(34%)]. The majority were female [112(64%)]. Most were maternal fetal medicine trained [53(30%)], followed by no fellowship [50(28%)]. Of the 178, 72 (40%) received additional NIH funding as a PI, 45 (25%) received at least one R01 and 23 (13%) received 2–5 R01s. Mean estimates of the proportion of scholars was 52(31%) with >10 first author publications, 66(39%) with >10 last author publications and 108(63%) had ≥ 25 publications. The highest RCR score was a median of 8.07 [IQR 4.20,15.16]. There were 121(71%) with RCR ≥5, indicating over 5-fold greater publication impact than other NIH-funded scientists in similar area of research. No differences by gender, institution or temporal trends were observed. Of the full cohort, 69 (45.7%) responded to the survey: the majority self-identified as women [50(73%)] and White [51(74%)].
Conclusion:
Our findings suggest that the infrastructure provided by an institutional K award is an advantageous CDA mechanism for obstetrician-gynecologists, a group of predominantly women surgeons. It may serve as a corrective for the known inequities in NIH funding by gender.
Keywords: National Institutes of Health (NIH) Institutional Career Development Award, (K12), obstetrician-gynecologist physician-scientist, grant funding, publication impact, bibliometrics, women’s reproductive health research
INTRODUCTION
Most specialized reproductive health care is delivered by obstetrician-gynecologists. Their field is both clinical and surgical in nature and in recent years has become dominated by women (nearly 85%).1 Cutting-edge research within medicine is necessary to advance the highest quality care, yet a dearth of obstetrican-gynecologist physician-scientists prompted an Institute of Medicine study,2 the findings of which led to the National Institute of Child Health and Human Development (NICHD) K12 Women’s Reproductive Health Research (WRHR) program in 1998. However, NIH funding awarded to address basic reproductive health for common female conditions remains disproportionately low,3,4 in part due to lower success rates of grant applications by obstetrician-gynecologists than pediatricians, internists, or general surgeons,5 and in part due to disproportionately low NIH funding for female investigators.6 Data suggests that obstetrician-gynecologist early stage investigators receive less than 1% of NIH funding overall.7
The contribution of National Institutes of Health (NIH) career development awards (CDAs) toward advancement of independent clinician-scientists is well-established, particularly for individual K awards.8,9 However, speculation remains as to whether individual K awards result in greater scientific advancements than funding from institutional K awards (2018).10 An evaluation of the impact of institutional vs individual NIH CDAs for physician-scientists engaged in reproductive women’s health research indicated similar proportions of successful R01s obtained by 23% of 159 K12 WRHR recipients, vs 32% of 43 K08 recipients (p=NSS).5
Our objectives were to update NIH funding success previously described5 but also to evaluate additional measures of scientific and academic success such as publication numbers and impact, promotions, and leadership roles at NIH, within academic institutions and nationally. We hypothesized that the NIH Institutional K12 WRHR centers would demonstrate a high level of academic productivity. Our findings may provide guidance for future NIH funding decisions.
MATERIALS AND METHODS
Study design and participants
We conducted a cohort study of WRHR scholars using historical data over a 15-year period, comparing temporal trends in scholarly productivity and leadership roles (Figure 1). WRHR grants are funded on 5-year cycles. There were 28 institutions with WRHR grants awarded: each institution appointed up to 4 scholars (December 1, 1998 - June 30, 2005), 3 scholars (July 1, 2005 - June 30, 2010) and 2 scholars (July 1, 2010 - June 30, 2015). Identified scholars (N= 223) were included if they had 2–5 years in training at one of 28 different institutions and at least 5-years follow-up after completion between June 30, 2000 and July 1, 2015. Current scholars and all graduates with completion dates after June 30, 2015 (n= 34) were excluded and duplicates or those subsequently found to have end dates outside of inclusion parameters were removed (n=11). The final cohort (N=178) was evaluated for scholarly productivity using publicly available data and data collected from a subset of individuals who completed an online survey administered by email between July 1 and September 30, 2021. This study qualified for exempt status by the University of Washington Institutional Review Board and was designed and reported using STROBE cohort guidelines.
Figure 1. Study Cohort.
There were 28 institutions with WRHR grants awarded between December 1, 1998 and June 30, 2015. Each institution appointed up to 4 scholars from December 1, 1998 through June 30, 2005, 3 scholars from July 1, 2005 through June 30 2010, and 2 scholars from July 1, 2010 through June 30, 2015 (n=223). WRHR funding announcements occurred in 1998–1999, 2003–2004 and 2009–2010 over the time period of interest. Three relatively even groups were created, using WRHR completion dates, creating training cohorts of up to approximately 5 years each: on or before June 30 of 2005 (early), July 1, 2005-June 30,2010 (middle) and Jully1, 2010-June 30, 2015 (recent).
Publicly available data from the American Board of Obstetrics and Gynecology identified scholars with Accreditation Council for Graduate Medical Education (ACGME) certified fellowships. Proportions of individuals categorized into ACGME fellowships: Maternal Fetal Medicine (MFM), No fellowship - Obstetrics and gynecology generalist (OB GYN), Reproductive Endocrinology and Infertility (REI), gynecologic oncology (GYN ONC) and urogynecology (UROGYN). Any fellowships in complex family planning ACGME accredited in 2021 and non-ACGME or non-ABOG approved fellowships including minimally invasive gynecologic surgery, pediatric and adolescent gynecology, clinical genetics, national research service award (NRSA) primary care research fellowship and a reproductive infectious diseases fellowship are not captured here, but were identified in survey data. Five individuals identified as complex family planning specialists. Two individuals completed 2 fellowships.
Primary and secondary outcomes (collected approximately 5, 10, or 15 years post-graduation)
Primary outcomes taken from publicly available sources included: 1) number of total and R01 NIH grants as PI, 2) numbers of total, first and last author publications, and 3) median and highest publication impact factor measured using the relative citation ratio (RCR). RCR represents the field- and time-normalized citation rate and is benchmarked to 1.0 for a typical (median) NIH paper in the corresponding year of publication. A paper with an RCR of 1.0 has received the same number of citations per year as the median NIH-funded paper in its field, a paper with an RCR of 2.0 has received twice as many citations per year as the median NIH-funded paper in its field, while an RCR of 0.5 has received half as many citations per year.11
Secondary outcomes from survey data included: total number of research grants, numbers of NIH grants as co-investigator; promotions and academic appointments; institutional, national and NIH leadership roles; and career and mentorship satisfaction.
Data Collection
Publicly available data sources were NICHD, NIH RePORTER, PubMed iCite, and American Board of Obstetrics and Gynecology (ABOG). Information regarding identification of prior WRHR Scholars was provided through the NIH Freedom of Information Act, Eunice Kennedy Shriver National Institute of Child Health and Human Development. NIH RePORTER was used to capture grant awards in which the K scholar was PI or multiple PI (MPI) for the years this was available, along with award dates. PubMed and iCite were utilized to capture journal articles and impact metrics. Scholars who completed fellowships accredited by the Accreditation Council for Graduate Medical Education (ACGME) were identified on the ABOG website.
Survey invitations were emailed to the cohort. We searched for email addresses using google search, contacted program directors at individual sites, and reached out via personal connections for contact information. Among email addresses identified (n=162), some were incorrect or could not be delivered (n=11); the remaining individuals received email invitations to complete the survey (n=151). Reminders to complete the survey were sent up to 5 times at 3-day intervals over 3 months until the survey was closed. Following email consent, survey data (34 questions and curriculum vitae [CV] upload) were collected and managed using Research Electronic Data Capture (REDCap). Three data abstractors (EM, DL, and EO) collated public and survey data and created a de-identified dataset.
Variables
Variables for the main cohort included gender, number of NIH awards, number of R01s, median number of publications, total number of publications (<25, 25–50 and > 50), median number of first author publications and total number (<5, 5–10, >10), and median number of last author publications and total number (<5, 5–10, >10). Publication impact was estimated by the median value of the highest RCR score; the median RCR score of each scholar’s total publications (categorized as <1, 1–2, >2), and the highest RCR score (categorized as <5, 5–10, >10). Gender was by self-report in the survey and estimated using publicly available data for the remaining 148 individuals using previously described methods.5
Additional variables collected on the survey subcohort included: self-described race and ethnicity, marital/cohabitation status at end of WRHR, parenting status during and after WRHR, primary career type (academics, industry, private practice), fellowship training, degrees, and post-doc training. In addition, number of federal grants not awarded by NIH, number of NIH grants as co-investigator (co-I), total numbers of grants, current academic titles, and leadership roles (NIH, national and academic) were abstracted from CVs.
Analysis
Variables were recorded at 5, 10, and 15 years post-graduation and aggregate anonymized data divided into 3 groups using WRHR completion dates: June 30 of 2005 (early), 2010 (middle) and 2015 (recent).
Temporal trends were assessed, comparing the 3 groups. Primary outcomes were compared between survey respondents and non-respondents to investigate potential response bias in the survey results. Results were stratified by gender, by those institutions awarded 1–2 cycles vs 3–4 cycles of funding and specialty type. There were no missing data for primary outcomes. Data are presented as N (%), mean (standard deviation), or median (interquartile range). Data were analyzed using Fisher exact or Pearson chi-square tests and Mantel-Haenszel test of trend. P values < 0.05 were considered statistically significant. All analyses were performed with SPSS version 28.0 (released 2021. Armonk, NY: IBM Corp.)
RESULTS
Full cohort assessed with publicly available data (N=178).
Characteristics of scholars in the full cohort included the time period of training and specialty type (Figure 1). Scholars were fairly evenly distributed into 3 groups by year of WRHR completion: 57 (32%) – on or before June 30, 2005; 60 (34%) from July 1, 2005 – June 30, 2010 and 61 (34%) from July 1, 2010 - June 30, 2015. The majority had completed a maternal fetal medicine (MFM) fellowship [53(30%)], followed by no fellowship [50(28%)]. Other fellowship training included: reproductive endocrinology and infertility (REI) [39(22%)], gynecologic oncology [25(14%)], and urogynecology [11(6%)].
Of the full cohort, 72(40%) received additional NIH funding as a PI, 45 (25%) received at least one R01 and 22(13%) received 2–5 R01s (Table 1). Of the full cohort, 108(63%) had over 25 publications. There were 52(31%) with >10 first author publications and 66(39%) with >10 last author publications. The median of the greatest RCR for each scholar was relatively high at 8.07 [IQR 4.20,15.16]. There were 121(71%) with RCR ≥5; 70(41%) had RCR >10 with the greatest proportion in the earliest cohort but temporal trends were not significant (p=0.22). No significant differences were observed when these primary outcomes were evaluated by gender or by the number of funding cycles awarded to the institutions from which the scholar graduated (1–2 vs 3–4) (Table 2). The proportions of individuals who received any NIH funding for benign gynecologic conditions appeared lower than that for pregnancy conditions, but differences were not significant (p=0.21); whereas proportions were significantly lower when compared with gynecologic oncology (p=0.03) (Table S1).
Table 1.
NIH Grants, PubMed citations and citation impact scores in the Full WRHR Alumni Cohort from 2000–2015, by training completion date, N=178
| WRHR completion date |
Total | |||||
|---|---|---|---|---|---|---|
| Outcomes | On or before 6/30/05 N (%) |
7/1/05 – 6/30/10 N (%) |
7/1/10 – 6/30/15 N (%) |
P Value for trend | ||
| 57 (32) | 60 (34) | 61 (34) | 178 (100) | |||
|
| ||||||
| NIH FUNDING | ||||||
|
| ||||||
| Number of all awards | 0 | 33 (58) | 40 (68) | 33 (54) | 106 (60) | |
| 1–3 | 13 (22) | 16 (27) | 17 (28) | 46 (26) | 0.72 | |
| 4–9 | 10 (18) | 3 (6) | 10 (16) | 23 (12) | ||
| ≥10 | 1(2) | 1(2) | 1(2) | 3(2) | ||
|
| ||||||
| Number of R01 grants | 0 | 42 (74) | 50 (83) | 41 (67) | 133 (75) | 0.39 |
|
|
||||||
| 1 | 8 (14) | 5 (8) | 10 (16) | 23 (13) | ||
|
|
||||||
| 2 | 1 (2) | 3 (5) | 6 (10) | 10 (6) | ||
|
|
||||||
| ≥ 3 | 8 (11) | 2 (4) | 4 (7) | 12 (7) | ||
|
| ||||||
| PUBLICATIONS | ||||||
|
| ||||||
| Any authorship, median * | 31 [13, 87] | 36 [18,86] | 32 [14, 51] | 35 [15,73] | 0.43 | |
|
| ||||||
| Any authorship, total number | ||||||
| < 25 | 20 (38) | 18 (31) | 24 (41) | 62 (37) | 0.27 | |
| 25 – 50 | 12 (23) | 16 (28) | 20 (34) | 48 (28) | ||
| > 50 | 21 (40) | 24 (41) | 15 (25) | 60 (35) | ||
|
| ||||||
| First author, median * | 6 [2,12] | 9 [4,20] | 6 [3,11] | 6 [3, 14] | 0.32 | |
|
| ||||||
| First author, total number | ||||||
| < 5 | 22 (42) | 19 (33) | 25 (42) | 66 (39) | 0.86 | |
| 5–10 | 17 (32) | 16 (28) | 19 (32) | 52 (31) | ||
| >10 | 14 (26) | 23 (40) | 15 (25) | 52 (31) | ||
|
| ||||||
| Last author, median * | 7 [2,21] | 10 [2,26] | 7 [2,13] | 7 [2,20] | 0.47 | |
|
| ||||||
| Last author, total number | ||||||
| < 5 | 20 (32) | 23 (40) | 21 (36) | 64 (38) | 0.51 | |
| 5–10 | 11 (21) | 8 (14) | 21 (36) | 40 (24) | ||
| >10 | 22 (42) | 27 (47) | 17 (29) | 66 (39) | ||
|
| ||||||
| RELATIVE CITATION RATIO † | ||||||
|
| ||||||
| Highest RCR score, median ‡ | 9.37 [5.41, 17.21] | 8.27 [3.82, 17.29] | 6.59 [4.22, 13.79] | 8.07 [4.20, 15.16] | 0.42 | |
|
| ||||||
| All RCR scores, median | ||||||
| < 1 | 23 (43) | 32 (55) | 33 (56) | 88 (52) | ||
| 1.00 – 2.00 | 26 (49) | 25 (43) | 25 (42) | 76 (45) | 0.92 | |
| > 2.00 | 4 (8) | 1 (2) | 1 (2) | 6 (4) | ||
|
| ||||||
| Highest RCR score | ||||||
| < 5 | 11 (21) | 20 (35) | 18 (31) | 49 (29) | 0.22 | |
| 5 – 10 | 18 (34) | 13 (22) | 20 (34) | 51 (30) | ||
| >10 | 24 (45) | 25 (43) | 21 (36) | 70 (41) | ||
Data compiled from public sources: NICHD, NIH RePORTER, PubMed and iCite. WRHR = Women’s Reproductive Health Research. NIH = National Institutes of Health. IQR = interquartile range.
median number [interquartile range]
RCR = Relative Citation Ratio and is defined as the field- and time-normalized citation rate benchmarked to 1.0 for a typical (median) NIH paper in the corresponding year of publication. A paper with an RCR of 1.0 has received the same number of citations per year as the median NIH-funded paper in its field, a paper with an RCR of 2.0 has received twice as many citations per year as the median NIH-funded paper in its field, while an RCR of 0.5 has received half as many citations per year.8
Highest RCR score, median [Interquartile range]
P-value for comparison of temporal trend tested using Mantel-Haenszel test.
Table 2.
WRHR scholars grants, publications and publication impact by gender and by number of WRHR 5-year grant cycles achieved at WRHR scholar’s institution
| Outcomes | Gender N (%) |
Number of grant cycles N (%) |
|||||
|---|---|---|---|---|---|---|---|
|
| |||||||
| Female | Male | p-value | 1 – 2 | 3 – 4 | p-value | ||
| 112 (64) | 64 (36) | 68 (38) | 110 (62) | ||||
|
| |||||||
| GRANT FUNDING | |||||||
|
| |||||||
| Number, all awards | 0 | 63 (56) | 41 (64) | 41 (60) | 65 (59) | 1.00 | |
| 1 – 20 | 49 (43) | 23 (36) | 0.34 | 27 (40) | 45 (41) | ||
|
| |||||||
| Number R01s | 0 | 78 (70) | 53 (83) | 49 (72) | 84 (76) | 0.60 | |
| 1 –5 | 34 (30) | 11 (17) | 0.07 | 19 (28) | 26 (24) | ||
|
| |||||||
| PUBLICATIONS | |||||||
|
| |||||||
| Any authorship, median * | 34 [17, 66] | 34 [13,96] | 0.87 | 27 [10,60] | 38 [21,87] | 0.12 | |
|
| |||||||
| Any authorship, total | |||||||
| < 25 | 37 (36) | 25 (39) | 0.20 | 30 (48) | 32 (30) | 0.06 | |
| 25 – 50 | 34 (33) | 13 (20) | 13 (21) | 35 (33) | |||
| > 50 | 33 (32) | 26 (41) | 20 (32) | 40 (37) | |||
|
| |||||||
| First author, median * | 8 [3,14] | 5 [3,15] | 0.58 | 5 [2,11] | 8 [4,16] | 0.05 | |
|
| |||||||
| First author, total | |||||||
| < 5 | 36 (35) | 30 (47) | 0.23 | 29 (46) | 37 (35) | 0.31 | |
| 5–10 | 35 (34) | 15 (23) | 18 (29) | 34 (32) | |||
| >10 | 33 (32) | 19 (30) | 16 (25) | 36 (34) | |||
|
| |||||||
| Last author median * | 7 [2,16] | 8 [2,21] | 0.97 | 6 [1,16] | 8 [2,12] | 0.40 | |
|
| |||||||
| Last author | |||||||
| < 5 | 39 (38) | 25 (39) | 0.54 | 26 (41) | 38 (36) | 0.53 | |
| 5–10 | 27 (26) | 12 (19) | 16 (25) | 24 (22) | |||
| >10 | 38 (37) | 27 (42) | 21 (35) | 45 (42) | |||
|
| |||||||
| RELATIVE CITATION RATIO † | |||||||
|
| |||||||
| Highest RCR score, median ‡ | 7.64 [4.60,15.41] | 8.46 [3.97,16.19] | 0.63 | 7.07 [3.25,14.68] | 8.46 [5.11,18.03] | 0.75 | |
|
| |||||||
| All RCR scores, median | |||||||
| < 1 | 54 (52) | 33 (52) | 0.97 | 32 (51) | 56 (52) | 0.31 | |
| 1.00 – 2.00 | 46 (44) | 29 (45) | 27 (43) | 49 (46) | |||
| > 2.00 | 4 (4) | 2 (3) | 4 (6) | 2 (2) | |||
|
| |||||||
| Highest RCR score | |||||||
| < 5 | 30 (29) | 19 (30) | 0.94 | 24 (38) | 25 (23) | 0.11 | |
| 5 – 10 | 32 (31) | 18 (28) | 15 (24) | 36 (34) | |||
| >10 | 42 (40) | 27 (42) | 24 (38) | 46 (43) | |||
Data compiled from public sources: NICHD, NIH RePORTER, PubMed and iCite. WRHR = Women’s Reproductive Health Research. NIH = National Institutes of Health. IQR = interquartile range.
median number [interquartile range]
RCR = Relative Citation Ratio and is defined as the field- and time-normalized citation rate benchmarked to 1.0 for a typical (median) NIH paper in the corresponding year of publication. A paper with an RCR of 1.0 has received the same number of citations per year as the median NIH-funded paper in its field, a paper with an RCR of 2.0 has received twice as many citations per year as the median NIH-funded paper in its field, while an RCR of 0.5 has received half as many citations per year.8
Highest RCR score, median [Interquartile range]
P-value for comparison of male vs female and 1–2 vs 3–4 awarded grant cycles at the awardees institution, using t-tests, Fisher exact or Pearson chi-square tests.
Subcohort of survey respondents (N=69).
Of the 151 individuals with available contact information, the survey response rate was 45.7% and was greatest among the most recent cohort of graduates 28(41%), followed by 24(35%) from the earliest and 17(25%) from the middle cohort (Table 3). Proportions of subcohort individuals who completed MFM 18(26%) and REI 17(25%) fellowships were comparable to the full cohort. Many held additional degrees: PhD 17(25%) and science or public health master’s degrees 31(45%). Most self-identified as women 50(73%) and White 51(74%); only 3(4% identified as Hispanic and 4(6%) identified as Black. By completion of WRHR training, the majority were married or in a cohabitating relationship 58(84%). The majority were parents during their training 53(76%), and 60(86%) were parents during or after their training.
Table 3.
Characteristics of WRHR K12 Survey Alumni Sub-cohort from 2000–2015, by Training Completion Date, N=69
| Characteristic | WRHR Completion date | |||
|---|---|---|---|---|
| On or before 6/30/05 N (%) |
7/1/05–6/30/10 N (%) |
7/1/10–6/30/15 N (%) |
All N (%) |
|
|
|
||||
| 24 (35) | 17 (25) | 28 (41) | 69 (100) | |
|
| ||||
| Gender | ||||
| Man | 8 (33) | 4 (24) | 3 (11) | 15 (22) |
| Woman | 16 (67) | 12 (71) | 22 (79) | 50 (73) |
| Missing or prefer not to disclose | 0 | 1 (6) | 3 (11) | 4 (6) |
|
| ||||
| Race | ||||
| Asian | 1 (4) | 3 (18) | 5 (18) | 9 (13) |
| Black or African American | 0 | 3 (18) | 1 (4) | 4 (6) |
| White | 23 (96) | 8 (47) | 20 (71) | 51 (74) |
| Two or more races | 0(0) | 1 (6) | 0(0) | 1 (1) |
| Missing or prefer not to disclose | 0 | 2 (12) | 2 (7) | 4 (6) |
|
| ||||
| Ethnicity | ||||
| Hispanic/Latinx | 1 (4) | 0(0) | 2 (7) | 3 (4) |
| Not Hispanic | 22 (92) | 16 (94) | 24 (86) | 62 (90) |
| Missing or prefer not to disclose | 1 (4) | 1 (6) | 2 (7) | 4 (6) |
|
| ||||
| Marital status at WRHR completion | ||||
| Single | 1 (5) | 0(0) | 1 (4) | 2 (3) |
| Married or cohabitating relationship | 20 (83) | 15 (88) | 23 (82) | 58 (84) |
| Divorced | 1 (5) | 1 (6) | 2 (7) | 4 (6) |
| Missing or prefer not to disclose | 2 (8) | 1 (6) | 2 (7) | 5 (7) |
|
| ||||
| Parent during and after WRHR | ||||
| No during/ no after | 2 (8) | 0 | 5 (18) | 7 (10) |
| No during/ yes after | 1 (4) | 2 (12) | 4 (14) | 7 (10) |
| Yes during | 20 (83) | 14 (83) | 19 (68) | 53 (76) |
|
| ||||
| Current primary career type | ||||
| Academic | 18 (75) | 15 (88) | 23 (82) | 56 (81) |
| Industry, Government, Nonprofit | 1(4) | 1(6) | 4 (14) | 6 (8) |
| Retired | 3 (13) | 0 | 0 | 3 (4) |
| Subspecialty/fellowship training | ||||
| Maternal-Fetal Medicine | 4 (17) | 5 (29) | 9 (32) | 18 (26) |
| Repro Endocrine and Infertility | 7 (29) | 3 (18) | 7 (25) | 17 (25) |
| Gynecologic Oncology | 1 (4) | 0 | 4 (14) | 5 (7) |
| Complex Family Planning | 1 (4) | 1 (6) | 3 (11) | 5 (7) |
| Urogynecology | 5 (21) | 2 (12) | 1 (4) | 8 (12) |
| MIGS | 0(0) | 1 (6) | 0(0) | 1 (1) |
| None of the above* | 1 (4) | 1 (6) | 1 (4) | 3 (4) |
| None† | 4 (18) | 3 (18) | 3 (11) | 10 (14) |
|
| ||||
| Additional Degrees | ||||
| PhD | 5 (21) | 2 (12) | 10 (36) | 17 (25) |
| MA/MS/MAS/MPH | 9 (38) | 10 (59) | 12 (43) | 31 (45) |
| Other | 2 (8) | 0 | 0 | 2 (3) |
| None | 7 (29) | 4 (2) | 6 (21) | 17 (24) |
|
| ||||
| Postdoctoral funding | ||||
| T32, F32, or F35 before WRHR | 21 (88) | 13 (77) | 20 (71) | 54 (78) |
Categories may not sum to 100% due to missing answers in 1–3 individuals. WRHR = Women’s Reproductive Health Research. MIGS = Minimally Invasive Gynecologic Surgery.
Other fellowships included: clinical genetics, NRSA primary care research, reproductive infectious disease. Several individuals completed multiple fellowships.
None refers to Generalist in Obstetrics and Gynecology.
Of the subcohort, 28(41%) received additional NIH funding as a PI, 20(29%) received at least one R01 and 10(15%) received 2–5 R01s (Table 3). Respondents reported mean number of federal research grants as 4.5 (SD 5.5) as PI, as 2.9 (SD 3.0) as co-I and mean number of non-federal research grants as 9.8 (SD 10.9) as PI/MPI/Co-I. Of survey respondents, 55(80%) had over 25 publications. The median of the highest RCR score in the subcohort was similar to the full cohort at 9.68 [IQR 5.33, 18.95].
In general, the mean estimates for the primary outcomes of the survey respondents were higher than non-respondents, particularly for publication numbers, but proportions of individuals who were PIs of any NIH awards or R01s did not differ between survey respondents and non-respondents (p=1.0 and 0.38, respectively) nor did median highest RCR (p=0.12) and all RCR values (p=0.09) (Figure 2, Table S2).
Figure 2. Scholarly success rates by survey respondents and non-respondents.
Y axis: Proportions of survey respondents (grey) and survey non-respondents (black) with on x axis, from left to right: at least one NIH award, at least one R01, at least 25 publications and relative citation index greater than 1.
Most WRHR graduates held an academic position 56(81%) at the time of the survey, and many 50(75%) currently held or had held leadership positions in their department and their institutions (Table 4). Almost one third had been vice chair, chair, or dean and 36(52%) took part in NIH study section.
Table 4.
Promotion, Leadership and Research Activity among K12 WRHR Alumni Survey Respondents by training completion date
| WRHR completion date | ||||
|---|---|---|---|---|
| On or before 6/30/05 N (%) |
7/1/05–6/30/10 N (%) |
7/1/10–6/30/15 N (%) |
Total N (%) |
|
| 24 (35) | 17 (25) | 28 (41) | 69 (100) | |
|
| ||||
| Promotion (current title) | ||||
|
| ||||
| Professor | 15 (63) | 12 (70) | 11 (39) | 38 (55) |
| Associate Professor | 1 (4) | 4 (24) | 11 (39) | 16 (23) |
| Assistant Professor | 1 (4) | 1 (6) | 1 (4) | 3 (4) |
| Clinical appointment | 2 (8) | 0 (0) | 1 (4) | 1 (1) |
| Retired | 3 (13) | 0 (0) | 0 (0) | 3 (4) |
| Non-academics | 2 (8) | 0 (0) | 4 (14) | 6 (9) |
|
| ||||
| Department or Institutional Leadership Roles Ever | ||||
|
| ||||
| At least one leadership role | 19(79) | 13(77) | 18(69) | 50(75) |
|
| ||||
| Types of leadership roles | ||||
|
| ||||
| Residency program director (or associate) | 2 (8) | 1 (6) | 1 (4) | 4 (6) |
|
| ||||
| Fellowship director (or associate) | 8 (33) | 7 (41) | 7 (25) | 22 (32) |
|
| ||||
| Medical director (or associate) | 10 (42) | 5 (30) | 6 (21) | 21 (30) |
|
| ||||
| Division director (or associate) | 14 (58) | 6 (35) | 6 (38) | 26 (38) |
|
| ||||
| Research section director (or associate) | 5 (21) | 5 (29) | 3 (11) | 13 (19) |
|
| ||||
| Vice chair | 6 (25) | 4 (24) | 4 (14) | 14 (20) |
|
| ||||
| Chair | 5 (21) | 0 (0) | 1 (4) | 6 (9) |
|
| ||||
| Dean | 0 (0) | 2 (12) | 0 (0) | 2 (3) |
|
| ||||
| Types of National Leadership Roles Ever | ||||
|
| ||||
| WRHR PI or RD | 4 (17) | 1 (6) | 5 (18) | 10 (14) |
|
| ||||
| Grant Reviewer | 15 (63) | 13 (76) | 17 (61) | 45 (65) |
|
| ||||
| NIH Study Section | 14 (58) | 9 (53) | 13 (46) | 36 (52) |
|
| ||||
| Elected leadership national organization | 12 (50) | 7 (41) | 17 (61) | 36 (52) |
WRHR = Women’s Reproductive Health Research. NIH = National Institutes of Health.
Qualitative data analyses of open-ended questions by survey respondents add to our findings. The mean response to the question, “How would you rate the quality of the mentorship you received during the five years immediately following your graduation from the WRHR program?” was 6.4 (SD 3.0), (Scale 0 [poor] to 10 [excellent]). Two main themes emerged from answers to the question, “Is there anything else you would like to tell us (e.g. how you think WRHR helped your career, how the WRHR experience might be improved)?” Most responded that WRHR was critical in launching their career by providing protected time and mentorship. A common suggestion for program improvement was to strengthen cross-institution networking during and after WRHR participation. Scholars reported that bridge funding and mentorship following graduation were critical to success.
COMMENT
Principal Findings
Among WRHR scholars who completed training from 2000 – 2015, 25% were subsequent R01 PI recipients and 40% achieved additional independent NIH funding as PIs. Sixty-three percent had at least 25 publications. The majority, 71%, had at least one publication with a high impact RCR score of 5 or more, demonstrating impact ratings for most WRHR graduates 5-fold greater than NIH funded scientists in their area of research.11 Temporal trends were notable. Though not statistically significant, as anticipated, high publication numbers and high impact scores were lowest among most recent graduates, but—surprisingly—recent graduates had higher rates of being a recipient of at least one R01 (33%), suggesting improved mentorship and perhaps recruitment of highly committed individuals over time as WRHR programs matured.
Detailed characteristics of a subset of WRHR alumni [69(45.7%)] supported our hypothesis that WRHR scholars provide the cornerstone of academic leadership in obstetrics and gynecology: 81% were academic faculty and 75% held leadership roles in their departments and institutions. Almost 2/3 served on grant review committees and 52% served in leadership roles of national professional societies. Despite only 5 years of follow up for some, 55% had been promoted to professor.
Results in the Context of What is Known
The new metrics provided by this study point to the value of the WRHR K12 program that extends beyond attainment of R01 funding as a PI, although grant funding for this cohort was strong and not significantly different from appropriate historic control groups.5,8,9,12 Although nearly 85% of recent obstetrician-gynecologist residents are female,1 only 64% of the full WRHR study cohort and 73% of the survey subcohort were women. These statistics are in direct contra-distinction to those reported for other K programs, of which the majority of awardees are male.9,12 Others have shown lower publication trajectories for female CDA awardees from Veterans Affairs, NIH, and Agency for Healthcare Research & Quality, as compared with male awardees.12 Moreover, higher physician burn out is well documented among female physicians.13 Burnout of female physicians has been linked to long work hours, inflexible schedules, managing a family and challenges related to pregnancy and infertility. Among the subcohort of mainly female physicians, 87% were co-habitating or married and 77% were parents at completion of training. Others have described the gap in the observed number of scientific works produced by women vs men in science, and the ensuing consequences for the retention and promotion of women in science.14 Some have suggested the gap is a result of productivity differences,15,16,17,18 with societal expectations that women shoulder more of the childcare and household duties while others suggest the gap is due to women’s contributions not being acknowledged.19,20 Recent evidence finds that at least in part, the disparity is due to the latter: women in science are significantly less likely to be credited with authorship than are men.21 Thus, this report on WRHR K12 graduate achievements suggests that NIH efforts to overcome gender bias, as well as increase the number of obstetrician-gynecologist physician-scientists is succeeding.
Research Implications
Our findings expand upon those of a report of R01 success rates among individual K recipients and K12 recipients that included WRHR scholars, but also Building Interdisciplinary Research Careers in Women’s Health (BIRCWH) and Reproductive Scientist Development Program (RSDP) K12 awardees.5 Among all obstetrician-gynecologist physician-scientists, K08 awardees received the largest proportion of R01 awards (32%), with lower proportions among K12 (21%) and K23 (17%) recipients. The proportion of WRHR scholar R01 recipients in our study of 179 individuals was slightly higher (25%) than prior K12 and K23 recipient findings. Overall evaluations of NICHD CDAs awarded to pediatricians, rehabilitation medicine specialists and obstetrician-gynecologists concluded that those with institutional K awards were less successful than those with individual K awards in achieving independent funding.10 The report did not separate findings by specialty group and gender data were not available; therefore, meaningful comparisons with the majority of individuals in our cohort who were women and, by specialty definition surgeons, cannot be made. Other reports combining MDs, MD/PhDs and PhDs from other specialties, are also not appropriate comparators but show R01 award recipients of around 40%.22 Data published from a single institution for all K recipients [MDs (56%), MD/PhDs (24%), and PhDs (20%)], reported similar attainment of R01 grants by KL2 (29%), K08 (29%) and K23 (20% awardees).9 It is important to evaluate CDAs taking into account their unique physician factors and scientific challenges in order to provide a fair assessment of their value.
Strengths and limitations
Publicly available databases are relatively accurate for capturing PIs of NIH grants, but older data did not capture co-PI or MPI status as this was not available at the beginning of our study period. Data in PubMed and iCite is quite accurate but we were not able to collect data on grants submitted and not funded. We used the RCR11 to assess the influence of articles; others have reported that among the top 100 obstetrics and gynecology articles with the highest relative citation ratios, most were observational studies (36%), reviews (26%), or consensus statements (21%).23 The data collected in our subcohort survey is not publicly available and therefore is new information. The survey response rate of 45.7%, while low, is fairly similar to other physician survey response rates.24 We acknowledge likely reporting bias by those who completed the survey, but for subsequent NIH funding and most publication impact scores, we noted similar outcomes between respondents and non-respondents. This suggests that respondents are more likely to be academicians, but among the non-respondents, we suspect there is a cadre of academicians too busy writing grants to complete our survey. Nearly 75% of respondents identified as white; only 4% identified as Hispanic and 6% identified as Black. Bias in the findings of secondary outcomes is possible, such as the number of individuals who were partnered or parents. Individuals in the subcohort self-reported their gender, whereas for the remaining 109 individuals, gender was estimated using methods previously described.5 This methodology best reflects the potential effect that a presumed gender might have on the outcome of grants and publication acceptance.
Conclusions
This study supports current levels of funding, or expansion, for the NICHD K12 WRHR training program in order to maintain the academic physician-scientist workforce, with proven overall scholarly productivity above or at least commensurate with that reported by others in the literature.12,15,16,17,18,19,20,21 Given known inequities in NIH funding by gender,6 with the proportions of women funded lagging behind men, our findings suggest the infrastructure provided by an institutional K award may be advantageous for obstetrician-gynecologists. Discussions at the national level among K12 Principal Investigators and Research Directors on how to improve scholarly success and the diversity of obstetrics and gynecology physician–scientists through careful targeted recruitment, developing programs that support faculty who are not only surgeons but parents, and providing mentorship and bridge funding are ongoing and echo other national conversations.25, 26 Enhancement of the racial and ethnic diversity of our ob-gyn academic workforce is a high priority for K12 leaders. Our findings support improving national networking programs and better “bridging of mentorship” in the years immediately following WRHR completion, providing improved K to R rates, but also improving overall career success. Consideration of mechanisms for improving NIH funding of benign gynecologic conditions may be warranted. Although competition for NIH K12 WRHR awards is steep, years of national collaborations and sharing of good practices have maintained a high level of success and supports the vision of NICHD.26
Supplementary Material
Tweetable statement:
The infrastructure provided by an institutional K award is an advantageous career development mechanism for obstetrician-gynecologists, a group of predominantly female surgeons.
AJOG at a Glance:
A. Why was this cohort study conducted?
Obstetrician-gynecologist early-stage investigators receive <1% of National Institutes of Health (NIH) overall funding
We aimed to evaluate scholarly productivity of the NIH Women’s Reproductive Health Research (WRHR) K12 career development award graduates
B. What are the key findings?
40% achieved additional NIH funding as PIs
25% were subsequent R01 PI recipients
63% had ≥25 publications
71% had a relative citation ratio score ≥5, indicating publication impact ratings ≥5-fold greater than other NIH-funded scientists in similar area of research
C. What does this study add to what is already known?
The WRHR K12 program is a successful career development program for a predominantly female obstetrician-gynecologist academic physician-scientist workforce, with proven overall scholarly productivity above or commensurate with peers
Acknowledgements:
We would like to thank Erica Lokken, PhD for her input on study design and Nicole Wothe, BS, Megan Lee, BA, Sally Sneden, JD, and Sonya Pien, BA for their assistance with study administration and manuscript preparation.
Funding:
This study was supported by Eunice Kennedy Shriver National Institute of Child Health and Human Development NICHD K12HD001264 and the Department of Obstetrics and Gynecology University of Washington School of Medicine. The University of Washington Institute for Translational Health Science National Institutes of Health NIH UL1 TR002319 supported REDCap data management. The funding source had no involvement in the study design, data collection, analysis and interpretation of the data, writing of the report or decision to submit for publication.
Footnotes
Disclosure potential conflicts of interest: Authors EEM, RK, DL, EO, AM, CGB, NS, EH, LH, SDR, BG have no conflicts to disclose.
Presentation: This manuscript was conceived by all WRHR centers in 2019. The data was presented and reviewed by all current WRHR Centers (PIs and RDs) at the 2022 Annual National WRHR meeting at Yale University, New Haven, Connecticut, November 30, 2022. Current centers include: Baylor - Michael A. Belfort, Kjersti M. Aagard, Chandrasekhar Yallampalli; Brigham and Women’s at Harvard University - Nawal Nour, Raina Fichorova; Duke – Matthew D. Barber, Evan Meyers; Johns Hopkins University – Andrew J. Satin, James H. Segars; Northwestern University - Serdar E. Bulun, Melissa Simon; Oregon Health Sciences University - Aaron B. Caughey, Jeffrey Jensen; Stanford - Leslee L. Subak, Virginia Winn; University of California San Diego - Cynthia Gyamfi-Bannerman, Louise C. Laurent; University of California San Francisco –Rebecca Jackson (Amy Murtha), Vanessa Jacoby, Synthia H. Mellon; University of Colorado Denver - Nanette Santoro, T. Rajendra Kumar, Jeanelle Sheeder; University of North Carolina Chapel Hill – Genevieve Neal-Perry, Benjamin H. Chi; University of Pennsylvania - Elizabeth A. Howell, Christos Coutifaris, University of Utah- Robert M. Silver, Ingrid Nygaard; University of Washington – Barbara A. Goff, Elizabeth M. Swisher; and Yale University - Hugh S. Taylor, Vikki M. Abrahams.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
References
- 1.https://www.aamc.org/news-insights/what-specialties-do-doctors-choose (accessed on February 10, 2023).
- 2.Townsend J. Ed. Institute of Medicine. Committee on Research Capabilities of Academic Departments of Obstetrics and Gynecology. Strengthening research in academic OB/GYN departments. Washington, DC: National Academy Press; 1992. [Google Scholar]
- 3.Parchem JG, Townsel CD, Wermoint SA, Afshar Y. More than grit: growing and sustaining physician-scientists in obstetrics and gynecology. Am J Obstet Gynecol 2022. Jan;226(1):1–11.. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Mirin AA. Gender Disparity in the Funding of Diseases by the U.S. National Institutes of Health. J Women’s Health 2021. Epub 2020 Nov 27.. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Okeigwe I, Wang C, Politch JA, Heffner LJ, Kuohung W. Physician-scientists in obstetrics and gynecology: predictors of success in obtaining independent research funding. Am J Obstet Gynecol. 2017. Jul;217(1):84.e1–84.e8. Epub 2017 Mar 16.. [DOI] [PubMed] [Google Scholar]
- 6.Lauer MS, Roychowdhury D. Inequalities in the distribution of National Institutes of Health research project grant funding. Elife 2021. Sep 3;10:e71712. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Rice LW, Cedars MI, Sadovshy Y, Siddiqui NY, Teal SB, Wright JD, et al. Increasing NIH funding for academic departments of obstetrics and gynecology: a call to action. Am J Obstet Gynecol 2020. Jul;223(1):79.e1–79.e8.. [DOI] [PubMed] [Google Scholar]
- 8.Nikaj S, Lund PK. The Impact of Individual Mentored Career Development (K) Awards on the Research Trajectories of Early-Career Scientists. Acad Med. 2019. May;94(5):708–714.. [DOI] [PubMed] [Google Scholar]
- 9.Amory JK, Louden D, McKinney C, Rich J, Long-Genovese S, Disis ML Scholarly productivity and professional advancement of junior researchers receiving KL2, K23, or K08 awards at a large public research institution. J Clin Transl Sci 2017. Apr;1(2), 140–143. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Twombly DA, Glavin SL, Guimond J, Taymans S, Spong CY, Bianchi DW. Association of National Institute of Child Health and Human Development Career Development Awards With Subsequent Research Project Grant Funding. JAMA Pediatr. 2018. Mar 1;172(3):226–231.. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Hutchins BI, Yuan X, Anderson JM, Santangelo GM. Relative Citation Ratio (RCR): A New Metric That Uses Citation Rates to Measure Influence at the Article Level. PLoS Biol. 2016. Sep 6;14(9):e1002541.. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Halvorson MA, Finlay AK, Cronkite RC, Bi X, Hayashi K, Maisel NC, Amundson EO, Weitlauf JC, Litt IF, Owens DK, Timko C, Cucciare MA, Finney JW. Ten-Year Publication Trajectories of Health Services Research Career Development Award Recipients: Collaboration, Awardee Characteristics, and Productivity Correlates. Eval Health Prof. 2016. Mar;39(1):49–64. Epub 2014 Jul 11.. [DOI] [PubMed] [Google Scholar]
- 13.Rittenberg E, Liebman JB, Rexrode KM. Primary Care Physician Gender and Electronic Health Record Workload. J Gen Intern Med 2022. Oct;37(13):3295–3301. Epub 2022 Jan 6.. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Huang J, Gates AJ, Sinatra R, Barabasi AL Historical comparison of gender inequality in scientific careers across countries and disciplines. Proc Natl Acad Sci U S A. 2020. Mar 3;117(9): p. 4609–4616.. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Tomaskovic-Devey D, Women’s Work: Gender Equality vs. Hierarchy in the Life Sciences. 417 Administrative Science Quarterly, 2005. 50(4): p. 661–662. [Book Review]. [Google Scholar]
- 16.Whittington KB and Smith-Doerr L Gender and Commercial Science: Women’s Patenting in the Life Sciences. J Technol Transfer, 2005. 30(4): p. 355–370. [Google Scholar]
- 17.Bostwick VK, Weinberg BA. Nevertheless She Persisted? Gender Peer Effects in Doctoral STEM Programs. National Bureau of Economic Research: NBER Working Paper Series. 2018. http://www.nber.org/papers/w25028. [DOI] [PMC free article] [PubMed]
- 18.Mairesse J, Pezzoni M Does Gender Affect Scientific Productivity? Revue économique, 423 2015. 66(1): p. 65–113. [Google Scholar]
- 19.Klug A Rosalind Franklin and the discovery of the structure of DNA. Nature, 1968. Aug 24;219(5156):808–810. [DOI] [PubMed] [Google Scholar]
- 20.Isaacson W The Code Breaker: Jennifer Doudna, Gene Editing, and the Future of the Human Race. 2021: Simon and Schuster. [DOI] [PubMed] [Google Scholar]
- 21.Ross MB, Glennon BM, Murciano-Goroff R, Berkes EG, Weinberg BA, Lane JI. Women are credited less in science than men. Nature 2022. Aug;608(7921):135–145. Epub 2022 Jun 22.. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Jagsi R, Motormura AR, Griffith KA, Rangarajan S, Ubel PA. Sex differences in attainment of independent funding by career development awardees. Ann Intern Med 2009. Dec 1;151(11):804–11.. [DOI] [PubMed] [Google Scholar]
- 23.Mitra AN, Aurora N, Grover S, Ananth CV, Brandt JS. A bibliometric analysis of obstetrics and gynecology articles with highest relative citation ratios, 1980 to 2019. Am J Obstet Gynecol MFM 2021 Jan;3(1):100293. Epub 2020 Dec 11.. [DOI] [PubMed] [Google Scholar]
- 24.Meyer VM, Benjamens S, Moumni ME, Lange JFM. Global Overview of Response Rates in Patient and Health Care Professional Surveys in Surgery: A Systematic Review. Ann Surg 2022. Jan 1;275(1):e75–e81.. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Jain MK, Cheung VG, Utz PJ, Kobika BK, Yamada T. Saving the Endangered Physician Scientist - A Plan for Accelerating Medical Breakthroughs. N Engl J Med 2018. Aug 1;381(5):399–402. [DOI] [PubMed] [Google Scholar]
- 26.Huang X, Dovat S, Mailman RB, Thiboutot DM, Berini D, Parent LJ. Building a System to Engage and Sustain Research Careers for Physicians. Acad Med 2021. Apr 1;96(4):490–494.. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.