Abstract
Abstract
Objectives
Most first-time biomedical research grant applications are not funded. In the challenging research funding climate, resubmitting a grant application is a necessary task for scientists. Identifying which factors influence their decision to resubmit and the success of resubmissions will inform funders and applicants. However, data on resubmissions are fragmented and under-reported. In this scoping review, we aimed to summarise (1) the outcomes of resubmitting biomedical research grant applications and (2) the demographic characteristics of scientists who resubmitted grant applications.
Design
Scoping review with reporting informed by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses.
Data sources
MEDLINE, CINAHL, EMBASE, Cochrane Central Registrar of Controlled Trials CENTRAL, PsycINFO, Web of Science and grey literature sources were searched through November 2022.
Eligibility criteria
We included peer-reviewed and grey literature records from the biomedical sciences that reported outcomes of the resubmission process (eg, resubmission success rate, rate of resubmission) and information about the scientists who resubmit grant applications (eg, sex, race, career stage).
Data extraction and synthesis
Data were extracted independently by two reviewers. The data were cross-referenced and any conflicts were resolved via consensus. Data were summarised descriptively and presented in tables and figures.
Results
Resubmissions represented a substantial proportion of applications (lowest prevalence rate: 4%; highest prevalence rate: 56%) in a given funding cycle and were reliably more successful than first-time applications (lowest success rate: 16%; highest success rate: 82%)—a phenomenon associated with several sociodemographic, institutional and project-related factors. There was conflicting evidence about the relationship of sociodemographic-related, institution-related and project-related factors to resubmission likelihood and success.
Conclusion
The resubmission process is a time-consuming and often frustrating experience for researchers. Our review identified opportunities to streamline and improve the process to enhance the biomedical research landscape.
Keywords: Health, Health policy, Research Design
STRENGTHS AND LIMITATIONS OF THIS STUDY.
Given the broad scope of our search strategy, we are confident that we have captured the available evidence related to biomedical grant resubmissions.
We adhered to open science principles that facilitate transparency and research integrity.
The institutions included in our analysis were located in high-income, mostly English-speaking, Western nations.
Data extracted from grey literature were primarily available in aggregate form, and we used Web Plot Digitizer (https://plotdigitizer.com/) to extract relevant values.
As this study was a scoping review, no risk of bias assessment was performed for the included studies.
Introduction
Research grants fund advances in human health, including new ways to prevent and treat disease. It has never been harder for a scientist to get funding. First-time application success rates have declined across the last two decades.1 In high-income Western nations, the success rate for a new grant application is usually below 20%2,4; in other places, substantially lower.5 As a result, scientists are well acquainted with resubmitting unsuccessful grant applications to obtain funding in the current hypercompetitive environment.
Funding agencies also encourage applicants to resubmit unsuccessful applications in future competitions.6 However, data on the number of successful resubmissions, factors related to the decision to resubmit, and what makes a resubmission more likely to be successful—all of which could strategically inform researchers’ decisions and agencies’ policies—are difficult to find, and published studies are often cross-sectional. This lack of longitudinal or historical trends is a key knowledge gap. Scientists are increasingly prompted to engage in open science practices, including data transparency and yet data from most funding agencies remain out of from the public domain. It is also unclear whether demographic and institutional factors predict who does and does not resubmit a grant application. Some under-represented groups might have to resubmit grant applications more times before getting funded or may be less likely to resubmit an unsuccessful application.7 Obtaining public research grant funding is becoming increasingly difficult, and women, racialised individuals and early-career researchers face greater barriers than others.8,10
We want to do more than simply encourage a researcher to persist with applying for grants—in this review, we chart the available data on resubmission processes, outcomes and the scientists who resubmit their grants. Better informed applicants will find it easier to navigate their resubmission decisions and funders may see ways to streamline and improve their granting processes.
The objectives of this scoping review were to summarise (1) the outcomes of resubmitting biomedical research grant applications to competitive funding agencies and (2) the demographic characteristics of scientists who resubmitted their grant applications.
Methods
The search strategies, data extraction instrument and protocol for this scoping review were prospectively registered on the Open Science Framework on 5 January 2023.11 The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA-ScR) statement guided the conduct and reporting of our review. We used scoping review methods due to the diverse nature of the available literature and the broad nature of our objectives.12 The methodological guidance for conducting a scoping review provided by the Joanna Briggs Institute was followed.13 Here, we provide a summary of our scoping review methods and direct readers to our protocol11 for additional information.
Information sources and search
In November 2022, we searched the following databases (platforms) for peer-reviewed academic literature on biomedical grant resubmissions: MEDLINE (Ovid), CINAHL (Ebsco), EMBASE (Ovid), Cochrane Central Registrar of Controlled Trials CENTRAL (Ovid), PsycINFO (Ebsco) and Web of Science (University of British Columbia Institutional Access). The search strategy was developed in consultation with a knowledge synthesis and medical liaison librarian (VK) and a Peer Review of Electronic Search Strategy was performed by an independent librarian. The search strategies used for each database are available in online supplemental appendix 1.
We searched grey literature in August/September 2022. The body of literature in this field has grown slowly over time so we do not expect there were substantial advances in this area that would change our conclusions since our search was performed. Two researchers (AML and HK) used a keyword search to explore websites for the 50 largest (by investment amount) international philanthropic and public research funding agencies (listed at https://www.healthresearchfunders.org/). Keywords were “resubmit”, “resubmits”, “resubmission” and “resubmitted”.
Article screening
All records were uploaded to Covidence (Veritas Health Innovation, Melbourne) ahead of eligibility screening. Duplicates were removed using Covidence’s algorithm and manual deduplication. Four researchers (AML, CLA, HK and JW), working in pairs, screened results from the database searches. One pair of reviewers (AML and HK) screened results from the grey literature searches. Conflicts were resolved via consensus; a third reviewer was available to resolve disagreements if required.
Eligibility criteria
In this review, we discuss resubmission in the context of an individual grant application that was initially unsuccessful and then submitted to the same funding agency in a subsequent competition (rather than a process where scientists apply for additional, renewal or other funding after an initial application or award).
Included sources must have reported at least one outcome measure related to the process of resubmitting a biomedical grant or the success of a biomedical grant resubmission. Resubmission data from fields outside the biomedical sciences (eg, social sciences, natural sciences) were excluded. In cases where resubmission data were aggregated with original grant submission data, and the resubmission data could not be separated, the source was excluded.
Studies were included regardless of funding agency type, study design or date of publication. Only studies published in English were included. Full details are available in the study protocol.11
Data charting
Data were independently extracted by AML and HK using a custom form.11 The data were cross-referenced, and any conflicts were resolved via consensus; a third reviewer was available to resolve disagreements if required.
Data items and synthesis
We organised the data according to themes that emerged through the data extraction process. These themes were organised according to our two main research objectives:
Objective 1: summarise the outcomes of resubmitting biomedical research grant applications
The themes were rate of resubmission (how many scientists resubmitted their unsuccessful grant applications?), resubmission prevalence (what proportion of grant applications were resubmissions?), resubmission success rate (how successful were resubmitted grant applications?), downstream funding success (was an applicant more likely to be successful in the long term if they resubmitted an unsuccessful application or submitted a new application?), applicants’ experiences (what issues did applicants identify in biomedical grant resubmission?) and time (what was the time cost of resubmitting?).
Objective 2: summarise the demographic characteristics of scientists who resubmitted their grant applications
The themes were as follows: variables associated with resubmission (characteristics associated with whether a scientist was more or less likely to resubmit) and variables associated with resubmission success (factors associated with higher or lower likelihood of resubmission success).
Applicant-related variables associated with the likelihood and success of resubmission were race, sex, gender, career stage and field of study. Application-related variables associated with the likelihood and success of resubmission were original submission scores and ranking, distance from the payline, clinical versus non-clinical research and human versus animal research. See online supplemental appendix 2 for a description of each of these variables.
The data were summarised descriptively and presented in tables and figures. Data were organised into a Microsoft Excel (Redmond, USA) spreadsheet and figures were generated using Flourish (https://flourish.studio/) (London, UK).
Patient involvement
Patient partners were not directly involved in the construction of the research question, selection of the study design and outcome measures, or interpretation of the results.
Results
We screened 16 864 bibliographic records and obtained 138 records for full-text review. We excluded 104 records that did not report resubmission data, 4 that did not disaggregate resubmission data from other submission data, 4 duplicate records and 1 non-English record. 96 results were returned from the grey literature search, 15 of which met the inclusion criteria; we excluded 85 records that did not report resubmission data. Two additional records were added through citation tracking.14 15 The search yielded 40 records for review. 31 studies (78%) reported data from an American funding agency and 27 (68%) reported data from the National Institutes of Health (NIH) or one of its members. The results from our search are presented using PRISMA-ScR flow diagram16 (online supplemental appendix 3). The figure was generated using PRISMA 2020.17
Study characteristics
The 40 records included in this scoping review reported data from the following research funders: The NIH, National Institute on Aging, the US Agency for Healthcare Research and Quality, National Heart Lung and Blood Institute, National Institute of Allergy and Infectious Disease, National Science Foundation, American Cancer Society, Canadian Institutes of Health Research (CIHR), the European Union Research Council, Netherlands Organization for Scientific Research, the UK National Health Service, Australian National Health and Medicine Research Council, and Swiss National Science Foundation (online supplemental appendix 4). The most common award studied is the NIH R01—‘an award made to support a discrete, specified, circumscribed project to be performed by the named investigator(s) in an area representing the investigator’s specific interest and competencies, based on the mission of the NIH’.18
Rate of resubmission (how many scientists resubmit their unsuccessful grant applications?)
Five studies reported the proportion of researchers who resubmitted their unsuccessful grant applications (figure 1). Values ranged from 18% to 50%; studies with larger samples reported lower rates of resubmission.
Figure 1. Rate of resubmission, represented as the proportion (%) of applicants (white text at the end of each bar) who resubmitted unsuccessful research grant applications, listed by funding institution, award type, nation and sample size (if reported). EU, European Union.
13 studies explored the applicant-related or application-related variables that were associated with the likelihood of resubmission. These findings are summarised in figure 2 and the variables are described in online supplemental appendix 2. There was no consistent reporting of effect sizes or statistical significance.
Figure 2. Variables associated with rate of resubmission. (Green=the variable is associated with a higher likelihood of resubmission; Grey=the variable is not associated with a change in likelihood of resubmission; Pink=the variable is associated with a lower likelihood of resubmission).
Original submissions, primarily those to the NIH, that were reviewed and rated more favourably were more likely to be resubmitted. The influence of sex and gender was heterogeneous: females and women were more, less and equally likely to resubmit their unsuccessful grants.8 19 20 African American/Black scientists were less likely to resubmit grants,8 largely explained by the discrepancy in original submission scoring.21 When original submission score was accounted for, African American/black and white scientists were resubmitting at roughly the same rate.
Resubmission prevalence (what proportion of grant applications are resubmissions?)
Figure 3 displays the proportion of resubmissions (as a percentage of total applications) over time in R01 competitions at the NIH, along with data points from the CIHR and one of the individual NIH institutes (The National Heart, Lung and Blood Institute). The highest resubmission prevalence was 56% for A1 submissions (first resubmissions) to the NIH in 2013 and the lowest was 4% for A2 submissions (second resubmissions, which were being phased out) to the NIH in 2012.
Figure 3. Proportion of resubmissions over time, represented as the proportion of resubmissions in a given application cycle. (A1=a first resubmission; A2=a second resubmission). CIHR, Canadian Institutes of Health Research; NHLBI, National Heart, Lung and Blood Institute; NIH, National Institutes of Health.
Resubmission success rate (how successful are resubmitted grant applications?)
Figure 4 shows historical trends in application success rate for R01 awards at the NIH.22 Resubmissions (A1 and A2) had a consistently higher rate of success than new applications (A0). While second resubmissions (A2) were phased out through a 2008 policy change, they had the greatest likelihood of success among all three types of applications while they were still permitted. Since second resubmissions were eliminated, first resubmissions (A1) have had a steadily higher likelihood of success when compared with new applications.
Figure 4. Success rates for resubmitted R01 applications at the National Institutes of Health (NIH). (A0=original submission; A1=first resubmission; A2=second resubmission).
11 studies discussed the success rate for resubmitted grant applications. Studies that directly reported a success rate for resubmissions are summarised in online supplemental appendix 5. In all but one study,23 resubmitted grant applications were more successful than first-time applications. The studies were heterogeneous and cross-sectional, although Lindman et al described the funding success of a small group of cardiovascular physician–scientists across a 4-year period.24 In data from 2011 to 2014, resubmissions consistently demonstrated a better success rate than new applications, although with significant variation.
A 2022 study by Souder et al reported the outcomes of a training course for physician–scientists applying for individual fellowship awards (F Awards) from the NIH.25 After completing the course, there was no significant change in the success rate of resubmitted F Award applications, although the sample size was limited to only two applications.
Five studies explored the applicant-related or application-related variables that were associated with the resubmission success rate (figure 5). The applicant-related variables reported were career stage, time spent on the resubmission, whether or not the applicant had a subsequent award, and whether or not the applicant had completed a training programme. Application-related variables were whether or not the original submission was discussed, its score, total requested funds, human versus animal sample and how many PIs were included in the application. More information about each variable is available in online supplemental appendix 2. Original submissions that were reviewed more favourably were more likely to be successful on resubmission. There was no consistent reporting of effect sizes or statistical significance across reported data.
Figure 5. Variables associated with resubmission success rate (Green=the variable is associated with a higher likelihood of resubmission; Grey=the variable is not associated with a change in likelihood of resubmission; Pink=the variable is associated with a lower likelihood of resubmission; PI=principal investigator).
Downstream funding success (is an applicant more likely to be successful in the long term if they resubmit an unsuccessful application or submit a new application?)
Doyle et al26 reported the downstream funding success of 11 808 early career R01 applicants at 3 and 5 years after their original submission. Applicants who resubmitted their original unfunded application were compared with applicants who submitted a new application, and the groups were stratified by whether or not the original application was discussed (discussed applications indicating a more favourable review). Resubmissions were more successful than new applications, irrespective of whether the original application was discussed (figure 6).
Figure 6. Downstream funding success of R01 applicants to the National Institutes of Health.
Applicants’ experiences (what issues do applicants identify in biomedical grant resubmission?)
NIH K-award recipients emphasised the importance of persistence in the face of rejection and how gender may influence an applicant’s response to negative feedback.15 Direct quotes are presented in online supplemental appendix 6.
Canadian scientists alluded to their logistical difficulties when resubmitting CIHR grant applications, a lack of information about the reviewers who would be judging their resubmitted grant applications, and how to address reviewers’ comments when comments conflicted or were inappropriate/unhelpful.27
In a recent study by our group (in review), we interviewed grant reviewers, committee chairs and scientific officers at the CIHR. On the topic of grant resubmissions, a committee chair commented on the discrepancy between resubmission policy and practice.
Time (what is the time cost of resubmitting?)
A 2009 NIH policy change limiting researchers to a single resubmission reduced the average time to award for applicants, from 93 to 56 weeks.28
At the Australian National Health and Medical Research Council,29 the estimated average time cost of resubmission was 28 working days. There was no significant relationship between time spent on the resubmitted application and likelihood of success.30
Discussion
We collected and summarised the available information related to resubmitted biomedical grant applications. We addressed questions about the likelihood of pursuing and succeeding with a resubmission versus a new application and have identified other questions that the data do not yet allow us to answer. Our most important finding is that resubmitting a biomedical grant application is a worthwhile endeavour (despite the time and frustration involved) particularly if the original submission was reviewed favourably. There was no clear relationship between applicant and application variables and the likelihood of resubmission or likelihood of resubmission success. The important caveat is how well the original submission performed: favourably reviewed original submissions were more likely to be resubmitted and more likely to be successful on resubmission. Applicants whose original submissions performed well should be highly motivated to resubmit. This finding is further supported by our recent analysis of the CIHR Project Grant competition.31 We encourage other funding agencies to follow the NIH and CIHR in making these data available to applicants and researchers.
Approximately 3 in every 10 applications in a given funding cycle are resubmitted grants, and resubmissions were over-represented among successful grant applications. We encourage applicants to consider resubmitting an unsuccessful grant (even if it was not initially discussed26), rather than beginning a new application. This echoes guidance provided by the NIH.6 What is not clear is whether resubmissions were more successful because they benefit from peer review or because reviewers believe that applications improve iteratively based on the reviews they provide. We wonder: are reviewers providing valuable, actionable feedback to applicants that strengthens their scientific rigour? Or are reviewers using an applicant’s adoption of their suggestions as a proxy of fundability? Or perhaps there is there another explanation for this relationship.
We observed demographic disparities, although the statistical significance of these disparities was often unclear. Sometimes, disparities seemed related to the resubmission process. However, it is possible they originated from the first rounds of peer review. For example, African-American/black scientists simultaneously had to resubmit applications more often to obtain success in the NIH and were less likely to resubmit their applications.9 Yet later, African-American/black scientists resubmitted grant applications at the same rate as other (non-racialised) applicants when accounting for the score of their initial application.21 Resubmission outcomes were likely a product of discrepancies during initial peer review, and that topics studied by black scientists tended to score less favourably.
We suggest that other demographic disparities need examining with nuance to clearly identify the associated, explanatory or mediating variables involved. To view these complex relationships in isolation or without proper context could mean misunderstanding both the problem and possible solutions. Zea and Bowleg32 speculated about reasons why scientists from under-represented minority groups were revising and resubmitting their grant applications to the NIH less frequently than white colleagues. They suggested a lack of mentorship was responsible, where ethnic and racial minorities did not have mentors to guide them in translating comments into a revised application. Institutional resources for researchers during the revision/resubmission process, including protected time, bridge funding and other supports, are also vital to scientists’ success.
We believe that many scientists who have applied for a biomedical research grant can relate to the experiences described in our review. Interview responses range from frustration with administrative tasks like formatting to difficulty interpreting conflicting comments from different reviewers. Considering the substantial time that scientists spend navigating the resubmission process, we suggest that funders have an opportunity to create efficiencies through strategic policy change. Funding agencies can pull the necessary levers to improve the granting process for applicants (by extending resubmission timelines) and agencies themselves (by reducing application burden) through data-informed policy changes.29 The 2022 synthesis by Recio-Saucedo et al summarised the real-life interventions that funders have employed, and we note that only the NIH and Research Council UK have published data related to resubmission policy change. Other funders may have evaluated how their own policies without making the data available. Unfortunately, the current landscape is data-deficient.
Regarding difficulty with interpreting reviewer’s comments, different groups of scientists (eg, under-represented minorities, early career researchers) internalise and respond to comments differently.33,35 This highlights another rich vein for future research to explore what explains the likelihood of funding success. Ongoing training, evaluation and quality improvement for peer review committees is required to ensure quality and professionalism in reviewer feedback.36
Future research
Outside the NIH, there was an absence of publicly available data related to biomedical grant resubmissions. We urge biomedical funding agencies to make data on grant resubmissions available, including measures of resubmission prevalence and resubmission success rate, and application-related and applicant-related variables. Transparency will help applicants make informed decisions about how to spend their grant-writing time and guide evidence-informed policies at funding agencies. Successful future research hinges on the accessibility of this data. For that reason, we do not suggest a systematic review is performed until funders significantly improve their grant-related data transparency.
Funders could begin by introducing or improving existing internal processes to track resubmitted grant applications and then reporting the data publicly. The NIH RePORTER system is exemplary. As funders continue to embrace open science practices—scientists are often required by funders to make their data publicly available—they could simultaneously improve their own information-sharing practices. Applicants benefit from knowing what the success rates are for first-time and resubmitted grant applications, along with resubmission success rates stratified by relevant applicant-related and application-related variables. These data may also help applicants recognise when their likelihood of resubmission success is low, allowing applicants to plan accordingly and reducing the application burden on funding agencies.
Conclusions
Our scoping review collected and summarised the publicly available information on biomedical research grant resubmission worldwide. We discovered a deeply incomplete and fragmented data landscape, which does not fully inform the research questions posed in this review or—perhaps more importantly—researchers navigating the grant application process. Better-informed applicants are a crucial piece of effective and efficient funding practices. Our review reveals rich opportunities for future research, calls for action on adopting open data policies and provides a signal that strategic policy changes might meaningfully impact the pursuit of high-impact scientific research.
supplementary material
Acknowledgements
We would like to thank Dr. Zahra Premji, Health Research Librarian at the University of Victoria (Canada) for providing our Peer Review of Electronic Search Strategies (PRESS).
Footnotes
Funding: This work was supported indirectly through funding provided by the Canadian Institutes of Health Research (CIHR) as a Research Operating Grant (Scientific Directors) held by KMK. CIHR provided no input or guidance on any aspect of the work reported in this scoping review.
Prepublication history and additional supplemental material for this paper are available online. To view these files, please visit the journal online (https://doi.org/10.1136/bmjopen-2024-089927).
Provenance and peer review: Not commissioned; externally peer reviewed.
Patient consent for publication: Not applicable.
Data availability free text: Data reported in this manuscript are available from Lasinsky, Anne; Wrightson, James; Khan, Hassan; Moher, David; Kitchin, Vanessa; Khan, Karim; Ardern, Clare, 2024, 'Replication Data for Biomedical Research Grant Resubmission: Rates and Factors Related to Success—A Scoping Review', https://doi.org/10.5683/SP3/VOK9AB, Borealis.
Patient and public involvement: Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Contributor Information
Anne M Lasinsky, Email: anne.lasinsky@ubc.ca.
James Wrightson, Email: jwrigh15@ubc.ca.
Hassan Khan, Email: hkhan103@uottawa.ca.
David Moher, Email: dmoher@ohri.ca.
Vanessa Kitchin, Email: vanessa.kitchin@ubc.ca.
Karim Khan, Email: karim.khan@ubc.ca.
Clare L Ardern, Email: clare.ardern@ubc.ca.
Data availability statement
Data are available in a public, open access repository.
References
- 1.Data from: NIH data book, research project grants: competing applications, awards, and success rates. 2024. www.report.nih.gov/nihdatabook/category/10 Available.
- 2.Crow JM. What to do when your grant is rejected. Nat New Biol. 2020;578:477–9. doi: 10.1038/d41586-020-00455-0. [DOI] [PubMed] [Google Scholar]
- 3.Cushman P, Hoeksema T, Kouveliotous C, et al. Impact of declining success rates on scientific productivity. Vol. 10. Public Library of Science; 2015. [Google Scholar]
- 4.Canadian Institutesof Health Research https://cihr-irsc.gc.ca/e/50070.html Available.
- 5.Swiss National Science Foundation https://www.snf.ch/en/93E43Wi7LIXzhqTN/news/snsf-starting-grants-2023-67-projects-approved Available.
- 6.National Institutes of Health https://nexus.od.nih.gov/all/2016/10/28/are-you-on-the-fence-about-whether-to-resubmit/ Available.
- 7.Ginther DK, Haak LL, Schaffer WT, et al. Are race, ethnicity, and medical school affiliation associated with NIH R01 type 1 award probability for physician investigators? Acad Med. 2012;87:1516–24. doi: 10.1097/ACM.0b013e31826d726b. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Ginther DK, Kahn S, Schaffer WT. Gender, Race/Ethnicity, and National Institutes of Health R01 Research Awards: Is There Evidence of a Double Bind for Women of Color? Acad Med. 2016;91:1098–107. doi: 10.1097/ACM.0000000000001278. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Ginther DK, Schaffer WT, Schnell J, et al. Race, Ethnicity, and NIH Research Awards. Science. 2011;333:1015–9. doi: 10.1126/science.1196783. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Gallo SA, Schmaling KB, Thompson LA, et al. Grant Review Feedback: Appropriateness and Usefulness. Sci Eng Ethics. 2021;27:18. doi: 10.1007/s11948-021-00295-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Lasinsky A, Ardern C, Kitchin V, et al. Grant resubmission scoping review. OSF; 2023. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Munn Z, Peters MDJ, Stern C, et al. Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Med Res Methodol. 2018;18:143. doi: 10.1186/s12874-018-0611-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Peters MDJ, Marnie C, Tricco AC, et al. Updated methodological guidance for the conduct of scoping reviews. JBI Evid Synth. 2020;18:2119–26. doi: 10.11124/JBIES-20-00167. [DOI] [PubMed] [Google Scholar]
- 14.Sims-Gould J, Lasinsky A, Mota A, et al. “I did not receive training for any of my roles”: threats to grant peer review. 2023
- 15.DeCastro R, Sambuco D, Ubel PA, et al. Batting 300 is good: perspectives of faculty researchers and their mentors on rejection, resilience, and persistence in academic medical careers. Acad Med. 2013;88:497–504. doi: 10.1097/ACM.0b013e318285f3c0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Tricco AC, Lillie E, Zarin W, et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation. Ann Intern Med. 2018;169:467–73. doi: 10.7326/M18-0850. [DOI] [PubMed] [Google Scholar]
- 17.Haddaway NR, Page MJ, Pritchard CC, et al. PRISMA2020: An R package and Shiny app for producing PRISMA 2020-compliant flow diagrams, with interactivity for optimised digital transparency and Open Synthesis. Campbell Syst Rev. 2022;18:e1230. doi: 10.1002/cl2.1230. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.NIH Research Project Grant Program (R01) grants.nih.gov. [09-Jul-2023]. https://grants.nih.gov/grants/funding/r01.htm Available. Accessed.
- 19.Waisbren SE, Bowles H, Hasan T, et al. Gender differences in research grant applications and funding outcomes for medical school faculty. J Womens Health (Larchmt) 2008;17:207–14. doi: 10.1089/jwh.2007.0412. [DOI] [PubMed] [Google Scholar]
- 20.van der Lee R, Ellemers N. Gender contributes to personal research funding success in The Netherlands. Proc Natl Acad Sci U S A. 2015;112:12349–53. doi: 10.1073/pnas.1510159112. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Hoppe TA, Litovitz A, Willis KA, et al. Topic choice contributes to the lower rate of NIH awards to African-American/black scientists. Sci Adv. 2019;5:eaaw7238. doi: 10.1126/sciadv.aaw7238. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.National Institutes of Health NIH RePORT: NIH data book. [03-Oct-2024]. https://report.nih.gov/nihdatabook/category/10 Available. Accessed.
- 23.Wisely J, Haines A. Commissioning a national programme of research and development on the interface between primary and secondary care. BMJ. 1995;311:1080–2. doi: 10.1136/bmj.311.7012.1080. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Lindman BR, Tong CW, Carlson DE, et al. National Institutes of Health Career Development Awards for Cardiovascular Physician-Scientists: Recent Trends and Strategies for Success. J Am Coll Cardiol. 2015;66:1816–27. doi: 10.1016/j.jacc.2015.08.858. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Souder JP, Pepin ME, Seay RL, et al. A novel curricular framework to develop grant writing skills among MD-PhD students. J Clin Transl Sci. 2022;6:e54. doi: 10.1017/cts.2022.384. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Doyle JM, Baiocchi MT, Kiernan M. Downstream funding success of early career researchers for resubmitted versus new applications: A matched cohort. PLoS ONE. 2021;16:e0257559. doi: 10.1371/journal.pone.0257559. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.CIHR Fall 2013 knowledge synthesis pilot: final report. 2013. [28-Jun-2023]. https://cihr-irsc.gc.ca/e/48940.html Available. Accessed.
- 28.NIH Extramural Nexus The A2 resubmission policy continues: a closer look at recent data. [28-Jun-2023]. https://nexus.od.nih.gov/all/2012/11/28/the-a2-resubmission-policy-continues-a-closer-look-at-recent-data/ Available. Accessed.
- 29.Recio-Saucedo A, Crane K, Meadmore K, et al. What works for peer review and decision-making in research funding: a realist synthesis. Res Integr Peer Rev. 2022;7:2. doi: 10.1186/s41073-022-00120-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Herbert DL, Barnett AG, Clarke P, et al. On the time spent preparing grant proposals: an observational study of Australian researchers. BMJ Open. 2013;3:e002800. doi: 10.1136/bmjopen-2013-002800. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Wrightson JG, Lasinsky A, Snell RR, et al. What factors are important to the success of resubmitted grant applications in health research? a retrospective study of over 20,000 applications to the canadian institutes of health research. Health Informatics. doi: 10.1101/2024.05.29.24308137. Preprint. [DOI] [Google Scholar]
- 32.Zea MC, Bowleg L. The Final Frontier-Transitions and Sustainability: From Mentored to Independent Research. AIDS Behav. 2016;20 Suppl 2:311–7. doi: 10.1007/s10461-016-1368-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Pier EL, Raclaw J, Kaatz A, et al. “Your comments are meaner than your score”: score calibration talk influences intra- and inter-panel variability during scientific grant peer review. Res Eval. 2017;26:1–14. doi: 10.1093/reseval/rvw025. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Silbiger NJ, Stubler AD. Unprofessional peer reviews disproportionately harm underrepresented groups in STEM. PeerJ. 2019;7:e8247. doi: 10.7717/peerj.8247. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Biernat M, Carnes M, Filut A, et al. Gender, Race, and Grant Reviews: Translating and Responding to Research Feedback. Pers Soc Psychol Bull. 2020;46:140–54. doi: 10.1177/0146167219845921. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Ardern CL, Martino N, Nag S, et al. Three years of quality assurance data assessing the performance of over 4000 grant peer review contributions to the Canadian Institutes of Health Research Project Grant Competition. Facets (Ott) 2023;8:1–14. doi: 10.1139/facets-2022-0175. [DOI] [Google Scholar]