We read the response of Albers (1) to the article by van der Lee and Ellemers (2) with interest. In our paper, we provide evidence of gender disparities in grant funding success for early career researchers in the Netherlands Organization for Scientific Research (NWO). Albers (1) argues that our conclusions are based on “inappropriate statistical procedure… due to… Simpson’s paradox” (3).
In our case, Simpson’s paradox would indicate that the overall gender effect in awarding rates is simply explained by the observation that more women tend to apply in disciplines where funding rates are relatively low. Correcting for scientific discipline indeed reduces the effect of applicant gender, so that the overall effect is no longer significant. However, there is a significant interaction between scientific discipline and applicant gender [Wald(8) = 17.574, P = 0.025], which warrants within-discipline testing. Importantly, as fully disclosed in table S1 in our article (2), these tests revealed uneven distributions of awarding rates within disciplines. Lower awarding rates for women compared with men were mainly observed in the disciplines with a high proportion of female applicants, and relatively low success rates overall. Simpson’s paradox cannot explain these gender disparities that we observed within disciplines.
In our publication, we explicitly note that differences in awarding rates were observed in the disciplines in which women are most strongly represented (with 51%), in which most applications are submitted, and in which the overall awarding rates are low. We refer to prior research suggesting that these particular conditions make evaluators more vulnerable to reliance on stereotypical expectations as cognitive heuristics in complex decision making. Further, Simpson’s paradox cannot explain that fewer women than men are selected for the next phase in each step of the review procedure, and, again, these disparities are most clearly visible in the disciplines with a relatively high proportion of female applicants. Additionally, we report that, overall, women were less likely to be prioritized for their “quality of researcher” than men, whereas no difference between male and female applicants was observed for the “quality of proposal” or “knowledge utilization” ratings. Finally, the instructional and evaluation materials reviewers received contain gendered language favoring male applicants over female applicants.
The main conclusion from our study, that applicant gender contributes to early career funding success, is based on all these findings taken together. Albers (1) ignores the convergence of evidence from different indicators and questions the validity of our overall conclusion, based on concerns regarding one statistical test. This response resonates with results from recent research revealing a broader tendency of scholars’ reluctance to accept empirical evidence demonstrating implicit gender bias in science (4). Unwarranted criticism such as posted here may disrupt attempts aiming to increase equal gender representation in academia.
Footnotes
The authors declare no conflict of interest.
References
- 1.Albers CJ. Dutch research funding, gender bias, and Simpson’s paradox. Proc Natl Acad Sci USA. 2015;112:E6828–E6829. doi: 10.1073/pnas.1518936112. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.van der Lee R, Ellemers N. Gender contributes to personal research funding success in The Netherlands. Proc Natl Acad Sci USA. 2015;112(40):12349–12353. doi: 10.1073/pnas.1510159112. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Simpson EH. The interpretation of interaction in contingency tables. J R Stat Soc Series B Stat Methodol. 1951;13(2):238–241. [Google Scholar]
- 4.Handley IM, Brown ER, Moss-Racusin CA, Smith JL. Quality of evidence revealing subtle gender biases in science is in the eye of the beholder. Proc Natl Acad Sci USA. 2015;112(43):13201–13206. doi: 10.1073/pnas.1510649112. [DOI] [PMC free article] [PubMed] [Google Scholar]