No evidence that gender contributes to personal research funding success in The Netherlands: A reaction to van der Lee and Ellemers

A recent PNAS article (1) argues that success rates for attaining research grants are gender-biased. However, the overall gender effect borders on statistical significance, despite the large sample. Moreover, their conclusion could be a prime example of Simpson’s paradox (2, 3); if a higher percentage of women apply for grants in more competitive scientific disciplines (i.e., with low application success rates for both men and women), then an analysis across all disciplines could incorrectly show “evidence” of gender inequality. Indeed, the social sciences and medical sciences are the two fields with a high proportion of female applicants as well as a low application success rate (table S1 in ref. 1). Moreover, multiple comparisons (across disciplines) are conducted without correcting for alpha inflation. Furthermore, it cannot be ruled out that the findings are artifacts due to unmeasured conditions, because no control variables were included. Finally, possible composition effects are ignored.

We analyzed data from the field of the social sciences in the Netherlands Organization of Scientific Research (NWO) consisting of 8,687 individual applications to all grants announced in the period between 2006 and 2013 (not just the Veni grant). Taking nesting within institutions and years into account (intraclass correlation coefficient = 14.5% in the empty model), bivariate analyses of the Veni grant application show no or just borderline significance (P = 0.062), whereas bivariate analyses of all applications show a highly significant result, which seems to support the conclusion of van der Lee and Ellemers (1). However, when type of grant and social scientific field are included—separately or together—the results show no evidence to reject the null hypothesis of gender equality. Also, no interaction is found between gender and these conditions.

In short, we find no convincing evidence for gender inequality. However, based on our findings, we also may not conclude that there is no gender inequality in NWO grant application success. Rather, it is too soon to spend public money on changing the evaluation procedures and gender balancing programs within the Science Foundation in The Netherlands. More in-depth analyses with statistical techniques that overcome the above-mentioned issues are needed before jumping to conclusions about gender inequality in grant awards.

Our analyses are summarized in Table 1 and more detailed analyses are available on request.

Table 1.

Binomial multilevel model on research grant funding in The Netherlands (n = 8,687)

Variables	Only Veni		All grant applications		Added: subdiscipline and type of grant
	Odds ratio	P value	Odds ratio	P value	Odds ratio	P value
Women	0.77	0.062	0.87	0.023	0.92	0.185
Grant type (reference = all other grants)
PhD					0.47	0.000
Veni					0.36	0.000
Vidi					0.40	0.000
Vici					0.31	0.000
Rubicon					0.68	0.000
Prog Edu					0.68	0.000
Ora					0.25	0.000
Social science discipline (reference = psychology)
Organizational Sciences					0.40	0.000
Administrative Sciences					0.65	0.002
Communication Sciences					0.49	0.012
Anthropology					0.61	0.022
Demography					0.91	0.781
Economics					0.73	0.008
Geography					0.88	0.294
Environmental sciences					0.22	0.000
Educational sciences					0.51	0.000
Law					0.60	0.000
Sociology					0.65	0.000
Unknown discipline					0.81	0.001
Intercept	0.17	0.00	0.22	0.00	0.58	0.000
Variance institution_year coefficient (SD)	0.03 (0.05)		0.10 (0.02)		0.08 (0.02)
(Wald) chi2	3.47	0.06	5.17	0.00	276.28	0.000