Joel et al.'s method systematically fails to detect large, consistent sex differences

In their widely publicized paper, Joel et al. (1) make two empirical claims about sex differences in features of the human brain: (i) “…internal consistency [in individuals’ sex-differentiated brain features] is rare” (p. 15472) and (ii) the amount of overlap in sex-differentiated features of male and female brains “undermines any attempt to distinguish between a ‘male’ and a ‘female’ form for specific brain features” (p. 15471). We argue that claim i is based on faulty methodology, and claim ii is misleading if extended to overall sex differences in brain structure.

In regard to claim ii, Joel et al. (1) did not conduct analyses (e.g., discriminant analyses) designed to test how well various brain features predicted participants’ sex. Performing such analyses on the data of Joel et al. (1), we found that brain features correctly predicted subjects’ sex about 69–77% of the time (2). Moreover, the multivariate overlap of female and male distributions based on the same variables was moderate (42% on average), and certainly not so large as to invalidate the idea of overall sex differences in brain structure.

As for claim i, the definition of “internal consistency” Joel et al. (1) use is so extreme that, in realistic conditions, it can only generate results consistent with their hypothesis. Via simulations, we systematically varied sample characteristics such as the magnitude of sex differences and correlations among variables. Some of the simulated scenarios were intentionally unrealistic, involving uniformly strong sexual dimorphism and/or extremely high correlations between variables (up to r = 0.90). Despite this, the proportion of “internally consistent” profiles remained low in all conditions. Under more realistic assumptions, the method of Joel et al. (1) virtually always returned the same pattern of results—a preponderance of “substantially variable” profiles, a minority of “intermediate profiles,” and a very small proportion (often close to zero) of “sex-typical” profiles (2).

These results are supported by Joel et al.’s own analysis of sex-typed activities (figure 2 in ref. 1, p. 15471; data from ref. 3). Taken together, activities showed a clear bimodal distribution with almost no overlap between the sexes. However, even here, Joel et al. (1) report that “55% of subjects showed substantial variability and only 1.2% were internally consistent.” In short, there seems to be no degree of sexual dimorphism in realistic datasets that will yield results that falsify the hypotheses of Joel et al. (1).

To further reinforce this point, we applied Joel et al.’s methods to facial morphology features in three species of monkeys (crab-eating macaques, grivets, and tufted capuchins). Our goal was to see what percent of individual monkeys would display internally consistent species-typical profiles of features (2). Across comparisons, only 1.1–5.1% of the monkeys showed consistent “species-typical” profiles, whereas 18.9–25.3% had “intermediate” profiles (as expected based on the extremely large size of species differences, there were no “substantially variable” profiles). If the methods of Joel et al. (1) cannot demonstrate consistency in morphological features that distinguish distinct species, is it any wonder that they cannot demonstrate within-individual consistency in sexually differentiated brain structures and behaviors in humans?