Challenges for determining the causal effects between social behavior and testosterone

The authors of “Effects of gendered behavior on testosterone in women and men” in PNAS (1) claim that they observed sex differences in testosterone (T) responses following stereotyped acting. We believe, however, that the article contains numerous flaws and that the conclusions are not warranted for several reasons. First, the SEs for the power-wielding conditions in figure 3 of ref. 1 suggest that there are no significant sex differences in the increase of T before and after acting. We think that should be the critical comparison (i.e., the within-subjects factor “increase in T due to acting a power-wielding situation” compared with the between-subjects factor “sex”), but van Anders et al. contrast the acting and watching documentary conditions (within subjects) and compare this for the two sexes. Problematically, the effect of watching a 5-min travel documentary (the neutral control condition) significantly lowered testosterone in women (at a level comparable to that of the acting condition, but in the opposite direction), but not in men. The paper fails to address why. However, this is critically important, because this latter effect contributed to the reported significant effects associated with wielding power.

A second issue is the small and unbalanced sample of participants (15 women and 26 men). Given the variability in males and females, studies of sex differences have typically larger samples. The van Anders et al. (1) study likely has unreliable group estimates, which might partially explain the larger T response following the documentary in women than in men.

A third issue relates to the T measurement. The intra-assay coefficient of variation of testosterone measures in men is 16%, which is large. This measurement error was much smaller in women (7%). More typically, the intra-assay coefficient of variation is higher in women because they have such low testosterone levels to begin with (e.g., ref. 2); in other words, it is harder to measure T in women because values hover around 20–40 pg/mL. Men, on the other hand, usually have T levels that are above 80 pg/mL, which are easier to measure. Furthermore, the researchers used the enzyme-immunoassay method of measurement for women, but the radio-immunoassay method of measurement for men, without providing any good rationale for this approach. These measurement issues may explain why the effect is less robust in men vs. women.

A fourth issue is that the conclusion of van Anders et al. (1) (“gender socialization can contribute to variation in human testosterone levels”) about life-time—hence, sustained effects—cannot be drawn based on this short experimental manipulation. Even if we would grant a significant sex difference in T response in this study, it would take a considerably more complex (e.g., longitudinal) design to support the claims made.