Importance of accurately measuring spatial abilities

Hoffman et al. (1) suggested that identifying the cultural foundations of sex differences in spatial abilities may uncover the reasons why women tend to be underrepresented in fields of science, technology, and engineering. The authors ingeniously selected their sample from two tribes from Northeast India that appear to differ little in terms of genetic heritage but live in patrilineal and matrilineal societies, respectively. Women from the patrilineal Karbi tribe took longer than their male counterparts to assemble a four-piece jigsaw puzzle. No sex difference in task performance was found in the matrilineal Khasi tribe. This finding suggests that nurture may affect women's spatial abilities, which could have important implications for determining career trajectories. Crucially, this inference rests on the assumption that the puzzle task used in this study is a reasonably valid measure that can reliably gauge substantive sex differences in spatial abilities. For several reasons, this assumption is questionable.

First, the puzzle task the authors selected fails to target the aspects of spatial ability in which notable sex differences have consistently been demonstrated. Large metaanalyses have shown men to outperform women repeatedly on mental rotation (d = 0.56–0.73) and spatial perception (d = 0.44) tasks (2, 3). However, few sex differences have been found on spatial visualization tasks, such as the four-piece puzzle task, that habitually involve deriving an image from a set of fragmented parts (e.g., Paper Form Board, Block Design) (d = 0.13–0.19) (2, 3). Given the contrast between the effect size expected from the extant literature and that identified in the current study [d = 0.13–0.19 vs. d = 0.31 (patrilineal Karbi tribe)], the authors have done little to rule out alternative explanations for the core result.

Fundamentally, this result may be a product of the unknown quality of the puzzle task combined with its vulnerability to extraneous influences. The authors have not benchmarked performance on the puzzle task against well-validated measures of spatial ability. Furthermore, the “one-shot” nature of the task contrasts sharply with traditional spatial reasoning assessments that include numerous trials to reduce measurement error and to reliably gauge ability. Critically, the authors rely solely on a measure of the time required to complete the puzzle task and do not assess the participants’ accuracy in making difficult spatial judgments. By using time-to-complete as the core outcome on an uncomplicated single-trial task, the authors may have inadvertently introduced numerous established culturally influenced motivational and personality sex-related biases. A broad set of such factors, including lack of assertiveness, low impulsivity and activity, poor self-esteem, and increased tenderness, could each explain why females took longer to complete the cognitively undemanding task in the patrilineal society (4).

To ensure the validity of the results presented by Hoffman et al. (1), it is therefore of vital importance that further steps be taken to ensure that the experimental task used can reliably measure spatial abilities and that the sex differences identified are robust to the influence of other key traits that are likely to have affected performance.