Total grit scale score does not represent perseverance

In PNAS, Jachimowicz et al. (1) claim that the results from a three-study investigation show that passion moderates the perseverance–performance relationship such that the relationship is stronger when passion is high. Their metaanalysis (study 1) is characterized by a series of questionable decisions and coding errors, but my primary concern is that the findings of all three studies are based on a statistical error. Because of space limitations, I focus this letter only on this primary concern.

The two grit scales (2, 3) were developed to measure two aspects of grit: perseverance (“perseverance of effort”) and passion (“consistency of interests”). While some researchers use an aggregate score to represent overall grit, others use separate scores for perseverance and passion in their analyses, in line with recent metaanalytic evidence (4) indicating that this is more appropriate. Jachimowicz et al. (1) base the results in all three studies on the highly unorthodox assumption that the total score on the grit scale (i.e., perseverance and passion combined) can be used to represent just perseverance. I am unware of anyone else who has interpreted the total grit scale in this manner, probably because half of the items clearly measure passion rather than perseverance. The authors indicated to me that they based their decision on confirmatory factor analyses of the responses to the total grit scale. Specifically, the authors state that a single-factor model exhibited much worse fit than a higher-order model with two first-order factors and one-second order factor (and an equality constraint). This justification reflects a misunderstanding of factor analysis because the higher-order model, as specified, is mathematically identical to the two-factor model that reflects the hypothetical structure of the scales, with perseverance and passion as separate factors. In other words, the authors’ own analyses confirm that the scale responses have a clear two-factor solution, not a one-factor solution. As such, their use of the overall grit score to represent just perseverance is incorrect in two ways: (i) the scale exhibits a clear two-dimensional structure, and (ii) the overall score would be combination of perseverance and passion.

If this error is corrected, then study 1 has little meaning since all of the effect sizes reflect the overall grit score, not perseverance as claimed. Further, in study 3, if perseverance is computed from the items that were intended to measure perseverance, then no moderation effect is evident. That is, perseverance does not interact with passion in predicting performance. The data for study 2 were not publicly available, but it is likely that the claimed moderation effect would be absent for this study too.