Determining false-positives requires considering the totality of evidence

Fowler and Montagnes (1) independently replicate one finding in Healy et al. (2): that college football wins increase incumbent vote share. Although we interpret this result as evidence of irrelevant events impacting voters’ decisions, which is consistent with established theory in the psychological and decision sciences literatures, Fowler and Montagnes (1) conclude that chance is responsible. False-positives can occur. Consequently, we performed several tests to address that possibility (2), but Fowler and Montagnes (1) surprisingly ignore these analyses. Although replication and reanalysis are important to scientific discovery, one cannot selectively consider pieces of evidence when evaluating past research. Our consideration of the totality of evidence [the full results in Healy et al. (2) and the new results in Fowler and Montagnes (1)] leads us to conclude that college football games influence elections.

First, Fowler and Montagnes (1) entirely ignore the most important analyses of the football data in Healy et al. (2): the ones using betting spreads to isolate surprise outcomes. The finding reported in the abstract of Healy et al. (a 1.61 percentage point effect) comes from analyses comparing the actual outcomes of the games to what was projected beforehand by betting markets. This approach creates a quasi-experiment that isolates the surprise component of game outcomes. These “surprise wins” identify a variation that has nothing to do with how good a team is in a given year altogether. This is the same approach used by Card and Dahl in their paper showing that football games affect domestic violence (3). These results mean that Fowler and Montagnes (1) can only be correct if teams systematically over many years happened to get lucky in both the game immediately before the election and the previous game (“surprise wins” result), and were better over the entire season (baseline result without point spreads). This scenario is exceedingly unlikely.

Second, Fowler and Montagnes (1) entirely ignore Healy et al.’s (2) replication with a different sample collected from another sport. Fowler and Montagnes (1) emphasize the importance of replication of empirical results. We agree. To confirm the original result, we conducted a survey experiment during the 2009 National Collegiate Athletic Association men’s basketball tournament. The results confirmed the general findings from the football data. Again, strangely, Fowler and Montagnes ignore these results.

Third, Fowler and Montagnes (1) conduct a series of auxiliary tests poorly designed to evaluate Healy et al.’s (2) results. The betting spreads analysis makes more sense than any of these tests. The main approach of Fowler and Montagnes (1) is to conduct a series of auxiliary tests, the results of which Fowler and Montagnes claim are inconsistent with Healy et al.’s (2) original findings. All of these tests are flawed and rely on voters being decidedly more capable than much research (not just ours) suggests. We do not have space to discuss these individual tests in this letter, but have posted a detailed response online (4).

We thank Fowler and Montagnes (1) for pushing us to reconsider the full set of empirical evidence. We have done so and conclude that college football wins increase incumbent vote share, consistent with a large literature on irrelevant events and voting. We look forward to future scholars evaluating the implications of these results for democracy, such as the excellent work of Ashworth and Bueno de Mesquita (5).