Does thinking about God increase acceptance of artificial intelligence in decision-making?

As AI becomes more powerful, it has become a more valuable decision-making aide. It is therefore increasingly important to understand what affects decision-makers’ likelihood of relying on AI.

Karataş and Cutright (1) present eight experiments (N = 2,462) suggesting that “God salience” (i.e., thinking about God) increases people’s willingness to rely on guidance from AI. The authors claim a consistent effect across decision domains and additionally report that these effects emerge for religious believers and nonbelievers alike.

Although the original studies were preregistered, a p-curve analysis (2, 3) reveals that they only had an average statistical power of 9% (90% CI: 5 to 48%). The studies were underpowered to detect the effects they report.

The importance of the research question, coupled with the underpowered studies, motivated us to replicate all online studies conducted by Karataş and Cutright (Studies 1, 2a, 2b, 3, and 5). The authors generously provided us with their research materials, enabling us to conduct direct replications. We preregistered our plans and employed samples (total N = 4143) which were at least two times larger than the five original studies we replicated (N = 1531); see ref. 4 for full results, materials, data, and code.

Fig. 1 shows that our replications produced weaker results than those reported in the original studies. In our replications, the effect of God salience on AI reliance is not statistically different from zero (Studies 1, 2a, 2b, 3, and 5 produced effect sizes, measured by r, of 0.03, 0.02, 0.03, 0.002, and 0.01, respectively—in contrast to the original effect sizes reported for each of the respective studies of 0.19, 0.15, 0.14, 0.11, and 0.19). Fig. 1 omits a conceptual replication of Study 1 that produced a nonsignificant effect in the opposite direction of the original Study 1, t(279) = −1.78, P = 0.077, r = −0.10 and a replication of Study 5 that successfully replicated the original result but whose experimental materials included a typographical error that presented inconsistent and potentially confusing information to participants in one experimental condition, t(872) = 2.26, P = 0.024, r = 0.09. Although the majority of our replications produced results directionally consistent with those of Karataş and Cutright, our results imply that if the effect exists, it is so small that the original studies lacked the power to detect it.

Fig. 1.

Original and replicated study effect sizes. The figure is a forest plot displaying the effect size (r) results for the original (circle) and replication (triangle) studies. Horizontal lines indicate the 95% CI.

There are some limitations to our approach. First, we attempted to replicate only the five online experiments, not the three additional experiments. Further, we cannot identify why the published effects may not have replicated; our close adherence to the authors’ original procedures means that any potential moderators are well hidden.

Does thinking about God increase reliance on AI in decision-making? The results of our replications suggest that the evidence is less compelling than the original paper purported.