Reply to Eckles et al.: Facebook’s optimization algorithms are highly unlikely to explain the effects of psychological targeting

We thank Eckles et al. (1) for their thoughtful comments. The authors point out that the optimization algorithms of Facebook’s advertising platform constitute a potential confound of campaign outcomes. We agree, in general, that such algorithms could pose a threat to the validity of field studies since they introduce unintended variance across the target audiences. However, as we demonstrate below, it is unlikely that such confounds account for the pattern of results presented in our original research (2).

First, the authors point toward “statistically significant” differences in the age and gender distributions of target groups. However, it is important to keep in mind that with sample sizes as large as ours (several million users viewed our ads), even the most trivial differences in demographic variables will be highly significant (3). An inspection of figure 1 of their Letter reveals that the actual differences across target groups were, in fact, small. For example, the highly significant difference in age distributions (P < 10⁻¹⁸, figure 1A) between the two extroverted target groups in study 1 is associated with miniscule differences (0.3–0.7%) in the relative frequencies for the different age groups. Most importantly, however, we controlled for age, gender, and their interaction with the ad version, and found the matching effects to be robust. Indeed, the effect size estimates were almost identical when including controls (4; study 1, B_original = 0.90 ± 0.21, B_controls = 0.91 ± 0.21, Z = −0.03, P = 0.976; study 2, B_original = 0.72 ± 0.22, B_controls = 0.72 ± 0.22, Z = −0.03, P = 0.998; see tables S6 and S10 in the appendix of ref. 2).

Second, in contrast to the study by Gordon et al. (5), our studies tested for interaction effects between target group and advertising content, not main effects. Even if the Facebook algorithm optimized the target audiences based on the initial performance of our ads, it is unclear why the same algorithm would systematically favor the matching over the mismatching campaigns. In other words, there is no reason to assume that Facebook would selectively boost the introverted ad for the introverted target audience, and the extroverted ad for the extroverted target audience, if those conditions were not “naturally” performing better.

Third, we replicated the effects of psychological targeting in several studies using different target audiences (with varying personality-related target Likes and geo-demographic specifications) and different advertising designs. In addition, our studies were conducted over a period of 3 y (2014–2017), in which the Facebook optimization algorithm has likely to have undergone several changes and developments, yet the results consistently show an effect of psychological matching. Hence, we consider it extremely unlikely that inherent Facebook optimization can consistently explain the interaction effects found across multiple studies with different methodologies.

Taken together, we believe that our findings provide robust evidence for the effectiveness of real-life psychological targeting. Having said that, we encourage other researchers to replicate and further generalize our findings and to conduct additional rigorous tests by leveraging some of the recently introduced Facebook features that allow for unbiased A/B testing of advertisements.