Figure 6.

Autocorrelation in the symmetric matching pennies. The solid lines show the autocorrelation between actions from the current and the previous trial for both players. The dashed lines indicate cross-correlations between one players action in the current trial with another player’s action in the previous trial. From left to right: experimental data, policy gradient, policy gradient with intrinsic costs, Q-learning, Q-learning with intrinsic costs. Created using MATLAB R2021a (https://www.mathworks.com)⁴³.