Table 1.

Model update rules

Model	Update rule
RL	Vt+1a = Vt + ηa(Rt − Vta)
Fictitious	pt+1* = pt* + η(Pt − pt*)
Influence	pt+1* = pt* + η(Pt − pt*) − κ(Qt − qt**)

The RL model updates the value of the chosen action a with a simple Rescorla–Wagner (35) prediction error (R_t − V_t^a) as the difference between received rewards and expected rewards, where η is the learning rate. The fictitious play model instead updates the state (strategy) of the opponent p_t* with a prediction error (P_t − p_t*) between the opponent's action and expected strategy. The influence model extends this approach by also including the influence (Q_t − q_t**) that a player's own action Q_t has on the opponent's strategy (see Methods).