Figure 3. Dependence of the forgetting curves on the model parameters.

A Left: The stationary belief state in the absence of observations (indicated by dots) moves along the direction of the arrows for increasing probability of the neutral state or increasing average reward . How fast the belief drifts towards the stationary state after receiving reward depends on the parameter that controls the “timescale of changes”. Right: Changing the probability of the neutral state only marginally affects the forgetting curve (solid and dashed line). A smaller rate of changes leads to slower forgetting (dash-dotted curve). A positive average reward leads to a higher fraction of agents choosing the appetitive reaction, which is here the conditioned response (dotted curve). B For a large variance of costs of responding (curve with scale parameter of the exponential distribution ) there are some agents that do not exhibit the conditioned response immediately after conditioning, since the costs of the conditioned response are too large. If the variance of the costs of responding is small (curve with ), most agents choose the conditioned response until their belief gets close to the stationary belief state.