Table 2.

Possible influences on decision making and learning of the model parameters in equations (2.2), (2.4) and (2.5), when these are understood as depending, potentially, on external stimuli (S, S′), on internal stimuli (X, X′) and on the behaviour performed (B). The internal stimuli referred to by X and X′ are motivational states, internal clocks, hormonal states and so on. These are not discussed explicitly in the text, but here we emphasize that they can also influence decision making and learning. The notation in the table refers to the sequence (S,X)→B→(S′,X′) in which external stimulus S is experienced together with internal stimulus X, then behaviour B is chosen and then the external and internal stimuli S′ and X′ are experienced.

parameter	may depend on	influences	enables genes to control
β, response bias	S, X, B	selection of behaviour	amount of exploration
			response bias (different βs for different B)
			context dependence (different βs for different S or X)
u, primary reinforcement	S′, X′	rate of change and maximum values of v and w	what responses can be learned, and how fast
			what the animal ultimately strives for (what is a reinforcer)
αv, learning rate	S, X, B, S′, X′	rate of change of v	what responses can be learned (αv>0), and how fast
			context dependence (different αvs for different S, X, B, S′, X′)
αw, learning rate	S, X, S′, X′	rate of change of w	what can become a conditioned reinforcement (αw>0), and how fast
			context dependence (different αvs for different S, X, B, S′, X′)
			behaviour sequences