FIGURE 1.

Models of first-order UAL. Functional units are depicted as circles; receptors and effectors are depicted as squares. Solid arrows denote bottom-up and lateral interactions; dashed arrows denote top–down interactions. The ellipses are the local loci of conjunction. I, II, and III are distinct hierarchical functional levels. R, receptor (e.g., retinal cell); M, effector (e.g., Muscle); S, primary units processing sensory information; CPG, Central Pattern Generator; SIU, Sensory Integrating Unit (all sensory exteroceptive and interoceptive information); MIU, Motor (action) Integrating Unit proprioceptive and interocpetive information); REIU, Reinforcing Integrating Unit; AU, Association Unit (motor-sensory integration); MEMU, dedicated Memory Unit. (A) World learning. Following an animal’s activity (e.g., saccades and exploratory touch) at time T₁, a compound CS (e.g., a black, round and rigid ball) is processed in the sensory integrating unit, SIU. The CS comprises several perceptual elements received by receptors R_1,2,3 and processed by sensory neural units S_1,2,3. The construction of the compound stimulus (in the SIU unit, blue) is obtained via hierarchical predictive coding: the novel aspects of bottom-up information (depicted by ascending solid black arrows) are combined with top–down predictions (depicted by dashed curved arrows), based on available perceptual information in SIU, and on mnemonic and contextual information in the memory unit, MEMU (green). At T₂, a reinforcing stimulus (a US such as food) activates the reinforcing unit, REIU (red), via receptor R₄ and sensory unit S₄. The SIU, MIU and the REIU systems activate the association unit, AU (violet), that constructs the updated model of the world at T₂, and sends a PE to the local association unit (the ellipse). The PE is based on the discrepancy signal sent from the SIU and the EC (efference copy) signal sent from MIU [PE is the difference between the received reward λ, and the reward prediction V at time T₁, λ(t) – V(t)]. When the discrepancy is non-zero, the strength of the association between the SIU and MIU will increase in proportion to the magnitude of the prediction error. This association circuit reconstructs engrams in MEMU. When the REIU activates the motor integrating unit MIU, it sends a signal to the effector, leading to the response via the activation of a lower-level central pattern generator unit (CPG₁), which activates the effector M₁. On later encounters with the stimulus, the SIU-MIU relation is primed for activity by the MEMU engrams. As a result, the compound CS will now elicit the adaptive M₁ activity. (B) Self-learning. At time T₁, a compound action carried out by M₁ (e.g., pressing a button, turning a dial, and jumping) is originated by unit MIU, based on temporal and spatial combinations of various action patterns (implemented by central pattern generators, CPGs). This compound action pattern, which is influenced by past behaviors (engrams in MEMU), has reinforcing effects depending on present context, and is memorized both at the level of the local circuit (ellipse) and in the dedicated memory unit MEMU. As in A, the relation between the compound motor pattern and the reinforcer is processed in AU, which send a PE to the association locus. (C) Combination of self and world learning, typical of most classical and operant conditioning. The action results in an altered world-state, which in turn leads to the perception of a compound CS at time T₂ (same process as in A). If at time T₃ reinforcement of both compound movement pattern and compound percept occurs, prediction-errors that are sent to the association loci between MIU and SIU are strengthened, and both the compound CS percept the compound action pattern directed toward it will be learned.