Figure 1.

Overview of model. A virtual arm with joint angles θ_sh and θ_el (θ_sh: angle of upper arm with respect to x-axis; θ_el: angle of forearm with respect to upper arm) controlled by two pairs of flexor and extensor muscles, is trained to reach toward a target. A proprioceptive (P) sensory area translates muscle lengths into an arm configuration representation. Plasticity is present in excitatory-to-excitatory recurrent connections within the higher-order sensory (S) and the motor (M) areas, in feedforward and feedback excitatory to excitatory connections between the higher-order sensory and the motor areas, and in feedforward connections from excitatory to inhibitory cells within each area. Motor units drive the muscles to change the joint angle. The actor is trained by the critic, which evaluates error and provides a global reward or punishment signal.