Figure 2.

Left plot: Agent architecture employed during the developmental period. No external reward is provided but the motivational system IM creates an intrinsic reward r_i. In parallel, new skills o are identified using the skill discovery module SD and added to the skill pool O. The policy π_i selects skills according to their intrinsic reward; both π_i and the policy π_o of the active skill are learned. Right plot: Agent architecture for learning to solve external tasks . A hierarchical policy π_e is learned based on the external reward r_e using the fixed set of skills O. The policy π_o of the active skill is also improved.