Figure 1.

Example scenario used to test the open-ended learning architecture proposed here. (A) Intrinsic phase: initial configuration of the environment containing nine 2D “objects.” Each object has a certain position and color (the “object state”). In this phase the agent can autonomously explore the objects for a certain time to acquire as much knowledge and skills as possible. (B) Extrinsic phase: example of a task that the agent has to solve, requiring to either move objects or change their colors to bring the “environment” to the state of the “goal image”; the agent has to be able to do this on the basis of the knowledge and skills acquired during the intrinsic phase. Note that in these images the objects are set on the vertices of a 3 × 3 regular grid, although during exploration and extrinsic-task solving they can occupy any non-overlapping position with their center within the white square frame.