Skip to main content
. 2020 Oct 28;3:509354. doi: 10.3389/frai.2020.509354

Figure 7.

Figure 7

Level 2 MDP. Level 2 of the hierarchical POMDP for scene construction. Hidden states consist of two factors, one encoding the scene identity and another encoding the eye position (i.e., current state of the oculomotor system). The first hidden state factor s(2),1 encodes the scene identity of the trial in terms of two unique RDM directions occupy two of the quadrants (four possible scenes as described in the top right panel) and spatial configuration (one of 12 unique ways to place two RDMs in four quadrants). This yields a dimensionality of 48 for this hidden state factor (4 scenes × 12 spatial configurations). The second hidden state factor s(2),2 encodes the eye position, which is initialized to be in the center of the quadrants (Location 1). The next four values of this factor index the four quadrants (2–5), and the last four are indices for the choice locations (the agent fixates one of these four options to guess the scene identity). As with the sampling state factor at Level 1, the eye position factor s(2),2 is controllable by the agent through the action-dependent transition matrices B(2),2. Outcomes at Level 2 are characterized by three modalities: the first modality o(2),1 indicates the visual stimulus (or lack thereof) at the currently-fixated location. Note that during belief updating, the observations of this modality o(2),1 are inferred hidden states over motion directions that are passed up after solving the Level 1 MDP (see Figure 6). An example likelihood matrix for this first modality is shown in the upper left, showing the conditional probabilities for visual outcomes when the 1st factor hidden state has the value 32. This corresponds to the scene identity DOWN-LEFT under spatial configuration 8 (the RDMs occupy quadrants indexed as Locations 2 and 4). The last two likelihood arrays A(2),2 and A(2),3 map to respective observation modalities o(2),2 and o(2),3, and are not shown for clarity; the A(2),2 likelihood encodes the joint probability of particular types of trial feedback (Null, Correct, Incorrect—encoded by o(2),2) as a function of the current hidden scene and the location of the agent's eyes, while A(2),3 is an unambiguous proprioceptive mapping that signals to the agent the location of its own eyes via o(2),3. Note that these two last observation modalities o(2),2 and o(2),3 are directly sampled from the environment, and are not passed up as “inferred observations” from Level 1.