Figure 4.

The inferential account of conscious sensory processing. Illustration of the hypothesis that conscious sensory processing is organized according to low, intermediate and high representational levels (see Pennartz, 2015). In this scheme, representations correspond to predictions or hypotheses (rendered here as H(…)). A low-level prediction or hypothesis (green) pertains to a singular feature within a sensory modality, pertaining to an object or location in the environment (e.g., H(color) is the hypothesis that a visual object is of a certain color). The intermediate level (blue) is exemplified by H(visual object), the hypothesis that a visual object has several properties integrated across the low-level predictions. The highest level of representation (red) integrates across several sensory modalities (vision, audition, somatosensory and (not shown) olfaction, taste, sense of balance) and is rendered here as the hypothesis on a multimodal object. For instance, a grabbing of a piece of paper in one’s hand generates a joint inference on its visual, tactile and auditory properties. Already at the lowest level, predictions are learned and generated within a feedforward and recurrent multi-layer architecture which may well stretch across several connected cortical areas (e.g., V1, V2, V3 and V5 for visual motion). Note that motor or situational aspects, including the object’s position in space, are not taken into account in this scheme. In addition, a prediction on one particular feature × may well depend on predictions on other features y and z (etc.), for instance when assessing object shape from colored patches, texture and disparity cues. Thus, hypotheses are can be conditional on each other, which may be implemented by interactions between sub-networks. Horizontal dotted lines denote that the potential list of relevant features or sensory modalities can be extended. Curved arrows indicate the various types of interaction between sub-networks (recurrent, feedforward and lateral).