Fig. 5.

Analysis by synthesis in AV speech integration. Visual speech inputs typically precede the auditory signals and elicit an abstract speech representation. The predictive value of the abstract token varies as a function of visual saliency and is updated as more visual information is made available. Incoming auditory speech inputs are evaluated against the prediction. Redundancy between predictor and auditory inputs are decorrelated such that greater redundancy leads to greater decorrelation. The stronger the predictor, the faster the auditory speech processing. The N1/P2 complex reflects the residual error of the evaluation process and is observed as amplitude decrease.