a, Assumed hierarchical generative model of the stimulus as maintained by the brain. G represents a higher visual area, encoding a global orientation variable, X represents sensory neuronal responses, which are activations of local Gabor filters, and I represents the visual input. The idea is that when visual input I is presented, the (simplified model of the) brain probabilistically infers a global orientation and the activations of the Gabor filters from the visual input via posterior inference, i.e., computes p(G,X|I) and samples from it. Posterior samples of X, i.e., of sensory neurons depend on the visual input I (likelihood, feedforward) as well as on G (prior, feedback). b, Gabor filters corresponding to the 6 model neurons as activated only by the prior, i.e., by G. In our model, we have 3 neurons with vertical Gabor filters and 3 others with horizontal Gabor filters. Gabor filters that have orientations similar to the global orientations have higher activations, as opposed to those that have orientations dissimilar to the global orientation. These activations are interpreted as sensory neuronal responses. We consider the neuron with a horizontal Gabor at the center as the “center neuron”. c, Collection of visual stimuli we present to the model. All stimuli, i.e., MEI, MEI & congruent surround, and MEI & incongruent surround are defined w.r.t the center neuron. d-f, Scatterplot of posterior samples from the center neuron under various stimulus presentations, reproducing the key experimental observations of surround-based excitation and inhibition.