The receptive field is dead. Long live the receptive field?

Abstract

Advances in experimental techniques, including behavioral paradigms using rich stimuli under closed loop conditions and the interfacing of neural systems with external inputs and outputs, reveal complex dynamics in the neural code and require a revisiting of standard concepts of representation. High-throughput recording and imaging methods along with the ability to observe and control neuronal subpopulations allow increasingly detailed access to the neural circuitry that subserves these representations and the computations they support. How do we harness theory to build biologically grounded models of complex neural function?

Sensory neurophysiology is dominated by the concept of stimulus representation. Our senses are impinged upon by a variety of stimuli. The nervous system captures these stimuli and filters them to extract and encode a myriad of features. These features are thought to be assembled and integrated across sensory modalities to form representations of increasing complexity, specificity and invariance. These hierarchically organized representations then in principle become accessible to perception and enable the lifelong construction and updating of internal models of the world about which we reason and which provide a basis for invention and imagination.

The concept of a heterogeneous feature basis that becomes increasingly sophisticated as it is propagated hierarchically [1] has gained powerful traction because of the extraordinary finding that the responses of many individual sensory neurons are indeed intelligible: it is frequently possible to find stimulus parameters with respect to which the response of a neuron varies systematically and fairly repeatably. While most clearly elaborated in the visual system, this picture roughly recurs across almost all sensory domains [2,3]. Olfaction may be a counterexample [4], in the sense that representations of intermediate complexity do not appear to exist [5]. Much theoretical progress has been made in the past several decades in developing methods to mine input/output data to determine variations of cascade models, which identify linear filters that extract relevant stimulus components, and predict the firing rate as a nonlinear function of the filtered stimulus [6]. In some cases, such models give excellent predictions of responses to restricted stimulus sets [7] from feature-based models.

Furthermore, theory has addressed not just what is encoded, but why the encoded features may assume the form they do. Two key principles have emerged: that these features may provide an efficient way to represent the specific statistical structure of the natural world [8], and that neural representations are sparse, in the sense that any natural input can be represented by the activation of relatively few neurons [9]. Further, it has been proposed that neural systems might use representations that facilitate computation [10,11] and that processes like adaptation can dynamically enhance the quality of representations [12,13,8]. The utility of such feature representation is validated by the rapid advance of “deep learning” networks in machine intelligence [14], which instantiate the principles of hierarchical feature selection learned from natural data, emergent high order features, and distributed and sparse representations. These advances have resulted in engineered networks that are now able to perform object and speech recognition tasks with unprecedented accuracy.

While this picture of sensory representation is compelling, there are many important caveats-- ones that will assume greater significance as more experiments move toward recording during natural behavior. The success of basic coding models in predicting responses is generally limited to certain stimulus regimes: models fitted using, for example, white noise do not generally accurately predict responses to natural inputs [15,16]. Even in the retina, the poster child for successful neural coding paradigms, the observation of complex feature selectivity such as sensitivity to figure/ground differences and multiple adaptation timescales has triggered the deployment of hybrid coding/dynamical models [16]. The hierarchical feature model, and its machine learning analog, is essentially feed-forward. In reality, feedback plays an enormous though not yet well-understood role in modulating responses by behavioral state, top-down effects and contextual cues [3], multimodality [17] and through interaction with signals of self-motion [2].

Given these complexities, is there an alternative way to think about neural representation? The diverse approaches to computational neuroscience that are expressed in this issue at times expose a tension between two paths to understanding brain function, one which might be seen as originating in computer science and the other in physics. In a computer science formulation, a circuit element implements an algorithmically defined function, a step in a logical chain. From a physical perspective, the state of such an element evolves according to dynamics specified by its interactions; neural circuits can be modeled as a set of differential equations driven by continuous and analogue inputs. This distinction could also be framed as one of function and mechanism. Of course, this somewhat artificial dichotomy between physics and computation becomes obvious when moving from the nervous system to the body: neural signals interface with biomechanics, which provide a fundamental contribution to the transformation from sensory inputs into behavior [18,19,2].

To bridge function and mechanism, we suggest an embellishment of Marr's famous three-level schema of Computation, Algorithm, Physical Implementation to give physics a more prominent role in further unpacking “implementation” into the true physical substrate and a comprehensible dynamical mechanism that can help to “explain” computation.

The three-part picture of Fig. 1 has been most extensively elaborated at the single neuron level. Experimentally well-founded conductance-based models describe the evolution of the voltage of single neurons as a function of inputs, depending on ion channel densities and morphology. These models can be reduced, analytically or numerically, to much simpler and highly predictive low-dimensional dynamical systems [20]. The resulting low-dimensional system is then amenable to analysis, leading to a coding model that expresses its computational properties [20,21,22]. Spike-triggering features approximately arise from the local linearization of the underlying nonlinear dynamical system. The threshold nature of excitability privileges certain stimulus components, reducing the dimensionality of the relevant feature space. Thus, quasi-linearity of the dynamical system establishes the system's filtering properties or feature selectivity, while the nonlinearity of spiking reduces the intrinsic dimensionality of the feature space [21].

Figure 1.

Taking computation here to subsume Marr's computation and algorithm, that is, a description of the system's function, a complete understanding of the neural mechanisms of computation should work at several levels. Data and detailed modeling provide a high-dimensional description of the system. To understand how this concrete implementation carries out a computation, it is useful to develop a low-dimensional description in which the fundamental mechanism of the computation is exposed. The transformation from high-dimensional implementation to low-dimensional model captures the parameter invariance or robustness of the implementation.

The example of the single neuron, a fundamental unit of information encoding, highlights the duality between a dynamical system and a feature-selecting coding model. Any choice of coding model “queries” the dynamical system in order to generate an input/output relationship with respect to a specific variable or set of variables. Despite the sophisticated methods available to guide the selection of this variable set [6], the result is necessarily an impoverishment of the full behavior of the nonlinear system. An example is that of contrast gain in single neurons. When stimulated by inputs that vary over a certain range, the input/output function of many sensory systems depends on the stimulus range: the dynamic range of the response is matched to the input range [13]. Some single neurons show the same effect, demonstrating that the property can arise from intrinsic neuronal nonlinearities [22]. Identifying a low-dimensional model that matches experimental data allows analysis of the dynamics that lead to this coding property.

Extending such a multifaceted approach beyond single neurons is challenging; high-dimensional biophysical models will always be underspecified [23]. However, given that it is now possible to visually identify, record from and manipulate specific cell types, the motivation to incorporate this information in models is there. The appropriate mathematics to perform the necessary reduction of such high-dimensional systems is emerging [24,25]. Studies undertaken in this spirit are beginning to address important open problems, such as the role of diverse cell types [24,26,27], pharmaceuticals [28], neuromodulation [29,30,31] and the statistics of connectivity [32,24] in shaping circuit dynamics and computation. The goal of extracting computation from detailed modeling motivates the use of high-resolution imaging to determine not just a connectivity matrix but also the morphologies of dendritic arbors that influence small circuit computation [33,34,35].

On the output end, the motor field is currently carefully scrutinizing classical tuning curve concepts. Perhaps surprisingly, brain-computer interfaces using decoded motor cortex activity to drive robotic limbs can show considerable success with quite simple decoding algorithms [36,37], which could be viewed as the ultimate validation of a coding picture. As researchers emphasize [38], however, this may be an outcome of remarkable plasticity: as brain-machine interface pioneer Eb Fetz showed decades ago, firing rates of individual neurons can be adjusted through feedback when connected to a decoder [39]. The success of simple linear decoders suggests that the network may adjust itself to the brain-machine interface if the decoding is easily learnable [40]. During normal (although overtrained) behavior, the complex, sometimes non-monotonic time dependence of multielectrode recordings of neuronal responses indicates that instantaneous tuning curves are certainly an impoverished description. Shenoy and colleagues [41] have argued for a return to a more fundamental dynamical systems approach. In this case, the high-dimensional description of Fig. 1 is not modeled but is represented by data; projecting this data into low-dimensional models may permit the inference of an effective dynamical model with explanatory power.

It is likely that the relative success of coding and decoding strategies both at the sensory input and motor output side are reflections of the existence of relatively low-dimensional structure in the underlying dynamical system. In more central processing areas, the situation appears murkier. Neural recordings from prefrontal (monkey) and parietal (rodent) cortex during sensory decision making tasks reveal distributed, multimodal responses that depend on task [42,43,44]. How can one infer the computation that is taking place from these mixed responses? One route is again to search for low-dimensional structure in the data that separates these influences and hopefully elucidates their interaction [44]. Training recurrent neural networks to solve the task and using dimensionality reduction to analyse the resulting network activity is also an intriguing way forward [45,43].

Thus, there are two major routes toward reducing the intrinsic high dimensionality of neural circuits to low-dimensional models in the search for computation: quasi-analytical simplifications of high-dimensional dynamical systems [20,24,46], or data-driven approaches that discover low dimensional structure in high dimensional data [41,45]. The form of these low-dimensional systems should inform or potentially eventually replace coding models that allow for modulation by behavioral state and top-down effects, and incorporate the nonlinearities that account for adaptation and context dependence. Such an approach might permit a breaking out of conventional thinking about visual representation to encompass emerging alternative views of the true function of V1 [47].

While the feature or receptive field concept, broadly defined, still has much to contribute in helping to organize thinking about representation and its transformation in the brain, many important caveats apply both in the application of the concept and its existence in reality. Passive stimulation protocols favor a relatively simple feedforward perspective. Newer experimental techniques present the possibility, and indeed the necessity, of understanding the more complete dynamical system. Capturing the relationship between dynamically modulated representations and the underlying high-dimensional nonlinear dynamical system in the form of a low-dimensional model will be critical in deciphering complex computations. While some apparent correspondences are emerging between neuronal and artificial networks designed for object recognition, it remains to be seen whether these similarities will prevail under natural recording conditions during complex behavior. As machine learning approaches move beyond object recognition into the realm of natural behavior [48,23,10], the solutions that arise might inform our interpretations of central processing. Ultimately, the development of methods to map the dynamics of the physical substrate onto the computational is the bottleneck in our ability to truly comprehend the biological mechanisms of intelligence.