Correlation-Based Development of Ocularly Matched Orientation and Ocular Dominance Maps: Determination of Required Input Activities

Abstract

We extend previous models for separate development of ocular dominance and orientation selectivity in cortical layer 4 by exploring conditions permitting combined organization of both properties. These conditions are expressed in terms of functions describing the degree of correlation in the firing of two inputs from the lateral geniculate nucleus (LGN), as a function of their retinotopic separation and their “type” (ON center or OFF center and left eye or right eye).

The development of ocular dominance requires that the correlations of an input with other inputs of the same eye be stronger than or equal to its correlations with inputs of the opposite eye and strictly stronger at small retinotopic separations. This must be true after summing correlations with inputs of both center types. The development of orientation-selective simple cells requires that (1) an input’s correlations with other inputs of the same center type be stronger than its correlations with inputs of the opposite center type at small retinotopic separation; and (2) this relationship reverse at larger retinotopic separations within an arbor radius (the radius over which LGN cells can project to a common cortical point). This must be true after summing correlations with inputs serving both eyes.

For orientations to become matched in the two eyes, correlated activity within the receptive fields must be maximized by specific between-eye alignments of ON and OFF subregions. Thus the correlations between the eyes must differ depending on center type, and this difference must vary with retinotopic separation within an arbor radius.

These principles are satisfied by a wide class of correlation functions. Combined development of ocularly matched orientation maps and ocular dominance maps can be achieved either simultaneously or sequentially. In the latter case, the model can produce a correlation between the locations of orientation map singularities and local ocular dominance peaks similar to that observed physiologically.

The model’s main prediction is that the above correlations should exist among inputs to cortical layer 4 simple cells before vision. In addition, mature simple cells are predicted to have certain relationships between the locations of the ON and OFF subregions of the left and right eyes’ receptive fields.

To what extent do cortical receptive fields (RFs) and maps result from simple, activity-dependent rules for synaptic development? A primary model system for addressing this is the development of monocular and binocular, orientation-selective simple cells and their arrangement into orientation (ORI) and ocular dominance (OD) maps in the primary visual cortex (Hubel, 1982; Wiesel, 1982).

In previous work, we studied the conditions required to develop either OD or ORI preference systems alone, under a general class of synaptic modification rules (Miller et al., 1989; Miller, 1994). This class, “correlation-based” rules (Miller, 1990a, 1997), includes mechanisms that have in common the dependence of the development of a structure on the correlations among the activities of its inputs. These mechanisms include simple versions of Hebbian synaptic modification, activity-dependent release, and uptake of diffusible modifying factors, or synaptic sprouting and retraction with activity-dependent stabilization. We now examine whether the coexistence of multiple features in single RFs and maps is compatible with correlation-based mechanisms by studying the conditions necessary for the combined development of OD and ORI.

A significant problem is raised by this combined development. ORI maps are continuous across OD column borders, regardless of whether the OD map contains many or few binocular cells, and binocular cells show nearly identical preferred ORIs and spatial frequencies in each eye’s RF (Skottun and Freeman, 1984). How can correlation-based development yield binocularly matched ORI maps, which presumably requires correlation between the activities of the two eyes, and also yield OD segregation, which presumably requires a sufficient lack of interocular correlation?

We address this question through study of the development of simple cells. These are cells with RFs composed of one or more spatially segregated, elongated subregions, each giving exclusively ON (response to light onset/dark offset) or exclusively OFF (response to light offset/dark onset) excitatory input (Hubel and Wiesel, 1962). These comprise most or all cells in layer 4 of cat primary visual cortex (Hubel and Wiesel, 1962; Gilbert, 1977; Bullier and Henry, 1979). In that layer, the segregation of a simple cell’s lateral geniculate nucleus (LGN) inputs appears to underlie its RF properties. Segregation of left- and right-eye inputs underlies OD (LeVay et al., 1978), whereas segregation of ON-center and OFF-center inputs underlies spatial RF structure (Tanaka, 1983; Ferster, 1988; Reid and Alonso, 1995) as proposed by Hubel and Wiesel (1962). This spatial RF structure in turn appears to underlie ORI selectivity (Movshon et al., 1978;Jones and Palmer, 1987; Ferster et al., 1996). Thus, we will study the development of the LGN inputs to cat layer 4 simple cells. We ignore for simplicity the simultaneous development of intracortical connections (Callaway and Katz, 1991; Löwel and Singer, 1992;Durack and Katz, 1996; Ruthazer and Stryker, 1996).

Our previous work studied segregation of only two LGN input types. We determined how OD maps can emerge from competition among inputs serving the left and right eyes (Miller et al., 1989) and how simple cells and ORI maps can emerge through competition among ON- and OFF-center inputs (Miller, 1994). We now study the outcome of correlation-based competition among four input types—ON- and OFF-center cells serving left and right eyes (see Fig. 1a) —to address combined development of ORI and OD.

The primary outcome of our model is a specification of the spatial patterns of correlated activity among these inputs that are sufficient to yield both binocularly matched ORI maps and OD segregation. The activity patterns are robust, not requiring fine tuning. They are simple enough to plausibly exist in spontaneous neural activity before the onset of vision, although they are also plausible attributes of visually induced activity. These results are shown to generalize the results of the earlier separate OD and ORI models: the earlier results remain applicable, but new results emerge, including the conditions for developing binocularly matched orientation maps and a novel prediction for the interocular relationships of ON and OFF RF subregions. These results should in turn remain applicable if further subcategorizations of LGN cell types are considered (see discussion in Appendix ).

It is possible that ORI and OD normally develop at separate times in response to differing activity patterns. We consider such sequential development and find that it increases the range of correlation structures compatible with combined ORI and OD development. It also induces relationships between the ORI and OD maps such as those observed experimentally (Crair et al., 1997b). Simple explanations of the origins of these relationships are proposed.

Although we focus on the development of connections from LGN to layer 4 of cat primary visual cortex, simple cells might also develop through the segregation of monocular, ON- and OFF-center cell inputs in other systems. Possible examples include the avian visual Wulst (Pettigrew, 1979) and the projections from visual cortical layer 4 to upper layers in species in which layer 4 cells are primarily not ORI-selective [these include monkey (Blasdel and Fitzpatrick, 1984), ferret (Chapman and Stryker, 1993), and tree shrew (Fitzpatrick, 1996)]. If so, our analysis could also apply to such systems.

Preliminary reports of this work have appeared (Erwin and Miller 1995,1996a,b).

MATERIALS AND METHODS

Model system

We use a model system based on those previously used to study activity-dependent formation of OD columns (Miller et al., 1989) and of simple-cell RFs and ORI columns (Miller, 1994; for review, see Miller, 1996a). Except for the change from two to four LGN input classes, and other very minor changes discussed below, the model is as in Miller (1994).

Model cortical cells are arranged in a 32 × 32 grid with positions denoted by Roman letters, e.g., x⃗ ory⃗ (Fig. 1a). Model LGN cells are arranged in a corresponding 32 × 32 grid, with positions denoted by Greek letters, e.g., α⃗ or β⃗. We use common, e.g., retinotopic, coordinates so that locations in the two grids can be equated. Each LGN grid location represents a “projection column” containing four classes of cells: LN, LF, RN, and RF. Here the first letter stands for the left (L) or right (R) eye layer, and the second letter stands for the ON (N) or OFF (F) center type of the LGN receptive field. We will use the variables E and C to represent eye and center type respectively: E ∈ {L, R}, C ∈ {N, F}. We do not assume that ON and OFF cells are segregated into separate LGN layers as in the illustration but only that cells of each center type, receiving input from each eye, are present for all retinotopic positions.

Fig. 1.

a, Sketch of the model. The synaptic weight variable S^RN(x⃗,α⃗) represents the total weight to cortical positionx⃗ from LGN cells at α⃗ representing eye R (right eye) of center type N (ON-center). Similarly, S^LN(y⃗, β⃗) represents the total left-eye, ON-center weight from β⃗ toy⃗. More generally, synaptic weights from α⃗ tox⃗ are written S^EC(x⃗,α⃗), where E represents eye, E ∈ {R, L} (right eye, left eye), and C represents center type, C ∈ {N, F} (ON center, OFF center). The correlation function C^RN,LN(α⃗, β⃗) measures the degree of correlation between the spiking activities of LGN neurons of type RN at position α⃗ and those of type LN at positionβ⃗. More generally, activity correlations between LGN neurons of type E, C at position α⃗, and those of type E′, C′ at position β⃗, are written C^EC,E′C′(α⃗, β⃗). The intracortical interaction function I(x⃗, y⃗) describes how activity at cortical location x⃗encourages or discourages the development of correlated synaptic connections at a nearby location y⃗. b, c, The correlation functions C^EC,E′C′(α⃗,β⃗) are defined in terms of two functions. b, A Gaussian function G₃(α⃗ − β⃗) (see Eq. 3) represents correlations that are purely positive and taper with distance; c, an oscillating or Mexican-hat function M(α⃗ − β⃗) (Eq. 4) represents correlations that change sign with distance. d, e, Arbor function A(x⃗ − α⃗) and the intracortical interaction function I(x⃗ − y⃗) used throughout this article. The horizontal axes are in units of grid intervals; 1 grid interval can be taken to be ∼100 μm (Miller, 1994). The vertical axes are in arbitrary units (changes in the size of A , I or C are equivalent to change in the size of the learning rate η (Eq. 2); hence the absolute size of these functions is arbitrary).

For simplicity, we ignore distinctions among LGN cells beyond these four types. Ignoring competition among X-, Y-, and W-cell inputs may be justified by considering the model to represent the dominant input to a cortical region: X-cells for area 17 and Y-cells for area 18 (Humphrey et al., 1985; Ferster, 1990a,b; Ferster and Jagadeesh, 1991; Boyd and Matsubara, 1996). Distinguishing between cells of differing temporal envelopes (e.g., lagged and nonlagged; Mastronarde 1987a,b) would likely only extend, not invalidate, the present results, as explained in Appendix .

Connectivity from LGN to cortex is modeled by a static arbor function, A(x⃗, α⃗), and an evolving synaptic weight function, S(x⃗, α⃗). The arbor function, A(x⃗, α⃗), models activity-independent biases in connectivity, e.g., the retinotopically allowed range of cortex over which an LGN afferent may arborize or sprout. It can be imagined roughly as being 1 for (x⃗, α⃗) such that there can be a connection from α⃗ to x⃗, and 0 otherwise. More generally, we consider a function that tapers from 1 to 0 with increasing retinotopic separation of α⃗ andx⃗ (Fig. 1b). The arbor function can be interpreted as a measure of the retinotopic affinity between α⃗and x⃗ as determined by activity-independent cues (Miller et al., 1989; Miller, 1997), or alternatively as the relative number of synapses initially made by an LGN cell at α⃗ to a cortical cell at x⃗ (Miller et al., 1989; Miller, 1990b). The synaptic weight variables, S^EC(x⃗, α⃗), represent the total efficacy or strength of the synaptic connections from LGN cells at position α⃗ in eye layer E and of center type C to cells at cortical position x⃗ (Fig. 1a). This efficacy is necessarily zero where A(x⃗, α⃗) is zero. The spatial RF of the cortical cell at x⃗ is determined by its pattern of geniculate input, represented by S^EC(x⃗,α⃗).

Model dynamics

Overview. We use a simple, linear learning rule, which we will refer to for simplicity as “Hebbian.” However, our equations can be derived from simple models of various underlying mechanisms, as mentioned in the introductory remarks (Miller et al., 1989; Miller, 1990b, 1997). Each component of the model may thus have multiple interpretations in terms of biological processes. Some of these are discussed here. For more details, see Miller (1994).

The dynamic variables in the model are the synaptic weights, S^EC, which define the cortical spatial RFs. These weights develop over time according to a linear “Hebbian” learning rule, described below, subject to several constraints. We have shown previously that under this rule, correlations in firing activity of LGN cells, (Fig. 1a, C) are the primary determinants of RF structure, whereas interactions between cortical cells (Fig. 1a, I) are the primary determinants of map structure.

We first present the Hebbian learning rule without any constraints. This learning rule involves the functions A, I, and C, so we next describe the forms of these functions we will use. Then we describe the saturation constraints, which keep individual synaptic strengths positive and bounded, and the competitive constraints, which cause less-correlated inputs onto a cortical cell to weaken in response to competition from more-correlated inputs onto the same cell. The constraint is intended to model the effects, but not the mechanism, of unknown physiological processes that achieve a competitive outcome. Finally we describe in detail the iterative numerical procedure used to integrate the differential equations of the model and to allow joint enforcement of the saturation and competitive constraints.

Unconstrained Hebbian learning. To study the pattern of synaptic weights that initially develops from a nearly uniform projection, it is not necessary to explicitly specify sequences of input activity patterns. Rather, we only need to specify “correlation functions,” C^EC,E′C′(α⃗,β⃗), each representing the time averaged pattern of correlation between pairs of LGN cells, one of type EC at locationα⃗ and another of type E′C′ at location β⃗(Fig. 1a) (for further details of the averaging procedure, see Miller, 1990b, and references therein). The time over which this averaging occurs is purposely kept vague: it corresponds to the time separation of two input activities that yields cooperative interactions under whatever plasticity rule is acting biologically. More generally, it would involve some weighted average over time separation, weighted by the degree of cooperative interaction. The assumptions being made are that (1) one can summarize the temporal details of activity of any input pair by a number describing their degree of cooperation under the plasticity rule; and (2) these numbers can on average be predicted simply from the separation of two inputs and their input types (as supported by, e.g., Mastronarde, 1983a,b; Meister et al., 1995; Wong and Oakley, 1996).

Unconstrained development under a Hebbian or other correlation-based synaptic modification rule then takes the form:

$$\frac{d}{\text{dt}} S^{EC}(\vec{x},\vec{\alpha },t)=H^{EC}[\mathbf{S}](\vec{x},\vec{\alpha },t),$$ Equation 1

where S represents the set of all synaptic weights, and:

$$H^{EC}[\mathbf{S}](\vec{x},\vec{\alpha },t)=\eta A(\vec{x},\vec{\alpha })\sum _{\vec{y}} I(\vec{x},\vec{y})\sum _{\vec{\beta },E^{\prime },C^{\prime }} C^{EC,E^{\prime }{C}^{\prime }}(\vec{\alpha },\vec{\beta })S^{E^{\prime }{C}^{\prime }}(\vec{y},\vec{\beta },t).$$ Equation 2

Here, η is a constant learning rate. We have explicitly shown the time dependence of S^EC(x⃗, α⃗, t), to distinguish it from the time-independent terms, but in most cases we will simply write S^EC(x⃗,α⃗).

Although this equation may look daunting, it expresses a simple rule. It states that the development of one synapse (S^EC(x⃗, α⃗)) is determined by a sum of the average influences exerted on it by all other synapses (e.g., S^E′C′(y⃗, β⃗)). The influence on one synapse by another is a product of three factors: (1) the synaptic strength of the “influencing” synapse, S^E′C′(y⃗, β⃗); (2) the strength of correlation in the firing of the two synapses, C^EC,E′C′(α⃗, β⃗); and (3) the average influence I(x⃗, y⃗) exerted across the cortex when the two synapses fire together. The influence between two synapses is attributable to their cooperation (via activity correlation, C) in producing a “reward” signal. This signal may be the firing of each synapse’s postsynaptic cell in a Hebbian model, or the activity-dependent production and uptake of a diffusible modification factor in other models. The propagation of this influence between cortical cells is summarized by I, which could represent the effects of intracortical connectivity in a Hebbian model, and/or of diffusion in a model involving diffusible factors. Multiplication by the arbor function, A(x⃗,α⃗), ensures that influence is modulated by the intrinsic retinotopic affinity between α⃗ and x⃗ and in particular ensures that no synapses can develop where they are retinotopically disallowed (i.e., where A(x⃗, α⃗) = 0).

The advantage of this formulation is that it allows the central determinants of development to be isolated, independent of underlying mechanism. That is, for any proposed mechanism, it may be possible to summarize the elements driving development as (1) a pattern of input correlations, (2) propagation of influence of correlated input activity between cortical cells, and (3) retinotopic affinity or retinotopic limitations on connectivity. Our formulation isolates and describes the influence of each of these three factors on the developmental outcomes and thus simultaneously describes the determinants of development under multiple mechanisms. This formulation is further described by, e.g.,Miller (1990a,b, 1997).

The use of a linear equation can be justified by considering early development of the differences between initially approximately equivalent input projections, e.g., left eye and right eye or ON and OFF (Miller, 1990a,b). To the extent to which initial differences are small, early development of these differences will be determined by linear equations. This early linear development determines many features of the final mature pattern (in particular, those features that are shared by the fastest growing patterns in the linear regime typically persist in the final pattern). However, nonlinearities can also play important roles in developmental outcome (Feidler et al., 1997). Furthermore the different input projections are not really equivalent, because initial projections show contralateral-eye and OFF-center dominance (Albus and Wolf, 1984; Braastad and Heggelund, 1985; Crair et al., 1997a). Thus, to some extent this simple approach is justified simply by its success in yielding insight into the biology. It is of value to understand the simplest models before adding complexity to them, the moreso when that complexity is not well constrained by experiment. Many insights gained from the simplest models will persist as more complexities are added; we comment at several points on which features of the outcome are likely to be most susceptible to alteration by nonlinearities.

The functions A, I, and C. We will show analytically that the arbor function, A, and the intracortical interaction function, I, play developmental roles in the present model similar to their previously studied roles in the two-input-type forms of the model (Miller et al., 1989; Miller, 1994). Thus in the present numerical simulations we hold constant the forms of A and I while we study the role of the correlation functions, C^EC,E′C′(α⃗, β⃗), within and between the four input types.

The correlation between two inputs of given types is assumed to depend only on their types and on the distance separating them: C^EC,E′C′(α⃗, β⃗) = C^EC,E′C′(‖α⃗ − β⃗‖). Such dependence is shown in experimental measurements of correlations (Mastronarde, 1983a,b; Meister et al., 1995; Wong and Oakley, 1996) and is sufficient to produce RFs with alternating ON/OFF subregions, such as simple cells.

Two basic functional forms will be used throughout the paper. The first form has purely positive correlations that taper with distance ‖α⃗ − β⃗‖, as given by a Gaussian function determined by a parameter γ:

$$G_{\gamma }(\Vert \vec{\alpha }-\vec{\beta }\Vert )=(1/{\gamma }^2)exp[-\Vert \vec{\alpha }-\vec{\beta }{\Vert }^2/((0.24\gamma )(6.5){)}^2].$$ Equation 3

This function is illustrated for γ = 3 in Figure 1b. A second functional form, a “Mexican-hat” function (that is, positive at small distances and negative at larger distances), as illustrated in Figure 1c, represents correlations that change sign with distance:

$$M(\Vert \vec{\alpha }-\vec{\beta }\Vert )=G_1(\Vert \vec{\alpha }-\vec{\beta }\Vert )-G_3(\Vert \vec{\alpha }-\vec{\beta }\Vert ).$$ Equation 4

In these definitions, the width of the Gaussian is specified as a fraction 0.24γ of the radius of an arbor, which is 6.5 grid intervals corresponding to ∼600 μm, as discussed by Miller (1994). The factor (1/γ²) ensures that varying γ varies the width of the Gaussian without altering its two-dimensional integral. The constant 0.24 in Equation 3 was chosen because it optimizes the orientation selectivities resulting when certain correlation functions (C^ORI+ and C^ORI−, defined in Fig. 2d, below), have the Mexican-hat form of Equation 4 (Miller, 1994).

For simplicity, we take the arbor function to be identical for each eye, E, and center type, C, and constant in time. We let A be radially symmetric, largest nearx⃗ = α⃗ and falling to zero as a function only of lateral distance, A(x⃗, α⃗) = A(‖x⃗ − α⃗‖). We use the default arbor function of Miller (1994), which is zero beyond a radius of 6.5 grid intervals, as illustrated in Figure 1d. Taking grid intervals to be on the order of 100 μm, arbor diameters are a little >1 mm (for more detailed discussion, see Miller, 1994, p 433). This arbor function imparts an initially smooth retinotopy on a coarse scale. Because the receptive fields that evolve are typically significantly smaller than allowed by the arbor function, fine scale variations in retinotopy can emerge from activity-dependent synaptic modification.

Intracortical interactions are modeled by a function, I(x⃗,y⃗). For simplicity, so that we can focus on the effects of varying input correlations and also to ease computation, we use a two-dimensional cortical architecture and take I to be a function of only the distance between two cortical column positions, I(‖x⃗ − y⃗‖). This function is not meant to be a realistic model of interactions attributable to cortical connectivity, which are three-dimensional and cell-specific and develop along with the geniculocortical weights. This grossly simplified implementation of cortical interactions means that our model focuses more on development of RFs than on details of map structure because, as discussed above, correlations are the primary determinants of RF structure, whereas intracortical interactions are the primary determinants of map structure (for extensive discussion, see Miller, 1994.)

Setting I to be a radially symmetric, Mexican-hat function of lateral distance, with the form illustrated in Figure 1e, is sufficient to cause the emergence of continuous, approximately periodic maps of RF properties. The form illustrated is used throughout this paper; it is identical to the function M defined in Equation 4, except with the factor 0.25 rather than 0.24 in Equation 3defining the underlying Gaussian functions. (The value 0.25 was chosen simply as a round number that gave reasonable orientation maps inMiller (1994); the similarity of the Mexican-hat functions used for I and C has no significance for the results presented here.) Because there is little evidence for such Mexican-hat-shaped intracortical connections, it should be noted that very similar model behavior is achieved if purely excitatory intracortical interactions are combined with an additional competitive “arbor” constraint ensuring that each presynaptic afferent with a given mean activity maintains approximately constant total synaptic projection strength (Miller et al., 1989; Miller, 1994; Miller and MacKay, 1994). Thus one can think of the Mexican-hat I as a computationally convenient stand-in for this combination of excitatory interactions and arbor constraints.

Saturation constraints. Synaptic weights must be constrained to remain positive and bounded. We refer to these as saturation constraints. We take these to be:

$$0\le S^{EC}(\vec{x},\vec{\alpha })\le 8A(\vec{x},\vec{\alpha }),\text{for all} E,C,\vec{x},\vec{\alpha }$$ Equation 5

(the effects of varying the value “8” are studied in Miller, 1994). Once a synapse S^EC(x⃗,α⃗) becomes saturated at either zero or 8A(x⃗,α⃗), it is no longer plastic. This prevents the Hebbian term H^EC[S](x⃗, α⃗, t) from pushing the synapse beyond the saturation limits (this term rarely if ever changes sign for a synapse once it has developed sufficiently to saturate, assuming the functions A, I, and C do not change). We let P(x⃗, t) denote the set of coordinates of the plastic (unsaturated) synapses onto the cortical cell at x⃗:

$$P(\vec{x},t)=\{(E,C,\vec{\alpha })\text{such that} 0<S^{EC}(\vec{x},\vec{\alpha },t)<8A(\vec{x},\vec{\alpha })\}.$$ Equation 6

Competitive constraints. Some sort of competitive mechanism must be included in any Hebbian learning scheme to model the fact that different correlated input patterns compete with one another, so that a single correlated pattern of inputs ultimately comes to dominate the RF of a given cell (Guillery, 1972; von der Malsburg, 1973; Bienenstock et al., 1982; Stryker and Strickland, 1984; Miller and MacKay, 1994; Miller, 1996b). For example, in OD segregation, each eye’s projection is capable of strengthening its innervation onto cortical cells, yet ultimately only one eye comes to dominate each layer 4 cell; where one eye wins, the other must lose. Competition was directly demonstrated by Guillery (1972), who showed that, after monocular deprivation, the deprived eye can maintain its projection in retinotopic locations that lack open-eye competition yet loses its projection where open-eye afferents are present.

Very little is known about the mechanism by which such competition is enforced physiologically (but see Davis and Goodman, 1998; Turrigiano et al., 1998). We therefore do not try to model a specific mechanism but instead simply enforce competition by demanding that the total synaptic efficacy projecting onto each cortical cell be held constant:

$$\frac{d}{\text{dt}} \sum _{E,C,\vec{\alpha }} S^{EC}(\vec{x},\vec{\alpha },t)=0.$$ Equation 7

We refer to this as a competitive constraint on the developmental dynamics. It ensures that when some synapses are strengthened, others must be correspondingly weakened. It is enforced by modifying our development equation, Equation 1, through subtraction of an additional term, so that it becomes:

$$\frac{d}{\text{dt}} S^{EC}(\vec{x},\vec{\alpha },t)=\left\{\begin{array}{c}{H}^{EC}[\mathbf{S}](\vec{x},\vec{\alpha },t)-\epsilon (\vec{x},t)A(\vec{x},\vec{\alpha }),\\ \hfill 0,\hfill \end{array} \begin{array}{c}\text{for} \{(E,C,\vec{\alpha })\}\in P(\vec{x},t),\\ \text{otherwise}.\hfill \end{array}\right.$$ Equation 8

The value of ε(x⃗, t) is chosen at each time so as to ensure that the constraint, Equation 7, is satisfied.

When the number of plastic synapses P(x⃗, t) at cortical position x⃗ is not changing, the value of ε(x⃗, t) that will satisfy the constraint (7) is:

$$\epsilon (\vec{x},t)=\left[\sum _{P(\vec{x},t)} H^{EC}[\mathbf{S}](\vec{x},\vec{\alpha },t)\right]/\sum _{P(\vec{x},t)} A(\vec{x},\vec{\alpha }).$$ Equation 9

In numerical simulations, for any time step over which the number of plastic synapses in P(x⃗, t) changes, we instead use the equation:

$$\epsilon (\vec{x},t)=\left[\zeta (\vec{x},t)+\sum _{P(\vec{x},t)} H^{EC}[\mathbf{S}](\vec{x},\vec{\alpha },t)\right]/\sum _{P(\vec{x},t)} A(\vec{x},\vec{\alpha }),$$ Equation 10

where the value of ζ(x⃗, t) must be determined by an iterative procedure, given in the next section, to ensure that Equation 7 remains true after application of Equation 8 and restriction of all synapses at x⃗ to lie within the saturation limits of Equation 5.

Initialization. Synapses are initialized as S^EC(x⃗, α⃗) = A(x⃗,α⃗)[1 + ξ(x⃗, α⃗)] where each ξ is drawn randomly from a distribution uniform between −0.2 and 0.2. Note that this embodies our assumption that initial differences between the projections of the different input types are small (see discussion above on use of a linear equation).

For ease of comparing the effects of changing parameters, a single random initial condition (same random seed) is used for all illustrated simulations. We have run simulations for many initial conditions; results are qualitatively invariant.

Numerical algorithm. This section describes in detail how the differential equation governing synaptic growth is numerically integrated while the saturation and competitive constraints are also enforced. These details are necessary for a complete description of the model sufficient to allow reconstruction of the results. This section may be safely skipped by most readers.

We integrate the differential equation, Equation 8, with a three-step difference method (Birkhoff and Rota, 1978). The initial condition is taken as time t = 0. Application of the equations proceeds as follows for each cell x⃗ at each time step t:

(1)

Calculate H^EC[S](x⃗,α⃗, t) from Equation 2.

(2)

Determine the sets of plastic synapses P(x⃗, t) from Equation 6.

(3)

Initially assume ζ(x⃗, t) = 0.

(4)

For a given value of ζ(x⃗, t):

(a)

Calculate ε(x⃗, t) from Equation 10.

(b)

For each of the four weights S^EC, calculate (d/dt)S^EC(t) using Equation8.

(c)

Calculate ΔS^EC(t) = Δt(f₀(d/dt)S^EC(t) + f₁(d/dt)S^EC(t − Δt) + f₂(d/dt)S^EC(t − 2Δt)). The factors are f₀ = 1, f₁ = f₂ = 0 for the first time step; f₀ = 2, f₁ = −1, f₂ = 0 for the second time step; and f₀ = 23/12, f₁ = −16/12, and f₂ = 5/12 for subsequent steps.

(d)

For each of the four weights S^EC, calculate S^EC(t + Δt) = S^EC(t) + ΔS^EC(t), followed by setting any weight S^EC(x⃗, α⃗, t + Δt) < 0 to 0, and any weight S^EC(x⃗, α⃗, t + Δt) > 8A(x⃗, α⃗) to 8A(x⃗,α⃗).

(e)

Calculate D(x⃗) = Σ_E,C,α⃗[S^EC(x⃗,α⃗, t + Δt) − S^EC(x⃗,α⃗, t)] to determine whether the input to cellx⃗ was held constant for this value of ζ(x⃗, t).

(f)

If D(x⃗) is zero (or smaller than our criterion 0.00001), then accept the trial values of all the S^EC(t + Δt) forx⃗ and continue from step 2 with the nextx⃗. Otherwise, discard these S^EC(t + Δt) and repeat from step 4a using a new value of ζ(x⃗) selected by the method of bisection (Press et al., 1992). This method is guaranteed to converge [to find, in a finite number of steps, a ζ(x⃗) that yields an arbitrarily small D(x⃗)], because D(x⃗) decreases monotonically with ζ(x⃗).

For the first four time steps, we take Δt = 1, but we double the time step to Δt = 2 afterward. Thus, after t = 4 no synaptic values are computed at odd-numbered time steps. [Similar results can be obtained using Euler one-step integration with a smaller time step. The three-step method allows the use of a larger Δt or η but requires the storage of the intermediate results (d/dt)S^EC(x⃗, α⃗, t − Δt) and (d/dt)S^EC(x⃗,α⃗, t − 2Δt).] In two-stage simulations, the correlation functions are changed at the beginning of the second stage. In this case we again use Δt = 1 for the first four time steps of the second stage and Δt = 2 thereafter.

As described by Miller and MacKay (1994), with the subtractive competitive constraint (Eq. 10), the development will cease only when all of the synapses have reached either the upper or lower cutoff specified in Equation 5. For efficiency, we terminate a simulation when 90% of the synapses have reached the upper or lower cutoff. We choose the learning rate η small enough such that each complete simulation will last at least 40 iterations. (Note: because Δt = 2 after the first four time steps, 40 iterations corresponds to t = 76.)

RF and map analysis

It is not possible to show images of the RFs and maps from each of the many simulations conducted. Thus, we define numerical measures through which we may study the effects of the model parameters. It is most convenient to describe map properties in terms of the composite weight vectors defined in Table 1. (These composite weight vectors are introduced and extensively discussed in Results.)

Table 1.

Definitions of the composite variables

Composite weight variablesSμ									Composite correlation functions Cμ
SSUM	=	(SRN	+	SRF)	+	(SLN	+	SLF)	CSUM	=	(CSESC	+	CSEOC)	+	(COESC	+	COEOC)
SOD	=	(SRN	+	SLF)	−	(SLN	+	SLF)	COD	=	(CSESC	+	CSEOC)	−	(COESC	+	COEOC)
SORI+	=	(SRN	−	SRF)	+	(SLN	−	SLF)	CORI+	=	(CSESC	−	CSEOC)	+	(COESC	−	COEOC)
SORI−	=	(SRN	−	SRF)	−	(SLN	−	SLF)	CORI−	=	(CSESC	−	CSEOC)	−	(COESC	−	COEOC)

The degree of OD segregation of a cell, m(x⃗), is defined as the difference in connection strength between the right and left eyes, divided by the sum of those connection strengths: m(x⃗) = Σ_α⃗S^OD(x⃗,α⃗)/Σ_α⃗S^SUM(x⃗,α⃗). Complete right- or left-eye dominance is indicated by m(x⃗) = +1 or −1. The overall degree of OD segregation 〈m〉 can be assessed as the root mean square of m(x⃗):

$$\langle m\rangle ={\left(\sum _{\vec{x}} m(\vec{x}{)}^2/N\right)}^{1/2}.$$ Equation 11

Here, N is the number of cortical locations.

We measure the degree to which ON and OFF inputs are segregated by an index Z:

$$Z=(1/N)\sum _{\vec{x},\vec{\alpha }}[\Vert S^{ORI+}(\vec{x},\vec{\alpha })\Vert /S^{SUM}(\vec{x},\vec{\alpha })].$$ Equation 12

We define preferred orientation of the R or L eye, φ_x⃗^R or φ_x⃗^L, respectively, as the orientation of the flashed sinusoidal grating stimulus that maximizes the direct LGN input to a cortical cell for stimulation of that eye. This orientation is computed as that corresponding to the peak of the Fourier transform of ON minus OFF input for that eye (Miller, 1994). Note that we do not directly calculate cortical activity, and we neglect input from other cortical cells for two reasons: (1) we are interested in the degree of orientation tuning arising from segregation of the LGN input; and (2) our model does not incorporate development of the intracortical connections, and if we interpret the static I( ) function as the strength of synaptic input between cortical cells, then including these inputs causes only subtle changes in cell responses and the orientation map (Miller, 1994).

Orientation selectivity, q_x⃗^R or q_x⃗^L, measures the degree to which a cell’s input to sinusoidal gratings is peaked at its preferred orientation. We use a measure q_x⃗^E, defined by Miller (1994). With the simple response model used here, in practice 0 ≤ q_x⃗^E ≤ 0.27. Mean cortical orientation selectivity, Q, is computed as a weighted average of monocular orientation selectivities q^R and q^Lacross cortex:

$$Q=\frac{1}{N} \sum _{\vec{x}} \frac{1}{2}[q_{\vec{x}}^R(1+m_{\vec{x}})+q_{\vec{x}}^L(1-m_{\vec{x}})].$$ Equation 13

In a few cases in which the preferred ORI is nearly the same in each eye of all binocular cells, we show a “binocular” ORI map, rather than monocular maps, to save space. Such a map is constructed by averaging, at each cortical position x⃗, the two monocular preferred orientations, φ_x⃗^E, E ∈ {L, R}, weighted by their ORI selectivities, q_x⃗^E, and by the corresponding eye’s proportion of synaptic strength at x⃗, (1 ± m_x⃗)/2. The “binocular” preferred orientation, φ_x⃗, and selectivity, q_x⃗, are found from

$$q_{\vec{x}}{e}^{2i{\phi }_{\vec{x}}}=q_{\vec{x}}^R{e}^{2i{\phi }_{\vec{x}}^R}(1+m_{\vec{x}})/2+q_{\vec{x}}^L{e}^{2i{\phi }_{\vec{x}}^L}(1-m_{\vec{x}})/2.$$

Positive and negative singularities in the ORI maps are defined as locations centered between four grid positions around which the sum of the changes in preferred orientation rotates through 180° in the same or opposite direction as the movement around the singularity.

To determine the similarity of any two ORI maps, A and B, we use a method similar to that which has been used in optical imaging experiments (Gödecke and Bonhoeffer, 1996). We first compute single-orientation response maps at 18 discrete orientations θ at 10° intervals. We define the responses, A(x⃗, θ) and B(x⃗, θ), as the maximum LGN input to cells at x⃗ attributable to any grating (of any spatial phase or frequency) with preferred orientation within ±5° of θ. Correlation coefficients (Pearson’s r; Press et al., 1992) are computed between the two single-orientation maps, A(x⃗, θ) and B(x⃗, θ), giving a measure r(θ), which is then averaged over θ to give a single measure of similarity between the maps.

RESULTS

The dynamical variables in our model are the four synaptic weights, S^EC, defined in Figure 1a. These represent the projection to layer 4 from LGN cells of center type C ∈ {N, F} (ON and OFF), and of eye E ∈ {R, L} (right and left) (for simplicity, we use “of eye E ” to mean “driven by eye E ”). These develop under Equation 8, as determined by the correlation functions C, which describe the patterns of correlation in the firing of pairs of LGN cells, and by the intracortical interaction function I (Fig. 1e), which describes interactions across cortical positions attributable either to synaptic connectivity or diffusible factors. This development occurs subject to constraints, including the limits on retinotopically appropriate connections specified by the arbor function A (Fig. 1d), competition for limited synaptic resources on each postsynaptic cell imposed by Equation 10, and minimum and maximum weight values (saturation limits) imposed by Equation 5.

The functions I and A and the saturation limits all play roles in the present model that are formally identical to their roles in earlier models (Miller et al., 1989; Miller, 1994), which described development of OD or ORI maps alone. Because the effects of changes in these functions were fully described there, we hold them constant here and focus on the role of the correlation functions.

We make the simplifying assumptions that the activity patterns in LGN of the right and left eyes are statistically indistinguishable, as are the activity patterns of the ON and OFF populations. In reality, there are differences between ON and OFF spontaneous activity patterns (in retina; Mastronarde, 1983a,b; Wong and Oakley, 1996). As previously noted, we also make the simplifying assumption that the arbor function is identical for all four projection types (in reality, the contralateral eye has a somewhat stronger anatomical projection than the ipsilateral eye; LeVay et al., 1978; Shatz and Stryker, 1978). We will refer to these, along with the assumption that the four input types begin in approximately equal, unstructured conditions (see Materials and Methods), as symmetry assumptions between the two eyes and between the two center types.

Correlations among input activities are in general defined by a set of 16 correlation functions, C^EC,E′C′(α⃗− β⃗), describing the correlation in activity between an input of eye E and center type C at positionα⃗, and one of eye E′ and center type C′ at position β⃗. Because of the symmetry assumptions, the functions C^EC,E′C′ depend only on whether E and E′ are the same or opposite eyes (S_E or O_E) and on whether C and C′ are the same or opposite center types (S_C or O_C). Thus, there are only four distinct correlation functions, which we describe as C^S_ES_C, C^S_EO_C, C^O_ES_C, and C^O_EO_C (Fig. 2a).

Fig. 2.

a, If we assume the left and right eyes may be treated equivalently, and likewise for the ON and OFF cells, then the 16 correlation functions C^EC,E′C′, E, E′ ∈ {R, L}, C, C′ ∈ {N, F}, reduce to only four distinct functions C^XY, as shown. Here X ∈ {S_E, O_E}, where S_E means same eye and O_E means opposite eye; and Y ∈ {S_C, O_C}, where S_C means same center type and O_C means opposite center type. b–d, Development is most easily described in terms of four composite synaptic weights or modes, S^μ, and four corresponding composite correlation functions, C^μ, μ ∈ {SUM, OD, ORI+, ORI−}. b, On the left, the four synaptic weight variables S^EC are shown in a square grid with columns labeled by eye, E, and center type, C. c, Similarly, the four correlation functions of a are also arranged in a square grid. d, To the right, there is one small grid for each μ. The symbols inside the grids define both S^μ and C^μ as sums of positively and negatively weighted entries in the corresponding positions in the larger grids to the left. For example, S^OD ≡ S^RN + S^RF− S^LN − S^LF, and C^ORI+ = C^S_ES_C − C^S_EO_C + C^O_ES_C − C^O_EO_C. Explicit definitions are in Table 1.

Composite weight modes and correlation functions

The four synaptic weight variables and correlation functions correspond to physical quantities that are, in principle, measurable. However, the behavior of the model is more easily described in terms of four composite synaptic weight variables, ormodes, called S^μ, where μ ∈ {SUM, OD, ORI+, ORI−}, and four corresponding correlation functions, C^μ. These are defined, as shown in Figure 2d and Table 1, as different linear combinations of the respective physical weight variables (Fig. 2b) and correlation functions (Fig. 2c). The modes can be understood as follows: (1) S^SUM(x⃗, α⃗) is the summed synaptic connection strength from all cell types atα⃗ to x⃗; (2) S^OD(x⃗, α⃗) is the total right-eye minus total left-eye synaptic strength from α⃗to x⃗; it describes monocularity and OD maps; (3) S^ORI+(x⃗, α⃗) is the total ON-center minus total OFF-center synaptic strength fromα⃗ to x⃗; it describes one type of ORI selective RFs and their ORI maps; and (4) S^ORI−(x⃗, α⃗) is the right-eye ON/OFF difference minus the left-eye ON/OFF difference from α⃗ to x⃗; it describes a second type of ORI-selective RFs and their ORI maps.

Near the beginning of a simulation, when most synapses are far from their maximum or minimum values, each mode S^μ develops independently of the other three, under the influence of its corresponding composite correlation function, C^μ. This can be seen in equations, by substituting the definitions of the composite weight vectors and correlation functions into Equation 8. Ignoring the effects of synaptic saturation (Eq. 5), this yields the following fourindependent dynamical equations:

$$\frac{d}{\text{dt}} S^{SUM}(\vec{x},\vec{\alpha })=\eta A(\vec{x},\vec{\alpha })\sum _{\vec{y},\vec{\beta }}I(\vec{x},\vec{y})C^{SUM}(\vec{\alpha },\vec{\beta })S^{SUM}(\vec{y},\vec{\beta })-4\epsilon (\vec{x})A(\vec{x},\vec{\alpha });$$ Equation 14a

$$\frac{d}{\text{dt}}{S}^{OD}(\vec{x},\vec{\alpha })=\eta A(\vec{x},\vec{\alpha })\sum _{\vec{y},\vec{\beta }}I(\vec{x},\vec{y})C^{OD}(\vec{\alpha },\vec{\beta })S^{OD}(\vec{y},\vec{\beta });$$ Equation 14b

$$\frac{d}{\text{dt}}{S}^{ORI+}(\vec{x},\vec{\alpha })=\eta A(\vec{x},\vec{\alpha })\sum _{\vec{y},\vec{\beta }}I(\vec{x},\vec{y})C^{ORI+}(\vec{\alpha },\vec{\beta })S^{ORI+}(\vec{y},\vec{\beta });$$ Equation 14c

$$\frac{d}{\text{dt}}{S}^{ORI-}(\vec{x},\vec{\alpha })=\eta A(\vec{x},\vec{\alpha })\sum _{\vec{y},\vec{\beta }}I(\vec{x},\vec{y})C^{ORI-}(\vec{\alpha },\vec{\beta })S^{ORI-}(\vec{y},\vec{\beta }).$$ Equation 14d

Note that all modes except S^SUM are described by a formally identical equation, Equations 14-14. This decomposition (Eqs. 14-14) was reported by Erwin and Miller (1996b)and was independently noted by Piepenbrock et al. (1996, 1997). The derivation is presented in more detail in Appendix .

When the symmetry assumptions apply, the outcome of development is most conveniently described in terms of the four modes, because they initially (before weights saturate) develop independently of one another. The final RF and map properties are largely determined by this early phase of development. In contrast, the development of each of the four original weight vectors, S^EC (Fig. 2b), depends on the other three original weight vectors. Thus, the modes S^μ and their corresponding composite correlation functions C^μ provide the natural language with which to characterize model behavior.

Because Equations 14-14 neglect the weight saturation constraints, Equation 5, they do not completely describe the model dynamics. It is awkward to express the saturation constraints in terms of the composite weight vectors. Thus, all simulations are actually carried out using the numerical algorithm described in Materials and Methods, which does not make reference to the modes or the composite correlation functions.

Independently growing synaptic patterns and their growth rates

Within each mode, S^μ, μ = {SUM, OD, ORI+, ORI−}, there are multiple, independently growing weight patterns, which we label S_i^μ. For example, one pattern of the OD mode, S₁^OD, might involve monocular cells with eye preference alternating in OD stripes having a period of four grid intervals, whereas another pattern, S₂^OD, might be similar but with a period of five grid intervals. Similarly, S₁^ORI+ might involve cells developing ON/OFF segregation with three horizontal subregions per RF, whereas another pattern, S₂^ORI+, might involve two vertical subregions per RF.

Each synaptic pattern, S_i^μ, has its own corresponding growth rate, λ_i^μ, which we number from largest to smallest: λ₀^μ ≥ λ₁^μ ≥ λ₂^μ ≥ … . Thus, the fastest-growing pattern of mode S^μis S₀^μ, with corresponding growth rate λ₀^μ. For fixed A and I, the shapes of the patterns of S_i^μ are determined by the shape of C^μ, whereas their growth rates λ_i^μ increase with the magnitude of C^μ and also depend on the shape of C^μ. [The growth rates are defined by the independent exponential growth of each pattern: letting S_i^μ(t) be the amplitude of S_i^μ at time t; then for small times t, i.e., before weights saturate, S_i^μ(t) = S_i^μ(0)e^λ _iμt. Technically, the S_i^μ and λ_i^μ are the eigenvectors andeigenvalues of Eqs. 14-14.]

The various synaptic patterns, S_i^μ, grow independently until synapses begin to saturate at the upper or lower cutoffs in Equation 5. The fastest growing patterns will dominate the overall weight pattern by the time saturation begins. Thus, the final RFs and map typically are determined by a mixing of only the fastest-growing weight patterns, so that central features of the outcome of development can be characterized by determining the structure of these patterns (this approach is discussed in more detail by Miller et al., 1989; Miller, 1990a, 1994). The fastest growing patterns may all belong to a single mode, such as OD or ORI, or may include members of several modes.

Overview

We will begin by discussing how each mode, S^μ, develops independently. This determines the conditions necessary for development of OD or of ORI selectivity alone. We then turn to the simultaneous development of several modes, for which we determine the conditions for binocular matching of preferred orientations and for combined development of OD and ORI. Finally, we consider the effects of sequential development, in which initially ORI develops alone, and OD development begins subsequently because of a change in input correlations. We find that sequential development does not qualitatively change the previously determined conditions but does quantitatively relax some of them. In addition, it leads to development of significant correlations between OD and ORI maps similar to those observed experimentally.

Our results are presented in terms of the form that the correlation functions C^μ must take to achieve a given developmental outcome. The results are very simple in this form. However, the C^μ functions are not directly accessible experimentally. As shown in Table 1 and Figure 2d, the C^μ functions are linear combinations of the correlation functions, C^S_ES_C, C^S_EO_C, C^O_ES_C, and C^O_EO_C, which correspond to more intuitive, experimentally measurable quantities. To allow our results to be understood in terms of these more intuitive quantities, we show in Figure3 the forms that C^S_ES_C, C^S_EO_C, C^O_ES_C, and C^O_EO_C take in our simulations. Figure 3 should be referred to along with each simulation. We have limited ourselves in the simulations to cases in which each of the C^μ functions can be expressed as a Gaussian (Eq. 3, Fig. 1b) or Mexican-hat (Eq.4, Fig. 1c) function.

Fig. 3.

Correlations in firing among LGN cells used in the simulations. In terms of composite correlation functions, these take the form C^ORI+ = r⁺M, C^ORI− = r⁻M, C^OD = dG₃, and C^SUM = 0 (see Fig. 1b,c). Here we illustrate these correlations in terms of the more intuitive, experimentally measurable functions C^S_ES_C, C^S_EO_C, C^O_ES_C, and C^O_EO_C (see Fig. 2a). a, LGN firing correlations corresponding to C^ORI+ and C^ORI− alone: d = 0. Model results for this case are shown in Figure 6. Correlation functions are shown for C^ORI+ dominant: r⁺ = 1, r⁻ ≤ r⁺. For C^ORI−dominant, r⁺ ≤ r⁻, the within-eye correlations would be identical to those shown here, whereas between-eye correlations would be of opposite signs to those shown here. b, LGN correlations attributable to C^OD with or without C^ORI+: r⁻ = 0. Functions of this form are used in Figures 5 and 7 and elsewhere. As in Figure 1b–e, the horizontal axes are in units of grid intervals (∼100 μm; Miller, 1994), and the vertical axes are in arbitrary units.

Separate development of individual modes

Each mode develops independently in the early part of development, before synaptic saturation begins to break the independence of Equations 14-14. In this section, we describe the determinants of the independent development of each mode. These results are summarized schematically in Figure 4. These results derive directly from the results of our previous two-input-type models, as explained in Appendix , so discussion here will be brief.

Fig. 4.

Top. Determinants of single-cell RF properties for cases in which development is dominated by a single correlation function. a, When C^OD(α⃗, β⃗) dominates the other correlation functions and is positive for distances within the radius of the arbor function A(x⃗, α⃗), monocular RFs tend to develop. The example RF has no connections to the right eye (shown as black) and high connection strength in the center of the left eye, RF, gradually falling to zero at the edges (shown as decreasing brightness). Yellow indicates that ON and OFF connections are mixed in the left RF. b, When C^ORI+(α⃗, β⃗) is dominant and oscillates from positive to negative within the arbor radius, as in the Mexican-hat function shown, both the left and right RFs of each cortical cell will develop segregated ON (green) and OFF (red) regions, giving each cell a preferred orientation. The ON and OFF regions in the left RF will be at the same locations as the ON and OFF regions in the right RF. c, When an oscillating C^ORI−(α⃗, β⃗) dominates, segregated, elongated ON and OFF regions in the left and right RFs again result. However, in this case the ON-center regions in one eye correspond to OFF-center regions in the other, and vice versa. Each form of between-eye matching of ON and OFF regions, in-phase (b) or antiphase (c), yields ocularly matched preferred orientations.

The basic intuition behind these results is that correlation-based development leads to RFs consisting of a maximally correlated subset of inputs (for the mathematical expression of this, see Miller and MacKay, 1994; Miller, 1997). Correlations are maximized when the spatial RF pattern of each S^μ closely follows the spatial structure of the corresponding C^μ: if a given C^μ is all of one sign, then so is the RF pattern that results for the corresponding S^μ; if a given C^μ oscillates in sign, so does the RF pattern of the corresponding S^μ, with a similar spatial period. In particular, the fastest growing patterns of each S^μ follow the form of the corresponding C^μ in this manner, and these patterns in turn determine the structure that develops in S^μ.

Development of ocular dominance

Ocular dominance is a net difference in total left-eye versus right-eye strength, summed over all input types of either eye. OD is represented only by the S^OD mode (Table1). Monocularity develops if cortical cells become dominated by a single eye throughout their RF, which means that S^OD has the same sign throughout the RF (positive for right-eye domination, negative for left-eye domination). The condition for this to occur is as follows (Fig. 4a):

Condition 1. For monocularity to develop, C^OD should be positive at least at small distances, and it should not significantly oscillate in sign within an arbor radius. (More precisely: the peak of the Fourier transform of C^OD must be at a wavelength long compared to the arbor diameter.) By arbor radius, we mean the radius over which LGN cells can project to a common cortical point.

Condition 1 was found by Miller et al. (1989). The condition there was expressed in terms of the net difference in total left-eye versus right-eye strength, which here is represented by S^OD; and in terms of the difference between within-eye and between-eye correlations, which here is represented by C^OD. Note that both S^OD and C^ODinclude a sum over ON and OFF center types (Table 1), whereas distinct center types were not considered by Miller et al. (1989). The applicability of those results to the present case is shown in Appendix.

The independent development of an OD map in the present four-input-type model is shown in Figure 5a, for a case in which only the S^OD mode develops (because all the C^μ values except C^OD are set to zero), and C^OD satisfies Condition 1. Here, C^OD was a broad Gaussian. Most cortical cells become monocular, as shown by the RFs of an 8 × 8 subset (Fig. 5c,d). ON- and OFF-center inputs develop identically. The form and spatial period of the OD map are jointly determined by the functions I and A precisely as in Miller et al. (1989).

Development of orientation selectivity:S^ORI+ and S^ORI−

A monocular RF develops ORI selectivity if it develops simple-cell structure: elongated, spatially segregated ON and OFF subregions. This means that the monocular RF must spatially oscillate, along one direction, between regions of ON domination and OFF domination; that is, the difference between ON-center and OFF-center strength must spatially oscillate in sign. For a binocular cell, each monocular RF should have such a structure, with matching preferred orientations.

S^ORI+ is the sum of the right-eye and left-eye ON/OFF differences. ORI selectivity develops if S^ORI+ spatially oscillates in sign within an RF. Oscillations in S^ORI+ correspond to ON/OFF segregation that is in phase in the two eyes: ON subregions in the right eye and ON subregions in the left eye grow in corresponding locations; likewise, OFF subregions in the two eyes are matched (Fig. 4b).

S^ORI− is the difference of the right-eye and left-eye ON/OFF differences. Just as for S^ORI+, ORI selectivity also develops if S^ORI− spatially oscillates in sign within an RF. Oscillations in S^ORI−correspond to ON/OFF segregation that is antiphase in the two eyes (Fig. 4c): ON subregions in the right eye and OFF subregions in the left eye grow in corresponding locations; likewise, OFF subregions in the right eye are matched to ON subregions in the left eye.

Only these two modes involve differences between ON-center and OFF-center innervation, and thus only these modes can yield simple cells. This yields the following condition (Fig. 4b,c):

Condition 2. For ORI selectivity to develop, the fastest growing patterns of either the ORI+ or ORI− mode must have RFs with simple-cell structure: elongated, spatially segregated ON and OFF subregions. This in turn means that the corresponding ORI correlation function, C^ORI+ or C^ORI−, should have a Mexican-hat or similar form that oscillates in sign within an arbor radius. (More precisely, it must have the peak of its Fourier transform at a wavelength corresponding to one or more cycles within an arbor diameter.)

Note that development of either S^ORI+ or S^ORI− alone yields RFs in which orientation is binocularly matched. Later, we will consider the case in which both modes develop and determine the more general conditions required for binocular matching of preferred orientation.

Condition 2 was found by Miller (1994), which considered two center types but ignored the existence of two eyes. The condition there was expressed in terms of the net difference in total ON-center versus OFF-center strength and the net difference between within-center-type and between-center-type correlations. The applicability of these results both to the binocular sum of these differences (S^ORI+ and C^ORI+, respectively) and to the binocular difference of these differences (S^ORI−and C^ORI−, respectively) is shown in Appendix .

The independent development of an ORI map in the present four-input-type model is shown in Figure 6a for a case in which only the S^ORI+ mode develops: C^ORI+ has a Mexican-hat shape, whereas C^SUM, C^OD, and C^ORI− are all set to zero. All cortical cells develop segregated ON and OFF RF subregions, as shown by the RFs of an 8 × 8 subset of cells (Fig. 6b), and this yields a continuous map of ORI preference (Fig. 6a). All cortical cells are binocular, with the two monocular RFs in phase: ON and OFF subregions and ORI maps are matched in the two eyes. If instead only the S^ORI− mode develops ( C^ORI− has a Mexican-hat shape, and C^SUM, C^OD, and C^ORI+ are all zero) an ocularly matched ORI map again develops. However, in this case the two monocular RFs develop in antiphase (Fig. 6c). In either case, the periodicity of the ORI map depends on C^ORI±, I, and A in the same complex way as did the maps of Miller (1994).

RFs corresponding to S^ORI− may seem functionally implausible: it may seem that they would not respond to binocular input, because excitation in the ON- or OFF-type region of one eye would be canceled by inhibition from the opposite-type region at the same location in the other eye. However, independent movements of the right and left eyes, as well as presentation of stimuli at various depths, complicate this simple picture. In fact, RF patterns such as S^ORI+ and S^ORI− have been proposed to underlie so-called “tuned-excitatory” and “tuned-inhibitory” cells, respectively (Freeman and Ohzawa, 1990; Nomura et al., 1990; DeAngelis et al., 1995). There is, however, some evidence that “tuned-inhibitory” cells are rare or absent in the cat (LeVay and Voigt, 1988).

Development of the summed synaptic strength

The S^SUM mode is the only mode affected by the competitive constraint, represented by the second, subtracted term in Equation 14a. This term ensures that the total synaptic strength to each cortical cell remains constant. This constrains the patterns of S^SUM that may grow: the strength of S^SUM to a cortical cell may increase from some LGN locations only if it correspondingly decreases from other LGN locations.

The development of S^SUM can be understood from the development of the summed synaptic strength in our previous two-input models (Appendix ), as discussed for example by Miller (1990a). A typical effect of non-zero C^SUM is refinement of the RFs, meaning that the RFs grow at their centers and decay at their peripheries without changing their total synaptic weight. Because our constraint is only a poor stand-in for the unknown mechanisms of biological competition, the details of growth of the S^SUM mode in our model are unlikely to be of biological relevance (also see Discussion). Growth of S^SUM is of relevance to us only insofar as it interacts with, and thus alters the growth of, the other modes. This interaction is discussed below.

Joint development of orientation and ocular dominance modes

We now consider the additional conditions on the correlation functions that must be met for both ORI and OD maps to co-develop, with binocularly matched ORI preferences. This requires study of the co-development of multiple modes.

The different modes grow independently until synapses begin to saturate at their upper or lower cutoffs (Eq. 5). As multiple synapses saturate, the overall synaptic pattern becomes locked in. Thus, the main effect of the different modes on one another involves their relative synaptic strengths at the time that saturation begins to halt development. This in turn depends on their relative growth rates. In general, for several modes to co-develop, their fastest growing patterns must have comparable growth rates, whereas modes with fastest growing patterns that have significantly smaller growth rates will have little influence.

Interaction of the two ORI modes

We have seen that each ORI mode alone yields binocularly matched ORI preferences. It turns out that the requirement for binocular matching of ORI preferences is that one of these modes should remain predominant; the mixing of the two modes destroys binocular matching. The general condition is as follows:

Condition 3. For ocular matching of ORI preferences to develop, the growth rates of one of S^ORI+and S^ORI− must sufficiently dominate those of the other. This in turn requires that between-eye correlations must be center-type-specific (C^O_ES_C and C^O_EO_C must differ significantly from one another).

To demonstrate this, we consider the case in which each is driven by correlations of identical Mexican-hat form but with independent magnitudes: C^ORI+ = r⁺ M; C^ORI− = r⁻ M (Fig. 3a). We have seen above that, if only r⁺ is positive (C^ORI+ dominant), left and right RFs will be in phase, whereas if only r⁻ is positive (C^ORI− dominant), left and right RFs will be in antiphase (Figs. 4b,c, 6a–c). One might imagine that, if r⁺ and r⁻ are both positive, the resulting codevelopment of the two ORI modes could lead to the development of ocularly matched ORI maps with other spatial phase relationships between the two eyes. However, this is not the case.

Instead, if S^ORI+ and S^ORI− codevelop with approximately equal growth rates, the two eyes develop independent, uncorrelated ORI maps and RFs (Fig. 6d,e). To understand this, consider first the case in which r⁺ = r⁻, so that C^ORI+ = C^ORI−. A glance at Table 1 shows that C^ORI+ = C^ORI− implies C^O_ES_C = C^O_EO_C. This means that an ON (or OFF) input of one eye cannot “tell the difference” (in terms of activity correlations) between an ON input and an OFF input of the other eye. In that case, although firing correlations within each eye have the form needed to generate simple-cell RFs, there is no mechanism to relate the structure of ON or OFF subregions in one eye to the ON or OFF subregions in the other eye. ON or OFF subregion structure, and thus preferred orientations, develop independently in each eye. When r⁺ and r⁻ are not exactly equal but have similar magnitude, the argument is the same: the intereye correlations differentiate the two center types only weakly, too weakly to cause the ON/OFF subregion structures that develop for each eye to become related. [These arguments can also be understood mathematically as follows: when the fastest growing patterns of S^ORI+ and S^ORI− have identical growth rates, then any two orthogonal linear combinations of these patterns also grow independently. But one such orthogonal pair is (S^ORI+ + S^ORI−)/2 = S^RN − S^RF, and (S^ORI+ − S^ORI−)/2 = S^LN − S^LF. Thus, the two eyes develop independent ON/OFF segregation patterns and ORI maps.]

In Figure 6f, we plot the degree of similarity between the two eyes’ maps as a function of λ₀^ORI−/λ₀^ORI+ = r⁻/r⁺, the ratio of the fastest growth rates of S^ORI+ and S^ORI−. When λ₀^ORI−≈ λ₀^ORI+, the left-eye and right-eye ORIs become independent. For the choices of functions explored here, the condition for ocular matching of ORI maps is λ₀^ORI− < (2/3) λ₀^ORI+ or λ₀^ORI+ < (2/3) λ₀^ORI− (Fig. 6f). The quantitative factor 2/3 may change for other choices of shapes for C^ORI+ and/or C^ORI− or for the functions A and I (although in limited exploration, we have found that it does not seem to vary much).

How much difference must exist between C^O_ES_C and C^O_EO_C for ORI preferences to become binocularly matched? Note that C^ORI± = C^S_EΔ_C ± C^O_EΔ_C, where C^S_EΔ_C = C^S_ES_C − C^S_EO_C and C^O_EΔ_C = C^O_ES_C − C^O_EO_C. Thus, the requirement is that C^O_EΔ_C, the difference between C^O_ES_C and C^O_EO_C, must be of significant size relative to C^S_EΔ_C. Furthermore, the difference between C^O_ES_C and C^O_EO_C must vary significantly within an arbor radius (and therefore, it cannot simply be attributable to differing mean rates of ON- and OFF-center activity). This can be seen in two ways. Intuitively, for the ORI maps of the two eyes to match, C^O_EΔ_C must be such that correct alignment of the two eyes’ RFs will significantly increase the between-eye correlations relative to other alignments. If C^O_ES_C and C^O_EO_C do not vary significantly within an arbor radius, then the between-eye correlations would not vary if one eye’s RF is rotated relative to the other eye’s RF, and so correct alignment would not be favored. Mathematically, C^O_EΔ_C must couple to the leading pattern of C^ORI+ or C^ORI−, whereas for orientation to develop, that fastest growing pattern should involve an oscillation across the RF. If C^O_EΔ_C were flat over an arbor radius, it would not have any influence on such a mode.

Small local variations in left- and right-eye dominance are visible in Figure 6e, even though C^OD = 0. This occurs primarily because the RFs of the left and right eyes begin with different random initial conditions, so S^OD begins with small random variations. Although S^OD does not change during linear development (because C^OD = 0), saturation nonlinearities can induce weak additional changes in S^OD. This does not yield strong or spatially structured OD segregation like that observed biologically; that can only emerge in the model as a pattern of S^OD driven by C^OD.

Interaction of the ORI and OD modes

For OD and ocularly matched ORI maps to jointly develop from a single, time-invariant set of correlation functions, the following condition must also be met:

Condition 4. The fastest growing patterns of OD and of ORI must have comparable growth rates.

We demonstrate this by varying the relative magnitudes of OD and ORI correlation functions while ensuring that the previous conditions are met.

To meet Condition 1, we set C^OD to the Gaussian function used to develop OD columns in Figure 5, scaled by a parameter d: C^OD = dG₃. To meet Conditions 2 and 3, we set one of the ORI correlation functions, which we arbitrarily choose to be C^ORI−, to zero and the remaining one, C^ORI+, to the Mexican-hat function M used to develop ORI columns in Figure 6, a and b. We continue to set C^SUM = 0; we will consider C^SUM more carefully in the next section. Changes in the correlations with d are illustrated in Figure 3b.

Fig. 5.

Bottom. a, An OD map developed when C^OD dominates: C^OD was set to a broad Gaussian, and the other correlation functions were all set to zero, as shown in Figure 3b, dashed curves (r⁺ = 0; d = 1). Bright and dark regions correspond to dominance by the right and left eyes, respectively. b, Color code used to represent strength of synaptic weights in all RF images. Strengths of ON- and OFF-center connections are represented by intensities of green and red, respectively. Minimum brightness (black) corresponds to strength zero; maximum brightness of red or green corresponds to maximum allowed synaptic strength 8. Yellow results from mixture of ON and OFF connections. c, d, Left- and right-eye RFs, respectively, of the 8 × 8 subset of cortical cells outlined in a. According to the arbor function, A(x⃗,α⃗), adjacent cortical cells receive input from adjacent, partially overlapping circular regions of LGN cells with diameter 13. Each circle in c and d shows synaptic strengths of these connections, S^EC(x⃗,α⃗), for one cortical cell.

Fig. 6.

Interaction of C^ORI+and C^ORI− determines the ocular matching of orientation maps. Here both functions are set proportional to a Mexican-hat function, C^ORI+ = r⁺M and C^ORI− = r⁻M (Fig. 1c), whereas C^OD = C^SUM= 0 (Fig. 3a). The maps and RFs that develop depend on the relative values of r⁺ and r⁻. a, b, For C^ORI+ dominant ( r⁺ = 1, r⁻ = 0), left- and right-eye RFs and ORI preference maps are virtually identical. a, Maps show preferred ORIs and ORI selectivities (represented by orientations and lengths, respectively, of line segments) of model cortical cells in the 32 × 32 periodic array. The longest line corresponds to an ORI selectivity Q (Eq. 13) of 0.20; mean selectivity is 0.11. Line lengths in all later ORI maps are normalized on the same scale. Preferred ORIs are also represented by hue, on a finer grid (128 × 128), with intermediate pixel ORIs determined by linear interpolation. Positive and negative singularities in the (interpolated) ORI maps are indicated by black circles. b, Left- and right-eye RFs of the 8 × 8 subset of cortical cells indicated by theboxes in a. c, RFs from maps qualitatively like those of a, but developed with C^ORI− dominant ( r⁺ = 0, r⁻ = 1). The left- and right-eye RFs have antiphase ON and OFF subregions but otherwise have virtually identical RFs and orientation maps (maps not shown, but see f). (Note that the RFs of b and c, are unrelated, because initial conditions of S^ORI+ and S^ORI− are each random and uncorrelated with one another.) d, e, When r⁺ and r⁻are of similar magnitude, the right and left eyes’ RFs develop independently. d, The independent left- and right-eye ORI maps for r⁺ = r⁻ = 1. e, Independent left- and right-eye RFs from simulation in d. f, Correlation (see Materials and Methods) between left- and right-eye responses to oriented stimuli as a function of λ₀^ORI−/λ₀^ORI+, the ratio of growth rates of S^ORI+ and S^ORI− (which here is equal to r⁻/r⁺). Responses, and thus orientation maps, are essentially identical (correlation of 1) for λ₀^ORI− < (2/3)λ₀^ORI+ or λ₀^ORI+ < (2/3)λ₀^ORI−. The correlation is slightly negative for λ₀^ORI+ = λ₀^ORI− because of spatial variation in overall strength of left and right connections: constraint on summed weight received by a cortical cell ensures that, on cells in which one eye has larger than average overall weight, the other eye has less than average.

Figure 7 illustrates the effect of varying the ratio of OD and ORI growth rates by varying d. Because only the ORI+ and OD modes have non-zero growth rates, the outcome is determined by the relative magnitudes of their fastest growing patterns. In this case, it turns out numerically that d is approximately equal to the ratio of growth rates, λ₀^OD/λ₀^ORI+ = 0.97 d, so that d can be used as a convenient stand-in for this ratio.

Fig. 7.

Combined development of OD segregation and ocularly matched ORI maps. Only C^OD and C^ORI+ are non-zero; C^ORI− = C^SUM = 0. As in Figure 6a, C^ORI+ is a Mexican-hat function, C^ORI+ = M (Fig. 1c), which leads to development of ORI-selective simple cells with matched orientations and phases in the two eyes. As in Figure 5, C^OD is a Gaussian function, which leads to OD segregation: C^OD(α⃗, β⃗) = dG₃(‖α⃗ − β⃗‖) (Fig. 1b). The magnitude d of C^OD is varied in a and b and c–f. For these functions, λ₀^OD/λ₀^ORI+ = 0.97 d ≈ d. When d is small, ORI dominates and OD segregation does not develop; when d is large, OD dominates and ON/OFF segregation and ORI do not develop. a, b, Binocular ORI preference maps (see Materials and Methods) for d = 1.0 and d = 1.6; note that preferred ORIs match in the two eyes for binocular cells. c–f, OD maps and sample RFs for d = 0.5, 1.0, 1.6, and 4.0. When an ORI map develops, its structure is largely unchanged by co-development of an OD map (compare Fig. 6a, which has d = 0, with a and b here). Similarly, to the extent to which an OD map develops, its structure is unchanged by co-development of an ORI map (compare Fig. 5a with d and e here). The sample RFs come from the 8 × 8 cells indicated byboxes in the maps.

When this ratio is small, as for d = 0.5 (Fig. 7c), no pattern of OD grows fast enough to compete with the rapidly growing ORI patterns. Thus the final map contains a map of ORI preference matched in the two eyes but no OD map. No cells are monocular, and most cells have well segregated ON and OFF subregions generating ORI preferences. At d = 1, patterns of OD and ORI grow at similar rates, generating concurrent representation of an ORI and an OD map (Fig. 7a,d). The map includes both binocular and monocular RFs. Very similar maps develop for d = 1.6, except that few cells remain binocular (Fig. 7b,e). As d is raised further, ORI patterns contribute less and less to the final outcome. Thus the final RFs for d = 4 (Fig. 7f) contain an OD map but show little separation into ON and OFF subregions and thus little ORI selectivity or map.

How similar must the growth rates of ORI and OD be to support co-development of both kinds of maps? To explore this, we now allow the shape as well as the amplitude of C^OD to vary: C^OD = dG_γ(see Eq. 3). The parameter γ controls the width of the Gaussian function forming C^OD: γ = 3 gives the function just studied in Figure 7, whereas larger γ yields a proportionately wider Gaussian. (Note from Eq. 3 that G_γ is normalized so that its magnitude, i.e., its two-dimensional integral, is independent of γ. Thus, γ controls width, whereas d controls magnitude.) For the functions used, we can numerically solve Equation 14b to find their independently growing patterns and their growth rates. The fastest growing pattern, S₀^OD, is always monocular for C^OD ∝ G_γ; its dominance in growth rate over binocular patterns increases with increasing γ (Miller, 1990a). We restrict attention to γ ≥ 2.5, a large enough value to ensure robust development of OD segregation. The growth rate, λ₀^OD, of S₀^ODdecreases with increasing γ and is proportional to the amplitude d.

The range of growth rate ratios, λ₀^OD/λ₀^ORI+, that allow co-development of ORI and OD is quantified in Figure8. As the growth rate ratio λ₀^OD/λ₀^ORI+ increases, the degree of OD segregation, 〈m〉 (Eq. 11), rises monotonically from near 0.0 to near 1.0 (Fig. 8a, solid curves), whereas the degree of ON/OFF segregation, Z (Eq. 12), falls monotonically (Fig. 8a, dotted curves). The ORI selectivity, Q (Eq. 13), also falls with increasing λ₀^OD/λ₀^ORI+ when this ratio is sufficiently large (Fig. 8b), in parallel with the decrease in ON/OFF segregation. Perhaps surprisingly, ORI selectivity increases with increasing λ₀^OD/λ₀^ORI+ for smaller values of this ratio. A likely explanation for this rise in ORI selectivity is that the patterns of ORI that are most impeded through competition with OD are those that have the lowest growth rates, and these generally have poorer ORI selectivity than the faster-growing ORI patterns. Modestly high OD growth rates, by out-competing the slower-growing ORI patterns, can thus increase ORI selectivity.

Fig. 8.

Quantitative measures of map properties for joint development of ORI and OD maps using correlation functions as in Figure7, except that we now also allow the width of C^OD to vary, proportional to the parameter γ (indicated by symbols). The growth rate ratio λ₀^OD/λ₀^ORI+ is varied by varying d. a, Solid curves, 〈m〉, the degree of OD segregation (see Eq. 11); dotted curves, Z, the degree of RF segregation of ON and OFF subregions (Eq. 12). b, Mean ORI selectivity, Q (Eq. 13). Higher values of λ₀^OD/λ₀^ORI+ favor development of OD, whereas lower values favor ON/OFF segregation and development of ORI selectivity. In all cases, ORI selectivities and maps are matched in the two eyes (because λ₀^ORI− = C^ORI− = 0; see Fig. 6f). c, The range of ratios λ₀^OD/λ₀^ORI+ that allows co-development of OD and ORI selectivity grows wider with increasing width, γ, of C^OD. We define the lower bound on the range of ratios as the value at which OD segregation 〈m〉 > 0.5 (from a) and the upper bound as the value at which ORI selectivity Q (from b) falls below the value it would have for λ₀^OD/λ₀^ORI+ = 0, i.e., for development with C^OD = 0. The growth rates (computed numerically) are given by: λ₀^OD = 14.04 d, 12.46d, 9.74d, 7.62d, and 3.94d for γ = 2.5, 3, 4, 5, and 8, respectively; λ₀^ORI+ = 12.84.

Different curves in Figure 8 correspond to different values of γ. Wider γ allows both ORI selectivity Q and ON/OFF segregation Z to develop for larger values of λ₀^OD/λ₀^ORI+, for the following reason. As γ widens, the growth rates of nonmonocular patterns of S^OD become smaller relative to those of the leading, monocular patterns. Therefore, for a fixed λ₀^OD/λ₀^ORI+, wider γ means that fewer OD patterns are competing against growth of S^ORI+, allowing fuller development of S^ORI+. The net effect of widening γ is that the range of relative growth rates that yields combined maps becomes less restricted, requiring less “fine tuning” of λ₀^OD/λ₀^ORI+. These ranges are quantified in Figure 8c.

ORI selectivity emerges more rapidly than OD, even though joint map development occurs only when ORI and OD modes grow at similar rates (Fig. 9). The similar rate of growth is reflected in the similar time courses of ON/OFF segregation Z and left/right segregation 〈m〉. However, ORI selectivity Q develops more rapidly than ON/OFF segregation: ORI selectivity can be strong as a result of only weak segregation of ON and OFF inputs. (The time courses of development of Z and Q can differ because the former is a linear function of the weights, whereas the latter is a nonlinear function.) The result is that the ORI map approaches its final state before OD segregation significantly develops. This is shown by the time course of the correlation, r, between the ORI map computed at any time step t and the final ORI map (Fig. 9a; compare time course of degree of OD segregation 〈m〉).

Fig. 9.

Emergence of map properties during a single simulation of joint ORI and OD map development. Correlation functions were as in Figure 7a,d. For these parameters, the fastest growing patterns of ORI and OD have nearly the same growth rate, λ₀^OD/λ₀^ORI+ = 0.97. a, Development of OD segregation, 〈m〉, ON/OFF segregation, Z, and the correlation, Pearson’s r, between the ORI map at any time step and the ORI map that will be reached at the final time step. Pearson’s r is computed from summed left- and right-eye responses to sinusoidal gratings, averaged over 18 orientations. b, Development of ORI selectivity, Q, occurs earlier than the development of either OD segregation, 〈m〉, or ON/OFF segregation, Z. High ORI selectivity is achieved by t ≈ 40, although ON/OFF segregation is low at this point. Even at this early time, the ORI map structure is already well correlated, r ≈ 65%, with its final structure. Learning rate is η = 0.008.

ORI selectivity develops before OD segregation over the entire range of λ₀^OD/λ₀^ORI+ in which OD and ORI maps jointly develop. This provides one possible explanation for the experimental fact that ORI selectivity reaches a reasonably high level before OD bands are observable (LeVay et al., 1978; Fregnac and Imbert, 1984; Crair et al., 1997a). However, this could also be explained if the correlations driving ORI development exist during a first developmental stage, before the onset of another correlation structure driving OD development; this “two-stage” developmental scenario is explored later in Results.

OD and ORI columns have been shown to preferentially intersect at sharp angles in primary visual cortex of both monkey (Obermayer and Blasdel, 1993) and cat (Crair et al., 1997b). This effect also occurs in many abstract feature map models (for review, see Erwin et al., 1995,Swindale 1996). Although the initial developments of OD and ORI map modes under Equation 14 are independent, relationships between them could develop after synapses have begun to saturate via Equation 5. In the simulations discussed so far, these saturation effects only occur during a short period near the end of a simulation, and we have found no statistically significant relationship between the OD and ORI map patterns. (Such relationships might weakly exist but might be lost in the noise because of the current low resolution of simulated ORI maps; testing this will require higher-resolution simulations.) We show later that a clearly demonstrable relationship between the maps does develop in two-stage development.

Influence of the SUM mode on ORI and OD development

Just as rapid growth in one of the OD or ORI modes can prevent significant growth in the other of these modes, rapid growth of patterns in the SUM mode can push some synaptic strengths to their limiting saturation values (Eq. 6) before OD and ORI map patterns have had time to develop. Thus we may state the final condition for joint development of ORI and OD maps:

Condition 5. Patterns of the SUM mode must not grow fast enough to prevent development of the OD or ORI modes.

The growth of the SUM mode is distinguished from that of the other modes in two ways. First, the competitive constraint in Equation 10requires that growth of S^SUM in some regions of any RF must be exactly canceled by decay of S^SUM in other regions of the same RF, whereas the other modes are unaffected by this constraint. Second, the initial condition of the SUM mode is close in shape to A(x⃗,α⃗), whereas the other modes are initially close to 0. This means that the SUM mode begins with some patterns [those similar to A(x⃗, α⃗)] having large size, whereas the other modes start with all patterns having approximately equal, small sizes. The initially large SUM patterns can dominate development through exponential growth, even with relatively modest growth rates. Both of these conditions make it difficult to predict when the SUM mode growth will interfere with growth of the other modes.

Because both the competitive constraint and the method of initializing weights are not likely to be correct in their details, we have restricted our analysis above to the case in which C^SUM = 0 and patterns of the SUM mode do not interfere. A more biologically accurate description of limits on synaptic growth is clearly needed before the effects of such limits on co-development of OD and ORI can be properly understood.

For the sake of completeness, however, we have made a limited study of the effect of variation in C^SUM on development. The results of this study are available as a supplement to the present paper (Erwin and Miller, 1998).

Time-varying correlation functions: sequential development of ORI and OD modes

We showed above that OD and ORI maps can jointly develop under a time-invariant set of correlation functions. Experimental evidence suggests that development of a binocular ORI map precedes development of an OD map (Crair et al., 1997a). Although this arises naturally in the above scenarios (Fig. 9), it also suggests an alternative developmental scenario in which the correlation functions change in time. In this scenario, a period of ORI but not OD development is followed by a period in which both ORI and OD are developing. This change could occur gradually, but it is easier to study a two-stage process with a sudden change. One might imagine, for example, that a sudden change in correlations occurs at the time of eye opening or when the vascular tunic of the eye’s lens begins to fall away [the latter approximately coincides with (Thorn et al., 1976) or somewhat precedes (Freeman and Lai, 1978) the onset of the critical period and the major onset of ocular dominance development (LeVay et al., 1978)].

Here we study the effect of such a two-stage scenario on our conclusions. We find that Conditions 1–5 for development of OD and ocularly matched ORI maps all remain true, but their application becomes more flexible. Conditions 1–3 must still each be met during at least one stage of development, but they need not all be met at the same time. Condition 4 remains qualitatively unchanged, as applied to the period in which both ORI and OD are developing, but the relative growth rates of OD and ORI modes can now cover a wider range.

We also find that two-stage development offers an additional benefit. It leads to stronger coupling of the OD and ORI maps, with relationships between their patterns similar to those that are experimentally observed. Although such coupling might also occur in the later time steps of a single-stage development, it is clearly demonstrable in maps from a two-stage process.

The two-stage simulations proceed as follows. The first stage, beginning from random initial conditions, involves development up to a specified time, t₁, with C^ORI+ the only non-zero correlation function, as in Figure 6a. Thus, only the S^ORI+ mode develops. The second stage continues the simulation with the same C^ORI+, but in addition now C^OD is non-zero: it is set to a broad Gaussian, with magnitude proportional to the parameter d, as in Figures 3b, 8, and 9.

An example of such a two-stage simulation is shown in Fig.10. Here, the first stage lasts only a short time (t₁ = 26). This short first stage yields only poorly organized, similar but nonidentical left- and right-eye ORI maps (Fig. 10a,b). Most synaptic connections are still weak, and there is only weak segregation of ON and OFF inputs (Fig. 10e).

Fig. 10.

Development of ORI and OD in a two-stage scenario. a, b, The first stage begins from random synaptic connections and develops under LGN correlation functions as in Figure 6a,b: C^ORI+ = M (Fig. 1c) and is the only non-zero composite correlation function. Weak ORI maps have developed, with little variation in OD, by t₁ = 26 with learning rate η = 0.008. c, d, The second stage continues from the synaptic weights in a and b, using correlation functions as in Figure8 with γ = 3 and d = 2 (λ^OD/λ^ORI+ = 1.9): C^OD is now a broad Gaussian, whereas C^ORI+ = M is unchanged. (A range of d values would give similar results; see Fig. 11.) c, ORI preference map at the end of the second stage. Along with map refinement, average ORI selectivity Q has increased from 0.069 (a, b) to 0.122 (c). Preferred ORI changes smoothly across OD boundaries, shown as black lines.d, OD map after the second stage. Preferred ORIs measured through the left and right eyes are shown as yellow andblue lines, respectively. e, f, Left- and right-eye RFs of the 8 × 8 subset of neurons outlined in a–d. Color scheme as in Figure 5b, with weights in e multiplied by 4 for visibility.

The second stage continues with C^ODnon-zero; in this case, d = 2. When the second stage terminates, the ORI map has fully developed (Fig. 10c), and an OD map has formed (Fig. 10d). Most synaptic connections have saturated at their upper or lower bounds (Fig. 10f).

The ORI map at the end of the second stage is shown as a binocular ORI map, because the ORI maps in the two eyes have become well matched. This can be seen by noting that, although most cells have connections to only one eye, the cells that are binocular have similar RFs in the two eyes (Fig. 10f), and the binocular ORI map is continuous across OD column borders (Fig. 10c,d).

Two-stage development allows greater flexibility than a one-stage process in the relative magnitudes of correlation functions that can be used to develop combined ORI and OD maps. We judge this range in the same way as in Figure 8. As the first stage is carried to longer times, t₁, joint OD and ORI maps develop in a progressively larger range of second-stage growth rate ratios, λ₀^OD/λ₀^ORI+, Fig.11. One-stage development allowed a range of 0.96 ≤ λ₀^OD/λ₀^ORI+≤ 2.0, that is, variation by a factor of 2.1. For two-stage development with t₁ = 26, this range expands slightly to a factor of 2.2; as t₁ increases to 66, the range expands to allow variation by a factor of 6.5 (Fig. 11e).

Fig. 11.

Two stage development, as in Figure 10, in which an ocularly matched, binocular ORI map begins to develop under one set of correlation functions and an OD map develops, along with continued development of ORI selectivity, under a second set of correlation functions. In the first stage, C^ORI+ = M (Fig. 1c) is the only non-zero composite correlation function. In the second stage, C^OD = dG₃ (Fig. 1b) is also non-zero, with d a variable parameter. Note λ₀^OD/λ₀^ORI+ ≈ d. In the left plots (filled symbols), C^ORI− remains zero throughout both stages, as in Figure 10. In the right plots(open symbols), C^ORI− = C^ORI+ in the second stage (which means the two eyes’ ORI maps develop independently during that stage; also see Fig. 12). The first phase lasts until time t₁ (indicated in the legend), with η = 0.008, and the second phase continues to completion with the same η. a, b, Degree of OD segregation, 〈m〉, and of segregation of ON and OFF subregions, Z. c, d, Mean orientation selectivity, Q. e, The range of λ₀^OD/λ₀^ORI+ that allows codevelopment of OD and ORI selectivity (computed as in Fig. 8c) grows wider with increasing length of the first stage t₁. f, The range for any t₁ is even wider when C^ORI− = C^ORI+ during stage 2. Filled symbols are repeated from e for comparison.

The two eyes’ ORI maps can develop independently in the second stage and still become matched: the degree of correlation between the two eyes’ maps shown in Figure 10, a and b, induced in the first stage, is sufficient to cause the two eyes to independently arrive at nearly identical ORI maps. The outcome of such development is shown in Figure 11, right column. Here we have made an additional change in the second-stage correlation functions: in addition to making C^OD non-zero, we also set C^ORI+ = C^ORI−. As shown previously, this equality implies that the left and right eyes’ orientation maps are developing independently, except for saturation effects. The range of variation of growth rate ratios, λ₀^OD/λ₀^ORI+, allowing joint development of OD and ORI is greater still under this scenario, growing to a factor of 8.8 for t₁ = 66 (Fig. 11f).

We have visually verified that such independent second-stage development leads to ocularly matched maps. However, it is difficult to characterize the correlation of the ORI maps in the two eyes when OD also develops, because most cells come to receive input from only a single eye. Therefore, to quantitatively demonstrate that independent development of the two eyes’ maps in the second stage leads to matched maps, we show the outcome of development of ORI alone (Fig.12). Here, the first stage is as before (only C^ORI+ is non-zero), whereas in the second stage, C^ORI− = C^ORI+, and C^OD remains zero. Thus, in the first stage, the two eyes’ ORI maps develop in a correlated fashion, yielding results such as in Figure 10, a and b; in the second stage, the two eyes’ ORI maps continue development completely independently of one another, and there is no OD development. As shown in Figure 12, the correlation between the maps of the two eyes continues to increase in the second stage. For t₁ ≥ 26, the two maps come to be essentially perfectly correlated. Once an ORI map has developed sufficiently far along a path, its outcome becomes well determined; the two eyes’ maps, having developed in a correlated way in early development, independently arrive at a common outcome.

Fig. 12.

Development of correlation between left- and right-eye maps for a two-stage simulation. Binocular ORI develops in the first stage, for which only C^ORI+ = M is non-zero. If C^ORI−remains zero (no symbol), the correlation rises monotonically and is near 1 within 40 steps. Open symbols show cases in which C^ORI− = C^ORI+ in a second stage, beginning at t₁; this means that the two eyes’ ORI maps develop independently in this stage. For t₁ ≥ 26, the correlated first-stage development has already created conditions such that each eye’s map, developing independently, will converge on a similar map. Thus, the correlation between the two eyes’ maps increases monotonically to a value near 1.

We can summarize by saying that the two-stage process removes the requirement for center-type-specific between-eye correlations in the second stage of development (recall that such correlations are required to correlate the development of the two eyes’ ORI maps; and that the condition C^ORI+ = C^ORI− implies that such correlations do not exist). It is sufficient that such correlations exist only in the first stage of development, e.g., up to the state illustrated in Figure10, a, b, and e.

A further benefit of the two-stage process is that it leads to a stronger coupling of the ORI and OD maps, in a direction that is observed biologically. The S^OD and S^ORI+ weights develop independently, under Equation 14, until the saturation constraints of Equation 5 begin to take effect, linking their development in complex ways. In one-stage development, the OD and ORI maps develop almost independently until they are nearly complete; therefore, coupling of the maps is very weak. In two-stage development, saturation may begin to couple OD and ORI development early in the second stage, i.e., early in OD map development. This coupling grows stronger for increasing t₁. The OD map that develops therefore can have a somewhat different form than that which develops in a one-stage process (compare Figs. 7d, 13a, developed with t₁ = 66).

Fig. 13.

Data from the end of a two-stage simulation as in Figure 10c,d, but with a longer first stage: t₁ = 66. a, The gray scale image shows the OD map after smoothing with the Gaussian filter exp (−x²), with x measured in cortical grid intervals. Circles mark locations of singularities in the ORI map. Each upward- ordownward-pointing triangle marks a local maximum (maximum of right-eye preference) or minimum (maximum of left-eye preference) of OD, m(x⃗), in the smoothed OD map. b, Cumulative histogram (rough curve) of the distances, as a fraction of map width, connecting each of the OD maxima and minima in the smoothed map to its nearest ORI singularity, for the map in a and for nine other two-stage simulations with identical parameters but different seeds for the random-number generator. Thesmooth curve shows the distribution expected if local extrema and singularities were each independently and randomly located with a uniform probability density (spatially Poisson; Chandrasekhar, 1954); this curve is proportional to p(r) = 1 − exp (−λπr²), where λ = 116.0 is the average number of singularities per ORI map. The observed distribution contains more small distances (mean, 0.0422) than the expected distribution (mean 1/2 $\sqrt{\lambda }$ = 0.0464). The two distributions are significantly different [p_KS < 8.17 × 10⁻⁸, where p_KS is the p value from Kolmolgorov–Smirnov (KS) test]. This effect weakens as the length of the first stage, t₁, is decreased and is very weak or absent for one-stage development. c, The “shuffled” cumulative histogram (rough curve) was obtained by matching pairs of ORI and OD maps from simulations using different seeds. This curve lies much nearer to and has the same mean value (0.0464) as the random distribution. Even though the histograms in b and c are clearly different, because there are many more data points underlying the shuffled histogram in c than the single-run histogram in b, the difference between shuffled and random distributions has only a slightly weaker p value (p_KS < 1.64 × 10⁻⁵) for c than b.

The relationship induced between the OD and ORI maps by the saturation constraints is similar to the relationship observed experimentally in cats (Crair et al., 1997b) in at least one respect: the most monocular regions of the OD map tend to lie closer to singularities in the ORI map than they would by chance (Fig. 13a–c). This coupling in the model is in the same direction as that observed biologically, but is weaker. Because the effect grows with increasing t₁, we illustrate the effect by showing the case t₁ = 66 (for t₁ = 26, the effect is barely visible).

The reason for this coupling of ORI and OD can be intuitively understood as follows. OD can grow most strongly where it has the least competition from ORI. Near pinwheels, ORI patterns on different nearby cells are least able to contribute to one another’s growth. OD patterns are oblivious to ON/OFF segregation patterns and hence grow equally well near pinwheels or near linear ORI regions. Hence, the OD patterns are able to develop most fully near pinwheels, where they face the least competition, or equivalently where synapses are least saturated when OD development occurs. It is likely that this tendency of pinwheels to form in the centers of OD bands is sufficient to explain the experimentally observed tendency toward perpendicular crossings of iso-ORI lines and OD borders. However, as in the one-stage simulations, we can find no statistically significant tendency toward perpendicular crossings in our two-stage maps; this may be attributable to the weakness of these effects in the model as well as to the noisiness in (low resolution of) the ORI maps.

Experimental predictions and tests

From these results we can extract two major sets of experimental predictions. One set relates to correlation structures, the other to the relation between the left- and right-eye RFs of mature simple cells.

Correlation structures

The correlation structures we have described should be observed in cat LGN at the time that ORI selectivity and OD are developing in layer 4 of cat V1. These structures would be measured by simultaneously recording pairs of LGN cells. The degree of correlations in each pair’s activity would be plotted against the retinotopic separation of the pair’s RFs. Separate such plots would be made for pairs of same or opposite eyes and same or opposite center types, yielding the four correlation functions C^S_ES_C, C^S_EO_C, C^O_ES_C, and C^O_EO_C (Fig. 2a).

The overall prediction is that Conditions 1–5 should be met. In more directly biological terms, the essential predictions are: (1) when ocularly-matched simple cell RF structure develops: (a) within eyes, the difference between same-center-type and opposite-center-type LGN correlations, C^S_ES_C − C^S_EO_C, should have Mexican-hat or oscillating structure; (b) intereye LGN correlations should be center-type-specific: C^O_ES_C and C^O_EO_C should be non-zero and differ from one another, in a manner that varies over an arbor radius; and (2) when OD develops, total within-eye LGN correlations, C^S_ES_C + C^S_EO_C, should be larger than total between-eye LGN correlations, C^O_ES_C + C^O_EO_C.

When and under what conditions should these correlations be measured? Optical measurements in cat show that ORI maps are ocularly well matched from their first appearance, at postnatal day 12 (P12), regardless of whether the animal has had visual experience (Crair et al., 1997a). Because this development is independent of vision, the relevant correlations would be in LGN spontaneous activity, in darkness (“dark activity”) or with eyelids closed. It is not precisely clear when ocularly matched ORI maps develop. P12 is only 1 d after synaptic connections from layer 4 to upper layers first cross into layer 3 (Callaway and Katz, 1992); optical measurements, which measure upper-layer activity, probably could not see an earlier map. With electrode studies, ORI-selective cells are found in deep layers as early as recording is possible (P6, before normal eye opening) (Albus and Wolf, 1984; Braastad and Heggelund, 1985) (earlier work reviewed byMovshon and Van Sluyters, 1981; Fregnac and Imbert, 1984). The optical results make it seem likely that the initial development of ORI preference in the deep layers is also ocularly correlated. Thus, the predicted correlation structure should probably be looked for in the period P6–P12, although it is possible that ocularly matched ORI maps develop even earlier.

Ocular dominance appears to develop somewhat later (LeVay et al., 1978;Crair et al., 1997a). Physiologically, some OD is first visible optically at P14 (Crair et al., 1997a). Until approximately P18–P20, visual experience appears to have no effect on ORI or OD development (Fregnac and Imbert, 1984; Crair et al., 1997a). The major development of OD, including equalization of the contralateral and ipsilateral projections and anatomical segregation, develops immediately after this time and depends strongly on visual experience (LeVay et al., 1978;Crair et al., 1997a). Our studies have shown that OD may appear later than ORI selectivity even if the relevant correlation structures are present at the same time. Thus, the OD-relevant correlations might be present in spontaneous activity as early as the ORI-relevant correlations and certainly should be present by P14. Such correlations should be present in visually induced activity after P20.

Binocular spatial phase relationships

Our model predicts “binocular subregion correspondence”: the ON and OFF subregions in the two eyes’ RFs for a given cell should be arranged with either a 0° (S^ORI+-dominant) or 180° (S^ORI−-dominant) spatial phase shift, measured in absolute LGN coordinates. This result is compatible with observed interocular phase shifts (Freeman and Ohzawa, 1990), because those phases are defined in relativecoordinates that move with the centers of each eye’s RF. Our model allows any value of such a relative phase shift, as long as it is accompanied by an appropriate position shift between the spatial envelopes of the two eyes’ RFs (Fig.14). Because of our choice of a narrow arbor function, only small position shifts between RFs of the two eyes develop in our simulations. Use of a wider arbor function relative to final RF size would allow development of larger position shifts (Berns et al., 1993) and thus a larger range of relative phase shifts.

Fig. 14.

Absolute and relative phases. The sinusoidal function in a has an absolute phase Φ defined relative to the coordinate axis. The Gabor function in b is composed of the sinusoid in a multiplied by a Gaussian envelope. It may be described by a relative phase φ₁, defined relative to the center of the Gaussian envelope. The Gabor function in c is composed of the same sinusoid (a), multiplied by a Gaussian envelope that is shifted in position relative to the one in b. Thus b and c show different relative phases, φ₁ and φ₂, even though they both have the same absolute phase, Φ, as the sinusoid in a.

Some support for subregion correspondence exists in the finding that most tuned excitatory cells in both cat (Ferster, 1981; LeVay and Voigt, 1988) and macaque V1 (Poggio and Fischer, 1977) have preferred disparity at 0°. This arises naturally from subregion correspondence, whereas it is not predicted if relative phase shifts and position shifts are uncorrelated (although non-zero preferred disparities can also arise from subregion correspondence; this will be discussed elsewhere). The same authors report that tuned excitatory cells are binocular, whereas near and far cells tend to be monocular (disparity tuning arising from inhibition from the nondominant eye). This is also consistent with the idea that when both eyes have significant input to a cell, their RFs are in subregion correspondence.

More realistic models of intracortical circuitry might lead to more complex possibilities. For example, multiple forms of co-existence between left- and right-eye RFs on individual cells might arise through chains or loops of excitatory and inhibitory network interactions. It seems inescapable, though, that the set of absolute spatial phases of left- versus right-eye RFs in individual layer 4 cells should not be consistent with a random distribution: there should be correlations between the absolute phase found in one eye’s RF and that found in the other eye’s for the preferred ORIs of the two eyes to become matched. This general prediction is robust and can be tested in adult binocular simple cells in cat layer 4.

We are aware of only one experiment (Anzai et al., 1997) that attempted to measure both position and phase shifts in the same set of cells. Unfortunately, we have found that the reference-cell-pair method used there is not sufficient to distinguish between the coordinated shifts of position and relative phase that we predict, and an uncorrelated distribution of these shifts (Erwin and Miller, 1997). Measuring many cells’ RFs simultaneously yields a stronger test. We will describe these results, and other consequences of binocular subregion correspondence that can be more easily tested, in detail elsewhere.

DISCUSSION

We have studied correlation-based competition among four input types (ON- and OFF-center cells from left and right eyes), and determined the conditions necessary for development of both (1) OD segregation and (2) ORI-selective simple cells with ocularly matched ORI preferences. These conditions, approximately summarized, are as follows:

(1)  For OD to develop, total within-eye activity correlations (summed between a pair of same-center-type and a pair of opposite-center-type LGN cells at a given separation) must be greater than or equal to total between-eye correlations for LGN input separations within the retinotopic equivalent of a geniculocortical arbor radius.

(2)  For ORI to develop, the activity correlations within each eye should have a “Mexican-hat” or oscillatory structure (correlations between same-center-type cell pairs greater than those between opposite-center-type cell pairs at small retinotopic separations but smaller at larger retinotopic separations within an arbor radius).

(3)  For preferred ORIs to be matched in the two eyes, between-eye activity correlations should be center-type-specific, in a manner that varies with input separation over an arbor radius. This ensures that maximal correlations arise when center-specific subregions in the two eyes are appropriately aligned; as a byproduct, this yields matching of preferred orientations.

(4)  For OD and ocularly matched ORI to develop together, the growth rates of OD and of ORI should be approximately comparable.

Conditions 1–3 are fairly general, met by a wide range of correlation structures. They may be met at different times, e.g., Condition 3 may be met only in an early stage in which binocularly matched ORI selectivity develops, whereas Condition 1 may be met only in a later stage in which OD develops. Condition 4 also does not require tight tuning: if ORI and OD develop under a single correlation structure, the ORI/OD growth rate ratio can vary by factors of ≥2–8; if ORI and OD develop in separate stages, this range can be increased as much as three times.

The maxima of OD in the model tend to arise near ORI singularities, as observed experimentally (Crair et al., 1997b), at least when ORI and OD develop in separate stages. A qualitative explanation is proposed: pinwheels slow the growth of ORI but not OD and hence are sites where OD development faces less competition.

Robustness of the results

Model results fall into two categories. Some are general to a large class of correlation-based learning rules; others depend on fairly arbitrary choices in modeling competition between modes.

The first category includes results describing the correlation structure needed to develop a given RF or map structure, e.g., Conditions 1–3 above and the qualitative explanation of the OD maxima–ORI pinwheel relationship. These results should be reasonably robust: they follow from the idea that development leads to RFs and maps that, in some sense, maximize input activity correlations.

Results in the second category involve competition between different modes, e.g., between OD and ORI, where each mode captures a different aspect of the input correlations. Examples include Condition 4 above, the fact that OD maxima–ORI pinwheel relationships are only demonstrable in two-stage and not one-stage simulations, or Condition 5 (stated only in Results), which involves suppression of OD or ORI development by growth of the SUM mode. These results depend on our simple-minded pictures of saturation and competition, in which different modes grow independently until they are coupled and then stabilized by saturation of synaptic strengths. Biology likely contains more subtle mechanisms of competition (Miller, 1996b; Davis and Goodman, 1998; Turrigiano et al., 1998). These might allow multiple aspects of input activity to be captured: strong development of one feature might allow subsequent development of another feature, rather than freezing development. Such behavior is seen in models in which cortical cells compete nonlinearly for activation (Ritter et al., 1992;Sirosh and Miikulainen, 1997; . Before model predictions in the second category can be given much weight, better understandings of competitive and nonlinear mechanisms are needed, both theoretically and biologically. This might considerably broaden the conditions allowing co-development of OD and ORI.

Piepenbrock and Obermayer, Piepenbrock et al. (1996, 1997), studying the present model, independently noted that nonlinearities determine the precise conditions allowing OD and ORI co-development (also see Erwin and Miller, 1996b). They concluded from this that codevelopment could not occur robustly. We have shown the opposite and believe the correct conclusion is simply that the quantitative details of conditions allowing co-development are not to be trusted.

Some nonlinearities might also modify results in the first category. For example, Feidler et al. (1997) report that certain nonlinearities can yield RFs that would not result from a linear rule. They studied development of direction tuning, but the results translate directly to the present model, with direction tuning replaced by disparity tuning (Wimbauer et al., 1997a,b). After translation, these results show that, in a one-dimensional environment in which the distribution of stimulus disparity has non-zero peaks symmetric about zero, certain nonlinearities allow intereye absolute phase differences other than 0 or 180° to develop. It will be interesting to explore these ideas in a 2-D environment with alternative intereye stimulus distributions.

Evaluation of assumptions

Our results depend on several simplifying assumptions. First, we have assumed symmetries between the two eyes and between the two center types. As discussed previously, this ignores actual asymmetries in projection strengths and activity patterns.

The symmetry between ON and OFF is not crucial. The “Mexican-hat” correlation structure is sufficient to robustly give simple-cell structure within each eye’s RFs. This yields an approximate balancing of total ON and OFF innervation, independent of asymmetries in overall activities of ON versus OFF inputs.

The symmetry between left and right is more important but can be regarded as a simple stand-in for the unknown mechanisms of competition, that is, for whatever forces lead to equalization of the projections of the two eyes. As discussed elsewhere (Miller and MacKay, 1994; Miller, 1996a), at least two mechanisms can achieve equalization: a Mexican-hat pattern of intracortical interactions, i.e., short-distance excitation and longer-distance inhibition; or a competitive process (e.g., competition for neurotrophins) that ensures that all afferents with approximately equal average activity achieve approximately equal projection strength. (Because we used a Mexican-hat cortical interaction here, we did not need to assume that the two eyes begin with equal strength; however, this assumption simplified analysis considerably). Given either symmetry assumptions or competitive mechanisms to render the eyes equal, the impact of correlations on development should be as described here.

The facts that there is initial contralateral dominance and that ocular equalization occurs only in the presence of vision have been suggested to require non-Hebbian mechanisms of development (Crair et al., 1997a). Instead, we believe they suggest that vision is required to induce adequate competition, e.g., adequate development of inhibition or release of neurotrophins.

We also assume that the LGN inputs from each eye can be divided into two homogeneous groups, ON and OFF (this could be extended to more groups; Appendix ). These groups are heterogeneous in their visual response properties; for example, the temporal phase of response to a drifting sinusoid is widely distributed among neurons of a single type (Saul and Humphrey, 1990). The key requirement of our model is an overall tendency to a Mexican-hat correlation structure between ON and OFF in darkness. A fair amount of diversity in the details, which amounts to addition of noise to the correlations assumed here, can be tolerated; the amount tolerable is a quantitative issue not explored here. Measurements have shown clear separations of ON from OFF retinal cells in their dark correlation properties (Mastronarde, 1983b; Meister et al., 1995; Wong and Oakley, 1996).

Possible sources of the postulated activity correlations

Because the development of ocularly matched ORI maps is independent of vision (Crair et al., 1997a), we presume it depends on LGN spontaneous activity. Are the correlation structures we predict plausible attributes of this activity?

For the ORI maps of the two eyes to become matched, we predict that the LGN must possess center-type-specific intereye correlations. In adults, both excitatory and inhibitory LGN interocular interactions exist (Xue et al., 1987; Guido et al., 1989; Schroeder et al., 1990; Moore et al., 1992; Tong et al., 1992). There are at least three potential sources of intereye LGN correlations at the developmentally appropriate times: the perigeniculate nucleus, the corticogeniculate feedback, and the inhibitory intrinsic connections within the LGN. The perigeniculate nucleus induces spindle waves that, in vitro, propagate across the retinotopic map of the LGN in register between the layers of the two eyes (Kim et al., 1995). These waves arise in ferret (McCormick et al., 1995) shortly before the major development of cortical ORI selectivity (Chapman and Stryker, 1993). Corticogeniculate feedback is established even earlier in development (Shatz and Rakic, 1981; Weber and Kalil, 1987; Johnson and Casagrande, 1993). Cortical simple cells, which retain center-type-specific information, project to LGN only in binocular regions (Tsumoto and Suda, 1980), suggesting such information may be important for LGN binocular interactions.

The existence of strong intereye correlations, apparently generated by corticogeniculate feedback, has recently been demonstrated in developing (P24–P27) ferret LGN, 0.5–1 week before the major development of ORI selectivity (Weliky and Katz, 1998). These correlations appear to have the proper form to drive the major features described thus far in ferret layer 4 (Zahs and Stryker, 1988): OD segregation (because the correlations are stronger within than between eyes) and ON/OFF segregation into cortical patches (because correlations within a single eye are stronger between cells of the same rather than opposite center type; note that differing ON and OFF mean activity levels could explain ON/OFF patch continuity across OD borders). It is not clear whether these correlations have the proper form to account for simple cells or their ocular matching, because retinotopic positions of the cells were not measured. However, it is also not clear whether such correlations should exist in ferret LGN: only 40% of the cells in ferret layer 4 are ORI-selective (Chapman and Stryker, 1993), and there is as yet no evidence as to whether these are simple cells.

For development of simple-cell RF structure, within-eye correlations should have Mexican-hat structure. As extensively discussed by Miller (1994), such correlations are expected from (1) the connectivity yielding center-surround receptive fields, combined with (2) the assumption that correlations in spontaneous activity are attributable to common input from photoreceptors (Mastronarde, 1983b). A possible flaw in this argument was also discussed: it relies on center-surround RF structure but discusses correlations in spontaneous activity, which may involve dark-adapted cells. In adult cats, dark-adapted cells tend to lose their surrounds in retina (Enroth-Cugell and Lennie, 1975;Barlow and Levick, 1976) and perhaps also LGN (Virsu et al., 1977;Kaplan et al., 1979). Recent studies in salamander retina demonstrate a Mexican-hat correlation structure between ON and OFF ganglion cells in spontaneous activity both in light and darkness (Meister et al., 1995). These cells also lose their surrounds in darkness (M. Meister, personal communication). Thus, at least in salamander, the circuitry that produces the RF surround in light may equally influence correlations in light and darkness.

These arguments focus on adult-like correlations based on spontaneous quantal events in photoreceptors (Mastronarde, 1983b, 1989). An alternative correlation structure exists in early developing retina, attributable to spatially traveling waves of activity (Wong et al., 1993; Wong and Oakley, 1996). It was argued by Miller (1994) that these waves probably are not involved in development of simple cells and ORI selectivity. The arguments were threefold: the waves are far too wide to produce simple cell subregions; they disappear about or slightly before the time that ORI selectivity first arises; and no ON/OFF difference had been observed in the waves. An ON/OFF difference has since been observed in ferrets (this has not yet been studied in cats) beginning at P14, when ON/OFF segregation begins in LGN (Wong and Oakley, 1996), but ∼2.5 weeks before significant development of ORI selectivity (Chapman and Stryker, 1993; Chapman et al., 1996). Although this difference is appropriate to drive ON/OFF segregation in ferret LGN (Lee and Wong, 1996; Miller, 1996b), the timing and size of the waves continue to render them unlikely candidates for a role in simple cell development.

Other models

Only one other model has addressed ocular matching of ORI preferences in the absence of vision (Wolf et al., 1996). That work concluded that boundary conditions such as those of cat area 18, but not area 17, could cause well correlated ORI maps to develop from activity-dependent mechanisms. We predict that interocular correlations in spontaneous activity induce matching of the two eyes’ maps in both areas 17 and 18; boundary conditions may also contribute in area 18 but are not needed given such correlations.

Previous models of joint OD and ORI map development are of two types. Some used reduced description of inputs and RFs in terms of a few low-dimensional “features,” such as OD, ORI, and retinotopic position (Obermayer et al., 1992; Swindale, 1992) (Durbin and Mitchison, 1990; as extended in Erwin et al., 1995). These models assume a single ORI map and thus do not address ocular matching of the maps. More generally, they do not explain how RF features are generated from the inputs of many separate synapses. Synapse-based models have been studied by several authors. Tanaka (1996) and Olson and Grossberg (1998) assumed between-eye correlations were zero and so obtained only ocularly uncorrelated orientation maps. Shouval et al. (1996) studied development of a single cortical cell’s RF in response to natural images (well correlated between the two eyes); the problem of ocular matching without vision was not addressed, and the approach was not extended to multiple cells and maps. The more general relationship of our separate ORI and OD models to these and other modeling approaches has been extensively discussed (Miller, 1990a, 1994, 1996a; also seeErwin et al., 1995; Swindale, 1996).

Conclusion

Our model is very simple and has many inadequacies. Nonetheless, the richness and diversity of the experimental results accounted for, starting from extremely simple assumptions of a correlation-based synaptic competition, suggest that the basic explanations proposed here may underlie the more complex results of biological development.

The model reveals that the two most salient RF properties of primary visual cortical cells, OD and ORI selectivity, arise naturally from competition among inputs representing two eyes and two center types. Given the symmetry assumptions, the natural variables describing development in this system are ocular difference (S^OD), center-type difference (S^ORI+), and their combination (S^ORI−). The neural organizations of OD and ORI arise as a result of simple properties of the input correlations driving these variables (Fig. 4). It seems possible that many RF properties may arise from such competitions between differing input types [e.g., lagged vs nonlagged, which may yield development of direction selectivity (Feidler et al., 1997; Wimbauer et al., 1997a,b) and color-selective input subtypes in the monkey parvo system; also see Appendix ]. As discussed by Miller (1994), this raises the possibility that such RF properties may arise, not through selection for their computational role in visual processing (although such selection shapes internally generated input correlation structures), but rather as byproducts of deeper and more general, modality-independent principles of cortical computation.

In conclusion, cortical self-organization via correlation-based synaptic competition can explain the development of linked ORI maps in the two eyes in the absence of vision, along with development of OD segregation, and can naturally account for the observed correlation between OD maxima and ORI singularities. The role of self-organization has two key tests: appropriate LGN correlations as described here must exist at the times that ORI selectivity and OD first develop; and binocular simple cells in adult layer 4 should have nonrandom absolute phase relationships between their right- and left-eye receptive fields.

FURTHER DETAILS OF MODE DECOMPOSITION

The decomposition of Equation 14 (which amounts to diagonalization of the matrix of correlation functions; e.g., see Erwin and Miller, 1996b) can be easily found as follows. We assume a symmetry of our equations under exchange of right-eye and left-eye labels and under exchange of ON-center and OFF-center labels. If our equations (e.g., Eq. 2) are invariant under such exchange, then the eigenvectors of those equations can be expressed as eigenvectors of the “exchange operators.”

Eigenvectors under exchange are easily found as the sum (Σ) or difference (Δ) of the weights designated by the two exchanged labels. For example, for the ocular label E ∈ {L, R}, the sum S^Σ_E = S^R+ S^L is equal to itself under L/R exchange and hence has eigenvalue +1 under such exchange, whereas the difference S^Δ_E = S^R − S^L is equal to its negative under L/R exchange and hence has eigenvalue −1 under such exchange. The same procedure applies to the center-type label C ∈ {N, F}. This provides a basis for finding the decomposition, by proceeding one label at a time. We let λ_E, λ_C be the eigenvalue under L/R or N/F exchange, respectively. Then the patterns are found as follows:

Begin with two labels, L and R. Eigenvectors are:

$$S^{{\mathrm{\Sigma }}_E}=S^R+S^L,{\lambda }_E=+1$$

$$S^{{\mathrm{\Delta }}_E}=S^R-S^L,{\lambda }_E=-1$$

Add additional two labels, N and F. Eigenvectors are:

$$S^{{\mathrm{\Sigma }}_E{\mathrm{\Sigma }}_C}=(S^{{\mathrm{\Sigma }}_E}{)}^N+(S^{{\mathrm{\Sigma }}_E}{)}^F,{\lambda }_E=+1,{\lambda }_C=+1$$

$$S^{{\mathrm{\Sigma }}_E{\mathrm{\Delta }}_C}=(S^{{\mathrm{\Sigma }}_E}{)}^N-(S^{{\mathrm{\Sigma }}_E}{)}^F,{\lambda }_E=+1,{\lambda }_C=-1$$

$$S^{{\mathrm{\Delta }}_E{\mathrm{\Sigma }}_C}=(S^{{\mathrm{\Delta }}_E}{)}^N+(S^{{\mathrm{\Delta }}_E}{)}^F,{\lambda }_E=-1,{\lambda }_C=+1$$

$$S^{{\mathrm{\Delta }}_E{\mathrm{\Delta }}_C}=(S^{{\mathrm{\Delta }}_E}{)}^N-(S^{{\mathrm{\Delta }}_E}{)}^F,{\lambda }_E=-1,{\lambda }_C=-1$$

Alternatively, begin with two labels, N and F. Eigenvectors are:

$$S^{{\mathrm{\Sigma }}_C}=S^N+S^F,{\lambda }_C=+1$$

$$S^{{\mathrm{\Delta }}_C}=S^N-S^F,{\lambda }_C=-1$$

Add additional two labels, L and R. Eigenvectors are:

$$S^{{\mathrm{\Sigma }}_C{\mathrm{\Sigma }}_E}=(S^{{\mathrm{\Sigma }}_C}{)}^R+(S^{{\mathrm{\Sigma }}_C}{)}^L,{\lambda }_C=+1,{\lambda }_E=+1$$

$$S^{{\mathrm{\Sigma }}_C{\mathrm{\Delta }}_E}=(S^{{\mathrm{\Sigma }}_C}{)}^R-(S^{{\mathrm{\Sigma }}_C}{)}^L,{\lambda }_C=+1,{\lambda }_E=-1$$

$$S^{{\mathrm{\Delta }}_C{\mathrm{\Sigma }}_E}=(S^{{\mathrm{\Delta }}_C}{)}^R+(S^{{\mathrm{\Delta }}_C}{)}^L,{\lambda }_C=-1,{\lambda }_E=+1$$

$$S^{{\mathrm{\Delta }}_C{\mathrm{\Delta }}_E}=(S^{{\mathrm{\Delta }}_C}{)}^R-(S^{{\mathrm{\Delta }}_C}{)}^L,{\lambda }_C=-1,{\lambda }_E=-1$$

[Note: (S^Σ_E)^N = (S^R + S^L)^N = S^RN + S^LN; (S^Δ_C)^R = (S^N − S^F)^R = S^RN − S^RF; etc.]. Note that eigenvectors with identical superscripts are equal, regardless of superscript order (e.g., S^Σ_EΣ_C = S^Σ_CΣ_E, S^Σ_EΔ_C = S^Δ_CΣ_E, etc.).

We now identify:

$$S^{SUM}=S^{{\mathrm{\Sigma }}_E{\mathrm{\Sigma }}_C}=S^{{\mathrm{\Sigma }}_C{\mathrm{\Sigma }}_E}$$ Equation 15

$$S^{OD}=S^{{\mathrm{\Delta }}_E{\mathrm{\Sigma }}_C}=S^{{\mathrm{\Sigma }}_C{\mathrm{\Delta }}_E}$$

$$S^{ORI+}=S^{{\mathrm{\Sigma }}_E{\mathrm{\Delta }}_C}=S^{{\mathrm{\Delta }}_C{\mathrm{\Sigma }}_E}$$

$$S^{ORI-}=S^{{\mathrm{\Delta }}_E{\mathrm{\Delta }}_C}=S^{{\mathrm{\Delta }}_C{\mathrm{\Delta }}_E}$$

The corresponding eigenvalues under the original equation (Eq. 2) are proportional to C^SUM, C^OD, C^ORI+, and C^ORI−, respectively; these can be found, it turns out, by applying the same procedure to the C values, if N, F, R, and L are respectively replaced by S_C, O_C, S_E, and O_E (these labels are defined in Fig. 2a).

This method can be applied to arbitrary dualistic symmetries of input types. For example, if there were eight input types corresponding to three symmetries (e.g., left/right, ON/OFF, and lagged/nonlagged), then there would be eight independently growing composite weights, corresponding to the four above with eigenvalue +1 and −1 under the new symmetry (i.e., each of the four composite S^xs above would split into two composite weights, [(S^x)^lagged ± (S^x)^nonlagged], where ± gives eigenvalue ±1 under lagged/nonlagged exchange; and the corresponding correlation functions would be (C^x)^Σ_T and (C^x)^Δ_T, where Σ_T and Δ_T stand for sum and difference, respectively, of same-temporal-type and opposite-temporal-type correlation functions).

More generally, no matter how many types of cells there may be and whether they form dualistic symmetries, as long as the symmetries hold between left and right and between ON and OFF, then, if we characterize inputs only by L/R/N/F (ignoring all other distinctions), then the development of L/R/N/F will still be described in the linear regime by the above four modes and composite correlation functions. However, the underlying correlation functions C^EC, E ∈ {S_E, O_E}, C ∈ {S_C, O_C} may be more difficult to determine, because they will involve various linear combinations over correlation functions involving the various other subtypes. Nonetheless, assuming we have calculated the C^EC correctly, our analysis will then correctly describe the development of L/R/N/F (ignoring all other distinctions) in terms of these correlation functions.

RELATIONSHIP OF TWO-INPUT-TYPE AND FOUR-INPUT-TYPE MODELS

We previously considered two-input-type models of OD (Miller et al., 1989) and ORI (Miller, 1994) development. We label the two input types S^I, I ∈ {1, 2}. In the OD model, S¹ and S² were right-eye and left-eye inputs, S^R and S^L, whereas in the ORI model, S¹ and S² were ON-center and OFF-center inputs, S^N and S^F. These models used equations identical to those of the present model (Eqs. 1, 2, 5-9), except for the use of only two rather than four input types (replacing the combination E, C by I and E′, C′ by I′, the equations become identical).

In both two-input-type models, we assumed a symmetry of the equations under exchange of S¹ and S², which implies that C¹¹ = C²² ≡ C^SAME and C¹² = C²¹ ≡ C^OPP. Development before saturation limits took effect was then reexpressed in terms of composite variables: S^S = S¹ + S², S^D = S¹ − S², C^S = C^SAME + C^OPP, and C^D = C^SAME− C^OPP. Again, the procedures of the present paper are identical except for the change in the number of input types.

The resulting equation for development of S^D in terms of C^D was identical to each of Equations14-14 for development of S^μ in terms of C^μ, μ ∈ {OD, ORI+, ORI−}. We previously showed how the independently growing patterns of S^D and their growth rates are determined by C^D. Because the equations are identical, these results apply directly to tell how the independently growing patterns of each S^μ, μ ∈ {OD, ORI+, ORI−}, and their growth rates are determined by the corresponding C^μ.

The resulting equation for development of S^S in terms of C^S was identical to Equation 14a for development of S^SUM in terms of C^SUM. [There is a superficial difference: a factor of 2 rather than 4 multiplying ε(x⃗), which reflects the number of input types. The equations become identical if Eq. 9 for ε is rewritten in terms of S^SUM (or S^S) rather than S^EC (or S^I).] Thus, our previous studies of the development of S^S can be applied directly to understand development of S^SUM under Equation 14a.

Beyond formal equivalence of the equations, the two-input-type models can also be seen as special cases of the present model. If we define S^R = S^RN + S^RF, S^L = S^LN + S^LF, C^SAME = C^S_ES_C + C^S_EO_C, and C^OPP = C^O_ES_C + C^O_EO_C, then S^D and C^Din the OD model are identical to S^OD and C^OD in the present model. Thus, the two-input-type OD model can be understood as a special case of the present model in which ON/OFF differences are zero, so that S^ORI+ = S^ORI− = 0. Similarly, if we define S^N = S^RN + S^LN, S^F = S^RF + S^LF, C^SAME = C^S_ESC + C^O_ES_C, and C^OPP = C^S_EO_C + C^O_EO_C, then S^D and C^Din the ON/OFF model are identical to S^ORI+ and C^ORI+ in the present model. So the two-input-type ON/OFF model can be understood as a special case of the present model in which left–right differences are zero, so that S^OD = S^ORI− = 0. These equivalences can also be seen from Appendix : setting all terms involving Δ_E to zero, we arrive at the ON/OFF model, whereas setting all terms involving Δ_C to zero, we arrive at the left/right model.