FIG. 2.

An illustrative two-dimensional problem. (a) Cartoon depicting two stimulus response patterns in a simple feedforward network with two input neurons and one driven neuron. (b) Since the driven neuron in panel (a) responds in one condition but not the other, we have one constrained dimension (magenta axis) and one semiconstrained dimension (green axis). The yellow ray depicts the space of weights, (w₁, w₂), that generate the stimulus transformation. The weight vector $\left(\frac{1}{2},\frac{1}{2}\right)$ (brown dot) would uniquely generate the neural responses in a linear network. We assume that the magnitude of the weight vector is bounded by W, such that all candidate weight vectors lie within a circle of that radius. A nonzero synapse x₂ → y exists in all solutions, but the x₁ → y synapse can be zero because the yellow ray intersects the w₁ = 0 axis.