Fig. 1.

Only balanced solutions can be robust to both input and output noise. Each panel depicts membrane potentials resulting from different input patterns in a classification task. Weights are unbalanced [$\left|\text{𝒘}\right|=\mathcal{O}(1/\sqrt{N})$, Top Left and Bottom Left] or balanced [$\left|\text{𝒘}\right|=\mathcal{O}\left(1\right)$, Top Right and Bottom Right]. The neuron is in an active state only if the membrane potential is greater than the threshold $V_{\mathrm{th}}$. The input pattern class (plus or minus) is specified by the squares underneath the horizontal axis. Each input pattern determines a membrane potential (mean, horizontal bars) that fluctuates from one presentation to another due to input noise (Top Left and Top Right) and output noise (Bottom Left and Bottom Right). Vertical bars depict the magnitude of the noise in each case. The variability of the mean $V_{\mathrm{PSP}}$ across input patterns (which is the signal differentiating input pattern classes) is proportional to $\left|\text{𝒘}\right|$. As a result, the mean $V_{\mathrm{PSP}}$s for unbalanced solutions (Top Left and Bottom Left) cluster close to the threshold [difference from threshold $\mathcal{O}(1/\sqrt{N})$]. For balanced solutions (Top Right and Bottom Right), the mean $V_{\mathrm{PSP}}$s have a larger spread [potential difference $\mathcal{O}\left(1\right)$]. Input noise (fluctuations of $x_i$, Top Left and Top Right) produces membrane potential fluctuations with SD that is proportional to $|\text{𝒘}|$, which is of $\mathcal{O}(1/\sqrt{N})$ for unbalanced solutions (Top Left) and of $\mathcal{O}\left(1\right)$ for balanced solutions (Top Right). Output noise (Bottom Left and Bottom Right) produces membrane potential fluctuations that are independent of $\left|\text{𝒘}\right|$, so it is of the same magnitude for both solution types. Thus, while both balanced and unbalanced solutions can be robust to input noise, only balanced solutions can also be robust to substantial output noise.