Fig. 5.

Optimal network size for linear and nonlinear networks in the presence of intrinsic synaptic noise. (A) Network expansion for a linear network, given by an embedding into a larger network, followed by a rotation of the weight matrix. This corresponds to transforming inputs $\mathbf{u}$ by a projection $\mathbf{B}$ and outputs $\mathbf{y}$ by a semiorthogonal mapping $\mathbf{D}$. (B) Plots show the dependence of $N_{\text{opt}}$ in linear and nonlinear networks using Eqs. 17 and 19. In both cases the learning rule has ${\gamma }_1=0.01$ and $T=2$. Low task-irrelevant plasticity corresponds to ${\gamma }_2=0.05$, while high task-irrelevant plasticity corresponds to ${\gamma }_2=1$.