Fig. 2.

Mean error rate achieved when optimizing the CE loss in the binary single-layer network, as predicted by the replica analysis, at various values of $\gamma$ (increasing from top to bottom). The figure also shows the points ${\alpha }_c\approx 0.83$ (up to which solutions exist) and ${\alpha }_U\approx 0.76$ (up to which nonisolated solutions exist). (Inset) Binary CE function $f_{\gamma }\left(x\right)$ for various values of $\gamma$ (increasing from top to bottom). For low values of $\gamma$, the loss is nonzero even for small positive values of the input, and thus the minimization procedure tends to favor more robust solutions. For large values of $\gamma$ the function tends to $max\left(-x,0\right)$. The dotted line shows the corresponding NE function, which is just 1 in case of an error and 0 otherwise (cf. Eq. 1).