Fig. 3.

Replicated SGD on a fully connected committee machine with $N=1$,605 synapses and $K=5$ units in the second layer, comparison between the noninteracting (i.e., standard SGD) and interacting versions, using $y=7$ replicas and a minibatch size of $80$ patterns. Each point shows averages and standard deviations on $10$ samples with optimal choice of the parameters, as a function of the training set size. (Top) Minimum training error rate achieved after ${10}^4$ epochs. (Bottom) Number of epochs required to find a solution. Only the cases with $100\%$ success rate are shown (note that the interacting case at $\alpha =0.6$ has $50\%$ success rate but an error rate of just $0.07\%$).