Figure 7.

Activity costs force RNNs to adopt low firing rates and ignore firing rate decrements. A, Schematic of a network trained to detect interleaved increases and decreases in contrast. The input is a single, oriented Gabor that changes in contrast, which is passed to 24 units that are tuned to orientation and contrast (12 excited or inhibited by contrast changes). The contrast-responsive layer passes input to a recurrent layer with 100 interconnected units (80 excitatory/20 inhibitory) that project to a single output node that controls lever releases. B, Increasing activity costs (x axis) drive recurrent networks to lower basal rates of firing (y axis). Points represent the results obtained from different networks trained with different activity costs. C, The proportion of output weights from the recurrent layer that are positive increases as function of activity cost. D, Normalized ROC (mean of all contrast changes). Range extends from −1 (all units decrement in firing) to 1 (all units increment). E, Task accuracy as a function of activity cost when all units with positive weights are removed. F, Task accuracy as a function of activity cost when all units with negative weights are removed.