Figure 4.

Effect of self-excitation on the sigmoidal activation unit defined by equation 2. (A) The input-output equilibrium function f_γ,k of the system (thick black curve) for the sigmoidal unit with self-excitation k = 0 and noise coefficient c set to 0; when c > 0, this function still describes the average behavior of the system. Upward and downward arrows and shading indicate system velocity at different input-output combinations (dark = negative; light = positive). (B) Self-excitation k = 1/(1 − γ) cancels leak and produces a vertical effective activation function and leakless temporal integration [k < 1/(1 − γ) produces leaky integration]. (C) Latching/switching behavior occurs for all k > 1/(1 − γ); here k = 2/(1 − γ). (D) A section of the complete catastrophe manifold defined over the space of input and self-excitation pairs. (E) Binarization diagram demonstrating separated ON (1) and OFF (0) areas of output activation and hysteresis. Dashed section of S-curve denotes unstable equilibria; solid sections denote stable equilibria.