Figure C.13.

Left) The gradient, $\overrightarrow{g}$, has two components: one for contrastive divergence, ${\overrightarrow{g}}_{CD}$, the other from L₁ regularization, ${\overrightarrow{g}}_{L_1}$ . If the dot product of ${\overrightarrow{g}}^{\mathsf{T}}{\overrightarrow{g}}_{CD}>{\overrightarrow{g}}^{\mathsf{T}}{\overrightarrow{g}}_{L_1}$, then contrastive divergence dominates. We measure the dot products at iterations W away from the solution, W* . Right) Dot products of the contrastive divergence and L₁ gradients, ${\overrightarrow{g}}_{CD}$ and ${\overrightarrow{g}}_{L_1}$ , onto the total gradient, $\overrightarrow{g}$, for real fMRI (top) and simulation (bottom) data across step distance from solution with SMs thresholded at 1 standard deviation. The L₁ gradient and CD gradients are equal near the solution, but CD quickly dominates farther away. This indicates that while L₁ as a stronger role close to the solution, the overall gradient is dominated by CD.