FIG. 13.

[Adapted from (Friedman et al., 2001)] Illustration of LASSO (left) and Ridge regression (right). The blue concentric ovals are the contours of the regression function while the red shaded regions represent the constraint functions: (left) |w₁| + |w₂| ≤ t and (right)$w_1^2+w_2^2\le t$. Intuitively, since the constraint function of LASSO has more protrusions, the ovals tend to intersect the constraint at the vertex, as shown on the left. Since the vertices correspond to parameter vectors w with only one non-vanishing component, LASSO tends to give sparse solution.