Algorithm 2 K-SVD algorithm

Task: Find the best dictionary to represent the data samples ${\left\{y_i\right\}}_{i=1}^N$, $y_iϵR^N$ as sparse compositions by solving: $$mi{n}_{D,X}\left\{{\Vert Y-DX\Vert }_F^2\right\}\mathrm{subject}\mathrm{to}{\forall }_i,{\Vert x_i\Vert }_0\le T_0.$$

Initialization: Set the dictionary matrix $D^{\left(0\right)}$$ϵ$$R^{n\times K}$ with $l^2$ normalized columns. Set $J=1$.

Iterations: Repeat until convergence: Sparse coding stage: Use any pursuit algorithm to compute the representation vectors $x_i$ for each sample $y_i$ by approximating the solution of $$i=1,2,\dots ,N,mi{n}_{x_i}\left\{{\Vert y_i-D{x}_i\Vert }_2^2\right\}\mathrm{subject}\mathrm{to}{\Vert x_i\Vert }_0\le T_0.$$ Dictionary update stage: For each column $k=1,2,\dots ,K$ in $D^{J-1}$, - Define the group of samples that use this atom, $w_k=\{i|1\le i\le N,x_T^k\left(i\right)\ne 0\}$ - Compute the overall representation error matrix, $E_k$, by $$E_k=Y-\sum _{j\ne k}{d}_j{x}_T^j$$ - Restrict $E_k$ by choosing only the columns corresponding to $w_k$, and obtain $E_k^R$. - Apply SVD decomposition $E_k^R=U\Delta V^T$. Choose the updated dictionary column $\widetilde{d_k}$ to be the first column of U. Update the coefficient vector $x_R^k$ to be the first column of V multiplied by $\Delta (1,1)$. Set $J=J+1$.