Algorithm 2.

Two-parameter backtracking line search

Input: ${\alpha }_0^p$, ${\alpha }_0^c$, F(k), ${\mathbf{G}}_p^{(k)}$, ${\mathbf{G}}_c^{(k)}$, lmax

Output: $\widehat{{\alpha }_k^p}$, $\widehat{{\alpha }_k^c}$

1: $\widehat{{\alpha }_k^p}\leftarrow {\alpha }_0^p$

2: found ← false

3: while not found do

4:  $\widetilde{{\mathbf{p}}_0}\leftarrow {\mathrm{prox}}_{\widehat{{\alpha }_k^p}\beta R}\left({\mathbf{p}}_0^{(k)}-\widehat{{\alpha }_k^p}{\mathbf{G}}_p^{(k)}\right)$

5:  if $F\left(\widetilde{{\mathbf{p}}_0},\mathrm{\Phi }\left({\mathbf{c}}_p^{(k)}\right)\right)>F^{(k)}+\langle {\mathbf{G}}_p^{(k)},\widetilde{{\mathbf{p}}_0}-{\mathbf{p}}_0^{(k)}\rangle +\frac{1}{2\widehat{{\alpha }_k^p}}{\Vert \widetilde{{\mathbf{p}}_0}-{\mathbf{p}}_0^{(k)}\Vert }_2^2$ then

6:   $\widehat{{\alpha }_k^p}\leftarrow \widehat{{\alpha }_k^p}/2$

7:  else

8:   found ← true

9:  end if

10: end while

11: l ← 0 {l is the line search step number.}

12: $\widehat{{\alpha }_k^C}\leftarrow {\alpha }_0^c$

13: found ← false

14: while not found and l < lmax do

15:  ${\tilde{\mathbf{c}}}_p\leftarrow {\mathbf{c}}_p^{(k)}-\widehat{{\alpha }_k^c}\mathbf{B}{\mathbf{G}}_c^{(k)}$

16:  if $F\left({\tilde{\mathbf{p}}}_0,\mathrm{\Phi }\left({\mathbf{c}}_p^{(k)}\right)\right)<F\left({\tilde{\mathbf{p}}}_0,\mathrm{\Phi }\left({\tilde{\mathbf{c}}}_p\right)\right)$ then

17:   $\widehat{{\alpha }_k^c}\leftarrow \widehat{{\alpha }_k^c}/2$

18:  else

19:   found ← true

20:  end if

21:  l ← l + 1

22: end while

23: if l is lmax then

24:  $\widehat{{\alpha }_k^c}\leftarrow 0$

25: end if