Algorithm 2.

Regularized dual averaging (RDA) method

Input: c0, λ
Output: $\widehat{c}$
1:	k ← 0 {k is the algorithm iteration number.}
2:	A−1 ← 0
3:	while stopping criterion is not satisfied do
4:	Draw wk according to chosen distribution.
5:	Calculate Gk ← ∇c f (ck, wk)
6:	Choose weight αk > 0 {Unweighted case: αk = 1}
7:	Ak ← Ak−1+αk
8:	${\overline{G}}_k\leftarrow \left(1-\frac{{\alpha }_k}{A_k}\right){\overline{G}}_{k-1}+\frac{{\alpha }_k}{A_k}{G}_k$ {Compute weighted average of gradient.}
9:	Choose μk {For example, μk = γAk, where γ > 0 is a constant.}
10:	$c_{k+1}\leftarrow c_0-{\mu }_k{\overline{G}}_k$
11:	$c_{k+1}\leftarrow {\text{prox}}_{\lambda {\mu }_kℛ}\left(c_{k+1}\right)$
12:	k ← k+1
13:	end while
14:	$\widehat{c}\leftarrow c_k$