Algorithm 1 Construction of synthetic network g in Papernot’s oracle based black-box attack [24]

1:Input: 2:        $\mathcal{O}$ represents black-box access to $F\left(f\right(·\left)\right)$ for target model f with output function F; 3:        ${\mathcal{X}}_0\subseteq \mathcal{X}$, where $\mathcal{X}$ is the training data set of target model f; 4:        substitute architecture S 5:        training method $\mathrm{M}$; 6:        constant $\lambda$; 7:        number N of synthetic training epochs 8:Output: 9:        synthetic model s defined by parameters ${\theta }_s$ 10:        (s also has output function F which selects the max confidence score; 11:        s fits architecture S) 12:  13:Algorithm: 14:for N iterations do 15:    $\mathcal{D}\leftarrow \{(x,\mathcal{O}\left(x\right)):x\in {\mathcal{X}}_t\}$ 16:    ${\theta }_s=\mathrm{M}(S,\mathcal{D})$ 17:    ${\mathcal{X}}_{t+1}\leftarrow \{x+\lambda ·\mathrm{sgn}\left(J_{{\theta }_s}\left(x\right)\left[\mathcal{O}\left(x\right)\right]\right):x\in {\mathcal{X}}_t\}\cup {\mathcal{X}}_t$ 18:end for 19:Output ${\theta }_s$