Algorithm 1 Proposed deep Q-learning algorithm

1:Initialize policy, target DQN with random w, $w^{\prime }$ 2:Initialize $ϵ$ 3:for episode do 4:    for instance do 5:        Select a channel matrix and add it to action space $A_t$ for present state space $S_t$ 6:        Observe immediate reward $R_t$, next state space $S_{t+1}$ 7:        Put $(S_t,A_t,R_t,S_{t+1})$→ ERM 8:        Form random sample mini batch of $(S_t,A_t,R_t,S_{t+1})$ from ERM 9:        for each tuple in mini batch do 10:           Calculate Q-values 11:           Approximate $Q^{*}$-values using target DNN 12:           Compute loss from Q and $Q^{*}$ 13:           Optimize w of policy DNN with Adam optimizer 14:    $w^{\prime }\leftarrow w$ after all time steps Ensure:  ${\mathbf{R}}_r\approx {\mathbf{R}}_{\max }$