Algorithm 1: The proposed improved soft actor-critic algorithm.

Initialization: Randomly initialize the parameters of the policy network and the two

Q networks. Set the experience reply D with size of 100,000.

Input: The current communication parameters of the communication parties and the

current jamming action of the jammer

1: for episode i = 1, 2, …, J do

2:     for step j = 1, 2, …, N do

3:          According to the state, $s_t$, input to the policy network sampling output

action, $a_t$;

4:          The proto-action, $a_t$, is input to the improved Wolpertinger architecture to

obtain the actual executed action, $a_t$;

5:          Executing action $a_t$;

6:          Obtaining the next state, $s_{t+1}$, and feedback and calculating the actual

reward, $r$;

7:          Storing ($s_t$,$a_t$,$s_{t+1}$,$r$) into experience pool D;

8:          Sampling the smallest batch, $N_B$, from experience pool D for training;

9:          Updating network parameters A and B for Q1 and Q2;

10:        Updating the parameters of the policy network;

11:        Updating the parameters of the target Q1 and target Q2 networks;

12:        Setting $s_t=s_{t+1}$;

13:    end for

14: end for

Output: Jamming action for the communication parties at the next moment