Table 5. Parameter configuration of the actor-critic neural networks and agent.

Parameter	Actor network	Critic network
$X_i$ Input layer size	19	19
Hidden layer $L_1$	150	150
Activation function $L_1$	Tanh	Tanh
Hidden layer $L_2$	150	150
Activation function $L_2$	Tanh	Tanh
Hidden layer $L_3$	100	100
Activation function $L_3$	ReLu	ReLu
$Y_i$ Output layer	27	1
AC agent parameters
Number of steps to look ahead	70	70
Learning rate	0.001	0.001
Entropy loss weight	0.25	0.25
Gradient threshold	1	1
Discount factor	0.91	0.91
Max number of episodes	5,000	5,000
Max steps per episode	4,000	4,000