Figure 4.

Structure of the actor network. The input layer receives a 44-dimensional state vector. The state vector passes through three fully connected layers: initially through 400 neurons, followed by two consecutive layers of 300 neurons each. The final output layer produces a 10-dimensional action vector. The fully connected layers transform the input state vector through these stages, ultimately producing the action vector as the output.