FIGURE 6.

Case1: Performance of the RL based controller without external perturbation during multiple squatting cycles. The first figure demonstrates the foot CoP reward (green and red lines), end-effector tracking reward (magenta line) and joint position tracking reward (black line) calculated according to Eqs 10–15. The bottom figure depicts the actions for the hip, knee ankle joint predicted from the neural network.