Table 2.
Runtime of the experiments.
| Network architecture | Cartpole | Ant | Ant-v4 | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Training (s) | Evaluation (s) | Training (s) | Evaluation (s) | Training (s) | Evaluation (s) | |||||||
| Mean | Std | Mean | Std | Mean | Std | Mean | Std | Mean | Std | Mean | Std | |
| [2] * 1 | 31.35 | 7.96 | 35.85 | 5.53 | 364.05 | 9.51 | 184.00 | 47.79 | – | – | – | – |
| [2] * 2 | 44.65 | 3.37 | 27.95 | 3.65 | 299.15 | 25.10 | 166.75 | 11.26 | – | – | – | – |
| [2] * 4 | 45.30 | 6.20 | 40.80 | 4.37 | 304.20 | 59.53 | 424.50 | 29.23 | – | – | – | – |
| [4] * 1 | 40.25 | 4.84 | 35.70 | 3.16 | 358.40 | 8.48 | 160.45 | 11.80 | – | – | – | – |
| [4] * 2 | 44.05 | 4.77 | 29.05 | 4.83 | 295.50 | 6.14 | 204.00 | 88.15 | – | – | – | – |
| [4] * 4 | 44.00 | 4.28 | 40.20 | 4.70 | 292.68 | 6.31 | 414.40 | 18.64 | – | – | – | – |
| [8] * 1 | 40.85 | 4.64 | 30.70 | 3.30 | 373.30 | 22.58 | 159.85 | 14.06 | – | – | – | – |
| [8] * 2 | 44.50 | 3.25 | 32.83 | 5.95 | 299.20 | 5.95 | 411.80 | 16.19 | – | – | – | – |
| [8] * 4 | 44.55 | 4.25 | 40.70 | 3.39 | 292.85 | 7.73 | 418.00 | 19.69 | – | – | – | – |
| [16] * 1 | 44.75 | 2.38 | 26.25 | 3.94 | 357.10 | 8.99 | 158.55 | 17.12 | – | – | – | – |
| [16] * 2 | 46.85 | 2.83 | 34.90 | 4.88 | 306.40 | 4.59 | 404.00 | 29.78 | – | – | – | – |
| [16] * 4 | 46.85 | 3.48 | 41.95 | 4.84 | 299.65 | 8.17 | 427.40 | 15.45 | – | – | – | – |
| [32] * 1 | 40.20 | 6.45 | 30.15 | 3.97 | 348.40 | 30.32 | 164.95 | 14.67 | – | – | – | – |
| [32] * 2 | 44.25 | 3.83 | 32.90 | 4.21 | 305.35 | 6.55 | 407.20 | 22.13 | – | – | – | – |
| [32] * 4 | 51.35 | 3.24 | 44.00 | 4.38 | 313.00 | 8.99 | 422.65 | 21.98 | – | – | – | – |
| [64] * 1 | 36.95 | 6.42 | 32.90 | 2.70 | 343.40 | 15.29 | 168.85 | 13.92 | – | – | – | – |
| [64] * 2 | 44.60 | 2.48 | 32.70 | 5.15 | 312.75 | 15.14 | 406.30 | 38.02 | – | – | – | – |
| [64] * 4 | 49.10 | 4.48 | 41.25 | 6.20 | 364.90 | 11.81 | 437.80 | 17.88 | – | – | – | – |
| [128] * 1 | 37.45 | 5.44 | 31.70 | 2.41 | 365.95 | 12.33 | 159.35 | 16.61 | – | – | – | – |
| [128] * 2 | 44.75 | 2.91 | 31.70 | 4.62 | 344.75 | 23.31 | 401.45 | 42.73 | – | – | – | – |
| [128] * 4 | 47.05 | 4.17 | 35.45 | 7.35 | 450.30 | 21.41 | 425.00 | 33.45 | – | – | – | – |
| [256] * 1 | 43.50 | 6.03 | 29.90 | 4.23 | 421.50 | 4.43 | 169.50 | 4.83 | – | – | – | – |
| [256] * 2 | 44.95 | 4.81 | 32.45 | 6.86 | 445.15 | 22.56 | 416.75 | 35.72 | – | – | – | – |
| [256] * 4 | 60.35 | 4.35 | 37.70 | 5.36 | 642.05 | 44.33 | 472.80 | 17.75 | – | – | – | – |
| [512] * 1 | 47.85 | 3.41 | 28.75 | 4.13 | 448.60 | 6.26 | 171.15 | 2.92 | 12 104.90 | 1509.36 | 11 054.05 | 458.07 |
| [512] * 2 | 53.70 | 4.23 | 31.55 | 6.30 | 643.20 | 50.28 | 420.95 | 42.93 | 11 831.70 | 465.60 | 11 621.60 | 776.31 |
| [512] * 4 | 83.00 | 5.86 | 34.10 | 5.53 | 1 072.55 | 99.10 | 515.80 | 69.64 | – | – | – | – |
| [1024] * 1 | 53.15 | 4.14 | 29.35 | 4.09 | 609.25 | 16.85 | 173.55 | 6.79 | – | – | – | – |
| [1024] * 2 | 75.15 | 4.74 | 30.30 | 3.95 | 1 133.60 | 113.40 | 491.00 | 61.05 | – | – | – | – |
| [2048] * 1 | 59.90 | 4.96 | 27.20 | 4.41 | 960.50 | 36.94 | 196.60 | 7.59 | – | – | – | – |
| Total | 47.84 | 11.27 | 33.70 | 6.68 | 459.96 | 238.18 | 313.56 | 135.10 | 11 970.25 | 1160.10 | 11,150.20 | 555.03 |