Figure 3.

Training results. (a, b) ⟨c_tot⟩ and ⟨b_tot⟩ computed using π_∗ are plotted versus policy iterations, respectively (solid blue squares). Solid horizontal lines show these quantities for the heuristic policy π_sd (red triangles) and the random policy (black circles). The larger cyan square shows ⟨c_tot⟩ after each tree had been searched for the best (lowest) target cost. Dashed vertical lines show points when ε was lowered.