Table 1.

Properties of various online methods.

Algorithm	ε-optimal	Anytime	Branch & Bound	Monte Carlo	Heuristic	Learning
RTBSS	yes	no	yes	no	no	no
McAllester	high probability	no	no	yes	no	no
Rollout	no	no	no	yes	no	no
Satia and Lave	yes	yes	yes	no	yes	no
Washington	acyclic graph	yes	implicit	no	yes	no
AEMS	yes	yes	implicit	no	yes	no
HSVI-BFS	yes	yes	implicit	no	yes	no
RTDP-Bel	no	no	no	no	no	yes
SOVI	yes	yes	no	no	no	yes