Figure 3.

Optimal control in feedback form, the value function, and the pitfalls of PMP. (a) A phase portrait of the optimal system dynamics. The vector field is shown by grey arrows over the optimal drugs-off (blue background) and drugs-on (yellow background) regions. A sample optimal trajectory (in green and red) corresponds to the initial state from figure 2. (b) Computation of the value function u (whose level curves are shown by light blue lines) is used to determine the optimal drugs-on and drugs-off regions (shown in yellow and dark blue, respectively). Optimal trajectories are not unique for initial states on the shockline (where the level curves of u are not smooth). Two such optimal trajectories are shown starting from an asterisk (*). The green-red trajectory takes longer to reach the recovery, but uses less drugs than the red (start-drugs-right-away) trajectory. The cumulative cost is the same for both of them. (c) For initial conditions off the shocklines of u, there can still be multiple locally optimal trajectories. We show an example of two such trajectories starting from a cross marker (×). The risk of applying the PMP method is that it might yield either of them, but only the red (start-drugs-right-away) is globally optimal. (Online version in colour.)