Seeing the Forest Despite the Trees: Large Scale Spatial-Temporal Decision Making
By Mark Crowley
We introduce a challenging real-world planning problem where actions must be taken at each location in a spatial area at each point in time. We use forestry planning as the motivating application. In Large Scale Spatial-Temporal (LSST) planning problems, the state and action spaces are defined as the cross-products of many local state and action spaces spread over a large spatial area such as a city or forest. These problems possess state uncertainty, have complex utility functions involving spatial constraints and we generally must rely on simulations rather than a explicit transition model. We present a policy gradient algorithm with an abstract policy formulation that exploits the structure of LSST problems to make useful abstractions. We compare this policy formulation with a more elementary policy that does not use LSST structure. We show that the abstract policy is more robust and achieves higher rewards with far fewer parameters than the elementary policy. This abstract policy is a better fit to the properties that practitioners in LSST problem domains require for such methods to be widely useful.