Robust Task-based Control Policies for Physics-based Characters

ACM Transaction on Graphics (Proceedings of SIGGRAPH ASIA 2009)

Stelian Coros    Philippe Beaudoin     Michiel van de Panne

University of British Columbia


We precompute task-specific control policies for real-time physics-based characters. The character moves efficiently towards the current goal, responds interactively to changes of the goal, and can respond to significant physical interaction with the environment.


We present a method for precomputing robust task-based control policies for physically simulated characters. This allows for characters that can demonstrate skill and purpose in completing a given task, such as walking to a target location, while physically interacting with the environment in significant ways. As input, the method assumes an abstract action vocabulary consisting of balance-aware, step-based controllers. A novel constrained state exploration phase is first used to define a character dynamics model as well as a finite volume of character states over which the control policy will be defined. An optimized control policy is then computed using reinforcement learning. The final policy spans the cross-product of the character state and task state, and is more robust than the conrollers it is constructed from. We demonstrate real-time results for six locomotion-based tasks and on three highly-varied bipedal characters. We further provide a game-scenario demonstration.


    PDF (1.23 Mb)


    High-Quality QuickTime Video (128M)





Smoothly walking around lamp posts.  
Physics-based interaction with the environment.  
Graphical user interface for controller authoring.  
    author = {Stelian Coros and Philippe Beaudoin and Michiel van de Panne},
    title = {Robust Task-based Control Policies for Physics-based Characters},
    journal = {ACM Trans. Graph. (Proc. SIGGRAPH Asia)},
    year = {2009},
    volume = {28},
    number = {5},
    pages = {Article 170}
   Code for SIMBICON controller editor and simulation
   NSERC (Natural Sciences and Engineering Research Council of Canada)
   FQRNT (Fonds québécois de recherche sur la nature et les technologies)