Lecture Schedule

Date Topic
Mar 12 -- Apr 7 Papers presentations Piazza post
Tue Mar 10 No class (Ottawa trip)
Feb 11 - Mar 5 Policy Gradient methods PolicyGradient.pdf
ModelBasedRL ModelBasedRL.pdf
Imitation Learning ImitationLearning.pfd
Sim-to-real sim2real.pdf
Hindsight Experience Replay (HER) and Advantage Actor Critic (AAC) HER-and-AAC.pdf
Thu Feb 6 RL slides up until now: RL-slides-Feb6-2020.pdf
Deep Q-learning (DQN); Deep Deterministic Policy Gradients (DDPG)
Assignment 3 (Tabular Q learning) is out. https://github.com/UBCMOCCA/CPSC533V_a3 (view and edit using: python notebook hw3.ipynb)
A good Jupyter notebook tutorial
Due next Friday, Fri Feb 14, by email to Michiel (van@cs)
Tue Feb 4 HJB equations     HJB methods thesis     level sets toolbox (Ian Mitchell)
Thu Jan 30 Basic RL: RL-2.pdf
returns, discount factor, policy cloning, episodic tasks, simple deterministic graph MDP, generalized policy iteration, state value function, bootstrapping, prioritized sweeping, experience tuples, stochastic dynamics function, Bellman backups, Bellman backup diagram, model-based vs model-free, policy objective functions
A2 out (due Fri Feb 7)
car drifting control: video     paper
2D manipulation using RL: project web page
Tue Jan 28 More dynamics: constrained rigid body dynamics: general form in maximal coords,
constraint stabilization, reduced coordinates, modeling collision impulses
physics-two-body-complete.pdf     physics-collisions.pdf    
Thu Jan 23 Two rigid bodies with a constraint
Tue Jan 21 Rigid body dynamics (continued)     physics-rigid-body-complete.pdf    
Thu Jan 16 Rigid Body Dynamics    
Tue Jan 14 Introduction to Control    
control-blank.pdf     a1.pdf
Thu Jan 9 Example of movement skills (video links to be posted);    
Tue Jan 7 Course overview; introductions; background survey.    
(various) Representations of movement
   motion-notations.pdf    prehistoric animated paintings (youtube)    prehistoric animated paintings: paper (PDF)
   animated point light horse display    visualizing human motion; motion sculptures    Pixar Zoetrope:
Animal movement
   leaping goat    chicken head tracking:    falcon    cat stepping
   snake climbing tree    goats on dam    Scaling of sensorimotor control in terrestrial mammals
Human movement
   baseball catching strategies    optimizing energetic cost during walking    unpowered exoskeleton
Passive dynamics
   passive dynamic walking    passive juggling    bicycle stability    split belt energy harvesting
   wheel-with-mass    diesel governor    steam governor
   Komura lab: high-speed robotics    fast running robot    triple pendulum    baking robot: 2'31 in
   parkour atlas
RL for movement
   NeurIPS 2019 learn to walk