**CPSC 533V: Learning to Move**
2021W1 (Sept-Dec 2021) - [UBC](http://www.ubc.ca) - [Department of Computer Science](http://cs.ubc.ca)
This course is about learning to control the movement of humans, animals, and robots, with
application to character animation, computer vision, robotics, and biological motor control.
The bulk of the course will focus on reinforcement learning (RL), which has seen many advances over the past five years.
: motion control problems, sequential decision problems, RL fundamentals, dynamic programming, tabular methods, deep Q-learning,
policy gradient methods, common policy gradient algorithms (A2C, A3C, TRPO, PPO), common Q-learning algorithms (DDPG, SAC, TD3),
model-based RL and model-predictive control, imitation learning, RL and representation learning, sim-to-real, RL frameworks,
forward/inverse kinematics and dynamics, linear quadratic regulators, advanced topics in RL
: [Michiel van de Panne](http://cs.ubc.ca/~van),
3-4:20pm |Lecture ICCS 246
3-4:20pm | Office hours ICCS x865
4-5pm [Weekly schedule](schedule.html) Accomodations for remote attendance are being made until at least October 8, 2021. In this context, the in-person __lectures and office hours will be live-streamed__ (from the classroom / office). A link will be distributed by email to registered students. Recordings may be available (TBD). Piazza will be used to handle most questions. [Sign up here](https://piazza.com/ubc.ca/winterterm12021/cpsc533v) Evaluation ======================================================================== Component | Percentage -------------------------------------|------------------- Assignments (5) | 45% Readings, Presentations, Discussion | 20% Project | 35% - Assignment 1: Warmup math [a1.html](a1.html) - [a1.pdf](a1.pdf) - Assignment 2: Tabular Q-learning [Python notebook (GitHub)](https://github.com/UBCMOCCA/CPSC533V_2021W1/blob/master/a2/a2.ipynb) - Assignment 3: DQN [Python notebook (GitHub)](https://github.com/UBCMOCCA/CPSC533V_2021W1/blob/master/a3/hw3.ipynb), due Fri Oct 15 - Assignment 4: Policy gradients and PPO [Python notebook (GitHub)](https://github.com/UBCMOCCA/CPSC533V_2021W1/tree/master/a4), due Fri Oct 29 Resources ======================================================================== See this [extended list of resources](resources.html). Highlights: - ["Reinforcement Learning: An Introduction" (Sutton & Barto, 2018)](http://incompleteideas.net/book/the-book-2nd.html) - ["An Introduction to Deep Reinforcement Learning"](https://arxiv.org/pdf/1811.12560.pdf) - [Open AI -- Spinning Up in Deep RL](https://spinningup.openai.com/en/latest/) Policies ======================================================================== _Non-medical masks_ : In keeping with [BC's mandate](https://www2.gov.bc.ca/gov/content/covid-19/info/restrictions#masks), masks are required for all in-person course activities and for all indoor public areas at UBC. Except where needed for your health, please refrain from eating and drinking in class. There are a few exemptions to the mask mandate; if you believe you are exempt, please contact the Centre for Accessibility (firstname.lastname@example.org).) _Vaccination_ : Vaccines are available to you, free, and [on campus](http://www.vch.ca/covid-19/covid-19-vaccine). The higher the rate of vaccination in our community overall, the lower the chance of spreading this virus. You are an important part of the UBC community. Please arrange to get vaccinated if you have not already done so. UBC will require those who have not been vaccinated to undergo regular testing. _Illness_ : You are allocated three late days for the course to deal with unforeseen circumstances. _Special Accomodations_ : Please contact the instructor. -->