**CPSC 533V: Learning to Move**
**[Weekly schedule](schedule.html)**
2023W2 (Jan-Apr 2024) - [UBC](http://www.ubc.ca) - [Department of Computer Science](http://cs.ubc.ca)
Course Description
========================================================================
This course is about learning to control the movement of humans, animals, and robots, with
application to character animation, computer vision, robotics, and biological motor control.
The bulk of the course will focus on reinforcement learning (RL), which has seen many advances over the past five years.
_Topics_
: motion control problems, sequential decision problems, RL fundamentals, dynamic programming, tabular methods, deep Q-learning,
policy gradient methods, common policy gradient algorithms (A2C, A3C, TRPO, PPO), common Q-learning algorithms (DDPG, SAC, TD3),
model-based RL and model-predictive control, imitation learning, RL and representation learning, sim-to-real, RL frameworks,
forward/inverse kinematics and dynamics, linear quadratic regulators, advanced topics in RL
People
========================================================================
_Instructor_
: [Michiel van de Panne](http://cs.ubc.ca/~van), , ICCS x865.
_Teaching Assistants_
: to be determined
Lectures
========================================================================
Monday | Wednesday | Thursday
-----------------------------------------|---------------------------------------------|--------------
Lecture FORW 519
1:30-3pm |Lecture FORW 519
1:30-3pm | Office hours
ICCS x865, time TBD
[Weekly schedule](schedule.html)
Sign up to [Piazza](http://piazza.com/ubc.ca/winterterm22023/cpsc533v), which will be used to handle many questions.
Evaluation
========================================================================
Component | Percentage
-------------------------------------|-------------------
Assignments (5) | 45%
[Readings, Presentations, Discussion](discussion.html) | 20%
[Project](project.html) | 35%
Assignments:
- [a1.pdf](a1.pdf) - [a1-soln-revised.pdf](a1-soln-revised.pdf)
- [Assignment 2](https://github.com/UBCMOCCA/CPSC533V_2023W2/blob/main/A2/hw2_Tabular_DQN.ipynb) Due Thu Feb 22
- [Assignment 3](https://github.com/UBCMOCCA/CPSC533V_2023W2/blob/main/A3/hw3_policy_gradients.ipynb) Due Tue Mar 5
Resources
========================================================================
See this [extended list of resources](resources.html).
Highlights:
- ["Reinforcement Learning: An Introduction" (Sutton & Barto, 2018)](http://incompleteideas.net/book/the-book-2nd.html)
- ["An Introduction to Deep Reinforcement Learning"](https://arxiv.org/pdf/1811.12560.pdf)
- [Open AI -- Spinning Up in Deep RL](https://spinningup.openai.com/en/latest/)
Policies
========================================================================
_Illness_
: You are allocated three late days for the course to deal with unforeseen circumstances.
_Special Accomodations_
: Please contact the instructor.
-->