Lectures will be held on Zoom. The link is available on Canvas (for registered students), and has been e-mailed to students on the wait list.

Instructor: Mark Schmidt.

Teaching Assistants: Setareh Cohan, Peyman Gholami, Nam Hee Gordon Kim, Frederik Kunstner, Shahriar Shayesteh, Betty Shea.

Tutorials and office hours calendar: Coming soon.

**Synopsis**: This course is intended as a second or third university-level course on machine learning,
a field that focuses on using automated data analysis for tasks like pattern recognition and prediction.
The class is intended as a continuation of CPSC 340 (or 532M), and will assume a strong background in math
and computer science. Topics will (roughly) include deep learning, generative models, latent-variable models,
Markov models, probabilistic graphical models, and Bayesian methods.

**Registration**: Graduate and undergraduate students from any department are welcome to take the class. Undergraduate students should enroll in CPSC 440
while graduate students should enroll in CPSC 540. Below are more details on registration for each course:

- The majority of the seats in 440 class are reserved for UBC computer science majors. For other students,
**to enroll in the course you need to sign up for the wait list**. - For reasons I do not understand or agree with, only a limited number of seats (20) have been allocated for CPSC 540 this year and these are full. If you do not already have one of these
20 seats, then
**to enroll in the course you need to sign up for the wait list**. Please read about the differences between 340/532M and 440/540 below, and**do not take this course "because 340/532M is full"**or not offered in a given term.

Starting in the second week of classes, we'll have weekly tutorials run by the TAs. These will do things like go through provided assignment code, review background material, review big concepts, and/or do exercises. You can register for particular tutorial sections if you want to save a seat at a particular time, but note that you do not need to register in a tutorial section.

**CPSC 340/532M vs. CPSC 440/540**:
CPSC 340 and CPSC 440 are roughly structured as one full-year course. CPSC 340 (which is also listed as CPSC 532M for graduate students) covers more data mining methods
and the methods that are most widely-used in applications of machine learning
while CPSC 440 (listed as CPSC 540 for graduate students) focuses on structured prediction
and probabilistic methods which appear in more niche applications.
It is strongly recommended that you take CPSC 340 first, as it covers the most fundamental ideas as well as the most common and practically-useful techniques.
In 440 it will be assumed that you are familiar with all the material in the current offering of CPSC 340,
and note that online machine learning courses (and courses from many other universities)
are *not* an adequate replacement for CPSC 340 (they typically have more overlap with our applied machine learning course, CPSC 330).

**Prerequisites**:

- CPSC 320: Intermediate Algorithm Design and Analysis. Note that this course itself has the following prerequisites:
- Basic algorithms and data structures (CPSC 221 or EECE 320).

- CPSC 340: Machine Learning and Data Mining. Note that this course itself has the following additional prerequisites:
- Linear algebra (one of MATH 152, 221, or 223).
- Probability (one of STAT 241, STAT 251, ECON 325, ECON 327, MATH 302, STAT 302, or MATH 318).
- Multivariate calculus (one of MATH 200, 217, 226, 253, or 263).

**Textbook**: There is no textbook for the course, but the textbook with the most extensive coverage of many of the course's topics is Kevin Murphy's Machine Learning: A Probabilistic Perspective (MLAPP). This book can be purchased from Amazon,
is on reserve in the CS Reading Room (ICCS 262), and can be accessed through the library here. Optional readings will be given out of this textbook, in addition to other free online resources.

**Grading**:
Assignments 40%, Final Exam 30%, Project 30%

Date | Lecture Slides | Related Readings and Links | Homework |
---|---|---|---|

Mon Jan 11 | Syllabus | MLAPP 1.1-1.2, 1.4, 6.5, 7.1-3, 7.5, 8.1-3 ML vs. Stats (2001, 2015) 3 Cultures of ML Essence of Linear Algebra Mathematics for Machine Learning |
Assignment 1 (a1.zip, a1.tex) |

Wed Jan 13 | 340 Overview Fundamentals of Learning |
MLAPP 1.4, 6.5, Probability Primer | |

Fri Jan 15 | More Fundamentals of Learning Convexity |
||

Mon Jan 18 | More Convexity | BV 2.1-2.3, 3.1-3.2 | |

Wed Jan 20 | How Much Data? Structured Prediction Motivation |
The Tradeoffs of Large Scale Learning | |

Fri Jan 22 | Density Estimation | MLAPP 2.3, Covariance Matrix | Assignment 1 due |

Mon Jan 25 | Multivariate Gaussians | MLAPP 2.4-5 and 4.1-3, Properties of Gaussians | Assignment 2 (a2.zip, a2.tex) |

Mon Jan 25 | More Gaussians | ||

Fri Jan 29 | Mixture Models | MLAPP 11.1-2 | |

Mon Feb 1 | Generative Classifers | MLAPP 3.5, 4.2, 8.6 | |

Wed Feb 3 | Expectation Maximization | MLAPP 11.3-4, 11.6 | |

Fri Feb 5 | More EM | MLAPP 11.3-4, 11.6 | |

Mon Feb 8 | Kernel Density Estimation | MLAPP 12.1-2, 14.7 | |

Wed Feb 10 | Markov Chains | MLAPP 17.1-2 | |

Fri Feb 12 | Monte Carlo Methods | MLAPP 23.1-2 | Assignment 2 due |

Mon Feb 22 | Message Passing | MLAPP 17.4 | Assignment 3 (a3.zip, a3.tex) |

Wed Feb 24 | Hidden Markov Models | MLAPP 17.3-5, 18.1-4 | |

Fri Feb 26 | DAG Models | MLAPP 10.1-2, 10.5 | |

Mon Mar 1 | More DAGs | MLAPP 10.3-4, 26.1-4 | |

Wed Mar 3 | More DAGs 2 | ||

Fri Mar 5 | More DAGs 3 | ||

Mon Mar 8 | Undirected Graphical Models | MLAPP 19.1-4, 20.1-4 | |

Wed Mar 10 | Approximate Inference | MLAPP 24.1-2 | |

Fri Mar 12 | More Approximate Inference | Assignment 3 due | |

Mon Mar 15 | Log-Linear Models | MLAPP 19.5 | Assignment 4 (a4.zip, a4.tex) |

Wed Mar 17 | Boltzmann Machines | MLAPP 27.7, 28.1-2, 19.6 | |

Fri Mar 19 | Bayesian Statistics | MLAPP 3.1-4, 4.4-6, 5.1-4, 7.6 | |

Mon Mar 22 | Empirical Bayes | MLAPP 5.6-7 | |

Wed Mar 24 | Conjugate Priors | ||

Fir Mar 26 | Topic Models | MLAPP 5.5, MLAPP 27.1, 27.3 | |

Mon Mar 29 | Rejection/Importance Sampling | MLAPP 23.3-4 | |

Wed Mar 31 | Metropolis-Hastings | MLAPP 24.3-4 | |

Wed Apr 7 | Neural Networks Implicit Regularization |
||

Fri Apr 9 | Double Descent Curves Deep Structured Models |
Assignment 4 due | |

Mon Apr 12 | Fully-Convolutional Networks | ||

Wed Apr 14 | Recurrent Neural Networks |

**Notes**: These are additional notes mentioned in the lectures and homeworks:

- Calculus
- Linear Algebra
- Probability
- Norms
- Max and Argmax
- Julia Commands
- Gradient and Hessian of Quadratics
- Expectation Maximization
- Forward-Backward for HMMs

**Related Courses**: Besides CPSC 340, other closely-related courses available at UBC include 500-level classes taught by Frank Wood, Leonid Sigal, and Helge Rhodin. Related courses from other departments include
EECE 360/592, EOSC 510/550, STAT 305/306/406/460/461, and most 500-level STAT courses. There is some discussion of how 340/440 relate to some of the undergraduate STAT classes written by a former student (Geoff Roeder) here.

Some related courses that have online notes are:

- Machine Learning (UBC 2013)
- Introduction to Machine Learning (Alberta)
- Machine Learning (MIT)
- Course in Machine Learning (Maryland)
- Apprentissage (ENS, in French)

- Machine Learning (Mathematical Monk)

Mark Schmidt > Courses > CPSC 440