CPSC 440/540: Advanced Machine Learning – 2022W2 (Jan-Apr 2023)

Schedule

• Calculus
• Linear Algebra
• Probability
• Norms
• Max and Argmax
• Gradient and Hessian of Quadratics
• Gaussians with Conjugate Priors
• Expectation Maximization
• Forward-Backward for HMMs

Italicized entries are tentative; in particular, the timing and even number of assignments might change. Textbook acronyms are explained below.

Overview

This course is intended as a second or third university-level course on machine learning, a field that focuses on using automated data analysis for tasks like pattern recognition and prediction. The class is intended as a continuation of CPSC 340 (or 532M), and will assume a strong background in math and computer science. Topics will (roughly) include deep learning, generative models, latent-variable models, Markov models, probabilistic graphical models, and Bayesian methods.

Logistics

The course meets in person in Swing 122. I plan to release recordings, but can't guarantee their quality, so please come to class.

Grading scheme:

• 50% assignments. A1 must be done alone; A2-4 can be done alone or with a partner.
• 50% chosen among a subset of:
  • Final: a traditional 2.5 hour exam, handwritten, in person.
  • Project: a course project, relatively limited scope, done in a group of 2-3 (details later).
  • Example lecture+assignment: develop a sample lecture and assignment, in a group of 2-3 (details later).
  440 students must do at least one of these three options; 50% of their grade is the maximum of these three options.
  540 students must do at least two of these three options; 25% of their grade is the maximum of these three, and 25% is the second-highest.

Further details in the syllabus slides.

Registration and Prerequisites

• The majority of the seats in 440 are reserved for UBC computer science majors. For other students, to enroll in the course you need to sign up for the wait list.
• The seats in 540 are reserved for graduate students in CPSC, EECE, or STAT. If these seats are full or you are from a different department, to enroll in the course you need to sign up for the wait list. Please read about the differences between 340/532M and 440/540 below, and do not take this course "because 340/532M is full" or not offered in a given term.

Starting in the second week of classes, we'll have weekly tutorials run by the TAs. These will do things like go through provided assignment code, review background material, review big concepts, and/or do exercises. You can register for particular tutorial sections if you want to save a seat at a particular time, but note that you do not need to register in a tutorial section.

CPSC 340/532M vs. CPSC 440/540: CPSC 340 and CPSC 440 are roughly structured as one full-year course. CPSC 340 (which is occasionally listed as CPSC 532M for graduate students) covers more data mining methods and the methods that are most widely-used in applications of machine learning while CPSC 440 (listed as CPSC 540 for graduate students) focuses on probabilistic methods which appear in more niche applications. It is strongly recommended that you take CPSC 340 first, as it covers the most fundamental ideas as well as the most common and practically-useful techniques. In 440 it will be assumed that you are familiar with all the material in the current offering of CPSC 340, and note that online machine learning courses (and courses from many other universities) are not an adequate replacement for CPSC 340 (they typically have more overlap with our applied machine learning course, CPSC 330).

Resources

Textbook: There is no textbook for the course, but the textbook with the most extensive coverage of many of the course's topics is Kevin Murphy's Probabilistic Machine Learning series. The one-volume 2012 version is Machine Learning: A Probabilistic Perspective (MLAPP; you can access a PDF through the UBC library, read a hardcopy version in the CS Reading Room [ICCS 262], or buy a hardcopy). Alternately, he has a very recent two-volume version (2022/2023), PML1 and PML2, both of which have free Creative Commons draft pdfs through those links. I'll try to refer to the relevant sections of both versions as we go, as well as links to various other free online resources.

If you need to refresh your linear algebra or other areas of math, check out Mathematics for Machine Learning (Marc Deisenroth, Aldo Faisal, Cheng Soon Ong; 2020).

Related courses: Besides CPSC340, there are several 500-level graduate courses in CPSC and STAT that are relevant: check out the graduate courses taught by people on the ML@UBC page and the MILD list. CPSC 422/425/436N, EECE 360/592, EOSC 510/550, and STAT 305/306/406/460/461 are also all relevant.

Some related courses that have online notes are:

• Machine Learning (UBC 2013)
• Introduction to Machine Learning (Alberta)
• Machine Learning (MIT)
• Course in Machine Learning (Maryland)
• Apprentissage (ENS, in French)

• Machine Learning (Mathematical Monk)