CPSC 532D: Statistical Learning Theory – Fall 2025 (25W1)

Instructor: Danica Sutherland (she): dsuth@cs.ubc.ca, ICICS X563.
TA: Matt Buchholz (he).
Lecture info: Mondays/Wednesdays, 14:00 - 15:30, Swing 206.
Office hours: TBD, hybrid in ICICS X563 + Zoom unless announced otherwise.
We'll use Piazza and Gradescope; links to come.

Previously offered in 24W1, 23W1, 22W1, and (with the name 532S) 21W2; this instance will be broadly similar.

This is a course on the mathematical foundations of machine learning. When should we expect ML algorithms to work (or not work), and what kinds of assumptions do we need to make to rigorously prove this?

Schedule

Italicized entries are tentative. The lecture notes are self-contained, but the supplements column also refers to the following books (all available as free pdfs) for more details / other perspectives: Lecture notes are available as individual chapters linked below, or as one big frequently-updated file.
DateTopicSupplemental Reading
M Sep  1No class: Labour Day
W Sep  3Course intro, ERMSSBD 1-2; MRT 2; Bach 2
W Sep  3Assignment 1 posted — loss functions, ERM, background
M Sep  8Uniform convergence: finite classesSSBD 2-4; MRT 2; Bach 4.4
W Sep 10Concentration inequalitiesSSBD B; MRT D; Bach 1.2
Zhang 2; Wainwright 2
M Sep 15Covering numbersBach 4.4.4; SSBD 27
Zhang 3.4/4/5
M Sep 15Assignment 1 due at 11:59pm
M Sep 15Drop deadline
M Sep 15Assignment 2 posted — finite classes, concentration, covering numbers
W Sep 17Rademacher complexityMRT 3.1; SSBD 26
Bach 4.5; Zhang 6
M Sep 22More Rademacher
W Sep 24VC dimensionSSBD 6; MRT 3.2-3.3
M Sep 29No free lunchSSBD 5; MRT 3.4
Bach 4.6/12; Zhang 12
W Oct  1PAC learning, the “fundamental theorem”SSBD 5
F Oct  3Assignment 2 due at 11:59pm
F Oct  3Assignment 3 posted — Rademacher, VC, PAC
M Oct  6Online learningSSBD 21; MRT 8
W Oct  8More online learning
M Oct 13No class: Thanksgiving Day
W Oct 15Approximation error:
structural risk minimization, min description length
SSBD 7; MRT 4
M Oct 20More approximation error: universal approximatorsTelgarsky 2; SSBD 20
Bach 9.3; SC 4.6
W Oct 22KernelsBach 7, MRT 6, SSBD 16
F Oct 24Assignment 3 due at 11:59pm
F Oct 24Withdrawal deadline
F Oct 24Assignment 4 posted — online, SRM
M Oct 27More kernels
W Oct 29Margins
M Nov  3More margins
W Nov  5Is ERM enough?; optimization
F Nov  7Assignment 4 due at 11:59pm
SuNov  7Paper-reading assignment: choice of papers posted
M Nov 10No class: midterm break
W Nov 12No class: midterm break
F Nov 14Assignment 5 posted — kernels, margins
M Nov 17More optimization
W Nov 19Neural tangent kernels
M Nov 24Implicit regularization
W Nov 26Stability
M Dec  1PAC-Bayes
? ?Paper-reading assignment: self-scheduled appointment
availability tba, maybe during Nov 20 — Dec 1 or similar
W Dec  3Grab-bag/wrap-up, or possibly canceled for NeurIPS
F Dec  5Assignment 5 due at 11:59pm
? Dec ??Final exam (in person, handwritten); date/time tbd but will be during Dec 9-20 inclusive

Logistics

The course meets in person in Swing 206, with possible rare exceptions (e.g. if I get sick but can still teach, I'll move it online). Note that this room does not have a recording setup, but if you need to miss class for some reason and would prefer some form of a janky recording to just reading the lecture notes, talk to me.

Grading scheme: 70% assignments, 30% final.

Prerequisites

There are no formal prerequisites. TL;DR: if you've done well in CPSC 340/540 or 440/550, didn't struggle with the probability stuff there, and are also pretty comfortable with proofs, you'll be fine. If not, keep reading.

I will roughly assume the following; if you're missing one of them, you can probably get by, but if you're missing multiple, talk to me about it.

If you have any specific questions about your background, please ask.

Resources

If you need to refresh your linear algebra or other areas of math:

In addition to the books above, some other points of view you might like:

Measure-theoretic probability is not required for this course, but there are instances and related areas where it could be helpful:

Some similar courses:

Policies

Academic integrity

The vast majority of you are grad students. The point of grad school classes is to learn things; grades are not the point. For PhD students, they're almost totally irrelevant to your life; for master's students, they barely matter if at all. Cheating, or skirting the boundaries of cheating, does not assist with your learning, but it does potentially set you down a slippery slope towards research miscoduct, which undermines the whole enterprise of science. So, don't cheat; talk to me about what you're struggling with that's making you feel like you need to cheat, and we can figure something out. (Or just take the lower grade.)

You can read more about general UBC policies: Academic Honesty, Academic Misconduct, and Disciplinary Measures. You should also read the Student Declaration and Responsibility to which you've already implicitly agreed.

For the purposes of this class specifically:

Penalties for academic dishonesty can be quite severe (including failing the course or being expelled from the program), and I can say from experience that the process is very unpleasant for everyone involved. Please just don't; talk to me instead, and we can work something out.

Positive Space

This course, including class / Piazza / office hours / etc, is a positive space both in the specific sense linked there and also in that I truly hope everyone can feel safe and supported in the class. If anyone is making you feel uncomfortable in any way, please let me know immediately. If I'm the one causing the issue or you would otherwise rather speak to someone other than me, please immediately talk to your departmental advisor or a CS department head (currently Margo Seltzer and Joanna McGrenere; email head@cs.ubc.ca.)