Office hours: Tuesdays 3:00-4 (ICCS 193) or by appointment.
Instructor: Mark Schmidt
Teaching Assistant: Mohamed Ahmed
Tutorials: Thursdays 3:00-4:30 (Frank Forward 519), Fridays 3:30-4:30 (ICCS 146)
Synopsis: This is a graduate-level course on machine learning, a field that focuses on using automated data analysis for tasks like pattern recognition and prediction. Roughly two thirds of the course will focus on the traditional core machine learning material and the later third of the course will focus on more trendy topics. Topics will (roughly) include regression, classification, model selection, regularization, kernels and Gaussian processes, convex and stochastic optimization, bootstrapping/boosting and random forests, mixture and latent variable models, missing data, Bayesian inference, graphical models, and deep learning.
Textbook: Suggested readings and some assignments will use Machine Learning: A Probabilistic Perspective (MLAPP). This book can be purchased from Amazon, is on reserve in the CS Reading Room (ICCS 262), and can be accessed through the library here.
Prerequisites: Linear algebra, multivariate calculus, probability, algorithms, programming experience (we will use Matlab), and undergraduate machine learning or statistics.
Registration: Graduate students from certain programs can directly register in the course, but the course is open to undergraduate and graduate students from other departments that satisfy the prerequisites. If you are not a computer science graduate student, registration information is available here.
Grading: Midterm 30%, Homeworks 30%, Coding Project 10%, Course Projects 30%.
Piazza for course-related questions
| Date | Topic | Scott's Notes | Reading | Homework |
|---|---|---|---|---|
| Tue Sep 2 |
Course Pitch | |||
| Wed Sep 3 |
Admin, Overview, Supervised Basics, K-Nearest Neighbours | Lec 1, Tut 1 | MLAPP Chapter 1 | Assignment 1 (due Sep 10), ass1.zip |
| Mon Sep 8 |
Probability, Naive Bayes, Maximum Likelihood | Lec 2 | MLAPP Section 2.2, Section 3.5 | Notes on Naive Bayes |
| Wed Sep 10 |
Gaussian Discriminant Analysis, Linear Regression | Lec 3 | MLAPP Sections 4.1-4.2, 7.1-7.3 | Assignment 1 due, Notes on Gaussian MLE |
| Mon Sep 15 |
Robust Regression, Change of Basis, Cross-Validation | Lec 4 | MLAPP Sections 7.3-7.4 | Assignment 2 (due Sep 24), ass2.zip, Assignment 1 solution posted, Notes on Least Squares Methods |
| Wed Sep 17 |
Ridge Regression, MAP, Kernels | Lec 5, Tut 2 | MLAPP Sections 7.5, 13.1, 14.1-14.2 | |
| Mon Sep 22 |
Kernels, Feature Selection, L1-Regularization | Lec 6 | MLAPP 13.3, 14.1-14.2 | Marked Assignment 1 due, Assignment 3 (due Oct 1), ass3.zip |
| Wed Sep 24 |
Logistic Regression | Lec 7, Tut 3 | MLAPP Section 8.1-8.3 | Assignment 2 due, Weighted Least Squares |
| Mon Sep 29 |
Support Vector Machines, Convex Functions | Lec 8 | MLAPP Section 14.5, BV Sections 2.1-2.3, 3.1-3.2, 3.5 | |
| Wed Oct 1 |
Convex and Stochastic Optimization | Lec 9 | MLAPP 8.3, 8.5, BV Sections 9.1-9.3 | Assignment 3 due, Assignment 4 (due Oct 15), ass4.zip, Preliminary Course Evaluation, Notes on Norms |
| Mon Oct 6 |
Neural Networks (Michael Gelbart) | Lec 10 | MLAPP Section 16.5 | Hand in preliminary course evaluation |
| Wed Oct 8 |
Deep Learning (Michael Gelbart) | Lec 11 | ||
| Mon Oct 13 |
No class (holiday) | |||
| Wed Oct 15 |
Convergence Rates | Lec 12 | BV Sections 9.1-9.3 | Assignment 4 due, Marked Assignment 2 due |
| Mon Oct 20 |
Proximal-Gradient, Fenchel Duality | Lec 13 | MLAPP Section 14.3, BV Sections 3.3, 5.1-5.2 | Marked Assignment 3 due, Assignment 5 (due October 29), ass5.zip |
| Tue Oct 21 |
Stochastic Average Gradient | |||
| Wed Oct 22 |
Ensemble Methods | Lec 14, Tut 5 | MLAPP Sections 2.8, 6.2, 16.2-4, 16.6 | |
| Mon Oct 27 |
Hidden Variables, Expectation Maximization | Lec 15 | MLAPP Sections 8.6, 11.2-11.6 | Marked Assignment 4 due, Assignment 6 (due November 12), ass6.zip |
| Wed Oct 29 |
Bayesian Learning | Lec 16 Tut 6 | MLAPP Sections 3.2-3.3, 5.2-5.4, 5.7 | Assignment 5 due |
| Mon Nov 3 |
Conjugate Priors, Type II Maximum Likelihood, Gaussian Processes | Lec 17 | MLAPP Sections 3.4, 4.6, 5.5-5.6, 7.6, 9.2, 13.7, 15.1-2 | Project Proposal due |
| Wed Nov 5 |
Directed Acyclic Graphical Models | Lec 18 | MLAPP Sections 10.1-5, Sections 17.1-3, 17.6, 18.1 | |
| Mon Nov 10 |
Special Guest Lecture: Dimitri Bertsekas (3-4pm) | |||
| Wed Nov 12 |
Undirected Graphical Models | Lec 19 Tut 7 | MLAPP Sections 4.3-4.4, 17.3, 19.1-4, 20.1-2, 27.7 | Marked Assignment 5 due, Assignment 6 due, Coding Project (due December 17) |
| Mon Nov 17 |
Midterm | Assignment 7 (due November 26), ass7.zip | ||
| Wed Nov 19 |
Inference in Loopy Models, Variational Inference | MLAPP Sections 20.4, 21.1-3, 21.5, 22.1-2 | Marked Assignment 6 due, matLearn.zip | |
| Mon Nov 24 |
Principle Component Analysis, Independent Component Analysis (Michael Gelbart) | Lec 21 | MLAPP, Section 12.1-12.3, 12 .6, 13.8 | |
| Wed Nov 26 |
Auto-Encdoers (Michael Gelbart) | Lec 22 | Section 27.1-3, 28.3 | Assignment 7 due |
| Mon Dec 1 |
Convex Relaxations, Monte Carlo Methods | Lec 23 | Section 19.5, 22.6, 23.1-4, 24.1-3 | Assignment 8 (due December 17), ass8.zip |
| Fri Dec 5 |
Tut 8 | Marked Assignment 7 due | ||
| Wed Dec 17 |
Assignment 8 and Coding Project and Final Project Due |
