Announcements
- Jun 29, 2020 Final grades posted to SSC. Congratulations for finishing this course!
This Semester
Instructor
Lectures Sections (beginning May 11, 2020)
- 9 - 11:30 am (Monday/Wednesday/Friday) online through Zoom.
Office hours
- Instructor: Wednesdays 11:30 to 12:30 (after the lectures).
- TAs:
- There 9 TA office hours each week.
- See the calendar below for the most accurate schedule. The TA office hours will be delivered through Canvas/Collaborate.
Tutorials (beginning May 13, 2020)
- There are 6 tutorials each week.
- The tutorials start on May 13th.
- T1A: Mon/Wed 1-2 pm.
- T1B: Mon/Wed 2-3 pm.
- T1C: Mon/Wed 3-4 pm.
- The tutorials will be delivered through Canvas/Collaborate.
Teaching assistants
Services
- Piazza for course-related questions and HW solutions (accessible through Canvas).
- Zoom for lecture presentations and CollaborateUltra for office hours.
- Gradescope for homework assignments (accessible through Canvas).
- Google Calendar for course schedule (see below).
Note Zoom is hosted on servers in the U.S., so your name and data about how you use the system will be stored on servers outside Canada. If you have privacy concerns about this data collection: a) don’t create an account with Zoom and provide only your first name or a nickname when you join a session, b) keep your camera off and microphone muted during sessions, and c) don’t share any identifying information about yourself.
Course Calendar
- I recommend that you subscribe to this calendar to always have the up-to-date schedule.
Registration
Undergraduate and graduate students from any department are welcome to take the course. Undergraduate students should enroll in CPSC 340 while graduate students should enroll in CPSC 532M (which has an extra small project component -- not offered in summers). Below are more details on registration for each course:
- The majority of the seats in 340 are reserved for UBC computer science majors. For other students, to enroll in the course you need to sign up for the wait list. Note that last year all students on the wait list were ultimately accepted into the course (but we did not have room for auditors.)
- Two-thirds of the seats in 532M will initially be reserved for UBC computer science grad students, while the remaining seats are open to graduate students from any department. Note that you will likely get a warning about prerequisities when you register (which you can ignore), but depending on your department you may later be contacted and asked to show how you satisfy the relevant prerequisites. As with 340, if 532M becomes full then signing up for the waiting list is the only way to enroll in the course. Last year all students on the wait list were ultimately accepted into the course.
Starting in the first week of classes, we will have weekly tutorials run by the TAs. These will do things like go through provided assignment code, review background material, review big concepts, and/or do exercises. You can register for particular tutorial sections if you want to save a seat at a particular time, but note that you do not need to register in a tutorial section.
Prerequisites
- Basic algorithms and data structures (CPSC 221, or both of CPSC 260 and EECE 320 as well as one of CPSC 210, EECE 201, or EECE 309).
- Linear algebra (one of MATH 152, 221, or 223).
- Probability (one of STAT 200, STAT 203, STAT 241, STAT 251, STAT 302, MATH 302, MATH 318, or BIOL 300).
- Multivariate calculus (one of MATH 200, 217, 226, 253, or 263).
Graduates students may receive a warning about prerequisites when registering and may need to follow additional steps described
here.
Textbook
There is no required textbook for the class. A introductory book that covers many (but not all) the topics we will discuss is the Artificial Intelligence book of Rusell and Norvig (AI:AMA) or the Artificial Intelligence book of Poole and Mackworth (you may need these for other classes). More advanced books include The Elements of Statistical Learning (ESL) by Hastie et al., Murphy's Machine Learning: A Probabilistic Perspective (ML:APP) which can be accessed through the library here, and Bishop's Pattern Recognition and Machine Learning (PRML). For books with a bigger focus on data mining, see Introduction to Data Mining (IDM) and Mining of Massive DataSets.
Related Courses
Related courses in statistics include:
STAT 305,
STAT 306,
STAT 406,
STAT 460,
STAT 461 (as well as
EOSC 510). A discussion of the difference between CPSC 340 and these various STAT classes written by a former student (Geoff Roeder) is available
here.
Grading
- Assignments 30%
- Midterm 30%
- Final 40%
List of topics
We will roughly cover the following topics:
- Data representation and summarization.
- Supervised learning with frequencies and distances.
- Data clustering, outlier detection, and association rules.
- Linear prediction, regularization, and kernels.
- Latent-factor models and collaborative filtering.
- Neural networks and deep learning.
Lectures
| Date |
Video |
Slides |
Related Readings and Links |
Notes |
| Mon May 11 |
may.11.2020.mp4 |
Motivation and Syllabus
|
What is Machine Learning? ·
Machine Learning Wiki ·
Rise of the Machines ·
Talking Machine Episode 1
|
|
|
Exploratory Data Analysis
|
Gotta Catch'em all · Why Not to Trust Statistics ·
Visualization Types · Google Chart Gallery ·
Other Tools
|
See assignment 1 below.
|
| Wed May 13 |
may.13.2020.mp4
|
Decision Trees
|
A Visual Introduction to Machine Learning ·
Decision Trees ·
Entropy
AI:AMA 18.2-3, ESL: 9.2, ML:APP 16.2
|
Big-O Notes
|
|
Fundamentals of Learning
|
7 Steps of Machine Learning ·
IID ·
Cross-validation ·
Bias-variance ·
No Free Lunch
AI: AMA 18.4-5, ESL 7.1-7.4, 7.10, ML:APP 1.4, 6.5
|
Course Notation Guide
|
| Fri May 15 |
may.15.2020.mp4
|
Probabilistic Classifiers
|
Conditional probability (demo) ·
Naive Bayes ·
Probabilities and Battleship
ESL 4.3, ML: APP 2.2, 3.5, 4.1-4.2
|
Probability Notes
Probability Slides
|
|
Non-Parametric Models
|
K-nearest neighbours ·
Decision Theory for Darts ·
Norms
AI: AMA 18.8, ESL 13.3, ML:APP 1.4
|
|
| Wed May 20 |
may.20.2020.mp4
|
Ensemble Methods
|
Ensemble Methods ·
Random Forests ·
Empirical Study ·
Kinect ·
Data Augmentation
AI: AMA 18.10, ESL: 7.11, 8.2, 15, 16.3, ML: APP 6.2.1, 16.2.5, 16.6
|
|
|
Clustering
|
Clustering ·
K-means clustering (demo) ·
K-Means++ (demo)
IDM 8.1-8.2, ESL: 14.3
|
|
Fri May 22
|
may.22.2020.mp4 |
More Clustering
|
DBSCAN
(video
demo) ·
Hierarchical Clustering ·
Phylogenetic Trees
IDM 8.4
|
|
|
Outlier Detection
|
Empirical Study
IDM 8.3, ESL 14.3.12, ML:APP 25.5 |
|
| | |
Finding Similar Items
(Bonus Lecture) |
MMD Chapter 3 |
|
| Mon May 25 |
may.25.2020.mp4 |
Least Squares
|
Linear Regression ·
(demo,
2D data, 2D video) ·
Least Squares
Essence of Calculus ·
Partial Derivative ·
Gradient
ESL 3.1-2, ML:APP 7.1-3, AI:AMA 18.6
|
Calculus Notes |
|
Nonlinear Regression
|
Why should one learn machine learning from scratch? ·
Essence of Linear Algebra ·
Matrix Differentiation ·
Fluid Simulation (video) ·
ESL 5.1(Poly basis), 6.3 (local regression)
|
Linear Algebra Notes
Linear/Quadratic Gradients |
Wed May 27
|
may.27.2020.mp4 |
Gradient Descent
|
Gradient Descent ·
Convex Functions
|
|
|
Robust Regression
|
ML:APP 7.4 |
|
| The midterm will be from the material above. |
Fri May 28
|
may.29.2020.mp4 |
Feature Selection
|
Genome-Wide Association Studies ·
AIC ·
BIC
ESL 3.3, 7.5-7
|
|
|
Regularization
|
ESL 3.4., ML:APP 7.5, AI:AMA 18.4
|
|
| Mon Jun 1 |
|
Midterm Exam in class. You will need a webcam to take the exam. |
Mon Jun 1
|
jun.1.2020.mp4 |
More Regularization
|
RBF video ·
RBF and Regularization video
ESL 6.7, ML:APP 13.3-4
|
HW 4 released |
| Wed Jun 3 |
jun.3.2020.mp4
|
Linear Classifiers
|
Perceptron
ESL 4.5, ML:APP 8.5
|
|
|
More Linear Classifiers
|
Support Vector Machines
ESL 4.4, 12.1-2, ML:APP 8.1-3, 9.5 14.5, AI:AMA 18.9
|
|
| Fri Jun 5 |
jun.5.2020.mp4 |
Feature Engineering
|
Gmail Priority Inbox
|
|
|
Convolutions + 1/2 Kernels
|
|
|
| Mon Jun 8 |
jun.8.2020.mp4 |
Kernel Trick (Bonus: all slides)
|
ESL 12.3, ML:APP 14.1-4
|
HW5 released |
|
1/2 Kernels + Stochastic Gradient
|
Stochastic Gradient
ML:APP 8.5
|
|
|
Boosting
|
AdaBoost (video) ·
XGBoost (video)
ML:APP 16.4
|
Max and Argmax Notes
|
Wed Jun 10
|
jun.10.2020.mp4 |
MLE and MAP
|
Maximum Likelihood Estimation
ML:APP 9.3-4
|
|
|
Principal Component Analysis
|
Principal Component Analysis
ESL 14.5, IDM B.1, ML:APP 12.2
|
|
Fri Jun 12
|
jun.12.2020.mp4 |
More PCA
|
Making Sense of PCA ·
SVD Eigenfaces
|
|
|
Sparse Matrix Factorization
|
Non-Negative Matrix Factorization · (original - access from UBC)
ESL 14.6, ML: APP 13.8
|
|
Mon Jun 15
|
jun.15.2020.mp4 |
Recommender Systems
|
Recommender Systems · Netflix Prize |
HW 6 released |
|
Multi-Dimensional Scaling
|
Nonlinear Dimensionality Reduction · t-SNE demo
ESL 14.8-9, IDM B.2 |
|
| The FINAL will be from the material above. |
Wed Jun 17
|
jun.17.2020.mp4 |
Deep Learning
|
Google Video · What is a Neural Network? · Interactive Guide
ML:APP 16.5, ESL 11.1-4, AI: AMA 18.7 |
|
|
More Deep Learning
|
Fortune Article · Deep Learning References ·
Alchemy ·
Convolutional Neural Networks
ML:APP 28.4, ESL 11.7
ML:APP 28.3, ESL 11.5 |
|
From
Mon Jun 22
8:00 AM |
To
Wed Jun 24
11:00 AM |
Final Exam
- Take-home
- Individually
- Open-book
See this Piazza post for more information. |
The acroynms in the table above refer to the following textbooks:
- AI:AMA: Artificial Intelligence: A Modern Approach by Russell and Norvig
- ESL: The Elements of Statistical Learning by Hastie et al.
- ML:APP: Machine Learning: A Probabilistic Perspective by Kevin Murphy
- PRML: Pattern Recognition and Machine Learning by Christopher Bishop
- IDM: Introduction to Data Mining by Steinbach et al.
Related courses that have online notes
Homework Assignments
| Post Date |
Due Date |
Files |
Notes/Links |
| Mon May 11 |
Fri May 15
Sun May 17
|
a1.pdf · a1.zip (contains the LaTeX template + code)
a1.v2.pdf
|
Setting up Python
v2 changes are highlighted in red.
|
| Mon May 18 |
Sun May 24
|
a2.pdf · a2.zip (contains the LaTeX template + code)
|
|
| Mon May 25 |
Sun May 31
|
a3.pdf · a3.zip (contains the LaTeX template + code)
|
|
| Mon Jun 1st |
Sun Jun 7th
|
a4.pdf · a4.zip (contains the LaTeX template + code)
|
|
| Mon Jun 8th |
Sun Jun 14th
|
a5.pdf · a5.zip (contains the LaTeX template + code)
|
|
| Mon Jun 14th |
Sun Jun 21st
|
a6.pdf · a6.zip (contains the LaTeX template + code)
|
Due to time constraints, no late-submission is allowed.
|
- Submission will happen through Gradescope. Please sign up through Canvas.
- Instructions Comprehensive submission instructions are here.
- Handwritten submissions are not allowed. You will need to use LaTeX.
- You must work alone for Assignment 1. For Assignment 2 onwards, you may work with a partner or work alone. You can make this decision independently for each assignment. Make your partnership details explicit in your report.
- Suggestion: if you’ve never used LaTeX and/or Python before, try to find a partner who has, especially for the early assignments when these tools are new to you.
- Late submissions Up to 2 late classes per assignment and up to 4 late classes across the entire term. See lecture 1 for details.
- Assignment updates It happens sometimes that the instructor needs to make changes to an assignment after it is posted. If this happens we will notify you on piazza and you can download a new version of the assignment as required.
- Citation/Acknowledgement If you use information from students outside your group or from online sources including code snippets from Stack Overflow, cite this at the start of each question. You will receive a mark of 0 for the assignment (and possibly other consequences) if you are found copying from other sources without citation.
- Regrading policy If you perceive a problem with your homework grade, you have 1 week to raise a concern, from the time that your grades were posted. After that, your grade is final.
- See lecture 1 for further info.
Previous Offerings
- We will be going through the same path as the previous offerings of the course. You may find the following links useful.
- Mike's CPSC 340 videos on YouTube and slides, notebooks, and readings here.
- Mark's CPSC 340 material here.
