AIL 7024: Machine Learning (26-01)

Course Contents

Below is a tentative list of topics:

  1. Sampling from Probability Distributions
  2. Foundations: Maximum Likelihood Estimation, Bias-Variance Tradeoff
  3. Classification
  4. Latent Variable Models and Expectation Maximisation
  5. Score-based Models
  6. Diffusion Models
  7. Flow-based Models
  8. Influence Functions

Textbooks:

  1. Pattern Recognition and Machine Learning, Bishop
  2. The Elements of Statistical Learning, Hastie, Tibshirani and Friedman
  3. Deep Learning, Bishop

Logistics

Time and Location:

  • Slot: C
  • Class Timings: Tue/Wed/Fri 8-9 am
  • Venue: LH 521

Communication:

We will use piazza as the forum for students to ask questions about both the course material as well as logistics.

Joining Link

Teaching Team:

Announcements

Schedule

The weekly schedule will be updated with lecture slides and pointers to additional references as we progress through the course.

Monday Tuesday Wednesday Thursday Friday Saturday Sunday
Week 1 July 20 July 21 July 22 July 23 July 24 July 25 July 26
Course overview Course overview
Slides
Week 2 July 27 July 28 July 29 July 30 July 31 Aug 1 Aug 2
Sampling

PRML Chapter 11 		Inverse Transform Sampling
Notebook
Notebook: Gaussian inverse CDF
 Sampling via change of variables Rejection Sampling
Notebook
Week 3 Aug 3 Aug 4 Aug 5 Aug 6 Aug 7 Aug 8 Aug 9
Statistical Principles

PRML Chapter 1 & 3 		Polynomial curve fitting, Max likelihood estimation
 Logistic Regression, Decision Theory: Regression & Classification
CS229 Notes Chapter 2 		Decision Theory (contd.),
Why MLE
Week 4 Aug 10 Aug 11 Aug 12 Aug 13 Aug 14 Aug 15 Aug 16
Model Selection

PRML Chapter 1 		Monte Carlo Estimator of log likelihood,
MAP estimate
 Degree of polynomial fit
Bias-Variance Decomposition
Slides
CS229 Notes Chapter 8
Week 5 Aug 17 Aug 18 Aug 19 Aug 20 Aug 21 Aug 22 Aug 23
Regularisation, Generative Classification
ESL Chapter 7
 Variance of OLS and regularised least squares
Bias OLS vs regularised, Regularisation visualisations,
Cross Validation
Slides
 Gaussian Discriminant Analysis
Week 6 Aug 24 Aug 25 Aug 26 Aug 27 Aug 28 Aug 29 Aug 30
Gaussian Discriminant Analysis
PRML Section 4.2
CS229 Chapter 4
 QDA & LDA with isotropic Gaussians, Gaussian density estimation
Slides
 Multivariate Gaussians
Slides
Week 7 Aug 31 Sep 1 Sep 2 Sep 3 Sep 4 Sep 5 Sep 6
GDA,
Naive Bayes
CS229 Chapter 4.2
 GDA with anisotropic Gaussians
Slides 		QDA vs LDA vs Logistic Regression,
Bayesian Networks
Slides 		Naive Bayes Classifier
Slides
Week 8 Sep 7 Sep 8 Sep 9 Sep 10 Sep 11 Sep 12 Sep 13
Gaussian Mixture Models
PRML Sections 2.3.9, 9.2
 GMM with complete data
Slides 		GMM with incomplete data
Week 9 Sep 14 Sep 15 Sep 16 Sep 17 Sep 18 Sep 19 Sep 20
Midsem
Week 10 Sep 21 Sep 22 Sep 23 Sep 24 Sep 25 Sep 26 Sep 27
Continous Latent Variables,
Probabilistic PCA
PRML Chapter 12
DL Chapter 16
 Mixture of Bernoullis, Latent Variables, Manifolds
Slides
Goodfellow DL Section 5.11
 Continous latent variables,
Probabilistic PCA
Week 11 Sep 28 Sep 29 Sep 30 Oct 1 Oct 2 Oct 3 Oct 4
Week 12 Oct 5 Oct 6 Oct 7 Oct 8 Oct 9 Oct 10 Oct 11
Topic Lecture
Lecture
Lecture
Lecture
Week 13 Oct 12 Oct 13 Oct 14 Oct 15 Oct 16 Oct 17 Oct 18
Topic Lecture
Lecture
Lecture
Week 14 Oct 19 Oct 20 Oct 21 Oct 22 Oct 23 Oct 24 Oct 25
Topic Lecture
Lecture
Week 15 Oct 26 Oct 27 Oct 28 Oct 29 Oct 30 Oct 31 Nov 1
Topic Lecture:
Lecture
Lecture
Week 16 Nov 2 Nov 3 Nov 4 Nov 5 Nov 6 Nov 7 Nov 8
Topic Lecture
Lecture
Lecture
Week 17 Nov 9 Nov 10 Nov 11 Nov 12 Nov 13 Nov 14 Nov 15
Topic Lecture
Lecture
Lecture
Week 18 Nov 16 Nov 17 Nov 18 Nov 19 Nov 20 Nov 21 Nov 22
Topic Lecture
Endsem

Grading Policy

Grading policy (tentative, upto ±10% on each component):

  1. Assignments: 40%
  2. Exams: 60%
  3. Audit policy: At least 50% in each and every exam

Attendance policy:

  1. If a student’s attendance is less than 75%, the student will be awarded one grade less than the actual grade that he/she has earned.
  2. A student cannot get NP for an audit course if his/her attendance is less than 75%.