EEL709: Pattern Recognition

If you're doing the course, please join the Piazza forum (the access code has been announced in class).

Note on nomenclature: This course is entitled as above for historical reasons. However, today the term Machine Learning is more widely used to denote the general body of statistical techniques for automatically detecting and modelling patterns in data. Pattern Recognition may at times refer to the use of a more conventional subset of these techniques, such as Neural Networks. This course should thus essentially be seen as an introduction to Machine Learning.
[Machine Learning: A very quick introduction.]
[Introduction to a similar course, written by Amos Storkey at the University of Edinburgh.]

Evaluation components

• Minors: 25% [Paper I; Solutions] [Paper II; Solutions]
• Major: 25% [Paper]
• Class attendance/participation: 10%
• Assignments: 40% [1] [2] [3] [4] [Quiz; Solutions; Re-Quiz]

Planned outline

• Primary reference: Christopher M. Bishop, Pattern Recognition and Machine Learning (PRML). Springer, 2006.
• Secondary reference: David J. C. MacKay, Information Theory, Inference, and Learning Algorithms. Cambridge University Press, 2003. [Free online, and apparently better than Harry Potter, unless one speaks Welsh or Latin.]

  Serial no.	Topics	Lecture nos.	PRML Chapters	Other References	Slides
  1	Introduction, Revision of Probability Theory and Distributions	1–7	1, 2	Review of Probability Theory; The Multivariate Gaussian Distribution	Very quick introduction; PRML Chapter 1
  2	Supervised Learning: Linear Regression Models	8–9	3	Regression notes	PRML Chapter 3
  3	Supervised Learning: Classification, Linear Discriminant Analysis	10–11	4	Generative learning notes	Linear models for classification
  4	Supervised Learning: Kernels, Support Vector Machines	12–16	6, 7	SVM notes; SVM tutorial; Properties of kernels; Platt's SMO paper; SVM demo
  5	Supervised Learning: Neural Networks, Deep Learning	17–21	5	ANN tutorial; Geoff Hinton lecture on deep learning; Convolutional neural nets: tutorial, demos; Sparse autoencoder notes; Primate visual system paper	Google's unsupervised deep learning
  6	Feature Selection	22
  7	Naïve Bayes, Probabilistic Graphical Models	23–25; 29	8	Naïve Bayes notes; MRF notes; Bayesian Networks notes	Graphical Models
  8	Unsupervised Learning: Clustering, Mixture Models, Expectation-Maximisation	26–28	9	K-means notes; GMM notes; EM notes	Clustering
  9	Unsupervised Learning: Latent Variables, Component Analysis	30–34	12	PCA notes; PCA tutorial; Factor analysis notes; ICA notes; ICA overview; Le et al. paper on reconstruction ICA	PCA; ICA for feature learning
  10	Hidden Markov Models	35–38	13	HMM notes; Microsoft's photo-real talking head [demo paper]
  11	Semi-Supervised Learning	39–40		SSL literature survey
  12	Computational Learning Theory	41		Jayadeva's MCM paper

  [Teaching home]