Machine Learning Module ML(M)

 

Semester 2, 2007

Lecturer: Professor. M. A .Girolami

 

Lectures:

Monday, 1.00pm, Maths Building (204)

Thursday, 1.00pm, Maths Building 515 (4B)

 

 

Laboratory & Tutorial:

Friday, 1.00pm, Boyd Orr, Level 4 Laboratory

 

Module Descriptor

Student Ratings of Module 2006

 

I feel I learned a lot from the course and it pushed me! - Robin Donaldson (Level 4 CS, 2006)

 

Altogether I really enjoyed the course and I learned a lot of useful techniques.

I really liked your lectures especially the tutorials and scripts - Clement Rodegast (Exchange Student CS, 2006)

 

 

Week No.

Lecture Slides

Lecture Notes

Lab Sheet

Tutorial

Additional Files

1.

Lecture 1

Lecture 2

Introduction & Linear Regression

Linear Regression

Vector Differentiation

wk_1_matlab.m

long_jump_data.txt

2.

Lecture 1

Lecture 2

Generalisation

Cross Validation

 

cross_val.m

cv_demo.m
long_jump_cv.m

3.

Lecture 1

Lecture 2

Probabilistic & Bayesian Methods

Bayesian Regression

 

max_like_demo.m

brdemo.m

regdemo.m

wk3_lab_1_sol.m

wk3_lab_2_sol.m

cross_val_wk3.m

gauss.m
kernel_func.m

4.

Lecture 1

Lecture 2

Probabilistic Classification Methods

Laplace Approximation, Logistic Regression & Nave Bayes

Coursework Handout

laplace_demo.m

logistic_classification_demo.m

lab_4_sol.m

nave_bayes_binary.m

20news_w100.mat

rip_dat_tr.txt

rip_dat_te.txt

5.

Lecture 1

Lecture 2

Non-Probabilistic Classification Methods

Support Vector Machines & K-Nearest Neighbours

 

knn_multi_class.m

svm_demo.m

svm_demo_kernels.m

monqp0.m

cout.m

cross_val_wk5.m

digits_3_8.mat

6.

Lecture1&2

Probability Density Estimation

EM Algorithm

 

gauss_density_est.m

gauss_mix_em_demo.m

mix_gauss_density.m

multi_var_gauss_sampler.m

Gauss_Mix_Data.mat

Lab_6_EM_Data.mat

7.

Lecture1&2

Principal Component Analysis

PCA on images

 

olivettifaces.mat

faces_demo.m

power_pca.m

8.

Lecture 1

Cluster Analysis

Image segmentation & nonlinear clustering

Girolami, M., Mercer kernel-based clustering in feature space, IEEE Trans NN, 13(3), 780-784, 2002.

kmeans.m

kernel_kmeans.m

wk8_demo_1.m

wk8_demo_2.m

wk8_demo_dat.mat

kernel_func.m

ollivettifaces.mat

wee_dog.jpg

water_lillies.jpg

9.

 

Independent Component Analysis

Noisy_Images

 

ICA Research Network

10.

 

Tutorial

 

Coursework Submission

 


Student Resources

Matlab Tutorials

  1. http://www.mathworks.com/academia/student_center/tutorials/launchpad.html
  2. http://www.duke.edu/~hpgavin/matlab.html
  3. http://www.mathworks.com/access/helpdesk/help/techdoc/learn_matlab/learn_matlab.shtml

Suggested Books

  1. The Elements of Statistical Learning: Data Mining, Inference, and Prediction., Hastie, Tibshirani & Friedman, Book Website
  2. Pattern Classification, 2nd Edition., Duda, Hart & Stork, Book Website
  3. Pattern Recognition & Machine Learning ., Bishop, Book Website

Data Repositories

  1. UCI Data Repository All the standard data collections used to illustrate various Machine Learning methods are found here
  2. DELVE Datasets for Evaluating and Comparing Learning Methods

Useful Reference Material

  1. Matrix Cookbook An encyclopaedic collection of just about every matrix and vector equality you will ever need
  2. Gaussian Identities Sam Roweis compiled a useful listing of some of the more common Gaussian identities you may encounter
  3. Nice Wikpedia entry for Machine Learning.