Schedule
Subject to change.
Wed 4 Sep 2013
Lecture 1: Introduction to Inference and Learning
Mon 9 Sep 2013
Lecture 2: Simple Discrete Models
[ discrete_coins.m | plot_beta.m | beta_coins.m ]
- Assignment 1 Out
- [required] Book: Murphy -- Chapter 2 -- Probability
- [required] Book: Murphy -- Chapter 3 -- Generative Models for Discrete Data
- [optional] Book: Bishop -- Chapter 2, Sections 2.1-2.2 -- Probability Distributions
- [optional] Book: MacKay -- Chapter 2 -- Probability, Entropy, and Inference
- [optional] Book: MacKay -- Chapter 3 -- More About Inference
- [optional] Book: Mackay -- Chapter 23 -- Useful Probability Distributions
- [optional] Video: Iain Murray -- Introduction to Machine Learning, Part 2
- [optional] Metacademy: Bayesian Parameter Estimation
- [optional] Metacademy: Dirichlet Distribution
Linear Models
Wed 11 Sep 2013
Lecture 3: Simple Gaussian Models
[ plot_bigauss1.m | plot_bigauss2.m ]
Fri 13 Sep 2013
Section 1: Math Review
[ notes ]
Mon 16 Sep 2013
Lecture 4: Bayesian Statistics
[ quiz ]
Wed 18 Sep 2013
Lecture 5: Linear Regression (Guest Lecturer: Matt Johnson)
[ demos ]
- Assignment 1 Help Session 5-7pm, Maxwell-Dworkin Second Floor Lounge
- [required] Book: Murphy -- Chapter 7 -- Linear Regression
- [optional] Book: Bishop -- Chapter 3 -- Linear Models for Regression
- [optional] Book: Hastie, Tibshirani, and Friedman -- Chapter 2 -- Overview of Supervised Learning
- [optional] Book: Hastie, Tibshirani, and Friedman -- Chapter 3 -- Linear Methods for Regression
- [optional] Metacademy: Linear Regression
Fri 20 Sep 2013
Section 2: Practical Optimization
[ notes ]
- Assignment 1 Due
- Assignment 2 Out
- [optional] Metacademy: Optimization
Mon 23 Sep 2013
Lecture 6: Linear Classifiers
[ quiz ]
- [required] Book: Murphy -- Chapter 8 -- Logistic Regression
- [optional] Book: Bishop -- Chapter 4 -- Linear Models for Classification
- [optional] Book: Hastie, Tibshirani, and Friedman -- Chapter 4 -- Linear Methods for Classification
- [optional] Metacademy: Binary Linear Classifiers
Wed 25 Sep 2013
Lecture 7: Generalized Linear Models
[ quiz ]
Unsupervised Bayesian Modeling
Fri 27 Sep 2013
Section 3: Undirected Graphical Models and Factor Graphs
[ notes ]
- Return Assignment 1
- [required] Book: Murphy -- Chapter 19, Sections 19.1-19.4 -- Undirected Graphical Models (Markov Random Fields)
- [optional] Book: Bishop -- Chapter 8, Sections 8.1-8.3 -- Graphical Models
- [optional] Book: Hastie, Tibshirani, and Friedman -- Chapter 17 -- Undirected Graphical Models
- [optional] Book: Koller and Friedman -- Chapter 4 -- Undirected Graphical Models
- [optional] Metacademy: Markov Random Fields
- [optional] Metacademy: Factor Graphs
Mon 30 Sep 2013
Lecture 8: Directed Graphical Models
[ quiz ]
- [required] Book: Murphy -- Chapter 10, Sections 10.1-10.5 -- Directed Graphical Models (Bayes Nets)
- [optional] Book: Bishop -- Chapter 8 -- Graphical Models
- [optional] Book: Koller and Friedman -- Chapter 3 -- The Bayesian Network Representation
- [optional] Paper: Martin J. Wainwright and Michael I. Jordan. Graphical Models, Exponential Families and Variational Inference. Foundations and Trends in Machine Learning 1(1-2):1-305, 2008.
- [optional] Paper: Michael I. Jordan. Graphical Models. Statistical Science 19(1):140-155, 2004.
- [optional] Video: Zoubin Ghahramani -- Graphical Models
- [optional] Video: Cedric Archambeau -- Graphical Models
- [optional] Metacademy: Bayesian Networks
Wed 2 Oct 2013
Lecture 9: Mixture Models
[ quiz ]
- Assignment 2 Help Session, 5-7pm, Maxwell-Dworkin Second Floor Lounge
- [required] Book: Murphy -- Chapter 11 -- Mixture Models and the EM Algorithm
- [optional] Book: Bishop -- Chapter 9 -- Mixture Models and EM
- [optional] Book: Mackay -- Chapter 20 -- An Example Inference Task: Clustering
- [optional] Book: Mackay -- Chapter 22 -- Maximum Likelihood and Clustering
- [optional] Metacademy: Expectation-Maximization Algorithm
Fri 4 Oct 2013
Section 4: Factor Analysis and PCA
[ notes ]
Mon 7 Oct 2013
Lecture 10: Sparse Linear Models
[ quiz ]
- [required] Book: Murphy -- Chapter 13, Sections 13.1-13.7 -- Sparse Linear Models
- [optional] Book: Hastie, Tibshirani, and Friedman -- Chapter 3 -- Linear Methods for Regression
- [optional] Metacademy: LASSO
Inference Procedures
Wed 9 Oct 2013
Lecture 11: Exact Inference
[ quiz ]
- [required] Book: Murphy -- Chapter 17, Section 17.1-17.4 -- Markov and Hidden Markov Models
- [required] Book: Murphy -- Chapter 20, Section 20.1-20.3 -- Exact Inference for Graphical Models
- [optional] Book: Murphy -- Chapter 17, Section 17.5-17.6 -- Markov and Hidden Markov Models
- [optional] Book: Bishop -- Chapter 8, Sections 8.4 -- Graphical Models
- [optional] Book: Bishop -- Chapter 13, Sections 13.1-13.2 -- Sequential Data
- [optional] Book: Mackay -- Chapter 16 -- Message Passing
- [optional] Book: Mackay -- Chapter 21 -- Exact Inference by Complete Enumeration
- [optional] Book: Mackay -- Chapter 24 -- Exact Marginalization
- [optional] Book: Mackay -- Chapter 26 -- Exact Marginalization in Graphs
- [optional] Book: Koller and Friedman -- Chapter 9 -- Exact Inference: Variable Elimination
- [optional] Book: Koller and Friedman -- Chapter 10 -- Exact Inference: Clique Trees
- [optional] Book: Koller and Friedman -- Chapter 13 -- MAP Inference
- [optional] Frank R. Kschischang, Brendan J. Frey and Hans-Andrea Loeliger. Factor Graphs and the Sum-Product Algorithm. IEEE Transactions on Information Theory 47(2):498-519, 2001.
- [optional] Metacademy: Variable Elimination
Fri 11 Oct 2013
Section 5: The Junction Tree Algorithm
[ notes ]
- Last Day for Assignment 1 Regrades
- Return Assignment 2
- [required] Book: Murphy -- Chapter 20, Section 20.4 -- Exact Inference for Graphical Models
- [optional] Metacademy: Junction Trees
Wed 16 Oct 2013
Lecture 12: Variational Inference (Guest Lecturer: Matt Johnson)
[ demo | quiz ]
Fri 18 Oct 2013
Section 6: Loopy Belief Propagation
Mon 21 Oct 2013
Lecture 13: Monte Carlo Basics (Guest Lecturer: Finale Doshi-Velez)
[ quiz ]
- [required] Book: Murphy -- Chapter 23, Section 23.1-23.4 -- Monte Carlo Inference
- [optional] Book: Bishop -- Chapter 11, Section 11.1 -- Sampling Methods
- [optional] Book: MacKay -- Chapter 29 -- Monte Carlo Methods
- [optional] Book: Devroye -- Chapter 2
- [optional] Metacademy: Monte Carlo Estimation
- [optional] Metacademy: Rejection Sampling
- [optional] Metacademy: Importance Sampling
Wed 23 Oct 2013
Midterm Exam
- Final Project Brainstorming Session 5-7pm, Maxwell-Dworkin Second Floor Lounge
Fri 25 Oct 2013
Section 7: Particle Filtering
Mon 28 Oct 2013
Lecture 14: Markov Chain Monte Carlo
[ quiz ]
Wed 30 Oct 2013
Lecture 15: Advanced Markov Chain Monte Carlo
[ quiz ]
Fri 1 Nov 2013
Section 8: MCMC Practicalities
- Final Project Proposal Feedback Available
- Midterms Back
Example Models
Mon 4 Nov 2013
Lecture 16: Latent Dirichlet Allocation
[ quiz ]
Wed 6 Nov 2013
Lecture 17: State Space Models
- Assignment 4 Help Session 5-7pm, Maxwell-Dworkin Second Floor Lounge
- [required] Book: Murphy -- Chapter 18, Sections 18.1-18.4 -- State Space Models
- [optional] Book: Murphy -- Chapter 18, Sections 18.5-18.6 -- State Space Models
- [optional] Book: Bishop -- Chapter 13, Sections 13.3 -- Sequential Data
- [optional] Paper: Eric A. Wan and Rudolph van der Merwe. The Unscented Kalman Filter for Nonlinear Estimation.
- [optional] Metacademy: Linear Dynamical Systems
Fri 8 Nov 2013
Section 9: Graph Models
[ notes ]
- Assignment 4 Due
- Assignment 5 Out
- Last Day for Assignment 3 Regrades
- [required] Book: Murphy -- Chapter 27, Sections 27.5-27.6 -- Latent Variable Models for Discrete Data
Nonparametric Models
Mon 11 Nov 2013
Lecture 18: Kernels
[ quiz ]
Wed 13 Nov 2013
Lecture 19: Gaussian Processes
[ quiz ]
- [required] Book: Murphy -- Chapter 15 -- Gaussian Processes
- [optional] Book: Rasmussen and Williams -- Chapter 2 -- Regression
- [optional] Book: Rasmussen and Williams -- Chapter 3 -- Classification
- [optional] Book: Bishop -- Chapter 6, Sections 6.4 -- Kernel Methods
- [optional] Book: MacKay -- Chapter 45 -- Gaussian Processes
- [required] Video: David MacKay -- Gaussian Process Basics
- [optional] Video: Carl Rasmussen -- Learning with Gaussian Processes
- [optional] Metacademy: Gaussian Process
Fri 15 Nov 2013
Section 10: Practical Gaussian Processes
Mon 18 Nov 2013
Lecture 20: Dirichlet Processes I
[ quiz ]
Wed 20 Nov 2013
Lecture 21: Dirichlet Processes II
[ quiz ]
- Assignment 5 Help Session 5-7pm, Maxwell-Dworkin Second Floor Lounge
- [optional] Paper: Jayaram Sethuraman. A Constructive Definition of Dirichlet Priors. Statistica Sinica 4, 1994.
- [optional] Paper: Yee Whye Teh, Michael I. Jordan, Matthew J. Beal and David M. Blei -- Hierarchical Dirichlet Processes, Journal of the American Statistical Association, 2006.
- [optional] Paper: Radford M. Neal -- Defining Priors for Distributions Using Dirichlet Diffusion Trees, Bayesian Statistics 7, 619-629, 2003.
- [optional] Paper: Zoubin Ghahramani, Thomas L. Griffiths and Peter Sollich -- Bayesian nonparametric latent feature models
- [optional] Metacademy: Hierarchical Dirichlet Process
Fri 22 Nov 2013
Section 11: Practical Dirichlet Processes
Deep Learning
Mon 25 Nov 2013
Lecture 22: Boltzmann Machines
Mon 2 Dec 2013
Lecture 23: Neural Networks
[ quiz ]
- Last Day for Assignment 4 Regrades
- Assignment 5 Back
- [required] Book: Murphy -- Chapter 16, Section 16.5 -- Adaptive Basis Function Models
- [optional] Book: Bishop -- Chapter 5 -- Neural Networks
- [optional] Book: MacKay -- Chapter 39 -- The Single Neuron as a Classifier
- [optional] Book: MacKay -- Chapter 40 -- Capacity of a Single Neuron
- [optional] Book: MacKay -- Chapter 41 -- Learning as Inference
- [optional] Book: MacKay -- Chapter 44 -- Supervised Learning in Multilayer Networks
- [optional] Book: Hastie, Tibshirani, and Friedman -- Chapter 11 -- Neural Networks
- [optional] Metacademy: Feedforward Neural Networks
Wed 4 Dec 2013
Lecture 24: Advanced Neural Networks
[ quiz ]
Thu 5 Dec 2013
- Final Project Poster Session
Wed 11 Dec 2013
- Final Project Reports Due
- Last Day for Assignment 5 Regrades