Theory of generalization
- A first model of learning [Slides, Notes ]
- The Bayes classifier, nearest neighbor classifiers, and consistency [Slides, Notes ]
- Measuring complexity and the growth function [Slides, Notes]
- VC dimension and VC generalization bounds [Slides, Notes]
- Overfitting and the bias variance decomposition [Slides, Notes ]
Methods for supervised learning
- Regression and regularization [Slides, Notes]
- Principal component analysis and the LASSO [Slides, Notes]
- Plugin classifiers: Linear discriminant analysis and logisitic regression [Slides, Notes]
- Plugin classifiers: Logistic regression (continued) and Naive Bayes [Slides, Python code]
- Overview of unconstrained optimization [Slides]
- The perceptron algorithm, maximum margin hyperplanes, and kernels [Slides, Notes on kernels]
- Overview of constrained optimization and support vector machines [Slides, Notes on SVMs, duality, and kernelization]
- Kernel regression and density estimation[Slides]
Unsupervised learning
- Multidimensional scaling and nonlinear dimensionality reduction [Slides]
- Structured matrix factorizations [Slides]
Supervised learning (Part 2)
- Model selection, cross validation, and the bootstrap [Slides]
- Decision trees and random forests [Slides]
- Neural networks [Slides]
- Neural networks in practice and in theory [Slides]
Unsupervised learning (Part 2)
- K-means clustering and Gaussian mixture models [Slides]
- Spectral clustering, density-based clustering, and hierarchical clustering [Slides]