Books on statistical learning
- Learning from Data, Abu-Mostafa, Magdon-Ismail, and Lin, 2012. This book is short and sweet. It doesn't cover everything we will talk about, but has a fantastic and very accessible overview of VC theory. (Amazon)
- The Elements of Statistical Learning, Hastie, Tibshirani, and Friedman, 2009. This book covers most of the material we will be covering in the class and is probably the best overall resource that is freely available on the internet. (Free online version, Amazon)
- Machine Learning: A Probabilistic Perspective, Murhpy. A close second to Hastie et al., this book is also great and is an excellent resource with some material covering more modern topics such as deep learning. (Amazon)
- Pattern Classification, Duda, Hart, and Stork, 2000. Another good overview of much of what we will cover in this course. This book was another close second to Hastie et al., but it isn't available online. (Amazon)
- Pattern Recognition and Machine Learning, Bishop, 2006. This is a classic. A good introduction to machine learning from a Bayesian perspective. I view Murphy to be a slightly more modern update, but for a lot of classic material this is still my go-to resource. (Amazon)
- A Probabilistic Theory of Pattern Recognition, Devroye, Gyorfi, and Lugosi, 1996. A (somewhat intense) book that considers the more theoretical aspects of statistical learning. A great resource if you want to know the details. (Amazon)
- Learning with Kernels, Scholkopf and Smola, 2001. The book on support vector machines and related kernel methods. (Amazon)
- Reinforcement Learning: An Introduction, Sutton and Barto, 1998. We will focus mostly on supervised and unsupervised learning, but this book is a good introduction to the key ideas in reinforcement learning. (Amazon)
- The Signal and the Noise, Silver, 2012. A higher level book that examines several applications of the kinds of statistical learning techniques, including weather, earthquakes, athletic performance, and more. (Amazon)
Books covering background material
- How to Solve It, Polya, 1945. Classic introduction to mathematical problem solving. (Amazon)
- Introduction to Probability, Bertsekas and Tsitsiklis, 2008. Good introduction to elementary probability theory. (Amazon)
- An Introduction to Probability Theory and its Applications, Feller, 1950. An absolute classic. A more advanced (but still accessible) introduction to probability theory. (Amazon)
- The Art of Probability, Hamming, 1994. Another introduction to probability theory written by a fellow engineer. (Amazon)
- All of Statistics, Wasserman, 2004. Exactly what the title says. (Amazon)
- Linear Algebra and Its Applications, Strang, 2005. A great introduction to linear algebra. (Amazon)
- Convex Optimization, Boyd, 2004. We will cover the tools from convex optimization that we need as part of the course, but if you want to know more, this is a great resource targeted towards electrical engineers. (Free online version, Amazon)