Review of machine learning textbooks
Kevin Murphy, March 2005.

- Hastie, Tibsharani, Friedman,
"The elements of statistical learning".

The problem with this book is that it is on the one hand too advanced,
and on the other hand too narrow in scope (no discussion of graphical models,
time series, Bayesian methods). Also, it feels to me like a cookbook of
techniques rather than a unified presentation. Nevertheless, it is
probably the best book on supervised learning currently available.


- Duda, Hart, Stork,
"Pattern classification" (2nd ed)

This has a great table of contents and great pictures, but
unfortunately I found several major errors, so I no longer trust
it. The material from the first edition (1973) is great, but many of the
new sections are poorly written. Also, it's $115 USD!


- Denison, Holmes, Mallick, Smith.
"Bayesian methods for nonlinear classification and regression"

This book is a beautiful exposition of Bayesian approaches to
supervised learning. Almost all the methods use the same basic
algorithm: reversible jump MCMC. This is both a strength and a
weakness. Unfortunately, the book is too narrow in scope to be used as a
textbook for an introductory class on machine learning.


- Russell and Norvig,
"AI: a modern approach" (2nd ed)

This book is about AI, but has an excellent introduction to machine
learning and probabilistic reasoning.
In particular, chapter 13 is one of the clearest introductions to
probability theory I've encountered. Similarly, chap 14 is a great
introduction to Bayes nets (no discussion of undirected graphical
models, though). Chap 15 is a good introduction to dynamic Bayes nets (HMMs
and LDSs). Chaps 16-17 are good introductions to decision
theory. Chaps 18-20 are good introductions to machine learning.
This is probably the best source for an undergraduate class on machine
learning.


- Mackay,
"Information theory, Inference and Learning algorithms".

This has some great individual chapters on topics related to machine
learning (eg chap 28 on Bayesian model selection),
but it's not really suitable as a textbook for a class.

- Mitchell,
"Machine learning"

I have not read this book, but it seems to be missing many of the
important recent (statistical) developments, like kernel methods and graphical
models. Also, it's $146 USD!!

- Jordan,
"An introduction to probabilistic graphical models"
(unpublished draft)

This has great material, but much of it is too advanced
for an introductory class on machine learning.

- Koller and Friedman,
"Bayes nets and beyond"
(unpublished draft)

This is similar to Jordan's book, but from an AI perspective,
but again the focus is not really on machine learning.

- Haykin,
"Neural networks"

I hear this is good, but have never read it.