“Not only defended but also applied”

On page 124 of his superb Introduction to Probability Theory book (volume 1), William Feller has this strange remark about Bayesian inference:

“Unfortunately Bayes’ rule has been somewhat discredited by metaphysical applications of the type described above. In routine practice, this kind of argument can be dangerous. A quality control engineer is concerned with one particular machine and not with an innite population of machines from which one was chosen at random. He has been advised to use Bayes’ rule on the grounds that it is logically acceptable and corresponds to our way of thinking. Plato used this type of argument to prove the existence of Atlantis, and philosophers used it to prove the absurdity of Newton’s mechanics. In our case it overlooks the circumstance that the engineer desires success and that he will do better by estimating and minimizing the sources of various types of errors in predicting and guessing.The modern method of statistical tests and estimation is less intuitive but more realistic. It may be not only defended but also applied.”

When we were discussing about this great book, Andrew Gelman pointed out to me this strong dismissal of Bayesian techniques (note that I had overlooked so far) and, given that it is still quoted as an argument against a Bayesian approach to inference, we ended up writing [well, mostly Andrew!] a short note on the motivations and implications of this remark, now published on arXiv. One of the points is that Feller’s sentence has the interesting feature that it is actually the opposite of the usual demarcation: typically it is the Bayesian who makes the claim for inference in a particular instance and the frequentist who restricts claims to infinite populations of replications. Another point is the naïve faith in the classical Neyman-Pearson theory to solve practical problems in statistics.

Actually, Persi Diaconis took a (deeper) look at Feller’s stance as well, as mentioned in this review of Jaynes’s Probability Theory. Using Amazon Look Inside tool,  I spotted Feller being mentioned more than 30 times in Jaynes’s book, one of the best quotes being “The date was 1956 when the author met Willy Feller“! More to the point, Jaynes identifies Feller’s dismissal of the “old wrong ways” (volume 2, p.76), which is to be opposed to the “modern method” above. (Persi Diaconis and Susan Holmes also wrote a nice piece entitled “A Bayesian peek into Feller volume 1” that does not relate directly to this issue.) In a loosely related point, Persi’s warning that he sees “a strong trend against measure theory in modern statistics departments: [he] had to fight to keep the measure theory requirement in Stanford’s statistics graduate program“, to which I completely subscribe, should be heard more widely…