“Not only defended but also applied”: The perceived absurdity of Bayesian inference
After a first unsuccessful attempt at publishing a note on the great Willliam Feller’s dismissive attitude towards Bayesian statistics, in An Introduction to Probability Theory and Its Applications, and more broadly about misconceptions on Bayesianism, jointly with Andrew Gelman, last year, we have rewritten some of it and resubmitted to The American Statistician. It has also been re-arXived. Here is the abstract:
Abstract. The missionary zeal of many Bayesians has been matched, in the other direction, by a view among some theoreticians that Bayesian methods are absurd—not merely misguided but obviously wrong in principle. We consider several examples, beginning with Feller’s classic text on probability theory and continuing with more recent cases such as the perceived Bayesian nature of the so-called doomsday argument. We analyze in this note the intellectual background behind various misconceptions about Bayesian statistics, without aiming at a complete historical coverage of the reasons for this dismissal.
Xi'an's Og 
April 16, 2012 at 3:16 pm
[…] positive and encouraging review by The American Statistician of our paper with Andrew Gelman on Feller’s misrepresentation of Bayesian statistics in the otherwise superb Introduction to Probability Theory , we have […]
October 17, 2011 at 2:59 pm
There are a lot of confusing arguments on both sides. My own working view is that Bayesian methods are often unreliable if used to infer, but extremely valuable if used to generate hypotheses to be tested. The Bletchley Park example is paradigmatic. They used Bayes’ rule, but they weren’t ‘Bayesian’ in any bad sense.
October 14, 2011 at 6:02 pm
I came across the doomsday argument in Stephen Baxter’s book “Space” (book one of the Manifold trilogy) where it was presented explicitly as an example of Bayesian inference.
In the context of the story, the purpose of the doomsday argument is to persuade the protagonist that a catastrophic population/resource collapse is imminent so he must start his own space programme now, rather than later when it will be too late.
I didn’t see the doomsday argument as either Bayesian or frequentist. I just thought it was wrong. The argument invites you to consider a future doomsday event that wipes out (most of) humanity and then work out how close you lie to the event given that you are one of the billions of humans who have ever lived. But conditioning on future, unobserved events is forbidden in a proper probabilistic framework for stochastic processes, and with good reason. I just thought that the argument illustrates the absurd conclusions that one reaches when one tries this, like the paradox of the unexpected hanging, but perhaps I did not think hard enough about the problem.
October 14, 2011 at 2:30 am
I see some problems with the references in the arXiv article:
Several authors mentioned in the paper don’t show in the reference list, such as Jaynes (footnote p.2), Rubin (p.5), Carter (p.6) and Neal (footnote p.6).
The third author of the Berger paper is missing.
Keynes in the reference list should be 1921, not 1920.
October 14, 2011 at 7:13 am
Thank you for checking those.
October 13, 2011 at 7:56 am
I cannot judge for philosophy of sciences, however I think you are at least making an historical contresens in terms of the “bullying and intimidation”. E.g., Fisher, Neyman and Pearson, Lehmann. E.g., up to the 80’s, it was difficult to publish Bayesian papers and some were rejected on the sole basis they were using Bayesian techniques. E.g., I remember being turned down at a job application in 1991 on the sole reason of my Bayesian tendencies. More globally, I do not see Bayesians involved in a conspiracy of “bullying and intimidation”! First, they would have to agree among themselves; second, the prevalence of non-Bayesian techniques in applied statistics for all fields shows the “bullying and intimidation”, if any, is clearly unsuccessful!
October 13, 2011 at 5:22 am
I’m sorry to say, but the truth is that the BayesianBrotherhood—as a group at least– have taken the lead in bullying and intimidation, at least in philosophy of science, but to some extent in statistics as well. That is why we frequentists are “in exile”! I find it especially disappointing in philosophy to see how afraid some are to raise their criticisms openly. Here I always thought we became philosophers because we couldn’t be bought….
October 13, 2011 at 1:05 pm
Mayo:
What Christian said. I can well believe that a simplistic Bayesianism could be dominant among philosophers of science, but my impression is that statistics is much more pluralistic, that in statistics it’s all about “what works.” But it wasn’t always that way! As the quotes from Feller indicate, statistics has a tradition of ignorant Bayes-bashing, a tradition that we thought worth exploring a bit.