May I believe I am a Bayesian?!

…the argument is false that because some ideal form of this approach to reasoning seems excellent n theory it therefore follows that in practice using this and only this approach to reasoning is the right thing to do.” Stephen Senn, 2011

Deborah Mayo, Aris Spanos, and Kent Staley have edited a special issue of Rationality, Markets and Morals (RMM) (a rather weird combination, esp. for a journal name!) on “Statistical Science and Philosophy of Science: Where Do (Should) They Meet in 2011 and Beyond?” for which comments are open. Stephen Senn has a paper therein entitled You May Believe You Are a Bayesian But You Are Probably Wrong in his usual witty, entertaining, and… Bayesian-bashing style! I find it very kind of him to allow us to remain in the wrong, very kind indeed…


Now, the paper somehow intersects with the comments Stephen made on our review of Harold Jeffreys’ Theory of Probability a while ago. It contains a nice introduction to the four great systems of statistical inference, embodied by de Finetti, Fisher, Jeffreys, and Neyman plus Pearson. The main criticism of Bayesianism à la de Finetti is that it is so perfect as to be outworldish. And, since this perfection is lost in the practical implementation, there is no compelling reason to be a Bayesian. Worse, that all practical Bayesian implementations conflict with Bayesian principles. Hence a Bayesian author “in practice is wrong”. Stephen concludes with a call for eclecticism, quite in line with his usual style since this is likely to antagonise everyone. (I wonder whether or not having no final dot to the paper has a philosophical meaning. Since I have been caught in over-interpreting book covers, I will not say more!) As I will try to explain below, I believe Stephen has paradoxically himself fallen victim of over-theorising/philosophising! (Referring the interested reader to the above post as well as to my comments on Don Fraser’s “Is Bayes posterior quick and dirty confidence?” for more related points. Esp. about Senn’s criticisms of objective Bayes on page 52 that are not so central to this discussion… Same thing for the different notions of probability [p.49] and the relative difficulties of the terms in (2) [p.50]. Deborah Mayo has a ‘deconstructed” version of Stephen’s paper on her blog, with a much deeper if deBayesian philosophical discussion. And then Andrew Jaffe wrote a post in reply to Stephen’s paper. Whose points I cannot discuss for lack of time, but with an interesting mention of Jaynes as missing in Senn’s pantheon.)


The Bayesian theory is a theory on how to remain perfect but it does not explain how to become good.” Stephen Senn, 2011

While associating theories with characters is a reasonable rethoretical device, especially with large scale characters as the one above!, I think it deters the reader from a philosophical questioning on the theory behind the (big) man. (In fact, it is a form of bullying or, more politely (?), of having big names shoved down your throat as a form of argument.)  In particular, Stephen freezes the (Bayesian reasoning about the) Bayesian paradigm in its de Finetti phase-state, arguing about what de Finetti thought and believed. While this is historically interesting, I do not see why we should care at the praxis level. (I have made similar comments on this blog about the unpleasant aspects of being associated with one character, esp. the mysterious Reverent Bayes!) But this is not my main point.

…in practice things are not so simple.” Stephen Senn, 2011

The core argument in Senn’s diatribe is that reality is always more complex than the theory allows for and thus that a Bayesian has to compromise on her/his perfect theory with reality/practice in order to reach decisions. A kind of philosophical equivalent to Achille and the tortoise. However, it seems to me that the very fact that the Bayesian paradigm is a learning principle implies that imprecisions and imperfections are naturally endowed into the decision process. Thus avoiding the apparent infinite regress (Regress ins Unendliche) of having to run a Bayesian analysis to derive the prior for the Bayesian analysis at the level below (which is how I interpret Stephen’s first paragraph in Section 3). By refusing the transformation of a perfect albeit ideal Bayesian into a practical if imperfect bayesian (or coherent learner or whatever name that does not sound like being a member of a sect!), Stephen falls short of incorporating the contrainte de réalité into his own paradigm. The further criticisms found about prior justification, construction, evaluation (pp.59-60) are also of that kind, namely preventing the statistician to incorporate a degree of (probabilistic) uncertainty into her/his analysis.

In conclusion, reading Stephen’s piece was a pleasant and thought-provoking moment. I am glad to be allowed to believe I am a Bayesian, even though I do not believe it is a belief! The praxis of thousands of scientists using Bayesian tools with their personal degree of subjective involvement is an evolutive organism that reaches much further than the highly stylised construct of de Finetti (or of de Finetti restaged by Stephen!). And appropriately getting away from claims to being perfect or right. Or even being more philosophical.

9 Responses to “May I believe I am a Bayesian?!”

  1. […] Max Albert Says: January22, 2012 at 4:08 pm […]

  2. […] have given bloglinks, I’ll just note them here and give a few brief responses: Christian Robert Mayo’s brief remarks: As I see it, Robert overlooks the most difficult challenge Senn […]

  3. […]  Max Albert Says: January22, 2012 at 4:08 pm […]

  4. […] Xi’an’s Og is still Bayesian even if Stephen Senn thinks otherwise. […]

  5. At the beginning of the piece I wrote and to which Xi’an refers I referred to George Barnard’s advice that one should, ‘be familiar with the four great systems of inference,’ and at the end of the piece I wrote ‘This leaves us, I maintain, with applied Bayesian analysis as currently practiced as one amongst a number of rough and ready tools that we have for looking at data. I think we need many such tools because we need mental conflict as much as mental coherence to spur us to creative thinking. When different
    systems give different answers it is a sign that we need to dig deeper’ (omitting, unintentionally the full stop to which Xi’an refers).

    It is thus clear that I am not hostile to applied Bayesian inference. I am hostile to extreme hype about such an approach. (The current literature on meta-analysis is a case in point where the most ridiculous claims circulate about an approach that produces pretty much what iteratively re-weighted least squares does.)

    In fact, I don’t really believe that Bayesian inference is wrong. (I am tempted to quip ‘it is not even wrong’.,,, and I suppose I have just yielded to temptation :-)) What I was claiming is that it is not usually in practice quite what it is claimed to be and if you tried to delude yourself that what passes for Bayesian analysis (often very good and interesting as applied statistics) really was what it is often claimed to be, you could be led astray.

    In theory, Bayesian analysis ought to be a very promising way of dealing with nuisance parameters. In practice I can only characterise much of the work in this direction as disappointing.

    If I can quote a great Bayesian quoting another, ” I agree with Herman Rubin’s remark, at the Waterloo conference on scientific inference in 1970, that a good Bayesian does better than a non-Bayesian but a Bad Bayesian gets clobbered.” Jack Good, Good Thinking, P139

  6. Max Albert Says:

    Dear xi’an,

    I am a bit surprised that you consider “Rationality, markets and morals” as a “rather weird combination”. It is a classical combination of topics in economics and philosophy. And it seems to me that just now all the world is talking and writing about it.

    Of course, we very much appreciate the fact that you have looked at our sprecial issue (or special topic, as we call it, since we have no issues). Reading, in addition, some of our other papers may convince you that our journal title makes sense.

    Best wishes

    Max Albert

    • Sorry for the pun: for me, markets, rationality, and morals (understood as the French translation into morale) do not seem to get together very well, esp. in the current times! This being said, I appreciate this special topic and do not mean in any way that the journal does not make sense! Again, sorry for the pun…

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: