Discussion paper on Search for Certainty

Following our various comments on Krzysztof Burdzy’s The Search for Certainty, Bayesian Analysis has now published the whole series as a discussion paper. The set of comments by Krzysztof Burdzy of my review is of course the most entertaining. (Some of its ingredients can be found in the earlier posted comments of his, as well as in comments on Andrew Gelman’s blog.) Indeed I strongly suggest reading those comments as some are genuinely hilarious. (I still do not understand, as an aside, why Krzysztof Burdzy values so highly this pretentious book by Douglas Hofstadter, Gödel, Escher, Bach: An Eternal Golden Braid, which still ranks 2000th on Amazon… Or maybe the pique at my denomination of vulgarisation is another level of sarcasm!) In conclusion, I think the amount of discussion spent (so far) on Burdzy’s The Search for Certainty is a good enough reply to his question, why did Robert bother to write his review?

A critical assessment now published

The review of Krzysztof Burdzy’s The Search for Certainty has been posted on Bayesian Analysis today. Given the submission date, this makes for an ultrafast acceptance and publishing! Thanks for the editorial team for this speedy processing. The additional comments by Andrew Gelman, Krzysztof Burdzy, and Larry Wasserman should follow soon. I am not sure I will be allowed for a reply to these comments, since they mostly state different opinions rather than facts about the book.

Review in

My revised review of Krzysztof Burdzy’s The Search for Certainty has been accepted by Bayesian Analysis:

Editor's Report on Submission 2010-Jan-27-01-R1 to Bayesian Analysis,
The Search for Certainty: A critical assessment by C. Robert
I am pleased to accept this article for publication. The next step will be to move on to the publication process.

Now, the remaining open question is whether or not Bayesian Analysis will also publish the comments that were posted on Andrew’s blog.

arXives

Yesterday, I finally arXived my notes on Keynes’ book A Treatise On Probability, but, due to the new way the arXiv website operates, there is no indication of the page associated with the submitted paper before it gets accepted and I cannot thus prepare an Og’ entry until this acceptance, wasting perfect timing! Anyway, this is the first draft of the notes and it has not yet been submitted to a journal. As the new user interface on the arXiv webpage now displays all past papers, I added a category on our 2007 Annals paper with Randal Douc, Arnaud Guillin and Jean-Michel Marin, which means it appeared again in today’s list…

Today I completed my revision of the review of Burdzy’s The Search for Certainty over for Bayesian Analysis, so the new version will be on arXiv tomorrow morning. The changes are really minor as Bayesian Analysis mostly requested smoothing down my criticisms. I also added a few more quotes and some sentences in the conclusion. I wonder if this paper will appear with a discussion, since three are already written!

At last, let me point out three recent interesting postings on arXiv if I do not have time to discuss them more in depth, one by Peter Green on Colouring and breaking sticks: random distributions and heterogeneous clustering, one by Nicolas Chopin, Tony Lelièvre et Gabriel Stolz on Free energy methods for efficient exploration of mixture posterior densities, and one by Sophie Donnet and Jean-Michel Marin on An empirical Bayes procedure for the selection of Gaussian graphical models.

Another review of Search for Certainty

The best thing about this book is that it will offend and annoy both frequentists and subjectivists. I implore my friends on both sides of the philosophical divide to read the book with an open mind.

Our comments, Andrew‘s and mine‘s, led Larry Wasserman to read Krzysztof Burdzy’s The Search for Certainty to make his own opinion and Andrew just posted Larry’s review. The review is highly positive, arguing that “this is an interesting and important book” and that “Burdzy makes a convincing case that the philosophy of probability is a complete failure“. While remaining utterly unconvinced (that the book has any bearing on the philosophical foundations of Statistics),  I will not engage here into another debate about The Search for Certainty as the readers can check for themselves the strength of Larry’s arguments. Needless to say, I cannot be convinced into redefining probability as an experimental science where Burdzy’s five laws would replace Kolmogorov’s three axioms…

Comments on the reply

It is quite nice that Professor Burdzy replied to our critical analyses of his book The Search for Certainty in such a kind way, but, overall, my opinion about the book is basically unchanged, just as Andrew’s. The book does not bring much to the way I perceive Statistics nor to the reasons for which I favour Bayesian over frequentist analysis. So, just as clearly, I am not interested in (i) (philosophical) criticisms of von Mises and de Finetti philosophical bases, in (ii) a new “scientific theory of probability”, or in (iii) educating scientists about philosophical theories of probability. ’tis not to say that I cannot launch into a (pseudo-)philosophical defence of the Bayesian approach at times, but I then focus on the coherence of the Bayesian principles and on their fundamental  interaction with decision, taking for granted the measure-theoretic foundations of probability objects…

I think the other “Facts” in the reply are basically a matter of imprecision in my text or in Burdzy’s, which could be settled by a revision of my paper (and the choice of a less unlikely picture by the publisher!). The same applies to the “Opinion” part, as, again, I do not think The Search for Certainty has any bearing on statistical practice (and I agree with Professor Burdzy that most statisticians do not care much about those philosophical issues). I find the remark about catholic theology of interest in that it implies a coherent and unique (or dual) basis for the philosophical justification of probability. One most exciting aspect about statistics—and one that makes the field so distinct from mathematics—is on the opposite that the same problem can be tackled in many different ways, based on very different premises. There is no “Vatican doctrine”, thank God—easy pun, I know!

Anyway, I hope the sum of those discussions can eventually make it into a (BA?) discussion paper!

A reply from Krzysztof Burdzy

Professor Burdzy sent us this reply to the critical analysis of his book, with his authorisation to post it on-line. Since it is quite long already, I will discuss the points in a later post.

Dear Professors Gelman and Robert,

First of all, thank you for reading my book The Search for Certainty. On the Clash of Science and Philosophy of Probability.

I hope it is OK if I reply to both “The Search for Certainty: a critical assessment” by Christian Robert (arXiv:1001.5109) and a post by Andrew Gelman in a single letter. I will refer to these reviews as [CR] and [AG]. I will refer to my book as [KB].

Since I referred to the foundations of probability as “one of the greatest intellectual failures of the twentieth century”, I should not have expected an enthusiastic reaction from the community or probabilists, statisticians and philosophers. Hence, your criticism is not a great surprise for me. In this reply, I will try to avoid opinions, as much as it is possible in this area—we obviously have different opinions and we all expressed them in public. I will try focus on facts, because this may help the readers of your reviews and the readers of my book.

FACTS

1. I witnessed the following event.  Wilfrid Kendall was asked a question at the end of a talk at a conference. He said “My answer will be very aggressive. I totally agree with you.” (He followed with more substantial remarks). My reaction to your reviews will be very aggressive—I totally agree with you. Well, to be honest, I totally agree on one point: “… [the book does not make] a significant contribution to the foundations of statistical inference in general and of Bayesian analysis in particular.” ([CR] p. 7). On page vii of [KB], I clearly state my three intellectual goals: (i) Criticism of von Mises and de Finetti, (ii) Presentation of my “scientific theory or probability”, and (iii) Education of scientists about philosophical theories of probability. The subtitle of [KB] (at least implicitly) indicates that the book is about the miscommunication between philosophers and scientists. On p. 199 of [KB] I say that it is not my ambition to reform statistics. I would have never stated my intellectual goals as a desire to make “a significant contribution to the foundations of statistical inference in general and of Bayesian analysis in particular.”

2. On p. 2 of [CR], there is a quote “no probability statement is falsifiable in any sense” (p. 22 of [KB]). This quote is referred to as “his falsifiability criterion”. Since the word “Popper” comes later in the sentence than the word “author”, the reader of [CR] may think that this quote is from Popper. In fact, [KB] contains a quote from Popper on p. 18 that I interpret as saying that events of very high probability ARE falsifiable. If the reader interprets “his falsifiability criterion” as a criterion proposed by KB, then this is false even more! I attribute the claim that “no probability statement is falsifiable in any sense” (p. 22 of [KB]) to de Finetti. I may be write or wrong, but I attributed this philosophical claim neither to Popper nor to me.3. On p. 2 of [CR] we find “Indeed, this is, in my opinion, the whole extent of the support for the criticism contained in the book about the failure of both von Mises’ and de Finetti’s theories.” This sentence in [CR] refers to Popper’s theory. Chapters 5 and 7 of [KB] are devoted to detailed criticism of the theories of von Mises and de Finetti. Both chapters stress (i) the fact that the two philosophical theories are weak, and (ii) the technical ideas (“collectives” and “consistency”) are pretty much useless. My arguments may be right or wrong but I do not see any of Popper’s philosophy in these chapters. Perhaps I used it at a subconscious level …

4. The following comes from p. 6 of [CR]: `Similarly, de Finetti’s over-exploited statement “Probability does not exist” cannot be seen as the core principle of many (any?) Bayesian statisticians and I certainly do not relate to his all-subjective stance for conducting Bayesian inference.’ This may suggest to the reader of [CR] that [KB] failed to notice that “Probability does not exist” is not universally or widely accepted by Bayesian statisticians. In fact, this is precisely what I argue in 8.1.1 and 8.1.2 of [KB].

5. On p. 6 of [CR], the following quote from [KB] is given: ““subjective theory does not provide any justification for the use of the Bayes theorem”. A very similar quote appears in the second to last paragraph of [AG]. I find it amusing that [CR] calls my claim “rather nonsensical” while [AG] says “fine by me, but of course nothing new.” I can’t win—f my claim is true, I failed to notice that it was well known; if it is false, I failed to notice that it was nonsensical.

6. A suggestion for improvement. On p. 7 of [CR], there are references to “dices”. The singular is “die” and the plural is “dice”.

7. The analysis of the cover picture on p. 7 of [CR] is unnecessary—there is little profound philosophy behind it. The publisher asked me for an image so I borrowed 7 dice from my son and I arranged them by hand so that they would form a more or less balanced picture. The message in the picture was supposed to be that there are everyday experiments that can be performed by anyone that involve events of very small probability, i.e., (such events are complements of “predictions”). I did not arrange the dice so that they would all display fours or fives on the side on purpose. I seem to recall that some dice had imperfections so I turned them so that the imperfections would not show up in the photo. The book cover is a highly transformed version of my photo, prepared by an artist hired by the publisher.

OPINION

8. On p. 3 of [CR] there is a remark on the “lack of involved examples”. I do not see how involved examples could have changed the perception of my book. I do not have any new frequency or Bayesian statistical methods to propose. I believe that all statisticians, probabilists and other scientists should use (L1)-(L5) to teach the basic principles of probability. Would involved examples make (L1)-(L5) more attractive to you?

9. On p. 6 of [CR] I am criticized for concentrating “on two very specific entries to frequentism and subjectivism, namely von Mises’ and de Finetti’s, respectively, while those are not your average statistician’s references.” I explain on p. 12 of [KB] why I chose the theories of von Mises and de Finetti. Let me repeat and rephrase my reasons. These two theories are more or less complete and more or less logically consistent philosophical theories created by people who are recognized by philosophers as the leading figures in frequency and subjective currents of philosophy of probability. De Finetti and von Mises wrote books that I could study and criticize. There is an implicit suggestion in [CR] that I have chosen the wrong theories to criticize and that statisticians apparently use other philosophical theories. As far as I can tell, statisticians have a multitude of philosophical opinions but that does not mean that these opinions add up to a logically consistent theory. If I ever want to criticize the Catholic theology, I will use the official Vatican doctrine as the target of my criticism. It is unquestionably true that the real religious beliefs of Catholics are quite often different from the Vatican doctrine, but the union of all beliefs of all Catholics does not add up to a logically consistent philosophy (as far as I can tell).

10. On p. 6 of [CR], the author admits that he had never encountered “collectives”. First of all, I am happy that my book achieved one of its three main intellectual goals—education. Second, this admission should be food for thought. In philosophical literature, von Mises is unquestionably considered the main (most significant) representative of the frequency philosophy. The confession about not knowing collectives illustrates one of the main points of [KB]—statisticians and philosophers do not communicate enough in view of the depth of philosophical controversies surrounding probability.

With best regards,

Chris
(Krzysztof Burdzy)