A reply from Krzysztof Burdzy
Professor Burdzy sent us this reply to the critical analysis of his book, with his authorisation to post it on-line. Since it is quite long already, I will discuss the points in a later post.
Dear Professors Gelman and Robert,
First of all, thank you for reading my book The Search for Certainty. On the Clash of Science and Philosophy of Probability.
I hope it is OK if I reply to both “The Search for Certainty: a critical assessment” by Christian Robert (arXiv:1001.5109) and a post by Andrew Gelman in a single letter. I will refer to these reviews as [CR] and [AG]. I will refer to my book as [KB].
Since I referred to the foundations of probability as “one of the greatest intellectual failures of the twentieth century”, I should not have expected an enthusiastic reaction from the community or probabilists, statisticians and philosophers. Hence, your criticism is not a great surprise for me. In this reply, I will try to avoid opinions, as much as it is possible in this area—we obviously have different opinions and we all expressed them in public. I will try focus on facts, because this may help the readers of your reviews and the readers of my book.
1. I witnessed the following event. Wilfrid Kendall was asked a question at the end of a talk at a conference. He said “My answer will be very aggressive. I totally agree with you.” (He followed with more substantial remarks). My reaction to your reviews will be very aggressive—I totally agree with you. Well, to be honest, I totally agree on one point: “… [the book does not make] a significant contribution to the foundations of statistical inference in general and of Bayesian analysis in particular.” ([CR] p. 7). On page vii of [KB], I clearly state my three intellectual goals: (i) Criticism of von Mises and de Finetti, (ii) Presentation of my “scientific theory or probability”, and (iii) Education of scientists about philosophical theories of probability. The subtitle of [KB] (at least implicitly) indicates that the book is about the miscommunication between philosophers and scientists. On p. 199 of [KB] I say that it is not my ambition to reform statistics. I would have never stated my intellectual goals as a desire to make “a significant contribution to the foundations of statistical inference in general and of Bayesian analysis in particular.”
2. On p. 2 of [CR], there is a quote “no probability statement is falsifiable in any sense” (p. 22 of [KB]). This quote is referred to as “his falsifiability criterion”. Since the word “Popper” comes later in the sentence than the word “author”, the reader of [CR] may think that this quote is from Popper. In fact, [KB] contains a quote from Popper on p. 18 that I interpret as saying that events of very high probability ARE falsifiable. If the reader interprets “his falsifiability criterion” as a criterion proposed by KB, then this is false even more! I attribute the claim that “no probability statement is falsifiable in any sense” (p. 22 of [KB]) to de Finetti. I may be write or wrong, but I attributed this philosophical claim neither to Popper nor to me.3. On p. 2 of [CR] we find “Indeed, this is, in my opinion, the whole extent of the support for the criticism contained in the book about the failure of both von Mises’ and de Finetti’s theories.” This sentence in [CR] refers to Popper’s theory. Chapters 5 and 7 of [KB] are devoted to detailed criticism of the theories of von Mises and de Finetti. Both chapters stress (i) the fact that the two philosophical theories are weak, and (ii) the technical ideas (“collectives” and “consistency”) are pretty much useless. My arguments may be right or wrong but I do not see any of Popper’s philosophy in these chapters. Perhaps I used it at a subconscious level …
4. The following comes from p. 6 of [CR]: `Similarly, de Finetti’s over-exploited statement “Probability does not exist” cannot be seen as the core principle of many (any?) Bayesian statisticians and I certainly do not relate to his all-subjective stance for conducting Bayesian inference.’ This may suggest to the reader of [CR] that [KB] failed to notice that “Probability does not exist” is not universally or widely accepted by Bayesian statisticians. In fact, this is precisely what I argue in 8.1.1 and 8.1.2 of [KB].
5. On p. 6 of [CR], the following quote from [KB] is given: ““subjective theory does not provide any justification for the use of the Bayes theorem”. A very similar quote appears in the second to last paragraph of [AG]. I find it amusing that [CR] calls my claim “rather nonsensical” while [AG] says “fine by me, but of course nothing new.” I can’t win—f my claim is true, I failed to notice that it was well known; if it is false, I failed to notice that it was nonsensical.
6. A suggestion for improvement. On p. 7 of [CR], there are references to “dices”. The singular is “die” and the plural is “dice”.
7. The analysis of the cover picture on p. 7 of [CR] is unnecessary—there is little profound philosophy behind it. The publisher asked me for an image so I borrowed 7 dice from my son and I arranged them by hand so that they would form a more or less balanced picture. The message in the picture was supposed to be that there are everyday experiments that can be performed by anyone that involve events of very small probability, i.e., (such events are complements of “predictions”). I did not arrange the dice so that they would all display fours or fives on the side on purpose. I seem to recall that some dice had imperfections so I turned them so that the imperfections would not show up in the photo. The book cover is a highly transformed version of my photo, prepared by an artist hired by the publisher.
8. On p. 3 of [CR] there is a remark on the “lack of involved examples”. I do not see how involved examples could have changed the perception of my book. I do not have any new frequency or Bayesian statistical methods to propose. I believe that all statisticians, probabilists and other scientists should use (L1)-(L5) to teach the basic principles of probability. Would involved examples make (L1)-(L5) more attractive to you?
9. On p. 6 of [CR] I am criticized for concentrating “on two very specific entries to frequentism and subjectivism, namely von Mises’ and de Finetti’s, respectively, while those are not your average statistician’s references.” I explain on p. 12 of [KB] why I chose the theories of von Mises and de Finetti. Let me repeat and rephrase my reasons. These two theories are more or less complete and more or less logically consistent philosophical theories created by people who are recognized by philosophers as the leading figures in frequency and subjective currents of philosophy of probability. De Finetti and von Mises wrote books that I could study and criticize. There is an implicit suggestion in [CR] that I have chosen the wrong theories to criticize and that statisticians apparently use other philosophical theories. As far as I can tell, statisticians have a multitude of philosophical opinions but that does not mean that these opinions add up to a logically consistent theory. If I ever want to criticize the Catholic theology, I will use the official Vatican doctrine as the target of my criticism. It is unquestionably true that the real religious beliefs of Catholics are quite often different from the Vatican doctrine, but the union of all beliefs of all Catholics does not add up to a logically consistent philosophy (as far as I can tell).
10. On p. 6 of [CR], the author admits that he had never encountered “collectives”. First of all, I am happy that my book achieved one of its three main intellectual goals—education. Second, this admission should be food for thought. In philosophical literature, von Mises is unquestionably considered the main (most significant) representative of the frequency philosophy. The confession about not knowing collectives illustrates one of the main points of [KB]—statisticians and philosophers do not communicate enough in view of the depth of philosophical controversies surrounding probability.
With best regards,