principles of uncertainty

Posted in Books, R, Statistics, University life with tags , , , , , , , , , , , , , , on October 14, 2011 by xi'an

Bayes Theorem is a simple consequence of the axioms of probability, and is therefore accepted by all as valid. However, some who challenge the use of personal probability reject certain applications of Bayes Theorem.”  J. Kadane, p.44

Principles of uncertainty by Joseph (“Jay”) Kadane (Carnegie Mellon University, Pittsburgh) is a profound and mesmerising book on the foundations and principles of subjectivist or behaviouristic Bayesian analysis. Jay Kadane wrote Principles of uncertainty over a period of several years and, more or less in his own words, it represents the legacy he wants to leave for the future. The book starts with a large section on Jay’s definition of a probability model, with rigorous mathematical derivations all the way to Lebesgue measure (or more exactly the McShane-Stieltjes measure). This section contains many side derivations that pertain to mathematical analysis, in order to explain the subtleties of infinite countable and uncountable sets, and the distinction between finitely additive and countably additive (probability) measures. Unsurprisingly, the role of utility is emphasized in this book that keeps stressing the personalistic entry to Bayesian statistics. Principles of uncertainty also contains a formal development on the validity of Markov chain Monte Carlo methods that is superb and missing in most equivalent textbooks. Overall, the book is a pleasure to read. And highly recommended for teaching as it can be used at many different levels. Continue reading

Posted in Statistics with tags , , , , on June 7, 2009 by xi'an

There was a poster by Timothy Wallstrom yesterday night at the O-Bayes09 poster session about marginalisation paradoxes and we had a nice chat about this topic. Marginalisation paradoxes are fascinating and I always mention them in my Bayesian class, because I think they illustrate the limitations of how much one can interpret an improper prior. There is a consequent literature on how to “solve” marginalisation paradoxes, following Jaynes’ comments on the foundational paper of David, Stone and Zidek (Journal of the Royal Statistical Society, 1974), but—and this is where I disagree with Timothy—I do not think they need to be “solved” either by uncovering the group action on the problem (left Haar versus right Haar) or by using different proper prior sequences. For me, the core of the “paradox” is that writing an improper prior as

$\pi(\theta,\zeta) = \pi_1(\theta) \pi_2(\zeta)$

does not imply that $\pi_2$ is the marginal prior on $\zeta$ when $\pi_1$ is improper. The interpretation of $\pi_2$ as such is what leads to the “paradox” but there is no mathematical difficulty in the issue. Starting with the joint improper prior $\pi(\theta,\zeta)$ leads to an undefined posterior if we only consider the part of the observations that depends on $\zeta$ because $\theta$ does not integrate out. Defining improper priors as limits of proper priors—as Jaynes and Timothy Wallstrom do—can also be attempted from a mathematical point of view, but (a) I do not think a global resolution is possible this way in that all Bayesian procedures for the improper prior cannot be constructed as limits from the corresponding Bayesian procedures for the proper prior sequence, think eg about testing, and (b) this is trying to give a probabilistic meaning to the improper priors and thus gets back to the over-interpretation danger mentioned above. Hence a very nice poster discussion!

O’Bayes 09 update

Posted in Statistics with tags , , , on June 4, 2009 by xi'an

The Objective Bayes meeting, O-Bayes09, is starting tomorrow in Philadelphia, at the Wharton Business School. Here are the slides I plan to present in the Jeffreys session, even though I do not see how I can cover the 148 of them in 45 minutes…

This means I will have more editing to do in the plane and tomorrow morning!

Theory of Probability revisited

Posted in Books, Statistics, University life with tags , , , , on April 4, 2009 by xi'an

Last year, I gave an advanced graduate course at CREST about Jeffreys’ Theory of Probability, where I tried to present the many advances contained in the book using modern concepts and notations. With Judith Rousseau and Nicolas Chopin who attended the course, we then decided to make a paper of those notes, because we thought [and think] that (a) this book was truly foundational for the field of Bayesian statistics and (b) a guide to the most relevant parts could be useful to get over the idiosyncrasies of a book published in 1939, that is, seventy years ago… Here are the slides of that course (Warning: huge file!)

The paper was submitted to Statistical Science and it (surprisingly) came back with very minimal requests for changes, which is great of course! We have now resubmitted the revised version, also posted on arXiv. Hopefully, the paper will be discussed as well and the discussion will make Jeffreys’ book even more appealing for new readers. Hopefully, contributors will also attend the Jeffreys’ session I am organising at the O’Bayes 09 meeting next June. It would be nice also if the paper could appear in the 2009 volume of Statistical Science, in order to make it a nice 70th anniversary celebration for this still relevant book. Next year, I may try re-reading and teaching Keynes’ A Treatise On Probability.