Jaynes’ re-read

On many technical issues we disagree strongly with de Finetti. It appears to us that his way of treating infinite sets has opened up a Pandora’s box of useless and unecessary paradoxes.”  E.T. Jaynes, PT, p.xxi

On Friday, despite the cancellation of the reading seminar on Jaynes’ Probability Theory, I completed my slides on Chapters 4 (Elementary hypothesis testing) to 14 (Simple applications of decision theory), plus of course Chapter 20 (Model comparison). I skipped Chapter 15 (Paradoxes of probability theory), despite its extensive and exciting coverage of the marginalisation paradoxes which saw Jaynes opposing David, Stone, and Zidek (and even the whole Establishment, page 470), as it would have taken me another morning at the very least… (Next year, maybe, if the seminar resumes?!)

There are no really trustworthy standards of rigor in a mathematics that embraced the theory of infinite sets. E.T. Jaynes, PT, p.xxvii

Except for the incomprehensible shots at formalised mathematics (Bourbakism), measure theory (as in Bertrand’s paradox) and Feller (although the later’s anti-Bayes stance  may have induced these attacks), I found the book quite pleasant and mostly in tune with my perception of Bayesian statistics (if strong on the militant side!). Jaynes did not think much of Bayes himself (an amateur!, on page 112), considering that Laplace had done much more to establish Bayesian-ism, and he clearly is a staunch supported of Jeffreys, if not of de Finetti. (Curiously, he also seems to hold Keynes in high respect, despite the later’s dismissal of Laplace.) Given that the book spans 40 years, it is quite modern in most aspects when compared with, Jeffreys’ or Keynes’ say. (For instance, the chapters on Decision theory include coverage of Jim Berger’s book.) It could therefore be read by anyone today, including students, and it would make for a good if challenging graduate textbook. Of course, if I was to teach from it, I would have my students go carefully over the anti-measure rants in order to see the limitations of a derivation of continuous variables solely based on limits. (Corey Yanofsky pointed out to me both a re-evaluation of the measure theoretic derivations in Chapter 15  and a list of erratas in Probability Theory by Kevin S. Van Horn.)

Inference must depend on the data that was observed, not on data sets that might have been observed but were not.” E.T. Jaynes, PT, p.167

Some of the most innovative points in the book are—besides Jaynes’ coherence throughout the book—(i) the coverage of testing before estimation, which makes complete sense from a decision-theoretic viewpoint; (ii) the proposal of weaker forms of sufficiency (even though examples are missing); (iii) the constant relation to Jeffreys’ principles (and not only priors); (iv) the well-oiled uncovering of the maxent principle, including the role played by Gibbs (even though the extension to the continuous case cannot produce a universal principle for picking the reference measure); (v) a very decent coverage of Decision theory (even though Jaynes was visibly no a great fan of the approach); (vi) an interesting move from Bayes’ factors to Ockham’s factors, replacing marginal likelihoods for model choices with the rescaled version

\left.\int \frac{L_i(\theta_i|x)}{\max_\theta L_i(\theta|x)}p(\theta_i|{\mathfrak M}_i)\text{d}\theta_i\right.

for each model (the only example provided in the book (page 613) is not conclusive, so I need to check on a simple Poisson/negative binomial example to understand the impact of the rescaling); and (vii) a very convincing explanation as to why Bayes factors are automatic Ockham’s razors!

Some 650 years ago the Franciscan monk William of Ockham perceived the logical error in the mind projection fallacy.” E.T. Jaynes, PT, p.601

All in all, going through Probability Theory was a very pleasant experience, even though I did not happen to find enough material for writing an equivalent to my re-reads of both Jeffreys’ and Keynes‘ on the past years. (But having had the reading seminar delivered could have changed this!)  Although I did not check the state of the manuscript (available here) when E.T Jaynes died in 1998, I must add that I quite impressed by the amount of work done by Larry Bretthorst and other friends for editing this manuscript into a perfectly finished and so much enjoyable book.

One Response to “Jaynes’ re-read”

  1. […] paper. Whose points I cannot discuss for lack of time, but with an interesting mention of Jaynes as missing in Senn’s […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: