## “Not only defended but also applied”

Posted in Statistics with tags , , , , on June 30, 2010 by xi'an

On page 124 of his superb Introduction to Probability Theory book (volume 1), William Feller has this strange remark about Bayesian inference:

“Unfortunately Bayes’ rule has been somewhat discredited by metaphysical applications of the type described above. In routine practice, this kind of argument can be dangerous. A quality control engineer is concerned with one particular machine and not with an in nite population of machines from which one was chosen at random. He has been advised to use Bayes’ rule on the grounds that it is logically acceptable and corresponds to our way of thinking. Plato used this type of argument to prove the existence of Atlantis, and philosophers used it to prove the absurdity of Newton’s mechanics. In our case it overlooks the circumstance that the engineer desires success and that he will do better by estimating and minimizing the sources of various types of errors in predicting and guessing.The modern method of statistical tests and estimation is less intuitive but more realistic. It may be not only defended but also applied.”

When we were discussing about this great book, Andrew Gelman pointed out to me this strong dismissal of Bayesian techniques (note that I had overlooked so far) and, given that it is still quoted as an argument against a Bayesian approach to inference, we ended up writing [well, mostly Andrew!] a short note on the motivations and implications of this remark, now published on arXiv. One of the points is that Feller’s sentence has the interesting feature that it is actually the opposite of the usual demarcation: typically it is the Bayesian who makes the claim for inference in a particular instance and the frequentist who restricts claims to infinite populations of replications. Another point is the naïve faith in the classical Neyman-Pearson theory to solve practical problems in statistics.

Actually, Persi Diaconis took a (deeper) look at Feller’s stance as well, as mentioned in this review of Jaynes’s Probability Theory. Using Amazon Look Inside tool,  I spotted Feller being mentioned more than 30 times in Jaynes’s book, one of the best quotes being “The date was 1956 when the author met Willy Feller“! More to the point, Jaynes identifies Feller’s dismissal of the “old wrong ways” (volume 2, p.76), which is to be opposed to the “modern method” above. (Persi Diaconis and Susan Holmes also wrote a nice piece entitled “A Bayesian peek into Feller volume 1″ that does not relate directly to this issue.) In a loosely related point, Persi’s warning that he sees “a strong trend against measure theory in modern statistics departments: [he] had to fight to keep the measure theory requirement in Stanford’s statistics graduate program“, to which I completely subscribe, should be heard more widely…

## Magical risotto!

Posted in Kids, Travel with tags , , on June 29, 2010 by xi'an

When Marilena Barbieri told me in Padova that she could make [good] risotto in her microwave oven, I was quite skeptical, both because I avoid using the microwave for cooking and because risotto is one of my favourite dishes! (Did I ever mention this outworldly risotto di seppie a la Veneziana that was so good it was almost a desert, served at an outrageous price at the Hostaria da Franz in Venezia!) However, Marilena sent me her recipe and I tried it last week. Now, I have to acknowledge that it is an excellent recipe, to the point that I will use it in the future (as long as my kids do not get tired of it, as happened with fajitas, crumbles and quiches in the past years…). Here is the core of Marilena’s recipe:

2 deciliters/cups hot chicken or vegetable broth/stock (or soup cube)
1 deciliter/cup arborio rice
1/2 teaspoon salt
chopped onion
risotto ingredients (e.g. minced vegetables or truffle bits)
oil or butter

Start with oil or butter and chopped onion in a covered bowl. Put in the microwave for 2 minutes. Add the other ingredients. Cover and put in the microwave for 2 additional minutes (or more, depending on the kind of vegetable you add). Then add broth, rice and cook at 750W for 3 minutes, then at 350W for 10 additional minutes. Stir only at the end. If the risotto looks too liquid, keep cooking one minute at a time until it tastes al dente.

## Two U of Toronto professors die on Denali

Posted in Mountains, Running, University life with tags , , , , on June 28, 2010 by xi'an

When looking for Radford Neal’s page in the CS department of U of T, I came upon this very sad item of news, namely that (climber and) Professor Avner Magen have been killed last month in an avalanche on Denali, along with Professor Andrew Herzenberg from the Faculty of Medicine at University Health Network. There is a donation site to support Avner’s family.

## “Bayesian model comparison in cosmology” on-line

Posted in Statistics, University life with tags , , , , , on June 27, 2010 by xi'an

I actually missed the piece of information that our our paper “Bayesian model comparison in cosmology with Population Monte Carlo” has been accepted by Monthly Notices of the Royal Astronomical Society on March 1! The abstract if not the whole paper is available on-line as early-view since mid-April… This is my last paper published in collaboration with the cosmologists of the Ecosstat 2005-2009 ANR program. Hopefully not the end of our collaboration as this was a very fruitful experience from my viewpoint, which happened to coincide with the golden years of population Monte Carlo, just as the Misgepop ANR program launched our foray into ABC methods. (In case you are unaware of the link, Scott Sisson has a twitter page posting news on ABC methods.)

## Another harmonic mean approximation

Posted in R, Statistics with tags , , , , , , on June 27, 2010 by xi'an

Martin Weinberg posted on arXiv a revision of his paper, Computing the Bayesian Factor from a Markov chain Monte Carlo Simulation of the Posterior Distribution, that is submitted to Bayesian Analysis. I have already mentioned this paper in a previous post, but I remain unconvinced of the appeal of the paper method, given that it recovers the harmonic mean approximation to the marginal likelihood… The method is very close to John Skilling’s nested sampling, except that the simulation is run from the posterior rather than from the prior, hence the averaging on the inverse likelihoods and hence the harmonic mean connection. The difficulty with the original (Michael Newton and Adrian Raftery’s) harmonic mean estimator is attributed to “a few outlying terms with abnormally small values of” the likelihood, while, as clearly spelled out by Radford Neal,  the poor behaviour of the harmonic mean estimator has nothing abnormal and is on the opposite easily explainable.

I must admit I found the paper difficult to read, partly because of the use of poor and ever-changing notations and partly because of the lack of mathematical rigour (see, e.g., eqn (11)). (And maybe also because of the current heat wave.) In addition to the switch from prior to posterior in the representation of the evidence, a novel perspective set in the paper seems to be an extension of the standard harmonic mean identity that relates to the general expression of Gelfand and Dey (1994, Journal of the Royal Statistical Society B) when using an indicator function as an instrumental function. There is therefore a connection with our proposal (made with Jean-Michel Marin) of considering an HPD region for excluding the tails of the likelihood, even though the set of integration is defined as “eliminating the divergent samples with $L_i \ll 1$“. This is essentially the numerical Lebesgue algorithm advanced as one of two innovative algorithms by Martin Weinberg. I wonder how closely related the second (volume tesselation) algorithm is to Huber and Schott’s TPA algorithm, in the sense that TPA also requires a “smaller” integral….