Bayes factor approximations

Following Andrew Gelman by a few days, we have now finished with Jean-Michel Marin a survey on importance sampling methods for Bayes factor approximations which is our contribution to a book of essays, Frontiers of Statistical Decision Making and Bayesian Analysis, in honour of Jim Berger, published in conjunction with a conference with the same name, next March, in San Antonio, Texas. When I was young, so much younger than today, I could not see the point of those celebration conferences. I am not sure I have gotten any wiser with age, but I do see a point in celebrating Jim Berger! First, Jim has contributed and is contributing immensely to statistical science from many respects, first and foremost in grounding Bayesian analysis within decision theory. His book Statistical Decision Theory and Bayesian Analysis has influenced many readers and this is certainly the (statistics) book that has had the most influence upon me, as can be seen in The Bayesian Choice. His works on Bayesian testing are fundamental for the understanding of the specific nature of this approach and his earlier advances in shrinkage estimations have determined the field. On a more personal basis, Jim Berger has always been immensely helpful and supportive, starting with his invitation to Purdue in 1987 when I was fresh out of my PhD… So, yes indeed !, I am now quite glad to see this conference in his honour taking place so that we can all thank him!

The survey with Jean-Michel, now posted on arXiv, is linked with some older work of Jim in that he was studying importance sampling methods with Man-Suk Oh at the time I was visiting Purdue, in the pre-MCMC days. The toolkit for approximating Bayes factors has considerably grown in the past twenty years but importance sampling, albeit in a refined format, remains a central methodology. In the probit example we provide in the survey, this is actually the best method, thanks to a very good approximation of the posterior by the normal asymptotic approximation to the likelihood, as already shown in the entry on Savage-Dickey posted a few days ago. In this same example, the harmonic mean estimator of the marginal is doing surprisingly well, again because of the fit of the approximation.