Bayes factor approximations

Following Andrew Gelman by a few days, we have now finished with Jean-Michel Marin a survey on importance sampling methods for Bayes factor approximations which is our contribution to a book of essays, Frontiers of Statistical Decision Making and Bayesian Analysis, in honour of Jim Berger, published in conjunction with a conference with the same name, next March, in San Antonio, Texas. When I was young, so much younger than today, I could not see the point of those celebration conferences. I am not sure I have gotten any wiser with age, but I do see a point in celebrating Jim Berger! First, Jim has contributed and is contributing immensely to statistical science from many respects, first and foremost in grounding Bayesian analysis within decision theory. His book Statistical Decision Theory and Bayesian Analysis has influenced many readers and this is certainly the (statistics) book that has had the most influence upon me, as can be seen in The Bayesian Choice. His works on Bayesian testing are fundamental for the understanding of the specific nature of this approach and his earlier advances in shrinkage estimations have determined the field. On a more personal basis, Jim Berger has always been immensely helpful and supportive, starting with his invitation to Purdue in 1987 when I was fresh out of my PhD… So, yes indeed !, I am now quite glad to see this conference in his honour taking place so that we can all thank him!

The survey with Jean-Michel, now posted on arXiv, is linked with some older work of Jim in that he was studying importance sampling methods with Man-Suk Oh at the time I was visiting Purdue, in the pre-MCMC days. The toolkit for approximating Bayes factors has considerably grown in the past twenty years but importance sampling, albeit in a refined format, remains a central methodology. In the probit example we provide in the survey, this is actually the best method, thanks to a very good approximation of the posterior by the normal asymptotic approximation to the likelihood, as already shown in the entry on Savage-Dickey posted a few days ago. In this same example, the harmonic mean estimator of the marginal is doing surprisingly well, again because of the fit of the approximation.

9 Responses to “Bayes factor approximations”

  1. […] nonparametric estimate is entered in an harmonic mean representation we previously exploited in our HPD proposal for evidence […]

  2. […] change to become a chapter on computational techniques for model choice. This means covering intra-model as well as inter-model computational tools like bridge, path, umbrella, nested sampling, harmonic […]

  3. Thank you for making your paper on Bayes factors freely available. As a practitioner who doesn’t know much theory, I found it a very useful introduction.

    However, after reading about various calculation techniques, it still seems that using “defensive” importance sampling on a 50/50 mix of the posterior and prior distributions should give good results and doesn’t require any custom coding. It seems this idea has gotten rejected because I only see it in older articles. Do you know what’s wrong with it?


    • There is nothing wrong with defensive sampling and I still consider the paper by Hesterberg (1995) an important contribution to the field. However, in practice, finding a defensive distribution with fat tails that manages to contribute to the defensive weight is challenging: using the prior is not necessarily a good solution in that most simulations from the prior have zero posterior densities…

  4. […] using an indicator function as an instrumental function. There is therefore a connection with our proposal (made with Jean-Michel Marin) of considering an HPD region for excluding the tails of the […]

  5. […] Note that the meeting will be held in the beautiful buildings of the  Conservatoire National des Arts et des Métiers,  which is also linked with Umberto Eco’s Foucault’s Pendulum since it partly takes place there. Another point of relevance is that Jean-Michel Marin will give a tutorial on Bayesian Core during Compstat 2010. (He will focus on Bayes factor approximations.) […]

  6. […] Tardella on arXiv. While working on phylogenetic trees, it does not consider an ABC approach for computing the evidence but instead relies on harmonic mean estimators (since thermodynamic alternatives take too long). It […]

  7. […] as the Verdinelli and Wasserman (1995) alternative on the Pima Indian benchmark used in our survey of Bayes factor approximation methods. The theoretical comparison between both Monte Carlo […]

  8. […] I will also teach a very short course on March 17 in San Antonio,  Texas, based on the book, in the first day of the meeting Frontiers of Statistical Decision Making and Bayesian Analysis, in honour of Jim Berger, […]

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: