Sequential Monte Carlo without likelihoods

ABC-PRC acceptance probability

While peeping at the slides of the working groups of the 2008-09 Program on Sequential Monte Carlo Methods, I came upon a short presentation of the ABC-PRC version of Sisson, Fan and Tanaka of the ABC algorithm that does not seem to be aware the bias exhibited in our paper with Marc Beaumont, Jean-Marie Cornuet and Jean-Michel Marin, following a first exchange of Marc with the authors. Since this appears to be the case for many people using ABC, I recap here our point.

The difficulty with the method centers at the acceptance probability above, that is derived from the SMC sampler of Del Moral, Doucet and Jasra (2006, JRSS B), with the difference that the likelihood is removed in a standard ABC fashion. However, the missing likelihood in the denominator is not compensated for and this creates the bias. The difficulty is not acknowledged in PNAS (which rejected our submission on the ground that the problem was “well-known”, which is apparently not that true). The update published on Scott Sisson’s webpage does not acknowledge the bias but rather puts the blame for poor performances on the fact that “poor choices of backward kernels such as L = K can in some cases result in importance weights with a very large or infinite variance“. Rather interestingly, the solution put forward in the update for the backward kernel L ends up with a form that is identical with the population Monte Carlo solution we propose, but for the wrong reason altogether!

14 Responses to “Sequential Monte Carlo without likelihoods”

  1. […] Incredibly ugly squalid pictures… Well, this is not a common teaser to attract readers, but a comment on one of my graphs in the second revision of our paper Adaptivity for approximate Bayesian computation algorithms: a population Monte Carlo approach, written with Marc Beaumont, Jean-Marie Cornuet, and Jean-Michel Marin, and (re-re-)submitted to Biometrika… Not something I’d like to hear about my graphs, thank you!, as the pdf version of the graph on the right actually looks better than than one…. Anyway, we revised the paper towards less squalidness, replacing histogram with density using the “h” type in R. The major request on the revision was to get under eight pages in order to fit inside the Miscelanea section of Biometrika. Changes are thus mostly cosmetic compared with the earlier version, as you can check on the arXiv list of versions. The background for the paper and the earlier paper of Sisson, Fan, and Tanaka (2007, PNAS) it analyses, is described in this earlier post. […]

  2. […] going on at the Université de Montpellier II, I will (hopefully) give a seminar there tomorrow on ABC methods. Here are my slides for those interested (or on […]

  3. […] a second round of reviews from Biometrika where the referees validated our analysis of the bias in ABC-PRC. The changes are quite minor, compared with the previous version and include an […]

  4. […] Marc Beaumont, University of Reading (whose paper is discussed on that post); […]

  5. […] the supplementary material in the Appendix of Sisson et al. (2007b) yesterday, I found I have an additional difficulty with the ABC-PRC algorithm that is related with the partial rejection control (PRC) […]

  6. […] for parameter inference and model selection in dynamical systems Another paper on ABC that is about to appear [in Journal of the Royal Society Interface, 2009] is “Approximate […]

  7. […] brings a new light on the SMC-PRC proposal of Sisson, Fan and Tanaka, already discussed in this post, in that both papers are based on the 2006 Series B paper idea of using a backward kernel […]

  8. […] proposed on ArXiv last september a generalisation of Beaumont et al.’s (Genetics, 2002) ABC where the local linear regression of the parameter θ on the sufficient (or summary) statistics s […]

  9. […] approximation in ABC A new paper about ABC was posted today by Richard Wilkinson on ArXiv. The main point in the paper is to replace the […]

  10. […] statistics for ABC The issue of picking quasi-suffificent statistics in ABC algorithms is quite important and Joyce and Marjoram have published (last August, sorry I’m […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.