efficient learning in ABC

Jean-Michel Marin just posted on arXiv a joint paper of ours, Efficient learning in ABC algorithms. This paper, to which elaboration [if not redaction] I contributed to, is one of the chapters of Mohammed Sedki’s thesis. (Mohammed is about to defend this thesis, currently with reviewers. A preliminary version of this paper was presented at ABC in London and it is in revision with Statistics and Computing. Hence missing the special issue!)

The paper builds on the sequential ABC scheme of Del Moral et al. (2012), already discussed in this post, where the tolerance level at each step is adapted from the previous iterations as a quantile of the distances. (The quantile level is based on a current effective sample size.) In a “systematic” step, the particles that are closest to the observations are preserved and duplicated, while those farther away are sampled randomly. The resulting population of particles is then perturbed by an adaptive (random walk) kernel and the algorithm stops when the tolerance level does not decrease any longer or when the acceptance rate of the Metropolis step is too low. Pierre Pudlo and Mohammed Sedki experimented a parallel implementation of the algorithm, with an almost linear improvement in the number of cores. It is less clear the same would work on a GPU because of the communication requirements. Overall, the new algorithm brings a significant improvement in computing time when compared with earlier versions, mainly because the number of simulations from the model is minimised. (It was tested on a rather complex population scenario retracing the invasion of honeybees in Europe (in connection with the previous post!)

ABC in Roma, May 30-31, 2013!!!

After ABC in Paris in 2009 and ABC in London in 2013, the ABC workshop on the current developments of ABC methodology continues its tour of European capitals! It will take place next year in Rome over two days to allow for travel from Paris and London (no EuRomaStar yet!). ABC in Rome is organised by Brunero Liseo and his colleagues at Roma 1 and Roma 3 Università, and sponsored by La Sapienza Università di Roma. Following the previous meetings, in Paris and London, the field still sees rapid methodology progress and an increased number of applications in a wider range of scientific areas, as no doubt readers of the ‘Og are aware! ABC in Rome (ABCiR) will hopefully bring together leading researchers in the field, with focus on

•  applications of ABC to real world problems
• recent computational advances in ABC
• comparative efficiency of ABC methods with respect to alternative methodologies
• model selection and model checking in the ABC framework.

As in the previous meetings, attendance is free of charge (free as in “no registration fee at all”!) but you do need to register as space is strictly limited. (For ABC in London, the waiting list was more than 100 persons long…) If you wish to present a poster then please email the organisation committee with a brief abstract. We particularly encourage posters from young participants and posters detailing recent software implementations of ABC methods and computational advance.

Adaptive ABC

Maxime Lenormand, Franck Jabot and Guillaume Deffuant have just posted on arXiv a paper about a refinement of the ABC-PMC algorithm we developed with Marc Beaumont, Jean-Marie Cornuet, and Jean-Michel Marin. The authors state in their introduction that ABC-PMC

presents two shortcomings which are particularly problematic for costly to simulate complex models. First, the sequence of tolerance levels ε₁,…,ε_T has to be provided to the ABC algorithm. In practice, this implies to do preliminary simulations of the model, a step which is computationally costly for complex models. Furthermore, a badly chosen sequence of tolerance levels may inflate the number of simulations required to reach a given precision as we will see below. A second shortcoming of the PMC-ABC algorithm is that it lacks a criterion to decide whether it has converged. The final tolerance level ε_T may be too large for the ABC approach to satisfactorily approximate the posterior distribution of the model. Inversely, a larger ε_T may be sufficient to obtain a good approximation of the posterior distribution, hence sparing a number of model simulations.

shortcomings which I thought were addressed by the ABC-SMC algorithm of Pierre Del Moral, Arnaud Doucet and Ajay Jasra [not referenced in the current paper], the similar algorithm of Chris Drovandi and Tony Pettitt, and our recent paper with  Jean-Michel Marin, Pierre Pudlo and Mohammed Sedki [presented at Statistics and Computing a few months ago, but alas not available on-line for unmentionable reasons linked to the growing dysfunctionality of one co-author…!]. It is correct that we did not address the choice of the ε_t‘s in the original paper, even though we already used an on-line selection as a quantile of the current sample of distances. In essence, given the fundamentally non-parametric nature of ABC, the tolerances ε_t should always be determined from the simulated samples, as regular bandwidths.

The paper essentially proposes the same scheme as in Del Moral et al., before applying it to the toy example of Sisson et al. (PNAS, 2007) and to a more complex job dynamic model in central France. Were I to referee this paper, I would thus suggest that the authors incorporate a comparison with both papers of Del Moral et al. and of Drovandi and Pettitt to highlight the methodological  novelty of their approach.

ABC and sufficient statistics

Chris Barnes, Sarah Filippi, Michael P.H. Stumpf, and Thomas Thorne posted a paper on arXiv on the selection of sufficient statistics towards ABC model choice. This paper, called Considerate Approaches to Achieving Sufficiency for ABC model selection, was presented by Chris Barnes during ABC in London two months ago. (Note that all talks of the meeting are now available in Nature Precedings. A neat concept by the way!) This paper of them builds on our earlier warning about (unfounded) ABC model selection to propose a selection of summary statistics that partly alleviates the  original problem. (The part about the discrepancy with the true posterior probability remains to be addressed. As does the issue of whether or not the selected collection of statistics provides a convergent model choice inference. We are currently working on it…) Their section “Resuscitating ABC model choice” states quite clearly the goal of the paper:

- this [use of inadequate summary statistics] mirrors problems that can also be observed in the parameter estimation context,
- for many important, and arguably the most important applications of ABC, this problem can in principle be avoided by using the whole data rather than summary statistics,
- in cases where summary statistics are required, we argue that we can construct approximately sufficient statistics in a disciplined manner,
- when all else fails, a change in perspective, allows us to nevertheless make use of the flexibility of the ABC framework

The driving idea in the paper is to use an entropy approximation to measure the lack of information due to the use of a given set of summary statistics. The corresponding algorithm then proceeds from a starting pool of summary statistics to build sequentially a collection of the most informative summary statistics (which, in a sense, reminded me of a variable selection procedure based on Kullback-Leibler, we developed with  Costas Goutis and Jérôme Dupuis). It is a very interesting advance in the issue of ABC model selection, even though it cannot eliminate all stumbling blocks. The interpretation that ABC should be processed as an inferential method on its own rather than an approximation to Bayesian inference is clearly appealing. (Fearnhead and Prangle, and Dean, Singh, Jasra and Peters could be quoted as well.)

ABC model choice by DIC

Yet another paper on ABC model choice was posted on arXiv a few days ago, just prior to the ABC in London meeting that ended in the pub above (most conveniently located next to my B&B!). It is written by Olivier Francois and Guillaume Laval and the approach relies on DIC for running model selection. Although I disagree with the reasons given for abandoning Bayes factors in favour of this more rudimentary indicator, I consider the paper (and the trend) an interesting and positive contribution to the idea already stressed by Oliver Ratmann and coauthors that model selection with ABC should be more exploratory than decisional…