Archive for SAME algorithm

ABC-MCMC for parallel tempering

Posted in Mountains, pictures, Statistics, Travel, University life with tags , , , , , , , , , , , on February 9, 2012 by xi'an

In this paper a new algorithm combining population-based MCMC methods with ABC requirements is proposed, using an analogy with the Parallel Tempering algorithm (Geyer, 1991).

Another of those arXiv papers that had sat on my to-read pile for way too long: Likelihood-free parallel tempering by Meïli Baragtti, Agnès Grimaud, and Denys Pommeret, from Luminy, Marseilles. The paper mentions our population Monte Carlo (PMC) algorithm (Beaumont et al., 2009) and other ABC-SMC algorithms, but opts instead for an ABC-MCMC basis. The purpose is to build a parallel tempering method. Tolerances and temperatures evolve simultaneously. I however fail to see where the tempering occurs in the algorithm (page 7): there is a set of temperatures T1,….,TN, but they do not appear within the algorithm. My first idea of a tempering mechanism in a likelihood-free setting was to replicate our SAME algorithm (Doucet, Godsill, and Robert, 2004), by creating Tj copies of the [pseudo-]observations to mimic the likelihood taken to the power Tj. But this is annealing, not tempering, and I cannot think of the opposite of copies of the data. Unless of course a power of the likelihood can be simulated (and even then, what would the equivalent be for the data…?) Maybe a natural solution would be to operate some kind of data-attrition, e.g. by subsampling the original vector of observations.

Discussing the issue with Jean-Michel Marin, during a visit to Montpellier today, I realised that the true tempering came from the tolerances εi, while the temperatures Tj were there to calibrate the proposal distributions. And that the major innovation contained in the thesis (if not so clearly in the paper) was to boost exchanges between different tolerances, improving upon the regular ABC-MCMC sampler by an equi-energy move.

Feedback on data cloning

Posted in Books, Statistics, Travel, University life with tags , , , , , , , , , , , , , on September 22, 2010 by xi'an

Following some discussions I had last week at Banff about data cloning, I re-read the 2007 “Data cloning” paper published in Ecology Letters by Lele, Dennis, and Lutscher. Once again, I see a strong similarity with our 2002 Statistics and Computing SAME algorithm, as well as with the subsequent (and equally similar) “A multiple-imputation Metropolis version of the EM algorithm” published in Biometrika by Gaetan and Yao in 2003—Biometrika to which Arnaud and I had earlier and unsuccessfully submitted this unpublished technical report on the convergence of the SAME algorithm… (The SAME algorithm is also described in detail in the 2005 book Inference in Hidden Markov Models, Chapter 13.)

Continue reading

10w2170, Banff [2]

Posted in R, Statistics, University life with tags , , , , , , , on September 14, 2010 by xi'an

Over the two days of the Hierarchical Bayesian Methods in Ecology workshop, we managed to cover normal models, testing, regression, Gibbs sampling, generalised linear models, Metropolis-Hastings algorithms and of course a fair dose of hierarchical modelling. At the end of the Saturday marathon session, we spent one and half discussing some models studied by the participants, which were obviously too complex to be solved on the spot but well-defined so that we could work on MCMC implementation and analysis. And on Sunday morning, a good example of Poisson regression proposed by Devin Goodman led to an exciting on-line programming of a random effect generalised model, with the lucky occurrence of detectable identifiability issues that we could play with… I am impressed at the resilience of the audience given the gruesome pace I pursued over those two days, covering the five first chapters of Bayesian Core, all the way to the mixtures! In retrospect, I think I need to improve my coverage of testing as the noninformative case presumably sounded messy. And unconvincing. I also fear the material on hierarchical models was not sufficiently developed. But, overall, the workshop provided a wonderful opportunity to exchange with bright PhD students from Ecology and Forestry about their models and (hierarchical) Bayesian modelling.

Continue reading