Archive for Montpellier

ABC model choice by random forests

Posted in pictures, R, Statistics, Travel, University life with tags , , , , , , , , , , , , , on June 25, 2014 by xi'an

treerise6After more than a year of collaboration, meetings, simulations, delays, switches,  visits, more delays, more simulations, discussions, and a final marathon wrapping day last Friday, Jean-Michel Marin, Pierre Pudlo,  and I at last completed our latest collaboration on ABC, with the central arguments that (a) using random forests is a good tool for choosing the most appropriate model and (b) evaluating the posterior misclassification error rather than the posterior probability of a model is an appropriate paradigm shift. The paper has been co-signed with our population genetics colleagues, Jean-Marie Cornuet and Arnaud Estoup, as they provided helpful advice on the tools and on the genetic illustrations and as they plan to include those new tools in their future analyses and DIYABC software.  ABC model choice via random forests is now arXived and very soon to be submitted…

truePPOne scientific reason for this fairly long conception is that it took us several iterations to understand the intrinsic nature of the random forest tool and how it could be most naturally embedded in ABC schemes. We first imagined it as a filter from a set of summary statistics to a subset of significant statistics (hence the automated ABC advertised in some of my past or future talks!), with the additional appeal of an associated distance induced by the forest. However, we later realised that (a) further ABC steps were counterproductive once the model was selected by the random forest and (b) including more summary statistics was always beneficial to the performances of the forest and (c) the connections between (i) the true posterior probability of a model, (ii) the ABC version of this probability, (iii) the random forest version of the above, were at best very loose. The above picture is taken from the paper: it shows how the true and the ABC probabilities (do not) relate in the example of an MA(q) model… We thus had another round of discussions and experiments before deciding the unthinkable, namely to give up the attempts to approximate the posterior probability in this setting and to come up with another assessment of the uncertainty associated with the decision. This led us to propose to compute a posterior predictive error as the error assessment for ABC model choice. This is mostly a classification error but (a) it is based on the ABC posterior distribution rather than on the prior and (b) it does not require extra-computations when compared with other empirical measures such as cross-validation, while avoiding the sin of using the data twice!

last Big MC [seminar] before summer [June 19, 3pm]

Posted in pictures, Statistics, University life with tags , , , , , , , , , , , on June 17, 2014 by xi'an

crossing Rue Soufflot on my way to IHP from Vieux Campeur, March 28, 2013Last session of our Big’MC seminar at Institut Henri Poincaré this year, on Tuesday Thursday, June 19, with

Chris Holmes (Oxford) at 3pm on

Robust statistical decisions via re-weighted Monte Carlo samples

and Pierre Pudlo (iC3M, Université de Montpellier 2) at 4:15pm on [our joint work]

ABC and machine learning

trip to Montpellier

Posted in pictures, Statistics, Travel, University life, Wines with tags , , , , , , on February 27, 2014 by xi'an

IMG_2477Last week, I flew down to Montpellier for two days of work on ABC model choice with Jean-Michel Marin and Pierre Pudlo. Although we missed the COLT 2014 deadline, we are now close to completing this work that will propose a rather radical change in our advocacy of how ABC model choice should be conducted. We actually spent the second day on the wonderful campus of INRA at Montferrier-sur-Lez, just outside Montpellier, discussing of the implications of this approach with our friends at CBGP, Jean-Marie Cornuet and Arnaud Estoup. With possible impact on the DIYABC software. It was a very profitable trip (not mentioning tasting great Grés de Montpellier wine!) and I hope to manage completing the paper with Pierre during the next week in Banff. Unfortunately, when I came back to my train station, I found some idiots had a go at my bike and bent the back wheel which then needed to be replaced…

art brut

Posted in pictures, Travel with tags , , , , on February 8, 2014 by xi'an

IMG_1514

Saint Christol white & red

Posted in Travel, Wines with tags , , , , , , on September 20, 2013 by xi'an

IMG_0233IMG_0235

ABC for design

Posted in Statistics with tags , , , , , , , on August 30, 2013 by xi'an

I wrote a comment on this arXived paper on simulation based design that starts from Müller (1999) and gets an ABC perspective a while ago on my iPad when travelling to Montpellier and then forgot to download it…

Hainy, [Wener] Müller, and Wagner recently arXived a paper called “Likelihood-free Simulation-based Optimal Design“, paper which relies on ABC to construct optimal designs . Remember that [Peter] Müller (1999) uses a natural simulated annealing that is quite similar to our MAP [SAME] algorithm with Arnaud Doucet and Simon Godsill, relying on multiple versions of the data set to get to the maximum. The paper also builds upon our 2006 JASA paper with my then PhD student Billy Amzal, Eric Parent, and Frederic Bois, paper that took advantage of the then emerging particle methods to improve upon a static horizon target. While our method is sequential in that it pursues a moving target, it does not rely on the generic methodology developed by del Moral et al. (2006), where a backward kernel brings more stability to the moves. The paper also implements a version of our population Monte Carlo ABC algorithm (Beaumont et al., 2009), as a first step before an MCMC simulation. Overall, the paper sounds more like a review than like a strongly directive entry into ABC based design in that it remains quite generic. Not that I have specific suggestions, mind!, but I fear a realistic implementation (as opposed to the linear model used in the paper) would require a certain amount of calibration. There are missing references of recent papers using ABC for design, including some by Michael Stumpf I think.

I did not know about the Kuck et al. reference… Which is reproducing our 2006 approach within the del Moral framework. It uses a continuous temperature scale that I find artificial and not that useful, again a maybe superficial comment as I didn’t get very much into the paper … Just that integer powers lead to multiples of the sample and have a nice algorithmic counterpart.

relevant statistics for Bayesian model choice (#4)

Posted in pictures, Statistics, Travel, University life with tags , , , , , , , , , on August 23, 2013 by xi'an

leaves from a pergola reflecting on a truck, Crès, near Montpellier, June 14, 2012I have just posted on arXiv the fourth (and hopefully final) version of our paper, Relevant statistics for Bayesian model choice, written jointly with Jean-Michel Marin, Natesh Pillai, and Judith Rousseau over the past two years. As we received a very positive return from the editorial team at JRSS Series B, I flew to Montpellier today to write & resubmit a revised version of the paper. The changes are only stylistic, since we could not answer in depth a query about the apparently different speeds of convergence of the posterior probabilities under the Gaussian and Laplace distributions in Figures 3 & 4 (see paper). This was a most interesting question in that the marginal likelihoods do indeed seem to converge at different speeds. However, the only precise information we can derive from our result (Theorem 1) is when the Bayes factor is not consistent. Otherwise, we only have a lower bound on its speed of convergence (under the correct model). Getting precise speeds in this case sounds beyond our reach… (Unless I am confused with time zones, this post should come alive just after the fourth version is announced on arXiv..)

Follow

Get every new post delivered to your Inbox.

Join 640 other followers