In a well-coordinated move, Oliver Ratmann and his co-authors published his reply to our letter on arXiv yesterday afternoon, right after the “viva” for his PhD thesis at Imperial. The arXiv reply is the longer version of the PNAS reply (just like our arXiv posting is the longer version of our letter). Maybe the most important feature of the PNAS paper is that the original motivation of running ABC for conducting inference on the parameters of a model is replaced by the alternative goal of running ABC for assessing a model from within, the parameters being then at best nuisance parameters.
While unwilling to restart the debate, I (obviously) still stick to the arguments presented in our letter, in particular that what serves as a proxy to a likelihood in ABCμ is not always a density in the observed variable if only because the use of summary statistics induces a decrease in the dimension of the vector. (A note about the Poisson example: when we mentioned the “truncation to positive values”, we meant that, since
is a positive integer valued random variable, its density given
should only be positive for positive integer values if it is to be interpreted as a density in
.) Maybe my strongest misgiving about using ABCμ for model criticism is that I do not see any reason why the discrepancies
should be centred at zero given the data
… In a frequentist approach
is indeed symmetric and thus centred at zero (if the model is correct) but from a Bayesian perspective this is not the case. (Hence my earlier criticism of the null hypothesis.) It would thus be nice to see the equivalent of Example 3 and Figures 1-2 when the data is truly from the tested model: zero could then be (more) covered by the posterior of
but there is no reason this posterior should be centred there! Anyway, I find this reply quite stimulating for further pursuing the potential of ABC methods in setups where there is no viable alternatives.
Reply to our PNAS letter
Posted in Statistics, University life with tags ABC, Bayesian model choice, Dickey-Savage ratio on December 17, 2009 by xi'anABC thesis
Posted in Statistics, University life with tags ABC, Bayes factor, Bayesian model choice, hypothesis testing, London, sufficient statistics on December 16, 2009 by xi'anI am currently in London for the thesis examination of Oliver Ratmann and, while I am more than disapointed at finding my favourite Indian restaurant closed!, I enjoy very much reading this very rich and unusual thesis. As posted earlier, I have disagreements with some of the choices made in this thesis, in particular the ambivalent role of the error , whose discussion is to appear in PNAS, but it opens a whole range of new directions. In particular, it proposes an examination of the successive inclusion of diverse statistics, a bit in the spirit of Joyce and Marjoram, discussed in this post. It also considers the impact of testing for the adequacy of a model as testing for the hypothesis
, which I find quite an interesting stance, even though I completely disagree with the approach! Indeed, a Bayes factor can be constructed almost formally for this hypothesis, thus formal Bayesian answers provided. But testing whether or not
does not make sense since, even when the model fits,
varies around zero. But this is nonetheless a very imaginative proposal! (Overall the thesis stands a very good chance for the Savage Prize 2011 if it is ever submitted!)
Big wall in Antartica
Posted in Mountains with tags Antartica, big wall, Holtanna, Queen Maud on December 14, 2009 by xi'anNested in, at last!
Posted in Statistics with tags Bayesian model choice, Biometrika, evidence, nested sampling, simulation on December 13, 2009 by xi'anAfter a rather long editorial process of about two years, our paper on an evaluation of nested sampling, written with Nicolas Chopin, has just been accepted by Biometrika. (This is the version currently posted on arXiv.) Besides the examples processed in this paper, I think it would be worthwhile examining the performances of nested sampling and of the alternatives for approximating evidence in astrophysical settings as those mentioned in this earlier post. (This should have been covered in the PMC based approximation paper arXived last week…) Especially the most recent occurences of the method like multi-nest that addresses multimodal targets.
Already a competitor?!
Posted in Books, Statistics with tags Bayesian computation, Bayesian statistics, Gibbs sampling, MCMC, Monte Carlo, R, simulation on December 12, 2009 by xi'an
When looking around on Amazon, I found that “Introducing Monte Carlo Methods with R” was associated with another very recently published (same day as ours!) book, “Understanding Computational Bayesian Statistics“, by William Bolstad, that seems to mostly cover the same ground as us (with some connections with Bayesian Core for prior modelling in regression and logistic models). Although R seems to be less proeminently advocated than in our Use R! volume, I am quite curious to see what exactly is in this book and how much of a competitor it is! (Given that it is the same length as ours (about 315 pages), I am however a bit surprised at the high $110’s asked for this book.)
