Archive for quasi-Monte Carlo

Optimization Monte Carlo: Efficient and embarrassingly parallel likelihood-free inference

Posted in Books, Statistics, Travel with tags , , , , , , , , on December 16, 2015 by xi'an

optiMC1AmstabcTed Meeds and Max Welling have not so recently written about an embarrassingly parallel approach to ABC that they call optimisation Monte Carlo. [Danke Ingmar for pointing out the reference to me.] They start from a rather innocuous rephrasing of the ABC posterior, writing the pseudo-observations as deterministic transforms of the parameter and of a vector of uniforms. Innocuous provided this does not involve an infinite number of uniforms, obviously. Then they suddenly switch to the perspective that, for a given uniform vector u, one should seek the parameter value θ that agrees with the observation y. A sort of Monte Carlo inverse regression: if


then invert this equation in θ. This is quite clever! Maybe closer to fiducial than true Bayesian statistics, since the prior does not occur directly [only as a weight p(θ)], but if this is manageable [and it all depends on the way f(θ,u) is constructed], this should perform better than ABC! After thinking about it a wee bit more in London, though, I realised this was close to impossible in the realistic examples I could think of. But I still like the idea and want to see if anything at all can be made of this…

“However, it is hard to detect if our optimization succeeded and we may therefore sometimes reject samples that should not have been rejected. Thus, one should be careful not to create a bias against samples u for which the optimization is difficult. This situation is similar to a sampler that will not mix to remote local optima in the posterior distribution.”

Now, the paper does not go that way but keeps the ε-ball approach as in regular ABC, to derive an approximation of the posterior density. For a while I was missing the difference between the centre of the ball and the inverse of the above equation, bottom of page 3. But then I realised the former was an approximation to the latter. When the authors discuss their approximation in terms of the error ε, I remain unconvinced by the transfer of the tolerance to the optimisation error, as those are completely different notions. This also applies to the use of a Jacobian in the weight, which seems out of place since this Jacobian appears in a term associated with (or replacing) the likelihood, f(θ,u), which is then multiplied by the prior p(θ). (Assuming a Jacobian exists, which is unclear when considering most simulation patterns use hard bounds and indicators.) When looking at the toy examples, it however makes sense to have a Jacobian since the selected θ’s are transforms of the u’s. And the p(θ)’s are simply importance weights correcting for the wrong target. Overall, the appeal of the method proposed in the paper remains unclear to me. Most likely because I did not spend enough time over it.

Latent Gaussian Models im Zürich [day 1]

Posted in R, Statistics with tags , , , , , on February 5, 2011 by xi'an

An interesting first day (for me) at the Latent Gaussian Models workshop in Zürich. The workshop is obviously centred at the INLA approach, with Havard Rue giving a short course on Wednesday then a wide ranging tour of the applications and extensions of INLA this afternoon. Thanks to his efforts in making the method completely accessible for many models through an R package, using mode description commands like

inla(formula, family="weibull", data=Kidney, control.inla=list(h=0.001))

there is now a growing community of INLA users. As exemplified by the attendees to this workshop. Chris Holmes gave another of his inspirational talks this afternoon when defending the use of quasi-Monte Carlo methods in Bayes factor approximations. The model choice session this morning showed interesting directions, including a calibration of the Hellinger distance by Bernoulli distributions, while the application session this afternoon covered owls, bulls, and woolly mammoths. I even managed to speak about ABC model choice, Gaussian approximations of Ising models, stochastic volatility modelling, and grey codes for variable selection, before calling it a (full and fruitful) day!