Archive for intractability

slice sampling for Dirichlet mixture process

Posted in Books, Statistics, University life with tags , , , , , , , on June 21, 2017 by xi'an

When working with my PhD student Changye in Dauphine this morning I realised that slice sampling also applies to discrete support distributions and could even be of use in such settings. That it works is (now) straightforward in that the missing variable representation behind the slice sampler also applies to densities defined with respect to a discrete measure. That this is useful transpires from the short paper of Stephen Walker (2007) where we saw this, as Stephen relies on the slice sampler to sample from the Dirichlet mixture model by eliminating the tail problem associated with this distribution. (This paper appeared in Communications in Statistics and it is through Pati & Dunson (2014) taking advantage of this trick that Changye found about its very existence. I may have known about it in an earlier life, but I had clearly forgotten everything!)

While the prior distribution (of the weights) of the Dirichlet mixture process is easy to generate via the stick breaking representation, the posterior distribution is trickier as the weights are multiplied by the values of the sampling distribution (likelihood) at the corresponding parameter values and they cannot be normalised. Introducing a uniform to replace all weights in the mixture with an indicator that the uniform is less than those weights corresponds to a (latent variable) completion [or a demarginalisation as we called this trick in Monte Carlo Statistical Methods]. As elaborated in the paper, the Gibbs steps corresponding to this completion are easy to implement, involving only a finite number of components. Meaning the allocation to a component of the mixture can be operated rather efficiently. Or not when considering that the weights in the Dirichlet mixture are not monotone, hence that a large number of them may need to be computed before picking the next index in the mixture when the uniform draw happens to be quite small.

still confronting intractability in Bristol…

Posted in pictures, Running, Statistics, Travel, University life with tags , , , , , , , , , , , , , , on April 19, 2012 by xi'an

Another definitely interesting and intense day at the Confronting Intractability in Statistical Inference workshop in Bristol: all talks there had a high informational content for me and even those I had heard previously [in Banff or elsewhere] had a significant impact! (One of the many good points of attending a conference in England is that there is no time difference and hence much less chances of my dozing during talks, which, alas!, now gets into an almost certainty for US conferences!) For instance, I am still coming to terms with Gareth’s importance sampling for continuous diffusions. (This was the first time I was hearing Arnaud’s talk on the estimation of the score vector and I definitely to hear it again, given its technicality!) Sumeet Singh gave a talk mixing ABC with maximum likelihood estimation for HMMS, in connection with his earlier paper, and I got more convince  by the idea of using a sequence of balls for keeping pseudo-data close to the true data when I realised it could be implemented sequentially. Nial Friel’s talk on the double intractable likelihoods was covering graphical models and social network models, maybe calling for a comparison with ABC, as done in the recent paper by Richard Everitt. I had too many slides and thus presumably failed to deliver an intelligible message about the selection of ABC summary statistics for testing, even though the population genetics new illustration presumably helped. In connection with our ABC paper, Dennis Prangle and Paul Fernhead presented a poster on using the Bayes factor as a summary statistics in this setup, in the spirit of their Read Paper of last December. And Richard Wilkinson concluded the day with a more philosophical talk on the dual nature of ABC inference, in a quite pleasant perspective (that related to the way ABC was received by econometricians during my talk in Princeton last week). The day ended up quite pleasantly in a south-Indian thali restaurant, a good preparation for Glasgow’s Ashoka tomorrow night!