My discussion on label switching
For the incoming conference on mixtures at ICMS in Edinburgh, Scotland, I have prepared a few slides as my discussion of the talks by Sylvia Früwirth-Schnatter and John Geweke on the controversial topic of label switching. I am afraid I am still in agreement with the radical breakup of our 2000 JASA paper, namely that most MCMC samplers fail to converge to the stationary distribution because they grossly fail to reproduce the symmetry predicted by the exchangeability. Here are the slides:
Xi'an's Og 
March 5, 2010 at 12:17 am
[…] is Ben Nevis on Monday, whose skyline really looks like a three component mixture!) and giving a discussion on label switching: I am implementing a Markov-Chain Monte Carlo method for Gibbs sampling from a […]
March 1, 2010 at 9:30 am
My initial reaction to this is only slightly more nuanced than “who cares”: as long as the indentifiable parts of the model converge, does it matter? You seem to be concentrating on the number of components in the mixture, but is that the only place there is a problem?
In practice, I tend to run more chains when I have mixture models, although that’s largely because it’s easier to spot lack of mixing.
March 4, 2010 at 6:01 am
Bob, if you take a look at the slides, I mention Chib’s approximation to evidence/marginal likelihood and hence to the Bayes factor. This is a perfectly valid approximation in the Monte Carlo sense that Chib’s representation converges to the true marginal likelihood when the number of MCMC iterations goes to infinity. And obviously when the MCMC chain is converging to the right stationary distribution. In the case of mixtures, it is well documented (see, e.g., Sylvia Frühwirth-Schnatter’s DeGroot Prize 2008 Finite Mixture and Markov Switching Models) that Chib’s approximation based on a Gibbs sampler fails to produce the correct numerical answer, being wrong by a factor of log(k!) in the case of well-separated modes…. So, yes, there are occurrences where the lack of label switching matters.