MCMC with mutually singular distributions
A paper that recently appeared in the latest issue of JCGS deals with this important issue of using mutually singular proposals, written by Raphael Gottardo and Adrian Raftery. I personally find the problem quite interesting because it covers major applications like Bayesian model choice and Bayesian nonparametric density estimation. The beginning of the paper recalls some basics about computing densities for measures that are mixtures wrt mutually exclusive measures. (This may sound obvious to many, but there is this important remark that, for measures involving Lebesgue and Dirac components at , the density of the Lebesgue part must be zero at all ‘s.) Once the density of the target is clearly defined, Gottardo and Raftery move to define a general Metropolis-Hastings algorithm. This sets the correct setup for running samplers across models, as in hypothesis testing or variable selection.
Now, I find it rather restrictive that in the paper both the target and the proposal are defined against the same dominating measure, while an almost standard MCMC algorithm (like Gibbs) would move only some components and thus be defined against another dominating measure. Maybe I did miss a major point but the paper seems to imply that Gibbs sampling works within this framework only if one defines the target as the posterior plus conditionals (?). Anyway, I would have thought that the proper setting would require both product measures and to be absolutely continuous wrt one another, rather than having to impose to all four components to be defined against the same dominating measure. Not that there is anything wrong in the current paper, but this choice seems to be limiting the appeal of the extension… (Note that the complexities of product measures on those variable dimension spaces are what prompted Peter Green to introduce reversible jump in 1995 in terms of a dominating symmetric measure.)