MCMC with mutually singular distributions

A paper that recently appeared in the latest issue of JCGS deals with this important issue of using mutually singular proposals, written by Raphael Gottardo and Adrian Raftery. I personally find the problem quite interesting because it covers major applications like Bayesian model choice and Bayesian nonparametric density estimation. The beginning of the paper recalls some basics about computing densities for measures that are mixtures wrt mutually exclusive measures. (This may sound obvious to many, but there is this important remark that, for measures involving Lebesgue and Dirac components at \alpha_i(1\le i\le n), the density of the Lebesgue part must be zero at all \alpha_i‘s.) Once the density of the target is clearly defined, Gottardo and Raftery move to define a general Metropolis-Hastings algorithm. This sets the correct setup for running samplers across models, as in hypothesis testing or variable selection.

Now, I find it rather restrictive that in the paper both the target and the proposal are defined against the same dominating measure\nu, while an almost standard MCMC algorithm (like Gibbs) would move only some components and thus be defined against another dominating measure. Maybe I did miss a major point but the paper seems to imply that Gibbs sampling works within this framework only if one defines the target as the posterior plus conditionals (?). Anyway, I would have thought that the proper setting would require both product measures \Pi(\text{d}x)Q(x,\text{d}y) and \Pi(\text{d}y)Q(y,\text{d}x) to be absolutely continuous wrt one another, rather than having to impose to all four components to be defined against the same dominating measure. Not that there is anything wrong in the current paper, but this choice seems to be limiting the appeal of the extension… (Note that the complexities of product measures on those variable dimension spaces are what prompted Peter Green to introduce reversible jump in 1995 in terms of a dominating symmetric measure.)

3 Responses to “MCMC with mutually singular distributions”

  1. The topic is quite hot on the Internet at the moment. What do you pay the most attention to when choosing what to write ?

  2. […] with mutually singular distributions (2) In connection with questions I had posted earlier about the interesting paper of Raphael Gottardo and Adrian Raftery, I got several emails […]

  3. Salut Xian,

    I told you I was going to post something so here it is, and sorry for the delay. As I said to you via email, you are not restricted to a proposal that is dominated by $\nu$. You could combine several moves with different proposals, not all dominated by $\nu$. This said, at least one of your proposals must be dominated by $\nu$ for the chain to be irreducible. You need to make sure that you have a positive prob. to visit all of the singular components (e.g. models). This is very similar to the within and between model moves in RJ-MCMC. In a sense, the between model proposal needs to be dominated by $\nu$. This is illustrated in the paper in several examples. Though, I agree that it may not have been terribly clear. For example, if you look at the paper, we have what we call the component-wise Gibbs, and the Gibbs. The component-wise Gibbs is a move that is not dominated by $\nu$ whereas the Gibbs is.

    Anyway, I’d be happy to post more to clarify some more things if needed. I enjoy your blog, so please keep it up. I have added a link in the ISBA bulletin.



Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.