MCMC and conditioning

Here is an interesting question from Tomas Iesmantas:

I have a question related to conditional decomposition in Bayesian hierarchical models. I’m dealing with a pretty nasty model, which involves nonlinearities. The hierarchy is made of two levels of parameters. After quite long calculations I derived the marginal distribution (up to a proportionality constant) for the first level parameters (I have 40 parameters there). Then I have the posterior  density (up to a proportionality constant) for the second level parameters conditioned on the first level parameters (i.e. a conditional posterior distribution for the second level parameters).

Am I right to simulate by MCMC from that [first level] marginal distribution and then to use the simulation to obtain a sample from the [second level] conditional distribution? I know that it would be valid to simulate from the marginal of the second level and then to use the sample for the conditional distribution of the first level. But I feel that somehow it is related to de Finetti’s exchangeability notion. I’m confused. Hope that you could point me to the right direction.

My reply was that, indeed, this implementation is correct. While we distinguish first level from second level for modelling purposes, they all are the same from a probabilistic perspective. In other words,

\pi(\theta_1,\theta_2|\mathcal{D})=\pi(\theta_1|\theta_2,\mathcal{D}) \pi(\theta_2|\mathcal{D}) = \pi(\theta_2|\theta_1) \pi(\theta_1|\mathcal{D})

(with hopefully obvious notations). If the marginal posterior for the first level parameters is available, MCMC can be directly applied to this posterior and simulations of the second level parameters are obtained “exactly” by simulating from the true full conditional posterior distribution. It can even wait till the MCMC simulation is over. (And I do not think de Finetti’s exchangeability matters for this property.)

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.