On the Dickey-Savage ratio estimator

While working on the estimation of our representation of the Dickey-Savage-like approximation to the Bayes factor,

B_{01} = \dfrac{\pi_1(\theta_0|x)}{\pi_1(\theta_0)}

with Jean-Michel Marin, we came upon a missing term in the Monte Carlo approximation

B_{01} \approx \dfrac{1}{T}\dfrac{\sum_t \tilde\pi_1(\theta_0|x,z^{(t)},\psi^{(t)})}{\pi_1(\theta_0)},

where the z^{(t)}‘s are simulated in one step of a two-stage or three-stage Gibbs sampler and \tilde\pi_1(\theta|x,z,\psi) is the full (completed) posterior derived from the  pseudo-posterior \tilde\pi. Indeed, simulating from

\tilde \pi_1(\theta,\psi|x)\propto \pi_0(\psi) \pi_1(\theta) f(x|\theta,\psi)

means that the normalising constant of this joint distribution is not m_1(x) but an alternative \tilde m_1(x). Therefore the proper approximation to the Bayes factor should be

B_{01} \approx \dfrac{1}{T}\dfrac{\sum_t \tilde\pi_1(\theta_0|x,z^{(t)},\psi^{(t)})}{\pi_1(\theta_0)}\,\dfrac{\tilde m_1(x)}{m_1(x)},

based on the same simulations. This obviously creates a local problem since both m_1(x) and \tilde m_1(x) are not available. However, this ratio can be estimated by regular bridge sampling from the same simulation experiment since

\mathbb{E}^{\tilde\pi_1(\theta,\psi|x)}\left[ \dfrac{\pi_1(\psi|\theta)}{\pi_0(\psi)} \right] = \dfrac{m_1(x)}{\tilde m_1(x)}

or from a sample from \pi_1(\theta,\psi|x) since

\mathbb{E}^{\pi_1(\theta,\psi|x)}\left[ \dfrac{\pi_0(\psi)}{\pi_1(\psi|\theta)}\right] = \dfrac{\tilde m_1(x)}{m_1(x)}

the later having the advantage of leading to an unbiased estimator of B_{01}. which is a rather uncommon feature! When applying both evaluations to the probit example, we ended up with both estimates of the correctng factor close to 1/1.17, and very stable, which means the picture in the previous post is next to being correct! Still, there was a clear mistake in the original representation I should not have made…

One Response to “On the Dickey-Savage ratio estimator”

  1. […] the Savage-Dickey paradox Following several posts on this topic, we eventually managed to write down a short note with Jean-Michel Marin, which is […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.