On the Dickey-Savage ratio estimator

While working on the estimation of our representation of the Dickey-Savage-like approximation to the Bayes factor,

$B_{01} = \dfrac{\pi_1(\theta_0|x)}{\pi_1(\theta_0)}$

with Jean-Michel Marin, we came upon a missing term in the Monte Carlo approximation

$B_{01} \approx \dfrac{1}{T}\dfrac{\sum_t \tilde\pi_1(\theta_0|x,z^{(t)},\psi^{(t)})}{\pi_1(\theta_0)},$

where the $z^{(t)}$‘s are simulated in one step of a two-stage or three-stage Gibbs sampler and $\tilde\pi_1(\theta|x,z,\psi)$ is the full (completed) posterior derived from the  pseudo-posterior $\tilde\pi$. Indeed, simulating from

$\tilde \pi_1(\theta,\psi|x)\propto \pi_0(\psi) \pi_1(\theta) f(x|\theta,\psi)$

means that the normalising constant of this joint distribution is not $m_1(x)$ but an alternative $\tilde m_1(x)$. Therefore the proper approximation to the Bayes factor should be

$B_{01} \approx \dfrac{1}{T}\dfrac{\sum_t \tilde\pi_1(\theta_0|x,z^{(t)},\psi^{(t)})}{\pi_1(\theta_0)}\,\dfrac{\tilde m_1(x)}{m_1(x)},$

based on the same simulations. This obviously creates a local problem since both $m_1(x)$ and $\tilde m_1(x)$ are not available. However, this ratio can be estimated by regular bridge sampling from the same simulation experiment since

$\mathbb{E}^{\tilde\pi_1(\theta,\psi|x)}\left[ \dfrac{\pi_1(\psi|\theta)}{\pi_0(\psi)} \right] = \dfrac{m_1(x)}{\tilde m_1(x)}$

or from a sample from $\pi_1(\theta,\psi|x)$ since

$\mathbb{E}^{\pi_1(\theta,\psi|x)}\left[ \dfrac{\pi_0(\psi)}{\pi_1(\psi|\theta)}\right] = \dfrac{\tilde m_1(x)}{m_1(x)}$

the later having the advantage of leading to an unbiased estimator of $B_{01}$. which is a rather uncommon feature! When applying both evaluations to the probit example, we ended up with both estimates of the correctng factor close to 1/1.17, and very stable, which means the picture in the previous post is next to being correct! Still, there was a clear mistake in the original representation I should not have made…

One Response to “On the Dickey-Savage ratio estimator”

1. […] the Savage-Dickey paradox Following several posts on this topic, we eventually managed to write down a short note with Jean-Michel Marin, which is […]

This site uses Akismet to reduce spam. Learn how your comment data is processed.