## On resolving the Savage-Dickey paradox

Posted in R, Statistics with tags , , , , , , , on November 16, 2009 by xi'an

After a long month of polishing our explanations and adding a full-scale comparison with the solution of Verdinelli and Wasserman (1995). we have now completed (and rearXived) our rewriting of the paper on the Savage-Dickey paradox. And we eventually made the submission to the Annals of Statistics, with the hope that the mix of measure-theory and computational techniques therein would appeal to the journal. In this extended version, we show that the Verdinelli and Wasserman (1995) representation

$B_{01} = \dfrac{\pi_1(\theta_0|x)}{\pi(\theta_0)} \, \mathbb{E}^{\pi_1(\psi|\theta_0,x)}[\pi_0(\psi)/\pi_1(\psi|\theta_0)]$

only depends on a specific version of the conditional density

$\pi_1(\psi|\theta_0,x) = \dfrac{f(x|\theta_0,\psi) \pi_1(\psi|\theta_0)\pi_1(\theta_0)}{m_1(x)\pi_1(\theta_0|x)}$

where $\pi_1(\psi|\theta_0)\,,\ \pi_1(\theta_0)\,,\ \pi_1(\theta_0|x)$ are arbitrary versions.

We also found that a quick-and-dirty sufficient condition for the above Verdinelli and Wasserman (1995) representation to apply is that Bayes’ theorem

$\dfrac{\pi_1(\theta|x)}{\pi_1(\theta)} = \dfrac{\int f(x|\theta,\psi) \pi_1(\psi|\theta) \text{d}\psi}{\int f(x|\theta,\psi) \pi_1(\psi|\theta) \pi_1(\theta)\text{d}\psi\text{d}\theta}$

holds in $\theta_0$ and not only almost everywhere. In terms of Monte Carlo approximations, the solution based on the generic Savage-Dickey representation

$B_{01} = \dfrac{\tilde\pi_1(\theta_0|x)}{\pi(\theta_0)} \, \mathbb{E}^{\pi_1(\theta,\psi|x)}[\pi_0(\psi)/\pi_1(\psi|\theta_0)]$

has the same unbiasedness and variability as the Verdinelli and Wasserman (1995) alternative on the Pima Indian benchmark used in our survey of Bayes factor approximation methods. The theoretical comparison between both Monte Carlo solutions remains an open question. The R objects needed to produce the boxplots above are available both in Dauphine and Montpellier.