On resolving the Savage-Dickey paradox

After a long month of polishing our explanations and adding a full-scale comparison with the solution of Verdinelli and Wasserman (1995). we have now completed (and rearXived) our rewriting of the paper on the Savage-Dickey paradox. And we eventually made the submission to the Annals of Statistics, with the hope that the mix of measure-theory and computational techniques therein would appeal to the journal. In this extended version, we show that the Verdinelli and Wasserman (1995) representation

B_{01} = \dfrac{\pi_1(\theta_0|x)}{\pi(\theta_0)} \, \mathbb{E}^{\pi_1(\psi|\theta_0,x)}[\pi_0(\psi)/\pi_1(\psi|\theta_0)]

only depends on a specific version of the conditional density

\pi_1(\psi|\theta_0,x) = \dfrac{f(x|\theta_0,\psi) \pi_1(\psi|\theta_0)\pi_1(\theta_0)}{m_1(x)\pi_1(\theta_0|x)}

where \pi_1(\psi|\theta_0)\,,\ \pi_1(\theta_0)\,,\ \pi_1(\theta_0|x) are arbitrary versions.

box-plot comparing Bayes factor approximationsWe also found that a quick-and-dirty sufficient condition for the above Verdinelli and Wasserman (1995) representation to apply is that Bayes’ theorem

\dfrac{\pi_1(\theta|x)}{\pi_1(\theta)} = \dfrac{\int f(x|\theta,\psi) \pi_1(\psi|\theta) \text{d}\psi}{\int f(x|\theta,\psi) \pi_1(\psi|\theta) \pi_1(\theta)\text{d}\psi\text{d}\theta}

holds in \theta_0 and not only almost everywhere. In terms of Monte Carlo approximations, the solution based on the generic Savage-Dickey representation

B_{01} = \dfrac{\tilde\pi_1(\theta_0|x)}{\pi(\theta_0)} \, \mathbb{E}^{\pi_1(\theta,\psi|x)}[\pi_0(\psi)/\pi_1(\psi|\theta_0)]

has the same unbiasedness and variability as the Verdinelli and Wasserman (1995) alternative on the Pima Indian benchmark used in our survey of Bayes factor approximation methods. The theoretical comparison between both Monte Carlo solutions remains an open question. The R objects needed to produce the boxplots above are available both in Dauphine and Montpellier.

6 Responses to “On resolving the Savage-Dickey paradox”

  1. […] and the computational aspects of the resulting Bayesian procedures, including evidence, the Savage-Dickey paradox, nested sampling, harmonic mean estimators, and […]

  2. […] hinted at a partial misunderstanding with the nature of the “paradox”. I am afraid the measure-theoretic difficulty with this Savage-Dickey paradox will not vanish once the paper is […]

  3. […] I received the very nice news from the Electronic Journal of Statistics that our paper on the Savage-Dickey paradox was in for revision. One referee recommended acceptance as is and the second referee asked for more […]

  4. […] Here are the slides for the Savage-Dickey paradox paper that I gave in San Antonio this […]

  5. […] Our Savage-Dickey paper has been rejected by the Annals of Statistics, for being too obscure. I completely understand […]

  6. […] that it seems to be primarily a probability seminar, I will insist on the Savage-Dickey paradox to see if the audience gets the point, contrary to the Annals’ screeners. Possibly related […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: