## Why is it necessary to sample from the posterior distribution if we already KNOW the posterior distribution?

I found this question on X validated somewhat hilarious, the more because of the shouted KNOW! And the confused impression that because one can write down π(θ|x) up to a constant, one KNOWS this distribution… It is actually one of the paradoxes of simulation that, from a mathematical perspective, once π(θ|x) is available as a function of (θ,x), all other quantities related with this distribution are mathematically perfectly and uniquely defined. From a numerical perspective, this does not help. Actually, when starting my MCMC course at ENSAE a few days later, I had the same question from a student who thought facing a density function like

f(x) ∞ exp{-||x||²-||x||⁴-||x||⁶}

was enough to immediately produce simulations from this distribution. (I also used this example to show the degeneracy of accept-reject as the dimension d of x increases, using for instance a Gamma proposal on y=||x||. The acceptance probability plunges to zero with d, with 9 acceptances out of 10⁷ for d=20.)

### 2 Responses to “Why is it necessary to sample from the posterior distribution if we already KNOW the posterior distribution?”

1. betanalpha Says:

Ah, one of the most frustrating and ubiquitous misunderstandings in statistics, especially amongst practitioners! The posterior is easy to construct (or at least the joint distribution over data and parameters) — it’s _using the posterior_ that’s hard. Let’s all shout this from the rooftops and emphasize it at the beginning of all of our talks. :-)

• Just found this sentence in another X validated question: “simulations of posterior densities are introduced early in Bayesian texts so that more advanced techniques like MCMC are more intuitive”…

This site uses Akismet to reduce spam. Learn how your comment data is processed.