## Why is it necessary to sample from the posterior distribution if we already KNOW the posterior distribution?

**I** found this question on X validated somewhat hilarious, the more because of the shouted KNOW! And the confused impression that because one can write down π(θ|x) up to a constant, one KNOWS this distribution… It is actually one of the paradoxes of simulation that, from a mathematical perspective, once π(θ|x) is available as a function of (θ,x), all other quantities related with this distribution are mathematically perfectly and uniquely defined. From a numerical perspective, this does not help. Actually, when starting my MCMC course at ENSAE a few days later, I had the same question from a student who thought facing a density function like

f(x) ∞ exp{-||x||²-||x||⁴-||x||⁶}

was enough to immediately produce simulations from this distribution. (I also used this example to show the degeneracy of accept-reject as the dimension d of x increases, using for instance a Gamma proposal on y=||x||. The acceptance probability plunges to zero with d, with 9 acceptances out of 10⁷ for d=20.)

October 27, 2017 at 2:33 am

Ah, one of the most frustrating and ubiquitous misunderstandings in statistics, especially amongst practitioners! The posterior is easy to construct (or at least the joint distribution over data and parameters) — it’s _using the posterior_ that’s hard. Let’s all shout this from the rooftops and emphasize it at the beginning of all of our talks. :-)

October 27, 2017 at 12:44 pm

Just found this sentence in another X validated question: “simulations of posterior densities are introduced early in Bayesian texts so that more advanced techniques like MCMC are more intuitive”…