**P**aulo (a.k.a., Zen) posted a comment in StackExchange on Larry Wasserman‘s paradox about Bayesians and likelihoodists (or likelihood-wallahs, to quote Basu!) being unable to solve the problem of estimating the normalising constant *c* of the sample density, *f*, known up to a constant

(Example 11.10, page 188, of *All of Statistics*)

**M**y own comment is that, with all due respect to Larry!, I do not see much appeal in this example, esp. as a potential criticism of Bayesians and likelihood-wallahs…. The constant *c* is known, being equal to

If *c* is the only “unknown” in the picture, given a sample *x*_{1},…,x_{n}, then there is no statistical issue whatsoever about the “problem” and I do not agree with the postulate that there exist *estimators* of *c*. Nor *priors* on *c* (other than the Dirac mass on the above value). This is not in the least a statistical problem but rather a *numerical *issue.That the sample *x*_{1},…,x_{n} can be (re)used through a (frequentist) density estimate to provide a numerical approximation of *c*

is a mere curiosity. Not a criticism of alternative statistical approaches: e.g., I could also use a Bayesian density estimate…

**F**urthermore, the estimate provided by the sample *x*_{1},…,x_{n} is not of particular interest since its precision is imposed by the sample size *n* (and converging at non-parametric rates, which is not a particularly relevant issue!), while I could use importance sampling (or even numerical integration) if I was truly interested in *c*. I however find the discussion interesting for many reasons

- it somehow relates to the infamous harmonic mean estimator issue, often discussed on the’Og!;
- it brings more light on the paradoxical differences between statistics and Monte Carlo methods, in that statistics is usually constrained by the sample while Monte Carlo methods have more freedom in generating samples (up to some budget limits). It does not make sense to speak of
*estimators* in Monte Carlo methods because there is no parameter in the picture, only “unknown” constants. Both fields rely on samples and probability theory, and share many features, but there is nothing like a “best unbiased estimator” in Monte Carlo integration, see the case of the “optimal importance function” leading to a zero variance;
- in connection with the previous point, the fascinating Bernoulli factory problem is not a statistical problem because it requires an infinite sequence of Bernoullis to operate;
- the discussion induced Chris Sims to contribute to StackExchange!

### Like this:

Like Loading...