Indeed. But, unless the prior is flat as well, it should normalise the likelihood to some extent. And if not MCMC should have a heyday running around the parameter space…

]]>However, isn’t there a class of functions which are not flat but have nearly flat plateaus which are comparably problematic? For example, this is a known numerical problem for Maximum Likelihood, per

Lima, Verônica MC, and Francisco Cribari–Neto. “Penalized maximum likelihood estimation in the modified extended Weibull distribution.” Communications in Statistics-Simulation and Computation 48, no. 2 (2019): 334-349

and is also emphasized in

Konishi, Sadanori, and Genshiro Kitagawa. Information criteria and statistical modeling. Springer Science & Business Media, 2008.

In particular Konishi and Kitagawa make the point that if likelihoods are monotonic in a region but have sufficiently small absolute slopes, the numerical uncertainty in the MLE is high, even if it exists, because the likelihood itself can be perturbed by variations in parameters.

]]>This may be the exceptional counter-example, though!

]]>