as you wrote later, numerical integration does not work when the nuisance parameter is highly dimensional; so there may be cases where this is the “only” possible approach. A different question from Nicolas during the seminar was about the purpose of this likelihood derivation: if a Bayesian analysis is not the final goal, one simply needs the mode and the curvature at the mode to run a likelihood analysis. So simpler algorithms could make more sense.

]]>(I seem to be leaving a lot of comments here)

Interesting, the hack-y solution (compute the effective support and put an easy (or appropriate) integration rule over it) matches well with the version of reference priors in Berger etc’s Annals of Statistics paper, in which they presented an algorithm for computing the reference prior at a discrete set of points.

I also know that this is how everything is implemented in INLA and I strongly suspect that everyone who’s ever written a “partially collapsed” Gibbs sampler (I love that terminology – it’s so evocative) does it this way (I think it’s in Chris Strickland’s PyMCMC package, but I may be wrong. Possibly also STAN, but I’m less sure about that… I know they do this for discrete variables, but it’s a bit different there [unless you approximate the sum by and integral that you approximate by a different sum])

]]>That comment is predicated on the parameter in question being fairly low dimensional (<10), but improper priors in high dimensions are, I'm pretty sure, quite weird. It's definitely true as the dimension goes to infinity (think of putting a prior on a function space. Most functions are really weird, and we probably don't want prior mass on those ones), but I feel like this "weirdness" kicks in pretty quickly as the dimension grows.

]]>Please feel free to add all of the usual comments about improper priors and how, if it’s easier to work with proper priors, it may be a thing worth considering…

]]>