Bob, any news on this comparison you were mentioning a while ago..?!

]]>emcee is a nice piece of software in that it lets you plug in an arbitrarily-defined density function in Python. So people are using it with large, externally-defined models that often have person-years invested in them, such as big physics models for astronomy or energy usage models simulating buildings — something you can’t do with BUGS/JAGS/Stan (though you could do with a simple Metropolis algorithm).

The argument for affine invariance is that you don’t have to do any adaptation or reparameterization gymnastics. My understanding is that Gibbs is scale invariant, but not rotation invariant, whereas HMC with fixed step size and unit mass matrix is rotation invariant, but not scale invariant. If you estimate a dense mass matrix for HMC, (I think) you can get global scale invariance and preserve rotation invariance. Of course, RHMC gives you the best of both worlds in that it’s not only scale and rotation invariant, it adapts to local curvature of the kind you get in hierarchical models (like Neal’s “textbook” example, the funnel [or “whirlpool” as Andrew would have it]).

It’d be nice to see some evaluations of these ensemble samplers compared to other methods. Especially for hierarchical models. We plan to implement both the ter Braak and Goodman/Weare samplers in Stan, so we should be able to do this for a large range of relatively simple models. But we probably won’t get to it soon (unless someone volunteers…).

]]>