unbiased HMC
Jeremy Heng and Pierre Jacob arXived last week a paper on unbiased Hamiltonian Monte Carlo by coupling, following the earlier paper of Pierre and co-authors on debiasing by coupling a few weeks ago. The coupling within the HMC amounts to running two HMC chains with common random numbers, plus subtleties!
“As with any other MCMC method, HMC estimators are justified in the limit of the number of iterations. Algorithms which rely on such asymptotics face the risk of becoming obsolete if computational power keeps increasing through the number of available processors and not through clock speed.”
The main difficulty here is to have both chains meet (exactly) with large probability, since coupled HMC can only bring these chain close to one another. The trick stands in using both coupled HMC and coupled Hastings-Metropolis kernels, since the coupled MH kernel allows for exact meetings when the chains are already close, after which they remain happily and forever together! The algorithm is implemented by choosing between the kernels at random at each iteration. (Unbiasedness follows by the Glynn-Rhee trick, which is eminently well-suited for coupling!) As pointed out from the start of the paper, the appeal of this unbiased version is that the algorithm can be (embarrassingly) parallelised since all processors in use return estimators that are iid copies of one another, hence easily merged into a better estimator.
September 25, 2017 at 2:01 pm
Thanks Christian! It’s good to read some supporting comments before seeing it rejected with insulting reviews by the most prestigious journals of our field!
September 25, 2017 at 5:45 pm
Hopefully it will not come to this dark outcome!
September 25, 2017 at 6:44 pm
Nah. Regardless of the quality of the paper (haven’t read it, but it’s on my [alarmingly large] pile), this is probably the most likely outcome.
It would be nice if it wasn’t the case.
September 25, 2017 at 9:51 pm
Thanks for your encouraging remarks, Dan!!!
September 26, 2017 at 4:59 am
I am nothing if not a source of positivity and joy in the world.
September 26, 2017 at 9:12 am
If. Not.