Non-reversible Markov Chains for Monte Carlo sampling

the pond in front of the Zeeman building, University of Warwick, July 01, 2014This “week in Warwick” was not chosen at random as I was aware there is a workshop on non-reversible MCMC going on. (Even though CRiSM sponsored so many workshops in September that almost any week would have worked for the above sentence!) It has always been kind of a mystery to me that non-reversibility could make a massive difference in practice, even though I am quite aware that it does. And I can grasp some of the theoretical arguments why it does. So it was quite rewarding to sit in this Warwick amphitheatre and learn about overdamped Langevin algorithms and other non-reversible diffusions, to see results where convergence times moved from n to √n, and to grasp some of the appeal of lifting albeit in finite state spaces. Plus, the cartoon presentation of Hamiltonian Monte Carlo by Michael Betancourt was a great moment, not only because of the satellite bursting into flames on the screen but also because it gave a very welcome intuition about why reversibility was inefficient and HMC appealing. So I am grateful to my two colleagues, Joris Bierkens and Gareth Roberts, for organising this exciting workshop, with a most profitable scheduling favouring long and few talks. My next visit to Warwick will also coincide with a workshop on intractable likelihood, next November. This time part of the new Alan Turing Institute programme.

4 Responses to “Non-reversible Markov Chains for Monte Carlo sampling”

  1. Dan Simpson Says:

    Ok. I’ve clearly fallen asleep at the wheel, because I always thought HMC Was a metropolis corrected Hamiltonian flow (and hence reversible). Is it not???

    • Michael Betancourt Says:

      Dan, you should have been at the workshop where I discussed exactly this point in detail! To summarize, Hamiltonian flow and the modified Hamiltonian flow of a symplectic integrator are both non-reversible hence effective at exploring the target. But if we want to unbias with a Metropolis the samples then we have to augment the flow (say with a momentum flip before and then a momentum resampling after) to make it reversible. The problem is that such an augmentation compromises the performance of the flow — for example if we apply corrections with a high frequency then we devolve into a Langevin diffusion. Only by integrating for long enough can we resolve that tension, and formalizing that idea very naturally motivates algorithms like NUTS.

    • I know nothing, but I do not think so.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: