a passage to & from India

Our trip from Paris (CDG) to Bengaluru got a wee bit (!) perturbed by 2x bad luck, with a first plane grounded for damages to a wing and a second plane flashing an alarm signal just as it was accelerating to take off, which induced an extra hour of tests, plus an unexpected long wait to get the e-visa at the Bengalore airport, resulting in an arrival in town at 5:30 am! A good thing that my talk was only the next day.

I was glad to be back at the (Tata) Indian Institute of Science and its wonderful campus for the IISA meeting (taking place alternately in India and in the US). The conference program was rich and with a large Bayesian component, but being sleep deprived and slightly sick did not help with my concentration during the talks… Had however nice discussions during the poster session, including one on a most unusual RJMCMC where the model-to-model transform was the identity. In a sense this voided (?) the need for RJMCMC, but it allowed for a fast & valid exploration of the different models.

Quite a contrast in my local lodging conditions, when compared with my previous visit,  since, rather than staying in the ideal visitors’ lodge located at the centre of the campus, I took the (bargain) offer (from IISA) of the nearby Sheraton (!) as the conference hotel with five star conditions, including a proper, outside, empty and non-heated swimming pool.

The (touristy) train trip to Mysore was most pleasant, on an air-conditioned carriage with food vendors proposing their wares all along the journey, great views of the countryside and an arrival sharp on time. The reverse trip to the airport was less successful as the FlyBus we took was crawling rather than flying, with heavy traffic all the way because/despite being New Year Eve’ning.  At some point, a truck carrying what looked like kindling was stuck in a pothole, blocking the highway, and a crane was brought on site to push the truck out of the hole, a strategy that surprisingly worked. But we managed to reach the airport just before midnight, when absolutely nothing happened in relation with the entry into 2023!

sampling, transport, and diffusions

This week, I am attending a very cool workshop at the Flatiron Institute (not in the Flatiron building!, but close enough) on Sampling, Transport, and Diffusions, organised by Bob Carpenter and Michael Albergo. It is quite exciting as I do not know most participants or their work! The Flatiron Institute is a private institute focussed on fundamental science funded by the Simons Foundation (in such working conditions universities cannot compete with!).

Eric Vanden-Eijden gave an introductory lecture on using optimal transport notion to improve sampling, with a PDE/ODE approach of continuously turning a base distribution into a target (formalised by the distribution at time one). This amounts to solving a velocity solution to an KL optimisation objective whose target value is zero. Velocity parameterised as a deep neural network density estimator. Using a score function in a reverse SDE inspired by Hyvärinnen (2005), with a surprising occurrence of Stein’s unbiased estimator, there for the same reasons of getting rid of an unknown element. In a lot of environments, simulating from the target is the goal and this can be achieved by MCMC sampling by normalising flows, learning the transform / pushforward map.

At the break, Yuling Yao made a very smart remark that testing between two models could also be seen as an optimal transport, trying to figure an optimal transform from one model to the next, rather than the bland mixture model we used in our mixtestin paper. At this point I have no idea about the practical difficulty of using / inferring the parameters of this continuum but one could start from normalising flows. Because of time continuity, one would need some driving principle.

Esteban Tabak gave another interest talk on simulating from a conditional distribution, which sounds like a no-problem when the conditional density is known but a challenge when only pairs are observed. The problem is seen as a transport problem to a barycentre obtained as a distribution independent from the conditioning z and then inverting. Constructing maps through flows. Very cool, even possibly providing an answer for causality questions.

Many of the transport talks involved normalizing flows. One by [Simons Fellow] Christopher Jazynski about adding to the Hamiltonian (in HMC) an artificial flow field  (Vaikuntanathan and Jarzynski, 2009) to make up for the Hamiltonian moving too fast for the simulation to keep track. Connected with Eric Vanden-Eijden’s talk in the end.

An interesting extension of delayed rejection for HMC by Chirag Modi, with a manageable correction à la Antonietta Mira. Johnatan Niles-Weed provided a nonparametric perspective on optimal transport following Hütter+Rigollet, 21 AoS. With forays into the Sinkhorn algorithm, mentioning Aude Genevay’s (Dauphine graduate) regularisation.

Michael Lindsey gave a great presentation on the estimation of the trace of a matrix by the Hutchinson estimator for sdp matrices using only matrix multiplication. Solution surprisingly relying on Gibbs sampling called thermal sampling.

And while it did not involve optimal transport, I gave a short (lightning) talk on our recent adaptive restore paper: although in retrospect a presentation of Wasserstein ABC could have been more suited to the audience.

sampling using adaptive regenerative processes

We just posted a new arXival on Sampling using Adaptive Regenerative Processes, written by Hector McKimm (Warwick), Andi Wang (soon Warwick), Murray Pollock (ex-Warwick), Gareth Roberts (Warwick) and myself. This is a collaborative that has been going on for a while, mostly via zoom in these Covid times. It builds upon the earlier paper of Wang et al.  (2021) constructing the regeneration process (Restore), by aiming at improving this process by adapting the regeneration distribution and hence dramatically reducing the number of regenerations. Gaining in addition the ability to sample from target distributions for which simulation under a fixed regeneration distribution is computationally intractable. This work is part of Hector’s PhD, written at Warwick.