transport, diffusions, and sampling

At the Sampling, Transport, and Diffusions workshop at the Flatiron Institute, on Day #2, Marilou Gabrié (École Polytechnique) gave the second introductory lecture on merging sampling and normalising flows targeting the target distribution, when driven by a divergence criterion like KL, that only requires the shape of the target density. I first wondered about ergodicity guarantees in simultaneous MCMC and map training due to the adaptation of the flow but the update of the map only depends on the current particle cloud in (8). From an MCMC perspective, it sounds somewhat paradoxical to see the independent sampler making such an unexpected come-back when considering that no insider information is available about the (complex) posterior to drive the [what-you-get-is-what-you-see] construction of the transport map. However, the proposed approach superposed local (random-walk like) and global (transport) proposals in Algorithm 1.

Qiang Liu followed on learning transport maps, with the  Interesting notion of causalizing a graph by removing intersections (which are impossible for an ODE, as discussed by Eric Vanden-Eijden’s talk yesterday) through  coupling. Which underlies his notion of rectified flows. Possibly connecting with the next lightning talk by Jonathan Weare on spurious modes created by a variational Monte Carlo sampler and the use of stochastic gradient, corrected by (case-dependent?) regularisation.

Then came a whole series of MCMC talks!

Sam Livingstone spoke on Barker’s proposal (an incoming Biometrika paper!) as part of a general class of transforms g of the MH ratio, using jump processes based on a nasty normalising constant related with g (tractable for the original Barker algorithm). I then realised I had missed his StatSci paper on how to speak to statistical physics researchers!

Charles Margossian spoke about using a massive number of short parallel runs (many-short-chain regime) from a recent paper written with Aki,  Andrew, and Lionel Riou-Durand (Warwick) among others. Which brings us back to the challenge of producing convergence diagnostics and precisely the Gelman-Rubin R statistic or its recent nR avatar (with its linear limitations and dependence on parameterisation, as opposed to fuller distributional criteria). The core of the approach is in using blocks of GPUs to improve and speed-up the estimation of the between-chain variance. (D for R².) I still wonder at a waste of simulations / computing power resulting from stopping the runs almost immediately after warm-up is over, since reaching the stationary regime or an approximation thereof should be exploited more efficiently. (Starting from a minimal discrepancy sample would also improve efficiency.)

Lu Zhang also talked on the issue of cutting down warmup, presenting a paper co-authored with Bob, Andrew, and Aki, recommending Laplace / variational approximations for reaching faster high-posterior-density regions, using an algorithm called Pathfinder that relies on ELBO checks to counter poor performances of Laplace approximations. In the spirit of the workshop, it could be profitable to further transform / push-forward the outcome by a transport map.

Yuling Yao (of stacking and Pareto smoothing fame!) gave an original and challenging (in a positive sense) talk on the many ways of bridging densities [linked with the remark he shared with me the day before] and their statistical significance. Questioning our usual reliance on arithmetic or geometric mixtures. Ignoring computational issues, selecting a bridging pattern sounds not different from choosing a parameterised family of embedding distributions. This new typology of models can then be endowed with properties that are more or less appealing. (Occurences of the Hyvärinen score and our mixtestin perspective in the talk!)

Miranda Holmes-Cerfon talked about MCMC on stratification (illustrated by this beautiful picture of nanoparticle random walks). Which means sampling under varying constraints and dimensions with associated densities under the respective Hausdorff measures. This sounds like a perfect setting for reversible jump and in a sense it is, as mentioned in the talks. Except that the moves between manifolds are driven by the proximity to said manifold, helping with a higher acceptance rate, and making the proposals easier to construct since projections (or the reverses) have a physical meaning. (But I could not tell from the talk why the approach was seemingly escaping the symmetry constraint set by Peter Green’s RJMCMC on the reciprocal moves between two given manifolds).

Fusion at CIRM

Today is the first day of the FUSION workshop Rémi Bardenet and myself organised. Due to schedule clashes, I will alas not be there, since [no alas!] at the BNP conference in Chili. The program and collection of participants is quite exciting and I hope more fusion will result from this meeting. Enjoy! (And beware of boars, cold water, and cliffs!!!)

Bayesian Methods for the Social Sciences [and for the Parisians]

[Reposting the announcement of a Bayesian conference in Paris, next October, with a fabulous list of friends speakers! Note that this is the week prior to our Fusion workshop at CIRM, Marseille. And to BNP.]

This three-day workshop will gather statisticians, mathematicians and social scientists around the theme of Bayesian statistical methods for the social sciences. This area has been growing rapidly in the past decade, and the speakers will include some of the leading researchers in the area from around the World.

The first day will consist of tutorial introductions to Bayesian inference, demography and social network analysis. Days 2 and 3 will consist of talks and posters on cutting-edge research in the area. The workshop is sponsored by the Fondation des Sciences Mathématiques de Paris (FSMP), and the tutorial sessions are organised jointly with the French Institute of Mathematics for Planet Earth.

It will be held in person at the Institut Henri Poincaré in Paris from October 19 to 21, 2022.

me no savi [travel madness]

Today, I left home in the wee hours, after watering my tomatoes!, quite excited to join the Safe, Anytime-Valid Inference (SAVI) workshop in Eindhoven, which was taking place after two years of postponement. I alas did not check the state of the train traffic beforehand and when I reached the train station I found that part of the line to De Gaulle airport was closed, due to some control cables being stolen last night. Things quickly deteriorated as the train management in Gare du Nord was pretty inefficient, meaning that the trains would stop for five minutes at each station, and that there was no rail alternative to reach Roissy. The taxi stand was a complete mess, with no queue whatsoever, and the Parisian taxis kept true to their reputation, by refusing to take people to the airport, asking for outrageous prices (60 euros per passenger), and stopping anywhere. I almost managed to get one but he refused to take me on top of the Swede family I had directed to this stand from the RER train, and this was simply my last opportunity. Über taxis were invisible and I soon realised I could not catch my flight. Later flights were outrageously expensive and there was not train seat whatsoever till the day after, so I gave up and returned home from this trip to nowhere…

Foundations of objective Bayesian methodology [21w5107]

After years in the making (!), our BIRS-CMO workshop on the foundations of O’Bayes is at last taking place! In an hybrid format as BIRS-CMO is restricting the attendance to 15 people on site, instead of the customary (i.e., pre-COVID) 35. Still, it is quite exciting to join this workshop and the friends who will gather in Mexico or on-line to discuss objective Bayesian tools and prospects. And of course to visit for the second time the city of Oaxaca, its temples and markets! (Hopefully managing the stray dogs when running. If running.)