coupling for the Gibbs sampler

At BNP13, Brian Trippe presented the AISTAT 2022 paper he recently wrote with Tin D. Nguyen and Tamara Broderick. Which made me read their 2021 paper on the topic. There, they note that coupling may prove challenging, which they blame on label switching. Considering a naïve Gibbs sampler on the space of partitions, meaning allocating each data-point to one of the existing partitions or to a singleton, they construct an optimal transport coupling under Hamming distance. Which appears to be achievable in O(NK³log{K}), if K is the maximal number of partitions among both chains. The paper does not goes deeply into the implementation, which involves [to quote] (a) computing the distances between each pair of partitions in the Cartesian product of supports of the Gibbs conditionals and (b) solving the optimal transport problem. Except in the appendix where the book-keeping necessary to achieve O(K²) for pairwise distances and the remaining complexity follows from the standard Orlin’s algorithm. What remains unclear from the paper is that, while the chains couple faster (fastest?), the resulting estimators do not necessarily improve upon budget-equivalent alternatives. (The reason for the failure of the single chain in Figure 2 is hard to fathom.)

not summiting Volcan Osorno

While in Puerto Varas for BNP13, I tried to summit the nearby Orsono volcano (2652m) but it did not work out… Indeed, Huella Andina, the guiding company, chose to postpone the climb by two consecutive days for fear of snowfall (which did not materialise) and ended up choosing the windiest day of the week. In the meanwhile, Zhu Yichen had joined me and we all three were at the start of the trail (at about 1000m) at the earliest, starting hiking by 4:30 under a magnificent Southern night sky on top of us, with a rich Milky Way. And the Southern X. Obviously all by ourselves. The wind was strong then but quite bearable and the trail of powdery ash quite smooth till we reached a small hut to put crampons and add one more layer, around 1600m.

When we started again the wind had strengthened with the incoming dawn, rising to a point where we could hardly stand up to it, and our guide eventually decided to turn back, as it was only to get worse in the remaining 600m to the top, confirming the earlier prediction of 70km/hour winds… Yichin was of the same opinion and hence we reluctantly turned back down to the bottom of the slope, with the wind mostly on our back and a pleasant steady gradient descent. (Although my tights are still sensitive three days after the climb!) Surprisingly I was not feeling particularly cold (and with the right face protection my nose was not close to frozen, which happened on Monte Rosa in 2016). Even after seeing my best North Face mitt been blown away within a second of loosing grip of it! (Anyone with a spare left hand down mitt?! Because it had to be the left hand, of course!)

Obviously, this was quite disappointing as I had planned the climb for weeks, brought all the layers to face a potential -20⁰ windchill, was in good enough shape, did not feel any altitude effect, and we did not even reach the glacier. It was most frustrating as the summiting attempt would have worked so nicely in either one of the previous two days, but these are the rules of the mountaineering game, especially in an oceanic climate as in Southern Chile. And with narrow windows as those allowed by attending a one week conference. At least we were back to the BNP conference in time for the morning coffee break! I wish the conference participants hoping to summit later had better luck.

mixtures at BNP [slides]

After chatting with some BNP13 participants at the Puerto Montt airport, I gave in (!) to their kind request to put my slides on-line and here is the link to the slideshare depository. It was quite the nice coincidence that Sanjib Basu (whom I met in Purdue in 1987!) gave the invited talk in our session since we were building on the under-appreciated Basu-Chib approximation of the evidence. Overall, this was an exhilarating week and I now have to recover from this sensory overload. (Incidentally, and uninterestingly, I got swindled by not one but two taxis on my way back to Santiago!)

Testing for mixtures at BNP 13 from Christian Robert

BNP13

BNP13 is set in this incredible location on a massive lake (almost as large as Lac Saint Jean!) facing several tantalizing snow-capped volcanoes… My trip from Paris to Puerto Varas was quite smooth if relatively longish (but I slept close to 8 hours on the first leg and busied myself with Biometrika submissions the rest of the way). Leaving from Paris at midnight proved a double advantage as this was one of the last flights leaving, with hardly anyone in the airport. On Sunday, I arrived early enough to take a quick dip in Lake Llanquihue which was fairly cold and choppy!

Overall the conference is quite exhilarating as all talks are of interest and often covering on-going research. This may be one of the most engaging meetings I have attended in the past years! Plus a refreshing variety of topics and seniority in the speakers.

To start with a bang!, Sonia Petrone (Bocconi) gave a very nice plenary lecture in the most auspicious manner, covering her recent works on Bayesian prediction as an alternative way to run Bayesian inference (in connection with the incoming Read Paper by Fong et al.). She covered so much ground that I got lost before long (jetlag did not help!). However, an interesting feature underlying her talk is that, under exchangeability, the sequence of predictives converges to a random probability measure, a de Finetti way to construct the prior that is based on predictives. Avoiding in a sense the model and the prior on the parameters of that process. (The parameter is derived from the infinite exchangeable [or conditionally iid] sequence, but the sequence of predictives need be defined.) The drawback is that this approach involves infinite sequences, with practical truncation to a finite horizon being an approximation whose precision / error may prove elusive to characterise. The predictive approach also allows to recover a limiting Normal distribution (not a Bernstein-von Mises type!) and hence credible intervals on parameters and distributions.

While this is indeed a BNP conference (!), I was surprised to see lot of talks paying attention to clustering and even to mixtures, with again a recurrent imprecision on the meaning of a cluster. (Maybe this was already the case for BNP11 in Paris but I may have been too busy helping with catering to notice!) For instance, Brian Trippe (MIT) gave a quick intro on his (AISTATS 2022) work on parallel MCMC with coupling. As unbiased MCMC strongly improving upon naïve parallel MCMC relative to the computing cost. With an interesting example where coupling is agnostic to the labeling of random partitions in clustering problems, involving optimal transport, manageable in O(K³log(K)) time when K is the number of clusters.

sunrise on Lago Llanquihue [jatp]