LGM 2012, Trondheim

A break from the “snapshots from Guérande” that will be a relief for all ‘ Og readers, I am sure: I am now in Trondheim, Norway, for the second Latent Gaussian model meeting, organised by Håvard Rue and his collaborators. As in the earlier edition in Zürich, the main approach to those models (that is adopted in the talks) is the INLA methodology of Rue, Martino and Chopin. I nonetheless (given the theme) gave a presentation on Rao-Blackwellisation techniques for MCMC algorithms. As I had not printed the program of the meeting prior to my departure (blame Guérande!), I had not realised I had only 20 minutes for my talk and kept adding remarks and slides during the flight from Amsterdam to Trondheim [where the clouds prevented me from seeing Jotunheimen]. (So I had to cut the second half of the talk below on parallelisation. Even with this cut, the 20 minutes went awfully fast!) Apart from my talk, I am afraid I was not in a sufficient state of awareness [due to a really early start] to give a comprehensive of the afternoon talks….

Trondheim is a nice city that sometimes feels like a village despite its size. Walking up to the university along typical wooden houses, then going around the town and along the river tonight while running a 10k loop left me with the impression of a very pleasant place (at least in the summer months).

snapshot from Guérande (4)

the universe in zero words

The universe in zero words: The story of mathematics as told through equations is a book with a very nice cover: in case you cannot see the details on the picture, what looks like stars on a bright night sky are actually equations discussed in the book (plus actual stars!)…

The universe in zero words is written by Dana Mackenzie (check his website!) and published by Princeton University Press. (I received it in the mail from John Wiley for review, prior to its publication on May 16, nice!) It reads well and quick: I took it with me in the métro one morning and was half-way through it the same evening, as the universe in zero words remains on the light side, esp. for readers with a high-school training in math. The book strongly reminded me (at times) of my high school years and of my fascination for Cardano’s formula and the non-Euclidean geometries. I was also reminded of studying quaternions for a short while as an undergraduate by the (arguably superfluous) chapter on Hamilton. So a pleasant if unsurprising read, with a writing style that is not always at its best, esp. after reading Bill Bryson’s “Seeing Further: The Story of Science, Discovery, and the Genius of the Royal Society“, and a book unlikely to bring major epiphanies to the mathematically inclined. If well-documented, free of typos, and engaging into some mathematical details (accepting to go against the folk rule that “For every equation you put in, you will lose half of your audience.” already mentioned in Diaconis and Graham’s book). With alas a fundamental omission: no trace is found therein of Bayes’ formula! (The very opposite of Bryson’s introduction, who could have arguably stayed away from it.) The closest connection with statistics is the final chapter on the Black-Scholes equation, which does not say much about probability…. It is of course the major difficulty with the exercise of picking 24 equations out of the history of maths and physics that some major and influential equations had to be set aside… Maybe the error was in covering (or trying to cover) formulas from physics as well as from maths. Now, rather paradoxically (?) I learned more from the physics chapters: for instance, the chapters on Maxwell’s, Einstein’s, and Dirac’s formulae are very well done. The chapter on the fundamental theorem of calculus is also appreciable.

snapshot from Guérande (3)

optimal direction Gibbs

An interesting paper appeared on arXiv today. Entitled On optimal direction gibbs sampling, by Andrés Christen, Colin Fox, Diego Andrés Pérez-Ruiz and Mario Santana-Cibrian, it defines optimality as picking the direction that brings the maximum independence between two successive realisations in the Gibbs sampler. More precisely, it aims at choosing the direction e that minimises the mutual information criterion

I have a bit of an issue about this choice because it clashes with measure theory. Indeed, in one Gibbs step associated with e the transition kernel is defined in terms of the Lebesgue measure over the line induced by e. Hence the joint density of the pair of successive realisations is defined in terms of the product of the Lebesgue measure on the overall space and of the Lebesgue measure over the line induced by e… While the product in the denominator is defined against the product of the Lebesgue measure on the overall space and itself. The two densities are therefore not comparable since not defined against equivalent measures… The difference between numerator and denominator is actually clearly expressed in the normal example (page 3) when the chain operates over a n dimensional space, but where the conditional distribution of the next realisation is one-dimensional, thus does not relate with the multivariate normal target on the denominator. I therefore do not agree with the derivation of the mutual information henceforth produced as (3).

The above difficulty is indirectly perceived by the authors, who note “we cannot simply choose the best direction: the resulting Gibbs sampler would not be irreducible” (page 5), an objection I had from an earlier page… They instead pick directions at random over the unit sphere and (for the normal case) suggest using a density over those directions such that

which cannot truly be called “optimal”.

More globally, searching for “optimal” directions (or more generally transforms) is quite a worthwhile idea, esp. when linked with adaptive strategies…