Chris Oates, Theodore Papamarkou, and Mark Girolami (all from the University of Warwick) just arXived a paper on a new form of thermodynamic integration for computing marginal likelihoods. (I had actually discussed this paper with the authors on a few occasions when visiting Warwick.) The other name of thermodynamic integration is path sampling (Gelman and Meng, 1998). In the current paper, the path goes from the prior to the posterior by a sequence of intermediary distributions using a power of the likelihood. While the path sampling technique is quite efficient a method, the authors propose to improve it through the recourse to control variates, in order to decrease the variance. The control variate is taken from Mira et al. (2013), namely a one-dimensional temperature-dependent transform of the score function. (Strictly speaking, this is an asymptotic control variate in that the mean is only asymptotically zero.) This control variate is then incorporated within the expectation inside the path sampling integral. Its arbitrary elements are then calibrated against the variance of the path sampling integral. Except for the temperature ladder where the authors use a standard geometric rate, as the approach does not account for Monte Carlo and quadrature errors. (The degree of the polynomials used in the control variates is also arbitrarily set.) Interestingly, the paper mixes a lot of recent advances, from the zero variance notion of Mira et al. (2013) to the manifold Metropolis-adjusted Langevin algorithm of Girolami and Calderhead (2011), uses as a base method pMCMC (Jasra et al., 2007). The examples processed in the paper are regression (where the controlled version truly has a zero variance!) and logistic regression (with the benchmarked Pima Indian dataset), with a counter-example of a PDE interestingly proposed in the discussion section. I quite agree with the authors that the method is difficult to envision in complex enough models. I also did not see mentions therein of the extra time involved in using this control variate idea.
Archive for the Books Category
Last month, I ordered several books on amazon, taking advantage of my amazon associate gains, and some of them were suggested by amazon algorithms based on my recent history. As I had recently read books involving thieves (like Giant Thief, or Broken Blade and the subsequent books), a lot of titles involved thieves or thievery related names… I picked Den of Thieves mainly for its cover as I did not know the author and the story sounded rather common. When I started reading the book, the story got more and more common, pertaining more to an extended Dungeons & Dragons scenario than to a genuine book! The theme of a bright young thief emerging from the gritty underworld of a close city has been over and over exploited in the fantasy literature, the best (?) example being The lies of Locke Lamora. (Whose third volume, The Republic of Thieves, is in my bag for Reykjavik!) This time, the thief does not appear particularly bright, except at times when he starts philosophy-sing with extremely dangerous enemies!, and the way he eventually overcomes insanely unbalanced odds is just too much. Most characters in the novel are not particularly engaging and way too much caricaturesque from the terribly evil sorcerer cavorting with she-demons to the rigid knight sticking to an idealistic vision of the world where ‘honour” and the code of chivalry is the solution to all problems. It is not even in the slightest sarcastic or tongue-in-cheek as the many novels by David Eddings and the main characters are mostly humourless. I wonder why the book did not get better edited as the weaknesses are very easy to spot! A good example where amazon software failed to make a worthy recommendation!
The English title of this 2007 book of Murakami is “What I talk about when I talk about running”. Which is a parody of Raymond Carver’s collection of [superb] short stories, “What we talk about when we talk about love”. (Murakami translated the complete œuvres of Raymond Carver in Japanese.) It is a sort of diary about Murakami’s running practice and the reasons why he is running. It definitely is not a novel and the style is quite loose or lazy, but this is not a drawback as the way the book is written somehow translates the way thoughts drift away and suddenly switch topics when one is running. At least during low-intensity practice, when I often realise I have been running for minutes without paying any attention to my route. Or when I cannot recall what I was thinking about for the past minutes. During races, the mind concentration is at a different level, first focussing on keeping the right pace, refraining from the deadly rush during the first km, then trying to merge with the right batch of runners, then fighting wind, slope, and eventually fatigue. While the book includes more general autobiographical entries than those related with Murakami’s runner’s life, there are many points most long-distance runners would relate with. From the righteous feeling of sticking to a strict training and diet, to the almost present depression catching us in the final kms of a race, to the very flimsy balance between under-training and over-training, to the strangely accurate control over one’s pace at the end of a training season, and, for us old runners, to the irremediable decline in one’s performances as years pass by… On a more personal basis, I also shared the pain of hitting one of the slopes in Central Park and the lack of nice long route along Boston’s Charles river. And shared the special pleasure of running near a river or seafront (which is completely uncorrelated with the fact it is flat, I believe!) Overall, what I think this book demonstrates is that there is no rational reason to run, which makes the title more than a parody, as fighting weight, age, health problems, depression, &tc. and seeking solitude, quiet, exhaustion, challenge, performances, zen, &tc. are only partial explanations. Maybe the reason stated in the book that I can relate the most with is this feeling of having an orderly structure one entirely controls (provided the body does not rebel!) at least once a day. Thus, I am not certain the book appeals to non-runners. And contrary to some reviews of the book, it certainly is not a training manual for novice runners. (Murakami clearly is a strong runner so some of his training practice could be harmful to weaker runners…)
After Broken Blade and its sequel Bared Blade, Kelly McCullough wrote Crossed Blades that I had ordered along with Bared Blade. And once again I read this volume within a few evenings. It is still very enjoyable, maybe the more given that there is a continuity in the characters and the plots. However, I did prefer Bared Blade to Crossed Blades as the former was creative in terms of plot and environment. Here, in Crossed Blades, the main character Aral is facing his past, from the destruction of his religious order and of his goddess to the possible treachery of former friends and mentors, to his attempt to drown this past in top quality whisky… While dealing with an adopted teenage daughter in the midst of a typical teenage crisis. This new instalment is thus full of introspection and reminiscence of past loves, and frankly a bit dull at times, even though there is a (spoiler warning!!) massive battle against the culprits for the destruction of the order. The very end is a bit disappointing, but it also hopefully closes a chapter in the hero’s life, which means that the next volume, Blade Reforged, may run into new territories and more into simili-detective stories. (Two more books in this Blade series are in the making!)
As the Le Monde mathematical puzzle of this week was a geometric one (the quadrangle ABCD is divided into two parts with the same area, &tc…) , with no clear R resolution, I chose to bypass it. In this April 3 issue, several items of interest: first, a report by Etienne Ghys on Yakov Sinaï’s Abel Prize for his work “between determinism and randomness”, centred on ergodic theory for dynamic systems, which sounded like the ultimate paradox the first time I heard my former colleague Denis Bosq give a talk about it in Paris 6. Then a frightening fact: the summer conditions have been so unusually harsh in Antarctica (or at least near the Dumont d’Urville French austral station) that none of the 15,000 Adélie penguin couples studied there managed to keep their chick alive. This was due to an ice shelf that did not melt at all over the summer, forcing the penguins to walk an extra 40k to reach the sea… Another entry on the legal obligation for all French universities to offer a second chance exam, no matter how students are evaluated in the first round. (Too bad, I always find writing a second round exam a nuisance.)