Chris Oates, Theodore Papamarkou, and Mark Girolami (all from the University of Warwick) just arXived a paper on a new form of thermodynamic integration for computing marginal likelihoods. (I had actually discussed this paper with the authors on a few occasions when visiting Warwick.) The other name of thermodynamic integration is path sampling (Gelman and Meng, 1998). In the current paper, the path goes from the prior to the posterior by a sequence of intermediary distributions using a power of the likelihood. While the path sampling technique is quite efficient a method, the authors propose to improve it through the recourse to control variates, in order to decrease the variance. The control variate is taken from Mira et al. (2013), namely a one-dimensional temperature-dependent transform of the score function. (Strictly speaking, this is an asymptotic control variate in that the mean is only asymptotically zero.) This control variate is then incorporated within the expectation inside the path sampling integral. Its arbitrary elements are then calibrated against the variance of the path sampling integral. Except for the temperature ladder where the authors use a standard geometric rate, as the approach does not account for Monte Carlo and quadrature errors. (The degree of the polynomials used in the control variates is also arbitrarily set.) Interestingly, the paper mixes a lot of recent advances, from the zero variance notion of Mira et al. (2013) to the manifold Metropolis-adjusted Langevin algorithm of Girolami and Calderhead (2011), uses as a base method pMCMC (Jasra et al., 2007). The examples processed in the paper are regression (where the controlled version truly has a zero variance!) and logistic regression (with the benchmarked Pima Indian dataset), with a counter-example of a PDE interestingly proposed in the discussion section. I quite agree with the authors that the method is difficult to envision in complex enough models. I also did not see mentions therein of the extra time involved in using this control variate idea.
Archive for the Running Category
First day at AISTATS 2014! After three Icelandic vacations days driving (a lot) and hinkg (too little) around South- and West-Iceland, I joined close to 300 attendees for this edition of the AISTATS conference series. I was quite happy to be there, if only because I had missed the conference last year (in Phoenix) and did not want this to become a tradition… Second, the mix of statistics, artificial intelligence and machine learning that characterises this conference is quite exciting, if challenging at time. What I most appreciated in this discovery of the conference is the central importance of the poster session, most talks being actually introductions to or oral presentations of posters! I find this feature terrific enough (is there such a notion as “terrific enough”?!) worth adopting in future conferences I am involved in. I just wish I had managed to tour the whole collection of posters today… The (first and) plenary lecture was delivered by Peter Bühlman, who spoke about a compelling if unusual (for me) version of causal inference. This was followed by sessions on Gaussian processes, graphical models, and mixed data sources. One highlight talk was the one by Marc Deisenroth, who showed impressive robotic fast learning based on Gaussian processes. At the end of this full day, I also attended an Amazon mixer where I learned about Amazon‘s entry on the local market, where it seems the company is getting a better picture of the current and future state of the U.S. economy than governmental services, thanks to a very fine analysis of the sales and entries on Amazon‘s entry. Then it was time to bike “home” on my rental bike, in the setting sun…
The English title of this 2007 book of Murakami is “What I talk about when I talk about running”. Which is a parody of Raymond Carver’s collection of [superb] short stories, “What we talk about when we talk about love”. (Murakami translated the complete œuvres of Raymond Carver in Japanese.) It is a sort of diary about Murakami’s running practice and the reasons why he is running. It definitely is not a novel and the style is quite loose or lazy, but this is not a drawback as the way the book is written somehow translates the way thoughts drift away and suddenly switch topics when one is running. At least during low-intensity practice, when I often realise I have been running for minutes without paying any attention to my route. Or when I cannot recall what I was thinking about for the past minutes. During races, the mind concentration is at a different level, first focussing on keeping the right pace, refraining from the deadly rush during the first km, then trying to merge with the right batch of runners, then fighting wind, slope, and eventually fatigue. While the book includes more general autobiographical entries than those related with Murakami’s runner’s life, there are many points most long-distance runners would relate with. From the righteous feeling of sticking to a strict training and diet, to the almost present depression catching us in the final kms of a race, to the very flimsy balance between under-training and over-training, to the strangely accurate control over one’s pace at the end of a training season, and, for us old runners, to the irremediable decline in one’s performances as years pass by… On a more personal basis, I also shared the pain of hitting one of the slopes in Central Park and the lack of nice long route along Boston’s Charles river. And shared the special pleasure of running near a river or seafront (which is completely uncorrelated with the fact it is flat, I believe!) Overall, what I think this book demonstrates is that there is no rational reason to run, which makes the title more than a parody, as fighting weight, age, health problems, depression, &tc. and seeking solitude, quiet, exhaustion, challenge, performances, zen, &tc. are only partial explanations. Maybe the reason stated in the book that I can relate the most with is this feeling of having an orderly structure one entirely controls (provided the body does not rebel!) at least once a day. Thus, I am not certain the book appeals to non-runners. And contrary to some reviews of the book, it certainly is not a training manual for novice runners. (Murakami clearly is a strong runner so some of his training practice could be harmful to weaker runners…)
As mentioned earlier, this was my very first MCqMC conference and I really enjoyed it, even though (or because) there were many topics that did not fall within my areas of interest. (By comparison, WSC is a serie of conferences too remote from those areas for my taste, as I realised in Berlin where we hardly attended any talk and hardly anyone attended my session!) Here I appreciated the exposure to different mathematical visions on Monte Carlo, without being swamped by applications as at WSC… Obviously, our own Bayesian computational community was much less represented than at, say, MCMSki! Nonetheless, I learned a lot during this conference for instance from Peter Glynn‘s fantastic talk, and I came back home with new problems and useful references [as well as a two-hour delay in the train ride from Brussels]. I also obviously enjoyed the college-town atmosphere of Leuven, the many historical landmarks and the easily-found running routes out of the town. I am thus quite eager to attend the next MCqMC 2016 meeting (in Stanford, an added bonus!) and even vaguely toying with the idea of organising MCqMC 2018 in Monaco (depending on the return for ISBA 2016 and ISBA 2018). In any case, thanks to the scientific committee for the invitation to give a plenary lecture in Leuven and to the local committee for a perfect organisation of the meeting.