Melbourne coastline [jatp]

Posted in pictures, Running, Travel with tags , , , , , on August 31, 2016 by xi'an

Florid’AISTATS

Posted in Statistics, University life, Travel, pictures, R with tags , , , , , , , , , on August 31, 2016 by xi'an

The next AISTATS conference is taking place in Florida, Fort Lauderdale, on April 20-22. (The website keeps the same address one conference after another, which means all my links to the AISTATS 2016 conference in Cadiz are no longer valid. And that the above sunset from Florida is named… cadiz.jpg!) The deadline for paper submission is October 13 and there are two novel features:

  1. Fast-track for Electronic Journal of Statistics: Authors of a small number of accepted papers will be invited to submit an extended version for fast-track publication in a special issue of the Electronic Journal of Statistics (EJS) after the AISTATS decisions are out. Details on how to prepare such extended journal paper submission will be announced after the AISTATS decisions.
  2. Review-sharing with NIPS: Papers previously submitted to NIPS 2016 are required to declare their previous NIPS paper ID, and optionally supply a one-page letter of revision (similar to a revision letter to journal editors; anonymized) in supplemental materials. AISTATS reviewers will have access to the previous anonymous NIPS reviews. Other than this, all submissions will be treated equally.

I find both initiatives worth applauding and replicating in other machine-learning conferences. Particularly in regard with the recent debate we had at Annals of Statistics.

parallel adaptive importance sampling

Posted in Statistics with tags , , , , , on August 30, 2016 by xi'an

Following Paul Russell’s talk at MCqMC 2016, I took a look at his recently arXived paper. In the plane to Sydney. The pseudo-code representation of the method is identical to our population Monte Carlo algorithm as is the suggestion to approximate the posterior by a mixture, but one novel aspect is to use Reich’s ensemble transportation at the resampling stage, in order to maximise the correlation between the original and the resampled versions of the particle systems. (As in our later versions of PMC, the authors also use as importance denominator the entire mixture rather than conditioning on the selected last-step particle.)

“The output of the resampling algorithm gives us a set of evenly weighted samples that we believe represents the target distribution well”

I disagree with this statement: Reweighting does not improve the quality of the posterior approximation, since it introduces more variability. If the original sample is found missing in its adequation to the target, so is the resampled one. Worse, by producing a sample with equal weights, this step may give a false impression of adequate representation…

Another unclear point in the pape relates to tuning the parameters of the mixture importance sampler. The paper discusses tuning these parameters during a burn-in stage, referring to “due to the constraints on adaptive MCMC algorithms”, which indeed is only pertinent for MCMC algorithms, since importance sampling can be constantly modified while remaining valid. This was a major point for advocating PMC. I am thus unsure what the authors mean by a burn-in period in such a context. Actually, I am also unsure on how they use effective sample size to select the new value of the importance parameter, e.g., the variance β in a random walk mixture: the effective sample size involves this variance implicitly through the realised sample hence changing β means changing the realised sample… This seems too costly to contemplate so I wonder at the way Figure 4.2 is produced.

“A popular approach for adaptive MCMC algorithms is to view the scaling parameter as a random variable which we can sample during the course of the MCMC iterations.”

While this is indeed an attractive notion [that I played with in the early days of adaptive MCMC, with the short-lived notion of cyber-parameters], I do not think it is of much help in optimising an MCMC algorithm, since the scaling parameter need be optimised, resulting into a time-inhomogeneous target. A more appropriate tool is thus stochastic optimisation à la Robbins-Monro, as exemplified in Andrieu and Moulines (2006). The paper however remains unclear as to how the scales are updated (see e.g. Section 4.2).

“Ideally, we would like to use a resampling algorithm which is not prohibitively costly for moderately or large sized ensembles, which preserves the mean of the samples, and which makes it much harder for the new samples to forget a significant region in the density.”

The paper also misses on the developments of the early 2000’s about more sophisticated resampling steps, especially Paul Fearnhead’s contributions (see also Nicolas Chopin’s thesis). There exist valid resampling methods that require a single uniform (0,1) to be drawn, rather than m. The proposed method has a flavour similar to systematic resampling, but I wonder at the validity of returning values that are averages of earlier simulations, since this modifies their distribution into ones with slimmer tails. (And it is parameterisation dependent.) Producing xi with probability pi is not the same as returning the average of the pixi‘s.

art brut

Posted in Books, Kids, Mountains, pictures, Travel with tags , , , , , on August 29, 2016 by xi'an

winning entry at MCqMC’16

Posted in Books, Kids, pictures, Statistics, Travel, University life with tags , , , , , , , on August 29, 2016 by xi'an

mcqmc4The nice logo of MCqMC 2016 was a collection of eight series of QMC dots on the unit (?) cube. The organisers set a competition to identify the principles behind those quasi-random sets and as I had no idea for most of them I entered very random sets unconnected with algorithmia, for which I got an honourable mention and a CD prize (if not the conference staff tee-shirt I was coveting!) Art Owen sent me back my entry, posted below and hopefully (or not!) readable.dots

Melbourne sunrise

Posted in pictures, Running, Travel with tags , , , , , , , , , , on August 28, 2016 by xi'an

Rifugio Vittorio Sella al Lauson

Posted in Kids, Mountains, pictures, Travel with tags , , , , , , , , on August 28, 2016 by xi'an

valnontey valleyTo sort of make up for the failed attempt at Monte Rosa, we stayed an extra day and took a hike in Vale d’Aosta, starting from Cogne where we had a summer school a few years ago. And from where we started for another failed attempt at La Grivola. It was a brilliant day and we climbed to the Rifugio Vittorio Stella (2588m) [along with many many other hikers], then lost the crowds to the Colle della Rossa (3195m), which meant a 1700m easy climb. By the end of the valley, we came across steinbocks (aka bouquetins, stambecchi) resting in the sun by a creek and unfazed by our cameras. (Abele Blanc told us later that they are usually staying there, licking whatever salt they can find on the stones.)

steinbocks near Colle della Rossa, Aosta, Jul 16, 2016 near Rifugio Vittorio Stella, Aosta, Jul 16, 2016

The final climb to the pass was a bit steeper but enormously rewarding, with views of the Western Swiss Alps in full glory (Matterhorn, Combin, Breithorn) and all to ourselves. From there it was a downhill hike all the way back to our car in Cogne, 1700m, with no technical difficulty once we had crossed the few hundred meters of residual snow. And with the added reward of seeing several herds of the shy chamois mountain goat.

end of the plateau, Rifugio Vittorio Stella, Aosta, July 16, 2016Except that my daughter’s rental mountaineering shoes started to make themselves heard and that she could barely walk downwards. (She eventually lost her big toe nails!) It thus took us forever to get down (despite me running to the car and back to get lighter shoes) and we came to the car at 8:30, too late to contemplate a drive back to Paris.

view from Colle Della Rossa, Aosta, July 16, 2016

Follow

Get every new post delivered to your Inbox.

Join 1,077 other followers