double descent

Last Friday, I [and a few hundred others!] went to the SMILE (Statistical Machine Learning in Paris) seminar where Francis Bach was giving a talk. (With a pleasant ride from Dauphine along the Seine river.) Fancis was talking about the double descent phenomenon observed in recent papers by Belkin & al. (2018, 2019), and Mei & Montanari (2019). (As the seminar room at INRIA was quite crowded and as I was sitting X-legged on the floor close to the screen, I took a few slides from below!) The phenomenon is that the usual U curve warning about over-fitting and reproduced in most statistics and machine-learning courses can under the right circumstances be followed by a second decrease in the testing error when the number of features goes beyond the number of observations. This is rather puzzling and counter-intuitive, so I briefkly checked the 2019 [8 pages] article by Belkin & al., who are studying two examples, including a standard “large p small n” Gaussian regression. where the authors state that

“However, as p grows beyond n, the test risk again decreases, provided that the model is fit using a suitable inductive bias (e.g., least norm solution). “

One explanation [I found after checking the paper] is that the variates (features) in the regression are selected at random rather than in an optimal sequential order. Double descent is missing with interpolating and deterministic estimators. Hence requiring on principle all candidate variates to be included to achieve minimal averaged error. The infinite spike is when the number p of variate is near the number n of observations. (The expectation accounts as well for the randomisation in T. Randomisation that remains an unclear feature in this framework…)

probabilistic methods in computational statistics [workshop]

A  one-day workshop is organised at Telecom Sudparis, Évry, on 22 November by R. Douc, F. Portier and F. Roueff. On the “hot topics” concerned with probabilistic methods in computational statistics. The workshop is funded by the project “Big-Pomm”, which strengthens the links between LTCI (Telecom Paristech) and SAMOVAR (Telecom Sudparis) around research projects implying partially observed Markov models. The participation to the workshop is free but registration is required for having access to the lunch buffet (40 participants max). (Évry is located 20km south of Paris, with trains on the RER C line.)

MHC2020

There is a conference on mixtures (M) and hidden Markov models (H) and clustering (C) taking place in Orsay on June 17-19, next year. Registration is free if compulsory. With about twenty confirmed speakers. (Irrelevant as the following remark is, this is the opportunity to recall the conference on mixtures I organised in Aussois 25 years before! Which website is amazingly still alive at Duke, thanks to Mike West, my co-organiser along with Kathryn Roeder and Gilles Celeux. When checking the abstracts, I found only two presenters common to both conferences, Christophe Biernaki and Jiahua Chen. And alas several names of departed friends.)

100ième victime

free Tuna Altinel

Tuna Altınel, an associate professor of mathematics at University Claude Bernard Lyon 1 since 1996 and a member of Institut Camille Jordan, was incarcerated by the Turkish authorities on Saturday 11 May, 2019. Internationally recognized as a mathematician, he is also a professor who stands as a model in clarity and dedication. He is also involved in the defence of human rights through the “Academics For Peace” movement.

Tuna Altınel is being prosecuted by the Turkish justice system for signing a petition entitled “We, Turkish academics, will not be a party to this crime”, in January 2016. This petition denounced the intervention of Turkish military forces in the south-east provinces of Turkey since the summer of 2015 (UN HCHR report) and called for the resumption of talks in order to restore peace.

As he arrived in Turkey on 12 April during French academic recess, his passport was confiscated. He was then arrested and incarcerated on Saturday 11 May. In February 2019, he participated in a public meeting—organized by an association legally recognized in France—, aiming to raise awareness about the consequences of military intervention on the civilian populations. His participation in that meeting appears to be one of the charges held against him.

Sign the petition

non-uniform Laplace generation

This year, the French Statistical Society (SFDS) Prix Laplace has been granted to Luc Devroye, author of the Non-Uniform Random Generation bible. among many achievements!, prize that he will receive during the 2019 meeting in Nancy, this very week.

ABC with Gibbs steps

With Grégoire Clarté, Robin Ryder and Julien Stoehr, all from Paris-Dauphine, we have just arXived a paper on the specifics of ABC-Gibbs, which is a version of ABC where the generic ABC accept-reject step is replaced by a sequence of n conditional ABC accept-reject steps, each aiming at an ABC version of a conditional distribution extracted from the joint and intractable target. Hence an ABC version of the standard Gibbs sampler. What makes it so special is that each conditional can (and should) be conditioning on a different statistic in order to decrease the dimension of this statistic, ideally down to the dimension of the corresponding component of the parameter. This successfully bypasses the curse of dimensionality but immediately meets with two difficulties. The first one is that the resulting sequence of conditionals is not coherent, since it is not a Gibbs sampler on the ABC target. The conditionals are thus incompatible and therefore convergence of the associated Markov chain becomes an issue. We produce sufficient conditions for the Gibbs sampler to converge to a stationary distribution using incompatible conditionals. The second problem is then that, provided it exists, the limiting and also intractable distribution does not enjoy a Bayesian interpretation, hence may fail to be justified from an inferential viewpoint. We however succeed in producing a version of ABC-Gibbs in a hierarchical model where the limiting distribution can be explicited and even better can be weighted towards recovering the original target. (At least with limiting zero tolerance.)