Archive for Annals of Statistics

statistical analysis of GANs

Posted in Books, Statistics with tags , , , , , , , , on May 24, 2021 by xi'an

My friend Gérard Biau and his coauthors have published a paper in the Annals of Statistics last year on the theoretical [statistical] analysis of GANs, which I had missed and recently read with a definitive interest in the issues. (With no image example!)

If the discriminator is unrestricted the unique optimal solution is the Bayes posterior probability

\dfrac{p^\star(x)}{p^\star(x)+p_\theta(x)}

when the model density is everywhere positive. And the optimal parameter θ corresponds to the closest model in terms of Kullback-Leibler divergence. The pseudo-true value of the parameter. This is however the ideal situation, while in practice D is restricted to a parametric family. In this case, if the family is wide enough to approximate the ideal discriminator in the sup norm, with error of order ε, and if the parameter space Θ is compact, the optimal parameter found under the restricted family approximates the pseudo-true value in the sense of the GAN loss, at the order ε². With a stronger assumption on the family ability to approximate any discriminator, the same property holds for the empirical version (and in expectation). (As an aside, the figure illustrating this property confusedly uses an histogramesque rectangle to indicate the expectation of the discriminator loss!) And both parameter (θ and α) estimators converge to the optimal ones with the sample size. An interesting foray from statisticians in a method whose statistical properties are rarely if ever investigated. Missing a comparison with alternative approaches, like MLE, though.

linearity, reversed

Posted in Books, Kids with tags , , , , , on September 19, 2020 by xi'an

While answering a question on X validated on the posterior mean being a weighted sum of the prior mean and of the maximum likelihood estimator, when the weights do not depend on the data, which is true in conjugate natural exponential family settings, I re-read this wonderful 1979 paper of Diaconis & Ylvisaker establishing the converse, namely that when the linear combination holds, the prior need be conjugate! This holds within exponential families, but I cannot think of a reasonable case outside exponential families where the linearity holds (again with constant weights, as otherwise it always holds in dimension one, albeit with weights possibly outside [0,1]).

Xmas tree at UCL, with a special gift

Posted in Books, pictures, Statistics, Travel, University life with tags , , , , , , , , , , , , , on November 26, 2019 by xi'an

Ph.D. students at UCL Statistics have made this Xmas tree out of bound and unbound volumes of statistics journals, not too hard to spot (especially the Current Indexes which I abandoned when I left my INSEE office a few years ago). An invisible present under the tree is the opening of several positions, namely two permanent lectureships and two three-year research fellowships, all in Statistics or Applied Probability, with the fellowship deadline being the 1st of December 2019!

running after my plane

Posted in Books, Kids, pictures, Running, Travel with tags , , , , , , , , , , , , on June 29, 2019 by xi'an

A bit of a hectic trip to Abidjan last Sunday, starting from Caen in the early morning where I was supporting my daughter, wife, mother, and mother-in-law for the annual Rochambelle women-only 5k race on the previous evening! With my daughter managing a fantastic 52nd position and ending up first of her category! As I was driven to the local train station to get back to my 63kg of Annals and my plane, the on-going 10k race kept preventing us from reaching it and I eventually decided 8 minutes before the deadline to leave the car and race to the station, actually running along the 10k racers for one kilometer and managing to enter the train just before it was leaving (too bad I could not finish the race, this start would have made for a great time despite my current injury!). And then when I eventually reached the CDG airport with my 63kg, I was told my bags were not good enough to carry them and that I had to drop some years of these Annals in the bin! So very frustrating… At least the remaining books reached their intended destination.The plane ride itself was uneventful and above a constant cover of clouds. While catching up with some unread arXivals, I also watched a great 2018 Japanese movie, Shoplifters, which sounded more like a modern tale than a realistic story, so outlying its characters were. With unexpected revelations towards the end and overall a moving and very subtle reflection on what ultimately makes a family. (Reading reviews on the movie later made me realise one scene had been censored for plane audiences…)

off to Abidjan, with a few Annals [62kg of ’em]

Posted in Books, Kids, Statistics, Travel, University life with tags , , , , , , , , on June 16, 2019 by xi'an