Archive for Ithaca

your GAN is secretly an energy-based model

Posted in Books, Statistics, University life with tags , , , , , , , , , , , , , on January 5, 2021 by xi'an

As I was reading this NeurIPS 2020 paper by Che et al., and trying to make sense of it, I came across a citation to our paper Casella, Robert and Wells (2004) on a generalized accept-reject sampling scheme where the proposal changes at each simulation that sounds surprising if appreciated! But after checking this paper also appears as the first reference on the Wikipedia page for rejection sampling, which makes me wonder if many actually read it. (On the side, we mostly wrote this paper on a drive from Baltimore to Ithaca, after JSM 1999.)

“We provide more evidence that it is beneficial to sample from the energy-based model defined both by the generator and the discriminator instead of from the generator only.”

The paper seems to propose a post-processing of the generator output by a GAN, generating from the mixture of both generator and discriminator, via a (unscented) Langevin algorithm. The core idea is that, if p(.) is the true data generating process, g(.) the estimated generator and d(.) the discriminator, then

p(x) ≈ p⁰(x)∝g(x) exp(d(x))

(The approximation would be exact the discriminator optimal.) The authors work with the latent z’s, in the GAN meaning that generating pseudo-data x from g means taking a deterministic transform of z, x=G(z). When considering the above p⁰, a generation from p⁰ can be seen as accept-reject with acceptance probability proportional to exp[d{G(z)}]. (On the side, Lemma 1 is the standard validation for accept-reject sampling schemes.)

Reading this paper made me realised how much the field had evolved since my previous GAN related read. With directions like Metropolis-Hastings GANs and Wasserstein GANs. (And I noticed a “broader impact” section past the conclusion section about possible misuses with societal consequences, which is a new requirement for NeurIPS publications.)

Grand Central Terminal

Posted in Books, pictures, Travel with tags , , , , , , , , , , , on April 22, 2020 by xi'an

not for the faint-hearted!

Posted in Books, Kids, pictures, Travel with tags , , , , , on March 21, 2016 by xi'an

While flying over to Boston yesterday, I had a look at The Martian on my seat screen but this proved too much of a hardship: after watching the early self-surgery scene, which is definitely realistic and somewhat gory, I just fainted. Really and truly fainted, which means I came back to my senses being dragged on the plane floor by two Air France flight attendants!, hearing and seeing them but being unable to react for a dozen seconds. There was a doctor in the plane who checked upon me while I was coming back to my senses and his final advice was to stop watching this “kind of movies”, as if I knew I was going to faint from watching a  PG-13 movie… (It actually happened to me once earlier, in that I came close to fainting from watching The Last Temptation of Christ in Ithaca in the 80’s, while protesters were demonstrating outside the cinema.) Quite an embarrassment, frankly! And I did not even watch the rest of the movie…

Cayuga 1989

Posted in Kids, Travel, Wines with tags , , , , , on August 20, 2015 by xi'an

cayuga

George’s dream

Posted in Kids, Travel with tags , , , , , on April 11, 2015 by xi'an

doublewashWhile I have shared this idea with many of my friends [in both senses that I mentioned it and that they shared the same feeling that it would be a great improvement], the first time I heard of the notion was in George Casella‘s kitchen in Ithaca, New York, in the early 1990’s… We were emptying the dishwasher together and George was reflecting that it would be so convenient to have a double dishwasher and remove the need to empty it altogether! Although, at the moral level, I think that we should do without dishwashers, I found this was a terrific idea and must have told the joke to most of my friends. I was nonetheless quite surprised and very pleased to receive the news from Nicole today that Fisher & Paykel (from Auckland, New Zealand) had gone all the way to produce a double dishwasher, or more exactly a double dishdrawer, perfectly suited to George’s wishes! (Pleased that she remembered the notion after all those years, not pleased with the prospect of buying a double dish washer for more than double the cost of [and a smaller volume than] a regular dishwasher!)