Archive for Chicago Booth School of Business

Metropolis-Hastings via classification

Posted in pictures, Statistics, Travel, University life with tags , , , , , , , , , , , , , on February 23, 2021 by xi'an

Veronicka Rockova (from Chicago Booth) gave a talk on this theme at the Oxford Stats seminar this afternoon. Starting with a survey of ABC, synthetic likelihoods, and pseudo-marginals, to motivate her approach via GANs, learning an approximation of the likelihood from the GAN discriminator. Her explanation for the GAN type estimate was crystal clear and made me wonder at the connection with Geyer’s 1994 logistic estimator of the likelihood (a form of discriminator with a fixed generator). She also expressed the ABC approximation hence created as the actual posterior times an exponential tilt. Which she proved is of order 1/n. And that a random variant of the algorithm (where the shift is averaged) is unbiased. Most interestingly requiring no calibration and no tolerance. Except indirectly when building the discriminator. And no summary statistic. Noteworthy tension between correct shape and correct location.

vertical likelihood Monte Carlo integration

Posted in Books, pictures, Running, Statistics, Travel, University life with tags , , , , , , , on April 17, 2015 by xi'an

A few months ago, Nick Polson and James Scott arXived a paper on one of my favourite problems, namely the approximation of normalising constants (and it went way under my radar, as I only became aware of it quite recently!, then it remained in my travel bag for an extra few weeks…). The method for approximating the constant Z draws from an analogy with the energy level sampling methods found in physics, like the Wang-Landau algorithm. The authors rely on a one-dimensional slice sampling representation of the posterior distribution and [main innovation in the paper] add a weight function on the auxiliary uniform. The choice of the weight function links the approach with the dreaded harmonic estimator (!), but also with power-posterior and bridge sampling. The paper recommends a specific weighting function, based on a “score-function heuristic” I do not get. Further, the optimal weight depends on intractable cumulative functions as in nested sampling. It would be fantastic if one could draw directly from the prior distribution of the likelihood function—rather than draw an x [from the prior or from something better, as suggested in our 2009 Biometrika paper] and transform it into L(x)—but as in all existing alternatives this alas is not the case. (Which is why I find the recommendations in the paper for practical implementation rather impractical, since, were the prior cdf of L(X) available, direct simulation of L(X) would be feasible. Maybe not the optimal choice though.)

“What is the distribution of the likelihood ordinates calculated via nested sampling? The answer is surprising: it is essentially the same as the distribution of likelihood ordinates by recommended weight function from Section 4.”

The approach is thus very much related to nested sampling, at least in spirit. As the authors later demonstrate, nested sampling is another case of weighting, Both versions require simulations under truncated likelihood values. Albeit with a possibility of going down [in likelihood values] with the current version. Actually, more weighting could prove [more] efficient as both the original nested and vertical sampling simulate from the prior under the likelihood constraint. Getting away from the prior should help. (I am quite curious to see how the method is received and applied.)


Posted in Wines with tags , , , on December 20, 2012 by xi'an

a day in Chicago

Posted in pictures, Statistics, Travel, University life with tags , , , , , , , , , on November 1, 2012 by xi'an

Another busy day as I visited the University of Chicago Booth School of Business. This was my first time visit to this impressive building (and my first visit to Chicago for 25 years…) I actually had to leave Ames at 4:30 (am!) to catch a plane in Des Moines at 6:50 and be at the Chicago Booth before my first appointment at 10… Everything worked out fine, despite the potential for disruption due to the storm Sandy (just spotted a few big waves along the waterfront on my way to the University), and I had a definitely productive sequence of discussions. The talk on ABC was again well-attended and, because this was an econometric seminar (as in Princeton), definitely lively with a flow of questions all along. (There were also a few people from Biology, for whom the focus on our consistency result was presumably less interesting than for econometricians.) As in Ames, I did not manage to reach the part on empirical likelihood. Fodder for another seminar! The day ended by a meal in a superb restaurant with my favourite wine, Saint-Joseph, after which I was ready for a few hours of sleep..! And then a few hours to spend in the Art Institute of Chicago before flying back to Paris. Direct, courtesy of Sandy.

Midwestern trip

Posted in pictures, Statistics, Travel, University life with tags , , , , , , , , , , , on October 24, 2012 by xi'an

Next week, I will visit both Iowa State University, in Ames—a funny item for French speaking readers is that I will first land in Des Moines before reaching (les) Ames!, a logical step if any, even though only the first name relates to the early French exploration of the area: Ames has apparently no [ethymological] connection with souls…—, and the University of Chicago Booth Business School, giving a seminar on ABC model choice and empirical likelihood in both places. (I have never been to Iowa before and the last time I visited Chicago—rather than just commuting through O’Hare—was in May 1988, when I drove a friend to the airport…!) Here are the time and places for the seminars (note that the seminar at Booth is on Tuesday rather than on the customary Thursday to accommodate my tight schedule!):

As a coincidence—not so much as he is currently assistant professor in Ames—, the previous seminar speaker in Ames is my friend Vivek Roy, talking on Monte Carlo Methods for Improper Target Distributions! Here is (again!) the current version of the slides: