Archive for University of Oxford

scalable Metropolis-Hastings, nested Monte Carlo, and normalising flows

Posted in Books, pictures, Statistics, University life with tags , , , , , , , , , , , , , , , , , , , , , , , , , on June 16, 2020 by xi'an

Over a sunny if quarantined Sunday, I started reading the PhD dissertation of Rob Cornish, Oxford University, as I am the external member of his viva committee. Ending up in a highly pleasant afternoon discussing this thesis over a (remote) viva yesterday. (If bemoaning a lost opportunity to visit Oxford!) The introduction to the viva was most helpful and set the results within the different time and geographical zones of the Ph.D since Rob had to switch from one group of advisors in Engineering to another group in Statistics. Plus an encompassing prospective discussion, expressing pessimism at exact MCMC for complex models and looking forward further advances in probabilistic programming.

Made of three papers, the thesis includes this ICML 2019 [remember the era when there were conferences?!] paper on scalable Metropolis-Hastings, by Rob Cornish, Paul Vanetti, Alexandre Bouchard-Côté, Georges Deligiannidis, and Arnaud Doucet, which I commented last year. Which achieves a remarkable and paradoxical O(1/√n) cost per iteration, provided (global) lower bounds are found on the (local) Metropolis-Hastings acceptance probabilities since they allow for Poisson thinning à la Devroye (1986) and  second order Taylor expansions constructed for all components of the target, with the third order derivatives providing bounds. However, the variability of the acceptance probability gets higher, which induces a longer but still manageable if the concentration of the posterior is in tune with the Bernstein von Mises asymptotics. I had not paid enough attention in my first read at the strong theoretical justification for the method, relying on the convergence of MAP estimates in well- and (some) mis-specified settings. Now, I would have liked to see the paper dealing with a more complex problem that logistic regression.

The second paper in the thesis is an ICML 2018 proceeding by Tom Rainforth, Robert Cornish, Hongseok Yang, Andrew Warrington, and Frank Wood, which considers Monte Carlo problems involving several nested expectations in a non-linear manner, meaning that (a) several levels of Monte Carlo approximations are required, with associated asymptotics, and (b) the resulting overall estimator is biased. This includes common doubly intractable posteriors, obviously, as well as (Bayesian) design and control problems. [And it has nothing to do with nested sampling.] The resolution chosen by the authors is strictly plug-in, in that they replace each level in the nesting with a Monte Carlo substitute and do not attempt to reduce the bias. Which means a wide range of solutions (other than the plug-in one) could have been investigated, including bootstrap maybe. For instance, Bayesian design is presented as an application of the approach, but since it relies on the log-evidence, there exist several versions for estimating (unbiasedly) this log-evidence. Similarly, the Forsythe-von Neumann technique applies to arbitrary transforms of a primary integral. The central discussion dwells on the optimal choice of the volume of simulations at each level, optimal in terms of asymptotic MSE. Or rather asymptotic bound on the MSE. The interesting result being that the outer expectation requires the square of the number of simulations for the other expectations. Which all need converge to infinity. A trick in finding an estimator for a polynomial transform reminded me of the SAME algorithm in that it duplicated the simulations as many times as the highest power of the polynomial. (The ‘Og briefly reported on this paper… four years ago.)

The third and last part of the thesis is a proposal [to appear in ICML 20] on relaxing bijectivity constraints in normalising flows with continuously index flows. (Or CIF. As Rob made a joke about this cleaning brand, let me add (?) to that joke by mentioning that looking at CIF and bijections is less dangerous in a Trump cum COVID era at CIF and injections!) With Anthony Caterini, George Deligiannidis and Arnaud Doucet as co-authors. I am much less familiar with this area and hence a wee bit puzzled at the purpose of removing what I understand to be an appealing side of normalising flows, namely to produce a manageable representation of density functions as a combination of bijective and differentiable functions of a baseline random vector, like a standard Normal vector. The argument made in the paper is that imposing this representation of the density imposes a constraint on the topology of its support since said support is homeomorphic to the support of the baseline random vector. While the supporting theoretical argument is a mathematical theorem that shows the Lipschitz bound on the transform should be infinity in the case the supports are topologically different, these arguments may be overly theoretical when faced with the practical implications of the replacement strategy. I somewhat miss its overall strength given that the whole point seems to be in approximating a density function, based on a finite sample.

Judith’s colloquium at Warwick

Posted in Statistics with tags , , , , , , , , on February 21, 2020 by xi'an

MCqMC2020 key dates

Posted in pictures, Statistics, Travel, University life with tags , , , , , , , , , on January 23, 2020 by xi'an

A reminder of the key dates for the incoming MCqMC2020 conference this summer in Oxford:

Feb 28, Special sessions/minisymposia submission
Mar 13, Contributed abstracts submission
Mar 27, Acceptance notification
Mar 27, Registration starts
May 8, End of early bird registration
June 12, Speaker registration deadline
Aug 9-14 Conference

and of the list of plenary speakers

Yves Atchadé (Boston University)
Jing Dong (Columbia University)
Pierre L’Ecuyer (Université de Montreal)
Mark Jerrum (Queen Mary University London)
Gerhard Larcher (JKU Linz)
Thomas Muller (NVIDIA)
David Pfau (Google DeepMind)
Claudia Schillings (University of Mannheim)
Mario Ullrich (JKU Linz)

rationality and superstition

Posted in Books with tags , , , , , , , , on December 4, 2019 by xi'an

As I am about to read The Secret Commonwealth, the second volume in his Book of Dust trilogy, I found that Philip Pullman wrote a fairly interesting piece inspired from a visit to an 2018 exhibition at the Ashmolean Museum in Oxford, dedicated to magic and witchcraft. Which I enjoyed reading even though I do not agree with most points. Even though the human tendency to see causes in everything, hidden or even supernatural if need be, explains for superstition and beliefs in magics, the Enlightenment and rise of rationality saw the end of the witch-hunt craze of the 16th and early 17th Centuries (with close to 50,000 executions throughout Europe.

“…rationalism doesn’t make the magical universe go away (…) When it comes to belief in lucky charms, or rings engraved with the names of angels, or talismans with magic squares, it’s impossible to defend it and absurd to attack it on rational grounds because it’s not the kind of material on which reason operates. Reason is the wrong tool. Trying to understand superstition rationally is like trying to pick up something made of wood by using a magnet.”

“Whether witches were “filthy quislings” or harmless village healers, they and those who believed in witchcraft and magic existed in a shared mental framework of hidden influences and meanings, of significances and correspondences, whether angelic, diabolic, or natural (…)  a penumbra of associations, memories, echoes and correspondences that extend far into the unknown. In this way of seeing things, the world is full of tenuous filaments of meaning, and the very worst way of trying to see these shadowy existences is to shine a light on them.”

“I simply can’t agree with (Richard Dawkins’): “We don’t have to invent wildly implausible stories: we have the joy and excitement of real, scientific investigation and discovery to keep our imaginations in line.” (The Magic of Reality, 2011). If we have to keep our imaginations in line, it’s because we don’t trust them not to misbehave. What’s more, only scientific investigation can disclose what’s real. On the contrary, I’d rather say that there are times when we have to keep our reason in line. I daresay that the state of Negative Capability, where imagination rules, is in fact where a good deal of scientific discovery begins. “

Florence Nightingale Bicentennial Fellowship and Tutor in Statistics and Probability in Oxford [call]

Posted in Statistics, Travel, University life with tags , , , , , on July 29, 2019 by xi'an

Reposted: The Department of Statistics is recruiting a Florence Nightingale Bicentennial Fellowship and Tutor in Statistics and Probability with effect from October 2019 or as soon as possible thereafter. The post holder will join the dynamic and collaborative Department of Statistics. The Department carries out world-leading research in applied statistics fields including statistical and population genetics and bioinformatics, as well as core theoretical statistics, computational statistics, machine learning and probability. This is an exciting time for the Department, which relocated to new premises on St Giles’ in the heart of the University of Oxford in 2015. Our newly-renovated building provides state-of-the-art teaching facilities and modern space to facilitate collaboration and integration, creating a highly visible centre for Statistics in Oxford. The successful candidate will hold a doctorate in the field of Statistics, Mathematics or a related subject. They will be an outstanding individual who has the potential to become a leader in their field. The post holder will have the skills and enthusiasm to teach at undergraduate and graduate level, within the Department of Statistics, and to supervise student projects. They will carry out and publish original research within their area of specialisation. We particularly encourage candidates working in areas that link with existing research groups in the department to apply. The deadline for application is September 30, 2019.

If you would like to discuss this post and find out more about joining the academic community in Oxford, please contact Professor Judith Rousseau or Professor Yee Whye Teh. All enquiries will be treated in strict confidence and will not form part of the selection decision.