Archive for Scotland

asymptotically exact inference in likelihood-free models

Posted in Books, pictures, Statistics with tags , , , , , , , on November 29, 2016 by xi'an

“We use the intuition that inference corresponds to integrating a density across the manifold corresponding to the set of inputs consistent with the observed outputs.”

Following my earlier post on that paper by Matt Graham and Amos Storkey (University of Edinburgh), I now read through it. The beginning is somewhat unsettling, albeit mildly!, as it starts by mentioning notions like variational auto-encoders, generative adversial nets, and simulator models, by which they mean generative models represented by a (differentiable) function g that essentially turn basic variates with density p into the variates of interest (with intractable density). A setting similar to Meeds’ and Welling’s optimisation Monte Carlo. Another proximity pointed out in the paper is Meeds et al.’s Hamiltonian ABC.

“…the probability of generating simulated data exactly matching the observed data is zero.”

The section on the standard ABC algorithms mentions the fact that ABC MCMC can be (re-)interpreted as a pseudo-marginal MCMC, albeit one targeting the ABC posterior instead of the original posterior. The starting point of the paper is the above quote, which echoes a conversation I had with Gabriel Stolz a few weeks ago, when he presented me his free energy method and when I could not see how to connect it with ABC, because having an exact match seemed to cancel the appeal of ABC, all parameter simulations then producing an exact match under the right constraint. However, the paper maintains this can be done, by looking at the joint distribution of the parameters, latent variables, and observables. Under the implicit restriction imposed by keeping the observables constant. Which defines a manifold. The mathematical validation is achieved by designing the density over this manifold, which looks like

p(u)\left|\frac{\partial g^0}{\partial u}\frac{\partial g^0}{\partial u}^\text{T}\right|^{-\textonehalf}

if the constraint can be rewritten as g⁰(u)=0. (This actually follows from a 2013 paper by Diaconis, Holmes, and Shahshahani.) In the paper, the simulation is conducted by Hamiltonian Monte Carlo (HMC), the leapfrog steps consisting of an unconstrained move followed by a projection onto the manifold. This however sounds somewhat intense in that it involves a quasi-Newton resolution at each step. I also find it surprising that this projection step does not jeopardise the stationary distribution of the process, as the argument found therein about the approximation of the approximation is not particularly deep. But the main thing that remains unclear to me after reading the paper is how the constraint that the pseudo-data be equal to the observable data can be turned into a closed form condition like g⁰(u)=0. As mentioned above, the authors assume a generative model based on uniform (or other simple) random inputs but this representation seems impossible to achieve in reasonably complex settings.

travelling from pub to pub [in a straight line]

Posted in pictures, Wines with tags , , , , , , , , , , , on October 30, 2016 by xi'an

Above is the solution produced by a team at the University of Waterloo to the travelling salesman problem of linking all pubs in the UK (which includes pubs in Northern Ireland as well as some Scottish islands—even though I doubt there is no pub at all on the Island of Skye! They also missed a lot of pubs in Glasgow! And worst gaffe of all, they did not include the Clachaigh Inn, probably the best pub on Earth…). This path links over 24 thousand pubs, which is less than the largest travelling salesman problem solved at the current time, except that this case used the exact distances provided by Google maps. Of course, it would somehow make more sense to increase the distances by random amounts as the pub visits increase, unless the visitor sticks to tonic. Or tea.

eagle and child

MCqMC 2016 [#4]

Posted in Mountains, pictures, Running, Statistics, Travel, University life with tags , , , , , , , , , , , , , , on August 21, 2016 by xi'an

In his plenary talk this morning, Arnaud Doucet discussed the application of pseudo-marginal techniques to the latent variable models he has been investigating for many years. And its limiting behaviour towards efficiency, with the idea of introducing correlation in the estimation of the likelihood ratio. Reducing complexity from O(T²) to O(T√T). With the very surprising conclusion that the correlation must go to 1 at a precise rate to get this reduction, since perfect correlation would induce a bias. A massive piece of work, indeed!

The next session of the morning was another instance of conflicting talks and I hoped from one room to the next to listen to Hani Doss’s empirical Bayes estimation with intractable constants (where maybe SAME could be of interest), Youssef Marzouk’s transport maps for MCMC, which sounds like an attractive idea provided the construction of the map remains manageable, and Paul Russel’s adaptive importance sampling that somehow sounded connected with our population Monte Carlo approach. (With the additional step of considering transform maps.)

An interesting item of information I got from the final announcements at MCqMC 2016 just before heading to Monash, Melbourne, is that MCqMC 2018 will take place in the city of Rennes, Brittany, on July 2-6. Not only it is a nice location on its own, but it is most conveniently located in space and time to attend ISBA 2018 in Edinburgh the week after! Just moving from one Celtic city to another Celtic city. Along with other planned satellite workshops, this occurrence should make ISBA 2018 more attractive [if need be!] for participants from oversea.

even dogs in the wild

Posted in Books, Mountains, Travel with tags , , , , , , , on August 10, 2016 by xi'an

A new Rankin, a new Rebus! (New as in 2015 since I waited to buy the paperback version.) Sounds like Ian Rankin cannot let his favourite character rest for his retirement and hence set in back into action, along with the new Malcom Fox [working in the Complaints] and most major characters of the Rebus series. Including the unbreakable villain, Big Ger Cafferty. This as classical as you get, borrows from half a dozen former Rebus novels, not to mention this neo-Holmes novel I reviewed a while ago. But it is gritty, deadly efficient and captivating. I read the book within a few days from returning from Warwick.

About the title, this is a song by The Associates that plays a role in the book. I did not this band, but looking for it got me to a clip that used an excerpt from the Night of the Hunter. Fantastic movie, one of my favourites.

London calling….

Posted in pictures with tags , , , , , , on June 24, 2016 by xi'an

The Day After… Most sadly, England massively voted against remaining in the EU, while Scotland even more massively supported the Remain option.

the comforts of a muddy Saturday [book review]

Posted in Books, Travel, University life with tags , , , , on March 12, 2016 by xi'an

Besides the fantastic No. 1 Ladies Detective Agency series, which takes place in Botswana, Alexander McCall Smith has also written another series located in Edinburgh and featuring Isabel Dalhousie, a philosopher plus occasional detective. While the detective story is light to the point of being evanescent (and me losing interest by the middle of the book), the book The comforts of a muddy Saturday was still pleasant to re-read as Isabel is the editor of a philosophy academic journal, Review of Applied Ethics, and reflects on her duties as editor as well as brings philosophical musings into the novel.

“In fact, sometimes we publish papers that I suspect next to nobody reads.”

There is also a somewhat melancholic tone to the book in that it takes place at a time when submissions and replies were sent by regular mails, and faxes were for administrative aspects and only those. The description of Isabel’s duties is such that I am not convinced she needs 37 hours per week (!) to handle the submissions and editorial duties connected with the journal, although she ponders and hesitates so much before sending a particularly poor piece on the trolley dilemma that this may indeed end up in a full time job! Light reading for a rainy Saturday afternoon, then…

fit for Les Calanques

Posted in Mountains, Running with tags , , , , , , , , , on March 1, 2016 by xi'an