ABC in Warwick [Arena, Blended, Committed]

control variates [seminar]

Today, Petros Dellaportas (whom I have know since the early days of MCMC, when we met in CIRM) gave a seminar at the Warwick algorithm seminar on control variates for MCMC, reminding me of his 2012 JRSS paper. Based on the Poisson equation and using a second control variate to stabilise the Monte Carlo approximation do the first control variate. The difference with usual control variates is finding a first approximate G(x)-q(y|x)G(Y) to F-πF. And the first Poisson equation is using α(x,y)q(y|x) rather than π. Then the second expands log α(x,y)q(y|x) to achieve a manageable term.

Abstract: We provide a general methodology to construct control variates for any discrete time random walk Metropolis and Metropolis-adjusted Langevin algorithm Markov chains that can achieve, in a post-processing manner and with a negligible additional computational cost, impressive variance reduction when compared to the standard MCMC ergodic averages. Our proposed estimators are based on an approximate solution of the Poisson equation for a multivariate Gaussian target densities of any dimension.

I wonder if there were a neural network version that would first build G from scratch and later optimise it towards solving the Poisson equation. As in this recent arXival I haven’t read (yet).

general perspective on the Metropolis–Hastings kernel

[My Bristol friends and co-authors] Christophe Andrieu, and Anthony Lee, along with Sam Livingstone arXived a massive paper on 01 January on the Metropolis-Hastings kernel.

“Our aim is to develop a framework making establishing correctness of complex Markov chain Monte Carlo kernels a purely mechanical or algebraic exercise, while making communication of ideas simpler and unambiguous by allowing a stronger focus on essential features (…) This framework can also be used to validate kernels that do not satisfy detailed balance, i.e. which are not reversible, but a modified version thereof.”

A central notion in this highly general framework is, extending Tierney (1998), to see an MCMC kernel as a triplet involving a probability measure μ (on an extended space), an involution transform φ generalising the proposal step (i.e. þ²=id), and an associated acceptance probability ð. Then μ-reversibility occurs for

with the rhs involving the push-forward measure induced by μ and φ. And furthermore there is always a choice of an acceptance probability ð ensuring for this equality to happen. Interestingly, the new framework allows for mostly seamless handling of more complex versions of MCMC such as reversible jump and parallel tempering. But also non-reversible kernels, incl. for instance delayed rejection. And HMC, incl. NUTS. And pseudo-marginal, multiple-try, PDMPs, &c., &c. it is remarkable to see such a general theory emerging a this (late?) stage of the evolution of the field (and I will need more time and attention to understand its consequences).

simulating the pandemic

Nature of 13 November has a general public article on simulating the COVID pandemic as benefiting from the experience gained by climate-modelling methodology.

“…researchers didn’t appreciate how sensitive CovidSim was to small changes in its inputs, their results overestimated the extent to which a lockdown was likely to reduce deaths…”

The argument is essentially Bayesian, namely rather than using a best guess of the parameters of the model, esp. given the state of the available data (and the worse for March). When I read

“…epidemiologists should stress-test their simulations by running ‘ensemble’ models, in which thousands of versions of the model are run with a range of assumptions and inputs, to provide a spread of scenarios with different probabilities…”

it sounds completely Bayesian. Even though there is no discussion of the prior modelling or of the degree of wrongness of the epidemic model itself. The researchers at UCL who conducted the multiple simulations and the assessment of sensitivity to the 940 various parameters found that 19 of them had a strong impact, mostly

“…the length of the latent period during which an infected person has no symptoms and can’t pass the virus on; the effectiveness of social distancing; and how long after getting infected a person goes into isolation…”

but this outcome is predictable (and interesting). Mentions of Bayesian methods appear at the end of the paper:

“…the uncertainty in CovidSim inputs [uses] Bayesian statistical tools — already common in some epidemiological models of illnesses such as the livestock disease foot-and-mouth.”

and

“Bayesian tools are an improvement, says Tim Palmer, a climate physicist at the University of Oxford, who pioneered the use of ensemble modelling in weather forecasting.”

along with ensemble modelling, which sounds a synonym for Bayesian model averaging… (The April issue on the topic had also Bayesian aspects that were explicitely mentionned.)

Haldane’s short autobiography

“I was born at Oxford, England, in 1892.  My father was Prof. J.S. Haldane, the physiologist.  I was educated at Eton and New College, Oxford.  I learned much of my science by apprenticeship, assisting my father from the age of eight onwards, and my university degree is in for classics, not science.  I was in a British infantry battalion from 1914 to 1919, and was twice wounded.  I began scientific research in 1910, and became a Fellow of New College, Oxford, in 1919.  I was at Cambridge from 1922-1932 as Reader in Biochemistry, and have been a professor in London University since 1933.  I was visiting professor in the University of Berkeley, Cal., in 1932.  In the same year I was elected a Fellow of the Royal Society of London.

My scientific work has been varied.  In the field of human physiology I am best known for my work on the effects of taking large amounts of ammonium chloride and other salts.  This has had some application in treating lead and radium poisoning.  In the field of genetics I was the first to discover linkage in mammals, to map a human chromosome, and (with Penrose) to measure the mutation rate of a human gene.  I have also made some minor discoveries in mathematics.

Whilst I may have been a credit to my universities, I have been a trial in other ways.  I was dismissed from Cambridge University in 1926 in connexion with a divorce case, but regained my post on appeal to a higher tribunal, which found that the university authorities had decided to dismiss me without hearing my case.  At present I have refused to evacuate University College, London, and, with two assistants am its sole academic occupant.  I am carrying on research there under difficulties.

Besides strictly scientific books I have written a number of popular works including a book of children’s stories.  I consider that a scientist, if he can do so, should help to render science intelligible to ordinary people, and have done my best to popularize it.

Till 1933 I tried to keep out of politics, but the support given by the British Government to Hitler and Mussolini forced me to enter the political field.  In 1936-1938 I spent three months in Republican Spain, first as an adviser on gas protection, and then as an observer of air raid precautions.  I was in the front line during fighting, and in several air raids behind the line.  Since then I have tried, with complete lack of success, to induce the British Government to adopt air raid protection measures which had proved their efficacy in Spain.

Mr. Chamberlain’s policy, and the recent developments in physics and biology, combined to convince me of the truth of the Marxist philosophy.  Though I am a member of no political party, I have of late years supported the communist party on a number of issues.  At present I am engaged on research in genetics, & research intended to save the lives of members of the British armed forces, and writing and public speaking designed to prevent the spreading of the present war, and if possible to bring about peace.  I am a fairly competent public speaker.

It will be seen that my life has been a full one.  I have been married for 14 years, measure 73 inches, weigh 245 pounds, and enjoy swimming and mountain walking.  I am bald and blue-eyed, a moderate drinker and a heavy smoker. I can read 11 languages and make public speeches in three, but am unmusical.”

J.B.S. Haldane, circa 1940