## scalable Metropolis-Hastings

Posted in Books, Statistics, Travel with tags , , , , , , , , , on February 12, 2019 by xi'an

Among the flury of arXived papers of last week (414!), including a fair chunk of papers submitted to ICML 2019, I spotted one entry by Cornish et al. on scalable Metropolis-Hastings, which Arnaud Doucet had mentioned to me yesterday when in Oxford. The paper builds on the delayed acceptance paper we wrote with Marco Banterlé, Clara Grazian and Anthony Lee, itself relying on a factorisation decomposition of the likelihood, combined with control variate accelerating techniques. The factorisation of both the target and the proposal allows for a (less efficient) Metropolis-Hastings acceptance ratio that is the product

$\prod_{i=1}^m \alpha_i(\theta,\theta')$

of individual Metropolis-Hastings acceptance ratios, but which allows for quicker rejection if one of the probabilities in the product is small, because the corresponding Bernoulli draw is zero with high probability. One advance made in Michel et al. (2017) [which I doubly missed] is that subsampling is achievable by thinning (as in PDMPs, where these authors have been quite active) through an algorithm of Shantikumar (1985) [described in Devroye’s bible]. Provided each Metropolis-Hastings probability can be lower bounded:

$\alpha_i(\theta,\theta') \ge \exp\{-\psi_i \phi(\theta,\theta')\}$

by a term where the transition φ does not depend on the index i in the product. The computing cost of the thinning process thus depends on the efficiency of the subsampling, namely whether or not the (Poisson) number of terms is much smaller than m, number of terms in the product. A neat trick in the current paper that extends the the Fukui-Todo procedure is to switch to the original Metropolis-Hastings when the overall lower bound is too small, recovering the geometric ergodicity of this original if it holds (Theorem 2.1). Another neat remark is that when using the naïve factorisation as the product of the n individual likelihoods, the resulting algorithm is sort of doomed as n grows, even with an optimal scaling of the proposals. To achieve scalability, the authors introduce a Taylor (i.e., Gaussian) approximation to each local target in the product and start the acceptance decomposition by using the resulting overall Gaussian approximation. Meaning that the remaining product is now made of ratios of targets over their local Taylor approximations, hence most likely close to one. And potentially lower-bounded by the remainder term in the Taylor expansion. Leading to the conclusion that, when everything goes well, meaning that the Taylor expansions can be conducted and the bounds derived for the appropriate expansion, the order of the Poisson scale is O(1/√n)..! The proposal for the Metropolis-Hastings move is actually tuned to the Gaussian approximation, appearing as a variant of the Langevin move or more exactly a discretization of an Hamiltonian move. Obviously, I cannot judge of the complexity in implementing this new scheme from just reading the paper, but this development on the split target is definitely an exciting prospect for handling huge datasets and their friends!

## Jeffreys priors for hypothesis testing [Bayesian reads #2]

Posted in Books, Statistics, University life with tags , , , , , , , , , , , , , , , , on February 9, 2019 by xi'an

A second (re)visit to a reference paper I gave to my OxWaSP students for the last round of this CDT joint program. Indeed, this may be my first complete read of Susie Bayarri and Gonzalo Garcia-Donato 2008 Series B paper, inspired by Jeffreys’, Zellner’s and Siow’s proposals in the Normal case. (Disclaimer: I was not the JRSS B editor for this paper.) Which I saw as a talk at the O’Bayes 2009 meeting in Phillie.

The paper aims at constructing formal rules for objective proper priors in testing embedded hypotheses, in the spirit of Jeffreys’ Theory of Probability “hidden gem” (Chapter 3). The proposal is based on symmetrised versions of the Kullback-Leibler divergence κ between null and alternative used in a transform like an inverse power of 1+κ. With a power large enough to make the prior proper. Eventually multiplied by a reference measure (i.e., the arbitrary choice of a dominating measure.) Can be generalised to any intrinsic loss (not to be confused with an intrinsic prior à la Berger and Pericchi!). Approximately Cauchy or Student’s t by a Taylor expansion. To be compared with Jeffreys’ original prior equal to the derivative of the atan transform of the root divergence (!). A delicate calibration by an effective sample size, lacking a general definition.

At the start the authors rightly insist on having the nuisance parameter v to differ for each model but… as we all often do they relapse back to having the “same ν” in both models for integrability reasons. Nuisance parameters make the definition of the divergence prior somewhat harder. Or somewhat arbitrary. Indeed, as in reference prior settings, the authors work first conditional on the nuisance then use a prior on ν that may be improper by the “same” argument. (Although conditioning is not the proper term if the marginal prior on ν is improper.)

The paper also contains an interesting case of the translated Exponential, where the prior is L¹ Student’s t with 2 degrees of freedom. And another one of mixture models albeit in the simple case of a location parameter on one component only.

## warm stone & cold morning light [jatp]

Posted in pictures, Travel, University life with tags , , , , , , on January 30, 2019 by xi'an

## BNP12

Posted in pictures, Statistics, Travel, University life with tags , , , , , , , , , , , , on October 9, 2018 by xi'an

The next BNP (Bayesian nonparametric) conference is taking place in Oxford (UK), prior to the O’Bayes 2019 conference in Warwick, in June 24-28 and June 29-July 2, respectively. At this stage, the Scientific Committee of BNP12 invites submissions for possible contributed talks. The deadline for submitting a title/abstract is 15th December 2018. And the submission of applications for travel support closes on 15th December 2018. Currently, there are 35 awards that could be either travel awards or accommodation awards. The support is for junior researchers (students currently enrolled in a Dphil (PhD) programme or having graduated after 1st October 2015). The applicant agrees to present her/his work at the conference as a poster or oraly if awarded the travel support.

As for O’Bayes 2019, we are currently composing the programme, following the 20 years tradition of these O’Bayes meetings of having the Scientific Committee (Marilena Barbieri, Ed George, Brunero Liseo, Luis Pericchi, Judith Rousseau and myself) inviting about 25 speakers to present their recent work and 25 discussants to… discuss these works. With a first day of introductory tutorials to Bayes, O’Bayes and beyond. I (successfully) proposed this date and location to the O’Bayes board to take advantage of the nonparametric Bayes community present in the vicinity so that they could attend both meetings at limited cost and carbon impact.

## postdoctoral position on the Malaria Atlas Project, Oxford [advert]

Posted in pictures, Statistics, Travel, University life with tags , , , , , on September 21, 2018 by xi'an

The Malaria Atlas Project is opening a postdoctoral position in Oxford in geospatial modelling toward collaborating with other scientists to develop probabilistic maps of malaria risk at national and sub-national level to evaluate the efficacy of past intervention strategies and to assist with the planning of future interventions. An understanding of spatiotemporal modelling and expertise in geostatistics, random-field models, or equivalent are essential. An understanding of the epidemiology of a vector-borne disease such as malaria is desirable but not essential. You must have a PhD or equivalent experience in mathematics, statistics, biostatistics, or a similar quantitative discipline.

You will contribute to and, as appropriate, lead in the preparation of scientific reports and journal articles for publication of research findings from this work in open access journals. Travel to collaborators in Europe, the United States, Africa, and Asia will be part of the role.

This full-time position is fixed-term until 31 December 2019 in the first instance. The closing date for this position will be 12.00 noon on Wednesday 17 October 2018.

## snapshot from Oxford [jatp]

Posted in pictures, Travel with tags , , , , , , , , on April 21, 2018 by xi'an

## five postdoc positions in top UK universities & Bayesian health data science

Posted in Statistics with tags , , , , , , , , , , , , , on March 30, 2018 by xi'an

The EPSRC programme New Approaches to Bayesian Data Science: Tackling Challenges from the Health Sciences, directed by Paul Fearnhead, is offering five 3 or 4 year PDRA positions at the Universities of Bristol, Cambridge, Lancaster, Oxford, and Warwick. Here is the complete call:

Salary:   £29,799 to £38,833
Closing Date:   Thursday 26 April 2018
Interview Date:   Friday 11 May 2018

We invite applications for Post-Doctoral Research Associates to join the New Approaches to Bayesian Data Science: Tackling Challenges from the Health Sciences programme. This is an exciting, cross-disciplinary research project that will develop new methods for Bayesian statistics that are fit-for-purpose to tackle contemporary Health Science challenges: such as real-time inference and prediction for large scale epidemics; or synthesizing information from distinct data sources for large scale studies such as the UK Biobank. Methodological challenges will be around making Bayesian methods scalable to big-data and robust to (unavoidable) model errors.

This £3M programme is funded by EPSRC, and brings together research groups from the Universities of Lancaster, Bristol, Cambridge, Oxford and Warwick. There is either a 4 or a 3 year position available at each of these five partner institutions.

You should have, or be close to completing, a PhD in Statistics or a related discipline. You will be experienced in one or more of the following areas: Bayesian statistics, computational statistics, statistical machine learning, statistical genetics, inference for epidemics. You will have demonstrated the ability to develop new statistical methodology. We are particularly keen to encourage applicants with strong computational skills, and are looking to put together a team of researchers with skills that cover theoretical, methodological and applied statistics. A demonstrable ability to produce academic writing of the highest publishable quality is essential.

Applicants must apply through Lancaster University’s website for the Lancaster, Oxford, Bristol and Warwick posts.  Please ensure you state clearly which position or positions you wish to be considered for when applying. For applications to the MRC Biostatistics Unit, University of Cambridge vacancy please go to their website.

Candidates who are considering making an application are strongly encouraged to contact Professor Paul Fearnhead (p.fearnhead@lancaster.ac.uk), Sylvia Richardson (sylvia.richardson@mrc-bsu.cam.ac.uk), Christophe Andrieu (c.andrieu@bristol.ac.uk), Chris Holmes (c.holmes@stats.ox.ac.uk) or Gareth Roberts (Gareth.O.Roberts@warwick.ac.uk) to discuss the programme in greater detail.

We welcome applications from people in all diversity groups.