connection between tempering & entropic mirror descent

The next One World ABC webinar is this  Thursday,  the 2nd May, at 9am UK time, with Francesca Crucinio (King’s College London, formerly CREST and even more formerly Warwick) presenting

“A connection between Tempering and Entropic Mirror Descent”.

a joint work with Nicolas Chopin and Anna Korba (both from CREST) whose abstract follows:

This work explores the connections between tempering (for Sequential Monte Carlo; SMC) and entropic mirror descent to sample from a target probability distribution whose unnormalized density is known. We establish that tempering SMC corresponds to entropic mirror descent applied to the reverse Kullback-Leibler (KL) divergence and obtain convergence rates for the tempering iterates. Our result motivates the tempering iterates from an optimization point of view, showing that tempering can be seen as a descent scheme of the KL divergence with respect to the Fisher-Rao geometry, in contrast to Langevin dynamics that perform descent of the KL with respect to the Wasserstein-2 geometry. We exploit the connection between tempering and mirror descent iterates to justify common practices in SMC and derive adaptive tempering rules that improve over other alternative benchmarks in the literature.

Warwickshire honey fungus [by Birmingham]

Warwick Stats recruits

The Department of Statistics at the University of Warwick is recruiting:

Assistant Professor, Statistics (3 positions in Applied, Methodological or Theoretical Statistics )

Assistant Professor, Computational Statistics or Machine Learning (2 positions)

Associate Professor (1 position, any area within the Department)

Applicants should have evidence or promise of world-class research excellence and ability to deliver high quality teaching across our broad range of degree programmes. At Associate Professor level, applicants should have an outstanding publication record. Other positive indicators include enthusiasm for engagement with other disciplines, within and outside the Department and, at Associate Professor level, a proven ability to secure research funding. Further details of the requirements for each of the positions can be found at https://warwick.ac.uk/statjobs.

The Department of Statistics is committed to promoting equality and diversity, holding an Athena SWAN Silver award which demonstrates this commitment. We welcome applicants from all sections of the community and will give due consideration to applicants seeking flexible working patterns, and to those who have taken a career break. Further information about working at the University of Warwick, including information about childcare provision, career development and relocation is at https://warwick.ac.uk/services/humanresources/workinghere/.

Informal enquires can be addressed to Professor Jon Forster (J.J.Forster@warwick.ac.uk) or to any other senior member of the Warwick Statistics Department.

Further information about the Department of Statistics: https://warwick.ac.uk/stats
Further information about the University of Warwick: https://www2.warwick.ac.uk/services/humanresources/jobsintro/furtherparticulars

 

foxhuntshire

[more than] everything you always wanted to know about marginal likelihood

Earlier this year, F. Llorente, L. Martino, D. Delgado, and J. Lopez-Santiago have arXived an updated version of their massive survey on marginal likelihood computation. Which I can only warmly recommend to anyone interested in the matter! Or looking for a base camp to initiate a graduate project. They break the methods into four families

1. Deterministic approximations (e.g., Laplace approximations)
2. Methods based on density estimation (e.g., Chib’s method, aka the candidate’s formula)
3. Importance sampling, including sequential Monte Carlo, with a subsection connecting with MCMC
4. Vertical representations (mostly, nested sampling)

Besides sheer computation, the survey also broaches upon issues like improper priors and alternatives to Bayes factors. The parts I would have done in more details are reversible jump MCMC and the long-lasting impact of Geyer’s reverse logistic regression (with the noise contrasting extension), even though the link with bridge sampling is briefly mentioned there. There is even a table reporting on the coverage of earlier surveys. Of course, the following postnote of the manuscript

The Christian Robert’s blog deserves a special mention , since Professor C. Robert has devoted several entries of his blog with very interesting comments regarding the marginal likelihood estimation and related topics.

does not in the least make me less objective! Some of the final recommendations

• use of Naive Monte Carlo [simulate from the prior] should be always considered [assuming a proper prior!]
  
• a multiple-try method is a good choice within the MCMC schemes
• optimal umbrella sampling estimator is difficult and costly to implement , so its best performance may not be achieved in practice
• adaptive importance sampling uses the posterior samples to build a suitable normalized proposal, so it benefits from localizing samples in regions of high posterior probability while preserving the properties of standard importance sampling
  
• Chib’s method is a good alternative, that provide very good performances [but is not always available]
• the success [of nested sampling] in the literature is surprising.