sample-efficient inference for simulators: complex noise models and time-series [One World ABC seminar]

The next One World ABC seminar will take place next Thursday, 23 Feb, at 9:30 UK time, with a talk by Alexander Aushev, on the above, based on a paper with Tran, Pesonen, Howes, and Kaski:

Simulators are becoming more complex, with their parameter inference requiring as few simulations as possible. This talk will go over two likelihood-free inference (LFI) challenges for computationally intensive simulators. The first challenge is modeling complex simulator noise, which is frequently oversimplified by existing methods or needs far too many simulations. I will discuss how LFI can handle multimodal, non-stationary, and heteroscedastic noise distributions in Bayesian Optimization by using deep Gaussian processes as surrogate models. The second challenge involves simulators in time-series settings, in which the observed time-series data is generated by an unknown stochastic process of simulator parameters. Modern LFI methods, in such cases, either require an accurate model of parameter transition dynamics (e.g. available for sampling) or assume it to be linear. In the last part of the talk, I will discuss the challenges and solutions for performing LFI in such time-series settings, which involve learning the unknown transition dynamics of simulator parameters.

ABC with path signatures [One World ABC seminar, 2/2/23]

The next One World ABC seminar is by Joel Dyer (Oxford) at 1:30pm (UK time) on 02 February.

Title: Approximate Bayesian Computation with Path Signatures

Abstract: Simulation models often lack tractable likelihood functions, making likelihood-free inference methods indispensable. Approximate Bayesian computation (ABC) generates likelihood-free posterior samples by comparing simulated and observed data through some distance measure, but existing approaches are often poorly suited to time series simulators, for example due to an independent and identically distributed data assumption. In this talk, we will discuss our work on the use of path signatures in ABC as a means to handling the sequential nature of time series data of different kinds. We will begin by discussing popular approaches to ABC and how they may be extended to time series simulators. We will then introduce path signatures, and discuss how signatures naturally lead to two instances of ABC for time series simulators. Finally, we will demonstrate that the resulting signature-based ABC procedures can produce competitive Bayesian parameter inference for simulators generating univariate, multivariate, irregularly spaced, and even non-Euclidean sequences.

Reference: J. Dyer, P. Cannon, S. M Schmon (2022). Approximate Bayesian Computation with Path Signatures. arXiv preprint 2106.12555

call for posters at BayesComp²³ satellite [AG:DC]

An urgent reminder that the early bird deadline for BayesComp²³ and the different satellites is 30 November (with a difference of $50) and also a call for poster presentations at our AG:DC (aka, Bayesian computing without exact likelihood) satellite workshop. Poster spots will be attributed to presenters on a first come – first served basis, so do not delay in sending me an abstract at my gmail account bayesianstatistics

the new DIYABC-RF

My friends and co-authors from Montpellier have released last month the third version of the DIYABC software, DIYABC-RF, which includes and promotes the use of random forests for parameter inference and model selection, in connection with Louis Raynal’s thesis. Intended as the earlier versions of DIYABC for population genetic applications. Bienvenue!!!

The software DIYABC Random Forest (hereafter DIYABC-RF) v1.0 is composed of three parts: the dataset simulator, the Random Forest inference engine and the graphical user interface. The whole is packaged as a standalone and user-friendly graphical application named DIYABC-RF GUI and available at https://diyabc.github.io. The different developer and user manuals for each component of the software are available on the same website. DIYABC-RF is a multithreaded software on three operating systems: GNU/Linux, Microsoft Windows and MacOS. One can use the program can be used through a modern and user-friendly graphical interface designed as an R shiny application (Chang et al. 2019). For a fluid and simplified user experience, this interface is available through a standalone application, which does not require installing R or any dependencies and hence can be used independently. The application is also implemented in an R package providing a standard shiny web application (with the same graphical interface) that can be run locally as any shiny application, or hosted as a web service to provide a DIYABC-RF server for multiple users.

marginal likelihood as exhaustive X validation

In the June issue of Biometrika (for which I am deputy editor) Edwin Fong and Chris Holmes have a short paper (that I did not process!) on the validation of the marginal likelihood as the unique coherent updating rule. Marginal in the general sense of Bissiri et al. (2016). Coherent in the sense of being invariant to the order of input of exchangeable data, if in a somewhat self-defining version (Definition 1). As a consequence, marginal likelihood arises as the unique prequential scoring rule under coherent belief updating in the Bayesian framework. (It is unique given the prior or its generalisation, obviously.)

“…we see that 10% of terms contributing to the marginal likelihood come from out-of-sample predictions, using on average less than 5% of the available training data.”

The paper also contains the interesting remark that the log marginal likelihood is the average leave-p-out X-validation score, across all values of p. Which shows that, provided the marginal can be approximated, the X validation assessment is feasible. Which leads to a highly relevant (imho) spotlight on how this expresses the (deadly) impact of the prior selection on the numerical value of the marginal likelihood. Leaving outsome of the least informative terms in the X-validation leads to exactly the log geometric intrinsic Bayes factor of Berger & Pericchi (1996). Most interesting connection with the Bayes factor community but one that depends on the choice of the dismissed fraction of p‘s.