futuristic statistical science [editorial]

This special issue of Statistical Science is devoted to the future of Bayesian computational statistics, from several perspectives. It involves a large group of researchers who contributed to collective articles, bringing their own perspectives and research interests into these surveys. Somewhat paradoxically, it starts with the past—and a conference on a Gold Coast beach. Martin, Frazier, and Robert first submitted a survey on the history of Bayesian computation, written after Gael Martin delivered a plenary lecture at Bayes on the Beach, a conference held in November 2017 in Surfers Paradise, Gold Coast, Queensland, and organised by Bayesian Research and Applications Group (BRAG), the Bayesian research group headed by Kerrie Mengersen at the Queensland University of Technology (QUT). Following a first round of reviews, this paper got split into two separate articles, Computing Bayes: From Then ‘Til Now , retracing some of the history of Bayesian computation, and Approximating Bayes in the 21st Century, which is both a survey and a prospective on the directions and trends of approximate Bayesian approaches (and not solely ABC). At this point, Sonia Petrone, editor of Statistical Science, suggested we had a special issue on the whole issue of trends of interest and promise for Bayesian computational statistics. Joining forces, after some delays and failures to convince others to engage, or to produce multilevel papers with distinct vignettes, we eventually put together an additional four papers, where lead authors gathered further authors to produce this diverse picture of some incoming advances in the field. We have deliberated avoided topics which have excellent recent reviews— such as Stein’s method, sequential Monte Carlo, piecewise deterministic Markov processes— and topics which are still in their infancy, such as the relationship of Bayesian approaches to large language models (LLMs) and foundation models.

Within this issue, Past, Present, and Future of Software for Bayesian Inference from Erik Štrumbelj & al covers the state of the art in the most popular Bayesian software, reminding us of the massive impact BUGS has had on the adoption of Bayesian tools since its early introduction in the early 1990s (which I remember discovering at the Fourth Valencia meeting on Bayesian statistics in April 1991). With an interesting distinction between first and second generations, and a light foray of the potential third generation, maybe missing the role of LLMs in coding that are already impacting the approach to computing and the less immediate revolution brought by quantum computing. Winter & al.’s The Future of Bayesian Computation [TITLE TO CHANCE] is making a link with machine learning techniques, without looking at the scariest issue of how Bayesian inference can survive in a machine learning world! While it produces an additional foray into the blurry division between proper sampling (à la MCMC) and approximations, additional to the historical Martin et al. (2024), it articulates these aspects within a (deep) machine learning perspective, emphasizing the role of summaries produced by generative models exploiting the power of neural network computation/optimization. And the pivotal reliance on variational Bayes, which is the most active common denominator with machine learning. With further entries on major issues like distributed computing, opening on the important aspect of data protection and guaranteed  privacy. We particularly like the clinical presentation of this paper with attention to automation and limitations. Normalizing flows actually link this paper with Heng, Bortoli and Doucet’s coverage of the Schrödinger bridge, which is a more focussed coverage of recent advances on possibly the next generation of posterior samplers. The final paper, Bayesian experimental design by Rainforth & al., provides a most convincing application of the methods exposed in the earlier papers in that the field of Bayesian design has hugely benefited from the occurrence of such tools to become a prevalent way of designing statistical experiments in real settings.

We feel the future of Bayesian computing is bright! The Monte Carlo revolution of the 1990s continues to be a huge influence on today’s work, and now is complemented by an exciting range of new directions informed by modern machine learning.

Dennis Prangle and Christian P Robert

Bayes on the Beach²⁴

One World ABC seminar [31.3.22]

The next One World ABC seminar is on Thursday 31 March, with David Warnes (from QUT) talking on Multifidelity multilevel Monte Carlo for approximate Bayesian computation It will take place at 10:30 CET (GMT+1).

Models of stochastic processes are widely used in almost all fields of science. However, data are almost always incomplete observations of reality. This leads to a great challenge for statistical inference because the likelihood function will be intractable for almost all partially observed stochastic processes. As a result, it is common to apply likelihood-free approaches that replace likelihood evaluations with realisations of the model and observation process. However, likelihood-free techniques are computationally expensive for accurate inference as they may require millions of high-fidelity, expensive stochastic simulations. To address this challenge, we develop a novel approach that combines the multilevel Monte Carlo telescoping summation, applied to a sequence of approximate Bayesian posterior targets, with a multifidelity rejection sampler that learns from low-fidelity, computationally inexpensive,
model approximations to minimise the number of high-fidelity, computationally expensive, simulations required for accurate inference. Using examples from systems biology, we demonstrate improvements of more than two orders of magnitude over standard rejection sampling techniques

Online workshop: Measuring the quality of MCMC output

[Here is an announcement about an online MCMC workshop, posted by Pierre Jacob on Statisfaction and reposted here with his permission. By the way, there is a competition for the conference logo if the above does not sound good enough!]

With Leah South from QUT we are organizing an online workshop on the topic of “Measuring the quality of MCMC output”. The event website is here with more info:

https://bayescomp-isba.github.io/measuringquality.html

This is part of ISBA BayesComp section’s efforts to organize activities while waiting for the next “big” in-person meeting, hopefully in 2023. The event benefits from the generous support of QUT Centre for Data Science. The event’s website will be regularly updated between now and the event in October 2021, with three live sessions:

• 11am-2pm UTC on Wednesday 6th October,
• 1pm-4pm UTC on Thursday 14th October,
• 3pm-6pm UTC on Friday 22nd October.

Registration is free but compulsory (form here) as we want to make sure the live sessions remain convivial and focused; hence the rather specific theme, but it’s an exciting topic with lots of very much open questions, which we hope will attract both practitioners and methodologists. Meanwhile some material will be available on the website to everyone, including video recordings of presentations, and posters, so that the workshop hopefully benefits the wider community.

If you have suggestions for this event, or would like to organize a similar event in the future, on another “BayesComp” topic, do not hesitate to get in touch. Our contact details are on the workshop’s website.

ABC in Svalbard [the day after]

The following and very kind email was sent to me the day after the workshop

thanks once again to make the conference possible. It was full of interesting studies within a friendly environment, I really enjoyed it. I think it is not easy to make a comfortable and inspiring conference in a remote version and across two continents, but this has been the result. I hope to be in presence (maybe in Svalbard!) the next edition.

and I fully agree to the talks behind full of interest and diverse. And to the scheduling of the talks across antipodal locations a wee bit of a challenge, mostly because of the daylight saving time  switches! And to seeing people together being a comfort (esp. since some were enjoying wine and cheese!).

I nonetheless found the experience somewhat daunting, only alleviated by sharing a room with a few others in Dauphine and having the opportunity to react immediately (and off-the-record) to the on-going talk. As a result I find myself getting rather scared by the prospect of the incoming ISBA 2021 World meeting. With parallel sessions and an extensive schedule from 5:30am till 9:30pm (in EDT time, i.e. GMT-4) that nicely accommodates the time zones of all speakers. I am thus thinking of (safely) organising a local cluster to attend the conference together and recover some of the social interactions that are such an essential component of [real] conferences, including students’ participation. It will of course depend on whether conference centres like CIRM reopen before the end of June. And if enough people see some appeal in this endeavour. In the meanwhile, remember to register for ISBA 2021 and for free!, before 01 May.