## distributed evidence

Posted in Books, pictures, Statistics, University life with tags , , , , , , , , , , , , , , , , , , on December 16, 2021 by xi'an

Alexander Buchholz (who did his PhD at CREST with Nicolas Chopin), Daniel Ahfock, and my friend Sylvia Richardson published a great paper on the distributed computation of Bayesian evidence in Bayesian Analysis. The setting is one of distributed data from several sources with no communication between them, which relates to consensus Monte Carlo even though model choice has not been particularly studied from that perspective. The authors operate under the assumption of conditionally conjugate models, i.e., the existence of a data augmentation scheme into an exponential family so that conjugate priors can be used. For a division of the data into S blocks, the fundamental identity in the paper is

$p(y) = \alpha^S \prod_{s=1}^S \tilde p(y_s) \int \prod_{s=1}^S \tilde p(\theta|y_s)\,\text d\theta$

where α is the normalising constant of the sub-prior exp{log[p(θ)]/S} and the other terms are associated with this prior. Under the conditionally conjugate assumption, the integral can be approximated based on the latent variables. Most interestingly, the associated variance is directly connected with the variance of

$p(z_{1:S}|y)\Big/\prod_{s=1}^S \tilde p(z_s|y_s)$

under the joint:

“The variance of the ratio measures the quality of the product of the conditional sub-posterior as an importance sample proposal distribution.”

Assuming this variance is finite (which is likely). An approximate alternative is proposed, namely to replace the exact sub-posterior with a Normal distribution, as in consensus Monte Carlo, which should obviously require some consideration as to which parameterisation of the model produces the “most normal” (or the least abnormal!) posterior. And ensures a finite variance in the importance sampling approximation (as ensured by the strong bounds in Proposition 5). A problem shared by the bridgesampling package.

“…if the error that comes from MCMC sampling is relatively small and that the shard sizes are large enough so that the quality of the subposterior normal approximation is reasonable, our suggested approach will result in good approximations of the full data set marginal likelihood.”

The resulting approximation can also be handy in conjunction with reversible jump MCMC, in the sense that RJMCMC algorithms can be run in parallel on different chunks or shards of the entire dataset. Although the computing gain may be reduced by the need for separate approximations.

## Bayesians at the helm!

Posted in pictures, Statistics, University life with tags , , , , , , , , on October 10, 2021 by xi'an

Just read the announcement that my friend (and former colleague at Warwick U) Mark Girolami became the Chief Scientist at The Alan Turing Institute, joining forces with Adrian Smith, currently Director and Chief Executive of the Turing Institute, into a Bayesian leadership!

## Le Monde on the “dangers” of mathematics

Posted in Books, Statistics, Travel, University life with tags , , , , on October 12, 2015 by xi'an

“La responsabilité des mathématiciens semble engagée.”

This post is presumably aiming at a very small (French speaking) audience, but Le Monde published a central Science leaflet this week on the dangers of using uncontrolled mathematical modelling. Resulting in a mismatch of platitudes and absurdities. Blaming mathematicians for about every misappropriate use of mathematics and even more statistics, from the lack of reproducibility in published psychology studies and the poor predictions of flu epidemics by Google to the sub-prime crisis and the prosecutor fallacy. Quoting judicial miscarriages like the case of Lucy de Berk when the statistical arguments were administrated by a psychologist, while a statistician, Richard Gill, was instrumental in reopening the case by demonstrating those arguments were wrong. Objecting to the use of logistic regression for profiling inmates on the probability of recidivism. &tc., &tc… The only item of interest in this really poor article is the announcement of a semester workshop at the Isaac Newton Institute on the use of mathematics in criminal sciences. Which after verification is a workshop on probability and statistics in forensic sciences. With Richard Gill as one of the organisers.

## Alan Turing Institute

Posted in Books, pictures, Running, Statistics, University life with tags , , , , , , , , , on February 10, 2015 by xi'an

The University of Warwick is one of the five UK Universities (Cambridge, Edinburgh, Oxford, Warwick and UCL) to be part of the new Alan Turing Institute.To quote from the University press release,  “The Institute will build on the UK’s existing academic strengths and help position the country as a world leader in the analysis and application of big data and algorithm research. Its headquarters will be based at the British Library at the centre of London’s Knowledge Quarter.” The Institute will gather researchers from mathematics, statistics, computer sciences, and connected fields towards collegial and focussed research , which means in particular that it will hire a fairly large number of researchers in stats and machine-learning in the coming months. The Department of Statistics at Warwick was strongly involved in answering the call for the Institute and my friend and colleague Mark Girolami will the University leading figure at the Institute, alas meaning that we will meet even less frequently! Note that the call for the Chair of the Alan Turing Institute is now open, with deadline on March 15. [As a personal aside, I find the recognition that Alan Turing’s genius played a pivotal role in cracking the codes that helped us win the Second World War. It is therefore only right that our country’s top universities are chosen to lead this new institute named in his honour. by the Business Secretary does not absolve the legal system that drove Turing to suicide….]

## Bill Fitzgerald (1948-2014)

Posted in Books, Statistics, University life with tags , , , , on April 4, 2014 by xi'an

Just heard a very sad item of news: our colleague and friend Bill Fitzgerald, Head of Research in the Signal Processing Laboratory in the Department of Engineering at the University of Cambridge, Fellow of Christ’s College, co-founder and Chairman of Featurespace, and fanatic guitar player, passed away yesterday. He wrote one of the very first books on MCMC with Joseph Ó Ruanaidh, Numerical Bayesian Methods Applied to Signal Processing, in 1996. On a more personal level, he invited me to Cambridge for my first visit there  in 1998 and he thus was influential in introducing me to my friends Christophe Andrieu and Arnaud Doucet. Farewell, Bill!, and may the blessing of the rain be on you…