Archive for Philosophical Transactions of the Royal Society

Bayesian inference: challenges, perspectives, and prospects

Posted in Books, Statistics, University life with tags , , , , , , , , , , , , , , , , , on March 29, 2023 by xi'an

Over the past year, Judith, Michael and I edited a special issue of Philosophical Transactions of the Royal Society on Bayesian inference: challenges, perspectives, and prospects, in celebration of the current President of the Royal Society, Adrian Smith, and his contributions to Bayesian analysis that have impacted the field up to this day. The issue is now out! The following is the beginning of our introduction of the series.

When contemplating his past achievements, it is striking to align the emergence of massive advances in these fields with some papers or books of his. For instance, Lindley’s & Smith’s ‘Bayes Estimates for the Linear Model’ (1971), a Read Paper at the Royal Statistical Society, is making the case for the Bayesian analysis of this most standard statistical model, as well as emphasizing the notion of exchangeability that is foundational in Bayesian statistics, and paving the way to the emergence of hierarchical Bayesian modelling. It thus makes a link between the early days of Bruno de Finetti, whose work Adrian Smith translated into English, and the current research in non-parametric and robust statistics. Bernardo’s & Smith’s masterpiece, Bayesian Theory (1994), sets statistical inference within decision- and information-theoretic frameworks in a most elegant and universal manner that could be deemed a Bourbaki volume for Bayesian statistics if this classification endeavour had reached further than pure mathematics. It also emphasizes the central role of hierarchical modelling in the construction of priors, as exemplified in Carlin’s et al.‘Hierarchical Bayesian analysis of change point problems’ (1992).

The series of papers published in 1990 by Alan Gelfand & Adrian Smith, esp. ‘Sampling-Based Approaches to Calculating Marginal Densities’ (1990), is overwhelmingly perceived as the birth date of modern Markov chain Monte Carlo (MCMC) methods, as itbrought to the whole statistics community (and the quickly wider communities) the realization that MCMC simulation was the sesame to unlock complex modelling issues. The consequences on the adoption of Bayesian modelling by non-specialists are enormous and long-lasting.Similarly, Gordon’set al.‘Novel approach to nonlinear/non-Gaussian Bayesian state estimation’ (1992) is considered as the birthplace of sequential Monte Carlo, aka particle filtering, with considerable consequences in tracking, robotics, econometrics and many other fields. Titterington’s, Smith’s & Makov’s reference book, ‘Statistical Analysis of Finite Mixtures(1984)  is a precursor in the formalization of heterogeneous data structures, paving the way for the incoming MCMC resolutions like Tanner & Wong (1987), Gelman & King (1990) and Diebolt & Robert (1990). Denison et al.’s book, ‘Bayesian methods for nonlinear classification and regression’ (2002) is another testimony to the influence of Adrian Smith on the field,stressing the emergence of robust and general classification and nonlinear regression methods to analyse complex data, prefiguring in a way the later emergence of machine-learning methods,with the additional Bayesian assessment of uncertainty. It is also bringing forward the capacity of operating Bayesian non-parametric modelling that is now broadly accepted, following a series of papers by Denison et al. in the late 1990s like CART and MARS.

We are quite grateful to the authors contributing to this volume, namely Joshua J. Bon, Adam Bretherton, Katie Buchhorn, Susanna Cramb, Christopher Drovandi, Conor Hassan, Adrianne L. Jenner, Helen J. Mayfield, James M. McGree, Kerrie Mengersen, Aiden Price, Robert Salomone, Edgar Santos-Fernandez, Julie Vercelloni and Xiaoyu Wang, Afonso S. Bandeira, Antoine Maillard, Richard Nickl and Sven Wang , Fan Li, Peng Ding and Fabrizia Mealli, Matthew Stephens, Peter D. Grünwald, Sumio Watanabe, Peter Müller, Noirrit K. Chandra and Abhra Sarkar, Kori Khan and Alicia Carriquiry, Arnaud Doucet, Eric Moulines and Achille Thin, Beatrice Franzolini, Andrea Cremaschi, Willem van den Boom and Maria De Iorio, Sandra Fortini and Sonia Petrone, Sylvia Frühwirth-Schnatter, Sara Wade, Chris C. Holmes and Stephen G. Walker, Lizhen Nie and Veronika Ročková. Some of the papers are open-access, if not all, hence enjoy them!

Galton’s 1904 paper in Nature

Posted in Books, Statistics, University life with tags , , , , , , , , , on October 11, 2022 by xi'an

Nature [28 September] posted an editorial apologizing for publishing Galton’s 1904 speech on Eugenics as part of “material that contributed to bias, exclusion and discrimination in research and society”. Apology that I do not find particularly pertinent from an historical viewpoint, given the massive time, academic, and societal distances we stand from this “paper”, which sounds more than a pamphlet than a scientific paper, by current standards. Reading these 1904 Nature articles show no more connection with a modern scientific journal than considering Isaac Newton’s alchemy notes in the early proceedings of the Royal Society.

“The aim of eugenics is to represent each class or sect by its best specimens; that done, to leave them to work out their common civilization in their own way.” F. Galton

Galton’s speech was published in extenso by the American Journal of Sociology, along with discussions from participants of the Sociological Society meeting. This set of discussions is rather illuminating as the views of the 1904 audience are quite heterogeneous, from complete adherence to a eugenic “golden’ future (see the zealous interventions of K. Pearson or B. Shaw] to misgivings about the ability to define the supposed ranking of members of society by worth or intelligence (H.G. Wells), to rejection that moral traits are genetically inherited (Mercier), to protests against the negation of individual freedom induced by a eugenic state (B. Kidd) and to common sense remarks that improvements in living conditions of the working classes were the key factor in improving society. But, overall, there was no disagreement therein on the very notion of races and on the supposed superiority of the Victorian civilization (with an almost complete exclusion of women from the picture), reflecting on the prejudices of the era and it is quite unlikely that this 1904 paper of Galton had any impact on these prejudices.

Price’s theorem?

Posted in Statistics with tags , , , , , , on March 16, 2013 by xi'an

A very interesting article by Martyn Hooper in Significance Feb. 2013 issue I just received. (It is available on-line for free.) It raises the question as to how much exactly Price contributed to the famous Essay… Given the percentage of the Essay that can be attributed to Price with certainty (Bayes’ part stops at page 14 out of 32 pages), given the lack of the original manuscript by Bayes, given the delay between the composition of this original manuscript (1755?), its delivery to Price (1761?) and its publication in 1763, given the absence of any other document published by Bayes on the topic, I tend to concur with Martyn Hooper (and Sharon McGrayne) that Price contributed quite significantly to the 1763 paper. Of course, it would sound quite bizarre to start calling our approach to Statistics Pricean or Pricey (or even Priceless!) Statistics, but this may constitute one of the most striking examples of Stigler’s Law of Eponymy!

reading classics (#9)

Posted in Books, Statistics, University life with tags , , , , , , , , on February 24, 2013 by xi'an

In today’s classics seminar, my student Bassoum Abou presented the 1981 paper written by Charles Stein for the Annals of Statistics, Estimating the mean of a normal distribution, recapitulating the advances he made on Stein estimators, minimaxity and his unbiased estimator of risk. Unfortunately; this student missed a lot about paper and did not introduce the necessary background…So I am unsure at how much the class got from this great paper… Here are his slides (watch out for typos!)

 Historically, this paper is important as this is one of the very few papers published by Charles Stein in a major statistics journal, the other publications being made in conference proceedings. It contains the derivation of the unbiased estimator of the loss, along with comparisons with posterior expected loss.

reading classics (#8)

Posted in Books, Statistics, University life with tags , , , , , , , , on February 1, 2013 by xi'an

In today’s classics seminar, my student Dong Wei presented the historical paper by Neyman and Pearson on efficient  tests: “On the problem of the most efficient tests of statistical hypotheses”, published in the Philosophical Transactions of the Royal Society, Series A. She had a very hard time with the paper… It is not an easy paper, to be sure, and it gets into convoluted and murky waters when it comes to the case of composite hypotheses testing. Once again, it would have been nice to broaden the view on testing by including some of the references given in Dong Wei’s slides:

Listening to this talk, while having neglected to read the original paper for many years (!), I was reflecting on the way tests, Type I & II, and critical regions were introduced, without leaving any space for a critical (!!) analysis of the pertinence of those concepts. This is an interesting paper also because it shows the limitations of such a notion of efficiency. Apart from the simplest cases, it is indeed close to impossible to achieve this efficiency because there is no most powerful procedure (without restricting the range of those procedures). I also noticed from the slides that Neyman and Pearson did not seem to use a Lagrange multiplier to achieve the optimal critical region. (Dong Wei also inverted the comparison of the sufficient and insufficient statistics for the test on the variance, as the one based on the sufficient statistic is more powerful.) In any case, I think I will not keep the paper in my list for next year, maybe replacing it with the Karlin-Rubin (1956) UMP paper…

%d bloggers like this: