Archive for Mike Titterington

Finite mixture models do not reliably learn the number of components

Posted in Books, Statistics, University life with tags , , , , , , , , , , , , , on October 15, 2022 by xi'an

When preparing my talk for Padova, I found that Diana Cai, Trevor Campbell, and Tamara Broderick wrote this ICML / PLMR paper last year on the impossible estimation of the number of components in a mixture.

“A natural check on a Bayesian mixture analysis is to establish that the Bayesian posterior on the number of components increasingly concentrates near the truth as the number of data points becomes arbitrarily large.” Cai, Campbell & Broderick (2021)

Which seems to contradict [my formerly-Glaswegian friend] Agostino Nobile  who showed in his thesis that the posterior on the number of components does concentrate at the true number of components, provided the prior contains that number in its support. As well as numerous papers on the consistency of the Bayes factor, including the one against an infinite mixture alternative, as we discussed in our recent paper with Adrien and Judith. And reminded me of the rebuke I got in 2001 from the late David McKay when mentioning that I did not believe in estimating the number of components, both because of the impact of the prior modelling and of the tendency of the data to push for more clusters as the sample size increased. (This was a most lively workshop Mike Titterington and I organised at ICMS in Edinburgh, where Radford Neal also delivered an impromptu talk to argue against using the Galaxy dataset as a benchmark!)

“In principle, the Bayes factor for the MFM versus the DPM could be used as an empirical criterion for choosing between the two models, and in fact, it is quite easy to compute an approximation to the Bayes factor using importance sampling” Miller & Harrison (2018)

This is however a point made in Miller & Harrison (2018) that the estimation of k logically goes south if the data is not from the assumed mixture model. In this paper, Cai et al. demonstrate that the posterior diverges, even when it depends on the sample size. Or even the sample as in empirical Bayes solutions.

David Cox (1924-2022)

Posted in Books, Statistics, University life with tags , , , , , , , , , , , , , , , , , , , , , , , , on January 20, 2022 by xi'an

It is with much sadness that I heard from Oxford yesterday night that David Cox had passed away. Hither goes a giant of the field, whose contributions to theoretical and methodological statistics are enormous and whose impact on society is truly exceptional. He was the first recipient of the International Prize in Statistics in 2016 (aka the “Nobel of Statistics”) among many awards and a Fellow of the Royal Society among many other recognitions. He was also the editor of Biometrika for 25 years (!) and was still submitting papers to the journal a few month ago. Statistical Science published a conversation between Nancy Reid and him that tells a lot about the man and his amazing modesty. While I had met him in 1989, when he was visiting Cornell University as a distinguished visitor (and when I drove him to the house of Anne and George Casella for dinner once), then again in the 1990s when he came on a two-day visit to CREST,  we only really had a significant conversation in 2011 (!), when David and I attended the colloquium in honour of Mike Titterington in Glasgow and he proved to be most interested in the ABC algorithm. He published a connected paper in Biometrika the year after, with Christiana Katsonaki. We met a few more times later, always in Oxford, to again discuss ABC. In each occasion, he was incredibly kind and considerate.

Biometrika, volume 100

Posted in Books, Statistics, University life with tags , , , , , , , , , , , , , , on March 5, 2013 by xi'an

I had been privileged to have a look at a preliminary version of the now-published retrospective written by Mike Titterington on the 100 first issues of Biometrika (more exactly, “from volume 28 onwards“, as the title state). Mike was the dedicated editor of Biometrika for many years and edited a nice book for the 100th anniversary of the journal. He started from the 100th most highly cited papers within the journal to build a coherent chronological coverage. From a Bayesian perspective, this retrospective starts with Maurice Kendall trying to reconcile frequentists and non-frequentists in 1949, while having a hard time with fiducial statistics. Then Dennis Lindley makes it to the top 100 in 1957 with the Lindley-Jeffreys paradox. From 1958 till 1961, Darroch is quoted several times for his (fine) formalisation of the capture-recapture experiments we were to study much later (Biometrika, 1992) with Ed George… In the 1960’s, Bayesian papers became more visible, including Don Fraser (1961) and Arthur Dempster’ Demspter-Shafer theory of evidence, as well as George Box and co-authors (1965, 1968) and Arnold Zellner (1964). Keith Hastings’ 1970 paper stands as the fifth most highly cited paper, even though it was ignored for almost two decades. The number of Bayesian papers kept increasing. including Binder’s (1978) cluster estimation, Efron and Morris’ (1972) James-Stein estimators, and Efron and Thisted’s (1978) terrific evaluation of Shakespeare’s vocabulary. From then, the number of Bayesian papers gets too large to cover in its entirety. The 1980’s saw papers by Julian Besag (1977, 1989, 1989 with Peter Clifford, which was yet another precursor MCMC) and Luke Tierney’s work (1989) on Laplace approximation. Carter and Kohn’s (1994) MCMC algorithm on state space models made it to the top 40, while Peter Green’s (1995) reversible jump algorithm came close to Hastings’ (1970) record, being the 8th most highly cited paper. Since the more recent papers do not make it to the top 100 list, Mike Titterington’s coverage gets more exhaustive as the years draw near, with an almost complete coverage for the final years. Overall, a fascinating journey through the years and the reasons why Biometrika is such a great journal and constantly so.

Reading classics (#5)

Posted in Books, Statistics, University life with tags , , , , , , , , , on December 14, 2012 by xi'an

This week, my student Dona Skanji gave a presentation of the paper of Hastings “Monte Carlo sampling methods using Markov chains and their applications“, which set the rules for running MCMC algorithms, much more so than the original paper by Metropolis et al. which presented an optimisation device. even though the latter clearly stated the Markovian principle of those algorithms and their use for integration. (This is definitely a classic, selected in the book Biometrika: One hundred years, by Mike Titterington and David Cox.) Here are her slides (the best Beamer slides so far!):

Given that I had already taught my lectures on Markov chains and on MCMC algorithms, the preliminary part of Dona’s talk was easier to compose and understanding the principles of the method was certainly more straightforward than for the other papers in the series. I think she nonetheless did a rather good job in summing up the paper, running this extra simulation for the Poisson distribution—with the interesting “mistake” of including the burnin time in the representation of the output and concluding about a poor convergence—and mentioning the Gibbs extension.I led the discussion of the seminar towards irreducibility conditions and Peskun’s ordering of Markov chains, which maybe could have been mentioned by Dona since she was aware Peskun was Hastings‘ student.

Colloquium for Mike Titterington

Posted in Statistics, Travel, University life with tags , , , , , , , , on June 3, 2011 by xi'an

The colloquium held today at Glasgow University in honour of Mike Titterington for his retiral was highly enjoyable! First, it was a pleasure to celebrate Mike’s achievements at this (early) stage of his career, along with people from Glasgow but also from all over the UK and even from Australia, among whom a lot of friends. Second, the (other) talks were highly interesting, with Peter Hall talking about the asymptotics of records, Byron Morgan about identifiability in capture-recapture models, Peter Green presenting a graphical diagnostic for spotting divergence between prior and likelihood in multivariate models, and Adrian Bowman illustrating advanced face analysis using principal curves on lips and faces. Third, I got a fair amount of questions and comments about ABC in general and ABC model choice in particular, including David Cox commenting that ABC was an important new topic and suggesting using goodness-of-fit tools for model comparison. The symposium per se ended up with a specially designed cake covering (in sugar!) some of Mike’s academic endeavours during the past years. While a formal affair for which I had to run to get a shirt, the diner was equally enjoyable, including a simultaneously witty and deep after-dinner talk paying tribute to Mike’s contributions by David Cox (who was Mike’s predecessor as editor of Biometrika) and a funny conclusion by John McColl who dug out a 1976 probability assignment he had from Mike that was the Monty Hall problem.

The next celebration of that kind I am taking part in is Hans Künsch’s 60th birthday in Zürich next October. Looking forward to it!

%d bloggers like this: