Bayes 2013

Among the many Bayes.250 meetings this year—including one at the Royal Statistical Society on June 19 and 20 I am co-organising with Chris Holmes—I just became aware of another Bayes 2013 meeting organised in Rotterdam by Emmanuel Lesaffre (I was going to write the ISBA Section on Biostatistics and Pharmaceutical Statistics but could not find a link on the webpage to a sponsorship of the section). Nice visual too! This could be an interesting start to a European tour of meetings in May and June, followed with the French statistical meeting in Toulouse, ABC in Rome, the Bayesian young statistician meeting in Milano, the 9th Conference on Bayesian Nonparametrics in Amsterdam, and then Bayes.250 at the RSS on June 19-20!

semi-automatic ABC [reply]

When I came back from LGM2012 in Trondheim, I found the latest issue of Series B on my desk. It is much thicker than in “my” days, with about 250 pages in this June 2012 issue! (One reason is that it contains two Read Papers with their discussions, amounting to 110 pages of the journal.) The first Read Paper was “Catching up faster by switching sooner” by van Erven, Grünwald and de Rooij, that we discussed with Nicolas Chopin. There are also comments (among others!) from Stephen Lauritzen, Iain Murray, and Aki Vehtari, who also spoke about Bayesian model evaluation tools at LGM2012. The second Read Paper is Fearnhead’s and Prangle’s semi-automatic ABC that I discussed last December. I have already posted about this Read Paper and used some of the discussion in preparing my ABC PhD class in Roma.  However, the remark we made in our discussion with Jean-Michel Marin that the Bayes factor would not be a pertinent summary statistic for model choice is wrong, as shown by Dennis Prangle in his poster at the workshop in Bristol. And, when reading the reply by Paul Fearnhead and Dennis Prangle, I do not see a satisfactory answer to my demand of more formal conditions for Theorem 2 and its corollary, the convergence of the noisy ABC posterior to the true parameter (page 425), to apply. (Such results exist in indirect inference.)

Large-scale Inference

Large-scale Inference by Brad Efron is the first IMS Monograph in this new series, coordinated by David Cox and published by Cambridge University Press. Since I read this book immediately after Cox’ and Donnelly’s Principles of Applied Statistics, I was thinking of drawing a parallel between the two books. However, while none of them can be classified as textbooks [even though Efron's has exercises], they differ very much in their intended audience and their purpose. As I wrote in the review of Principles of Applied Statistics, the book has an encompassing scope with the goal of covering all the methodological steps  required by a statistical study. In Large-scale Inference, Efron focus on empirical Bayes methodology for large-scale inference, by which he mostly means multiple testing (rather than, say, data mining). As a result, the book is centred on mathematical statistics and is more technical. (Which does not mean it less of an exciting read!) The book was recently reviewed by Jordi Prats for Significance. Akin to the previous reviewer, and unsurprisingly, I found the book nicely written, with a wealth of R (colour!) graphs (the R programs and dataset are available on Brad Efron’s home page).

“I have perhaps abused the “mono” in monograph by featuring methods from my own work of the past decade.” (p.xi)

Sadly, I cannot remember if I read my first Efron’s paper via his 1977 introduction to the Stein phenomenon with Carl Morris in Pour la Science (the French translation of Scientific American) or through his 1983 Pour la Science paper with Persi Diaconis on computer intensive methods. (I would bet on the later though.) In any case, I certainly read a lot of the Efron’s papers on the Stein phenomenon during my thesis and it was thus with great pleasure that I saw he introduced empirical Bayes notions through the Stein phenomenon (Chapter 1). It actually took me a while but I eventually (by page 90) realised that empirical Bayes was a proper subtitle to Large-Scale Inference in that the large samples were giving some weight to the validation of empirical Bayes analyses. In the sense of reducing the importance of a genuine Bayesian modelling (even though I do not see why this genuine Bayesian modelling could not be implemented in the cases covered in the book).

“Large N isn’t infinity and empirical Bayes isn’t Bayes.” (p.90)

The core of Large-scale Inference is multiple testing and the empirical Bayes justification/construction of Fdr’s (false discovery rates). Efron wrote more than a dozen papers on this topic, covered in the book and building on the groundbreaking and highly cited Series B 1995 paper by Benjamini and Hochberg. (In retrospect, it should have been a Read Paper and so was made a “retrospective read paper” by the Research Section of the RSS.) Frd are essentially posterior probabilities and therefore open to empirical Bayes approximations when priors are not selected. Before reaching the concept of Fdr’s in Chapter 4, Efron goes over earlier procedures for removing multiple testing biases. As shown by a section title (“Is FDR Control “Hypothesis Testing”?”, p.58), one major point in the book is that an Fdr is more of an estimation procedure than a significance-testing object. (This is not a surprise from a Bayesian perspective since the posterior probability is an estimate as well.)

“Scientific applications of single-test theory most often suppose, or hope for rejection of the null hypothesis (…) Large-scale studies are usually carried out with the expectation that most of the N cases will accept the null hypothesis.” (p.89)

On the innovations proposed by Efron and described in Large-scale Inference, I particularly enjoyed the notions of local Fdrs in Chapter 5 (essentially pluggin posterior probabilities that a given observation stems from the null component of the mixture) and of the (Bayesian) improvement brought by empirical null estimation in Chapter 6 (“not something one estimates in classical hypothesis testing”, p.97) and the explanation for the inaccuracy of the bootstrap (which “stems from a simpler cause”, p.139), but found less crystal-clear the empirical evaluation of the accuracy of Fdr estimates (Chapter 7, ‘independence is only a dream”, p.113), maybe in relation with my early career inability to explain Morris’s (1983) correction for empirical Bayes confidence intervals (pp. 12-13). I also discovered the notion of enrichment in Chapter 9, with permutation tests resembling some low-key bootstrap, and multiclass models in Chapter 10, which appear as if they could benefit from a hierarchical Bayes perspective. The last chapter happily concludes with one of my preferred stories, namely the missing species problem (on which I hope to work this very Spring).

comments in ABC PhD course

Following my reading the discussions of the Read Paper by Fearnhead and Prangle, I included some of their points in my course this morning. Which ended up with me spending the whole two hours on this topic (and finally getting a grasp on calibration!). Here is [hopefully] the final version of the slides.

ABC Read Paper at the Royal Statistical Society [slides]

Here are the slides (modified from the earlier version) I should be presenting right now, more or less!