nested sampling X check

Andrew Fowlie, Will Handley and Liangliang Su have recently arXived a new paper on checking the convergence of nested sampling by a uniformity test. The argument goes as follows: if the draw from the prior under the likelihood restriction (at the core of the nested sampling principle) is correctly generated, the rank of the realised value of the associated likelihood should be uniformly distributed among the remaining likelihoods. Obviously, the opposite does not hold:  a perfectly uniform distribution can happen even when the sampler misses a particularly well-hidden mode of the target disstribution or when it systematically stops too early, using for instance a misspecified bound on the likelihood. One particular setting when uniformity fails is when the likelihood surface plateaus in a particular region of the parameter space. (As a French speaker, writing plateaus makes me cringe since the plural of plateau is plateaux! Pardon my French!) When reaching the plateau the algorithm starts accumulating at the limiting value (or else completely ignores the plateau and its prior mass). I actually wonder if the existence of plateaux is not a sufficient reason for invalidating nested sampling, at least in its original version, since it assumes a continuous distribution on the likelihood values… If no plateau comes to hinder the algorithm, the rank test could be used to calibrate the exploration algorithm as for instance in the determination of the number of MCMC steps, running in parallel T random walks until the rank test across these runs turns green. The authors of the paper suggest using a Kolmogorov-Smirnov test, which strikes me as not the most appropriate solution, given the discrete nature of the theoretical distribution and the existence of uniformity tests in the pseudo random generation literature. At a conceptual level, I am also wondering at the sequential use of the test (as opposed to a parallel version at each iteration) since the target distribution is changing at every step (and so does the approximate method used to reproduce the prior simulation under the likelihood restriction).

feu rouge

teaching in English

A strange (if very French!) debate is taking place these days in the French main chamber, where some socialist deputies are contesting an incoming change in the regulation of university studies that would allow some courses to be taught in… English! Quelle horreur!!! Since this option has been implemented by many universities, incl. Dauphine, it means that we all are acting outside the law! I do not fear in the least being indicted for teaching R and Bayesian statistics in English… However, I find the action of these deputies missing the point: just like most other Western countries, we need to attract bright students from emerging countries in order to keep our departments open. It is unrealistic to think that those students will accept to learn French in addition to English, just because our universities are that attractive (and they are not!). Plus, our own students are asking for courses in English as they realise that their English level is not that great and that this training is more efficient than regular English courses… This position was better expressed in a Le Monde tribune a few days ago signed by several university professors, incl. Cédric Villani.

Uneducated guesses

I received this book, Uneducated Guesses: Using Evidence to Uncover Misguided Education Policies by Howard Wainer, from Princeton University Press for review in CHANCE. Alas, I am presumably one of the least likely adequate reviewers for the book in that

• having done all of my academic training in France (except for my most useful post-doctoral training in Purdue and in Cornell), I never took any of those ACT/SAT/&tc tests (except for the GRE at the very end of my Ph.D. towards a post-doctoral grant I did not get!);
• teaching in a French university, I never used any of those tests to compare undergraduate or graduates applicants;
• I am very marginally aware of the hiring process in US universities at the undergraduate, even though I knew about the early admission policy;
• there is no equivalent in the French high school system, given that high school students have to undergo a national week-long exam, le baccalauréat, to enter higher education and that most curricula actually decide on the basis of the high school record, prior to [but conditional on] the baccalauréat.

Thus, this review of Wainer’s Uneducated Guesses is to be taken with pinches (or even tablespoons) of salt. And to be opposed to other reviews. Esp. in Statistics journals (I could not find any).

“My role in this parallels Spock’s when he explained `Nowhere am I so desperately needed as among a shipload of illogical humans.‘” (page 157)

First, the book is very pleasant to read, with a witty and whimsical way of pushing strong (and well-argued) opinions. Even as a complete bystander, I found the arguments advanced for keeping SAT as the preferential tool for student selection quite engaging, as were the later ones against teacher and college rankings equally making sense. So the book should appeal to a large chunk of the public, as prospective students, parents, high school teachers or college selection committees. (Scholars on entrance tests may already have seen the arguments since most of the chapter are based on earlier papers of  Howard Wainer.) Second, and this is yet another reason why I feel remote from the topic, the statistical part of the analysis is simply not covered in the book. There are tables and there are graphs, there are regressions and there are interpolation curves, there is a box-plot and there are normal densities, but I am missing a statistical model that would push us further than the common sense that permeates the whole book. After reading the book, my thirst about the modelling of education tests and ranking is thus far from being quenched! (Note I am not saying the author is ignorant of such matters, since he published in psychometrics, educational statistics and other statistics journals, and taught Statistics at Wharton. The technical side of the argument does exist, but it is not included in the book. The author refers to Gelman et al., 1995, and to the fruitful Bayesian approach on page 69.)

Beta translation done!

Once my team of four translators had handed back to me all the chapters of the French version of Introducing Monte Carlo Methods with R to me, I had to go over the book to ensure some minimal consistency between the chapters. I started the editing in the plane to Vancouver but did not get much done until last Monday when arriving in Long Beach. After three days of hard work, here at home, I am now done with the beta version of the translation and I have sent it to the French Springer editor… I hope he will not ask for deep changes as I have absolutely no time left in my schedule for the coming months!