Archive for expertise

ERC panel [step #2]

Posted in Kids, pictures, Statistics, Travel, University life with tags , , , , , , , , , on April 2, 2019 by xi'an

Another post that was written ages ago, about the second round of the European Research Council (ERC) panel on starting grants for mathematics in which I took part as an expert and not as an applicant. While anonymity possibly fell apart for the several dozens of applicants who were shortlisted for interview, in particular more like those few from my own field, the official list of the panel only came out much later. The interviews were quite interesting, obviously, with a strict attention to time and questions to make all interviews as “equal” as possible. And sometimes painful to attend as the candidates were visibly stressed and more over-prepared than not. Which did not necessarily help as the preparation, presumably with the help of local consultants out of maths, had removed some of the enthusiasm behind the project and too much of the maths. I think we all stopped breathing when one applicant broke mid-sentence, as in a theatre play when one actor forgets one’s lines… The rehearsal does not work so well for later questions, even though preparing for these is also essential,  and some upgrading or downgrading may then occur because of a single answer. An unavoidable limitation of the exercise.

Overall I remain impressed by the quality of the collective work of the panel [despite a gruelling schedule on interview days] and of the overall selection of eleven projects, even though it sounds like more theoretical and abstract topics seem privileged, in a bias that seems difficult to counteract. And [not because no statistics proposal was selected this time] making me (and others) wonder whether or not a separate statistics section of the ERC would not be more appropriate, since statistics proposals are not uniquely and solely centred on the maths aspects.

ERC panel [step #1]

Posted in Kids, pictures, Statistics, Travel, University life with tags , , , , , , , , , on March 16, 2019 by xi'an

Although this post was written ages ago, regulations of the European Research Council (ERC) prevented me from posting it until now, for confidentiality reasons. I was indeed nominated as an expert member of the ERC panel on starting grants for mathematics [a denomination including statistics, obviously, but also quantum physics or some aspects of it], which means evaluating a hundred-ish applications of young researchers (five years from PhD) to select about ten of them to be richly funded for the coming five years. The reason for secrecy is that the panel members have to be protected from pursuits from the candidates (or, more likely, from their senior mentors). While this is a pretty heavy commitment, above 20 days total, the evaluation process gets quite interesting and the most annoying part is to have to reject proposals that should be funded, were more funds available. (For obvious reasons, I cannot get into the details of individual proposals, but let me just bemoan that there were too few proposals connected to statistics!) I may however get into my appreciation of the collective work of the panel during the first step selection process. I actually knew no other member prior to my joining the panel and was impressed at how smoothly we managed to work together and incorporate different opinions in a joint perspective. When I re-read these sentences, it feels like langue de bois (double talk), really!, but they truly represent my feelings at the end of the meeting. Making me (almost) looking forward the second step of interviewing the selected candidates in another week-long meeting, again in Brussels, for the interviews and final selection and ranking. (Which is when anonymity falls apart.)

 

Bayes, reproducibility, and the quest for truth

Posted in Books, Kids, Statistics, University life with tags , , , , , , , , , on September 2, 2016 by xi'an

“Avoid opinion priors, you could be held legally or otherwise responsible.”

Don Fraser, Mylène Bedard, Augustine Wong, Wei Lin, and Ailana Fraser wrote a paper to appear in Statistical Science, with the above title. This paper is a continuation of Don’s assessment of Bayes procedures in earlier Statistical Science [which I discussed] and Science 2013 papers, which I would qualify with all due respect of a demolition enterprise [of the Bayesian approach to statistics]…  The argument therein is similar in that “reproducibility” is to be understood therein as providing frequentist confidence assessment. The authors also use “accuracy” in this sense. (As far as I know, there is no definition of reproducibility to be found in the paper.) Some priors are matching priors, in the (restricted) sense that they give second-order accurate frequentist coverage. Most are not matching and none is third-order accurate, a level that may be attained by alternative approaches. As far as the abstract goes, this seems to be the crux of the paper. Which is fine, but does not qualify in my opinion as a criticism of the Bayesian paradigm, given that (a) it makes no claim at frequentist coverage and (b) I see no reason in proper coverage being connected with “truth” or “accuracy”. It truly makes no sense to me to attempt either to put a frequentist hat on posterior distributions or to check whether or not the posterior is “valid”, “true” or “actual”. I similarly consider that Efron‘s “genuine priors” do not belong to the Bayesian paradigm but are on the opposite anti-Bayesian in that they suggest all priors should stem from frequency modelling, to borrow the terms from the current paper. (This is also the position of the authors, who consider they have “no Bayes content”.)

Among their arguments, the authors refer to two tragic real cases: the earthquake at L’Aquila, where seismologists were charged (and then discharged) with manslaughter for asserting there was little risk of a major earthquake, and the indictment of the pharmaceutical company Merck for the deadly side-effects of their drug Vioxx. The paper however never return to those cases and fails to explain in which sense this is connected with the lack of reproducibility or of truth(fullness) of Bayesian procedures. If anything, the morale of the Aquila story is that statisticians should not draw definitive conclusions like there is no risk of a major earthquake or that it was improbable. There is a strange if human tendency for experts to reach definitive conclusions and to omit the many layers of uncertainty in their models and analyses. In the earthquake case, seismologists do not know how to predict major quakes from the previous activity and that should have been the [non-]conclusion of the experts. Which could possibly have been reached by a Bayesian modelling that always includes uncertainty. But the current paper is not at all operating at this (epistemic?) level, as it never ever questions the impact of the choice of a likelihood function or of a statistical model in the reproducibility framework. First, third or 47th order accuracy nonetheless operates strictly within the referential of the chosen model and providing the data to another group of scientists, experts or statisticians will invariably produce a different statistical modelling. So much for reproducibility or truth.

Anonymous fame: all wrong!

Posted in Statistics, University life with tags , , on January 30, 2010 by xi'an

Yesterday, the French daily Liberation ran a story about an appeal trial. As I happened to contribute to an expertise about this murder, I went pestering my office neighbours in Dauphine with the story and my two words of “fame” («très probablement»). The funniest thing is that our conclusion is quoted all wrong! We determined (by a simple Bayesian conjugate analysis of a Binomial experiment) that a cell-phone call was very unlikely to have been given from the place the main suspect said she was, given the antenna that got this call, while Liberation [and maybe the court] reports that the suspect was very likely to be at the location of the murder… No wonder when considering that statistics nor probability is taught in Law schools. Nor directly used in trials, as far as I know.

Ps- It is also comes as a surprise to me that this trial is still going on when considering that the murder took place in 1997 and that we sent our report in early 2000.