Archive for ASA

errors, blunders, and lies [book review]

Posted in Books, Kids, Statistics, University life with tags , , , , , , , , , on July 9, 2017 by xi'an

This new book by David Salsburg is the first one in the ASA-CRC Series on Statistical Reasoning in Science and Society. Which explains why I heard about it both from CRC Press [as a suggested material for a review in CHANCE] and from the ASA [as mass emailing]. The name of the author did not ring a bell until I saw the line about his earlier The Lady Tasting Tea book,  a best-seller in the category of “soft [meaning math- and formula-free] introduction to Statistics through picturesque characters”. Which I did not read either [but Bob Carpenter did].

The current book is of the same flavour, albeit with some maths formulas [each preceded by a lengthy apology for using maths and symbols]. The topic is the one advertised in the title, covering statistical errors and the way to take advantage of them, model mis-specification and robustness, and the detection of biases and data massaging. I read the short book in one quick go, waiting for the results of the French Legislative elections, and found no particular appeal in the litany of examples, historical entries, pitfalls, and models I feel I have already read so many times in the story-telling approach to statistics. (Naked Statistics comes to mind.)

It is not that there anything terrible with the book, which is partly based on the author’s own experience in a pharmaceutical company, but it does not seem to bring out any novelty for engaging into the study of statistics or for handling data in a more rational fashion. And I do not see which portion of the readership is targeted by the book, which is too allusive for academics and too academic for a general audience, who is not necessarily fascinated by the finer details of the history (and stories) of the field. As in The Lady Tasting Tea, the chapters constitute a collection of vignettes, rather than a coherent discourse leading to a theory or defending an overall argument. Some chapters are rather poor, like the initial chapter explaining the distinction between lies, blunders, and errors through the story of the measure of the distance from Earth to Sun by observing the transit of Venus, not that the story is uninteresting, far from it!, but I find it lacking in connecting with statistics [e.g., the meaning of a “correct” observation is never explained]. Or the chapter on the Princeton robustness study, where little is explained about the nature of the wrong distributions, which end up as specific contaminations impacting mostly the variance. And some examples are hardly convincing, like those on text analysis (Chapters 13, 14, 15), where there is little backup for using Benford’s law on such short datasets.  Big data is understood only under the focus of large p, small n, which is small data in my opinion! (Not to mention a minor crime de lèse-majesté in calling Pierre-Simon Laplace Simon-Pierre Laplace! I would also have left the Marquis de aside as this title came to him during the Bourbon Restauration, despite him having served Napoléon for his entire reign.) And, as mentioned above, the book contains apologetic mathematics, which never cease to annoy me since apologies are not needed. While the maths formulas are needed.

not an ASA’s statement on p-values

Posted in Books, Kids, Statistics, University life with tags , , , , on March 18, 2016 by xi'an


This may be a coincidence, but a few days after the ASA statement got published, Yuri Gurevich and Vladimir Vovk arXived a note on the Fundamentals of p-values. Which actually does not contribute to the debate. The paper is written in a Q&A manner. And defines a sort of peculiar logic related with [some] p-values. A second and more general paper is in the making, which may shed more light on the potential appeal of this formalism…

ASA’s statement on p-values [#2]

Posted in Books, Kids, Statistics, University life with tags , , , , , , , , on March 9, 2016 by xi'an


It took a visit on FiveThirtyEight to realise the ASA statement I mentioned yesterday was followed by individual entries from most members of the panel, much more diverse and deeper than the statement itself! Without discussing each and all comments, some points I subscribe to

  • it does not make sense to try to replace the p-value and the 5% boundary by something else but of the same nature. This was the main line of our criticism of Valen Johnson’s PNAS paper with Andrew.
  • it does not either make sense to try to come up with a hard set answer about whether or not a certain parameter satisfies a certain constraint. A comparison of predictive performances at or around the observed data sounds much more sensible, if less definitive.
  • the Bayes factor is often advanced as a viable alternative to the p-value in those comments, but it suffers from difficulties exposed in our recent testing by mixture paper, one being the lack of absolute scale.
  • we seem unable to escape the landscape set by Neyman and Pearson when constructing their testing formalism, including the highly unrealistic 0-1 loss function. And the grossly asymmetric opposition between null and alternative hypotheses.
  • the behaviour of any procedure of choice should be evaluated under different scenarios, most likely by simulation, including some accounting for misspecified models. Which may require an extra bit of non-parametrics. And we should abstain from considering further than evaluating whether or not the data looks compatible with each of the scenarios. Or how much through the mixture representation.

ASA’s statement on p-values

Posted in Books, Statistics, University life with tags , , , , , on March 8, 2016 by xi'an


Last night I received an email from the ASA signed by Jessica Utts and Ron Wasserstein with the following sentence

“Widespread use of ‘statistical significance’ (generally interpreted as ‘p

In short, we envision a new era, in which the broad scientific community recognizes what statisticians have been advocating for many years. In this “post p

Is such an era beyond reach? We think not, but we need your help in making sure this opportunity is not lost.”

which is obviously missing important bits. The email was pointing out a free access American Statistician article warning about the misuses and over-interpretations of p-values. Which contains rather basic “principles” that p-values are not probabilities that the null is true, that there is no golden level against which to compare the p-value, that nominal p-values may be far from actual p-values, that they do not provide a measure of evidence per se, &tc. As written in the conclusion, “Nothing in the ASA statement is new”. But, besides calling for caution and the cumulative use of different assessments of evidence, this statement may leave the non-statistician completely nonplussed about how to proceed when testing hypotheses or comparing models. And make the decision of Basic and Applied Social Psychology of rejecting all arguments based on p-values sound sensible.

Incidentally, the article contains the completion of the first sentence [in red below], if not of the second:

“Widespread use of ‘statistical significance’ (generally interpreted as ‘p≤ 0.05”) as a license for making a claim of a scientific finding (or implied truth) leads to considerable distortion of the scientific process.


JSM 2015 [day #4]

Posted in pictures, Running, Statistics, Travel, University life with tags , , , , , , , , , , , , , , , , , , on August 13, 2015 by xi'an

My first session today was Markov Chain Monte Carlo for Contemporary Statistical Applications with a heap of interesting directions in MCMC research! Now, without any possible bias (!), I would definitely nominate Murray Pollock (incidentally from Warwick) as the winner for best slides, funniest presentation, and most enjoyable accent! More seriously, the scalable Langevin algorithm he developed with Paul Fearnhead, Adam Johansen, and Gareth Roberts, is quite impressive in avoiding computing costly likelihoods. With of course caveats on which targets it applies to. Murali Haran showed a new proposal to handle high dimension random effect models by a projection trick that reduces the dimension. Natesh Pillai introduced us (or at least me!) to a spectral clustering that allowed for an automated partition of the target space, itself the starting point to his parallel MCMC algorithm. Quite exciting, even though I do not perceive partitions as an ideal solution to this problem. The final talk in the session was Galin Jones’ presentation of consistency results and conditions for multivariate quantities which is a surprisingly unexplored domain. MCMC is still alive and running!

The second MCMC session of the morning, Monte Carlo Methods Facing New Challenges in Statistics and Science, was equally diverse, with Lynn Kuo’s talk on the HAWK approach, where we discovered that harmonic mean estimators are still in use, e.g., in MrBayes software employed in phylogenetic inference. The proposal to replace this awful estimator that should never be seen again (!) was rather closely related to an earlier solution of us for marginal likelihood approximation, based there on a partition of the whole space rather than an HPD region in our case… Then, Michael Betancourt brilliantly acted as a proxy for Andrew to present the STAN language, with a flashy trailer he most recently designed. Featuring Andrew as the sole actor. And with great arguments for using it, including the potential to run expectation propagation (as a way of life). In fine, Faming Liang proposed a bootstrap subsampling version of the Metropolis-Hastings algorithm, where the likelihood acknowledging the resulting bias in the limiting distribution.

My first afternoon session was another entry on Statistical Phylogenetics, somewhat continued from yesterday’s session. Making me realised I had not seen a single talk on ABC for the entire meeting! The issues discussed in the session were linked with aligning sequences and comparing  many trees. Again in settings where likelihoods can be computed more or less explicitly. Without any expertise in the matter, I wondered at a construction that would turn all trees, like  into realizations of a continuous model. For instance by growing one branch at a time while removing the MRCA root… And maybe using a particle like method to grow trees. As an aside, Vladimir Minin told me yesterday night about genetic mutations that could switch on and off phenotypes repeatedly across generations… For instance  the ability to glow in the dark for species of deep sea fish.

When stating that I did not see a single talk about ABC, I omitted Steve Fienberg’s Fisher Lecture R.A. Fisher and the Statistical ABCs, keeping the morceau de choix for the end! Even though of course Steve did not mention the algorithm! A was for asymptotics, or ancilarity, B for Bayesian (or biducial??), C for causation (or cuffiency???)… Among other germs, I appreciated that Steve mentioned my great-grand father Darmois in connection with exponential families! And the connection with Jon Wellner’s LeCam Lecture from a few days ago. And reminding us that Savage was a Fisher lecturer himself. And that Fisher introduced fiducial distributions quite early. And for defending the Bayesian perspective. Steve also set some challenges like asymptotics for networks, Bayesian model assessment (I liked the notion of stepping out of the model), and randomization when experimenting with networks. And for big data issues. And for personalized medicine, building on his cancer treatment. No trace of the ABC algorithm, obviously, but a wonderful Fisher’s lecture, also most obviously!! Bravo, Steve, keep thriving!!!

Ebola virus [and Mr. Bayes]

Posted in Statistics, Travel, University life with tags , , , , , , , on August 12, 2014 by xi'an

Just like after the Malaysian Airlines flight 370 disappearance, the current Ebola virus outbreak makes me feel we are sorely missing an emergency statistical force to react on urgent issues… It would indeed be quite valuable to have a team of statisticians at the ready to quantify risks and posterior probabilities and avoid media approximations. The situations calling for this reactive force abound. A few days ago I was reading about the unknown number of missing pro-West activists in Eastern Ukraine. Maybe statistical societies could join forces to set such an emergency team?! Whose goals are somewhat different from the great Statistics without Borders

As a side remark, the above philogeny is taken from Dudas and Rambaut’s recent paper in PLOS reassessing the family tree of the current Ebola virus(es) acting in Guinea. The tree is found using MrBayes, which delivers a posterior probability of 1 to this filiation! And concluding “that the rooting of this clade using the very divergent other ebolavirus species is very problematic.”

Nobel prize in statistics???

Posted in Kids, Statistics, University life with tags , , , , on January 4, 2014 by xi'an

Xiao-Li Meng asked this question in his latest XL column, to which Andrew replied faster than I. And in the same mood as mine. I had taken part to a recent discussion on this topic within the IMS Council, namely whether or not the IMS should associate with other organisations like ASA towards funding and supporting this potential prize. My initial reaction was one of surprise that we could consider mimicking/hijacking the Nobel for our field. First, I dislike the whole spirit of most prizes, from the personalisation to the media frenzy and distortion, to the notion that we could rank discoveries and research careers within a whole field. And separate what is clearly due to a single individual from what is due to a team of researchers.

Being clueless about those fields, I will not get into a discussion of who should have gotten a Nobel Prize in medicine, physics, or chemistry. And who should not have. But there are certainly many worthy competitors to the actual winners. And this is not the point: I do not see how any of this fights the downfall of scientific students in most of the Western World. That is, how a teenager can get more enticed to undertake maths or physics studies because she saw a couple old guys wearing weird clothes getting a medal and a check in Sweden. I have no actual data, but could Xiao-Li give me a quantitative assessment of the fact that Nobel Prizes “attract future talent”? Chemistry departments keep closing for lack of a sufficient number of students, (pure) maths and physics departments threatened with the same fate… Even the Fields Medal, which has at least the appeal of being delivered to younger researchers, does not seem to fit Xiao-Li’s argument. (To take a specific example: The recent Fields medallist Cédric Villani is a great communicator and took advantage of his medal to promote maths throughout France, in conferences, the medias, and by launching all kinds of initiative. I still remain sceptical about the overall impact on recruiting young blood in maths programs [again with no data to back up my feeling).) I will even less mention Nobel prizes for literature and peace, as there clearly is a political agenda in the nomination. (And selecting Sartre for the Nobel prize for literature definitely discredited it. At least for me.)

“…the media and public have given much more attention to the Fields Medal than to the COPSS Award, even though the former has hardly been about direct or even indirect impact on everyday life.” XL

Well, I do not see this other point of Xiao-Li’s. Nobel prizes are not prestigious for their impact on society, as most people do not understand at all what the rewarded research (career) is about. The most extreme example is the Sveriges Riksbank Prize in Economic Sciences in Memory of Alfred Nobel: On the one hand, Xiao-Li is right in pointing out that this is a very successful post-Alfred creation of a “Nobel Prize”. On the other hand, the fact that some years see two competing theories simultaneously win leads me to consider that this prize gives priority to theoretical construct above any impact on the World’s economy. Obviously, this statement is a bit of shooting our field in the foot since the only statisticians who got a Nobel Prize are econometricians and game-theorists! Nonetheless, it also shows that the happy few statisticians who entered the Nobel Olympus did not bring a bonus to the field… I am thus  remaining my usual pessimistic self on the impact of a whatever-company Prize in Statistical Sciences in Memory of Alfred Nobel.

Another remark is the opposition between the COPSS Award, which remains completely ignored by the media (despite a wealth of great nominees with various domains of achievements) and the Fields Medal (which is not ignored). This has been a curse of Statistics that has been discussed at large, namely the difficulty to separate what is math and what is outside math within the field. The Fields Medal is clearly very unlikely to nominate a statistician, even a highly theoretical statistician, as there will always be “sexier” maths results, i.e. corpora of work that will be seen as higher maths than, say, the invention of the Lasso or the creation of generalized linear models. So there is no hope to reach for an alternative Fields Medal with the same shine. Just like the Nobel Prize.

Other issues I could have mentioned, but for the length of the current rant, are the creation of rewards for solving a specific problem (as some found in Machine Learning), for involving multidisciplinary and multicountry research teams, and for reaching new orders of magnitude in processing large data problems.