the end of statistics [not!]

Last week I spotted this tribune in The Guardian, with the witty title of statistics loosing its power, and sort of over-reacted by trying to gather enough momentum from colleagues towards writing a counter-column. After a few days of decantation and a few more readings (reads?) of the tribune, I cooled down towards a more lenient perspective, even though I still dislike the [catastrophic and journalistic] title. The paper is actually mostly right (!), from its historical recap of the evolution of (official) statistics across centuries, to the different nature of the “big data” statistics. (The author is “William Davies, a sociologist and political economist. His books include The Limits of Neoliberalism and The Happiness Industry.”)

“Despite these criticisms, the aspiration to depict a society in its entirety, and to do so in an objective fashion, has meant that various progressive ideals have been attached to statistics.”

A central point is that public opinion has less confidence in (official) statistics than it used to be. (warning: Major understatement, here!) For many reasons, from numbers used to support any argument and its opposite, to statistics (-ians) being associated with experts, found at every corner of news and medias, hence with the “elite” arch-enemy, to a growing innumeracy of both the general public and of the said “elites”—like this “expert” in a debate about the 15th anniversary of the Euro currency on the French NPR last week equating a raise from 2.4 Francs to 6.5 Francs to 700%…—favouring rhetoric over facts, to a disintegration of the social structure that elevates one’s community over others and dismisses arguments from those others, especially those addressed at the entire society. The current debate—and the very fact there can even be a debate about it!—about post-truths and alternative facts is a sad illustration of this regression in the public discourse. The overall perspective in the tribune is one of a sociologist on statistics, but nothing to strongly object to.

“These data analysts are often physicists or mathematicians, whose skills are not developed for the study of society at all.”

The second part of the paper is about the perceived shift from (official) statistics to another and much more dangerous type of data analysis. Which is not a new view on the field, as shown by Weapons of Math Destruction. I tend to disagree with this perception that data handled by private companies for private purposes is inherently evil. The reticence in trusting the conclusions drawn from such datasets also extends to publicly available datasets and is not primarily linked to the lack of reproducibility of such analyses (which would be a perfectly rational argument!). It is neither due to physicists or mathematicians running those, instead of quantitative sociologists! The roots of the mistrust are rather to be found in an anti-scientism that has been growing in the past decades, in a paradox of an equally growing technological society fuelled by scientific advances. Hence, calling for a governmental office of big data or some similar institution is very much unlikely to solve the issue. I do not know what could, actually, but continuing to develop better statistical methodology cannot hurt!

major confUSion

In a recent evening talk-show on France Inter, the French national public radio, the debate was about the [bad] surprise election of the donald and the fact that the media had missed the result, (self-)blaming a disconnection with the “real” country. One of the discussants, Julia Cagé, Professor of Economics at Science Po’, started the discussion with the amazing confusion [at 5’55”] between the probability that Hillary Clinton would win [evaluated at 84% on the last day] and the percentage of votes in her favour [which was around that figure in Manhattan]…

On a related if minor theme, my post on Flaxman et al.’s early [if preliminary] analysis of the said election got so many views that it became the most popular post for 2016! (If not competing with Ross Ihaka’s call to simply start over with R!)

And yet another related entry today in Libération, blaming the disastrous result partly on the social media and their algorithms (again!) that favour items of information (or dis-information) from the same perspective and do not rank those items by their reliability… The author of the tribune is an econometrician at Essec, but there is no methodological content in this ideological entry that seems to call for a super-monitor which would impose (how?) diversity and (which?) ranking on social media. A post-truth era, for sure! Shifting the blame from the deplorable voters themselves to anything else…