human data

Due to a mishap in the handling of my Nature subscription, I received six issues at once when returning from vacations… Among which the 8 July issue with a special focus on computational social science, including five “perspective” articles on this topic.

“The need to blend expertise in the social sciences with the skills required to collect, clean and analyse large data sets means that computational social science requires teams of researchers who can field a remarkably diverse set of expertise and skills”

These perspective articles are however rather bland and mostly generic, beyond the feel-good call to protect privacy, counter biases, acknowledge the observational nature of most of the available data and the behavioural impact of the many algorithms interacting with us. The only formula appearing in the collection is the linear regression prediction ŷ=βx, which offers four different interpretations whether or no there are interventions, and if the fovus is on explanation or prediction (referring to Breiman’s two cultures in passing).

“Using computers to analyse large data sets dates back to the earliest mainframe computers — and has been central to the work of actuaries and national statistics offices, both of which have long been important resources for studies of society and people.”

I am also very chagrined by the almost complete absence of references to statistics, the sole article clearly mentioning the term actually involving statistical physics! With a reference to Ising and Potts models to analyse social networks. While the one on algorithmically infused societies sees statistical and machine learning models as potentially harmful. Or as being misleadingly explanatory. But no discussion of the statistical meaning of model assessment and of multiscale model building. (To be fair, the references list all include links to statistics papers.)

random forests [reading group]

Here are the slides I prepared (and recycled) over the weekend for the reading group on machine-learning that recently started in Warwick. Where I am for two consecutive weeks.

years (and years) of data science

In preparation for the round table at the start of the MCMSkv conference, this afternoon, Anto sent us a paper written by David Donoho for the Tukey Centennial workshop, held in Princeton last September. Entitled 50 years of Data Science. And which attracted a whole round of comments, judging from the Google search results. So much that I decided not to read any of them before parsing through the paper. But almost certainly reproducing here with my two cents some of the previous comments.

“John Tukey’s definition of `Big Data’ was `anything that won’t fit on one device’.”

The complaint that data science is essentially statistics that does not dare to spell out statistics as if it were a ten letter word (p.5) is not new, if appropriate. In this paper, David Donoho evacuates the memes that supposedly separate data science from statistics, like “big data” (although I doubt non-statisticians would accept the quick rejection that easily, wondering at the ability of statisticians to develop big models), skills like parallel programming (which ineluctably leads to more rudimentary algorithms and inferential techniques), jobs requiring such a vast array of skills and experience that no graduate student sounds properly trained for it…

“A call to action, from a statistician who fells `the train is leaving the station’.” (p.12)

One point of the paper is to see 1962 John Tukey’s “The Future of Data Analysis” as prophetical of the “Big Data” and “Data Science” crises. Which makes a lot of sense when considering the four driving forces advanced by Tukey (p.11):

1. formal statistics
2. advanced computing and graphical devices
3. the ability to face ever-growing data flows
4. its adoption by an ever-wider range of fields

“Science about data science will grow dramatically in significance.”

David Donoho then moves on to incorporate   Leo Breiman’s 2001 Two Cultures paper. Which separates machine learning and prediction from statistics and inference, leading to the “big chasm”! And he sees the combination of prediction with “common task framework” as the “secret sauce” of machine learning, because of the possibility of objective comparison of methods on a testing dataset. Which does not seem to me as the explanation for the current (real or perceived) disaffection for statistics and correlated attraction for more computer-related solutions. A code that wins a Kaggle challenge clearly has some efficient characteristics, but this tells me nothing of the abilities of the methodology behind that code. If any. Self-learning how to play chess within 72 hours is great, but is the principle behind able to handle go at the same level?  Plus, I remain worried about the (screaming) absence of model (or models) in predictive approaches. Or at least skeptical. For the same reason it does not help in producing a generic approach to problems. Nor an approximation to the underlying mechanism. I thus see nothing but a black box in many “predictive models”, which tells me nothing about the uncertainty, imprecision or reproducibility of such tools. “Tool evaluation” cannot be reduced to a final score on a testing benchmark. The paper concludes with the prediction that the validation of scientific methodology will solely be empirical (p.37). This leaves little ground if any for probability and uncertainty quantification, as reflected their absence in the paper.