10 Little’s simple ideas

“I still feel that too much of academic statistics values complex mathematics over elegant simplicity — it is necessary for a research paper to be complicated in order to be published.” Roderick Little, JASA, p.359

Roderick Little wrote his Fisher lecture, recently published in JASA, around ten simple ideas for statistics. Its title is “In praise of simplicity not mathematistry! Ten simple powerful ideas for the statistical scientist”. While this title is rather antagonistic, blaming mathematical statistics for the rise of mathematistry in the field (a term borrowed from Fisher, who also invented the adjective ‘Bayesian’), the paper focus on those 10 ideas and very little on why there is (would be) too much mathematics in statistics:

  1. Make outcomes univariate
  2. Bayes rule, for inference under an assumed model
  3. Calibrated Bayes, to keep inference honest
  4. Embrace well-designed simulation experiments
  5. Distinguish the model/estimand, the principles of estimation, and computational methods
  6. Parsimony — seek a good simple model, not the “right” model
  7. Model the Inclusion/Assignment and try to make it ignorable
  8. Consider dropping parts of the likelihood to reduce the modeling part
  9. Potential outcomes and principal stratification for causal inferenc
  10. Statistics is basically a missing data problem

“The mathematics of problems with infinite parameters is interesting, but with finite sample sizes, I would rather have a parametric model. “Mathematistry” may eschew parametric models because the asymptotic theory is too simple, but they often work well in practice.” Roderick Little, JASA, p.365

Both those rules and the illustrations that abund in the paper are reflecting upon Little’s research focus and obviously apply to his model in a fairly coherent way. However, while a mostly parametric model user myself, I fear the rejection of non-parametric techniques is far too radical. It is more and more my convinction that we cannot handle the full complexity of a realistic structure in a standard Bayesian manner and that we have to give up on the coherence and completeness goals at some point… Using non-parametrics and/or machine learning on some bits and pieces then makes sense, even though it hurts elegance and simplicity.

“However, fully Bayes inference requires detailed probability modeling, which is often a daunting task. It seems worth sacrifycing some Bayesian inferential purity if the task can be simplified.” Roderick Little, JASA, p.366

I will not discuss those ideas in detail, as some of them make complete sense to me (like Bayesian statistics laying its assumptions in the open) and others remain obscure (e.g., causality) or with limited applicability. It is overall a commendable Fisher lecture that focus on methodology and the practice of statistical science, rather than on theory. I however do not see the reason why maths should be blamed for this state of the field. Nor why mathematical statistics journals like AoS would carry some responsibility in the lack of further applicability in other fields.  Students of statistics do need a strong background in mathematics and I fear we are losing ground in this respect, at least judging by the growing difficulty in finding measure theory courses abroad for our exchange undergradutes from Paris-Dauphine. (I also find the model misspecification aspects mostly missing from this list.)

6 Responses to “10 Little’s simple ideas”

  1. Rod Little Says:

    Thanks for your comments on my paper. BTW as my paper notes George Box coined the term “mathematistry” not Fisher. Also I am not “blaming” mathematical statistics for anything, it is clearly indispensible for our subject, I just feel simplicity is a dimension that is often not accorded sufficient respect. Rod Little

  2. Dan Simpson Says:

    I’m also most probably wrong about all of this.

    The flip side is “how do you judge utility”?

    Well, let’s imagine someone proved something that required an MCMC-type scheme with asymptotic MSE of O(n^{3/4}). I would say that’s useless. SUch things don’t exist etc etc.

    But indications are I’m wrong. People like Ian Sloan and Francis Kuo etc have made dimension independent, unbiased randomised QMC schemes that have MSE O(n^{1-delta}), which suggests that it could be possible to construct posterior sampling schemes with a similar error rate. (Also – that pure Monte Carlo is finally dead!)

    So, I don’t know.

    • I am back from the IMACS conference on Monte Carlo methods and the QMC people do achieve this O(n^{1-ε}) rate. Maybe not immensely practical for our problems though as the coefficient in front of the rate may be humongous… But this made me ponder whether we should not have QMC steps in our codes at some point or another. Thanks for the discussion, I am playing devil’s advocate about utility as I also think I/we should get more involved at the data level…

  3. Dan Simpson Says:

    Maybe a reasonable question is whether the average AoS article has any impact in statistics (as the science of data). I don’t honestly know the answer, but I have my biases and the experience that my least “useful” and most “mathematical” work coincides quite nicely. But if AoS does not have, on average, an impact outside on (or within or outside of) practical statistics, maybe we should think of re-branding. (Is AoS a “worthwhile” journal if its only reach is theoreticians?)

    I think that we need to work to prove that our community’s love of maths mirrors the needs of the discipline lest we find ourselves faced with another “machine learning”-shaped debacle. (Ie a situation where a new field springs up around questions that we should be answering)

    To Mis-quote David Cox (at a future of mathematical statistics session at the last ISI), the future of mathematical statistics is to stop thinking of mathematical statistics as an actual field. (NB: my memory of what he said possilble reflects me more than him)

    • Hmmm, you seem to be falling into the utilitarian fallacy: don’t publish until you prove (or even better a scientist from a field applying statistics) that it is useful (or already used). I do not think papers published in AoS have to be stamped “useful” before being accepted, no more than papers published in Annals of Probability or Annals of Mathematics… The field has grown considerably in the past decenies and allows for a spectrum that ranges from mathematics to computer science. While I agree that we have to keep and nurture connections with machine learning at “all” costs, this does not mean banishing mathematical statistics to one of the circles of Dante’s Hell…

      • Dan Simpson Says:

        I’m not sure i meant to say that things should not be published until proven useful. By that metric I would have no papers! But I do think that mathematical statisticians (and journals like AoS) should be concerned with applicability.

        I actually think that it’s the connection to data and practical problem classes that should be watched and nurtured. (ML is an example of this, but not the only one) That doesn’t mean that I think everyone should be doing data-driven research, but “data-driven”-driven research may not be a bad aim.

        There are only a (relatively) small number of people who can do these things and, as you said, statistics is an enormous field, so surely they should be deployed in the most “useful” manner. (I think the way people have latched onto ABC is a successful example of this)

        Maybe the response to his lecture is not to take the maths out, but to “double down” and start blurring the lines between the vanguard of stats as a data science and that of stats as a mathematical discipline.

        And, incidentally, if you want to watch a smaller version of the machine learning thing happen, wander over to inverse problems and uncertainty quantification. Massive models (cf big data) beyond, but not far beyond, the classical GLM-type framework. We should be on top of this as a community, but it’s actually the applied and comp maths people who are pushing it ahead (mostly). They have their own journals, conferences etc.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 667 other followers