## eliminating an important obstacle to creative thinking: statistics…

Posted in Books, Kids, Statistics, University life with tags , , , , , , , , , , , , , on March 12, 2015 by xi'an

“We hope and anticipate that banning the NHSTP will have the effect of increasing the quality of submitted manuscripts by liberating authors from the stultified structure of NHSTP thinking thereby eliminating an important obstacle to creative thinking.”

About a month ago, David Trafimow and Michael Marks, the current editors of the journal Basic and Applied Social Psychology published an editorial banning all null hypothesis significance testing procedures (acronym-ed into the ugly NHSTP which sounds like a particularly nasty venereal disease!) from papers published by the journal. My first reaction was “Great! This will bring more substance to the papers by preventing significance fishing and undisclosed multiple testing! Power to the statisticians!” However, after reading the said editorial, I realised it was inspired by a nihilistic anti-statistical stance, backed by an apparent lack of understanding of the nature of statistical inference, rather than a call for saner and safer statistical practice. The editors most clearly state that inferential statistical procedures are no longer needed to publish in the journal, only “strong descriptive statistics”. Maybe to keep in tune with the “Basic” in the name of the journal!

“In the NHSTP, the problem is in traversing the distance from the probability of the finding, given the null hypothesis, to the probability of the null hypothesis, given the finding. Regarding confidence intervals, the problem is that, for example, a 95% confidence interval does not indicate that the parameter of interest has a 95% probability of being within the interval.”

The above quote could be a motivation for a Bayesian approach to the testing problem, a revolutionary stance for journal editors!, but it only illustrate that the editors wish for a procedure that would eliminate the uncertainty inherent to statistical inference, i.e., to decision making under… erm, uncertainty: “The state of the art remains uncertain.” To fail to separate significance from certainty is fairly appalling from an epistemological perspective and should be a case for impeachment, were any such thing to exist for a journal board. This means the editors cannot distinguish data from parameter and model from reality! Even more fundamentally, to bar statistical procedures from being used in a scientific study is nothing short of reactionary. While encouraging the inclusion of data is a step forward, restricting the validation or in-validation of hypotheses to gazing at descriptive statistics is many steps backward and does completely jeopardize the academic reputation of the journal, which editorial may end up being the last quoted paper. Is deconstruction now reaching psychology journals?! To quote from a critic of this approach, “Thus, the general weaknesses of the deconstructive enterprise become self-justifying. With such an approach I am indeed not sympathetic.” (Searle, 1983).

“The usual problem with Bayesian procedures is that they depend on some sort of Laplacian assumption to generate numbers where none exist (…) With respect to Bayesian procedures, we reserve the right to make case-by-case judgments, and thus Bayesian procedures are neither required nor banned from BASP.”

The section of Bayesian approaches is trying to be sympathetic to the Bayesian paradigm but again reflects upon the poor understanding of the authors. By “Laplacian assumption”, they mean Laplace´s Principle of Indifference, i.e., the use of uniform priors, which is not seriously considered as a sound principle since the mid-1930’s. Except maybe in recent papers of Trafimow. I also love the notion of “generat[ing] numbers when none exist”, as if the prior distribution had to be grounded in some physical reality! Although it is meaningless, it has some poetic value… (Plus, bringing Popper and Fisher to the rescue sounds like shooting Bayes himself in the foot.)  At least, the fact that the editors will consider Bayesian papers in a case-by-case basis indicate they may engage in a subjective Bayesian analysis of each paper rather than using an automated p-value against the 100% rejection bound!

[Note: this entry was suggested by Alexandra Schmidt, current ISBA President, towards an incoming column on this decision of Basic and Applied Social Psychology for the ISBA Bulletin.]

## re-re-relevant statistics for ABC model choice

Posted in Books, Statistics, University life with tags , , , , , , on March 18, 2013 by xi'an

After a very, very long delay, we eventually re-revised our paper about necessary and sufficient conditions on summary statistics to be relevant for model choice (i.e. to lead to consistent tests). Reasons, both good and bad, abound for this delay! Some (rather bad) were driven by the completion of a certain new edition… Some (fairly good) are connected with the requests from the Series B editorial team, towards improving our methodological input.  As a result we put more emphasis on the post-ABC cross-checking for the relevance of the summary choice, via a predictive posterior evaluation of the means of the summary statistic under both models and a test for mean equality. And re-ran a series of experiments on a three population population genetic example. Plus, on the side, simplified some of our assumptions. I dearly hope the paper can make it through but am also looking forward the opinion of the Series B editorial team  The next version of Relevant statistics for Bayesian model choice should be arXived by now (meaning when this post appears!).

## appliBUGS (wet)

Posted in Statistics, University life with tags , , , , , , , , , on December 27, 2012 by xi'an

This morning I gave my talk on ABC; computation or inference? at the appliBUGS seminar. Here, in Paris, BUGS stands for Bayesian United Group of Statisticians! Presumably in connection with a strong football culture, since the talk after mine was Jean-Louis Foulley’s ranking of the Euro 2012 teams. Quite an interesting talk (even though I am not particularly interested in football and even though I dozed a little, steaming out the downpour I had received on my bike-ride there…) I am also sorry I missed the next talk by Jean-Louis on Galton’s quincunx. (Unfortunately, his slides are not [yet?] on-line.)

As a coincidence, after launching a BayesComp page on Google+ (as an aside, I am quite nonplussed by the purpose of Google-), Nicolas Chopin also just started a Bayes in Paris webpage, in connection with our informal seminar/reading group at CREST. With the appropriate picture this time, i.e. a street plaque remembering…Laplace! May I suggest the RER stop Laplace and his statue in the Paris observatory as additional illustrations for the other pages…

## Bayes by the Bay

Posted in Books, Statistics, Travel, University life with tags , , , , , , , , , , , , , on August 28, 2012 by xi'an

No, no, this is not an announcement for a meeting on an Australian beach (which is Bayes on the Beach, taking place next November (6-8) on the Sunshine Coast and is organised by Kerrie Mengersen’s BRAG, at QUT, that I just left! With Robert Wolpert as the international keynote speaker and Matt Wand as the Australian keynote speaker.) Bayes by the Bay is “a pedagogical workshop on Bayesian methods in Science” organised by the Institute of Mathematical Sciences, based in the CIT campus in Chennai. It is taking place on January 4-8, 2013, in Pondichéry. (To use the French spelling of this former comptoir of French India…) Just prior to the ISBA Varanasi meeting on Bayesian Statistics.

Great: the webpage for the workshop uses the attached picture of Pierre-Simon (de) Laplace, rather than the unlikely picture of Thomas Bayes found all over the place (incl. this blog!). This was also the case in Christensen et al.’s Bayesian ideas and data analysis. So maybe there is a trend there. I also like the name “Bayes by the Bay“, as it reminds me of a kid song we used to sing to/with our kids when they were young, “down by the bay“, after a summer vacation with Anne and George Casella…

Coincidentally, my re-read of Laplace’s Théorie Analytique des Probabilitiés just appeared (in English) in the Boletim ISBrA, the dynamic Brazilian branch of ISBA.

## why noninformative priors?

Posted in Books, Statistics, University life with tags , , , , on May 9, 2012 by xi'an

Answering a question around this theme on StackExchange, I wrote the following reply:

The debate about non-informative priors has been going on for ages, at least since the end of the 19th century with criticisms by Bertrand and de Morgan about the lack of invariance of Laplace’s uniform priors (the same criticism reported by Stéphane Laurent in the above comments). This lack of invariance sounded like a death stroke for the Bayesian approach and, while some Bayesians were desperately trying to cling to specific distributions, using less-than-formal arguments, others had a wider vision of a larger picture where priors could be used in situations where there was hardly any prior information, beyond the shape of the likelihood itself. (This was even before Abraham Wald established his admissibility and complete class results about Bayes procedures. And at about the same time as E.J.G. Pitman gave an “objective” derivation of the best invariant estimator as a Bayes estimator against the corresponding Haar measure…)

This vision is best represented by Jeffreys’ distributions, where the information matrix of the sampling model, $I(\theta)$, is turned into a prior distribution

$\pi(\theta) \propto |I(\theta)|^{1/2}$

which is most often improper, i.e. does not integrate to a finite value. The label “non-informative” associated with Jeffreys’ priors is rather unfortunate, as they represent an input from the statistician, hence are informative about something! Similarly, “objective” has an authoritative weight I dislike… I thus prefer the label “reference prior”, used for instance by José Bernado.

Those priors indeed give a reference against which one can compute either the reference estimator/test/prediction or one’s own estimator/test/prediction using a different prior motivated by subjective and objective items of information. To answer directly the question, “why not use only informative priors?”, there is actually no answer. A prior distribution is a choice made by the statistician, neither a state of Nature nor a hidden variable. In other words, there is no “best prior” that one “should use”. Because this is the nature of statistical inference that there is no “best answer”.

Hence my defence of the noninformative/reference choice! It is providing the same range of inferential tools as other priors, but gives answers that are only inspired by the shape of the likelihood function, rather than induced by some opinion about the range of the unknown parameters.

## Théorie analytique des probabilités

Posted in Books, Statistics, University life with tags , , , , , , on March 26, 2012 by xi'an

The Brazilian society for Bayesian Analysis (ISBrA, whose annual meeting is taking place at this very time!) asked me to write a review on Pierre Simon Laplace’s book, Théorie Analytique des Probabilités, a book that was initially published in 1812, exactly two centuries ago. I promptly accepted this request as (a) I had never looked at this book and so this provided me with a perfect opportunity to do so, (b) while in Vancouver, Julien Cornebise had bought for me a 1967 reproduction of the 1812 edition,  (c) I was curious to see how much of the book had permeated modern probability and statistics or, conversely, how much of Laplace’s perspective was still understandable by modern day standards. (Note that the link on the book leads to a free version of the 1814, not 1812, edition of the book, as free as the kindle version on amazon.)

Je m’attache surtout, à déterminer la probabilité des causes et des résultats indiqués par événemens considérés en grand nombre.” P.S. Laplace, Théorie Analytique des Probabilités, page 3

First, I must acknowledge I found the book rather difficult to read and this for several reasons: (a) as is the case for books from older times, the ratio text-to-formulae is very high, with an inconvenient typography and page layout (ar least for actual standards), so speed-reading is impossible; (b) the themes offered in succession are often abruptly brought and uncorrelated with the previous ones; (c) the mathematical notations are 18th-century, so sums are indicated by S, exponentials by c, and so on, which again slows down reading and understanding; (d) for all of the above reasons, I often missed the big picture and got mired into technical details until they made sense or I gave up; (e) I never quite understood whether or not Laplace was interested in the analytics like generating functions only to provide precise numerical approximations or for their own sake. Hence a form of disappointment by the end of the book, most likely due to my insufficient investment in the project (on which I mostly spent an Amsterdam/Calgary flight and jet-lagged nights at BIRS…), even though I got excited by finding the bits and pieces about Bayesian estimation and testing. Continue reading

## Bayesian ideas and data analysis

Posted in Books, R, Statistics, Travel, University life with tags , , , , , , , , , , , , , on October 31, 2011 by xi'an

Here is [yet!] another Bayesian textbook that appeared recently. I read it in the past few days and, despite my obvious biases and prejudices, I liked it very much! It has a lot in common (at least in spirit) with our Bayesian Core, which may explain why I feel so benevolent towards Bayesian ideas and data analysis. Just like ours, the book by Ron Christensen, Wes Johnson, Adam Branscum, and Timothy Hanson is indeed focused on explaining the Bayesian ideas through (real) examples and it covers a lot of regression models, all the way to non-parametrics. It contains a good proportion of WinBugs and R codes. It intermingles methodology and computational chapters in the first part, before moving to the serious business of analysing more and more complex regression models. Exercises appear throughout the text rather than at the end of the chapters. As the volume of their book is more important (over 500 pages), the authors spend more time on analysing various datasets for each chapter and, more importantly, provide a rather unique entry on prior assessment and construction. Especially in the regression chapters. The author index is rather original in that it links the authors with more than one entry to the topics they are connected with (Ron Christensen winning the game with the highest number of entries).  Continue reading