art brut

snapshot from Crès, Croatia (#2)

on noninformative priors

A few weeks ago, Larry Wasserman posted on Normal Deviate an entry on noninformative priors as a lost cause for statistics. I first reacted rather angrily to this post, then decided against posting my reply. After a relaxing week in Budapest, and the prospect of the incoming summer break, I went back to the post and edited it towards more constructive  goals… The post also got discussed by Andrew and Entsophy, generating in each case a heap of heated discussions. (Enjoy your summer, winter is coming!)

Although Larry wrote he wanted to restrain from only posting on Bayesian statistics, he does seem attracted to them like a moth to a candle… This time, it is about the “lost cause of noninformative priors”. While Larry is 200% entitled to post about whatever he likes or dislikes, the post does not really bring new fuel to the debate, if debate there is. First, I think everyone agrees that there is no such thing as a noninformative prior or a prior representing ignorance. (To quote from Jeffreys: “A prior probability used to express ignorance is merely the formal statement of ignorance” (ToP, VIII, x8.1). Every prior brings something into the game and this is reflected in the posterior inference. Sometimes, the impact is enormous and we may be unaware of it. Take for instance Bayesian nonparametrics. It is thus essential to keep this in mind. (And to keep calm!) Which does not mean we should not use them. Indeed, noninformative priors are a way of setting a reference measure, from which one can start evaluating the impact of picking this or that prior. Just a measure. (No-one gets emotional when hearing the Lebesgue measure mentioned, right?!) And if the reference prior is a σ-finite measure, one cannot even put a meaning to events like θ>0. This reference measure is required to set the Bayesian crank turning, here or there depending on one’s prior beliefs or information. If we reject those reference priors for accepting only the cases when the prior is provided along with the data and the model, I think everyone is a Bayesian. Even Feller. Even Larry (?).

Second, there is alas too much pathos or unintended meaning put in names like noninformative, ignorance, objective, &tc. And this may be the major message in Larry’s post. We should call those reference priors Linear A priors in reference to the mostly undeciphered Minoan alphabet. Or whatever name with no emotional content whatsoever in order not to drive people crazy. Noninformative is not even a word, to start with… And I dunno how to define ignorance in a mathematical manner.Once more in connection with the EMS 2013 meeting in Budapest, I do not see why one should object more to reference priors than to the so-called “subjective” priors, as the former provide a baseline against which to test the latter, using e.g. Xiao Li’s approach. I am actually much more annoyed by the use of a specific proper prior in a statistical analysis when this prior is neither justified nor assessed in terms of robustness. And I see nothing wrong in establishing either asymptotic or frequentist properties about some procedures connected with some of those reference priors: I became a Bayesian this way, after all.

Anyway, have a nice (end of the) summer if you are in the Northern Hemisphere, and expect delays (or snapshots!) on the ‘Og for the coming fortnight…

Le Monde puzzle [#825]

The current puzzle is the last one before the summer break and not exciting enough to take along:

Take the first ten digits, create five pairs out of those, and for each pair (x,y) derive the quantity (min(x,y)+1.5max(x,y)). What is the collection of pairs that maximises the product of those quantities?

I wrote the following R code in the train back from Annecy:

res=0
bonus=1.5
for (t in 1:1000){
x=matrix(sample(1:10),ncol=2)
mix=apply(x,1,min);mox=apply(x,1,max)
rex=prod(mix+bonus*mox)
if (rex>res){
  print(x)
  res=rex}
}

and got:

     [,1] [,2]
[1,]    4    7
[2,]    5    6
[3,]    8    3
[4,]    2    9
[5,]    1   10

Thus the sum of the pairs is always equal to 11. And by changing bonus you can check this is true for all values of the coefficient that are larger than one (one included).

one of those coincidences…

I was at a small faculty meeting today at Paris-Dauphine and introduced myself to my neighbour who shook hands with me and told me that last time he saw me it was in Agra! As it transpired, this colleague from the Law School, whom I had never met before, was visiting Agra at the same time as we were, stayed in the same hotel, and saw my Paris-Dauphine sweat-shirt at dinner (he was presumably wearing more neutral clothes)! Given the number of tourists visiting Agra at this time of the year, it is not such an immense coincidence that I met someone from Paris-Dauphine (after all, my other neighbour had also visited Agra a little while ago), but it still is a remarkable one that we ended up in the same hotel and that I chose to wear my Paris-Dauphine sweat-shirt that night…