AI and Statistics 2014

Today, I am leaving Paris for a 8 day stay in Iceland! This is quite exciting, for many reasons: first, I missed the AISTATS 2013 last year as I was still in the hospital;  second, I am giving a short short tutorial on ABC methods which will be more like a long (two hours)  talk; third, it gives me the fantastic opportunity to visit Iceland for a few days, a place that was top of my wish list of countries to visit. The weather forecast is rather bleak but I am carrying enough waterproof layers to withstand a wee bit of snow and rain… The conference proper starts next Tuesday, April 22, with the tutorials taking place next Friday, April 25. Hence leaving me three completely free days for exploring the area near Reykjavik.

art brut

snapshot from Crès, Croatia (#2)

on noninformative priors

A few weeks ago, Larry Wasserman posted on Normal Deviate an entry on noninformative priors as a lost cause for statistics. I first reacted rather angrily to this post, then decided against posting my reply. After a relaxing week in Budapest, and the prospect of the incoming summer break, I went back to the post and edited it towards more constructive  goals… The post also got discussed by Andrew and Entsophy, generating in each case a heap of heated discussions. (Enjoy your summer, winter is coming!)

Although Larry wrote he wanted to restrain from only posting on Bayesian statistics, he does seem attracted to them like a moth to a candle… This time, it is about the “lost cause of noninformative priors”. While Larry is 200% entitled to post about whatever he likes or dislikes, the post does not really bring new fuel to the debate, if debate there is. First, I think everyone agrees that there is no such thing as a noninformative prior or a prior representing ignorance. (To quote from Jeffreys: “A prior probability used to express ignorance is merely the formal statement of ignorance” (ToP, VIII, x8.1). Every prior brings something into the game and this is reflected in the posterior inference. Sometimes, the impact is enormous and we may be unaware of it. Take for instance Bayesian nonparametrics. It is thus essential to keep this in mind. (And to keep calm!) Which does not mean we should not use them. Indeed, noninformative priors are a way of setting a reference measure, from which one can start evaluating the impact of picking this or that prior. Just a measure. (No-one gets emotional when hearing the Lebesgue measure mentioned, right?!) And if the reference prior is a σ-finite measure, one cannot even put a meaning to events like θ>0. This reference measure is required to set the Bayesian crank turning, here or there depending on one’s prior beliefs or information. If we reject those reference priors for accepting only the cases when the prior is provided along with the data and the model, I think everyone is a Bayesian. Even Feller. Even Larry (?).

Second, there is alas too much pathos or unintended meaning put in names like noninformative, ignorance, objective, &tc. And this may be the major message in Larry’s post. We should call those reference priors Linear A priors in reference to the mostly undeciphered Minoan alphabet. Or whatever name with no emotional content whatsoever in order not to drive people crazy. Noninformative is not even a word, to start with… And I dunno how to define ignorance in a mathematical manner.Once more in connection with the EMS 2013 meeting in Budapest, I do not see why one should object more to reference priors than to the so-called “subjective” priors, as the former provide a baseline against which to test the latter, using e.g. Xiao Li’s approach. I am actually much more annoyed by the use of a specific proper prior in a statistical analysis when this prior is neither justified nor assessed in terms of robustness. And I see nothing wrong in establishing either asymptotic or frequentist properties about some procedures connected with some of those reference priors: I became a Bayesian this way, after all.

Anyway, have a nice (end of the) summer if you are in the Northern Hemisphere, and expect delays (or snapshots!) on the ‘Og for the coming fortnight…

Le Monde puzzle [#825]

The current puzzle is the last one before the summer break and not exciting enough to take along:

Take the first ten digits, create five pairs out of those, and for each pair (x,y) derive the quantity (min(x,y)+1.5max(x,y)). What is the collection of pairs that maximises the product of those quantities?

I wrote the following R code in the train back from Annecy:

res=0
bonus=1.5
for (t in 1:1000){
x=matrix(sample(1:10),ncol=2)
mix=apply(x,1,min);mox=apply(x,1,max)
rex=prod(mix+bonus*mox)
if (rex>res){
  print(x)
  res=rex}
}

and got:

     [,1] [,2]
[1,]    4    7
[2,]    5    6
[3,]    8    3
[4,]    2    9
[5,]    1   10

Thus the sum of the pairs is always equal to 11. And by changing bonus you can check this is true for all values of the coefficient that are larger than one (one included).