Ah. Thanks!

]]>I think I’m of the “condition on X until it doesn’t work and then panic” school….

]]>linear independence and near colinearity:I had not though of this problem indeed. In the paper, we pick the column indices at random. This is, I think, related with the overall debate as to whether or not we should condition on X (as opposed to modelling X as well). I am of the “condition on X” school.

]]>Training data: this is a sample with the smallest possible size so that the posterior is proper. The posterior associated with the reference prior associated with model i. Which usually is improper. If you look at the four steps on page 5, each sounds clear enough to me. Mind that the reference prior is an objective Bayes prior associated with the model $M_i$ per se, not the reference prior we are seeking. Maybe this explains for your confusion…

]]>Uh?! A prior that gives “bad” results?! Whazat?!

We define “objective” [testing] priors as a result of an information minimisation goal. The principle was laid by Pérez & Berger (2002) and we follow it in this less manageable setting of binomial regression.. I kind of like it for the reason that it allows for the ‘improper prior sin’ in testing, offering a way out or rather a way in for improper priors. The implementation issue is not part of this question.

Now, I agree with you [?] that we could have conducted experiments where we knew the “truth” and had the possibility of finding the error rate of a model selection principle based on integral priors. A nice proposal for a summer project.

]]>How far from a “bad” prior (aka a prior that gives bad results) are the integral priors?

And, to answer my own question, I think that they’re quite far away, in the sense that you’re solving a well-posed problem (an integral equation of the second kind) to get the prior, so a “nearby” prior should be the solution of a “nearby” integral equation.

]]>It’s a little different in this case. It is (moderately well) understood what MCMC does for posterior inference, but here you’re propagating this through another layer of inference machinery. So I guess it’s worth thinking about.

It’s probably just an awkward way of framing a “prior sensitivity” question. The prior that you’re actually using is a perturbation of a theoretically motivated prior, so it’s worth checking how good/bad that is.

]]>Interesting suggestion about the numerical or even analytical resolution: I would think the answer is highly dependent on the model, although it may be that Beta/Bernoulli models can be handled rather easily… I am less sure I get the remark about MC(MC) error vs. numerical approximation error: it seems to apply every time MCMC is used!

]]>A different thing: drawing linearly independent columns isn’t, to my knowlege, trivial, especially in the big data context. Isn’t that part of why g-priors exist? (The X^T X but deals with the approximate colinearity) is there a similar trick here? I imagine drawing independent but almost colinear columns would be a bad thing…

]]>Then I’d probably stick the resulting approximate priors into INLA (but that’s personal preference :p)

Did you look at how the MC error upsets the balance ? (i.e. are the priors still neutral ) Because 10k chains will (if you’re lucky) give you 1 significant figure (maaaaybe 2).

(NB – I’ve only read the start and the end… Apologies if this was addressed in the middle (pp 5-10)… I’m getting to it presently)

A more general question: is this the sort of things scientists want? As opposed to designing objective priors on the whole of 2^X (X=set of covariates) and then leaping around the model space with gay abandon? Or is it more common/practical/useful to test given groupwise in/out hypotheses?

]]>