Incoherent phyleogeographic inference [reply]
“Logical overlap is the norm for the complex models analyzed with ABC, so many ABC posterior model probabilities published to date are wrong.” Alan R. Templeton, PNAS, doi:10.1073/pnas.1009012107
Our letter in PNAS about Templeton’s surprising diatribe on Bayesian inference is now appeared in the early edition, along with Templeton’s reply. This reply is unfortunately missing any novelty element compared with the original paper. First, he maintains that the critcism is about ABC (which is, in case you do not know, a computational technique and not a specific statistical methodology!). Second, he insists on the inappropriate Venn diagram analogy by reproducing the basic identity
(presumably in case we had lost sight of it!) to argue that using instead
is incoherent (hence rejecting Bayes factors, Bayesian model averaging and so on). I am not particularly surprised by this immutable stance, but it means that there is little point in debate when starting from such positions… Our main goal in publishing this letter was actually to stress that the earlier tribune had no statistical ground and I think we achieved this goal.
Xi'an's Og 
October 6, 2010 at 11:13 pm
Templeton is clearly simply sending out arguments like soldiers to fight for his cause, right or wrong. (Boo!) But there is something to be said for trying to “separate” models so they do not, in Templeton’s terms, “overlap logically”.
I’m thinking here of Johnson and Rossell’s non-local priors (J. R. Statist. Soc. B (2010), 72, Part 2, pp. 143–170). Their priors don’t change the consistency of Bayesian model selection, but do address a bizarre asymmetry that occurs when the prior under the alternative puts a lot of probability mass in the neighborhood of the null, which they term a “local prior”. The asymmetry is in the rates of accumulation of evidence for the true model when the null is true versus when the alternative is true. Under local priors evidence accumulates much more slowly for a true null, because the alternative looks a lot like it. Non-local priors move probability mass out of the neighborhood of the null, speeding up evidence accumulation for a true null.
September 30, 2010 at 3:57 am
Hi Christian, I’d like your explicit take on Templeton’s assertion that P(A)+P(B)+P(C) is incoherent in the case of nested models. In terms, for example, that a knowledgeable statistical layman might explain to grad students, for instance. I trust your interpretation, as the letter’s authors are about tops as far as I’m concerned, but I’d love to hear it explained in straightforward terms why Templeton is wrong here. I’m thinking about running a paper reading group on “The Nested Clade Wars”, and I’d like to be able to address this issue without worry of mis-statement.
September 30, 2010 at 7:31 am
As maybe explained in more details in my initial draft of our letter, the analogy with the Venn diagram and the overlapping models does not apply to model comparison. In a Bayesian framework, when several models are under comparison, the model index becomes one (unknown) parameter M and the parameter(s) of the model is defined conditionally on the value of M. Since M cannot take two values simultaneously, two models cannot “co-exist”. This is also why considering parameters that are “common” to all models does not make sense (even though I may use this sentence from time to time in justifying the call to a “common” prior on a variance parameter say for all models). Another reason for using waterproof separation between models is that the purpose for selecting a model is to…select a model and hence work within this model once the decision is made. Having the properties of other (rejected) models interfering with the inference on the chosen model is not coherent. (Great title for the reading group!)