## Statistical evidence for revised standards

**I**n yet another permutation of the original title (!), Andrew Gelman posted the answer Val Johnson sent him after our (submitted) letter to PNAS. As Val did not send me a copy (although Andrew did!), I will not reproduce it here and I rather refer the interested readers to Andrews’ blog… In addition to Andrew’s (sensible) points, here are a few idle (post-X’mas and pre-skiing) reflections:

makes me wonder in which metric this exponential rate (in γ?) occurs;*“evidence against a false null hypothesis accrues exponentially fast”*- that
is difficult to accept as an argument since there is no trace of a decision-theoretic argument in the whole paper;*“most decision-theoretic analyses of the optimal threshold to use for declaring a significant finding would lead to evidence thresholds that are substantially greater than 5 (and probably also greater 25)”* - Val rejects our minimaxity argument on the basis that
but the prior that corresponds to those tests is minimising the integrated probability of not rejecting at threshold level γ, a loss function integrated against parameter and observation, a Bayes risk in other words… Point masses or spike priors are clearly characteristics of minimax priors. Furthermore, the additional argument that*“[UMPBTs] do not involve minimization of maximum loss”*has been used by many to refute the Bayesian perspective and makes me wonder what are the arguments left in using a (pseudo-)Bayesian approach;*“in most applications, however, a unique loss function/prior distribution combination does not exist”* - the next paragraph is pure tautology: the fact that
is a paraphrase of the definition of UMPBTs, not an argument. I do not see we should solely*“no other test, based on either a subjectively or objectively specified alternative hypothesis, is as likely to produce a Bayes factor that exceeds the specified evidence threshold”*, since minimising those should lead to a point mass on the null (or, more seriously, should not lead to the minimax-like selection of the prior under the alternative).*“worry about false negatives”*

March 15, 2014 at 10:46 am

[…] Further comments (which I agree with) here from […]