## Archive for testing of hypotheses

## at last the type IX error

Posted in Statistics with tags comics, false discovery rate, Neyman-Pearson tests, significance test, testing of hypotheses, Type I error, Type II error, xkcd on May 11, 2020 by xi'an## severe testing or severe sabotage? [not a book review]

Posted in Books, pictures, Statistics, University life with tags Cambridge University Press, commercial editor, cup, Deborah Mayo, philosophy of sciences, print on demand, severe testing, statistical inference, statistics wars, testing of hypotheses on October 16, 2018 by xi'an**L**ast week, I received this new book of Deborah Mayo, which I was looking forward reading and annotating!, but thrice alas, the book had been sabotaged: except for the preface and acknowledgements, the entire book is printed upside down [a minor issue since the entire book is concerned] and with some part of the text cut on each side [a few letters each time but enough to make reading a chore!]. I am thus waiting for a tested copy of the book to start reading it in earnest!

## relativity is the keyword

Posted in Books, Statistics, University life with tags Bayes factor, model posterior probabilities, OxWaSP, relativity, Saint Giles cemetery, testing of hypotheses, The Bayesian Choice, University of Oxford on February 1, 2017 by xi'an**A**s I was teaching my introduction to Bayesian Statistics this morning, ending up with the chapter on tests of hypotheses, I found reflecting [out loud] on the relative nature of posterior quantities. Just like when I introduced the role of priors in Bayesian analysis the day before, I stressed the relativity of quantities coming out of the BBB [Big Bayesian Black Box], namely that whatever happens as a Bayesian procedure is to be understood, scaled, and relativised against the prior equivalent, i.e., that the reference measure or gauge is the prior. This is sort of obvious, clearly, but bringing the argument forward from the start avoids all sorts of misunderstanding and disagreement, in that it excludes the claims of absolute and certainty that may come with the production of a posterior distribution. It also removes the endless debate about the determination of *the* prior, by making *each* prior a reference on its own. With an additional possibility of calibration by simulation under the assumed model. Or an alternative. Again nothing new there, but I got rather excited by this presentation choice, as it seems to clarify the path to Bayesian modelling and avoid misapprehensions.

Further, the curious case of the Bayes factor (or of the posterior probability) could possibly be resolved most satisfactorily in this framework, as the [dreaded] dependence on the model prior probabilities then becomes a matter of relativity! Those posterior probabilities depend directly and almost linearly on the prior probabilities, but they should not be interpreted in an *absolute* sense as the ultimate and unique probability of the hypothesis (which anyway does not mean anything in terms of the observed experiment). In other words, this posterior probability does not need to be scaled against a U(0,1) distribution. Or against the *p*-value if anyone wishes to do so. By the end of the lecture, I was even wondering [not so loudly] whether or not this perspective was allowing for a resolution of the Lindley-Jeffreys paradox, as the resulting number could be set relative to the choice of the [arbitrary] normalising constant. Continue reading

## not an ASA’s statement on p-values

Posted in Books, Kids, Statistics, University life with tags ASA, p-values, statistical significance, testing of hypotheses, Vladimir Vovk on March 18, 2016 by xi'an

**T**his may be a coincidence, but a few days after the ASA statement got published, Yuri Gurevich and Vladimir Vovk arXived a note on the Fundamentals of p-values. Which actually does not contribute to the debate. The paper is written in a Q&A manner. And defines a sort of peculiar logic related with [some] p-values. A second and more general paper is in the making, which may shed more light on the potential appeal of this formalism…