It’s the selection’s fault not the p-values’… [seminar]
Yoav Benjamini will give a seminar talk in Paris next Monday on the above (full title: “The replicability crisis in science: It’s the selection’s fault not the p-values’“). (That I will miss for being in Warwick at the time.) With a fairly terse abstract:
I shall discuss the problem of lack of replicability of results in science, and point at selective inference as a statistical root cause. I shall then present a few strategies for addressing selective inference, and their application in genomics, brain research and earlier phases of clinical trials where both primary and secondary endpoints are being used.
Details: February 8, 2016, 16h, Université Pierre & Marie Curie, campus Jussieu, salle 15-16-101.
Xi'an's Og 
February 8, 2016 at 7:49 pm
I looked at some of their work presented at JSM 2014 and exchanged some emails.
Some interesting points are raised but they seem to want to dichotomise replication into present or absent by each study study and estimate the number of studies where it was present (as just more than one or not).
From a recent paper “In this work we suggested enhancing the systematic reviews meta-analyses, …, with a measure that quantifies the strength of replicability,i.e., the r-value. In the reporting, if the r-value is small we have evidence that the conclusion is based on more than one study, i.e., that the effect was replicated across studies.” http://arxiv.org/pdf/1502.00088v2.pdf
To me, replication of studies just means the prior probabilities (from a common prior) were moved in similar directions to a posterior just based on each study, i.e. Posterior.i / Prior.0. Most definitely not a present or absent thing to be counted or estimated.
Keith O’Rourke
February 5, 2016 at 2:33 pm
I saw a version of this talk a while back in Lofoten. It was very interesting. It’s similar ground to Gelman’s “garden of forking paths”, but coming from a very different angle.
February 5, 2016 at 3:24 pm
This is 20 years old in Genetics, and still the most frequent trap in the field:
http://genetics.org/content/165/4/2259