sunrise

Posted in pictures, Statistics, Travel, University life with tags , , , , , on November 30, 2011 by xi'an

is the p-value a good measure of evidence?

Posted in Statistics, University life with tags , , , , , , , on November 30, 2011 by xi'an

Statistics abounds criteria for assessing quality of estimators, tests, forecasting rules, classification algorithms, but besides the likelihood principle discussions, it seems to be almost silent on what criteria should a good measure of evidence satisfy.” M. Grendár

A short note (4 pages) appeared on arXiv a few days ago, entitled “is the p-value a good measure of evidence? an asymptotic consistency criterion” by M. Grendár. It is rather puzzling in that it defines the consistency of an evidence measure ε(H1,H2,Xn) (for the hypothesis H1 relative to the alternative H2) by

$\lim_{n\rightarrow\infty} P(H_1|\epsilon(\neg H_1,H_2,X^n)\in S) =0$

where S is “the category of the most extreme values of the evidence measure (…) that corresponds to the strongest evidence” (p.2) and which is interpreted as “the probability [of the first hypothesis H1], given that the measure of evidence strongly testifies against H1, relative to H2 should go to zero” (p.2). So this definition requires a probability measure on the parameter  spaces or at least on the set of model indices, but it is not explicitly stated in the paper. The proofs that the p-value is inconsistent and that the likelihood ratio is consistent do involve model/hypothesis prior probabilities and weights, p(.) and w. However, the last section on the consistency of the Bayes factor states “it is open to debate whether a measure of evidence can depend on a prior information” (p.3) and it uses another notation, q(.), for the prior distribution…  Furthermore, it reproduces the argument found in Templeton that larger evidence should be attributed to larger hypotheses. And it misses our 1992 analysis of p-values from a decision-theoretic perspective, where we show they are inadmissible for two-sided tests, answering the question asked in the quote above.

most hated airport in the World!

Posted in pictures, Travel with tags , , , on November 29, 2011 by xi'an

The Charles de Gaulle airport, on which I posted my list of complaints a while ago, has been nominated “the most hated airport in the world” by CNN Go. They mentioned

“Grimy washrooms with missing toilet seats don’t help. Nor do broken scanning machines and an overall lack of signage, gate information screens and Paris-worthy bars, restaurants or cafés. The baffling circular layout is worsened by warrens of tunnel-like structures, dismissive staff and seething travelers waiting forever in the wrong queue.”

This is about Terminal 1, whose circular design is indeed nonsensical since there are only two exits to the circle. The picture on the site is in Terminal 2, with the huge posterboard at the exit of the train station. My complaint with Terminal 2 (I rarely use Terminal 1 for frequent-flyer reasons) is not about the toilets (they are fine by French standards), nor about scanning machines (I actually registered for an automated passport scan that cuts queues dramatically when both entering and leaving the country), nor about the bars and restaurants (I do not eat, nor drink), but rather about the poor design (or rather the outgrowth of the design:) a linear layout of the airport that forces travellers to walk long distances (often doubled by the fact that the luggage room exit is as far as possible from the train station) since there is no inner train, a flight density that often induces bussing passengers for dozens of minutes (this is always the case when flying to the UK), and a very poor train connection to down town Paris (there is no direct train, all trains stop in a myriad of Northern suburban cities).

Error and Inference [arXived]

Posted in Books, Statistics, University life with tags , , , , , , , on November 29, 2011 by xi'an

Following my never-ending series of posts on the book Error and Inference, (edited) by Deborah Mayo and Ari Spanos (and kindly sent to me by Deborah), I decided to edit those posts into a (slightly) more coherent document, now posted on arXiv. And to submit it as a book review to Siam Review, even though I had not high expectations it fits the purpose of the journal: the review was rejected between the submission to arXiv and the publication of this post!

R exam

Posted in Kids, pictures, Statistics, University life with tags , , , , , , , on November 28, 2011 by xi'an

Following a long tradition (!) of changing the modus vivendi of each exam in our exploratory statistics with R class, we decided this year to give the students a large collection of exercises prior to the exam and to pick five among them to the exam, the students having to solve two and only two of them. (The exercises are available in French on my webpage.) This worked beyond our expectations in that the overwhelming majority of students went over all the exercises and did really (too) well at the exam! Next year, we will hopefully increase the collection of exercises and also prohibit written notes during the exam (to avoid a possible division of labour among the students).

Incidentally, we found a few (true) gems in the solutions, incl. an harmonic mean resolution of the approximation of the integral

$\int_2^\infty x^4 e^{-x}\,\text{d}x=\Gamma(5,2)$

since some students generated from the distribution with density f proportional to the integrand over [2,∞) [a truncated gamma] and then took the estimator

$\dfrac{1-e^{-2}}{\frac{1}{n}\,\sum_{i=1}^n y_i^{-4}}\approx\dfrac{\int_2^\infty e^{-x}\,\text{d}x}{\mathbb{E}[X^{-4}]}\quad\text{when}\quad X\sim f$

although we expected them to simulate directly from the exponential and average the sample to the fourth power… In this specific situation, the (dreaded) harmonic mean estimator has a finite variance! To wit;

> y=rgamma(shape=5,n=10^5)
> pgamma(2,5,low=FALSE)*gamma(5)
[1] 22.73633
> integrate(f=function(x){x^4*exp(-x)},2,Inf)
22.73633 with absolute error < 0.0017
> pgamma(2,1,low=FALSE)/mean(y[y>2]^{-4})
[1] 22.92461
> z=rgamma(shape=1,n=10^5)
> mean((z>2)*z^4)
[1] 23.92876


So the harmonic means does better than the regular Monte Carlo estimate in this case!