Archive for Laplace

“Who is the statistician or scientist you admire the most?”

Posted in Books, Statistics, University life with tags , , , , , , , on April 5, 2011 by xi'an

As for the previous ISBA Bulletin, Luke Bornn sent me this (impossible!) question, to be answered in a few hundred words. First, let me exclude all living statisticians and scientists to avoid making a choice among all those people I admire and hurting anyone’s feeling (and also because this is somehow unfair to younger researchers). So let us stick to dead individuals! Second, I am quite hesitant to choose between a scientist (broad category!) and a statistician (restrictive category!). Again, let me [first?] stick to statisticians, avoiding the impossible choice between Albert Einstein, Marie Curie, Srinivasa Ramanujan, Henri Poincaré, Evariste Galois, Ada Byron,  and others… Continue reading

Keynes’ derivations

Posted in Books, Statistics with tags , , , , , on March 29, 2010 by xi'an

Chapter XVII of Keynes’ A Treatise On Probability contains Keynes’ most noteworthy contribution to Statistics, namely the classification of probability distributions such that the arithmetic/geometric/harmonic empirical mean/empirical median is also the maximum likelihood estimator. This problem was first stated by Laplace and Gauss (leading to Laplace distribution in connection with the median and to the Gaussian distribution for the arithmetic mean). The derivation of the densities f(x,\theta) of those probability distributions is based on the constraint the likelihood equation

\sum_{i=1}^n \dfrac{\partial}{\partial\theta}\log f(y_i,\theta) = 0

is satisfied for one of the four empirical estimate, using differential calculus (despite the fact that Keynes earlier derived Bayes’ theorem by assuming the parameter space to be discrete). Under regularity assumptions, in the case of the arithmetic mean, my colleague Eric Séré showed me this indeed leads to the family of distributions

f(x,\theta) = \exp\left\{ \phi^\prime(\theta) (x-\theta) - \phi(\theta) + \psi(x) \right\}\,,

where \phi and \psi are almost arbitrary functions under the constraints that \phi is twice differentiable and f(x,\theta) is a density in x. This means that \phi satisfies

\phi(\theta) = \log \int \exp \left\{ \phi^\prime(\theta) (x-\theta) + \psi(x)\right\}\, \text{d}x\,,

a constraint missed by Keynes.

While I cannot judge of the level of novelty in Keynes’ derivation with respect to earlier works, this derivation therefore produces a generic form of unidimensional exponential family, twenty-five years before their rederivation by Darmois (1935), Pitman (1936) and Koopman (1936) as characterising distributions with sufficient statistics of constant dimensions. The derivation of the distributions for which the geometric or the harmonic means are MLEs then follows by a change of variables, y=\log x,\,\lambda=\log \theta or y=1/x,\,\lambda=1/\theta, respectively. In those different derivations, the normalisation issue is treated quite off-handedly by Keynes, witness the function

f(x,\theta) = A \left( \dfrac{\theta}{x} \right)^{k\theta} e^{-k\theta}

at the bottom of page 198, which is not integrable in x unless its support is bounded away from 0 or \infty. Similarly, the derivation of the log-normal density on page 199 is missing the Jacobian factor 1/x (or 1/y_q in Keynes’ notations) and the same problem arises for the inverse-normal density, which should be

f(x,\theta) = A e^{-k^2(x-\theta)^2/\theta^2 x^2} \dfrac{1}{x^2}\,,

instead of A\exp k^2(\theta-x)^2/x (page 200). At last, I find the derivation of the distributions linked with the median rather dubious since Keynes’ general solution

f(x,\theta) = A \exp \left\{ \displaystyle{\int \dfrac{y-\theta}{|y-\theta|}\,\phi^{\prime\prime}(\theta)\,\text{d}\theta +\psi(x) }\right\}

(where the integral ought to be interpreted as a primitive) is such that the recovery of Laplace’s distribution, f(x,\theta)\propto \exp-k^2|x-\theta| involves setting (page 201)

\psi(x) = \dfrac{\theta-x}{|x-\theta|}\,k^2 x\,,

hence making \psi a function of \theta as well. The summary two pages later actually produces an alternative generic form, namely

f(x,\theta) = A \exp\left\{ \phi^\prime(\theta)\dfrac{x-\theta}{|x-\theta|}+\psi(x) \right\}\,,

with the difficulties that the distribution only vaguely depends on \theta, being then a step function times exp(\psi(x)) and that, unless \phi is properly calibrated, A also depends on \theta.

Given that this part is the most technical section of the book, this post shows why I am fairly disappointed at having picked this book for my reading seminar. There is no further section with innovative methodological substance in the remainder of the book, which now appears to me as no better than a graduate dissertation on the probabilistic and statistical literature of the (not that) late 19th century, modulo the (inappropriate) highly critical tone.

“Pour la Science” almost goes Bayesian!

Posted in Books, Statistics with tags , , , , on October 22, 2009 by xi'an

After the strange views held on Bayesian statistics by the popular science magazine La Recherche, it is  more than comforting to see the other popular science magazine Pour la Science to publish a more balanced paper on the role of statistical evidence, both frequentist and Bayesian. And by Andrew Gelman! This paper is actually a translation into French of a paper of Andrew with David Wiekliem, published earlier in American Scientist. I can only make one complaint about a missing reference to Laplace (the true father of Bayesian statistics!) who did study the difference between male and female births in his Théorie Analytique des Probabilités.

Extra-“Ordinary” meeting as well!!!

Posted in Statistics with tags , , , , , , on October 19, 2008 by xi'an

The “ordinary” meeting of the Royal Statistical Society last Wednesday was a tremendous success! The Read Paper by Rue, Martino and Chopin attracted a large crowd, surely partly thanks to the pre-ordinary meeting organised by the Young Statistician Section, and we are likely to see a nice collection of discussions in JRSS B as a result, if the number of discussions at the meeting can be used as a gauge. While I played my role of seconder by pointing out in my discussion the radical viewpoint of the paper according to which all simulation aspects can be erased, I noticed in a second discussion with Roberto Casarin that the Gaussian approximation to the marginal posterior is quite accurate in the stochastic volatility model. I am also looking forward the written discussion by Omiros Papaspiliopoulos where he points out connections with exact simulation methods and marginal representations such as Chib’s estimate of marginal likelihoods. In conclusion, this is certainly one of the most exciting Read Papers of the past years!!!