Archive for October, 2008

Computing evidence

Posted in Statistics with tags , , , on October 30, 2008 by xi'an

Yesterday, I gave this talk in the Model Assessment working group seminar at SAMSI, in connection with the Sequential Monte Carlo program that SAMSI runs this year. The format was quite nice as the two-hour schedule allowed for a lot of questions and interruptions (as well as my experimenting with smart board writing!). The talk is based on several papers written with Jean-Michel Marin this year and on the nested sampling paper with Nicolas Chopin discussed there a few days ago. (This will also be the topic of my advanced graduate course at CREST next February.) The methods that generated the most comments were

  1. reverse importance sampling, à la Gelfand & Dey (1994), which is one very elegant method, even though it may be prone to misbehaviour as it relates to harmonic means (see Radford Neal’s point). (Interestingly [?], googling on that term leads to links to Ò Ruanaidh & Fitzgerald’s (1996) book.)
  2. bridge sampling, à la Gelman & Meng (1998), especially for its curious connection with mixture sampling and defensive sampling.
  3. Chib’s (1995) marginal likelihood estimator for latent variable models, both because of the label switching difficulty (that was maybe lost on part of the audience) and because of the direct permutation fix.
  4. nested sampling, for both its formal simplicity and not-so-simple implementation.

The part about cross-model methods did not seem so interesting to the audience, maybe because it is mostly negative. The overall sobering conclusion, however, was that most of those methods were likely to fail in large dimensions, which is true when using (as we do) importance functions derived from nonparametric principles.

Dance Hall of the Dead

Posted in Books with tags , , on October 28, 2008 by xi'an

Tony Hillerman, author of a series of detective novels centered at the Navajo nation, died on Sunday. The series is made of 18 novels that feature Joe Leaphorn and Jim Chee from the Navajo Tribal Police and that give more insights in the Navajo culture than in police work. The set is to be read chronologically as the life and love story of the two main characters unfold along the way. One of my close friends finds the books too “New Age” but I think this truly reflects on the highly metaphysical Navajo perception of the World as well as the fragile preservation of this culture against exterior and devastating influences. While not all of the novels are of the same quality, the later ones being too centered on the main characters and somehow unrealistic in their conclusion, the early ones are truly terrific and I believe “A Thief of Time” to be Hillerman’s masterpiece.

Nested sampling, the return

Posted in Statistics with tags , , , , on October 25, 2008 by xi'an

After a fairly lenghty period of pondering whether or not we should invest in a revision, thanks to some scathing referee’s reports, Nicolas Chopin and I have at last made the jump and we just completed this revision of our assessment of nested sampling and resubmitted it to Biometrika.

To recap, nested sampling was introduced by John Skilling, in 2004 or so [if this page is a clue], presented at the Valencia 8 meeting in 2006 as an invited paper and consecutively published in Bayesian Analysis the same year under a slightly modified format. The method is a stochastic ascending climb of the likelihood function that approximates the marginal likelihood (or the evidence dear to Jeffreys) by a Riemann like representation, using the prior distribution as a proposal. It has been adopted enthusiastically by astronomers (I first heard from it at the Bayesian Cosmology meeting in Sussex in June 2006) and physicists, but less so by statisticians, to the point that it remains mostly unknown within our community. When the paper was presented, Nicolas Chopin and I were rather unconvinced by some of the claims made there, partly because of the un-orthodox style of John Skilling’s writing, and we wrote a skeptical discussion for the Valencia volume that led to an exchange of emails with the author. In order to study more in details the exact properties of the method, we embarked upon a larger experiment where we ended up with the conclusions that

  1. nested sampling enjoys the same speed of convergence as regular Monte Carlo methods, a fact already noted in the discussion of Evans in the Valencia volume, and a normal asymptotic approximation as well;
  2. there exists an importance sampling version of nested sampling where simulating under the likelihood constraint is straightforward;
  3. the unidimensional features of the methods are not absolute in that the computational effort is still in d3, if d is the dimension of the space;
  4. the implementation requires computational efforts that are equivalent to a specific kind of slice sampling since the method simulates from the prior under a minimum likelihood constraint;
  5. for a given computational effort, nested sampling does not necessarily dominate alternative approaches to evidence approximation like bridge sampling or reverse importance sampling.

We however botched the illustration of those points with poor programming choices (like using an infinite variance proposal in the MCMC step!) and got promptly rejected from Biometrika! Besides valid criticisms on the programming choices and surprising ones on the (ir)relevance of the CLT, we also got a fairly helpful suggestion on a possible closed form implementation of the importance sampling version nested sampling that kills one of the error terms in the Riemann approximation. When applied to a standard probit posterior, this version of nested sampling proved itself to be indeed very efficient, bypassing a well-tuned importance sampler.

Extra-“Ordinary” meeting as well!!!

Posted in Statistics with tags , , , , , , on October 19, 2008 by xi'an

The “ordinary” meeting of the Royal Statistical Society last Wednesday was a tremendous success! The Read Paper by Rue, Martino and Chopin attracted a large crowd, surely partly thanks to the pre-ordinary meeting organised by the Young Statistician Section, and we are likely to see a nice collection of discussions in JRSS B as a result, if the number of discussions at the meeting can be used as a gauge. While I played my role of seconder by pointing out in my discussion the radical viewpoint of the paper according to which all simulation aspects can be erased, I noticed in a second discussion with Roberto Casarin that the Gaussian approximation to the marginal posterior is quite accurate in the stochastic volatility model. I am also looking forward the written discussion by Omiros Papaspiliopoulos where he points out connections with exact simulation methods and marginal representations such as Chib’s estimate of marginal likelihoods. In conclusion, this is certainly one of the most exciting Read Papers of the past years!!!

Extraordinary “Pre-ordinary Meeting” at the RSS

Posted in Statistics with tags , on October 13, 2008 by xi'an

The Young Statisticians Section at the Royal Statistical Society is organising a pre-meeting on October 15 to introduce the Read Paper of the day at the Royal Statistical Society, namely the discussion paper by Havard Rue, Sandra Martino and Nicolas Chopin “Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations”. One of the speakers, Nicolas Chopin in the current case, is presenting a summary of the paper, followed by a (critical?) presentation by another researcher, Nigel Frial in this case.

I think this is a terrific idea and as all great ideas only wish it would have been brought forward earlier! And hope this will be continued for all discussion papers in the future. It will help younger members of the statistical community to get aquainted with the topics discussed during the meeting and hopefully encourage them to contribute either orally or by written means to the discussion in the Journal of the Royal Statistical Society. Best wishes to this valuable endeavour!

The great’08 Pascal challenge

Posted in Statistics with tags , , , , , , , on October 8, 2008 by xi'an

In order to make advances in the processing of their datasets and experiments, and in the understanding of the fundamental parameters driving the general relativity model, cosmologists are lauching a competition called the great’08 challenge through the Pascal European network. Details about the challenge are available on an arXiv:0802.1214 document, the model being clearly defined from a statistical point of view as a combination of lensing shear (the phenomenon of interest) and of various (=three) convolution noises that make the analysis so challenging, and the date being a collection of images of galaxies. The fundamental problem is to identify a 2d-linear distortion applied to all images within a certain region of the space, up (or down) to a precision of 0.003, the distortion being identified by an isotonic assumption over the un-distrorted images. The solution must be efficient too in that it is to be tested on 27 million galaxies! A standard MCMC mixture analysis on each galaxy is thus unlikely to converge before the challenge is over, next April. I think the challenge is worth considering by statistical teams, even though this represents a considerable involvement over the next six months….

A caricature of Bayesian statistics

Posted in Statistics with tags , on October 5, 2008 by xi'an

Following a rather shocking rendering of Bayesian statistics in the French popular Science magazine La Recherche last June by a French statistician, Nicolas Vayatis, I decided to send the following protest letter to its Editor as the ISBA president (my translation of a googslation, pardon my French!).

“As a statistician and as the current (2008) President of the International Society for Bayesian Analysis (ISBA), I would like to protest against the presentation and incredibly simplistic caricature made by Nicolas Vayatis of Bayesian statistics in No. 421 (p.26) of La Recherche.

Firstly, the Bayesian approach has not developed “along with the improvement of computers” but, rather, this approach has been dominant throughout the nineteenth century, from Laplace to Poincaré. Also, the contrast between Bayesian and frequent approaches is definitely not “philosophical”; on the opposite, it deals with the effectiveness and universality of procedures built therein.

Then, the presentation of Bayesian statistical inference made in the article got it all wrong: in the simple exemple of inference on the probability p of a coin hitting head during 100 tests on the same coin, the probability p also remains a “real” value, both unique and unknown, in the Bayesian approach which merely updates the [posterior] distribution of the probability parameter p given the number of heads obtained during the experiment. This probability distribution is not a belief but a rational construct that update information on p by integrating the pieces of information contained in the comments. Now this distribution is used to deduce an estimate of p as well as to test if the coin is biased or not.

Secondly, contrary to the title, the sectarian image spread by the paper (“mostly Anglo-Saxon”, “self-centered”, “Bayesian hymns”, “journals that do not accept the frequentist views”) decidedly is a thick-skinned caricature in the community even though it does not stand against examination: there are well-established Bayesian statisticians in many other countries than anglo-saxon countries, such as Chile, Brazil, Spain, or Italy, and obviously France. (For instance, a series of global conferences sees Bayesian statisticians gather every four years in Spain.) First-rate international journals of mathematical statistics, computational statistics or methodological statistics all include between 20% and 30% of Bayesian articles, they all have had Bayesian editors at one time or another, without a noticeable impact on or the quality of these journal. The only journal mostly devoted to Bayesian techniques is Bayesian Analysis, an electronic journal that was only recently created. Similarly, most national or international statistical societies have had or have chairs that are Bayesians and they apparently survived without collateral damage.

Finally, and it is indeed a story that did not belong in the article, the only Bayesian songs that you can hear at Bayesian conferences are those pastiches sung at the “cabaret” which concludes the conference banquet. This is a (indeed) Anglo-Saxon tradition of self-derision that is also found in many other conferences outside Bayesian statisticians.”

What is amazing is not that a popular Science magazine talks non-sense, this happens all the time (for instance, the same La Recherche confuses statististical Physics with Statistics in its September issue!). Nor is it the fact that editors build articles from browsing mostly at random ArXiv (this seems to be the case for most entries in the Mathematics section…), even though the current case is rather extreme (a communication made by a philosopher at a 2003 meeting that “focus on the pragmatic point of view of scientists’ day-to-day practices”, resulting in a poorly edited document that does not bring much novelty to the debate and further attributes Bayesian tendencies to Karl Popper). No, it is the illustration of the medias turning anyone and everyone into an expert without any checking and of those anyone and everyone gleefully endossing the expert coat (see all those so-called Economics experts unable to explain the impact of the subprime crisis on European banks, so we’d be better using sticky figures instead of experts)! In fact, in the current case, the expert is specialised in Machine Learning and non-parametric Statistics and he should know better about the basics of Bayesian Statistics… In any case, he is talking non-sense about Bayesians publishing in their own journals (like what?! Annals of Statistics? JASA? JRSS Series B?) and singing Bayesian anthems at the opening of their meetings (simply demonstrating he never attended a Bayesian meeting). This must be an effect of the expert’s self-intoxication of being able to make statements without any moderation by peers…