“Not only defended but also applied”

Posted in Statistics with tags , , , , on June 30, 2010 by xi'an

On page 124 of his superb Introduction to Probability Theory book (volume 1), William Feller has this strange remark about Bayesian inference:

“Unfortunately Bayes’ rule has been somewhat discredited by metaphysical applications of the type described above. In routine practice, this kind of argument can be dangerous. A quality control engineer is concerned with one particular machine and not with an in nite population of machines from which one was chosen at random. He has been advised to use Bayes’ rule on the grounds that it is logically acceptable and corresponds to our way of thinking. Plato used this type of argument to prove the existence of Atlantis, and philosophers used it to prove the absurdity of Newton’s mechanics. In our case it overlooks the circumstance that the engineer desires success and that he will do better by estimating and minimizing the sources of various types of errors in predicting and guessing.The modern method of statistical tests and estimation is less intuitive but more realistic. It may be not only defended but also applied.”

When we were discussing about this great book, Andrew Gelman pointed out to me this strong dismissal of Bayesian techniques (note that I had overlooked so far) and, given that it is still quoted as an argument against a Bayesian approach to inference, we ended up writing [well, mostly Andrew!] a short note on the motivations and implications of this remark, now published on arXiv. One of the points is that Feller’s sentence has the interesting feature that it is actually the opposite of the usual demarcation: typically it is the Bayesian who makes the claim for inference in a particular instance and the frequentist who restricts claims to infinite populations of replications. Another point is the naïve faith in the classical Neyman-Pearson theory to solve practical problems in statistics.

Actually, Persi Diaconis took a (deeper) look at Feller’s stance as well, as mentioned in this review of Jaynes’s Probability Theory. Using Amazon Look Inside tool,  I spotted Feller being mentioned more than 30 times in Jaynes’s book, one of the best quotes being “The date was 1956 when the author met Willy Feller“! More to the point, Jaynes identifies Feller’s dismissal of the “old wrong ways” (volume 2, p.76), which is to be opposed to the “modern method” above. (Persi Diaconis and Susan Holmes also wrote a nice piece entitled “A Bayesian peek into Feller volume 1″ that does not relate directly to this issue.) In a loosely related point, Persi’s warning that he sees “a strong trend against measure theory in modern statistics departments: [he] had to fight to keep the measure theory requirement in Stanford’s statistics graduate program“, to which I completely subscribe, should be heard more widely…

Magical risotto!

Posted in Kids, Travel with tags , , on June 29, 2010 by xi'an

When Marilena Barbieri told me in Padova that she could make [good] risotto in her microwave oven, I was quite skeptical, both because I avoid using the microwave for cooking and because risotto is one of my favourite dishes! (Did I ever mention this outworldly risotto di seppie a la Veneziana that was so good it was almost a desert, served at an outrageous price at the Hostaria da Franz in Venezia!) However, Marilena sent me her recipe and I tried it last week. Now, I have to acknowledge that it is an excellent recipe, to the point that I will use it in the future (as long as my kids do not get tired of it, as happened with fajitas, crumbles and quiches in the past years…). Here is the core of Marilena’s recipe:

2 deciliters/cups hot chicken or vegetable broth/stock (or soup cube)
1 deciliter/cup arborio rice
1/2 teaspoon salt
chopped onion
risotto ingredients (e.g. minced vegetables or truffle bits)
oil or butter

Start with oil or butter and chopped onion in a covered bowl. Put in the microwave for 2 minutes. Add the other ingredients. Cover and put in the microwave for 2 additional minutes (or more, depending on the kind of vegetable you add). Then add broth, rice and cook at 750W for 3 minutes, then at 350W for 10 additional minutes. Stir only at the end. If the risotto looks too liquid, keep cooking one minute at a time until it tastes al dente.

Two U of Toronto professors die on Denali

Posted in Mountains, Running, University life with tags , , , , on June 28, 2010 by xi'an

When looking for Radford Neal’s page in the CS department of U of T, I came upon this very sad item of news, namely that (climber and) Professor Avner Magen have been killed last month in an avalanche on Denali, along with Professor Andrew Herzenberg from the Faculty of Medicine at University Health Network. There is a donation site to support Avner’s family.

“Bayesian model comparison in cosmology” on-line

Posted in Statistics, University life with tags , , , , , on June 27, 2010 by xi'an

I actually missed the piece of information that our our paper “Bayesian model comparison in cosmology with Population Monte Carlo” has been accepted by Monthly Notices of the Royal Astronomical Society on March 1! The abstract if not the whole paper is available on-line as early-view since mid-April… This is my last paper published in collaboration with the cosmologists of the Ecosstat 2005-2009 ANR program. Hopefully not the end of our collaboration as this was a very fruitful experience from my viewpoint, which happened to coincide with the golden years of population Monte Carlo, just as the Misgepop ANR program launched our foray into ABC methods. (In case you are unaware of the link, Scott Sisson has a twitter page posting news on ABC methods.)

Another harmonic mean approximation

Posted in R, Statistics with tags , , , , , , on June 27, 2010 by xi'an

Martin Weinberg posted on arXiv a revision of his paper, Computing the Bayesian Factor from a Markov chain Monte Carlo Simulation of the Posterior Distribution, that is submitted to Bayesian Analysis. I have already mentioned this paper in a previous post, but I remain unconvinced of the appeal of the paper method, given that it recovers the harmonic mean approximation to the marginal likelihood… The method is very close to John Skilling’s nested sampling, except that the simulation is run from the posterior rather than from the prior, hence the averaging on the inverse likelihoods and hence the harmonic mean connection. The difficulty with the original (Michael Newton and Adrian Raftery’s) harmonic mean estimator is attributed to “a few outlying terms with abnormally small values of” the likelihood, while, as clearly spelled out by Radford Neal,  the poor behaviour of the harmonic mean estimator has nothing abnormal and is on the opposite easily explainable.

I must admit I found the paper difficult to read, partly because of the use of poor and ever-changing notations and partly because of the lack of mathematical rigour (see, e.g., eqn (11)). (And maybe also because of the current heat wave.) In addition to the switch from prior to posterior in the representation of the evidence, a novel perspective set in the paper seems to be an extension of the standard harmonic mean identity that relates to the general expression of Gelfand and Dey (1994, Journal of the Royal Statistical Society B) when using an indicator function as an instrumental function. There is therefore a connection with our proposal (made with Jean-Michel Marin) of considering an HPD region for excluding the tails of the likelihood, even though the set of integration is defined as “eliminating the divergent samples with $L_i \ll 1$“. This is essentially the numerical Lebesgue algorithm advanced as one of two innovative algorithms by Martin Weinberg. I wonder how closely related the second (volume tesselation) algorithm is to Huber and Schott’s TPA algorithm, in the sense that TPA also requires a “smaller” integral….

The Millennium Trilogy (tome 3)

Posted in Books with tags , , , , on June 26, 2010 by xi'an

“Trinity and Bob the Dog devoted the best part of a week to identifying and separating out Ekström’s mobile from the background noise of about 200,000 other mobile telephones. They used a technique called Random Frequency Tracking System.” Stieg Larsson, The Girl who kicked the hornets’ nest

While I was reading the second volume of The Millennium Trilogy, I [addictedly!] ordered the third volume , The Girl who kicked the hornets’ nest, and found it in my mailbox on my return from Padova. I started reading it on Saturday night and [addictedly!] kept reading and reading till it was over, on Monday early morning! The conclusion is that…The Millennium Trilogy is indeed truly addictive, although not very well-written nor even altogether convincing. In somewhat of a contradiction with several of my friends, I actually preferred the second volume of the series, the first one being too brutal and the last one too predictable.

Plague ran Ekström’s digitized voice through a program called V.P.R.. When he had five separate examples of a word, he charted it with respect to the time it took to speak the word, what tone of voice and frequency range it took to speak the word, and a dozen other markers. The result was a graph.” Stieg Larsson, The Girl who kicked the hornets’ nest

This last novel has several interesting literary features but still shares many of the defects of the previous volumes. For instance, the poor habit of launching into useless descriptions. This time, instead of the unabridged Ikea catalog, we are given the complete tour of the home protection company… Similarly, the sudden relation of Blomkvist with a policewoman has been announced by red flags for dozens of pages in advance (even though it is quite a hilarious tryst!). The gun-battle in the restaurant is rather implausible, even though its role in the plot is meaningful. Maybe the least convincing part of this plot is the counterstrike at the secret cell within the National Security Agency, S.I.S., coming from the same agency but from “good” agents, as they manage to dismantle a well-organised if small secret unit that has been operated for fifty years in complete anonymity. The pace at which this reaction of the legal side takes place is gripping and helps very much at making the book addictive!, but I feel the story is stretched quite thin at this stage. The [bad] secret [secret] agents are also a bit too stupid to have survived fifty years of this regime, while Blomkvist’ lawyer sister is [again] too smart for the defence of Salander (even thought the availability of hard proofs like Burjman’s video are a big help!)…

“She opened the door wide and let him into her life again.” Stieg Larsson, The Girl who kicked the hornets’ nest

On the positive side, I appreciate the return to the main characters Blomkvist and Berger acting as engaged journalists. The passage of Berger at the main Swedish daily, Svenske Morgon-Posten, is quite convincing, both in terms of handling a large team of professionals with short-term goals and of facing antagonism from male colleagues (and the resolution of the stalking sub-story is quite surprising!). The most appealing part of the book is the unité de lieu imposed on Salander by her stay in the hospital as this forces the author to focus on psychological descriptions of Salander rather than detailing her grocery bills… The trick imagined to get her to communicate with the outside world (incl. Blomkvist and Berger) is quite good and the connection she builds with the neurosurgeon is also believable. I also loved discovering the fact that Paolo Roberto is a real boxer (who also plays his own role in the movies!) while the book was making fun of a reverse situation where Blomkvist runs into an actor who plays the role of a detective. (I am not sure I am clear enough there!) The death of Salander’s father comes as a complete surprise (even though we have to surmise that the old security agent is deeply sick). And involving the Prime Minister is also a nice move by Stieg Larssen! At last, the final battle of Salander and Niederman is nailed down in the most original (if gory) manner! Even Fermat’s last theorem comes back with a twist that makes me reconsider the second volume in a much more positive manner. I thus [reluctantly] conclude at a good readable fast-paced story with the shortcomings of the genre (not only the inconsistencies, and the too many coincidences, but also the perturbing fact that the main characters have this vigilante impulse to make justice outside the legal system)… A perfect book for a long plane ride.

Top 15 all-timers?

Posted in Statistics, University life with tags on June 25, 2010 by xi'an

Next year, I will organise a reading seminar for my graduate students so that they all read the “classics” of Statistics. Originally,  I planned to use the book Biometrika: One Hundred Years by Mike Titterington and David Cox, but there are not enough papers in it, so I expanded the list and ended with the following. This is a poll so please vote for the 15 top papers all students should have read or add the missing ones. (Of course, those are supposed to be “classics”, so 2009 preprints are not appropriate!)