## Carnon [and Core, end]

Posted in Books, Kids, pictures, R, Running, Statistics, Travel, University life with tags , , , , , , , , , , , , , on June 16, 2012 by xi'an

Yet another full day working on Bayesian Core with Jean-Michel in Carnon… This morning, I ran along the canal for about an hour and at last saw some pink flamingos close enough to take pictures (if only to convince my daughter that there were flamingos in the area!). Then I worked full-time on the spatial statistics chapter, using a small dataset on sedges that we found in Gaetan and Guyon’s Spatial Statistics and Modelling. I am almost done tonight, with both path sampling and ABC R codes documented and working for this dataset. But I’d like to re-run both codes for longer to achieve smoother outcomes.

## yet more questions about Monte Carlo Statistical Methods

Posted in Books, Statistics, University life with tags , , , , , , , , , , on December 8, 2011 by xi'an

As a coincidence, here is the third email I this week about typos in Monte Carlo Statistical Method, from Peng Yu this time. (Which suits me well in terms of posts as  I am currently travelling to Provo, Utah!)

I’m reading the section on importance sampling. But there are a few cases in your book MCSM2 that are not clear to me.

On page 96: “Theorem 3.12 suggests looking for distributions g for which |h|f/g is almost constant with finite variance.”

What is the precise meaning of “almost constant”? If |h|f/g is almost constant, how come its variance is not finite?

“Almost constant” is not a well-defined property, I am afraid. By this sentence on page 96 we meant using densities g that made |h|f/g as little varying as possible while being manageable. Hence the insistence on the finite variance. Of course, the closer |h|f/g is to a constant function the more likely the variance is to be finite.

“It is important to note that although the finite variance constraint is not necessary for the convergence of (3.8) and of (3.11), importance sampling performs quite poorly when (3.12) ….”

It is not obvious to me why when (3.12) importance sampling performs poorly. I might have overlooked some very simple facts. Would you please remind me why it is the case? From the previous discussion in the same section, it seems that h(x) is missing in (3.12). I think that (3.12) should be (please compare with the first equation in section 3.3.2)

$\int h^2(x) f^2(x) / g(x) \text{d}x = + \infty$

The preference for a finite variance of f/g and against (3.12) is that we would like the importance function g to work well for most integrable functions h. Hence a requirement that the importance weight f/g itself behaves well. It guarantees some robustness across the h‘s and also avoids checking for the finite variance (as in your displayed equation) for all functions h that are square-integrable against g, by virtue of the Cauchy-Schwarz inequality.

## Time series

Posted in Books, R, Statistics with tags , , , , , , on March 29, 2011 by xi'an

(This post got published on The Statistics Forum yesterday.)

The short book review section of the International Statistical Review sent me Raquel Prado’s and Mike West’s book, Time Series (Modeling, Computation, and Inference) to review. The current post is not about this specific book, but rather on why I am unsatisfied with the textbooks in this area (and correlatively why I am always reluctant to teach a graduate course on the topic). Again, I stress that the following is not specifically about the book by Raquel Prado and Mike West!

With the noticeable exception of Brockwell and Davis’ Time Series: Theory and Methods, most time-series books seem to suffer (in my opinion) from the same difficulty, which sums up as being unable to provide the reader with a coherent and logical description of/introduction to the field. (This echoes a complaint made by Håvard Rue a few weeks ago in Zurich.) Instead, time-series books appear to haphazardly pile up notions and techniques, theory and methods, without paying much attention to the coherency of the presentation. That’s how I was introduced to the field (even though it was by a fantastic teacher!) and the feeling has not left me since then. It may be due to the fact that the field stemmed partly from signal processing in engineering and partly from econometrics, but such presentations never achieve a Unitarian front on how to handle time-series. In particular, the opposition between the time domain and the frequency domain always escapes me. This is presumably due to my inability to see the relevance of the spectral approach, as harmonic regression simply appears (to me) as a special type of non-linear regression with sinusoidal regressors and with a well-defined likelihood that does not require Fourier frequencies nor periodogram (nor either spectral density estimation). Even within the time domain, I find the handling of stationarity  by time-series book to be mostly cavalier. Why stationarity is important is never addressed, which leads to the reader being left with the hard choice between imposing stationarity and not imposing stationarity. (My original feeling was to let the issue being decided by the data, but this is not possible!) Similarly, causality is often invoked as a reason to set constraints on MA coefficients, even though this resorts to a non-mathematical justification, namely preventing dependence on the future. I thus wonder if being an Unitarian (i.e. following a single logical process for analysing time-series data) is at all possible in the time-series world! E.g., in Bayesian Core, we processed AR, MA, ARMA models in a single perspective, conditioning on the initial values of the series and imposing all the usual constraints on the roots of the lag polynomials but this choice was far from perfectly justified…

## Andrew’s criticisms

Posted in Books, Statistics with tags , , , , , , , , on January 23, 2010 by xi'an

Andrew Gelman has just written a most entertaining review of “Introducing Monte Carlo Methods with R” on his blog. The first sentence is ominous as the book seemingly reminded him of communists and fascists…! The explanation for this frightening debut is that the connection between the components of statistics

… ↔ Probability theory ↔ Theoretical statistics↔Statistical methodology ↔ Applications ↔ Computation ↔ Probability theory ↔ …

may be seen as a torus just as the range of political ideologies, the argument being that both George and I switched from proving mathematical minimaxity theorems about James-Stein estimators to proving convergence theorems. about Metropolis-Hastings algorithms. After pondering Andrew’s lines for a while, I am far from sure this is a positive assessment of Introducing Monte Carlo Methods with R! Indeed, at the first glance, it may give the blog reader the feeling that this is yet another theoretical book about Monte Carlo methods, written by theorists and mainly for theorists (Andrew wrote “applied researchers such as myself will get much more use out of theory as applied to computation“)… While we strive to distance ourselves from making a baby version of Monte Carlo Statistical Method, choosing the format of a Use R! book to clarify even further the purpose of the book: to lead (our students and) our readers to understand Monte Carlo methods through worked-out examples to the point of developping their own methods, while keeping the theory at bay.

A second read shows that Andrew’s point is much more subtle, namely that as (formerly?) mathematical statisticians, we have adopted a terse style that (maybe unconsciously) shy way from giving too much detail and explanations: once a definition is provided, it should suffice to itself! This leads to what Andrew calls little puzzles, where the reader needs to stop and reason out why things are as they are. (“I noticed a bunch of other examples of this sort, where the narrative just flows by and, as a reader, you have to stop and grab it. Lots of fun.”)  I noticed the same reactions from my students, so I quite agree with this point. When learning with a book, you need to sit with a piece of paper on one side (if the margins are too narrow), your computer on the other side and test everything for yourself. This is actually an intended feature, if not spelled out more clearly, and I thus appreciate very much Andrew’s conclusion that “it would also be an excellent book for a course on statistical computing“!

There is also Andrew’s comment that the book is ugly, which stings, but again can be seen in a different light.I obviously do not find Introducing Monte Carlo Methods with R ugly but the printing could have been indeed nicer and the fact that the printers used the jpeg versions of the figures instead of the postscript or pdf versions did not help. The raw R output presented verbatim in most pages is not particularly beautiful either, but this is truly intended, for readers who cannot test the code immediately (as when reading in the metro or listening to the course at the same time). The R programs are far from perfect R programs, but examples of what a “standard” beginner would do. I also agree with the suggestion of an epilogue: we wrote several times during the course of the book that we were not providing the big picture and that many aspects of the Monte Carlo methodology were not covered, but this would be worth repeating at the end, along with the few general recommendations we can make about better R programming. Another thing to add in the next edition!

A final interesting remark is that the very first comment on Andrew’s post was about solutions! This is a strong request from readers. nowadays, and thus seems like a compulsory element of publishing books with exercises. (As we discovered a wee too late for Bayesian Core!)