computational methods for numerical analysis with R [book review]

This is a book by James P. Howard, II, I received from CRC Press for review in CHANCE. (As usual, the customary warning applies: most of this blog post will appear later in my book review column in CHANCE.) It consists in a traditional introduction to numerical analysis with backup from R codes and packages. The early chapters are setting the scenery, from basics on R to notions of numerical errors, before moving to linear algebra, interpolation, optimisation, integration, differentiation, and ODEs. The book comes with a package cmna that reproduces algorithms and testing. While I do not find much originality in the book, given its adherence to simple resolutions of the above topics, I could nonetheless use it for an elementary course in our first year classes. With maybe the exception of the linear algebra chapter that I did not find very helpful.

“…you can have a solution fast, cheap, or correct, provided you only pick two.” (p.27)

The (minor) issue I have with the book and that a potential mathematically keen student could face as well is that there is little in the way of justifying a particular approach to a given numerical problem (as opposed to others) and in characterising the limitations and failures of the presented methods (although this happens from time to time as e.g. for gradient descent, p.191). [Seeping in my Gallic “mal-être”, I am prone to over-criticise methods during classing, to the (increased) despair of my students!, but I also feel that avoiding over-rosy presentations is a good way to avoid later disappointments or even disasters.] In the case of this book, finding [more] ways of detecting would-be disasters would have been nice.

An uninteresting and highly idiosyncratic side comment is that the author preferred the French style for long division to the American one, reminding me of my first exposure to the latter, a few months ago! Another comment from a statistician is that mentioning time series inter- or extra-polation without a statistical model sounds close to anathema! And makes extrapolation a weapon without a cause.

“…we know, a priori, exactly how long the [simulated annealing] process will take since it is a function of the temperature and the cooling rate.” (p.199)

Unsurprisingly, the section on Monte Carlo integration is disappointing for a statistician/probabilistic numericist like me,  as it fails to give a complete enough picture of the methodology. All simulations seem to proceed there from a large enough hypercube. And recommending the “fantastic” (p.171) R function integrate as a default is scary, given the ability of the selected integration bounds to misled its users. Similarly, I feel that the simulated annealing section is not providing enough of a cautionary tale about the highly sensitive impact of cooling rates and absolute temperatures. It is only through the raw output of the algorithm applied to the travelling salesman problem that the novice reader can perceive the impact of some of these factors. (The acceptance bound on the jump (6.9) is incidentally wrongly called a probability on p.199, since it can take values larger than one.)

[Disclaimer about potential self-plagiarism: this post or an edited version will eventually appear in my Books Review section in CHANCE.]

Vectorize!

Here is an email sent by one of my students a few days ago:

Do you know how to integrate a function with an  “if”?
For instance:
>X=rnorm(100)
>Femp=function(x){
+   return(sum(X<x))
+}
>integrate(Femp,0,1)$value

does not work.

My reply was that the fundamental reason it does not work is that integrate (or curve for instance) computes the function in several points of a grid by calling the function on a vector x and the comparison X<x does not make sense when both X and x are vectors. If one rewrites the code as

X=rnorm(100)
Femp=function(x){
   return((apply(as.matrix(outer(X,x,"-")<0),2,sum))}

the function can be integrated

> integrate(Femp,-2,2)
1.877735 with absolute error < 5.6e-05

However, this kind of syntactic gymnastic can be avoided by a simple call to Vectorize.

X=rnorm(100)
Femp=function(x){
   return(sum(X<x))}
> integrate(Vectorize(Femp),-2,2)
1.877735 with absolute error < 5.6e-05

More typos in Chapter 5

Following Ashley’s latest comments on Chapter 5 of Introducing Monte Carlo Methods with R, I realised Example 5.5 was totally off-the-mark! Not only the representation of the likelihood should have used prod instead of mean, not only the constant should call the val argument of integrate, not only integrate  uses lower and upper rather than from and to, but also the approximation is missing a scale factor of 10, squared root of the sample size… The corrected R code is thus

> cau=rcauchy(10^2)
> mcau=median(cau)
> rcau=diff(quantile(cau,c(.25,.75)))/sqrt(10^2)
> f=function(x){
+   z=dcauchy(outer(x,cau,FUN="-"))
+   apply(z,1,prod)}
> fcst=integrate(f,lower=-20,upper=20)$val
> ft=function(x){f(x)/fcst}
> g=function(x){dt((x-mcau)/rcau,df=49)/rcau}
> curve(ft,from=-1,to=1,xlab="",ylab="",lwd=2)
> curve(g,add=T,lty=2,col="steelblue",lwd=2)

and the corrected Figure 5.5 is therefore as follows. Note that the fit by the t distribution is not as perfect as before. A normal approximation would do better.

This mistake is most embarrassing and I cannot fathom how I came with this unoperating program! (The more embarrassing as Cauchy‘s house is about 1k away from mine…) I am thus quite grateful to Ashley for her detailed study of this example.