Of black swans and bleak prospects

Following a review by Dennis Lindley in Significance (March 2008) and several entries on Andrew Gelman’s blog, I decided to read The Black Swan: The Impact of the Highly Improbable by Nassim Taleb in order to check by myself why the analyses of those two (admittedly very different) Bayesians were so dissonant. Not very suprisingly, almost immediately after starting the book, I found myself much more in agreement with Dennis’ negative views. While I think his review is an elegant and precise view of the book that is enough by itself, I want to add a few points of details below.

First, I found the tone of the book immensely annoying, to the point of almost giving up reading it several times, mostly because of its intense anti-intellectualism, including a populist dismissal of academics (including almost all economists, most philosophers and mathematicians, and apparently all statisticians, except for Jaynes!), as well as because of the inflated I-know-better-than-thou ego of the author. It is only thanks to being stuck several days a week in the black swan of the inpredictable Paris metro that I was able to reach the last page… (Once you get used to it, the permanent [tongue-in-cheek] French-bashing found in the book is quite funny!)

While I could discuss the wider picture at length, I think that the book can be criticised solely from a statistical point of view as mostly missing the point. For instance, the notions of probable/improbable and randomness [that are constantly in use within the book] are always used in a vague sense and they thus mostly loose their meaning. (The distinction between random—that is, driven by a probability distribution—and fortuitous—that is, lacking any kind of reproducibility to be considered as a probability outcome—comes so late within the book as to be rather useless.) The extreme events that are called black swans are never analysed in terms of model shift, although they mostly correspond to cases where the background model had changed but the players were not aware of it. This somehow gives the impression that the author expects there exists a (deterministic) model that should explain even the most extreme phenomena. When considering some examples in the book like 09/11, this sounds ludicrous: the attack on 09/11 has nothing to do with randomness or a probabilistic model! Similarly, there is no discussion of the possible non-homogeneous nature of the time series leading to black swans.

The (heavy) anti-Statistics discourse focusses on the “bell” curve, which gives the (wrong) impression that all (necessarily mediocre) statisticians (honestly or not) think that a Gaussian distribution should fit everything and all. As pointed out by Dennis Lindley, extreme value theory has been invented quite a while ago and actuaries do rely on it with some amount of success. Furthermore, Nassim Taleb’s main criticism of the Gaussian distribution seems to center on the CLT that cannot allow for extreme events in the asymptotic. But then alternatives to the mean like the median and to Gaussian like the Cauchy could have been brought into the game. A sentence like `A true random system is in fact random and does not have any predictable property‘ (page 198) is representative of a certain lack of understanding of Statistics by the author, just like his criticism that model checking is a vicious circle in that it cannot be validated outside the checked model… Not so! There are other misunderstandings / misrepresentations of basic principles as for instance the confusion between the random walk and the CLT just before introducing Galton’s Quincunx—Galton who is accused of being innocent of mathematics! (The power law explanation in the Notes on page 322 is also completely confused/confusing since the cdf F seems to be replaced with the pdf f, while the power is the wrong sign.)

Obviously, the picture is not completely bleak in that the relevance of conditioning on the whole past rather than solely on the events that agree with one’s theory is well-explained (a mention of the misdirected analysis of the O-rings leading to the Challenger disaster would have nicely fitted at this point). The strong and blind reliance of financial markets on formalised mathematical models is certainly to blame and I am sure the author feels vindicated to have been such an accurate doomsayer the year before the Big Crash of last Fall (or was it the Big Fall of last Crash?!). I actually think one of the issues in this reliance is the quasi-absence of statisticians (as opposed to probabilists) in those financial structures, in agreement with Taleb’s point that predictions were rarely accompanied with error ranges and, when they were, they were based on inappropriate Gaussian approximations. Actually, I never took the Black-Scholes formula to be an accurate representation of reality, but rather a gentleman’s agreement between traders that served to agree on prices, the “proof” being that they never seemed to estimate anything about this model! That it did not allow for big jumps was overlooked by most, to their eventual sorrow… (At another level, the book also interestingly alludes to the philosophical debate about the nature of inference, one of Popper’s pet topics, but this would need another post by itself.)

It is obviously a difficult exercise to write about popular Science without being populist and it must be almost inevitable to oversimplify one’s discourse by emphasizing a few examples over others, but I think the book overdoes it! By a fair margin. Worse, by attacking modelling tools like the Gaussian, models and modelers as a conglomerate of “charlatans”, it contributes to the anti-Scientist discourse that is unfortunately so prevalent today. Being a skeptic is commendable and scientists should never cease questioning their models, but throwing all models to the winds and using only “facts” to drive one’s decisions is not very helpful. As put by George Box (or by someone else before him), “all models are wrong, but some models are useful” and we (as statisticians) can devise tools to assess how wrong and how useful. Encouraging a total mistrust of anything scientific or academic is not helping in solving issues, but most surely pushes people in the arms of charlatans with ready answers.