Of black swans and bleak prospects

Following a review by Dennis Lindley in Significance (March 2008) and several entries on Andrew Gelman’s blog, I decided to read The Black Swan: The Impact of the Highly Improbable by Nassim Taleb in order to check by myself why the analyses of those two (admittedly very different) Bayesians were so dissonant. Not very suprisingly, almost immediately after starting the book, I found myself much more in agreement with Dennis’ negative views. While I think his review is an elegant and precise view of the book that is enough by itself, I want to add a few points of details below.

First, I found the tone of the book immensely annoying, to the point of almost giving up reading it several times, mostly because of its intense anti-intellectualism, including a populist dismissal of academics (including almost all economists, most philosophers and mathematicians, and apparently all statisticians, except for Jaynes!), as well as because of the inflated I-know-better-than-thou ego of the author. It is only thanks to being stuck several days a week in the black swan of the inpredictable Paris metro that I was able to reach the last page… (Once you get used to it, the permanent [tongue-in-cheek] French-bashing found in the book is quite funny!)

While I could discuss the wider picture at length, I think that the book can be criticised solely from a statistical point of view as mostly missing the point. For instance, the notions of probable/improbable and randomness [that are constantly in use within the book] are always used in a vague sense and they thus mostly loose their meaning. (The distinction between random—that is, driven by a probability distribution—and fortuitous—that is, lacking any kind of reproducibility to be considered as a probability outcome—comes so late within the book as to be rather useless.) The extreme events that are called black swans are never analysed in terms of model shift, although they mostly correspond to cases where the background model had changed but the players were not aware of it. This somehow gives the impression that the author expects there exists a (deterministic) model that should explain even the most extreme phenomena. When considering some examples in the book like 09/11, this sounds ludicrous: the attack on 09/11 has nothing to do with randomness or a probabilistic model! Similarly, there is no discussion of the possible non-homogeneous nature of the time series leading to black swans.

The (heavy) anti-Statistics discourse focusses on the “bell” curve, which gives the (wrong) impression that all (necessarily mediocre) statisticians (honestly or not) think that a Gaussian distribution should fit everything and all. As pointed out by Dennis Lindley, extreme value theory has been invented quite a while ago and actuaries do rely on it with some amount of success. Furthermore, Nassim Taleb’s main criticism of the Gaussian distribution seems to center on the CLT that cannot allow for extreme events in the asymptotic. But then alternatives to the mean like the median and to Gaussian like the Cauchy could have been brought into the game. A sentence like `A true random system is in fact random and does not have any predictable property‘ (page 198) is representative of a certain lack of understanding of Statistics by the author, just like his criticism that model checking is a vicious circle in that it cannot be validated outside the checked model… Not so! There are other misunderstandings / misrepresentations of basic principles as for instance the confusion between the random walk and the CLT just before introducing Galton’s Quincunx—Galton who is accused of being innocent of mathematics! (The power law explanation in the Notes on page 322 is also completely confused/confusing since the cdf F seems to be replaced with the pdf f, while the power is the wrong sign.)

Obviously, the picture is not completely bleak in that the relevance of conditioning on the whole past rather than solely on the events that agree with one’s theory is well-explained (a mention of the misdirected analysis of the O-rings leading to the Challenger disaster would have nicely fitted at this point). The strong and blind reliance of financial markets on formalised mathematical models is certainly to blame and I am sure the author feels vindicated to have been such an accurate doomsayer the year before the Big Crash of last Fall (or was it the Big Fall of last Crash?!). I actually think one of the issues in this reliance is the quasi-absence of statisticians (as opposed to probabilists) in those financial structures, in agreement with Taleb’s point that predictions were rarely accompanied with error ranges and, when they were, they were based on inappropriate Gaussian approximations. Actually, I never took the Black-Scholes formula to be an accurate representation of reality, but rather a gentleman’s agreement between traders that served to agree on prices, the “proof” being that they never seemed to estimate anything about this model! That it did not allow for big jumps was overlooked by most, to their eventual sorrow… (At another level, the book also interestingly alludes to the philosophical debate about the nature of inference, one of Popper’s pet topics, but this would need another post by itself.)

It is obviously a difficult exercise to write about popular Science without being populist and it must be almost inevitable to oversimplify one’s discourse by emphasizing a few examples over others, but I think the book overdoes it! By a fair margin. Worse, by attacking modelling tools like the Gaussian, models and modelers as a conglomerate of “charlatans”, it contributes to the anti-Scientist discourse that is unfortunately so prevalent today. Being a skeptic is commendable and scientists should never cease questioning their models, but throwing all models to the winds and using only “facts” to drive one’s decisions is not very helpful. As put by George Box (or by someone else before him), “all models are wrong, but some models are useful” and we (as statisticians) can devise tools to assess how wrong and how useful. Encouraging a total mistrust of anything scientific or academic is not helping in solving issues, but most surely pushes people in the arms of charlatans with ready answers.

22 Responses to “Of black swans and bleak prospects”

  1. […] somehow reminds me of the criticisms on the normal/Gaussian distribution in Nassim Taleb’s (outrageous) Black Swan. I would think the same type of criticism applies here: The interview mentions the fact […]

  2. […] The Bed of Procrustes: Philosophical and Practical Aphorisms (only surprising given my aversion for Taleb’s style) […]

  3. […] but this sounds like an interesting discussion! At this stage, I cannot decide whether this is yet again a point about model shifts or if there is a more fundamental issue at stake. (Thankfully, Popper is […]

  4. […] your world is pleasant and engaging, at the other end of the stylistic spectrum from Taleb’s Black Swan. Fung’s point is obviously the opposite of Taleb’s: he is showing the reader how well […]

  5. Nobody, I think you missed my point: I do not aim at defending the Gaussian distribution, but at criticising the fact that Taleb depicts statisticians as obsessed with this distribution. Statistics has come a long way from the all-linear all-Gaussian perspective of the 30’s…

  6. About the tone of the book, I just need to say that most people, including (or should I say specially?) academics, think with the same arrogance expressed in the book but usually don’t spell it out for education (or fear?), then one should not take it personally and focus on analyzing the main ideas.

    About the third paragraph, Taleb does not think that a deterministic model exist, his point is that once you do not have a model able to take into account extreme events, you should not use one, because the very unfortunate outcome that you did not protect yourself against (since you were using a inadequate model) can put you out of the game. And that is the main idea of the book that the blog´s author does not seem to understand, specially when he mention that the extreme value theory is used by actuaries with some amount of success. First of all, most of the actuarial job don’t face the issues that Taleb is worried about, that is, the extreme event that can put you out of the game, for example the idea of going bankruptcy if you are option trader (which is not likely to happen to an insurance firm, or any other that actuaries are worried about). Again, regarding those black swans of the book, some amount of success is not enough. And the way to work with it is not finding the perfect model (that is why he critics statistician, because they are always worried about the model) but to protect against it, by not putting all your money in what your “imperfect” model says, for example.

    To sum up, the book has a lot of interesting ideas, but you were so blind by the attack on the Gaussian distribution, that you forgot assimilate them.

  7. […] Indeed, I found the book both annoying and unconvincing, for reasons not very different from the criticisms addressed at Taleb’s The Black Swan. The book aims at demonstrating that the philosophical […]

  8. Thank you for your blog

    A review by David Aldous :

    http://www.stat.berkeley.edu/~aldous/157/Books/taleb.html

  9. […] an easy pun related to earlier posts… There is an article in The New York Times of yesterday about unemployed traders turning into […]

  10. […] are probably tired of my rants about what I think of Taleb’s work and what I think he’s gotten wrong.  But really, I find his FT article interesting as it’s giving us “principles” […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.