Archive for Nassim Taleb

Texan black swan

Posted in Books, pictures with tags , , , , , , , , on September 12, 2017 by xi'an

“Un événement improbable aux conséquences d’autant plus désastreuses que l’on ne s’y est pas préparé.”

This weekend, there was a short article in Le Monde about the Harvey storm as a Texan illustration of Taleb’s black swan. An analysis that would imply every extreme event like this “once-in-a-thousand year” event (?) can be called a black swan… “An improbable event with catastrophic consequences, the more because it had not been provisioned”, as the above quote translates. Ironically, there is another article in the same journal, about the catastrophe being “ordinary” and “not unexpected”! While such massive floods are indeed impacting a huge number of people and companies, because the storm happened to pour an unusual amount of rain right on top of Houston, they indeed remain within the predictable and not so improbable in terms of the amount of water deposited in the area and in terms of damages, given the amount and style of construction over flood plains. For instance, Houston is less than 50 feet above sea level, has fairly old drainage and pipe systems, and lacks a zoning code. With mostly one or two-story high buildings rather than higher rises. (Incidentally, I appreciated the juxtaposition of the article with the add for Le Monde des Religions and its picture of a devilesque black goat!)


book based on a single well-known concept [smbc repost]

Posted in Books, Kids with tags , , , , , on December 26, 2016 by xi'an

the latest Significance: Astrostats, black swans, and pregnant drivers [and zombies]

Posted in Books, Kids, pictures, Statistics, Travel, University life with tags , , , , , , , , , , on February 4, 2015 by xi'an

Reading Significance is always an enjoyable moment, when I can find time to skim through the articles (before my wife gets hold of it!). This time, I lost my copy between my office and home, and borrowed it from Tom Nichols at Warwick with four mornings to read it during breakfast. This December issue is definitely interesting, as it contains several introduction articles on astro- and cosmo-statistics! One thing I had not noticed before is how a large fraction of the papers is written by authors of books, giving a quick entry or interview about their book. For instance, I found out that Roberto Trotta had written a general public book called the Edge of the Sky (All You Need to Know About the All-There-Is) which exposes the fundamentals of cosmology through the 1000 most common words in the English Language.. So Universe is replaced with All-There-Is! I can understand and to some extent applaud the intention, but it nonetheless makes for a painful read, judging from the excerpt, when researcher and telescope are not part of the accepted vocabulary. Reading the corresponding article in Significance let me a bit bemused at the reason provided for the existence of a multiverse, i.e., of multiple replicas of our universe, all with different conditions: multiplying the universes makes our more likely, while it sounds almost impossible on its own! This sounds like a very frequentist argument… and I am not even certain it would convince a frequentist. The other articles in this special astrostatistics section were of a more statistical nature, from estimating the number of galaxies to the chances of a big asteroid impact. Even though I found the graphical representation of the meteorite impacts in the past century because of the impact drawing in the background. However, when I checked the link to Carlo Zapponi’s website, I found the picture was a still of a neat animation of meteorites falling since the first report.

Continue reading

on the Piketty craze [not a book review]

Posted in Books, University life with tags , , , , , , on June 15, 2014 by xi'an

The controversy about the data in Piketty’s Capital in the 21st Century fed a dozen articles in the NYT last week, not to mention other newspapers… To the point of Bloomberg producing this parody of a glitzy popular press cover. I have not read the book, nor intend to read it, but I find the buzz and controversies rather surprising. (Thomas Piketty will actually give a public BBC interview in Paris-Dauphine and in English two weeks from now, on June 19.) Indeed, the book sounds to me like a data analysis of extensive (and elaborately processed) datasets rather than the construct of a new economic theory. And I could not see how the criticisms from the Financial Time were anywhere beyond the obvious ideological.  I was thus pleased to read Francis Diebold’s blog on this craze. And to see the repeated acknowledgements that Piketty’s data being made public was a great thing. Although somewhat dismayed that such a painstakenly gathered dataset was not submitted to a more advanced econometric/statistical processing than the one offered by Excel… Definitely looking forward a more [statistically] involved analysis  of the data. (Unsurprisingly, Nassim Taleb reanalysed the whole craze as missing the fat tail features of wealth accumulation. With new aphorisms like “the fastest road to bankruptcy in foreign exchange was an economics degree”.)

bias in estimating bracketed quantile contributions

Posted in Books, R, Statistics, University life with tags , , , , , , on May 16, 2014 by xi'an

“Vilfredo Pareto noticed that 80% of the land in Italy belonged to 20% of the population, and vice-versa, thus both giving birth to the power law class of distributions and the popular saying 80/20.”

Yesterday, in “one of those” coincidences, I voluntarily dropped Nassim Taleb’s The Bed of Procrustes in a suburban café as my latest contribution to the book-crossing (or bXing!) concept and spotted a newly arXived paper by Taleb and Douadi. Paper which full title is “On the Biases and Variability in the Estimation of Concentration Using Bracketed Quantile Contributions” and which central idea is that estimating

\kappa_\alpha = \alpha\mathbb{E}[X|X>q_\alpha]\big/\mathbb{E}[X]

(where qα is the α-level quantile of X) by the ratio

\sum_{i=1}^n \mathbb{I}_{X_i>\hat{q_\alpha}} X_i \big/ \sum_{i=1}^n X_i

can be strongly biased. And that the fatter the tail (i.e. the lower the power β for a power law tail), the worse the bias. This is definitely correct, if not entirely surprising given that the estimating ratio involves a ratio of estimators, plus an estimator of qα. And that both numerator and denominator have finite variances when the power β is less than 2.  The paper contains a simulation experiment easily reproduced by the following R code

#biased estimator of kappa(.01)
alpha=.01 #tail
T=10^4    #simulations
n=10^3    #sample size
beta=1.1  #Pareto parameter

for (t in 1:T){

What is somewhat surprising though is that the paper deems it necessary to run T=10¹² simulations to assess the bias when this bias is already visible in the first digit of κα. Given that the simulation experiment goes as high as n=10⁸, this means the authors simulated 10²⁰ Pareto variables to exhibit a bias a few thousand replicas could have produced. Checking the numerators and denominators in the above collection of ratios also shows that they may take unbelievably large values.)

“…some theories are built based on claims of such `increase’ in inequality, as in Piketti (2014), without taking into account the true nature of κ, and promulgating theories about the `variation’ of inequality without reference to the stochasticity of the estimation—and the lack of consistency of κ across time and sub-units.”

The more relevant questions about this issue of estimating κα are, in my opinion, (a) why this quantity is of enough practical importance to consider its estimation and to seek estimators that would remain robust as the power β varies arbitrarily close to 1; (b) in which sense there is anything more to the phenomenon than the difficulty in estimating β itself;  and (c) what is the efficient asymptotic variance for estimating κα (since there is no particular reason to only consider the most natural estimator). Despite the above quote, that the paper constitutes  a major refutation of Piketty’s Capital in the Twenty-First Century is rather unlikely!

Le Monde puzzle [#852]

Posted in Books, Kids, Statistics with tags , , , , , on February 16, 2014 by xi'an

A number theory Le Monde mathematical puzzle:

Integers n of type A are such that the set {1,…,3n} can be written as the union of n sets of three integers of the form {a,b,a+b}. Integers n of type B are such that the set {1,…,3n} can be written as the union of n sets of three integers of the form {a,b,c} with a+b+c constant. What are the integers of type B? The smallest integer of both type A and type B is 1. What are the next two integers? Is it true that for n of type A, 4n and 4n+1 are of type A?

Again a case when writing a light R code proves out of (my) reach. When n grows, a brute-force search of all partitions quickly gets impossible. So no R solution provided here! (Feel free to suggest one.)

The Feb. 12, 2014, edition of the Sciences&Medicine leaflet is not that exciting either: mostly about medical topics. It however confirmed my solution for the #853 puzzle (44 and 42, esp.), with a tribune of Marco Zito who read the Edge 2014 annual question on What scientific idea is ready for retirement: he was [surprisingly] disappointed that those entries of a few thousand signs lacked validation and were mere speculations… (One of the entries was written by Nassim Taleb on standard deviation, which should be replaced with MAD, which Taleb mistakenly called mean absolute deviation, instead of median absolute deviation. Gerd Gireneizer has another (short) entry against the blind use of p-values, poorly titled as “Scientific Inference Via Statistical Rituals” as it sounds directed against the whole of statistics.) And then a short but interesting article on a large number of French universities cancelling their subscriptions to major journals like Science or Physical Review Letters. As they cannot follow the insane inflation in the prices of journals, thanks to the unrestricted greed of commercial editors (sounds like the Elsevier boycott did not have such an impact on the profession). The article contains this absurd quote from a Science editor who advances that all articles of the journal are freely available one year after their publication. One year is certainly better than two, ten or an infinity of years. Nonetheless, edge research is about what is published now rather than last year. A final mention for a tribune (in the Economics leaflet) complaining about the impact of the arrival of continental “scientific” business students and  mathematicians on the City, which deprived it from bright students from humanities and led to the financial crisis of 2008… Apparently, some are still looking for scapegoats!

RSS conference in Newcastle

Posted in Books, pictures, Running, Statistics, Travel, University life with tags , , , , , , , on September 5, 2013 by xi'an

IMG_1697Although I could not stay at the RSS Annual Conference for the three days, I would have liked to do so, as there were several interesting sessions, from MCMC talks by Axel Finke, Din-Houn Lau, Anthony Lee and Michael Betancourt, to the session on Anti-fragility, the concept produced by Nassim Taleb in his latest book (reviewed before completion by Larry Wasserman). I find it rather surprising that the RSS is dedicating a whole session to this, but the usually anti-statistic stance of Taleb (esp. in The Black Swan) may explain for it (and the equally surprising debate between a “pro-Taleb” and a “pro-Silver”. I will also miss Sharon McGrayne‘s talk on the Bayesian revolution, but look forward to hear it at the Bayes-250 day in Duke next December. And I could have certainly benefited from the training session about building a package in R. It seemed, however, that one-day attendance was a choice made by many participants to the conference, judging from the ability to register for one or two days and from the (biased) sample of my friends.

Incidentally, the conference gave me the opportunity to discover Newcastle and Tynemouth, enjoying the architecture of Grey Street and running on the huge meadows almost at the city centre, among herds of cows in the morning fog. (I wish I had had more time to reach the neighbourly Hadrian wall and Durham, that I only spotted from the train to B’ham!)