Archive for variability

visualising bias and unbiasedness

Posted in Books, Kids, pictures, R, Statistics, University life with tags , , , , , , , , , on April 29, 2019 by xi'an

A question on X validated led me to wonder at the point made by Christopher Bishop in his Pattern Recognition and Machine Learning book about the MLE of the Normal variance being biased. As it is illustrated by the above graph that opposes the true and green distribution of the data (made of two points) against the estimated and red distribution. While it is true that the MLE under-estimates the variance on average, the pictures are cartoonist caricatures in their deviance permanence across three replicas. When looking at 10⁵ replicas, rather than three, and at samples of size 10, rather than 2, the distinction between using the MLE (left) and the unbiased estimator of σ² (right).

When looking more specifically at the case n=2, the humongous variability of the density estimate completely dwarfs the bias issue:

Even when averaging over all 10⁵ replications, the difference is hard to spot (and both estimations are more dispersed than the truth!):

statistics in Nature [a tale of the two Steves]

Posted in Books, pictures, Statistics with tags , , , , , , , , , on January 15, 2019 by xi'an

In the 29 November issue of Nature, Stephen Senn (formerly at Glasgow) wrote an article about the pitfalls of personalized medicine, for the statistics behind the reasoning are flawed.

“What I take issue with is the de facto assumption that the differential response to a drug is consistent for each individual, predictable and based on some stable property, such as a yet-to-be-discovered genetic variant.”S. Senn

One (striking) reason being that the studies rest on a sort of low-level determinism that does not account for many sources of variability. Over-confidence in causality results. Stephen argues that improvement lies in insisting on repeated experiments on the same subjects (with an increased challenge in modelling since this requires longitudinal models with dependent observations). And to “drop the use of dichotomies”, favouring instead continuous modeling of measurements.

And in the 6 December issue, Steven Goodman calls (in the World view tribune) for probability statements to be attached as confidence indices to scientific claims. That he takes great pain to distinguish from p-values and links with Bayesian analysis. (Bayesian analysis that Stephen regularly objects to.) While I applaud the call, I am quite pessimistic about the follow-up it will generate, the primary reply being that posterior probabilities can be manipulated as well as p-values. And that Bayesian probabilities are not “real” probabilities (dixit Don Fraser or Deborah Mayo).

Numbers rule your world

Posted in Books, Statistics with tags , , , , , , , , , , , on February 22, 2010 by xi'an

Andrew Gelman gave me a copy of the recent book Numbers rule your world by Kaiser Fung, along with the comment that it was a nice book but not for us. I spend my “lazy Sunday” morning reading the book at the breakfast table and agree with Andrew on his assessment. (waiting for the  incoming blog review!). Numbers rule your world is unlikely to bring enlightment to professional or academic statisticians, but it provides a nice and soft introduction to the use of statistics in everyday’s life, to the point I would encourage my second and third year students to read it. It covers a few topics that are central to Statistics via ten newspaper-ised stories that make for a very light read, but nonetheless make the point. The themes in Numbers rule your world are

  • variability matters more than average, as illustrated by queuing phenomena;
  • correlation is not causation, but is often good enough to uncover patterns, as illustrated by epidemiology and credit scoring;
  • Simpson’s paradox explains for apparent bias in group differences, as illustrated by SAT score differences between black students and white students;
  • false positives and false negatives have different impacts on the error (here comes Bayes theorem!), depending on population sizes and settings, as illustrated by the (great!) case of cheating athletes and polygraph tests (with a reference to Steve Fienberg‘s work);
  • extreme events may exhibit causes, or not, as illustrated by a cheating lottery case (involving Jeff Rosenthal as the expert, not the cheater!) and a series of air crashes.

The overall tone of Numbers rule your world is pleasant and engaging, at the other end of the stylistic spectrum from Taleb’s Black Swan. Fung’s point is obviously the opposite of Taleb‘s: he is showing the reader how well statistical modelling can explain for apparently paradoxical behaviour. Fung is also adopting a very neutral tone, again a major change from Taleb, maybe being even too positive (no the only mention is made of the current housing crisis in the pages Numbers rule your world dedicates to credit scoring comes in the conclusion, pp. 176-7). Now, in terms of novelty, I cannot judge of the amount of innovation when compared with (numerous) other popular science books on the topic. For instance, I think Jeff Rosenthal’s Struck by Lightning brings a rather deeper perspective, but maybe thus restricts the readership further…