cynical trials

against method

A vitriolic tribune in Le Monde this weekend by the microbologist Didier Raoult, head of the Institut Hospitalo-Universitaire Méditerranée Marseille, campaining for the immediate use of the hydroxychloroquine sulfate drug for coronavirus patients. Arguing that the major emergency of the coronavirus pandemic calls for this use without waiting for confirmation from clinical trials, without controlled comparison with other drugs or a placebo. Solely a study of patients being administered the drug, outside the usual practice of clinical trials.

“Enfin, l’envahissement des méthodologistes amène à avoir des reflexions purement mathématiques.” [At last, the invasion of methodologists leads to purely mathematical reasonings.]

“Ce modèle, qui a nourri une quantité de méthodologistes, est devenu une dictature morale.” [This model, which has fed quantity of methodologists, has become a moral dictatorship.]

“…il faut nous débarasser des mathématiciens, des metéorologistes [sic] dans ce domaine.” [we must get rid of mathematicians and meteorologists in this domain]

“…conseil scientifique dans lequel on trouvait deux modélisateurs de l’avenir (qui pour moi représentent l’équivalent de l’astrologie), des maniaques de la méthodologie. Les médecins confrontés au problème du soin représentaient une minorité qui n’avait pas nécessairement l’habitude de s’exprimer et qui se trouvait noyée par cet habillage pseudo-scientifique.” […scientific committee including two modelisators of the future (equivalent in my opinion to astrologers), manics of methodology. Physicians facing treatment problems were a minority not necessarily used to intervene, overwhelmed by this pseudo-scientific babbling.]

Obviously I have no expertise in drug development or even in epidemiology, but the name-calling tone of this tribune, as illustrated by the above quotes, is appalling and populist, more in the spirit of Trump than of a rational search for scientific evidence. On the opposite, the arguments therein are a-scientific and reject the use of mathematical and statistical methodology for being… mathematical. And resort to name-calling, while not considering the more than philosophical aspect that opting for this drug rather than another one may be reducing survival chances for some groups of patients. (While the title chosen for this post reflects the title of Raoult’s tribune, with its philosophical pretenses, let me stress that Feyerabend’s book is not mentioned therein and that the article contains no indication that the author relates to Feyerabend’s views.)

statistics in Nature [a tale of the two Steves]

In the 29 November issue of Nature, Stephen Senn (formerly at Glasgow) wrote an article about the pitfalls of personalized medicine, for the statistics behind the reasoning are flawed.

“What I take issue with is the de facto assumption that the differential response to a drug is consistent for each individual, predictable and based on some stable property, such as a yet-to-be-discovered genetic variant.”S. Senn

One (striking) reason being that the studies rest on a sort of low-level determinism that does not account for many sources of variability. Over-confidence in causality results. Stephen argues that improvement lies in insisting on repeated experiments on the same subjects (with an increased challenge in modelling since this requires longitudinal models with dependent observations). And to “drop the use of dichotomies”, favouring instead continuous modeling of measurements.

And in the 6 December issue, Steven Goodman calls (in the World view tribune) for probability statements to be attached as confidence indices to scientific claims. That he takes great pain to distinguish from p-values and links with Bayesian analysis. (Bayesian analysis that Stephen regularly objects to.) While I applaud the call, I am quite pessimistic about the follow-up it will generate, the primary reply being that posterior probabilities can be manipulated as well as p-values. And that Bayesian probabilities are not “real” probabilities (dixit Don Fraser or Deborah Mayo).

no publication without confirmation

“Our proposal is a new type of paper for animal studies (…) that incorporates an independent, statistically rigorous confirmation of a researcher’s central hypothesis.” (p.409)

A comment tribune in Nature of Feb 23, 2017, suggests running clinical trials in three stages towards meeting higher standards in statistical validation. The idea is to impose a preclinical trial run by an independent team following an initial research showing some potential for some new treatment. The three stages are thus (i) to generate hypotheses; (ii) to test hypotheses; (iii) to test broader application of hypotheses (p.410). While I am skeptical of the chances of this proposal reaching adoption (for various reasons, like, what would the incentive of the second team be [of the B team be?!], especially if the hypothesis is dis-proved, how would both teams share the authorship and presumably patenting rights of the final study?, and how could independence be certain were the B team contracted by the A team?), the statistical arguments put forward in the tribune are rather weak (in my opinion). Repeating experiments with a larger sample size and an hypothesis set a priori rather than cherry-picked is obviously positive, but moving from a p-value boundary of 0.05 to one of 0.01 and to a power of 80% is more a cosmetic than a foundational change. As Andrew and I pointed out in our PNAS discussion of Johnson two years ago.

“the earlier experiments would not need to be held to the same rigid standards.” (p.410)

The article contains a vignette on “the maths of predictive value” that makes intuitive sense but only superficially. First, “the positive predictive value is the probability that a positive result is truly positive” (p.411) A statement that implies a distribution of probability on the space of hypotheses, although I see no Bayesian hint throughout the paper. Second, this (ersatz of a) probability is computed by a ratio of the number of positive results under the hypothesis over the total number of positive results. Which does not make much sense outside a Bayesian framework and even then cannot be assessed experimentally or by simulation without defining a distribution of the output under both hypotheses. Simplistic pictures are the above are not necessarily meaningful. And Nature should certainly invest into a statistical editor!

le bayésianisme aujourd’hui [book review]

It is quite rare to see a book published in French about Bayesian statistics and even rarer to find one that connects philosophy of science, foundations of probability, statistics, and applications in neurosciences and artificial intelligence. Le bayésianisme aujourd’hui (Bayesianism today) was edited by Isabelle Drouet, a Reader in Philosophy at La Sorbonne. And includes a chapter of mine on the basics of Bayesian inference (à la Bayesian Choice), written in French like the rest of the book.

The title of the book is rather surprising (to me) as I had never heard the term Bayesianism mentioned before. As shown by this link, the term apparently exists. (Even though I dislike the sound of it!) The notion is one of a probabilistic structure of knowledge and learning, à la Poincaré. As described in the beginning of the book. But I fear the arguments minimising the subjectivity of the Bayesian approach should not be advanced, following my new stance on the relativity of probabilistic statements, if only because they are defensive and open the path all too easily to counterarguments. Similarly, the argument according to which the “Big Data” era makesp the impact of the prior negligible and paradoxically justifies the use of Bayesian methods is limited to the case of little Big Data, i.e., when the observations are more or less iid with a limited number of parameters. Not when the number of parameters explodes. Another set of arguments that I find both more modern and compelling [for being modern is not necessarily a plus!] is the ease with which the Bayesian framework allows for integrative and cooperative learning. Along with its ultimate modularity, since each component of the learning mechanism can be extracted and replaced with an alternative.