Nature snapshots

In this 6 June issue of Nature, which I read on my way to O’Bayes, an editorial on the scary move by the WHO to incorporate traditional Chinese medicine remedies in its classification as this includes drugs made from protected and endangered species and as such remedies have not been evidence tested. A news brief on India abandoning the requirement for PhD students to get a paper published prior to been awarded the degree, presumably much to the sorrow of predatory publishers. A delay to Plan S (a European project to make all funded research freely available) reported to 21 January 2021. A review of the latest and yet unpublished book by Neal Stephenson, Fall. Which I obviously ordered immediately! A paper in the British Journal of Anasthesia published along with an independent assessment of the same study (methods and results). Some letters protesting the “public’s phobia” induced by the series Chernobyl. Which recoups an email from one of my colleagues on the same complaining theme, since “only 20 deaths” can be attributed to the disaster with certainty! A revisit of the “cold fusion” with no evidence of the claimed phenomenon that led to a scientific outcry in 1989.

natural LaTeX

Nature must have been out of inspiration in the past weeks for running a two-page article on L^aT_eX [as the toolbox column] and how it compares with… Word! Which is not so obvious since most articles in Nature are not involving equations (much) and are from fields where Word prevails. Besides the long-running whine that LaTeX is not he selling argument for this article seemed to be the increasing facility to use (basic) LaTeX commands in forums (like Stack Exchange) and blogs (like WordPress) via MathJax. But the author also pushes for the lighter (R?)Markdown as, “in LaTeX, there is a greater risk that contributors will make changes that prevent the code compiling into a PDF” (which does not make sense to me). This tribune also got me to find out that there is a blog dedicated to the “LaTeX fetish”, which sounds to me like an perfect illustration of Internet vigilantism, especially with arguments like “free and open source software has a strong tendency towards being difficult to install and get up and running”.

aftermaths of retiring significance

Beyond mentions in the general press of the retire significance paper, as in Retraction Watch, Bloomberg, The Guardian, Vox, and NPR, not to mention the large number of comments on Andrew’s blog, and Deborah Mayo’s tribune on a ban on free speech (!), Nature of “the week after” contained three letters from Ioannidis, calling for more stringent thresholds, Johnson, essentially if unclearly stating the same, and my friends from Amsterdam, Alexander Ly and E.J. Wagenmakers, along with Julia Haaf, getting back to the Great Old Ones, to defend the usefulness of testing versus estimation.

the joy of stats [book review]

David Spiegelhalter‘s latest book, The Art of Statistics: How to Learn from Data, has made it to Nature Book Review main entry this week. Under the title “the joy of stats”,  written by Evelyn Lamb, a freelance math and science writer from Salt Lake City, Utah. (I noticed that the book made it to Amazon #1 bestseller, albeit in the Craps category!, which I am unsure is completely adequate!, especially since the book is not yet for sale on the US branch of Amazon!, and further Amazon #1 in the Probability and Statistics category in the UK.) I have not read the book yet and here are a few excerpts from the review, quoted verbatim:

“The book is part of a trend in statistics education towards emphasizing conceptual understanding rather than computational fluency. Statistics software can now perform a battery of tests and crunch any measure from large data sets in the blink of an eye. Thus, being able to compute the standard deviation of a sample the long way is seen as less essential than understanding how to design and interpret scientific studies with a rigorous eye.”

“…a main takeaway from the book is a sense of circumspection about our confidence in what is known. As Spiegelhalter writes, the point of statistical science is to ease us through the stages of extrapolation from a controlled study to an understanding of the real world, `and finally, with due humility, be able to say what we can and cannot learn from data’. That humility can be lacking when statistics are used in debates about contentious issues such as the costs and benefits of cancer screening.

abandon ship [value]!!!

The Abandon Statistical Significance paper we wrote with Blakeley B. McShane, David Gal, Andrew Gelman, and Jennifer L. Tackett has now appeared in a special issue of The American Statistician, “Statistical Inference in the 21st Century: A World Beyond p < 0.05“.  A 400 page special issue with 43 papers available on-line and open-source! Food for thought likely to be discussed further here (and elsewhere). The paper and the ideas within have been discussed quite a lot on Andrew’s blog and I will not repeat them here, simply quoting from the conclusion of the paper

In this article, we have proposed to abandon statistical significance and offered recommendations for how this can be implemented in the scientific publication process as well as in statistical decision making more broadly. We reiterate that we have no desire to “ban” p-values or other purely statistical measures. Rather, we believe that such measures should not be thresholded and that, thresholded or not, they should not take priority over the currently subordinate factors.

Which also introduced in a comment by Valentin Amrhein, Sander Greenland, and Blake McShane published in Nature today (and supported by 800+ signatures). Again discussed on Andrew’s blog.

Nature tea[dbits]

A very special issue of Nature (7 February 2019, vol. 556, no. 7742). With an outlook section on tea, plus a few research papers (and ads) on my principal beverage. News about the REF, Elsevier’s and Huawei’s woes with the University of California, the dangerous weakening of Title IX by the Trump administration, and a long report on the statistical analysis of Hurricane Maria deaths, involving mostly epidemiologists, but also Patrick Ball who took part in our Bayes for Good workshop at CIRM. Plus China’s food crisis and ways to reduce cropland losses and food waste. Concerning the tea part(y), a philogenetic study of different samples led to the theory that tea was domesticated thrice, twice in Yunnan (China) and once in Assam (India), with a divergence estimated at more than twenty thousand years ago. Another article on Pu-Ehr, with the potential impacts of climate change on this very unique tea. With a further remark that higher altitudes increase the anti-oxydant level of tea… And a fascinating description of agro-forestry where tea and vegetables are grown in a forest that regulates sun exposure, moisture evaporation, and soil nutrients.

how a hiring quota failed [or not]

This week, Nature has a “career news” section dedicated to how hiring quotas [may have] failed for French university hiring. And based solely on a technical report by a Science Po’ Paris researcher. The hiring quota means that every hiring committee for a French public university hiring committee must be made of at least 40% members of each gender.  (Plus at least 50% of external members.) Which has been reduced to 30% in some severely imbalanced fields like mathematics. The main conclusion of the report is that the reform has had a negative impact on the hiring imbalance between men and women in French universities, with “the higher the share of women in a committee, the lower women are ranked” (p.2). As head of the hiring board in maths at Dauphine, which officiates as a secretarial committee for assembling all hiring committee, I was interested in the reasons for this perceived impact, as I had not observed it at my [first order remote] level. As a warning the discussion that follows makes little sense without a prior glance at the paper.

“Deschamps estimated that without the reform, 21 men and 12 women would have been hired in the field of mathematics. But with the reform, committees whose membership met the quota hired 30 men and 3 women” Nature,  07 February 2019

Skipping the non-quantitative and somewhat ideological part of the report, as well as descriptive statistics, I looked mostly at the modelling behind the conclusions, as reported for instance in the above definite statement in Nature. Starting with a collection of assumptions and simplifications. A first dubious such assumption is that fields and even less universities where the more than 40% quota was already existing before (the 2015 reform) could be used as “control groups”, given the huge potential for confounders, especially the huge imbalance in female-to-male ratios in diverse fields. Second, the data only covers hiring histories for three French universities (out of 63 total) over the years 2009-2018 and furthermore merges assistant (Maître de Conférence) and full professors, where hiring is de facto much more involved, with often one candidate being contacted [prior to the official advertising of the position] by the department as an expression of interest (or the reverse). Third, the remark that

“there are no significant differences between the percentage of women who apply and those who are hired” (p.9)

seems to make the all discussion moot… and contradict both the conclusion and the above assertion! Fourth, the candidate’s qualification (or quality) is equated with the h-index, which is highly reductive and, once again, open to considerable biases in terms of seniority degree and of field. Depending on the publication lag and also the percentage of publications in English versus the vernacular in the given field. And the type of publications (from an average of 2.94 in business to 9.96 on physics]. Fifth, the report equates academic connections [that may bias the ranking] with having the supervisor present in the hiring committee [which sounds like a clear conflict of interest] or the candidate applying in the [same] university that delivered his or her PhD. Missing a myriad of other connections that make committee members often prone to impact the ranking by reporting facts from outside the application form.

“…controlling for field fixed effects and connections make the coefficient [of the percentage of women in the committee] statistically insignificant, though the point estimate remains high.” (p.17)

The models used by Pierre Deschamps are multivariate logit and probit regressions, where each jury attaches a utility to each of its candidates, made of a qualification term [for the position] and of a gender bias most surprisingly multiplying candidate gender and jury gender dummies. The qualification term is expressed as a [jury free] linear regression on covariates plus a jury fixed effect. Plus an error distributed as a Gumbel extreme variate that leads to a closed-form likelihood [and this seems to be the only reason for picking this highly skewed distribution]. The probit model is used to model the probability that one candidate has a better utility than another. The main issue with this modelling is the agglomeration of independence assumptions, as (i) candidates and hired ones are not independent, from being evaluated over several positions all at once, with earlier selections and rankings all public, to having to rank themselves all the positions where they are eligible, to possibly being co-authors of other candidates; (ii) jurys are not independent either, as the limited pool of external members, esp. in gender-imbalanced fields, means that the same faculty often ends up in several jurys at once and hence evaluates the same candidates as a result, plus decides on local ranking in connection with earlier rankings; (iii) independence between several jurys of the same university when this university may try to impose a certain if unofficial gender quota, a variate obviously impossible to fill . Plus again a unique modelling across disciplines. A side but not solely technical remark is that among the covariates used to predict ranking or first position for a female candidate, the percentage of female candidates appears, while being exogenous. Again, using a univariate probit to predict the probability that a candidate is ranked first ignores the comparison between a dozen candidates, both male and female, operated by the jury. Overall, I find little reason to give (significant) weight to the indicator that the president is a woman in the logistic regression and even less to believe that a better gender balance in the jurys has led to a worse gender balance in the hirings. From one model to the next the coefficients change from being significant to non-significant and, again, I find the definition of the control group fairly crude and unsatisfactory, if only because jurys move from one session to the next (and there is little reason to believe one field more gender biased than another, with everything else accounted for). And for another my own experience within hiring committees in Dauphine or elsewhere has never been one where the president strongly impacts the decision. If anything, the president is often more neutral (and never ever imoe makes use of the additional vote to break ties!)…