Archive for impact factor

Elsevier in the frontline

Posted in Books, Statistics, University life with tags , , , , , , , , , , , , , on January 27, 2017 by xi'an

“Viewed this way, the logo represents, in classical symbolism, the symbiotic relationship between publisher and scholar. The addition of the Non Solus inscription reinforces the message that publishers, like the elm tree, are needed to provide sturdy support for scholars, just as surely as scholars, the vine, are needed to produce fruit. Publishers and scholars cannot do it alone. They need each other. This remains as apt a representation of the relationship between Elsevier and its authors today – neither dependent, nor independent, but interdependent.”

There were two items of news related with the publishark Elsevier in the latest issue of Nature I read. One was that Germany, Peru, and Taiwan had no longer access to Elsevier journals, after negotiations or funding stopped. Meaning the scientists there have to find alternative ways to procure the papers, from the authors’ webpage [I do not get why authors fail to provide their papers through their publication webpage!] to peer-to-peer platforms like Sci-Hub. Beyond this short term solution, I hope this pushes for the development of arXiv-based journals, like Gower’s Discrete Analysis. Actually, we [statisticians] should start planing a Statistics version of it!

The second item is about  Elsevier developing its own impact factor index, CiteScore. While I do not deem the competition any more relevant for assessing research “worth”, seeing a publishark developing its own metrics sounds about as appropriate as Breithart News starting an ethical index for fake news. I checked the assessment of Series B on that platform, which returns the journal as ranking third, with the surprising inclusion of the Annual Review of Statistics and its Application [sic], a review journal that only started two years ago, of Annals of Mathematics, which does not seem to pertain to the category of Statistics, Probability, and Uncertainty, and of Statistics Surveys, an IMS review journal that started in 2009 (of which I was blissfully unaware). And the article in Nature points out that, “scientists at the Eigenfactor project, a research group at the University of Washington, published a preliminary calculation finding that Elsevier’s portfolio of journals gains a 25% boost relative to others if CiteScore is used instead of the JIF“. Not particularly surprising, eh?!

When looking for an illustration of this post, I came upon the hilarious quote given at the top: I particularly enjoy the newspeak reversal between the tree and the vine,  the parasite publishark becoming the support and the academics the (invasive) vine… Just brilliant! (As a last note, the same issue of Nature mentions New Zealand aiming at getting rid of all invasive predators: I wonder if publishing predators are also included!)

a discovery that mean can be impacted by extreme values

Posted in University life with tags , , , , , , on August 6, 2016 by xi'an

A surprising editorial in Nature about the misleading uses of impact factors, since as means they are heavily impacted by extreme values. With the realisation that the mean is not the median for skewed distributions…

To be fair(er), Nature published a subsequent paper this week about publishing additional metrics like the two-year median.

statistical modelling of citation exchange between statistics journals

Posted in Books, Statistics, University life with tags , , , , , on April 10, 2015 by xi'an

Cristiano Varin, Manuela Cattelan and David Firth (Warwick) have written a paper on the statistical analysis of citations and index factors, paper that is going to be Read at the Royal Statistical Society next May the 13th. And hence is completely open to contributed discussions. Now, I have written several entries on the ‘Og about the limited trust I set to citation indicators, as well as about the abuse made of those. However I do not think I will contribute to the discussion as my reservations are about the whole bibliometrics excesses and not about the methodology used in the paper.

The paper builds several models on the citation data provided by the “Web of Science” compiled by Thompson Reuters. The focus is on 47 Statistics journals, with a citation horizon of ten years, which is much more reasonable than the two years in the regular impact factor. A first feature of interest in the descriptive analysis of the data is that all journals have a majority of citations from and to journals outside statistics or at least outside the list. Which I find quite surprising. The authors also build a cluster based on the exchange of citations, resulting in rather predictable clusters, even though JCGS and Statistics and Computing escape the computational cluster to end up in theory and methods along Annals of Statistics and JRSS Series B.

In addition to the unsavoury impact factor, a ranking method discussed in the paper is the eigenfactor score that starts with a Markov exploration of articles by going at random to one of the papers in the reference list and so on. (Which shares drawbacks with the impact factor, e.g., in that it does not account for the good or bad reason the paper is cited.) Most methods produce the Big Four at the top, with Series B ranked #1, and Communications in Statistics A and B at the bottom, along with Journal of Applied Statistics. Again, rather anticlimactic.

The major modelling input is based on Stephen Stigler’s model, a generalised linear model on the log-odds of cross citations. The Big Four once again receive high scores, with Series B still much ahead. (The authors later question the bias due to the Read Paper effect, but cannot easily evaluate this impact. While some Read Papers like Spiegelhalter et al. 2002 DIC do generate enormous citation traffic, to the point of getting re-read!, other journals also contain discussion papers. And are free to include an on-line contributed discussion section if they wish.) Using an extra ranking lasso step does not change things.

In order to check the relevance of such rankings, the authors also look at the connection with the conclusions of the (UK) 2008 Research Assessment Exercise. They conclude that the normalised eigenfactor score and Stigler model are more correlated with the RAE ranking than the other indicators.  Which means either that the scores are good predictors or that the RAE panel relied too heavily on bibliometrics! The more global conclusion is that clusters of journals or researchers have very close indicators, hence that ranking should be conducted with more caution that it is currently. And, more importantly, that reverting the indices from journals to researchers has no validation and little information.

Series B reaches 5.721 impact factor!

Posted in Books, Statistics, University life with tags , , , on September 15, 2014 by xi'an

I received this email from Wiley with the great figure that JRSS Series B has now reached a 5.721 impact factor. Which makes it the first journal in Statistics from this perspective. Congrats to editors Gareth Roberts, Piotr Fryzlewicz and Ingrid Van Keilegom for this achievement! An amazing jump from the 2009 figure of 2.84…!

news from Elsevier

Posted in Books, Statistics, University life with tags , on July 4, 2012 by xi'an

Here is an email I got today from Elsevier:

We are pleased to present the latest Impact Factors for Elsevier’s Mathematics and Statistics journals.

Statistics and Probability Letters

0.498

Journal of Statistical Planning and Inference

0.716

Journal of Multivariate Analysis

0.879

Computational Statistics and Data Analysis

1.028

So there are very few journals published by Elsevier in the statistics field, which may explain for the lack of strong support for the boycott launched by Tim Gowers and others. Also, the impact factors are not that great either. Not so suprising for Statistics and Probability Letters, given that they publish a high number of papers of uneven quality, but also gets a minimal 1. So it does not make too much sense for Elsevier to flout such data. (Once again, impact factors should not be used for assessing the quality of a journal and even less of a paper!)

epidemiology in Le Monde

Posted in Books, Statistics, University life with tags , , , , , , , , , on February 19, 2012 by xi'an

Quite an interesting weekend Le Monde issue: a fourth (2 pages!) of the science folder is devoted to epidemiology… In the statistical sense. (The subtitle is actually Strengths and limitations of Statistics.) The paper does not delve into technical statistical issues but points out the logical divergence between a case-by-case study and an epidemiological study. The impression that the higher the conditioning (i.e. the more covariates), the better the explanation is a statistical fallacy some of the opponents interviewed in the paper do not grasp. (Which reminded me of Keynes seemingly going the same way.) The short paragraph written on causality and Hill’s criteria is vague enough to concur to the overall remark that causality can never been proved or disproved… The fourth examples illustrating the strengths and limitations are tobacco vs. lung cancer, a clear case except for R.A. Fisher!, mobile phones vs. brain tumors, a not yet conclusive setting, hepatitis B vaccine vs. sclerosis, lacking data (the pre-2006 records were destroyed for legal reasons), and leukemia vs. nuclear plants, with a significant [?!] correlation between the number of cases and the distance to a nuclear plant. (The paper was inspired by a report recently published by the French Académie de Médecine on epidemiology in France.) The science folder also includes a review of a recent Science paper by Wilhite and Fong on the coercive strategies used by some journals/editors to increase their impact factor, e.g., “you cite Leukemia [once in 42 references]. Consequently, we kindly ask you to add references of articles published in Leukemia to your present article”.

Citation abuses

Posted in Statistics with tags , , , , , , on October 21, 2009 by xi'an

“There is a belief that citation statistics are
inherently more accurate because they
substitute simple numbers for complex
judgments, and hence overcome the
possible subjectivity of peer review.
But this belief is unfounded.”

A very interesting report appeared in the latest issue of Statistical Science about bibliometrics and its abuses (or “bibliometrics as an abuse per se”!). It was commissioned by the IMS, the IMU and the ICIAM. Along with the set of comments (by Bernard Silverman, David Spiegelhalter, Peter Hall and others) also posted in arXiv, it is a must-read!

“even a casual inspection of the h-index and its variants shows
that these are naïve attempts to understand complicated citation
records. While they capture a small amount of information about
the distribution of a scientist’s citations, they lose crucial
information that is essential for the assessment of research.”

The issue is not gratuitous. While having Series B ranked with a high impact factor is an indicator of the relevance of a majority of papers published in the journal, there are deeper and more important issues at stake. Our grant allocations, our promotions, our salary are more and more dependent on these  “objective” summary or “comprehensive” factors. The misuse of bibliometrics stems from government bodies and other funding agencies wishing to come up with assessments of the quality of a researcher that bypass peer reviews and, more to the point, are easy to come by.

The report points out the many shortcomings of journal impact factors. Its two-year horizon is very short-sighted in mathematics and statistics. As an average, it is strongly influenced by outliers, like controversial papers or broad surveys, as shown by the yearly variations of the thing. Commercial productions like Thomson’s misses a large part of the journals that could quote a given paper and this is particularly true for fields at the interface between disciplines and for emergent topics. The variation in magnitude between disciplines is enormous and based on the impact factor I’d rather publish one paper in Bioinformatics than four in the Annals of Statistics… The second issue is that the “quality” of the journal does not automatically extend to all papers it publishes: multiplying papers by the journal impact factor is thus ignoring variation to an immense extent. The report illustrates this with the fact that a paper published in a journal with half the impact factor of another journal has a 62% probability to be more quoted than if it had been published in this other journal! The h-factor is similarly criticised by the report.  More fundamentally, the report also analyses the multicriteria nature of citations, which cannot be reflected (only) as a measure of worth of the quoted papers.