Archive for refereeing

Nature on predatory journals

Posted in Books, University life with tags , , , , on January 24, 2020 by xi'an

A (long) comment published in Nature this week studies the impact of predatory journals, with a definition (made by the 32 authors of the comment and 9 others at a special meeting in Ottawa) of what constitutes a predatory journal.

Predatory journals and publishers are entities that prioritize self-interest at the expense of scholarship and are characterized by false or misleading information, deviation from best editorial and publication practices, a lack of transparency, and/or the use of aggressive and indiscriminate solicitation practices.”

The article discusses each term in the definition, terms that remain vague (like what’s a “deviation from best”? A terrible website? May be due to English not being the first language of the journal editors…). In my opinion, the main criterion is the “aggressive and indiscriminate solicitation”  and a lack of actual peer review, which can be easily detected when the paper is accepted within a very short time period. (Which is not to state that journals with a very quick rejection time should be a priori considered as predatory!) High publishing fees are certainly part of the predatory landscape but difficult to detect from established journals, even those backed by national or international societies. My only experience with predatory “publishers”, beyond the constant flow of proposals to send a paper, to edit a special edition and so forth, is a paper sent to a journal with the same title as a regular (Elsevier) journal, modulo a permutation!, and a threat of legal action from another source, which I described as “predatory” for proposing to write a general public paper in their glossy magazine. For the first occurrence, the paper was accepted within a day, we never signed any copyright form, and despite requests to withdraw the paper, it almost immediately got published. Even though we never paid the requested fees.

“Efforts to counter predatory publishing need to be constant and adaptable. The threat is unlikely to disappear as long as universities use how many publications a scholar has produced as a criterion for graduation or career advancement. The publish-or-perish culture, a lack of awareness of predatory publishing and difficulty in discerning legitimate from illegitimate publications fosters an environment for predatory publications to exist. Predatory journals are also quick to adapt to policies and measures designed to foil them.”

It certainly feels impossible to completely counter predatory actions, especially when some researchers seek such publications, but detecting some could be achieved by sending decoys to them, in the form of low-content pseudo-articles that could not pass any serious assessment by a genuine referee. Because no publication is intended, the same decoy could be used over and over by the society initiating the action…

No review this summer

Posted in Books, Statistics, University life with tags , , , , , , , , on September 19, 2019 by xi'an

A recent editorial in Nature was a declaration by a biologist from UCL on her refusal to accept refereeing requests during the summer (or was it the summer break), which was motivated by a need to reconnect with her son. Which is a good enough reason (!), but reflects sadly on the increasing pressure on one’s schedule to juggle teaching, research, administration, grant hunting, society service, along with a balanced enough family life. (Although I have been rather privileged in this regard!) Given that refereeing or journal editing is neither visible nor rewarded, it comes as the first task to be postponed or abandoned, even though most of us realise it is essential to keep science working as a whole and to make our own papers published. I have actually noticed an increasing difficulty in the past decade to get (good) referees to accept new reviews, often asking for deadlines that are hurting the authors, like six months. Making them practically unavailable. As I mentioned earlier on this blog, it could be that publishing referees’ reports as discussions would help, since they would become recognised as (unreviewed!) publications, but it is unclear this is the solution. If judging from the similar difficulty in getting discussions for discussed papers. (As an aside, there are two exciting papers coming up for discussion in Series B, ‘Unbiased Markov chain Monte Carlo methods with couplings’ by  Pierre E. Jacob, John O’Leary and Yves F. Atchadé and in Bayesian Analysis, Latent nested nonparametric priors by Frederico Camerlenghi, David Dunson, Antonio Lijoi, Igor Prünster, and Abel Rodríguez). Which is surprising when considering the willingness of a part of the community to engage into forii discussions, sometimes of a considerable length as illustrated on Andrew’s blog.

Another entry in Nature mentioned the case of two University of København tenured professors in geology who were fired for either using a private email address (?!) or being away on field work during an exam and at a conference without permission from the administration. Which does not even remotely sound like a faulty behaviour to me or else I would have been fired eons ago..!

open reviews

Posted in Statistics with tags , , , , , , on September 13, 2019 by xi'an

When looking at a question on X validated, on the expected Metropolis-Hastings ratio being one (not all the time!), I was somewhat bemused at the OP linking to an anonymised paper under review for ICLR, as I thought this was breaching standard confidentiality rules for reviews. Digging a wee bit deeper, I realised this was a paper from the previous ICLR conference, already published both on arXiv and in the 2018 conference proceedings, and that ICLR was actually resorting to an open review policy where both papers and reviews were available and even better where anyone could comment on the paper while it was under review. And after. Which I think is a great idea, the worst possible situation being a poor paper remaining un-discussed. While I am not a big fan of the brutalist approach of many machine-learning conferences, where the restrictive format of both submissions and reviews is essentially preventing in-depth reviews, this feature should be added to statistics journal webpages (until PCIs become the norm).

peer reviews on-line or peer community?

Posted in Statistics with tags , , , , , , , , , on September 20, 2018 by xi'an

Nature (or more precisely some researchers through Nature, associated with the UK Wellcome Trust, the US Howard Hughes Medical Institute (hhmo), and ASAPbio) has (have) launched a call for publishing reviews next to accept papers, one way or another, which is something I (and many others) have supported for quite a while. Including for rejected papers, not only because making these reviews public diminishes on principle the time involved in re-reviewing re-submitted papers but also because this should induce authors to revise papers with obvious flaws and missing references (?). Or abstain from re-submitting. Or publish a rejoinder addressing the criticisms. Anything that increases the communication between all parties, as well as the perspectives on a given paper. (This year, NIPS allows for the posting of reviews of rejected submissions, which I find a positive trend!)

In connection with this entry, I am still most sorry that I could not pursue the [superior in my opinion] project of Peer Community in computational statistics, for the time requested by Biometrika editing is just too important [given my current stamina!] for me to handle another journal (or the better alternative to a journal!). I hope someone else can take over the project and create the editorial team needed to run it.

And yet again in connection with this post (!), Andrew posted an announcement about the launch of res3archers.one, an on-line publication forum launched by Harry Crane and Ryan Martin, where the authors handle the peer review process from A to Z, including choosing the reviewers, whose reviews may be public or not, taken into account or not. Once published, the papers are open to comments from users, which constitutes a form of post-publication peer-review. Albeit a weak one in my opinion as the weakness of all such open depositories is the potential lack of interest of and reaction from the community. Incidentally, there is a $10 fee per submission for maintenance. Contrary to Peer Community in… the copyright is partly transferred to res3archers.one, which apparently prevents further publication in another journal.

running ABC when the likelihood is available

Posted in Statistics with tags , , , , , on September 19, 2017 by xi'an

Today I refereed a paper where the authors used ABC to bypass convergence (and implementation) difficulties with their MCMC algorithm. And I am still pondering whether or not this strategy makes sense. If only because ABC needs to handle the same complexity and the same amount of parameters as an MCMC algorithm. While shooting “in the dark” by using the prior or a coarse substitute to the posterior. And I wonder at the relevance of simulating new data when the [true] likelihood value [at the observed data] can be computed. This would sound to me like the relevant and unique “statistics” worth considering…

crowd-based peer review

Posted in Statistics with tags , , , , , , , , , , on June 20, 2017 by xi'an

In clear connection with my earlier post on Peer Community In… and my visit this week to Montpellier towards starting a Peer Community In Computational Statistics, I read a tribune in Nature (1 June, p.9) by the editor of Synlett, Benjamin List, describing an experiment conducted by this journal in chemical synthesis. The approach was to post (volunteered) submitted papers on a platform accessible to a list of 100 reviewers, nominated by the editorial board, who could anonymously comment on the papers and read others’ equally anonymous comments. With a 72 hours deadline! According to Benjamin List (and based on  a large dataset of … 10 papers!), the outcome of the experiment is one of better quality return than with traditional reviewing policies. While Peer Community In… does not work exactly this way, and does not aim at operating as a journal, it is exciting and encouraging to see such experiments unfold!

peer community in evolutionary biology

Posted in Statistics with tags , , , , , , , on May 18, 2017 by xi'an

My friends (and co-authors) from Montpellier pointed out the existence of PCI Evolutionary Biology, which is a preprint and postprint validation forum [so far only] in the field of Evolutionary Biology. Authors of a preprint or of a published paper request a recommendation from the forum. If someone from the board finds the paper of interest, this person initiates a quick refereeing process with one or two referees and returns a review to the authors, with possible requests for modification, and if the lead reviewer is happy with the new version, the link to the paper and the reviews are published on PCI Evol Biol, which thus gives a stamp of validation to the contents in the paper. The paper can then be submitted for publication in any journal, as can be seen from the papers in the list.

This sounds like a great initiative and since PCI is calling for little brothers and sisters to PCI Evol Biol, I think we should try to build its equivalent in Statistics or maybe just Computational Statistics.