Archive for editor

eliminating an important obstacle to creative thinking: statistics…

Posted in Books, Kids, Statistics, University life with tags , , , , , , , , , , , , , on March 12, 2015 by xi'an

“We hope and anticipate that banning the NHSTP will have the effect of increasing the quality of submitted manuscripts by liberating authors from the stultified structure of NHSTP thinking thereby eliminating an important obstacle to creative thinking.”

About a month ago, David Trafimow and Michael Marks, the current editors of the journal Basic and Applied Social Psychology published an editorial banning all null hypothesis significance testing procedures (acronym-ed into the ugly NHSTP which sounds like a particularly nasty venereal disease!) from papers published by the journal. My first reaction was “Great! This will bring more substance to the papers by preventing significance fishing and undisclosed multiple testing! Power to the statisticians!” However, after reading the said editorial, I realised it was inspired by a nihilistic anti-statistical stance, backed by an apparent lack of understanding of the nature of statistical inference, rather than a call for saner and safer statistical practice. The editors most clearly state that inferential statistical procedures are no longer needed to publish in the journal, only “strong descriptive statistics”. Maybe to keep in tune with the “Basic” in the name of the journal!

“In the NHSTP, the problem is in traversing the distance from the probability of the finding, given the null hypothesis, to the probability of the null hypothesis, given the finding. Regarding confidence intervals, the problem is that, for example, a 95% confidence interval does not indicate that the parameter of interest has a 95% probability of being within the interval.”

The above quote could be a motivation for a Bayesian approach to the testing problem, a revolutionary stance for journal editors!, but it only illustrate that the editors wish for a procedure that would eliminate the uncertainty inherent to statistical inference, i.e., to decision making under… erm, uncertainty: “The state of the art remains uncertain.” To fail to separate significance from certainty is fairly appalling from an epistemological perspective and should be a case for impeachment, were any such thing to exist for a journal board. This means the editors cannot distinguish data from parameter and model from reality! Even more fundamentally, to bar statistical procedures from being used in a scientific study is nothing short of reactionary. While encouraging the inclusion of data is a step forward, restricting the validation or in-validation of hypotheses to gazing at descriptive statistics is many steps backward and does completely jeopardize the academic reputation of the journal, which editorial may end up being the last quoted paper. Is deconstruction now reaching psychology journals?! To quote from a critic of this approach, “Thus, the general weaknesses of the deconstructive enterprise become self-justifying. With such an approach I am indeed not sympathetic.” (Searle, 1983).

“The usual problem with Bayesian procedures is that they depend on some sort of Laplacian assumption to generate numbers where none exist (…) With respect to Bayesian procedures, we reserve the right to make case-by-case judgments, and thus Bayesian procedures are neither required nor banned from BASP.”

The section of Bayesian approaches is trying to be sympathetic to the Bayesian paradigm but again reflects upon the poor understanding of the authors. By “Laplacian assumption”, they mean Laplace´s Principle of Indifference, i.e., the use of uniform priors, which is not seriously considered as a sound principle since the mid-1930’s. Except maybe in recent papers of Trafimow. I also love the notion of “generat[ing] numbers when none exist”, as if the prior distribution had to be grounded in some physical reality! Although it is meaningless, it has some poetic value… (Plus, bringing Popper and Fisher to the rescue sounds like shooting Bayes himself in the foot.)  At least, the fact that the editors will consider Bayesian papers in a case-by-case basis indicate they may engage in a subjective Bayesian analysis of each paper rather than using an automated p-value against the 100% rejection bound!

[Note: this entry was suggested by Alexandra Schmidt, current ISBA President, towards an incoming column on this decision of Basic and Applied Social Psychology for the ISBA Bulletin.]

 

down with referees, up with ???

Posted in Books, Statistics, University life, Wines with tags , , , , , , , , , , on April 18, 2012 by xi'an

Statisfaction made me realise I had missed the latest ISBA Bulletin when I read what Julyan posted about Larry’s tribune on a World without referees. While I agree on many of Larry’s points, first and foremost on his criticisms of the refereeing process which seems to worsen and worsen, here are a few items of dissension…

The argument that the system is 350 years old and thus must be replaced may be ok at the rethoretical level, but does not carry any serious weight! First, what is the right scale for a change: 100 years?! 200 years?! Should I burn down my great-grand-mother’s house because it is from the 1800’s and buy a camping-car instead?! Should I smash my 1690 Stradivarius and buy a Fender Stratocaster?! Further, given the intensity and the often under-the-belt level of the Newton vs. Leibniz dispute, maybe refereeing and publishing in the Philosophical Transactions of the Royal Society should have been abolished right away from the start. Anyway, this is about rethoric, not matter. (Same thing about the wine store ellipse. It is not even a good one:  Indeed, when I go to a wine store, I have to rely on (a) well-known brands; (b) brands I have already tried and appreciated; (c) someone else’s advice, like the owner, or friends, or Robert Parker…. In the former case, it can prove great or disastrous. But this is the most usual way to pick wines as one cannot hope [dream?] to sample all wines in the shop.)

My main issue with doing away with referees is the problem of sifting through the chaff. The amount of research documents published everyday is overwhelming. There is a maximal amount of time I can dedicate to looking at websites, blogs, twitter accounts like Scott Sisson’s and Richard Everitt’s, and such. And there clearly is a limited amount of trust I put in the opinions expressed in a blog (e.g., take the ‘Og where this anonymous X’racter writes about everything, mostly non-scientific stuff, and reviews papers with a definite bias!) Even keeping track of new arXiv postings sometimes get overwhelming. So, Larry’s “if you don’t check arXiv for new papers every day, then you are really missing out” means to me that missing arXiv for a few days and I cannot recover. One week away at an intense workshop or on vacations and I am letting some papers going by forever, even though I carry them in my bag for a while…. Noll’s suggestion to publish only on one’s own website is even more unrealistic: why should anyone bother to comment on poor or wrong papers, except when looking for ‘Og’s fodder?! So the fundamental problem is separating the wheat from the chaff, given the amount of chaff and the connected tendency to choke on it! Getting rid of referees and journals to rely on depositories like [the great, terrific, essential] arXiv forces me to also rely on other sources for ranking, selecting, and eliminating papers. Again with a component of arbitrariness, subjectivity, bias, variation, randomness, peer pressure, &tc. In addition, having no prior check of papers means reading a new paper a tremendous chore as one would have to check the references as well, leading to a sort of infinite regress… and forcing one to rely on reputation and peer opinions, once again! And imagine the inflation in reference letters! I already feel I have to write too many reference letters at the moment, but a world without (good and bad) journals would be the Hell of non-stop reference letters. I definitely prefer to referee (except for Elsevier!) and even more being a journal editor, because I can get an idea of the themes in the field and sometimes spot new trends, rather than writing over and over again about an old friend’s research achievements or having to assess from scratch the worth of a younger colleague’s work…

Furthermore, and this is a more general issue, I do not believe that the multiplication of blogs, websites, opinion posts, tribunes, &tc., is necessarily a “much more open, democratic approach”: everyone voicing an opinion on the Internet does not always get listened to and the loudest ones (or most popular ones) are not always the most reliable ones. A complete egalitarian principle means everyone talks/writes and no one listens/reads: I’d rather stick to the principles set by the Philosophical Transactions of the Royal Society!

Anyway, thanks to Larry for launching a worthwhile debate into discovering new ways of making academia a more rational and scientific place!

[super]power to X-edit

Posted in Kids, Statistics, University life with tags , , , on February 22, 2012 by xi'an

Having reached 2000 reputation credits on Cross Validated, I now have the privilege to edit others’ posts:

We believe in the power of community editing. That means once you’ve generated enough reputation, we trust you to edit anything in the system without it going through the peer review system. Not just your posts—anyone’s posts!

Which I already did in the past with superusers’ supervision. Hopefully, this promotion will not induce spending even more time on the forum… And increase my stress at reading often too often questions written by people obviously too lazy to open a stat manual. (I know, I know, no one forces me to read them!)