New APPS: Art, Politics, Philosophy, Science

Recently discovered: 100+ computer-generated papers. Relation to spamference?

On the Nature website, Richard Van Noorden reports that a French computer scientist, Cyril Labbé, has discovered over 120 computer-generated papers that have been published in conference proceedings between 2008 and 2013. Over 100 of these papers were published by the Institute of Electrical and Electronic Engineers (IEEE) and 16 others appeared in Springer publications.

The papers have been composed using SCIgen, which only requires the user to input author names, and automatically generates random papers that look like Computer Science, but which are actually meaningless. Cyril Labbé has written a program that is able to recognize papers that have been generated by SCIgen. (The program compares the vocabulary of a text to that of a reference corpus; in particular, it measures the inter-textual distance as the proportion of word-tokens shared by two texts. For details of the method, see Labbé's 2012 paper published in Scientometrics.)

The proceedings issues that appeared in Springer publications were (supposed to be) peer-reviewed; for the IEEE proceedings, it is less clear whether they underwent peer review. In any case, the former examples show that the peer review system is not always watertight, not just in the case of open-access journals (which was also discussed here at NewApps).

Most of the conferences took place in China and most of the authors have Chinese affiliations. Of course, it remains to be checked whether the author names correspond to real scholars and if so, whether they were aware of the submission in their name. Nature was able to contact one actual researcher: he does not know why his name appeared in the author list of such a computer-generated paper.

Below the fold, I offer a speculation on the motivation behind the submission of these fake papers.

Is peer-reviewing really tracking quality in philosophy?

On Facebook, Jason Stanley writes:

I'm reviewing Kieran Healy's citation data, and it reminds me again how weird journal acceptance is. My book *Knowledge and Practical Interests* is the fifth most cited work of philosophy since 2000 in Phil Review, Mind, Nous, and the Journal of Philosophy (book or article). Yet the book itself is the result of three revise and resubmits, and finally a rejection from Phil Review. One of those drafts was also rejected from Mind, and also from Nous. All of those journals have accepted papers discussing, in many cases very centrally, a work those very journals have deemed unpublishable.

It is a familiar issue: the extent to which peer reviewing really does track quality. As has been noted before in discussions prompted by Healy’s data, in terms of publication venues, we are a highly conservative discipline, one where publishing in the ‘Healy Four’ – Phil Review, Mind, Nous, the Journal of Philosophy – is often viewed as the ultimate goal to be pursued. The assumption seems to be that publishing in these journals is a necessary and perhaps even sufficient condition for philosophical excellence. However, at least two of them have notoriously problematic refereeing practices (though it seems that efforts were made on that front); those that do have a reputation of fair refereeing practices end up overburdened and often need to call for a moratorium on submissions. All of them end up with minuscule rates of acceptance, much lower than in some of the top journals in other disciplines (such as Science and Nature).