On Facebook, Jason Stanley writes:
I'm reviewing Kieran Healy's citation data, and it reminds me again how weird journal acceptance is. My book *Knowledge and Practical Interests* is the fifth most cited work of philosophy since 2000 in Phil Review, Mind, Nous, and the Journal of Philosophy (book or article). Yet the book itself is the result of three revise and resubmits, and finally a rejection from Phil Review. One of those drafts was also rejected from Mind, and also from Nous. All of those journals have accepted papers discussing, in many cases very centrally, a work those very journals have deemed unpublishable.
It is a familiar issue: the extent to which peer reviewing really does track quality. As has been noted before in discussions prompted by Healy’s data, in terms of publication venues, we are a highly conservative discipline, one where publishing in the ‘Healy Four’ – Phil Review, Mind, Nous, the Journal of Philosophy – is often viewed as the ultimate goal to be pursued. The assumption seems to be that publishing in these journals is a necessary and perhaps even sufficient condition for philosophical excellence. However, at least two of them have notoriously problematic refereeing practices (though it seems that efforts were made on that front); those that do have a reputation of fair refereeing practices end up overburdened and often need to call for a moratorium on submissions. All of them end up with minuscule rates of acceptance, much lower than in some of the top journals in other disciplines (such as Science and Nature).
What are the odds that this system is actually tracking the very best philosophical work being done at any given time? From other disciplines, there are increasingly disturbing signs that peer-reviewing is in no way a reliable method to discern good from bad research, including a number of high-profile fraudsters and an even larger number of post-publication retractions (see here). For example, a review on the evidence on peer reviewing in the biomedical sciences has concluded: “At present, little empirical evidence is available to support the use of editorial peer review as a mechanism to ensure quality of biomedical research.”
Within philosophy, which is a comparatively small discipline, one of the problems becomes: how to find referees of the ‘appropriate level’? It is a usual practice among journals to recruit referees among those who have already published in the journal in question. But given the comparatively small number of authors who have published in the Healy Four and the rivers of submissions they receive, even if they do not send every paper received to referees, it is hard to imagine that this small number of philosophers could possibly act as referees for all these submissions. This sometimes leads to the somewhat surreal situation described by philosopher Panu Raatikainen in comments at Stanley’s FB post: “Another funny repeated experience with some of these top journals: shortly after rejecting my paper, the journal has asked me to act as a referee on the same topic.”
So it seems that given the scarcity of available referees (and the difficulties in getting people to actually accept the referee request and do it in a timely fashion, as often discussed by Brit), many potentially groundbreaking papers might not be getting the chance they deserve. Relatedly, it might seem that, while other areas display the phenomenon of over-publication, with a lot of bad stuff being published, philosophy, at least in terms of the highly respected venues, suffers from the phenomenon of under-publication. (Not to mention the relatively narrow focus of the topics usually covered by the Healy Four, and a certain aversion to innovative – aka ‘weird’ – perspectives.)
All these observations taken together got me thinking, and led me to a decision with respect to my practice as a referee (which I want to distinguish for now from my practice as journal editor for RSL and Ergo): from now on, I’m going to be much more easy-going on my recommendations for editors than I’ve been so far. Of course, I will still make whatever constructive comments I think may improve the quality of the paper (which of course remains a judgment call), but I’m going to set a lower bar between an R&R and a rejection, and between a conditional acceptance and an R&R.
The thought behind this is that there is a lot of good work out there that deserves a chance to be published. Following my long-standing Feyerabendian instincts, I believe that, once the paper is published and perused by a much larger number of people than the editor and the referee(s), a more reliable consensus on the actual merits of the paper will emerge. This seems to me to be the moral of Jason Stanley’s anecdote above (or at least one of them). (As it turns out, last week I got my first request to referee for one of the Healy Four, so it seems like the right time for a new resolution!)
Recent Comments