Refereed Publications: A Modest Proposal

Helen's recent post on the difficulty of finding referees made me wonder why we bother any more. Why does the institution of refereed publication persist?

There was a time when refereeing was supposed to be a guarantee of quality. You read the top journals, because supposedly the high submission rate combined with excellent refereeing and the high cost of publication to guarantee very high quality. This is not particularly plausible any more.

Of course, refereeing ensures that somebody has read the paper and isn't prepared to say that it's awful. But increasingly, the choice of articles seems arbitrary and capricious. Moreover, a lot of publication attends to tiny details, because this is the stuff that's easy to get refereed. Because of the pressure to publish, the time pressure on referees is intolerable, and their attention to detail seems to diminish by the day. Plus there is some of the referee-manipulation that Eric mentions.

Helen's example of pre-refereeing is interesting: to recapitulate, there is a site to which authors upload, other scholars (perhaps a pre-selected board?) are alerted by keywords, and contribute reports. Journal editors pick papers for publication based on the reports on the site. An interesting model, as I say, but (a) what would a referee's motivation be? (Perhaps, by agreeing to be on the board, they agree to read a certain number of papers every year?) And (b) why think this would solve the problem of poor refereeing?

Nowadays, many people self-publish papers and drafts on their own websites or on public sites such as PhilPapers. What would be wrong if this system replaced peer-review? I can think of two objections.

a. There is no guarantee of quality. Well, this is true, but what if sites allowed qualified professionals to submit brief comments, and the author continually revised as well as replied? The comments could be quite focussed and short, and would not be an assessment or guarantee of quality—they wouldn't take as much time as referee reports, and they wouldn't pretend to objectivity. Hopefully, papers would gradually get better by responding (selectively) to comments.

b. Universities would not be able properly to assess their faculty's research. Now, this is a genuine problem. But I suspect that in academia today, this is the tail that wags the dog. Refereed publications are about people getting jobs, tenure, and merit pay. They are not about guaranteeing scholarly quality. But if this is the purpose for which referees are needed, then why not get referees just for this purpose? Why have everything refereed twice: once for publication, again for hiring and promotion.

Abolish refereeing. That's my modest proposal.