[This post was inspired and encouraged by Sebastian Lutz; he should not be held responsible for the informal presentation of these matters.--ES]
Refereeing is a crucial professional obligation, especially now that most philosophers depend on (multiple) publication(s) for appointments and advancement. It also plays a non-trivial role in improving the quality of what is published. Yet, (a) refereeing is rarely taught. Even (b) experienced referees almost never get feedback on the quality of refereeing, so it is difficult to learn to calibrate one’s judgment. Moreover, (c) the relationship between the referee report and editorial decision-making is often opaque: (i) few journals tell referees of the decision; (ii) sometimes the referee report and the decision seem not well connected; (iii) sometimes the referee report and the final publication seem not well connected. This last feature also makes refereeing unsatisfying (and may well be connected with the difficulty finding referees reported by Brogaard). I continue to referee, perhaps out of a sense of duty, but primarily because it forces me to read, engage with and reflect on recent work beyond my circle of intellectual friends before publication.
Now, recently I refereed a paper for a journal that asked me to also referee the revised paper (after resubmission). I was amazed to discover a whole package that included the original paper, all the original referee reports (including my own), and the revised paper. I found it fascinating to read the two other original referee reports. (It was also amusing because one of them suggested that the author should engage with Schliesser!) It turned out that the three of us all saw what mattered about the paper, while also raising interesting (I think) objections that were subtly different. I was really impressed by the professionalism and thorough refereeing of the other two referees. But this got me thinking, Why can’t referees see each other’s anonymous (!) reports after submitting their reports?* I write 'after' because the reports ought to be independent. It’s seems feasible with manuscript-central software. Can anybody offer an argument against this proposal?
Refereeing is a crucial professional obligation, especially now that most philosophers depend on (multiple) publication(s) for appointments and advancement. It also plays a non-trivial role in improving the quality of what is published. Yet, (a) refereeing is rarely taught. Even (b) experienced referees almost never get feedback on the quality of refereeing, so it is difficult to learn to calibrate one’s judgment. Moreover, (c) the relationship between the referee report and editorial decision-making is often opaque: (i) few journals tell referees of the decision; (ii) sometimes the referee report and the decision seem not well connected; (iii) sometimes the referee report and the final publication seem not well connected. This last feature also makes refereeing unsatisfying (and may well be connected with the difficulty finding referees reported by Brogaard). I continue to referee, perhaps out of a sense of duty, but primarily because it forces me to read, engage with and reflect on recent work beyond my circle of intellectual friends before publication.
Now, recently I refereed a paper for a journal that asked me to also referee the revised paper (after resubmission). I was amazed to discover a whole package that included the original paper, all the original referee reports (including my own), and the revised paper. I found it fascinating to read the two other original referee reports. (It was also amusing because one of them suggested that the author should engage with Schliesser!) It turned out that the three of us all saw what mattered about the paper, while also raising interesting (I think) objections that were subtly different. I was really impressed by the professionalism and thorough refereeing of the other two referees. But this got me thinking, Why can’t referees see each other’s anonymous (!) reports after submitting their reports?* I write 'after' because the reports ought to be independent. It’s seems feasible with manuscript-central software. Can anybody offer an argument against this proposal?
The only concern with the proposal I can identify is that it might make editors more vulnerable to potential criticism and second-guessing (because now the referees are in position to see the full dossier). But I do not see how this would occur often, and it might make all the agents involved more judicious. On the up-side, referees learn from each other and get a better sense of the nature of their contribution to the discipline; this might motivate us to keep up the hard work and entice a few to stay engaged.
*Note that this proposal is unrelated to my advocacy of (a) making referee reports available alongside accepted papers and (b) removing anonymity at this stage.
*Note that this proposal is unrelated to my advocacy of (a) making referee reports available alongside accepted papers and (b) removing anonymity at this stage.
Recent Comments