Helen has a very interesting post up on the problematic culture of grant-funded research. (Full disclosure: I have been on research grants pretty much for my entire career, and I see many advantages in this system.) She emphasized the fact that grant-writing is a very time-consuming endeavor, and that many researchers spend so much time securing research money for their group that they virtually stop doing research of their own. In this short post, I want to outline yet another way in which this culture entails waste of precious research time: the time it takes panel members and referees to evaluate the huge number of proposals that get written.
This year, I was a member of the panel for one of the grant schemes of the Netherlands Organisation for Scientific Research (NWO). I can report that carefully reading the big pile of research proposals has taken something like 50 to 60 hours, not including the actual meetings to discuss the proposals and to interview the candidates. All in all, it must have amounted to 2 full weeks of work, during which time I could have written a new paper or worked on my own research more generally. In fact, I have the feeling that the first 3 months of this year were entirely spent on evaluating other people’s work rather than doing my own: besides the work for NWO, I also wrote multiple reports on (other) research proposals and applications, and countless referee reports on papers submitted to journals -- not to mention my work as an editor for the Review of Symbolic Logic (as every journal editor knows, it is very hard to find referees for papers these days, as everyone is already so overburdened). These three months were exceptionally (or so I hope!) busy in this respect, and I’m sure there are many others who do just as much or even more work of this kind. Still, this just seems wrong.
I can’t help the feeling that the culture of peer-reviewing, with all its numerous advantages (certainly over models where what counts is political influence or prestige) has the perverse side-effect of a lot of time spent on evaluating other people’s work rather than doing your own. Often peer-reviewing has the productive dimension of potential useful feedback for authors (I have certainly benefited from quite a few very helpful referee reports on my papers over the years), but in many cases authors never get to see such reports (especially in the case of grant proposals). The Kafkaesque limit of this model is the situation where there is no more original work to evaluate given that everyone is spending so much time evaluating the work of others.
Alas, I don’t have any concrete proposals on how to counter this effect, and would be curious to hear thoughts that readers may have on this matter. (And on this note, let me go write a paper of my own!)