On Being A Particular Scholar's Gate-Keeper

About a decade ago I refereed a paper which seemed familiar. I decided to check my archive (that is my email account), and I found a rather lengthy and what seemed (to me) generous referee report on an earlier version of the same, barely changed paper. I filed a second, somewhat more curmudgeonly report (mostly cut and paste from the first one), but alerted the editor of the (second) journal that I had seen this paper before (with apologies I had not alerted him to it earlier in the stage). A few months later I was sent the same paper by a third journal. The paper was essentially unchanged from the earlier versions. I told the editor that it was pointless for me to referee it again because I had refereed the paper twice before and that in my opinion the author was not responsive to my criticism. The editor decided to send the paper to another referee.

At the time I had no idea who the author of the paper was, although I have to admit that I had developed a certain amount of irritation about him. In my imagination the author was a stubborn scholar immune to my helpful attempts to call attention to detailed evidence that undermined his position. I also thought of him as ungrateful because I had put in a lot of work to offer (splendid!) advice on the paper--work that would go unrewarded professionally and, now, unnoticed, too.

The case I describe occurred in history of philosophy--where most journals practice so-called double-blind or (in deference to my wife--an eye surgeon) double-masked review. By contrast, in some fields, say, economics and history of economics, when one is refereeing one nearly always knows the author's identity (so-called single-blind, etc.). This is pretty problematic, and perhaps I should stop reviewing for such journals. But here I ignore that it is problematic and what I can do about it. (Mea culpa: I have been pretty lethargic in protesting this state of affairs.) A while ago, I noticed that during the last half decade I was often being asked to review the work of particular young scholars, who were otherwise unknown to me. None of the cases involved anything like the situation described in the previous two paragraphs. (That is, I read the papers, write reports, and then the experience merges into the mush that is my professional life.) In some cases I hope I'll meet the young scholar in person one day because I was excited about their work.

Even so, I have started to wonder what it means that for some junior scholars (I can think of at least four), I am literally one of the referees on the vast majority of the refereed papers they have published. (That's an easy thing to check given that most young scholars have updated versions of their CV online.) Part of me thinks this can't be healthy for them (and the field). I am happy to be gate-keeper, but now I worry that my scholarly niches and complete strangers (whose careers are on the line) have been shaped by the idiosyncratic nature of my intellectual sensibilities. (Yes, I recognize that the self-image may be a delusion, but it's a better delusion than those that think of their own work as un-problematic mainstream or cutting edge.)

Now, journals have lots of ways of preventing over-reliance on particular referees. But they are somewhat powerless to prevent different editors of different journals of over-using the same referee for particular papers (absent a data-base which would likely undermine anonymity).* [In fact, some scholarly conglomerates now find ways to ensure that you review a paper again (by having their software ask you if you would be willing to review for a sister journal).] For, while it is not a coincidence that I get asked to review these papers, the particular journal editors may call on me rather rarely.

My current inclination is to decline requests to referee when I notice that I am a frequent referee for a particular, junior person. This stance, if it is more widely shared, may also encourage editors to think outside the box and draw on a wider range of referees. Having just written the previous two sentences, it occurs to me that such a stance may also make it harder for journals to find (willing) referees and, thereby, undermine the opportunities for professional advancement of the younger scholars and the wider circulation of their ideas (which hurts the cause of truth). So, now I am  unsure about what to do. I welcome hearing what readers think.

Before I close this Digression, one final remark: I learned the identity of the stubborn and ungrateful author years later after the paper was published pretty much unchanged in the third journal. (I didn't look for the paper, but scholar.google brought it to my attention.) As it happens, by then I had many pleasant and stimulating interactions with the person at various conferences. Even so, I harbor a modest resentment and suspicion toward this person (now happily tenured) and I will not review any other papers by him if I can avoid it.

*Regular readers know I am no friend of anonymous review. But that's for different occasion (recall).

Comments

[Jonathan Ichikawa]

Huh. Without knowing the specifics of what you said in the report I can't know how to feel about this particular case. But I would certainly be willing to say that sometimes, when submitting to a new journal, it can be appropriate to ignore previous referees' reports. If I get an R&R I'll feel obliged to respond to the referee whether I agree with them or not, but if my paper is rejected and I'm submitting somewhere else, I'm perfectly willing to just ignore the previous reports if I don't find them valuable. Indeed, when I was in grad school, we were explicitly advised to do just that. (They called it the 'five envelope rule'—no-named back when papers were actually submitted to journals as physical objects.)

[Graham White]

It's an odd relationship, but it can work out well: I'm very grateful for changes that referees have suggested to several of my papers. So, of course, optimisation is another goal of the refereeing system, as well as gatekeeping. And (thinking of the blog past after this) it's a way of teaching authors not to be too attached to their own ideas: How Would Seneca Review This? might be a good question to ask.