Letters of recommendation: Comparing X to Y

It is common for those writing letters of recommendation, whether for faculty going up for tenure or for students going on the job market, to compare the subject of the letter X to a faculty member or student Y at the same career stage.  I've heard it suggested that this is one of the few concrete things that a letter of recommendation can say.  Otherwise, the letter reader ends up trying to divine adjectives like "excellent," "outstanding," "superb," etc.

However, I've long been uncomfortable with this practice.  It's a bit hard for me to say why, so this post is my attempt to put my vague discomforts into words, toss them out into the blogosphere, and see what other people think.  So let me be clear at the outset – I am not saying that comparing X to Y is unethical, or should be stopped, or anything along those lines.  Just that the practice is... troubling.  I offer several reasons for concern below (both as a letter writer and as a reader of letters), in no particular order. They raise the specter of bias, among other issues.

Consider the scenario where you are reading a letter of recommendation for a person outside of your field.  The letter writer compares X to Y, but you have no idea who Y is.  In this scenario, the comparison might simply be seen to be uninformative, which is frustrating but not especially problematic.  Yet it can happen that other members of the review committee do know who Y is.  They are very impressed that X compares favorably to Y and advocate strongly for X on those grounds.  This might end up favoring certain areas of philosophy over others, especially if Y is in a so-called "core" area of philosophy.  At the least, it puts you at a loss in the conversations with your colleagues over X because you've been given information that is useless to you.  Your colleagues can insist that Y really is fabulous, but are they making the comparison on the same grounds as the letter writer?  This leads to my next concern.

Comparing X to Y allows decisions to be made on unclear grounds; exactly how does X compare to Y?  Comparing two papers is fairly straightforward, but comparing two people is far more complex.  You have to consider the argumentation, clarity, and creativity of multiple papers; the scope and interest of the research project; participation in the field through conferences and in other ways; collegiality, etc.  To do this properly requires a pretty thorough analysis and deep knowledge of both X and Y.  Are letter writers really doing an analysis this thorough?  And if they are not, what kind of biases might creep into the judgment of X's comparability to Y?  Note that it is because of concerns over such biases that best practice for evaluation of candidates requires explicit consideration of each criterion on its own, not just an overall shoot-from-the-hip X-is-similar-to-Y judgment.  So, how much credence should we put in the claim of X's comparability to Y?

The letter writer can, of course, specify the grounds for the comparison, but in my experience, this is often not done.  And if the letter writer is going to do this, it means talking about X's writing, presentations, collegiality, etc., in detail.  So, why not just do that?  Is there really much information added to the comparison that cannot be gleaned from analysis of the specifics of X's case?  It doesn't seem to me that much information is added (and certainly, no one is going to provide the same level of detail for Y so that the comparison can be fully substantiated).  I think we can all agree that  adjectives like "outstanding" on their own are not very helpful.  But adjectives supported by specific examples from X's portfolio can be extremely illuminating.

I also wonder, who gets to be Y?  How often do women and other unrepresented groups in philosophy get to be Y?  Are we entrenching biases by making such comparisons?  I don't pretend to know the answers to these questions, not having done an analysis of letters of recommendation, but I suspect that Y is most often a straight white male.  If X is not a straight white male, then does X gain status from being being compared to Y?  Yuck. 

Which leads me to my final concern – as a letter writer, comparing specific individuals makes me extremely uncomfortable.  The vast majority of letters that I write are for people in philosophy of biology.  Suppose, as is frequently the case (given the size of the field), I know X and her potential Ys quite well. I've now put, in writing, what I think about the relative merits of my good colleagues X and Y.  I don't like doing that, in part because of the (hopefully small!) possibility that it might get back to them or out to someone else, but also because it can start to feel disloyal and possibly hurtful.  If I say that X is better than Y, but Y is a good colleague, what have I done to Y?  I don't want to hurt Y just to help X.

Ok, enough.  Maybe someone can tell me why my concerns are unfounded – why we would not be well served simply by writing detailed letters that discuss X's specifics.  If we need comparisons, they can be of the form "X is among the top 5% of students I have taught" (or something similar).  It seems to me that such statements serve essentially the same purpose without suffering from the concerns raised above.