Now that the future of PGR is hanging in the balance, some folk are thinking a future not just with multiple rankings (recall Weatherson and my response), but no philosophy rankings [HT Winsberg & Huebner on Facebook]. Several of my friends in the discipline have expressed surprise that I defend rankings in our future and that I have even expressed some desire to keep a modestly reformed PGR as one of the more important rankings in the discipline. I am not ignorant of the technical problems with the PGR -- and it is really astounding that in a discipline with so much knowledge about, and so often pays lip-service to, scientific method these problems have not caused more concerns -- and, given that I always insist that incentives matter, I am not sanguine about the perverse incentives PGR has generated in its own ecology, and the harm done to those on its margins and those excluded. For all I know, it has deformed the lives of some that thrive under it (I am not joking--it really deforms characters and moral sentiments). So, why defend it?
One defense might be that by generating competition for scarce talent along a whole range of sub-niches, it has elevated the salaries of people who would otherwise could never expect to rank as special (or distinguished, etc.). I have trotted out this defense, and I think it true. But I recognize that for the vast majority of folk this argument from self-interest will seem not very compelling, if only because they may have justified fear that these increases come at the expense of their salaries (or tenure, etc.). Given that I am stipulating a zero-sum environment, I don't have a good response to this concern (except to note that the expense may be felt by disciplines outside philosophy). (I thank Enzo Rossi for discussion.)
Rather, the most important thing to remember is that we live in an imperfect world, where scarcity matters and where, among the many constraints that influence our choices, judgments of quality need to have a place at the table,--or they will be simply ignored. The key virtue behind PGR is that it has given a wide enough group of very good professional philosophers a chance at shaping some disciplinary autonomy when it comes to judgments of quality. Rather than farming this judgment out to supposedly objective scientific metrics, or to outside news agencies, or more informal local judgments, the PGR gives a flawed and imperfect and often unfair, but decent snapshot of the evolving sensus communis. It's an imperfect proxy, but still a proxy that tracks perceptions of quality. To put the point differently: we will be ranked anyway by Deans, by grant agencies, by donors, etc. So, perhaps it's because I spent my time in a grant environment trying to track quality by silly metrics, but I am convinced it is better to have some disciplinary control over the rankings.
Does, my better a flawed autonomous disciplinary ranking than no ranking absolve us all from the complicity in the many petty (and a few larger) injustices and biases embedded in them and their application in the real world of hiring, tenuring, and graduate school choices? No, of course not. We -- that is, the inner core of a thousand to two-thousand professional philosophers that really benefit from the PGR (even if some never admit it) -- could have all done much more to encourage the systematic, ongoing adoption of known best practices into the PGR and more thoughtful ways to expand the PGR ecological reach in ways beneficial to more professional philosophers. The intellectual sparkle of even the most eminent of our humanitarian thinkers will be tainted and dimmed forever by their quiet acceptance of the PGR culture during the last decade(s); but even so, the embrace of the PGR is defensible.
I dont necessarily disagree. But kudos to you for dropping the canard that we do this for the students. The students don't really need this much detail.
Posted by: Eric Winsberg | 09/27/2014 at 08:29 PM
The PGR can be useful to prospective students, but I agree they could be reached by other means, too.
Posted by: Eric Schliesser | 09/27/2014 at 08:32 PM
In order to give a "decent snapshot of the evolving sensus communis," don't the rankers have to be a representative sample of the community? Do we have reason to believe that they are? (I don't think we do). And isn't the question of "who is in the community" part of what is at stake?
Posted by: Roberta Millstein | 09/27/2014 at 08:57 PM
Thanks, Eric. I still haven't quite made up my mind on the whole question. It seems to me that there's a kind of paradox of the PGR: initially the PGR disrupted entrenched snobberies and prejudice, by pointing out that some state schools were better for philosophy than most Ivies, etc. That was good. But as the PGR became successful it replaced the old snobberies with its own kind of snobbery. So now I imagine it must be very hard for an evaluator to prescind from the entrenched knowledge that, say, Rutgers is a very Leiter-strong programme.
Posted by: Enzo Rossi | 09/27/2014 at 09:10 PM
On the final question: yes. And I advocate that the PGR ecology ought to be creatively expanded and become more porous.
Posted by: Eric Schliesser | 09/27/2014 at 09:15 PM
Yes, that is a necessary, albeit not sufficient, change.
Posted by: Roberta Millstein | 09/27/2014 at 09:17 PM
The PGR erases the professional standing of the many of us who do not teach in graduate programs. (I did, got tenure, left, and I understand how the question of where they stood shaped the thinking of the department I left. I assume this is somewhat generalizable.) My current department--U Mass Boston--used to get listed in the PGR short list of strong departments without grad programs, the only public institution so listed. I don't know why other publics weren't there and why we got dropped, and honestly I don't care. This discussion--not specifically to Eric--is extremely biased toward philosophers who train other professional philosophers, and does not actually represent the discipline as a whole. There are many, many good reasons to step away from teaching in a grad program, so I disagree with Eric's claim that the PGR has "given a wide enough group of very good professional philosophers a chance at shaping some disciplinary autonomy when it comes to judgments of quality". NOT a wide enough group, and if that group dictates, then the autonomy of those excluded is damaged.
Posted by: Lynne Tirrell | 09/27/2014 at 09:28 PM
I am not denying the harms you mention, Lynne, and I have never claimed the PGR represents the discipline as a whole (In fact, I have repeatedly stressed that the PGR ecology is not representative.)
Posted by: Eric Schliesser | 09/27/2014 at 09:33 PM
I guess there can be value in snobbery, Enzo, when it is directed at proper objects.
Posted by: Eric Schliesser | 09/27/2014 at 09:38 PM
Eric: This is your claim that I am disagreeing with: "The key virtue behind PGR is that it has given a wide enough group of very good professional philosophers a chance at shaping some disciplinary autonomy when it comes to judgments of quality." I'm saying: NOT "wide enough." Because of the many philosophers whose work is overlooked and whose views of the profession are excluded, I don't think the rankings have any real legitimacy.
Posted by: Lynne Tirrell | 09/27/2014 at 09:55 PM
Lynne, I am stipulating that there will be overlooked philosophers (for all I know the vast majority), but I am claiming it does a decent enough (albeit improvable, flawed, biased, etc.) job at tracking the views of folk within the PGR ecology.
Posted by: Eric Schliesser | 09/27/2014 at 09:57 PM
There can be value in some forms of elitism, yes. But if one of the motivations for the PGR was that entrenched snobbery was distorting because it was entrenched, then now that the PGR is itself entrenched it's hard to see how it can avoid falling on its own sword. I mean, it's not as if the pre-PGR pro-Ivy (say) snobbery was based on nothing.
(Sorry to replicate the FB discussion, should've come here in the first place!)
Posted by: Enzo Rossi | 09/27/2014 at 09:58 PM
Well, the Ivies were not first movers on meritocracy.
Posted by: Eric Schliesser | 09/27/2014 at 10:10 PM
Eric, do we really need *rankings*, or would it also be sufficient to have other form of measuring and clustering information about which departments are good/medium/not-so-good in different sub disciplines, without numbers running from 1 to x? Maybe such groupings would have less detrimental side-effects, and could be more inclusive. But on another point: we do have "rankings" (broadly understood) in the sense that we have "top journals" and it is very easy to find out who publishes there. I don't know enough about the PGR to really judge this, but doesn't it sort of track these other informations that are out there anyway? And given that there are many well-known problems with those journals, does it reinforce the same hierarchies and patterns of inclusion and exclusion? This is a honest question, as I really don't know how the PGR rankers make their judgments and what information apart from journal publications they take into account in what ways. Ceteris paribus, I would say that we should try to have various kinds of measures, which are not redundant to one another.
Posted by: Lisa Herzog | 09/27/2014 at 10:15 PM
Lisa, I think it be great if we experiment with other forms of measures. We can then learn how they do.
In practice, lots of hierarchies get reinforced, some pull in different directions.
Posted by: Eric Schliesser | 09/27/2014 at 10:24 PM
But....it's a ranking of graduate programs. I agree there is value in having professional autonomy over such a ranking, and that such a ranking can play an important role in how graduate programs ask for funding. But as a ranking of graduate programs it would naturally be for students interested in going to graduate school.
All graduate education these days is exploitative in some sense. The universities that run graduate programs get way more out of the arrangement than the people enrolled. Now if it is true that people in professional philosophy will judge you in any way by the place you went to graduate school then the people considering graduate school need to know the content of those judgments so they can factor it in to where to go, and whether to go. Most other fields have this provided by US News and World Report, or by some other body. Maybe this need could be satisfied by something like a customizable ranking using data the APA is already collecting, or any of the other suggestions out there. I'm not sure.
But if we want a disciplinary ranking of quality that plays some role in hiring and tenure decisions more generally then we should rank all philosophy departments. If we are ranking graduate programs then we are ranking graduate programs.
I teach primarily low income and first generation college students. The walls of my department are lined with posters for master's degree programs (and some phd) just waiting to take their money (in the form of loans of course). Now, if we would be just as likely to hire them coming out of any of these programs then there's no harm. But if we know that we take some programs more seriously than others, and we don't just tell them that upfront, than that is on us.
Posted by: Kristina Meshelski | 09/27/2014 at 10:34 PM
Kristina, I think there is a lot that ought to be told folk upfront. As I say above I recognize that a PGR can be useful to prospective graduate students, but I would not defend it on those grounds.
Posted by: Eric Schliesser | 09/27/2014 at 10:59 PM
DO you have evidence for the salary-improving claim? I doubt it very much.
Paragraph beginning with "rather". Again, evidence? Was philosophy more or less well supported by administrations prior to and after the introduction of PGR. I have no data to offer but my strong sense is that it has gotten worse, that rankings are used against us more than for us - where "us" includes all departments, not the "top ten". In the past, you had to show that you attracted students and published a lot. Now, you show that and they say - but you are only ranked 35th.
So I'm pretty much unmoved here. (SOrry if this is repeating from earlier comments which I haven't time to read - heading off to a race this morning followed by masses of grading. Feel free not to publish this if it is totally redundant.)
Posted by: Mark Lance | 09/28/2014 at 02:18 PM
Eric,
I think Lynne (and Mark) are on to something really important. If the accepted unit of professional evaluation is the department, then belonging to a particular department adds luster or takes away luster from a member of a department independent of their own accomplishments. In the case of excellent philosophers who are not members of departments with luster, for example their current department is not ranked at all because it doesn't have a graduate program, this seems to me to lead to a lot of unfair bias. Philosophers seem incapable of making the selective judgments that allow them to overcome this irrational biasing effect. Given that such a vast quantity of excellent philosophers are in this position, i.e. not in graduate departments, I think it ought to be gotten rid of. In particular if it results in less ability for these philosophers to move to graduate programs on the basis of their real accomplishments, and rather allows the usual candidates to continue to recirculate due to their ability to bring some #4 luster over to #8. [I know it sounds like some kind of magic cologne]. And note, no blame to Brian Leiter. Like many people I think the initial effect of the PGR was to break up an even more serious and unjust version of this effect. And nothing wrong with the candidate from 4, I'm sure they are super terrific.
Posted by: Aaron Garrett | 09/28/2014 at 03:02 PM
Mark, on the salary part; I have my own experience, but it's not an official part of my argument. I am making no comparative claims about the past (I think); higher education is different now than it was when PGR got introduced.
I agree that a measure can be used against departments in the way you describe. (I would hope it's an argument for more resources--look we're only ranked X, but could move up with strategic hiring.) I am convinced it has been used against departments outside the PGR ecology; these are real harms.
Posted by: Eric Schliesser | 09/28/2014 at 03:12 PM
Aaron, I think you are partially right here. (Partial because given field/specialty rankings individual contributions in ranked departments can be tracked in some imperfect way.) I agree that folk in un-ranked departments but within PGR ecology are harmed in the way you describe. (My own solution is to expand PGR so that more folk are trapped in the ranking!)
We do disagree about move-ability; I think before the financial crisis quite a few non-top15 departments hire senior folk out of unranked departments. But I am unsure if that would happen more w/o a ranking.
Posted by: Eric Schliesser | 09/28/2014 at 03:23 PM
I posted this on Facebook but wanted to add this to the coversations here. Thanks.
I do believe that there is a moral issue still on the table, namely, if we acknowledge harms, how are we justifying the harms that are being done? The claims being made to justify them are apparently prudential ones: having rankings tells us something about quality, which can then be used as leverage by philosophers to improve their working conditions and salaries. However, even if we set aside the rather serious issue of claiming sufficient knowledge and wisdom to justify harms to others for the benefit of some, there is little ground for believing that rankings are a better measure of quality than other tools that we have available to us. (As many have pointed out, it's very difficult to do rankings in an impartial and cosmopolitan fashion. Provincialism of various kinds seems to just work its way into ranking systems. All the more true in a profession that defines itself in part by arguing about what constitutes the legitimate parameters and methods of the discipline. Talk about the wrong discipline to be doing rankings! Maybe we should make a deal: we will do rankings after we all agree about what Philosophy is.)
Further, the issue of leverage is a bit of a joke. Yes, I am sure that in certain situations some deans somewhere have said, "oh, we need to provide more money to the Philosophy Department because they rank high in Leiter's rankings." But who would these departments be? I doubt that they are going to be very many of the truly "prestige" departments in prestigious schools. Their sense of self worth would actually be diminished by appealing to something so hoi polloi as rankings. And of course all of the non-top ranked schools can't use the rankings, except to plea they need more support because they are poorly ranked. (See how far that one will get you.) So, we are back to the question, who are we actually helping here (in the face of the harms being done), and can't we help them in other ways that create less harms?
Posted by: Mitchell Aboulafia | 09/28/2014 at 10:03 PM
No, the claims are not merely prudential. I think having a measure of quality in the decision-mix is also an intrinsic good of high value. I am open to encouraging other measures of quality, feel free to suggest them and develop them. But without specifics I can't judge if they are better or worse than the status quo. (I think scientific metrics are definitely worse--and I say that as somebody who thrives in an ecology built around them.)
I don't think you can claim that the issue of leverage is a joke because that undercuts the harms claim, too. Like all measures it can be used and abused, but I think it is special pleading to stipulate the good uses away.
Feel free to suggest ways that may be less harmful; I am genuinely interested in those.
Posted by: Eric Schliesser | 09/28/2014 at 10:12 PM
I think arguments that there should not be any rankings are understandable in the abstract and in principle, but in the concrete context in which grad programs have had rankings, it is not going to be the case that there won't be rankings in the future. It's like wishing the Internet would go away.
I mentioned this briefly on FB but it was not taken quite as I intended: I'm still surprised that the accessibility of the Internet has not resulted in a variety of rankings on many different measures. I'm surprised it has taken this long for someone to try to assess the top-placing-in-TT-jobs programs, the most-publications-per-capita programs, the ratio-of-faculty-to-students programs, etc. One of my puzzlements with the Gourmet when it was still young was that I could not quite discern how reputation isn't a somewhat perpetually self-fulfilling measure, but surely there are more quantitative measurements of many things that could be ranked by many different people.
I hope the future holds something like this.
Posted by: Kate Norlock | 09/28/2014 at 11:18 PM
Just came back to see what's new and have to say: YES, Aaron!!
The only thing is...not everyone might want to move to a grad program anyway, for a variety of reasons. But you are right, that mobility is limited. In addition, offers for giving talks and other sorts of professional opportunities decline (no institutional luster from that speaker, so why waste the slot?). I don't think this classism is explicit or conscious, but it's real. We are fools if we don't think philosophy is just as much a rich-get-richer system as our economy. When I went on the job market from Pitt, Brandom told me not to look askance at any of the non-grad schools interviewing me, because there are terrific philosophers at places I might never have heard of. So go with respect and listen to their questions and learn. He was right, and I have carried that message with me all these years. We have deeper benches than the PGR imagines. (And the benches aren't all benched, but actively engaged.)
Posted by: Lynne Tirrell | 09/29/2014 at 04:49 PM