We seem to have a never-ending fondness for ranking. We rank schools, we rank hotels, we rank the cost of living in different countries, and so on. There is something about the competitive streak in human nature that makes us want to compare ourselves with everybody else.
The Shanghai Jiao Tong University ranked the top 500 universities in the world, and published their findings in September 2004. Their methodology involved looking at four main criteria:
a) The quality of education (alumni winning Nobel or Field Prizes)
b) The quality of the faculty (staff with Nobel or Field Prizes, highly cited researchers)
c) The research output (articles published in top journals eg. Nature, Science, etc.; articles cited in Science or Social Science Citation Indices)
d) The size of institution (academic performance with respect to size of institution)
The Times Higher Education Supplement published its ranking of the top 200 universities in the world on 5 November 2004. They explained that their ranking was based on:
a) Peer Review (by 1,300 academics in 88 countries)
b) Research impact (calculated by measuring citations per faculty member)
c) International orientation (the number of international students and faculty members)
Ranking of the World’s Top Ten Universities 2004:
Times Higher Education Supplement (THES)
1. Harvard University
2. University of California, Berkeley
3. Massachusetts Institute of Technology (MIT)
4. California Institute of Technology
5. Oxford University
6. Cambridge University
7. Stanford University
8. Yale University
9. Princeton University
10. ETH Zurich
Shanghai Jiao Tong University (SJTU)
1. Harvard University
2. Stanford University
3. Cambridge University
4. University of California, Berkeley
5. Massachusetts Institute of Technology (MIT)
6. California Intitute of Technology
7. Princeton University
8. Oxford University
9. Columbia University
10. Chicago University
The criteria used for ranking must perforce be arbitrary. Thus, not unnaturally, people’s eyebrows will be raised when they see this list. And one can forever debate whether or not this university is really better than that. Which ranking list is the more accurate? I don’t know.
Let us look at some of the general problems associated with ranking. The SJTU survey places an inordinate weight on individual achievements. For example, Nobel and Field Prizewinners will greatly influence a university’s ranking. The prizes are usually awarded long after the research has been completed. This will unfairly discriminate against newer universities.
The THES analysis relies on peer review quite heavily. Unfortunately this tends to be very subjective. The older universities would tend to have an advantage because usually they would have a number of famous professors, many of whom have written books used by undergraduates and postgraduate students. Newer universities would be disadvantaged by this system.
Both the SJTU and THES look at the impact of research publications by counting the number of citations in Science and Social Science Citation indices. In general scientists are more prolific in their publications, and they tend to quote their own previous publications more frequently. Hence the use of citation frequency is probably valid when it comes to Science and Social Science publications. But it is not so useful when it comes to the humanities.
One must not forget that most scientific publications are in English, and therefore non-English publications tend to be less frequently cited. Universities that teach and publish in languages other than English may tend to be ranked lower as a result.
Other parameters used to rank universities include the faculty-to-student ratio. This will favor the better endowed universities with greater funding for staff. Another index is the degree of internationality of the student and faculty population.
Although not ostensibly used by the SJTU and THES, other ranking systems look at the ratio of applicants per place available, or the minimum requirements needed to gain admission. This is an assessment on the difficulty of entry. Again this is not a very useful guide, because the reasons why students choose one university rather than another are often quite idiosyncratic.
My major concerns about university league tables are several. First, they have not been validated statistically. In earlier surveys, it has been found that the differences between institutions are statistically insignificant. Secondly, the use of the various indicators, and their weighting, are arbitrary. Minor changes in weighting can result in major changes in ranking. Thirdly, league tables can give an institution an undeservedly high or low reputation. They may skew students and their parents’ perception of a university, so that they might not apply for a perfectly good university, in favor of another. Fourthly, it has been known in the past that some universities changed their policies so that they could be better ranked in such surveys.
Related to ranking of universities is the question as to what these institutions are trying to achieve. What are their aims? Succinctly, the purpose of a university is to preserve, advance and disseminate knowledge. For the undergraduate, the purpose of a university is to imbue the student with knowledge, tolerance and vision, and to create a lifelong sense of curiosity and a love of learning.
Unfortunately the assessment and ranking of universities does not really take all this into account. Ranking will continue nevertheless. One cynic said that the only beneficiaries of such university league tables are the top-ranked universities that will attract more students and therefore raise their fees, and newspapers that will sell more copies by writing about the ranking.
I think it is important for one to be aware of the shortcomings of university ranking tables. Best to have a good laugh, and forget about them promptly!