We seem to have a never-ending fondness for ranking. We rank schools, we rank hotels, we rank the cost of living in different countries, and so on. There is something about the competitive streak in human nature that makes us want to compare ourselves with everybody else.

The Shanghai Jiao Tong University ranked the top 500 universities in the world, and published their findings in September 2004. Their methodology involved looking at four main criteria:
a) The quality of education (alumni winning Nobel or Field Prizes)
b) The quality of the faculty (staff with Nobel or Field Prizes, highly cited researchers)
c) The research output (articles published in top journals eg. Nature, Science, etc.; articles cited in Science or Social Science Citation Indices)
d) The size of institution (academic performance with respect to size of institution)

The Times Higher Education Supplement published its ranking of the top 200 universities in the world on 5 November 2004. They explained that their ranking was based on:
a) Peer Review (by 1,300 academics in 88 countries)
b) Research impact (calculated by measuring citations per faculty member)
c) International orientation (the number of international students and faculty members)

Ranking of the World's Top Ten Universities 2004:
Times Higher Education Supplement (THES)
1. Harvard University
2. University of California, Berkeley
3. Massachusetts Institute of Technology (MIT)
4. California Institute of Technology
5. Oxford University
6. Cambridge University
7. Stanford University
8. Yale University
9. Princeton University
10. ETH Zurich

Shanghai Jiao Tong University (SJTU)
1. Harvard University
2. Stanford University
3. Cambridge University
4. University of California, Berkeley
5. Massachusetts Institute of Technology (MIT)
6. California Intitute of Technology
7. Princeton University
8. Oxford University
9. Columbia University
10. Chicago University

The criteria used for ranking must perforce be arbitrary. Thus, not unnaturally, people's eyebrows will be raised when they see this list. And one can forever debate whether or not this university is really better than that. Which ranking list is the more accurate? I don't know.

Let us look at some of the general problems associated with ranking. The SJTU survey places an inordinate weight on individual achievements. For example, Nobel and Field Prizewinners will greatly influence a university's ranking. The prizes are usually awarded long after the research has been completed. This will unfairly discriminate against newer universities.

The THES analysis relies on peer review quite heavily. Unfortunately this tends to be very subjective. The older universities would tend to have an advantage because usually they would have a number of famous professors, many of whom have written books used by undergraduates and postgraduate students. Newer universities would be disadvantaged by this system.

Both the SJTU and THES look at the impact of research publications by counting the number of citations in Science and Social Science Citation indices. In general scientists are more prolific in their publications, and they tend to quote their own previous publications more frequently. Hence the use of citation frequency is probably valid when it comes to Science and Social Science publications. But it is not so useful when it comes to the humanities.