September 2, 2014
University rankings: multiple missions, lots of discrepancy
Tertiary institutions’ multiple missions undermine overall value of global comparisons
The world of higher education welcomed yet another large-scale global ranking this week, with the Centre for World University Rankings (CWUR) placing the University of Buenos Aires (UBA) third in Latin America and 378th in the world. It’s the first time the Saudi Arabia-based institution has assessed more than 100 universities.
Experts, however, question the relevance of global rankings, noting the difficulty of measuring academic quality across regions.
Founded in 2012, CWUR says it publishes the “only global university ranking that measures the quality of education and training of students” as well as the “prestige of the faculty members and the quality of their research without relying on surveys and university data submissions, criticizing QS for its “reliance on reputational indicators.”
CWUR’s rankings are based on eight criteria: “quality of education, alumni employment, quality of faculty, publications, influence, citations, broad impact and patents.”
The UBA was in May downgraded in QS’ rankings of Latin American universities, one of world's several barometers for international higher education institutions, raising concern that the institution had suffered a drop in quality.
The companies that conduct the rankings themselves admit to this, noting differences in their methodologies and missions, just like the characteristics that often make universities around the globe unique themselves.
“Any ranking ultimately serves the information needs of a particular audience, and methodological priorities are balanced accordingly,” Danny Byrne, senior editor at QS acknowledged.
Earlier this month the rankings by QS, a private British company, Shanghai Jao Tong University and The Times (London) were singled out by the New York Times, highlighting their prominence and influence as the leaders of ranking industry. The Times, however, has yet to include Argentine universities within its rankings, although that is likely to change in the near future.
QS releases an annual “World University Rankings” that placed UBA 209th in the world in 2009, its latest global report, while Shanghai Jiao Tong University’s Academic “Ranking of World Universities” has the institution in the 151-200 category, well into the top 500.
UBA has stayed flat in Shanghai’s table over the last few years, but the institution plunged seven places to 19th in the QS 2014 Latin America Rankings. It still remains the country’s most prominent on either ranking.
“The Shanghai rankings were originally designed to benchmark the research capacity of Chinese universities against their international competitors, so are weighed almost entirely on scientific research,” Byrne said, adding “the QS rankings are specifically targeted at prospective students and parents … the things we measure, such as academic and employer reputation and student-faculty ratios, are of paramount importance to students.”
This would appear to explain the better placement of UBA on the Shanghai system, as the institution conducts 30 percent of the country’s scientific research.
As its dean, Alberto Barbieri, recently outlined in an opinion piece for La Nación that UBA currently has an astounding 351,200 undergraduate students, or 25 percent of those total studying at public higher education institutions throughout the country.
For Barbieri, “rankings attempt to represent reality in a simplified manner and do not cover the complexity behind the activities carried out by our universities, which often cannot be quantified.”
Despite these difficulties, The Times argues that useful rankings that include complementary qualitative data can be elaborated.
“We understand that many public South American universities have different missions, including social mobility, and face huge demands and pressure on the ground — there are dramatic variations in systems and cultures that challenge the application of global criteria,” noted Times Higher Education World University Rankings editor Phil Baty, who told the Herald the London-based firm met with UBA representatives earlier this month to work on incorporating the university into the ranking.
For The Times and Thomson Reuters, which work together on rankings, what is most important is how many times papers by an institution’s researchers have been cited over a five-year period, Baty added, indicating a certain affinity to the approach taken by Shanghai Jiao Tong University.
The Times is “very keen on reaching out” to diverse universities “and understanding their needs, developing new metrics to understand local context and provide valuable data to communities,” Baty said.
A broken model?
For some, though, the whole business of rankings should not be seen as anything other than an academic exercise, because the qualitative difference of institutions proves too much of a grey area.
"I think we have to move away from paying too much attention to global rankings because they involve comparisons that use different criteria, excluding some and taking the point of view of central countries that have very different universities than ours,” Claudio Suasnábar, a public university expert and professor at Universidad de San Andrés, told the Herald.
Regardless of qualitative distinctions, specific criteria are also given varying weights, fuelling the criticisms of those who question their overall worth.
“Job placement after graduation should be relevant when assessing the quality of an institution,” but in a country like “Argentina, with its cycles of expansion and contraction in employment rates, it’s difficult to use this as an indicator of quality, especially since statistics show that the abundance of professionals in certain areas makes employers hire overqualified individuals at lower wages,” Suasnábar said.
Another fundamental uniqueness of the Argentine system that affects international comparisons is that most courses are designed to allow for students to be able to work full-time, meaning higher education programmes are often designed with flexible hours, thus affecting the duration of a person’s degree.
Acording to a 2010 census, at UBA for instance, 57 percent of students worked 36 hours or more per week.
Yet despite these particularities, QS still believes Argentine institutions generally fall short.
“Argentine, Mexico and Colombia excel in certain areas, but lack the consistency of leading institutions in Chile and Brazil," said QS' Ben Sowter.
In addition to the intangible qualitative uniqueness of universities and the different signficance of certain criteria, critics add that lobbies are strong within the rankings industry.
In the UK, for instance, being placed within the top 10 could mean securing greater interest from prospective students from abroad.
As might be expected, ranking firms strongly deny lobbying plays a role in their procedures.
While Baty acknowledged The Times comes “under a lot of pressure from government leaders and institutions” seeking higher rankings to attract higher paying foreigners, Byrne said QS’ system “is based on a combination of objective data and performance in surveys with very large response rates – and which are consequently extremely difficult to manipulate.”
Yet whether that “objective data” manages to tell the full story is still up for debate.