From University Rankings to Education System Rankings

University rankings

It seems to have become a predictable outcome for the Irish higher education sector whenever a new set of league tables is released.

No matter it is the QS rankings, or Shanghai Jiaotong rankings, or the Times Higher Education rankings, the performance of Irish universities has been, overall, falling.

In the Times Higher Education rankings which were released late yesterday, no Irish third level institution made into the top 200, the first time since the publication of the rankings.

The most important factor which is considered to have contributed to the decline is the funding cut to the sector throughout almost the last decade.

A few hours before the release (Wednesday evening), everyone at Trinity College Dublin received an email from the University, informing that at this moment, TCD would not be included in the ranking due to some data error.

The email did not specify what the error was, but it did highlight that the data error probably had negatively impacted on the ranking of TCD over the last two years.

Therefore, it is still possible to see, when the data error will be corrected and the rankings will be updated in the way as informed, one Irish university among the top 200. Nevertheless, even this is not guaranteed.

More often now we see a growing sense in Ireland that maybe we should not care about the rankings after all.

This is an interesting argument. Partly, it is true that all the rankings are more focused on research performance than on education outcome. The quality of education, for sure, is much more difficult to measure. Let alone to compare it worldwide.

Education system rankings

Instead of ranking individual institutions, the recently released OECD Education at a Glance 2016 report measures the performance of national education systems.

Overall, in this broadly defined and education-focused ranking, Ireland is also among the bottom performers.

In particular, the performance is measured against a total of 10 Sustainable Development Goal (SDG) targets for education by 2030, showing countries’ efforts to achieve “inclusive and equitable quality education and promote lifelong learning opportunities for all”.

These 10 targets are:

  1. Percentage of 15-year old students performing at level 2 or higher on the PISA math scale (threshold 80%);
  2. Enrolment rate in pre-primary and primary education at age 5 (threshold 95%);
  3. First-time tertiary entry rates (threshold 60%);
  4. Percentage of adults with a high degree of skills and readiness to use ICT for problem solving (threshold 60%);
  5. PISA inclusion index (threshold 75%);
  6. Percentage of adults with high levels of literacy (threshold 50%);
  7. Percentage of students at level A, B and C in the PISA environmental science performance index (threshold 70%);
  8. Computers for educational purposes per student, mean index (threshold 0.7);
  9. Difference in scholarships and student costs in donor countries between 2012 and 2014, in millions (threshold USD 0); and
  10. Percentage of lower secondary teachers having completed teacher education of training programmes (threshold 95%).

 

Out of these 10 targets, Ireland has available data for 8 of them. As some of the top performers, e.g. Belgium and the Netherlands, have available data for all of the 10 targets, it is more likely that they achieve more targets.

To compare the performance of Ireland with those top performers, it is more reasonable to narrow the focus down to the 8 targets all of the countries have available data.

In particular, I compare Ireland with the following 9 countries which are top performers according to the OECD report: Australia, Canada, Belgium, Netherlands, Norway, New Zealand, Denmark, Japan and Sweden.

Figure 1 below shows GDP Per Capita (PPP) in these 10 countries in 2015. As the World Factbook by the CIA shows, Ireland ($55,500) is ranked at the 3rd place in the 10-country group, following behind only Norway ($68,400) and Australia ($65,400).

Given this context, the overall under-performance of Ireland against these countries, most of which have lower levels of GDP Per Capita (PPP), is even more worrying.

figure-1
Figure 1

In Table 1 below, I show the performance of the 10 countries in the 8 targets where Ireland has available data (indeed, all of the 10 countries have available data in these 8 targets).

Table 1: Countries’ progress towards the education SDG targets

Education SDG targets 1 2 4 5 6 7 8 9 No. of targets above benchmark
Benchmark 80 95 60 75 50 70 0.7 0 n/a
Australia 80 101 66 77 58 71 1.5 38 8
Canada 86 93 65 83 51 75 0.8 13 7
Netherlands 85 99 73 82 60 70 0.7 -30 7
Denmark 83 98 70 82 50 63 0.8 0 7
New Zealand 77 98 75 78 58 68 1.1 7 6
Belgium 81 98 65 72 53 67 0.7 0 6
Norway 78 98 72 91 58 59 0.8 1 6
Japan 89 96 53 78 72 75 0.6 -1 5
Sweden 73 95 72 87 58 62 0.6 9 5
Ireland 83 100 51 80 45 66 0.6 1 4

Ireland is ranked at the bottom of the group.

In Ireland, the percentage of adults with a high degree of skills and readiness to use ICT for problem solving is 51%, while the threshold for this target is 60%. The percentage of adults with high levels of literacy in Ireland is just 45%, making Ireland the only country under the threshold of 50% in the group.

Challenges

While those who criticise the methodologies of world university rankings might have rightfully pointed out the limits of the ranking systems, it should not become any excuse for the overall under-performance of the Irish education system (including primary, secondary, and high education).

The challenges facing the Irish sector would not be resolved by ignoring the imperfect university rankings altogether. Nor should we.

Advertisements

Universities and Global Rankings: Linking University Performance with National Economy

Nowadays, the launch of the global university rankings has become more like a cultural event, widely reported by mainstream media and hotly debated by academics and policy makers. There has been a significant increase of the number of academic articles on the rankings of universities. Two things, which might seem contradictory to each other at first glance, stand out from the studies. On the one hand, all of the ranking schemes, especially their methodologies, have received heavy criticism for being partial, subjective, and sometimes, manipulated. On the other hand, however, almost all academics seem to admit, and accept, the fact that the rankings have become an unstoppable phenomenon as they have generated too much influence across the globe to be simply ignored. Indeed, if we think of the rankings as a ‘game’, no matter how flawed it might be, it is better to be part of it rather than being excluded from it.

As a former student and now a university employee, I have followed the rankings, in particular the ranking of the institution where I have been based at that specific time. Here is an example. In 2010, when I started my PhD study in Wales, I was enrolled at the then University of Wales, Institute of Cardiff (it has been renamed as Cardiff Metropolitan University in 2011). When finishing the first year study, I got an opportunity to be able to transfer my study to Cardiff University. Although these two universities are in the same city, they are ranked significantly differently in both the national league tables and international rankings. I think I later recalled the transfer as ‘lucky’ (although I did enjoy my study in both institutions).

Of course, different people focus on rankings for different reasons. In Ireland, the news articles on the performance of Irish universities have been written in a worrying tone, as they reported that Irish universities have been falling down the rankings over the last few years. In general, the articles were based on the fact that fewer Irish universities made into, say, top 100 or top 500 universities in the world. This is the most straightforward way to look at the rankings: how many universities in each nation/country are ranked among top 100 or top 500. Nevertheless, we are not surprised at all to know that the USA has most top-ranked universities in the world, as it has the largest and best higher education sector in the world, and is the largest economy in the world as well. Instead, we might be more impressed by Hong Kong, which is a city with just over 7 million people, has 4 universities among top 100 in the QS World University Rankings 2015, same as countries such as Canada and China (Mainland).

An alternative way to look at the rankings, as proposed by Marginson (2007) who examined the implications of rankings for Australian Universities, is to compare the nation shares of top 100 and top 500 universities with their shares of ‘world economic capacity’. If the nation shares of top universities are higher than their shares of world economic capacity, they are performing ‘better’ than assumed; otherwise they are performing ‘worse’. Marginson (2007) defined that, ‘World economic capacity is measured as an aggregate of the individual nations’ economic capacity, defined as GNI multiplied by GNI per head.’

Using this definition, I recalculated the national performance of university rankings for the 3 most widely recognised ranking systems, namely the QS World University Rankings 2015, the Times Higher Education World University Rankings 2015 and the Shanghai Jiao Tong Academic Ranking of World Universities 2015. The economic data of nations was sourced from the World Bank.

Table 1: Number of top 100 universities in the QS, THE, and ARWU rankings 2015 (in a descending order of the number of top 100 universities, QS 2015)

Nation Number of Top 100 universities, QS 2015 Number of Top 100 universities, THE 2015 Number of Top 100 universities, ARWU 2015
USA 30 39 51
UK 18 16 9
Australia 7 6 4
Japan 5 2 4
Netherlands 5 8 4
Canada 4 4 4
China 4 2 0
China Hong Kong 4 2 0
Germany 4 9 4
Switzerland 4 2 4
South Korea 3 1 0
France 2 1 4
Singapore 2 2 0
Sweden 2 3 3
Belgium 1 1 2
China Taiwan 1 0 0
Denmark 1 1 2
Finland 1 1 1
Ireland 1 0 0
New Zealand 1 0 0
Argentina 0 0 0
Austria 0 0 0
Brazil 0 0 0
Chile 0 0 0
Czech Republic 0 0 0
Egypt 0 0 0
Greece 0 0 0
Hungary 0 0 0
India 0 0 0
Iran 0 0 0
Israel 0 0 2
Italy 0 0 0
Malaysia 0 0 0
Mexico 0 0 0
Norway 0 0 1
Poland 0 0 0
Portugal 0 0 0
Russia 0 0 1
Saudi Arabia 0 0 0
Serbia 0 0 0
Slovenia 0 0 0
South Africa 0 0 0
Spain 0 0 0
Turkey 0 0 0

Table 1 shows the number of top 100 universities by nation. Overall, the USA and the UK dominate the table, followed by Australia, Japan, Netherlands, Canada, China, China Hong Kong, Germany and Switzerland. There are significant differences across the three rankings for each nation though. While 51 American universities made into top 100 in the ARWU rankings, that number drops to 39 in the THE rankings and further to 30 in the QS rankings. The UK shows the contrasting pattern from the US. In the ARWU rankings, there are only 9 UK universities ranked among top 100, but the QS rankings labelled 18 UK universities as top 100. There is only 1 Irish university ranked among top 100 in the QS rankings, while none of universities in Ireland made into top 100 in the THE rankings and the ARWU rankings.

Table 2: Number of top 500 universities in the QS, THE, and ARWU rankings 2015 (in a descending order of the number of top 500 universities in the QS 2015)

Nation Number of Top 500 universities, QS 2015 Number of Top 500 universities, THE 2015 Number of Top 500 universities, ARWU 2015
USA 95 123 146
UK 51 56 37
Germany 35 36 39
China 25 11 32
France 23 19 22
Australia 21 27 20
Canada 20 21 20
Japan 15 11 18
Netherlands 13 13 12
South Korea 13 11 12
Italy 12 33 20
Spain 12 9 13
China Taiwan 11 7 7
Finland 9 8 6
India 9 5 1
Russia 9 7 2
New Zealand 8 5 2
Sweden 8 11 11
Switzerland 8 10 7
Belgium 7 7 7
Brazil 7 2 6
Argentina 6 1 1
China Hong Kong 6 6 5
Ireland 6 7 3
Denmark 5 6 5
Malaysia 5 1 2
Turkey 5 3 1
Austria 4 7 6
Israel 4 4 6
Norway 4 4 3
Portugal 4 5 3
Chile 3 2 2
Saudi Arabia 3 1 4
South Africa 3 4 4
Czech Republic 2 3 1
Greece 2 3 2
Mexico 2 1 1
Poland 2 1 2
Singapore 2 2 2
Egypt 1 0 1
Iran 1 2 2
Hungary 0 0 2
Serbia 0 0 1
Slovenia 0 0 1
All other nations 19 5 0

In Table 2, I compare the number of top 500 universities in different nations. The picture remains relatively unchanged when it comes to the dominance of the USA and the UK. These two nations combined, a total of 183 universities are among top 500 in the ARWU rankings, 179 in the THE rankings, and 146 in the QS rankings. But there are significant changes in terms of the positions of the top 10 nations. For instance, Germany has 39 and 35 universities among top 500 in the ARWU and the QS rankings respectively, and was ranked at the third place only after the USA and the US in terms of number of top 500 universities. South Korea has no top 100 university in any the of three rankings, but it has 12 top 500 universities in the ARWU rankings.

Data from the World Bank was sourced to calculate world economic capacity. National data of GNI (PPP, Constant 2011 international $b) were used: for most countries the 2014 data were used, for a dozen of countries where the 2014 data were unavailable the data in the most recent year were used instead. The World Bank does not have GNI data for a total of 29 nations, but most of them are very small in the economic scale, and they are thus excluded from the analysis. In the end, I calculated the GNI, the population, and the GNI per head for 185 nations.

Table 3: National shares of top 100 universities and their shares of world economic capacity, 2014-15

Nation Share of Top 100 universities, QS 2015 Share of Top 100 universities, THE 2015 Share of Top 100 universities, ARWU 2015 Share of world economic capacity, %
USA 30.0 39.0 51.0 31.7
UK 18.0 16.0 9.0 3.7
Australia 7.0 6.0 4.0 1.5
Japan 5.0 2.0 4.0 6.4
Netherlands 5.0 8.0 4.0 1.3
Canada 4.0 4.0 4.0 2.3
China 4.0 2.0 0.0 6.9
China Hong Kong 4.0 2.0 0.0 0.8
Germany 4.0 9.0 4.0 5.8
Switzerland 4.0 2.0 4.0 0.9
South Korea 3.0 1.0 0.0 2.1
France 2.0 1.0 4.0 3.5
Singapore 2.0 2.0 0.0 1.2
Sweden 2.0 3.0 3.0 0.7
Belgium 1.0 1.0 2.0 0.7
Denmark 1.0 1.0 2.0 0.4
Finland 1.0 1.0 1.0 0.3
Ireland 1.0 0.0 0.0 0.2
New Zealand 1.0 0.0 0.0 0.2
Argentina 0.0 0.0 0.0 0.5
Austria 0.0 0.0 0.0 0.6
Brazil 0.0 0.0 0.0 1.7
Chile 0.0 0.0 0.0 0.3
Czech Republic 0.0 0.0 0.0 0.3
Egypt 0.0 0.0 0.0 0.3
Greece 0.0 0.0 0.0 0.2
Hungary 0.0 0.0 0.0 0.2
India 0.0 0.0 0.0 1.4
Israel 0.0 0.0 2.0 0.3
Italy 0.0 0.0 0.0 2.5
Malaysia 0.0 0.0 0.0 0.6
Mexico 0.0 0.0 0.0 1.2
Norway 0.0 0.0 1.0 0.8
Poland 0.0 0.0 0.0 0.7
Portugal 0.0 0.0 0.0 0.3
Russia 0.0 0.0 1.0 2.7
Saudi Arabia 0.0 0.0 0.0 2.7
Serbia 0.0 0.0 0.0 0.1
Slovenia 0.0 0.0 0.0 0.1
South Africa 0.0 0.0 0.0 0.3
Spain 0.0 0.0 0.0 1.7
Turkey 0.0 0.0 0.0 1.0

Table 3 compares the national shares of top 100 universities and their shares of world economic capacity. USA performs well in the THE and ARWU rankings but not so well in the QS rankings. Countries such as the UK, Australia, the Netherlands, Canada, Switzerland, Belgium, Denmark and Finland perform well in all of the 3 rankings, with their shares of top 100 universities larger than their shares of world economic capacity.

Table 4: National shares of top 500 universities and their shares of world economic capacity, 2014-15

Nation Share of Top 500 universities, QS 2015 Share of Top 500 universities, THE 2015 Share of Top 500 universities, ARWU 2015 Share of world economic capacity, %
USA 19.0 24.6 29.2 31.7
UK 10.2 11.2 7.4 3.7
Germany 7.0 7.2 7.8 5.8
China 5.0 2.2 6.4 6.9
France 4.6 3.8 4.4 3.5
Australia 4.2 5.4 4.0 1.5
Canada 4.0 4.2 4.0 2.3
Japan 3.0 2.2 3.6 6.4
Netherlands 2.6 2.6 2.4 1.3
South Korea 2.6 2.2 2.4 2.1
Italy 2.4 6.6 4.0 2.5
Spain 2.4 1.8 2.6 1.7
China Taiwan 2.2 1.4 1.4 N/A
Finland 1.8 1.6 1.2 0.3
India 1.8 1.0 0.2 1.4
Russia 1.8 1.4 0.4 2.7
New Zealand 1.6 1.0 0.4 0.2
Sweden 1.6 2.2 2.2 0.7
Switzerland 1.6 2.0 1.4 0.9
Belgium 1.4 1.4 1.4 0.7
Brazil 1.4 0.4 1.2 1.7
Argentina 1.2 0.2 0.2 0.5
China Hong Kong 1.2 1.2 1.0 0.8
Ireland 1.2 1.4 0.6 0.2
Denmark 1.0 1.2 1.0 0.4
Malaysia 1.0 0.2 0.4 0.6
Turkey 1.0 0.6 0.2 1.0
Austria 0.8 1.4 1.2 0.6
Israel 0.8 0.8 1.2 0.3
Norway 0.8 0.8 0.6 0.8
Portugal 0.8 1.0 0.6 0.3
Chile 0.6 0.4 0.4 0.3
Saudi Arabia 0.6 0.2 0.8 2.7
South Africa 0.6 0.8 0.8 0.3
Czech Republic 0.4 0.6 0.2 0.3
Greece 0.4 0.6 0.4 0.2
Mexico 0.4 0.2 0.2 1.2
Poland 0.4 0.2 0.4 0.7
Singapore 0.4 0.4 0.4 1.2
Egypt 0.2 0.0 0.2 0.3
Iran 0.2 0.4 0.4 N/A
Hungary 0.0 0.0 0.4 0.2
Serbia 0.0 0.0 0.2 0.1
Slovenia 0.0 0.0 0.2 0.1
All other nations 3.8 1.0 0.0 9.0

In Table 4 I map the situation of top 500 universities. In this case, the USA performs worse than ‘it should’ in all of the 3 rankings, so do other large economies like China and Japan. Those countries who perform well in all of the 3 rankings in terms of their shares of top 100 universities also do well when it comes to their shares of top 500 universities. Many other nations perform well in Table 4 (but not in Table 3) in all of the 3 rankings, including: Germany, South Korea, Spain, Sweden, China Hong Kong, Ireland, Austria and Israel. For these nations, although they might not have a large number of top 100 universities, their universities are still among top 500.

In the case of Ireland, although there is reason to worry about Trinity College Dublin falling out of top 100 in the rankings (except for the QS rankings), it is also, maybe more, important to build a world-leading higher education system, with the overall quality of universities is among the best in the world.