www.anrt.asso.fr

download report

Transcript www.anrt.asso.fr

A few remarks on ARWU*
Edouard Mathieu
Head of the Benchmarking Center
Invest in France Agency
* ARWU: Academic Ranking of World Universities 2005
Some remarks on methodological issues
 ARWU is a useful idea. The difficulties are honestly
documented by prof. Liu. Some will be hard to overcome.
 Among difficulties: the attribution of citations (example in
Paris: Paris-6 or UPMC, careers in CNRS laboratories, College
de France, Polytechnique…). Does Shanghai Jiao Tong
University benefit from a partner in France to help with the
French attributions?
 Nobel and Fields medals are skewed indicators: only 12
universities score > 50 for alumni ; 305 universities score 0 ;
372 universities score 0 at awarded staff. Beyond the top 10,
these variables do not discriminate academic performances.
 Only one measure for all world diverse universities and
schools makes global ranking difficult to interpret. We should
rather compare comparable units.
© AFII 2005
2, avenue Vélasquez
75008 Paris
2
How to gain 10 ranks? Paris-6 versus UT at Austin
 Ranking. Paris-6 (Université Pierre et Marie Curie, UPMC) ranks
46, UT Austin (The University of Texas at Austin) ranks 36. Why?
 Enrollment. Paris-6 30,000 (science and medical students);
UT Austin 50,000 (architecture, liberal arts, business, law, natural
sciences, engineering, pharmacy, nursing…)
 Employees. Paris-6 : 4,000 faculty and 3,000 staff; UT Austin :
3,000 faculty and 18.000 staff (laboratories, museum, library,
sports…). UT Austin enjoys big budget and managing powers.
 Scores. Paris-6 much better at alumni, awards, size (per capita)
and equal at SCI. UT Austin better only at HiCi and N&S.
 Conclusion. Paris-6 matches UT Austin at academic
performances. Size not an issue here. Strong English language
bias.
© AFII 2005
2, avenue Vélasquez
75008 Paris
3
Remarks on communication issues
 ARWU is meant to measure scientific competences, but it is
generally perceived as a global ranking, including teaching
performances and efficiency.
 Academic indicators are included because they are related to
the “reputation “. It implies that ARWU reinforces initial
reputations ; a ranking should rather help detect good
practices.
 The relationships between academic performances, teaching
performances and efficiency at meeting the economy needs
should be somewhat elaborated.
 Beyond the top ten, it would be much more appropriate to
communicate only classes of equivalent level universities :
UT Austin and Paris-6 would play in the same league (World
class) beyond the World top twenty, and not ranked 36/46.
ARWU already uses such classes beyond 100: classes 101152, 301-400.
© AFII 2005
2, avenue Vélasquez
75008 Paris
4
Impact on university management
 Most indicators are absolute numbers, and so ARWU favors
big size. Is bigger better than smaller? No universal answer.
 Paris (as other French cities) boasts many specialized
universities, schools and organizations. This dispersion may
send the wrong message through ARWU. Some experiences
are made to join forces for better visibility: Paris Tech…
 Prof. Liu intends to increase the weight of size (per capita
criteria). This will make ARWU more neutral on the size
question. But comparable evaluations of faculty and staff
numbers will not be easy to elaborate.
 Same difficulties with the specialization / concentration
management alternatives and corresponding rankings.
© AFII 2005
2, avenue Vélasquez
75008 Paris
5
Impact on student international mobility
 ARWU should be a helpful tool for mobility (both ways). Is it?
 Probably a significant impact: 2000 visitors on the Web a day
(Chinese? From developed countries?)
 Probably less impact on the top excellent students who
receive direct offers and information from universities and
schools.
 Impact on professors / researchers national and international
mobility ? (which would boost future impact on student
mobility).
© AFII 2005
2, avenue Vélasquez
75008 Paris
6
Academic ranking and countries' attractiveness
 Top 500 Universities list gives a ranking of countries for
research : US (1st), UK, Japan, Germany, Canada, France (6th),
Sweden, … China (19). How it compares with rankings in the
“France attractiveness scoreboard”?
 Proportion of international students enrolled in tertiary
education: Australia, Belgium, UK, Germany, France, USA,
Netherlands, Spain, Japan…
 R&D expenditure/GDP (average 2001-2003): Japan, USA,
Germany, France and Belgium (ex.), Netherlands, UK…
 ARWU and the Attractiveness Scoreboard agree on the global
picture. But France’s 6th global position is a more solid
assertion than separate universities rankings, with a 46th
ranking for Paris-6, 101-152d ranking for College de France.
© AFII 2005
2, avenue Vélasquez
75008 Paris
7
Edouard Mathieu
Head of the Benchmarking Center
Invest in France Agency