Transcript presentation

Research evaluation strategies
in Finland and Estonia
Practical management of
research evaluation in Estonia
Tiit Laasberg
Estonian Higher Education
Accreditation Centre
1
2
Research evaluation in Finnland
Finnish universities are carrying out the evaluation
of all its research every six years
The evaluation combines an external assessment by
international Evaluation Panels with an internal selfassessment exercise.
The purpose of the evaluation is to support the
development of research in future.
The main objectives of the external evaluation are: to
examine the quality of the research of the units
during last five-years period, and to advise how to
develop the University’s research in the future.
The results of the external evaluation of the quality of
research will have an effect on the funds within the
University.
3
Research evaluation in Finland
Summary of the self-evaluation documents
1. Staff
1a. Statistics on all staff of the Unit
1b. List of research active staff
2. Publications
2a. Description of the research profile
2b. Complete list of publications
2c. Selected compilation of publications
2d. The best publications
3. Doctoral theses and degrees
3a. List of doctoral theses
3b. Number of doctoral degrees
•
•
•
•
•
•
•
•
4
Research evaluation in Finland
4. Self-assessment exercise
• 4a. Implementation of the ’99 recommendations
• 4b. Evaluation of strengths, weaknesses,
opportunities and challenges
• 4c. Interaction between the research of the Unit and
society
5. Other academic activity and collaboration
• 5a. Visits abroad
• 5b. Visiting academics
• 5c. Invited talks
• 5d. Editorial and other academic assignments
5
Research evaluation in Finland
6. Funding
• 6a. Funding from the budget of the university
• 6b. External funding
7. Description of the Programme (concerns only the
Research Programmes of the Faculty of Medicine)?
• 8. Description of the mission and structure of the
Unit (concerns only Units which will be evaluated by
a Sub-Panel)?
9. Signature of the Head of the Unit
6
Research evaluation in Finland
Evaluation Report
Name of the unit of assessment:
Panel:
1 Scientific quality of the research
Rating (1–7):
Supporting Comments:
2 Interaction between research and society
3 Overall evaluation: strengths, weaknesses, opportunities and challenges
3.1 Strengths
3.2 Weaknesses
3.3 Opportunities and challenges
4 Panel’s recommendations for th e future
4.1 Research - both single disciplinary and interdisciplinary research
4.2 Development of research environment and infrastructure
4.3 Research active staff
4.4 The role of doctoral/post-doctoral training in research
4.5 Other issues
7
Research evaluation in Estonia
First international evaluation was organised 19921993 by Estonian Science Foundation. The
external evaluators were Swedish colleagues.
Unfortunately, the results of the evaluation were
not used.
The second international evaluation were 20002004. This was organized by the Ministry of
Education and the Estonian Science Competence
Council. Practical arrangement was made by the
Estonian Higher Education Accreditation Centre.
8
Research evaluation in Estonia
The purpose of the evaluation shall be to:
1) judge the activities of research and development institutions
and the research topics implemented by them to ensure the
state funding for internationally recognised research and
development;
2) identify deficiencies in the activities of research and
development institutions;
3) give recommendations on the development concerning
research and development and research areas necessary
to the state of Estonia
Strategy – simultaneusly were evaluated all research groups
in the same specific field, for example in physics,
mathematics, organic chemistry etc.
Process – self-evaluationreport
→ site visit → expert report
52 experts teams evaluated 418 reseach groups
9
1
1
2
Island
Slovenia
Switzerland
Netherlands
Portugal
France
70
20
Finland
30
Sweden
UK
Latvia
Germany
6
Belgium
1
5
Denmark
1
5
Estonia
1
5
Hungary
1
Italy
0
1
3
Chech
10
Austria
EXPERTS OF THE RESEARCH EVALUATION
66
60
50
40
23
20
10 11
10
0
mining
hum.geogr./demography
semiotics/culturology
politology/publ.adm.
psychology
physics
arts
natural geography
power and elect.engineer.
astronomy/atm.physics
philosophy, theology
philology
ethno./social antr./cult.
literature, folkloristics
MB, G, MI, FY, VI
mathematics
mechanics
history
physical chemistry
foristry
geology
IT and systems
zoology
material science
sociology
pharmaco./pharmacy/toxic.
law
mechan/apparatus eng.
hydrobiology
ecology
botany/mycology
educational sciences
envir.techn./chem.engin.
org./bio./biorg. chem.
biomedicine
veterinary medicine
geodesy and build. techn.
clinical and other med.
agriculture
animal breeding
agricult.engineering
bio and food technolgy
fishery
Percentage of research group recieved grade above avarage
100
80
60
40
20
11
1
2,0
12
astronomy./atm.phys.
mathematics
physics
arts
literature, folkloristics
linguistics
history
philosophy, theology
3,4
4,6
4,7
4,5
4,3
4,0
3,8
4,7
hum.geogr./demography
5,5
6
semiotics/culturology
4,6
4,0
3,7
3,4
3,3
3,0
3,8
4,7
ethnol./sotsial antr./cult.
psychology
politology/publ.admin.
law
sociology
educatational science
phys. chem.
MB, G, MI, FY, VI
2,5
4,0
mechanics
3,2
4,0
material sciences
org./bio./biorg. chem.
4,0
3,4
3,2
3,0
2,8
2,5
2,3
power and elect. engin.
mechan./apparatus eng.
IT and systems engineer.
mining
geodesy and build. techn
bio and food technology
envir.techn./chemic.engin.
agricult. engineering
3,8
5
natural geography
zoology
3,6
botany/mycology
3,1
3,5
2,7
4
geology
ecology
hydrology
fishery
3,3
3,0
2,8
pharmac/pharmac/toxic.
2,7
biomedicine
0,9
2,3
3
clinical and other med.
forestry
animal breeding
1,7
0
veterinary medicine
1
1,7
2
agriculture
RESULTS OF THE RESEARCH EVALUATION
ist
ry
av
ar
so
ag
cia
e
ls
ci
en
ce
s
hu
m
an
itie
ex
s
ac
ts
ci
en
ce
s
Es
to
ni
an
ch
em
te
ch
ni
cs
bi
os
ci
ag
ric
ul
tu
re
m
ed
ic
in
e
COMPARISION OF THE RESEARCH FIELDS
5
4,5
4
3,5
3
2,5
2
1,5
1
0,5
0
13
The self-assessment report of a research and development
institution must contain:
data on the academic career of head of a research topic;
data on the qualification of main performers of a research
topic;
aims of research topics;
data on scientific publications and other results of research;
data on research grants and the implementation of scientific
agreements;
data on development activities;
data on the introduction of the research results;
the financial report;
judgement on facilities and data on the existence of
equipment for fundamental research;
data on maintaining dissertations;
the development plan of the research and development
14
institution.
•
•
•
•
•
•
•
•
•
•
•
The summary of work of the evaluation committee must
contain the judgement based on the following criteria of
evaluation:
•the novelty of the results of research and development;
•the quality of research and development;
•the strategy and perspective of research;
•the competence of research groups and their capability for
development;
•success in applying for funds and grants;
•national and international co-operation;
•the implementation opportunities for the research results and
their importance for the Estonian society;
•the correspondence of research end development to the
international level.
15
16
Costs of the research evaluation?
•
•
•
•
2000 - 1720 000.2001* - 1200 000.2002 - 1400 000.2003 - 1785 000.-
ALL: 6105 000.- kr appr 400 000 EUR
* - evaluation the centres of excellent ca 500 000 kr
17