Transcript PRTLI in Ireland - Amazon Web Services

Evaluation of Science Foundation Ireland: A Case
Study of a Programme Designed to Enhance National
Research Quality
Dr. Jim Ryan
Michael Fitzgibbon
CIRCA Group Europe
CIRCA is an international consulting company, established in
1991 in Dublin, Ireland.
Member of ETAN (European Technology Assessment Network)
and the European S&T Observatory (ESTO).
12 Consultants in different areas of S&T, all with significant
experience of European S&T issues and systems.
The major Irish consultancy company specializing in S&T policy
and management issues.
www.circa.ie
2
About Forfás
• Forfás is the national board responsible for providing policy
advice to Government on enterprise, trade, science,
technology and innovation in Ireland.
• Legal responsibility for the promotion and development of
these sectors is vested in Forfás by the State, through the
Department of Trade, Enterprise and Employment.
www.forfas.ie
[email protected]
3
www.circa.ie
3
Presentation Outline
 Irish infrastructure – the context
 Background to Science Foundation Ireland

Evaluation Process
www.circa.ie
4
Ireland in Summary



Population 3.98 million
Total at work 1.81 million (was 1.1 million in 1990)
GDP per capita €33,919

A GROWING ECONOMY - 2003





GDP
Exports
Imports
Trade Surplus
€135 bn
€113 bn
€92 bn
€21 bn - 15.5% of GDP
The average annual rate of increase in Irish GDP
has been 9% over the 7 year period 1994 - 2000,
the highest in the EU and the OECD
www.circa.ie
5
S&T Context: Some notable features
1.
Ireland has historically had very low S&T investment
2.
Universities & Institutes of Technology are almost the sole public source of
technology & RTD services: Only Public RTD institute is in Food/Agric.
3.
S&T was a low priority, and the agency structure was under-developed and
staffed, until very recent times
www.circa.ie
6
Profile of Enterprise in Ireland
Most of the products
we manufacture are
designed elsewhere
Bulk of our exports are
marketed /sold by
organisations
outside Ireland
www.circa.ie
7
Changing the Capability Profile of Enterprise in Ireland
 Need to Build World Class
Expertise
 International Sales &
Marketing
 Operations
 R&D/Technology
 Focus on Sectors & Niches
www.circa.ie
8
Technology Foresight Ireland:
Conducted 1998; published by Forfás 1999
“A world class research capability in selected niches
of … enabling technologies is an essential foundation
for future growth. These key technologies require
new skills, and in particular the development of a
cadre of world class S&T personnel.”
www.circa.ie
9
Ireland –S&T & the Economy
1.
Foreign Direct Investment is important to economy (~130,000
jobs; 14% of GDP)
2. Innovative capacity seen as important to the attraction and
retention of FDI firms
3. Also vital to the growth of indigenous companies
Investment in RTD was seen as important:
o
o
To provide an incentive for Foreign firms
To develop indigenous high-technology industries
ENTERPRISE
IRELAND
www.circa.ie
10
“.... There will be a major accelerated
increase in Research, Technological
Development and Innovation (RTDI)
investment with the objectives:
 to strengthen the capacity of Irish third
level institutions and other research
establishments to conduct research relevant
to the needs of the Irish economy;
 to strengthen the capacity of Irish firms
to assimilate the results of R&D into their
products and processes, and;
 to provide support for sectoral research
in agriculture, food, marine and the
environment....”
www.circa.ie
11
RTD and Innovation Funding in
National Development Plan 2000 – 2006
Total : €2.5 billion
PRTLI
Capital Facilities &
Research
Infrastructure in
Universities
€690m
Other Sectoral &
Industry Supports
Science Foundation Ireland.
Quality and Scale of Basic
R&D
€1,160m
€650m
www.circa.ie
12
Research Community view: SFI v PRTLI
PRTLI provides
infrastructure for
research performance
Bricks & Mortar
Science Foundation
Ireland provide
researchers
Grey Matter
www.circa.ie
13
Programme for Research in Third Level Institutions
(PRTLI)
Forced Universities to develop Strategic Plan for RTD
Built 33 new Research Centres
Provided an additional 90,000 m2 research space
Funded 62 new or expanded research programmes
Increased undergraduate Post-grad places by approx 1,500
Encouraged inter-disciplinary research and inter-institutional collaboration
Encouraged greater interaction between research and teaching
Conway Institute –
Dublin
Nanofabrication facility
– Cork
Materials & Surface
Science Institute –
Limerick
www.circa.ie
14
‘Helping Ireland Recruit and Retain Research Groups’

SFI was established in 2000 and became operational in
2001

‘SFI is investing €646 million in academic researchers who
are likely to generate new knowledge, leading edge
technologies, and competitive enterprises in the
fields underpinning two broad areas:
 Biotechnology
 Information & Communications Technology’
Vision Statement
Through strategic investments in the people, ideas and partnerships essential to
outstanding research in strategic areas, SFI will help build research of globally recognised
excellence and nationally significant economic importance.
www.circa.ie
15

Grants are allocated solely on the criterion of excellence as
determined by external, international peer review: relatively
unusual in EU

Have established 163 research groups led by principal investigators
of whom 34 have come to Ireland from laboratories abroad

SFI groups employ over 1150 research staff and 450 PhD students

Established 6 major R&D Centres in collaboration with industry
Schemes
Committed €m
2001-2005 (April)
No. of Awards
2001-2005
Investigators and Fellows
254
128
CSETs
108
6
Research Professorships
30
10
Research Frontiers
12
107
Young Researcher
5
4
Other
46.5
182
Total
462.5
437
www.circa.ie
16
Six Centres for Science, Engineering and Technology
(CSETs) have been established in core thematic fields and
with considerable industry involvement.
The CSETs are in the following areas :






Adaptive Nanostructures and Nanodevices
Regenerative Medicine
Digital Enterprise Research
Alimentary Pharmabiotics
Telecommunications Value Chain
Human Proteomics
(CRANN in Trinity College Dublin)
(REMEDI in NUI Galway)
(DERI in the NUI Galway)
(APC in University College Cork)
(Trinity College Dublin)
(College of Surgeons, Dublin)
www.circa.ie
17
Review of the performance and impact of SFI to date.
 Requested by Department of Enterprise, Trade & Employment at end ‘04
 Managed by Forfás.
 Department ‘Accept that SFI is a long-term investment ...
 …. but also know that the early stages are critical’
 it is therefore ‘not too early to form an initial impression about the
operational performance of SFI to date’
www.circa.ie
18
Review of the performance and impact of SFI to date.
Evaluation Methodology:
1.
International review panel led the evaluation. Small size of Ireland
2.
Consulting groups were commissioned (by public tender) to provide
support and information inputs to the panel.
makes it difficult to find local independent panel-members for such
evaluations. External panels are therefore the norm.
 CIRCA Group conducted the Peer Review Assessment and the
Bibliometric analysis ( with CWTS, Leiden, NL)
 Technopolis (UK) conducted a survey of industry views
www.circa.ie
19
Members of International Panel
 Professor Sir Richard Brook (Chairman)
Director of the Leverhulme Trust (UK)
 Prof. Karin Markides
Deputy Director General of Vinnova, Swedish agency for Innovation (Sw)
 Dr. David Clark
Chair of Computer Science & Telecomm Board, US National Academies (US)
 Dr. Wilhelm Krull
Secretary General of the Volkswagen Foundation (D)
 Prof. David Finnegan
Institute of Cell & Molecular Biology, Univ. of Edinburgh (UK)
 Mr. Pat Toole
Retired Senior VP of Corporate Manufacturing & Technology of IBM (US)
www.circa.ie
20
Terms of Reference
Appropriateness or Efficacy:
Are the objectives still consistent with the Irish research system and with national
research and innovation policies?
Is the investment in SFI appropriate to meet its objectives?
Are there sufficient and appropriate complementary measures to SFI which would allow the
desired economic effects to materialise?
Effectiveness:
Is SFI effective in meeting its objectives?
Are its programmes and activities likely to lead to the desired outcomes?
What are the outputs and impacts of its activities?
What impact is SFI having on the research system as a whole?
Efficiency:
Examine the operational efficiency of SFI in relation to a range of relevant issues (e.g.
application and review procedures, monitoring).
www.circa.ie
21
Panel Methodology
Conducted discussions with a representative cross-section of
relevant groups:
 SFI-Funded Researchers (44) , post-doctoral staff (33)
and students (34) funded under SFI awards;
 SFI management and administration (5)
 Management staff of institutions housing SFI researchers
(13)
 Other key stakeholders: Public agencies, National
Research Funding bodies and private sector (6)
www.circa.ie
22
Evaluation of the Peer Review system
CIRCA Group
Process Design
 Peer selection: qualifications, nationality*, experience, gender, etc
 Peer process: criteria used, information provided to peers; ranking
and rating processes;
 Internal SFI Process: mechanisms and speed of decision-making;
rejection rate etc
 Supports provided to successful and unsuccessful applicants
 Oversight and monitoring: conflict-of-interest rules; appeals process;
internal audit/evaluations
Process Operation
 Random Case files were examined to assess the practical operation of
the review process. These were balanced by discipline, university,
successful/unsuccessful applicants, and by type of award.
* No Irish reviewers are used
www.circa.ie
23
Bibliometric Analysis
CIRCA Group and CWTS (Leiden)
Two studies were commissioned to assess:

Quality of the SFI-funded researchers from non-Irish institutions
prior to their coming to Ireland. This was to determine whether the
SFI system had selected excellent researchers

The publication record of researchers who had been working in Irish
institutions prior to funding. This was to assess whether the receipt
of funding had affected their publication output and/or quality
Note that appointments of researchers by SFI commenced in 2001
www.circa.ie
24
Percentage of SFI-funded researchers with a higher than
average number of papers in the top 10% and Top 5% of papers
in their respective fields
Biotech
Top 10%
ICT
Top 5%
Top 10%
%
Top 5%
%
Foreign*
Researchers
82
64
73
80
Resident
Researchers
68
64
75
50
* i.e. researchers who were working outside Ireland when receiving SFI award
www.circa.ie
25
Industry View of SFI
Technopolis (UK)
In-depth face-to-face or telephone interviews with 24 companies
were conducted in January/ February 2005.
The interviewees were from a mix of large foreign-owned
multinationals, large indigenous firms and small high technology startups, all operating in business sectors with an interest in biotechnology
or information and communications technology research.



Generally very positive about SFI approach and activities
Some concern about lack of incentives in HE sector re IP
protection etc
Recognise that time is required for policy to bear fruit
www.circa.ie
26
Overall View of Panel
 SFI is a ‘bold and well-designed path to transform Irish research’
 There are ‘persuasive indications of the suitability and effectiveness
of its process’
 The defining aspect of SFI is its ‘demand for excellence in research
quality’
 Continued success will need ‘ongoing commitment to sustenance of a
research culture by government and universities’
Report to be formally published by Forfás in November 2005
www.circa.ie
27