Transcript Nessun titolo diapositiva

International Conference on
Technology Policy and Innovation 2011
Bogotà, Colombia
September 8 - 9, 2011
Indicators for Science, Technology and
Innovation as a tool to evaluate effectiveness of
policies and programs
Giorgio Sirilli
National Research Council of Italy
The evolution of S&T indicators
The first attempt to measure S&T in 1957
Frascati Manual (1963)
The Frascati manual “family”
A continous process of broadening and deepening
The role of international organisations
The dialogue between producers and users
Lord Kelvin
“If you can not measure it, you can not
improve it.”
Lord Kelvin
Hariolf Grupp
“If you can not measure it, don’t talk
about it.”
Ariolf Grupp
The founding fathers - 1968
Villa Falconieri in the 60s
OECD Group of National Experts on
Science and Technology Indicators NESTI
NESTI
NESTI: Working Party of National Experts on Science and Technology
Indicators (1962)
Mission:
Produce methodologies, statistics and analyses
Co-ordinating body
Clearing house
S&T indicators: the triangle
The triangle: data producers, analysts, policy makers
producers measure the past
analysts interpret today
policy makers design the future
Fred Gault
“Policy analysts should be both literate and numerate, able to put a
case using innovation indicators. Not only should the analysits have
such a skill set, but they also require some knowledge of the subject. It
is in this environment that monitoring, benchmarking and evaluation
lead to policy learning and to more effective policies.”
Keith Pavitt
“One would think that the political agenda determines
the collection and analysis of indicators. In reality it is
the other way round: it is the availability of indicators
which steers the political discourse.”
What is a Manual?
Manuals are codified knowledge
They are guidelines for the collection and interpretation of data
and for international comparisons of data, statistics and
indicators.
They are supported by an international infrastructure (ISIC, ISCED
and ISCO)
Manuals provide a language of discourse and they behave like a
technology (our guiding assumption)
International organisations
Organisations active in the field of science, technology and innovation
are:
Eurostat and the relevant Commission DGs and member states of
the EU
OECD and member countries
RICYT and member countries
United Nations Institute of Statistics
NEPAD is moving in this direction
Indicators
Indicators are a technology, a product, which
- governs behaviour
- is modified by users (outside of the producer community)
- develops in response to user needs
Indicators
Data sources
Surveys, Administrative data, Case studies
Data collection is informed by manuals
Data populate statistics which can be indicators
Decisions are taken on the basis of indicators
Science and technology
Science and technology is different from innovation
Science and technology includes R&D and related S&T activities
There are ministers of S&T, rarely of innovation
S&T happens in
Higher education, research labs and in business
Innovation involves people in firms connecting to the market
They do not necessarily do R&D
These are fundamental differences
Innovation
A systems approach
Actors: Governments, education and health institutions, business,
foreign institutions
Activities: R&D, invention, innovation, diffusion of technologies and
practices, HR development
Linkages: Contracts, collaborations, co-publication, grants,
monitoring
Outcomes: Jobs, growth, wealth
Impacts: Wellbeing, culture change, global influence
The activity of innovation is dynamic, complex, non-linear and
global
Research and development – R&D
R&D, OECD (2002)
The Frascati Manual
Provides a definition of R&D, and examples of what it includes
and does not include
R&D is a performance measure, not a funding measure
Designates sectors for measurement: government, higher
education (includes hospitals); business enterprise; private
non-profit; and, abroad (funding only)
R&D
R&D expenditure is a measure of the formal creation of knowledge
R&D human resources (FTE or Head Count) are an indication of the
size of the commitment to the creation of knowledge.
The aggregate measure is the GERD
Gross (not Net) Domestic (not National) Expenditure on
(performance, not funding) R&D
R&D
GERD is used for target setting
Africa 1% of GDP
EU 3% (2% from business, 1% from public sector)
U.S. > 3%
R&D performance is supported by public policy
Grants, contracts, tax credits, technical assistance, scholarships
and training support
Capitalisation of R&D
Some studies show:
US: “GDP would have been an average of 2.9 per cent higher
between 1959 and 2004 if R&D spending was treated as
investment “
UK: increase of GDP by 1.5%
Canada: increase of GDP by 2.9%
Innovation
Oslo Manual, Eurostat-OECD (2005)
Guidelines on collecting and interpreting data on innovation
Manufacturing
Services
Organisational and marketing innovation
New: industrial design
Innovation
What is innovation?
An innovation is the implementation of a new or significantly
improved product (good or service), or process, a new
marketing method, or a new organizational method in business
practices, workplace organization or external relations.
Implementation connects innovation to the market
The activity of innovation is not an isolated event
Measurement
Bogotá process: to include in the Oslo manual
Service industries
Agriculture
Innovation and the Development Agenda
Moving agriculture from subsistence to a knowledge intensive
industry (Juma)
Patents
Patents, OECD (2009)
The Patent Manual
Guidelines on collecting and interpreting data on triadic families of
patents. (Patents filed at the European Patent Office (EPO) and the
Japan Patent Office (JPO) and granted by the US Patent and
Trademark Office (USPTO))
Resulting data provide a rich resource for analysis about existing and
emerging technologies – and the relative specialization of countries
Technological balance of payments
TBP Manual, OECD (1990)
Guidelines on collecting and interpreting data on payments and
receipts for
R&D services
Intellectual property licenses for patents, copyrights, trademarks
and technical assistance
Brings together R&D and IP measurement
Raises interesting analytical questions
What does a positive balance mean? Is it good?
What about a negative balance?
Human resources for S&T
HRST, OECD / Eurostat (1995)
The Canberra Manual
Identifies HRST by S&T occupation (ISCO2 and3) and by qualification
(3rd level in S&T, ISCED 5 and 6)
HRST can be
Qualified & employed
Qualified & not employed
Not qualified but employed
Human resources for S&T
Used to analyse HRST
Stocks and flows
Employment
Nationality
Data are collected and distributed through Eurostat for the EU
OECD and partners work on the ‘Career Paths of Doctorate Holders’
Bibliometrics
There is no OECD bibliometrics manual
Bibliometrics is the study of:
Publications in selected journals
Citations of articles published and, on occasion,
Co-publication
Bibliometrics and similar patent analysis can provide S&T indicators
Technology
There has been work on the production and use of technologies and
practices
Advanced Technologies
Biotech
ICTs (goods and services)
Nanotech
Emerging technologies (and practices)
Knowledge Management
New activities
Understanding the ‘market’
Public sector ‘innovation’
Including services to agriculture
User innovation
Sharing of knowledge in the peer group
Social innovation
Improving measurement
The relationship producers-users
After WWII the main issue in S&T policy was the social responsibility of
science. Now the attention is placed on the social return of investment
on S&T and, in particular, on innovation
Weak innovation theories do not allow a straighforward interpretation of
indicators
A tension: the simplification of policy makers (e.g. the 3% R&D/GDP
ratio) versus deepening of analysts
The time dimension: demand of indicators and analyses (quick and dirty)
but …. the construction of indicators is a “heavy ship” (more than 10
years to build)
Chris Freeman and Luc Soete
“The link between the measurement of national STI (science, technology
and innovation) activities and their national economic impact, while
always subject to debate, particularly in the context af small countries,
has now become so loose that national STI indicators are in danger of
non longer providing relevant economic policy insights”
The policy advisor
Advising has become the preserve of applied economists (no more
physicists and sociologists)
Advisors need to be – and to appear – experts who guarantee a
competent and independent approach
The compromise between engagement and integrity of analysts
Frustration of advisors is part of the game
The paradox: too many or too little indicators?
The pressure of vested interests on official statisticians
Indicators community
The members of the indicators community have to play
the difficult game of finding a proper balance between
data collection, analysis and policy making – preserving
their intellectual integrity
Use of indicators
(Benoit Godin)
The rhetoric of numbers
Policy prescriptions based on shaking statistical evidence (New
Economy)
The “umbrella” concept, slogans, buzzwords, which shape new ways
to arrange old indicators
Use of indicators
(Benoit Godin)
As OECD admitted: “Monitoring and benchmarking are not coupled
with policy evaluation (…). They are seldom used for evaluation
purposes (…) but to analyse [counties’] position vis-à-vis competing
countries and to motivate adaptation or more intense policy efforts
(…).”
“Official statistics mainly served discourse purposes, and in this sense
the accounting framework and the statistics presented within it
were influential because they fit perfectly well with the policy
discourse on rationality, efficiency and accountability: it aligns and
frames the science system, by way of statistics, as goal-oriented
and accountable. As it actually is, the accounting in official statistics
on science is a metaphor, not an accounting exercise as such”
Conclusions
Challenges
International cooperation is essential
Resource constraints
Increase of areas of work
Enlargement with ensuing management problems
Support to non-OECD countries
Equilibrium between producers and users
Stay creative, autonomous, and risk-taking
Thank you for your attention
[email protected]