Transcript ESRC and Impact - All Hands Meeting 2011

Demonstrating the impact of UK e-Research;
a research council perspective
Drs. Astrid Wissenburg, ESRC
Workshop 11: Profiling UK e-Research: Mapping Communities and
Measuring Impacts
UK e-Science
ALL HANDS MEETING 2008
Demonstrating the impact of UK e-Research;
a research council perspective

Definitions

Context and drivers

Current approaches - examples

Outstanding issues
What is (economic) impact? (1)
“An action or activity has an economic impact when it
affects the welfare of consumers, the profits of firms and/or
the revenue of government. Economic impacts range from
those that are readily quantifiable, in terms of greater
wealth, cheaper prices and more revenue, to those less
easily quantifiable, such as effects on the environment,
public health and quality of life. “ (Treasury)
“improvements for UK society and the economy” (RCUK)
What is (economic) impact? (2)
“The concept of 'impact' in the social sciences applies to
all sectors: public, private and voluntary. It embraces
economic and societal impact in the sense of direct and
often quantifiable economic benefits; wider social impacts
that will benefit society more generally such as effects on
the environment, public health or quality of life; and
impacts on government policy, the third sector and
professional practice. (….) Economic and societal impact
builds on academic impact: developing social science
knowledge and methods.” (ESRC)
Impact aspects

Academic and non-academic

On all sectors: public, private, voluntary – and
general public

Quantifiable and non-quantifiable

Direct and indirect

Policy, practice, products
Strategic drivers for impact agenda

Government’s Science and Innovation investment
frameworks emphasize the maximisation of public
investment in science on the economy

Research councils have been challenged to
demonstrate an increase in their economic impact

Major research challenges are interdisciplinary, and
require collaboration and co-production to ensure
quality research with impact
Purposes of identifying and measuring
impact

To learn lessons, specifically to understand, improve
and support the processes which create impact

To make the case for the science budget

To inform funding decisions
Identifying and measuring impact
Growing portfolio of evaluation methodologies to
‘measure’ economic and societal impact:

Across all types of ‘research activities’ – including eResearch

At all levels: national science budget, activity type,
funding mode, programmes, individual projects

Across the whole life-cycle of activities, including pre
and post
Example 1: RCUK Economic Impact study
(SQW/PA, 2007)

Series of 18 case studies across training, research
and facilities investments from all research councils.

Methodology: stakeholder interviews was
supplemented by data and secondary research

Used a classification of impact types:
 Development of human capital
 Business and commercial
 Policy
 Quality of life
Example 1: RCUK study - impacts
Examples of impacts from case studies include:

Use of scientific facilities by industry:
 direct income through selling access
 indirect impact through businesses' exploitation of
knowledge that is developed by using the facilities.

Software development and licensing revenue

Spin-off companies
Example 1: RCUK study - findings

A wide diversity of impacts emerging from research
funding, many not part of the original rationale for the
specific investment

Not possible to robustly extrapolate from the case
study findings to total spend of research councils,
partly because the methodology is resource intensive
and expensive

Lack of ‘management information’ about outputs from
investments, especially post project.
Example 2: DIUS study on large scale
facilities (2008)
Review of economic impacts relating to the location of
large-scale science facilities in the UK

Five facilities covered

Analysis of economic impacts arising from
employment, expenditure and knowledge transfer
and scientific benefits

Focus on benefit of the location of facilities in the UK
as opposed to access to similar facilities abroad.
Example 2: DIUS study on large scale
facilities (2008)
The major economic impacts arise from:

employment of relatively highly paid staff, most of whom reside
close to the facility

the awarding of contracts to UK-based suppliers
Plus:

contributing to local technology clusters but generally a
relatively small component

Some examples of transferring knowledge and technologies to
suppliers, but no evidence that this is on a substantial scale
Example 3: ESRC policy and practice
impact evaluations
Case study approach intended to cover different investments and
different methodologies, including:

“Payback” method: this method examines the stages of a
project throughout its life, and categorises its “paybacks”.

"tracking forward" from research outputs to the way in which
these have been incorporated into practice in the public and
private sectors.

an analysis of end of award reports to assess the potential for
impact, followed by an analysis based on "unobtrusive" or "nonreactive" measures involving web-based searches, and
databases of publications

a logic chain, mapping resource inputs to shorter term research
outputs and to longer term outcomes.
Example 3: ESRC policy and practice
impact evaluations

qualitative studies with mixed methods captures the
complexities and allows for triangulation

models that anticipate the communication flows
between researchers and potential users provide a
framework for analysis, but linear models assume a
trail of evidence which is seldom present
Example 3: ESRC policy and practice
impact evaluations – next steps
Economic impact study of 2 Research Centres:

Identify achieved economic impacts (through
research, dissemination, networking, etc);

Apply appropriate techniques to calculate values for
the economic impact of selected aspects

Present the results within a broader analysis of the
Centres’ policy and practice impacts;

Identify lessons for future economic evaluations
Methodological issues

How can we address the problems of
 Attribution
 Traceability
 Measurability
 Time lags?

How can we move beyond inputs and outputs as
proxies for actual impact?

How do we represent the receptiveness of users?
Research funders issues

Can/should we establish any robust baseline?

What is the relationship between individual projects to
programmes to investment portfolios to the science budget?
What we can do:

More systematic data gathering of output data and ‘impact’
stories, during and post projects

More testing of economic impact valuations

Extract the lessons about good pre-conditions and practice for
impact creation
E-Research Issues

Are the current approaches applicable to assess the
impact of e-Research?

Understand pathways to impact, both direct and
indirect, from e-Research

Impact of e-Research on the research process itself
and the implications for impact measurement

Can some e-Research tools be sued as part of a
portfolio to collect and analyse ‘impact’ information
Thank you
[email protected]