Transcript Ex post evaluation – WP 6c

Ex post evaluation – WP 6c
Support to enterprise – a
counterfactual approach
Daniel Mouqué
Evaluation Unit, DG REGIO
1
Enterprise & innovation support
• Some €79 billion in 2007-13: the largest
broad category of expenditure
• Key instrument: the grant (others include:
loans/VC, advice, networking, incubators)
• Previously, evidence for success was mainly:
– Monitoring data
– Interviews on deadweight
2
The evaluation
• 2 grants (modernisation, R&D) in E. Germany
• 2 databases: IAB enterprise « panel », GEFRA
survey of innovation in Thuringia
• Focus groups with project and prog managers
• 2 scientific experts
3
What is a counterfactual?
• Used in many methods (but often implicit or
qualitative)
• New tool borrowed from medicine and science.
Key feature: control group (think drug trial)
• Harnessing the power of statistics (pro: credible,
con: data-heavy)
• Selecting matches: can range from very simple to
very complex.
4
Main methods of this evaluation
• Controlled difference in difference (take a simple
method, add regression)
• Propensity Score Matching (using statistics to
find “twins”)
• Instrumental Variable (using grants to female
entrepreneurs)
5
Results: investment grants
Investments in euros/employee
Impact of investment grants
20000
18000
16000
14000
12000
10000
8000
6000
4000
2000
0
Additional investment
from firm
Contribution from grant
Baseline
Non-assisted firms
Assisted firms
6
Results: R&D grants
Investment in euros/employee
Impact of R&D grants
14000
12000
10000
8000
Contribution from grant
6000
Firm investment
4000
2000
0
Non-assisted firms
Assisted firms
7
Results: employment
• An estimated 27,000 jobs created
• Significantly lower than monitoring data for jobs
created (107,000). Reconciliation: gross/net.
• Cannot reconcile with number of jobs safeguarded
(439,000)
=> Strong conclusion: main effect of grants is
investment (and productivity) change, not jobs
8
In conclusion
• Interesting results
• An interesting method (potentially more
rigorous, gives clear headline figures)
• But data-heavy, so not appropriate in every
case
9
Where to find the evaluation
Inforegio, then the following steps:
> The Policy
> Impacts and results
> Evaluation
> Evaluations undertaken for the Commission
> 2000-2006
10