Transcript Causal inference in the social sciences

Causal inference in the
social sciences
Why mechanisms matter and how to find out about them
by Attilia Ruzzene
Erasmus Institute for Philosophy and Economics, Rotterdam
[email protected]
About me:
 Phd dissertation: “Causality in economics. A
methodological inquiry”
◦ (defended Feb 2010, department of economics Cognetti de Martiis,
Torino)


Currently, Phd student at Erasmus Institute for Philosophy
and Economics (EIPE, Rotterdam):
“Tracing the process: why and how mechanisms matter in
the social sciences”
Today presentation:
◦ Brief summary of my past research
◦ Insights to pursue further
◦ Pending problems in need of a solution
Overview
General motivation to my (first) project:
 Relevance of causal knowledge to the variety of purposes economists
(and social scientists as well) pursue:
◦ (To differ extent) explanation, prediction, intervention.

Difficulties in having that knowledge reliable

Hence, the questions:
◦ J.S. Mill (1843), on the difficulties of applying its methods for causal inference
to a science that is not experimental
◦ Marshall (1870), on the complexity of causal forces as to their number, (lack
of) regularity, way of combining (chemical rather than mechanical)
◦ How do economists attain causal information (the methods in use);
◦ What type of causal information this is (do different methods provide causal
knowledge that is diverse?);
◦ What purposes it is intended to serve (what type of causal knowledge serves
what type of purposes).
“Causality in economics. A methodological
inquiry”

Economists use a variety of methods to draw causal
inferences:
◦ Statistical methods (SEM, Bayes Nets);
◦ Experiments (thought ex., natural ex., laboratory ex.);
◦ Case-based methods (process tracing).

I selected two cases –instances- of scientific practice
and analysed them in detail to see:
◦
◦
◦
◦
How the methods work in the specific context;
What assumptions they rely on;
What causal information they provide;
What the specific causal information is (thought) useful for.
The methods in use




Case study research: Regional 
Advantage. Culture and competition
in Silicon Valley and Route 128 by
A. Saxenian 1994;

Techniques of causal analysis:
small-n method of comparison
and process tracing;
Type of evidence: correlation 
and mechanistic;
Research purpose: explaining 
socio-economic phenomenon
(asymmetry in regional
development).
The sample
Structural econometric model:
Marshallian causal functions by J.
Heckman 2000, 2001, 2008;
Technique of causal analysis:
controlled (ceteris paribus)
variation in thought
experiments;
Type of evidence: causal
effects
Research purpose: measuring
the impact of economic and
social policies on outcomes of
interest (intervention).

Mechanisms play a crucial role in the case-study research: they help causal
inference.
◦ In particular, the mechanism gives structure (and order) to the causal backbone (the
web of causal relations) around which the historical narrative is articulated.
◦ If the mechanism was not detected the narrative would not be possibly recounted.
◦ Thus, process tracing –and the mechanism it detects- are necessary in the circumstances,
though not sufficient by themselves, for the causal inference.
 The small-n method plays a complementary and equally necessary role.

Mechanisms play no role in Heckman’s proposal for evaluating intervention.
However, they should.
◦ MCF for causal effects are irrelevant in the range of circumstances in which
interventions affects the causal structure in intended and unintended ways .
◦ These alterations need to be taken into account if one wants the model to evaluate the
impact of intervention.
◦ I believe that what is needed in these circumstances is a model for causal mechanisms.
Some conclusions
on the role of mechanisms

Mechanisms matter in the social sciences.
They help us attain epistemic and non epistemic goals by:
1.
2.
3.


Helping causal inference ;
Facilitating extrapolation;
Evaluating interventions.
This formulation is too vague: are mechanisms necessary and sufficient for 1,
2 and 3?
No. Whether and how they matter depends on the context.
There is no general demand for mechanisms (Kincaid 1997).
◦ Sensible proposal. By itself, however, it does not tell us when and how mechanisms
matter. It simply relocates the methodological investigation and places it closer to the
actual scientific practice.

What are the contextual features that make the search for mechanisms
desirable?
◦ Daniel Steel argues that it depends on the available information. In the social sciences it
happens sometimes that we can more easily access the mechanisms and their
components than the causal relationships at the aggregate level (Steel 2008).
Mechanisms matter; but, how?
What types of clues suggest that the search for mechanisms is desirable in
the circumstances?
 General clues: field of investigation, research purposes, subject of
study, applicable methods, background knowledge.
 More specifically, consider Heckman’s MCF for measuring
intervention. (If my conclusions are correct) causal effects are
irrelevant in the circumstances in which:
◦ The causal structure is affected by the intervention;
◦ The intervention operates by jiggling distant rather than proximate causes of
the effect of interest.

These are contextual clues. As such, they

The need for clues is consistent with the idea that in the social
sciences reliable knowledge of mechanisms is not easy to obtain...
◦ do not require one to have full knowledge of the causal structure in place;
◦ point out the irrelevance of a model for causal effects and the need of a
model for mechanisms.
Contextual clues

Mechanisms help scientists attaining epistemic and non epistemic purposes
under conditions:
◦ They help causal inference only if they solve the problem of confounders (spurious
causal relations);
◦ They help extrapolation only if they bear external validity besides being internally valid;
◦ They help assessing interventions only if they are relevant to the policy implemented.
The conditions above can be fulfilled only if mechanisms are detected correctly

This is a major issue for the social sciences because social mechanisms are
not there ready to be picked up:
◦
◦
◦
◦
◦

They are not directly observable;
They consist of social practices that have to be interpreted (Steel 2004, 2008);
They are bound to uncertainty;
They are context-dependent;
Various mechanisms can operate simultaneously.
Therefore, scientists need first to find out methods that give reliable evidence
of causal mechanisms if they are to use those mechanisms as valid evidence
for causal and policy hypotheses.
Mechanisms matter only if valid

Process tracing is the method for detecting mechanisms in
the social sciences.
◦ Whether process tracing provides valid evidence for mechanisms
and how (at what conditions) is related with how we define
mechanisms in the first place;
◦ If we understand social mechanisms as regular sequences of events
(Little 1991, Bunge 2004, Mayntz 2004), process tracing should test
the stability in the association of events;
◦ but then it would fall prey of the same problems that plague
statistical inferences (spurious correlation).
◦ If we understand social mechanisms as sets of social practices
(Steel 2004 2008), process tracing is to provide evidence for the
existence of those practices. What we need to establish is then how
social practices are to be interpreted unambiguously.
The troubles of process tracing
To sum up:
 Social scientists make extensive use of mechanisms and
mechanistic language,
 And philosophers and methodologists promote the use of
mechanisms for epistemic and non epistemic purposes.
 However, how mechanisms actually matter and to what
extent seem to depend on the context.
 Moreover, they can help solve the aforementioned
problems if correctly detected. Hence, the importance of
1.
2.
Identifying the relevant contextual features:
Clues that tell the researcher that the search for mechanisms is fruitful
in the circumstances;
Identifying the conditions under which process tracing secures
the validity of the mechanisms it detects.
To conclude