Transcript Instrumental Variable Estimation

Causality in Social Work Research
via Instrumental Variable
Estimation
Roderick A. Rose
Senior Research Associate
Carolina Institute for Public Policy
The University of North Carolina at Chapel Hill
Summary
• Rationale: endogeneity & causality
• Instrumental variable estimation
• The exclusion restriction: justification and
identification; logic; example
• Recommendations
Rationale:
Endogeneity & Causality
Endogeneity is pervasive throughout nonexperimental research.
What is Endogeneity?
This is a Directed Acyclic Graph, or DAG. Causality flows in the
direction of the arrow. This DAG represents what we might
want to show (3 examples), representing a causal relationship
between X and Y.
X
Y
Rental housing assistance
Employment
Career-relevant instruction
Math achievement
Health insurance
Health at time t
What is Endogeneity?
Endogeneity occurs as an undesired relationship between
variables in this causal relationship and other variables not
accounted for in the model.
Education Level
C
X
Rental housing assistance
Y
Employment
Eliminating endogeneity requires removing one of the arrows.
What is Endogeneity?
The arrows corresponding to the confounder may go in any
direction; endogeneity has no regard for the direction of the arrow.
Teaching quality
C
X
Career-relevant instruction
Y
Math achievement
Note that any relationships not modeled occur through the error
term. This is the definition of endogeneity: a correlation between an
effect an the error.
What is Endogeneity?
When simultaneity or reverse causality are needed, they are
represented in a DAG by lagged versions of variables.
Health at time t-1
C
b<0
X
Public Health insurance
Y
b>0
Health at time t
A Common Risk in Social Work Research?
When simultaneity occurs in the context of cross-sectional data
collection (i.e., one time point), we cannot be sure which effect we
are observing.
X
Public Health insurance
Y
b>0
or
b<0
Health
Note that this is not technically a DAG (because of the two headed
arrow).
Description vs. Causality
It is entirely appropriate to use regression methods to describe
relationships among variables without inferring causality.
Race/ethnicity
C
X
Y
Public Health insurance
Health at time t
This curved line represents an unspecified relationship between two
variables; implicit in this is that the relationship occurs through or
because of other variables.
Instrumental Variable Estimation
Notation
X = EV = endogenous variable
Y = DV = dependent variable
Z = IV = instrumental variable or instrument
The Instrumental Variable or IV
In large samples, the instrument is a variable Z that is highly
correlated with X and uncorrelated with the error.
Health at time t-1
Not measured (x-sect)
Z
C
X
Y
Public Health insurance
Health at time t
The IV must effectively randomize participants to conditions of X.
Like random assignment, it cannot be confounded by any C.
IV Estimator using Covariance (Diagram)
Circles represent variance; Source: Kennedy, 2003.
error
in Y
Y = DV
Used to
estimate
βOLS
Note the
loss of
efficiency.
Used to
estimate βIV
X = Endog Var
Note overlap
with error
Z = Instr Var
No overlap with error
No overlap with Y outside of X
IV Estimator (Diagram)
Circles represent variance; Source: Kennedy, 2003.
Y = DV
Predict X’s
covariance
with Y
Predicted
X
X = Endog Var
Z = Instr Var
IV Estimator (Diagram)
Circles represent variance; Source: Kennedy, 2003.
Y = DV
r
X
X = Endog Var
Z = Instr Var
Common Instruments
• “Natural” phenomena including births, weather events, geographical
boundaries. Katrina is a good example.
• Randomized treatments. These are a particular class of IVE that are
used when doing mediation or dosage type models (local average
treatment effects). This is also (under certain assumptions) the Bloom
Estimator (Bloom, 1984): Divide the ITT of the outcome by the ITT of
the dosage.
• Functions and aggregates of observed characteristics. A person’s ethnic
background will not satisfy exclusion, but membership in a “cluster”
defined by ethnicity, age, income, job and other characteristics might.
See Cutler & Gruber, 1996.
• Taking the cluster idea one step further, multilevel data yield “natural”
clusters that can be used, under certain assumptions, to produce useful
instruments. This is often referred to as a “Wald instrument” because it
accords with Wald’s hypothesis. See Cobb-Clark & Crossley, 2003.
The Wald Estimator and IVE
•
“If Z is a dummy variable indicating membership in one of
two groups, then zy′/n captures the difference in the average
outcomes of the two groups and zd′/n captures the
difference in the treatment intensity across the two groups.
Thus the IV estimator simply scales the difference in average
outcomes by the difference in treatment intensities, as our
intuition suggests.”
-Cobb-Clark and Crossley, 2003. page 504.
The Exclusion Restriction:
Justification and Identification
Controversy Surrounding IVs
• The most problematic aspect of IVE: search for a variable
satisfying the requirements of no correlation with error and
high correlation with endogenous variable.
• Even some economists are skeptical of the value of certain approaches
taken to estimating IVs: Heckman (1990; 1997).
• Heckman even questions random assignment as an instrument!
• Inadequate attention to ensuring exclusion is a frequent
problem.
• This is changing particularly as researchers outside of
economics begin to see the value of the method.
Can Exclusion be Proven?
The (flawed) logic of a test for exclusion is that if Z is associated with Y
after conditioning on X, then it is not an IV, because that means it must
operate through C (or any other unmeasured variable).
Health at time t-1
Not measured (x-sect)
C
Z
X
Y
Food stamp receipt
Public Health insurance
Health at time t
The problem is that this test always fails, because conditioning on X creates
an artifactual association between Z and C. Exclusion must be “proven” by
inductive methods only.
Can Exclusion be Proven?
Propensity to receive food stamps
1
0
Health at time t-1
Green: does not have public health insurance
pink: has public health insurance
Refer to: Morgan & Winship, 2007; p. 67.
Exclusion Logic: Example
• A school district recently implemented a policy change,
promoting students to middle school in 7th grade rather than 6th.
• You are an evaluator given the task of determining whether this
policy is beneficial to students, based on 7th grade exams.
• Promotion is endogenous because it is determined partly by
performance on end-of-grade exams.
• Having been hired after implementation, randomization is not an
option.
• If a valid IV can be identified, you might have a causal estimate.
• Birth date is random, right? (Right?)
Exclusion Logic: Example of Potential IV
Jan 1
Aug 31 Sept 1
Dec 31
Yr A
Yr B
Yr C
• Blue cohort: advances in 6th grade.
• Green cohort: advances in 7th grade.
• Students in different cohorts nearest the star are most alike, and thus less
subjected to random differences.
• In calendar Year B, birth date determines cohort membership.
• Correlation between birth date and cohort determined by dispersion over
year-long period before/after star.
•  Quality of IV (correlation with cohort; influence of error) determined by
the distance of each student’s birth from the star.
Discussion:
Heintze, Berger, Nadich & Meyers (2006)
• Question: Is housing assistance directly associated with
employment among low-income single mother households?
• Endogeneity problem?
• “The instruments used here are lagged measures of state-level
housing assistance generosity. They are constructed from HUD data
(HUD 1998). The lagged measure of housing assistance generosity
for the 1997 sample is the ratio of the number of federally
subsidized units to the number of poor households in the state in
1993.” (p 649)
• How is exclusion assumed here?
• How might exclusion fail?
• Did the authors sufficient examine exclusion?
• What alternatives might exist?
Recommendations
Recommendations: Endogeneity
• Never include a known endogenous variable as an
intermediary between your focal causal variable and your
outcome (e.g., do not include dosage when modeling
treatment).
• The seeming exception to this is mediation, though in this
case the mediator is the focal causal variable.
• There are many methods for dealing with endogeneity,
including fixed effects, interrupted time series, PSM,
regression discontinuity; some of these remain relatively
untapped by social work researchers.
Recommendations: IVE
• Knowing what an IV is and how it works will be important for social
work evaluators and researchers as the disciplines converge on
scarce grant funds for research projects.
• When doing mediation or dosage (efficacy; i.e., not intent-to-treat)
analysis, IVE is a potentially valuable tool, particularly when the
treatment (as an antecedent to dosage) is randomly assigned.
 Look for next paper on local average treatment effects.
 This method also appropriate when ethics prevents randomization directly
to treatment.
• Think about what instruments might be useful in your work; look
for examples of instruments by other researchers in the literature.
Be critical; a lousy instrument does not promote good science.
Recommended Reading
• Morgan, S. L. & Winship, C. (2007). Counterfactuals and Causal Inference:
Methods and Principals for Social Research. Cambridge University Press.
• There are about a dozen articles in the social work literature that use or
refer to IVE. These are listed in the appendix of the Stone article on
endogeneity.
• The papers I’ve referred to:
 Cutler, D.M., and J. Gruber. 1996. The Effect of Expanding the Medicaid Program on
Public Insurance, Private Insurance, and Redistribution. American Economic
Review: Papers and Pro-ceedings 86(2): 368–373.
 Cobb-Clark, D. A. & Crossley, T. (2003). Econometrics for evaluations: An
introduction to recent developments. The Economic Record, 79(247), 491-511.
• A three part series in Social Science and Medicine by myself & Susan
Parish & colleagues, and Michael Foster:
 Rose, R. A., Parish, S. L., Yoo, J. (2010). Observation, correlation and policy analysis:
A response to Foster. Social Science & Medicine, 70, 1274-1276.
 Foster, E. M. (2010). A commentary on Rose et al. Social Science & Medicine, 70,
1271-1273.
 Rose, R. A., Parish, S. L., Yoo, J., Grady, M.D., Andrews, M. E., Powell, S. E., & HicksSangster, T. K. (2010). Medicaid suppresses racial disparities in health care among
children with special health care needs: Evidence from a multi-level mediation
modeling analysis. Social Science & Medicine, 70, 1263-1270.
SSWR in Washington DC, Jan 2012
• I will be presenting a paper using LATE via IVE methods in a
symposium for CareerStart that has several papers by myself,
Mike Woolley and Jay Unick.
• While my time for explaining the intricacies of LATE will be
limited, I will have (by this time) briefs and probably a short
report.
• The paper I am presenting will constitute one of my
dissertation papers.
Closing Comments & Questions