Transcript lecture0010_2004

URBDP 591 A Lecture 10: Causality
Objectives
• Internal Validity
• Threats to Internal Validity
• Causality
• Bayesian Networks
Internal validity
The extent to which the hypothesized
relationship between 2 or more variables is
supported by evidence in a given study.
Validity
Internal Validity is the approximate truth about inferences
regarding cause-effect or causal relationships.
External Validity: Assuming that there is a causal relationship
in this study between the constructs of the cause and the effect,
can we generalize this effect to other persons, places or times?
Statistical validity has to do with basing conclusions on
proper use of statistics.
Construct validity refers to the degree to which inferences
can legitimately be made from your study to the theoretical
constructs on which those operationalizations were based.
Internal validity
External Validity
Statistical Validity
In Reality
We Conclude
We accept H0
We reject H1
We reject H0
We accept H1
H0 (null hypothesis) true
H1 (alternative hypothesis) false
1-  (e.g., .95)
THE CONFIDENCE LEVEL
The probability we say there
is no relationship when there is not
 (e.g., .05)
Type I Error
The probability we say there
is a relationship when there is not
H0 (null hypothesis) false
H1 (alternative hypothesis) True
 (e.g., 20)
Type II Error
The probability we say there
is no relationship when there is one
1-  (e.g., 80)
THE POWER
The probability we say there
is a relationship when there is one
Construct Validity
Threats to the Validity of Research
Threats to internal validity
Although it is claimed that the independent variable caused change in the
dependent variable, the changes in the dependent variable may have actually
been caused by a confounding variable.
Threats to external validity
Although claimed that the results are more general, the observed effects may
actually only be found under limited conditions or for specific groups of people.
Threats to construct validity
Although it is claimed that the measured variables and the experimental
manipulations relate to the conceptual variables of interests, they actually
may not.
Threats to statistical validity
Conclusions regarding the research may be incorrect because a Type 1 or
Type 2 error was made.
Challenges to Internal Validity
a. History
b. Maturation
c. Experimental mortality
d. Instrumentation
e. Testing
f. Interactions with selection
History
Any events that occur during the course of the
experiment which might effect outcome.
Example: An important event not related
to the experiment affects the
measurement of pre and post-test.
Maturation
Changes in the subjects over the course of the
experiment.
Example: Age, experience, physical
development of participants that leads to
increase in knowledge and understanding of
the world or behavior which can affect
program results.
Experimental Mortality
Dropouts from the experiment; especially when
the dropouts systematically bias the comparisons
Example: If your include pretest subsequent dropouts in
the pretest and not in the posttest you will bias the
test based on the characteristics of dropouts. And,
you won't necessarily solve this problem by
comparing pre-post averages for only those who
stayed in the study. This subsample would certainly
not be representative even of the original entire
sample
Instrumentation
Any way in which the instrument used for
observation or collecting data changes from the
pre-test to the post-test.
Example: Test, Interview, Measurement
Technique or Instrument.
Regression Threat
A regression threat, also known as a
"regression artifact" or "regression to the
mean" is a statistical phenomenon that
occurs whenever you have a nonrandom
sample from a population and two
measures that are imperfectly correlated.
Regression Threat
The highest and lowest
scorers will regress toward the
mean at a higher rate than
those
who scored close to the
mean. There will be a higher
degree of regression for
unreliable
measures than for more
reliable ones.
Testing
The observations gathered may influence the way
subjects behave, thus effecting the outcome.
Example: having had the experience of taking the
GRE once, without any additional preparation, you
are more likely to improve your score on a re-take.
Interactions with Selection
When there is a relationship between the treatment
and the selection of subjects, this causes a
systematic bias which affects causal inference
Example: A selection threat is any factor other than
the program that leads to posttest differences between
groups. These include: history, maturation, test,
instrumentation, mortality, and regression.
SIMPSON’S PARADOX
Any statistical relationship between two variables
may be reversed by including additional factors in
the analysis.
SIMPSON’S PARADOX
Example by Judea Pearl
The classical case demonstrating Simpson's paradox
took place in 1975, when UC Berkeley was
investigated for sex bias in graduate admission. In
this study, overall data showed a higher rate of
admission among male applicants, but, broken down
by departments, data showed a slight bias in favor of
admitting female applicants. The explanation is
simple: female applicants tended to apply to more
competitive departments than males, and in these
departments, the rate of admission was low for both
males and females. .
FISHNET (JUDEA PEARL)
Which factors SHOULD be included in
the analysis?
All conclusions are extremely sensitive to which variables we
choose to hold constant when we are comparing, and that is
why the adjustment problem is so critical in the analysis of
observational studies.
According to Judea Pearl, such factors can now be identified
by simple graphical means.
Bayes Theorem
The basic foundation of probability theory follows from the following intuitive
definition of conditional probability.
P(A,B) = P(A|B)P(B)
In this definition events A and B are simultaneous an have no (explicit)
temporal order we can write
P(A,B) = P(B,A) = P(B|A)P(A)
This leads us to a common form of Bayes Theory, the equation:
P(A) = P(B|A)P(B)/P(A|B) (marginalization)
which allows us to compute the probability of one event in terms of
observations of another and knowledge of joint distributions.
Use of Bayes Rule
Often one is interested in particular conditional probability and discovers that
The reverse conditional probabilities are more easily obtained.
Example: One is interested in the P(disease | symptom), but typically
P (symptom | disease) is better known.
Causality Example: One is interested in the P(cause | effect), but typically
P (effect | cause) is better known.
Bayes Theorem
The heart of Bayesian inference lies in the inversion formula which
States that the belief we accord to a hypothesis H upon obtaining
Evidence e can be comnputed by multiplying our previous belief P(H)
by the likelhood P(e H) that e will materialize if H is true.
P(H | e) = P(e | H) P(H) / P(e)
P(H | e) = posterior probability
P(H) = prior probability
From the definition of condition probability
P(A | B) = P(A,B) / P(B)
P(B | A) = P(A,B) / P(A)
Bayesian networks
Graphs in probabilistic form
• To provide convenient means of espressing
substantive assumptions
• To facilitate economical representation of joint
probability functions, and
• To facilitate efficient inferences from
observation
Bayesian networks provide a language for qualitatively representing the conditional
independence properties of a distribution. This allows a natural and compact
representation of the distribution, eases knowledge acquisition, and supports
effective inference algorithms.
Bayesian networks
Bayesian Network is a Directed Acyclic Graph
(DAG) in which
1. Each node denotes some random variable X
2. Each node has a conditional probability
distribution P(X | parent X)
The intuitive meaning of an arc from node X
to node Y is that X directly influences Y.
Typical Bayesian Network
A Bayesian network representing dependencies among five variables:
X1 = season of the year
X2 = rain
X3 = sprinkler is on
X4 = pavement get wet
X5 = pavement will be slippery
Causal Relationships and their stability
S1: Turning the sprinkler on would not affect the rain
S2: Belief in the state of the rain is independent of knowledge in the state of the sprinkler
S2: Unless we learn what season it is
Given X1, S2 changes from true to false once we know that the pavement is wet
S1: Remain true regardless of X1 or X4
CONDITIONAL INDEPENDENCE
Two events, A and B, are said to be independent if
P(A and B both happening)=P(A happening).P(B happening).
The two events do not affect one another.
A more general result can be derived for conditional probabilities.
Two events, A and B, are said to be conditionally independent if
P(A and B both happening | Another event C) = P(A happening | C).P(B happening | C).
Where we already have information about the situation (through our knowledge of event C),
knowledge of event A will not enable us to change our estimate of the probability of event B.
Types of Connections
• Serial Connections
A
B
C
• Diverging Connections
B
• Converging Connections
B
C
A
D
A
C
D
D-separation
• Two variables A and B are d-separated if for
all paths between A and B, there is an
intermediate variable V such that either:
– The connection is serial or diverging and V is
known
– The connection is converging and neither V nor
any of V’s descendants have received evidence
• If A and B are d-separated, then changes in
the certainty of A have no impact on the
certainty of B
Bayesian network inference
Compute the posterior probability distribution for
a set of query variables, given values for some
evidence variables.
Explaining away
R = rain
R
W
S
H
S = sprinkler
W = Watson’s grass is
wet
H = Holmes grass is wet
Explaining Away:
More probable explanation given the data can explain
away other explanations.
When nothing is known, R and S are independent.
However when we obtain information about H, R and S
are dependant.