Transcript Quantitative Provenance. Using Bayesian Networks to Help Quantify

Quantitative Provenance
3
2
1
0
1
2
3
Using Bayesian Networks to Help Quantify
the Weight of Evidence In Fine Arts Investigations
A Case Study: Red Black and Silver
Outline
• Probability Theory and Bayes’ Theorem
• Likelihood Ratios and the Weight of Evidence
• Decision Theory and its implementation: Bayesian
Networks
• Simple example of a BN: Why is the grass wet?
• Taroni Bayesian Network for trace evidence
• The Bayesian Network for Red, Black and Silver
• Stress testing: Sensitivity analysis
• Recommendation for RBS
Probability Theory
“The actual science of logic is conversant at present only
with things either certain [or] impossible. Therefore the
true logic for this world is the calculus of Probabilities,
which takes account of the magnitude of the probability
which is in a reasonable man’s mind.”
— James Clerk Maxwell, 1850C
Probability theory is nothing but common sense
reduced to calculation.”— Laplace, 1819L
Probability Theory
Probability: “A particular scale on which
degrees of plausibility can be measured.”
“They are a means of describing the
information given in the statement of a
problem”
— E.T. Jaynes, 1996J
Probability Theory
• Probability theory forms the rules of
reasoning
• Using probability theory we can explore
the logical consequences of our
propositions
• Probabilities can be updated in light of
new evidence via Bayes theorem.
Bayesian Statistics
• The basic Bayesian philosophy:
Prior Knowledge × Data = Updated Knowledge
A better understanding
of the world
Prior × Data = Posterior
The “Bayesian Framework”
• Bayes’ Theorem to Compare Theories:
•
•
•
•
Ha = Theory A (the “prosecution’s” hypothesisAT)
Hb = Theory B (the “defence’s” hypothesisAT)
E = any evidence
I = any background information
Pr(E | H a, I )
Pr(H a | E, I) =
Pr(H a , I )
Pr(E)
Pr(E | H b, I)
Pr(H b | E, I) =
Pr(H b , I)
Pr(E)
The “Bayesian Framework”
• Odd’s form of Bayes’ Rule:
Likelihood Ratio
{
Posterior odds in favour of
Theory A
{
{
Pr(H a | E, I ) Pr(E | H a , I) Pr(H a , I )
=
´
Pr(H b | E, I ) Pr(E | H b , I) Pr(H b , I)
Prior odds in favour of
Theory A
Posterior Odds = Likelihood Ratio × Prior Odds
The “Bayesian Framework”
• The likelihood ratio has largely come to be the
main quantity of interest in the forensic statistics
literature:
Pr(E | H a , I)
LR =
Pr(E | H b, I )
• A measure of how much “weight” or “support”
the “evidence” gives to Theory A relative to
Theory BAT
The “Bayesian Framework”
Pr(E | H a , I)
LR =
Pr(E | H b, I )
• Likelihood ratio ranges from 0 to infinity
• Points of interest on the LR scale:
LR
Jeffreys ScaleJ
<1
Evidence supports for Theory B
1 to 3
Evidence barely supports Theory A
3 to 10
Evidence substantially supports Theory A
10 to 30
Evidence strongly supports Theory A
30 to 100 Evidence very strongly supports Theory A
> 100
Evidence decisively supports Theory A
LR
Kass-Raftery ScaleKR
<1
Evidence supports for Theory B
1 to 3
Evidence barely supports Theory A
3 to 20
Evidence positively supports Theory A
20 to 150 Evidence strongly supports Theory A
> 150 Evidence very strongly supports Theory A
Decision Theory
• Frame decision problem (scenario)
• List possibilities and options
• Quantify the uncertainty with available
information
• Domain specific expertise
• Historical data if available
• Combine information respecting the laws
of probability to arrive at a
decision/recommendation
Bayesian Networks
• A “scenario” is represented by a joint probability
function
• Contains variables relevant to a situation which represent
uncertain information
• Contain “dependencies” between variables that describe how
they influence each other.
• A graphical way to represent the joint probability
function is with nodes and directed lines
• Called a Bayesian NetworkPearl
Bayesian Networks
• (A Very!!) Simple exampleWiki:
• What is the probability the Grass is Wet?
• Influenced by the possibility of Rain
• Influenced by the possibility of Sprinkler action
• Sprinkler action influenced by possibility of Rain
• Construct joint probability function to answer
questions about this scenario:
• Pr(Grass Wet, Rain, Sprinkler)
Bayesian Networks
Pr(Sprinkler | Rain)
Sprinkler
:
Rain:
yes
was on
was off
40%
60%
no
Pr(Rain)
Pr(Sprinkler)
1%
Pr(Rain)
99%
Pr(Grass Wet)
Pr(Grass Wet | Rain, Sprinkler)
Grass
Wet:
Sprinkler:
Rain:
was on
yes
was on
no
was off
yes
was off
no
yes
no
99%
1%
90%
10%
80%
80%
0%
100%
Rain: yes
no
20%
80%
Bayesian Networks
Pr(Sprinkler)
Pr(Rain)
Other probabilities
are adjusted given
the observation
Pr(Grass Wet)
You observe
grass is wet.
Bayesian Networks
• Likelihood Ratio can be obtained from the BN once evidence is
entered
• Use the odd’s form of Bayes’ Theorem:
Probabilities of the theories after
we entered the evidence
Probabilities of the theories before
we entered the evidence
Bayesian Networks
• Areas where Bayesian Networks are used
• Medical recommendation/diagnosis
• IBM/Watson, Massachusetts General Hospital/DXplain
• Image processing
• Business decision support
• Boeing, Intel, United Technologies, Oracle, Philips
• Information search algorithms and on-line recommendation
engines
• Space vehicle diagnostics
• NASA
• Search and rescue planning
• US Military
• Requires software. Some free stuff:
• GeNIe (University of Pittsburgh)G,
• SamIam (UCLA)S
• Hugin (Free only for a few nodes)H
• gR R-packagesgR
Taroni Model for Trace Evidence
• Taroni et al. have prescribed a general BN fragment that
can model trace evidence transfer scenariosT:
• H: Theory (Hypothesis) node
• X: Trace associated with (a) “suspect”
node
• TS: Mediating node to allow for chance
match between suspect’s trace and trace
from an alternative source
• T: Trace transfer node
• Y: Trace associated with the “crime
scene” node
Trace Evidence BN for RBS case
• Theories are that Pollock or someone else associated with
him in summer 1956 made the painting
• The are two “suspects”
• Use a Taroni fragment for each of:
• Group of wool carpet fibers
• Human hair
• Polar bear hair
• Use a modified Taroni fragment (no suspect node) for
each of:
• Beach grass seeds
• Garnet
Trace Evidence BN for RBS case
• Link the garnet and seeds fragment together directly
• They a very likely to co-occur
• Link all the fragments together with the Theory (Painter)
node and a Location node
Trace Evidence BN for RBS case
• Enter the evidence:
Sensitivity Analysis
• Local sensitivityC
• Posterior’s sensitivity to small changes in the
model’s parameters.
Threshold > 1
Sensitivity Analysis
• Global sensitivityC
• Posterior’s sensitivity to large changes in the
model’s parameters.
Threshold < 0.1
• Parameter 24 is: “the probability of a transfer of polar bear hair, given the painting was made
outside of Springs by Pollock and he had little potential of shedding the hair”.
Conservative Recommendation
• Considering the Likelihood ratio calculated with
the “Red, Black and Silver” trace evidence
network coupled with the sensitivity analysis
results:
• The physical evidence is more in support of the
theory that Pollock made RBS vs. someone else
made RBS:
•
•
“Strongly” – “Very Strongly” (Kass-Raftery Scale)
“Very Strongly” – “Decisively” (Jeffreys Scale)
References
• C
•
•
•
•
•
•
•
•
•
•
•
•
•
Lewis Campbell. The Life of James Clerk Maxwell: With Selections from His
Correspondence and Occasional Writings, Nabu Press, 2012.
L Pierre Simon Laplace. Théorie Analytique des Probabilités. Nabu Press, 2010.
J
E. T. Jaynes. Probability Theory: The Logic of Science. Cambridge University Press,
2003.
AT C. G. G. Aitken, F. Taroni. Statistics and the Evaluation of Evidence for Forensic
Scientists. 2nd ed. Wiley, 2004.
J
Harold Jeffreys. Theory of Probability. 3rd ed. Oxford University Press, 1998.
KR R. Kass, A. Raftery. Bayes Factors. J Amer Stat Assoc 90(430) 773-795, 1995.
P Judea Pearl. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible
Inference. Morgan Kaufmann Publishers, San Mateo, California, 1988.
Wiki http://en.wikipedia.org/wiki/Bayesian_network
T
F. Taroni, A. Biedermann, S. Bozza, P. Garbolino, C. G. G. Aitken. Bayesian Networks
for Probabilistic Inference and Decision Analysis in Forensic Science. 2nd ed. Wiley,
2014.
C
Veerle M. H. Coupe, Finn V. Jensen, Uffe Kjaerulff, and Linda C. van der Gaag. A
computational architecture for n-way sensitivity analysis of Bayesian networks.
Technical report, people.cs.aau.dk/~uk/papers/coupe-etal-00.ps.gz, 2000.
G
http://genie.sis.pitt.edu/
S
http://reasoning.cs.ucla.edu/samiam/
H
http://www.hugin.com/
gR Claus Dethlefsen, Søren Højsgaard. A Common Platform for Graphical Models in R:
The gRbase Package. J Stat Soft http://www.jstatsoft.org/v14/i17/, 2005.
Fin