Transcript Reporting Status or Progress

Bayesian Networks for
Risk Assessment
Government Actuary's Department
18 November 2014
Norman Fenton
Queen Mary University of London
and
Agena Ltd
Slide 1
Outline
Overview of Bayes and Bayesian
networks
Why Bayesian networks are
needed for risk assessment
Examples and real applications
in financial risk
Challenges and the future
Slide 2
Our book
www.BayesianRisk.com
Slide 3
Overview of Bayes and
Bayesian Networks
Slide 4
A classic risk assessment
problem
A particular disease has a 1 in 1000
rate of occurrence
A screening test for the disease is
100% accurate for those with the
disease; 95% accurate for those
without
What is the probability a person
has the disease if they test
positive?
Slide 5
Bayes Theorem
Have a prior P(H) (“person has disease”)
Now get some evidence E (“test result positive”)
H
(hypothesis)
E
(evidence)
But we want the posterior P(H|E)
We know P(E|H)
P(E|H)*P(H)
P(H|E) = P(E|H)*P(H) =
P(E)
P(E|H)*P(H) + P(E|not H)*P(not H)
P(H|E)
=
1*0.001
1*0.001 + 0.05*0.999
=
0.001
0.5005
 0.02
Slide 6
A Classic BN
Slide 7
Bayesian Propagation
Applying Bayes theorem to
update all probabilities when
new evidence is entered
Intractable even for small BNs
Breakthrough in late 1980s - fast
algorithms
Tools implement efficient
propagation
Slide 8
A Classic BN: Marginals
Slide 9
Dyspnoea observed
Slide 10
Also non-smoker
Slide 11
Positive x-ray
Slide 12
..but recent visit to Asia
Slide 13
The power of BNs
Explicitly model causal factors
Reason from effect to cause and vice versa
‘Explaining away’
Overturn previous beliefs
Make predictions with incomplete data
Combine diverse types of evidence
Visible auditable reasoning
Can deal with high impact low probability
events (we do not require massive datasets)
Slide 14
Why causal Bayesian
networks are needed for
risk assessment
Slide 15
Problems with regression driven
‘risk assessment’
N  2.144  T  243.55
Irrational for risk assessment
Rational for risk assessment
Slide 16
‘Standard’ definition of risk
“An event that can have negative
consequences”
Measured (or even defined by):
Slide 17
Slide 17
..but this does not tell us tell us
what we need to know
Armageddon risk:
Large meteor strikes the Earth
The ‘standard approach’ makes no sense at all
Slide 18
Risk using causal analysis
A risk is an event that can be characterised
by a causal chain involving (at least):
The event itself
At least one consequence event that
characterises the impact
One or more trigger (i.e. initiating) events
One or more control events which may
stop the trigger event from causing the
risk event
One or more mitigating events which help
avoid the consequence event (for risk)
Slide 19
Bayesian Net with causal view of risk
Trigger
Meteor on
collision course
with Earth
Risk event
Meteor strikes
Earth
Blow up
Meteor
Build
Underground
cities
Control
Mitigant
Consequence
Loss of Life
Slide 20
Examples and real
applications in financial
risk
Slide 21
Causal Risk Register
Note that
‘common
causes’ are
easily
modelled
Slide 22
Simple stress test interest payment example
Assumes capital sum $100m
and a 10-month loan
Expected value of resulting payment is
$12m with 95% percentile at $26m
Regulator stress test: “at least 4%
interest rate”
Slide 23
Simple stress test interest payment example
Expected value of resulting payment
in stress testing scenario is $59m with
95% percentile at $83m
This model can be built in a couple
of minutes with AgenaRisk
Slide 24
Stress testing with causal dependency
Slide 25
Stress testing with causal dependency
Slide 26
Op Risk Loss Event Model
Slide 27
Operational Risk VAR Models
Aggregate
scenario
outcome
Contributing
outcomes
Scenario
dynamics
Slide 28
Stress and Scenario Modelling
Travel Disruption
Pandemic
Reverse Stress
Civil Unrest
Slide 29
Business Performance
Holistic map of business enhances
understanding of interrelationships
between risks and provides
candidate model structure
Risk Register entries
help explain uncertainty
associated with
business processes
Business Performance
Indicators serve as ex-post
indicators, we can then use
the model to explain the
drivers underlying business
outcomes
KPIs inform
the current
state of the
system
Slide 30
Policyholder Behaviour
Slide 31
The challenges
Slide 32
Challenge 1:
Resistance to Bayes’ subjective
probabilities
“.. even if I accept the calculations
are ‘correct’ I don’t accept subjective
priors”
There is no such thing as
a truly objective
frequentist approach
Slide 33
Challenge 2:
Building realistic models
Common method:
Structure and probability
tables all learnt from data
only (‘machine learning’)
DOES NOT WORK EVEN
WHEN WE HAVE LOTS OF
‘RELEVANT’ DATA!!!!!!!!!!!!!!!
Slide 34
A typical data-driven study
Age
Delay in
arrival
Injury
type
Brain
scan
result
Arterial
pressure
Pupil
dilation
Outcome
(death y/n)
17
25
A
N
L
Y
N
39
20
B
N
M
Y
N
23
65
A
N
L
N
Y
21
80
C
Y
H
Y
N
68
20
B
Y
M
Y
N
22
30
A
N
M
N
Y
…
…
…
..
…
…
Slide 35
A typical data-driven study
Injury
type
Brain scan
result
Arterial
pressure
Delay in
arrival
Pupil
dilation
Age
Outcome
Purely data driven
machine learning
algorithms will be
inaccurate and produce
counterintuitive results
e.g. outcome more likely
to be OK in the worst
Slide 36
scenarios
Causal model with intervention
Injury
type
Brain scan
result
Arterial
pressure
Delay in
arrival
Pupil
dilation
Age
Danger
level
TREATMENT
Outcome
..crucial variables missing
from the data
Slide 37
Challenge 2:
Building realistic models
Need to incorporate experts
judgment:
Structure informed by experts,
probability tables learnt from
data
Structure and tables built by
experts
Fenton NE, Neil M, and Caballero JG, "Using Ranked nodes to
model qualitative judgements in Bayesian Networks“, IEEE TKDE
19(10), 1420-1432, Oct 2007
Slide 38
Challenge 3:
Handling continuous nodes
Static discretisation: inefficient
and devastatingly inaccurate
Our developments in dynamic
discretisation starting to have a
revolutionary effect
Neil, M., Tailor, M., & Marquez, D. (2007). “Inference in hybrid Bayesian
networks using dynamic discretization”. Statistics and Computing, 17(3),
219–233.
Neil, M., Tailor, M., Marquez, D., Fenton, N. E., & Hearty, P. (2008).
“Modelling dependable systems using hybrid Bayesian networks”.
Reliability Engineering and System Safety, 93(7), 933–939
Slide 39
Challenge 4: Risk Aggregation
Estimate sum of a collection of financial
assets or events, where each asset or
event is modelled as a random variable
Methods not designed to cope with the
presence of Discrete Causally Connected
Random Variables
Solution: Bayesian Factorization and
Elimination (BFE) algorithm - exploits
advances in BNs and is as accurate on
conventional problems as competing
methods.
Peng Lin, Martin Neil and Norman Fenton (2014). “Risk
aggregation in the presence of discrete causally connected
random variables”. Annals of Actuarial Science, 8, pp 298-319
Slide 40
Conclusions
Genuine risk assessment requires
causal Bayesian networks
Bayesian networks now used
effectively in a range of real world
problems
Must involve experts and not rely
only on data
No major remaining technical
barrier to widespread
Slide 41
Follow up
Get the book
www.BayesianRisk.com
Try the free BN software and all the
models
www.AgenaRisk.com
Propose case study for ERC Project BAYES-KNOWLEDGE
www.eecs.qmul.ac.uk/~norman/projects/B_Knowledge.html
Slide 42