Decision Theoretic Analysis of Improving Epidemic Detection
Download
Report
Transcript Decision Theoretic Analysis of Improving Epidemic Detection
Decision Theoretic Analysis of
Improving Epidemic Detection
Izadi, M.
Buckeridge, D.
AMIA 2007,Symposium Proceedings 2007
Overview
Objective: Improve the accuracy of current
detection methods
Observation: Quantifying the potential costs
and effects of intervention can be used to
optimize the alarm function
Method: Use Partially Observable Markov
Decision Processes on the outbreak detection
method
The Surveillance Cycle*
1. Identifying
individual cases
2. Detecting population
patterns
Individual Event
Definitions
Event
Detection
Algorithm
3. Conveying information for
action
Population Pattern
Definitions
Event
Reports
Data Describing
Population
(Buckeridge DL & Cadieux G, 2007)
Intervention
Guidelines
Pattern
Report
Pattern
Detection
Algorithm
Population Under
Surveillance
Intervention
Decision
Public
Health
Action
Usual Detection Methods*
Methods are non-specific – they look for
anything unusual in the data
Design a baseline.
Define an aberration when some
statistics are more than expected values
by the baseline.
Detection Method Example*
Define a threshold for the number of Emergency
Department visits per day.
Signal an alarm when the number of ED visits per day
exceeds the threshold.
Num ber of ED Visits per Day
Number of ED Visits
50
40
30
20
10
0
1
10 19
28 37 46
55 64
Day Num ber
73 82 91 100
Sensitivity and Specificity Tradeoff
Sensitivity is the probability of alarm
given an outbreak P(A+|O+)
Specificity is the probability of no alarm
given no out break P(A-|O-)
Timeliness is time between outbreak and
detection
Challenge: Increasing sensitivity and
improving timeliness decreases specificity
Approach Overview*
Instead
of trying to improve the
detection method, ‘post-process’
the signals:
Use
a standard detection method to
provide signals
Feed this signal to a decision support
model to find the optimal action
Quick Introduction to POMDPs*
What goes on:
st-1
What we see:
ot-1
What we infer:
bt-1
States: sS
Actions: aA
Observations: oO
Transition probabilities: Pr(s’|s,a)
Observation probabilities: Pr(o|s,a)
Rewards: R(s,a)
Belief state: b(s)
st
at-1
ot
bt
at
Model Components
States: - True epidemic state
No Outbreak
Day1
...
Day4
Observations: Output from the detection
algorithm:
Alarm
No-Alarm
Model Components (continued)
Actions
1.
2.
3.
4.
Do nothing
More Systematic Studies (e.g. get more patient
files from ED)
More Investigation (done by human expert)
Declare outbreak
Transition and Observation Probabilities
Calculated based on expert knowledge
Model Components (continued)*
Costs
Investigation (false and true positive)
Intervention (false and true positive)
Outbreak by day (false negative)
(# deaths* future earnings) + (# hospitalized *
cost of hospitalization) + (# outpatient visits * cost
of visit)
Model Components (continued)*
Rewards
Preventable loss at each day
Outbreak Detection as a POMDP*
No Outbreak
D1
D2
D3
D4
Outbreak
detected
Do nothing
Review records
Investigate cases
Declare outbreak
Experimental Design
Compare a detection method (moving
average) with and without addition of
POMDP
Consider a fixed Specificity of 0.97
The comparison is over 10 years
simulation
Not exactly clear how the data is generated
Experimental Results*
Small size outbreak
Day of Outbreak
Experimental Results*
Larger size outbreak
Day of Outbreak
Conclusion
POMDPs can improve the accuracy of the
current outbreak detection methods
We can use the potential costs and effects of
intervention to learn a decision process
P(A-|O-) = 0.97 P(A|O-) = 0.03
In every 100 days , we will have 3 false alarms!
Is this acceptable?