Transcript Oberlin02-06

Meeting the Future in Managing
Chronic Disorders: Individually
Tailored Strategies
S.A. Murphy
Univ. of Michigan
Oberlin College, Feb. 20, 2006
1
Outline
–
–
–
–
Three apparently dissimilar problems
Myopic decision making
Unknown, unobserved causes
Discussion
2
Three Apparently Dissimilar
Problems
– Artificial Intelligence: Autonomous Helicopter
Flight
– Management of Chronic Mental Illnesses
– Management of a Welfare Program
3
Artificial Intelligence
• Autonomous Helicopter Flight
– Observations: characteristics of the helicopter (position,
orientation, velocity, angular velocity, ….),
characteristics of the environment (wind speed, wind
angle, turbulence….)
– Actions/treatments: cyclic pitch (causes
forward/backward and sideways acceleration), tilt angle
of main rotor blades (direction), tail rotor pitch control
(turning)
– Rewards: Closeness of helicopter’s flight path to the
desired path; avoidance of crashes(!)
4
Andrew Ng’s Helicopter: http://ai.stanford.edu/~ang/
5
The Management of Chronic Mental
Illnesses
• Treating Patients with Opioid Dependence
(heroin)
– Observations: characteristics of the individual
(withdrawal symptoms, craving, attendance at
counseling sessions, results of urine tests….),
characteristics of the environment (housing,
employment.…)
– Actions/treatments: methadone dose, amount of
weekly group counseling sessions, daily dosing time of
methadone, individual counseling sessions, methadone
taper
– Rewards: minimizing opioid use and maximizing
6
health, minimizing cost
http://www.nida.nih.gov/perspectives/vol1no1.html
7
Management of a Welfare Program
• “Jobs First” Program in Connecticut
– Observations: characteristics of the individual (assets,
income, age, health, employment), characteristics of the
environment (domestic violence, incapacitated family
member, # children, living arrangement…)
– Actions/treatments: child care, job search skills
training, amount of cash benefit, medical assistance,
education
– Rewards: maximizing employment/independence.
8
9
The Common Thread: Sequential
Decision Making
• Observation, action, observation, action,
observation, action,…………………….
• A strategy tells us how to use the
observations to choose the actions.
• We’d like to develop strategies that
maximize the rewards.
10
Role of the Statistician
• What kinds of data are most useful for developing
strategies?
• How do we design an experiment that will
produce the most useful data?
• How do we use the data to construct good
strategies?
(A strategy tells us how to use the observations to
choose the actions.)
11
Myopic Decision Making
12
Myopic Decision Making
• In myopic decision making, decision makers use
strategies that seek to maximize immediate
rewards. Problems:
– Longer term consequences of present actions.
– Ignore the range of feasible future actions/treatments
(A strategy tells us how to use the observations to
choose the actions.)
13
Autonomous Helicopter Flight
The helicopter has veered from flight plan.
•
Myopic action: Choose an acceleration and direction that will
ASAP bring us back to the flight plan.
•
The result: The myopic action results in the helicopter overshooting
the planned flight path and in drastic situations may lead to the
helicopter cycling out of control.
•
The mistake: We did not consider the range of actions we can take
following the initial action. The ability to slow down is
mechanically limited.
•
The message: Use an acceleration that will not quickly return us to
the planned flight path but will take into account the ability of the
helicopter to slow down and reduce the overshoot.
14
Myopic Decision Making
• The message: The fields of robotics and artificial
intelligence teach us that we should try to
construct strategies that are not myopic!
– Pay attention to the longer term consequences of
present actions.
– Do not ignore the range of feasible future
actions/treatments
(A strategy tells us how to use the observations to
15
choose the actions.)
Treatment of Psychosis
•
Myopic action: Choose a medication, say A, that reduces psychosis
for as many people as possible.
•
The result: Some patients are not helped and/or experience abnormal
movements of the voluntary muscles (TDs). The class of subsequent
medications is greatly reduced.
•
The mistake: We should have taken into account the variety of
treatments available to those for whom the first treatment is
ineffective.
•
The message: Use an initial medication that may not have as large a
success rate but that will be less likely to cause TDs.
16
Unknown, Unobserved Causes
17
Artificial Intelligence
• Scientists who construct strategies that will
be used for autonomous helicopter flights
can use physical laws: momentum=m*v,
W=F*d*cos(θ)………
• Scientists know many (most?) of the causes
of the observations and know how the
observations relate to one another.
18
Conceptual Structure in the
Behavioral/Social/Medical Sciences
Unknown
Causes
Observations
Unknown
Causes
Action or
Treatment 1
Observations
Time 2
Action or
Treatment 2
Reward
Time 3
19
Unknown, Unobserved Causes
• Scientists who want to use data on
individuals to construct treatment strategies
must confront the fact that non-causal
“relationships” occur due to the unknown
causes.
20
Unknown, Unobserved Causes
Unknown
Causes
Observations
Unknown
Causes
Action or
Treatment 1
Observations
Time 2
Action or
Treatment 2
Reward
Time 3
21
Unknown, Unobserved Causes
Unknown
Causes
Observations
Unknown
Causes
Action or
Treatment 1
Observations
Time 2
Action or
Treatment 2
Reward
Time 3
22
Unknown, Unobserved Causes
Unknown
Causes
Maturity
of Student
+
-
Observations
Action or
Treatment 1
binge drinking
Time 2
Action or
Treatment 2
Grade
Time 3
23
Unknown, Unobserved Causes
• Problem: Non-causal associations between
observations and rewards are likely (due to
the unknown causes).
• Solution: Construct strategies using data
sets collected on representative students
(representative of all college students).
24
Unknown, Unobserved Causes
Unknown
Causes
Decision
to join "Adult"
Society
+
+
Observations
Action or
Treatment 1
Binge Drinking
Time 2
Read web-based
Counseling
Material
Grades
Time 3
25
Unknown, Unobserved Causes
• Problem: Non-causal associations between
“treatments” and rewards are likely (due to
the unknown causes).
• Solution: Construct strategies using data
sets in which a coin was tossed in order to
assign students to treatments. This breaks
the non-causal associations yet permits
causal associations.
26
Unknown, Unobserved Causes
Unknown
Causes
Decision
to join "Adult"
Society
"+"
Observations
Action or
Treatment 1
Binge Drinking
Time 2
Read web-based
Counseling
Material
Grades
Time 3
27
Unknown, Unobserved Causes
(Constructing Sequences of Treatment)
Unknown
Causes
Observations
Unknown
Causes
Action or
Treatment 1
Observations
Time 2
Action or
Treatment 2
Reward
Time 3
28
Unknown, Unobserved Causes
Unknown
Causes
Decision
to join "Adult"
Society
+
-
Binge Drinking
Mandated
Intensive
Counseling
-
Binge Drinking
Time 2
Sanctions
+ counseling
Grades
Time 3
29
Unknown, Unobserved Causes
Unknown
Causes
High SAT
Scores
+
+
Observations
Student
is an superior
athlete
+
Student
admitted to
University
Action or
Treatment 2
Grades
30
Unknown, Unobserved Causes
• The problem: Even when treatments are
randomized (flip coin to assign treatment) noncausal associations can occur in the data.
• The solution: Develop statistical and mathematical
methods that construct strategies but are able to
ignore the non-causal “associations” between
treatment and reward.
31
Summary of Solutions
• Subjects in your sample should be
representative of population of subjects.
• Experiments should randomize actions.
• Use statistical methods that avoid being
influenced by non-causal associations yet
help you construct the strategy.
• Scientists in the fields of robotics and
artificial intelligence should pay attention to
32
our field!
Some Experiments
33
ExTENd
• Ongoing study at U. Pennsylvania (D.
Oslin)
• Goal is to learn how best to help alcohol
dependent individuals reduce alcohol
consumption.
34
ExTENd
When to
abandon
initial treatment?
Observation
Secondary T xt
TDM +
Treatment
is working
R
Prescription
Prescription
Counseling
Quickly
Treatment is
not working
R
Naltrexone +
Counseling
Provide
R
Naltrexone
Treatment
is working
TDM +
R
Prescription
Prescription
Slowly
Counseling
Treatment is
not working
R
Naltrexone +
Counseling
35
STAR*D (www.star-d.org)
• This trial is over and the data is being
analyzed (J. Rush).
• One goal of the trial is construct good
treatment sequences for patients suffering
from treatment resistant depression.
36
37
Discussion
• When thinking how best to manage chronic
disorders (poverty, mental illness, other
medical conditions) we need to
– Allow for longer term effects of the treatments
– When comparing treatment options take into
account the effect of future treatments
– Use data and good statistical methods to
develop the strategies.
38
This seminar can be found at:
http://www.stat.lsa.umich.edu/~samurphy/seminars/Oberlin
02-06.ppt
Research
[email protected]
39