Transcript What Now? - Carnegie Mellon University

Robust Decision Making
Robert Lempert
RAND
HDGC Seminar
February 13, 2004
How Should Climate-Change Uncertainties
Be Characterized for Decisionmakers?

Key climate change uncertainties include





“Basecase” emissions
Behavior of perturbed climate system
Value our descendants place on impacts of climate
change
Costs of abatement with future technology
Climate-change decisionmakers must understand
uncertainties to make effective choices
11-15-03
2
Analytic Tools Often Vital to Clarify Thinking,
But Can Illuminate Trees Rather Than Forest

Analytic tools often vital in improving complicated decisions:



Can successfully summarize vast quantities of information
Help address flaws in human reasoning
Traditional analytic methods assume well-characterized risks and
policy choices based on predictions
Predict

Act
But strategic decisions can go awry if decision-makers assume
risks are well-characterized when they are not



Uncertainties are underestimated
Strategies can be brittle
Misplaced concreteness helps blind decision-makers to surprise
11-15-03
3
Scenarios Capture and Communicate Information
About Future, But Hard to Link to Actions

Global Scenario Group offers three families of
sustainability scenarios
 Conventional worlds
 Barbarization
 Great Transformations

These scenarios
 Capture a wide range of factors which may affect the
future
 Attempt to make an argument for a particular riskmanagement strategy
11-15-03
4
Should Analysts Put Probabilities on Scenarios
Such as Those Developed by SRES?

Pros
 Necessary to make
policy
 Others will provide
likelihood estimates if
experts don’t

Cons
 Little evidence to support
judgments about
probabilities
 Arguing over likelihoods
distracts from reaching
consensus on near-term
actions
Desire for concreteness driving IPCC towards
placing probabilities on scenarios
11-15-03
5
Outline

Robust decision making

Example of robust decisionmaking
as a means of characterizing
uncertainty

Conclusions
11-15-03
6
Climate Change is a Problem of
Decisionmaking Under Deep Uncertainty

Deep uncertainty is:


When we do not know, and/or key parties to the decision do
not agree on, the system model, prior probabilities, and/or
“cost” function
Under conditions of deep uncertainty, decisionmakers:


Often seek robust strategies, ones which perform reasonably
well compared to the alternatives across a wide range of
plausible futures, evaluated with a range of values
Robust strategies are often (but not always) adaptive, that is
they evolve over time in response to new information
Often use choice of strategy, not additional information, to
reduce uncertainty
11-15-03
7
Robust Decisionmaking (RDM)

Robust decisionmaking

Is an iterative, analytic process that identifies
• Strategies that perform well over a wide range of futures
• Remaining vulnerabilities of these strategies


Made possible by advances in computational
capabilities
Characterizes uncertainties most important to the
choice among strategies
11-15-03
8
Four Key Elements of
Robust Decision Making
Consider large ensembles (hundreds to millions)
of scenarios
 Seek robust, not optimal strategies
 Achieve robustness with adaptivity
 Design analysis for interactive exploration of a
multiplicity of plausible futures

11-15-03
9
Consider Ensembles of Many Scenarios
On the occasion of the 1893
World Columbian Exposition,
74 experts wrote essays
predicting what the United
States would look like in 1993

Most were wrong

But some were strangely
close to the truth
11-15-03
1
0
Use Robustness Criteria to Judge
Alternative Strategies

Under deep uncertainty, decision makers often seek
robust strategies that work reasonably well over a wide
range of plausible futures

We measure robustness according to degree of “regret,”
which is defined as the difference between

the performance of a strategy in a given future, and

the performance of the best strategy in that future
11-15-03
1
1
Combine Human and Machine Capabilities
Landscape of
plausible futures
X
Alternative
strategies
Ensemble of
scenarios
Robust
strategies
11-15-03
1
2
Exploratory Modeling Software
Supports This Process

Exploratory modeling software enables users to


navigate through large numbers of scenarios and
Formulate rigorous policy arguments based on these explorations
Tools to
represent
information

Tools to draw
meaning from
information
Users
CARsTM is java-based exploratory modeling software
that:


Links to virtually any type of model and/or data
Supports interactive use of search and visualization to create, explore,
compare, and understand large scenario ensembles
11-15-03
1
3
Outline

Robust decision making

Example of robust decisionmaking
as a means of characterizing
uncertainty

Conclusions
11-15-03
1
4
Example Application of
Robust Decisionmaking
Example: What choice of near-term actions will help ensure
strong economic growth and a healthy environment over the
course of the 21st century?

The RDM approach employed a simple method of representing
information
 “Toy” systems-dynamics model with 41 input parameters
representing uncertainties about
• future economic, demographic, and environmental trends
• values and capabilities of future decisionmakers
 Simple agent-based model of future decisionmakers
 Four value functions based loosely on UN Human Development
Index, which reflects interests of a range of stakeholders
 Near-term strategies affect “decoupling” rate
11-15-03
1
5
Visualizations Capture Key Relationships
Among Plausible Futures
Landscape of plausible futures helps illuminate key challenges to
ensuring strong economic growth and a healthy environment over the
course of the 21st century.
5.0
Decoupling
rate
4.0
China
since
1960
Conventional World
scenario
3.0
U.S.
since 1950
Great Transition
2.0
scenario
U.S.
in 20th century
1.0
0
Russia
since 1993
Brazil since 1980
U.S. 1890-1930
Barbarization scenario
–1.0
India since 1960
0
1.0
2.0
3.0
4.0
Economic growth rate
11-15-03
1
6
Compare “Fixed” Near-Term Strategies
Across Scenarios
Assume near-term policy continues until
changed by future generations
Near Term
Choose policies
Future
Future decisionmakers recognize
and correct our
mistakes
11-15-03
1
7
Look for Robust Strategies
Landscape of
plausible futures
X
Alternative
strategies
Ensemble of
scenarios
Robust
strategies
11-15-03
1
8
Strategies Should Be Robust
Across Multiple Measures of “Goodness”

Use measures inspired by UN’s Human Development Index (HDI)
• Discounted, average rate of improvement in GDP/capita, longevity,
and environmental quality (but no education level) time series
• Four different weightings
N$: North GDP/capita and
longevity
NG: North GDP/capita,
longevity, and environmental
quality
W$: Global GDP/capita and
longevity
WG: Global GDP/capita,
longevity, and environmental
quality
11-15-03
1
9
Speeding Decoupling Performs Well in Many
Futures Using North HDI Measure
Slight speed-up
5.0
N$ W$
4.0
Decoupling
Rate
NG WG
U.S. since 1950
Conventional
world scenario
3.0
2.0
U.S. in 20th
century
U.S. in 19th
century
1.0
0
–1.0
0
1.0
2.0
3.0
4.0
Economic growth rate
No regret
Mild
A lot
Overwhelming
11-15-03
2
0
But Often Fails for Global Green Measure
Slight speed-up
5.0
N$ W$
4.0
Decoupling
rate
NG WG
Conventional
world scenario
3.0
2.0
1.0
0
–1.0
0
1.0
2.0
3.0
4.0
Economic growth rate
No regret
Mild
A lot
Overwhelming
11-15-03
2
1
Exploration Demonstrates
No “Fixed” Strategy Is Robust
Stay the Course
Decoupling
rate
Crash Effort
5.0
5.0
4.0
4.0
Conventional
world
scenario
3.0
3.0
2.0
2.0
1.0
1.0
0
0
–1.0
–1.0
0
1.0
2.0
Conventional
world
scenario
3.0
4.0
0
1.0
2.0
3.0
4.0
11-15-03
2
2
Economic growth
rate
N$ W$
NG WG
No regret
Mild
A lot
Overwhelming
Design and Examine Additional Strategies
Landscape of
plausible futures
X
Alternative
strategies
Ensemble of
scenarios
Robust
strategies
11-15-03
2
3
Start with a Milestone, but Evaluate Progress Early
and Modify Milestone If Necessary (Safety Valve)
Present
Future
Select near-term
milestone
Determine best policy
to meet milestone
Implement policy
Is milestone
achievable with
current approach? YES
NO
Does the carrying
capacity change?
NO
YES
Choose policies
to maximize
utility
Relax milestone
11-15-03
2
4
“Safety Valve” Strategy Appears
Highly Robust
Decoupling rate (%)
Safety valve
5.0
5.0
4.0
4.0
U.S. since 1950
3.0
2.0
U.S. in 19th
century
1.0
2.0
U.S. in 20th
century
0
–1.0
–1.0
1.0
2.0
3.0
4.0
Economic growth rate (%)
U.S. in 19th
century
1.0
0
0
U.S. since 1950
3.0
0
U.S. in 20th
Worst
century
Case
1.0
2.0
3.0
4.0
Economic growth rate (%)
N$ W$
N$ W$
NG WG
+
No regret
Mild
A lot
Overwhelming
NG WG
11-15-03
2
5
Even Simple Scenario Generator Implies a
High Dimensional Uncertainty Space
Uncertainties
Levers
Economic Parameters (N&S)
14 parameters
6 parameters
Demographic Parameters
8 parameters
Environment Parameters (N&S)
7 parameters
Future Generations (N&S)
10 parameters
Measures
Relationships
4 measures
14 state equations
11-15-03
2
6
RDM Employs an Iterative Process
Suggest candidate robust
strategy
 Initial choice is contingent on
probability weighting across
futures
Characterize breaking scenarios
 i.e., clusters of futures where
strategy performs poorly
independent of assumed
weightings
Identify tradeoffs among wellhedged strategies
11-15-03
2
7
Scanning Across All Scenarios
Suggests a Candidate Robust Strategy
Regret
0.95
0.75
0.55
0.35
0.15
Regret
0.00
M00
No
M0X
All
increase
North
North
M12 &
M13
Mostly
somePolicy North
South
Policy
MX0
All
South
MXXthe
Stay
course
11-15-03
2
8
Analytic Tools Generate “Narrative”
Scenarios

RDM identifies low-dimensional, easy-to-interpret regions
where candidate strategy performs poorly



Used Friedman and Fisher’s (1999) Patient Rule Induction Method
(PRIM)
“Low Global Decoupling” scenario is defined by 3 of 41 parameters
Scenario suggests important data for consideration by decisionmakers
North's Innovation Rate
1950-99 (U.S.)
1890-1930 (U.S.)
-0.01
Difference in Innovation
Rate bet. the N. and S.
1960-99
(India)
0.0139
1963-99
(Brazil)
-0.03
North's Economic
Growth Rate
0.05
1993-99
(Russia)
-0.00288
1978-99
(China)
0.03
1890-1930 & 1950-99 (U.S.)
0.0004
0.00812
0.04
11-15-03
2
9
RDM Analysis Helps Policymakers Focus on
a Small Number of Key Tradeoffs
Assessment of adaptive “milestone” sustainability strategies over
two computer-generated scenarios
Safety valve strategy
Milestone strategy
Regret in Low Global Decoupling Futures
0.08
Regret in low-globaldecoupling futures
0.06
M12
M22
0.04
SV01.005.002
0.02
M13
M0X
SV02.005.015
SV01.010.015
SV02.010.015
0.00
0.000
0.002
0.004
0.006
0.008
0.010
Regret
in SV01.005.002
Satisficing Futures
Regret
in SV01.005.002
“Satisficing”
Futures
0.012
11-15-03
3
0
Analysis Ends by Characterizing
Uncertainties which Drive Policy Choices
Stringent/stringent
SV01-0.5%-0.2%
SV01-1%-1.5%
SV02-1%-1.5%
Stringent/lax
Lax/lax
Stringent milestones
and stringent cost
constraints
(SV01-0.5%-0.2%)
SVab-x%y%
a = N milestone
b = S milestone
x% = N cost threshold
y% = S cost threshold
Expected
Regret
}
Robust
Regions
Lax milestones
and lax cost constraints
(SV02-1%-1.5%)
Stringent milestones
and lax cost constraints
(SV01-1%-1.5%)
1:100
1:10
1:1
10:1
100:1
Relative Odds of A Low Decoupling Future
11-15-03
3
1
Outline

Robust decision making

Example of robust decisionmaking
as a means of characterizing
uncertainty

Conclusions
11-15-03
3
2
Different Methods Appropriate in
Different Circumstances
Predict-Then-Act
Robust Decisions
Complexity
High
Scenario
Planning
Few
Low
Wellcharacterized
Deep
Many
Hedging
Opportunities
Uncertainty
11-15-03
3
3
Robust Decision Making Adds Another Means to
Characterize Uncertainty for Decisionmakers

Information about future characterized by
identifying robust strategies and their
vulnerabilities

Complicated technology supports simple
operational concept

Focus on alternative policies may require

Closer coordination between analyst and
decisionmakers

Changes in process in organizations that use analysis
11-15-03
3
4
11-15-03
3
5
What About Surprises?
Landscape of
plausible futures
X
Alternative
strategies
Ensemble of
scenarios
Robust
strategies
11-15-03
3
6
The Advisory Panel Suggested Several
Potentially Stressing Surprises

Rapid technological advance that eliminates
emissions

Plague that decimates population for twenty
years

Future generations whose values (utility) are
completely disconnected from concern about the
environment
11-15-03
3
7
“Safety Valve” Strategy Is Still Robust,
Even with Surprises
Rate of change in
emissions intensity
5.0
4.0
3.0
2.0
1.0
0
–1.0
5.0
4.0
3.0
2.0
1.0
0
–1.0
No surprise
5.0
4.0
3.0
2.0
1.0
0
–1.0
5.0
4.0
3.0
2.0
1.0
0
–1.0
Population surprise
0
1.0
2.0
3.0
Technological surprise
4.0
Value surprise
0
Economic growth rate
1.0
2.0
3.0
4.0
N$ W$
NG WG
11-15-03
3
8