Transcript No Slide Title

Carnegie Mellon University
Risk Analysis and Communication:
What can we learn from research?
M. Granger Morgan
Head, Department of
Engineering and Public Policy
Carnegie Mellon University
Pittsburgh PA 15213 USA
tel: 412-268-2672
e-mail: [email protected]
Department of Engineering and Public Policy
1
Carnegie Mellon University
First, Two Slides on
Carnegie Mellon's Department of
Engineering and Public Policy
http://www.epp.cmu.edu
A department in the Engineering college at Carnegie Mellon
University.
Faculty: Total of 41. Include true 50:50 joint appointments with
all five engineering departments as well as joint appointments
with four different social science units in three other colleges.
Undergraduate double major degrees with traditional
departments (574 BS graduates to date).
Graduate program is a research-oriented Ph.D. focused on
problem in which the technical details really matter (current
enrollment 45, 119 Ph.D. graduates to date).
Department of Engineering and Public Policy
2
Carnegie Mellon University
Research
Four major areas:
1. energy and environment.
2. risk analysis and communication.
3. telecommunication and information policy.
4. technology policy.
In the context of these four areas, we also work on issues in
technology and development (China and India) and on issues
in dual-use technology, arms control, and defense policy.
EPP currently has several large collaborative group efforts:
•
•
•
•
•
•
•
•
Center for Integrated Study of the Human Dimensions of Global Change.
The Electricity Industry Center.
Green Design Initiative.
Center for Energy and Environmental Systems.
Center for the Study and Improvement of Regulation.
Brownfields Center.
IT and telecommunications policy.
Risk analysis, ranking, communication.
Department of Engineering and Public Policy
3
Carnegie Mellon University
Today I will talk about:
• What is risk?
• Basic ideas in risk analysis.
• A few details on the characterization and treatment of
uncertainty.
• Basic ideas in risk management.
• Basic ideas in risk communication.
I've also included some slides which I will not show which
summarize a few results from four different recent research
projects which I'd be happy to discuss individually.
• Some recent topics of research.
Department of Engineering and Public Policy
4
Carnegie Mellon University
Risk
My dictionary defines risk as follows:
risk - n. 1. The exposure to the chance of injury
or loss; a hazard or dangerous chance: he decided
to take the risk.
Note:
• negative outcome;
• uncertain;
• not just the probability of loss.
While this looks relatively simple, in the real world,
things get more complicated…
Department of Engineering and Public Policy
5
Carnegie Mellon University
Let's perform
a little thought experiment
Suppose I have a new product.
I've done careful market research and know:
I could sell Q devices at a price P, for total
revenues QP.
I'd make a profit R.
BUT, the product will have a net impact on
national mortality of D excess deaths/year.
When would I be justified in introducing this product?
Department of Engineering and Public Policy
6
Carnegie Mellon University
Some of the things that may matter...
-
A few people bear the risks and many get the benefits.
The product is frivolous.
The deaths do not occur immediately.
I can identify the individuals before/after the deaths.
The deaths all occur at once/are spread out.
The people are socially related.
D = N-M where N is deaths caused and M is deaths
prevented.
- The effects are uncertain, <P(n)> = N.
- In addition to mortality, there is morbidity; environmental
impact; etc.
In short, risk is a "multi-attribute concept…7
Department of Engineering and Public Policy
Carnegie Mellon University
Risks can be reliably sorted
in terms of such factors...
QuickTime™ and a TIFF (Uncompressed) decompressor are needed to see this pi cture.
QuickTime™ and a TIFF (Uncompressed) decompressor are needed to see this pi cture.
Source: Slovic et al., in Readings in Risk, 1980.
Department of Engineering and Public Policy
8
Carnegie Mellon University
A useful framework for
thinking about risk
nature
Exposure
processes
Eff ects
processes
Perception
processes
Evaluation
processes
benef its
and costs
human
activity
Objects and
systems exposed
to possibility of
change
Changes that
occur
Perceived
changes
Department of Engineering and Public Policy
9
Carnegie Mellon University
A specific example
w eather
natural air
pollutants
coal-f ired
pow er plant
Transport,
dif fusion,
oxidation
and
deposition
culture, political climate
Eff ects
processes
Material in atmos.
Plants and people
exposed
Perception
processes
Sunsets get
redder; crops
grow n in sulf ate
poor soil increase
yield; people
develop
respiratory
dif ficulties.
Evaluation
processes
benef its
and costs
Perceived
changes
Department of Engineering and Public Policy
10
Carnegie Mellon University
The role of values
Value-f ree analysis
possible in principle.
Value-f ree analysis
inherently not possible.
nature
Exposure
processes
Eff ects
processes
Perception
processes
Evaluation
processes
benef its
and costs
human
activity
Objects and
systems exposed
to possibility of
change
Changes that
occur
Perceived
changes
Department of Engineering and Public Policy
11
Carnegie Mellon University
Today I will talk about:
• What is risk?
• Basic ideas in risk analysis
• A few details on the characterization and treatment of
uncertainty.
• Basic ideas in risk management.
• Basic ideas in risk communication.
• Some recent topics of research.
Department of Engineering and Public Policy
12
Carnegie Mellon University
A highly simplified taxonomy
of tools for risk assessment
SO2
SO4
wet and
dry deposition
risks f rom
continuous
exposure
risks f rom
discrete
events
transport and
dispersion
models
doseresponse
models
f ailure mode
and eff ects
analysis
f ault tree
analysis
Department of Engineering and Public Policy
13
Carnegie Mellon University
Transport and Dispersion Models
QuickTime™ and a TIFF (U ncompressed) decompressor are needed to see thi s picture.
Source: Morgan and McMichael, Policy Sciences, 1981.
Department of Engineering and Public Policy
14
Carnegie Mellon University
Non-linear, without (A)
and with (B) threshold.
Response (e.g. cancer incidence)
Linear, without (A)
and with (B) threshold.
Response (e.g. cancer incidence)
Dose Response Functions
A
B
threshold
Dose (e. g. concent ration times t ime)
A
B
Dose (e. g. concent ration times t ime)
Department of Engineering and Public Policy
15
Complications
Variable sensitivity among
individuals.
Response (e.g. cancer incidence)
Carnegie Mellon University
Healthy
C xt= D
time
concentration
Response depends in a timedynamic way on exposure
(i.e., no one-to-one mapping
between cumulative
exposure and response).
concentration
Dose (e.g. concentration times time)
C xt= D
Dead
time
Department of Engineering and Public Policy
16
Carnegie Mellon University
Examples of real dose-response functions
QuickTime™ and a TIFF (Uncompressed) decompressor are needed to see this picture.
QuickTime™ and a TIFF (Uncompressed) decompressor are needed to see this picture.
Source: Morgan, IEEE Spectrum, 1981.
Department of Engineering and Public Policy
17
Carnegie Mellon University
We could go through a similar
discussion for discrete events
Event trees:
QuickTime™ and a TIFF (U ncompressed) decompressor are needed to see this picture.
Fault trees:
QuickTime™ and a TIFF (Uncompressed) decompressor are needed to see this picture.
Source: Rasmussen, Ann. Rev. of Energy, 1981.
Department of Engineering and Public Policy
18
Carnegie Mellon University
Today I will talk about:
• What is risk?
• Basic ideas in risk analysis
• A few details on the characterization and treatment of
uncertainty.
M. Granger Morgan, Max Henrion, with a chapter by Mitchell Small, Uncertainty: A
guide to dealing with uncertainty in quantitative risk and policy analysis, 332pp.,
Cambridge University Press, New York, 1990. (Paperback edition 1992. Latest printing
(with revised Chapter 10) 1998.)
• Basic ideas in risk management.
• Basic ideas in risk communication.
• Some recent topics of research.
Department of Engineering and Public Policy
19
Carnegie Mellon University
Probability
Probability is the basic language of uncertainty.
I will adopt a personalistic view of probability
(sometimes also called a subjectivist or Bayesian view).
In this view, probability is a statement of the degree of
belief that a person has that a specified event will occur
given all the relevant information currently known by that
person.
P(X|i) where:
X is the uncertain event
i is the person's state of information.
Department of Engineering and Public Policy
20
Carnegie Mellon University
The clairvoyant test
Even if we take a personalist view of probability, the event
or quantity of interest must be well specified for a
probability, or a probability distribution, to be meaningful.
"The retail price of gasoline in 2008" does not pass this
test. A clairvoyant would need to know things such as:
• Where will the gas be purchased?
• At what time of year?
• What octane?
Department of Engineering and Public Policy
21
Carnegie Mellon University
Does a subjectivist view mean
your probability can be arbitrary?
NO, because if they are legitimate probabilities,
they must
• conform with the axioms of probability
• be consistent with available empirical data.
Lots of people ask, why deal with probability? Why not
just use subjective words such as "likely" and "unlikely" to
describe uncertainties? There are very good reasons not to
do this.
Department of Engineering and Public Policy
22
Carnegie Mellon University
The risks of using qualitative
uncertainty language
Qualitative uncertainty language is inadequate
because:
- the same words can mean very different things to
different people.
- the same words can mean very different things to
the same person in different contexts.
- important differences in experts' judgments about
mechanisms (functional relationships), and about
how well key coefficients are known, can be
easily masked in qualitative discussions.
Department of Engineering and Public Policy
23
Carnegie Mellon University
This figure shows the
range of probabilities that
people are asked to
assign probabilities to
words, absent any
specific context.
Qualitative description of uncertainty used
Mapping
words to
probabilities
range of
individual
upper bound
est imat es
range from upper
t o lower m edian
est imat e
range of
individual
lower bound
est imat es
Alm ost certain
P robable
Likely
Good chance
P ossible
T ossup
Unlikely
Improbable
Doubt ful
Alm ost im possible
1.0
0.8
0.6
0.4
0.2
0.0
Probability that subjects associated
with the qualitative description
Figure adapted from Wallsten et al., 1986.
Department of Engineering and Public Policy
24
Carnegie Mellon University
Key :
Ex Com of
EPA SAB
el
li k
t
no
en y
we i kel
t
be t l
ng d no
i
eth n
m ly a
o
s ke
li
y
Other m ee ting pa rticipa nts:
0.000001
0.00001
0.0001
0.001
0.01
0.05
0.1
0.3
0.5
0.7
0.9
Figure from Morgan, HERA, 1998.
y
SAB m e m be rs:
1.0
The minimum probability
associated with the word
"likely" spanned four orders of
magnitude.
The maximum probability
associated with the word "not
likely" spanned more than five
orders of magnitude.
There was an overlap of the
probability associated with the
word "likely" and that
associated with the word
"unlikely"!
el
li k
P roba bility tha t the m ate rial
is a hum an c ar cinoge n
Department of Engineering and Public Policy
25
Carnegie Mellon University
The bottom line
Without at least some quantification, qualitative descriptions of
uncertainty convey little, if any, useful information.
Here are two examples from the climate assessment
community:
Schneider and Moss have worked to get a better treatment of
uncertainty incorporated into the past and current round of
IPCC. Progress is uneven, but awareness is growing.
Individual investigators are pushing the process along.
The U.S. National Assessment Synthesis Team gave
quantitative definitions to five probability words and tried to
use them consistently throughout their overview report.
Department of Engineering and Public Policy
26
Carnegie Mellon University
In doing risk analysis, we must consider
two quite different kinds of uncertainty
1. Situations in which we know the relevant variables
and the functional relationships among them, but we
no not know the values of key coefficients (e.g., the
"climate sensitivity").
2. Situations in which we are not sure what all the
relevant variables are, or the functional relationships
among them (e.g., will rising energy prices induce
more technical innovation?).
Both are challenging, but the first is much more easily
addressed than the second.
Department of Engineering and Public Policy
27
Carnegie Mellon University
Uncertainty about quantities
From Morgan and Henrion, Uncertainty, Cambridge, 1990/99.
Department of Engineering and Public Policy
28
Carnegie Mellon University
One should…
…use available data, and well-established physical and
statistical theory, to describe uncertainty whenever either or
both are available.
However, often the available data and theory are not exactly
relevant to the problem at hand, or they are not sufficiently
complete to support the full objective construction of a
probability distribution.
In such cases, one may have to rely on expert judgment.
This brings us to the problem of how to "elicit" expert
judgment.
Department of Engineering and Public Policy
29
Carnegie Mellon University
Expert elicitation
takes time and care
Eliciting subjective probabilistic judgments requires careful
preparation and execution.
Developing and testing an appropriate interview protocol
typically takes several months. Each interview is likely to
require several hours.
When addressing complex, scientifically subtle questions of the
sorts involved with most problems in climate change, there are
no satisfactory short cuts. Attempts to simplify and speed up the
process almost always lead to shoddy results.
The next eight slides (which I will skip because time is short) talk
about the important issues of overconfidence and the impacts of
"cognitive heuristics." These are critically important topics for
anyone who actually plans to do expert elicitation. <Skip to 37.>
Department of Engineering and Public Policy
30
Carnegie Mellon University
Extra slide - will not show
Over
Confidence
Source: Morgan and Henrion,
1990/99.
Department of Engineering and Public Policy
31
Carnegie Mellon University
Extra slide - will not show
Over
Confidence
Source: Morgan and Henrion, 1990/99.
Department of Engineering and Public Policy
32
Extra slide - will not show
Carnegie Mellon University
Cognitive heuristics
When ordinary people or experts make judgments about
uncertain events, such as numbers of deaths from chance
events, they use simple mental rules of thumb called
"cognitive heuristics."
In many day-to-day circumstances, these serve us very well,
but in some instances they can lead to bias - such as over
confidence - in the judgments we make.
This can be a problem for experts too.
The three slides that follow illustrate three key heuristics:
"availability," "anchoring and adjustment," and
"representativeness."
Department of Engineering and Public Policy
33
Extra slide - will not show
Carnegie Mellon University
Cognitive bias
from Lichtenstein et al., 1978.
Availability: probability judgment is driven by ease with
which people can think of previous occurrences of the event
or can imagine such occurrences.
Department of Engineering and Public Policy
34
Carnegie Mellon University
Extra slide - will not show
Cognitive bias…(Cont.)
from Lichtenstein et al., 1978.
Anchoring and adjustment: probability judgment is frequently
driven by the starting point which becomes an "anchor."
Department of Engineering and Public Policy
35
Extra slide - will not show
Carnegie Mellon University
Cognitive bias…(Cont.)
I flip a fair coin 8 times. Which of the following two
outcomes is more likely?
Outcome 1: T, T, T, T, H, H, H, H
Outcome 2: T, H, T, H, H, T, H, T
Of course, the two specific sequences are equally
likely...but the second seems more likely because it
looks more representative of the underlying random
process.
Representativeness: people judge the likelihood that an object
belongs to a particular class in terms of how much it resembles
that class.
Department of Engineering and Public Policy
36
Carnegie Mellon University
Extra slide - will not show
Expert elicitation…(Cont.)
In all our elicitation studies, we've focused on creating a process
that allows the experts to provide their carefully considered
judgment, supported by all the resources they may care to use.
Thus, we have:
- Prepared a background review of the relevant literatures.
- Carefully iterated the questions with selected experts and run pilot
studies with younger (Post-doc) experts to distil and refine the
questions.
- Conducted interviews in experts' offices with full resources at hand.
- Provide ample opportunity for subsequent review and revision of the
judgments provided.
All of these efforts have involved the development of new
question formats that fit the issues at hand.
Department of Engineering and Public Policy
37
Carnegie Mellon University
Expert elicitation …(Cont.)
Over the past two decades, my colleagues and I have
developed and performed a number of substantively detailed
expert elicitations. These have been designed to obtain
experts’ considered judgments. Examples include work on:
Health effects of air pollution from
coal-fired power plants.
•
M. Granger Morgan, Samuel C. Morris, Alan K. Meier and Debra L. Shenk, "A
Probabilistic Methodology for Estimating Air Pollution Health Effects from Coal-Fired
Power Plants," Energy Systems and Policy, 2, 287-310, 1978.
•
M. Granger Morgan, Samuel C. Morris, William R. Rish and Alan K. Meier, "Sulfur
Control in Coal-Fired Power Plants: A Probabilistic Approach to Policy Analysis,"
Journal of the Air Pollution Control Association, 28, 993-997, 1978.
•
M. Granger Morgan, Samuel C. Morris, Max Henrion, Deborah A.L. Amaral and
William R. Rish, "Technical Uncertainty in Quantitative Policy Analysis: A Sulfur
Air Pollution Example," Risk Analysis, 4, 201-216, 1984 September.
•
M. Granger Morgan, Samuel C. Morris, Max Henrion and Deborah A. L. Amaral,
"Uncertainty in Environmental Risk Assessment: A case study involving sulfur
transport and health effects," Environmental Science & Technology, 19, 662-667, 1985
August.
Department of Engineering and Public Policy
38
Carnegie Mellon University
Expert elicitation…(Cont.)
Climate science, climate impacts
and mitigation technology:
•
M. Granger Morgan and David Keith, "Subjective Judgments by Climate Experts,"
Environmental Science & Technology, 29(10), 468-476, October 1995.
•
Elizabeth A. Casman, M. Granger Morgan and Hadi Dowlatabadi, “Mixed Levels of
Uncertainty in Complex Policy Models,” Risk Analysis, 19(1), 33-42, 1999.
•
M. Granger Morgan, Louis F. Pitelka and Elena Shevliakova, "Elicitation of Expert
Judgments of Climate Change Impacts on Forest Ecosystems," Climatic Change, 49,
279-307, 2001.
•
Anand B. Rao, Edward S. Rubin and M. Granger Morgan, "Evaluation of
Potential Cost Reductions from Improved CO2 Capture Systems,"Proceedings of the
2nd National Conference on Carbon Sequestration, Alexandria, VA, May 5-8, 2003.
Bounding uncertain health risks:
•
M. Granger Morgan, "The Neglected Art of Bounding Analysis," Environmental
Science & Technology, 35, 162A-164A, April 1, 2001.
•
Minh Ha-Duong, Elizabeth A. Casman, and M. Granger Morgan, "Bounding Poorly
Characterized Risks: A lung cancer example," Risk Analysis, in press.
Department of Engineering and Public Policy
39
Warming for
2x[CO2]
expert
Carnegie Mellon University
-10
-5
0
5
10
15
20
0
5
10
15
20
1
2
2 with st at e change
3
4
4 with "surprise"
5
6
7
8
9
10
11
12
13
14
15
16
-10
-5
Temperature response given 2x[CO
Source:
Morgan and Keith, ES&T, 1995.
2
] (K)
…and, lest you conclude that most of these
40
Department
Engineering
and Public Policy
experts
areofin
basic agreement…
Pole to equator
temperature
gradient for
2x[CO2]
expert
Carnegie Mellon University
1
2
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
30
35
40
w/cli mate state change
na
na
na
30
35
40
Meridinal temperature gradient given 2x[CO
2
] (K)
Source: Morgan and Keith, ES&T, 1995.
Department of Engineering and Public Policy
41
Carnegie Mellon University
Biomass in Northern Forests
w/ 2xCO2 climate change
B.
Change
in soil carbon
1
Expert
A.
Change
in standing biomass
2
North A merica
North A merica
Eurasia
Eurasia
"tri vial"
North A merica
3
Eurasia
4
5
North A merica
North A merica
Eurasia
Eurasia
6
w/ o permafrost
w/ permafrost
w/ o permafrost
w/ permafrost
7
8
9
North A merica
North A merica
10
Eurasia
Eurasia
11
0. 4
0. 6
0. 8
1. 0
North A merica and E urasia E of the Urals
North A merica and E urasia E of the Urals
Europe west of the Urals
Europe west of the Urals
1. 2
1. 4
1. 6
1. 8
2. 0
2. 2
Change in standing biomass in minimally disturbed Northern Forests
bet ween 45°N and 65°N under specified 2x[ CO2] climat e change.
Source: Morgan et al., Climatic Change, 2001.
2. 4
0. 2
0. 4
0. 6
0. 8
1. 0
1. 2
1. 4
Change in soil carbon in minimally disturbed
Northern Forests between 45°N and 65°N
under specified 2x[CO 2 ] climat e change.
Department of Engineering and Public Policy
42
Carnegie Mellon University
Biomass in Tropical Forests
w/ 2xCO2 climate change
A.
Change in standing
biomass
soil carbon
Expert
1
B.
Change in
2
3
4
5
6
7
8
9
10
11
0.4
0.6
0.8
1.0
1.2
1.4
Change in standing bi omas s in minimall y
dis turbed Tropic al Forests between
20°N and 20°S under spec ifi ed
2x[CO 2 ] cli mate c hange.
0.4
0.6
0.8
1.0
1.2
1.4
Change in soil carbon in minimall y
dis turbed Tropic al Forests between
20°N and 20°S under spec ifi ed
2x[CO 2 ] cli mate c hange.
Source: Morgan et al., Climatic Change, 2001.
Department of Engineering and Public Policy
43
Carnegie Mellon University
Uncertainty about model form
Often uncertainty about model form is as or more important
than uncertainty about values of coefficients. Until recently
there had been little practical progress in dealing with such
uncertainty, but now there are several good examples:
• John Evans and his colleagues at the Harvard School of
Public Health (Evans et al., 1994).
• Alan Cornell and others in the seismic risk (Budnitz et
al., 1995).
• Hadi Dowlatabadi and colleagues at Carnegie Mellon in
Integrated Assessment of Climate Change (ICAM)
(Morgan and Dowlatabadi, 1996).
Department of Engineering and Public Policy
44
Carnegie Mellon University
ICAM
Integrated Climate
Assessment Model
T o run the model:
1 - Double click on INP UT S to set up t he scenario inputs;
2 - Double click on ST RUCT URE to set up the model;
3 - Double click on OUT P UT S and evaluate t he indicat ors.
Demogr aphi cs
& Economi cs
Impacts of
Cli mate Change
I NTERVENTI ON
Atmospheri c
Composi tion &
Cli mate
Ener gy &
Emissions
Inputs
Structure
Outputs
See for example:
Hadi Dowlatabadi and M. Granger Morgan, "A Model Framework for
Integrated Studies of the Climate Problem," Energy Policy, 21(3), 209221, March 1993.
and
M. Granger Morgan and Hadi Dowlatabadi, "Learning from Integrated
Assessment of Climate Change," Climatic Change, 34, 337-368, 1996.
A very large hierarchically
organized stochastic
simulation model built
in Analytica®.
Ae r os ol
Mode l
GHG
Mode ls
RITS
Conc
Change in
Short Wave
Forcing
Change in
Long Wave
Forcing
Forcing
Elicite d
Clim ate Mode l
Department of Engineering and Public Policy
Regional
²T
45
Carnegie Mellon University
ICAM deals with...
…both of the types of uncertainty I've talked about:
1. It deals with uncertain coefficients by assigning PDFs
to them and then performing stochastic simulation to
propagate the uncertainty through the model.
2. It deals with uncertainty about model functional form
(e.g., will rising energy prices induce more technical
innovation?) by introducing multiple alternative
models which can be chosen by throwing "switches."
Department of Engineering and Public Policy
46
Carnegie Mellon University
ICAM
There is not enough time to present any details from our work
with the ICAM integrated assessment model. Here are a few
conclusions from that work:
• Different sets of plausible model assumptions give
dramatically different results.
• No policy we have looked at is dominant over the wide
range of plausible futures we’ve examined.
• The regional differences in outcomes are so vast that few
if any policies would pass muster globally for similar
decision rules.
• Different metrics of aggregate outcomes (e.g., $s versus
hours of labor) skew the results to reflect the OECD or
developing regional issues respectively.
Department of Engineering and Public Policy
47
Carnegie Mellon University
These findings lead us...
...to switch from trying to project and examine the future, to
using the modeling framework as a test-bed to evaluate the
relative robustness, across a wide range of plausible model
futures, of alternative strategies that regional actors in the
model might adopt.
We populated the model's regions with simple decision agents
and asked, which behavioral strategies are robust in the face
of uncertain futures, which get us in trouble.
Thus, for example, it turns out that tracking and responding to
atmospheric concentration is more likely to lead regional
policy makers in the model to stable strategies than tracking
and responding to emissions.
Department of Engineering and Public Policy
48
Carnegie Mellon University
Our conclusion
Prediction and policy optimization are pretty silly analytical
objectives for much assessment and analysis related to the
climate problem.
It makes much more sense to:
• Acknowledge that describing and bounding a range of
futures may often be the best we can do.
• Recognize that climate is not the only thing that is
changing, and address the problem in that context.
• Focus on developing adaptive strategies and evaluating
their likely robustness in the face of a range of possible
climate, social, economic and ecological futures.
Department of Engineering and Public Policy
49
Carnegie Mellon University
Today I will talk about:
• What is risk?
• Basic ideas in risk analysis
• A few details on the characterization and treatment of
uncertainty.
• Basic ideas in risk management.
• Basic ideas in risk communication.
• Some recent topics of research.
Department of Engineering and Public Policy
50
Carnegie Mellon University
Strategies for
managing risk
Modify the
hum an
activity
Modify the
e xpos ure
pr oce s s
Modify the
e ffe cts
pr oce s s
Modify
hum an
pe r ce ptions
Modify
hum an
valuations
Mitigate
or
com pe ns ate
nature
Exposure
processes
Eff ects
processes
Perception
processes
Evaluation
processes
benef its
and costs
human
activity
Objects and
systems exposed
to possibility of
change
Changes that
occur
Perceived
changes
Department of Engineering and Public Policy
51
Carnegie Mellon University
Examples
No strategy is best everywhere. Depending on context,
different strategies do best in different settings.
Avoid or
modify
exposure
processes
Avoid or
modify effects
processes
Wear seat belts. Carry auto
insurance.
Install protective systems. Operate good
emergency
medical system.
Sue other
driver.
Risk
Modify natural
or human
environment
Risk of vehicle
occupant
injury in auto
accident.
Live close to
work and walk.
Build rapid
transit systems.
Change speed
limits.
Get tough with
DUIs.
Train people to
drive
defensively.
Risk of getting
shot by
someone with
handgun.
Eliminate
poverty,
inequality,
anger, mental
illness, and so
on.
Ban handguns.
Impose harsh
penalties for use
in crimes.
Stay out of high
crime areas.
Wear protective
bulletproof
clothing.
Duck.
Mitigate or
compensate
for effe cts
Carry health
insurance.
Operate good
emergency
medical system.
Sue person who
shot you.
Department of Engineering and Public Policy
52
Carnegie Mellon University
Strategies for managing risk
• Tort and other common law (e.g., negligence, liability,
nuisance, trespass).
• Insurance (private, public or hybrid).
• Voluntary standard setting organizations (UL, ASTM, etc.).
• Individual and collective corporate initiatives (CMA Responsible Care).
• Information-based strategies (e.g, TRI, green labels).
• Mandatory government standards and other regulations
(performance standards, design standards).
• Market-based solutions (emissions taxes, tradable permits).
Department of Engineering and Public Policy
53
Carnegie Mellon University
Today I will talk about:
• What is risk?
• Basic ideas in risk analysis
• A few details on the characterization and treatment of
uncertainty.
• Basic ideas in risk management.
• Basic ideas in risk communication.
M. Granger Morgan, Baruch Fischhoff, Ann Bostrom and Cynthia Atman, Risk
Communication: A mental models approach, 351pp., Cambridge University Press, New
York, 2002.
• Some recent topics of research.
Department of Engineering and Public Policy
54
Carnegie Mellon University
The traditional approach…
…to risk communication involves two steps:
1. Ask an expert what people should be told.
2. Get a "communications expert" to package it.
And, if you are being really fancy:
3. Run some tests to see how people like it.
However, if you think about it for a few minutes, this
approach ignores two critical issues:
•
What the people receiving the message already
"know" about the topic.
•
What they need to know to make the decisions they
face.
Department of Engineering and Public Policy
55
Carnegie Mellon University
Finding out what people know…
…is not simple! If I give people a questionnaire, I have to
put information in my questions.
People are smart. They will start making inferences
based on the information in my questions.
Pretty soon I won't know if their answers reflect what
they already knew, or the new ideas they have come up
with because of the information I have given them.
I need a better, less intrusive way to learn what they
know. The method we have developed to do this is called
the mental model interview.
Department of Engineering and Public Policy
56
Carnegie Mellon University
Five step process:
In work done over the past 15 years at Carnegie Mellon, we
have developed a five-step approach to risk communication,
based on people's mental models of risk processes:
2. Conduct an openended elicitation of
people's beliefs
about a hazard,
allowing the
expression of both
accurate and
inaccurate concepts.
Map the results to
the expert influence
diagram.
Cumulative number of concepts
1. Develop an "influence diagram" to structure expert
knowledge.
160
B2
120
S1
B3
S2
S3
S4
80
B1
A
40
0
1
10
20
30
Number of interviews
Department of Engineering and Public Policy
57
Carnegie Mellon University
Five steps…(Cont.)
3. On the basis of results from the open-ended interviews,
develop and administer a closed-form questionnaire to a much
larger group in order to determine the prevalence of these
beliefs.
4. Develop a draft communication based on both a decision
analytic assessment of what people need to know in order to
make informed decisions and a psychological assessment of
their current beliefs.
5. Iteratively test successive versions of those communications
using open-ended, closed-form, and problem-solving
instruments, administered before, during, and after the receipt
of the message.
In 1994, we applied these methods to study public understanding
of climate change. While the results are now getting old, I will
show you a few since they are unlikely to have changed much.
Department of Engineering and Public Policy
58
Carnegie Mellon University
Example of opening response
in interview
Interviewer: "I'd like you to tell me all about the issue of
climate change."
Subject: "Climate change. Do you mean global warming?"
Interviewer: "Climate change."
Subject: "OK. Let's see. What do I know. The earth is getting
warmer because there are holes in the atmosphere and this is
global warming and the greenhouse effect. Um... I really
don't know very much about it, but it does seem to be true.
The temperatures do seem to be kind of warm in the winters.
They do seem to be warmer than in the past.. and.. hmm..
That's all I know about global warming.
Department of Engineering and Public Policy
59
Carnegie Mellon University
Another example…
Interviewer: "Tell me all about the issue of climate
change."
Subject: "I'm pretty interested in it... The ice caps are
melting -- the hole in the ozone layer. They think
pollution from cars and aerosol cans are the cause of
all that. I think the space shuttle might have
something to do with it too, because they always
send that up through the earth, to get out in outer
space. So I think that would have something to do
with it, too."
Department of Engineering and Public Policy
60
Carnegie Mellon University
Another example…
Interviewer: "Tell me all about the issue of climate
change."
Subject: "Climate change? Like, what about it? Like, as far
as the ozone layer and ice caps melting, water level
raising, rainforest going down, oxygen going down
because of that? All of that kind of stuff?"
Interviewer: "Anything else?"
Subject: "Well, erosion all over the place. Um, topsoils
going down into everywhere. Fertilizer poisoning.
"Interviewer: "Anything else that comes to mind related
to climate change?Subject: "Climate change. Winter's
ain't like they used to be. Nothing's as severe. Not as
much snow. Nothing like that."
Department of Engineering and Public Policy
61
Carnegie Mellon University
Schematic of the approach
21
Conduct openended interv iews
of represent ativ e
lay people.
Develop/ref ine
representations of
lay mental models.
32
Validat e representation(s)
and prev alence of mental
model(s) wit h closed-f orm
questionnaires.
Structure
model of
lay v alues.
Structure decision problems
lay people f ace.
Structure
risk managment
strat egy.
Ev aluate risk
communicat ion
with open-ended,
closed-f orm and
problem solv ing
instruments.
Design risk
communicat ion
(if required).
4
3
1
Evaluate risk
managment
strat egy (eg.
changes in
behavior, et c. ).
Develop expert
model of t he risk
problem.
Other social
and economic
considerat ions.
5
4
Department of Engineering and Public Policy
62
Carnegie Mellon University
Based on what we have learned…
…in our studies of public understanding of various risks we
have produced various public communication materials:
Light-weight "Details Books"
in pockets at the back of each of the
three sections provide a third level
of details.
After the initial two-page overview,
three two-page spreads concisely
summarize the key ideas.
h ang e?
li mate c
An initial two-page spread
hits the main points and lists
common misconceptions
• •
t1 1
ar art
•P •P
t1 1
ar rt
•P •Pa
t1 1
ar art
•P •P
t 1 t1
ar ar
•P •P
t1 1
ar art
•P 1•P
rt
a
•P
c
Wh at is
Part 1:
I have brought a
few examples for
people who are
interested.
Department of Engineering and Public Policy
63
Carnegie Mellon University
A few conclusions
There is no such thing as an "expert" in public
communication who can simply tell you what to do.
An empirical approach is essential.
Before developing a communication one must learn
what the public knows and thinks.
The mental models method offers a promising
strategy for doing this.
Uncertainty must be quantified.
Department of Engineering and Public Policy
64
Carnegie Mellon University
Today I will talk about:
• What is risk?
• Basic ideas in risk analysis
• A few details on the characterization and treatment of
uncertainty.
• Basic ideas in risk management.
• Basic ideas in risk communication.
• Some recent topics of research.
Department of Engineering and Public Policy
65
Carnegie Mellon University
In the Department of Engineering and
Public Policy at Carnegie Mellon…
…we've had an active program in risk-related research for 30
years. Since time is very short, I've included a few illustrative
slides on four recent projects:
• Dealing with mixed and extreme uncertainty.
Elizabeth A. Casman, M. Granger Morgan and Hadi Dowlatabadi, “Mixed Levels of Uncertainty in
Complex Policy Models,” Risk Analysis, 19(1), 33-42, 1999.
• The use of bounding analysis.
Minh Ha-Duong, Elizabeth A. Casman, and M. Granger Morgan, “Bounding Poorly Characterized Risks: A
Lung Cancer Example,” Risk Analysis, in press.
• Risk ranking.
H. Keith Florig et al. "A Deliberative Method for Ranking Risks (I): Overview and test bed development,"
Risk Analysis, 21(5), 913-921, 2001 and Kara M. Morgan et al. "A Deliberative Method for Ranking Risks
(II): Evaluation of validity and agreement among risk managers," Risk Analysis, 21(5), 923-937, 2001.
• Public perceptions of deep geological disposal of CO2.
Claire R. Palmgren, M. Granger Morgan, Wändi Bruine de Bruin, and David W. Keith "Initial Public
Perceptions of Deep Geological and Oceanic Disposal of Carbon Dioxide," in review at ES&T.
Department of Engineering and Public Policy
66
Carnegie Mellon University
Acknowledgments
Over the years my work on risk has been greatly assisted by
collaborations with many colleagues including:
Elizabeth Casman
Michael DeKay
Hadi Dowlatabadi
Paul Fischbeck
Baruch Fischhoff
Keith Florig
Max Henrion
Karen Jenni
David Keith
Lester Lave
Kara Morgan
Louis Pitelka
Elena Shevliakova
Patti Steranchak
Henry Willis
Department of Engineering and Public Policy
67
Mean change in
global temperature,°C
Limited
domain of
model validity
6
5
Expert 1
4
Probability of a change
in the climate state
Extra slide - will not show
Carnegie Mellon University
0.30
0.20
3
2
0.10
1
0.00
2000
2025
2050
2075
2100
Probability of a change
in the climate state
Mean change in
global temperature,°C
Year
6
5
Expert 8
0.30
4
0.20
3
2
0.10
1
0
1975
0.00
2000
2025
2050
2075
2100
Probability of a change
in the climate state
Year
Mean change in
global temperature,°C
Examples of warming
estimated via the
ICAM model (dark
curves) and probability
that the associated
climate forcing will
induce a state change
in the climate system
(light curves) using the
probabilistic judgments
of three different
climate experts.
0
1975
6
5
Expert 15
0.30
4
0.20
3
2
0.10
1
0
1975
0.00
2000
2025
2050
2075
2100
Year
Department of Engineering and Public Policy
68
Carnegie Mellon University
Extra slide - will not show
Model
switching
Schematic illustration of
the strategy of switching
to progressively simpler
models as one moves into
less well understood
regions of the problem
phase space, in this case,
over time.
Department of Engineering and Public Policy
69
Carnegie Mellon University
Extra slide - will not show
Illustration
of model
switching
Results of applying
the model switchover strategy to the
ICAM demographic
model (until about
2050) and an
estimate of the
upper-bound
estimate of global
population carrying
capacity based on
J. S. Cohen.
Department of Engineering and Public Policy
70
Extra slide - will not show
Carnegie Mellon University
Bounding
When the science is poorly understood, probabilistic risk
analysis is routinely used to obtain estimates of health impacts,
with results typically reported in the form of a very broad
subjective probability density function.
For impacts with multiple causes, such estimates are usually
made separately, by different investigators, for each cause of
interest.
However, if those separate probabilistic estimates were brought
together and summed, the results could sometimes substantially
exceed the numbers of cases actually observed.
Department of Engineering and Public Policy
71
Extra slide - will not show
Carnegie Mellon University
Bounding analysis
In a commentary in ES&T in April of 2001, I argued that
methods of bounding analysis could be used in environmental
risk analysis to avoid such problems. For health endpoints with
multiple external causes, the available knowledge can be used
to constrain estimates of the magnitude of the poorly
characterized risks.
If most risks were known with precision, this would be a simple
subtraction problem. But health risks from environmental
causes often involve high uncertainty. However, in many cases,
there is agreement on the general magnitude of the impacts of
the best-studied causes. The idea is to use this knowledge to
bound the sum of the other less well known risks.
M. Granger Morgan, "The Neglected Art of Bounding Analysis," Viewpoint,
Environmental Science & Technology, 35, 162A-164A, April 1, 2001.
Department of Engineering and Public Policy
72
Carnegie Mellon University
Extra slide - will not show
To illustrate the method…
…we have developed an example that harmonizes informed
beliefs about the relative contribution made by different causal
factors using total current U.S. lung cancer mortality.
The goal of the analysis is to generate an upper bound on the
mortality attributed to the group of poorly-characterized factors,
derived from information on the group of well-characterized
risk factors.
To perform this bounding analysis it is first necessary to
apportion lung cancer mortalities among the various known
causes, or groups of causes, according to current scientific
knowledge and opinion.
Department of Engineering and Public Policy
73
Extra slide - will not show
Carnegie Mellon University
All lung cancers:
Lung cancers attributable to:
tobacco:
radon:
Illustration
asbestos:
all others:
Department of Engineering and Public Policy
74
Extra slide - will not show
Carnegie Mellon University
In the example…
…I am presenting today, we have developed the needed
bounds by reviewing the relevant literatures and making
our own judgments.
We are currently running an expert elicitation to seek the
carefully informed judgment of a number of lung cancer
experts.
Department of Engineering and Public Policy
75
Extra slide - will not show
Carnegie Mellon University
We partition…
…the lung cancer deaths into non-intersecting categories of
single causative factors and causal factor groupings and
constrain the sum of all mortality across those partitions to
equal the observed mortality.
In addition to causal categories, we define a "background"
category for the lung cancer deaths that would have occurred
in the absence of exposure to all carcinogens.
While there is no way to measure this quantity, clearly it exists
in principle, and it can be bounded through consideration of
lung cancer deaths in groups with low exposures to known
carcinogens.
Department of Engineering and Public Policy
76
Extra slide - will not show
Carnegie Mellon University
In our illustration…
… the set of possible causes of lung cancer () consists of
active, former, and passive smoking (C); domestic exposure to
radon (R); occupational exposure to inhaled asbestos (A); and
the group of all other environmental risk factors (X).
Note that our analysis deals with annual lung cancer mortality
in the U. S. population as a whole and not in subpopulations
that experience different exposure histories, display different
susceptibilities and have different access to health care.
Since genetic factors play a role in all cancers, we consider
them to be a priori, non-manipulable attributes of the overall
U.S. population.
Department of Engineering and Public Policy
77
Extra slide - will not show
Carnegie Mellon University
Graphically
The basic stat istic n, showing only risk factors C, R, and X, for simplicity. N is the total
number of lung cancer fatalities. n is the number of fatalities apportioned to each risk
factor or combinat ion of factors. n() is the background number of lung cancer deaths
that would occur absent all the various risk fact ors. n() = n(CRX) and represents the
number of cases for which no risk fact or can be excluded.
N
n()
n(C)
n(R)
n(CR)
n(X)
n(XC)
n(XR)
n()
Department of Engineering and Public Policy
78
Extra slide - will not show
Carnegie Mellon University
Smoking
Smoking-Attributable Mortalit y (SAM), Cancers of the Trachea, Lung and Bronchus,
1999 (Source: CDC)
Mortality from
Lung Cancer
Active & Passive
Neoplasms of
Deat hs from
Smoking
T rachea, Lung,
Secondhand
At tribut able
and Bronchus
Smoking(28)
Fraction
Total deaths
89,337
Males
SAM
78,459
1,110
percent
87.8%
89.1%
Total deaths
62,613
Females
SAM
44,727
1,890
percent
71.4%
74.4%
Total deaths
151,950
Males
and
SAM
123,186
3,000
83.0%
Females
percent
81.1%
2%
Department of Engineering and Public Policy
79
Extra slide - will not show
Carnegie Mellon University
In summary
Based on our review of the literature, we have constructed a
set of judgments apportioning lung cancer deaths among
these causes which imply the following constraints:
f u (C) = 0.70, f l(C) = 0.95
f u (R) = 0.21, f l(R) = 0.02
f u (A) = 0.05, f l (A) = 0.01
where f u denotes an upper bound on the fraction of lung
cancer deaths due to a particular cause and f l the lower
bound.
Department of Engineering and Public Policy
80
Extra slide - will not show
Carnegie Mellon University
From the bounds…
…on the proportions of annual lung cancer mortality due to
the major lung cancer causes (C, R, and A), information about
the magnitude of risk from all other lung cancer causes (X) is
C = N - n( )
then inferred, using a
consistency constraint on the
total number of deaths.
i.e. C = 100%
Graphically we perform:
X = 0%
R = N - n( )
i.e. R = 100%
Pn
C = 0%
R = 0%
X = N - n( )
i.e. X = 100%
Department of Engineering and Public Policy
81
Extra slide - will not show
Carnegie Mellon University
Example results
Results of Optimization: Upper and Lower Bounds on Att ributable Fractions and
Relative Risks of Cancer Risk Factors
C
R
A
X
Upper bound on af
95%
21%
5.0%
3.2%
Lower bound on af
70%
2%
1.0%
0.0%
Exposure probability
45%(a)
50%*(b)
5%(c)
5%**
Upper bound on rr
43.2
1.53
2.05
1.66
Lower bound on rr
6.19
1.04
1.20
1.00
a. National Center for Health St atist ics, 2001.
b. NRC, 1999.
c. Dept . of Labor, 2002.
* The fraction of U. S. homes with radon concent rations at or above 25 Bqm-1 .
** Our est imate.
Department of Engineering and Public Policy
82
Carnegie Mellon University
Extra slide - will not show
Example results…(Cont.)
This example calculation assigns between 0% and
3.2% of lung cancer deaths to X, the group of
unspecified occupational and environmental
pollutants.
Thus, for the group of known and suspected lung
carcinogens other than C, A, and R, if one is confident
in the bounds assigned to the well understood risk
factors, the sum of the effects of the poorly
understood factors collectively should account for no
more than 3.2% of total lung cancer mortality.
This provides a constraint on estimates of those risks
produced by more conventional risk analysis.
Department of Engineering and Public Policy
83
Carnegie Mellon University
Extra slide - will not show
Checking diesel estimates
Two national associations of air quality control offices published
a projection of the number of cancers due to exposure to diesel
exhaust.
They estimated that diesel would be responsible for 125,110
cancers for all metropolitan and non-metropolitan areas of the
U.S. (over 70-year lifetimes) for an annual rate of 1,787 cancers.
This figure is below 3% (4,716 deaths) of the projected 2003
lung cancer mortality rate of 157,200, even without adjusting for
the non-lung cancer mortality inherent in their risk estimate, and
therefore, to first order, would pass our plausibility test.
Department of Engineering and Public Policy
84
Extra slide - will not show
Carnegie Mellon University
Our objective...
...in the work we've done on risk ranking has been to develop and
demonstrate a method which:
• Uses experts to analyze and characterize the risks (because
they have the necessary knowledge).
• Uses modern risk communication methods to describe the
risks in multi-attribute terms (so that they will be
understandable to educated members of the general
public).
• Uses representative groups of laypeople to perform the
actual ranking (because ranking requires the application of
social values).
• Produces a fairly "thick" description of the deliberations,
including ranks which are robust, and which are useful as
an input in risk management decision making.
Department of Engineering and Public Policy
85
Extra slide - will not show
Carnegie Mellon University
Steps in the risk-ranking method
Step A
Def ine and
cat egorize the
risks to be
ranked.
Step B
Ident if y the risk
attribut es that
should be
considered.
Step C
Describe t he
risks in t erms of
the att ributes in
risk summary
sheets.
Step D
Select
participant s
and perf orm
the risk
rankings.
Step E
Describe t he
issues
ident if ied and
the resulting
rankings.
Department of Engineering and Public Policy
86
Carnegie Mellon University
Extra slide - will not show
First experimental test-bed:
risks in schools
We chose risks in a hypothetical middle school as our testbed because:
• most people know and care about the topic -- major
efforts are not required to get lay subjects briefed
before participating in studies;
• risks in schools are not the responsibility of any
single existing U.S. Government risk management
agency;
• the topic offers opportunities to address a wide range
of physical and chemical risks as well as important
social issues;
• there are a number of recent studies on which we can
build.
Department of Engineering and Public Policy
87
Carnegie Mellon University
Extra slide - will not show
Example of
a risk summary
sheet
Risk name
Brief summary
description of the risk .
Tabular summary, evaluating
the risk in terms of the
standard attributes and
description of the level of
uncertainty.
General description o f the risk,
discussion of the risk in
thespecific context of the
Centerville Middle Sch ool, and
discussion of what the school
has don e to deal with the risk.
Department of Engineering and Public Policy
88
Extra slide - will not show
Carnegie Mellon University
Town of Centerville
clif f
air
f ield
Water treatment
plant
uR
usa
Wa
clif f
iver
Schuman Hill
regional park
w ater
tow er
C&LL
Alvarez Expressw ay
Rail
Road
to Centerville Heights
230kV transmission line
Centerville
Middle School
C&
LL
R
city
hall
ail
Ro
ad
f ire
station
police
Centerville
Elementary
School
W
au
sa
u
R
ive
r
Centerville
High School
Centerville
Landing
City of Centerville
0
Sew age
treatment
plant
0.5
1.0
Scale in miles
Department of Engineering and Public Policy
89
Extra slide - will not show
Carnegie Mellon University
reta
inin
gw
all
Centerville
Middle School
utility
tunnel
loading
dock
Room 141
janitorial
Room 110
classroom
Room 121
kitchen
Room 139
classroom
Room 140
utility
stage
serving area
Room 109
classroom
Room 138
classroom
Room 120
cafeteria
elevator
Room 108
classroom
Room 137
classroom
Room 130
auditorium
Room 135
library
Room 106
classroom
Room 107
girl's
washroom
Room 136
classroom
Room 119
boy's
washroom
Room 105
classroom
Room 104
classroom
Room 134
classroom
Rooms 100 to 103
administrative offices
Room 131
classroom
Room 133
classroom
Room 132
classroom
Centerville Middle School
First Floor
Scale in feet
0
10
20
30
40
50
Department of Engineering and Public Policy
90
Extra slide - will not show
Carnegie Mellon University
Second test-bed
The middle school test-bed involved only health and safety
risks.
After developing and refining the method in that context,
we then developed a second test-bed (based on a
hypothetical county) in order to extend the work to include
ecological and environmental risks.
A major part of the work in creating this second test-bed
has involved developing an appropriate set of relevant
attributes. That's a fairly long story that I will not go into,
but Ph.D. student Henry Willis has done a great job with
this problem.
Department of Engineering and Public Policy
91
Extra slide - will not show
Carnegie Mellon University
De Paul County
0
DePaul
County
2
6
4
Scale in miles
8
C&LL RR
State
Route
246
I- 82
For est
u
Wa
3
ive
uR
sa
r
5
er
R iv
ue
q
u
b
Du
7
4
I- 82
Ce nte r ville
2
1
For est
6
C&LL RR
For est
Harris
State Park
Cry stal
Lake
Au Cla
1.
2.
3.
4.
5.
6.
7.
er
ir e Riv
Fishkill Pow er Plant
Site of f ormer Johnson's Wood Products Plant
DeKay County Sanatary Land Fill
Cebulka's Premium Chickens
Ganley Estates
Feldman's Feeds
Site of proposed Tw in Pines Mall
Department of Engineering and Public Policy
92
Carnegie Mellon University
Extra slide - will not show
Risks used in the two test-beds
The middle school test-bed
• Accidental injuries
• Airplane Crash
• Allergens
• Asbestos
• Bites and Stings
• Building Collapse
• Commuting to school on foot, bike
or by car
• Common infectious diseases
• Drowning
• Electrical Power
• Electromagnetic fields (EMF)
• Fire and Explosion
• Food Poisoning
• Hazardous material transport
• Intentional Injury
• Lead poisoning
• Less common infectious diseases
• Lightning
• Radon gas
• School bus accidents
• Self-Inflicted Injury
• Team Sports
The county-level test-bed
• Agricultural runoff
• Air pollution from electric power generation
• Food poisoning
• Genetically-modified corn
• Invasive species
• Land filling municipal solid waste
• Motor vehicle accidents
• Recreational motor boating
• Road salt and road salt runoff
• Transporting hazardous materials by truck
Department of Engineering and Public Policy
93
Extra slide - will not show
Carnegie Mellon University
Typical ranking procedure
Study
all
materials
Individual
holistic
ranking
Start
group
ranking
Individual
MA
ranking
Revise
group
ranking
Final
individual
H and MA
rankings
Department of Engineering and Public Policy
94
Extra slide - will not show
Carnegie Mellon University
Summary of findings
with school test-bed:
Consistency between the rankings that have resulted from the
holistic and multiattribute procedures has been good for both
individuals and for groups, suggesting that these procedures
capture an underlying construct of riskiness.
Rankings of risks were similar across individuals and groups,
even though individuals and groups did not always agree on
the relative importance of risk attributes.
Lower consistency between the risk rankings from the holistic
and multiattribute procedures and lower agreement among
individuals and groups regarding these rankings were observed
for a set of high-variance risks.
Department of Engineering and Public Policy
95
Extra slide - will not show
Carnegie Mellon University
The story in numbers:
218 individuals (in 43 groups) have performed risk rankings for
health and safety risks.
0.863
Initial Indiv idual
Holistic
Risk Rankings
0.770
Group
Holistic
Risk Rankings
0.915
Final Indiv idual
Holistic
Risk Rankings
(Step D1)
(Step D3)
(Step D5)
0.595
0.829
0.860
0.595
0.686
Initial Indiv idual
Multi-Attribute
Risk Rankings
(f rom Step D2)
0.759
0.658
Group
Multi-Attribute
Risk Rankings
0.858
(f rom Step D4)
0.913
Final Indiv idual
Multi-Attribute
Risk Rankings
0.915
(f rom Step D6)
0.857
0.920
Department of Engineering and Public Policy
96
Extra slide - will not show
Carnegie Mellon University
Summary...(Cont.)
Participants reported high levels of satisfaction with their
groups’ decision-making processes and the resulting
rankings, and these reports were corroborated by
regression analyses.
Because of the generally high levels of consistency,
satisfaction, and agreement we have observed we
conclude that this deliberative method is capable of
producing risk rankings that can serve as informative
inputs to public risk-management decision making.
Department of Engineering and Public Policy
97
Carnegie Mellon University
Extra slide - will not show
Findings from
the environmental/ecological studies
• Participants are able to complete expanded ranking
task.
• Holistic and multiattribute rankings are consistent with
each other.
• Agreement shown between individuals and groups.
• Though less satisfied with attribute ranking process,
participants revealed satisfaction with final group risk
rankings.
• Results parallel previous findings from health and
safety test-bed.
Department of Engineering and Public Policy
98
Carnegie Mellon University
Extra slide - will not show
In order to study public perceptions…
…of CCD, we had to modify the basic mental model interview
approach since the typical member of the general public
knows nothing about this technology.
Part 1: Using language that we made as neutral as we could, we explained the
motivation of wishing to reduce the accumulation of atmospheric CO2 and
briefly outlined the basic design options of CCD technologies. We then asked
a series of questions to elicit reactions.
Part 2: Briefly discussed three questions: 1) "Can the technology to separate
and dispose of carbon dioxide be made practical, and cheap enough"; 2) "Once
the CO2 is put down deep in rock formations or deep in the oceans, will it stay
there?" and 3) "If the technology can be made cheap and reliable, will the
energy industry adopt it and use it widely?" Again, we asked a series of
questions to elicit reactions.
Part 3: Discussed "some of the concerns that critics might raise about these
technologies." Topics covered including CO2 pipeline issues, slow leaks, fast
releases, issues related to ocean ecology and issues related to hydrogen safety.
We asked a series of specific evaluative questions.
Department of Engineering and Public Policy
99
Carnegie Mellon University
Extra slide - will not show
Results from our pilot study suggested...
...that the standard concerns about siting would arise but may
not be dramatically different than those associated with other
large technologies.
Fears about rupture of high pressure CO2 pipelines did not
appear to be any greater than those associated with natural gas
pipelines.
Some concerns were expressed about large rapid releases from
deep geological formations, but they looked manageable.
However, while deep geological injection appeared like it
would prove to be publicly acceptable, the pilot results
suggested that proposals for deep ocean injection were likely
to lead to vigorous public opposition.
Department of Engineering and Public Policy
100
Carnegie Mellon University
Extra slide - will not show
Ocean sequestration example comments
I think the concern that really strikes me the most would have to be pumping
it under very high pressure into the deep ocean . . . I know the ocean is very
big and is very deep but I’m wondering what kind of affect it would have on
our oceans. [S1]
That, if this extra CO2 is absorbed into the ocean, would it disrupt whatever
balance is in the ocean? That it might be harmful to things that live in the
ocean. [S3]
Well, where are they going to build these? Do they have to be near the
ocean? Or are they going to build big pipelines into the ocean to flush the
stuff away? In the process of doing this, is there going to be pollution
occurring, from this process? [S5]
So, I don’t necessarily like the fact that it’s being pumped down deep in the
ocean, kind of like out of sight, out of mind. [S7]
So, if we were to put it, like, in the ocean, we could be messing with
some form of life that’s on the bottom. I don’t think we have much
knowledge of what’s down there. Because we really can’t explore that deep.
So we’d be messing with something we have no knowledge of. [S8]
Department of Engineering and Public Policy
101
Carnegie Mellon University
One of the things we noted…
…in our pilot studies was that subjects wanted to consider
alternatives to CCD.
Accordingly, in designing the closed form survey, we included
questions which allowed us to explore preferences across several
alternative strategies for reducing CO2 emissions.
Survey outline:
1.
2a.
2b.
3.
4.
5.
5.1
5.2
6.
7.
8.
Background
Climate change
General issues including climate change
Options for limiting CO2
Energy systems that dispose of CO2
Places to dispose of CO2
Deep rock formations
Disposal of CO2 in the deep ocean
Final evaluation
Environmental issues (NEPS = New Ecological Paradigm Scale)
A few questions about yourself
Department of Engineering and Public Policy
102
Extra slide - will not show
Carnegie Mellon University
Views on climate change
T he cont inuing release of CO2 into t he
earth's at mosphere during t his cent ury
may result in serious clim at e change.
4.6
4.7
Government regulation
should begin t o
significantly limit t he am ount of CO2 that
is released into t he earth's at mosphere.
Government regulation
will begin t o
significantly limit t he am ount of CO2 that
is released into t he earth's at mosphere at
som e time in t he next 20 years.
1
completely
disagree
4.0
4
7
Department of Engineering and Public Policy
103
Carnegie Mellon University
Extra slide - will not show
Not high
on respondent's
list of concerns
This is consistent with
previous studies that we
and others have done in
the U.S. Results are quite
different in Europe.
BUT, remember, despite
this result, climate change
clearly has political legs
in the U.S. in many states.
Department of Engineering and Public Policy
104
Carnegie Mellon University
Options
"Whatever your own beliefs are about
climate change, imagine that the U.S.
government has decided that we must cut
in half the amount of CO2 that is released
by generating electric power.
Suppose that your electricity supplier has
different methods of meeting the goal to
reduce CO2 emissions by 50%. Some of
these methods use a mixture of generation
systems that produce little or no CO2,
combined with regular coal-burning power
plants. These methods do not all cost the
same because some ways of making
electricity with less CO2 are cheaper than
others.
The supplier will be offering their
customers a choice of how they would like
them to meet this reduction. Some of the
options to reduce CO2 emissions will be
more expensive than the way we produce
electricity today, so we would like you to
tell us which method you would be willing
to pay the most for."
Extra slide - will not show
Options that will reduce CO2 emissions by 50%
from your energy consumption
Rank
1 is what you
would pay the
most for.
Biomass B lend:
50% f rom biomass, 50% from regular coal
Half of the energy comes from regular coal, and the other half comes from bioma ss. Biomass
refers to harvesting and burning special fast growing grasses and trees to make energy. Biomass
releases CO 2 that plants have absorbed from the atmosphere so no new CO2 is produced.
Coal Blend with deep geological disposal of CO2:
50% f rom coal with deep geological disposal of CO2, 50% f rom regular coal
Half of the energy comes from regular coal. The other half also comes from coal, but won't let the
CO2 enter the atmosphere. Instead they will separate the CO2 and dispose of it thousands of feet
underground.
Coal Blend with deep ocean disposal of CO2:
50% f rom coal with deep ocean disposal of CO2, 50% f rom regular coal
Half of the energy comes from regular coal. The other half also comes from coal, but won't let the
CO2 enter the atmosphere. Instead they will separate the CO2 and dispose of it deep in the ocean.
Energy Ef f iciency:
100% regular coal
All of the energy comes from regular coal. This option will replace appliances and lights in your
house with much more effi cient ones so that you use half as much electricity while enjoying using
your lights and appliances just as much as you do now. Using less electricity will result in less
CO2 being produced.
Hydro-electric B lend:
50% f rom hydro-electric, 50% from regular coal
Half of the energy comes from regular coal, and the other half comes from hydro-electric. Hydroelectric refers to making electricity with water power from dams . Hydro-electric produces no
CO2.
Natural gas Blend:
100% f rom natural gas
The energy comes from burning natural gas in highly efficient plants. Burning natural gas
efficiently produces about half as much CO2 as burning coal.
Nuclear power Blend:
50% f rom nuclear power, 50% from regular coal
Half of the energy comes from regular coal, and the other half comes nuclear power.
Nuclear power produces no CO2.
Solar power Blend:
50% f rom solar power, 50% from regular coal
Half of the energy comes from regular coal, and the other half comes from solar. Solar power
refers to making electricity with energy fr om the sun. Solar power produces no CO2.
Wind power Blend:
50% f rom wind power, 50% from regular coal
Half of the energy comes from regular coal, and the other half comes from wind. Wind power
refers to making electricity with energy fr om the wind. Wind power produces no CO2.
Department of Engineering and Public Policy
105
Carnegie Mellon University
Extra slide - will not show
Rank order
of options
Options used different
generation mixes (base of
coal) in order to reduce
CO2 emissions by 50%.
Department of Engineering and Public Policy
106
Carnegie Mellon University
Extra slide - will not show
Evaluation
of
specifics
The next three
slides, which I will
not show, give you
the numbers on
these and other
results.
Department of Engineering and Public Policy
107
Extra slide - will not show
Carnegie Mellon University
Summary evaluations
Geological disposal:
before details
after details
1
completely
oppose
4
7
completely
favor
Ocean disposal:
before details
after details
1
completely
oppose
4
7
completely
favor
Department of Engineering and Public Policy
108
Extra slide - will not show
Carnegie Mellon University
CCD Conclusions
The results of this study suggest that, at best, the public is
likely to view this technology with mixed feelings.
High levels of public acceptance will almost certainly require:
• broader public understanding of the need to limit carbon
dioxide emissions and alternative options for carbon
management;
• a much stronger scientific understanding and a larger
empirical base for claims about the likely efficacy and
safety of disposal; and
• an approach to public communication, regulation,
monitoring, and emergency response which is open and
respectful of public concerns.
Department of Engineering and Public Policy
109
Carnegie Mellon University
Extra slide - will not show
CCD Conclusions…(Cont.)
An open and inclusive approach does not guarantee
success.
However, an arrogant approach such as the one adopted
in the past by the industries responsible for nuclear
power and genetically modified crops, could create a
level of public distrust that makes the wide-spread
implementation of geological carbon disposal in the U.S.
difficult, if not impossible.
Department of Engineering and Public Policy
110