Transcript Bridging Academic Integrity to Professional Ethics

RCR Forum
Humility and Hubris in Science: The
Rewards and Risks of Emerging
Technologies
Daniel A. Vallero, Ph.D.
Duke University, Pratt School of Engineering
March 2, 2010
1
Much is expected….
The Brain—is wider than the Sky—
For—put them side by side—
The one the other will contain
With ease—and You—beside—
The Brain is deeper than the sea—
For—hold them—Blue to Blue—
The one the other will absorb—
As Sponges—Buckets—do—
The Brain is just the weight of God—
For—Heft them—Pound for Pound—
And they will differ—if they do—
As Syllable from Sound—
Emily Dickinson (1862) “Wider than the Sky.”
2
3 Parables
1. Trolleys and Medicine
3
3 Parables
1. Trolleys and Medicine
4
3 Parables
1. Trolleys and Medicine
5
The Double Effect
• Equation with 2 constraints.
1. Socrates & Hippocrates: “First do no harm.”
2. Saint Thomas Aquinas: “Good is to be done and promoted and
evil is to be avoided.”
• Act leading to good but with negative side-effects
permitted, but deliberate harm (even for good causes) is
wrong.
• Humans are not “objects.”
• Good effect must be at least as directly an effect of the
action as is the negative bad effect.
– Bad must not cause the good effect.
• Benefit of the bad must not outweigh the benefit of the
good.
• Other ethically acceptable approach without the side
effects must not be available.
6
3 Parables
1. Trolleys and Medicine
2. Rolheiser’s “Good” Town
7
3 Parables
1. Trolleys and Medicine
2. Rolheiser’s “Good” Town
3. Hardin’s “Tragedy of the Commons”
8
Exercise
•
•
•
•
•
•
Assess expertise and interests
Read summary
Discuss
Appoint spokesperson
Answer questions
Reach consensus, if possible, but log
minority views as well
• Summarize for all of us
9
An 6-step approach to ethical
decision making (from Resnik)
1.
2.
3.
4.
State or define the problem/issue
Gather information
Delineate options.
Apply different values, rules, principles,
regulations to the different options.
5. Resolve conflicts among values, rules,
etc.
6. Make a decision and act
10
“Settled Science”
•
•
•
•
•
Climategate
Lessons?
Is science a social enterprise?
Is science’s value instrumental?
Is it inherent?
11
Chaos
Chain of events
Initial event
Present
Mitigating measures
Subsequent
event
series1…n
Subsequent
event
series1…p
Subsequent
event
series1…q
Subsequent
outcome
series1…r
Actual outcome
Probability of outcome
at outset
Desired
environmental
outcome
0.970
Fortuitous, positive
environmental
impact
0.003
Neutral
environmental
impact
0.026
Unplanned negative
environmental
impact
0.001
Future
12
Improve desired outcome, but at
what cost?
Chain of events
Actual outcome
Probability of outcome
at outset
Desired outcome
0.975
Fortuitous, positive
impact
0.002
Subsequent
event
series1…n
Subsequent
event
series1…p
Initial event
Subsequent
event
series1…q
Subsequent
outcome
series1…r
Present
Neutral impact
0.020
Unplanned negative
impact
0.003
Future
13
Decision Tree (hypothetical)
Importance
Likelihood
First Order
Outcome
Second Order
Outcome
Environment
0.810
1
1
0.005
5
2
3
Biodiversity effects
0.001
5
3
2
Pest resistance
0.010
3
2
4
Crop damage
0.020
3
3
5
3
5
4
3
5
4
Non-target effects
Spores and
crystalline
insecticidal
proteins
Efficacious with
agricultural effects
Direct poisoning*
Efficacious with human
health impacts, but
without ecological
impacts
Food
Production
1
Efficacious with no impacts
Efficacious with no human
health impacts, but with
ecological impacts
Public
Health
0.002
Indirect contamination (e.g.
track-in)
0.030
Cross-resistant bacteria
0.002
5
5
5
0.020
3
3
5
0.100
NA
NA
5
Transgenic food problems
Nonefficacious
1 = Best; 5 = Worst
*This has its own decision tree according to vulnerability index, i.e. percentile exposure (high to no exposure) and sensitive
subpopulations (children, elderly, asthmatic, etc.)
14
Precaution versus Evidence-Based
Views
• What’s the difference?
• Which is better for….
– Environmental protection?
15
The environment can be seen as series of reactors….
Antimicrobial Use
Reactor
Level 1
Animal
microbial
populations
Human
microbial
populations
Microbes introduced
Level 2
Confined
feeding
operations,
aquaculture,
farms, etc.
Microbial
Level 3
Wastes,
effluents,
emissions, drift
Microbial
Level 4
Ground &
surface waters
into the environment
Healthcare
facilities, longterm care,
daycare centers,
etc.
genetic mixing
Wastewater
treatment plants,
sewers, septic
tanks, etc.
genetic mixing
Soil &
sediments
16
Precaution versus Evidence-Based
Views
• What’s the difference?
• Which is better for….
– Environmental protection?
– Biotechnologies?
17
European classes of risks posed genetically
modified microorganisms
Hazard Level
Description of Microbial Hazard
Least
Never identified as causative agents of disease in humans and that offer
any threat to the environment.
Hazardous when
contained, low
human risk
May cause disease in human and which might, therefore, offer a hazard to
laboratory workers. They are unlikely to spread in the environment.
Prophylactics are available and treatment is effective.
Severe when
contained, moderate
human risk
Severe threat to the health of laboratory workers, but a comparatively
small risk to the population at large. Prophylactics are available and
treatment is effective.
High human
population risk
Severe illness in humans and serious hazard to laboratory workers and to
people at large. In general, effective prophylactics are not available
and no effective treatment is known.
Greatest ecological
and human
population risk
Most severe threat to the environment, beyond humans. May lead to heavy
economic losses. Includes several classes, Epl, Ep2, Ep3 (see Table 1.2
for descriptions) to accommodate plant pathogens.
18
European classes of microbes
causing diseases in plants.
Biotechnology
Class
Description of Microbes in Class
Ep 1.
May cause diseases in plants but have only local significance. They
may be mentioned in a list of pathogens for the individual countries
concerned. Very often they are endemic plant pathogens and do not
require any special physical containment. However, it may be
advisable to employ good microbiological techniques
Ep 2.
Known to cause outbreaks of disease in crops as well as in ornamental
plants. These pathogens are subject to regulations for species listed by
authorities in the country concerned
Ep 3.
Mentioned in quarantine lists. Importation and handling are generally
forbidden. The regulatory authorities must be consulted by
prospective users
19
Precaution versus Evidence-Based
Views
• What’s the difference?
• Which is better for….
– Environmental protection?
– Biotechnologies?
– Drug safety?
– Devices?
20
Precaution versus Evidence-Based
Views
• What’s the difference?
• Which is better for….
– Environmental protection?
– Biotechnologies?
– Drug safety?
– Devices?
– Sustainability?
21
Disaster
A
Escape
Release of
disinfected wastes
Transformation of
indigenous
microbes
Colonization
Persistence
Transmission of DNA to
other organisms
Deleterious Effects
Project
B
Escape
Intentional sustenance
Colonization
Persistence
Hmmmm……
Transmission of DNA to
other organisms
Beneficial Effects
22
Precaution versus Evidence-Based
Views
• What’s the difference?
• Which is better for….
– Environmental protection?
– Biotechnologies?
– Drug safety?
– Devices?
– Sustainability?
– Greatest good for the greatest number?
23
Precaution versus Evidence-Based
Views
• What’s the difference?
• Which is better for….
–
–
–
–
–
–
–
–
–
Environmental protection?
Biotechnologies?
Drug safety?
Devices?
Sustainability?
Greatest good for the greatest number?
Professional duty?
Research integrity?
Advancement of knowledge?
24
So, then what is risk, really?
• Definition: Probability of harm or loss
• Part of our everyday lives
– Different for each of us
– Basis for decision-making
• But is it quantifiable?
25
… depends on how you ask
but,...
• Yes it is quantifiable:
Risk = f(Hazard x Exposure)
• A probability, a fraction
• Part of our everyday lives
– Different for each of us
– Basis for decision-making
26
Risk assessment is informed by science (you)…
27
Different Processes at Work: Not everyone
thinks like you do….*
Analytical Phase
Identifying risk
Risk Assessment
Processes
Risk Perception
Processes
Physical, chemical, and
biological monitoring
and measuring of the
event
Personal awareness
Deductive reasoning
Intuition
Statistical inference
Estimating risk
Magnitude, frequency and
duration calculations
Personal experience
Cost estimation and
damage assessment
Intangible losses and nonmonetized valuation
Economic costs
Evaluating risk
Cost/benefit analysis
Personality factors
Community policy analysis
Individual action
*Adapted from K. Smith, 1992
28
Perception is crucial
•Which line is longer?
The Müller-Lyer Illusion.
29
Perception is complex
•Which line is longer?
The Müller-Lyer Illusion.
30
Perception is variable
•Which line is longer?
The Müller-Lyer Illusion.
31
Perception is a
benchmark
•Which line is longer?
The Müller-Lyer Illusion.
32
Perception is personal
•Which line is longer?
The Müller-Lyer Illusion.
33
Perception just is….
•Which line is longer?
The Müller-Lyer Illusion.
34
And sometimes, perception is
pretty accurate….
Source: Pardon, ca. 1970.
35
Factors increasing perceived risk:
(after Covello, 1992)
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
Possible Severely Negative or Catastrophic Outcome
Unfamiliarity with Situation and Potential Risks
Inability to Explain Processes and Mechanism
Little Certainty in the Science and Engineering
Perception of Personal Control
Involuntary Exposures to Risks
Risk to Children and Sensitive Groups
Long-term Exposures, Latency Periods, Chronic Risk
Possible Transgenerational Exposures and Risks
Uncertainty about Potential Victims
36
… and the risk is perceived to
increase even more when …
• Greater “dread”
– Major problem for nuclear power industry
• Mistrust of corporate or governmental partners
– Guilt by association
• Negative media attention
• A history of accidents and failures at this site or in
similar situations
• Benefits are not clear
• Mistakes are irreversible
– Global climate change, for example
37
Discussion Questions
• What can you do to ensure your work
meets ethical standards?
• How can you make sure that it is not
misused by others?
38
A few final words…
•
•
•
•
Do you agree with the risk paradigm?
Should it be evidence based?
How about the precautionary principle?
Hubris is thinking you know when you
don’t….
39
Trust depends on humility –
knowing that you don’t know….
• Ethike aretai
– Engineering needs character
– Engineering needs skill
• Credat emptor!
• Resolve today to keep building
competence and character.
40
If you have questions, contact me….
• [email protected]
• 919-541-3306
41