Transcript Application of Lean Principles to Project Management

New Risk Management
Avoiding Bias and Blindness
Presentation to Project Management Institute
Monterey Bay Chapter
March 21, 2013
Bill Wiltschko
We are Mere Humans
A. We (even statisticians) are terrible at
statistical thinking
B. We are worse when thinking about
extreme events
C. We are worse yet at understanding
the game around probabilistic events
--but there are measures we can take to compensate
Page 2
What Would You Do – Case 1?
Get $900 for sure
Or
90% Chance to get $1,000
Page 3
What Would You Do – Case 2?
Lose $900 for sure
Or
90% Chance to lose $1,000
Page 4
The Four-Fold Pattern
Gains
Losses
High
Probability
95% chance to win $10,000
Fear of disappointment
RISK AVERSE
Accept unfavorable settlement
e.g. Winning Litigant
95% chance to lose $10,000
Hope to avoid loss
RISK SEEKING
Reject favorable settlement
e.g. Losing litigant
Low
Probability
5% chance to win $10,000
Hope of large gain
RISK SEEKING
Reject favorable settlement
e.g. Lottery
5% chance to lose $10,000
Fear of large loss
RISK AVERSE
Accept unfavorable settlement
e.g. Insurance
Page 5
Which is Scarier?
The risk of permanent disability from a given
child vaccination is 0.001%
Or
One of 100,000 children will be permanently
disabled by a given vaccination.
Page 6
What is the Ratio of Deaths
from These Causes?
A
B
Stroke
Accidents
Tornados
Asthma
Lightning
Botulism
Disease
Accidents
Accidents
Diabetes
Ratio? (A:B)
Page 7
Alar
Love Canal
Page 8
Boeing 787 Batteries
•
“The battery that burst into flames two months ago on a Boeing Co. 787 jet
in Boston experienced dramatic power fluctuations and other failures its
designers had considered practically impossible, according to the latest
report by U.S. air-safety regulators.”
•
“In another section, the report reiterates that Boeing initially determined
that the likelihood of a 787 battery emitting smoke, without catching on fire,
was one in 10 million flight hours.”
Wall Street Journal, March 7, 2013,
Based on NTSB Interim Factual Report 03-07-2013
Page 9
Fukushima
•
“In a discussion of the earthquake and tsunami that produced the 2011
Fukushima nuclear disaster in Japan, Taleb writes: "Not seeing a tsunami or
an economic event coming is excusable; building something fragile to them is
not." And in the case of the Fukushima disaster, authorities seem to be
responding appropriately: not by developing better predictive models, but by
building smaller and less vulnerable reactors. “
Nassim Nicholas Taleb on Accepting Uncertainty, Embracing Volatility,
Knowledge@Wharton, December 17, 2012
•
“According to NRC documents, between 42 percent and 59 percent of the
most risk-significant accident scenarios aren't even modeled in nuclear risk
assessments. The NRC and the nuclear industry have relied on risk models
that leave them half blind to the very events they're attempting to avoid.”
US News, July 1, 2011
Page 10
Black Swan
• The Story
• Black Swan characteristics
– Outlier, high-consequence, rational only in retrospect
• One Black Swan Lesson
Page 11
One Black Swan Lesson
Absence of Evidence
≠
Evidence of Absence
Page 12
Black Swan Quadrant
Simple Payoffs
Complex Payoffs
Thin-tailed
distributions
(e.g. Gaussian)
Extremely robust
to Black Swans
Quite robust to
Black Swans
Thick-tailed
distributions,
or unknown
Quite robust to
Black Swans
Limits of statistics –
extreme fragility to
Black Swans
Complex payoff is nonlinear function of value x, say x2 or ex or worse
Page 13
Payoffs
• “True and False [or right and wrong] play a poor,
secondary role in human decisions; it is the
payoff [benefit or harm] from the True and False
that dominates – and it is almost always
asymmetric.”
Page 14
Human Phenomena Are Often Not Gaussian
• “Fat tails,” e.g. Power Law
• Non-linear payoffs
• Lack of independence
– Of variables, instances, and with other
systems
Page 15
Fragility “Triad”
Triad
Meaning
Fragile
Benefits are small and visible, negative side effects are large and invisible
Robust
Benefits and costs are moderate, side effects are small and visible
Antifragile
Cost are small and visible, positive side effects are large and invisible
Page 16
Examples
Fragile
Robust
Antifragile
Black Swan
Exposed to negative Black Swans
Exposed to positive Black Swans
Businesses
Banking
Si Valley:
“fail fast”, “be foolish”
Biological and
economic systems
Efficiency, optimized
Redundancy
Degeneracy (functional redundancy)
Errors
Hates mistakes
Mistakes are just information
Loves mistakes (since they are small)
Science
Directed research
Opportunistic research
Stochastic tinkering (“bricolage”)
Ethics
System without skin in the game
System with skin in the game
System with soul in the game
Regulation
Rules
Principles
Virtue
Finance
Debt
Equity
Venture Capital
Financial dependence Corporate worker
Dentist, niche worker,
minimum-wage worker
Taxi driver, artisan, prostitute
Literature
E-book
Book
Oral tradition
Psychological wellbeing
Post-traumatic stress
Philosophy and
science
Rationalism
Post-traumatic growth
Empiricism
Skeptical, subtractive empiricism
Page 17
What Fragility Looks Like
Zero
Benefit
Benefit
This is
Fragility
Increasing values
Page 18
What Antifragility Looks Like
Benefit
Zero
Benefit
Increasing values
•
•
•
“Antifragility implies more to gain than to lose, equals more
upside than downside, equals favorable asymmetry.”
Equivalent to buying a put or call option
Having “out-sized” benefit is dependent on
–
–
Recognizing antifragility
Recognizing when you have a right-half situation
Page 19
How To Detect Fragility
• Do a sensitivity analysis
– For example, if add 1000 cars to traffic in a city, does
drive time become linearly affected or drastically
affected?
– For example, if predictions are wrong by double the
error band,
• Will the benefit/harm increase non-linearly?
• Will benefit increase more than harm, or vice versa?
(compare results of the two sides of the error band)
Page 20
Antifragility Lessons
• “Fools game” to guess probability or statistics of extreme
events (cannot be measured or estimated)
–
–
–
–
“What’s the worst that has happened in the past?”
“Let’s use the statistics tools in Excel”
“All our observations are independent”
“We will use error bands (e.g. ± 5%) to guard against bad
predictions”
• Vulnerability can be measured and improved
– Failure Modes and Effects Analysis (FMEA)
– Mitigation of effects (i.e. risk)
Page 21
Forecasting - Two Teams
•
Team 1 - For your reference, you have been provided with the following formula for
calculating the total amount of money (T) the investment will make three months after
the initial investment (I) given the rate of return (R):
T=I*R
•
Team 2 - For your reference, you have been provided with the following formula for
calculating the total amount of money An the investment will make three months after
the initial investment An-1 given the rate of return r.
𝐴𝑛 = 𝐴𝑛−1 +(𝑛 + 1)
•
𝑗
𝑛−1
𝑗=1 [𝐴𝑗 𝑟𝑗 𝑛2 −𝑛+𝑗
− 𝑗𝐴𝑗−1 𝑟𝑗−1
1
𝑗+ 𝑛−1
2 +𝑛−2
+ 𝐴𝑗 𝑟𝑗−1
1
]
𝑗+(𝑛−1)2 +𝑛−2
How did the two teams’ behavior differ?
Page 22
Last Word(s)
•
•
“Technology is the result of antifragility, exploited by risk-takers in the form
of tinkering and trial and error, with nerd-driven design confined to the
backstage. Engineers and tinkerers develop things while history books are
written by academics;”
Suggested National Entrepreneur Day message:
“Most of you will fail, disrespected, impoverished, but we are grateful for the risks you are
taking and the sacrifices you are making for the sake of economic growth of the planet and
pulling others out of poverty. You are at the source of our antifragility. Our nation thanks you.
[italics in original]
•
•
•
“To be sophisticated you need to accept that you are not so.”
“While the gene is antifragile, since it is information, the carrier of the gene
is fragile, and needs to be so for the gene to get stronger.”
“If you take risks and face your fate with dignity, there is nothing you can do
that can make you small.”
Page 23
References
•
•
•
Thinking, Fast and Slow by Daniel Kahneman
The Black Swan by Nassim Nicholas Taleb
Antifragile by Nassim Nicholas Taleb
Bill Wiltschko
[email protected]
831 596-5073
Page 24