Transcript Beyond Monte Carlo Analysis

Beyond Monte Carlo
An Algorithmic Replacement for a
Misunderstood Practice
Shawn Brayman
President, PlanPlus Inc.
Shawn Brayman
• PlanPlus Founder
• 20+ years experience in financial services
IT sector
• B.Sc. (Applied & Computational Math),
MES (Expert Systems) York University
• Canadian Institute of Financial Planning
(1994) prescribed course of study
Shawn Brayman
President
PlanPlus Inc.
• Software, training, speaking on financial
planning in 9 countries
Beyond Monte Carlo
Our industry is filled with
conflicting information.
For both advisors and clients it is almost
impossible to sort out “truth” from
“fiction”.
Beyond Monte Carlo
Executive Summary
SECTION ONE – MCS Myths




MCS challenges on assumptions
MCS tests higher & lower returns
MCS tests bad years, client goals (sequence risk)
Randomizing life expectancy tests longevity risk
Beyond Monte Carlo
Executive Summary
SECTION TWO – Reliability Forecast




Precision challenges with MCS
Creating an algorithmic solution
Validating the algorithmic solution
Improved sensitivity analysis for a financial plan
Beyond Monte Carlo
Disclaimer!
1.
MCS is simply a statistical procedure and as such
cannot “be wrong”. It is how it is applied and
interpreted that there may be issues.
2.
My research specifically looked at MCS for longterm financial planning. I am sure many academics
and researchers have valid applications of MCS in
specific instances.
Beyond Monte Carlo
A Last Aside
•
•
•
•
FPA Seattle had 7 of 30 sessions that deal directly
with Monte Carlo or aspects of it (just over 23%)
3 are supportive/use it, 3 question it and 1 is “yes,
but buyer beware”
I went back and to FPA conference programs from
Nashville, San Diego and Denver
Did a randomization with 100,000 simulations and
determined in 4 years the entire FPA conference will
be about MCS with no “winner”
Beyond Monte Carlo
A Last Aside
•
•
•
•
FPA Seattle had 7 of 30 sessions that deal directly
with Monte Carlo or aspects of it (just over 23%)
3 are supportive/use it, 3 question it and 1 is “yes,
but buyer beware”
I went back and to FPA conference programs from
Nashville, San Diego and Denver
Did a randomization with 100,000 simulations and
determined in 4 years the entire FPA conference will
be about MCS with no “winner”
Beyond Monte Carlo
A Last Aside
Beyond Monte Carlo
Financial Planning is a Balancing Act
Beyond Monte Carlo
Financial Planning is a Balancing Act
Return
Risk
Sequence
Risk
Beyond Monte Carlo
Financial Planning is a Balancing Act
Return
Risk
Longevity
Risk
Sequence
Risk
Beyond Monte Carlo
Financial Planning is a Balancing Act
Return
Risk
Longevity
Risk
Sequence
Risk
Beyond Monte Carlo
Inflation & Return Assumptions
Beyond Monte Carlo
Case 1: Tom and Mary







Both age 60
Assume mortality at age 85;
Indexed pensions of $5,675 at age 60, $4,653 at 65
Both have $400,000 in retirement savings
They have $5,000 in a joint savings
Goal $60,000 after-tax during their lifetime;
Assume a 15% average tax rate
Beyond Monte Carlo
Case 1: Tom and Mary
Portfolio is
 Cash 15%
 Fixed income 40%
 Canadian equity 25%
 US equity 20%
Beyond Monte Carlo
What Have the Returns Looked Like?
Beyond Monte Carlo
Case 1: Tom and Mary
Based on data from 1950 to 2005 (56 years)
 Inflation averaged 3.94% (geometric mean)
 Return on this mix averaged 8.96% (geometric
mean)
 Real Return is
(Nominal-Inflation)/(1+inflation) = 4.83%
Beyond Monte Carlo
Case 1: Tom and Mary
Results
 Tom and Mary’s plan is perfect and they run out of
money at the end of age 85, leaving an estate of $0
to their cat.
Beyond Monte Carlo
Monte Carlo
Beyond Monte Carlo
What is Monte Carlo?

A statistical procedure where we pass an expected
return and standard deviation and get random
series of returns that “average” to our assumptions.

If we passed a standard deviation of 0%, it would
be identical to an algorithmic solution.
Beyond Monte Carlo
Monte Carlo and Returns – 3 Benefits

It tests different returns – higher and lower?

It tests for bad years up front or the impact of the
order of returns? (Sequence risk)

Gives an overall chance of success?
Beyond Monte Carlo
Run a Monte Carlo


Enter 8.96% with standard deviation of 8.07%
Results indicate 65% failure – timing?
Beyond Monte Carlo
Run a Monte Carlo


Enter 8.96% with standard deviation of 8.07%
Results indicate 65% failure – timing?
NO
• You just entered a
different return than
your plan (8.55%)
Beyond Monte Carlo
Geometric vs. Arithmetic Mean



Planning uses geometric means
Monte Carlo uses arithmetic mean
G approx R – V/(2(1+R))
Many advisors attribute increased failures to timing
when really different assumptions!
Beyond Monte Carlo
Test Higher and Lower Returns?


For Tom and Mary, 8.96% return was “perfect”.
The geometric mean is by definition the point of
50/50 probability.
Beyond Monte Carlo
Test Higher and Lower Returns?

Enter proper 9.26% rate, 50.5% failures
Beyond Monte Carlo
Test Higher and Lower Returns?

What if we increase arithmetic return and standard
deviation with maintain the same 8.96% geometric
mean?
?
Beyond Monte Carlo
Test Higher and Lower Returns?

What if we increase arithmetic return and standard
deviation with maintain the same 8.96% geometric
mean?
!
Beyond Monte Carlo
Test Higher and Lower Returns?
Conclusions:
 MCS results are strictly a function of the geometric
mean and not sensitive to the level of risk.
 If MCS reports a single “probability of success”, it will
average out results so no “test” of the range of returns
is occurring.
Recommendation:
 The success/failure result must be subtotaled to allow
reporting based on higher or lower returns.
Beyond Monte Carlo
Does MCS Test Sequence Risk?
1.
2.
3.
4.
5.
6.
7.
Even withdrawal of $52,159 indexed;
$57,423 for the first 5 years (10% increase) then reduce the
withdrawal to $49,932 for the duration of the plan.
$65,000 for the first 5 years (25% more);
$78,000 for the first 5 years (50% more);
$104,300 for the first 5 years (100% more);
Withdraw an additional lump sum of $100,000 in the 12th
year of the plan;
Withdraw an additional lump sum of $100,000 in the 20th
year of the plan.
All scenarios have the same PV for the withdrawals.
Beyond Monte Carlo
Test Sequence Risk?
Beyond Monte Carlo
Test Sequence Risk?

So all withdrawal strategies with same PV of
withdrawals resulted in the same 52% failure rate.

Repeated the same 7 withdrawal strategies using
11% return +/- 20.63% (8.96% geometric mean).
From 51.8% to 52.1% failures. No change.

Why??
Beyond Monte Carlo
Test Poor Initial Returns?

Same PV of withdrawal and geometric mean results
in same failures
Beyond Monte Carlo
Test Poor Initial Returns?

Conclusion: If the PV of the withdrawals is constant,
no difference in success/failure.

Yes MCS can creates initial poor returns but no more
so than great returns. If reports a single “probability of
success”, it will average out results so no “test”
regardless.

Observation: Unless strategies exist to deal with initial
poor returns without compromising initial great returns
– so what?
Beyond Monte Carlo
Sidebar on Inflation



Given consistent indexation of revenues and goals,
planning is totally driven by the real rate of return.
Changing inflation assumptions and return assumptions
only matter to the extent they change the real rate of
return.
10,000 simulations of inflation at 3.99% +/- 3.3%
resulted in 49.1 success, assuming savings &
withdrawals use the random variable for the indexation
rate and constant real return.
Beyond Monte Carlo
Mortality Assumptions/Longevity Risk
Beyond Monte Carlo
Mortality Assumptions
What mortality assumption would you use in financial
plans?
Mortality tables (77 for Tom)
• Age 85
• Age 90
• Age 99
Beyond Monte Carlo
Mortality

Tom and Mary are 60

Tom’s life expectancy is age 77

Mary’s life expectancy is age 85
Beyond Monte Carlo
What is the Impact of Mortality?
Mortality
Capital
Required
85
$0
90
$69,000
95
$122,500
99
$158,200
Beyond Monte Carlo
How do we Randomize?
1.
Select a random number between 1 and the
number of lives (i.e. 87,813)
2.
Map to the tables to determine mortality
Beyond Monte Carlo
Results of Randomized Mortality

When we randomize mortality, ½ will be less than
expected mortality and ½ will be more.

If we average the results, we have generated a plan
based on the average life expectancy.
If we sum Success/Failure, any scenario with
mortality 85 or less succeeds, 86 or more fails
Our results are 31% failure, 69% success – the
same as the initial mortality assumption


Beyond Monte Carlo
Another Planning Risk




In respect to life expectancy, aside from longevity
risk, need to consider RAK-IT Risk
Retire And Kick IT (IT = The Bucket!)
When we obsess on Longevity Risk we may be
planning for our compliance department, not the
client
I would say “Forget longevity risk if it means
delaying my retirement and getting to my boat!”
Beyond Monte Carlo
Another Planning Risk
Beyond Monte Carlo
Longevity Risk



Conclusion:By randomizing mortality to represent
longevity, advisors are, on average, basing their
planning on a lower mortality assumption than would
be used in traditional plans.
The success/failure is identical to initial mortality
assumptions
Observation: Results can be subtotaled based on the
age parameter, which would allow reporting based on
mortality bands.
Beyond Monte Carlo
Section One Summary




Use of arithmetic means is subject to error and
misinterpretation
The practice of reporting a single result washes out
all concept of testing extremes and results in a plan
based on the mean.
People talk as if randomization tested only a portion
of the distribution – poor returns initially, living longer
than planned, but it is a statistical process that
equally looks at great initial returns or dieing sooner.
MCS returns the same results as the geometric
mean when consistent assumptions are maintained.
Beyond Monte Carlo
Section 2: Reliability Forecasts
Pre-Decease Age 85
(69%)
Outlive Age 85 (31%)
Returns
Above 8.96
(50%)
A 34 ½ % chance the client
will die before age 85 and
have higher than projected
returns (Surplus – Success)
A 15 ½ % chance the client
will outlive age 85 with
higher than projected returns
(Unclear outcome)
Returns
Below 8.96
(50%)
A 34 ½ % chance the client
will die before age 85 but
have lower than projected
returns (Unclear outcome)
A 15 ½ % chance the client
will outlive age 85 with lower
than projected returns
(Capital shortfall – Failure)
MCS provides an interesting “partial probabilities” removing
ambiguity around the “unclear outcomes”
Beyond Monte Carlo
MCS for Returns & Mortality
1.
2.
3.
4.
10,000 simulations
9.26% return +/- 8.07% or 8.96% geometric
Randomized mortality
72.9% successful and 27.1 failed sims
Is this good?
• Dying at age 61 is success because $ remain
• Living to 105 with funds exhausted at 103 is a
failure
Beyond Monte Carlo
A Challenge with Precision…
Simulated
Returns/Age
Planning
Tool
Beyond Monte Carlo
A Challenge with Precision…
Simulated
Returns/Age
Planning
Tool
Win/Loose Count
Beyond Monte Carlo
A Challenge with Precision…
Simulated
Returns/Age
Planning
Tool
Win/Loose Count
Beyond Monte Carlo
A Challenge with Precision…
Simulated
Returns/Age
Planning
Tool
Win/Loose Count
The more comprehensive the planning algorithm,
the longer per trial
Beyond Monte Carlo
A Challenge with Precision…
Simulated
Returns/Age
Planning
Tool
Win/Loose Count
The more comprehensive the planning algorithm,
the longer per trial
“While running the requisite 100,000 scenarios that would provide a minimal
margin of error is computationally not feasible within most real-time engines,
running as little as a few hundred scenarios can be woefully inadequate.”
Beyond Monte Carlo
100 Sims Distribution
Beyond Monte Carlo
Goals for an Algorithmic Solution
1.
2.
3.
4.
To generate partial probabilities like MCS
To remove issues of speed and therefore precision
To generate more usable results than a single
likelihood of success
To make the model easier to maintain
Beyond Monte Carlo
2 Critical Observations
Simulated
Returns/Age
Planning
Tool
•
The randomized variables are independent of the
plan! (This does not mean the plan is independent
of these variables.)
•
MCS returns the same cumulative result as the
geometric mean assumption, so…
Beyond Monte Carlo
So…
Beyond Monte Carlo
So…
X
Beyond Monte Carlo
=
Likelihood of Age x ROI
0.1% to 0.15%
chance of
dieing at 93
with a 10.46%
to 10.96%
return
0.35% to 0.4%
chance of
dieing at 87
with a 8.46% to
8.96% return
Beyond Monte Carlo
At each return, when are $ exhausted?
Return
4.96
5.46
5.96
6.46
6.96
7.46
7.96
Age Funds
Depleted
76.5
77
78
79
80
81
82.5
Remember age 85 and 8.96% return?
Return
8.46
8.96
9.46
9.96
10.46
10.96
Age Funds
Depleted
84
85
89
93
99
100+
So we run our planning model (in this case 13 times) to
calculate the boundary for success and failure
Beyond Monte Carlo
Our Reliability Forecast
Success
Failure
Beyond Monte Carlo
Reliability Forecast Results
1.
2.
Subtotal of cells in the “Success” region is 74.5%
This compares to 72.9% +/- 1.57% with MCS and
10,000 simulations
Beyond Monte Carlo
Reliability Forecast Results (2)
1.
Tested with withdrawal of $78,000 for 5 years and
reduced $ thereafter. No change in RF solution
(74.5%), 73.7% success with MCS.
2.
Tested with $100,000 withdrawal in 12th year of
plan. Same for RF (74.5%), 73.5% success for
MCS.
Beyond Monte Carlo
Reliability Forecast Results (3)
1.
2.
Tested accelerated withdrawal of $58,633 and
funds gone at age 80.
RF with new boundary gave 59.5% success. MCS
resulted in 60.3% success.
Beyond Monte Carlo
Reliability Forecast Results (4)
1.
Tested withdrawal of $65,000, 10% return and 11%
standard deviation (9.5% geometric mean).
2.
RF with new boundary gave 51.6% success. MCS
resulted in 51.2% success with 10,000 sims.
The Reliability Forecast provides the same partial
probabilities as MCS with a dozen iterations of the
planning model.
Beyond Monte Carlo
Subtotaling Results
What if the client lives to age 90?
Beyond Monte Carlo
Subtotaling Results
What if the client averages at least 6.96% return?
Beyond Monte Carlo
Improved Distribution Matrix
Beyond Monte Carlo
Sensitivity Analysis
Sensitivity Analysis
Age
Probability Success
70
80.7%
100.0%
75
65.7%
97.8%
80
47.1%
83.4%
85
27.4%
68.3%
90
11.5%
58.9%
95
2.6%
54.6%
Return Probability Success
11.5
22.4%
89.2%
10.5
37.2%
77.8%
9.5
54.4%
68.7%
8.5
70.9%
61.9%
7.5
84.0%
57.9%
Beyond Monte Carlo
Failure
0.0%
2.2%
16.6%
31.7%
41.1%
45.4%
Failure
10.8%
22.2%
31.3%
38.1%
42.1%
Sensitivity Analysis
Probability of Success
Age/Return
7.5
70
100.0%
75
100.0%
80
88.6%
85
73.6%
90
63.8%
95
59.2%
8.5
100.0%
100.0%
92.3%
78.0%
68.0%
63.2%
9.5
100.0%
100.0%
97.6%
85.2%
75.1%
70.1%
Beyond Monte Carlo
10.5
100.0%
100.0%
100.0%
93.4%
84.2%
79.3%
11.5
100.0%
100.0%
100.0%
100.0%
94.7%
90.6%
Sensitivity Analysis
Beyond Monte Carlo
Sensitivity Analysis
Bad!
Beyond Monte Carlo
Sensitivity Analysis
Annual Reviews
Beyond Monte Carlo
Summary of Findings
Issue
Return assumptions
Monte Carlo
Reliability Forecast
Based on arithmetic mean.
Estimation technique can convert.
Advisors do not understand
Not as utilized. Requires additional
computation and subtotaling of
results in return bands.
Misunderstood.
Misunderstanding. Does not occur.
Based on geometric mean, same
as financial plan
Test longevity using randomized
mortality
Not as utilized. Could add
additional layer of subtotaling.
Easily subtotaled.
Precision
Poor due to computational load.
Require 100,000 simulations.
High. 12 calculations are necessary
to generate the boundary line.
Ability to generate overall
probabilities of success
Yes
Yes
Ability to provide improved
granularity on sensitivity analysis
Requires additional computation
and subtotaling logic.
Yes, easily subtotaled.
Ability to maintain the planning tool
Complex .May restrict certain
capabilities.
Simpler. The algorithmic solution is
easier to maintain and audit.
Test higher and lower returns
Test impact of withdrawals and
order of returns (sequence risk)
Beyond Monte Carlo
Easily subtotaled.
Not calculated.
Conclusion
The Reliability Forecast approach outlined provides all
the analytical results available using MCS in a fraction
of the calculations, using more common assumptions,
leading to higher precision.
Beyond Monte Carlo
Questions
?
Beyond Monte Carlo