Transcript Lecture_1

STOCHASTIC MODELS
LECTURE 1
MARKOV CHAINS
Nan Chen
MSc Program in Financial Engineering
The Chinese University of Hong Kong
(ShenZhen)
Sept. 6, 2016
Outline
1. Introduction
2. Chapman-Kolmogrov
Equations
3. The Gambler’s ruin Problem
1.1 INTRODUCTION
What is a Stochastic Process?
• A stochastic process is a collection of random
variables that are indexed by time. Usually,
we denote it by
–
– (Xt , t ³ 0).
or
• Examples:
– Daily average temperature on CUHK-SZ campus
– Real-time stock price of Google
Motivation of Markov Chains
• Stochastic processes are widely used to
characterize the temporal relationship
between random variables.
• The simplest model should be that Xn are
independent of each other.
• But, the above model may not be able to
provide a reasonable approximation to
financial markets.
What is a Markov Chain?
• Let
be a stochastic process that
takes on a finite/countable number of
possible states. We call it by a Markov chain,
if the conditional distribution of Xn+1 depends
on the past observations
only
through Xn . Namely,
for all n.
The Markovian Property
• It can be shown that the definition of Markov
chains is equivalent to stating that
• In words, given the current state of the
process, its future and historical movements
are independent.
Financial Rationale: Efficient Market
Hypothesis
• The Markovian property turns out to be
highly relevant to financial modeling in light
of one of the most profound theory in the
history of modern finance --- efficient market
hypothesis.
• It states
– Market information, such as the information
reflected in the past record or the information
published in financial press, must be absorbed
and reflected quickly in the stock price.
More about EMH: a Thought
Experiment
• Let us start with the following thought
experiment:
Assume that Prof. Chen had invented an
formula which we could use to predict the
movements of Google stock price very
accurately.
What would happen if this formula was
disclosed to the public?
More about EMH: a Thought
Experiment
• Suppose that it predicted that Google’s
stock price would rise dramatically in three
days to US$700 from US$650.
– The prediction would induce a great wave of
immediate buy orders.
– Huge demands on Google’s stocks would push
its price to jump to $700 immediately.
– The formula fails!
• A true story of Edward Thorp and the BlackScholes formula
Implication of Efficient Market
Hypothesis
• One implication of EMH is that given the
current stock price, knowing its history will
help very little in predicting its future.
• Therefore, we should use Markov processes
to model the dynamic of financial variables.
Transition Matrix
• In this lecture, we only consider timehomogenous Markov chains; that is, the
transition probabilities P(Xn+1 = j | Xn = in ) are
independent of time n.
• Denote pij := P(Xn+1 = j | Xn = i). We then can use
the following matrix to characterize the
process.
Transition Matrix
• The transition matrix of a Markov chain must
be a stochastic matrix:
– pij ³ 0.
n
–
å pij = 1.
j=1
Example I: Forecasting the Weather
• Suppose that the chance of rain tomorrow in
Shenzhen depends on previous weather
conditions only through whether or not it is
raining today.
• Assume that if it rains today, then it will rain
tomorrow with probability 70%; and if it does
not rain today, then it will rain tomorrow with
prob. 50%.
• How can we use a Markov chain to model it?
Example II: 1-dimensional Random
Walk
• A Markov chain whose state space is given by
the integers 0,±1,±2,... is said to be a random
walk if, for some number 0 < p <1,
Pi,i+1 = p =1- Pi,i-1, i = 0,±1,±2,...
• We say the random walk is symmetric if
p =1/ 2 ; asymmetric if p ¹1/ 2.
1.2 CHAPMANKOLMOGOROV EQUATIONS
The Chapman-Kolmogorov Equations
• The CK equations provide a method for
computing the n -step transition probabilities
of a Markov chain.
•
Pijn+m = P ( Xn+m = j | X0 = i) = å Pikn Pkjm
k
or in a matrix form,
P n+m = P n P m.
Example III: Rain Probability
• Reconsider the situation in Example I. Given
that it is raining today, what is the probability
that it will rain four days from today?
Example IV: Urn and Balls
• An urn always contains 2 balls. Ball colors are
red and blue. At each stage a ball is randomly
chosen and then replaced by a new ball,
which with prob. 80% is the same color, and
with prob. 20% is the opposite color.
• If initially both balls are red, find the
probability that the fifth ball selected is red.
1.3 THE GAMBLER’S RUIN
PROBLEM
The Gambler’s Ruin Problem
• Consider a gambler who at each play of the
game has probability p of winning one dollar
and probability q =1- p of losing one dollar.
Assuming that successive plays of the game
are independent, what is the probability that,
starting with i dollars, the gambler fortune
will win N dollars before he ruins (i.e., his
fortune reaches 0)?
Markov Description of the Model
• If we let Xn denote the player’s fortune at
time n , then the process {Xn, n =1, 2,...} is a
Markov chain with transition probabilities
– P00 = PNN =1
– Pi,i+1 = p =1- Pi,i-1 , i =1,2,..., N -1.
• The Markov chain has three classes:
{0},{1, 2,..., N -1},{N}
Solution
• Let Pi be the probability that, starting with i
dollars, the gambler fortune will eventually
reach N.
• By conditioning on the outcome of the initial
play
Pi = pPi+1 + qPi-1 , i =1,2,..., N -1.
and P0 = 0, PN =1.
Solution (Continued)
• Hence, we obtain from the preceding slide
that
–
–
q
q
P2 - P1 = (P1 - P0 ) = P1;
p
p 2
æqö
q
P3 - P2 = (P2 - P1 ) = ç ÷ P1;
p
è pø
– ……
–
æqö
q
PN - PN-1 = (PN-1 - PN-2 ) = ç ÷
p
è pø
N-1
P1.
Solution (Continued)
• Adding all the equalities up, we obtain
–
–
ì
1 / N,
ï
P1 = í 1- (q / p)
ï 1- (q / p) N ,
î
ì
i / N,
ï
Pi = í 1- (q / p)i
,
ï
N
î 1- (q / p)
p =1/ 2;
p ¹1/ 2.
p =1/ 2;
p ¹1/ 2.
Solution (Continued)
• Note that, as N ®+¥,
ì
0,
ïï
i
Pi ® í
æqö
ï 1- ç ÷ ,
è pø
ïî
•
p £1/ 2;
p >1/ 2.
Thus, if p >1/ 2 , there is a positive probability that the gambler’s fortune
will increase indefinitely; while if p £1/ 2, the gambler will, with
probability 1, go ruin against an infinitely rich adversary (say, a casino).
Homework Assignments
• Read Ross Chapter 4.1, 4.2, and 4.5.1.
• Answer Questions:
– Exercises 5, 6 (Page 261, Ross)
– Exercises 13, 14 (Page 262, Ross)
– Exercises 56, 57 (Page 270, Ross)
• Due on Sept. 13, Tue.