Transcript Markov Processes - Oldham Sixth Form College

Markov Processes
Aim Higher
What Are They Used For?


Markov Processes are used to make
predictions and decisions where results are
partly random but may also be influenced by
known factors
Applications include weather forecasting,
economic forecasting, manufacturing and
robotics
What Are They?


Markov Processes use a series of matrices to
predict the outcome of a chain of random events
which may be influenced by known factors
These matrices predict the probability of a
system changing between states in one time step
based on probabilities observed in the past
Examples of Application




When predicting the weather it may be sensible,
based on past observations, to assume that it is
more likely to rain tomorrow given that it is raining
today.
Probabilities can be indicated for a given time step
P(R2|R1) > P(R2|S1)
or
PRR > PRS
This is not an accurate forecasting method but it can
give some indication of the likely probability of the
weather changing from one state – rain, sun, cloud,
snow, etc – to another.
Creating A Markov System


An initial transition
matrix is required to
show the probability
of state changes in
one time step:
One time step in this
case could be
decided as 24 hours
Rain
Cloud
Sun
Rain
0.6
0.4
0.2
Cloud
0.3
0.4
0.3
Sun
0.1
0.3
0.6
Weather Forecasting
Cloud
Sun
Rain
Rain
0.6
0.4
0.2
Cloud

We can now predict
tomorrow’s weather using
these probabilities and
applying them to today’s
weather.
If it is raining today, there is
a 60% chance of rain
tomorrow and only a 20%
chance of sun
0.3
0.4
0.3
0.1
0.3
0.6
Sun

Distribution Vectors



The number of units in each state depends on
both the transition probability and the number in
each state initially.
For example, on the stock market the number of
shares an investor owns in four different
companies may change with time
However, the total number he owns in each one
will depend how many of each he begins with.
Distribution Vectors: Shares

The Distribution after n
time steps can be
obtained as: vPn
200
175
500
=
50
170
0.2
0.7
0.1
0
0.4
0.2
0.2
0.2
0.1
0.3
0.2
0.4
0.2
0.1
0.4
0.3
330
175
250