Transcript Lecture 5 - Richard Clegg

Lecture 5
 This lecture is about:
 Introduction to Queuing Theory
 Queuing Theory Notation
 Bertsekas/Gallager: Section 3.3
 Kleinrock (Book I)
 Basics of Markov Chains
 Bertsekas/Gallager: Appendix A
 Kleinrock (Book I)
 Markov Chains by J. R. Norris
Queuing Theory
 Queuing Theory deals with systems of the
following type:
Server Process(es)
Input
Process
Output
 Typically we are interested in how much
queuing occurs or in the delays at the servers.
Queuing Theory Notation
 A standard notation is used in queuing theory to denote
the type of system we are dealing with.
 Typical examples are:




M/M/1
M/G/1
D/G/n
E/G/
Poisson Input/Poisson Server/1 Server
Poisson Input/General Server/1 Server
Deterministic Input/General Server/n Servers
Erlangian Input/General Server/Inf. Servers
 The first letter indicates the input process, the second
letter is the server process and the number is the
number of servers.
 (M = Memoryless = Poisson)
The M/M/1 Queue
 The simplest queue is the M/M/1 queue.
 Recall that a Poisson process has the following
characteristics:

P{ A(t   )  A(t )  n}  e
( ) n
n!
 Where A(t) is the number of events (arrivals) up to time
t.
 Let us assume that the arrival process is a Poisson with
mean  and the service process is a Poisson with a
mean 
Poisson Processes (a refresher)
 Interarrival times are i.i.d. and exponentially
distributed with parameter .
 tn is the time of packet n and n= tn+1 - tn then:
 s
P{ n  s}  1e
s0
 For every t  0 and   0:
P{ A(t   )  A(t )  0}  1    o( )
P{ A(t   )  A(t )  1}    o( )
P{ A(t   )  A(t )  2}  o( )
Poisson Processes (a refresher)
 If two or more Poisson processes (A1,A2...Ak) with
different means(1, 2... k) are merged then the
resultant process has a mean  given by:
  i 1 i
k
 If a Poisson process is split into two (or more) by
independently assigning arrivals to streams then the
resultant processes are both Poisson.
 Because of the memoryless property of the Poisson
process, an ideal tool for investigating this type of
system is the Markov chain.
On the Buses (a paradoxical property
of Poisson Processes)
 You are waiting for a bus. The timetable says that buses are
every 30 minutes. (But who believes bus timetables?)
 As a mathematician, you have observed that, in fact, the buses
are a Poisson process with a mean arrival rate such that the
expectation time between buses is 30 minutes.
 You arrived at a random time at the bus stop. What is your
expected wait for a bus? What is the expected time since the last
bus?
 15 minutes. After all, they are, on average, 30 minutes apart.
 30 minutes. As we have said, a Poisson Process is
memoryless so logically, the expected waiting time must be
the same whether we arrive just after a previous bus or a full
hour since the previous bus.
Introduction to Markov Chains
 Some process (or time series) {Xn| n= 0,1,2,...} takes
values in nonnegative integers.
 The process is a Markov chain if, whenever it is in state
i, the probability of being in state j next is pij
pij  P{ X n1  j | X n  i, X n1  in1 ,...X 0  i0 }
 P{ X n1  j | X n  i}
 This is, of course, another way of saying that a Markov
Chain is memoryless.
 pij are the transition probabilities.
Visualising Markov Chains (the
confused hippy hitcher example)
A
1/3
1/4
3/4
1/2
A hitchhiking hippy begins at A
town. For some reason he has
2/3
poor short-term memory and
travels at random according
B
1/2
to the probabilities shown. What
is the chance he is back at A after 2
days? What about after 3 days? Where is he likely to end up?
C
The Hippy Hitcher (continued)
 After 1 day he will be in B town with probability 3/4 or
C town with probability 1/4
 The probability of returning to A via B after 1 day is
3/12 and via C is 2/12 total 5/12
 We can perform similar
A
calculations for 3 or 4 days
but it will quickly
1/3
1/4
become tricky and
1/2
3/4
finding which city he
2/3
is most likely to end up
B
C
in is impossible.
1/2
Transition Matrix
 Instead we can represent the transitions as a
matrix
Prob of going to B from A
 0 3 / 4 1/ 4
P  1 / 3 0 2 / 3
1 / 2 1 / 2
0 
A
1/3
1/4
1/2
3/4
Prob of going to A from C
B
2/3
1/2
C
Markov Chain Transition Basics
 pij are the transition probabilities of a chain.
They have the following properties:

pij  0,  pij  1,
i  0,1....
j 0
 The corresponding probability matrix is:
 p00
p
P   10
 

 pn 0
p01
p02 
p11

p12

pn1
pn 2 


p0 n 
p1n 
 

pnn 
Transition Matrix
 Define n as a distribution vector representing
the probabilities of each state at time step n.
 We can now define 1 step in our chain as:
n1  n P
 And clearly, by iterating this, after m steps we
have:
nm  n P
m
The Return of the Hippy Hitcher
1 
What does this imply for our hippy?
0  0
We know the initial state vector:
0
So we can calculate n with a little drudge work.
(If you get bored raising P to the power n then
you can use a computer)
 But which city is the hippy likely to end up in?
 We want to know   lim n




n 
Invariant (or equilibrium)
probabilities)
  lim n
n 
 Assuming the limit exists, the distribution vector  is
known as the invariant or equilibrium probabilities.
 We might think of them as being the proportion of the
time that the system spends in each state or
alternatively, as the probability of finding the system in
a given state at a particular time.
 They can be found by finding a distribution which
solves the equation:
  P
 We will formalise these ideas in a subsequent lecture.
Some Notation for Markov Chains

Formally, a process Xn is Markov chain with
initial distribution  and transition matrix P if:
1.
2.



P{X0=i} = i (where i is the ith element of )
P{Xn+1=j| Xn=i, Xn-1=xn-1,...X0=x0}= P{Xn+1=j| Xn=i }=pij
For short we say Xn is Markov (,P)
We now introduce the notation for an n step
( n)
p
transition:
ij  P{X mn  j | X m  i}

And note in passing that:
pij( n m )   pik( n ) pkj( m )
k 0
This is the Chapman-Kolmogorov equation