Transcript Introduction to Multimedia Systems

Multimedia Over IP Networks -- II
Hao Jiang
Computer Science Department
Boston College
Nov. 8, 2007
CS335 Principles of Multimedia Systems
Real-time Transport Protocol
 Real-time transport protocol (RTP) is an Internetstandard protocol for transmitting real-time
multimedia over IP network.
 RTP is (often) built on top of UDP.
UDP
header
RTP
header
Multimedia Payload
 A RTP packet is embedded in a UDP packet and
provides additional information to support real-time
multimedia communications
– Timestamp.
– Sequence number and source id.
CS335 Principles of Multimedia Systems
The RTP Packet Header
Timestamp is a 32bit integer placed on the RTP packet.
It is incremented based on the local system clock. For
video it is usually 90kHz.
Sequence number is a 16bit integer incremented by 1
for each generated RTP packet.
PT: payload type. For example, 7 is for LPC audio, and 34 is for H263 video.
CS335 Principles of Multimedia Systems
Example: Motion JPEG Payload Packet
 Timestamp is incremented at 90KHz. The marker M
bit is set at the last packet for each frame.
Main JPEG header following the RTP header
There are other headers to provide information
like quantization table, etc.
CS335 Principles of Multimedia Systems
RTCP
 Real Time Control Protocol (RTCP) packets can be
generated by both a sender and a receiver.
V
P
IC
PT
Length
Format Specific Information
RCTP Packet Format
 RTCP uses a different UDP port from RTP.
 RTCP packets can be used to generate reports for
network statistics such as round trip time, jitter and to
synchronize different information sources.
CS335 Principles of Multimedia Systems
RTCP Receiver Report
 Payload type = 201.
 RTCP RR packet reports include
–
–
–
–
Loss fraction: ratio of lost packets # / expect packets #.
Cumulative number of packets lost.
Highest sequence number received.
Inter-arrival jitter
D(i) = (R(i) – R(i-1)) – (S(i) - S(i-1))
J(i+1) = J(i) * 15/16 + | D(i) | * 1/16
– Last send report timestamp (LSR) and delay since last send
report (DLSR) [NTP time]
round trip time = current timestamp - LSR - DLSR
sender
LSR
receiver
Current Timestamp
DLSR
CS335 Principles of Multimedia Systems
RTCP Sender Report
 Payload type = 200.
 RTCP SR packet contains
– NTP timestamp.
– RTP timestamp.
– Reporter SSRC and others.
 The most important usage of RTCP SR packet is to
synchronize different media streams.
CS335 Principles of Multimedia Systems
RTP Example Application
encoding buffer
Audio
Codec,
Packing,
Depacking
capturing
Playing back
Playback buffer
Internet
RTP Packet buffer
Client n
Client k
Client i
Audio Conferencing based on RTP
CS335 Principles of Multimedia Systems
RTP Media Sender
Buffer until send time comes
…
Compressed
media frames
with relative time
stamps
Adding sequence
number and other
fields to make RTP
packets
…
fragmentation
Send to UDP
module
System clock
…
…
Media buffer
CS335 Principles of Multimedia Systems
RTP Media Receiver
RTP packet with timestamp t > t1and t < t2
Packet buffer
(Packet with timestamp t1)
Play back
buffer
(Packet with timestamp t2)
Scheduled
to play out
decompress
We usually need another buffer for de-fragmentation.
CS335 Principles of Multimedia Systems
Timing
Sender
clock
20ms audio
20ms audio
20ms audio
encoding
making
a packet
Across
The network
20ms audio
Receiver
clock
What happens if receiver clock and sender clock
has different
frequency?
CS335 Principles
of Multimedia Systems
Methods to Compensate for Clock Skew
 A method is to directly compute the skew:
sender timestamp n1 = (t1 – tx)/dtx
packet arrive timestamp n2 = (t2 – ty)/dty
Assume we have 2 packets a and b, then
n1(a) – n1(b) = [t1(a) – t1(b)]/dtx
n2(a) – n2(b) = [t2(a) – t2(b)]/dty
=>
(n2(a) – n2(b)) / (n1(a) – n1(b))
= dtx/dty + D(a,b)/(t1(a) – t1(b))*dtx/dty
If (t1(a) – t1(b)) is constant, we can do a long term average to
remove the noise term.
CS335 Principles of Multimedia Systems
Compensation for Jitter
 The leaky bucket model:
Similar to a leaky bucket,
we can use a buffer to shape
the network traffic to generate
a smooth playback.
Buffer
…
CS335 Principles of Multimedia Systems
Compensation for Jitter
# of
Accumulated
Packets
t1
N/(t2-t1)
CS335 Principles of Multimedia Systems
t2
Compensation for Jitter
# of
Accumulated
Packets
Production Curve
Consumption Curve
t1
N/(t2-t1)
CS335 Principles of Multimedia Systems
t2
Compensation for Jitter
# of
Accumulated
Packets
Production Curve
Consumption Curve
t1
N/(t2-t1)
CS335 Principles of Multimedia Systems
t2
Compensation for Jitter
# of
Accumulated
Packets
Production Curve
Consumption Curve
t1
t2
CS335 Principles of Multimedia Systems
Media Synchronization
 In RTP, different media streams (audio and video) are
sent separately.
 Media synchronization relies on the correspondence
of timestamp and NTP time on the local machine.
 All the information sources need to be synchronized
so that they have the same NTP time base.
 RCTP SR packet is used to align the timestamp with
the NTP time in media synchronization.
CS335 Principles of Multimedia Systems
Useful Links
 http://www.realnetworks.com/support/education/sdk.h
tml
 http://www.w3.org/AudioVideo/Activity
 http://www.networksorcery.com/enp/protocol/rtp.htm
CS335 Principles of Multimedia Systems