Transcript Document

Streaming Video Over the Internet
Andreas Panteli
03/05/2012
ECE 654 Advanced Computer Networks
1
Streaming Video Over The Internet
03 May 2012
Introduction

Video principles
 When an image appears on the retina it is retained for some


millisecond before decaying
If a sequence of images is drawn line by line at 50 images/sec the
eye does not notice that it is looking at discrete images
Streaming Video
 Video content need not be downloaded in full
 It is being played out while parts of the content are being received
and decoded


Real-time multimedia have timing constraints
 Audio and video data must be played out continuously
 Bandwidth, delay and loss requirements
No QoS guarantees to streaming video, from the current
best-effort Internet!
2
Streaming Video Over The Internet
03 May 2012
An Architecture for Video Streaming
Streaming Server
Client/Receiver
Video
Decoder
Media
Synchronization
Audio
Decoder
Storage Device
Raw Video
Raw Audio
Video
Compression
Audio
Compression
Compressed
Video
Compressed
Audio
Application-layer
QoS Control
Transport
Protocol
Application-layer
QoS Control
Transport
Protocol
Internet
(Continuous media distribution services)
3
Streaming Video Over The Internet
03 May 2012
Key Areas

Video Compression

Application-Layer QoS Control
 Continuous Media Distribution Services

Media Synchronization Mechanisms

Protocols for Streaming Media
4
Streaming Video Over The Internet
03 May 2012
Key Areas

Video Compression
 JPEG standard
 MPEG standard
 Application-Layer QoS Control
 Continuous Media Distribution Services
 Media Synchronization Mechanisms
 Protocols for Streaming Media
5
Streaming Video Over The Internet
03 May 2012
Digital Video





Digital video is represented as a sequent of frames, each
consisting of a rectangular grid of picture elements (pixels)
For color video, 8 bits for each of the RGB colors are being
used
 24 bits/pixel => 16 million colors
To produce smooth motion, digital video, we must display at
least 25 frames/sec, but to avoid flickering we need to
repaint each frame on the screen twice
Smoothness and flickering affects the bandwidth
requirements
 Consider the lower video configuration for computer monitor
1024x768 with 24 bits/pixel and 25 frames/sec
 Need to be fed at 472 Mbps
We need video compression!
6
Streaming Video Over The Internet
03 May 2012
Video Compression


Two algorithms are needed
 Encoding and Decoding
Asymmetries of the algorithms
 Acceptable for the encoding algorithm to be slow and require

7
expensive HW, if the decoding algorithm is fast
 Only for video on demand
 For real-time multimedia like video conferencing, slow encoding
is unacceptable
Acceptable to have a slightly different video signal than the original,
after encoding and decoding (Lossy system)
 Lossy systems are important
 Accepting a small amount of information loss can give a huge
payoff in terms of the compression ratio possible
Streaming Video Over The Internet
03 May 2012
The JPEG Standard

Compression algorithm for still images
 Could be applied to each image in succession to achieve video
compression
 Compute the luminance Y and the two chrominances I and Q



Y = 0.30R + 0.59G + 0.11B
I = 0.60R – 0.28G - 0.32B
Q = 0.21R – 0.52G + 0.31B
 Compress the two chrominances twice than the luminance

Lossy reduction, but the eye barely notices it since it responds to luminance
more than to chrominance
 JPEG often produces 20:1 compression or better
 Decoding a JPEG image requires running the algorithm backwards
8
Streaming Video Over The Internet
03 May 2012
The MPEG Standard (1/2)



Main algorithms for video compression and international
standards since 1993
Can compress both audio and video
Three parts in MPEG-1
 Audio
 Video
 System (integrates the other two)
9
Streaming Video Over The Internet
03 May 2012
The MPEG Standard (2/2)


Two kinds of redundancies in videos
 Spatial redundancy
 Coding each frame separately with JPEG
 Temporal redundancy
 Consecutive frames are often almost identical
MPEG-1 output consists of four kinds of frames
 I (Intracoded) frames: Self-contained JPEG-encoded still pictures
 P (Predictive) frames: Block-by-Block difference with the last frame
 B (Bidirectional) frames: Differences between the last and the next
frame
D (DC-coded) frames: Block averages used for fast-forward



MPEG-1 can achieve a compression factor of 40
Decoding has about the same complexity as encoding
10
Streaming Video Over The Internet
03 May 2012
Key Areas


Video Compression



Continuous Media Distribution Services
Application-Layer QoS Control
 Congestion control
 Error control
Media Synchronization Mechanisms
Protocols for Streaming Media
11
Streaming Video Over The Internet
03 May 2012
Application-Layer QoS Control
 Avoids congestion and maximizes video quality in
the presence to packet loss
Congestion Control
 Rate Control
 Source-Based Rate Control
 Receiver-Based Rate

12
Control
Hybrid-Rate Control

Rate Shaping
 Frame-dropping filter
 Layer-dropping filter
 Frequency filter
 Re quantization filter
Error Control
 FEC
 Delay-Constrained
Retransmission
 Error-Resilient Encoding
 Error-Concealment
Streaming Video Over The Internet
03 May 2012
Rate Control (1/2)


Technique to determine the sending rate of video traffic
based on the estimated available bandwidth of the network
Source-based rate control
 The sender is responsible for adapting the video transmission rate


13
based on feedback from the receivers
Probe-based approach
 Sending rate adjustment to maintain the packet loss ratio below a
certain threshold
Model-based approach
Streaming Video Over The Internet
03 May 2012
Rate Control (2/2)

Receiver-based rate control
 The receivers regulate the receiving rate of video streams by



adding/dropping channels
Probe-based approach
 When no congestion is detected, join a channel
 => increase of it’s receiving rate
 When congestion is detected, drop a channel
 => reduction of it’s receiving rate
Model-based approach
 Explicit estimation for the available network bandwidth
 Also based on Equation (1)
Hybrid rate Control
 Combination of the above mentioned techniques
14
Streaming Video Over The Internet
03 May 2012
Rate Shaping



Matches the rate of a pre-compressed video bit stream to
the target rate constraint
A rate shaper (or filter) is required for source based rate
control
Types of filters




15
Codec filter
 Performs transcoding between different
compression schemes
Frame-dropping filter
 I, P ,B , D frames in MPEG
Frequency filter
 Operates in the frequency domain (DCT
coefficients)
Re-quantization filter
 Re-quantizes the DCT coefficients with a
larger quantization step
Streaming Video Over The Internet
03 May 2012
Error Control (1/2)

Forward Error Correction (FEC)
 Add redundant information so that original message can be



16
reconstructed in the presence of packet loss
Channel coding
 Stream is chopped into segments
 Each segment is packetized into k packets
 A block code generates an n-packet block, where n>k
 User only needs to receive any k packets out of n of the block
Source coding
 N-th group of blocks contains redundant information of (n-1)th
group of blocks
Joint source/channel coding
 Optimal rate allocation between Channel and Source coding
Streaming Video Over The Internet
03 May 2012
Error Control (2/2)
 Delay-constraint Retransmission
When the receiver detects the loss of packet N
if (Tc + RTT + Ds < Td(N))
send the request for packet N to the sender
Where
Tc
current time
RTT estimated round-trip time
Ds
a slack term (e.g receiver’s decoding delay)
Td(N) time when packet N is scheduled for display
 Error-resilient Encoding




Enhances robustness of compressed video to packet loss
A video is compressed into multiple streams (descriptions)
Each description provides acceptable visual quality
Combined descriptions provide a better visual quality
 Error Concealment
 Performed by the receiver when packet loss has already occurred
 Spatial interpolation
 Missing pixel values are reconstructed using neighboring pixels
 Temporal interpolation
 Loss data is reconstructed using data from the previous frame
17
Streaming Video Over The Internet
03 May 2012
Key Areas
 Video Compression
 Application-Layer QoS Control
 Continuous Media Distribution Services
 Network Filtering
 Application-level Multicast
 Content Replication
 Media Synchronization Mechanisms
 Protocols for Streaming Media
18
Streaming Video Over The Internet
03 May 2012
Network Filtering

Adequate support from the network is critical in order to
provide quality multimedia presentations

Routers have no knowledge of the
format of the media streams
 Randomly discard packets
Filters receive the client’s requests and
adapt the stream sent by the server
accordingly
Typically, frame-dropping filters are used
 Drop packets in a way that gracefully



19
R: Router
degrades the stream’s quality
Increase bandwidth efficiency by discarding
late frames
Streaming Video Over The Internet
03 May 2012
Application-level Multicast
Unicast
Multicast
 Internet’s original design fails to effectively support streaming-media

multicast
IP multicast has barriers
 Scalability, network management, support for higher layer functionality (e.g. error,
flow and congestion control)
 Application-level multicast to the rescue
 Builds a multicast service on top of the Internet
 Media bridge
 Routing in the application layer
 Each Media Bridge is interconnected with one or more neighboring Media
20
Bridges through explicit configuration
Collectively employ a distributed application-level multicast routing algorthm
Streaming Video Over The Internet
03 May 2012
Content Replication
 Advantages
 Increased network’s bandwidth efficiency
 Reduced load on streaming servers
 Reduced latency for clients
 Increased availability
Mirroring
 Scatter copies of the
original multimedia files in
different locations around
the Internet
 Clients retrieve data from
nearest duplicate server
21
Caching




Exploits temporal locality
Cache retrieves data from
server
Clients retrieve data from
cache if available
Cache sharing and cache
hierarchies
Streaming Video Over The Internet
03 May 2012
Key Areas





Video Compression
Application-Layer QoS Control
Continuous Media Distribution Services
Media Synchronization Mechanisms



Intra-stream Synchronization
Inter-stream Synchronization
Inter-object Synchronization
Protocols for Streaming Media
22
Streaming Video Over The Internet
03 May 2012
Media Synchronization



Multimedia applications

Various integrated media streams that must be presented in a
synchronized fashion
Media streams may lose synchronization after moving from the
server to the client
Three layers of synchronization



23
Intra-stream synchronization
 Maintain continuity of logical data units
Inter-stream synchronization
 Maintain temporal relationships among different continuous media
 e.g. movements of the lips of a speaker do not correspond to the
presented audio
Inter-object synchronization
 Relation with time-independent data, such as text, still image and
streams
 e.g. during a presentation, audio is commenting one slide while
another slide is presented
Streaming Video Over The Internet
03 May 2012
Media Synchronization Mechanisms



Delay in the network is unpredictable due to the best-effort
nature of the Internet
Axes-based specifications, or time-stamping
 At the source, a stream is time-stamped to keep temporal
information within the stream and with respect to other streams
Mechanisms implemented on the end systems
 Preventive mechanisms
 Minimize latencies and jitters
 i.e. disk-reading scheduling algorithms, network transport

24
protocols, operating systems and synchronization schedulers
Corrective mechanisms
 Recover synchronization in the presence of synchronization
errors
 i.e. Stream synchronization protocol (SSP)
 Concept of ”intentional delay”
Streaming Video Over The Internet
03 May 2012
Key Areas

Video Compression

Application-Layer QoS Control
 Continuous Media Distribution Services

Media Synchronization Mechanisms

Protocols for Streaming Media
 Transport Protocols
 Session Control Protocols
25
Streaming Video Over The Internet
03 May 2012
Protocols for Streaming Video

Data Plane
 Compressed video/audio is



packetized at the real-time
transport protocol (RTP)
layer
RTP packetized streams are
then passed to the
UDP/TCP layer and IP layer
IP packets are transported
over the Internet
Control Plane
 Real-time control protocol (RTCP) and real-time streaming protocol

26
(RTSP) packets are multiplexed at the UDP/TCP layer
IP packets are transported over the Internet
Streaming Video Over The Internet
03 May 2012
Transport Protocols
 UDP and TCP provide basic transport functions
 UDP and TCP
 Multiplexing, Error control (e.g. checksum)
 TCP only:
 Retransmission, Congestion control, Flow control
 UDP is typically employed as the transport protocol for video streams
 No packet delivery guarrantees
 Receiver needs to rely in upper layer to detect packet loss
 RTP and RTCP run on top of UDP/TCP
 RTP : does not guarantee
 RTCP : designed to work in
QoS or reliable delivery
 Time stamping
 Sequence numbering
 Payload type identification
 Source identification
27
conjunction with RTP
 QoS feedback
 Participant identification
 Control packets scaling
 Inter-media synchronization
 Minimal session control
information
Streaming Video Over The Internet
03 May 2012
Session Control Protocols

RTSP (Real-Time Streaming Protocol)
 Support VCR-like control operations
 (stop, pause/resume, fast forward, fast backward)
 Choosing delivery channel (UDT, multicast UDT, TCP)
 Establish and control streams of continuous audio and video media
between the media servers and the clients
 Media retrieval: ask server to setup a session to send the
requested media data
 Adding media to an existing session: server or client can notify
each other about any additional media becoming available to the
established session

SIP (Session Initiation Protocol)
 Similar to RTSP
 Support user mobility by proxying and redirecting request
28
Streaming Video Over The Internet
03 May 2012
References




A. S. Tanenbaum, Computer Networks, 4th ed. New Jersey:
Prentice Hall, 2002.
Dapeng Wu, Hou, Y.T., Wenwu Zhu, Ya-Qin Zhang, Peha,
J.M., "Streaming video over the Internet: approaches and
directions“, Circuits and Systems for Video Technology,
IEEE Transactions on , vol.11, no.3, pp.282-300, Mar 2001
Dapeng Wu, Yiwei Thoms Hou, Ya-Qin Zhang, "Transporting
real-time video over the Internet: challenges and
approaches," Proceedings of the IEEE , vol.88, no.12,
pp.1855-1877, Dec 2000
Dapeng Wu and Y. Thomas Hou and Jason Yao and Y.
Thomas and Hou Jason Yao and H. Jonathan Chao, “Realtime Video over the Internet: A Big Picture”, IEEE
NetWorld+Interop, 2000
29
Streaming Video Over The Internet
03 May 2012
Questions?
Thank you!
30
Streaming Video Over The Internet
03 May 2012