Transcript Internet Media Delivery Systems

Internet Media Delivery Systems:
Current Design, Open Issues,
Research Problems.
G. S. Greenbaum, Ph.D
RealNetworks – Codec Group
Stanford University
Feb 13, 2001
Outline

Introduction to Internet-based Media
Distribution Systems
– Components

A Real Example of a Media System:
RealSystem iQ
– RealProducer 8.5
– RealSystem Sever 8 and Proxy 8 with Neuralcast
Technologies
– RealPlayer 8

Open Problems
– source/channel coding
– server storage/complexity tradeoffs
– network traffic optimization problems
Introduction:: Ambitions of a Media
Delivery System


The Overall Goal is to Maximize the End-User
Subjective Experience to Streaming
Multimedia
A Successful Internet Media Delivery System
Must Provide:
– Tools for the easy re-purposing of new and extant
content distribution over digital networks
– Robust, Distributed, Scalable, Secure, and
Flexible Architecture for content distribution
– Ubiquitous client distribution supporting diverse
data types
Introduction:: Components
Capture Encoding Serving Internet distribution Playback
Real
Producer
Real
Server
SDK
Plug-ins
SDK
Plug-ins
IP network
Real
Player
SDK
Plug-ins
Real
Proxy
Plug-ins
Introduction:: Components

Content Creation: Digitization and Encoding
– Most general purpose content starts life in analog
form which needs conversion to digital form
– Rich multimedia data types: audio, video, image,
VRML, etc.
• Lossy and lossless modes of compression
• Download vs Broadcast (streamed) modes
– Encoding tools typically have little knowledge of the
distribution network or computational resources of
the client playback device
– Metadata for efficient search and retrieval
– Rights of content provider must be insured
Introduction:: Components

Transmission and Distribution
– Live and pre-recorded Broadcast and VOD modes
of operation
– Redundant, Intelligent, and Reliable architecture
needed
• Redundant:: Zero points of failure at each critical node of
the distribution chain
• Intelligent: Load balancing (dynamic capacity allocation)
for efficient distribution
• Reliable: Stream distributions that traverse public and
private network segments; fortified against loss and
congested
– Architecture usually contains splitters/repeaters
and caches for efficient usage of network/gateway
capacity working in both a push and pull mode
Introduction:: Components

Client-side Player residing on different
appliances and diverse networks
– DSL, Cable Modem, Intranets/firewalls,
POTS, ISDN, etc.
– PC’s, cellular devices, PDA’s, set-top
boxes, digital VCRs, etc.
RealSystem iQ:: RealProducer 8.5


State of the Art Narrowband and Broadband
Audio and Video Codecs: RealVideo 8 and
RealAudio 8
Pre-processing:
–
–
–
–

Noise removal
Resampling
Inverse-Telecine
Deinterlacing
Rate control:
– Two-pass, VBR

Channel adaptation:
– SureStream, forward unequal ECC, data interleaving
Broadcast
SLTA library
File System
Producer
File Format
“Storage”
Live Text
Remote Broadcast Library
Internet/Intranet
Live Pix
RealSystem iQ:: Server 8

Adaptive serving:
–
–
–
–

Bandwidth detection
Stream switching
Thinning
multicast
Designed for distributed
delivery networks in mind:
– proxy plug-ins
– edge-server plug-ins
– distributed licensing

Features:
– Ad Insertion, Authentication extensions, remote
administration, firewall proxy, etc. (see
www.realnetworks.com for more details)
RealSystem iQ:: Server 8
Broadcast
Applications
Broadcast
Plug-in
Packet Sink
Plug-in
File Format
Plug-in
“Storage”
Admin, monitoring
Applications
File System
Plug-in
Generic Plug-ins
(logging, monitoring, etc.)
Server Core
Network Services
Server Registry
Allowance
Plug-in
Internet/Intranet
Server Clustering
Originate & Deliver from Any Where
An architecture free from ‘origins’ and ‘edges’
Distributed Architectures
RealServer
(Extra-Net)
Storage Array
RealServer
Cache
Router
Public
Internet
(Terestrial)
Router
Storage Array
ENTERPRISE
RealServer
RealProxy
Firew all Router
BROADCASTER: NON
SUBSCRIBER BASED
CONTENT DELIVERY
NETWORK POP
NON SUBSCRIBER BASED
BROADCAST NETWORK
SUBSCRIBER BASED
RealServer
Router
SWITCH
Storage Array
Storage Array
RealServer
SUBSCRIBER ACCESS PROVIDER
- ISP
Storage Array
Router
Cache
RealServer
CLEC/TELCO
RealProxy
SWITCH
Router
CABLE
Storage
RealServer
Router
CLEC/TELCO
RealSystem iQ:: Server 8
Neuralcast Technologies

NeuralCast Communications Protocol
– The ability for a network of RealSystem Servers to become selfaware by exchanging information and making decisions. Current
abilities include capacity sharing and capacity fail over in the event
of a network or equipment issue.

NeuralCast Live Distribution
– The intelligent and reliable delivery of broadcasts through a network
or RealSystem Servers. Current abilities include multi-protocol
transmission between servers, error correcting methods for
streams, terrestrial and satellite multicast support.

NeuralCast Live Redundancy
– From encoder to server or from server to server, the ability to send
redundant streams, providing a fail-over feed in the event of a
network or equipment outage.
RealSystem iQ:: Server 8
Neuralcast Technologies

NeuralCast Live Distribution
– Push and Pull models
– Re-Transmission requests made at receiver not
originating transmitter
– FEC configurable
– Multiple Transport Support allows the same
broadcast to be transmitted via conventional
unicast UPD or TCP across non-multicast enabled
segments
Encoder/
SLTA
live.rm
RealServer
transmitter
Unidirectional channel
(UDP/multicast)
Unidirectional channel
(UDP/unicast)
RealServer
receiver
RealServer
receiver
RealServer
receiver
RealSystem iQ:: Server 8
Neuralcast Technologies
 NeuralCast Live Redundancy
– Stateless protocol decouples live broadcast
packet contents from the transmitting network
Encoder/
SLTA
– Live broadcast streams that are sent
redundantly over different networks may be reconverged at the Server receiver
RealSystem iQ::
Live Webcast Deployment
Server Splitter Relays
RealSystem iQ:: Real Proxy 8

Access Provider Deployment Goals
– Optimize gateway bandwidth for streaming media
• Eliminate redundant requests for media from subscribers
for content that is back hauled from the public Internet
– Improve playback quality experience by migrating
content closer to the viewing subscriber

Deployed Technologies
– RealSystem Proxy near Telco/Cable cross
connects or at data center
RealSystem iQ::
Real Proxy 8 Deployment
Traffic Management
Cache Acquisition (Cold)
RealSystem iQ::
Real Proxy 8 Deployment
Cache Playback (Hot)
Live Splitting
RealSystem iQ:: RealPlayer 8

Post-Filters
–
–
–
–
–
–

Sharpening
color correction
graphic equalizer
Rescaling
Bandwidth Simulator
Reverb
Network Aware
– Monitors Network connection
– Auto-update enabled to obtain latest player components
– Auto-configures for best transport method

Features
– RV8, RA8, Visualizations, Radio Tuner, Media Search,
localizations for different languages, etc. see
www.real.com for a complete list.
RealSystem iQ::
Real Client Architecture
RealPlayer
RealJukebox
Audio Services
Video Surface
Rendering
Plug-in
Netscape Plug-in
Client Core
ActiveX
Network Services
Client Registry
Generic Plug-ins
File Format
Plug-in
Internet/Intranet
“Storage”
File System Plug-in
Open Problems...
Overall minimum-distortion optimization
Source/Channel coding Internet traffic problems
Real
Producer
Real
Server
IP network
Real
Player
Real
Proxy
Server storage vs.
complexity tradeoff
Traffic optimization in
dedicated delivery network
Source Coding Issues:
The Challange
Video: no
adequate
distortion
measure
 Human retina an ultimate
120:1 lossy
compressor

Human eye has around 6M cones + 120M rods;
but only around 1M ganglion fibers in the
optical nerve.
Source Coding Issues:
The Challange

Some other facts from physiology:
– visual recognition requires power of ~30% of
the cortex
– hearing -- uses just 3% of the cortex

Ultimate video decoder = 1/3 of a human
(adequately educated and trained)
Source Coding Issues

Constraints:
– Bounded delay (1-10 s and/or MB constraint)
– Targeting many discrete, fixed bit rate connections
– Random-access is necessary
• Seek/rewind on pre-recorded media
• Ability to join live stream
– Packetization: Each packet must contain sufficient
information to be decoded.
– Low complexity decoding
– Real-time encoding
– Need to assume inherently lossy network
transport
Source Coding Issues

Bounded Delay
– Not as much an issue for Downloadable Media
– Leveraged to solve inhomogeneity in content bit
allocation (VBR), network jitter, random access,
re-transmission requests, etc.
– Imposes constraints on look-ahead dependencies
– Imposes constraints on interleaving techniques for
error mitigation
– Cannot replicated the “remote-control” paradigm
of broadcast television.
Source Coding Issues

Bandwidth Scalability
– Layered Scalability, e.g. FGS, annex O.
• Introduces additional delay
• Hit in coding efficiency
– Embedded techniques, e.g. FGS, wavelets
• Introduces additional delay
• Possible hit in coding efficiency
– Multi-Encoding e.g. SureStream
• Increased file size and possible delivery costs
• Optimal coding efficiency for targeted rate
–?
Source Coding Issues

How to facilitate random access
decoding?
– independent encoding
– pre-encoded transitions
– pre-encoded correlated fragments
combined with Slepian-Wolf encoding
– use of multiple-description codes
– something else???
?
Source Coding Issues

Packetization
– Not an issue for downloadable media
– Imposes constraints on dependencies for
streaming content leading to a decrease in coding
efficiency
– Due to IP and packet reconstruction overhead,
packet size should be nearly as large as the
packet fragmentation limit imposed by routers,
1500 bytes. => increased latency of stream

Complexity Scalability
– How does the algorithm scale on limited resource
devices for the encoder and decoder?
• Scalable Post-filters
• Limited decode: B-frames dropped or I-frame only
• Short-cuts: limited ME/MC
Channel Coding Issues






IP have been designed to relieve engineers from
thinking about physical connection and channel coding
problems
Ironically enough, with streaming media we now have to
go back to basics…
UDP is mainly used=> packet erasures
– ARQ and ECC
– Client side mitigation techniques
Ideally, we want the server to do adaptive channel
coding/ transmission. Cost is reduction in server load.
How much of this work can be done ahead of the time
by encoder?
Is there a way to store all necessary information in a
compact form? Is SureStream the ideal?
Source/Channel Coding Issues
 Separation
Theorem may not hold:
– packets have finite lengths
– packets can be sent via multiuser channels
 Should
we use joint source/channel
coding?
– How to combine distortion due to lossy
compression and one due to channel errors
– can we design a universal lossy joint
source/channel code?
Source/Channel coding problems

Another way to deal with IP packets is to use
multiple-description coding:

Ozarow (1980) The achievable set of quantuples
(R1,R2 ,d1,d2,d3) is given by:
d1  exp  2 R1 
d 2  exp  2 R2 
d3 
1

exp  2  R1  R2  
1  d1 1  d2  
d1d 2  exp  2  R1  R2  

2
Summary



Several embodiments of Media Delivery
Systems exist today, but the technology is still
in its toddler years.
Many more issues, problems, and questions
than answers.
Plenty of opportunities for grad students,
post-docs, and professors! (and job security
for those outside of academia)