Transcript Issues in Wireless Multimedia

Challenges
in Wireless Multimedia
CSE Department Seminar Series
September 26, 2003
Borko Furht
OUTLINE




Scalable Video Source Coding
Channel Coding and Error Control
Power-Aware Coding and
Transmission Techniques
Networking Issues



Rate control
Multimedia Security
Application: Virtual Workplace
Mobile Internet Access
1200
1000
800
Internet
Subscribers
(millions)
Fixed
600
Mobile
400
200
Source: Ericsson
Year
6
2
0
0
5
2
0
0
4
2
0
0
3
2
0
0
2
2
0
0
1
0
0
2
2
0
0
0
0
Wireless Multimedia Architecture
Bandwidth Problem
“Bandwidth
is like money and sex only too much seems to be enough.”
Arnold Penzias,
former chief scientist of Bell Labs
Generations of WAN Air Interfaces
Based on Access Technologies

1G: FDMA (Frequency Division Multiple Access)


1980s - each caller has a dedicated frequency
channel (3 callers use 3 channels)
2G: TDMA (Time Division Multiple Access) and GSM
(Groupe Speciale Mobile)


1990s - callers timeshare a frequency channel (9
callers use 3 channels)
3G: CDMA (Code Division Multiple Access) and WCDMA
(Wide Code Division Multiple Access)
 1990s - callers use a shorter bandwidth
 2000s - “spread spectrum”. Each code is spread, randomly
broken down and mixed (14 callers use the full bandwidth of 1
channel)
Data Services
Data Transmission Speed - k bps
2,000
1G
3G
2G
Video Streaming Remote
Medical
Video
Service
Conference (Medical
(High quality) image)
384
Audio Streaming
144
Text Messaging
128
Still
Imaging
Video on
Demand:
Sports, News
Weather
Video
Conference
(Lower quality)
Image Mobile TV
Video Surveillance,
Video Mail, Travel
Voice
64
Electronic
Newspaper
Voice
Mail
32
Fax
JPEG
Still Photos
Electronic
Publishing
E-Mail
Karaoke
Mobile
Radio
9.6
Telephone
(Voice)
0
Data
Weather, Traffic, News,
Sports, Stock updates
Audio
Voice-driven Web Pages
Streaming Audio
E-Commerce
M-Commerce Applications




Transaction Management
Digital Content Delivery
Telemetry Services
Searching for Killer Applications!
Transaction Management

On-line shopping tailored to mobile
phones and PDAs





on-line catalogs
shopping carts
back office functions
Initiate and pay for purchases and
services
Micro-transactions - subway fees,
digital cash
Digital Content Delivery

Information browsing








weather
transit schedules
sport scores
ticket availability
market prices
Downloading entertainment products
Transferring software, high-resolution
images, and full-motion video
Innovative video applications
Telemetry Services




Wide range of new applications
Transmission of receipt of status,
sensing, and measurement information
Communication with various devices
from homes, offices, or in the field
Activation of remote recording devices
or service systems
AT&T Wireless
Welcome to mlife
Get the latest weather forecasts
Get the latest weather forecasts
Find breaking
news,
flight information,
entertainment..
Get the business and
Get the investments
business and news
investments news
Future of Wireless Technology

Mobile networks have already begun the
migration to IP-based networks


4G, New spectrum, and Emerging wireless
air interfaces (very high bandwidth 10
Mbps+)


IP as the routing protocol
It may entirely be IP-based and packet-switched
Increasing usage of wireless spectrum

On average, the number of channels has doubled
every 30 months since 1985 (Cooper’s law)
Wireless Multimedia
Challenges
•
Adaptive Decoding - Optimizing rich digital media
for mobile information devices with limited
processing power, limited battery life and varying
display sizes
•
Error Resilience - Delivering rich digital media over
wireless networks that have high error rates and low
and varying transmission speeds
•
Network Access - Delivering rich digital media
without adversely affecting the delivery of voice and
data services
•
Negotiable QoS for IP multimedia sessions as well
as for individual media components
Components of a
Wireless Video System
Input
Video
Transport + Network Layer
Video
Encoder
Packetizer
Channel
Encoder
Modulator
Wireless
Channel
Output
Video
Video
Decoder
Depacketizer
Channel
Decoder
Demodulator
Tradeoff: Throughput, Reliability, Delay
Source and Channel Coding
Trade-off

Classic goal of source coding


Classic goal of channel coding


Achieve the lowest possible distortion for a given
target bit rate
Deliver reliable information at a rate that is as close
as possible to the channel capacity
Shannon’s separation principle:


It is possible to independently consider source and
channel coding without loss in performance
The separation principle applies only to point-topoint communications and it is not valid for multiuser
or broadcast scenarios
Pragmatic Approach

Keep the source coder and channel coder
separate, but optimize their parameters jointly


Key problem in this optimization is the bit allocation
between the source and channel coder
Joint source-channel coding schemes



In the infancy today
Exploit the redundancy in the source signal for
channel decoding (Source-controlled channel
decoding)
Designing the source codec for a given channel
characteristic (Channel-optimized source coding)
Characteristics of
a Wireless Video System



The capacity of wireless channel is limited by
the available bandwidth of the radio spectrum
and various types of noise and interference
The wireless channel is the weakest link of
multimedia networks – mobility causes fading
and error bursts
Resulting transmission errors require error
control techniques (such as FEC - forward error
control and ARQ – automatic repeat request)
The Case for
Scalable Video Coding

In emerging wireless applications, multimedia
data will be streamed:



over various access networks (GPRS, UMTS,
WLANs, etc.)
to a variety of devices (PCs, TVs, PDAs, cellular
phones, etc.)
The transmission of multimedia data need to
cope with unpredictable bandwidth variations:


due to heterogeneous access technologies of
receivers (3G, 802.11a, etc.) or
due to dynamic changes of network conditions
(interference, etc.)
Scalable Video Coding
Techniques



Scalable video coding methods can adapt in real time to
the bandwidth variations over heterogeneous networks
and to the terminal capabilities while using the same
pre-encoded system.
Scalable video coding uses multiple bit streams –
layered video coding
For example, in a two-layer coding, the codec generates
two bit streams:



Base layer – the most vital video information
Enhancement layer – the residual information to enhance the
quality of the base layer image
This form of two-layer coding is known as SNR scalability
Scalability Techniques





Data partitioning
SNR scalability
Spatial scalability
Temporal scalability
Hybrid scalability
Data Partitioning


Data partitioning is used when two channels
are available for transmission (it is not true
scalable coding)
Divides the bitstream of a single layer into
two parts, or layers.
Base-layer
bitstream
Video in
Single layer
encoder
Data
Partitioner
Enhancementlayer bitstream
Output
bitstream
Multiplexer
Block Diagram
Two-Layer SNR Scalable Coder
Base layer
bitstream
Video in
Base layer
Encoder
(MPEG 1)
+
-
Base layer
Decoder
(MPEG 1)
Output
bitstream
Multiplexer
Enhancement layer
Encoder
(MPEG 2)
Enhancement
layer bitstream
Adaptive Video Coder
Based on 3D-DCT
Original video cube 8x8x8
3D Discrete Cosine Transform


Frames
0
1
8
8
2
8
8
3
8
8
4
8
8
5
8
8
6
8
8
7
8
8
8
8
7
8X8X8
video
cube
7
7
F (u, v , w)  C u C v  C w    f  x , y , z
x 0 y 0 z 0


cos 2 x  1 u  cos 2 y  1v cos 2 z  1 w 
16
16
16
Motion Analysis for Various Blocks

Partition image into NxN inspection areas

Examine each area for motion content based on
Normalized Pixel Difference (NPD) between frames
1 and 8


Three motion types defined:

No Motion

Low Motion, and

High Motion
3D-DCT block size adapts based on determined motion
content
Example of a Video
Hallway Clip

8 Frames of luminance
(Y) component


Inspection area size =>
16x16
Inspection areas used
to determine NPD
thresholds
Video Example, 4:2:0
Original
Cr=190
Cr=120
Cr=408
Architecture of 3D-DCT
Adaptive Encoder
Video
cube
Forward
3D-DCT
Quantizer
Huffman
Encoder
Compressed
video
Input
video
Selection
of cube size
Level of
motion
Video
cube
Motion
Analyzer
.
Huffman
Tables
.
Quantization
Tables
Selection
of Q tables
Figure 2. The architecture of the adaptive 3D-DCT encoder.
Example of a Scalable Coding
Adaptive 3D-DCT Coder
Original
Adaptive 3D-DCT Coder
Layer 1: Cr=164
(in vehicles, 144 Kbps)
Adaptive 3D-DCT Coder
Adding Enhancement Layer 2: Cr=96
(For pedestrians, 384 Kbps)
Adaptive 3D-DCT Coder
Adding Enhancement Layer 3: Cr=54
(for indoor use, 2 Mbps)
Channel Coding and Error Control
Effects of Transmission Errors

Error-free frame
Example 2: Corrupted group number
causing a GOB misplacement

Example 1: The extra insertion
bit causing the loss of the first
GOB

Example 3: Corruption of the group
quantizer parameter that resulted in
employing the wrong quantizer in
decoder
Channel Coding and Error Control




Trade-off between throughput, reliability, and
delay
Forward Error Correction (FEC)
Automatic Repeat Request (ARQ)
Error Resilience Techniques for Low Bit
Rate Video


Techniques that reduce the amount of
introduced errors for a given error event
(Resynchronization)
Techniques that limit interframe error
propagation
Recovery From Packet Loss
FEC scheme
• “Piggyback lower
quality stream”
• Send lower resolution
audio stream as the
redundant information
• For example, nominal
stream PCM at 64 kbps
and redundant stream
GSM at 13 kbps.
• Sender creates
packet by taking the
nth chunk from nominal
stream and appending
to it the (n-1)st chunk
from redundant
stream.
• Whenever there is non-consecutive loss, the
receiver can conceal the loss.
• Only two packets need to be received before
playback
• Can also append (n-1)st and (n-2)nd low-bit rate
chunk
Joint Source Coding and
Transmission Power Management


Goal: to limit the amount of distortion in the
received video sequence, while minimizing
transmission energy
Combines:



Error resilience and concealment techniques at
the source coding level, and
Transmission power management at the
physical layer
Optimization problem: Minimizing the
energy required to transmit video under
distortion and delay constraints
Joint Source Coding and
Transmission Power Management
Decoder
Concealment
Strategy
Channel State
Information
Control coding
parameters
Controller
Video in
Channel
Encoder
Video
Encoder
Control power
Modulator
Wireless
Channel
Video out
Video
Decoder
Channel
Decoder
Demodulator
Goal: to limit the amount of distortion in the received video sequence,
while minimizing transmission energy
Transmission Energy

Total energy to transmit all the packets
in a frame:
K
K
k
B k
Etot   E   P
k 1
k 1 R
k

The algorithm calculates the power
needed to achieve the desired
probability of loss
Controlling the Bit Rate




Most video codecs use variable-length coding
techniques
Most existing mobile radio systems transmit at a fixed
bit rate
Goal: Constant signaling rate leading to a different
constant bit rate for each modulation scheme
Rate Control Techniques - determine the sending rate of
video traffic based on the estimated bandwidth in the
network



Source-Based Rate Control
Receiver-Based Rate Control
Hybrid-Based Rate Control
Rate Shaping Techniques


Techniques that adapt the rate of precompressed video stream to a target rate
constraint
Rate shaper is an interface (or filter)
between the compression layer and the
network transport layer
Variable
rate
Constant
bit rate
Video in
Compression
Layer
Rate
Shaper
Network
Transport
Layer
Rate Shapers





Codec filters
Frame-dropping filters (dropping B,P, or I
frames)
Layer-dropping filters (in scalable video
coding schemes)
Frequency filters (discard DCT coefficients of
the highest frequency)
Requantization filters (reqauntizes the DCT
coefficients with a larger quantizers, resulting
in rate reduction)
Multimedia Content Security




Access control in applications such as video-ondemand and videoconferencing, so only selected
users can access the data
Established encryption algorithms (DES or AES)
are very complicated and involve large number of
computations.
Software implementations of these schemes are not
fast enough to process the large amount of
multimedia data
Hardware implementations require additional costs
to both data generation and receivers
General Architecture
Selective Encryption System
Example of Video Encryption
MPEG Encoder
Secret Key
Secret Key
Permutation of
the Huffman codeword list
Selective encryption algorithm
That operates on sign bits of DC coefficients
I frame
Compressed data
10001110000...
Color
space
convertor
Quantization
FDCT
Entropy
encoder
RGB --> YUV
P/B frame Color
space
convertor
Error
terms
Compressed data
00111100101...
FDCT
Entropy
encoder
+
+
-
RGB --> YUV
Reference
frames
Motion
estimator
Secret Key
Randomly change the sign
bits of motion vectors
Example: Encrypting Frames of
a MPEG-4 Video Sequence
Original frame
Encrypted VLC only
Encrypted FLC only Encrypted VLC and FLC
Virtues of the Virtual Workplace
Universal
access to information,
applications, services, processes, and
people, from any device, over any
network connection - wired, wireless,
or Web
Virtual Workplace Video Clip





Wireless Internet and Web
Wireless appliances
Security
Redundant systems
Wireless applications:
videoconferencing
The Portable Office
Take the office with you, wherever you go
Secure Authentication
High Security Authentication,
including Bio-Authentication
Integrated
Messaging and Communication
Integrated messaging (eg. voice, chat), voice to
text, with intelligent alerting
Information Portability
Access information over any connection –
wired or wireless, regardless of form factor
Business Collaboration
Collaborative capabilities allow on-line
information sharing and communication
Business Continuity
Resilient to network interruptions
Further Readings





Hanzo, Cherriman, and Streit, “Wireless Video
Communications,” IEEE Press, 2001.
IEEE Trans. On Circuits and Systems for Video
Technology, Special Issue on Wireless Video, June
2002.
Sun and Reibman, “Compressed Video over
Networks,” Marcel Dekker, 2001
Wang, Ostermann, and Zhang, “Video Processing
and Communications, Prentice Hall, 2002.
Furht and Ilyas, “Wireless Internet Handbook,” CRC
Press, 2003.