Transcript NXP PowerPoint template Guidelines for presentations

Audio/Video compression
An introduction
Alain Bouffioux
December, 20, 2006
Agenda
Introduction - The evolution of Audio/Video consumer products and
the role of compression techniques.
Audio & Video compression principles
Audio demonstration
Video demonstration
2
AV Compression / Alain Bouffioux
December, 20, 2006
Agenda
Introduction - The evolution of Audio/Video consumer products
and the role of compression techniques.
Audio & Video compression principles
Audio demonstration
Video demonstration
3
AV Compression / Alain Bouffioux
December, 20, 2006
Moore’s law
Number of
transistors per
square inch
doubles every 18
months
4
AV Compression / Alain Bouffioux
December, 20, 2006
Moore’s law today
Cost of a transistor divided by one million in 30 years
5
AV Compression / Alain Bouffioux
December, 20, 2006
Moore’s law today (2)
Itanium 2 (2004,
592,000,000)
“Self-fulfulling prophecy”
= roadmap for the semiconductor industry
6
AV Compression / Alain Bouffioux
December, 20, 2006
Moore’s law today (3)
Roadmap for semiconductor industry
= only certainty in the current undefined future
Progress in semiconductors
= fuels the innovation
= fuels the software revolution
= fuels the wireless revolution
(WLAN, WPAN, WBAN, …)
Examples:
WBAN & sensors, RFID applications, camera to swallow, flexible display…
New products & related needs motivate semiconductor industry
( Self-fulfilling prophecy)
Moore’s law will continue to apply: 10 years, 20 years ?
– Economical limitation ? Investment (fixed) cost / globalisation
– Power consumption (Moore’s low in reverse direction)
– Architectural gap between IP-blocks & application (middleware still more
complex…)
7
AV Compression / Alain Bouffioux
December, 20, 2006
The evolution of CE products (1)
Past: every CE product was analogue
~1983: Music becomes digital (CD players)
Early 90ies: microprocessor enters CE devices
(Early DVD players incorporated processing power equivalent to their
comtemporary PC)
Late 90ies: Communication features incorporated within a CE devices.
Computer
Consumer
DVD
CD
Receiver Digital TV or
Residential gateway
Communication
8
AV Compression / Alain Bouffioux
December, 20, 2006
The evolution of CE products (2)
The Residential Gateway (Set-Top-Box, ADSL modem?) as the
link between the home and the world-wide information
infrastructure.
World-wide
communication
infrastructure
RG
Home Network
9
AV Compression / Alain Bouffioux
December, 20, 2006
The evolution of CE products (3)
The Residential gateway (in home) as the gateway to various
services.
Local Server provides 2 kind of services:
– Broadcast
Analogue & digital TV, NVOD, PPV
– Point-to-point (Home to local server)
Home shopping, VOD, e-mail, Web browsing, PC connection...
About 1000 homes
Local server
Network
Internet
Local server
10
AV Compression / Alain Bouffioux
December, 20, 2006
The evolution of CE products (4)
The Residential gateway as a key element of the home network
To telephone Network
Computer
To satellite
Network
Residential
Gateway
To cable Network
Television
Home Network
Disk
Recorder
DVD
Jukebox
11
AV Compression / Alain Bouffioux
December, 20, 2006
The evolution of CE products (5)
Everything will become digital: audio, telephone, video, photography,
newspaper…
The question is not if a selected product will really become digital; the
question is when?
Consumer/Computer/Communication Convergence is progressive
New products combine all 3 domains
(e.g. New GSM devices – Television on mobile)
Products always more and more complex
Products have always new features
Lifetime of products is always shorter
12
AV Compression / Alain Bouffioux
December, 20, 2006
Factors enabling such evolution
Compression is one among the various factors (all powered by
semiconductor progresses) that enable multimedia technologies.
Audio/Video
Compression
(e.g. CCIR601 vs MPEG)
Disc capacity (DVD), communication
going digital (xDSL, DVB, IP......)
Multimedia
Electronics
(Memory capacity,
clock frequency,
Software (methodology,
µP architecture, ...
user interface ...)
--> decoding at low cost
International cooperation
(interoperability &
economy of scale)
13
AV Compression / Alain Bouffioux
December, 20, 2006
BUT !!
Convergence of technologies (consumer, communication, computer)
All products combine all three technologies
BUT !
Divergence of applications
– Home consumer, Multimedia phone, Camera, PDA, Office
computer, Automotive…
– High number of potential products
Technology push Market pull (user centric approach)
14
AV Compression / Alain Bouffioux
December, 20, 2006
Agenda
Introduction - The evolution of Audio/Video consumer products and
the role of compression techniques.
Audio & Video compression principles
Audio demonstration
Video demonstration
15
AV Compression / Alain Bouffioux
December, 20, 2006
Compression in first A/V CE Products (1)
First Audio/Video products made compression without knowing it was
compression.
How ?
By removal of irrelevancies (sampling rate, quantization)
Audio and Video characteristics
Spectral
Sensitivity
Spatial
Sensitivity
Audio
Good
Video
Bad
Bad
Good
16
AV Compression / Alain Bouffioux
December, 20, 2006
Compression in first A/V CE Products (2)
Audio products
From 2 to 7.1 channels are enough to provide the spatial resolution.
Video products
Three colours (RGB) are enough to provide the spectral resolution.
17
AV Compression / Alain Bouffioux
December, 20, 2006
The need for more compression (1/5)
Audio: Compression needed in spectral domain
Bitrate of a stereo audio source (CD-DA encoding)
A ud oi w a ve of m
r (m
ti e )
m
ti e
Sampling frequency : 44.1 kHz
Stereo
16-bit per sample
Bitrate = 44100 * 2 * 16 = 1.41 Mbit/sec
18
AV Compression / Alain Bouffioux
December, 20, 2006
The need for more compression (2/5)
Video: Compression needed in spatial domain
Bitrate of a video source (CCIR 601 - 50 Hz countries)
V di eo mi age
720 sam p el s
576
lni e s
25 images per second
YUV coding (Y: luminance - U,V : Chrominance)
Y: 8 bit per pixel - U,V: 1 pixel on 2 coded, 8 bit per pixel
Bitrate = (576*720)*25*16 = 166 Mbit/sec
19
AV Compression / Alain Bouffioux
December, 20, 2006
The need for more compression (3/5)
Channels availables for AV transmission
– Analog television channel (compatibility)
Cable (bandwidth = 8 MHz)
Satellite (Bandwidth = 30-40 MHz)
 Capacity around 40 Mbit/sec
– Compact disc (CD)
For 74 min. play time : 1.41 Mbit/sec
20
AV Compression / Alain Bouffioux
December, 20, 2006
The need for more compression (4/5)
MPEG-1 target (Moving Picture Expert Group)
(Video-CD : 74 min. constraints)
V di eo :166M b it/se c
1 4. M b it/se c
A ud oi :1 4. M b it/se c
C om p re s s oi n
But quality was judged too poor (about VHS quality)
21
AV Compression / Alain Bouffioux
December, 20, 2006
The need for more compression (5/5)
MPEG-2 target
– Program stream (DVD)
1 p ro g ram
(v di e o ,
m u ltci h a n n e l
a u d oi , ....)
3 -9 M b it/se c (va rai b el b itra et )
(b u th gi h e r q u a lity ht a nM P EG -1 )
C om p re s s oi n
= m o tvi a toi n of r ht e ca p a c ity
ni c re a se o f ht e C D (--> D VD )
– Transport stream (DVB)
n p rog ram s
(v di eo ,
m u lt ci hanne l
aud oi , . . . .)
ab ou t 40 M b it /se c (con s tan t b itra te )
D
( V B -S a te llite & D VB C
- ab el )
C om p re ss oi n
22
AV Compression / Alain Bouffioux
December, 20, 2006
Principles of compression (1/2)
Compression (or source coding) is achieved by suppressing
information :
– redundant information
– irrelevant information
Suppression of redundant information
 lossless compression
example: PCM to DPCM,DCT
The original signal and the one obtained after encoding and
decoding are identical
F c (x ,y ,t)
C om p re s s oi n
R c kb p s
D e com p re s s oi n
R i< R c
F p (x ,y ,t) = F c (x ,y ,t)
Rp =R c
23
AV Compression / Alain Bouffioux
December, 20, 2006
Principles of compression (2/2)
Suppression of irrelevant information
 lossy compression
Example: bandwidth limitation, masking in audio
The original signal and the one obtained after encoding and
decoding are different but are perceived as identical
F c (x ,y ,t)
C om p re s s oi n
R c kb p s
D e com p re s s oi n
R i< R c
F p (x ,y ,t) <> F c (x ,y ,t)
Rp =R c
24
AV Compression / Alain Bouffioux
December, 20, 2006
Agenda
Introduction - The evolution of Audio/Video consumer products and
the role of compression techniques.
Audio & Video compression principles
Audio demonstration
Video demonstration
25
AV Compression / Alain Bouffioux
December, 20, 2006
Audio Demonstration
From “Borderline” Madonna - Stereo - 16 bit/channel
Compression used AAC
Original
705 kbps
Compression
128
kbps
64
kbps
32
kbps
16
kbps
Decompression
26
AV Compression / Alain Bouffioux
December, 20, 2006
MOS scale (1/2)
Signal distortion is not a good measure of the performance of a lossy
compression method
 an other method is necessary: MOS scale (Mean Opinion Score)
The five-grade CCIR impairment scale (Rec.562)
1(Very annoying), 2(Annoying), 3(Slightly annoying), 4(Perceptible but
not annoying), 5(Imperceptible)
Example:Double blind test
27
AV Compression / Alain Bouffioux
December, 20, 2006
MOS scale (2/2)
O rg
i n
i a ls g
i na l
C om p re sse d s g
i na l
E xcha n g e b o x
R
( a nd om o
f r ls
i e
t ne r)
Se e
l co
t r
(co n tro le
l d b y ls
i e
t ne r)
Ls
i e
t n e r a n sw e rs o
t :
1W
. hc
ih sg
i na l s
i h
t e o rg
i n
i a l? 2 o r 3 ?
2 .G ra d e h
teoh
t e r on e ?
5
m
I pa m
ir e n t
s ca e
l
Co
m p r
e s se d
O rg
i n
i a l
M a x va u
l e
4
M e a n va u
l e
M
n
i va u
l e
3
1
2
3
S e q ue n ce
28
AV Compression / Alain Bouffioux
December, 20, 2006
Agenda
Introduction - The evolution of Audio/Video consumer products and
the role of compression techniques.
Audio & Video compression principles
Audio demonstration
Video demonstration
29
AV Compression / Alain Bouffioux
December, 20, 2006
Compression to VBR or to CBR
B it
ra e
t
C on s a
t n t
b it ra e
t
C on s a
t n t
q u a l ity
C om p e
l x
Sm
i pe
l
D s
i o
t rs o
in
CBR (Constant Bit Rate) vs VBR (Variable Bit Rate)
Scene more complex Higher bit rate for same quality
CBR  variable quality (example : Video CD artefact)
Constant quality  VBR necessary (e.g.: DVD-Video)
30
AV Compression / Alain Bouffioux
December, 20, 2006
Video demonstration
MPEG-1 Example
Video CD standard
(288*352)video – CBR 1.4 Mbps
MPEG-2 Example
DVD standard
(576*720)video – VBR ~3Mbps
31
AV Compression / Alain Bouffioux
December, 20, 2006
The compression trade-off
Compression techniques are still making progress
Trade-off Complexity/Quality/Bit Rate
New technique may result in new trade-off
Complexity
Quality
MPEG Layer 2
MPEG Layer 1
MPEG Layer 3
Other Technique
Speech coding
MPEG AAC
Bitrate
32
AV Compression / Alain Bouffioux
December, 20, 2006
33
AV Compression / Alain Bouffioux
December, 20, 2006