Transcript Note - Square Server

Video Transmission System
Heejune AHN
Embedded Communications Laboratory
Seoul National Univ. of Technology
Fall 2008
Last updated 2008. 11. 23
Agenda



QoS Requirement and Constraints
Impacts on Video Coding
Representative Video Transmission Systems


MPEG-2 PS & TS System
Internet-based Video Transmission System
Heejune AHN: Image and Video Compression
p. 2
1. Transmission & Video coding

Transmission constraints



Impact on the video coding system: cross-layer concepts
e.g. rate-control, un-equal FEC, scalable coding etc
QoS & QoE

QoS (Quality of Service)
• network layer performance measures
• Bit-rate (mean, variation), delay (mean, jitter), loss (bit, packet)

QoE (Quality of Experience)
• Application (user) layer performance measures
network/storage
Input
Video
encoder
Bit-rate
Delay
Loss
Heejune AHN: Image and Video Compression
decoder
Input
Video
p. 3
QoS: Data rate

Required


Offered

Mean Rate
• 384kbps (WCDMA)
• DMB, DTV
• Required Quality
• TV (2~5Mbps)

Circuit Switched (constant)

Rate-Variation
• Scene complexity (inherent)
• Coding scheme
2.5
Packet-switched (variable)
• Internet
• LAN
• ADSL connected to Internet
2.0
1.5
화면당 트래픽 (x 105 bits/frame)
1.0
4000
4200
4400
4600
4800
5000
(a) JPEG 기법으로 부호화
3
2
1
0
4000
4200
4400
4600
4800
5000
4800
5000
(b) H.261 기법으로 부호화
3
2
1
0
4000
4200
4400
4600
(c) MPEG-1 기법으로 부호화
화면 번호
Heejune AHN: Image and Video Compression
p. 4
QoS: Error (distortion)

Required

Low transmission
distortion

Offered

Circuit switched
• low random bit error
• very seldom burst errors

Wired Packet-switched
• almost no bit error
• low packet loss
• some burst packet loss
– Due to network Congestion

Wireless packet-switched
• Some bit loss
• Some packet loss
• Some burst packet loss
– Due to channel fading
Heejune AHN: Image and Video Compression
p. 5
QoS: Delay

Required

One-way application

Offered

• usu. low transmission delay
• Not in sattlelite
• Constant delay & low variation
• e.g. DTV

Two-way application
• Low delay & low variation
• e.g. Videophone (< 400ms)

Interactive
• Low delay and low feedback
delay
• e.g. VoD TV
Heejune AHN: Image and Video Compression
Circuit Switched

Packet-switched
• Variable delay
• Due to congestion and rerouting in Internet
• Due to ARQ in wireless comm.
p. 6
2. Impact of trans. on Video coding

Data Rate

Rate control & output buffering
• constant bit-rate for circuit-switching network
• Smoothed bit-rate for packet switching network


High activity scene has lower quality
Error

Error propagation
• VLC error
– Bit error => the corresponding VLC decoding error
• Spatial error propagation
– VLC error => the successive VLC decoding error
– Resync. marker for every slices, piictures, GOPs.
• Temporal error propagation
– Wrong motion-compensation of Blocks in successive frame
Heejune AHN: Image and Video Compression
p. 7

Error-Concealment
• Use Reversible VLC
• Use Spatial domain smoothing
– POCS (projection onto convex set)
• Use temporal domain: MV estimation
– Use zero vector from previous frame
– Use median vector
– Re-estimation using boundary pixels
Heejune AHN: Image and Video Compression
p. 8

Feedback based control
• Due to low delay requirement
• Error tracking & intra-coding (H.263 Annex N)
• Reference picture selection (H.263 Annex U)
intracoding
Heejune AHN: Image and Video Compression
p. 9

Delay

Delay components
• Capture delay
– one frame, but can reduce it
• Encoder delay
– Depends on encoder performance, less than one frame
– B picture introduces extra delay
• Output buffer
– Depends on smoothing and rate control
– Max. out buffer delay = buffer size / tx rate
• Network delay
– Depends on network types and network conditions
• Input buffer
• Decoding delay
• Display buffer

Low delay case
• Select appropriate network and QoS negotiation
• no B picture, low output buffer
Heejune AHN: Image and Video Compression
p. 10
3. Video Transport Systems

MPEG-2 system

Provides Multiplexing and synchronization mechanism
•
•
•
•
•

MPEG-2 = System + Video + Audio
Build PS & TS from ES
Application environment
Fixed, guaranteed bit-rate, predictable delay and predictable errors
Used in Digital Cable TV, T/S-DTV, DVD etc
Internet Multimedia transport System

H.324-based system
• ITU-T’s Internet conference system
• Used in Serom’s Dialpad etc

RTSP (Real-time Streaming protocol)
• VCR remocon DESC, SET-UP, PLAY, STOP, TEAR-DOWN

SIP
• simple Session Initiation Protocol, current VoIP

RTP
• transport protocol for multimedia data
Heejune AHN: Image and Video Compression
p. 11

PS stream



ES (elementary steam) : video bytes, audio bytes streams
PES (packetized ES) : timestamped ES packet
Program Stream = MPEG-1 system
• One program (video, audio, etc), no loss assumption
• Variable and long Packet (called PACK)
• Pack header includes “SCR (system clock reference)

PS stream
encoded audio
PES-1
encoded video
PES-2
etc
PES-1
PACK
PES-3
Heejune AHN: Image and Video Compression
DVD
PACK
encoded audio
PES-2
encoded video
PES-3
etc
p. 12

TS (Transport stream)





Time stamp and clock info is supported
Multiple program is muxed
Fixed size (188 Byte) TS packet
inner coding (Reed-soloman) and outer coding (convolutional)
Program table info is added
RS, Conv
Program (KBS)
PES-1
PES-2
RS, Conv
TS
TS
tx
PES-1
PES-2
rx
Program (MBC)
Program (MBC)
PES-3
Program map
Heejune AHN: Image and Video Compression
PES-3
Program map
p. 13
RTP based transmission

H.323 system components





H.263 terminal
Gateway : to PSTN
Gatekeeper : call and BW broker
MCU: media mixer and trsnscoder
H.263 terminal protocol architectures


Signaling
Data transport
Heejune AHN: Image and Video Compression
p. 14
RTP

Real time protocol
Supports time-stamp, not guarantees the real time transmission
 “He………llo” is different from “Hell ……..o”

Internet
cam
00.00
00.10
00.20
00.32
00.01
packet 1
00.08
packet 2
00.14
packet 3
00.27
00.28
00.39
00.40
전송 시간
Heejune AHN: Image and Video Compression
00.18
도착 시간
재생버퍼
packet 1
packet 2
packet 3
재생 시간
p. 15
RTP 의 위상
 RTP






IP/UDP header
Format
V: Version (현재 버전은 ‘2’)
P: Padding 유/ 무 (‘1’/’0’)
Ver P X
CC
M
PT
Sequence Number
• 패딩의 마지막 바이트는 패딩의
길이
Time-Stamp
X: 확장 헤더 유/ 무 (‘1’/’0’)
CC: Contributor(CSRC ID)의 수 (
0~15 )
M: Marker bit : frame end/ silent
period
PT: Payload Type
Synchronization Source Identifier (SSRC)
• Fixed : 0: PCMu Audio, 33:
MPEG2 Video
• Dynamic 97+
Contributing Source Identifier (CSRC)
.
:
Contributing Source Identifier (CSRC)
(payload header)
Video/audio data
Heejune AHN: Image and Video Compression
p. 16

SN (sequence number)
• 첫번째 패킷- random하게 설정 , 전송시마다 1 씩 증가

TS (time-stamp)
• 첫번째 패킷- random/negotiated value
• 이전 timestamp값 + 재생되어야 하는 시간 (클럭틱에 의존)

SSRC & CSRC
• Mixing 시에 사용
End system
SSRC=‘9’
PCMu
Audio
203.246.81.51
End system
SSRC=’15’
H.261
Translater
G.721
Audio
203.246.81.10
Mixer
SSRC=’5’
SSRC=’5’
CSRC=’9’ / ’15’
203.246.81.50
203.246.81.54
Heejune AHN: Image and Video Compression
p. 17
 RTCP
(Real-time control protocol)
Help RTP function
 Use rtp port + 1

RTCP
Message types
Sender Report
200
Receiver Report
201
Source Description Message
202
Bye Message
203
Application Specific Message
204
<Type>
Heejune AHN: Image and Video Compression
p. 18
RTCP
End system
SSRC=‘9’
PCMu
Audio
203.246.81.51
End system
SSRC=’15’
H.261
Translater
G.721
Audio
203.246.81.10
Mixer
SSRC=’5’
SSRC=’5’
CSRC=’9’ / ’15’
203.246.81.50
203.246.81.54
Heejune AHN: Image and Video Compression
p. 19
MPEG-4 file format
Heejune AHN: Image and Video Compression
p. 20

Box (Element)







ftyp - describes the file type and compatibility of the MP4. Always
present - a top level atom which always comes first.
moov - contains all the item metadata. at beginning of the MP4 file
after ftyp. Contains all descriptive and technical metadata, to allow
the player to use appropriate codec(s) for the various elementary
streams, identify them correctly etc.
mvhd - the master header describing the movie content
trak - a data 'track' or stream - description of one of the elementary
streams: video, audio, subtitles
udta - user data box (eg the box containing iTunes metadata)
data - data portion of some types of user box (eg the picture data of
a covr cover art atom)
mdat - contains the multiplexed media data stream (usually by far
the biggest box)
Heejune AHN: Image and Video Compression
p. 21
Streaming Industry
Heejune AHN: Image and Video Compression
p. 22
RTSP

Real-time Streaming Protocol

Provides Remote Control function
Heejune AHN: Image and Video Compression
p. 23

Transaction example
C=>S: SETUP rtsp://audio.example.com/xena/audio RTSP/1.0
Transport: rtp/udp; compression; port=3056; mode=PLAY
C<=S: RTSP/1.0 200 1 OK
Session 4231
C=>S: PLAY rtsp://audio.example.com/xena/audio.en/lofi RTSP/1.0
Session: 4231
Range: npt=0 (npt = normal play time)
C=>S: PAUSE rtsp://audio.example.com/xena/audio.en/lofi RTSP/1.0
Session: 4231
Range: npt=37
C=>S: TEARDOWN rtsp://audio.example.com/xena/audio.en/lofi
RTSP/1.0
Session: 4231
S: 200 3 OK
Heejune AHN: Image and Video Compression
p. 24
HTTP Streaming


Scholars believed that multimedia over UDP (delay)
But, since 2000, industry starts streaming over TCP (HTTP)

Why HTTP?
• Firewall traversal
– No guarantees except web traffic
• Hardware performance
– FMS needs high performance system but most mobile device can‟t
have.
• No need to upgrade infra
– Network facilities can‟t be upgraded easily.
• Ease of distribution
– traditional inexpensive HTTP server can be used.
– (cf. Adobe‟s FMS has a license fee of $995)
• Scalability
– Caching each segments into CDN
Heejune AHN: Image and Video Compression
p. 25
3GPP HTTP Streaming
Media Presentation on HTTP
Server
Segment
DASH Client
Media
Presentation
Description
DASH
Control
Engine
Resources
located by
HTTP-URLs
HTTP/1.1
Heejune AHN: Image and Video Compression
on-time http
requests to
segments
HTTP
Access
Client
Media
Engines
p. 26
Comparison
Heejune AHN: Image and Video Compression
p. 27
Conclusion

Congratulation !




You have finished
“From basics, through standard, to application on video coding”
The course was not 100% perfect, but ok.
I hope you could find any interesting the video coding and
information theoretic topics.
Any further research cooperation is welcomed.
Heejune AHN: Image and Video Compression
p. 28