Transcript TEIN2-WS-200807-uhyo-dv

Realizing High Quality
Digital Video Over Internet
July 16th 2008
TEIN2 NOC Workshop
Kazunori Sugiura
( [email protected] )
Digitized Video
Transport System
How can we transport
live multi-media contents?
Hardware
MPEG-2
Media Format
H.264
DV
Raw-HD
WMV
MOV
RM
WAV
MP3
Ogg
Analog
Meta-Data
TXT
New Generation Internet: GLOBAL COMPUTING ENVIRONMENT
To fulfill the IT requirements of every social and living activities in Asia, This project aims new generation
Internet infrastructure by inter-connecting every activities.
Digital Broadcast
Contents on Demand
Metro, Regional Transportation
Sea, and Air Transportation
Cars, Bikes
Aerospace and Satellite
Social Services
Houses, Apartments,
Buildings
Global Computing Environment in Asia
Core Technology for New Asia Internet
Society
Education
Digital Media and Contents
Life style and Livings
Social and Public Activities
Once upon a time
Streaming ?!
1996 Internet World Exposition
Rally Raid Mongolia
1996 Internet World Expo
Frame rate and Resolution
• fps (frames per second)
– NTSC：30fps (29.97fps)
– PAL：25fps
– Films：24fps
• Resolutions ( dot resolutions)
30picts.
30 Picts
30fps
30fps
1 Second
1 Second
Quality and Contents
• By means of media contents
– Quality ( SD to QHD )
– Bandwidth ( 1.5M to 4.5G up)
– Encoding Characteristics
• ( High Computation, Delays…)
We are here
100BaseT
802.11g
DV World
(720x480)
TCP
Friendly
IPv4/IPv6
Embedded
DVD
HDV World
(MPEG2 720p 1080i)
802.11n
1000BT
Multicast
FEC
We are
here
HD World
(MPEG2，H.264 1080p)
DRM
NGN
IPTV
Real HD World
Uncompressed
(1920 x 1080 p)
Digital Cinema
(MJ 4096x2160, 2048x1080)
QHD
(3840x2160)
HDDVD
Blu-ray
10G
Digital
BS
16HD(SHD)
(7680x4320)
64HD(VSHD)
(15360x8640)
10G
Stack
Resolution and Bandwidth
Resolution
Pixel Dot
32Bit
1Frame
30fps
byte
Bandwidth
bps
640x480
307,200
1200K
35.16M
281M
800x600
480,000
1875K
54.93M
439M
1024x768
786,432
3M
90M
720M
1280x1024
1,310,720
5M
150M
1200M
1600x1200
1,920,000
7.32M
219.6M
1.72G
1920x1080
2,073,600
7.91M
237.3M
1.85G
1920x1200
3960x2400
2,304,000
9,504,000
8.78M
36.25M
263.4M
1087.5M
2.06G
8.50G
Summary
• Media streaming over Internet is a challenge
– Bandwidth
– Computational power
– Resolution and video quality
• Advances in media compression technique
• Advances in packet transport technology
Compression Technique
Compression Technique
• Frame based Compression Method
– Motion JPEG
– DV
• Inter Frame Compression Method
– MPEG2
– MPEG4
– H.264
JPEG/Motion JPEG
• JPEG
– Lossy data compression picture format
– DCT(Discrete Cosine Transform) algorithm
• Cutting High frequency
• Motion JPEG
– Combining JPEG pictures to make frame based
video
Compress even more
• Using higher compression method
– Motion based lossy compression
• No movement
– background
• Movement
– Target
• Using the recently used
• Inter Frame compression
MPEG
• Video CODEC
– MPEG1
– MPEG2
– (MPEG3 is gone )
– MPEG4 (H.264)
• Data standard
– MPEG7
• Data handling
– MPEG21
MPEG1
• Specifications
– Bandwidth: 1.5Mbps
– Standards for Video CD
• MPEG1 uses SIF Format
– Non Interlace
– 360×240×29.97Hz(NTSC)
GOP(Group Of Picture) structure
• GOP
–
–
–
–
Frame combination
I picture (Intra picture)
P picture (Predictive picture)
B picture (Bi-directional predictive picture)
Bi-directional
Forward prediction
prediction
I Pictire
B
B
B Pictire
15Picture/1GOP
I
B
B
P
B
B
P
P Pictire
B
B
P
B
B
P
MPEG2
• Higher quality encoding compared to MPEG1
– Broadcast Quality
• Examples
– DVD
– HDV
– Digital Broadcast
Resolutions for MPEG2
MP@HL
1920×1152
×60
4:2:0
HP@HL
1920×1152
×60
4:2:2
MP@H-14
1440×1152
×60
4:2:0
SP@ML
720×576×30
4:2:0
Bピクチャなし
Spt@H-14
1440×1152
×60
4:2:0
MP@ML
720×576×30
4:2:0
SNR@ML
720×576×30
4:2:0
MP@LL
352×288×30
4:2:0
SNR@LL
352×288×30
4:2:0
HP@H-14
1440×1152
×60
4:2:2
HP@ML
720×576×30
4:2:2
MPEG4
• Higher Compression method than MPEG2
• 1998 –
• Object based encoding
• For low bandwidth
• MPEG-4 AVC (Part 10 ) H.264
DV
• DV(Digital Video) 35.382Mbps
– IEC61834 (1999)
• Resolution：720x480(NTSC) 25.146Mbps
• Audio 1.536Mbps
– 48kHz/16bit 2 channel
– 32kHz/12bit 4 channel
– Frame Compression using DCT
• DV Cassette
• mini-DV Cassette
HDV
• Canon, Sharp, Sony, Victor (2003)
• Resolution
– 1280x720 (720p) 19.7Mbps
– 1440x1080 (1080i) 25Mbps
• 1080/25p 1080/30p 1080/24p
– Audio：48kHz/16bit 2 channel
• MPEG1 Audio Layer2 (384Kbps)
– MPEG2
• Inter Frame Compression
– DV, mini-DV
DVCPRO
• Panasonic (1996)
– DVCPRO
• 480i(NTSC) 25Mbps
– DVCPRO 50
• 480i(NTSC) 50Mbps
– DVCPRO P
• 480p(EDTV-II) 50Mbps
– DVCPRO HD
• 1080i/720p 100MBps
– Interoperability with DV Format
• DVCPRO Cassette
What is Streaming?
• Streaming
– Receive Data while playing
– Does not store data as a file ( DRM (Digital Rights
Management) perspective
• Download Playback
– Download contents as a file, and playback afterwards
Internet
Download Playback
Internet
Streaming
Streaming Technology
• To absorb characteristics of the Internet
• Some of the software technologies
– Buffering ( Delays)
– Session Management
– Multiple Bit-rate ( Multiple Quality)
– Bandwidth reservation, QoS
– Error Correction
– Contents Protection
History in Streaming
• 1994
– StreamWorks1.0(First Commercial streaming applications)
– VIC, VAT, RAT developed by UCL (University College London)
• 1995
– Real Audio1.0
• 1996
– Internet World Exposition IWE’96
– RTP(RFC1889)
• 1997
– MS NetShow2.0
History in Streaming (contd.)
• 1999
– QuickTime4.0(ストリーミングへ対応)
– VLC (Video Lan Client)
– DVTS
• 2002
– Helix( Open source )
• 2003
– Windows Media 9
Streaming Theory
TCP and streaming
• TCP
– Retransmission method
– Packet receiving order guarantee
– Congestion Management
• Using TCP for streaming
– How can we manage the real-time when
• Retransmission occurs?
• Congestion Management happens?
UDP Streaming
• UDP
– No Retransmission
– No Packet order guarantee
– No Congestion Management
• To maintain some sort of packet transport
guarantee to End-toEnd
– RTP/RTCP
RTP/RTCP
•
•
•
•
RTP Real-time Transport Protocol
RTCP Real-time Transport Control Protocol
Standardized in RFC1890
Standardization to transport real-time data
through the network
RTP Real-time Transport Protocol RFC1889
• Standard protocol for streaming
• Common information required to send
real-time data
– Sequence Number
– Timestamp
• Dedicate RFC for data dependent part
– Many payload format dependent on their
compression method and media formats
• RTP itself does not reserve resources or
manage and guarantee the QoS
RTP Packet
X
v pExtension
CC
CSRC
Count
Marker
Payload
Type
Sequence Number
Time Stamp
Synchronization Source (SSRC)
Contribution Source (CSRC)
RTP related RFCs
RTP Basic Specifications
RFC1889
RTP: A Transport Protocol for Real-Time Applications.
RFC1890
RTP Profile for Audio and Video Conferences with Minimal Control.
Other Payload Specifications
RFC2198
RTP Payload for Redundant Audio Data.
RFC2793
RTP Payload for Text Conversation.
RFC2833
RTP Payload for DTMF Digits, Telephony Tones and Telephony Signals.
RTP Generals
RFC2508
Compressing IP/UDP/RTP Headers for Low-Speed Serial Links.
RTP Payload Format
Payload Format
RFC2029
Sun's CellB Video Encoding.
RFC2032
H.261 Video Streams.
RFC2035
JPEG-compressed Video.
RFC2038
MPEG1/MPEG2 Video.
RFC2190
H.263 Video Streams.
RFC2250
MPEG1/MPEG2 Video.
RFC2343
Bundled MPEG EXPERIMENTAL
RFC2429
the 1998 Version of ITU-T Rec. H.263 Video (H.263+).
RFC2431
BT.656 Video Encoding.
RFC2435
JPEG-compressed Video.
RFC2658
PureVoice(tm) Audio.
RFC2862
Real-Time Pointers.
RFC3016
MPEG-4 Audio/Visual Streams.
RFC3047
ITU-T Recommendation G.722.1.
RTP Payload Type
Payload Type
Encoding
Audio / Video
Frequency (Hz)
Channel (音声)
0
PCMU
A
8000
1
1
1016
A
8000
1
2
G.721
A
8000
1
3
GSM
A
8000
1
5
DV14
A
8000
1
6
DV14
A
16000
1
7
LPC
A
8000
1
8
PCMA
A
8000
1
9
G.722
A
8000
1
10
L16
A
44100
2
11
L16
A
44100
1
14
MPA
A
90000
15
G.728
A
8000
25
CelB
V
90000
28
Nv
V
90000
31
H.261
V
90000
1
RTCP(Realtime Control Protocol)
• RFC1889
– To Control RTP
• Packet flow control, clock transmission between sender
and receiver
– Quality report by receiver
• Jitter
• Lost packet
• RTCP Sender Report
– Reports to receiver or the 3rd application for status
report on sending conditions
RTCP Packet
v=2 P
RC
Payload Type
SR=200
Length
Header
Sender Synchronization Source SSRC
NTP Timestamp Fore 32bit(Send time for reports)
NTP Timestamp Lower 32bit
RTP Timestamp
Sender Packet count
Sender Octet Count
• Report block comes afterwards
Sender
Info
RTCP Method
• RTCP Receiver Report
– Packet Loss rate, Sum of packet losses, Sequence
number of received packet, jitter, last timestamp
of sender report (LSR), and Delay between LSR
(DLSR)
• BYE
• APP
– Application Extension
RTCP Method
• SDES
– Receiver information
– CNAME
• Receiver identifier
– NAME
• Receiver name
– EMAIL
• Mail address
– PHONE
• Phone No.
– LOC
• Location
– TOOL
• Application name
– NOTE
• User condition
– PRIV
• Application extension
Summary
• Streaming application over Internet uses
– UDP / IP
– RTP
– RTCP
• Ensure real-time isochronous transmission
DVTS Design
Video format used in IEEE1394
• DV Format
•
•
•
•
•
A.K.A “DV Camera”
Frame base compression
30Mbps
720 x 480 Pixels (NTSC)
720 x 576 Pixels (PAL)
• HDV Format
• MPEG2-TS(PS) compressed High Definition DV
• Inter Frame compression ( = Latency )
• Somewhat different from “HDTV”
– 1080i (1440 x 1080)25Mbps, (1920 x 1080 )
– 720p (1280 x 720) 19Mbps
• Less Bandwidth compared to DV 30 > 25
Simple Mechanism
Packet Losses
• What happens when packet losses occur:
– Reuse the past frame data
Past frame data
Is used
Past frame
Packet losses
Present frame
Reused frame
Frame rate reduction and Bandwidth
in DV Format
Bandwidth
(Mbps)
35.00
30.00
25.00
20.00
15.00
10.00
5.00
0.00
full
1/3
1/5
1/7
1/9
Frame Rate
Video Bandwidth
Audio Bandwidth
Network Friendly streaming
• Packet shaping and burst-less transmission
Packets
time
Shaping packets
BURSTY
PACKETS
BURSTY
PACKETS
BURSTY
PACKETS
VIDEO
VIDEO
VIDEO
AUDIO
AUDIO
AUDIO
AUDIO
AUDIO
AUDIO
Fragmented Video Frames
AUDIO
AUDIO
AUDIO
AUDIO
AUDIO
AUDIO
Packet Flow Control
Congestion mechanism
Gateway / Router
Drop
End application based packet shaping
(Avoiding burst traffic)
End application
Application
Large packet buffer pool
Jitter caused by priority
mismatch
Sender host
Application
Waiting
for
Waiting for
audio packet
audio
packet
Audio packet
Video packet
Separation of packet
Sender host
Audio Packet
Video packet
Application
Audio queue
Video queue
Priority audio
packet
TCP＋UDP Traffic
TCP sender
packets
TCP packet
UDP packet
congestion
UDP sender
packets
TCP＋UDP Traffic
TCP sender
packets
TCP packet
UDP packet
congestion
UDP sender
packets
TCP＋UDP Traffic
TCP sender
packets
Decrease packets to
avoid congestion
UDP sender
packets
TCP packet
UDP packet
TCP＋UDP Traffic
TCP sender
packets
Decrease packets to
avoid congestion
UDP sender
TCP packet
UDP packet
Rate will not change
(No congestion avoidance)
packets
TCP＋UDP Traffic
TCP sender
packets
Decrease packets to
avoid congestion
TCP packet
UDP packet
UDP sender
Rate will not change
(No congestion avoidance)
packets
summary
• DVTS is developed with software technology
to
– Adapt into various condition of network
– Try to be friendly with other packet / networks
• Avoiding bursty packets
• Using TCP Friendly algorithms
• Using past packet / loss of packets
DVTS Update
DVTS supports HDV streaming
• Support Video Format
– MPEG2-TS 1080i and 720p
• Support Operating-System
– Linux (kernel 2.4.27 or later and 2.6.x)
• Newest Libraw1394 driver is required.
– Windows XP
– VLC
Transmit
DVTS for
Linux
DVTS for
Windows
Video LAN
Client
IEEE1394
Output
Display
Output
Recording
DVTSng Rev.2
• Merge DVTS and HiDVTS applications on one package
– DV -> use DV mode application
– JVC HDV -> use HDV-JVC mode application
– Sony HDV -> use HDV-SONY mode application
other device will be supported on next revision
• IPv6 multicast (ASM/SSM) update
– Rev.1 cannot use IPv6 multicast function
• getaddrinfo() doesn’t run -> re-enable
• Download URL
– http://beta.dvts.info/setup-0.0.0-2.exe
• このURLはmuscat.sfc.wide.ad.jpのalias
• このままでもいいですし，適宜移動してもOKです
DV Mode Application
Support IEEE1394 output
Not support IEEE1394 output
Please use HDVout tool instead!!
HDV Mode Application
HiDVTS / Camera Output Tool
• Functions
– HDV(MPEG2) RTP data receive
• IPv4, IPv6
• receive port
• unicast, multicast (ASM/SSM)
– IEEE1394(HDV device) output
• only support SONY device (maybe)
• Download URL
– http://beta.dvts.info/hdvout-setup-1.0.0-1.exe
Select HDV device (camera/VCR)
Start/Stop running
Select IP version
Quit application
If you want to use multicast,
Specify multicast address, interface and source address (SSM)
DVTS supports Multicast subsets
(including IGMPv3 and MLDv2)
• Linux kernel 2.6.x supports v4/v6 SSM
• Windows
– XP supports only v4 SSM(IGMPv3)
– Vista supports v4/v6 SSM(IGMPv3,MLDv2)
IGMPv3
DVTS for
Linux
DVTS for
Vista
DVTS for
Windows
XP
Video LAN
MLDv2
MacOS Update
• gldvshow was released
– No Audio Support
– FEC Support
– OpenGL error monitor Support
• DVTS1.0g was released
– Support MacOS 10.5 Leopard
cbrmaker(1/3)
• Constant Bit Rate UDP stream making stream in
Application Layer
– You can check
•
•
•
•
Bandwidth
Transmission delay
Packet arrival interval time
Packet loss
• You can get
– http://www.sfc.wide.ad.jp/dvts/software/cbrmaker/
• Support platform
– Linux i386 only
cbrmaker(2/3)
• You can draw graphs from logs of cbrmaker
– Transmission delay
– Packet arrival interval time
– Packet loss
cbrmaker(3/3)
• Parameter
–
–
–
–
bps (bit per second)
pps(packet per second)
Packet size
usleep time between packets
• Example of the utility
– OS / network driver test
– Network bandwidth test
– Network quality test
Global Studio
Global Studio Location(2008)
NEW(June,2007)
Stanford U.
US
Changes on existing studio
• Echo canceller installation
– To provide better audio environment
– Schedule
• Cambridge(September 07)
• Yonsei(March 08)
• Tsinguha(March 08)
• Replacing video transmission PCs
– To reduce audio noise
– More CPU power(for HD level video transmission)
– Schedule
• Cambridge(September 07)
• Updating equipments at Mita campus
– Improving audio and video quality
• Digital Audio for noiseless audio environment
• HD video
– Collaboration with SOI-Asia
The Stanford Studio
• Installed in June 2007
– Collaboration with Stanford university.
• Location
– The Wallenberg Hall
• Mobile Equipments
– 1st floor learning theater or other conference room
Mobile Equipment cabinet
Mobile cabinet(outside)
Handle to carry the cabinet
Removable doors
Wheels
Mobile cabinet(inside)
Polycom EF2241 Audio echo canceller
Cisco Catalyst 2960
Audio Patch panel(All XLR)
DVTS PCs(Laptop PC is used after 2007)
DMC Studio @Keio Mita
 Location: Tokyo, Japan
 Operated by: Keio University DMC Institute
 DVTS and Polycom / Multipoint capable
 IPv4/IPv6
Studio at Tsinghua University
 Location: Beijing, China
 Operated by: Tsinghua University
 Time difference: 1 hour
Studio at Yonsei University
 Location: Seoul, Korea
 Operated by: Yonsei University
 Connecting to KOREN (Korea
Advanced Research Network)
 Established in February 2006
Studio in San Francisco
Location: California, USA
•15 minutes drive from SFO
Airport
•in Data Center
Stanford University
Time difference: 17 hours
Studio at the University of Cambridge
 Location: Cambridge, UK
 Operated by: University of
Cambridge
 Time-zone: -9 hour (JST)
Studio in New York
 Location: New York, USA
• near United Nations HQ
 Operated by: Japan Society
 Time difference: 13 hours
Configuration Plan for
Cambridge Studio
AC TAP
Polycom
VSX 7000S
Panel
Monitor
ADVC-110
Behringer
FBQ2496
DVTS/HDVTS
S L Systems
Projector
Audio Mixer
WIN-XP
HD Note-PC
POSTERS
Behringer
MX882
20-182 B-U Converter
Network
Camera(SD)
Video Light MIC
WIDE Conversion
Lenz
Sony
HVR-Z1J
IEEE1394
Long Cable
Ethernet Switch
8 to 16 port Giga
DVTS/HDVTS
S L Systems
UPLINK
Ethernet
RGB / DVI
IEEE1394
Accessory Kit Headphone Tripod
Version 1.0 uhyo
ADVC-110
DVTS/HDVTS
S L Systems
S-Video
Audio
DMC Manifesto
Mozilla 24
• 24 hour World Wide continuous event
– Connects Mozilla community member on the earth for
collaboration.
• Different Time zone
• Distributed location
• DMC provide the “Global Studio” as communication environments.
– DVTS based conference
– Real video streaming
• Date & Location
– September 15th -16th
– Japan/US/France
– SOI-ASIA
www.mozilla24.com
Discussion at Paris
Closing session at Mita
Cool Japan
• Support NHK TV program “Cool Japan”
– In November 2007
• Provide DVTS video conference environment
– San Francisco, Cambridge, Seoul and Mita
• On Air
– December 12th
– DVD is available
• Contact kudo
Received video at Cambridge
Received video at Mita
Other Events
• The 6th DMC symposium
– Connect Beijing for Prof. Murai
– December 2007
• Keio/Kyoto MoU ceremony
– Connect Mita and Kyoto
– September 2007
• etc
Feedback and Evolve
体験
EXPERIENCE
マネジメント
DESIGN
MANAGEMENT
TECHNOLOGY
市場
Market
Society
デザイン
テクノロジ
製品化
PRODUCT
社会
フィードバック
FEEDBACK
企画
IDEA
ポリシー
POLICY
検証
TESTBED
試作
PROTOTYPE
91
SIGCOMM2007 IPv4/v6 dual stack
Multicast Live Streaming
using Consumer HD Video
outline
•
•
•
•
SIGCOMM2007 live streaming
Inter-Domain Multicast Routing Status
DVTS for HDV(MPEG2-TS HD)
Next step to deploy IPv6 Multicast
SIGCOMM 2007 Live Streaming
• WIDE Project(AS2500) provided Internet
connectivity for SIGCOMM2007 participates.
• We also provided 2 workshops and 3 conferences
live streaming to world wide researchers who
could receive.
• We prepared following streams.
– IPv4 and IPv6 MPEG2-TS 1080i streaming to world
wide researchers.
– IPv6 MPEG4 streaming to TEIN2 and SOI-ASIA
members.
Inter-Domain Multicast Routing
• Some Asian Academic AS have been enabled
IDMR since SIGCOMM2007. 
– Tein2-SG and Tein2-HK has been connected to
Tein2-JP.
– Koren has been connected to APAN-JP.
– Good content makes us promote IDMR.
• World Wide IDMR ring was constructed!
– Most of Academic AS have multiple AS-Path.
World WIDE Multi-Home IDMR MAP
GEANT2
APAN-JP
TEIN2-JP
TEIN2-HK
TEIN2-SG
AARNET3
TransPAC2
Abilene
IPv4 Listener AS
(Confirmed at SIGCOMM2007)
WIDE
KOREN
APAN-JP
TEIN2-JP
TEIN2-HK
TEIN2-SG
AARNET3
TransPAC2
Abilene
IPv6 Listener AS
(Confirmed at SIGCOMM2007)
UNINETT
NORDNET
WIDE
GEANT2
APAN-JP
TEIN2-JP
TEIN2-HK
TEIN2-SG
TransPAC2
Abilene
SIGCOMM2007
ASIA IPv6 Multicast Live Streaming
13 Mbps UDL
RP
DVTS
SIGCOMM2007
@ Kyoto, Japan
30 Mbps
MPEG4
1Mbps
VideoLAN
AI3/SOI Asia
@Keio, Japan
Uninet members
APAN-JP
MPEG4 VideoLAN
1 and 3Mbps
ThaiREN/Uninet
@Thailand
24 SOI Asia partner Universities
@ 12 Asia countries
TEIN2-JP
RP
INHERENT members
TEIN2-SG
INHERENT
@Indonesia
Backbone is ready! and then…
How to deploy IPv6 multicast?
• Backbone is ready to provide IPv4 and IPv6
multicast.
• BUT! Users don’t care of IP version, they want
to receive good “contents”.
• We have to prepare the IPv4/v6 transparent
multicast environment for the users.
• In addition, IPv6 has a priority to use IPv4:p
DVTS supports HDV streaming
• Support Video Format
– MPEG2-TS 1080i and 720p
• Support Operating-System
– Linux (kernel 2.4.27 or later and 2.6.x)
• Newest Libraw1394 driver is required.
– Windows XP
– VLC
Transmit
DVTS for
Linux
DVTS for
Windows
Video LAN
Client
IEEE1394
Output
Display
Output
Recording
Minimal Settings
• Just prepare 1 Camera and 2 PCs.
– Optional: HDV Deck.
IEEE1394
HDV Camera
$2,000 - $10,000
UDP/RTP
IP Network
HDV VCR
IEEE1394
DVTS supports Multicast subsets
(including IGMPv3 and MLDv2)
• Linux kernel 2.6.x supports v4/v6 SSM
• Windows
– XP supports only v4 SSM(IGMPv3)
– Vista supports v4/v6 SSM(IGMPv3,MLDv2)
IGMPv3
DVTS for
Linux
DVTS for
Vista
DVTS for
Windows XP
Video LAN
Client
MLDv2
Canon XL H1
SIGCOMM2007 Live Settings
Roland v-440 HD
SONY HVR-Z1J
Component/1080i
Component/1080i
Speaker’s
Laptop
RGB-15pin
Component/1080i
Component/1080i
Audio Mixer
Roland VC300-HD
Canon
Hall PA
Canon
Roland VC300-HD
IEEE1394(HDV)
IEEE1394(DV)
hdvsend for Linux
dvsend for Linux
IPv4/v6 Multicast
To Global Multicast Cloud
DV RTP
To SFC
ADVC-110
IEEE1394(DV)
Composite
IPv6 Multicast
To Tein2 via AI3
Connect from APAN-JP
MBGP AS-Path Table from UNINETT NORWAY
AS2500
WIDE
AS7660
APAN-JP
Live Streaming Equipments
SIGCOMM2007
ASIA IPv6 Multicast Live Streaming
HI VISION AND BEYOND
i-Visto
1.5Gbps非圧縮HDTVをはじめとする各種の高品質映像信号を、IPネットワーク上の複
数の拠点間でリアルタイム伝送するシステム。HD-SDI等をIPに変換し送受信する装置
（Gateway）や各種ビットレートの映像ストリームをリアルタイムに蓄積・配信する映像
サーバ（eXmedia server）から構成。
i-Visto media
converter
i-Visto camera
IP network
(LAN/WAN)
HD-SDI
HDTV equipment
i-Visto
eXmedia server
SD-SDI
SDTV equipment
HDTV monitor
i-Visto gateway XG
i-Visto product lineup
i-Visto
manager
Traffic(cont.)
HTB(10GE)
Incomming from Sapporo NW center
(v6-multi)
Sapporo NW center(10GE)
Outgoing to KDDI Sapporo
(v4-uni)
KDDI Sapporo(1GE*2)
Outgoing to KDDI Sendai
(v4-uni)
Traffic(cont.)
ABC(10GE)
Incoming (v4-uni)
Outgoing (v6-multi)
NTT Otemachi(10GE)
Forwarding (v4-uni&v6-multi)
i-Visto(cont.)
• Packetize each line
– Not each frame
• Configuration
– Resolution：1080i or 760p
– Color：8bit(1Gbps) or 10bit(1.6Gbps)
• Network interface: 2 * 1GE
– Usually use 10GE NIC
– Over subscribe 1GE*2 circuit
– Segmentation for each 1GE
• Testing IPv6 Multicast send/receive at 1470byte(most
suitable frame size)
– Pre-Test
– Packet loss occurred by lack of performance at receiving side
Layered Multicast
Layered multicast for
adaptive modulation
• Problem statement
– With WiMAX multicast, multicast data transmission rate depends on the link
capacity of the node whose link quality is worst
• Nodes with good wireless connection cannot get high-quality multicast data
• Combination of layered multicast and adaptive modulation
– Nodes come to be able to receive the multicast data with proper quality
depends on their wireless connection quality
• Key Technolgies
– Layered multicast
• Network resource aware multicast data transmission method
– Multicast group management
• Dynamic IP multicast group management based on the nodes’ current physical
layer modulation
• Mapping between an IP multicast group to a modulation based WiMAX
multicast group
10
Layered Multicast
• Hierarchical data structure
– Data are divided into several “Layer”
10/10
8
• Layer is multimedia data unit
• Video quality control selecting the number
6
of accepting layers
8/10
1
6/10
– Layer
• Base Layer
– Base Multimedia data
• Enhanced Layer
– Quality enhancement data
Enhanced n
• Multicast tree structure
– Data are deliverd with IP multicast
– Senders provide maximum quality data
– Intermediate nodes decrease layers to send
adapting to the network resource
Enhanced 4
Enhanced 3
Enhanced 2
Enhanced 1
Base
6
6/10
6
1/10
6/10
WiMAX electric wave method
64QAM ready
16QAM read
QPSK ready
BPSK ready
Modulation based multicast group
management image
64QAM +
16QAM +
QPSK +
BPSK zone
16QAM +
QPSK +
BPSK zone
BS
QPSK +
BPSK
zone
BPSK
zone
Modulation based multicast group
management
Signal type
64QAM
(Enhanced 3)
Base +
Enhanced 1 + 2 + 3
Base +
Enhanced 1 + 2
16QAM
(Enhanced 2)
Base +
Enhanced 1
QPSK (Enhanced 1)
Base only
BPSK (Base Layer)
distance
Research-Slide
Research background:
The growth of mixed environment
User
requirement
User requirement
Live
download
VoIP
High quality
Event relay
Network
Collaboration game
IPTV
Digital Cinema
Service
characteristic
Service or user-tool
Unicast
Multicast
Resource management
Priority control
P2P
authentication
End
control
virtual-network
AS
Real-network
AS
10G
40,100G 802.11a/b/g
AS
security
End
control
Network
AS
characteristic
802.16e
NGN
Motivation
• Due to the use of high speed Network(1Gbps, 10Gbps, etc),
high bandwidth (high video audio quality) streaming will
become common on the internet
• What is a requirement for high-bandwidth streaming?, and
impact on other applications?
– Ex) What is the tradeoff point on user-oriented application?
What is streaming application problem in nearly future ?
• What is the solution?
studio
HD Camera
studio
HD Camera
Adaptive Real-time Streaming
• Real-time Streaming
– High interactiveness
• Video conference
• Online game, 3D graphics contents
• Need to maintain strict packet interval time
– Difficult to maintain best possible quality
• Network condition change effect on largely
– Important to precisely detect a network condition change.
• Network associated system vs. End-to-End system
– The former merit: easy to quickly and precisely detect network condition (ex.
Bottleneck ling) possible to adjust according to it
• Priority control、Bandwidth guarantee
– The latter merit: not need for specific router or switch implementation
• Loss based (or RTT-based) adaptation
Network Estimation
Network Controller
End-Node controller
Quality Adaptation
•
Research point (1/2)
How can various streaming flows conduct best adaptation
according to network condition and change??
– Network estimation
• Various network environment
– Various network characteristic
• Various congestion
– The number or type of competing flows
– Network load
– User-oriented quality adaptation
Network estimation
• High network utilization
– Effectively consume the network
resource
Network Condition
• Contents quality protection
– Maintaining quality for end-usage
• Stability
– Frequently quality change would degrade it’s quality
– User-friendliness (impact on other applications)
• Scalability (Congestion Control)
– TCP-fairness  high network utilization
» Various definition of fairness
Quality adaptation
(Congestion Control)
Research point (2/2)
• Quality adaptation
– High network utilization  Contents quality protection, Scalability
• The way of changing transmission rate
• The way of maintaining quality on changeable network condition
• TCP-friendly could be best solution ?
• Network estimation
– Network indicator
• What is necessary and effectual?
• Need new indicator or indicator cooperation
– Signaling mechanism
• Responsiveness  accuracy
Transmission Protocol
Video format
Vide/Audio data
Application/service
policy
network
Quality
protection
Media source
client
Network indicator
scalability
Congestion
Control
Previous work
•
Specializing in quality protection
– High network utilization
– The mechanism of minimizing packet loss
•
Multiple control using FEC
– Quality adaptation
• Rate Control (discarding frame, so depends on video format)
• Dynamic FEC (Reed-Solomon Code)
– Network condition estimation
• Packet loss rate on a interval time
• The number of consecutive lost packets ( means a network load)
• FEC recovery rate
Application policy
network
Rate Control
Quality
protection
Media source
Dynamic FEC
Congestion
control
client
・Packet loss rate
・The number of consecutive loss packets
・FEC recovery rate
Future work
• Verification using a simulation
– Definite the mechanism tradeoff point
• High-network utilization, quality protection, stability and scalability
• Network condition (bandwidth and RTT, etc), the number and type of TCPflow and UDP-flow R
– TFRC effectiveness ?
• How can it be change on network condition ?
– My algorithm effectiveness?
• Investigate “effective” adaptation mechanism
• The mechanism of knowing network condition and change.
– Survey
– Verification of detection and learning adaptation mechanism
Research-Slide
Support material
The problem of quality reduction
Sender
Receiver
bursty
Internet
congestion
DV/RTP packet
• According to network condition, pktloss happens
– Physical bandwidth or available bandwidth
• Σ(DVTS traffic + other traffic ) > available bandwidth
– Congestion
– Quality reducing
The existing solution with DVTS(1/2)
• Reuse the past frame data
• Rate Control to save the consumption bandwidth
Σ(DVTS traffic + other traffic ) bandwidth < available bandwidth
35.00
30.00
25.00
20.00
15.00
10.00
5.00
0.00
full
1/3
1/5
1/7
1/9
Video Bandwidth
Audio Bandwidth
The existing solution with DVTS(1/2)
• Static FEC to packet loss recovery
DV/RTP packet
FEC packet
FEC technology
congestion
FEC decode!!
Packet loss recovery
The demerit of existing solutions
• In the condition of network congestion
– Reuse the past frame data and Rate Control
• Solve the block noise
• Cannot solve the reducing video quality
– Smoothness of frequently moving scene
– Static FEC
• Increasing the total consumption bandwidth
– There is the case of quality reducing more !!
– Users adapt the transport method by hand
• What is How to provide the best possible Video and
Play quality ???
Motivation
• Dynamic Adaptation
– To provide the best possible quality
– The user doesn’t have to change the DVTS options
– According to the network condition change,
adaptive DVTS dynamically adapt the transport
method
• DV full rate + FEC 10% ???
• DV half rate + FEC 30% ???
• DV 1/3 rate + FEC 20% ???
approach
• Rate Control with Dynamic FEC Mechanism
– Frame rate control
• reducing bandwidth in case of fatal bandwidth
conditions
 To save the total consumption bandwidth
– Dynamically adapting FEC rate
• keeping play and video quality
• Dynamic available bandwidth proving via FEC
sender
– What happens when network condition get well ??
– Adapting Frame rate to current available bandwidth
» Providing best possible video quality
Internet
Video frame data
Changing each rate
FEC data
Adaptive DVTS design
sender
・Network
state
・Current
flow type
・before
flow type
Rate control
module
receiver
Receive
report
module
Flow
control
module
FEC encode
module
RTCP RR
Packet
・Network
state
Report transmit
module
・pktloss
pattern
・Flow type
Network state
measurement
module
FEC decode
module
Transmit
Buffer
RTP Packet (data, FEC) tmp buffer
Play Buffer
The number of consecutively lost packet
and FEC recovery rate
Frec(%) =
L － L’
L
0
(L > L’, L≠0)
(L≦L’, L≠0)
Frec: FEC recovery rate
Fenc: FEC encoding rate
L: (the number of non-recoverd
UDP packets within 5 seconds)
Adaptive DVTS vs Normal DVTS
BEST CASE
Background bandwith:90M
Anticipating available bandwidth
関連研究
•
•
Streaming Adaptation
– “Adjusting Forward Error Correction with Quality Scaling for streaming MPEG”,
huahui NOSSDAV 2005 ACM, june
– A Rate Control Scheme for Adaptive Real-Time Applications in IP Networks
With Lossy Links and Long Round Trip Times
– “A Dynamic Congestion Control Mechanism for Real Time Streams over RTP”,
Ahmad, N, ICACT 2007, Feb
– A Rate Control Scheme for Adaptive Video Streaming over the Internet,
Panagiotis, IEEE ICC 2007 proceegings
ネットワーク特性/streaming品質特性に関する論文
– “End-to-End Internet Packet Dynamics”, Vern Paxson LBNL-404488,
june23,1997
– “Analysis of Audio Packet Loss in the Internet “, INRIA B.P 93,Jean Bolot,
HUgues Crepin, Andres Vega Garda 06902 Sophia-Antipolis Cedex
– “Improving Accuracy in End-to-End Packet loss Measurement”, SIGCOMM2005
Network States and Flow Types in Full rate(30Mbps)
State
Loss rate
Consecutive loss
F1
0
0
F2
0<R<1
0%
10%
20%
30%
FEC0%
FEC0%
FEC0%
FEC0%
3 ≦ C < 10
FEC20%
FEC10%
FEC10%
FEC10%
10 ≦ C
FEC30%
FEC20%
FEC20%
FEC10%
C < 10
FEC20%
FEC20%
FEC10%
FEC10%
F5
10≦ C < 25
FEC20%
FEC20%
FEC20%
FEC20%
F6
25≦ C
FEC30%
FEC30%
FEC30%
FEC30%
C < 30
FEC20%
FEC10%
FEC10%
FEC10%
F8
30≦ C < 70
FEC30%
FEC20%
FEC20%
FEC20%
F9
70 ≦ C
Half-FEC50%
FEC30%
FEC30%
FEC30%
C < 100
FEC30%
FEC20%
FEC20%
FEC20%
F11
100≦ C < 170
Half-FEC50%
FEC30%
FEC20%
FEC20%
F12
170≦ C
Half-FEC60%
Half-FEC50%
FEC30%
FEC30%
C < 180
Half-FEC60%
Half-FEC50%
FEC30%
FEC30%
F14
180≦ C < 250
Half-FEC70%
Half-FEC60%
Half-FEC50%
Half-FEC50%
F15
250≦ C
Half-FEC80%
Half-FEC70%
Half-FEC60%
Half-FEC60%
C < 300
Half-FEC80%
Half-FEC70%
Half-FEC60%
Half-FEC60%
F17
300≦ C < 380
Half-FEC90%
Half-FEC80%
Half-FEC80%
Half-FEC70%
F18
380 ≦ C
Half-FEC90%
Half-FEC90%
Half-FEC90%
Half-FEC80%
C = any
Half-FEC90%
Half-FEC90%
Half-FEC90%
Half-FEC90%
F3
F4
F7
F10
F13
F16
F19
1≦ R< 3
3≦ R< 8
8≦ R< 13
13≦ R< 18
18≦ R< 23
23≦ R
Network States and Flow Types in Half Rate (15Mbps)
State
Loss rate
H1
0
H2
0<R<1
50%
60%
70%
80%
90%
100%
FEC100%
FEC100%
FEC100%
FEC100%
FEC100%
Normal-Full
3 ≦ C < 10
FEC100%
FEC100%
FEC100%
FEC100%
FEC100%
Full=FEC10%
10 ≦ C
FEC100%
FEC100%
FEC100%
FEC100%
FEC100%
Full-FEC20%
C < 10
FEC100%
FEC100%
FEC100%
FEC100%
FEC100%
Full-FEC30%
H5
10≦ C < 25
FEC100%
FEC100%
FEC100%
FEC100%
FEC100%
Full-FEC20%
H6
25≦ C
FEC90%
FEC100%
FEC100%
FEC100%
FEC100%
Full-FEC20%
C < 30
FEC90%
FEC90%
FEC90%
FEC90%
FEC100%
Full-FEC20%
H8
30≦ C < 70
FEC90%
FEC90%
FEC90%
FEC90%
FEC100%
Full-FEC30%
H9
70 ≦ C
FEC90%
FEC90%
FEC90%
FEC80%
FEC70%
FEC60%
C < 100
FEC90%
FEC90%
FEC90%
FEC90%
FEC100%
Full-FEC30%
H11
100≦ C < 170
FEC90%
FEC90%
FEC90%
FEC70%
FEC70%
FEC60%
H12
170≦ C
FEC90%
FEC90%
FEC90%
FEC60%
FEC60%
FEC60%
C < 180
FEC60%
FEC60%
FEC60%
FEC50%
FEC50%
FEC50%
H14
180≦ C < 250
FEC80%
FEC80%
FEC70%
FEC70%
FEC70%
FEC70%
H15
250≦ C
FEC90%
FEC90%
FEC90%
FEC80%
FEC80%
FEC80%
C < 300
FEC90%
FEC90%
FEC90%
FEC80%
FEC80%
FEC80%
H17
300≦ C < 380
FEC90%
FEC90%
FEC90%
FEC90%
FEC80%
FEC80%
H18
380 ≦ C
FEC100%
FEC100%
FEC100%
FEC90%
FEC90%
FEC90%
C < 420
FEC100%
FEC100%
FEC90%
FEC90%
FEC90%
FEC90%
H20
420≦ C < 600
FEC100%
FEC100%
FEC100%
FEC90%
FEC90%
FEC90%
H21
600 ≦ C
FEC100%
FEC100%
FEC100%
FEC100%
FEC100%
FEC100%
C < 750
FEC100%
FEC100%
FEC100%
FEC100%
FEC90%
FEC90%
750≦ C < 1000
FEC100%
FEC100%
FEC100%
FEC100%
FEC100%
FEC100%
C < 1300
FEC100%
FEC100%
FEC100%
FEC100%
FEC100%
FEC90%
1300 ≦ C
FEC100%
FEC100%
FEC100%
FEC100%
FEC100%
FEC100%
H3
H4
H7
H10
H13
H16
H19
H22
1≦ R< 3
3≦ R< 8
8≦ R< 13
13≦ R<18
18≦ R<23
23≦ R<28
28≦ R<33
H23
H24
H25
33≦ R
Consecutive loss
*************
G8 summit operation using FEC
DVTS
Overview
Hokkaido
Receiving each G8
national broadcast
kanagawa
FEC DVTS
Receiver
Encoder
converter
FEC DVTS
Sender
FEC DVTS
Receiver
DV streaming with FEC 30%
converter
FEC DVTS
Sender
Backbone
Te 3/4
Vlan 3750
203.178.138.129
SNet / JGN2plus
1Gbps
札幌 IMC Net
203.178.138.128/28
T-LEX
10Gbps
Gi 7/13-16
Vlan 3758
203.178.139.97
Cisco2.notemachi
Foundry6.otemachi
Foundry4.nezu
Nec2.yagami
Cisco2.fujisawa
L2
Gi 2/1/2
203.178.139.65
L3
SFC ITC L2 Backbone
100M
1G
10G
1Gbps * 1
SFC λ館 Net
203.178.139.64/28
NTT Business Ether * 4
100Mbps * 4
本牧ハマーズ Net
203.178.139.96/27
Hammers in kanagawa
Cisco2.notemachi
Untag vlan 3758
203.178.139.97/27
Gi1/0/1
Cat3750-1
BBC
CNNI
Gi7/13
Gi7/14
Gi7/16
Gi7/15
Gi2/0/1
Fa0/1
Cat3750-2
TV-5
DWTV
Cat2950-1
RAITV
RTR
Fa0/1
Cat2950-2
NHK
World
YOBI
Sapporo IMC
Cisco2.notemachi
Te 3/4 Vlan 3750
203.178.138.129
T-LEX
catalyst3750.sapporo
tag vlan 3750
203.178.138.142
1/0/1 1/0/2
1/0/15
1/0/14
1/0/3
1/0/7
1/0/5
1/0/13
1/0/4
1/0/6
BBCR
CNN
I-R
TV5-R
DWT
V-R
RAITV-R
RTR
-R
NHKWorld
-R
YOBI
1
YOBI
2
YOBI
3
SFC (backup stream)
Cisco2.fujisawa
Gi 2/1/2
203.178.139.65
203.178.139.78
ax2430.sfc-ramda
0/22
TV5-S
0/23
DWTV
System overview in Sapporo
NHKメディコン後
プレビュー
NHKエンコーダ
直前プレビュー
WIDE
NHK
WIDE Rack
NTSC
PAL
DVモニタ DVモニタ
(WIDE)
(NHK)
BBC
CNN
TV5 YOBI-2
YOBI-3
DW
RAI
RTR
NHK YOBI-1
工具・シリアル
NHKスイッチ(to留寿都)
WIDEスイッチ(toJGN)
port assign and IP address
1/0/1:203.178.138.130:BBC-R
1/0/2:203.178.138.131:CNNI-R
1/0/3:203.178.138.132:TV-5-R
1/0/4:203.178.138.133:DWTV-R
1/0/5:203.178.138.134:RAI-TV-R
1/0/6:203.178.138.135:RPR-R
1/0/7:203.178.138.136:NHK-World-R
1/0/13:203.178.138.137:YOBI-1
1/0/14:203.178.138.138:YOBI-2
1/0/15:203.178.138.139:YOBI-3
Monitor Preview in Sapporo