Transcript Immersive Media Delivery and Access Grid

Proposed Collaboration on Access
Grid / Ubiquitous Computing
JongWon Kim and Juwon Park
The Temporary Genkai-Hyeonhae Meeting
Feb. 27th, 2003
Networked Media Laboratory
Dept. of Information & Communications
Kwang-Ju Institute of Science & Technology (K-JIST)
[email protected]
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
http://netmedia.kjist.ac.kr
Contents
Collaboration on Access Grid
Collaboration on Ubiquitous
Computing
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Collaboration
on
Access Grid
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Access Grid
Access GRID
oSupport group-to-group interaction across the Grid
Components of an AG Node
RGB Video
Digital Video
NETWORK
Display
Computer
Digital Video
Video
Capture
Computer
Audio
Capture
Computer
NTSC Video
Analog Audio
Digital Audio
Mixer
Control Computer
Echo
Canceller
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Usage Areas
 The Academic and Research, Government, Private Sectors
 Making Remote Collaborations Work across Boundaries
 Check “Multi-Sector Collaboration over the Access Grid”
by J. T. von Hoffman (Boston Univ)”
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Collaboration Issue for H-G Joint Projects
 Access Grid (#4)
Status 2
Contact: Koji Okamura (Kyushu Univ.)
JongWon Kim (K-JIST)
 Participants
Kyushu Nat`l Univ, AIST
KISTI, K-JIST, Chonnam U, HYU
 Description (to be revised)
The Access Grid, as a communication channel for users from remote sites, can give
researchers and scientists in both Korea and Japan a seamless and immersive collaboration
environment. With this project, we want to explore the potential of Access Grid in supporting
collaborative research between two countries. To realize it, we will collaborate to research ways to
build low-cost Access Grid system (e.g. mini-AG or PIG) and to promote its deployment. Also, we
will conduct a joint research to improve current AG systems to better support enhanced
video/audio quality, interaction, security, and inter-operablity.
 Expected Results (to be revised)
- Provide a seamless colloaboration channel for researchers and scientists.
- Promote the adoption of multicast technology over H-G link.
 Required bandwidth
- Dependent on the number of site, camera per site, video rates
(normally full access node requires bandwidth around 1.5Mbps)
- Multicast support
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Relation with AG and other projects
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
N*Grid Project lead by KISTI
GGF Grid
APAN APGrid
Euro APGrid
ANL/NCSA
APEC APGrid
Grid Tech-center
National Grid Infra
N*Grid
KISTI(2002~2006)
Grid Standard
N Supercmpt Grid
N Clusters Grid
N Appls. Grid
Access Grid
VR Grid
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
GNOC
Grid Middleware Research
N BT Grid
N NT Grid
N CT Grid
Security
Resource Mngt.
Data Mngt.
N ET Grid
N IT Grid
N ST Grid
Inf. Service
GFK
KISTI AG
 First AG in Korea (2002)
Int. AG
Community
KISTI’s Access Grid Node
Digital Video
RGB Video
Display
Computer
Network
enabled
IP
multicast
Shared App.
Control
NTSC Video
Digital Video
Video Capture
Computer
Analog
Audio
Digital Audio
Mixer
Audio Capture
Computer
RS232 Serial
Control
Computer
Echo
Canceller
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
K-JIST Mini AG
 Personal Interface to Access Grid (PIG) (Oct. 2002)
 With AG 1.2 release (note that PIG software is still very delicate)
KIST
AG Node @ KISTI
1st PIG Node
In KOREA
KISTI
1st AG Node
in KOREA
K-JIST
PIG @ K-JIST
KIST-KISTI-K-JIST AG demo (2002.10.30)
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Towards Enhanced Access Grid: e-AG
 An experimental, on-going trial to enhance the functionalities of
Access Grid by augmenting improved support for immersive 3D
Display, Human Computer Interaction, and Networking.
 With e-AG, we hope support
o All functionalities of Access Grid
o Stereo video delivery and display
o Collaborative use of 3D computer graphics.
Enhanced Part
Stereo
Camera
Input Device
Computer
silver
screen
3D
Display
Computer
Network
Access Grid
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
2D
Video Capture
Computer
Audio Capture
Computer
Control
Computer
Echo
Immersiveness Enhancement for Access Grid via
Stereo Video Delivery
 For providing the immersiveness, we include the stereo video delivery
functionality to EAG
 Developing 3D video delivery module enables the reliable and real-time
 The developed system supports
o Reliable networking of real-time media with reliable transport via ARQ, FEC,
and their hybrid
o Versatile display of stereo video in Left, Interlaced, Left & Right, and Above
& Below views.
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Immersiveness Enhancement
for Access Grid via Computer Graphics
 For the graphics side of 3D, EAG is being enhanced by supporting CG-based
sharing in various communication architectures.
 Combination of push repository in server memory and file repository method is
implemented to support multi-user collaborative environment.
 UIC EVL’s QUANTA is adopted as a baseline networking and data management
module
 VR software tools such as CAVELib or VR Juggler are also selected and utilized to
build immersive 3D display.
Server
Shared Memory
CMD, ID, Coord.
Client A (VR Juggler)
3D Display
Input
Networking
(Shared Memory Client)
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
CMD, ID, Coord.
Client B (VR Juggler)
3D Display
Input
Networking
(Shared Memory Client)
Network
CMD, ID, Coord.
Client C (CAVELib)
3D Display
Input
Networking
(Shared Memory Client)
Collaboration Issue for H-G Joint Projects
 On-line Joint Technical Conference on Access Grid
 Joint Project (Long-term)
o Joint participation in AG 2.0 and beyond
Research Network in Korea
Korean Partner Carrier
KJCN
Japanese Partner Carriers
Fukuoka Gigabit Highway
GENKAI Project
QGPOP part
Shimonoseki
KIST
Korea
KJIST
KISTI
Busan
Gwangju
Repeaterles
s KJCN
To Tokyo
GENKAI/Hyeonhae
Project
Kitakyushu City
High-speed LAN
GbE
150Mbps
APAN
Internet2
Univ. of Fishery
250 km
2.4Gbps
QGPOP
(ISIT)
Fukuok
a
Kurume
FGH AP
Submarine cable landing station
NETWORKed MEDIA LAB. Network equipment (Routers
etc.)
DEPT. OF INFO. & COMM., K-JIST
Ohmuta
Kitakyushu
Nogata
Iizuka
Tagawa
Japan
Collaboration
on
Ubiquitous Computing
- Seamless Mobile Service? NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
What’s UbiComp?
What’s Ubiquitous Computing (UbiComp)?
Distributed
Systems
•Remote communication
•Fault tolerance
•High availability
•Remote info access
+
Mobile
Computing
+
Ubiquitous
Computing
•Smart space: context-aware
•Invisible: smart agent
•Localized scalability: physical distance-based
•Uneven condition: heterogeneous infrastructure
•Mobile networking: mobile IP, ad hoc protocol, etc.
•Mobile information access: BW-adaptive, data consistency, etc.
•Energy-aware systems: process scheduling, memory management, etc.
•Location-sensitive systems: location sensing, location-based services, etc.
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Courtesy of K-JIST U-VR Lab
UbiComp Challenges: ubiquitous
Networking
 Infrastructure: ubi-networking
o Beyond desktops/servers
o Connected to networks
Network
 Networked appliances
 Sensors
 Historical sites & other locations
Mobile
Embedded
Location
Interactive
Sensor
Flight
Simulator
Wearable
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Active
badge
Collaboration Issue for H-G Joint Projects
 Ubiquitous (Wireline/wireless) Computing (#18)
Status 2
Contact: Morioka ([email protected])
JW Kim ([email protected])
 Participants
ISIT
KJIST
 Description
We are planning to build a prototype version of ubiquitous environment utilizing
Mobile IP and IEEE 802.11a/b (and other L2 media) connecting Fukuoka, JAPAN and
Gwangju, KOREA.
On top of the built ubiquitous environment we will explore example applications such as
user tracking
and IP mobile video streaming.
 Expected Result
In our joint effort, instead of trying to build hot spots, we will focus on building real
ubiquitous
environement with mobility and security support. We hope to let people in both countries
to better locate each other and communicate ubiquitously.
 Bandwidth
Peak 100 Mbps / Average 10 Mbps
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Project Scope & Objective
 Mobile Media Delivery via Wireless Multicast and Network
Adaptation
Efficiency:
Better Bandwidth
Utilization
Wireless Aware
Media Application
Wireless Network
Adaptation
Reliability:
Better Quality
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Group
Communication
(1-to-N, M-to-N)
Mobile IP /
Wireless
Multicast
Reliable
Multicast
Advanced Streaming
(Wireless + Multicast)
Project Scope & Objective (Cont.)
 For Streaming Media Applications over Wired/Wireless
Integrated Networks
o Real-time streaming of stored contents only (but can include on-line
streaming) to enable buffering (up to 3sec)
o Maybe extended to conferencing (limited scope) at later phase of
project
 Will address the challenges such as
o Mobility: How to track user’s movement and sustain seamless
streaming?
o Multicasting: How to enable sharing of wireless channel efficiently?
o End-to-end QoS: How to maintain the quality of media streaming
over highly fluctuating wireless network?
 Anticipated Duration: 2002. 12 ~ 2006.12 (4 years)
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Implementation Phase I (2003): One-to-one Streaming
 Environment
o Mobile IPv4
o IEEE 802.11b
o Server: Darwin or Helix
Server
o Client – Notebook
o MPEG-2/4 streaming
Media Server
 Objectives
o Implement the
streaming server/client
that provides seamless
streaming (i.e., no
service disruption
during handoff).
FA
FA
Adaptive streaming client
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Implementation Phase II (2004): One-to-many Streaming
 Environment
o
o
o
o
o
Media Server
Mobile IPv6
IEEE 802.11b or 802.11a
Client Extension to PDA?
MPEG-2/4 streaming
Source Specific Multicast
(SSM)
 Objectives
o Implement the streaming
server/client providing
that provides seamless
multicast streaming.
Multicast
FA
Adaptive streaming client
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
FA
Mobility Challenge: Mobile IP
CN
 Smooth and Fast Handoff ?
Forwarding buffer
7
6
5
4
4
3
3
2
2
FAold
forwarding
Bind
4
FAnew
2
Output buffer
to FAnew
Handoff
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
3
Mobility Challenge: Handoff-aware
Streaming Client
 Adaptive buffer control
o To reduce the discontinuity of streaming
o Increasing the buffer level of client before handoff
 Question
o What is a proper buffer level?
o What is a handoff duration?
Handoff is initiated
Handoff is initiated
Buffer Level
Buffer Level
No playback
time
Handoff is completed
< No buffer level control >
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
time
Handoff is completed
< Buffer level control >
Mobility Challenge: Handoff-aware Streaming
Client (Cont.)
 Estimation of handoff duration by
adopting K-JIST’s Transient Time
Analysis
o
o
o
o
Collecting
parameters
Source traffic rate
Handoff load
Background traffic load
Beacon time period (router
advertisement)
Calculating
HTP
Estimating
Buffer Level
Bc: Current Buffer Level
Bt: Target Buffer Level
 Buffer level control
o Using the handoff duration (HTP) to
calculate the proper buffer level.
o Minimizing the buffer level
o Providing no media discontinuity
 No service disruption during handoff
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Bc<Bt
Increase
Buffer
Decrease
Buffer
K-JIST Transient Time Analysis: How long time
does Handoff take?
Source Traffic Turning Point
 Handoff Transient Time
Handoff
Initialization
Case 1
Old stream
New stream
New stream
Old stream
STP
HTP
to
tn
tf
Case 2
STP
UTP
HTP
Case 3
Old Router
New Router
UTP=Unstable Time Period
UTP
STP
HTP
STP=Silence Time Period
HTP=Handoff Time Period
HTP may be a good reference to the buffer level.
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
K-JIST Transient Time Analysis: Handoff Period
Time (HTP)
 Timing Analysis



UTP = max(0,Tlast_of_old- Tfirst_of_new)
= max(0, (to - tn) + qo + tf)
STP = min (Tfirst_of_old, Tfirst_of_new)
HTP = UTP +STP + |Tfirst_of_new – Tfirst_of_old|
 Queueing Model
qo
(a) Before handoff
(b) After handoff
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
K-JIST Transient Time Analysis: Analysis
Result
0.40
1.0
Priority: tco= tcn=50ms
FIFO: tco= tcn=100ms
Priority: tco= tcn=100ms
FIFO: tco= tcn=150ms
Priority: tco= tcn=150ms
FIFO: tco= tcn=200ms
Priority: tco= tcn=200ms
FIFO: Ton=1ms, BP=300
FIFO: Ton=10ms, BP=300
0.38
FIFO: Ton=1ms, BP=500
HTP(Second)
HTP (Second)
0.8
FIFO: tco= tcn=50ms
0.6
FIFO: Ton=10ms, BP=500
Priority: Ton=1ms
0.36
Priority: Ton=10ms
0.34
0.4
0.32
0.2
0.30
0
200
400
600
800
0
1000
FIFO: tco=tcn=50ms
20
25
30
FIFO: tco= tcn=50ms
3.5
Required Buffer Size for WLM
Priority: tco=tcn=50ms
FIFO: tco=tcn=100ms
HTP(Second)
15
4.0
1.3
0.9
10
Handoff Rate (the number of handoffs per second)
Background packet arrival rate (BP)
1.1
5
Priority: tco=tcn=100ms
FIFO: tco=tcn=200ms
Priority: tco=tcn=200ms
0.7
0.5
0.3
FIFO: tco= tcn=100ms
FIFO: tco= tcn=150ms
3.0
FIFO: tco= tcn=200ms
Priority: tco= tcn=200ms
2.5
2.0
1.5
1.0
0.5
0.1
0.0
0.0
0.1
0.2
0.3
tbeacon(Second)
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
0.4
0.5
0.6
100
300
500
700
Background Traffic (BP)
900
35
Seamless Streaming Aid by Adaptive Playout?
Merits of Client
Adaptation
 Role of Clients for Adaptive Playback
•Can prevent excessive
buffer overflow and
underflow
•Can reduce
discontinuous playback
time
•Can maintain small
buffer size.
Playback Speed
MAW
 p
MIN
an
cy
ps
Buffer level
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Bu
ffe
rO
cc
up
 p
Tim
e ( s)
Seamless Streaming : Network Adaptation
 Robust/Scalable Media for Wireless Media Delivery and
Associated Network Adaptation
Layered Video Encoding
Network Adaptation
R-D /
Corruption
R-D
Model
Video
preprocessing
Frame
Complexity
Analysis
Layered
Encoding
Quality
Control
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Layered
RPI
Delivery
Source
Rate/
Error
Resilience
Inform
Bandwidth
&
App.
feedback
With
Network
Adaptation
(Rate, Delay,
Loss)
Network
Monitoring
&
Feedback
Handling
Receiver
Wireless
Network
Video packet
Stream
Receiver
adaptation
Network
Feedback
End-to-End Feedback
Application Feedback
End-to-end QoS Challenge: Wireless LAN
 Relative Differentiated Services and the Proportional Differentiation Model
for IEEE 802.11x WLAN
o Service Differentiation : Upstream
 Different IFS (Inter Frame Space)
 Different Back-off Time
 Different Frame Size
o Service Differentiation : Downstream
 Scheduler in Access Point
qi
c
 i
qj
cj
q1
Scheduler
q2
Classifier
q3
AP
– qi : Performance measure for class I
– c1 < c2 <….< cN : Quality differentiation parameter
 Proportional Delay Differentiation Model
 Proportional Loss Differentiation Model
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Collaboration Issue for H-G Joint Projects
 With H. Morioka@ISIT, Fukuoka, Japan
 How to combine ISIT’s Fast and Secure Handoff Idea ? (Shortterm)
o With implementation: Deploy ISIT’s handoff at K-JIST and test its
performance for seamless streaming
o MISP (mobile Internet Service Protocol) for Authentication, Address
Assignment, and Encryption
Access Point
Access Point
Communication
 Joint Project (Long-term)
Scanning & Connection
WLAN Interfaces
Mobile Node ?
o Build ubiquitous computing environment
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
ISIT: Handover with 2 Interfaces (1)
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
ISIT: Handover with 2 Interfaces (2)
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
ISIT: Handover with 2 Interfaces (3)
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
ISIT: Fukuoka Mobile Broadband (FMBB)
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Thank You!
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Research Network in Korea
Korean Partner Carrier
KJCN
Japanese Partner Carriers
Fukuoka Gigabit Highway
GENKAI Project
QGPOP part
KIST
Korea
KISTI
Repeaterless
KJCN
KJIST
Busan
250 km
Gwangju
To Tokyo
GENKAI/Hyeonhae
Project
Shimonoseki
Univ. of Fishery
Kitakyushu City
High-speed LAN
GbE
Kitakyushu
150Mbps
2.4Gbps
Fukuoka
APAN
Internet2
QGPOP
(ISIT)
Japan
Kurume
FGH AP
Submarine cable landing station
NETWORKed
MEDIA
LAB. (Routers etc.)
Network
equipment
DEPT. OF INFO. & COMM., K-JIST
Nogata
Iizuka
Tagawa
Ohmuta