Transcript (slowly) Towards Production Mode, Jon MacLaren

asdb
Challenges in Moving (slowly) Towards
Production Mode
Jon MacLaren
GHPN-RG at OGF20, Manchester, UK
9 May 2007
NSF seed-funded project
EnLIGHTened Introduction
•
•
•
•
Network research, driven by concrete application projects, all of which
critically require progress in network technologies and tools that utilize them
EnLIGHTened testbed: 10 Gbps optical networks running over NLR and
Louisiana Optical Network Initiative (LONI), connected via four all-photonic
Calient switches, all using GMPLS control plane technologies
Global alliance of partners
Will develop, test, and disseminate advanced software and underlying
technologies to:
– Provide generic applications with the ability to be aware of their network, Grid
environment and capabilities, and to make dynamic, adaptive and optimized use
(monitor & abstract, request & control) of networks connecting various high end
resources
– Provide vertical integration from the application to the optical control plane,
including extending GMPLS
•
Will examine how to distribute the network intelligence among the network
control plane, management plane, and the Grid middleware
CENTER FOR COMPUTATION & TECHNOLOGY AT LOUISIANA STATE UNIVERSITY
EnLIGHTened testbed connectivity diagram, with partners
To Canada
To Asia
To Europe
SEA
POR
BOI
CAVE wave
OGD
EnLIGHTened wave
(Cisco/NLR)
DEN
KAN
SVL
PIT
CHI
CLE
WDC
Cisco/UltraLight wave
LONI wave
San Diego
TUL
VCL @NCSU
DAL
Members:
- MCNC GCNS
- LSU CCT
- NCSU
- RENCI
Official Partners:
- AT&T Research
- SURA
- NRL
- Cisco Systems
- Calient Networks
- IBM
HOU
NSF Project Partners
- OptIPuter
- UltraLight
- DRAGON
- Cheetah
International
Partners
•Phosphorus - EC
•G-lambda - Japan
-GLIF
CENTER FOR COMPUTATION & TECHNOLOGY AT LOUISIANA STATE UNIVERSITY
Applications
Edge
Routers
Workflow
Engines
Application Abstraction Layer (API)
Translate app request to policy
Policy
Resource Manager
Co-Scheduler
Feedback Loop
Resource
Allocation
For SLA
Monitoring
Abstraction
Resource
Monitoring
•Discovery
•Performance
•Policy
Policy
HARC: Highly Available Resource Co-allocator
• Extensible, open-sourced co-allocation system
• Can already reserve:
– Time on supercomputers (advance reservation), and
– Dedicated paths on GMPLS-based networks with simple topologies
• Uses Paxos Commit to atomically reserve multiple resources, while
providing a highly-available service
• Used to coordinate bookings across
EnLIGHTened and G-lambda testbeds
in largest demonstration
of its kind to date (more later)
• Used for setting up the
network for Thomas Sterling’s
HPC Class which
goes out live in
HD (more later)
CENTER FOR COMPUTATION & TECHNOLOGY AT LOUISIANA STATE UNIVERSITY
Network Components
CENTER FOR COMPUTATION & TECHNOLOGY AT LOUISIANA STATE UNIVERSITY
NRM View of Network
CENTER FOR COMPUTATION & TECHNOLOGY AT LOUISIANA STATE UNIVERSITY
Scheduling a link
•
•
•
•
•
•
Scheduling is only one part of the puzzle
Input is a request to make a connection from one endpoint to another, e.g.
RA1 to BT2
The NRM “decides” the path that this will use
Currently it does this by looking up the path in a table
Sends the Explicit Route Object (ERO) as part of the TL1 command
So the middleware is deciding the path, not GMPLS
•
XML snippet:
<Work>
<Path>
<From>RA1</From><To>BT2</To>
<BandwidthMbs>10240</BandwidthMbs>
</Path>
</Work>
CENTER FOR COMPUTATION & TECHNOLOGY AT LOUISIANA STATE UNIVERSITY
Resource Map
CH1
(SL)
KMF
FUK
VC1
(NCSU)
RA1
(MCNC)
BT1
(LSU)
Pelican
Back
up
TKB
6509
(UR1)
0.11a.6
(UO2)
(UO1)
4G
KAN
X1N 5G 5G
X2N
2G
(UO3)
X1U
BT2
(LSU)
Santaka
0.11a.2
(UR2)
(UO4)
X1
Japan North
0.11a.7
US
10.16a.2
(UR3)
X1S
X2
LA
Foundry
X2S
AKB
OSA
Japan South
NR3
BT3
(LSU)
Viz
Machine
Client
LA1
(Caltech)
Credit: Tomohiro Kudoh
KHN
CENTER FOR COMPUTATION & TECHNOLOGY AT LOUISIANA STATE UNIVERSITY
EL-GL Middleware Interoperability
Japan
Application
EL
Grid Resource
Coordinator
GNS-WSI
KDDI
NRM
NTT
NRM
GL→EL
GNS-WSI
wrapper
HARC
Acceptor
US
GL→EL
CRM
wrapper
JAPAN
GL
Grid Resource
Scheduler
US
Application
EL
App. Launcher
EL→GL
wrapper
EL
NRM
CRM
CRM
CRM
CRM
CRM
CRM
CRM
CRM
Cluster
Cluster
Cluster
Cluster
Cluster
Cluster
Cluster
Cluster
Credit: Tomohiro Kudoh
GL: G-lambda
EL: Enlightened Computing
CRM: Compute Resource Manager
HARC: Highly-Available Resource Co-allocator
GNS-WSI: Grid Network Service-Web Services Interface
NRM: Network Resource Manager
CENTER FOR COMPUTATION & TECHNOLOGY AT LOUISIANA STATE UNIVERSITY
RNDS Display of G-lambda Reservations
Credit: Tomohiro Kudoh
CENTER FOR COMPUTATION & TECHNOLOGY AT LOUISIANA STATE UNIVERSITY
G-Lambda and EnLIGHTened
GMPLS E-NNI Demonstrations
• Collaborative effort between NTT, KDDI Research and EnLIGHTened
Computing
– Goal: to investigate potential for interdomain provisioning
• GLIF2006 and SC06 - demonstrated single-vendor interoperation
between Japan North (KDDI Research) and Enlightened
– Automated simultaneous in-advance reservation of network bandwidth
between the US and Japan, and computing resources in the US and
Japan
– World’s first inter-domain coordination of resource mangers for in-advance
reservation
– Resource managers have different interfaces and are independently
developed
• December 2006 - tested three domain multi-vendor provisioning
between Japan South (NTT) and Enlightened, with Japan North as
transit domain.
•
•
GL’s GNS-WSI between the Grid Resource Scheduler (GRS)
and the Network Resource Manager (NRM)
EL’s Highly-Available Resource Co-allocator (HARC)
uses Paxos Commit to reserve resources
CENTER FOR COMPUTATION & TECHNOLOGY AT LOUISIANA STATE UNIVERSITY
HD Class
• Spring 2007 - HPC Class
• Thomas Sterling, LSU, instructor
• Students at LSU (2 sites), LA Tech, University of Arkansas,
Masaryk University
• Simulcast as HDVC, Access Grid, QT Streaming (NCast), Webex
(graphics only), iChat (text only for inbound messages)
• Broadcast streams
• 1:N HD video broadcast from LSU (each 1.5 Gbps)
• N:1 HD video broadcast to LSU
• N:N audio distribution
• Uses EnLIGHTened middleware to build and tear-down network
twice weekly
• System developed as collaboration between CCT/LSU and
Masaryk Univ. (Brno, Czech Republic)
CENTER FOR COMPUTATION & TECHNOLOGY AT LOUISIANA STATE UNIVERSITY
HD Video over IP Transmission System
•
Packetization specified in RFC 4175 (“RTP Payload Format for Uncompressed
Video”)
– encapsulation: payload/RTP/UDP/IP
– augmenting common RTP headers with payload headers (e.g., additional packet
numbers because of fast RTP counter wrap-arounds – 0.5 s for HD-SDI)
•
Reasonable to use jumbo frames (best > 8500B)
– decreases packetization size overhead
– decreases host load due to decreasing number of pps
•
•
Linux-based implementation
Comes from UltraGrid by Perkins & Gharai
– extended to support full-HD 1080i
– support for SW display including color space down-sampling and eld de-interlacing
(assembly optimized for AMD)
– other enhancements
•
Used with DVS Centaurus HD capture cards
– problems with latency, since the card doesn’t support DMA and requires buffering at
least 4 fields for reliable operation
– quite expensive
– there are other cards but not supported in Linux :(
•
End-to-end (camera-to-display) latency: 175±5 ms
CENTER FOR COMPUTATION & TECHNOLOGY AT LOUISIANA STATE UNIVERSITY
HDTV Video Distribution
• Based on UDP packet reflectors
– designed as user-empowered solution
– relatively scalable with respect to number of users
• Simple design for HDTV video distribution
– read from network socket, write to other network sockets
(loop)
– optimizations for 1.5 Gbps streams - reduced per-packet
overhead (system calls/packet)
– can multiply the 1.5 Gbps stream for 4 users (limited by
the network card performance)
– latency increase as low as 13 ms
CENTER FOR COMPUTATION & TECHNOLOGY AT LOUISIANA STATE UNIVERSITY
HDTV Video Class Advantages
• Low network latencies
– Measurements based on ICMP ping averages with 8500 B
packets:
• LSU – StarLight: 30.631 ms
• StarLight – Masaryk University: 115.481 ms
• LSU – Masaryk University: 145.720 ms
• Multi-way interaction
• Quality far exceeds Access Grid
– “you can see the students’ facial expressions”
CENTER FOR COMPUTATION & TECHNOLOGY AT LOUISIANA STATE UNIVERSITY
HDTV Video Distribution
CENTER FOR COMPUTATION & TECHNOLOGY AT LOUISIANA STATE UNIVERSITY
Configuration Changes...
CENTER FOR COMPUTATION & TECHNOLOGY AT LOUISIANA STATE UNIVERSITY
The EnLIGHTened Team
•
•
•
•
•
•
•
•
•
•
Yufeng Xin
Steve Thorpe
Bonnie Hurst
Joel Dunn
Gigi Karmous-Edwards
Mark Johnson
John Moore
Carla Hunt
Lina Battestilli
Andrew Mabe
•
•
•
•
•
•
•
•
•
•
Ed Seidel
Gabrielle Allen
Seung Jong Park
Jon MacLaren
Andrei Hutanu
Lonnie Leger
Dan Katz
Savera Tanwir
Harry Perros
Mladen Vouk
•
•
Olivier Jerphagnon
John Bowers
•Steven Hunter
•Dan Reed
•Alan Blatecky
•Chris Heermann
•
•
•
•
•
Javad Boroumand
Russ Gyurek
Wayne Clark
Kevin McGrattan
Peter Tompsu
•Rick Schlichting
•John Strand
•Matti Hiltunen
•Yang Xia
•Xun Su
CENTER FOR COMPUTATION & TECHNOLOGY AT LOUISIANA STATE UNIVERSITY