Transcript AARnet-SX Transport IEEAF Link

Translight/PacificWave
John Silvester
Executive Director of Center for Scholarly Technology
Professor or Computer Engineering, USC
Chair, CENIC
NSF OCI-IRNC Cooperative Agreement [#0441119]
Briefing to NSF-OISE, April 28, 2006
Translight/PacificWave (TL/PW)

Cooperative Agreement [#0441119] under NSF
IRNC Program
 PI John Silvester (USC); Co-PI Ron Johnson
(UW)
 Other Key Partners





CENIC, Jim Dolgonas
PNWG, Jacqueline Brown
University of Hawaii, David Lassner
AARnet, George McLaughlin
Approx $1M per year for 5 years
 Awarded March 1, 2005
TL/PW Primary Objectives







Facilitate international R&E connections on US Pacific
Coast
Assist in the termination (equipment, backhaul) of
AARnet SXTransport links to Hawaii, Seattle and Los
Angeles
Assist in build-out of Hawaii connectivity to Telescopes
Further develop PacificWave exchange capabilities
Assist in operation of IEEAF link Tokyo-Seattle
Assist in operation and support of other International
R&E networks participating in PW
Develop and operate advanced capabilities to support
optical interconnect and exchange needs of R&E
networks
What is Pacific Wave?

Pacific Wave is a state-of-the-art international
peering facility designed to serve advanced
research and education.

Primary focus is interconnectivity among US
R&E networks and International R&E networks
from the Pacific Rim

One of the objectives was the desire for all
participants to be able to peer with each other
which led to the “distributed exchange” design
Context of Pacific Wave




The US States on the Pacific Coast [Alaska,
Washington, Oregon, California, Hawaii] have
collaborated in networking for several years. (e.g. US
Pacific Consortium is an affiliate of APAN)
Seattle, Los Angeles, and Sunnyvale are “natural” places
for exchange points
Many international links enter the US through
Washington and California (and to a certain extent
Hawaii although they tend to remain local there)
Led to the development of Pacific Wave – a collaboration
between CENIC and PNWG to build out exchange
capabilities on West Coast.
Who Operates Pacific Wave?
A joint project of CENIC and Pacific Northwest Gigapop
In collaboration with University of Southern California
and University of Washington
Pacific Wave Layer 2 Exchange










Extensible peering exchange to allow layer 3 networks to peer easily
over a layer 2 fabric
Nodes (currently) in Seattle, Sunnyvale, and Los Angeles.
Connected by a 10GbE wave provisioned over National LambdaRail
(2,241 kilometers)
AUP free
Supports IPv4 and IPv6; multicast enabled
Provides 24x7 NOC support.
Priced consistently from node to node
Allows participants to self-select their peering
Allows participants to connect to one-location and access
participants at all Pacific Wave nodes
Supports advance applications
Welcomes any research or development network that can meet the
minimum network configuration requirements (connect with a router;
support BGP; support jumbo-frames)
International Participation
“Layer 3” R&E Networks and Pacific Wave
CA*NET4
AARNET
GEMNET
GLORIAD
IEEAF
KREONET
MIMOS (Malaysia)
QATAR
SINET
SINGAREN
TANET/TWAREN
TRANSPAC2
Pacific
Wave
CLARA
CUDI
ABILENE (Internet2)
CALREN (HPR)
DREN ESNET
NASA
NLR-Packetnet
PNWG
PacificWave Seattle
NLR-Packetnet
NLR Lambda to
STARLIGHT
AARNet
Abilene
ATTBI/Comcast
TANET2/TWAREN
CA*NET4
SingAREN
DREN
ESnet
PNWGP
Seattle Westin
CISCO 6509
Microsoft
GEMnet
KREONet2
WIDE/TLEX
(IEEAF)
1 Gigabit
NLR Lambda to LAX
(via Sunnyvale)
Does not show “lightpath” connections
10 Gigabit
Pacific Wave - Sunnyvale
NLR TO
SEATTLE PWAVE
ESNET
PAIX
L3 SVL
6509
CENIC SVL HPR
NLR TO
LA PWAVE
1 Gigabit
Does not show “lightpath” connections
10 Gigabit
Pacific Wave - Los Angeles
10 GE NLR Lambda
To Seattle PWave
(via Sunnyvale)
Mimos Berhad
(Malaysia)
Abilene
Qatar Foundation
1 Wilshire
6509
Los Nettos
818 W 7th
6509
600 W 7th
6509
(3)
Singaren
NII/SINET
TRANSPAC2
TWAREN
Cinegrid
AARNet
NLR
Packetnet
CalREN-HPR
1 Gigabit
Does not show “lightpath” connections
10 Gigabit
Network Topology
The VLAN Architecture
IRNC TL/PW Pacific Connections
CA*Net4 POP
PW-Seattle
TλEX Tokyo
PW-LA
Oahu
IEEAF Link
Hawaii
AARnet-SX Transport
AARnet
POP Sydney
CLARA, CUDI
POPs (Tijuana)
TL-PW – West Coast Detail
NLR link
SEA
Hillsboro
(OR)
AARNet-SX(N)
To Kahe Point (HI)
PW-Seattle
(NLR)
Portland
(OR)
Pacific Wave
N-S link
SNV
Los Osos
(CA)
(CalREN-XD)
SLO
(CA)
AARNet-SX(S)
To Spencer Beach (HI)
Santa Barbara
(CA)
Optics to carry
new lambda
over CalREN-XD
from SLO to LA
LAX
PW-Los Angeles
TL/PW – Hawaii Detail
SCCN
OC-192
Kahe Point
(Oahu)
AARNet-SX(N)
To Portland
Hawaii Gigapop
(Oahu)
AARNetSX(S)
To Sydney
Spencer Beach
(Big Island)
AARNet-SX(S)
To Sydney
SCCN
OC-192
AARNet-SX(S)
To Los Osos
Waimea
Hilo
Mauna Kea
MKOCN (Summit)
Mauna Kea Observatories Communication Network
CSO
Gemini
JCMT
UKIRT
SMA
IRTF
Subaru
UH-2.2
UH-0.6
CFHT
Keck
VLBA
56kb
Verizon Summit
UH-Managed Shared
Network for All Observatories
Hilo Base Facilities
VLBA
JAC
Waimea Base Facilities
DS3
CSO
Verizon Hilo
Ke
ck
Verizon
Humuula
CFHT
Verizon
Waimea
Gemini
UH Hilo Campus
UNIVERSITY
SMA
UH IfA
Hilo
Subaru
Leased Private Links
Carrier Infrastructure
for leased shared & private links
Leased DS3 (shared)
UH managed fiber links
Proposed New GbE
Status of TL/PW as of April 2006

Engineering and technical support through
PNWG and CENIC for all International
connections into Pacific Wave
 Northern AARnet Link


Operational and in use
Southern AARnet Link




Backhaul from Los Osos through San Luis Obispo to
PW-LA over CALREN as OC-192 in place
ETE equipment for HI and LA on order (arriving as we
speak).
Local connectivity in Hawaii – in process
Expected operation (test mode) end of May
Changing Requirements for
Exchanges

The Exchange points need to provide a wider
array of services:






Layer 3 Interconnect (routed)
Layer 3 Interconnect (over layer 2 switch)
Layer 2 Interconnect (Ethernet switching)
Layer 1 Interconnect (wave switching)
Layer 0 Interconnect (fiber interconnect)
PacificWave is moving in this direction to allow
us to continue to participate in the world of
Global R&E networking
Pacific Wave Lightpath Interconnect





Primary focus has been on Layer 3 interconnect
There is a lot of interest in Layer 2 (“lightpaths”)
and Layer 1 (“waves” and “lambdas”)
interconnect. [GLIF, etc]
SXTransport(S) will be configured as 8x1GE
PW is deploying additional OC192 from LA to
SEA configured as 8x1GE to allow interconnect
for AARnet and other nets as demand grows.
Also plans in place to extend PW connectivity to
Starlight (Chicago) to further enhance layer 3
and lightpath interconnect capability
Pacific Wave Lightpath Exchange
To Canarie
NLR To Chicago and Starlight
IEEAF to TλEX
SEA
HOPI
SNY
HOPI
AARnet to AU, via HI
LA
NLR To Phoenix
San Diego
Lightpath Networks and PW
CA*NET4
NLR Waves
NLR (Layer 2)
TRANSPAC (?)
AARNET (S)
(to TλEX) IEEAF
Pacific
Wave
CALREN-XD
HOPI (Internet2)
PNWG
Others in the planning stages
to SLO
AARNet S Path
PW Lightpath Interconnect
to SNY, SEA
PW-L2X (8x1G)
LAX-SNY-SEA
CENIC 15808
PW 15454
CENIC 15808
6509
AARnet
Router
Abilene
Singaren
Twaren
6509
818
6509
NLR Packetnet
PW-L3X:10G
LAX-SNY
CalREN
Router
PW-L3X-LA
CENIC 15808
to SD
Some Current Projects
Activities
 Basically
all International R&E traffic to
Pacific Rim flows through PW
 Recent





Highlights
iGRID
SC’05
PRAGMA
APAN
INTERNET2 meetings
Examples

January 2005: Huygens Titan probe data transfer:
3Tbytes of satellite data distributed and collated for
analysis to locations in Amsterdam and Australia
 January 2005: Tele-microscopy demo between
University of Hawaii and UCSD (Lariat network)
 January 2005: PTC Honolulu 270Mbps interactive HD
video conferencing with Australia and Seattle
 January 2005: Seattle—Tokyo, uncompressed HDTV 1.5
Gbps Smarr HD lecture between Seattle and JGN
meeting
 June 2005: Chicago-Seattle-UCSD OptiPuter HD
permanent interactive uncompressed HDTV videoconference link
Examples – iGRID05

4k interactive HD between Tokyo & SD
 Concurrent n-way real-time low-latency uncompressed
1080 line (ie >1gb/s per stream) HDTV teleconferencing
between Australia, Japan, Korea, Ann Arbor, Seattle and
San Diego
 100 megapixel remote viz. Illinois-SD
 Real-time flat-panel stereo, remotely rendered
 Clusters in Chicago & Amsterdam loosely coupled
compute demo: performance = to single machine room
alternative!!
 Neptune/‘Looking’ real-time HDTV from 2km below the
surface showing volcanic vents in the northeast pacific
Examples

November 14-17, 2005: SC05 Seattle. About 50 10gb/s
waves, petabytes of storage, leading edge visualization
systems and teraflops of computing assembled for a
week computational sciences etc conference for this.

‘Persistent’ OptIPuter

Wide Area Storage GRIDs

The Next Disaster Recovery Architectures that really
work!?

Neptune Canada ‘in the water’
“Neptune” Ocean Observatories
‘Dry’ Headend – Nodes & Access points
Recent Stereo Video

I2 Spring MM from Stanford Medical School (Margaret
Krebs, Testbed Mgr, Stanford University Medical Media
and Information Technologies) ran a stereo video
program test with CSIRO in Sydney on March 30th
via Abilene and SX.
 They ran about a 90Mbps stream for the test.
No problems through AARnet N and across AARnet.
iGrid 2005, San Diego Sept.
2005

iGrid 2005, Over 10 Gbps bidirectional traffic coexisted
with production exchange traffic without detriment



4k line interactive Super-HD between Keio University and UCSD
Live HD from the sea floor 100 miles off the Pacific Coast
N-way uncompressed multicast HD video conferencing
ResearchChannel N-Way HD
Multicast Video Conferencing
Does it Really Perform in
Practice? …Yes!

7.5 Gbps of unicast
HDTV over IP traffic
(10 750 Mbps flows)

3 Gbps of multicast
replicated in Seattle
switch node using
PIM snooping

Traffic test performed
in Cisco POC lab
before the event
An Amazing Feat at SC05
47 10 Gbps Lambdas to be
Exact
HDTV Conferencing Spanning Two
Oceans Enabled by Pacific Wave
Traffic Flows at Supercomputing
 Among
other things, 5-way multicast
replication of 3 Gbps, over 12 Gbps total
traffic across the switch backplanes
Thank you
John Silvester, [email protected]