Transcript ESnet Update ESnet/Internet2 Joint Techs Albuquerque, New

ESnet Update
ESnet/Internet2 Joint Techs
Albuquerque, New Mexico
February 6, 2005
R. Kevin Oberman
ESnet Network Engineer
Lawrence Berkeley National Laboratory
1
Outline
 Physical Connectivity
o What’s new
o BAMAN in production
o LIMAN nearing production
o CHIMAN under construction
o New Circuits
 Traffic Driving:
o Interconnectivity
o Architecture
2
ESnet SF Bay Area
MAN Ring
NLR to Seattle
• 2 λs (2 X 10 Gb/s channels)
in a ring configuration, and
delivered as 10 GigEther
circuits
- 10-50X current site
bandwidth
Joint
Genome
Institute
10 Gb/s
optical channels
λ1 production IP
LBNL
λ2 SDN/circuits
NERSC
λ3 future
• Dual site connection
(independent “east” and
“west” connections) to each
site
λ4 future
SF Bay
Area
• Used as a 10 Gb/s
production IP ring and
2 X 10 Gb/s paths (for
circuit services) to each site
• Project completed 9/2005,
on time and on budget
IP core to
Chicago (Qwest)
LLNL
SNLL
SLAC
DOE Ultra
Science Net
(research net)
Level 3
hub
Qwest /
ESnet hub
ESnet MAN
ring (Qwest
circuits)
ESnet hubs
and sites
NLR to San
Diego
NASA
Ames
IP core to El Paso
Starlight
Long Island MAN Initial Configuration
CERN
Chicago
(Qwest hub)
Europe
GEANT
USLHCnet
MAN LAN
Abilene
NYSERNet
SINet (Japan)
CANARIE (Canada)
HEANet (Ireland)
Qatar
Brookhaven
National Lab, Upton, NY
ESnet
boarder
demarcation
device
10GE
circuit
LI MAN DWDM
Cisco 6509
32 AoA, NYC
T320
ESnet
IP core
1 GE or
10 GE
BNL
IP gateway
ESnet IP core
(Qwest)
Washington
(Qwest
hub)
10 Gb/s circuits
International
LHCnet circuits (proposed)
production IP core
SDN/provisioned virtual circuits
4
Chicago MAN
USLHCnet
T320
4x10G
T320
Starlight
455 N. City Front
3x10G
ESnet IP
SDN
Fermilab
Argonne
5
New 10G Circuits
• ESnet is nearing completion of a process to turn up 44 new
10G circuits in less than 12 months including:
o
MANs
- 16 circuits composing the San Francisco Bay Area MAN
- 2 circuits comprising the Long Island MAN (New York to Brookhaven)
- 15 circuits comprising the Chicago MAN (FNAL, ANL and StarLight)
o
NLR
- 3 circuits: San Diego to Sunnyvale, Sunnyvale to Seattle, hub to hub in
Sunnyvale
o
Sites
- 1 circuit JLAB (via, MAX/NGIX-E, MATP, E-LITE)
- 1 circuit between NYC and DC to reach GEANT
- 1 circuit ORNL – Starlight - CHI-HUB(NBC)
o
Peering
- 1 circuit GEANT
- 1 circuit SNV – NASA-AMES
6
Other Circuits
•
•
Completed:
o
OC48 ALB-ELP
o
DS3 IARC-SNV (New Site)
o
OC3 ALB-KCP
In the works:
o
OC3 ALB-HUB-PANTEX <-- In build-out
7
The Office of Science U.S. Community*
Pacific Northwest
National Laboratory
Idaho National
Laboratory
Ames Laboratory
Argonne National
Laboratory
Fermi
National
Accelerator
Laboratory
Lawrence
Berkeley
National
Laboratory
Brookhaven
National
Laboratory
Stanford
Linear
Accelerator
Center
Lawrence
Livermore
&
Sandia
National
Laboratories
Princeton
Plasma
Physics
Laboratory
Thomas Jefferson
National
Accelerator Facility
General
Atomics
Oak Ridge
National
Laboratory
Sandia
National
Laboratory
Institutions supported by SC
Major User Facilities
DOE Specific-Mission Laboratories
DOE Program-Dedicated Laboratories
DOE Multiprogram Laboratories
Los Alamos
National
Laboratory
National
Renewable Energy
Laboratory
SC Program sites
8
NLR Supplied Circuits
ESnet’s Physical Connectivity (Spring 2006)*
Japan (SINet)
Australia (AARNet)
Canada (CA*net4
Taiwan (TANet2)
Singaren
CA*net4
France
GLORIAD
(Russia, China)
Korea (Kreonet2
MREN
Netherlands
StarTap
Taiwan (TANet2)
UltraLight
GÉANT
- France, Germany,
Italy, UK, etc
SINet (Japan)
Russia (BINP)
CERN
(USLHCnet
CERN+DOE funded)
LIGO
PNNL
AU
MAN rings
ESnet IP core
MIT
USN
JGI
LVK
LLNL
SNLL
LBNL
NERSC
SLAC
PAIX-PA
Equinix, etc.
NASA
Ames
USN
YUCCA MT
NREL
42 end user sites
Office Of Science Sponsored (22)
NNSA Sponsored (12)
Joint Sponsored (4)
Other Sponsored (NSF LIGO, NOAA)
Laboratory Sponsored (6)
commercial and R&E peering points
ESnet core hubs
MAE-E
Equinix
KCP
JLAB
OSTI
LANL
GA
PPPL
ORAU DC
LBNL DC
LLNL/LANL
DC Offices
OSC GTN
NNSA
IARC
AU
AMPATH
FNAL
ANL
AMES
BNL
ARM
SNLA
ORNL
ORAU
NOAA
SRS
AMPATH
(S. America)
ESnet IP core: Packet over
SONET Optical Ring and Hubs
high-speed peering points with Internet2/Abilene
International (high speed)
Lab Supplied
10 Gb/s SDN core
10G/s IP core
2.5 Gb/s IP core
MAN rings (≥ 10 G/s)
OC12 / GigEthernet
OC3 (155 Mb/s)
45 Mb/s and less
9
Network Utilization
ESnet Accepted Traffic (Bytes)
Jan, 1990 to Dec., 2005
F
e
b
,
0
5
A
u
g
,
0
2
F
e
b
,
0
0
A
u
g
,
9
7
F
e
b
,
9
5
A
u
g
,
9
2
800
750
700
650
600
550
500
450
400
350
300
250
200
150
100
50
0
F
e
b
,
9
0
TBytes/Month
ESnet is Currently Transporting About 750 Terabytes/mo.
and this volume is increasing exponentially
10
0.0
Aug, 05
1000.0
Feb, 05
Aug, 04
Feb, 04
Aug, 03
Feb, 03
Aug, 02
Feb, 02
Aug,01
Feb, 01
Aug, 00
Jul., 1998
Feb, 00
Aug, 99
Feb, 99
Aug, 98
Feb, 98
Aug, 97
Feb, 97
Aug, 96
Feb, 96
Oct., 1993
Aug, 95
Feb, 95
Aug, 94
Feb, 94
Aug, 93
Feb, 93
Aug, 92
Aug., 1990
Feb, 92
Aug, 91
Feb, 91
Aug, 90
Feb, 90
TBytes/Month
Network Utilization
ESnet traffic has increased by about 10X every 46 months, on average,
since 1990
Dec., 2001
R2 = 0.991
42 months
100.0
57 months
10.0
39 months
1.0
0.1
11
OSCARS: Guaranteed Bandwidth VC Service For SC Science
•
ESnet On-demand Secured Circuits and Advanced Reservation System
(OSCARS)
•
To ensure compatibility, the design and implementation is done in collaboration
with the other major science R&E networks and end sites
o
Internet2: Bandwidth Reservation for User Work (BRUW)
- Development of common code base
o
GEANT: Bandwidth on Demand (GN2-JRA3), Performance and Allocated Capacity for
End-users (SA3-PACE) and Advance Multi-domain Provisioning System (AMPS)
- Extends to NRENs
BNL: TeraPaths - A QoS Enabled Collaborative Data Sharing Infrastructure for Petascale Computing Research
o GA: Network Quality of Service for Magnetic Fusion Research
o SLAC: Internet End-to-end Performance Monitoring (IEPM)
o USN: Experimental Ultra-Scale Network Testbed for Large-Scale Science
o
•
In its current phase this effort is being funded as a research project by the Office
of Science, Mathematical, Information, and Computational Sciences (MICS)
Network R&D Program
•
A prototype service has been deployed as a proof of concept
To date more then 20 accounts have been created for beta users, collaborators, and
developers
o More then 100 reservation requests have been processed
o
12
Observation
•
•
In 4 years, we can expect a 10x increase in traffic
over current levels without the addition of production
LHC traffic
o
Nominal average load on busiest backbone links is
~1.5Gbps today
o
In 4 years that figure will be ~15Gbps if current trends
continue
Measurements of this kind are science-agnostic
o
•
It doesn’t matter who the users are, the traffic load on the
backbone is increasing
Bandwidth trends drive requirement for a new
network architecture
13
ESnet Target Architecture:
IP Core+Science Data Network Core+Metro Area Rings
Asia-Pacific
Canada
(CANARIE)
Canada
(CANARIE)
Europe
(GEANT)
CERN
CERN
Europe
(GEANT)
Australia
SDN Core
Aus.
Sunnyvale
New York
Denver
Metropolitan
Area Rings
IP Core
Loop off
Backbone
Washington,
DC
LA
San Diego
IP core hubs
South America
(AMPATH)
Primary DOE Labs
SDN hubs
possible hubs
Albuquerque
South America
(AMPATH)
10-50 Gb/s circuits
Production IP core
Science Data Network core
Metropolitan Area Networks
International connections
14