Transcript Network - SPRACE

SPRACE KyaTera / UltraLight Proposal
VI D0SAR Workshop
São Paulo, Brazil
September 16, 2005
Rogério L. Iope
Universidade de Sao Paulo
(Grad. Research Assistant for SPRACE)
e-Science: Data Gathering, Analysis, Simulation, Collaboration
 Scientific discoveries increasingly driven by data collection
•
•
•
•
Computationally intensive analyses
Massive data collections
Data distributed across networks of varying capability
Internationally distributed collaborations
 New approaches to enquiry based on
•
•
•
•
Deep analysis of huge quantities of data
Interdisciplinary collaboration
Large-scale simulation
Smart instrumentation
 e-Science methods no longer optional but
now vital to scientific competitiveness
e-Science: Driving Global Cyberinfrastructure
CMS
TOTEM
ALICE : HI
ATLAS
LHCb: B-physics
 e-Science is about providing significantly enhanced research infrastructure
by utilizing distributed resources such as computers, storage devices,
scientific instruments, and experts using information technology
e-Science: Driving Global Cyberinfrastructure

The enormous speedups of computers and networks have enabled simulations of
far more complex systems and phenomena, as well as visualizing the results from
many perspectives

Advanced computing no longer restricted to a few research groups in a few fields,
but pervades scientific and engineering research

New data-intensive applications are driving seemingly insatiable demand for more
bandwidth

Groups collaborate across institutions and time zones, sharing data,
complementary expertise, ideas, and access to special facilities without traveling

Optical Networks are key to this vision
•
•
•
Massive scalable bandwidth
Protocol and bit-rate independence
The ability to launch and scale new services on demand
 Photonic Networking: the way to cope with IP traffic explosion
Overview of the UltraLight Project
 UltraLight is
• A collaboration of experimental physicists, computer scientists, and network
engineers from BNL, Caltech, CERN, UF, FIU, FNAL, Internet2, UM, MIT, SLAC...
• …to provide the network advances required to enable petabyte-scale analysis
of globally distributed data
• An application-driven network R&D program to explore the integration of
cutting-edge network technology with the Grid computing and data
infrastructure of HEP/Astronomy
• A non-standard core network with dynamic and varying bandwidth
interconnecting globally distributed nodes
• An NSF-funded 4 year program to deliver a new, high-performance, networkintegrated infrastructure
 Two primary, synergistic activities (source: S. McKee)
• Network “Backbone”: Perform R&D / engineering
• Application “Driver”: System Services R&D / engineering
Overview of the UltraLight Project
 Main goals
• Engineer and operate a trans- and intercontinental optical network testbed
• Promote the network as an actively managed component
• Develop and deploy prototype global services which broaden existing Grid
computing systems
• Enable physics analysis and discoveries by integrating and testing UltraLight
in Grid-based physics production and analysis systems currently under
development in ATLAS and CMS
 A three-phased plan
• Phase 1 (12 months): Implementation of network, equipment and initial
services
• Phase 2 (18 Months): Integration and footprint expansion
• Phase 3 (18 Months): Transition to production (LHC physics + eVLBI
astronomy)
Overview of the UltraLight Project
 Project Management Team
• PI: Harvey Newman (Caltech)
•
•
•
•
•
•
Project Coordinator: Rick Cavanaugh (UF)
Network Coordinator: Shawn McKee (UM)
Applications Coordinator: Frank van Lingen (Caltech)
Education&Outreach Coordinator: Laird Kramer (FIU)
Physics Analysis User Community Coordinator: Dimitri Bourilkov (UF)
“Wan-In-Lab”: Steven Low (Caltech)
 Project Coordination activities
•
•
•
•
•
Regularly scheduled phone and video meetings
Periodic face-to-face focus workshops (semi-annually or quarterly)
Persistent VRVS room for collaboration
Mail-lists
Web-page portal (first prototype)
Overview of the UltraLight Project
 Some important UltraLight R&D goals
•
•
•
•
•
•
•
•
•
•
•
Basic Network Services
Data transport protocols
MPLS/QoS Services and Planning
Optical Path Management Plans
Optical Testbed
Optical Exchange Point
Network Monitoring
Network Management and AAA
Disk-to-disk data transfers
Wan-In-Lab / DISUN
HEP Application Services
Connectivity Diagram for UltraLight
Source: http://ultralight.caltech.edu/
The KyaTera Project
 A cooperative program proposed by FAPESP, as part of the TIDIA Program
 Main goal:
 The establishment of an optical fiber network infrastructure connecting
laboratories for research, development and demonstrations of technologies for
advanced Internet applications
 Network infrastructure based upon the concept of dark fibers reaching
directly to the research laboratories (FTTLab)
 The name KyaTera comes from
• Kya (“net” in Tupi-Guarani)
• Tera (greek teras = monster)
The KyaTera Project
 Composed by a dark fiber mesh spread over several cities among the State
of São Paulo
• A large, geographically distributed laboratory facility for experimental tests of new
network concepts and optical devices, new network protocols and services
• A platform for developing and deploying new high performance e-Science
applications
 A stable, high performance network always co-exists with the experimental
network
• new developments in the last do not interfere with the operation of the first
The KyaTera Project
 Research subjects for KyaTera organized in 3 layers
• Physical Layer
 optical communications, new developments on fiber infrastructure
• Transport Layer
 protocols, interface standards, maanagement, monitoring, interoperability,
etc, in optical networks
• Applications Layer
 automation and computer control of scientific instruments, Grid
applications, HDTV, etc
WDM Fundamentals
 Wavelength-Division Multiplexing – WDM
• An approach that can exploit the huge bandwidth available on fiber optic links
• Can manyfold the capacity of existing networks by transmitting many channels
simultaneously on a single fiber optic line
• The optical transmition spectrum is carved up into a number of nonoverlapping wavelength (or frequency) bands
• Multiple WDM channels from different end-users may be multiplexed on the
same fiber
 Each wavelength supports a single communication channel operating at
peak electronic speed
 By allowing multiple WDM channels to coexist on a single fiber, one can
tap into the huge fiber bandwidth
 A more cost-effective alternative compared to laying more fibers
WDM - Parallelism on Optical Networking
(WDM)
Source: Steve Wallach, Chiaro Networks
“Lambdas”
Parallel lambdas will drive this decade
the way parallel processors drove the 1990s !
WDM Fundamentals
 WDM building blocks
•
•
•
•
•
•
•
Light sources (laser diodes) and detectors (photodetectors, filters)
Optical fibers (single-mode, multi-mode)
Multiplexers and Demultiplexers
Optical Add/Drop Multiplexers
Optical amplifiers (e.g. EDFA)
Photonic cross-connect switches
Transponders
WDM Fundamentals
 A wavelength-routed optical WDM network consists of a photonic
switching fabric comprising active optical switches connected by fiber
links forming any arbitrary physical topology
 Each node equipped with a set of transmitters and receivers (which may
be “wavelength tunable”)
 The basic mechanism of communication in such a network is a lightpath
• Lightpath: an all-optical communication channel (a path) between 2 nodes (it can
span more than one fiber link!)
 The intermediate nodes in this fiber path route the lightpath in the optical
domain using their active optical (photonic) switches
 The end-nodes of the lightpath access the signal with transmitters and
receivers that are tuned to the wavelength on which the lightpath operates
WDM Fundamentals
 Photonic switches & protocols like GMPLS are key elements to
address new goals, and implement a multi-tiered and scalable
IP/Optical network
WDM Fundamentals
 In wavelength-routed WDM networks, a control mechanism is needed to
set up and take down the optical connections (lightpaths)
 A successful data transfer event between 2 nodes has three phases
• Connection establishment
• Data transfer
• Connection release
 During first phase, a few control signaling packets are exchanged between
network resources, aiming to establish a lightpath with an assigned wavelength
 If it succeeds, a lightpath is established, and data transfer occur through this
circuit from source to destination
 When the transfer is completed, control packets are again exchanged between
the nodes, and the resources are released and made ready to be assigned for
another connection
WDM Fundamentals
 A challenging networking problem is that, given a set of lightpaths that
need to be established on the network, and given a constraint on the
number of wavelengths,
• determine the routes over which these lightpaths should be set up
• determine the wavelengths that should be assigned to them so that the
maximum number of lightpaths may be established
 If any switching/router node is also equipped with a wavelength-converter
facility, then lightpaths can be established using diferent wavelengths on
their routes from origin to destination
• This problem is referred to as the RWA problem
WDM Systems: General layout
Transmissor
DWDM
EDFA
OXC
.
.
.
EDFA
n
.
.
.
n
c1
DEMUX
CWDM
DEMUX
CWDM
Router
Border
Router
Router
OADM n
OADM n
(Source: M. Stanton - GIGA Project)
Router
c1
CWDM
.
.
.
.
.
.
CWDM
Border
OADM 1
c1
MUX
CWDM
Border
Border
cn
OADM 1
Transponder
CWDM
cn
cn
MUX
CWDM
Transponder
CWDM
Core
Router
GBIC n
.
.
.
1
cn
GBIC 1
GBIC n
fiber
fiber
1
Core
Router
.
.
.
DEMUX
DWDM
GBIC 1
MUX
DWDM
c1
Transponder
DWDM
WDM Systems: R-OADM Conceptual Architecture
Software Controlled Selectors
(Pass-through/Add/Block)
Pass
West
Pass-Through Wavelengths
Splitter
DWDM
Signal
Add
Pass
Software
Controlled
DEMUX
Add
block drop block drop
Add
Wavelengths
Transponder
Module
Network
Element
1
3
Network
Element
Drop
Wavelengths
drop block drop block
DWDM
Signal
Software
Controlled
DEMUX
Drop
Wavelengths
Network
Element
3
Network
Element
1
Transponder
Module
Add
Wavelengths
Add
Pass
Splitter
Pass-Through Wavelengths
Add
Pass
Software Controlled Selectors
(Pass-through/Add/Block)
East
The KyaTera testbed: Reference Architecture
IP Router
10 GbE <->
MUX/DEMUX
&
R-OADM
Ethernet
Aggregation
Switch
Photonic
Switch
Ethernet
Aggregation
Switch

MUX/DEMUX
&
R-OADM
MUX/DEMUX
&
R-OADM
MUX/DEMUX
&
R-OADM
IP Router
10 GbE <->

Photonic
Switch
Ethernet
Aggregation
Switch
MUX/DEMUX
&
R-OADM
MUX/DEMUX
&
R-OADM
Photonic
Switch
IP Router
10 GbE <->

The KyaTera testbed (example of a proposed solution)
Enabling e-Science: The KyaTera / UltraLight Proposal
 Network support: a critical aspect of Grid-enabled environments
 Commodity Internet is based on a best-effort delivery model, a vehicle
excessively slow and unreliable for the huge masses of data being
generated in emerging e-Science applications
 Deployment of Grids on wide-area scales is being severely restricted
Enabling e-Science: The KyaTera / UltraLight Proposal
 Optical networing: a promising solution to these limitations
• Emerging lightpaths technologies are becoming more and more popular in
the Grid community
• They can include the network resources as an integral Grid component,
controlled by Grid schedulers in the same way as computing elements and
storage resources
 The challenge:
• A new management technology is needed to allow end-users to acquire
network resources on demand, control end-to-end interconnections
between peers (lightpaths), and share unused bandwidth in a flexible and
collaborative way
The KyaTera / UltraLight Proposal
 Our project proposal:
• To work on the problem of monitoring, managing, and optimizing the use of the
networking resources present in next-generation user-controlled optical
networks in real time
• To work in close partnership with the UltraLight Project and the KyaTera
Project
• To use the optical networking infrastructure that is being made available by
the KyaTera Project
 The KyaTera network insfrastructure, enhanced by an intelligent optical control
plane middleware, will provide the basement for the deployment of the Gridenabled Analysis Environment Service Architecture (GAE), a project being
developed at Caltech and University of Florida, coordinated by Prof. Harvey
Newman
The KyaTera / UltraLight Proposal
 Research will be done on provisioning end-to-end survivable optical
connections in the testbed, as in a Grid environment, with an innovative use of
the GMPLS control plane
 (this will be accomplished in a close partnership with the OptiNet lab experts)
(Drawing and text courtesy of Gustavo Pavani – OptiNet / UNICAMP)
Project Planning: Milestones and Timeframe

Milestones
I.
Provisioning of end-to-end optical connections between pairs of nodes
II.
Provisioning multilayer protocols and intelligent monitoring software agents,
and research on RWA algorithms
III. Deployment of routing/switching and control protocols to locate suitable
lightpaths and schedule the networking resources
IV. Deployment of Grid Analysis Environment
V. Job submissions and data transfers between sites over the distributed
computing infrastructure looking for failures, malfunctioning and bottlenecks
Project Planning: Milestones and Timeframe
First Year
Tasks
1st
1
I
2
3
4
II
5
6
7
III
8
9
10
IV
11
12
V
13
14
2nd
Second Year
3rd
4th
5th
6th
7th
8th