Transcript presentation - NORDUnet Networking Conferences

CA*net 4
Optical Networking and Third Wave
of the Internet
Bill St. Arnaud
CANARIE Inc – www.canarie.ca
[email protected]
Overview
> 1. CA*net 4 Network Architecture
> 2. Applications that may drive require a new network
architecture
CA*net 4 Drivers-1
Reduce cost of Internet
> Set up lightpaths to no cost peering exchanges
– Most lambda sales in Canada and USA are for “Remote peering” to
no cost peering points
– Allows for considerable savings in Internet transit costs
– Each lightpath is directly connected to a high volume peer and
bypasses peering router
– Good example is “STAR LIGHT” where high volume peers have
direct connect and small volume peers use a router
– CA*net 4 “customer controlled patch panel” allows peers to change
peering relationship remotely without contacting technical staff at
peering exchange
• Very similar in concept to WorldCom “Peermaker” at MAEs which use
e-mail to setup peers
CA*net 4 Drivers-2
Reduce cost of routers
>
Eliminate expensive high end routers and replace them with lower cost
optical switches
– But circuits are NOT intended to replace packet networks
– Use rich mesh of circuits between edge routers to eliminate high
cost of 10GbE core routers
•
•
10Gbe routers ~ $500K with interfaces at ~$200k each
10Gbe switches ~$25K with interfaces at ~$20k each
– Trade off between cost of multiple lightpaths and loss of
multiplexing versus cost of high end core routers
– 10Gbe wavelengths ~$1000/km for 5 years (lifetime of router)
– Assume 1 GbE lightpaths per edge institution then
•
One 5000km Gbe lightpath (or 8 x 600km GbE) lightpaths per
institution is cheaper than routers
CA*net 4 Drivers-3
A VPN alternative to MPLS
>
Allows customer to create “customer owned and managed”
networks with resource heterogeneity
– Integration of wavelengths and dark fiber from different carriers
– Create customer controlled VPNs for downstream users and
overlay networks across multiple suppliers
– Customers can manage their own restoral and protection schemes
– Allows for inter-domain end to end setup of VPNs
– End users do not need to to signal carrier for VPN management
•
•
•
Create VPNs
Cross connect VPNs from independent users
Partition or spawn VPNs
•
Establish VPNs across multiple management domains
CA*net 4 Drivers-4
Application Specific Networks
> Lambda Grids - “Underlay” networks to support Grids and overlay
projects like PlanetLab and Oceanstore
– A lot of exciting research into overlay networks
– At some point in time when traffic volume is sufficient in overlay network
to setup its own direct path
> Soon high end grid applications will have sufficient traffic volume to
require their own underlay networks ”Complementing” routed
networks
– Not a replacement for routed networks – only increasing the direct
peering mesh of the routed network for specific applications
– But peering may be more dynamic (and not globally advertised) than
traditional IP BGP peering
> Discipline or applications specific networks
– VBLI grids like European EVN
– High energy physics grid – Ultralight
– NEES grid, Bio-informatics Grid, etc
Example – EVN traffic flows
over GEANT
SE
NORDUnet
JANET
UK
JIVE
PL
SURFnet
CZ
PSNC
BE
NL
DFN
DE2
DE1
FR
2.5G
10G
CH
AT
IT
GARR
Provided courtesy of Dai Davies
Issues
>
>
>
How do you charge for bandwidth and usage when single
application traffic dwarfs all other IP traffic?
Who pays for the traffic volume when it sinks into one NREN?
Possible solutions:
1. GMPLS (with QoS)
•
•
•
Requires expensive routers and complex coordinated central
management to setup and tear down tunnels
Does not address issue of traffic charging
Interdomain still unproven
2. Optical overlay/underlay –ASON
–
same problems as GMPLS
3. Application specific optical BGP networks
–
Increase BGP mesh for specific applications or disciplines
OBGP applied to EVN
Express route
SE
NORDUnet
JANET
UK
JIVE
PL
SURFnet
CZ
PSNC
BE
NL
DFN
DE2
DE1
FR
CH
EVN sites will see 2 BGP routes to SURFnet:
IT
-the normal IP route over GEANT
-Express route using dedicated lightpaths (in green)
AT
GARR
CA*net 4 Drivers-5
QoS
>
Spatial QoS
– TCP throughput over long fat pipes very susceptible to packet
loss, MTU, TCP kernel, Buffer memory, AQM optimized for
commodity Internet, Auto negotiating Ethernet, etc
– May also require consistent and similar TCP throughput for
multiple sites to maintain coherency for grids and SANs
– Some exciting new TCP protocols like FAST, XCP, etc
•
•
Mice and Elephant problem
Without careful design may look like a DOS attack on a router network
– Many commercial SAN/Grid products will only work with QoS
network
– Some users want to have super jumbo MTU (64K) or protocols
other than IP
Spatial QoS
x.x.x.1
Normal BGP path
y.y.y.1
Optical “Peermaker”
Only y.y.y.1
advertised to
x.x.x.1 via
OBGP path
Application or end user
controls peering of BGP
optical paths for transfer
of elephants!!!
OBGP path
Only x.x.x.1
advertised to
y.y.y.1 via
OBGP path
CA*net 4 Drivers-6
Extend E2E principle to circuits
> Extend the Internet end to end principle to circuit based networks
– The success of the Internet is largely attributable to the classic e2e principle
– Allowed development of exciting new applications or services
> E2E principle presumes all users behave honorably
– Otherwise you get spam and DDoS attacks
– This ungentlemanly behavior can be reasonably contained on commercial
Internet but pose big challenge on research networks
> What happens if you have a malicious implementation of FAST or XCP on a
big pipe network?
> Can the Internet e2e principle be applied to circuit based networks?
– Will it engender the same creativity in new applications and services?
> MPLS and ASON are classic network state based solutions for VPNs
– CA*net 4 architecture is an alternate approach
– All VPNs are BGP direct static routes using lightpaths that are setup and
controlled by end user
CA*net 4 is NOT a network
> It is an aggregation of point to point 10 Gbps wavelengths from a
number of carriers
> CA*net 4 is made up of may parallel application or discipline
specific networks that may (or may not) BGP peer with each other
> The wavelengths and switches are partitioned into smaller
lightpaths with user control of the switch partition which are used
for a variety of applications particularly grids
– International Grid Testbed – 10 Gbe server to server to CERN
– WESTgrid – 1 Gbe lightpaths for distributed backplane
– Numerous lightpaths to support direct peering between regional networks
and universities
– Lightpaths to support TransLight projects between North America, Europe
and Asia
– Many, many more coming – Virtual Astronomy, HDTV video walls, etc
Applications for E2E Lightpaths
The three waves
– The first wave of the Internet consisted primarily of text and
data services such as e-mail and FTP.
– The second wave was the web which improved ease of use
and facilitated the transfer of images, sound and video.
– The third wave is the integration of applications, p2p
networking, open source, distributed computing enabled by
next generation web services, semantic web and high speed
networks
What is the Third Wave?
> Before the Web on-line information was only available through a
small number of information providers who charged high fees
– Compuserve, Dialogic, etc
> The Web allowed millions of creators of information to make it
easily accessible to all others at very low cost, bypassing the
information middleman
> The Third Wave proposes to extend the WEB paradigm to
processes, applications and content
> Third Wave is about creation of tools and applications (i.e.
services) in variety of fields such as eLearning, eBusiness,
eScience, eHealth, etc that can make these services easily
available to all others
– At there are millions of web sites, there will be millions of Third
wave services
Today’s Network
The network is
subservient to the
computer
The application is
tightly bound to the OS
Application
Network
Application
User
User
OS
OS
Data
The network is a
mechanism for
applications to
communicate with each
other
Data
Third Wave Network
Application and data exist on the
network and are uncoupled from any
specific machine or location
Third Wave
Third Wave
Network
OS
OS
The computer is
subservient to
the network
Application and Data
Third Wave
Third Wave
Third Wave
Third Wave
OS
OS
OS
OS
Data
Data
Data
Data
A new way of doing science
>
Science used to about test tubes, wet labs and big instruments
>
But increasingly science is moving to networks and computers
>
Science is now longer bound by bricks and mortar or geography
>
NSF has announced “Cyber Infrastructure” initiative
– https://worktools.si.umich.edu/workspaces/datkins/001.nsf
>
DOE SciDAC “Scientific Discovery through Advanced Computing”
– http://www.er.doe.gov/feature_articles_2001/august/SCIAC/SciDAC_announcement.h
tm
>
Recognition that more and more science is network and computationally
based
>
Grids using web services will be foundation of this new research
methodology
Many e-Research Projects Coming
ALMA
LHC
Sloan Digital Sky
Survey
ATLAS
International Grid Testbed
~PBytes/sec
~100 MBytes/sec
Online System
SpecInt95 equivalents
ATLAS
Offline Processor Farm
~100 MBytes/sec
~20 TIPS
~622 Mbits/sec per channel
Tier 1
France Regional
Centre
1 TIPS is approximately 25,000
Tier 0
Germany Regional
Centre
Italy Regional
Centre
CERN Computer Centre
FermiLab ~4 TIPS
622 Mbits/sec
~
Tier 2
~622 Mbits/sec per channel
InstituteInstitute Institute
~0.25TIPS
Physics data cache
Tier2 Centre
Tier2 Centre
Tier2 Centre
Tier2 Centre
~1 TIPS ~1 TIPS ~1 TIPS ~1 TIPS
Physicists work on analysis “channels”.
Institute
~1 MBytes/sec
Tier 4
Physicist workstations
Caltech
~1 TIPS
Each institute will have ~10 physicists working on one or more
channels; data for these channels should be cached by the
institute server
International Grid Testbed
> First production use of international e2e lightpaths
to transfer Forward Calorimeter data from CERN to
Canada
> Experiments planned to directly transfer low level
trigger data from CERN to Canadian computers
> 10 GbE server to server experiments
> RDMA, TCP/IP offload, etc
Virtual Observatory
> http://www.us-vo.org/
> Discovery process will rely
on advanced visualization
and data mining tools
> Not tied to a single brick and
mortar location
> Will cross correlate existing
multi-spectral databases
petabytes in size
> Web services will integrate
data and applications
No new telescopes or radio dishes.
Just big networks interconnecting
large databases
Canada Virtual Observatory
Data Flows
>
>
>
>
Flow to Terapix : 1TB/month
Flow to science centres: 0.5 TB/month
Return from Terapix: 2 TB/month
Over 5 years users will download full dataset (100
TB) ~ 30 times: 50 TB per month
> Average flow 154 Mbps for five years
> User network load is the largest, least predictable,
and least manageable component of the network
traffic
> Accessible to students at schools
Canadian Forestry Grid
> SAFORAH (System of
Agents for Forest
Observation Research
with Automation
Hierarchies).
> SAFORAH connects five
locations across the
country to support the
monitoring of Canada's
forests
> Together, all five locations
will generate data equaling
40 terabytes (TB) per
month
Grids for Kids
> The ultimate goal of Grids for Kids is to allow
students and eventually members of the general
public to be full participants in scientific discovery
and innovation.
> Will allow increasing number of computationally or
networked research experiments to be seamlessly
integrated with the computer capabilities of
thousands of PCs located at our schools
> Some early primitive examples…
FightAIDS@Home
•Scientists at The Scripps Research Institute
(TSRI) are using computational methods to
identify drugs that have the right shape and
interaction characteristics to fight diseases such
as AIDS.
•Once such candidates are identified, they can be
synthesized in a laboratory, tested according to
FDA guidelines, and released as prescription
drugs to benefit the public.
•Such computations require a vast number of trial
dockings, testing variations in the target protein
and the trial drug molecules
Folding@home
>
>
>
This "virtual supercomputer"
uses peer-to-peer technology
to make unprecedented
amounts of processing power
available to medical
researchers to accelerate the
development of improved
treatments and drugs that
could potentially cure
diseases.
Rapid new discoveries in
cancer research
Two projects in Canada:
– Smallpox cure at UWO
– Leukemia research at Mt Sinai
Climate Prediction
>
>
>
Predict future climate due to greenhouse affect
Distribute climate model to thousands of PCs worldwide
www.climateprediction.com
ALTA Cosmic Ray eScience
>
>
>
Collaborative scientific
research project involving the
University of Alberta Center for
Subatomic Research and over
50 high schools across Canada
in the area of cosmic ray
detection.
Teachers and students actively
contribute to the physics
research while learning about
an exciting area of modern
science.
Distributed computing at
schools required to analyze
data from sensors in near real
time
Neptune/Venus Grid
>
>
>
Joint US-Canadian project for undersea dark fiber network off west
coast of USA and Canada
Undersea network will connect instrumentation devices, robotic
submarines, sensors, under sea cameras, etc
All devices available to students and researchers connected to CA*net 4
and Internet 2 networks
Distributed computing and
data storage devices on
CA*net 4 and Internet 2 will
be used to analyze and store
data
Faulkes Telescope
>
>
>
>
>
>
Provide UK schools with access to a
research class telescope in Hawaii
Provides an exciting resource for
teachers to use via the Web
To provide a real-time experience of
astronomy, through live use of a
telescope
To allow students to participate in
real research programs, mentored by
professional astronomers
Provides other public interest
groups, such as amateurs, access to
high quality astronomical data
http://www.faulkes-telescope.com/
More Information
> http://www.canarie.ca/canet4/library/canet4design.ht
ml
> http://www.canarie.ca/canet4/obgp/index.html
> http://www.canarie.ca/canet4/library/customer.html
> Thanks to the design teams at Carleton U, Ottawa U,
CRC, UQAM, UoWaterloo, Montague, etc
Background Slides
User Controlled Service
Provisioning Layer
Using Third Wave Service to manage a network
Grid
Application
Web Server
User Access Layer
Grid Service Interface
LPO Grid
Service
GT3 Hosting Environment
LPO
Factory
Service
LPO
Delegate
Service
RMI
RMI
LPO
Service
EJB
Home
LPO
Service
Implementation
LPO
Service
EJB
Remote
J2EE Application Server
JDBC
RMI
DB
Resource Management Layer
Create service
Access service
OGSI-conformant services
OGSI-conformant services
LPO advertisement
 LPO query
 LPO termination
 LPO access
 LPO reconfiguration
 LPO spawning
 LPO concatenation
 End-to-end LPO establishment

Resource Management
Layer
Service Provisioning Layer
RMI
Request
Controller
LPO Controller
Programmable
LPO Controller
Controller
Resource Agent
LPO
Controller
LPO Controller
LPO Controller
Switch
Interface
TL1
CA*net 4
RMI
LPO Space