Transcript The OptIPuter

OptIPuter
Larry Smarr, PI
Tom DeFanti, Jason Leigh, Mark Ellisman, Phil Papadopoulos, Co-PIs
Maxine Brown, Project Manager
Knowing the User’s
Bandwidth Requirements
A -> Need full Internet routing
B -> Need VPN services on/and full Internet
routing
C -> Need very fat pipes, limited multiple
Virtual Organizations
Bandwidth
consumed
A
Number of
users
B
DSL
C
GigE LAN
Source: Cees de Laat, UvA
The OptIPuter is a Distributed “Infostructure” for
Data-Intensive Scientific Research and Collaboration
“A global economy designed to waste transistors, power, and silicon area – and
conserve bandwidth above all – is breaking apart and reorganizing itself to waste
bandwidth and conserve power, silicon area, and transistors.”
— George Gilder, Telecosm (2000)
The OptIPuter: A Philosophy
• Bandwidth is getting cheaper, faster than storage.
• Storage is getting cheaper, faster than computing.
• The exponentials are crossing – we are moving from a processorcentric world, to one centered on optical bandwidth, where the
networks will be faster than the computational resources they connect.
The OptIPuter: A Paradigm
• Supercomputers maximize computing, and minimize bandwidth use.
• The OptIPuter maximizes bandwidth use: A 32-node IA-64 Linux cluster
assumes 1GigE per node (32 Gbps I/O). When clusters are optically
connected at 40 Gbps (for example), bandwidth is >100% the
throughput!
A Useful and Usable Tool for Data-Intensive
Application Drivers: BioScience and GeoScience
The OptIPuter project has two application drivers where scientists are
generating multi-gigabytes of 3D volumetric data objects that reside on
distributed archives that they want to correlate, analyze and visualize.
NIH Biomedical Informatics
Research Network
Initially a multi-scale brain imaging
federated repository, but to be expanded
to other organs of the body.
NSF EarthScope
The acquisition, processing and scientific
interpretation of satellite-derived remote
sensing, near-real-time environmental, and
active source data.
siovizcenter.ucsd.edu/library/gallery/shoot1/index.shtml
http://ncmir.ucsd.edu/gallery.html
A Radical New Architecture for
Scientific Cyber “Infostructure”
The OptIPuter is a “virtual” parallel computer in which the individual
“processors” are widely distributed clusters; the “memory” is in the form
of large distributed data repositories; “peripherals” are very-large
scientific instruments, visualization displays and/or sensor arrays; and
the “motherboard” uses standard IP delivered over multiple dedicated
lambdas.
Think of the OptIPuter as a giant graphics card, connected to a giant disk
system, via a system bus that happens to be an extremely high-speed
optical network. One major design goal is to provide scientists with
advanced interactive querying and visualization tools, to enable them to
explore massive amounts of previously uncorrelated data in near real
time.
Computing ● Data ● Visualization ● Networking
A 21st Century Amplified Work Environment:
The Continuum
Passive stereo display
AccessGrid
Tiled display
Digital white board
Cluster Visualization
San Diego OptIPuter Design:
LambdaGrid Enabled by Chiaro Router (Spring 2003)
Medical Imaging
and Microscopy
Chemistry,
Engineering, Arts
switch
switch
Chiaro
Enstara
switch
San Diego
Supercomputer Center
switch
Scripps Institution of
Oceanography
What is a Lambda?
• A lambda, in networking, is a fully dedicated
wavelength of light in an optical network,
typically used today for 1-10Gbps.
• Lambdas are circuit-based technology, but can
carry packet-based information.
• We are now mostly working with 1Gb dedicated
layer2 circuits that act like lambdas
Switching Lambdas make the OptIPuter
Possible
•
•
•
•
•
•
•
Visualization getting very good on PCs
Networking getting very fast on PCs (10GigE)
Disk getting cheaper and faster
PCIX bus, 64-bit architectures available
Cost of bandwidth is plummeting
Optical switching is coming
We don’t have to scale to reach everybody in
their homes
Why Optical Switching?
• No need to look at every packet when
transferring a terabyte of information
– 1% the cost of routing
– 10% the cost of switching
– 64x64 10Gb:
• $100,000 O-O-O switched
• $1,000,000 O-E-O switched
• $10,000,000 O-E-O Routed
• Spend the savings on computing and
collaboration systems instead!
• Replaces patch panels; allows rapid
reconfiguration of 1 and 10Gb experiments
Large-Scale International Application Development
Guaranteed
Latency
Access Grid
Guaranteed
Scheduling
Guaranteed
Bandwidth
●
USA, Canada, The Netherlands, UK, Italy, Germany, Russia, Australia,
China, Korea, Thailand, Taiwan, Japan, Brazil
BABAR
●
The D0 Experiment
●
USA and internationally
USA, CERN, Germany, France, Japan and other worldwide collaborators
GiDVN: Global Internet Digital Video Network
●
●
CCIRN DVWG worldwide membership
Hubble Space Telescope
●
USA, France and others worldwide
SC Global
●
USA and International
Sloan Digital Sky Survey (SDSS)
●
USA, France and worldwide
Virtual Room Videoconferencing System (VRVS)
●
CERN, Switzerland; Caltech, USA; Others
vlbiGrid
USA, The Netherlands, Finland, UK and worldwide
●
Large-Scale International Middleware and Toolkit Development
EU DataGrid
Guaranteed
Latency
Guaranteed
Scheduling
Guaranteed
Bandwidth
●
●
●
CERN, France, Italy, The Netherlands, UK, Czech Republic, Finland,
Germany, Hungary, Spain, Sweden (in cooperation with US grid projects,
notably GriPhyN, PPDG and iVGL
EU DataTAG
●
Globally Interconnected Object Databases
(GIOD)
●
USA and Europe
USA and CERN
Globus
●
●
●
USA, Sweden, others internationally
MONARC for LHC Experiments
●
CERN, Switzerland; Caltech, USA; Others
UK e-Science Programme
UK and USA
●
●
●
StarLight :
Perhaps the World’s Largest 1GigE and 10GigE Exchange
StarLight is an experimental
optical infrastructure and
proving ground for network
services optimized for highperformance applications.
Operational since summer
2001, StarLight is a 1GigE and
10GigE switch/router facility
for high-performance access
to participating networks and
is becoming a true optical
switching facility for
wavelengths.
Abbott Hall, Northwestern University’s
Chicago downtown campus
StarLight’s Dutch and Canadian Partners
Kees Neggers, SURFnet
Bill St. Arnaud, CA*net4
TransPAC/
APAN
CA*net4
GÉANT
Abilene
Euro-Link
SurfNET
Euro-Link
CERN
Euro-Link/Trans-Light and
TransPAC/Trans-Light Supplement
Funded, Co-Funded and Matching Links
Euro-Link Funded 10Gb Link
Euro-Link Co-Funded 10Gb Link
TransPAC Co-Funded 10Gb Link
Partner Matching 10Gb Link
GÉANT Matching 2.5Gb Links
Hard Problems
• Internet is not designed for single largescale users—TCP is not usable for long fat
applications
• Circuits are not “scalable”
• All intelligence has to be on the edge
• Tuning compute, data, visualization,
networking using clusters to get order of
magnitude improvement
• Security at 10Gb line speed
The OptIPuter Project will
• Investigate, procure and install optical and electronic switches.
• Explore protocol stack development (for VLAN/optical
management, control and data planes).
• Make progress in removing hierarchical protocol layers, moving
toward network transparency.
• Optimize network-attached computational PC clusters for largescale data mining and visualization.
• Facilitate measuring and monitoring 1-10Gb networks in
experimental and production configurations.
• Compare strategies for grids moving large data over moderately
congested links vs. dedicated links.
• Investigate the dynamic provisioning of light paths by highperformance applications on optical networks.
• Explore mechanisms to communicate and deliver Class of Service
in high-speed networks.
• Debate the security advantages of switched lambdas
Questions?