Transcript ppt - California Institute for Telecommunications and Information

The OptIPuter—
Lambda Coupled Distributed Computing,
Peer-to-Peer Storage,
and Volume Visualization
Dr. Larry Smarr,
Director, California Institute for Telecommunications
and Information Technology
Professor, Computer Science and Engineering,
Jacobs School of Engineering, UCSD
Two New Cal-(IT)2 Buildings
Approved by Legislature This Week!
Bioengineering • Will Create New Laboratory Facilities
UC Irvine
–
–
–
–
Clean Rooms for Nanotech and BioMEMS
Computer Arts Virtual Reality
Wireless and Optical Networking
Interdisciplinary Teams
UC San Diego
Future Systems Will “Waste” Bandwidth and Storage,
While Conserving Computing
Scientific American, January 2001
S-Curves of Exponential Technology Growth
Lambda Grids
Experimental
Networks
Production/
Mass Market
DWDM
100%
Technology
Penetration
Internet2 Abilene
Experimental/
Early Adopters
Connections Program
0%
Research
Time
Technology S-Curve
Gigabit Testbeds
~1990s
2000
2010
Networking Technology S-Curves
A LambdaGrid Will Be
the Backbone for an e-Science Network
Apps Middleware
•
Metro Area
Laboratories
Springing Up
Worldwide
•
Developing
GigE and
10GigE
Applications
and Services
•
Testing Optical
Switches
•
Metro Optical
Testbeds-the
next GigaPOP?
Clusters
Dynamically
Allocated
Lightpaths
Switch Fabrics
Physical
Monitoring
C
O
N
T
R
O
L
P
L
A
N
E
Source: Joe Mambretti, NU
NSF Defines Three Classes of Networks
Beyond the Commodity Internet
•
Production Networks (e.g. Internet2)
– High-Performance Networks
– Reaches All US Researchers
– 24 / 7 Reliable
•
Experimental Networks
– Trials of Cutting-Edge High-Performance Networks
– Deliver Advanced Application Needs Unsupported by Production Networks
– Robust Enough to Support Application-Dictated Development:
– Software Application Toolkits,
– Middleware,
– Computing and Networking
•
Research Networks
–
–
–
–
–
Smaller-Scale Network Prototypes
Enable Basic Scientific and Engineering Network Research
Testing of Component Technologies, Protocols, Network Architectures
Not Expected to Be Persistent
Not Expected to Support Production Applications
www.evl.uic.edu/activity/NSF/index.html
Local and Regional Lambda Experimental Networks
Are Achievable and Practical
• Several GigaPOPs Are Building Multi-Lambda Metropolitan
Experimental Networks by Lighting up Their Own Dark Fiber
– With Hundreds of Lambdas by 2010
• LambdaGrid Software to Dynamically Couple Applications to
Metro and Resources Is Possible
• Metro Experimental Networks Will Be Sensitized to:
– End Application Needs
– Local Connectivity Issues
– Current National Production Networks Have Not Begun to Satisfy
• Substantial State and Local Funds Can Be Heavily Leveraged by
an NSF Experimental Networks Program
–
–
–
–
–
Cross-country Inter-Connection
Persistent Support of Emerging Experimental Networks
First NSF Workshop UIC December 2001
Second NSF Workshop UCI May 2002
Expected NSF RFP by Fall 2002
Cal-(IT)2 is Developing a Metro-Scale
Experimental Optical Network
• Driven by Data-Intensive
Applications
– Real Time Seismic
– Emergency Response
– Medical Imaging
• Linked UCSD and SDSU
– Dedication March 4, 2002
Linking Control Rooms
UCSD
SDSU
44 Miles of Cox Fiber
Cox, Panoram,
SAIC, SGI, IBM,
TeraBurst Networks
SD Telecom Council
Beginning of Cal-(IT)2
Metro Optical Networking Laboratory
TeraBurst
TeraBurst
Next Step– California Must Have a State-Wide
Experimental Optical Network
• The Institutes are Creating a Joint Plan
– Led by Cal-(IT)2 & CITRIS
– Involving QB3 and CNSI
• Leveraging Today’s CENIC Investment
– Provides California Internet Connectivity
– K-12 and Universities
• Necessary for Data-Intensive Science
– Widely Available to Many Disciplines
• California is Not the Leader Today!
– Illinois, Indiana, and Canada Already There
CENIC and CISI May Create a Dark Fiber
Experimental and Research Network
The SoCal Component
Some Scientific Applications
Require Experimental Optical Networks
• Challenges
–
–
–
–
Large Data Challenges in Neuro and Earth Sciences
Each Data Object is 3D and Gigabytes
Data are Generated and Stored in Distributed Archives
Research is Carried Out on Federated Repository
• Requirements
–
–
–
–
Computing Requirements  PC Clusters
Communications  Dedicated Lambdas
Data  Large Peer-to-Peer Lambda Attached Storage
Visualization  Collaborative Volume Algorithms
• Response
– Cal-(IT)2 OptIPuter Research Project
NIH is Funding a National-Scale Grid
Which is an OptIPuter Application Driver
Biomedical Informatics
Research Network
(BIRN)
Part of the UCSD CRBS
National Partnership for Advanced Computational Infrastructure
Center for Research on Biological Structure
NIH Plans to Expand
to Other Organs
and Many Laboratories
The OptIPuter is
an Experimental Network Research Project
• Multiple Lambdas Linking Clusters and Storage
–
–
–
–
–
Integration with InfiniBand PC Clusters
Rethink TCP/IP Protocols
Peer to Peer Storage
LambdaGrid Software Stack
Interactive Collaborative Volume Visualization
• NSF Large Information Technology Research Proposal
– Larry Smarr, PI
– UCSD and UIC Lead Campuses
– USC, UCI, SDSU, NW Partnering Campuses
– Industrial Partners: IBM, Telcordia/SAIC, Chiaro
Networks, CENIC
– San Diego Telecom Council Letter of Support
– Submitted April 4, 2002
– Seeking $15M over 5 Years
The OptIPuter Research Team and Its
Regional, National, Int’l Extensions
Asia
Pacific
Vancouver
Seattle
Portland
CA*net4
Pacific
Light
Rail
Chicago
UIC
NU
San Francisco
Asia
Pacific
SURFnet
CERN
PSC
NYC
NCSA
USC
Los Angeles UCI
UCSD, SDSU
San Diego
(SDSC)
Atlanta
AMPATH
Source: Tom DeFanti and Maxine Brown, UIC
OMNInet Optical Switching Trial
Now Underway in Chicago Metro Area
Univ. Illinois at Chicago
8x1GE
Application
Cluster
2x10G
E
2x10GE
Passport
8600
OPTera
Metro
5200
Optical
Switching
Platform
Application
Cluster
Passport
8600
Optical
Switching
Platform
8x1GE
Passport
8600
Application
Cluster
CA*net3/4--Chicago
StarLight
8x1GE
Northwestern Univ.
8x1GE
2x10GE
2x10GE
Optical
Switching
Platform
Optical
Switching
Platform
Passport
8600
Application
Cluster
• A Four-Site Network in Chicago -- The First 10GE Service Trial!
• A Test Bed for All-Optical Switching and Advanced High-speed Services
• Partners: SBC, Nortel, iCAIR at Northwestern, UIC, CANARIE, ANL
Planned Chicago Metro
Electronic Switching OptIPuter Laboratory
Internationals: Canada, Holland, CERN, GTRN, AmPATH, Asia…
Int’l GE, 10GE
16x1 GE
16x10 GE
Metro GE, 10GE
16-Processor McKinley
at University of Illinois
at Chicago
10x1 GE
+
1x10GE
Nat’l GE,
16-Processor
Montecito/Chivano
at Northwestern
10GE
StarLight
Nationals: Illinois, California, Wisconsin, Indiana,
Abilene, FedNets. Washington, Pennsylvania…
Source: Tom DeFanti
Plan for UCSD Campus-Scale OptIPuter
Initially Connect:
Schools of Medicine
and Engineering
with SIO and SDSC
Philip Papadopoulos, SDSC
OptIPuter Optical Research Topics
•
Integrate InfiniBand and IP Over Lambdas
– Connectivity, Addressing, Configuration, Recovery and Resource Discovery
•
Design Evaluation, Monitoring and Analysis Tools for
– Optimizing InfiniBand Over Dedicated Lambdas
•
Devise Network Control and Management (NC&M) and Traffic Engineering
– Gather Cluster Demand and Signal the Reconfiguration of the Network
•
Extend Signaling
– Permit OptIPuter Clusters to Establish and Teardown Connections to Peer
Clusters Across the IP/Lambda Network
– Integrate This Signaling With Network-based NC&M
•
Extend the IP/Lambda Network to Support Very Fast Switches
– Participate in the Optical Switching of Label Switched Paths (LSPS) That
Transport InfiniBand Messages
•
Develop Common-Control-Plane Optical Transport Architectures
– With Generalized Multi-Protocol Label Switching (GMPLS)
– Transport Traffic Over Multiple User Planes With Variable Switching Modes,
Namely Lambda Switching
– Burst Switching
– Inverse Multiplexing (Enabling One Application to Use Multiple Lambdas)
Other OptIPuter Research Topics
• Software Architecture
– LambdaGrid Middleware Architecture
– Application Communication Abstractions and Cluster Bundle
Composition
– Real-Time, Dependable Execution Environment
• Distribute Storage While Optimizing Storewidth:
–
–
–
–
Distribute Massive Pools of Physical RAM (Network Memory)
Develop Visual TeraMining Techniques to Mine Petabytes of Data
Enable Ultrafast Image Rendering
Create for Optical Storage Area Networks (OSANs)
• Enhance Security Mechanisms:
– End-to-End Integrity Check of Data Streams
– Access Multiple Locations With Trusted Authentication Mechanisms
– Use LambdaGrid Middleware for Authentication, Authorization,
Validation, Encryption and Forensic Analysis of Multiple Systems and
Administrative Domains