Transcript ppt - Calit2

"The OptIPuter: Enabling Campus, State,
National, and Planetary-Scale LambdaGrids"
LONI Forum
Baton Rouge, LA
September 2, 2004
Dr. Larry Smarr
Director, California Institute for Telecommunications and
Information Technologies
Harry E. Gruber Professor,
Dept. of Computer Science and Engineering
Jacobs School of Engineering, UCSD
OptIPuter Application Motivation:
Interactive Analysis of Remote Multi-Gigabyte Data Objects
• Hundreds Of Megapixels 2-D Images
– Satellite Imaging and Remote Sensing
– Telescopes or Microscopy
• GigaZone 3-D (1k x 1k x 1k) Objects
– Supercomputer Simulations
– Seismic or Medical Imaging
• Goal: Remote Interactive Analysis and Visualization
– Virtually Impossible Today
– New Technology of Dedicated Lightpaths
– Bring These Capabilities to Laboratory Researchers
Cosmic Simulator
with a Billion Zone and Gigaparticle Resolution
•
•
•
•
5123 AMR or 10243 Unigrid
8-64 Times Mass Resolution
Can Simulate First Galaxies
One Gigazone Run:
Source: Mike Norman, UCSD
10243 Unigrid
– Output ~10 TeraByte
– “Snapshot” is 100 GB
– Must Visually Analyze
Compare with Sloan Survey
SDSC Blue Horizon (2004)
OptIPuter Driver: On-Line Microscopes Creating
Very Large Biological Montage Images
• 2-Photon Laser
Confocal Microscope
IBM 9M Pixels
– High Speed On-line
Capability
• Montage Image Sizes
Exceed 16x Highest
Resolution Monitors
– ~150 Million Pixels!
• Use Graphics Cluster
with Multiple GigEs to
Drive Tiled Displays
Source: David Lee, NCMIR, UCSD
Earth System Enterprise-Data Lives in
Distributed Active Archive Centers (DAAC)
NSIDC (67 TB)
ASF (256 TB)
Cryosphere
Polar Processes
LPDAAC-EDC (1143 TB)
SAR Products
Sea Ice
Polar Processes
Land Processes
& Features
SEDAC (0.1 TB)
Human Interactions in
Global Change
GES DAAC-GSFC
(1334 TB)
Upper Atmosphere
Atmospheric Dynamics, Ocean
Color, Global Biosphere,
Hydrology, Radiance Data
ASDC-LaRC (340 TB)
ORNL (1 TB)
PODAAC-JPL (6 TB)
Ocean Circulation
Air-Sea Interactions
GHRC (4TB)
Global
Hydrology
Biogeochemical
Dynamics
EOS Land Validation
Radiation Budget,Clouds
Aerosols, Tropospheric
Chemistry
EOS Aura Satellite Has Been Launched
Challenge is How to Evolve to New Technologies
However, Average Throughput of NASA Data Products
to End User is Only 50-100 Megabits/s
Tested from GSFC-ICESAT
August 2004
http://ensight.eos.nasa.gov/Missions/icesat/index.shtml
High Resolution Aerial Photography Generates Images
With 10,000 Times More Data than Landsat7
Landsat7 Imagery
100 Foot Resolution
Draped on elevation data
Shane DeGross, Telesis
USGS
New USGS Aerial Imagery
At 1-Foot Resolution
~350,000x350,000 Pixel Images of 350 US Cities
100 Billion Pixel Images!
States are Acquiring Their Own Dark Fiber Networks -Illinois’s I-WIRE and Indiana’s I-LIGHT
Indiana
Today Nearly Two
Dozen States are
Creating Dark Fiber
Networks—Including
the Most Advanced
LONI!
Source: Charlie Catlett, ANL
Regional Network
Evolution
• The Fundamental Nature of Regional
Networking is Rapidly Changing
– The GigaPoP Model Based on Provisioned, HighCapacity Services Steadily Is Being Replaced
– On The Metro and Regional Scales
• A Model of Facility-Based Networking Built with
Owned Assets has Emerged
– Regional Optical Networks (RONs)
– Notably, this Change Increases the Importance of
Regional Networks in the Traditional Three-Level
Hierarchy of U.S. R&E Advanced Networking
Source: Steve Corbató, Internet2
Leading & Emerging
Regional Optical Networks
• California (CALREN)
• Colorado (FRGP/BRAN)
• Connecticut (Conn. Education
Network)
• Florida (Florida LambdaRail)
• Georgia (Southern Light Rail)
• Indiana (I-LIGHT)
• Illinois (I-WIRE)
• Louisiana (LONI)
• Maryland, D.C. & northern
Virginia (MAX)
• Michigan (MiLR)
• Minnesota
• New York + New England
Region (NEREN)
• North Carolina (NC
LambdaRail)
• Ohio (Third Frontier Network)
• Oklahoma (OneNet)
• Oregon
• Pacific Northwest (Lariat –
Supported by NIH)
• Rhode Island (OSHEAN)
• SURA Crossroads
Southeastern U.S.)
• Texas (LEARN)
• Utah
• Virginia (MATP)
• Wisconsin (WEROC)
Source: Steve Corbató, Internet2
NLR Will Provide an Experimental Network
Infrastructure for U.S. Scientists & Researchers
“National LambdaRail” Partnership
Serves Very High-End Experimental and Research Applications
4 x 10Gb Wavelengths Initially
Capable of 40 x 10Gb wavelengths at Buildout
Starting Fall 2004
Dark Fiber: Gauging
Community-Wide Progress
• Aggregate Dark Fiber Assets Acquired by/for U.S. R&E Optical
Initiatives
– CENIC (for CalREN & NLR)
– FiberCo (via Level 3 for NLR & RONs)
– SURA (via AT&T)
– Plus 2,000 Route-Miles for Research
– NLR Phase 2 (WilTel & Qwest)
– OARnet
– ORNL (via Qwest)
– Other Projects (IN,IL,MI,OR, …)
• Total (conservative estimate)
– Over 55% of these Assets are in Regionals
– Remainder held by NLR (11,250 Route-Miles)
Source: Steve Corbató, Internet2
6,200 (Route-miles)
5,650
6,000
4,000
1,600
900
1,500+
25,850+
Global Lambda Integrated Facility (GLIF)
Creates Metacomputers on the Scale of Planet Earth
10 Gbit/s
New York
MANLAN
Stockholm
NorthernLight
IEEAF
10 Gbit/s
2.5 Gbit/s
10 Gbit/s
2.5 Gbit/s
10 Gbit/s
CA*net4
Tokyo
WIDE
10
Gbit/s
IEEAF
10 Gbit/s
Chicago
StarLight
SURFnet
10 Gbit/s
Amsterdam
NetherLight
NSF
10 Gbit/s
10 Gbit/s
DWDM
SURFnet
Dwingeloo
ASTRON/JIVE
10
Gbit/s
2.5 Gbit/s
SURFnet
10 Gbit/s
Tokyo
APAN
2.5 Gbit/s
10 Gbit/s
London
UKLight
Geneva
CERN
Source: Kees Neggers,
www.glif.is
SURFnet
Created in Reykjavik, Iceland 2003
Prague
CzechLight
The OptIPuter Project –
Removing Bandwidth as an Obstacle In Data Intensive Sciences
• NSF Large Information Technology Research Proposal
– Cal-(IT)2 and UIC Lead Campuses—Larry Smarr PI
– USC, SDSU, NW, Texas A&M, Univ. Amsterdam Partnering Campuses
• Industrial Partners
– IBM, Sun, Telcordia/SAIC, Chiaro Networks, Calient, Glimmerglass
• $13.5 Million Over Five Years
• Optical IP Streams From Lab Clusters to Large Data Objects
NIH Biomedical Informatics
Research Network
NSF EarthScope
and ORION
http://ncmir.ucsd.edu/gallery.html
siovizcenter.ucsd.edu/library/gallery/shoot1/index.shtml
OptIPuter Middleware Architecture
for Distributed Virtual Computers
OptIPuter Applications
DVC/
Middleware
Visualization
DVC #1
Higher Level
Grid Services
DVC #2
Security
Models
DVC #3
Data Services: Real-Time Layer 5: SABUL, RBUDP,
DWTPHigh-Speed
Objects
Fast, GTP
Transport
Grid and Web Middleware – (Globus/OGSA/WebServices/J2EE)
Layer 4: XCP
Optical
Signaling/Mgmtl-configuration, Net Management
Node Operating Systems
Physical Resources
From Grids to LambdaGrids
Source: Andrew Chien, UCSD
OptIPuter Software Systems Architect
Creating a Model for
a Campus LambdaGrid
OptIPuter UCSD / San Diego Network and Nodes -- 2004
SDSC
HP
28-node
cluster
(shared)
CSE
8-node
cluster
(shared)
Infiniband
64 nodes
Infiniband
4 nodes
Sun
Sun
24-32-node 128-node
compute
compute
cluster
cluster
IBM
48-node
storage
cluster
JSOE
Preuss
Sun
24-32-node
compute
cluster
HP
4-node
control
Dell Viz
10
Dell 5224
Chiaro
Extreme 400
Enstara
4
2
1
Extreme 400
9-node
cluster 7-node cluster
(shared)
(shared)
To UCI and ISI via
CalREN-HPR Juniper T320
10
Extreme 400
Dell 5224
CRCA
1
1
Bonded
GigE
1
Geowall 2
Tiled Display
IBM 9 mpixel
display pairs
1
3-node
Viz cluster
Cisco 6509
4
Dell
Geowall
4
Dell 5224
Dell 5224
SIO
Dell 5224
IBM 128-node
CPU cluster
IBM 9-node
Viz cluster
Sun
Sun
24-32-node 5-node
cluster
cluster
IBM 9 mpixel
display pairs
SOM
Sun
22-node
Viz cluster
6th
College
Dell 5224
OMNInet
The Metro Area OOO Tesbed
DWDM
RAM
C
NTONC
NTON
10 Gb Lambdas
10GE OptIPuter CAVEWAVE
Will Help Launch the National LambdaRail
EVL
Next Step: Coupling
Source: Tom DeFanti, OptIPuter co-PI
NASA Centers
to NSF OptIPuter
OptIPuter Scalable Visualization Systems:
End User Clusters Connected With Gigabit Flows
Earth Sciences are an OptIPuter Driver
Major NASA Goddard Visit This Week
20 Megapixels
3 Megapixels
Linux Cluster with
Nvidia Graphics Cards
High Bandwidth Optical Fibers Will Enable
High Definition Global Virtual Teaming
UC Irvine
UC San Diego
In 2005
Cal-(IT)2 will Link Its Two Buildings
Creating a Collaboration Laboratory
Ultra-Resolution OptIPuter Displays Utilizing
Photonic Multicasting --Scaling to 100 Million Pixels
UIC-EVL
Glimmerglass
Switch Used to
Multicast and Direct
TeraVision Stream
from One Tile to
Another on the
Geowall-2
Driven
by
Linux
Graphics
Clusters
Glimmerglass Switch
30 Megapixel High-Resolution Visualizations
= ~1 Gigapixel at 30fps = ~30Gb/s Bandwidth
Currently Developing OptIPuter Software
to Coherently Drive 100 MegaPixel Displays
•
55-Panel Display
– 100 Megapixel
– 11x5 21” LCDs
•
Developed at EVL
•
Driven by 30 DualOpterons (64-bit)
•
60 TB Disk
•
30 10GE interfaces
•
Linked to OptIPuter
•
Complementary to
NASA ARC HyperWall
Special issue of Communications of the ACM (CACM):
Blueprint for the Future of High-Performance Networking
•
Introduction
– Maxine Brown (guest editor)
•
TransLight: A Global-scale LambdaGrid for eScience
– Tom DeFanti, Cees de Laat, Joe Mambretti,
Kees Neggers, Bill St. Arnaud
•
Transport Protocols for High Performance
– Aaron Falk, Ted Faber, Joseph Bannister,
Andrew Chien, Bob Grossman, Jason Leigh
•
Data Integration in a Bandwidth-Rich World
– Ian Foster, Robert Grossman
•
The OptIPuter
– Larry Smarr, Andrew Chien, Tom DeFanti,
Jason Leigh, Philip Papadopoulos
•
Data-Intensive e-Science Frontier Research
– Harvey Newman, Mark Ellisman, John Orcutt
www.acm.org/cacm