Transcript The California Institute for Telecommunications and

“OptIPuter Tech Transfer to
the Broader e-Science
and HPC Communities"
OptIPuter All Hands Meeting
Calit2@UCSD
La Jolla, CA
December 20, 2006
Dr. Larry Smarr
Director, California Institute for Telecommunications and
Information Technology;
Harry E. Gruber Professor,
Dept. of Computer Science and Engineering
Jacobs School of Engineering, UCSD
Informing the Cyberinfrastructure Initiative
www.ctwatch.org
Global Lambda Integrated Facility (GLIF)
International Group Innovating the LambdaGrid
• GLIF Technical and Control Plane Working Groups
– Interim Meetings on 8-9 February 2006
– University of New Mexico, Albuquerque, United States
• GLIF Research and Applications Working Group
– Maxine Brown and Larry Smarr
– Abstract Taxonomy from 50 iGrid Demos
• GLIF Full Meeting
– Tokyo, Japan
– September 11-15, 2006
– Interim Deadline for SC06 Demos
OptIPuter Will Have
Coordinated Approach to SC06
Lambdas Give End Users Sustained
~ 10 Gbps Data Flow Rates
On August 5, 2005, GSFC’s
Bill Fink simultaneously
conducted two 15-minuteduration UDP-based 4.5Gbps flow tests, with one
flow between GSFC-UCSD
and the other between
GSFC-StarLight/Chicago.
This filled both the
NLR/WASH-STAR and
DRAGON/channel49
lambdas to 90% of capacity.
Flows were also tested in
both directions. He
measured greater than
9-Gbps aggregate in each
direction and no-tonegligible packet losses.
GSFC Scientific and Engineering Network (SEN)
Mrtg-based `Daily' Graph (5 Minute Average)
Bits per second In and Out
On Selected Interfaces
DRAGON 10Gig DWDM XFP
5 August 2005
200 Times Faster
Than Standard
Internet2!
chance1 10Gig (eth1 Intel
Pro/10GbE)
5 August 2005
chance2 10Gig (eth1 Intel
Pro/10GbE)
5 August 2005
Source: Pat Gary,
NASA GSFC
5
OptIPuter Experiments Accelerated
Future NASA NREN Over NLR
GRC
NLR Cleveland
StarLight
NGIX-East
ARC/NGIX-West
NLR Sunnyvale
NLR Chicago
GSFC
MAX
LRC
JPL
NLR Los Angeles
NLR MSFC
NLR Jacksonville
SSC
KSC
JSC
NLR Baton Rouge
NLR Houston
NREN Site
Peering Points
1 GE
10 GE
Linking OptIPuter to the DRAGON Testbed
Washington, D.C. Metropolitan Area
University of Maryland College Park
(UMCP)
Goddard Space Flight Center
(GSFC)
MAX
MIT Haystack Observatory
(HAYS)
U. S. Naval Observatory
(USNO)
CLPK
DCNE
Bossnet
DCGW
ARLG
HOPI / NLR
MCLN
DCNE
National Computational
Science Aliance (NCSA)
Univ of Southern California/
Information Sciences Institute
(ISIE)
Global e-VLBI
iGrid / SC05
•
Goal: Real-Time VLBI Radio Telescope Data Correlation
– from the USA (MIT Haystack, GGAO),
– Japan (Kashima) and
– Europe (Onsala in Sweden, Jodrell in the UK, Westerbork in The Netherlands)
•
•
Achieved 512Mb Transfers from USA and Sweden to MIT,
Results Streamed to iGrid.
Sloan Sky Survey
Data Mining and Visualizing Data Using OptIPuter
•
SDSS-I
– Imaged > 8,000 Square Degrees of the Sky in Five Bandpasses
– Detecting Nearly 200 Million Celestial Objects
– Measured Spectra Of:
– > 675,000 galaxies
– 90,000 quasars
– 185,000 stars
•
SDSS-II
– Underway till 2008
iGRID2005
From Federal Express to Lambdas:
Transporting Sloan Digital Sky Survey
Data Using UDT
Robert Grossman, UIC
www.sdss.org
Evolution is the Principle of Biological Systems:
Most of Evolutionary Time Was in the Microbial World
You
Are
Here
Much of Genome
Work Has
Occurred in
Animals
Source: Carl Woese, et al
$24.5M Over 7 Years
PI Larry Smarr
Announcing Tuesday January 17, 2006
The Sargasso Sea Experiment
The Power of Environmental Metagenomics
•
•
•
•
MODIS-Aqua satellite image of
ocean chlorophyll in the Sargasso
Sea grid about the BATS site from
22 February 2003
Yielded a Total of Over 1 billion Base Pairs
of Non-Redundant Sequence
Displayed the Gene Content, Diversity, &
Relative Abundance of the Organisms
Sequences from at Least 1800 Genomic
Species, including 148 Previously Unknown
Identified over 1.2 Million Unknown Genes
J. Craig Venter,
et al.
Science
2 April 2004:
Vol. 304.
pp. 66 - 74
Marine Genome Sequencing Project
Measuring the Genetic Diversity of Ocean Microbes
CAMERA will include
All Sorcerer II Metagenomic Data
Metagenomics “Extreme Assembly”
Requires Large Amount of Pixel Real Estate
Prochlorococcus
Microbacterium
Rhodobacter
SAR-86
unknown
Burkholderia
unknown
Source: Karin Remington
J. Craig Venter Institute
Calit2’s Direct Access Core Architecture
Will Create Next Generation Metagenomics Server
Sargasso Sea Data
Moore Marine
Microbial Project
NASA Goddard
Satellite Data
Community Microbial
Metagenomics Data
DataBase
Farm
Flat File
Server
Farm
10 GigE
Fabric
Request
+ Web Services
JGI Community
Sequencing Project
W E B PORTAL
Sorcerer II Expedition
(GOS)
Traditional
User
Dedicated
Compute Farm
(100s of CPUs)
Response
Direct
Access
Lambda
Cnxns
Local
Environment
Web
(other service)
Local
Cluster
TeraGrid: Cyberinfrastructure Backplane
(scheduled activities, e.g. all by all comparison)
(10000s of CPUs)
Source: Phil Papadopoulos, SDSC, Calit2
First Implementation of
the CAMERA Complex
Compute
Database &
Storage
Calit2/SIO will Establish Persistent
OptIPuter Collaboratory with Venter Institute
www.calit2.net/articles/article.php?id=660
August 8, 2005
25 Miles
SIO/UCSD
OptIPuter
Visualized
Data
HDTV Over
Lambda
Venter
Institute
NASA
Goddard
Calit2/SDSC Proposal to Create a UC Cyberinfrastructure
of “On-Ramps” to National LambdaRail Resources
OptIPuter + CalREN-XD
+ TeraGrid = “OptiGrid”
UC Davis
UC San Francisco
UC Berkeley
UC Merced
UC Santa Cruz
UC Los Angeles
UC Santa Barbara
UC Riverside
UC Irvine
UC San Diego
Creating a Critical Mass of End Users
on a Secure LambdaGrid
Source: Fran Berman, SDSC , Larry Smarr, Calit2