Transcript including - Internet2

International Collaborations
in Logistical Networking
Micah Beck and Terry Moore
Internet2 Member Meeting
April 10, 2003
Logistical Networking
Research
University of
Tennessee
• Micah Beck
• James S. Plank
• Jack Dongarra
University of
California,
Santa Barbara
• Rich Wolski
 LoCI Lab, including
• T. Moore
• A. Bassi
 Funding
• Dept. of Energy
SciDAC
• National Science
Foundation ANIR
• UT Center for Info
Technology Research
What is Logistical
Networking?
 A scalable mechanism for deploying shared
storage resources throughout the network
 An general store-and-forward overlay networking
infrastructure
 A way to break long transfers into segments and
employ heterogeneous network technologies
 P2P storage and content delivery that doesn’t use
endpoint storage or bandwidth
Why “Logistical
Networking”
 Analogy to logistics in distribution of
industrial and military personnel & materiel
 Fast highways alone are not enough
• Goods are also stored in warehouses for transfer or
local distribution
 Fast networks alone are not enough
• Data must be stored in buffers/files for transfer or
local distribution
 Conventional vs logistical networking
• Datagram routers make spatial choices
• Storage depots enable temporal choices
Network Storage Stack
 Model network storage
architecture on the IP stack
 Each level encapsulates details
from the lower levels, while
exposing details to higher levels
 Layers already in place: IBP,
exNode, L-Bone, Logistical
Runtime System
Applications
Logistical File System
Logistical Runtime Sys
L-Bone
exNode
IBP
Local Access
Physical
IBP: The Internet Backplane
Protocol
 Storage provisioned on community “depots”
 Very primitive service (similar to block service,
but more sharable)
• Goal is to be a common platform (exposed)
• Also part of end-to-end design
 Best effort service – no heroic measures
• Availability, reliability, security, performance
 Allocations are time-limited!
• Leases are respected, can be renewed
• Permanent storage is to strong to share!
exNode vs inode
IBP Allocations
 Metadata container for
aggregating storage
resources
 Encoded as XML for
network interoperability
exNode + Lbone + LoRS: Download
The Current L-Bone
160 depots, 16 countries, 26 US states, 10TB
Logistical Networking
Application Areas
 Multimedia
 Temporary storage
 Reducing (BWdelay)
 Reliable multicast
 Content Distribution
 Remote access to
structured data
 Heterogeneous
networks
 Managing computation
state
 Very large data sets
 Bandwidth adaption
 Source routing
 Collaborative
computing &
visualization
Planet Lab
 Global overlay network for developing and
accessing new network services
 Seeded by Intel Research, currently 96 sites
 IBP depots are running on them all and
account for a lot of the use of PlantLab
Lyon
 Home base of LoCI Europe (Alex Bassi)
 Tamanoir: HP networking project (INRIA –
ENS)• Using IBP for local data caching
• IBP Service deployed node start up
• Focus: Multimedia data distribution (semantic and
active cache, stream adaptation, data placement)
• Leaders: Laurent Lefevre (ENS)
 IBCP (Institute for Biology and Protein
Chemistry): Distribution of genome data
E-Toile project
 A computational Grid of several University,
Research and Industrial sites in France
 7 participating sites, 4 of which will have
IBP depot facilities at the beginning of the
project
 Depots are have arrived. Each will be a
dedicated machine (e.g. the ENS at Lyon
one will be a Sun with 1.3 Tb storage
available)
E-Toile Grid Architecture
ARCHITECTURE PHYSIQUE DE LA GRILLE
e-TOILE
12 PC bipro
1 Gb/s
IRISA
Rennes
1 Gb/s
CEA
Saclay
1 Gb/s
VTHD
2.5Gb/s
SUN
Grenoble
1 Gb/s
ENS
Lyon
EDF
Clamart
1 cluster de PC
Machine SMP
Serveur bipro
1,2 Ghz
Routeur actif
Service de dépôt de
données transientes IBP
Serveur 8 processeurs
1 Gb/s
1 Gb/s
Clusters de PC 1,9 Ghz PRiSM
Versailles
250 PC en cluster
2 Gb/s
6 PC bipro
16 PC bipro
ID-IMAG
Grenoble
1 cluster Myrinet de 10 PCs
1 cluster de 8 PCs
8 x 2 PC
linked by SCI
Serveur bipro
IBCP
MP760
16 power PC
linked by Myrinet
16 Sun Cobalt
Serveur 3* bipro
CERN DataTag project
 DataTag project: High performance Grid
infrastructure for sustained, reliable, high
performance data movement replication
 Minimum of 100Mbps in the early stages
 European Collaborators: CERN, INFN, INRIA,
University of Amsterdam
 VLAN via STARLIGHT with 10GigE interface to
Abilene
 Recently installed an IBP depot with 2.5Gbps
connection to Chicago
Italy
 GARR: Italian Academic & Research Network
• Collaborator in the International 6Bone project
• Coming up in the next few weeks: 3 dedicated servers
to be put on the PoPs (each one around 140 Gb for the
beginning).
• Gabriella Paolini
 Universita del Piemonte Orientale
• Depots at UPO, Surfnet, Tennessee
• Cosimo Anglino – Data Transfer Scheduling
Optimization
Initial GARR depot locations
PoP 6net
TORINO
Milano RT
GEANT
6NET
GARR
Milano RTG
PoP 6net
PISA
6NET
BACKBONE
MILANO
Pisa RTG
GigaPoP Milano
Via Lancetti
Bologna RTG
PoP 6net
BOLOGNA
Roma RTG
PoP 6net
ROMA
GigaPoP Roma
P. A.Moro
Other projects
 Univ. of San Paulo:
• Research to enhance app
performance on Brazillian
research networks (Dr. Sergio
Kofuji, Flavio Silva)
Nanyang Technological
Univ. (Singapore)
• Grid Replica Catalogue
over IBP (Dr. Francis Lee &
Ming Tang)
 University of Puerto Rico: Thailand
• Biomedical Research
Infrastructure Networks in
Puerto Rico (BRIN-BR) (Guy
Cormier)
• Putchong Uthayopas of
Kasetsart University
Australian National U.
• Content distribution
(Markus Buckhorn)
International Project
Futures
Proposals in development for the European
commission
 Active Grid Networks (AGridNet)
• Programmable Grid infrastructure focused more on
research applications
 Polymorphic Internet
• Large public/private consortium led by Alex Gallis
(University of London)
• Programmable cyberinfrastructure focused on
commercial applications
Conclusion: Join Us!
 As a user
• Open infrastructure
• Tools are free and easy to use
 As a provider
• Storage is cheap (<5k per TB)
• Depots are easy to deploy
• Server many application communities at once
 More information and software available at
http://loci.cs.utk.edu