Transcript IBP and Condor - Computer Sciences Dept.

IBP and Condor
Micah Beck
Assoc. Prof. & Director
Condor Week, Madison
May 6, 2003
Logistical Networking
Research at UTK
University of
Tennessee
• Micah Beck
• James S. Plank
• Jack Dongarra
University of
California,
Santa Barbara
• Rich Wolski
Funding
• Dept. of Energy
SciDAC
• National Science
Foundation ANIR
• UT Center for Info
Technology
Research
IBP: The Internet Backplane
Protocol
• A scalable mechanism for deploying shared
storage resources throughout the network
• A general store-and-forward overlay
networking infrastructure
• A state management infrastructure for
distributed applications and active services
The Network Storage Stack
Applications
• Our
adaption of the
network stack architecture
for storage
• Like the IP Stack
• Each level encapsulates
details from the lower
levels, while still exposing
details to higher levels
Logistical File System
Logistical Tools
L-Bone
exNode
IBP
Local Access
Physical
IBP: How it Works
• Storage provisioned on community “depots”
• Very primitive service (similar to block service,
but more sharable)
• Goal is to be a common platform (exposed)
• Also part of end-to-end design
• Best effort service – no heroic measures
• Availability, reliability, security, performance
• Allocations are time-limited!
• Leases are respected, can be renewed
• Permanent storage is to strong to share!
The Network Storage Stack
LoRS: The Logistical Runtime System:
Aggregation tools and methodologies
The L-bone:
Resource Discovery
& Proximity queries
The exNode:
A data structure
for aggregation
IBP: Allocating and managing network
storage (like a network malloc)
L-Bone: January 2003
Current Storage Capacity: 13 TB
Multithreaded Transfers
Caching/Staging
Point-to-Multipoint
Heterogeneous Multicast
Relationship to Work of
Condor Group
• Wide Area File Management/Access
• Management of Computation State
• As a Storage Allocation Layer for
• Kangaroo, NeSt
• DiskRouter
• If routers can have disks, how about disks
with processors?
Routers, Depots and the
Network Functional Unit
in
in
send
router
out
NFU
disk/RAM
in
in
store
in
depot
out
in
depot
out
load
execute
RAM
Scalable Operations
• IBP Depots with NFU define a State
Transformation Substrate
• Processes run at “endpoints” but can use
NFUs to transform data in network
• Is it processor-in-storage or active
networking?
• All state is exposed
Logistical Computing and
Internetworking
http://loci.cs.utk.edu
Micah Beck
[email protected]