Transcript An exposed approach to reliable multicast in heterogeneous

An Exposed Approach to Reliable Multicast
in Heterogeneous Logistical Networks
Micah Beck, Assoc. Prof. & Director
Logistical Computing & Internetworking (LoCI) Lab
Grids and Advanced Networking
Tokyo, 14 May 2003
Credits
• Authors
•
•
•
•
Micah Beck
Ying Ding
Erika Fuentes
Sharmila
Kancherla
• LoCI Lab
•
•
•
•
•
James S. Plank
Terry Moore
Alex Bassi
Yong Zheng
Hunter Hagewood
• PlanetLab
Logistical Networking
Research at UTK
University of
Tennessee
• Micah Beck
• James S. Plank
• Jack Dongarra
University of
California,
Santa Barbara
• Rich Wolski
Funding
• Dept. of Energy
SciDAC
• National Science
Foundation ANIR
• UT Center for Info
Technology
Research
What is Logistical
Networking?
• A scalable mechanism for deploying shared
storage resources throughout the network
• A general store-and-forward overlay
networking infrastructure
• A way to break transfers into segments and
employ heterogeneous network
technologies on the pieces
Why “Logistical
Networking”
• Analogy to logistics in distribution of
industrial and military personnel &
materiel
• Fast highways alone are not enough

Goods are also stored in warehouses for
transfer or local distribution
• Fast networks alone are not enough

Data must be stored in buffers/files for
transfer or local distribution
The Network Storage
Stack
Applications
• Our
adaptation of the
network stack architecture
for storage
• Like the IP Stack
• Each level encapsulates
details from the lower
levels, while still exposing
details to higher levels
Logistical File System
Logistical Tools
L-Bone
exNode
IBP
Local Access
Physical
IBP: The Internet Backplane
Protocol
• Storage provisioned on community “depots”
• Very primitive service (similar to block service,
but more sharable)
• Goal is to be a common platform (exposed)
• Also part of end-to-end design
• Best effort service – no heroic measures
• Availability, reliability, security, performance
• Allocations are time-limited!
• Leases are respected, can be renewed
• Permanent storage is to strong to share!
Data Movers
• Module implementing standard point-tomultipoint transfer between IBP allocations
• Uniform API allows independence from the
underlying data transfer protocol
• Not every DM can apply to every transfer
• Caller responsible for determining validity
• Current options: Multi-TCP, Multi-UDP
(reliable), UDP Multicast (unreliable)
mcopy operation
• Encapsulates shared buffering, management
of multiple low level transfers
Memory
Receiving Depots
Sending Depot
1. Buffering
File System
2. Parallel
Transfers
Heterogeneity in mcopy
•
•
•
•
TCP connections
Unreliable UDP multicast
Reliable UDP with flow control, retransmit
Reliable UDP with TCP control channel
• SABUL (R. Grossman, University of Chicago)
• Reliability must be end-to-end!
Comparison of Sending
Rates in the LAN
Mb/s
60
UDP MCAST
40
Pt-to-pt TCP
20
Pt-to-pt UDP
0
3
4
5
Destinations
6
Heterogeneous Multicast
End-to-End Reliability
through Retransmission
IBP depots
2. IBP mcasts
1. IBP
upload
source
5. TCP
retransmission
3. IBP
download
4. TCP control
|channel
destination
Other Approaches to
Reliable Multicast
• Retransmission in orginal group
• Multiple groups for retransmission assigned
dynamically to sets of missed blocks
• Retransmission from intermediate nodes
• Application-dependent approaches
• Video doesn’t need perfect reliability
• Time deadlines alter retransmission priorities
Exposed Approach to
Multicast
• Many important elements are under the
control of an endpoint (the source)
• Topology of multicast tree
• Choice of mcast operation types
• Handling of intermediate errors
• Performance optimization
• Global & app-specific strategies possible
Limitations of Exposed
Approach
• Scalability problems
•
•
•
•
Control from one end-point is limiting
Not sufficient for public media distribution
A distributed control infrastructure is required
Active routers provide a natural platform
• Tamanoir project of ENS-Lyon may provide
a testbed for this architecture
• Laurent Lefevre, Jean-Patrick Gelas
Topology and Performance
• Choosing tree nodes (can we detect
underlying Layer 2 topology?)
• Where is UDP multicast enabled?
• Where is are UDP flooding protocols legal?
• Evaluating reliability, performance of
component mcasts
• Trading off scalability for reliability and
performance
Experiment:
Three Approaches
• 10 recievers
• Direct Unicast TCP to all nodes
• Pure TCP overlay multicast
• TCP Data Mover used at every tree node
• Mixed TCP/UDP multicast
• TCP Data Mover used in backbone
• UDP multicast in edge networks
• Caveat: Measurements are not end-to-end!
Direct Unicast TCP
A
1
S
D
2
5
B
C
3
4
6
Pure TCP Overlay
Multicast
A
1
S
D
2
5
B
C
3
4
6
Mixed TCP in Backbone/
UDP Mcast at Edge
Experimental Results
Direct TCP vs Overlay
• 10 simultaneous TCP streams/connection
• 50 MB transfers
• Sending rate (not scaled by recievers)
• Direct TCP Unicast
• Pure TCP Overlay Multicast
• Speedup obtained: 50%
3.4 Mb/s
5.1Mb/s
Experimental Results
Overlay TCP vs Mixed
• 10 recievers
• No rate control on UDP Multicast, can’t run
multiple streams
• Comparing Overlay TCP with single TCP
stream/connection to Mixed, there is a 15%
speedup
• UDP at edge offers some speedup over TCP
Conclusions
• Logistical Networking implements a
scalable overlay networking infrastructure
• Data Movers provides support
heterogeneity even within a single transfer
• Exposed & heterogeneous multicast can
achieve speedups in the WAN
• Defining the tree and managing it for
reliability and performance is a challenge
L-Bone: January 2003
Current Storage Capacity: 13 TB
http://loci.cs.utk.edu
Micah Beck
[email protected]