Transcript N:Y - The ACIS Lab - University of Florida

IP over P2P: Enabling Selfconfiguring Virtual IP
Networks
for
Grid Computing
Arijit Ganguly, Abhishek Agrawal,
P. Oscar Boykin, Renato Figueiredo
University of Florida
IPDPS 2006
Advanced Computing and Information Systems laboratory
What is the talk about?


Convergence of Grid and P2P
technologies1
Context of network virtualization
1 On death, taxes, and the convergence of peer-to-peer and
Grid Computing. Foster et al. IPTPS 2003
Advanced Computing and Information Systems laboratory
2
Outline





Virtual networking and Grid Computing
Related work
Our approach – IP over P2P
Experimental evaluation
Conclusion and Future work
Advanced Computing and Information Systems laboratory
3
Background - Virtual Private Networks
Rhodes, Greece


Install Cisco VPN client
Connect to VPN gateway
Tunnel
Internet access
Internet
User inside
ACIS private
network
router
VPN gateway NAT/Firewall
printers
Advanced Computing and Information Systems laboratory
Files, emails,
compute cycles
4
Grid scenario
Issues:
 Idiosyncrasies of heterogeneous access
Purdue
 Network Address Translation
NAT
 Traffic generated
by untrusted code from
router
LSUGrid users – DoS attacks, viruses router
router
LSU
NAT/Firewall
router
Internet
router
Internet
Purdue
NAT
router
router
Firewall
router
Firewall
NAT/Firewall
Florida
Florida
Grid
Griduser
user
Advanced Computing and Information Systems laboratory
Northwestern
Northwestern
SSH only
SSH only
5
Virtual network of Grid resources
Virtual Network
Purdue
LSU
router
router
NAT
router
Internet
router
Firewall
NAT/Firewall
Florida
Grid user
Advanced Computing and Information Systems laboratory
SSH only
Northwestern
6
Virtual networking for Grids

VNET (Northwestern University)
• Bridge a remote Virtual Machine (VM) to a
client network

VIOLIN (Purdue University)

ViNe (University of Florida)
• Virtualized network components
• Isolated from real physical network
• Virtual IP network of Grid resources
• To be presented on Friday (Session 32)
Common technology: Overlay tunneling
What differentiates us: P2P routing
Advanced Computing and Information Systems laboratory
7
Motivations for P2P



Scalability and Self-configurability
• Manual effort required to add a new node
constant
• Independent of size of the network
Resiliency
• Robust P2P routing
Accessibility
• Ability to traverse NAT
• Hole punching1
1 RFC 3489 - STUN - Simple traversal of User
Datagram Protocol through Network Address Translators
Advanced Computing and Information Systems laboratory
8
Our approach – IP- over-P2P (IPOP)

Isolation
#affiliation
Virtual addresscondor_wow
space decoupled from Internet
#transport
address spaceudp
#port
 Self-configurability
15000
Automatic setup
of routes
andTAs
topologies
#number
of remote
2
Decentralized
#list of TAs
• No global state
• No central brunet.udp://planetlab-01.bu.edu:15000
points of failure
brunet.udp://planetlab1.cs.purdue.edu:15000
VM mobility #virtual interface
 Decentralizedtap0
NAT
traversal
#virtual IP address of tap0
No changes to172.16.1.5
NAT configuration
#MAC address
of tap0
No globally deployed
STUN
servers
CB:DF:E7:20:60:35
•
•
•
•
•
•
Advanced Computing and Information Systems laboratory
9
IPOP - Architecture Overview

IP tunneling over P2P overlay networks

Virtual IP packet capture and injection
through tap interface

Builds upon Brunet P2P library
• UDP, TCP
Advanced Computing and Information Systems laboratory
10
IPOP – Packet capture and routing


Extract IP from Ethernet
Encapsulate IP inside P2P
application
Node
X


Extract IP from P2P
Encapsulate in Ethernet
Node
application
Y
IPOP
IPOP
Y
X
tap0
tap0
(172.16.0.9)
(172.16.0.10)
eth0
Socket s =
new Socket(“172.16.0.10:3000”);
s.connect();
Advanced Computing and Information Systems laboratory
eth0
ServerSocket serv =
new ServerSocket(“172.16.0.10,3000);
serv.accept()
11
Brunet P2P architecture


Ring-structured overlay network topology
• Nodes ordered on 160-bit addresses
Overlay link:
• Near: neighbor connections
• Far: connections across ring
U
V
Multi-hop path
between X and Y
X
Y
Far connection
Near connection
Advanced Computing and Information Systems laboratory
12
Brunet P2P architecture (2)

Routing
• Constant number of connections
• O(log2(n)) overlay hops
• O(log(n)) connections
• O(log(n)) overlay hops
• n connections
• 1-hop

C# library, supports:
• Connection setup and maintenance
• NAT traversal
Advanced Computing and Information Systems laboratory
13
Network Address Translation (NAT)
216.239.37.99
128.227.56.83
10.5.144.69
Sends packet:
Host A Src = 10.5.144.69:5000
Dst = 216.239.37.99:80
NAT
Sends packet:
Src = 216.239.37.99:80
Dst = 10.5.144.69:5000
Translated:
Src = 128.227.56.83:5126
Dst = 216.239.37.99:80
Public
host
Sends packet:
Src = 216.239.37.99:80
Dst = 128.227.56.83:5126
NAT Tables
10.5.144.69:5000  128.227.56.83:5126
Outgoing packet to 128.227.56.83:5126
Applications on NATed hosts can learn
their NAT assigned IP:port
Advanced Computing and Information Systems laboratory
14
NAT traversal – Behind NATs
N:Y  S:B
Outgoing packet to M:X
(hole punched)
R:A  M:X
Outgoing packet to N:Y
(hole punched)
Exchange each other’s NAT assigned IP:port
Dropped
Src = S:B
Dst = M:X
Src = N:Y
Dst = M:X
Allow
R:A
M:X
Src = R:A
Dst = N:Y
N:Y
Src = M:X
Dst = N:Y
NAT M
128.227.56.83
Advanced Computing and Information Systems laboratory
S:B
Src = M:X
Dst = S:B
NAT N
128.139.156.90
15
Experiments

Latency overhead and throughput of single
overlay link
•


LAN and WAN
MPI application over IPOP
•
Light Scattering Spectroscopy (LSS)
Multi-hop routing experiments
•
More than 100 node network on PlanetLab
Advanced Computing and Information Systems laboratory
16
Latency (single IPOP link)

Two IPOP nodes separated by single overlay
hop
ACIS – ACIS for LAN
ACIS – VIMS for WAN

Ping times between two nodes

6ms-11ms overhead per packet for ICMP ping
Relative overhead is smaller in Wide-Area

ACIS: Florida
VIMS: Virginia
Advanced Computing and Information Systems laboratory
17
Latency overhead - analysis

Reasons for high LAN overhead:
• Double traversal of kernel stack
• C# runtime
• User-level overlay – context switches
• Other user-level overlays (VNET, Violin)
report few-ms latency overheads
Advanced Computing and Information Systems laboratory
18
Throughput (single IPOP link)

Two IPOP nodes separated by single overlay
hop
ACIS – ACIS for LAN
ACIS – VIMS for WAN

“ttcp”
 file transfer sizes (13.09 MB, 92.97 MB)

1.9MB/s LAN bandwidth (20% of physical 9.4
MB/s)
1.2MB/s WAN bandwidth (80% of physical 1.5
MB/s)

ACIS: Florida
VIMS: Virginia
Advanced Computing and Information Systems laboratory
19
Real Application – Parallel LSS

MPI + NFS + SSH11
LSU
router
Florida
Firewall
NAT/Firewall
router
Internet
router
NAT
VIMS
NAT/Firewall
1 Support for Data-Intensive, Variable-Granularity Grid Applications via
Distributed File System Virtualization - A Case Study of Light Scattering
Spectroscopy. Figueiredo et al. CLADE 2004
Advanced Computing and Information Systems laboratory
20
Real Application – Parallel LSS


With IPOP, could run “parallel LSS”
unmodified
No changes to NAT/Firewall rules
Achieve parallel speedup
Advanced Computing and Information Systems laboratory
21
PlanetLab experiments




Demonstrate ease of adding a new node
and achieving IP routability in WAN
environment
118 node TCP-based overlay on
PlanetLab
Connect two IPOP nodes in ACIS lab to
PlanetLab network
Measure ping times between nodes
• Average: 1617 ms; Std Dev: 2098 ms
Advanced Computing and Information Systems laboratory
22
Planetlab experiments (analysis)

Issues:
• High-load (>10) on nodes in routing path
• Geographically unaware p2p routing
• Packets between machines in Florida routed
through machines in California

Improvements:
• Direct overlay link setup between
communicating nodes
• No concerns of load and inefficient p2p routing
Advanced Computing and Information Systems laboratory
23
Conclusion

Our contribution:
• Novel virtual IP network based on P2P
overlay
• Scalable and Self-configurable
• Resilient
• NAT traversal
• Experiments showed feasibility of using
P2P approach for virtual networking
Advanced Computing and Information Systems laboratory
24
Future work

Overhead of TCP or UDP
• Raw sockets or Ethernet-based
overlay edges

Kernel level extensions
• Tap module with encapsulation and
bridging
• Reduce context switches
Advanced Computing and Information Systems laboratory
25
Related Work

Virtual Networking

Internet Indirection Infrastructure (i3)

IPv6 tunneling
• VIOLIN
• VNET
• ViNe (Session 32)
• Support for mobility, multicast, anycast
• Decouples packet sending from receiving
• Based on Chord p2p protocol
• IPv6 over UDP (Teredo protocol)
• IPv6 over P2P (P6P)
Advanced Computing and Information Systems laboratory
26
Acknowledgments


In-VIGO team at UFL
National Science Foundation
•
•
Middleware Initiative (http://www.nsf-middleware.org)
Research Resources Program
• nCn center


Resources
•
•
Peter Dinda (Northwestern University)
SURA/SCOOP
IBM Shared University Research
Questions?
Advanced Computing and Information Systems laboratory
27
Thank You
Advanced Computing and Information Systems laboratory
28