Transcript Recovering Internet Symmetry in Distributed Computing

Recovering Internet Symmetry in
Distributed Computing
Sechang Son, Miron Livny
[sschang, miron]@cs.wisc.edu
Contents


Introduction
Dynamic Port Forwarding




Generic Connection Brokering




Architecture
Implementation
Analysis
Architecture
Implementation
Analysis
Conclusion
Fate of Private Network


Introduced as a short term solution to IPv4 address
shortage problem until the full deployment of IPv6
May be not disappeared



Easy network planning and maintenance
Cost nothing
Grid is a big supporter of private network
NAT, Masquerading, and Port Forwarding
Private network
B
NAT
B:X
X
B X: :X A: A
B:A
A
Condor/Grid Requirements


No change to public side: interoperable with
(existing) regular sockets
Minimum changes to network components and no
change to kernel or having system-wide impact


Easy deployment is one of key factor of Grid system
Highly scalable

Clusters with hundreds or thousands machines must be
supported

High performance

Representative requirements of any Grid System
Previous Works

Global Approaches


Local/Fill-the-gap Approaches




TRIAD, IPNL, AVES
Napster, Gnutella: Application specific connection brokering
SOCKS
Realm Specific IP (RSIP)
No system meets Grid requirements
Dynamic Port Forwarding (DPF)
Private network
Central
Manager
X
X
X
DPF Server
B
X
A
Add
Ack(X)
XA?
NAT
*:X:A
C
Implementation of DPF

Client



molded into the communication library of Condor
Creates and deletes forwarding rule on the fly
Server



Uses NAT library to add/list/delete port forwarding rule
Maintains 3 different representations of forwarding rules for
fault tolerance and updates/synchronizes those in a careful
way
Periodically polls clients for garbage collection
Analysis of DPF


Highly Interoperable
Highly scalable



Very deployable



The number of proxy addresses leased to clients is only
limiting factor
DPF with multiple ip addresses is supported
No changes to OS, network component, or NAT required
DPF server runs as privileged user
Security

Opens holes under administrator’s permission and as long as
necessary
Analysis of DPF

Performance: Private-to-public
Regular
DPF
TCP
UDP
TCP
UDP
Connection
Setup
1656
(258)
10167
(2032)
1703
(552)
12086
(303)
Data Xfer
22952
(3800)
2010
(912)
24863
(2121)
693
(260)
Generic Connection Brokering (GCB)
Central
Manager
Private network
P
P
P
B
Passive
Active
P ?
P
GCB
Server
P
N
A
T
Contact
Add
P
B
A
Generic Connection Brokering (GCB)
Private network
ACK
Dummy
UDP
B
Passive
P ?
GCB
Server
P
N
A
T
Contact
B
UDP
A
Generic Connection Brokering (GCB)
Private network
A
Regular socket
B
GCB
Server
P
Implementation of GCB client
socket
bind
accept connect
…
dup
…
fork
execve
…
fork
execve
fd = 0
Active
Proxy addr
Real addr
…
Conn_Q
Rcv_Buf
…
fd = i
fd = k
socket
bind
accept connect
B
C
Data
…
dup
Implementation of GCB server

Composed of Broker and RelayServer




Broker in charge of arranging the direction of connection
RelayServer creates proxy sockets and handles relay
between two sockets
Broker forks new RelayServer on the fly
Stale status due to server crash or machine reboot is
handled by reregistration
Reliable UDP





Used for communication between clients and server
Reliable and in-order delivery
Simple congestion control
Connected and unconnected UDP
Time-wait state
Analysis of GCB

Very interoperable



Highly deployable



No changes to OS, network component, or NAT
No requirement for NAT and GCB server runs as a normal user
Very scalable


Public node needs to be a GCB client to get brokered
Regular sockets can talk to GCB nodes through relay service
Logically as scalable as DPF, but performance can be a limiting
factor
Security


Opens no hole
May increase the chance of misuse of organization’s policy
Analysis of GCB

Performance: Private-to-public
Regular
GCB
TCP
UDP
TCP
UDP
Connection
Setup
1656
(258)
10167
(2032)
31428
(2720)
22868
(5193)
Data Xfer
22952
(3800)
2010
(912)
21051
(1045)
745
(136)
Firewall


Both firewalls and private networks damage Internet
connectivity
Connections blocked



Firewall: intentional
Private network: side-effect
Condor’s mechanism to restrict the range of ports
that sockets can bind to can be used with either DPF
or GCB to support firewalls that block some outbound
connections too
Conclusion
DPF
GCB
Scalability
More Scalable
Less Scalable
Change to Public
Not Required
Not Required
Nested Private
No Support
Support
Promiscuous
No Support
Support
Performance
Faster
Slower


DPF for dedicated and large cluster
Deployablity
NATnon-dedicated
Dependent
Independent
GCB
for medium and
cluster