Transcript GCB Tutorial OGF 2007

GCB Tutorial
OGF 2007
Todd Tannenbaum
Condor Team
http://www.cs.wisc.edu/condor
What is GCB?
› GCB is the Generic Connection Broker
Included in Condor 6.7.13 (Nov 2005) and later
Linux-only
› It solves the “firewall traversal problem”
› So what is the firewall traversal problem?
www.cs.wisc.edu/condor
A Simple Condor Pool
Communication is initiated
in two directions
Matchmaker
Executor
Submitter
Note: This is a subset of
communication in Condor
www.cs.wisc.edu/condor
What If There Is A Firewall?
› Firewalls usually block incoming traffic on
most ports
› “Incoming” depends on your perspective:
Organizations have firewalls to protect from
computers outside the organization
Individual computers have firewalls to
protect from other computers
www.cs.wisc.edu/condor
A Condor Pool With Firewall
X
Matchmaker
X
Submitter
Executor
www.cs.wisc.edu/condor
How Can You Traverse Firewalls?
› Punch a hole
Configure firewall to allow traffic on a
certain range of ports to come through
Tell Condor to restrict itself to use only this
range
Bummer: Condor can use many ports
Bummer: Punching holes makes people
nervous
www.cs.wisc.edu/condor
How Can You Traverse Firewalls?
› Use Condor-C
Put host on network edge
Open a couple of ports for it
Delegate jobs to this host
Matchmaker
Submitter
Re-Submitter
www.cs.wisc.edu/condor
Executor
How Can You Traverse Firewalls?
› Change Condor to always use outgoing
traffic
What if there are two firewalls or private
networks?
Which direction is “outgoing”?
› GCB automates this solution
It knows which direction is outgoing
It can proxy if there are two firewalls
www.cs.wisc.edu/condor
GCB: Contacting Executor
(One Possible Scenario)
1. Executor registers with GCB
(Permanent TCP connection)
2. Executor advertises to
matchmaker (GCB IP address)
3. After match, submitter
contacts executor, via GCB
2
Matchmaker
3
GCB
4
4. GCB tell executor to open
connection
1
Executor
5. Executor opens connection to
submitter
5
Submitter
www.cs.wisc.edu/condor
GCB
(Acting as Proxy)
1. Assume 1 port open for
matchmaker. (Can avoid…)
1
3
2. Executor advertises with
GCB (permanent connection)
3. Executor advertises to Matchmaker
matchmaker (GCB IP address)
4
4. After match, submitter
contacts executor, via GCB
5. Communication flows
through GCB, using both
connections
GCB
2
5
Submitter
www.cs.wisc.edu/condor
Executor
GCB Advantages
› Good connectivity
Works with multiple private networks
Works with network address translation
› Don’t need to punch holes in firewall
› GCB does not need to be run as root
› No changes to firewall configuration
www.cs.wisc.edu/condor
GCB Disadvantages
› GCB is a point of failure
 All communications through GCB, so if GCB fails…
› Computers behind a firewall share an IP
address (of GCB)
 Makes host-based security difficult
› Doesn’t work with Kerberos security
› Can slow down network performance
› Scalability issues
 A single GCB server is limited by number of ports
available on computer
› Complex to configure and debug
www.cs.wisc.edu/condor
Now for the Nitty Gritty…
www.cs.wisc.edu/condor
Setting Up GCB
1. Install GCB
2. Configure GCB
3. Configure Condor to use GCB
www.cs.wisc.edu/condor
Install GCB
› GCB comes with Condor
› GCB has two programs
gcb_broker: The “big brains” of GCB
gcb_relay_server: proxy for private net
to private net communication
› GCB was written independently of Condor
Can’t read condor_config directly
So create environment in condor_config
GCB reads from environment
www.cs.wisc.edu/condor
Install GCB
› GCB should be on computer with no other
services
GCB can use lots of ports, so avoid port
competition with other programs
Using GCB can slow down communication, so
keeping GCB on its own computer helps speed
› GCB needs to be on edge of network
On public network and private network
At least one GCB per private network
www.cs.wisc.edu/condor
Configure GCB
› To run from condor_master:
# Specify that you only want the master
# and the broker running
DAEMON_LIST = MASTER, GCB_BROKER
# Define the path to the broker binary
# for the master to spawn
GCB_BROKER=$(RELEASE_DIR)/libexec/gcb_broker
www.cs.wisc.edu/condor
Configure GCB
› GCB expects configuration in
environment. Sample:
GCB_BROKER_ENVIRONMENT =
## Provide
the full path to the gcb_relay_server
Provide the full path to the gcb_relay_server
GCB_BROKER_ENVIRONMENT = =
GCB_RELAY_SERVER=$(GCB_RELAY)
GCB_BROKER_ENV
GCB_RELAY_SERVER=$(GCB_RELAY)
# Tell GCB to write all log files into the
# Tell GCB to write all log files into the Condor log
## Condor
directory log directory
Note: more configuration options are available.
GCB_BROKER_ENVIRONMENT=(GCB_BROKER_ENVIRONMENT);GCB_LOG_DIR=$(LOG)
GCB_BROKER_ENV=$(GCB_BROKER_ENV);GCB_LOG_DIR=$(LOG)
# Tell GCB it
connect
to private network
Seecan
manual
for details
# Tell GCB it can connect=to private network
GCB_BROKER_ENV
GCB_BROKER_ENVIRONMENT=$(GCB_BROKER_ENVIRONMENT);GCB_ACTIVE_TO_CLIENT=yes
$(GCB_BROKER_ENV);GCB_ACTIVE_TO_CLIENT=yes
## Set
Setpublic
public
IPforaddress
IP address
GCB broker for GCB broker
GCB_BROKER_ARGS = -i 123.123.123.123
GCB_BROKER_ARGS
= -i 123.123.123.123
www.cs.wisc.edu/condor
Configure Condor to Use GCB
› In condor_config:
Turn on GCB:
NET_REMAP_ENABLE = true
NET_REMAP_SERVICE = GCB
# Point to GCB
NET_REMAP_INAGENT = 123.123.123.123
# Routing Table
NET_REMAP_ROUTE = /full/path/gcbroutes
www.cs.wisc.edu/condor
Set Up Routing Table
Public Network
123.123.123.*
Private Network
192.168.2.*
GCB Broker
123.123.123.123
Routing Table
123.123.123.123/32 GCB
*/0
direct
www.cs.wisc.edu/condor
Set Up Routing Table
Public Network
123.123.123.*
Routing Table
123.123.123.65/32
123.123.123.66/32
*/0
GCB Broker
123.123.123.65
GCB Broker
123.123.123.66
GCB
GCB
direct
Private Network
192.168.2.*
Private Network
192.168.2.*
www.cs.wisc.edu/condor
Security Implications
› Hosts in private network look like they
share a single IP Address (the address
of the GCB broker)
› If you use host-based security, you can’t
distinguish hosts in the private network
› GCB does not authenticate who it is
providing its proxy service for.
www.cs.wisc.edu/condor
More Information
› Section 3.8 of the Condor manual “Networking”
› http://www.cs.wisc.edu/~sschang/firewall/gcb
Thank You!!!
www.cs.wisc.edu/condor