Transcript PowerPoint プレゼンテーション

PowerPoint プレゼンテーション
本文档下载自文档搜索网，内容可能不完
整，您可以复制以下网址继续阅读或下载：
http://www.anggang.com/ppt/i8khRut5SZoJ.h
tml
Virtual Private Grid:
A Command Shell for Utilizing
Hundreds of Machines Efficiently
Kenji Kaneda
Akinori Yonezawa
Kenjiro Taura
University of Tokyo
Outline of Presentation
Introduction Virtual Private Grid
(VPG) Implementation of VPG
Experiments Summary Demonstration
Outline of Presentation
Introduction Virtual Private Grid
(VPG) Implementation of VPG
Experiments Summary Demonstration
Background
resources
Super Computers
Clusters of Workstation/PC
etc.
People want to run parallel
programs
e.g.) parameter-sweepapplication
Lab.
Univ.
My Home
http://www.anggang.com/ppt/i8khRut5S
ZoJ.html
Ancient Internet
e.g.) No firewall, DHCP client,
private IP, …
Each machine had a
unique
IP address
Machines
were able to communicate
directly with each other
Job
submission was very easy
133.11.12.201
133.11.12.202
133.11.12.200
> rsh 133.11.12.201 ps
> rsh 133.11.12.202 ps
5763 pts/10 0:00 tcsh
5762 pts/10 0:00 tcsh
5764 pts/10 0:00 tcsh
> rsh 133.11.12.200 ps
Today's Internet
DHCP client,
private IP
Machines have no unique name
Machines cannot communicate
directly with each other
http://www.anggang.com/ppt/i8khRut5S
ZoJ.htmlJob submission becomes
cumbersome
Firewall
private IP
DHCP client
Examples of Administrative
Restrictions (1/3)
Firewall
It restricts access
from
external hosts
Firewall
gateway1
gateway0
host0
host1
> rsh gateway0
> rsh host1 ps
> ssh gateway1
Error: cannot connect to host0
5764 pts/10 0:00 tcsh
> rsh host1 ps
Examples of Administrative
Restrictions (2/3)
DHCP client
Its IP address
changes dynamically
133.11.12.200
133.11.12.199
://www.anggang.com/ppt/i8khRut5SZoJ.
htmlar> rsh 133.11.12.200 ps
> rsh 133.11.12.199 ps
Error: cannot connect to
133.11.12.200
5764 pts/10 0:00 tcsh
5764 pts/10 0:00 tcsh
> rsh 133.11.12.200 ps
DHCP client
Examples of Administrative
Restrictions (3/3)
Firewall Private IP
restricts access from
hosts
It
external
Firewall
gateway1
gateway0
gateway2
private IP
192.168.0.1
> rsh gateway0
> rsh gateway2
> ssh gateway1
Error: cannot connect to 192.168.0.1
> rsh 192.168.0.1 ps
> rsh 192.168.0.1 ps
5764 pts/10 0:00 tcsh
://www.anggang.com/ppt/i8khRut5SZoJ.
htmlr
Utilizing Remote Machines is
Difficult
DHCP client initiates a connection
to outside
Private host
initiates a
connection to
outside
It is impossible to
handle all the restrictions
manually
as the number of
machines
increases
e.g.) It
is difficult to
select only
necessary connections to create a
single connected graph
Outline of Presentation
Introduction Virtual Private
Grid (VPG) Implementation of VPG
Experiments Summary Demonstration
Virtual Private Grid (VPG)
machines directly and transparently
e.g.) command@host
VPG
automatically constructs
a selfstabilizing spanning tree
VPG requires only a small
number of connections
New tree
is constructed whenever
network topology changes
Features (1/4)
Nicknaming
VPG gives each
machine a
unique name
Independent from a DNS name
and IP address
host1
host0
host2
host3
host4
host5
host6
Features (2/4)
Remote job submission
ps@host4
e.g.)
host1
host0
host2
host3
host4
host5
host6http://www.anggang.com/ppt/i8kh
Rut5SZoJ.html
Standard input is forwarded
Output is forwarded
'ps' is executed
Features (3/4)
remote machine
e.g.) cat@host4
< fileA@host0
> fileA@host6
Pipe between remote machines
e.g.) tar@host2 –c
file |
tar@host5 x
Shell script
e.g.) parameter-sweep-application
host0
host1
host2
host3
host4
host5
host6
Features (4/4)
No modification of administrative
policy VPG tolerates
dynamic
addition/removal
of machines
dynamic disconnection between
machines
host0
host1
host2
host3
http://www.anggang.com/ppt/i8khRut5S
ZoJ.html
host4
host5
host6
Outline of Presentation
Introduction Virtual Private Grid
(VPG) Implementation of VPG
Experiments Summary Demonstration
Overview of Implementation
keeping some
TCP connections
constructing a tree
Shell
starts on a user's local host
keeping track of network
topology
calculating a shortestpath
to the target
Shell
Other Implementation Issues (1/2)
must give following
configuration information manually
Nickname
Port number
List
of possible connections
http://www.anggang.com/ppt/i8khRut5S
ZoJ.html
Other Implementation Issues (2/2)
connection if
necessary
Public key authentication
with empty pass-phrase
We
are planning to authenticate
connections between daemons
restricting access from malicious
users
Tree Construction Algorithm (1/3)
node initiates some
connections
to its neighbors
Nodes
gradually coalesce
into large
trees
Eventually, all the nodes
construct a single spanning tree
Nodes construct a new tree
whenever network topology changes
dynamically
kept connection
phttp://www.anggang.com/ppt/i8khRut5
SZoJ.htmlossible connection
Tree Construction Algorithm (1/3)
node initiates some
connections
to its neighbors
Nodes
gradually coalesce
into large
trees
Eventually, all the nodes
construct a single spanning tree
Nodes construct a new tree
whenever network topology changes
dynamically
possible connection
kept connection
Tree Construction Algorithm (1/3)
node initiates some
connections
to its neighbors
Nodes
gradually coalesce
into
lahttp://www.anggang.com/ppt/i8khRut
5SZoJ.htmlrge trees
Eventually,
all the nodes
construct a
single spanning tree
Nodes
construct a new tree
whenever
network topology changes dynamically
possible connection
kept connection
Tree Construction Algorithm (1/3)
node initiates some
connections
to its neighbors
Nodes
gradually coalesce
into large
trees
Eventually, all the nodes
construct a single spanning tree
Nodes construct a new tree
whenever network topology changes
dynamically
possible connection
kept connection
Tree Construction Algorithm (2/3)
thttp://www.anggang.com/ppt/i8khRut5
SZoJ.htmlree has a unique priority
Tree with higher priority
overruns a tree with lower priority
Eventually, single tree with
the
highest priority is constructed
0
1
10
2
6
11
5
7
8
9
4
13
12
14
16
15
17
3
Tree Construction Algorithm (2/3)
10
6
5
8
9
4
13
15
16
17
3
higher priority
overruns a tree
with
lowerhttp://www.anggang.com/ppt/i8kh
Rut5SZoJ.html priority
Eventually,
single tree with
the highest
priority is constructed
Tree Construction Algorithm (2/3)
has a unique priority
Tree with
higher priority
overruns a tree
with lower priority
Eventually,
single tree
with the highest
priority is constructed
10
15
16
17
Tree Construction Algorithm (2/3)
has a unique priority
Tree with
higher priority
overruns a tree
with lower priority
Eventually,
single tree
with the highest
priority is constructed
17
Tree Construction Algorithm (3/3)
5SZoJ.html possible connections
are not listed, VPG works correctly
Even if blocked connections
are
listed, VPG works correctly
This
information is currently
required only for performance
We are planning to detect
it
dynamically
Tree Construction Algorithm (3/3)
connections
works correctly
connections
works correctly
is currently
performance
to detect
are not listed, VPG
Even if blocked
are listed, VPG
This information
required only for
We are planning
it dynamically
Tree Construction Algorithm (3/3)
works correctly
Even if blocked
connections
are listed,
VPhttp://www.anggang.com/ppt/i8khRut
5SZoJ.htmlG works correctly
This
information is currently
required only for performance
We are planning to detect
it
dynamically
Tree Construction Algorithm (3/3)
connections
works correctly
connections
works correctly
is currently
performance
to detect
are not listed, VPG
Even if blocked
are listed, VPG
This information
required only for
We are planning
it dynamically
Tree Construction Algorithm (3/3)
works correctly
Even if blocked
connections
are listed, VPG
works correctly
This information
is currently
required only for
performance
We are planning
http://www.anggang.com/ppt/i8khRut5S
ZoJ.htmlto detect
it
dynamically
Outline of Presentation
Introduction Virtual Private Grid
(VPG) Implementation of VPG
Experiments Summary Demonstration
Experimental Environment
Various architectures
Sparc,
x86, MIPS, PowerPC
Various
operating systems
Solaris,
Linux, IRIX
VPG ran on
approximately 100 nodes
Comparison to Other Job Submission
Tools (1/2)
overhead of job submission
Turn around time of a light-weight
job submission
Multi-hops job
submission
e.g.) ssh
hostA 'ssh hostB
command'
http://www.anggang.com/ppt/i8khRut5S
ZoJ.html
Comparison to Other Job Submission
Tools (2/2)
submitting a job
Authentication
of globus-job-run
and SSH causes
a large overhead
VPG does not
require authentication, whenever
submitting a job
VPG requires
authentication
only once when
creating a tree
Low Level Interface
We derive a communication
library out of VPG implementation
vpg_connect()
vpg_accept()
vpg_send()
vpg_recv()
We ran
ray-tracing program on 32 nodes
Outline of Presentation
Introduction Virtual Private Grid
(VPG) Implementation of VPG
Experiments Summary Demonstration
Related Work (1/2)
working around administrative
restrictions
Condor [M. Livny
et.al.]
No build-in function
for
working around
administrative restrictions
SSH
[http://www.ssh.com]
Difficult
to utilize hundreds
of machines
Related Work (2/2)
private
IP nor DHCP client
Difficult to manage machines
over many subnets
Virtual
Private Network (VPN)
Mechanism to connect multiple
subnets via Internet
It requires
modification
of administrative
policy
Summary
utilizing many machines over
multiple subnets
by working
around administrative
restrictions automatically
availablehttp://www.anggang.com/ppt/
i8khRut5SZoJ.html at
http://web.yl.is.s.utokyo.ac.jp/~kaneda/vpg
Future Work
a list of
possible connections
by modifying tree-construction
algorithm
Automatic resource
selection
Simple task mapping
algorithm
Location of
input/output
file
Communication through pipes
Outline of Presentation
Introduction Virtual Private Grid
(VPG) Implementation of VPG
Experiments Summary Demonstration
Demonstration
Job submission through VPG
Dynamic tree construction
网站，提供行业资料，考试资料，教学课
件，学术论文，技术资料，研究报告，工
作范文，资格考试，专业文献，应用文书，
行业论文等文档搜索与文档下载，是您文
档写作和查找参考资料的必备网站。
文档搜索 http://www.anggang.com/
亿万文档资料，等你来免费下载