Transcript Document

Dynamic Topology Adaptation of
Virtual Networks of Virtual Machines
Ananth I. Sundararaj
Ashish Gupta
Peter A. Dinda
Prescience Lab
Department of Computer Science
Northwestern University
http://virtuoso.cs.northwestern.edu
Summary
• Dynamically adapt applications in virtual
environments to available resources
• Demonstrate the feasibility of adaptation at the
level of collection of VMs connected by VNET
• Show that its benefits can be significant for the
case of BSP applications
• Studying the extent of applications for which our
approach is effective
2
Outline
•
•
•
•
•
•
•
•
Virtual machine grid computing
Virtuoso system
Networking challenges in Virtuoso
Enter VNET
VNET, VTTIFAdaptive virtual network
Experiments
Current Status
Conclusions
3
Virtual Machine Grid Computing
1
arbitrary amounts of
AimDeliver
computational power to perform
distributed and parallel computations
Traditional
Paradigm
New
Paradigm
2
Resource multiplexing using
Grid OS level mechanism
Computing
3b
5
Grid Computing
using virtual
machines
4
3a
6a
Problem1:
6b
Complexity from resource
Solution
user’s perspective
Problem2:
Complexity from resource
owner’s perspective
Virtual Machines
What are they?
How to leverage
them?
4
Virtual Machines
Virtual machine monitors (VMMs)
•Raw machine is the abstraction
•VM represented by a single
image
•VMware GSX Server
5
The Simplified Virtuoso Model
User’s
LAN
Virtual networking ties the
machine back to user’s
home network
Orders a raw
machine
VM
Specific hardware and
performance
Basic software
installation available
Virtuoso continuously monitors and adapts
User
6
User’s View in Virtuoso Model
User’s
LAN
VM
User
7
Outline
•
•
•
•
•
•
•
•
Virtual machine grid computing
Virtuoso system
Networking challenges in Virtuoso
Enter VNET
VNET, VTTIFAdaptive virtual network
Experiments
Current Status
Conclusions
8
Why VNET? A Scenario
Foreign hostile
LAN
User’s friendly
LAN
IP network
User has just bought
Virtual Machine
9
Why VNET? A Scenario
VM traffic going
out on foreign
LAN
Foreign hostile
LAN
User’s friendly
LAN
X
IP network
Host
Proxy
Virtual Machine
A machine is suddenly plugged into a foreign
network. What happens?
•
Does it get an IP address?
•
Is it a routeable address?
•
Does firewall let its traffic
through? To any port?
VNET: A bridge with long wires
10
VNET startup topology Foreign LAN 1
TCP Connections
User’s LAN
VM 1
Host 1
+
VNET
IP network
Proxy
+
VNET
VM 4
Foreign LAN 4
Foreign LAN 2
Host 4
+
VNET
VM 3
Host 3
+
VNET
Foreign LAN 3
VM 2
Host 2
+
11 VNET
A VNET Link
Ethernet Packet Captured by
Interface in Promiscuous mode
First link Second link (to proxy)
VM
“eth0”
ethy
ethz
“Host Only”
Network
VM
“eth0”
vmnet0
vmnet0
VNET
Host
Ethernet Packet is Matched
against the Forwarding Table
on that VNET
IP Network
VNET
Ethernet Packet Tunneled
over TCP/SSL Connection
Host
Ethernet Packet is Matched
against the Forwarding
Table on that VNET
Local traffic matrix inferred by
VTTIF
Periodically sent to the VNET on
the Proxy
12
VTTIF
• Topology inference and traffic
characterization for applications
• Ethernet-level traffic monitoring
• VNET daemons collectively aggregate a
global traffic matrix for all VMs
• Application topology is recovered using
normalization and pruning algorithms
13
VTTIF Operation
Synced Parallel Traffic Monitoring
Traffic Filtering and Matrix
Generation
Matrix Analysis and Topology
Characterization
14
Dynamic Topology Inference
VNET
Daemons
VNET
Daemons
VTTIF parameters
•Update rate
•Smoothing interval
•Detection threshold
1. Fast updates
Smoothed
Traffic Matrix
Topology change output
2. Low Pass Filter
Aggregation
3. Threshold
change detection
15
Reaction time of VTTIF
0.7
Reported Change Rate (Hz)
0.6
0.5
Knee of Curve Depends
0.4
On VTTIF Threshold
and Smoothing Parameters
0.3
0.2
0.1
0
0
0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8
Topology Change Rate (Hz)
2
16
Outline
•
•
•
•
•
•
•
•
Virtual machine grid computing
Virtuoso system
Networking challenges in Virtuoso
Enter VNET
VNET, VTTIFAdaptive virtual network
Experiments
Current Status
Conclusions
17
Adaptation
Adapt to available resources
Virtuoso presents tremendous opportunities and challenges
Network
and host
monitoring
Challenges
Monitor
application
Adequacy of
Infer goals of
available
application
mechanisms
Challenges interrelated
To determine subset of applications
for which such adaptation succeeds
We demonstrate that the
subset is not empty
18
Experiments
• Focus on a specific instance
– Application : Patterns, a synthetic benchmark
– Monitoring : Application topology inferred by VTTIF
– Aim
: Minimize running time of patterns
– Mechanism : Add links and corresponding
forwarding rules to VNET topology
Performance of BSP applications significantly
enhanced by adapting VNET topology, guided by
topology inferred by VTTIF
19
Illustration of dynamic adaptation in Virtuoso
Resilient Star Backbone
Fast-path links amongst
the VNETs hosting VMs
User’s
LAN
Foreign host
LAN 1
VM 1
IP network
Proxy
+
Merged
matrix as VNET
inferred
by VTTIF
VM 4
Foreign host
LAN 4
Host 1
+
VNET
Foreign host
LAN 2
VM 2
Host 4
+
VNET
VM 3
Foreign host
LAN 3
Host 3
+
VNET
Host 2
20 +
VNET
Evaluation
• Reaction time of VNET
• Benefits of adaptation (performance
speedup)
– Eight VMs on a single cluster, all-all
topology
– Eight VMs spread over two clusters over
MAN, bus topology
– Eight VMs spread over WAN, all-all
topology
21
Reaction Time
3.5
3.23
2.92
3
Seconds
2.5
2.268
2
1.6
1.5
1
0.94
0.5
0
nt
e
ili
s
Re tar
S
th
th
st
h
a
t
a
a
p
a
p
lf
t
p
t
l
s
t
s
-A ks
as
fa
f
a
o
f
t
n
h
s
l
li
es s
ng
Al th
Bu ks
i
M k
R ks
pa
lin
n
i
lin
22
Benefits of Adaptation
Benefits accrued as a function of the number of fast-path links added
1800
1600
Dynamic measurement and
reconfiguration
1200
1000
800
600
Full all-to-all network after
startup measurement
+ reconfiguration cost
Full all-to-all from
beginning of run
400
•Patterns has an all-all
topology
•Eight VMs are used
•All VMs are hosted on
the same cluster
200
0
ideal
complete
star
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
Run Time (Seconds)
1400
No Fast Path Topology
Number of Fast Path Links in Virtual Topology
23
Benefits of Adaptation
Benefits accrued as a function of the number of fast-path links added
900
No Fast Path Topology
800
Dynamic measurement and
reconfiguration
•Patterns has a bus
topology
600
Full bus network after startup
measurement + reconfiguration cost
500
400
•Eight VMs are used
Full bus from beginning of run
300
•VMs spread over two
clusters over a MAN
200
7
6
5
4
3
2
1
star
0
complete
100
ideal
Run Time (Seconds)
700
24
Number of Fast Path Links in Virtual Topology
Benefits of Adaptation
Benefits accrued as a function of the number of fast-path links added
1800
No Fast Path Topology
1600
Dynamic measurement and
reconfiguration
1200
1000
800
•Patterns has an all-all
topology
Full all-to-all network after startup
measurement + reconfiguration cost
•Eight VMs are used
Full all-to-all from
beginning of run
600
• VMs are spread over
WAN
400
200
0
ideal
complete
star
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
Run Time (Seconds)
1400
Number of Fast Path Links in Virtual Topology
25
Outline
•
•
•
•
•
•
•
•
Virtual machine grid computing
Virtuoso system
Networking challenges in Virtuoso
Enter VNET
VNET, VTTIFAdaptive virtual network
Experiments
Current Status
Conclusions
26
Current Status
• Applications: Transactional web ecommerce
application
• Mechanisms: VM migration
27
Conclusions
• Demonstrated the feasibility of adaptation at
the level of collection of VMs connected by
VNET
• Showed that its benefits can be significant for
the case of BSP applications
• Studying the extent of applications for which
our approach is effective
• Moving ahead to use other adaptation
mechanisms
28
• For More Information
– Prescience Lab (Northwestern University)
• http://plab.cs.northwestern.edu
– Virtuoso: Resource Management and
Prediction for Distributed Computing using
Virtual Machines
• http://virtuoso.cs.northwestern.edu
• VNET is publicly available from
• http://virtuoso.cs.northwestern.edu
29
Isn’t It Going to Be Too Slow?
Small relative
virtualization
overhead;
compute-intensive
Relative
overheads < 5%
Application
Resource ExecTime Overhead
(10^3 s)
SpecHPC Physical
Seismic
VM, local
(serial,
medium) VM, Grid
16.4
N/A
16.6
1.2%
16.8
2.0%
9.31
N/A
9.68
4.0%
9.70
4.2%
virtual FS
SpecHPC Physical
Climate VM, local
(serial,
medium) VM, Grid
virtual FS
Experimental setup: physical: dual Pentium III 933MHz, 512MB memory, RedHat 7.1,
30GB disk; virtual: Vmware Workstation 3.0a, 128MB memory, 2GB virtual disk, RedHat 2.0
NFS-based grid virtual file system between UFL (client) and NWU (server)
30
Isn’t It Going To Be Too Slow?
3
2.5
2
1.5
1
0.5
Tasks on
Physical
Machine
Tasks on
Virtual
Machine
Tasks on
Physical
Machine
Tasks on
Virtual
Machine
Tasks on
Physical
Machine
Tasks on
Virtual
Machine
0
No Load
Light Load
Heavy Load
Synthetic benchmark: exponentially arrivals of compute bound
tasks, background load provided by playback of traces from PSC
Relative overheads < 10%
31
Isn’t It Going To Be Too Slow?
• Virtualized NICs have very similar
bandwidth, slightly higher latencies
– J. Sugerman, G. Venkitachalam, B-H Lim, “Virtualizing I/O Devices
on VMware Workstation’s Hosted Virtual Machine Monitor”,
USENIX 2001
• Disk-intensive workloads (kernel build,
web service): 30% slowdown
– S. King, G. Dunlap, P. Chen, “OS support for Virtual Machines”,
USENIX 2003
However: May not scale with faster NIC or disk
32