Transcript ossupport-ashish

Operating System Support for
Virtual Machines
Samuel King, George Dunlap,
Peter Chen
Univ of Michigan
Ashish Gupta
Two classifications for VM
1
VM/370
VMWare
VMWare
Guest tools
VAX VMM
Security
Kernel
Higher Level Interface
UMLinux
SimOS
Xen
Denali
u-kernels
JVM
Two classifications for VM
Convenience
Performance
2
VM/370
VMWare ESX
Disco
Denali
Xen
Type I
Underlying Platform
VMWare Workstation
VirtualPC
SimOS
UMLinux
Type II
UMLinux
• Higher level interface slightly different
• Guest OS needs to be modified
– Simple device drivers added
– Emulation of certain instructions (iret and in/out)
– Kernel Re-linked to different address
• 17,000 lines of change
• ptrace  virtualization
– Intercepts guest system calls
– Tracks transitions
Advantage of Type II VM
Guest Machine
Process
Virtual CPU
Host files and
devices
Virtual I/O
Devices
Host Signals
Virtual
Interrupts
mmap
munmap
Virtual MMU
The problem
Compiling the Linux Kernel
+ 510 lines to Host OS
Compiling the Linux Kernel
+ 510 lines to Host OS
Optimization One
System calls
Lots of context switches between
VMM < -- > Guest machine process
Use VMM as a
Kernel module
Modification to
Host OS also…
?
18
16
Normalized Runtime
14
12
VMware Workstation 3.1
10
Original UMLinux
8
UMLinux + VMM in host
6
4
2
0
POV-Ray
SPECweb 99
Kernel Build
Optimization Two
Memory protection
Frequent switching
between Guest Kernel
and Guest application
Guest Kernel to
Guest User
Guest User to
Guest Kernel
Through mmap,
munmap and
mprotect
Very expensive…
Host Linux Memory Management
• x86 paging provides built-in protection to memory
pages
• Linux uses page tables for translation and protection
• Segments used only to switch between privilege
levels
• Uses supervisor bit to disallow ring 3 to access
certain pages
The idea: segments bound features
are relatively unused
Solution:
Change Segment
bounds for each
mode
18
16
Normalized Runtime
14
VMware Workstation 3.1
12
Original UMLinux
10
8
UMLinux + VMM in host
6
UMLinux + VMM in host + seg.
bounds prot.
4
2
0
POV-Ray
SPECweb 99
Kernel Build
Optimization Three
Context Switching
• The problem with context switching:
– Have to remap user process’s virtual memory to
the “virtual” physical memory
– Generates large number of mmaps  costly
• The solution:
– Allow one process to maintain multiple addressspaces
– Each address space  different set of page tables
– New system call : switch guest, whenever context
switching
Multiple Page Table Sets
guest proc
proc ab
guest
Guest OS
switchguest syscall
Page Table Ptr
Host operating system
18
Normalized Runtime
16
14
VMware Workstation 3.1
12
Original UMLinux
10
UMLinux + VMM in host
8
UMLinux + VMM in host + seg.
bounds prot.
6
Fully optimized UMLinux
4
2
0
POV-Ray
SPECweb 99
Kernel Build
Conclusion
• Type II VMM CAN be as fast as type I
by modifying the Host OS
• Is the title of paper justified ?
Virtualizing I/O Devices on
VMware Workstation’s
Hosted VMM
Jeremy Sugerman, Ganesh Venkitachalam and Beng-Hong Lim
VMware, Inc.
Introduction
• VM Definition from IBM:
– a “virtual machine” is a fully protected and isolated
copy of the underlying physical machine’s
hardware.
• The choice for hosted architecture
– Relies upon host OS for device support
• Primary Advantage
– Copes with diversity of hardware
– Compatible with pre-existing PC software
– Near native performance for CPU intensive
workloads
The major tradeoff
• I/O performance degradation
• I/O emulation done in host world
– Switching between the host world and the VMM
world
How I/O works
Application
Portion
Privileged
Portion
VM App
VMM
VM
Driver
CPU Virtualization
I/O Virtualization
I/O Request
Interrupt
reasserted
H/w interrupt
I/O Virtualization
• VMM intercepts all I/O operations
– Usually privileged IN , OUT operations
• Emulated either in VMM on in VMApp
• Host OS drivers understand the semantics of port
I/O, VMM doesn’t
• Physical Hardware I/O must be handled in Host OS
• Lot of Overhead from world switching
– Which devices get affected ?
– CPU gets saturated before I/O…
The Goal of this paper
I/O CPU
I/O CPU
The Network Card
• Virtual NIC appears as a full fledged PCI Ethernet
Controller, with its own MAC address
• Connection implemented by a VMNet driver loaded in
the Host OS
• Virtual NIC : a combination of code in the VMM and
VMApp
– Virtual I/O Ports and Virtual IRQs
V
M
M
H
O
S
T
Sending a Packet
H
O
S
T
Receiving a Packet
V
M
M
H
O
S
T
Experimental Setup
Nettest: throughput tests
Time profiling
Extra work:
• Switching worlds for every I/O instruction: most
expensive
• I/O interrupt for every packet sent and received:
– VMM, host and guest interrupt handlers are run !
• Packet trans: two device drivers
• Packet copy on transmit
Optimization One
• Primary aim: Reduce world switches
• Idea: Only a third of the I/O instructions trigger
packet trans.
– Emulate the rest in VMM
• The Lance NIC address I/O has memory
semantics
– I/O  MOV !
– Strips away several layers of virtualization
Optimization Two
• Very high interrupt rate for data trans.
• When does a world switch occur:
– A packet is to be transmitted
– A real interrupt occurs e.g. timer interrupt
• The Idea: Piggyback the packet interrupts on
the real interrupts
– Queue the packets in a ring buffer
– Transmit all buffered packets on next switch
• Works well for I/O intensive workloads
Packet Transmit
Real Interrupt
Optimization Three
• Reduce host system calls for packet sends
and receives
• Idea: Instead of select, use a shared bit-vector,
to indicate packet availability
• Eliminates costly select()
?
Summary of three optimizations
Native
VM/733 MHz
Optimized
VM/733 MHz
Version 2.0
Guest OS idles
Summary of three optimizations
Native
VM/350 MHz
Optimized
VM/350 MHz
Version 2.0
Most effective Optimization ?
• Emulating IN and OUT to Lance I/O ports
directly in VMM
• Why ?
– Eliminates lots of world switches
– I/O changed to MOV instruction
Further avenues for Optimization ?
• Modify the Guest OS
– Substitute expensive-to-virtualize instructions e.g.
MMU instructions . Example ??
– Import some OS functionality into VMM
– Tradeoff: can use off-the-shelf Oses
• An idealized virtual NIC (Example ??)
– Only one I/O for packet transmit instead of 12 !
– Cost: custom device drivers for every OS
– VMWare Server version
Further avenues for Optimization ?
• Modify the Host OS: Example ??
– Change the Linux networking stack
• Poor buffer management
– Cost: requires co-operation from OS Vendors
• Direct Control of Hardware: VMWare ESX
– Fundamental limitations of Hosted Architecture
– Idea: Let VMM drive I/O directly, no switching
– Cost ??