Transcript Document

Using Virtualization to Improve
Development and Testing
Anil Desai
Austin .NET User’s Group
January 9th, 2006
ADNUG: 01/09/2006
1
Outline
I.
II.
III.
Virtualization Benefits and Scenarios
Virtual Server Architecture
Automating Virtual Server
•
•
•
•
•
IV.
Virtual Machines
Virtual Hard Disks
Removable Media
Virtual Networks
CPU / Memory
Virtualization Best Practices
ADNUG: 01/09/2006
2
Presentation Overview

Each section will cover:






Technical Overview
Virtual Server Walkthrough
Scripting Overview
Script Example
Optimization Tips
Sections:






Managing Virtual Machines
Managing Virtual Hard Disks
Working with media
CPU and Memory Settings
Virtual Networks
VMRC Automation
ADNUG: 01/09/2006
3
Virtualization Benefits and
Scenarios
How you can use virtualization to
improve software development and
testing
ADNUG: 01/09/2006
4
Introduction

The question: “Is hardware getting faster faster than software is
getting slower?”

The Answer:



The Old Days: Software would see linear speed increases.
Today: Hardware is often left underutilized
Business Case

The modern IT environment:


Capital costs for hardware are significant, but reasonable
The real challenges today:



Administration costs
Server management
Datacenter costs
ADNUG: 01/09/2006
5
Virtualization Features

Benefits







Simplified administration
Hardware independence/portability
Increased hardware utilization
Server consolidation
Decreased provisioning times
Improved security
Scenarios





Software Development
Testing / Quality Assurance
Product evaluations / demonstrations
Training
Disaster Recovery
ADNUG: 01/09/2006
6
Development/Testing-Related
Scenarios

Setting up a complex test lab




Rapid setup and tear-down of test environments




Testing multi-tier applications
Isolated networks
Simulating multiple clients
Consistent configurations
Ability to easily roll-back changes
Automation of VM setup and software installation
Other Benefits



Portability of VMs
Duplication of errors
Simplified administration
ADNUG: 01/09/2006
7
Virtualization Candidates

Getting Started



Migrate existing apps / physical machines
Create new virtual machines from scratch
Virtualization may not work well for:

Resource-intensive applications




Virtual Machines are limited to 1 CPU and 3.6GB RAM
Performance testing
Hardware compatibility testing
Specific hardware requirements


USB
Custom hardware devices
ADNUG: 01/09/2006
8
Estimating VM Resource
Requirements

Number of VMs
 Estimated memory
per VM
 Estimated disk
space per VM
 Resource usage
estimates
ADNUG: 01/09/2006
VM Name
CPU
Memory
Disk
Network
WebApp01
Low
Low
Medium
High
WebApp02
Low
High
Low
Low
WinXPTest
Low
Low
Medium
Low
FileServer01
Low
Medium
High
High
9
Virtual Server Architecture
An Overview of Virtual Server 2005
ADNUG: 01/09/2006
10
Virtual Server Overview
ADNUG: 01/09/2006
11
Virtual Server Components

Virtual Server Components





Virtual Server Service (vssrvc.exe)
Virtual Machine Helper (vmh.exe)
Virtual Server Administration Web Site
Virtual Machine Remote Console (VMRC)
New Features in Virtual Server 2005 R2


64-bit Host OS Support
Improved Guest OS Support



Updated Virtual Machine Additions
Improved performance
Other Features:



Virtual SCSI Driver
Virtual Disk Precompactor
Support for PXE booting
ADNUG: 01/09/2006
12
Virtual Server Licensing

General:





Details




OS license for the Host OS
OS license for each Guest OS
Virtual Server license
All applicable client licenses
Up to four free guest OS’s on Win2003 R2 Enterprise Ed.
Only running VMs require licenses
For more info, see the MS Virtual Server Licensing Page
Virtual Server Editions




Standard Ed. = Up to 4 CPUs
Enterprise Ed. = Up to OS limit
Standard Ed. is included with an MSDN Subscription
180-day Trial version is also available
ADNUG: 01/09/2006
13
Virtual Server vs. Virtual PC

Virtual Server Features:
 Improved performance
(multi-threaded)
 Web-Based
Administration
 Scripting and automation
support (COM API)
 Improved security
 Advanced networking
 Manageability
 SCSI Device Support
 Reliability/availability
features (clustering)
ADNUG: 01/09/2006

Virtual PC Features:
 Optimized for one or a
few concurrent VMs
(single-threaded)
 Simplified administration
 Sound card emulation
 Easier file sharing
 Automatic client desktop
resizing
14
VM Compatibility
(VPC vs. Virtual Server)

Compatible:






Virtual machine hardware platform
VM configuration files (.vmc)
Virtual hard disk files (.vhd)
Differencing disks
Undo disks (.vud)
Incompatible:





Saved-state files (.vsv)
SCSI-mounted virtual hard disks
CD/DVD-ROM devices (VirtualPC = 1; Virtual Server = 4)
Sound card support (not available in VirtualPC)
Network files
ADNUG: 01/09/2006
15
Installing Virtual Server

Pre-Installation:

Admin Web Site Requires IIS (Application Server Role)




Warning: Network connections will be temporarily dropped during
installation
Administration Web Site:


On Windows XP, new Virtual Directory will be created
On Windows Server, new web site will be created (port 1024)
Default URL: http://ServerName:1024/VirtualServer/VSWebApp.exe
Basic Virtual Server Settings



Search Paths
Enabling the VMRC Server
Viewing the Event Log
ADNUG: 01/09/2006
16
Emulated Hardware Platform
Component
Specifications
Chipset
Intel 440BX
CPU
Based on the host CPU (single CPU only)
BIOS
AMI BIOS
Video
S3 Trio 32/64 with 4MB of video memory
Memory
Up to 3.6GB
Input Devices
PS/2 Mouse and Keyboard
Floppy Drives
Up to two 1.44MB 3.5” floppy drives
Communications
Ports
2 Serial Port
1 Parallel Port
IDE Controllers
Two channels, each of which supports up to two devices each (up to four IDE total
devices)
SCSI Controllers
Up to four Adaptec 2940 SCSI Controllers (each supports up to 7 devices; 28 devices
total)
Network Interfaces
Up to four Intel 21140 10/100Mb network interface cards
ADNUG: 01/09/2006
17
Creating a New VM

VM Hardware Configuration (demo)

OS Installation Options:





Installing the Virtual Machine Additions





1) Physical Media
2) ISO Files
3) Network-Based Installations
4) Automated deployment
Improved general performance
Improved mouse and keyboard support
Time synchronization with the host
VM “heartbeats”
Enabling Remote Management in the Guest
ADNUG: 01/09/2006
18
Virtual Server Files
File Type
Filename
Extension
Virtual
Machine
Configuration File
.vmc
An XML-based file that specifies disk, memory, and other settings for the virtual machine.
Virtual
File
.vhd
This includes fixed-size hard disks, dynamically expanding hard disks and differencing disks.
Saved State File
.vsv
Stores the contents of memory for a virtual machine that has been placed in the “Saved” state
By default, these files will be stored in the same location as the .vmc file.
Virtual Server
configuration file
(options.xml)
Options.xml
Stores information about the configuration of Virtual Server, including search paths, VMRC settings,
and other options.
Virtual network
configuration
.vnc
Undo disk files
.vud
Hard
Disk
ADNUG: 01/09/2006
/
Description
By default, these files will be stored in the same location as the .vmc file.
19
Automating Virtual Server
Accessing the Virtual Server COM
API
ADNUG: 01/09/2006
20
Introduction to the Object Model

Setup installs the “Virtual Server 2005 Type Library”



VirtualServer object






Can be accessed directly using VBScript
Can be added as a COM reference in Visual Studio 6.0 / Visual Studio .NET
Connect to a local or remote instance of Virtual Server
Add/Remove virtual machines
.VirtualMachines collection provides list of defined VMs
.FindVirtualMachine method used for getting a VM object by name
Create “global” objects: virtual networks, virtual hard disks, etc.
VirtualMachine object




Represents an existing virtual machine
Control VM power states (turn on, turn off, etc.)
Contains properties and collections for the virtual hardware configuration
.Accountant can be used to get performance statistics
ADNUG: 01/09/2006
21
Setup: Visual Studio .NET

Add the COM reference for “Virtual Server 2005 R2 Type Library”



COM Security:




Virtual Server requires “Impersonation” (or higher) level
“Identification” is the default, therefore must call CoIntializeEx(), then
CoInitializeSecurity() before accessing the Virtual Server Interfaces
Alternatively, use DComCnfg.exe and modify the Default Properties to set the Default
Impersonation Level to “Impersonate”
Visual Studio 2005:



Add the namespace of Microsoft.VirtualServer.Interop
Should use the multi-threaded apartment (MTA) model (STA may cause performance
problems)
Disable debugging under the Visual Studio host process to prevent COM security errors
In Visual Studio, click on the Properties of a project, then uncheck “Enable the Visual
Studio hosting process.”
More information:


Virtual Server Programmer’s Guide (full sample code for C#)
Blog: “Programming Virtual Server from a managed application” (Ben Armstrong)
ADNUG: 01/09/2006
22
Virtual Machines: Overview

A virtual machine consists of:





VM Configuration file (.vmc)
Associated VHD files (.vhd)
Virtual network file (.vnc)
Other hardware options
Demonstration




Enumerating VMs
Creating a New VM
Adding a VM
Changing VM Properties
ADNUG: 01/09/2006
23
VM Power States

Virtual Machine power operations:






Guest OS Power Management




Turn On / Turn Off
Reset
Shut Down Guest OS
Save State
Pause
Standby
Suspend
Hibernate
By default, VMs are placed in Saved State when the Virtual
Server service is stopped

VMs are not automatically started, by default
ADNUG: 01/09/2006
24
Virtual Server Scripting Events

Server-Level Scripts:












Virtual Server starts
Virtual Server stops
Any Virtual Machine is turned on
Any virtual machine is restored
Any virtual machine is turned off (saved)
Any virtual machine is turned off (not
saved)
Any virtual machine is turned off within
the guest environment
Any virtual machine is reset
No heartbeat is detected for any virtual
machine: If the Guest operating system
stops responding for three minutes, this
event is raised.
Any virtual machine experiences a
guest processor error
Any virtual machine receives a warning
due to low disk space on the physical
computer: The warning is generated
when physical disk space on the host is
under 100MB.
Any virtual machine receives an error
due to low disk space on the physical
computer
ADNUG: 01/09/2006

Virtual Machine Scripts:










This virtual machine is turned on
This virtual machine is restored
This virtual machine is saved and
turned off
This virtual machine is turned off (and
not saved)
This virtual machine is turned off within
the guest environment
This virtual machine is reset
No heartbeat is detected for this virtual
machine
This virtual machine experiences a
guest processor error
This virtual machine receives a warning
due to low disk space on the physical
computer
This virtual machine receives an error to
do low disk space on the physical
computer
25
Virtual Machines: Tips

Whenever possible, store files for each VM in a single
folder

Moving/Copying VMs:
1.
2.
3.
4.
5.
Shut down (or save state of) the virtual machine.
Move or copy all required files to the destination server.
Add the virtual machine on the destination server (using the
Virtual Server Administration web site)
If the path to any physical VHD file has changed, edit the
configuration of the virtual machine to point to the new
physical location
(Optional) Remove the VM from the source server (to prevent
conflicts)
ADNUG: 01/09/2006
26
Managing Media: Overview

Changing media:



Hardware:



Media can be mounted/changed while the VM is running
Media devices can only be added or removed when the VM is not
running
Can have up to four virtual CD/DVD devices and one floppy device
Can capture a physical device on the host or capture image files
Image File Types:

ISO Images:



Can represent CD or DVD media
Read-only
Floppy Disk Images


Can be created within Virtual Server
Read/Write
ADNUG: 01/09/2006
27
Managing Media: Scripts

Adding/Removing DVD ROM drives:




Capturing Media:




objVirtualMachine.DVDROMDrives: Collection of VMDVDDrive
objects
objVirtualMachine.AddDVDROMDrive(BusType, BusNumber,
DeviceNumber)
objVirtualMachine.RemoveDVDROMDrive(VMDVDDrive object)
objDVDRomDrive.AttachHostDrive(string HostDriveLetter)
objDVDRomDrive.AttachImage(string path)
objDVDRomDrive.ReleaseImage(string path)
Tips:


Mounting media files is useful for software deployment/updates
Use startup Registry keys to automatically install software
ADNUG: 01/09/2006
28
Disk Architecture: Technical
Overview
ADNUG: 01/09/2006
29
Base Virtual Hard Disk Types

Base Disk Types:

Fixed-Size VHDs


Dynamically-Expanding VHDs



Overhead due to expansion and fragmentation
Can be compacted to reclaim space
Linked Disks



Best performance, lowest space utilization
Links to an entire physical disk (not just a volume)
Used primarily for physical-to-virtual (P2V) conversions
Virtual Attachment Options:

IDE



2 channels * 2 devices each
Up to 127GB disk size
SCSI


Up to 4 controllers * 7 devices each
Up to 2.0TB disk size
ADNUG: 01/09/2006
30
State-Related Disks

Undo Disks (.vud)


Applies to an entire virtual machine
Options:




Differencing Disks (.vhd)




Keep
Commit
Discard
Can be based on fixed-size, dynamically-expanding, or linked hard
disks
Allows the creation of a disk hierarchy
Parent disks must be read-only
Saved-state files (.vsv)


Contains contents of memory
Virtual Server automatically reserves disk space based on memory
allocation
ADNUG: 01/09/2006
31
Undo Disks: Example
ADNUG: 01/09/2006
32
Differencing Disks: Example
ADNUG: 01/09/2006
33
Merging Differencing Disks
ADNUG: 01/09/2006
34
Managing Virtual Hard Disks

Converting VHDs



Compacting VHDs




Fixed-Size  Dynamically-Expanding
Linked disk to Fixed-size or dynamically-expanding
Disk space requirements
Use the Virtual Disk Precompactor
Can be very resource intensive
Disk Defragmentation


Host: Optimizes VHD file placement
Guest: Reduces IO within the VM
ADNUG: 01/09/2006
35
Disk Management: Scripting

Enumerating virtual hard disk files

objVirtualMachine.HardDiskConnections


objHDConnection.HardDisk.File


Phyiscal filename/path of the VHD file
Creating virtual hard disks




Collection of Hard Disk Connection objects
objVirtualMachine.CreateDynamicVirtualHardDisk(path, size)
objVirtualMachine.CreateFixedVirtualHardDisk(path, size)
objVirtualServer.CreateDifferencingVirtualHardDisk (path,
ParentPath)
Adding a virtual hard disk:

objVirtualMachine.AddHardDiskConnection (path, BusType,
BusNumber, DeviceNumber)
ADNUG: 01/09/2006
36
Disk Management Tips:

Monitor Disk Space



PerfMon: Logical Disk  % Free Space
Use Virtual Server / Virtual Machine events to send alerts
Host Interface Types:





IDE
Serial ATA (SATA)
SCSI
RAID Controllers
Performance Tips



Separate files across multiple spindles
Use SCSI disks within the virtual machine
Monitor disk resource usage to better distribute virtual machines
ADNUG: 01/09/2006
37
Network Architecture: Overview
ADNUG: 01/09/2006
38
Network Architecture: Details

Virtual network options:




Guest-only
Host Network
Guest-to-host only (through Microsoft Loopback Adapter)
Network Adapters




All Ethernet NIC types are supported
The Virtual Machine Networking Service must be enabled for Virtual Server to
use the adapter
Each VM can have up to four virtual NICs
A dedicated NIC can be used for Virtual Server


Only the Virtual Machine Networking Service must be bound
More Features


Virtual DHCP Server for each virtual network
Integration with Internet Connection Firewall (ICF) and Internet Connection
Sharing (ICS)
ADNUG: 01/09/2006
39
Networking: Scripting

Managing Virtual Networks:

objVirtualServer.VirtualNetworks




Collection of virtual networks
objVirtualServer.CreateVirtualNetwork (name, path)
objVirtualServer.RegisterVirtualNetwork (name, path)
Managing Virtual Network Adapters

objVirtualMachine.NetworkAdapters




Collection of network adapter objects
objNetworkAdapter.AttachToVirtualNetwork()
objNetworkAdapter.AttachToVirtualNetwork(NetworkName)
objNetworkAdapter.DetachFromVirtualNetwork()
ADNUG: 01/09/2006
40
Network-Based Storage

Improving Performance:

Use NIC Teaming





Dedicated “appliances” that provide file-level I/O
Storage Area Networks (SANs)



CIFS / SMB (UNC Shares)
Multiple VMs can access the same files
Network-Attached Storage (NAS)


Use Jumbo Frames
Remote Storage


Fail-over
Load-balancing
Fibre Channel-based, block-level I/O
High performance, but highest cost
iSCSI



Block-level I/O over Ethernet
Performance benefits of SANs with much lower costs (uses copper)
Supported on the Host OS as well as from within Guest OS’s
ADNUG: 01/09/2006
41
Networking: ICF and ICW
ADNUG: 01/09/2006
42
Networking: Firewalls
TCP Port
Number
Purpose
Notes
1024
Virtual Server Administration
Web Site.
The port can be changed during the installation of Virtual Server, or afterwards
(see Chapter #9 for details). Administrators can also enable SSL within
IIS.
5900
VMRC Server
By default, the VMRC server is disabled. The port number can be modified
using the Virtual Server Administration Web Site (see Chapter #9).
88, 137, 138
VMRC Kerberos Authentication
These ports are used only when Kerberos is being used (for example, when
Active Directory-based users are attempting to authenticate to the VMRC
server).
137 – 139
NetBIOS over TCP/IP
This is only necessary if NetBIOS is being used for file transfers across the
firewall.
ADNUG: 01/09/2006
43
CPU Management

CPU Resource Allocation

Weight-Based Resource Allocation



Constraint-Based Resource Allocation




Reserved capacity
Maximum capacity
More complicated, but finer-grained control
Host CPU Options:




Hyper-Threading
Dual-Core / Multi-Core
64-Bit (Large Memory Support)
Virtualization Support



Simpler method (single number to adjust)
Ensures full resource usage
AMD’s “Pacifica”
Intel’s VT
Heat-Related Issues

CPUs will automatically step-down when heat thresholds are exceeded
ADNUG: 01/09/2006
44
CPU Management: Example
ADNUG: 01/09/2006
45
CPU Management: Scripting

CPU Performance History

objVirtualMachine.Account.CPUUtilization


objVirtualMachine.Account.CPUUtilizationHistory


Array of CPU util. stats
Modifying CPU Settings

objVirtualMachine.Accountant.





Last CPU utilization statistic
.relativeWeight
.reservedSystemCapacity
.allowableMaximumSystemCapacity
.allowableReservedSystemCapacity
.NET Apps can also easily use system performance counters
ADNUG: 01/09/2006
46
Managing Memory

Memory Architecture

Virtual Server can use all of the memory avaialble to the host OS








For 32-bit, large memory support, enable PAE
For large installations, 64-bit is better
Up to 3.6GB of physical memory per VM
Memory settings can be changed only when VM is turned off
Be sure to leave physical memory for the host OS
Allow an additional ~32MB for each virtual machine (virtualization
overhead)
Physical memory cannot be over-committed
Scripting Memory Settings

objVirtualMachine.Memory property

Amount of Memory (in bytes) (read-write property)
ADNUG: 01/09/2006
47
VMRC: Overview

Configuring the VMRC Server

VMRC Server is disabled by


Potential security issues with default configuration:




Unencrypted communications
NTLM security
Can customize TCP address and ports
Can enable SSL/TLS security



Enabled in Server Properties (VS Admin Web Site)
Encryption
Certificates
VMRC Client


ActiveX control that can be hosted in IE or within a client app
COM Reference:



Virtual Machine Remote Control (VMRC) ActiveX client type library
Added to .NET namespace VMRCClientControl.Interop
In Visual Studio, add the ActiveX control using Tools  Choose Toolbox Items.
Under “COM Components”, enable “Microsoft Virtual Server VMRC Control”
ADNUG: 01/09/2006
48
Virtualization Best Practices
Tips and techniques for making the
most of virtualization
ADNUG: 01/09/2006
49
Managing Performance

Profile your applications

Windows System Monitor







Counter Logs
Event Viewer
Task Manager
Windows Management Instrumentation (WMI)
Microsoft Operations Manager (MOM)
Systems Management Server (SMS)
Monitoring Approach:


Host Server: Aggregate performance data
Guest OS: Detailed resource usage
ADNUG: 01/09/2006
50
VM Maintenance

Establish a base library of images





Ensures consistency
Helps in troubleshooting
Prepare VMs for duplication (SysPrep)
Create a Library of ISO files
Manage virtual machines as if they were physical machines

Security updates






Patches
Corporate security standards
Configure anti-spyware / anti-malware exceptions


Automatic Updates (if supported in Guest OS)
Microsoft Baseline Security Analyzer (MBSA)
Windows Server Update Services (SUS)
*.vhd, *.vud, *.vsv, *.iso
Automate common tasks
ADNUG: 01/09/2006
51
VM Backup and Recovery

Backup and Recovery

Option 1:




Option 2:


Shut down/save state of the VM
Copy all required files.
Resume the VM
Use Backup Agents within the Guest OS.
Automating backups:

Build scripts/applications to automate virtual machine backup
and recovery:



1) Save state of the virtual machine
2) Copy all related files
3) Resume the virtual machine
ADNUG: 01/09/2006
52
More Best Practices

“Virtualization on a stick!”


Treat virtual machines like physical ones


Use removable media for creating portable VMs
Standard security and maintenance practices
Consider performing Physical to Virtual (p2v)
migrations


Virtual Server Migration Toolkit (VSMT)
Third-party utilities
ADNUG: 01/09/2006
53
The Future of Virtualization

Other Virtualization Platforms:

VMWare:





XEN Open Source Virtualization Platform
CPUs:





Workstation
GSX Server
ESX Server
Intel
AMD
Sun
Third-Party Vendors
Windows Vista / Longhorn Server

Virtualization will use a “hypervisor” approach and will be included in
the platform
ADNUG: 01/09/2006
54
References

Microsoft Resources

Microsoft Virtual Server Web Site



Microsoft Virtual Server Script Repository



Downloadable White Papers and updates
Pricing and licensing information
Hundreds of sample scripts (VBScript format)
“Program Customized Testing Environments Without Trashing
Your Machine” (Ben Waldron, MSDN Magazine)
Community Resources

http://AnilDesai.net (content planned for 01/2006)

Virtualization.info (blog)
Virtual PC Guy’s WebLog (Ben Armstrong)

ADNUG: 01/09/2006
55
For More Information

Realizing the Power of Virtualization


The Rational Guide to Managing Microsoft
Virtual Server 2005


Q1, 2006 (Anil Desai)
The Rational Guide to Scripting Virtual
Server 2005


Microsoft TechNet Magazine (February,
2006)
Q2, 2006 (Anil Desai)
Contact Anil Desai


E-Mail: [email protected]
Web Site: http://AnilDesai.net

Copies of the slides and source code
ADNUG: 01/09/2006
56