Transcript 6_CCAT- Troubleshooting Training Provisioning v1.0x

CCAT Troubleshooting Training –
Provisioning Services
April 2012
Citrix Consulting Architecture Team
Agenda
• PVS Troubleshooting – Key Areas
• WireShark Analysis
• System Memory Dumps
• Key Takeaways
Troubleshooting PVS Issues
PVS Ports used - Recap
Service
Protocol
Port
Comments
DHCP
UDP
67
DHCP Server port
UDP
68
DHCP Client port
UDP
67
If PXE separate from DHCP server
UDP
4011
If PXE on DHCP server
TFTP
UDP
69
Trivial File Transfer Protocol
TSB
UDP
6969
Two-Stage Boot Service
Console
TCP
54321-54322 SOAP Service
Database
TCP
1433
Microsoft SQL Server
Stream
Service
UDP
6890-6909
Inter-server communications
UDP
6910-6930
Streaming, initial login on port 6910
PXE
Troubleshooting
• Narrowing down scope of issue by determining where in the
boot cycle the issue is occurring.
• Packet Traces
• Capturing Kernel Dump Files
Boot Stages
• DHCP – PXE boot downloading of bootstrap file
• Login to the PVS Server
• Streaming of the I/O Pre-BNIStack (Citrix Target Device
driver)
• BNIStack up burst traffic
Using Wireshark – Packet
Trace Analysis
Analyzing Packet Captures
Packet Capture PXE TFTP download
Provisioning Server 10.1.1.1
TFTP
DHCP
PXE
Target 10.1.1.100
Common Issues During DHCP TFTP Download
Target does not receive IP address
Target gets IP but times out to TFTP
PXE Errors No boostrap found
Provisioning Server 10.1.1.1
TFTP
DHCP
PXE
Target 10.1.1.100
Login Process Traffic
Provisioning Server 10.1.1.1
TFTP
TFTP - Login
Stream
Process
Target 10.1.1.100
Common issues during login process
Incorrect Bootstrap IP information
Login port blocked
Target not entered in Database
Stream Service failed to start
Provisioning Server 10.1.1.1
TFTP
DHCP
PXE
Target 10.1.1.100
Streaming Pre-BniStack
TFTP
OS Booting UNDI Driver
Stream
Process
Target 10.1.1.100
Common issues pre BNIStack
• Large Send Offload
• Network instability
• Conflicts with UNDI Driver
Determining if BNIStack Driver is up
Provisioning Server 10.1.1.1
TFTP
OS Booting BNISTACK Driver
Stream
Process
Target 10.1.1.100
Common issues post BNIStack
• Filter Driver conflicts
• Firewall/Antivirus Services kickoff
• Antivirus Updates being loaded
• Domain Profile creation
Capturing Windows System
Dumps
Windows Kernel Dump files
Capturing Kernel Dump Files
• Capturing Kernel Dump files using Win2k3 and XP
• Capturing Kernel Dump files using Win2k8 Vista and Windows 7
• Capturing Kernel Dump files in XenServer and ESX
Capturing Kernel Dump Files Win2k3 and XP
Limitations
• Crash dumps to be on same drive as pagefile
• Page file to be on the system drive
• Redirection of dump not supported
• Live debug can be accomplished via serial cable.
Capturing Kernel Dump Files Win2k8 Vista and
Windows 7
• Enhancements allow kernel dumps to be directed to a location other
then a system drive
• Requires a local hard drive with enough free space for the dump file.
(Pagefile Size)
Capturing Kernel Dump Files Win2k8 Vista and
Windows 7
• Create a new string value under
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\CrashControl
named DedicatedDumpFile
• Create Value data box, type <drive>:\<dedicateddumpfile.sys>
• Create DWORD Value DumpFileSize use decimal under base and type the
name of dump file in megabytes
• More detailed information in this process can be found:
http://support.microsoft.com/kb/969028
http://support.citrix.com/article/ctx123642
Capturing a Kernel dump via XenServer
• Vista, Windows 7, Win2k8 redirection only possible if XenTools are
not installed
• Live Debug via WinDBG possible by redirecting serial to DOM 0
Capturing a Kernel Dump Via Xenserver
DOM 0
XEN SERVER
Serial
Redirect
WinDBG
SockPipe.EXE
http://support.citrix.com/article/ctx124979
Capturing Kernel dump via ESX
• Workstation 7 utility vmss2core
• Suspend hung VM
• Download suspended VM session (.vmss)
• Retrieve downloaded file and run command vmss2core –W
filename.vmss
http://support.citrix.com/article/ctx125086
Booting Target Devices – TFTP vs. BDM
• Whiteboard Discussion
PVS Retries – Potential Causes
• Network Traffic
•
•
•
•
•
High bandwidth utilization
Congested pipe
Application layer filtering
Physical network problems
UDP (LSO, Checksum Offload) Issues
• Check the vDisk Properties (System Tray)
• How to fix these issues? (Discussion)
Key Takeaways
Key Takeaways
• Stream Process Workflow
• How to identify where in the boot process a PVS Target is
failing
• Steps to capture a kernel level dump file from a provisioned
target