Transcript eortizchayes3FinalPresx

In-Band Detection of Virtual
Machines
Estefan Ortiz & Cory Hayes
Computer Science and Engineering
Graduate Operating Systems
December 16, 2011
1
Introduction
 Malicious programs (malware) need to know if they are
in a virtual environment so they can modify their behavior
and avoid detection
 Related work
 Red Pill Tests: Examine byte-level behavior of instructions for
physical and emulated CPUs. If any disagreements in
output, create one or more “red pills” that can avoid
detection
 SubVirt: Virtual machine-based rootkit installed underneath
host OS that runs OS as a guest to remain nearly
undetectable
2
Our Approach
 Similar to Red Pill and SubVirt, but client-server based
 Idea: Instead of monitoring system call discrepancies,
analyze network data sent to/from physical and virtual
machines
 Goal: Determine if there are sufficient differences in
network traffic to detect if a client/server is being run on
a virtual machine
3
Goal
Byte
0
Byte k1
Difference Found
Byte k2
Byte n
Client <-> Native TCP/IP Packet
Client <-> Virtual Machine TCP/IP
Packet
4
General Setup
5
Actual Setup
Functions as the “MITM”
Network output
saved for analysis
6
Experiment Setup
 Using Wireshark, capture and compare the raw info of
TCP/IP packets sent back and forth between a client and
a physical/virtual server running Apache
 Bits 1-160: IP
 Remainder: TCP
 Virtual machine OS matches the OS of the host (UbuntuUbuntu, Vista-Vista)
 Use a small set of Matlab commands to send regular and
malformed packets
 Dynex 5-port 10/100/1000 Gigabit Ethernet Switch
7
Sample Captured Wireshark Output
8th Packet sent between Client & VM running Apache
VM
Host
Client
8th Packet sent between Client & Host running
Apache
8
Metrics
Bit Difference Comparison:
Fractional Hamming distance
between two packets
9
Metrics (cont.)
Round trip time: Time from
SYN request sent by client to
received ACK from server
10
Metrics (cont.)*
Pairwise Packet Length
Comparison: Number of
concurrent packet pairs that
differ in length
11
Experiment #1
 Client: Windows Vista (4GB RAM, 2.6GHz)
 Server: Ubuntu 11.04 32-bit w/ Apache Web Server 2.2
 Server: Host OS Ubuntu: VirtualBox w/ Ubuntu running
Apache
 On isolated switch network (no other traffic)
12
Exp. #1: Frac. Hamming Distance
13
Exp. #1: Round-trip Timing
14
Example: Packet #9
These bits
correspond to
the header
length & flags
in the TCP
header
15
Experiment #2
 Client: Mac (4GB RAM, 2.4GHz, MacOSX 10.6.8)
 Server: Windows Vista 32-bit w/ Apache Web Server 2.2
 Server: Host OS Windows Vista: VirtualBox w/ Windows Vista
running Apache
 On isolated switch network (no other traffic)
16
Exp. #2: Frac. Hamming Distance
17
Exp. #2: Round-trip Timing
18
Example: Packet #4
Destination
Address in
IP header
Flags in TCP
header
19
Experiment #3
 Client: Windows Vista (4GB RAM, 2.6GHz)
 Server: Ubuntu 11.04 32-bit w/ Apache Web Server 2.2
 Server: Host OS Ubuntu: VirtualBox w/ Ubuntu running
Apache
 Both client and server on CVRL subnet (at ~3:00 am)
20
Exp. #3: Frac. Hamming Distance
21
Exp. #3: Round-trip Timing
22
Example: Packet #3
Destination
Address in
IP header
23
Experiment #4
ND/CVRL subnet
24
Experiment #4
 Client: Windows Vista (4GB RAM, 2.6GHz)
 Server: Ubuntu 11.04 32-bit w/ Apache Web Server 2.2
 Server: Host OS Ubuntu: VirtualBox w/ Ubuntu running
Apache
 Could not monitor packet information; only ping tests
 Varied number of bytes sent using ping
 Performed 100 per fixed byte amount
 Calculated avg. & std. dev
 Executed at ~3:30 am
25
Exp. #4: Ping Timing
26
Conclusion
 Examined packet information from a high level (packetlength) down to specific bit difference comparisons
 Packet length provided no insight
 Timing tests didn’t provide conclusive evidence of a
connection to a virtual machine
 Fractional hamming dist. provided first level of insight
 Further analysis of differences at the bit level provided
clues where to look for VM traces
27
Future Direction
 Experiments 1-3 were conducted under somewhat
“ideal” scenarios
 More realistic approach would be packet analysis on
multi-hop connections with knowledge of which sections
of the TCP/IP packets to monitor
28