Transcript An I/O Simulator for Windows Systems

An I/O Simulator for
Windows Systems
Jalil Boukhobza, Claude Timsit
[email protected]
27/10/2004
Versailles Saint Quentin University
laboratory
Outline







Introduction
Overview of the Windows I/O system
architecture
Description of the simulator’s architecture
(WinIOSim)
What’s new in WinIOSim?
Inputs / Outputs of the simulator
Validation of WinIOSim
Summary
27/10/2004
PRiSM Lab/ University of Versailles
2
Introduction



Windows I/O system is poorly studied
CreateFile(): many file access modes
different caching algorithms
big performance
fluctuations for a given workload.
Disk subsystems built independently from OS
 Interaction
with the OS are not easily predictable
What is the performance of a given workload on
a given system architecture for a defined I/O
strategy on Windows systems ?
27/10/2004
PRiSM Lab/ University of Versailles
3
Overview of the Windows I/O
system architecture

Different file access modes
in the CreateFile()
function:

Without using the file system
cache: no buffer mode
I/O request
FastIO
File System Driver
(FILE_FLAG_NO_BUFFERING)

Using the file system cache:
sequential, normal, write
through modes.
FILE_FLAG_SEQUENTIAL_SCAN,
FILE_ATTRIBUTE_NORMAL,
FILE_FLAG_WRITE_THROUGH
27/10/2004
Storage Device
Driver
Cache Manager
Page miss
Storage
Device
PRiSM Lab/ University of Versailles
Virtual Memory
Manager
4
Description of the WinIOSim
architecture (OMNET++)
Operating system modules
I/O generator 1
I/O generator 2
File system cache
Process
memory
System process
Application
process
Disk buffer
I/O scheduler
I/O generator n
Application modules
27/10/2004
Disk
Storage device subsystem
modules
PRiSM Lab/ University of Versailles
5
The WinIOSim modules

I/O generators: flexible workload generator
Request types, sizes, numbers, inter arrival times
Access modes, requested addresses, etc.
 Different possible distributions for each parameter (poisson,
uniform, exponential, etc.) thanks to OMNET++




Possible to plug real traces
Implemented request criticality (synchronous and asynchronous
requests)
The process memory and file system cache modules:
simulating the data copy operations, updating policies,
etc.
27/10/2004
PRiSM Lab/ University of Versailles
6
The WinIOSim modules (2)

The application and system processes:





Request flow control
Request grouping and splitting
File system cache prefetching algorithms, lazy write and write
through algorithms depending on access modes
Both communicate to issue the final request sequence (as seen
by the disk)
The different buses: simulating bus throughput, delays,
sharing.
27/10/2004
PRiSM Lab/ University of Versailles
7
The WinIOSim modules (3)

IO scheduler module: controls the flow of requests to the
disk subsystem


The disk module: very detailed model


Queuing system: FIFO, SCAN, LOOK, SSPF, C-LOOK, CSCAN, etc.
Mapping, zoning, spare area, number of platters, seek times,
rotational speed, head switching times, track and cylinder skew,
etc.
The disk cache module:

Segmentation, read ahead algorithms, lazy write and write
through algorithms, cache updating policies, etc.
27/10/2004
PRiSM Lab/ University of Versailles
8
What’s new in WinIOSim?

Implementation of Windows specific cache
algorithms depending on access modes
identified by reverse engineering work
 Specific
sequences of data issued by the system and
application process for each access mode

Disk subsystem reactions to these algorithms
 Specific
reactions for specific sequences depending
on the disk
27/10/2004
PRiSM Lab/ University of Versailles
9
Windows cache / disk cache

Windows prefetching algorithms:

No buffer mode
Read operations:

Sequential mode: loading data sequentially Bn, Bn+1, Bn+2, Bn+3, etc.

Normal mode (default) System process:
B1
B2
B3
B4
1 2 3 1 2 31 2 3
What are the disk
cache reactions?
Will it load a part
of these data ?
27/10/2004
The final sequence of request
blocks is: B1,1, B1,2, B1,3, B3,1, B2,1,
B3,2,B2,2, B3,3, B2,3, B4,1 , B4,2, ..
PRiSM Lab/ University of Versailles
One requested block:
3 blocks of 64KB
64KB block loaded by
the system process
64KB block loaded by
the application process
10
Windows cache / disk cache (2)

Write operations:

Sequential and normal modes : for one request some blocks are flushed on
the disk and the others on the file system cache (later on flushed on the disk).
Request
block
Application
How will the disk
react to those
sequences of
blocks to write ?
64KB block
Disk
flush

Write through mode

27/10/2004
File system
cache
Each written block -> file system cache -> disk cache -> disk +
modification of a system file on the disk -> acknowledge.
PRiSM Lab/ University of Versailles
11
Inputs / Outputs of the simulator

The inputs

I/O generator configuration



Modeled by the user
Real I/O traces
The simulated architecture definition

If existing:



Obtained from manufacturers (rarely complete)
Obtained using the WIOTest parameter extraction tool we developed.
The outputs


Response times and throughputs (2 main metrics for I/Os)
The different states of the modules at each stage of the
simulation
27/10/2004
PRiSM Lab/ University of Versailles
12
Validation of WinIOSim
Response time measures /
simulations of file read/write
operation on a sequential file with
different access modes.
Less than 4% variation

Measured Vs simulated response times for the "no buffer" mode read
for 64KB, 128KB, 192KB and 256KB (config 3)
9
8
response times (ms)

7
6
5
4
3
2
1
0
0
5
Measured vs simulated "normal" mode write with 192KB,
Measured vs simulated "normal / sequential" mode write with
320KB and 512KB requests (between peaks)
192KB requests
20
25
simulated
Measured Vs simulated response times for the "normal"
mode read / 192KB config3
Response time (ms)
Response time (ms)
160
30
140
25
120
100
20
15 80
60
10
40
5
20
0 0
response times (ms)
60
5
20
10
40
60 15
80
Request
numnum
ber ber
Request
measured
measured
27/10/2004
15
measured
35
0 0
10
request number
20
100
25 120
50
40
30
20
10
0
0
simulated
simulated
PRiSM Lab/ University of Versailles
10
20
30
40
request number
Measured
50
60
simulated
13
Summary




Efficient tool for Windows I/O
system simulation.
Very accurate and flexible
simulations of the whole
Windows IO system: from
application to disk.
Simulation of the interactions
between the modules for
example file system cache
and disk cache.
Unitary response time
validation.
Automatic parameter
extraction tool for the
simulation
Simulation Tool
I/O Generator
Storage
architecture
to simulate
I/O sequence
strategy defined
by the user / real
traces
Behavior of the
storage system
for the specified
I/O strategy
Developed tools
27/10/2004
PRiSM Lab/ University of Versailles
14
Thank you !
Questions ?
www.prism.uvsq.fr/users/jboukh
[email protected]
27/10/2004
PRiSM Lab/ University of Versailles
15