Performance and bench-marking tools

So you need to do some disk performance testing?   Maybe some benchmarking?  What tools are out there to help you out?   Well I am glad you asked… here are some that I use on my daily travels:

IOmeter

IOmeter is an old classic, with emphasis on the word old.    At time of writing, the most recent update was from 2006.   However it remains very popular mainly because it is free and easy to use.

Some tips when using IOmeter:

On Windows, IOmeter needs to be run as an Administrator, which seems to be the most common mistake people make (not running as Administrator means you don’t see any drives).  You can only run one instance of IOmeter in Windows, which means if multiple users logon to the same server, only one user can run IOmeter.   You also really need to run IOmeter with a queue depth ( or number of outstanding I/Os) greater than one, with multiple workers.  If you don’t, you will not be able to drive the storage to saturation.   For instance here are some results running 75% read I/O, 0% random, 4 KB blocks on a Windows 2008 machine with 4 workers.  In each case against the same 128 GB volume on a Storwize V7000 backended by  4 x 300 GB SSDs in a RAID10 array.  In each case I let the machine run for 10 minutes before taking the screen capture to ensure the performance was steady state and not peaking.

Firstly I used a queue depth of one.   Aggregate performance was around 27000 IOPS.

Then I used a queue depth of 10.  Aggregate performance was around 81000 IOPS.

I then used a queue depth of 20.  Aggregate performance was around 113000 IOPS.

What I am trying to show is that taking the defaults (one worker with a queue depth of 1) will not drive the storage to a useful value for comparison… you need to do some tuning and some experimenting to get valid results.  At some point increasing queue depths will not improve performance (it may actually decrease it).

You can get IOmeter from here:

http://www.iometer.org/

There is a great how to document for IOmeter here:

http://greg.porter.name/wiki/HowTo:iometer

IOrate

There is an  alternative to IOmeter called IOrate (created by an EMC employee).   It is also very popular and appears to still be in active development.  It is not unusual to see IBM performance whitepapers that used IOrate to generate the workload.

You can download it from here:

http://iorate.org/

I/O Analyzer

This is a fairly recent tool that I have not had a chance to try out (due to time pressures).   The tool uses virtual machines under VMware to generate the I/O and includes some very nice workload capture and playback tools as well as reporting tools.

http://labs.vmware.com/flings/io-analyzer

There is a good write-up on some results and experiences here:

http://gabrielchapman.com/?p=82

Jetstress

Jetstress is a benchmarking tool created by Microsoft to simulate Microsoft Exchange workloads.   I like the fact you can configure it to run for very long periods and it has a more real world feel about it than just running empty I/Os.   You can get the base software here, but you will also need some files from a Microsoft Exchange install DVD (or from an installed instance of Microsoft Exchange).  If you cannot get to those files you cannot complete the startup process inside Jeststress.

You can get Jetstress itself from here:

http://www.microsoft.com/download/en/details.aspx?id=4167 

There is a great how-to guide for Jetstress here:

http://gallery.technet.microsoft.com/Jetstress-Field-Guide-1602d64c

Orion tool

Oracle offer a tool on their website called Orion, which will simulate the workload of an Oracle database.   You can get the tool from here (although you will need to create a free Oracle user account before you can download it).

http://www.oracle.com/technetwork/topics/index-089595.html

SDelete

SDelete is not a benchmarking tool or a performance modelling tool.  But it is a great way to generate I/O with very little effort.   Just create a new drive in Windows and then run SDelete against it with the -c parameter.  This parameter is used for secure deletion, so generates random patterns (which is real traffic – albeit 100% sequential writes).  The syntax is like:

sdelete -c e:\

You can get SDelete from here:

http://technet.microsoft.com/en-us/sysinternals/bb897443 

(updated April 20, 2012 – I found in version 1.6 of SDelete the meaning of the -z and -c parameters got swapped.  In version 1.6 if you want random patterns use -c, if you want zeros use -z.  In previous versions it is the other way around!).

—————————————————————————————-

Just doing file copies is probably the worst way to generate benchmarks, especially as a single copy is usually a single threaded operation.

I am sure there are plenty of other tools out there to generate benchmarks and simulate workload.   My main concern with many of them is that synthetic (artificial) workloads do not reflect real world workloads.

So what are your favourite tools and why?

About these ads

About Anthony Vandewerdt

I am an IT Professional who lives and works in Melbourne Australia. This blog is totally my own work. It does not represent the views of any corporation. Constructive and useful comments are very very welcome.
This entry was posted in Brocade, Cisco, DS8800, IBM, IBM Storage, IBM XIV, Storwize V7000, SVC, Tivoli Storage, vmware and tagged , , , , , , , , , , , . Bookmark the permalink.

3 Responses to Performance and bench-marking tools

  1. Troels Arvin says:

    When I benchmark, I use Orion (as you mentioned) and: fio: http://freecode.com/projects/fio
    Fio is really powerful and lets you get results from I/O which can fairly easily be configured to resemble real-world I/O (e.g. a suitable combination of reads and writes, performed by several threads in parallel).

  2. Piw says:

    My favorite benchmark tool is VDbench (http://vdbench.sourceforge.net/).
    It’s written in java (so you can run in on all platforms) and can put workload on raw devices, file systems or single files.
    You can control many aspects of IO stream, as well you can “record” your real application streams, and reply them to test storage (with SWAT).

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s