Riddle me this: When is a SAS drive not a SAS drive?

Here are two common statement I often hear from clients:

1. I don’t just want SAS drives, I also want SATA drives.  SATA drives are cheaper than SAS drives.
2. Nearline SAS drives are just SATA drives with some sort of converter on them.

So is this right?  Is this the actual situation?

First up, if your storage uses a SAS based controller with a SAS backplane, then normally you can plug SAS drives into that enclosure, or you can plug SATA drives into that enclosure.    This is great because when you plug SATA drives into a SAS backplane, you can actually send SCSI commands to the drive plus you can send native SATA commands t00 (which is  handy when you are writing software for RAID array drivers).

But (and this is a big but) what we do know is that equivalent (size and RPM) SAS drives perform better than SATA drives.  This is because:

• SAS is full-duplex, SATA is simplex.
• SAS uses the native SCSI command set which has more functionality (which leads to the next point).
• A SAS drive uses SCSI error checking and reporting which is much more robust than the SATA error reporting.  This allows your storage system to collect richer information from the drive if errors are occurring (such as a failing or marginal disk).
• SAS drives are dual ported which is vital in dual controller enclosures.

So given a choice (and a very small price differential), why choose SATA over SAS?  SAS is the clear winner.     What we should instead differentiate on is speed (7.2K RPM vs 10K RPM vs 15K RPM vs SSD) and size (2.5″ vs 3.5″ form factor).

Which leads us to Nearline SAS

It is a common belief, that if you buy a Nearline SAS  (or NL-SAS) drive it is really a SATA drive with a SAS connector (interposer) stuck on it.  But this is confusion from the past.

What led to the confusion?

Most midrange and enterprise storage controllers and enclosures up until recent years, used disks that had fibre channel interfaces on them.  We plugged those disks into fibre channel enclosures.  Examples include the DS4700 or the DS8100.  And yet these devices also offered SATA drives.  How did they do this?

They took a SATA drive and added a SATA to Fibre Channel converter card to the disk.   We call this extra piece of hardware an interposer or bridge card.  So people start assuming that this is common practice in every product.  In fact we are now seeing SAS drives being put into  fibre channel disk enclosures by using a SAS to fibre channel interposer.

There are in indeed older products that did take a SATA drive and add a SATA to SAS interposer to achieve a similar thing.   But that really is not necessary any more.  The reason?   The same hard drive can now be ordered from the factory as either a SAS drive or a SATA drive.

Lets look at an example.   If you head over to the Seagate website and look at one of their ranges of 3.5″ Enterprise Drives, you should hopefully make it to this URL:
http://www.seagate.com/www/en-us/products/enterprise-hard-drives/constellation-es/constellation-es-1/

Seagate have a nice selector tool to let you see all their possible combinations. For instance you can order a  2 TB drive with a 6 Gbps SAS interface, which is a model ST32000444SS:

Or you can order a 2 TB drive with a 6 Gbps SATA interface, which is a model ST2000NM0011:

So what you get is very similar drive hardware (same spindles, heads, motors) but with different adapter hardware, built with the desired adapter at manufacture time. Meaning that if we install this drive into a SAS enclosure, there is no need to add an  interposer or bridge card to the drive after you bought it.

This leads to the next question:

OK.  So this is good, so Nearline SAS drives are MADE as SAS drives.  Does that mean a drive manufactured with a SAS adapter is a SAS drive or a Nearline SAS drive?

Now we are mixing up two different things.  SAS as a standard is a combination of a connection technology (the Serial Attached part) and a command set (the SCSI part). Actually SCSI as a standard also defines both connection methods and command sets.    So SAS is really talking about how we connect to the disk and what command set we use to control the disk.

Nearline on the other hand is a statement about the disks rotational speed and it’s mean time between failure (MBTF).   A Nearline-SAS drive is Nearline because:

• It rotates slower (7200 RPM)  than the higher specified Enterprise drives (that spin at 10 K or 15K RPM).   Because they are slower they can also hold way more data.
• It has a lower MBTF (1.2 million hours) than the higher specified Enterprise drives (which are normally specified at 1.6 million hours).

So we have now gone full circle.   A Nearline-SAS drive can use the same physical disk hardware as a SATA drive, but with a superior adapter that uses a superior command set, built onto the drive at manufacture time.

Still confused or want to read some more?  Check out these links:

http://storagebuddhist.wordpress.com/2011/07/01/nearline-sas-who-dares-wins/

http://www.seagate.com/docs/pdf/whitepaper/tp_sas_benefits_to_tier_2_storage.pdf

http://enterprise.media.seagate.com/2011/07/inside-it-storage/sas-mythbusters-data-highways-and-sas-vs-sata/

http://www.tomsitpro.com/articles/seagate-serial-attached-scsi-disk-drive-data-storage,2-119.html

http://en.wikipedia.org/wiki/Serial_attached_SCSI#Nearline_SAS

Xanax made my dog go mad

Since it’s Friday (well here at least), here is a totally non-tech story about drug side affects and one of my dogs (called Maili).   Here she is in one of her hairier moments:

Maili is a beagle crossed with a fox terrier crossed with some sort of Rastafarian dog (don’t ask, we have no idea).

We like to keep her well groomed, so after a shave she looks more like this:

So it turns out Maili doesn’t like thunder storms:  she hides under the bed, she cowers, she becomes very clingy and won’t leave your side.  If you are sitting at a desk she will lie on your feet.

Recently this has gotten worse, so we took her to the vet, who prescribed Xanax, a drug that helps with anxiety.  We tried it once and it seemed to work, she was definitely calmer.

Last night in Melbourne we had a fairly violent thunder storm, so we immediately gave some Xanax to Maili to help calm her nerves.

The result was not what we expected (or had seen before):  Maili went nuts.

She ran around the garden barking crazy style, she tried to break into the kitchen cupboards and any bags left on the floor, she tore all the clothes off the clothes line and attacked an old bone she has ignored for months, growling and barking at it.    The thunder storm was no longer an issue for her!

This morning Maili has returned to normal (thankfully) but it certainly made for an eventful evening.   It was a great example of those potential side affects they list on the packet, but which you never quite think will happen.

Have a good weekend.

Using XIV with Symantec Dynamic Multi-Pathing

I got a question about Veritas DMP and XIV, so I thought I would write a quick post with some details on the subject.

A fundamental requirement for a host attached to a fibre channel SAN, is the use of multi-pathing software.  One alternative to achieve this (that IBM support for most operating systems attaching to XIV) is Symantec Dynamic Multi Pathing (DMP).   A nice way to find out whether this is the case for your particular operating system is to head to the SSIC, choose Enterprise Disk →  XIV Storage System → Your product version and then Export the Selected Product Version to get a spreadsheet of every supported environment.  Now under the multi-path heading of each page you will see what choices are supported.

Two key benefits of purchasing Veritas DMP  are that:

• It works with heterogeneous storage and server platforms (so you could have EMC and IBM attached to the same server at the same time).
• You can centrally manage all storage paths from one central management GUI.

Then the question becomes,  if I choose to go down the DMP route, do we still need the XIV Host Attachment Kit (HAK)?

The answer is a definite yes!

Veritas DMP and Solaris

If you’re using DMP with Solaris, when you run XIV HAK wizard, it will scan for existing dynamic multi-pathing solutions.  Valid  solutions for the Solaris operating system are Solaris Multiplexed I/O (MPxIO) or Veritas Dynamic-Multipathing (VxDMP). If VxDMP is already installed and configured on the host, it is preferred over MPxIO.

Veritas DMP and Windows

For a Windows host the important point is that Veritas Storage Foundation Dynamic Multipathing (DMP) does not rely on the native multipath I/O (MPIO) capabilities of the Windows Server operating system. Instead, it provides its own custom multipath I/O solution. Because these two solutions cannot co‐exist on the same host, perform the following procedure if you intend to use the Veritas solution:

1. Install the Veritas Storage Foundation package (if it is not already installed).
2. Restart the host.
3. Install the IBM XIV Host Attachment Kit (or run the portable version).
4. The HAK will perform whatever system changes it detects are necessary while still allowing DMP to perform the multipathing.  This may require a reboot (to install Windows hot fixes).

If the version of the Veritas Storage Foundation that you are using does not support the IBM XIV Storage System, install the Device Driver Installation Package 1 (DDI‐1) for Veritas Storage Foundation 5.1.  This update supports the IBM XIV Storage System.

As I said, the HAK will ensure that the required hot fixes are present.  These hot fixes are fairly important.  To understand what tasks the HAK will want to perform WITHOUT performing them, use the portable HAK and run:

xiv_fc_admin -i

This will tell you what tasks will be undertaken when you run the command without the -i parameter.  I detailed this behaviour here.

One benefit of the HAK is the wonderful xiv_devlist command.   Even if you are using DMP, the xiv_devlist command will still work, although you may need to specify veritas as per this example:

xiv_devlist -m veritas

Need more documentation?

This is all documented in the XIV Host Attachment Users Guide which you can find here
.

You lost, stole, gave away or sold your Storwize V7000 USB key?

I love USB keys, I love free ones, conferences give away ones and ones shaped like Lego blocks.   The exciting thing (for me) is that if you buy a Storwize V7000, you also get a USB key:   A key which has two fundamental purposes:

1. It’s used to make installation very quick and easy (which it does very well!).
2. It’s used to reset the superuser password (in case you forget what it is) or to set the service IP addresses (in case you didn’t set them like I suggested you do ).

This is all well and good but what happens when you lose it, borrow it or accidentally throw it out? (oops)   If you are searching for it, yours may well have looked like this:

So what to do?  The answer is:   It’s OK, there is nothing magic about this key.  In fact the key contains just one piece of software, which you can get from here.  Just download the initialization tool and copy it onto your own USB key.  The original key also had an Autorun file, but you don’t need that (actually I object to auto-running USB keys anyway).

BUT… and there is always a but…  I cannot guarantee that EVERY USB key you try will work.   Why not?   Because some USB keys are formatted strangely or insist on running unique applications before they will work.   There is some good, simple advice on the InfoCenter that you can find here.  The main trick is to use a USB key that is formatted with the FAT32, EXT2, or EXT3 file system on its first partition and does not need to auto-run any applications before working.

XIV Release 3.1 materials – including a really cool video

If you are interested in the current state of play with XIV, there are a huge number of new resources that have been created or updated as part of the XIV 3.1 update, so I thought I would give you a list.   If you are a customer then please scan down to see if there is anything here that interests you.  If you an IBMer or IBM Business Partner (or IBM competitor!), this is all mandatory reading.   Either way,  check out the new YouTube video, it is very cool.

New!   XIV – VMware solution brief

New!   XIV – Asynchronous mirroring white paper   (link should be live very soon)

Updated!   XIV – Performance white paper (updated with SSD information)   (link should be added very soon)

XIV Cloud white paper

XIV Hyper-V white paper

IDC analyst report:  XIV as Tier 1

ITG analyst report: TCO comparison: XIV and EMC VMAX/VMAXe

XIV-related case studies

A hot off-the-press walk-thru demo, of the XIV GUI

As promised here is the new video on YouTube that shows the new XIV iPhone App!

I just checked the Apple App Store and cannot see the application yet (only the iPad version).  I will update you the moment the iPhone version becomes available for download (and yes it will have a demo mode).

For more XIV related materials (white papers, demos, videos, case studies…), I invite you to pop over to the XIV area of the ibm web site: ibm.com/storage/disk/xiv. You’ll find  links to materials throughout, such as the SPC report and ISV white papers; click on the Resources tab for a consolidated list of the most recent materials.

Finally, the official announcement letters for version 11.1 and 10.2.4e of the XIV software (which enables all the new functions) are here and here.

XIV Release 3.1 – SSD Read Cache is go!

In July last year I blogged about a brief history of XIV.

In that post I detailed how the XIV began as the Nextra, was then released as the IBM XIV and then updated to the XIV Gen3. So this means last year we saw release 3.0 of the XIV.

At the risk of getting over excited, some of the achievements of the IBM XIV have been truly remarkable:

• There are 59 Clients with more than 1 PB each of usable XIV capacity
• There are 16 Clients with more than 2 PB each of usable XIV capacity

I am sure some competitors will find larger numbers to try to drown out this achievement, but the point is this: These are FANTASTIC numbers. It shows that despite all the FUD, the XIV is a success for IBM and a success for IBM’s customers.

So at the time of the Gen3 release, IBM made no secret of the fact that they planned to add the option of SSD as a read cache layer. In fact each and every Gen3 shipped so far has the mounting and attachment hardware needed to support those SSDs.

Now with release 3.1 IBM turns that promise into a reality.

So… to answer some possible questions:

How can I get some of this SSD goodness?

Order the feature! For existing machines, IBM will need to update the firmware of your XIV Gen3 (non-disruptively) to add SSD support. There will also be an updated version of the XIV GUI. Once those are in place, an IBM Service Representative will add an SSD to each interface module.  They just slide into the mounting kit on each Gen3 module, and are then brought on-line as a group.  No module replacements (or phasing in and out of modules) are necessary and no data needs to be redistributed at any point.  All of this will thus be completed without interruption to your operations.

How much read cache will I get?

Each XIV Gen3 Module already has 24 GB of server RAM. Since an XIV can vary from 6 to 15 modules (based on capacity), that gives you between 144 GB and 360 GB of server RAM to provide read and write cache. If you add the SSD option you will get a 400 GB SSD per module. This means we get between 2.4 TB to 6 TB of additional read cache (depending on module count). The SSDs are not used as write cache.

What administration will I need to perform?

How about none? This is XIV: it’s all about making it simple. It’s no surprise that practically every IBM Storage device now uses the XIV GUI. These guys wrote the book on making things easy to use.

But seriously, no administration? Well… there are two things you may want to do:

1. Check how many SSD based read hits you are getting (versus memory based read hits). It’s always nice to see just how effective these SSDs are proving themselves to be.
2. Turn SSD read caching off or on at a per volume level (by default it is on for all volumes). I don’t anticipate many clients will need or want to do this, but the option is there and it is very easy to do.

Won’t these SSDs wear out or slow down over time?

These are the two great fears of SSD… and XIV development has combined their art with some great work from IBM Research to make sure this is not an issue. The way data is written out to the SSD is handled in a very sophisticated manner. The end result will be consistent and predictable performance with a very long operational life. I will give you more details about exactly how this is done in a future post.

What happens if one of these SSD fails?

Nothing bad.

Because the SSD is not used as write cache, no data can be lost. Data in memory cache is de-staged by that module to both SAS disk and asynchronously to SSD (although not all data will necessarily go to SSD). So there are no bottlenecks and there is no risk. The other modules will keep using their SSDs and IBM will replace the failed SSD non-disruptively.

What sort of performance improvement will I see?

Depending on application and data patterns you should see your IOPS more than double. A three times improvement is quite possible. Response times could drop by more than two-thirds. In many ways these are obvious results.

IBM intend to demonstrate using industry standard benchmarks what the performance of an XIV Gen3 with SSD will be. I can tell you these numbers are going to be very impressive. Watch this space.

Is that it? Any thing else in this release?

Release 3.1  (more correctly known as firmware version 11.1 as per here) also adds:

• The ability to mirror between Generation 2 and Gen3 XIVs.
• All the base support for IPv6 is now in place (although there are still some certification tests to complete)
• Improvements to system thresholds (such as maximum pool size)
• GUI enhancements (mainly to add panels for the SSD cache)
• A new iPhone app (in addition to the existing iPad app)

It’s like Christmas in February.

Report generation tools for XIV

I have previously blogged about two XIV report generation tools that you can download and start using.   This is just a short update to let you know there are updated versions of both tools, plus a new one that has just been added.   These tools are all on my files section at the IBM developerWorks site (where you can also find my Visios).

To sum up what these tools do:

XIV Capacity Report

This Script creates an XLS or CSV file that contains 4 very useful tabs: Systems, Pools, Hosts, Volumes.  You can use this to report on your storage, find un-mapped or un-mirrored volumes, check your consumption, etc.   Clients, Business Partners and Cloud providers love this nice and simple tool.

It is currently up to version 4.6 and you can find it here.

 XIV Performance Report

This Script creates an XLS or CSV file that gives the same information as the XIV Top utility but for a range of days (so we are looking at historic versus current performance).  You could for example see what were the most busy volumes for the past 3 days or for the previous week.   You can easily spot if host HBAs are not being used or if XIV interface traffic is not being balanced.

It is currently up to version 7.2 and you can find it here.

XIV Usage Report – NEW!

This Script creates an Excel file that shows you the current and historic usage of your volumes and pools.   It also gives a trend prediction that will help estimate when your pools or volumes will be full of data.   This is great for trend and growth analysis.

It is currently on version 1.0 and you can find it here.

______________________________________________________________

So get downloading!   Although if you don’t have an XIV, these tools will not be of much use to you.

More details on how to setup and use these tools can be found here.

On a side note, I have found some downloads from the developerWorks site fail if you are using the Chrome browser.  If you are having issues, switch to Firefox (just for that task).