Your DS3500 needs new firmware to support T10-PI

For those of you who use the IBM DS3500 (a midrange storage controller), you should ensure all your machines are on firmware release or higher since this adds support for T10-PI.   This is because new additional or replacement disk drives may require that support.   Inserting high-level drives into down-level machines can result in a failed drive replacement or unexpected errors.   Ideally you should not be upgrading your machines while there is a failed component, so I recommend you pro-actively upgrade your DS3500s, particularly if you are ordering new drives or additional enclosures.

Note that IBM recommend on this page.  New firmware can be downloaded from here.

If you wondering what on earth T10-PI is, check out this blog here.    If you use AIX there is also a short write up here.  It does not mention DS3500, but I think this is due to the age of the post.

You can tell that T10-PI support is enabled for an array very easily in the upgraded GUI.

T10-PI Screen Cap

Posted in Uncategorized | 1 Comment

Innovate, emulate or evaporate.


The IT Industry is changing rapidly.   New disruptive technologies are changing the whole playing field and vendors who just talk about backup are going the way of the dinosaur.   Actifio saw this more than four years ago and began a new era of Copy Data Management. Finally the other guys are starting to realize the ground has shifted below their feet and have begun talking about doing exactly the same thing (without actually changing anything that they currently do).

But don’t just listen to me, have a read of Chris Mellors analysis and watch the EMC video.   Then talk to Actifio and get today what EMC cannot deliver tomorrow.

Posted in Actifio | Tagged , , , , | Leave a comment

Backblaze Blog » How long do disk drives last?

This is fascinating stuff that pretty well exactly matches my experience with almost any IT product.  Congrats to Backblaze for collecting and sharing this information.

Backblaze Blog » How long do disk drives last?.

Posted in advice | Leave a comment

Your SanDisk USB stick is no longer removable – and it’s Microsofts fault!

I came across something curious last week that failed the question:
Can I find a quick answer for this in Google?

I had purchased a new SanDisk USB stick to boot Linux.   The script I was using to configure it had a check to make sure the disk target (e.g./dev/sdb)  I was installing Linux onto was removable media.   The script kept complaining that my new USB stick was not! I found an older SanDisk USB stick and with that one, my script ran without error.

Had I bought a bad USB stick?   It formatted without issue and I could write to it and read from it without any errors.

I then compared the dmesg output for the two sticks:

Newer USB stick:

[833126.592645] scsi 10:0:0:0: Direct-Access SanDisk Cruzer Switch 1.26 PQ: 0 ANSI: 6
[833126.592794] sd 10:0:0:0: Attached scsi generic sg2 type 0
[833126.593499] usb-storage: device scan complete
[833126.596965] sd 10:0:0:0: [sdb] 15633408 512-byte logical blocks: (8.00 GB/7.45 GiB)
[833126.601274] sd 10:0:0:0: [sdb] Write Protect is off
[833126.601276] sd 10:0:0:0: [sdb] Mode Sense: 43 00 00 00
[833126.601278] sd 10:0:0:0: [sdb] Assuming drive cache: write through
[833126.609031] sd 10:0:0:0: [sdb] Assuming drive cache: write through
[833126.609042] sdb: sdb1
[833126.634951] sd 10:0:0:0: [sdb] Assuming drive cache: write through
[833126.634954] sd 10:0:0:0: [sdb] Attached SCSI disk

Older USB stick:

[151567.256075] scsi 2:0:0:0: Direct-Access SanDisk Cruzer Switch 1.20 PQ: 0 ANSI: 5
[151567.256497] sd 2:0:0:0: Attached scsi generic sg2 type 0
[151567.257279] usb-storage: device scan complete
[151567.261777] sd 2:0:0:0: [sdb] 7821312 512-byte logical blocks: (4.00 GB/3.72 GiB)
[151567.265883] sd 2:0:0:0: [sdb] Write Protect is off
[151567.265888] sd 2:0:0:0: [sdb] Mode Sense: 43 00 00 00
[151567.265897] sd 2:0:0:0: [sdb] Assuming drive cache: write through
[151567.278033] sd 2:0:0:0: [sdb] Assuming drive cache: write through
[151567.278041] sdb: sdb1
[151567.287932] sd 2:0:0:0: [sdb] Assuming drive cache: write through
[151567.287935] sd 2:0:0:0: [sdb] Attached SCSI removable disk

I noticed the difference between the newer SanDisk USB stick and the older USB stick was the killer line at the end.  The old stick was identified as a ‘removable disk’ while the new stick was identified as a ‘disk’.

The other difference was simpler to spot.  The ‘removable’ variable was ‘1’ on the old stick and ‘0’ on the new stick.

 # cat /sys/block/sdb/removable

I then found this on the SanDisk web site which explained what I was seeing and confirmed that there was nothing wrong with this new USB stick:

Flash Drive shows as Hard Disk Drive (Fixed Disk) in Windows Explorer

Where it told me:

NOTE: SanDisk is beginning production of flash drives configured as fixed disk in 2012 to meet new requirements for Windows 8 Certification.

Historically, flash drives have been configured as removable disks and Windows Explorer displays them as ‘Removable Media’. Windows 8 Certification requires flash drive manufacturers to configure flash drives as fixed disks. Flash drives configured as fixed disk will show up in Windows Explorer as ‘Hard Disk Drives’. Flash drives configured as fixed disks still function the same as those configured as removable disks.

NOTE: SanDisk does NOT support configuring flash drives as bootable device for running an OS.

This change (regardless of what SanDisk says) does not stop you using these new sticks as USB boot media.  But it does mean my Linux install script can no longer programmatically identify SanDisk USB sticks as removable media… and its all because of Microsoft.

I can only imagine Steve Ballmers reaction on learning this:

Posted in advice | Tagged , , , , , , | 18 Comments

Different story but the plot remains the same

I just finished the latest Jack Reacher novel, “Never Go Back”, by Lee Child, and it was everything I expected.   If you have ever read any of the Jack Reacher novels you will know what I mean.   Every Jack Reacher novel contains pretty well the same elements: there are baddies (normally corrupt politicians, businessmen or military officers); their henchmen will foolishly tangle with Jack; some of these bad guys will end up with broken limbs, some will end up dead (some by Jack’s hand);  Jack will show his physical and military prowess; Jack will normally sleep with someone perfect…   and..  well…  it’s perfect.   But at the end, despite the romance, once everything is wrapped up nicely,  Jack disappears into the sunset, on to another town, another corrupt baddie, another perfect love interest.

Its a great formula and it makes for great reading.   Despite knowing that each novel will read much like the previous one, I keep reading them.

EMC announcements are a bit like that, there are goodies and baddies, but always the same story with no surprises.   The story always ends the same way.

Take their recent Backup is Broken pitch.   It’s worth taking the time to watch it (it is four and half minutes).      It sounds great on the surface, but you don’t have to dig too far to start seeing that nothing has changed.   Its funny to watch a pitch that starts off blaming the backup team and their current generation of tools, kindly supplied by EMC, for a whole raft of business shortcomings.   The solution is to then buy warmed over versions of the same products.   Sadly what doesn’t change is that EMC cannot fix the issue that the market has too many point solutions (many of them EMC products); that their backup products do nothing to help slow the growth in test and dev environments and that their product line is all designed from the perspective of EMC and the companies they have acquired, not from the perspective of the client.   It really highlights the innovators dilemma, how can EMC innovate when they also have to protect the revenue base from their existing platforms?

If you are still keen to watch videos, have a listen to what this client has to say about Actifio and how it compares with EMC for backup and recovery.     It makes for an interesting comparison, a story with a much better ending.

Posted in Actifio | Tagged , | Leave a comment

Pure Storage hits the Australian market!

Going to vendor presentations is always interesting on three fronts:

  1. What new technology is hitting the market?   The IT industry is constantly changing, which means to survive you need to be constantly learning.
  2. How are other vendors presenting their message?    Can I pick up any good ideas?
  3. How will their new innovations affect the market?   Are we all about to be unemployed?

What led me to this musing is a presentation I  attended today by Pure Storage.   It was a breakfast presentation, but they made it run fast, with a nominal start time of 7:30; talking starting at 7:45 and done by 9am!   Frankly this shows a lot of respect for the audience.   People simply don’t have the time for a half day seminar let alone a whole day.

The location (the Sofitel) was top dollar so their marketing budget clearly showed, as did their hiring budget.  They had A-Team Executives there including President David Hatfield, their CMO, the amazing Vaughn Stewart and Michael Cornwell their VP for APAC.   These guys have clearly assembled a killer leadership team and are hiring big (so big that EMC are suing people – check that story out here).

The bulk of the presentation was by David Hatfield who spoke really well and presented a great slide deck (top marks for presentation).

New entrants always struggle to get past the ‘your new, why should I trust you’  problem, so they immediately countered by telling us that they are on their third generation box – having started in analytics.      They then talked up their huge well placed and independent funding.   Its clear these guys are funded to go big (and are spending big right now to get there).

The opened the presentation with images of iPhones and of Google racks.   While these were aimed (I think) to  show the pervasiveness of flash, they also served to associate Pure Storage with ease of use which was one of their major messages.

But their main message was clear:   The disk is the bottleneck.   That hybrid (cache/flash/disk)  technology still allows for cache misses and still require too much setup and consulting and that all-flash devices using Consumer Grade MLC is the way to go.   They manage cost by 6:1 dedup inline ingest (depending on data type).  This means you can store 6 GB of client data in 1 GB of flash.  

They said their array would last 7-10 years as every component could be upgraded on the fly with no impact which is an impressive engineering goal.

They showed a case study of Paylocity, a Payroll Company who got 800% better SQL response ones, dropped backup times from 1.5 hours to 10 minutes, went from 8x 30 amp to 2 x 15 amp, reduced rack space from  58U to 10U, took 150TB of 3PAR and HDS down to 35TB of Pure Storage  Flash and achieved 4:1 dedup. 

The CEO of their local reference (Charles Tym from Harbour IT),  was part of a panel they then ran.  Charles spoke about how they bought UCS and Nexus but they could not get more than 10% out of their network because of disk latency.   They increased their disk footprint but this didn’t help (he didn’t say which vendor/brand).   They tried Pure Storage and got 10:1 dedup and amazing performance and are now retiring their disk based storage to be used as backup and DR disk.

So what did I think?

  1. They gave a great pitch but need to keep the focus on how they solve business problems (not just reduce cost).   There are too many flash entrants to just focus on speeds/feeds and on heat, space, noise and cost reduction.
  2. Their simplicity  message is key….  it’s what the market wants.
  3. They need to sell big… it seems to me that with the big investment in manpower they are making right now they will need to start showing results very quickly, regardless of how well they are funded.

The flash vendor wars are now in full swing.   Pure Storage are showing all the signs of being a big winner.   I wish them luck.

Posted in Uncategorized | 4 Comments

Safety Switches in the Data Center?


If you are living in a house in Australia built since some point after the year 2000, your home is almost certainly being protected by a safety switch – normally an RCD or Residual Current Device.  The job of the RCD is to cut off power if there is an imbalance between the Active and Neutral conductors, normally caused by a short to ground. The good news is that the RCD is keeping your home safe, protecting your family and making it 40% less likely your home will burn down due to fire.  The RCD in my own house has tripped occasionally, normally because a household kitchen device like an electric frypan or the element in my clothes dryer has ‘blown up’ but occasionally it trips for reasons unknown.

I had always presumed that RCDs did not have a role to play in the data centre, but I recently learned that this is not the case.  I attended a conference entitled “Meet the Data Centre Experts” where David Morley, Data Centre Engineer from CITEC  in Brisbane explained that many of their data centre racks are now protected by RCDs. This change was made after the release of the AS/NZS 3000: 2007 Wiring Rules & subsequent revisions. These revisions specify requirements for Residual Current Protection to be installed to practically all new and modified power circuits; this includes those within the Data Centre.

The only racks at CITEC that do not have RCDs are those where power consumption exceeds the capability of the RCD or where the equipment in the rack is so sensitive (due to its role in say ‘Emergency Services’) that the equipment is quarantined from RCD use.

Because power supply filters create a lot of transient noise and leakage an improved version of the RCD known as a Super immunised RCD is being used to eliminate nuisance trips. They test their RCDs every 6-24 months which means a rolling series of power interruptions on one ‘side’ of each rack. Rather than do rail1 followed by rail2, they normally test all the rail1s in a series of racks and then all the rail2s. This gives a ‘decent interval’ between power supply interruptions so the affected equipment can reset/recover from a power input loss error condition.

Interestingly the main issue that has come out of this is the human factor. Pre-checks occasionally find mis-cabled racks where data centre equipment either has only one power supply or where the equipment has dual power, both cords are attached to the same rail. Even racks that were previously ‘compliant’ can be found to be out of compliance when new equipment is added or worse, when existing equipment is maintained or repaired and the vendor doing the work does not correctly reconnect the power cords. Finally both power supplies need to be operational, so failed power supplies are also unearthed in this process.

The RCDs in use at CITEC appear to come from Geist who coincidentally sponsored the presentation.

Posted in advice | Tagged , , , , , , , | 1 Comment