Brocade 8 Gbps Fibre Channel Switches and Fill Words

I thought I would write a quick post about an issue that’s not new, but is certainly worth being aware of….

One of the interesting tricks with the change to 8 Gbps Fibre Channel is that it required a change to the way the switch handles its idle time… the quiet time when no one is speaking and nothing is said.     In these periods of quiet contemplation, a fibre channel switch will send idles.   When the speed of the link increased from 4 Gbps to 8 Gbps, the bit pattern used in these idles proved to not always be suitable, so a different fill pattern was adopted, known as an ARB.   All of this came to intrude on our lives when it became apparent that some 8 Gbps storage devices were having trouble connecting to IBM branded 8 Gbps capable Brocade switches because of this change.  This led to two things:

  1. IBM released several alerts regarding how to handle the connection of 8 Gbps capable devices to 8 Gbps capable fibre channel switches.
  2. Brocade changed their firmware to better handle this situation.

An example of what was said?

“Starting with FOS levels v6.2.0, v6.2.0a & v6.2.0b, Brocade introduced arbff-arbff as the new default fillword setting. This caused problems with any connected 8Gb SVC ports and these levels are unsupported for use with SVC or Storwize V7000.

In 6.2.0c Brocade reintroduced idle-idle as the default fillword and the also added the ability to change the fillword setting from the default of idle-idle to arbff-arbff using the portcfgfillword command. For levels between 6.2.0c and 6.3.1 the setting for SVC and Storwize V7000 should remain at default mode 0.

From FOS v6.3.1a onwards Brocade added two new fillword modes with mode 3 being the new preferred mode which works with all 8Gb devices. This is the recommended setting for SVC and Storwize V7000″

So there are several tips that I will point you to, depending on your product of interest:

Brocade Release Notes

For most environments, Brocade recommends using Mode 3, as it provides more flexibility and compatibility with a wide range of devices. In the event that the default setting or Mode 3 does not work with a particular device, contact your switch vendor for further assistance.  IBM publishes all the release notes for Brocade Fabric OS here.

DS3500/DS3950/DS5000

Check out this link if your connecting an 8 Gbps capable DS3500, DS3950 or DS5000 to a 8 Gbps capable switch:  http://www-947.ibm.com/support/entry/portal/docdisplay?brand=5000028&lndocid=MIGR-5083089

DS8800

There is no tip for the DS8800 but the advice remains effectively the same as for the Storwize V7000.   I can confirm that using a fill word setting of 3 works without issue.

SAN Volume Controller or Storwize V7000

Check out this link if your connecting a Storwize V7000 or CF8 or CG8 SVC node to an 8 Gbps capable switch:  https://www-304.ibm.com/support/docview.wss?uid=ssg1S1003699&wv=1

XIV Gen3

The XIV Gen3 comes with 8 Gbps capable Fibre Channel connections.   It does not support idle Fill Words meaning that the portCfgFillWord value should not be set to 0.

FICON Express-8

When an IBM System z server attaches an 8 Gbps capable FICON Express-8 CHPID to a Brocade switch with 8 Gbps capable SFPs,  you should upgrade your switches or directors to Fabric OS (FOS)  6.4.0c or 6.4.2a and set the fill word to 3 (ARBff).

LTO5 and TS1140

IBM have two tape drives that are capable of 8 Gbps, the LTO-5 drive and the TS1140.  Setting the fill word to 3 can actually cause issues with these drives.   To avoid issues do one of the following (you only have to do one of these, not all three):

  • Load the tape drive with firmware that has the access fairness algorithm fix for loop:
    – LTO5 drives should be on BBN0 and beyond (you may need to contact IBM support to get this code.
    – TS1140 drives should be on drive firmware 5CD or beyond.
  • Change the Fibre Channel topology to point-to-point (N port) (as opposed to L or NL).   This is my preferred option.
  • Change the Fibre Channel speed to 4Gbps.  This sounds slightly retrograde, but it is very rare for an individual drive to sustain a speed above 400 MBps (unless your data is very very compressible).

**** UPDATED 28 Feb 2012 – Added System z FICON and Tape info ****

Advertisements

About Anthony Vandewerdt

I am an IT Professional who lives and works in Melbourne Australia. This blog is totally my own work. It does not represent the views of any corporation. Constructive and useful comments are very very welcome.
This entry was posted in DS8800, IBM Storage, IBM XIV, SAN, Storwize V7000, SVC and tagged . Bookmark the permalink.

8 Responses to Brocade 8 Gbps Fibre Channel Switches and Fill Words

  1. Pingback: Brocade 8 Gbps Fibre Channel Switches and Fill Words | Storage CH Blog

  2. Urban says:

    Did Brocade or Cisco do something non-standard? I find it interesting Cisco SAN switches never had this problem.

    • avandewerdt says:

      Your correct that CIsco have never had this issue. A search of the Cisco support site did not yield one hit on terms like “Fill Word” and ARB. The bottom line is that Brocade appear to have wandered off the reservation somewhat. I got this extra info from IBM SAN Support in Europe that throws additional light on this:

      – In fact mode 2 is according to the FC standard. But as HBA Vendors already had to cope with the wrong Brocade implementation, there are devices out there relying on the wrong way to do it. So as you already stated, mode 3 is the best compromise.
      – Although described to affect all connections, this setting is only important for Non-ISL ports. If you have for example a longdistance ISL, the fillword depends on the VC Link Init parameter. You will even get a warning if your longdistance config is contrary to the fillword setting, but it’s meaningless. But of course, if you change the fillword modes on your switches anyway, there’s no harm to have the ISLs on mode 3, too.
      – If you have no chance to update to the latest codes and have to stick with either mode 0 or 1, mode 0 is recommended for 8G ports. Dont worry if you see “Invalid transmission words” (bad_os) increasing in portstatsshow. The switch just complains about getting the wrong fill word here. That’s not a problem unless you have portfencing enabled, which could cause the switch to automatically disable the port because of it. If you use 8G with fillword mode 0, don’t use portfencing

  3. Von Mazzillo says:

    Hi, I can’t determine ways to add your web site in my RSS feed reader. Can you tell me what I’m doing wrong, please.

  4. Thierry B says:

    This blog is really a gold mine, thanks a lot!

  5. cheap toms says:

    Admiring the time and effort you put into your website and
    detailed information you present. It’s good to come across a blog every once in a while that isn’t the same
    outdated rehashed material. Excellent read! I’ve saved your site and I’m including your RSS feeds to my Google
    account.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s