There’s a quiet revolution going on in large data centers.  It’s not as visible or flashy as virtualization or deduplication, but at least equal in important.

As its name implies, SAN “fabric” is a dedicated network that allows servers, storage arrays, backup & recovery systems, replication devices, and other equipment to pass data between systems.  Traditionally this has been comprised of 4Gbps Fibre Channel and 1Gbps Ethernet channels.  However, a new family of 8Gbps and 16Gbps Fibre Channel, 6Gbps and 12Gbps SAS, and 10Gbps Ethernet are quietly replacing legacy fabric with links capable of 2 – 4 times the performance.

The following is a comparison of the maximum throughput rates of various SAN fabric links:

A comparison of available SAN channel speeds.

Performance ranges from the relatively outdated 1Gbps channel (Ethernet or FC) capable of supporting data transfers of up to 100 MB per second, to 16Gbps Fibre Channel capable of handling 1940 MB per second.  Since all are capable of full duplex (bi-directional) operations, the sustainable throughput rate is actually twice the speed indicated in the chart.  If these blazing new speeds are still insufficient, 10Gbps Ethernet, 12Gbps SAS, and 16Gbps Fibre Channel can be “trunked” – bundled together to produce an aggregate bandwidth equal to the number of individual channels tied together.  (For example, eight 16Gbps FC channels can be bundled to create a 128Gbps “trunk”.)

In addition to high channel speeds, 10Gbps Ethernet and 16Gbps Fibre Channel both implement a 64b/66b encoding scheme, rather than the 8b/10b encoding scheme used by lower performance channels.  The encoding process improves the quality of the data transmission, but at a cost.  An 8b/10b encoding process decreases available bandwidth by 20%, while 64b/66b encoding only reduces bandwidth by 3.03%.  This significantly increases data transfer efficiency.

While 8/16Gbps Fibre Channel and 10Gbps Ethernet are changing the game at the front-end, SAS is revolutionizing the back-end disk drive connections as well.  For over a decade, enterprise-grade disks had 2Gbps or 4Gbps ports, and were attached to a Fiber Channel Arbitrated Loop (FC-AL).  Like any technologies using loop technology, low traffic enjoyed maximum speed but performance dropped off as demand increased.  Under heavy load conditions, the back-end bus could become a bottle-neck.

SAS will change that for two reasons.  First it uses switched technology, so every device attached to the controller “owns” 100% of the bus bandwidth.  The latency “dog leg pattern” found on busy FC-AL busses is eliminated.  Secondly current SAS drives are shipping with 6Gbps ports, which are 50% faster than 4Gbps Fibre Channel.  Just over the horizon are 12Gbps SAS speeds that will offer a 300% increase in bandwidth to the disks, and do it over switched (isolated) channels.

Recent improvements in fabric performance will support emerging SSD technology, and allow SANs to gracefully scale to support storage arrays staggering under a growth rate of 40% – 50% per year.