NOTE: My original article contained embedded calculation errors that significantly distorted the end results. These problems have since been corrected. I apologize to anyone who was accidentally misled by this information, and sincerely thank those diligent readers who brought the issues to my attention.
Some issues seem so obvious they’re hardly worth considering. Everyone knows that Solid State Drives (SSD) are more energy-efficient than spinning disk. They don’t employ rotating platters, electro-mechanical motors and mechanical head movement for data storage, so they must consume less power – right? However, everyone also knows the cost of SSD is so outrageous that they can only be deployed for super-critical high performance applications. But does the reputation of having exorbitant prices still apply?
While these considerations may seem intuitive, they are not entirely accurate. Comparing the Total Cost of Ownership (TCO) for traditional electro-mechanical disks vs. Solid State Disks provides a clearer picture of the comparative costs of each technology.
- For accuracy, this analysis compares the purchase price (CAPEX) and power consumption (OPEX) of only the disk drives, and does not include the expense of entire storage arrays, rack space, cooling equipment, etc.
- It uses the drive’s current “street price” for comparison. Individual vendor pricing may be significantly different, but the ratio between disk and SSD cost should remain fairly constant.
- The dollar amounts shown on the graph represent a 5-year operational lifecycle, which is fairly typical for production storage equipment.
- Energy consumption for cooling has also been included in the cost estimate, since it requires roughly the amount of energy as the drives consume to maintain them in an operational state.
- 100 TB of storage capacity was arbitrarily selected to illustrate the effect of cost on a typical mid-sized SAN storage array.
The following graph illustrates the combined purchase price, plus energy consumption costs for several popular electro-mechanical and Solid State Devices.
From the above comparison, several conclusions can be drawn:
SSDs are Still Expensive – Solid State Drives remain an expensive alternative storage medium, but the price differential between SSD and electro-mechanical drives is coming down. As of this writing there is only an x5 price difference between the 800GB SSD and the 600GB, 15K RPM drive. While this is still a significant gap, it is far less that the staggering x10 to x20 price differential seen 3-4 years ago.
SSDs are very “Green” – A comparison of the Watts consumed during a drive’s “typical operation” indicate that SSD consumes about 25% less energy than 10K RPM, 2.5-inch drives, and about 75% less power than 15K RPM, 3.5-inch disks. Given that a) each Watt used by the disk requires roughly 1 Watt of power for cooling to remove the heat that is produced, and b) the cost per Kwh continues to rise every year, this significant difference become a factor over a storage array-s 5-year lifecycle.
Extreme IOPS is a Bonus – Although more expensive, SSDs are capable of delivering from 10- to 20-times more I/O’s-per-second, potentially providing a dramatic increase in storage performance.
Electro-Mechanical Disks Cost Differential – There is a surprisingly small cost differential between 3.5 inch, 15K RPM drives and 2.5 inch 10K RPM drives. This may justify eliminating 10K disks altogether and deploying a less complex 2-tiered array using only 15K RPM disks and 7.2K disks.
Legacy 3.5 Inch Disks – Low capacity legacy storage devices (<146GB) in a 3-5-inch drive form-factor consume too much energy to be practical in a modern, energy-efficient data center (this includes server internal disks). Any legacy disk drive smaller than 300 GB should be retired.
SATA/NL-SAS Disks are Inexpensive – This simply re-affirms what’s already known about SATA/NL-SAS disks. They are specifically designed to be inexpensive, modest performance devices capable of storing vast amounts of low-demand content on-line.
The incursion of Solid State Disks into the industry’s storage mainstream will have interesting ramifications not only for the current SAN/NAS arrays, but also may impact a diverse set of technologies that have been designed to tolerate the limitations of an electro-mechanical storage world. As they say, “It’s a brave new world”.
Widespread deployment of SSD will have a dramatic impact on the storage technology itself. If SSDs can be implemented in a cost-effective fashion, why would anyone need an expensive and complex automated tiering system to decrement data across multiple layers of disk? Because of its speed, will our current efforts to reduce RAID rebuild times still be necessary? If I/O bottlenecks are eliminated at the disk drive, what impact will it have on array controllers, data fabric, and HBAs/NICs residing upstream of the arrays?
While it is disappointing to find SSD technology still commands a healthy premium over electro-mechanical drives, don’t expect that to remain the case forever. As the technology matures prices will decline when user acceptance grows and production volumes increase. Don’t be surprised to see SSD technology eventually eliminate the mechanical disk’s 40-year dominance over the computer industry.
For those of you interested in examining the comparison calculations, I’ve included the following spreadsheet excerpts contain detailed information used to create the graph.
One of the more promising technologies for improving applications and databases performance is the PCIe Flash card. This technology uses a standard half or full-height PCI-e interface card with a Solid State Disk mounted on it. It allows SSD to be added to a workstation or server by simply plugging a card into an available PCIe bus slot.
What makes PCIe Flash card approach superior to array-based SSD is its close proximity to the system memory and the elimination of latency-adding components to the I/O stream. In a normal SAN or NAS array, data is transferred to storage across the SAN fabric. Bytes of data move across the systems PCIe I/O bus, where it is read by the HBAs (or NICs if it’s NAS), translated into the appropriate protocol, converted to a serial stream, and sent across the SAN fabric. In most SANs the signal is read and retransmitted one or more times by edge switches and directors, then sent to disk array controllers. From there it is converted from a serial stream to a parallel data, translated from the SAN fabric protocol, given a block-level addressing, possibly stored in array cache, re-serialized for transmission to the disks, received and re-ordered by disks for efficient write processes, and finally written to the devices. For a data read, the process is reversed via a similar process.
Like other technologies, however, there are pros and cons to using PCIe Flash storage:
- Plugs directly into the PCIe bus, eliminating latency from the HBAs, network protocols, SAN fabric, array controller latencies, and disk tray connections.
- PCIe is a point-to-point architecture, so each device connects to the host with its own serial link
- PCIe Gen 2 supports 8Gbps/sec., which is 25% faster than the 6Gbps SAS interface
- Little or no additional infrastructure is required to capitalize on flash storage performance
- Very simple to deploy and configure
- Extremely low power consumption, as compared with traditional 3.5-inch hard disks.
- Positions data in very close proximity to the system processors and cache structure
- Requires no additional physical space in the storage equipment rack
- The number of SSD storage deployed is limited by the physical number of slots
- Some PCIe Flash cards are “tall” enough to block the availability of an adjacent slot
- Recent PCIe bus technology is required to support top performance (x4 or above)
- Internal PCIe storage cannot be shared by other servers like a shared SAN resource
- May require specialized software for the server to utilize it as internal cache or mass storage
- PCIe Flash may suffer quality issues if an Enterprise-Grade product is not purchased
- If the server goes down, content on the installed PCIe Flash becomes inaccessible
Like other SSD devices, Flash PCIe cards are expensive when compared to traditional disk storage. In Qtr4 of 2012, representative prices for 800GB of PCIe Flash storage are in the range of $3800 to $4500 each. Since a 15K RPM hard disk of similar capacity sells for $300 to $450 each, Flash memory remains about ten times more expensive on a cost-per-GB basis. However, since Solid State Disk (SSD) is about 21 times faster than electro-mechanical disk, it may be worth the investment if extremely fast performance is of upmost importance.