|2013-07 Storage Connectivity SAS & PCIe|
This article attempts to provide a basis for assessing the fitness of computer platforms for desktops and servers in terms of storage. A desirable pre-requisite article to read is “Computing Subsystem Performance” for a fuller context. This article focuses on internal storage connectivity. Technologies discussed are SAS2, SAS3, RAID, and PCIe. Storage devices discussed include SSD and HDD.
SAS (Serial Attached SCSI)
o SAS2 or 6G/s SAS has been around in the market since 2010 and SAS3 or 12G/s SAS will be introduced in late 2013 or early 2014 timeframe. Do we need to care about these standards? Yes in case storage performance is critical for the application.
o SAS2 came in at a time when SSD started receiving public attention. Assuming SAS2 has a bandwidth of 600MB/s, it is not useful at all for most HDD which achieved a maximum read rate of around 100MB/s. HDD did not hit the wall in a SAS1 pipe (half of SAS2), and therefore the widening of the pipe did not allow HDD to perform higher or allowed a marginal performance gain of HDD at most. However SSD achieved a higher maximum read rate than HDD that is close to SAS1 pipe width. The widening of SAS to 600MB/s saw SSD achieving 560MB/s of continuously read rate. Will another doubling of SAS pipe facilitate another doubling of SSD read rate?
PCIe (Peripheral Connect Interface Express)
o Some SSD vendors have been very ambitious in breaking the SAS2 limit by aggregating multiple SSD onto a PCIe card. PCIe v2 has a bandwidth of 500MB/s per lane or 4GB/s for 8 lanes aggregate. SSD vendors have borrowed this aggregation idea to aggregate multiple SSD units together. A read rate of 3GB/s has been published for products with an interface of PCIe v2x8.
o Bandwidth aggregation is not new for HDD in RAID arrangements. RAID controllers are able to put 3, 4, 5 or more low cost SATA HDD units together and fill up a SAS channel. In this respect, RAID is a tool for reducing the cost of achieving high read/write performance beyond the fundamental purposes of providing data security and integrity.
o To understand how RAID vendors maximize their opportunities, see this example. Compucon 4UD72 big data system (72 x 4TB = 288TB) can be configured with 3 RAID control cards each with 8 SAS2 ports (8 x 600MB/s) on PCIe v2 x8 (4GB/s) for a total of 20 SAS2 channels (20 x 600MB/s = 12GB/s). This is an extreme case of one 4U PC providing 288TB of storage with 12GB/s of theoretical max read output.
PCIe v3 has been around for a couple of years and we expect PCIe SSD and RAID vendors to be more aggressive in this respect as soon as SAS3 is released.
SAS3 = 1.2GB/s