EPCIe marketing hype
by Alan Okey
on
Jun 26, 2008 at 7:21:07 pm
Disclaimer: Please don't take this as a rip on CalDigit. I love their products and I hope to purchase an HDPro or HDOne in the near future.
- anyway -
I find it interesting how the External PCIe connection standard has become the latest marketing buzz for storage vendors. While it's true that EPCIe has much greater bandwidth than eSATA, it doesn't eliminate the need to bridge SATA drives to the PCIe bus. It's simply a matter of where the conversion physically takes place. Prior to EPCIe, the conversion happened on a SATA host card in a PCIe slot. Now, the conversion happens on a bridge chip in the array enclosure.
There's no such thing as a native PCIe drive. Somewhere, there must be a bridge chip between SATA (or SAS) and PCIe. Moving the bridge outside the host system to the external enclosure doesn't eliminate the conversion, it merely shifts the location of where the conversion happens.
From CalDigit's HDPro page:
Unlike Fibre Channel, SCSI, FireWire and eSATA RAID storage, CalDigit HDPro is a native PCI Express using a direct connection to the computers bridge chip and memory system combined with high link speeds. This direct link eliminates latency introduced by the conversion of other interfaces to PCI Express and provides superior bandwidth, availability and deployment flexibility over earlier-generation SCSI and Fibre Channel technology.
From Ciprico's site:
Based on the new PCI Express (PCIe) external expansion and cable connection standard, the MV5100 series expands your PC, server or workstations core logic PCIe bus outside of the box, enabling the ultimate in direct attached storage. At a raw speed of 20Gbps, overheads from protocol conversion to serial protocols such as Fibre Channel, Infiniband or Ethernet are totally eliminated.
I find the bandwidth comparison graphic on the HDPro page to be a little disingenuous, if not irrelevant. The interface isn't the bottleneck, the drives are. The Ciprico description also ignores the fact that SATA drives are serial protocol devices by definition (Serial ATA), so a conversion must still take place. Unless a native PCIe drive connection emerges, there's going to be a protocol conversion somewhere in the data chain.
I can see where the introduction of a third transfer protocol like Fibre Channel or Firewire might introduce latency, but how does using an EPCIe interface vs. an eSATA interface improve on the performance of an external SATA RAID? What are the technical advantages of using EPCIe as an interface from the host system to the array?
I'm guessing that reduced internal heat in the host system would be one potential advantage of using the EPCIe interface card. Perhaps the EPCIe cabling standard has better performance over distance than eSATA? Is there anything else I'm missing?
Re: EPCIe marketing hype by Jared Picune on Jun 26, 2008 at 8:03:38 pm
Alan,
You make a very interesting assessment of external PCIe. It's a little bit of a conundrum if you think about it. If you were to use eSATA, you would have to add a SATA card to your PCIe bus.
But where the real advantage of speed comes, is from the RAID itself. I'm going to over simplify, but let look at the HDPro. It has 8 SATA II drives. Each have a bandwidth of 3Gbs. Now the drives themselves are not that fast. But when you add 8 together in a TURE hardware RAID (independent CPU and cache) you are able to achieve speeds that are almost double the bandwidth of the SATA II bus.
To get all of this supper fast data to the computer you need a fat pipe. Fibre has been used by other companies, but the catch is that this high throughput is close to it's maximum bandwidth. With ePCIe you have 20Gbs. That leaves plenty of overhead.
So in summary the type drives used in the RAID become irrelevant. SATA II drives are fast and affordable, which is why they are used. You could use SAS drives in the same fashion and achieve even greater speeds, ePCIe allows for that overhead.
The cable type and length does vary between ePCIe and eSATA. Also the ePCIe connection is a lot more substantial than eSATA. I also find eSATA to be flakey while ePCIe is rock solid. But the major advantage is the speed difference eSATA 3Gbs and ePCIe 20Gbs. You just can't have a RAID that runs at 400+ MBs over an eSATA connection.
Re: EPCIe marketing hype by Alan Okey on Jun 26, 2008 at 8:09:59 pm
[Jared Picune]"But the major advantage is the speed difference eSATA 3Gbs and ePCIe 20Gbs. You just can't have a RAID that runs at 400+ MBs over an eSATA connection."
A typical eSATA RAID would use several port-multiplied 3Gbps connections, and aggregate the bandwidth.
All things being equal, however (which they're not), I'll take a single fat pipe over bundled skinny pipes any day. In short, I'm glad that we have ePCIe as an option.
Re: EPCIe marketing hype by Jared Picune on Jun 26, 2008 at 8:29:02 pm
[Alan Okey]"A typical eSATA RAID would use several port-multiplied 3Gbps connections, and aggregate the bandwidth. "
An eSATA solution like that would likely be software RAID.
There is a big difference between Hardware and Software RAID. With a hardware RAID, you have an independent CPU and cache that focus all of their energy and attention to the RAID, this gives you maximum performance and protection.
This is why the HDPro/HDOne are unique and ePCIe is the obvious option at this level of performance. I guess what I'm saying is there is a lot more to overall performance than just the connection. Generally in this category bigger is better.
Re: EPCIe marketing hype by Alan Okey on Jun 26, 2008 at 8:40:35 pm
[Jared Picune]"An eSATA solution like that would likely be software RAID."
Unless it were configured with an Apple, CalDigit or other vendor's hardware RAID PCIe card. Isn't the HDElement precisely that configuration?
As the cost of a hardware RAID eSATA solution approaches the entry price point for the better ePCIe options that you mentioned, and when you consider the added benefit of not adding more heat to the host system, I'd personally opt for the ePCIe solution.
Re: EPCIe marketing hype by Jared Picune on Jun 26, 2008 at 9:07:34 pm
[Alan Okey]"Unless it were configured with an Apple, CalDigit or other vendor's hardware RAID PCIe card. Isn't the HDElement precisely that configuration?
"
Well not exactly, the HDElement connects to the CalDigit RAID Card via a mini-SAS cable. So neither the Apple or CalDigit RAID Card connect to drives via external SATA.
The CalDigit RAID card is essentially the same as the "guts" of the HDOne. The main advantage of a card like this is the flexibility that it can give you. Instead of just 8 drives, you can start your RAID at 2 drives and migrate all the way to 16.
[Alan Okey]"As the cost of a hardware RAID eSATA solution approaches the entry price point for the better ePCIe options that you mentioned, and when you consider the added benefit of not adding more heat to the host system, I'd personally opt for the ePCIe solution. "
ePCIe is awesome, and I think it has a long future much like fibre has had.
Alan thanks for this discussion, I think it will be very educational to many people. ePCIe tends to get a little confusing at time.
Re: EPCIe marketing hype by Morten Stjernholm on Jun 27, 2008 at 7:40:33 am
With Fibre, SCSI etc there is a ADDITIONAL translation of the interface causing some latency. With the HDPro you can actually feel the difference on your timeline.
