Storage options for a SAS-enclosure owner in a Thunderbolt world
I've got an iStorage Pro 8 bay filled with 2 GB drives, and an ATTO R380 running on my 2009 Mac Pro in a RAID5 config (I have since realized that I should be using RAID6 for the more favorable reliability / performance tradeoff), and like many of you I’m eagerly awaiting the new mac pro. I've been researching various storage options, and would like to offer these options up for comment and criticism from the group.
My use case is single-user enthusiast-level media production, with the only firm constraint being the ability to sustain a modest 450 MB/s or so write to achieve reliable 120 fps capture of video game footage at 1920 x 1200. My current setup does 500 to 550 MB/s, which is enough, but is slower than it could be, possibly because the R380 is in slot 3 instead of slot 2, so it runs at 4x instead of 8x; I don't remember why I set it up this way, and I should probably do something about it… Anyway :)
I see three general approaches for moving to a thunderbolt-based mac, two of which retain my existing enclosure and storage: 1) External Thunderbolt --> PCI chassis to house my R380, such as perhaps the mLogic mLink, keeping the iStorage Pro, or 2) Retire the R380 and replace it with something like the ATTO ThunderStream SC 3808D, which provides two SAS ports and operates at 6 Gb, still keeping the iStorage Pro, or 3) get an entirely new thunderbolt RAID + enclosure + disks, which would probably offer a performance boost (that I don't *really* need).
1) I confirmed with ATTO and mLogic that the R380 with the newest drivers (4.0.1) is supported in thunderbolt topologies. An mLogic mLink is fairly inexpensive at about $400, and provides a single PCI slot that seems like it should be fast enough to let the R380 top out. However, I’m slightly concerned about inserting an additional device between the host and the platters; should I expect a significant latency / throughput hit as a result? This option is probably the least flexible and has the shortest useful lifespan of the three, but is also the cheapest by far.
2) An ATTO ThunderStream SC 3808D is more than twice the cost of an mLink at about $1000, and would replace the R380 and operate at 6 Gb instead of 3 Gb. This also leaves the door open for future SAS expansion via daisy chaining. I have no problem leaning on something like a ThunderStream for as long as it’s viable to do so, provided it’s reliable and performant enough.
3) Get a new thunderbolt RAID enclosure including new drives. At first I thought this was vastly more expensive than a SAS-connected solution, but maybe it’s not… To compare: on the one hand we have $3600 for a 3 TB Pegasus2 R8, and on the other hand, $1000 (ThunderStream SC 3808D) + $1400 (another iStorage T8 SAS enclosure) + ~$1200 (8 x 3 GB drives) = $3600. As the price is roughly a wash, going with the newer, faster solution with fewer discrete components is a no-brainer. There is also the fact that my existing drives have all just crossed the 3 year mark - no problems yet, but… check out the data from Backblaze that shows a marked increase in HDD failure rate after 3 years. Yet another reason for me to get a new enclosure + drives: I don’t have enough capacity elsewhere to back up or offload the 7.2 TB of data I currently have (like I said, enthusiast ;), which is somewhat unsettling given that it’s RAID5 and not RAID6.
So, those are the options I’ve been considering. Having written it all out, I think I’m leaning towards biting the bullet on a Pegasus2 24 TB R8, however I’d greatly appreciate any comments / pot-shots / hazing from the crowd.
so exactly what are you asking ? You have a wonderful storage enclosure. You have a wonderful computer. You have a wonderful ATTO host adaptor card. All the products that you mention are all wonderful. You could buy an expansion chassis - would that work - yes. Remember, your chassis and drives are 3g, not 6g, so adding a 6g host adaptor will not increase the speed of your array. But you are happy with the array, as many people are.
Well, as I am so often criticized, I am only drawing one conclusion from your post, and I will attack you. But first, the professional response. Is buying anything that you suggested a good idea - of course it is. Will a new Promise Pegasus 8 bay Thunderbolt 2 product be a nice, fast product - of course it will. Will new technology continue to come out, even after you "bite the bullet", and obsolete whatever you purchase ? Of course it will. but look at what you have now - you have wonderful professional equipment, that works just fine. Do you really need to buy anything right now ? Certainly, your drives are probably getting old, and finding matching 3g drives will be a difficult chore. But all of your research is excellent, all of your equipment is excellent, and either way, you can't go wrong.
But now for the attack. Exactly what is the point of your post - you seem very knowledgeable, and very researched. I believe that you are asking (what so many people ask) - what is the LEAST EXPENSIVE WAY I CAN DO THIS so I am not constantly pouring money into my system, and I can have some hope of having some money in the bank, while having a top performing system. WELL FORGET IT. THE ELECTRONICS INDUSTRY IS OUT TO GET YOUR MONEY. They will figure out ways to empty your wallet, by obsoleting your perfectly good 3g drives, and 3g ATTO host adaptor card, get rid of all the current Thunderbolt 1 products (they are SO last week ago) - make any Thunderbolt 1 expansion chassis obsolete, and make your cry that your equipment is now "obsolete" (it's really not, but that's what they want you to believe). So you will say "maybe I should bite the bullet and buy the new Promise TBolt 2 drive array" - well, this is just the beginning of your spending. And do you not think that Tbolt3 will come out 2 years after you buy the TBolt 2 drive array ? Everything you own will be obsolete because THAT IS THE MASTER PLAN - to obsolete everything, and make you keep spending money. Just look at the G5 (which some people STILL use) - it was a great computer. But you can't find any PATA drives for it, you can't find any PCIx cards for it (maybe on ebay) and you can't find any modern software or OS to run on it.
SO - if you are nervous, then DO NOTHING, and keep using your current gear, because it sounds like it is all excellent (and move your ATTO R380 into the x16 slot, so you can get 600 MB/sec on the array). And just sit back, and wait it out, to see what new products come out - because let me assure you, EVERY DAMN DRIVE MANUFACTURER will have TBolt 2 drive arrays, and they will all compete for price, so buying day 1 is probably not a great idea.
Rescue 1, Inc.
Thanks for the reply. Comments / answers inline:
[Bob Zelin] "so exactly what are you asking? ... Do you really need to buy anything right now ?"
Somehow I forgot this important information. I will sheepishly add three soft constraints to the hard constraint of throughput minimum I already mentioned: 1) I do intend to acquire a new mac pro reasonably soon after they are available, 2) The new mac pro needs access to some bulk storage to be fully useful to me, and 3) I would prefer to not have multiple mac pro machines online, for heat / power / noise reasons (this stuff lives in my bedroom), and also because somebody is waiting for a hand-me-down of my current machine. 2 and 3 together two would seem to rule out re-purposing the existing machine as a file server, which would otherwise be technically feasible, but... I'm calling these 'soft' requirements for a reason - more on that later.
[Bob Zelin] "I believe that you are asking (what so many people ask) - what is the LEAST EXPENSIVE WAY I CAN DO THIS"
This is a perfectly reasonable assumption. However, in my case, overall cost is not a top-level constraint, at least not in this ballpark. What can I say... I work in high-tech and live frugally :) What *is* important to me is VALUE, but value still ranks behind my other constraints. As I seem to have identified multiple ways to satisfy the primary constraints, this thread is about maximizing value.
My notion of value includes consideration of the longevity of the solution. I don't want to be re-rolling stuff every 1 to 2 years, which also means that I tend to avoid adding new requirements once the system is built. My current setup would last another couple years at least were it not for my pesky and arguably capricious requirement of upgrading to a new mac pro :) In the long term, another platform-level requirement is the ability to run new versions of OS X as they become available, which I need for my profession. It's hard to estimate longevity of the new mac pro in this respect. I find myself wanting to make optimistic estimates (e.g. major architecture upheavals seem to have died down after the intel and 64 bit transitions), but I will stop short of doing that, because I do agree with your points about how technology industries work :)
[Bob Zelin] "EVERY DAMN DRIVE MANUFACTURER will have TBolt 2 drive arrays, and they will all compete for price, so buying day 1 is probably not a great idea."
This is a very strong argument for using a stop-gap solution of keeping my existing mac pro online as a file server, at least until there are more tbolt2 options. I have an inkling suspicion that Promise won't budge on their prices, and if so... bully for them. I'm not set on buying Promise, and I can certainly handle the rough edges of unwieldy administration tools and / or bad English that seem to be more common with other vendors, provided the product is performant and robust. Right now, cost and value of the Pegasus2 stuff seems acceptable to me, but I like the idea of waiting to see how things develop, given that I have a stop-gap option, even though it violates my third soft constraint.
Thanks again for your perspective and sanity check of my findings so far. I did move my R380 to slot #2 last night, and indeed it's running faster now!
Just a few comments:
1-The R380 as you know has been obsolete quite a while now,
can hardly get replacements and so on.......not sure if I
would depend on using the R380 anymore....ATTO I am sure
could care less of issues with it.
2-We have tested and have seen ~20% performance improvement even using 3Gb
drives/chassis/expanders if you switch to using a R680 which
is still supported by ATTO. We have replaced R380's
for R680's over the last 2 years for some of our customers
so they have better performance and have a product that
is supported fully by ATTO.
3-Besides an X8/X16 slot for the ATTO card, make sure
that the I/O size is 4MBytes instead of the default
of 32KBytes that ATTO defaults to. Easy to do when using
the 3.38 Config tool or with a NVRAM command.
... time passes ...
I'm now using a 2013 Mac Pro (6c / d700 / 32GB / 1 TB) and a pegasus2 r6 with 2TB drives. Not a cheap upgrade by any means, but it should last many years (goal is 6 or better).
Here's a short writeup on this new combo, including some benchmarks:
FCP X is just so nice with a dual GPU config:
I know none of this is news at this point, but I wanted to add some data points for anyone else who may be considering such a transition.