Infortrend SAS RAID6 Running very slow
Hi everyone. I have an old infortrend A08S-C2132 8 bay SAS CUBE and it's running very slow. At the moment I'm getting about 135MB a second from it according to both the AJA and BM speed tests. I'm running it through a Thunderlink 1068 on a spec'd out late 2013 iMac but also have a 2008 MacPro with an LSI HBA in it and that gives similar results. The raid has got 8 Seagate 1TB enterprise drives in it, setup as RAID6. Bought in 2009. I'm having trouble getting any kind of support for it but do know that it used to do around 450MB/s write and almost 500Mb/s read when I first got it.
I'm wondering where I start to debug this thing? I've freed up space on it so there's almost 2TB free (6TB total as RAiD 6). If one drive in the set was running slow would that slow the whole thing down? Do RAIDs just get slower as they get older? Should I just dump the whole thing to LTO and totally rebuild, maybe pulling each drive, formatting them in OSX and testing them individually for speed? I was hoping to get a bit more time out of this thing if possible and was hoping to upgrade it to 2TB drives (as this is the max it can handle) and try and get another 2-3 years out of it. I haven't really done a speed test for while so it could've slowed down a little while ago. I noticed slower than usual copies last weekend and non real time playback when trying to play PRORES HQ UHD movies, which is why I investigated. Up til then I've mainly been doing 1-2 layer pro res HD stuff so haven't really noticed the slowness.
Any suggestions or thoughts are greatly appreciated. Thanks heaps.
Spinning rust drives get slower as they get full, assuming the file system allocates sectors from outside to inside of the platter, which is usually how it works. And the difference is pretty significant. Some companies include the min and max in the spec, most don't. I've seen the difference be upwards of 40%. What you're reporting is a lot more than that though, more like a 70% drop in performance.
So that makes me wonder if one or more drives is having problems writing data and it's having to retry a lot. I'd like to think that the RAID software has some testing options, maybe even a way to issue SMART commands to each drive, and get some idea what's going on. If not then it means breaking the array and doing both read and write tests to individual drives, and that is a destructive test.
Jason, I concur with Chris - you have one or more drives that are going bad and the RAID controller keeps on retrying whatever operation is failing (most likely a write). Now would be a great time to back this whole thing to your LTO tape drive. ASAP. It is your insurance policy. After that you can either upgrade the drives to 2TB ones, I recommend Hitachi Enterprise drives, or pull your Seagate drives out and test them one by one by connecting them directly to a PC and running the SeaTools Hard Drive diagnostics utility on each drive. You should be able to isolate the bad drive this way. You can then put the remaining good drives back in your array and it should come up in a degraded state. Get a replacement drive, rebuild the RAID Volume and you'll have the performance back.
Lucid Technology, Inc. / 801 West Bay Dr. Suite 465 / Largo, FL 33770
"Enterprise Data Storage for Everyone!"
Yes I think you're right. I'm going to run the backup this weekend. Haven't had time to get back to it yet. See if I can work out what's exactly going on. I've been looking at the 7k400's and they seem to have the best lifetime specs. It has taken me ages to get the software to be able to talk to the RAID. Infortrend have been super helpful via their online web chat. I'll let you know how I go.
Hmmm. After further testing I've found today that it seems to be more complex than I thought. On my OLD Mac Pro with a LSI SAS3801E card I get upwards of 400 MB/s.. On my iMac, no more than 135MB/s. On my iMac I'm using an ATTO ThunderLink SH 1068. I've put on the latest drivers and Firmware and am still getting those speeds.. So it looks like my RAID is fine and it's something to do with the Thunderlink. As far as I can tell they both have similar specs. The ATTO is meant to be up to 800MB/s throughput. Both the LSI and the ATTO have 2 x4 external SFF-8088 mini-SAS connectors.. So now I'm not sure where to start..
Any ideas anyone? Has anyone out there got an ATTO Thunderlink? I'm wondering if I've missed something basic..
Hi everyone. For those who are interested finally got this solved. It ended up going to ATTO as a bug and they took their sweet time solving it. Here's the fix.
After some extensive replication and testing of your configuration ATTO have confirmed that there are some settings which should increase the performance of the 1068.
It is advisable to limit the transfer size that the 1068 driver uses. By default, the 1068 driver transfer size is unlimited. This unlimited transfer size seems to not work well with the Infortrend interleave (stripe size). Therefore, by limiting the transfer size in our driver, you are lining up the transfer sizes with the stripe size. I found that limiting transfer size to 256k or 512k was best in my setup. I suggest you try values between 256k and 2M on your setup to see what works best. To limit the transfer size, you need to issue the following command in terminal:
sudo nvram atto-args-tlsashba="-i 0x00080000"
Then reboot. (NOTE: You must enter the command exactly as seen above).
The above example is for a 512k transfer size. The following values are what you should try:
256k = 0x00040000
512k = 0x00080000
1M = 0x00100000
2M = 0x00200000
You need to reboot the Mac after issuing the command each time. Therefore, issue the "sudo nvram" command above for one of the suggested values, reboot, then try and test. Then repeat for each value until you find the one that works best in your environment. 512 will usually give the best results for both read and write and 1M gives the best results for reads.
This solved my problems instantly and my RAID now runs as expected!! Took almost 4 months for the fix which was a simple terminal command! Hopefully they add these options to the driver at some stage so you don't need to use terminal to change them.