FORUMS: list search recent posts

RAID 50 loses half the expected speed?

COW Forums : RAID Set-Up

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
Blase Theodore
RAID 50 loses half the expected speed?
on Jan 21, 2014 at 9:31:26 pm

pertinent info first….

HARDWARE SETUP:
24 WD RE3 enterprise SATA 3Gb/s drives
ATTO R380 in x8 slot (3Gb SAS)
dual xtore 12bay JBOD expander units (3Gb SAS)
each xtore feeds a single SAS connection to the R380
OSX 10.8.5

RAID SETUP:
12 drives > xtore_unit1 > SAS1 > RAID5 > "RaidGroup1"
12 drives > xtore_unit2 > SAS2 > RAID5 > "RaidGroup2"
Interleave=512 / Sector size=4kb / Speedread=always / prefetch=0
Disk utility > "RaidGroup1" + "RaidGroup2"

USAGE:
HD Post production & online finishing


Hi everyone,

I have a 24 drive RAID50 thats giving me about 5-700MB/s R/W. which is about what I get from an 8 drive RAID5. The RAID50 is made from 2 RAID5 segments striped in disk utility. Both RAID5 segments give me 6-800Mb/s individually, so I was expecting that striping them together should get me around 12-1400Mb/s. But the result is actually slightly slower than the individual RAID5's themselves.

Am I missing something? Thoughts? I'd certainly appreciate the feedback.

Thanks,
Blase

(In case its relevant for testing, I also have a spare r380 card and a spare xtore 16 bay expander unit.)


Return to posts index

Alex Gerulaitis
Re: RAID 50 loses half the expected speed?
on Jan 21, 2014 at 11:44:25 pm

Any chance the R380 is in a 4x slot? I believe it's PCIe 1.0 and the 4x config could partially explain the 800MB/s speed cap... I think...

-- Alex Gerulaitis | Systems Engineer | DV411 - Los Angeles, CA


Return to posts index

Blase Theodore
Re: RAID 50 loses half the expected speed?
on Jan 21, 2014 at 11:48:11 pm
Last Edited By Blase Theodore on Jan 21, 2014 at 11:49:52 pm

Nope its in slot2 on a MacPro 5,1. Which is an x16 slot. And definitely an x8 connection, as verified by the connection speed in the ATTO util and the osx system info.


Return to posts index


Alex Gerulaitis
Re: RAID 50 loses half the expected speed?
on Jan 22, 2014 at 12:10:29 am

I've never tried anything similar on a R380 so all I can offer are wild guesses.

Is it possible to reformat the array in RAID0 across all drives and test speeds? (It'd be a PITA to initialize RAID5 again - yet this would be a good test to measure R380 max throughput.)

What does ATTO say about R380 performance ceiling in RAID5?

Also, R380 seems to support RAID50 natively. Any reason you decided on soft (OS) striping?


Return to posts index

Blase Theodore
Re: RAID 50 loses half the expected speed?
on Jan 22, 2014 at 12:26:48 am

Thanks, I should have mentioned that as well…
I did a test with RAID0's instead of RAID5's, but the results were the same.

The spec sheet shows native RAID50 support, but they don't actually support it.


Return to posts index

Alex Gerulaitis
Re: RAID 50 loses half the expected speed?
on Jan 22, 2014 at 3:16:25 am

It looks like R380 flatlines at 800MB/s. Puzzling though that RAID50 gives you worse performance vs. RAID5 with half the drives. Possibly it slows down as it works with more drives / RAID groups.

R680 then, or perhaps ARC-1882x.

-- Alex Gerulaitis | Systems Engineer | DV411 - Los Angeles, CA


Return to posts index


Jon Schilling
Re: RAID 50 loses half the expected speed?
on Jan 22, 2014 at 6:32:29 pm

Sounds like a issue to pose to ATTO. R380 = 3G speed. R680 = 6G speed. Alex is right, I'd try an R680 card, or Areca 1882X card.

Jonathan Schilling
Vertical Sales Manager
Proavio Storage by Enhance Technology Inc.
12221 Florence Ave.
Santa Fe Springs, CA 90670
Dir: 562-777-3498
Main: 562-777-3488 X106
Fax: 562-777-3499
Email: jon@proavio.com





Return to posts index

Blase Theodore
Re: RAID 50 loses half the expected speed?
on Jan 22, 2014 at 6:40:48 pm

To be clear, we're all on the same page of the specific problem I'm asking about?

OSX software striping fails.
700Mb/s + 700Mb/s (striped) = 500Mb/s.
It should equal 1200-1400.

I have 2 separate R380 cards. Each with its own RAID5, getting 700Mb/s.

I stripe them together through OSX and I get 500Mb/s. The ATTO cards are clearly working and doing their job correctly. The failure is in the last step.


Return to posts index

Alex Gerulaitis
Re: RAID 50 loses half the expected speed?
on Jan 22, 2014 at 7:13:29 pm

[Blase Theodore] "I have 2 separate R380 cards"

Not what you said initially:

[Blase Theodore] "each xtore feeds a single SAS connection to the R380
OSX 10.8.5

RAID SETUP:
12 drives > xtore_unit1 > SAS1 > RAID5 > "RaidGroup1"
12 drives > xtore_unit2 > SAS2 > RAID5 > "RaidGroup2""


The above implies and describes a single R380.

You also mentioned that RAID0 exhibits the same behavior, which points to the card as the culprit.

If you actually need help figuring out why striping two RAID5 groups across two R380 cards causes slower performance, perhaps you'd want to list your configuration more precisely?


Return to posts index


Blase Theodore
Re: RAID 50 loses half the expected speed?
on Jan 22, 2014 at 8:07:20 pm

Sorry Alex, I thought I had posted a follow up in which I repeated the setup, but split across 2 cards, and the result was the same.

I must have hit "preview post" but then never actually posted. I'll retype it below...

ALTERNATE TEST SETUP:

HARDWARE SETUP:
24 WD RE3 enterprise SATA 3Gb/s drives
dual ATTO R380 cards (3Gb SAS)
dual xtore 12bay JBOD expander units (3Gb SAS)
each xtore feeds a single SAS connection to a separate R380
OSX 10.8.5

RAID SETUP:
12 drives > xtore_unit1 > R380-1 > RAID5 > "RaidGroup1"
12 drives > xtore_unit2 > R380-2 > RAID5 > "RaidGroup2"
Interleave=512 / Sector size=4kb / Speedread=always / prefetch=0
Disk utility > "RaidGroup1" + "RaidGroup2"

In the alternate test setup each RAIDgroup is about 700Mb/s R/W. However once again, when striped together by disk utility, the resulting speed is only 5-600Mb/s. I would expect 12-1400Mb/s.



And to clarify the RAID5 vs RAID0..

I repeated both hardware test setups using RAID0's instead of RAID5's. The RAID0's performed exactly as the RAID5's did, and the overall results of all 4 tests were identical. Everything was always successful until the last step of striping them together.


Return to posts index

Bob Zelin
Re: RAID 50 loses half the expected speed?
on Jan 22, 2014 at 10:15:42 pm

let be me even more clear, so there is no mis-interpretation of what is going on here.

Blasé has TWO ATTO R380 cards. He has TWO AIC XStor 3gig chassis with 3gig drives. So this is not new equipment, that is capable of doing the usual 1500 MB/sec that we are used to seeing. He wants to COMBINE the two chassis in a RAID 50, for the SOLE PURPOSE of increasing his speed, so he does not have to go out and purchase new host cards, new 6g chassis, and new 6g drives. The drives on their own work perfectly fine. But when he merges the two together to create a single RAID 50, the speed DOES NOT INCREASE. This is his only problem.

You could make the same argument about this with all new equipment, (and an ATTO R680 or Areca 1882x, using TWO of these new cards, and two NEW 6g chassis with all 6g drives). The idea is - no matter how fast your single drive array is - if you merge both of them as a RAID 50 (or 60), shouldn't the speed INCREASE when it appears as a single volume ?

Bob Zelin

Bob Zelin
Rescue 1, Inc.
maxavid@cfl.rr.com


Return to posts index

Blase Theodore
Re: RAID 50 loses half the expected speed?
on Jan 22, 2014 at 10:22:11 pm

Yes exactly.


Return to posts index


Alex Gerulaitis
Re: RAID 50 loses half the expected speed?
on Jan 22, 2014 at 10:48:08 pm

[Bob Zelin] "if you merge both of them as a RAID 50 (or 60), shouldn't the speed INCREASE when it appears as a single volume ?"

The easiest tests would be:
- on a different host system to see if the problem is machine-specific
- to borrow an R680 or 1882x and perform similar tests

Until then, it's shooting in the dark.

P.S. Understanding the problem was never the issue, getting the relevant information was.

We did find out that 800MB/s is R380's performance ceiling, only to later find out there was a test done on two of them, with similar results - although we do not know if both cards were working in PCIe 8x mode.

We still don't know how that RAID0 test was done - a single stripe set all done in R380, two stripe sets then soft-striped in the OS (with one or two R380s). We still don't know the specs of the host machine. We don't know what other tests the OP did.

I've asked if the OP contacted ATTO about it, and heard no response.

We don't know if the system hits memory or CPU utilization ceilings during speed testing possibly pointing to configuration or performance problems.

The root cause of the slowdown is likely the OS - but we won't know unless we we have all the relevant info about the system, and until proper tests are done.


Return to posts index

Bob Zelin
Re: RAID 50 loses half the expected speed?
on Jan 24, 2014 at 1:00:01 pm

An R680 or an Areca 1882x test is pointless because -
1) these are 3g chassis with 3g drives
and
2) the client does not want to spend any money.

So the only question here is as follows -
no matter what host card you have, no matter what chassis you have, no matter what drives you have -
if they are all the same generation (3g, 6g, whatever) -

if you use TWO RAID cards on the same host computer (Mac Pro) connected to TWO RAID arrays (RAID0, RAID5, whatever), and stripe the two together (RAID 10, 50, 60) - SHOULD THE SPEED INCREASE ?
Yes, or no ?

And for the record, if you go with a single new ATTO R680 or Areca 1882x (with of course 6g chassis, and 6g drives), then yes - of course, the performance will be dramatically greater than any outcome of the R380 tests. Only one problem here - the client does NOT want to spend one penny on anything new. He just wants more performance with the equipment he currently owns.

Bob Zelin
Rescue 1, Inc.
maxavid@cfl.rr.com


Return to posts index

Alex Gerulaitis
Re: RAID 50 loses half the expected speed?
on Jan 24, 2014 at 6:53:49 pm
Last Edited By Alex Gerulaitis on Jan 27, 2014 at 6:40:01 am

Individual drives' link speeds (3G or 6G) don't matter. Drives don't saturate these link speeds.

Neither does the backplane: a 4-lane 3G SAS connection means 12Gbs in full duplex, i.e. roughly 1.2GB/s line speed per connection, and he has two of them: 2.4GB/s.

Testing this setup with a different (faster) controller will determine whether the bottleneck is in the controller or the host system.


Return to posts index


Chris Murphy
Re: RAID 50 loses half the expected speed?
on Jan 28, 2014 at 6:41:11 am

I agree with the earlier questions and I'll add some of my own:

What computer make/model/RAM?

What benchmarking tool?

Rerun the test on the raid50 volume, while running the Terminal command: 'top -s5 -n10 -o cpu' it will take 5 seconds for this to accumulate data, give it 10-15 seconds and take a screen shot (command-shift-4 then spacebar, hover-highlight the terminal window, mouse click)

Rerun the test on a single raid5 set (i.e. break the raid50, run the test on one of the raid5 arrays without software raid), while running the same Terminal command above and screen shot after 10-15 seconds for settling.

In Disk Utility when the raid0 set was created, what was the chunk size? I'm pretty sure the default is 32KB?

In Disk utility created the raid0 set with two hw raid5s, make sure the chunk size is a lot bigger or a lot smaller than whatever you used before. Dollars to donuts it was at the default of 32KB which is normally fine, but 32KB translates into a pile of IOs round robin between those two raid5s that's totally unnecessary and might just be amping the kernel doing a lot of work it doesn't need to do. So I'd try something almost obscene like 1MB if the chunk size goes that high and then redo your tests. I'm assuming your average file size is well above 1MB anyway for this raid50?


Return to posts index

Chris Murphy
Re: RAID 50 loses half the expected speed?
on Jan 28, 2014 at 6:44:34 am

Oh and for that matter I'd like to know what the chunk size is for the two R380 raid5 sets too.


Return to posts index

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
© 2017 CreativeCOW.net All Rights Reserved
[TOP]