FORUMS: list search recent posts

I could use opinions about my RAID set up plan - Thanks!

COW Forums : RAID Set-Up

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
Alex Vargas
I could use opinions about my RAID set up plan - Thanks!
on Apr 2, 2013 at 6:23:05 am

What I need
This is my first post and I am looking for opinions from anybody with experience/knowledge choosing the needed parts listed below...before I go crazy with my plastic card.

What I have

  • MacPro 12 core, 9.93Ghz with 48 GB RAM
  • 512GB RAM SSD (Apple stock) - boot drive
  • 1TB - audio drive 7200
  • 3TB - Scratch disk 7200
  • 4TB - back up drive. Hitachi sata III 7200

Where I am now
I am currently finishing production on a feature film and soon to start post. This movie is shot on XDCAM EX1, 1080p HD 30fps and edited on a Prores timeline (not LT or HQ).
For post, I am looking at setting up my Mac for fastest results. Post will include editing the movie including considerable amounts of compositing, lots of voiceovers and scoring.
I edit with FCS7 and Logic 9, both with a bunch of third party plugs and (therefore) still at 32bit on Snow Leopard.
My main focus with setting up an internal RAID is to make my work less tedious throughout the next 10 months. But after this movie I will be shooting 2.5K RAW and want to be able to use this set up, with possible upgrades changing or adding drives only.
I have not taken any clients in six months and survived through production by selling most of my sound studio gear, so don't think I am rich, but I will buy what I need to simplify my next 10 months within the limitations FC7 has for me. (After that I'll move to Premiere or FCPX...)

What I want to set up

While all my research came to this, I have never set up a RAID before and could use a professional's opinion. I plan to configure a RAID 5 plus one hot spare drive.

Is what I'm planning a good solution for best performance?
Are there any better alternatives?
Will all this set up actually better my current Mac performance considerably?


All opinons and/or suggestions are very, very appreciated...

Alex Vargas
filmmaker/composer
Macpro 5,1 12 core 48GB RAM - Macbookpro 15" - FCP 7+/Logic 9


Return to posts index

Rainer Wirth
Re: I could use opinions about my RAID set up plan - Thanks!
on Apr 2, 2013 at 1:48:36 pm

Hi Alex,

why do you want to go for an internal raid? Internal raids create problems after let's say 3 years of intensive cutting. I worked for a production company which had internal raids on 10 workstations. After nearly exact 3 years one after the other went bad. I think that the cooling within the mac is not good enough for a raid.
I would go for an external 4x8gb FC raid 16/24 bay Raid 6+ spare and Hitachi Ultrastars as HDs. The Atto FC 4x8gb card is a burner. You end up with speeds over 1200MB/s. When you start editing 2K and above plus compositing you need a reliable system like this. It is definitely the most expensive solution but also the most reliable and fastest solution. These systems run for years without a flicker. The rest of your equipment is just fine. Personally I wouldn't go for FCPX, Premiere or FCP7 are okay. We have switched to Avid MC6 from FCP7 and I have to admit Avid works fine for us.
The cheaper option would be SAS (8 or 16bay enclosure) Raid 5/6+spare and enterprise class drives. The speed is half of the FC raid. I think that you might get slight problems with a workflow above 2K.

cheers

Rainer

factstory
Rainer Wirth
phone_0049-177-2156086
Mac pro 8core
Adobe,FCP,Avid
several raid systems


Return to posts index

Alex Vargas
Re: I could use opinions about my RAID set up plan - Thanks!
on Apr 2, 2013 at 3:34:21 pm

Thanks Rainer,

You response helps me consider the cost much better and clarify my "wants". I am also happy to hear FCP7 is still preferred by some above X. (I will learn X if I have to, but will probably go with Premiere too).

The main reasons for wanting the internal RAID are:

- The ability to use full SATA III capability from each one of my internal drives.
- Space saving (my "studio" and home is UNUSUALLY small until done with this movie).
- Being able to finish this movie without bottlenecks that I have had on my previous Octocore.
- Cost
- Stability

I also want the possibility for this set up to be good for editing RAW next year, but seeing your response, I must accept that I can't afford it at the moment. I looked into the card you suggest, but it would cost more than my 18 month old mac...the card (external box) alone.
I apologize for my initial post sounding as if I were ready to buy a $15,000 set up. I am not. I will correct it (if I can do that).

This is the first time that I read about internal RAID set ups all failing after around three years of life, so thank you for sharing that. So far all I have read about is what they are good for. Real experiences are valuable. Thanks!

My first conclusion is that I should forget editing 2.5K RAW on this set up and focus instead on my upcoming edit (XDCAM EX HD 1080p 30 fps on a Prores timeline in FCP7).

Will the system I listed be able to handle it?
Do you or anybody else foresee any compatibility issues?
(I am particularly curious about the Accelsior E2 card as a boot drive with FCP7)
Any suggested alternatives for similar speeds and reliability in this planned RAID?

Thanks again!

Alex Vargas
filmmaker/composer
Macpro 5,1 12 core 48GB RAM - Macbookpro 15" - FCP 7+/Logic 9


Return to posts index


Alex Gerulaitis
Re: I could use opinions about my RAID set up plan - Thanks!
on Apr 2, 2013 at 5:51:31 pm

[Alex Vargas] "4 x 2TB SATA III enterprise drives, plus two more of the same in my optical drive's empty space."

Am I hearing you right that you plan to have six of those drives in RAID5 plus hot spare, on R608? (Equivalent to five drives in RAID5.)

I think it'll give you speeds of around 400MBs (when empty) going down to around 200MB/s at 90% full. This is enough to handle BMD 2.5K raw (about 150MB/s at 30fps if I am not mistaken). What I would change is set the drives up in RAID6 instead - which should increase the speeds a little bit. If my hunch is right, and RAID6 is slightly faster, then you also get the benefit of better redundancy.

HTH.

Alex Gerulaitis
Systems Engineer
DV411 - Los Angeles, CA


Return to posts index

Alex Vargas
Re: I could use opinions about my RAID set up plan - Thanks!
on Apr 2, 2013 at 6:35:01 pm

Thanks Alex Gerulaitis!

If your calculations are correct, you are saying that I may be able to edit 2.5K RAW with this set up.
That sounds good. It means I am all good for editing XDCAM EX HD 1080p... :)

In your opinion, will this set up improve my current SATA II limitations considerably? As obvious as it seems to be, I am concerned that it may not improve much due to any compatibility factors that I may be ignorant about.

My boot drive would be the Accelsior E2 PCIe card (please refer to links on the original post) at much higher speeds than SATA II allows, as would be all other 6 drives through the ATTO R608 for my media (scratch), so all data flow would be above SATA II limitations, but will all this work out to be faster and stable with FCP7? Or are there any known incompatibilities?

Your time is appreciated!

Alex Vargas
filmmaker/composer
Macpro 5,1 12 core 48GB RAM - Macbookpro 15" - FCP 7+/Logic 9


Return to posts index

Alex Gerulaitis
Re: I could use opinions about my RAID set up plan - Thanks!
on Apr 2, 2013 at 6:47:36 pm

[Alex Vargas] "My boot drive would be the Accelsior E2 PCIe card (please refer to links on the original post) at much higher speeds than SATA II allows"

The SATA II limitation only applies to drives connected directly to Mac Pro SATA ports. None of your drives will be. In addition, even if they were, the limitation is only for individual links, not aggregated for multiple SATA connections. Bottom line, nothing to worry about here. :)

[Alex Vargas] "will all this work out to be faster and stable with FCP7? Or are there any known incompatibilities? "

I don't know whether and how does FCP7 handle raw BMD CNG files... Does it do it well? Other than that, and purely on the storage side, I can't think of any potential problems except heat and serviceability.

Heat: those are fast drives and having six of them in a chassis not designed to handle that many drives may cause heat build up.

Serviceability: having them all internally means (1) not being able to see individual activity lights, (2) needing to open the tower to access the drives. What it means is that it may be a bear to determine which drive failed and needs replacement if it happened to fail. With external storage towers, it's easy. This is why I always try to find the budget (and space) for an external storage box even if the budget (and space) is tight.

Alex Gerulaitis
Systems Engineer
DV411 - Los Angeles, CA


Return to posts index


Rainer Wirth
Re: I could use opinions about my RAID set up plan - Thanks!
on Apr 2, 2013 at 10:06:04 pm

Alex's advice is always of great value. But I doubt that you get enough speed to handle 2k material properly. Especially when you have lots of compositing effects around it. We work mainly with ProRes422 and our SAS raid is sometimes too slow to handle multiple streams together with compositing. So real 200 MB/s is enough to handle one track but I doubt that it will handle multiple tracks with 2K in an appropriate way. I'm also concerned about the heat side inside your computer.

cheers

Rainer

factstory
Rainer Wirth
phone_0049-177-2156086
Mac pro 8core
Adobe,FCP,Avid
several raid systems


Return to posts index

Alex Vargas
Re: I could use opinions about my RAID set up plan - Thanks!
on Apr 2, 2013 at 11:04:14 pm

Thanks Rainer!

I am certainly listening to all opinions. I have been editing Prores 422 on an MacPro Octocore 3.0GHz 2,1 without a RAID, compositing, multiple tracks when needed...and it has worked fine, but slow. I will be doing the same for the next 10 months on a 12 core 9.93GHz MacPro with the RAID set up I mentioned. I apologize for not clarifying that editing RAW would be for my next project 15 months in the future.

Right now, it is XDCAM EX HD 1080p 30 fps on a Prores timeline. I am also considering an external RAID, but I am 100% clear that I cannot afford the number of drives that you mentioned would be ideal for smooth processing of RAW 2.5K material, so I will focus on getting what I need to make my current project work and wait until next year to see what money I make then before I think 16 - 24 bay external RAID.

And yes, the heat issue...I am on to it. Either a small external enclosure (though I don't know where to put it...and I mean it) or a very good way to keep it all cool internally.

Thank you for taking the time to share your experience Rainer!

:)

Alex Vargas
filmmaker/composer
Macpro 5,1 12 core 48GB RAM - Macbookpro 15" - FCP 7+/Logic 9


Return to posts index

Alex Vargas
Re: I could use opinions about my RAID set up plan - Thanks!
on Apr 2, 2013 at 10:48:35 pm

Thanks again Alex Gerulaitis!

Yes, what I meant about the Accelsior is that it will be faster than my current boot drive. So I'm good with that.

And yes, it will run hot and blind. But it will set an alarm of course if there is something going down. I am good with that. If and when it happens, it will happen, be it with lights or alarm. Once there is a hard drive down I will be informed as it begins the re-build on the hot spare. As to figuring out which one is the failing drive, I have to go with ATTO's monitoring software which seems to be pretty straight forward. I just called them and they went through the application to clarify the basics, even though I have not yet bought the card. That was cool. And opening the cover to replace a drive...not a problem. Hopefully it won't be often. :) Keeping it cool however will be essential. I'll figure something out for that.

The software side is OK. The RAW files are processed in Resolve, Prores in FC7 and back to Resolve for final color. I was more worried about compatibility between FC and the RAID...ignorant as I am...

Thank you for your thorough response Alex. My main issue, editing my upcoming feature, seems to be resolved with this set up.
I hope nobody pitches in to mention a known lack of compatibility! (Or actually, I hope they do if they know of it...)

I will consider the possibility of an external array, though I can't see the space "opening"...I will reconsider all possibilities before I buy.

Much appreciated! :)

Alex Vargas
filmmaker/composer
Macpro 5,1 12 core 48GB RAM - Macbookpro 15" - FCP 7+/Logic 9


Return to posts index


Alex Vargas
Re: I could use opinions about my RAID set up plan - Thanks!
on Apr 5, 2013 at 6:04:56 am

OK, all enclosure considerations made.

iT8 6G SAS Expander is the winner.

If all goes well, I'll get another (or two) next year. By then, space won't be an issue. This solution allows for proper cooling, visual monitoring AND expansion for a reasonable price while at the same time using 6Gb/s technology and its SATA III lovers.

Now I only need to figure which ATTO RAID controller is the best option these days to connect with this Enclosure. (One call to ATTO should do).

I am pretty happy with my findings. And yes, after reading a lot of posts, I realize what an excellent tool this site is. I won't make the same mistakes during set up. I feel the pain... damn.

Thank you both Alex and Rainer...and to the rest of you who read this post and chose to let me make my own mistakes...or bleeding read all other related posts! HA!

Alex Vargas
filmmaker/composer
Macpro 5,1 12 core 48GB RAM - Macbookpro 15" - FCP 7+/Logic 9


Return to posts index

Alex Gerulaitis
Re: I could use opinions about my RAID set up plan - Thanks!
on Apr 5, 2013 at 7:15:22 am

[Alex Vargas] "iT8 6G SAS Expander is the winner."

Just curious: why not CR-T08E?

[Alex Vargas] "Now I only need to figure which ATTO RAID controller is the best option these days to connect with this Enclosure."

R680. Or Areca 1882x.


Return to posts index

Alex Vargas
Re: I could use opinions about my RAID set up plan - Thanks!
on Apr 5, 2013 at 8:05:04 am

Definitely that ATTO R680!
Agreed...I have read a lot of threads about it...
I am just curious as if a 16 port card would work better for a 16 bay box or two 8 bay boxes...
About speed right...? So I have that question for ATTO tomorrow... :)

Unless you can tell me that I get a better box with the CR-T08E, so far cost and the fact that I was able to see the T8 6G SAS Expander quite available online (in different forms, but basically the same box) made me choose it. If there is better performance in the CR-T08E I would like to know that...but at what cost...and so on...

I am already overextending...you know...

Alex Vargas
filmmaker/composer
Macpro 5,1 12 core 48GB RAM - Macbookpro 15" - FCP 7+/Logic 9


Return to posts index


Eric Hansen
Re: I could use opinions about my RAID set up plan - Thanks!
on May 1, 2013 at 9:39:35 pm

hey Alex

glad to see you went with the iT8 instead of the internal RAID. WAY easier and the hard drives will last longer. I suggest RAID6 and no hot spares. ATTO's 16 channel cards don't have external ports. the R680 is the best for your use.

going way back to the beginning of the thread and your question about the Accelsior card with FCP7. it WILL NOT make a difference. here's why:

When i first started installing SSDs in edit systems, the first thing I checked was performance in FCP7. In starting the app, the SSDs were only slightly faster than a regular spinning boot hard drive. Even opening large project files didn't make a difference. FCP7 is just slow and a faster boot drive didn't really help. Once you're in FCP7 and editing, the boot drive makes no difference in how FCP7 operates. It's all about media storage speed. Yes the Accelsior is faster than an SSD on a 3G SATA connection, but it won't make FCP7 any faster. Most other apps will be faster. Your computer will most likely boot faster. But FCP7 itself, nope.

the only time i would consider an Accelsior is if I'm working in Photoshop on really large projects where each Save is a few GB. But, you now have a RAID which will go 600MB/s if you put 8 drives in it. as an editor, i think your money is better spent on the RAID than on the Accelsior. you already have the stock Apple SSD as your boot.

btw, XDCAM EX is only 35Mb/s. You don't have to get a super screaming fast RAID to edit with that.

e

Eric Hansen
Production Workflow Designer / Consultant / Colorist / DIT
http://www.erichansen.tv


Return to posts index

Alex Vargas
Re: I could use opinions about my RAID set up plan - Thanks!
on May 2, 2013 at 12:33:41 am

Thanks Eric,

( Skal jeg sige Tak?)
Anyway, I digress... :)
I went with a 16 bay from iStorage, the ATTO R680 and 1 TB Toshiba 6G drives. This should push my EX footage more than I need, but it will also allow for 2.5K and 4K editing in my next project.

I am happy to know that the Accelsior is not a must. In that regard, I added a bootable card to run four internal drives at 6G, though for the time being, anything I am transferring does not need it. I still couldn't help myself turning my 12 core into 6G capable. :)

Why do you suggest RAID 6 with no hot spares over RAID 5 with spares? And I mean that considering that I now have a 16 bay enclosure...
I am thinking RAID 5 with two spares for this. Do you still suggest RAID 6 with not hot spares?

Thanks!"

Alex :)

Alex Vargas
filmmaker/composer
Macpro 5,1 12 core 48GB RAM - Macbookpro 15" - FCP 7+/Logic 9


Return to posts index

Eric Hansen
Re: I could use opinions about my RAID set up plan - Thanks!
on May 2, 2013 at 1:01:35 am

ooh, Dansk. You're welcome. to be honest, I don't know my lineage that well. I know my family is from Scandinavia, and that's about it.

i know what you mean about wanting 6G in the Mac Pro. curious, are you using a 4 drive RAID for boot, or are you saying you just made your 4 internal bays 6G-capable? an SSD makes a huge difference as a boot drive, but the difference between 3G and 6G is less of a jump than spinning 3G hard drive to 3G SSD. so all my systems have 3G SSDs (or 6G SSDs on the 3G bus). and most of them have all the PCI slots full anyway.

definitely still recommend RAID6 over RAID5 with hot spare:

years ago, all the RAIDs i installed were set up as RAID5. Since there was always someone around, I figured if a drive failed, it would be swapped right away, which means RAID6 was just a waste of a drive.

but

3 years ago I had a drive fail in an 8 drive RAID5. during the rebuild, ANOTHER drive failed. i almost had a heart attack. this was footage that hadn't been backed up yet. luckily ATTO support was able to help me recover the RAID and rebuild without losing any data. Their support is second to none. That's one of the risks. When a drive fails and the hot spare takes over, it still needs to rebuild and your RAID is vulnerable during that time. It also runs slower (or you can use it at full speed and the rebuild takes a lot longer). my take is that if you're putting the drive in there as a hot spare, you might as well put it to good use and make it part of the RAID. now all my RAIDs are 6. when a client tells me they want RAID5 to get more space, i tell them that story and ask them if they intend to fully back up their entire RAID all the time. since that answer is usually NO (a whole other topic in and of itself), they agree on RAID6.

plus with RAID6, you're spreading out the risk. You're assuming that your hot spares will work when called upon. what if they don't?

oh, and I always setup my 16 drive RAIDs as single RAID6 volumes. some controllers will run faster as two RAID5s or one RAID50, but 16 drive RAID6 is what I usually do.

e

Eric Hansen
Production Workflow Designer / Consultant / Colorist / DIT
http://www.erichansen.tv


Return to posts index


Alex Vargas
Re: I could use opinions about my RAID set up plan - Thanks!
on May 2, 2013 at 1:25:37 am

Thanks Eric!

Thats great info. :)
For my current 1TB 16 drives, RAID 6 will be good. (your experience is respected) When I go to 4K I'll have to add 4TB drives instead, but that is a talk for next year...as will be backing up all that data...

Alex :)

Alex Vargas
filmmaker/composer
Macpro 5,1 12 core 48GB RAM - Macbookpro 15" - FCP 7+/Logic 9


Return to posts index

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
© 2017 CreativeCOW.net All Rights Reserved
[TOP]