FORUMS: list search recent posts

Best storage setup for video editing

COW Forums : RAID Set-Up

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
Tom Grant
Best storage setup for video editing
on May 4, 2013 at 7:03:30 pm

Dear all,

I am a novice with video editing but we are currently starting a weekly local tv broadcast and I'm interested in people's opinions on the best storage solution for such an endeavor.

Our current setup is a Mac Pro quad core with a 1TB internal drive with the OS and applications and a 2TB internal drive for media storage, projects, scratch, etc.

I would like to design the ideal system. I've spent some time researching the options and it seems depending on who's giving the advice, I see different answers. Some suggest raid 5, others say stay away from raid 5. Some suggest raid 3 or raid 10. Others say stay away from raid all together since I don't appear to be collecting massive quantities of data. If I do go with Raid, I'd love some suggestions on controllers, since obviously I'd use a hardware controller. I'd also like to know what is the best organizational scheme, i.e. OS on which drive, scratch on which drive, etc.

I should point out I currently have no backup system in place. So far we've only been using our collected footage for practice with editing so it hasn't been a priority. Now, however, it is. So we need an appropriate solution to backup raw media and final edited projects, and preferably do so automatically.

The broadcast requirements are 50Mbps and we'll collect about an hour of footage per week with a final edited length of about 30 minutes. Currently we're recording in HDV 1080i with a Sony s270u camera. I think HDV only utilizes 25 Mbps, so does that mean I need to change my recording format to DVCAM or something with higher bitrates, or can something be done on the back end in Compressor and allow me to stick with HDV? We usually get about 13 GB/hr, and I'd like to keep it as low as possible, but if I have to change the format I will. I know HDV is acceptable for most broadcasting in the US, but I'm not sure how they would use it if they require 50Mbps.

Obviously another important piece is budget. I think we have a decent budget for now, between $3k - $5k. Hopefully that will allow enough storage.

We would also like something scalable, where in the future it is easy to add disks for more storage.

Thank you all for your advice.

Tom


Return to posts index

Bob Zelin
Re: Best storage setup for video editing
on May 5, 2013 at 12:16:56 pm

I hate your vague question
When you buy a car, most will ask -should I get a camaro ot a mustang?
When you buy a guitar, most will ask -should I buy a strat or a les Paul?

So when you ask your vague question without stating any of your opinions or research, I have
Only one answer for you -buy any brand you se advertised on creative cow, because they are all
Excellent. And if you ask "what about brand x that I found on amazon that only costs $45 bucks?"
Well, then I will not answer any more of your questions

Bob Zelin



Return to posts index

Tom Grant
Re: Best storage setup for video editing
on May 6, 2013 at 1:04:47 am

Was my question really that bad to warrant such a tactless response? I tend to not use forums simply because I find that enough research reveals that other people have already answered the question. I've only posted on Creative Cow once about two years ago for another newbie question. I've perused Creative Cow hundreds of times finding useful answers to questions and my inability to answer my own question this time was simply a result of the lack of a consensus of opinion from various different people, including many forum answers not on Creative Cow, and not laziness or incompetence. I thought that maybe the different answers were partly due to the variety of user needs and each case is unique. So I thought I'd post my question on a forum where I know I'd get recommendations regarding my specific needs from experts far more knowledgeable about a topic that, I must say, is quite daunting for the uninitiated. But I will attempt to improve my question with more detail regarding the research I've performed, which I initially left out simply because my post was already quite lengthy, which this one now is.

While I'm not specifically asking for a brand recommendation, though if anyone has experience I'd appreciate the insight, what I'm more asking for is advice on my specific set up. As I mentioned, my research has resulted in many conflicting views based on individual user experience. I need something flexible for future expansion, but as turn-key as possible since I don't really know what I'm doing in terms of configuration. I need something reliable that I wont have to spend much time, if any, troubleshooting, something that I won't have to mess with more than replace a drive every 6 months or so, because I simply don't have the time. I've read up a lot on Drobo, Pegasus and Synology systems to see if any of these would work for me, as well as on various RAID configurations, but am still unsure which, if any, is best for my needs. To give some perspective, when I say I am unexperienced in all this, I mean that as of two months ago I didn't know what RAID was or that it even existed, so I've spent much time researching just to figure out what question I needed answered in the first place.

I was initially very interested in Drobo as advertised, since it seemed like it was exactly what I needed: easy to use, flexible, reliable. Unfortunately, what I found by reading hundreds of reviews online was that while many people said it was great, just as many others experienced not only drive failures during initial setup (which is not necessarily Drobo's fault), but failures of the Drobo unit itself, plus many people said it was too slow for video editing. The consensus seems to be that while when it works, it works great, it doesn't always work. Plus the idea of a proprietary system seems somewhat dangerous to me and many others.

Many of those disenchanted by Drobo have turned to Pegasus since it's based on conventional RAID and seems to be much speedier. Downsides seem to be a slightly more difficult setup procedure, less flexibility in terms of swapping disks, and capacity limitations (<12TB, however that should be enough for what I need), and I couldn't find a version with USB3, only thunderbolt, which is useless to me since I have a Mac Pro and can't add thunderbolt, but I can add USB3 with a PCIe card.

Synology may be more reliable, but it also seems to be the most difficult to set up out of the three. I would like to make sure the reliability of my storage isn't dependent on my limited knowledge of proper configuration.

In terms of less turn-key solutions, I'd be happy to set up a RAID 5 configuration, RAID 10, or RAID XX if others thought that I, a capable but ignorant consumer, would be able to accomplish such a feat. Do RAID set ups like this tend to be more or less reliable than something like Drobo? How about for a knowledgeable IT guy to set up vs. someone inexperienced such as myself?

I have read that many people prefer to have an online backup solution, so what are people's opinions on that considering the ~25 - 50 GBs/week of storage I'd require?

Thank you all for reading such a long post and for your thoughts.

Tom


Return to posts index


Bob Zelin
Re: Best storage setup for video editing
on May 6, 2013 at 1:40:59 am

I need something flexible for future expansion, but as turn-key as possible since I don't really know what I'm doing in terms of configuration. I need something reliable that I wont have to spend much time, if any, troubleshooting, something that I won't have to mess with more than replace a drive every 6 months or so, because I simply don't have the time.

REPLY - everything is flexible. A single drive can be removed inside your Mac Pro, and a second one can be put in. You can use an eSATA toaster from OWC, or CRU Dataport, to pop in and out individual drives. Drives are not reliable. They all fail. There are no more good brands. I use Enterprise series Hitachi Ultrastar drives, and they are maybe marginally better than any Western Digital Caviar series or Seagate Constellation, but they all stink, and they all fail, and they will all give you aggrivation. No one has the time do deal with failing drives, but everyone has to. As I mentioned in my first post, all brands you see advertised on Creative Cow are good brands.



I've read up a lot on Drobo, Pegasus and Synology systems to see if any of these would work for me, as well as on various RAID configurations, but am still unsure which, if any, is best for my needs.

REPLY - Of the three brands you have just mentioned, the Promise Pegasus will give you the best performance in a RAID enviornment. You will connect via Thunderbolt (only choice), which will give you between 600 - 800 MB/sec. The Drobo and entry level Synology are network attached storage devices that connect via ethernet. They will give you 1/6 to 1/8 the performance of the Promise Pegasus (about 90 - 100MB/sec), so if this is for a local Mac workstation, I see no reason to go thru this aggrivation, when you can just plug in a Pegasus and get teriffic performance. There are other brands that make Thunderbolt drive arrays (G-Tech, Lacie, Cal Digit, Maxx Digital, etc.) that will give you very good performance as well, with varying prices, depending on how much storage that you want. The Drobo makes a wonderful "nearline" or backup solution, but is not a wise choise for primary editing. The Synology (and QNAP) are more involved NAS systems, that can be configured for a shared storage enviornment, but it's not as easy as you think. From the three brands that you suggested, The Pegasus is your best choice. And remember, there are other brands as well.




I was initially very interested in Drobo as advertised, since it seemed like it was exactly what I needed: easy to use, flexible, reliable. Unfortunately, what I found by reading hundreds of reviews online was that while many people said it was great, just as many others experienced not only drive failures during initial setup (which is not necessarily Drobo's fault), but failures of the Drobo unit itself, plus many people said it was too slow for video editing. The consensus seems to be that while when it works, it works great, it doesn't always work. Plus the idea of a proprietary system seems somewhat dangerous to me and many others.

REPLY - you see drive failures during initial setup, because drives fail. You find bad drives often at the beginning of the setup process. When you buy a box that has no drives, and you go to Newegg or OWC or Frys to buy your drives, this is what you suffer thru. And when you buy Drobo, QNAP or Synology, who don't provide prebuilt systems, this is what you will possibly experience. This is exactly what the manufacturers that you see advertise on Creative Cow experience too, but they do the painful work, so you don't have to, and they charge more money because of this. As you know, they don't sell a Promise Pegasus empty - all the work is done for you, and the odds are that you will receive a working system. The same can be said for 100% of the brands that you see advertised on Creative Cow. These are all professional brands, and when you buy them for "the big bucks" (which are in your $3 - $5000 price range), you will receive a working product. Build it yourself, and you are on your own.



Many of those disenchanted by Drobo have turned to Pegasus since it's based on conventional RAID and seems to be much speedier. Downsides seem to be a slightly more difficult setup procedure, less flexibility in terms of swapping disks, and capacity limitations (<12TB, however that should be enough for what I need), and I couldn't find a version with USB3, only thunderbolt, which is useless to me since I have a Mac Pro and can't add thunderbolt, but I can add USB3 with a PCIe card.

REPLY - you can't swap drives on ANY RAID system. I have no idea of what nonsense Drobo is doing with their non conventional "raid" that you can randomly pop out a drive, but you can't do this with Promise, QNAP, Synology,or anyone else that you see advertising on Creative Cow that makes professional RAID systems. Modern RAID systems usually run at RAID5 or RAID6, and the only time you pop out a drive is when you have a drive failure. Run out of room ? Buy more storage. Can't afford more storage? Then BACKUP your RAID array with single eSATA drives and a toaster, or an LTO tape solution. That's how it's done. Forget Drobo, unless that is your backup solution.


Synology may be more reliable, but it also seems to be the most difficult to set up out of the three. I would like to make sure the reliability of my storage isn't dependent on my limited knowledge of proper configuration.

REPLY - this is a correct observation. If you don't need any sharing between multiple computers, buy the Pegasus, or any dedicated RAID solution that you see advertised on Creative Cow.


In terms of less turn-key solutions, I'd be happy to set up a RAID 5 configuration, RAID 10, or RAID XX if others thought that I, a capable but ignorant consumer, would be able to accomplish such a feat. Do RAID set ups like this tend to be more or less reliable than something like Drobo? How about for a knowledgeable IT guy to set up vs. someone inexperienced such as myself?

REPLY - even if you were an IT genius, you would buy a RELIABLE solution from a professional company that will give you lots of support. If you are a student, buy single SATA drives for 70 bucks and call it a day. If you are a professional, and make your living with this equipment, buy PROFESSIONAL equipment that has professional support. You already mentioned three brands that will all work. The Pegasus is within your budget, will arrive in working condition, where you can plug it in and go to work. And there are plenty of other brands that you can do this with too. when you build it yourself, you will be lost, and NO ONE will support you.
I believe that you already stated that you have betweeen 3 - 5 thousand dollars to work with. This is plenty of money to get a professional RAID5/RAID6 drive solution that will do exactly what you want to do.


I have read that many people prefer to have an online backup solution, so what are people's opinions on that considering the ~25 - 50 GBs/week of storage I'd require?

REPLY - having a cloud backup site for full res media is rediculous today, unless you are rich and have an 100MB/sec uplink. The only realistic thing you can do for cloud backup is with h.264 files, not full res media. The solution you need is cheap eSATA drives with an eSATA toaster from OWC for 70 bucks.

What else ?

Bob Zelin



Return to posts index

Rainer Wirth
Re: Best storage setup for video editing
on May 6, 2013 at 5:43:35 pm

Bob's right.
Now your question - Best storage setup for video editing?
First of all: Raid 5 or Raid 6 plus spare gives you enough redundancy, together with a good piece of hardware.
I would go for a expandable FC solution. Sonnet, Dulce, Small Tree, or Yotta3 together with a 4x4GB or 4x8GB FC card (Atto).
For back up we use an eSata Sonnet R400, you can also use a similar brand (Areca etc.) These 4bay raid systems have an internal controller and work with eSata, FW800 and USB.
As a big Raid you can also use SAS, which is very fast and reliable (the same brands). Best harddiscs to me are Hitachi Ultrastars (now Toshiba).

Cheers

Rainer

factstory
Rainer Wirth
phone_0049-177-2156086
Mac pro 8core
Adobe,FCP,Avid
several raid systems


Return to posts index

Jon Schilling
Re: Best storage setup for video editing
on May 6, 2013 at 6:00:24 pm

Rainer,

Actually the Ultrastar drives are now Western Digital, the Deskstar Drives are Toshiba. See my previous post on that.

http://forums.creativecow.net/readpost/292/2490

Jonathan Schilling
Vertical Sales Manager
Proavio Storage by Enhance Technology Inc.
12221 Florence Ave.
Santa Fe Springs, CA 90670
Dir: 562-777-3498
Main: 562-777-3488 X106
Fax: 562-777-3499
Email: jon@proavio.com





Return to posts index


Fred Jodry
Re: Best storage setup for video editing
on May 10, 2013 at 5:00:50 pm

Tom, here`s what works for me.
1. Pick a good box advertised here like, Active Storage (from the AJA event; I saw the Owner of the company demonstrate it) or Small tree. Two total winners amongst the sea of software, firmware, and hardware seaweed.
2. Hitch old Dobbin horse to the wagon. Stop complaining unless you sound really groovy.
3. Buy OWC (Other World Computing) SSD hard drives, and toss them in as you wallet goes down the road. It`s normally no reason to go RAID 0, 1, 0 + 1, et ce tera, because most hard drives and computing mistakes are primarily too much steam, a poor connection (usually a power cable), or a hard drive going below 50 degrees Fahrenheit or too hot. Keep it simple, stupid, and wieldly powerful.
4. Add backup, restore, storage to it like, tape drives running RAID 1 (not 0); or removable hard drives, or Pioneer DVD or BDR drives and a tested brand of disc.
Again: works for me. Fred Jodry


Return to posts index

Tom Grant
Re: Best storage setup for video editing
on May 10, 2013 at 7:20:22 pm

Thank you all for your responses, particularly Bob for your very thorough and extremely useful response.

While I would've gone with Pegasus, since it seems like it's exactly what we need and in our price range, it unfortunately only comes with Thunderbolt, and I have a Mac Pro which doesn't have Thunderbolt, so I can't use it. So I've decided to go with the G-Tech 16 TB G-Speed eS Pro with the ATTO SAS RAID Adapter in a RAID 5 configuration. I'll post here after using the product to give interested people my opinion.

Thanks,
Tom


Return to posts index

Dan Zinn
Re: Best storage setup for video editing
on Oct 24, 2013 at 5:39:49 am

Hi Tom,

Any updates on how the g-tech is working? I had the same issue with the Pegasus, so am planning on going with the G-Tech 12TB. Would be great to hear from someone who has used one.


Return to posts index


Tom Grant
Re: Best storage setup for video editing
on Oct 24, 2013 at 2:18:39 pm

It seems to work pretty well. No major issues so far. Right now we have it in a RAID 10 config (8 TB usable of 16 TB total). It's certainly speedy, which is great. I'm able to transfer ~100 GB of video in about 15 minutes, which, if I'm calculating correctly, works out to about 900 Mb/s, and I have yet to run into any lag, jitter or frame-drops when running four simultaneous 50 Mb/s video streams with a discrete audio channel in Final Cut X with some relatively basic graphics I created in Motion.

The only thing that's concerning is that when we first set it up, I turned off the G-tech before shutting down the computer. Even though I properly ejected it, like I would a typical external drive, when I turned off the enclosure, I received tons of errors and alerts that I had disconnected the enclosure improperly. It caused the RAID to fail and I had to rebuild it from a backup.

This is mildly concerning to me since the system is used by a group of people, and all of us are beginners, so I don't want someone to accidentally turn off the enclosure and cause the RAID to fail again. I'm a little amazed to be honest that something like this is not dealt with properly by the ATTO card, but that's probably just my ignorance about the subject.

Other than that problem though, the ATTO seems to work well. The set up is simple, even for a beginner like me, and rebuilding the RAID is straight forward. It took less than a day to rebuild the 5 TB (actually 10 TB including the mirror) when the RAID failed

HTH,
Tom


Return to posts index

Tom Grant
Re: Best storage setup for video editing
on Jul 15, 2013 at 5:32:15 pm

Am I? I don't mean to sound angry. My last reply as a thanks to Bob was very genuine. His was a very helpful response and I didn't mean it to sound sarcastic in any way.

My initial response to Bob I'll agree was a little curt, but not angry.


Return to posts index

Vincent Strader
Re: Best storage setup for video editing
on Dec 31, 2013 at 4:47:36 pm

Thanks to Tom Grant for this post. We're looking at the same options right now. I'll be sure to copy and send this to my manager.

And no, you did not sound angry in any of your posts.


Return to posts index


Tangier Clarke
Re: Best storage setup for video editing
on Jan 14, 2014 at 12:51:55 am

I see a lot of reviews about Pegasus, G-Tech, OWC and a handful of other RAIDs for thunderbolt, but never one that I've liked very much - Rocstor's Arcticroc.

Right now we have an Arcticroc 4T for our server but I was thinking of getting one of the Hybrid 7T models to go along with a new Mac Pro. I've been very happy with their product build, reliability, quiet fan (compared to some others) and customer service, but haven't used it in an NLE environment.

Have any of you used these this way or any of their similar products and or considered them?

Tangier


Return to posts index

Jeff Chamlee
Re: Best storage setup for video editing
on Dec 15, 2015 at 3:27:37 pm

Hello,
I'm fairly new to the Cow but have been reading through this thread and wanted to put myself out there to get Bob's (or others) take on a workflow we're considering.

We're a small post house that's taken on a larger project that will involve 3 Premiere Pro editors collaborating, and need to centralize our media in a shared storage environment. We're working with footage shot on the Sony F3 using the XDCam EX codec (35Mb/s), with a mix of AE Dynamic links, photos and stock footage on the timeline.

We're considering the Synology DiskStation DS1815+ 24TB (8 x 3TB) 8-Bay NAS Server, setup in a RAID 5,6, or 10, where we would all direct connect via 1Gbe. I know jumping up to the 10Gbe model would yield better performance, but for an entry level solution at our $2-4K budget, we're wondering if the performance and speed (~90-100MB/s) would be sufficient given the scope of our project size, codec and limited number of users?

For the sake of argument, we've got an IT consultant who deals with advanced networking to configure and set up the box for us.

I've seen a fair amount of chatter about the QNAP NAS boxes in production environments, but not as much about the Synology stuff. Any thoughts or alternatives suggestions are welcomed.

Thanks!
jc


Return to posts index

bryce arroyo
Re: Best storage setup for video editing
on Dec 15, 2015 at 6:14:41 pm

Tom,

How about this, lets get on a call with Bob. Bob is our tech guy here at Maxx Digital, we can talk over the phone and get you pricing and get you the right system that you need.

Bryce Arroyo
Maxx Digital
714-374-4944


Return to posts index

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
© 2017 CreativeCOW.net All Rights Reserved
[TOP]