RAID necessity?
by Steve Brame
on
Aug 16, 2009 at 4:17:54 pm
I know that it is strongly recommended to utilize a RAID when working with HD footage, but is this only necessary for video capture? What about simply the I/O needs for editing? Isn't a SATA drive with a 95MB/sec read speed going to be able to handle that?
I ask because I have had 2 RAIDS fail before, causing a loss of everything I was working on. While I now have a solid daily backup routine, I still have a bad taste for RAIDs, and would rather not use unless absolutely necessary. I just began working on my first project that is primarily made up of HDV footage, and was able to capture via firewire w/o a problem - but playback is a little stuttery unless I render.
System specs
Asus P5E3 Deluxe
Core2 Duo 2.66GHz
4 GB DDR3 1333MHz (/3BG switch enabled)
Re: RAID necessity? by Vince Becquiot on Aug 16, 2009 at 7:41:06 pm
First on losing data.
That should not happen since you should only be storing captured files that you already have a backup of, and that rarely changes during a project, so really, you would only have to back up new renders after the initial backup. All other files such as project files and other important data should be somewhere else.
If you are working in HD, you are bound to work with uncompressed HD, be it from After Effects, or a 3D app, or even Chroma key renders.
That single drive won't cut it. Premiere also often accesses a lot of data at once, so a bigger pipeline will avoid slow downs or a frozen timeline during that access time.
Finally, I would recommend a standalone system, not something from a motherboard which is almost always software driven, CPU intensive and quite unreliable.
An outside RAID will allow a fast RAID 5 which makes losing data even less likely.
Re: RAID necessity? by Mark Hollis on Aug 17, 2009 at 2:12:24 pm
Vince is absolutely correct here and you should be looking for an external RAID array, controlled by a card in a slot in your computer.
It's all about throughput, really. You need to be able to play back HD files without dropping frames and today's 7200 RPM drives, as well as the speedier 10,000 and 15,000 RPM drives just cannot keep up with uncompressed HD.
You have been burned once and I know just what happened. You had a RAID stripe set to RAID 0, which is interleave stripe where every other block of data was placed on the other drive. This allows the two drives to (more) fully saturate the disk drive controller's pipe and that gets data into your RAM where your CPU can use it lightning fast. It's a good solution but, if one drive fails, you lose 100% of your data.
If you don't want to risk the data loss, there are other means by which you can stripe the drives to give you some data protection and RAID 5 (with enough disk drives) will do the trick. RAID 5 creates an array with a "checksum" drive that will rebuild the array if one drive fails. You just replace the bad drive and keep on editing. The array will do a rebuild.
You can only get this kind of array in an external chassis, as the maximum amount of drives you can put inside of any computer is four.
I would encourage a quick read of http://en.wikipedia.org/wiki/RAID'>WikiPedia's article on RAID arrays. You don't start getting any decent speed out of an array until you have 4 drives or more. RAID 5 can be made to deliver close to RAID 0 performance and Dave Altavilla did testing of a RAID 5 array that is capable of delivering speed that is appropriate for HD. His article is here and it is highly recommended reading as he specifies drives to be used in the array as well as the controller card he uses. What he doesn't offer you is a chassis. Also the drives he's all fired up about don't have industry standard connectors (as of June).
Investing in a RAID array is kind of a big deal. But you should be able to charge for that investment if you are editing HD and you're not just a hobbyist.
If you are a hobbyist, keep your tapes handy in case of drive failure, check your drives' status every time you initiate a new project (before you load in your material) and do a RAID 0 stripe. You simply will not get reliable uncompressed HD playback from a single drive. They just aren't built to give you the raw throughput.
Re: RAID necessity? by Steve Brame on Aug 17, 2009 at 2:52:11 pm
Yep, I'm quite familiar with RAID structure. I was just questioning whether the necessity of one in HD editing was specifically for the capture, or for the actual read during editing and render. We are soon to move to a solid state workflow, but we'd still receive HDV tapes from time to time from some of our contract videographers. I've had no problem with the tape capture to a single drive, but as I stated, playback(unrendered) stutters. I'm now pricing out some external RAID's. Gonna be pricey for our 6 bays.
Steve Brame
creative illusions Productions
VODSouth
Re: RAID necessity? by Mark Hollis on Aug 17, 2009 at 7:59:58 pm
If it's that pricey, look at the SAN solutions. That way one bay can edit promos for the same material that another bay is cutting a package using.
I can highly recommend Facilis Technology's Terrablock systems. They look like hard drives attached to your PC and not a SAN, so applications (like Avid's) that will only connect to really expensive and proprietary SANs won't know the difference.
Terrablock will support a facility with heterogenius editing systems, as long as they can all work with each other's codecs (and I recommend the use of Avid's free DNxHD and MXF for easy sharing between systems). And Facilis' systems will support HD. Last thing I did on HD was off a Terrablock array. Aired on ESPN.
