Storage of large production RED footage
Im not sure if this is the right forum for this but i have two questions regarding storage and backup of RED ONE footage.
Im currently in pre production of a major documentary. It will be shot all over the world during a year. We will use the RED ONE as our main camera and capture 4K 2:1 footage. We will also use a Canon 5D Mk II as a secondary camera (1080p h264). Editing will be done in final cut pro 7 on 8-Core Mac Pros (16 GB RAM).
As we are in production we will need some kind of portable RAID solution to store and backup the media. The production team needs to be flexible and mobile, so any big and heavy system is out of the question. The team will hold about 4 TB of footage at most. We have looked at both the G-Raid mini 2 and the Sonnet Fusion F2 drive. What kind of experience do you guys have with these drives? Are they reliable enough? And is there any other solution we have missed?
We calculate to have about 12 TB (100 hours) of footage when we move into post production. We need a RAID system that is capable of storing this media but we have no possibility of installing a rack-mounted solution. Here we are looking at purchasing two ProAVIO Editbox EB10 (10TB) to hold the R3D files and use smaller disks for render files etc. We will edit the project offline for as long as possible so we don’t need to store any additional ProRes files. Is this a reliable solution? Are there any great downsides we need to consider? What’s most interesting about the ProAVIO is the price and the fact that is fast and not that big.
I am thankful for any input!
There are a couple of key principles that you should follow.
First, create two copies of footage. While the two drive approach in the field is common, I'd suggest getting one copy to LTO for long term archiving as soon as possible.
For this much RED footage, you should really look at the REDRocket. I would suggest processing everything to ProRes Proxy for offline editing, but then that depends on how you're going to be doing the finishing work.
If you don't edit from the R3D files, then you don't need to have them on a fast RAID storage. You can handle just having them on plain external drives (as long as you have two copies of them).
Just a couple of things to consider.
Colorist/Digital Cinema Specialist
Salt Lake City, UT
I am not a Red expert, but this is what I did for a Red feature I was involved with recently. G-Tech on the camera during shooting for R3D files. In the field, a second copy was made on a second G-Tech FW800 drive. The "backup" G-Tech was brought to our post facility, where TWO seperate eSATA copies were made (Hitachi enterprise Saturn drives), from the G-Tech drive. These were made with Sonnet E4P host cards, and older Maxx Digital 5 bay port multipler chassis (but we did not use stripped drives - just single 1TB eSATA drives - and two seperate backups).
The media was then transcoded using Red Rushes to ProRes422HQ onto an 8TB Maxx EVO2 RAID array. This is how the movie was cut. The movie will be conformed at full res at CineWorks in Miami from one of
the original drives.
All I can say is drives fail - even top quality enterprise level drives. Make backups of your data. We did not use an LTO, but many people use LTO4A, and the Cache-A is the current product for this (but it's $7995).
Yup- I also recommend the immediate field backup to at least two sets of RAIDS. Here's a utility to simplify the process with way better reliability than dragging/dropping in the Finder:
Then either nightly or at least weekly back up everything to LTO. This is really critical because drives die and when they are restored it's not always perfect.
I also recommend having enough RED-DRIVEs or CF cards to provide at least a 24 hour delay between re-using media in the camera. This should give the data wranglers enough time to backup everything and make sure all takes are accounted for before a drive has to be re-used. Last thing you want is to be shooting on a drive or card that has footage that wasn't confirmed as transferred/backed up 100%.
Check out my book: RED: The Ultimate Guide to Using the Revolutionary Camera!. Unlock the secrets of the 24p, HD and Final Cut Studio.
Now featuring the Sony EX1 Guidebook, Panasonic HVX200, and Panasonic DVX100.
I would avoid using the dual drive portable RAIDs for field backup, unless using RAID 1 (mirrored). Other wise you are doubling your chances of drive failure. (If you loose one drive in a 2 drive RAID 0, you loose everything.) We have had very good luck using single FW800 drives in the field. Typically, using the drives from macsales.com, we'll make two copies (on separate drives) in the field before releasing the CF card to be reused. If timeliness is an issue, we'll fedex one drive back to the office in a pelican case, where it is copied to the working RAID and the footage is ingested or transcoded as needed. This puts a copy with the production unit, a copy on the shelf in the studio, and a copy on the editor's working drives. On our most recent Red job, the production unit was using small Lacie FW800 drives, the kind that get their power from the Firewire bus. These, or the G-Drive mini, or OWC On-the-go might be ideal for a doc production because you could run everything off a MacBook, including power, with minimal gear.
I love the idea of an LTO backup, but every production I've been associated with opted for redundancy in drives for a fraction of the cost, increased speed, and convenience. What's the new saying: fast, cheap, reliable... pick two. (It's really not that bad. I've had drives die on me, but never two drives of the same footage at the same time.) Also, you mentioned that the crew would have no more than 4TB of footage. I would definitely avoid keeping up to 4TB of footage and backups with the production crew. Ship a copy home as soon as a 500GB drive is full. Spread that responsibility out if possible. Think of it as cash... you don't want to carry that much cash, deposit it in the bank (via a very well packaged pelican case). (Or think of it as unexposed film...)
As far as your big RAID for post, a RAID tower like the Editbox will do you right. A port multiplier RAID works shockingly well and typically has plenty of throughput for ProRes and Red footage, which itself is shockingly efficient considering what it is. I believe Bob Zelin, in the past year, wrote an article for Cow about building one yourself... The magic is in the PCI card and the drives, not as much the box.
-------Stuck On On--------
Audio and Video Post Production