Akitio vs. Promise
Just got set up with the latest top spec 27" Imac, stuffed it with ram, and about to take on an extended hiatus documentary project that has been brought back to life by exciting new developments. Apparently dreams do come true after all.
The project is spread out over 12+ one tb disks, plus some other stuff in random external drives. Most of the footie was converted to Prores so it would run on my 2008 hackintosh before it exploded, but there was funky stuff going with the transcoding and my last rough cut was terrible, so starting fresh is probably a good idea. I intend to re-import only the masters and limit transcoding to what is necessary using the 10.3 organizational workflow.
Pegusus3 R6 is $2300 on b&h right now and comes with an extra 4tb backup thingie = 20gb raid5
Akitio Thunder3 quad is around $1200 when stuffed with 6tb drives= 18gb raid5
Pegasus will probably less prone to failure, but that difference is lens money. Anything I'm missing before I make the decision?
1. This project is so old that I started out with miniDV, then 720, then 1080 in several different flavors from gopro to avchd to xdcam. There's definitely new material coming in, including possible 4k b-roll, which explains the urgency of "oh crap, I need to figure out where to put this stuff!" as well as do a rescue of the aging drives and cross my fingers that I made enough backups. Speaking of which how to you back up the new Library files? They could get pretty huge to the point where a single one could fill up several disks.
2. 8 years ago I was forced to transcode everything just to get it all to play together, but sounds like it's a new era where a lot of that stuff will exist on the same timeline (or project or whatever they call it now) at least until it's time to color. Maybe so much storage isn't necessary but I'd rather be on the safe side, especially since some potential new clients are sniffing around. Is this overkill?
Your wisdom is appreciated!
I'd go for the Akitio, because cheaper. Keep in mind that you'd have to buy SoftRAID if you want anything other than RAID 0 or 1. If you buy a Pegasus I'd look for alternative configs. I think I've seen R4s with 4x 10TB for the same price or cheaper as an R6 with 6x3.
Regarding organisation and backup: Keep the media files external, do not copy everything into the library bundle - Use the option to leave files in place on import. That way the library files stay very small, a few MB, and are easy to back up.
I'd probably split the media into multiple folders which are filled up to 3-6TB (whatever size your backup HDDs are). Fill up media folder1 --> USB-Backup1, then media folder 2 to another backup drive and so on. Then a projects-folder which is backed up hourly or so.
This way you'd only have to backup the majority of the footage once after import and after that only the newest media folder and project folder.
I hope that makes sense.
[Jacob Holcomb] "latest top spec 27" Imac...documentary project....is spread out over 12+ one tb disks, plus some other stuff in random external drives. Most of the footie was converted to Prores so it would run on my 2008 hackintosh before it exploded...I intend to re-import only the masters and limit transcoding to what is necessary using the 10.3 organizational workflow....Pegusus3 R6 is $2300 on b&h right now and comes with an extra 4tb backup thingie = 20gb raid5...
I'm a documentary 1st AE so I deal with lots of technical, media and workflow issues. Your new iMac is vastly faster than your old Hackintosh, especially on H264 since it has Quick Sync. Promise arrays are great (I have an R4) but I have switched to OWC Thunderbay 4 arrays since SoftRAID is just as fast and it avoids locking you into a proprietary hardware RAID format. E.g, if any of my Thunderbay chassis fail I can put those drives in any other JBOD chassis and access the data. If a Promise chassis fails those drives can only be accessed in another Pegasus. However that is an uncommon scenario. OTOH if you need 20TB you'd have to get a 32TB Thunderbay 4 to provide 24TB of RAID-5 capacity.
Yet another viewpoint is just ditch RAID-5 and use RAID-0 which gives more capacity and speed. Normally you'd never do this on a four drive array but in a sense your media itself doesn't need redundancy since you obviously already have it somewhere -- that itself is your backup. You mainly need to backup your library and edits. I'm not saying do this but it's another possibility. But if your current media is on aging and questionable drives, those need backing up -- even IF you copy it to RAID-5. So RAID-5 does not eliminate the need for backup.
But there is nothing wrong with the R6 -- it is a great array -- very quiet and good performance. It is a known and trusted item and lots of productions use those.
[Jacob Holcomb] ...started out with miniDV, then 720, then 1080 in several different flavors from gopro to avchd to xdcam. There's definitely new material coming in, including possible 4k b-roll, which explains the urgency of "oh crap, I need to figure out where to put this stuff!" as well as do a rescue of the aging drives and cross my fingers that I made enough backups....how to you back up the new Library files? They could get pretty huge to the point where a single one could fill up several disks....
In general you want to import your media with "leave files in place", and you don't normally need to generate optimized or proxy media. This makes the import much faster and avoids taking extra disk space and time for transcodes. FCPX is fast enough to smoothly edit most camera native content. But there are a few exceptions.
If you have AVCHD content be absolutely certain you do not import those with "leave files in place". FCPX does not handle that well and it can cause I/O performance problems. If you have the original folder trees FCPX will not allow import in place but will copy those to the library. If you only have the .MTS files you can import in place but never do this. The best solution for AVCHD (whether trees or files) is rewrap those using the EdityReady utility, then import them with "leave files in place". This is the fastest approach and takes the least space: https://www.divergentmedia.com/editready
Even your top-spec 2017 iMac 27 may not be fast enough to smoothly edit or scrub 4k H264, depending on the exact codec variant. This may require transcoding to proxy. You do this after import, so you can evaluate the performance beforehand. By default proxies go into your library so greatly swells the size. There is a procedure to keep the proxies outside that's documented in Ripple Training's 10.3 Media Management class, but IMO it's somewhat cumbersome: http://www.rippletraining.com/products/final-cut-pro/media-management-in-fi...
[Jacob Holcomb] ...8 years ago I was forced to transcode everything just to get it all to play together, but sounds like it's a new era where a lot of that stuff will exist on the same timeline (or project or whatever they call it now) at least until it's time to color....
FCPX is a great tool for documentary since it allows rapidly skimming through vast amounts of content and rating or keywording this. In general FCPX can handle a wide variety of media. However with such diverse media you will probably encounter a few bumps and will need to figure out a few things.
For interlaced DV content it's often best to import this into a progressive timeline and manually deinterlace that. Select the clip in the timeline, in the Inspector at the bottom right change from Basic to Settings, then above that select the deinterlace checkbox. For an interlaced project this doesn't do anything since by definition it must remain interlaced. For a progressive project FCPX does not automatically deinterlace, so this can help.
When dealing with mixed progressive and interlaced content, always do test exports and evaluate this with a tool like VLC which allows selective control over deinterlaced playback. Mistakes on interlaced content are common. Just yesterday Fox News posted this piece which has "baked in" interlacing artifacts. This is especially ironic since Fox, ABC and ESPN record and broadcast in 720p/60: #sp=show-clips'>http://video.foxnews.com/v/5487990681001/?#sp=show-clips
Thanks guys. Both great answers and right in time, too. Got the green light and have been pulling in new footage for the past several days without a set workflow, but the really exciting stuff is coming up so it's not too late.
The EditReady looks like a key component for keeping space use to a minimum. That was a big help in trying to figure out the backup puzzle with camera archives vs libraries. Etc. My brain is fried and I have to get back to the action. I'll double check with you before I order.