Sorry if this is a repeat topic, I am sure there are many suggestions and solutions and would appreciate any advice. I am not entirely how to search the forum for the information I am looking for so if people have suggested threads or suggestions anything would be appreciated.
I am beginning working in 720p30 I believe. I am working off of the HMC-150 and I know it only has 1 720 option which I believe is 30P. I am transcoding my AVCCAM files to ProRes422 which is working out to .5 GB per minute of video. Right now I have plenty of storage space but will run out quickly.
I have the option to archive my footage on to DV tapes through an HDV camera and the transfer to tape option in FCP. I am wondering how viable a solution this is without losing a noticeable amount of quality.
I am also currently making backups of the memory card before transcoding. I was considering just deleting the ProRes files after the project is completed and is a few months old. Then if I need to I can just re-transcode the clips if something comes up with that project.
Sorry for the long post, any suggestions for the short term would be appreciated. I would also appreciate some affordable longterm solutions - RAID, internal drives, other suggestions?
Currently my backup solution is a 500 GB and a 2 TB time capsule.
Thanks for any and all suggestions and help,
[Mark Laslo] "I have the option to archive my footage on to DV tapes through an HDV camera and the transfer to tape option in FCP. I am wondering how viable a solution this is without losing a noticeable amount of quality."
No...you will notice a HUGE drop in quality. From HD to the lowest format of broadcast SD out there. This would be the second worse way to archive...next to burning DVDs.
Why archive the IMPORTED footage? Why not archive the CAMERA ORIGINALS? Please tell me you backed those up. If not, that's a major *headdesk* moment. Or *facepalm* as my kids like to say.
[Mark Laslo] "I am also currently making backups of the memory card before transcoding"
OK...whew. Scared me for a second there. THIS is what you archive and back up. Takes up FAR LESS space than the transcoded footage.
[Mark Laslo] "I was considering just deleting the ProRes files after the project is completed and is a few months old. Then if I need to I can just re-transcode the clips if something comes up with that project. "
YES! Precisely...exactly. THAT's how it is done.
[Mark Laslo] " I would also appreciate some affordable longterm solutions - RAID, internal drives, other suggestions?"
Bare SATA drives and a SATA-USB docking station. 1TB drives for $70. Do them in mirrored sets, meaning a copy of the footage on TWO drives.
GETTING ORGANIZED WITH FINAL CUT PRO DVD...don't miss it.
Read my blog, Little Frog in High Def
You could also consider backing up card-wise to Blu Ray. 23 Gb or 50 Gb and the price of these writers and disks is falling fast.
Of course, you will back up the original AVCHD files, not the ProRes converts.
FCP Editor, Edit systems consultant
Thanks Shane and Neil,
I will consider both of those options.
Shane unfortunately the first camera we worked with was the HDV tape camera and to save cost we imported the entire tape in SD and then after the project was dormant for a few months reuse the tape. The original imported files still exist (though we have no way to get the HD footage back but that is not a huge issue for us as we are just now making the push to HD on our website). Going forward I will be making backups of the AVC files.
I like the idea of the 1 TB drives and a external dock. What is your suggestion for migrating the media - just copy the sequence and all of the original avc media into a folder or should I be using media manager at all? My only concern is we use a lot of pictures in some of our projects but they are stored in region specific folders instead of project specific folders as they are reused from time to time.
[Mark Laslo] "unfortunately the first camera we worked with was the HDV tape camera and to save cost we imported the entire tape in SD and then after the project was dormant for a few months reuse the tape."
I'm sorry to be blunt here, but that was completely stupid. Why would you/they do such a dumb thing? DV tapes are $5 each! You'd go and record OVER footage that you are using in a project? Just in general...why do that? $5 vs losing the footage forever. How cheap are you guys? Besides that, reusing tapes makes them more prone to drop outs...glitches. So you risk making your NEW footage have digital hits and errors. Is $5 really worth all that?
[Mark Laslo] "The original imported files still exist (though we have no way to get the HD footage back but that is not a huge issue for us as we are just now making the push to HD on our website). "
There is NO WAY you are going to get DV to look like HD. Even if you shot HD, if you captured as DV, it is DV. So this push for HD on your site will not happen with this footage. Yes, you can use INSTANT HD or COMPRESSOR to upconvert the footage, but that won't add detail where detail was removed. DV is the lowest end SD broadcast format...you are not going to make that look HD.
[Mark Laslo] "What is your suggestion for migrating the media - just copy the sequence and all of the original avc media into a folder or should I be using media manager at all?"
I am HIGHLY ORGANIZED when I go my projects. I make a single folder that I capture and render to, make folders for music and still pictures, graphic elements that I use in the project. So that I can back up everything, or just grab the non-tape or tapeless footage and just back up that. Good organization is key on so many levels.
[Mark Laslo] "My only concern is we use a lot of pictures in some of our projects but they are stored in region specific folders instead of project specific folders as they are reused from time to time."
COPY them to the project folder. This way you have everything you use in one location, and still keep a copy in a generic location. Having assets spread all over the place makes archiving a long and involved process.
GETTING ORGANIZED WITH FINAL CUT PRO DVD...don't miss it.
Read my blog, Little Frog in High Def
The tapes were only reused after the project was completed and put up on the web, but I do understand what you are saying about price vs being able to use/have the footage forever. Honestly it was a rookie mistake that I learned from and am learning from.
I understand that I cannot make the jump from DV to HD in any capacity besides upscaling the original footage and getting no improved quality. This is not a major concern to us as most of these projects are time sensitive and will not be on our site much longer and our current streaming solution does not support anything higher than 512k flv streaming.
What I am most interested at this point is your organizational strategy as this is something I've struggled with. I work for a non-profit and we are trying to keep price as low as possible so while I understand that drives are cheap, we lack the ability to regularly buy drives to add storage space. Right now I have internal one 700 GB and one 1 TB drive. The 700 GB drive is my OS drive and stores my project files and system files. The 1 TB drive stores pictures, music, video, render, and other misc project and FCP files.
I also have a 2 TB time capsule that is being used as a system backup including all Media except render files.
In terms of organization to avoid duplicate files I keep all of the pictures organized by region (we operate in 7 states), video organized by region with individual folders for each project, music separated by original source - Soundtrack, purchased, CD. My goal is to try and avoid duplicate files and end up using unnecessary hard drive space by having pictures in a general folder as well as each individual project, the same with video and music.
Sorry if that is long winded or confusing, I am just looking to get the best possible solution to allow for archiving. If you think that I am better off making duplicates then I am willing to consider that option but want to make sure I don't fill drives up unnecessarily.
I've been using Blu-Ray as an archiving solution for the last year or so. Since starting to work with R3D files we've noticed project files exploding in size, even beyond the practical limit of dual-layer Blu-Ray (a platter of 20 discs for a single project, for example. After a full day of shooting I have to spend another day just burning everything to blu-ray (twice). And we've had failures as well (which is why we do everything twice). And before anyone recommends LTO, we've been hosed by high capacity tape in the past as well no matter how carefully we store them.
We're considering moving to a bare SATA solution for better speed & cost. We even found a source for hard drive cases. I still don't trust bare drives as an archiving solution, but it looks like the best fit for us until solid state becomes more affordable (assuming that sd drives would be more reliable and durable). Once that glorious day comes, we can transfer the bare drives and be done.
I may not have a decent night's sleep until that happens but it seems like our best logical choice.
Obviously, the archiving method is largely dependent upon the shooting format. For people using a RED workflow, blu-ray probably doesn't make as much sense because of the size limitations.
However, for us smaller scale folks using AVCHD, I think blu-ray is THE best mid to even longer range archiving solution. Hands down. A year or two ago this may not have been the case, but now I think it is a no-brainer when compared to using HDDs (LTO isn't really practical for me given its cost and the current size of my enterprise). So cost AND reliability being factors, blu-ray is, imho, the clear winner. Especially after doing the math. What I figured was:
1) Hard drives fail. So I'd need at a MINIMUM two drives in a raid 1 to combat this, which immediately doubles my cost per gigabyte. If I want even more redundancy, then I triple or even quadruple this figure. And even at that, this is arguably still not as safe as properly stored optical media.
2) Bottom shelf cost for a low end 1TB hard drive currently is about $70. For archiving I'd need at least two of these (still not safe). But even with just two my cost per terabyte = $140. Per gig = 14 cents. (And this doesn't factor in the cost of an external enclosure either). But still assuming it is only .14 per gigabyte, that means that to archive 25 GB (a standard 1 layer bd-r's worth of data) it is going to cost me $3.50.
As of now, the cost of a bd-r (if you buy bulk and find the right merchant) is less than $1.50. I use Memorex bd-rs, have had no problems so far, and get these at this price or cheaper. (http://www.chiefvalue.com/product/productdetails.aspx?submit=&item=CE00158184010060&ATTCD%20DVD%20Media)
That's less than half the cost of archiving with HDDs, for a solution that is eminently safer and more portable. True, the time it takes to archive via bd-r is longer, but given the fact that I can easily organize my projects by disc, give them to clients without having to loan them my hard drive(s), and have the piece of mind that all of my projects are individually backed up (ie the failure of a single RAID array kills ALL your projects...whereas every bd-r is stand alone), I think there is no better (current) solution for folks working in consumer/prosumer formats like AVCHD.
Just my $.02 :)
http://www.gobomedia.net (under construction)
"Philosophy is questioning without answers; religion is answers without questioning." - Anonymous
I agree. However, I've learned the hard way to archive to blu-ray twice (what we call a "RAID level oops") since we've had a few failures in the last year. The extra cost is nothing compared to the extra time of archiving x2 but it's saved my rear more than once.
We toyed with the idea of a blu-ray robot for auto-archiving huge data sets but by the time you do that you might as well buy LTO.
And now that the discs cost half what they did when we started, we'll probably stick with it. The SATA system still seems tempting but only as a means of hot-swapping huge projects in and out when we get really, really busy. And most of those issues can be addressed with aggressive media management.