Media Storage - Disaster Recovery
I'm part of a small film production company that has been operating for about 10 years. We're working on developing a disaster recovery process for our media and various other files. So if we were to have a fire on the premises (or something like that) we wouldn't lose all of our archives, and current projects.
We’re planning on backing up all of our fcp files, etc, on a cloud storage system, but we’re not sure of the best way to back up all of our original footage, especially the footage we are currently working with. We currently have all of our completed projects archived on 10 2TB external hard drives, and we keep them at the office. We have our current projects on a 3TB server, also located at the office. We would really like to avoid having to take external hard-drives off-site at the end of everyday.
Any ideas? Is there a good (quick/easy) way to preform daily back-ups without using external hard-drives (which is becoming very time and space consuming)?
Hand Crank Films
Actually backing up your proj. files to the cloud is a great idea.
You don't have to worry about all the Qt's so much as you'll have the EDL's to reference back to the master reels.
Are all your sources from Tape? Where are the master tapes stored?
Are they card based media?
I'd store one Hard Drive backup on site and one off site.
Once a month take a hard drive to a safety deposit box.
Video Atlanta LLC
We have eSATA JBOD enclosures to put hard drives on sleds in for backups. We have currently 30 hard drives from 500 gigs to 1 terabyte. For each of our in house drives I have a clone of that drive that I keep in a fire case at my home in cool, dry storage. I run the drives every 2-3 months. We use a simple drive catalog software (DiscCatalog Maker) on the Mac.
This has worked for us. The drives are convenient to load up and make quick changes if a client asks for a DVD with last year's spots for example. This just happened. I loaded the backup drive, encoded the files right from the drive, built the DVD back to that drive and made some discs. Could not have done this from the cloud.
I'm sticking to this until tape backup can store 100 gigs or more of data per tape and can mount like a drive on the desktop and be easily added to and read from. Might be a few years....
Tilt Media Inc.
Video Production, Post, Studio Sound Stage
Get yourself a huge, fireproof gun safe and have someone run an offshoot line from the sprinkler system. Also, put a 32Tb backup system in the safe, in a rack and fireproof your cable entry. Pretty simple actually. Carrying your hard drives off site is probably the worst thing you could possible do. Accidents, thefts, forgetfulness could hurt you.
It's a dry heat!
Sony HDCAM F-900 & HDW-2000/1 deck
5 Final Cut (not quite PRO) systems
Sony HVR-M25 HDV deck
2-Sony EX-1 HD .
Ask me how to Market Yourself using Send Out Cards
I haven't seen any fire proof safes (outside of vaults perhaps), only fire resistant safes, rated on an hourly basis. Magnetic or optical media cooking for a couple hours doesn't sound very reliable. Even if it lasts through the fire, how much longer will it last after?
For your archives, you should have two copies. One on site, the other offsite. They are old archives so don't need to be rotated. They only take one trip offsite and stay there.
The rest of your data is cycled on a daily, weekly, monthly, quarterly, and annual backup rotation. If you don't like harddrives (and you shouldn't because they don't do revision control well, and have built in and fallible moving parts), then you should be using tapes, which are the super industry standard (across all industries). LTO are the highest capacity, giving you up to 1.5tb per tape. Use enterprise class backup software to manage them, like backup exec, and if you have the data demands and budget for it, an autoloader.
Rich, you should reconsider your position on tapes. 100gb happened a long time ago. However they don't mount like a drive, and they shouldn't. Tapes and backup software provide revision control; harddrives do not (without revision control software).
In any case, your backup process is only useful if it is documented accurately and followed to the letter. When you write it out, detail every step of each process. There will be many processes: The process for a daily backup, What happens for the weekly backup, The process for the monthly, quarterly and annual backup; when tapes are rotated, when they are taken off site, when data is removed from the backup cycle and turns into archive media. How to recover a single file. How to recover a whole project (you have some of it on the cloud, and perhaps some of it on the tape. How to recover files from the cloud, vs tape. Once you have your processes, test and fix them; you forgot something. Assume that the person creating the process will not be the one recovering the system.
How much do you take off site? Talk with your company principals and figure out how much risk exposure you have in order to determine how anal to be with your backups. If you are shooting new projects once a week, then taking your raw footage offsite once a week may be appropriate, twice a week is over kill. If you are doing 3 shoots every day, then taking that raw footage offsite once a week means you could potentially loose 15 shoots worth of information. How much did that cost vs employees time to take another backup?
A typical, reasonable backup cycle for a company is like this:
Daily (Sun-Thurs) - differential backup, single copy, on site
Weekly (Fri) - Full backup, copy 1, on site
Weekly (Sat) - Full backup, copy 2, taken off site and swapped with one of the offsite sets
Have as many sets of tapes as you desire for your weekly backups. Some can be on site, some off site as desired. Once a month, one of these weekly backups is designated the monthly backup. Retain monthlies up to a quarter. Once a quarter, one of the monthly backups is named the quarterly backup. And so on. At any given time you can have a single annual, a single quarter or multiple, multiple months, multiple weeks, and in the drive the daily differentials for the last one or two weeks.
You don't need to have a huge amount of tape sets, as you can recycle them (when the last quarter becomes the annual, the old annual goes back into the cycle). You just need enough to give you all the backup retention you desire. Scale the resolution up or down according to your cost/benefit assessment derived from your risk exposure analysis.
I've done this for public companies as an IT manager. The auditors loved my facilities because they ran smoothly and were well documented. I made my own job very easy as well.
sorry Megan, you are not going to get what you want. Everyone is giving you realistic ways to do things that you DO NOT want to do - you want to keep your media on a cloud (because you don't want to buy the correct backup systems or "gun safe" that currently will actually work in real life.
You want to back up 3 TB to a cloud. Well, get started now, and you should be done 6 months from now. FTP and uplink to cloud sites is SLOW. Have you ever tried to upload a tiny file to YouTube ? Have you ever tried to FTP a 2 Gig file to YouSendIt ? Do you know how long it takes to upload even a small file ? You are NEVER going to upload 3TB of full res media to a cloud, unless you mail it to them, and you won't get it down, unless "they " create proxy files for you. When Apple unveils their data center later this year, it's for h.264 files, not ProRes422HQ files.
With todays technology, unless you and your cloud site have Signiant or Aspera dedicated hardware (or a dedicated satellite link like Pathfire) - you are NOT going to get what you want. There is NO CHEAP WAY OUT. People that think that they are going to edit on the cloud (without proxy files in between) are on drugs. Not in 2011.
Do you know when this will happen Megan ? When the cable companies run fibre into everyone's business. And do you know when this will happen - not while you are employed at your company.
I saw a vid
by Chase Jarvis on what he does at his place. Bit of an ad for G-Technology, but he goes with what works.
Crikey. I just previewed this and there it is mid-sentence.
Thanks All! Some really great ideas out there. You've been a big help.
Hand Crank Films