Backup / Archive Advice?
I just started working at a place that does not have a very good backup/archive system in place and I'm wondering what others are doing.
I'm using AVID on a PC. I'm editing to a 10TB raid 5 tower that is also mirrored by another raid 5 tower. The situation is there are many shoots and several projects use footage from all these shoots. I'm thinking when a project is complete I'll archive it, including all the clips from these shoots used in the project. Possibly put that on a Raid 1 drive. But I'm not sure the best way to archive the shoot footage once it's not being used anymore. Also, there will be a lot of duplicate archiving with the shoot being archived, and the projects archived with copies of the clips used.
How do you all handle archiving/backups when there's a lot of footage coming in?
[Jerry Sneede] "But I'm not sure the best way to archive the shoot footage once it's not being used anymore"
Archive the camera masters, always. LTO6 is a good SOLID choice. Or, just archive it the way you mentioned, RAID 1 drives. Always archive the camera masters. If you will need to use that footage again, and don't want to re-ingest, then archive it to separate RAID 1 drives.
Where I work, we archive the camera masters to LTO6, and footage in the shows to RAID 1 drives from OWC. Because we always end up going "You know, I need that footage from that one series we worked on last summer." Or a second or third season comes along and we need to have access to all the previous footage.
Little Frog Post
Read my blog, Little Frog in High Def
LTO is one of your best bets. I consider hardrives as storage not archives. At some point you will either want that space back or it simply wont mount anymore. LTOs are battle tested,dont take a lot of space, and are fairly cheap for the storage. I have an lto 6 drive,but find the price change from lto 5 to 6 media vs the size difference still makes more sense to use lto 5 tapes.
Just make sure you pick software that is easy to use...proven... and doesnt tie your hands down the road.
The LTO looks like the most robust solution, but kind of expensive at first.
As for archiving MC projects, what's the best solution? There doesn't seem to be an archive tool like FCP has. I've heard of software that will do this, any suggestions?
Yeah, there is a method like FCP has, and it's far more robust. Select the sequence, and then in the CLIP menu choose CONSOLIDATE. Choose your handle length, destination drive and away you go. Best to make a new bin and put a copy of the sequence in that bin...as that bin will populate with all the clips linked to the new media that you are consolidating over. Then you can label that bin CONSOLIDATED LOCKED PICTURE MEDIA and put the sequence in a bin and call it CONSOLIDATED LOCKED CUT or something.
Little Frog Post
Read my blog, Little Frog in High Def
Would love to help you out on a system that fits your needs and budget. Feel free to give me a call. 714-374-4944
Also remember to backup your elements.
Nothing worse than needing to go back 1 step on something, and you don't have the photos you started with , or something like that.
When I do an archive, i have everything i brought into the system as far as music, stills, animations, scripts, notes, lists.
If it was associated with the project, it gets put into my archive.
I usually leave the archive of the camera masters to the producer/camera op, unless they specifically request it.
I also archive a digital copy of my master output.
As far as LTO equipment goes, there are cheaper ways of building a system.
We used to have a Cache, which was quite expensive, and was never my favourite.
I've since built my own system, and its been running perfectly for a couple of years now, at about half the price.
Expansion chassis with mini sas interface
Quantum LTO 6 Drive
software - I'm using BRU PE, and find it very simple, the catalog system is great, and if your client wants their LTO tape back, there is no charge for a restore only version of the software. In fact you have the option of burning it to disc when you do your initial archive for them.
I also do some LTFS archives, but find them much harder to go back to if i need to pull out a couple of files. A lot more scanning of the tape when you use that system. Software for it is usually free though. Do some research and (eventually) some testing.
I'm still a bit unclear about how to archive the MC projects. I'm used to FCP's media manager, also used to having all the files for a project in a project folder, unlike Avid putting all the transcoded footage in one folder. I know there is third party software for this, anyone know of anything good? I heard about this Russian free software but the URL does not seem to go anywhere anymore. Also, what do you usually archive, the transcodes or the original media (or both)? Thanks for any help on this.
From an Avid standpoint, assuming you are using Avid Media, not AMA links, or even fast imported items, as they may give you different results as well, but for a basic workflow, I would go into my Avid MediaFiles Folder, and rename the "1" folder to something else...perhaps "2"
I would then go into Avid and Consolidate my timeline, with handles, as was pointed out here earlier, duplicate it first, and put it into a new bin. Name this "consolidated Sequence"
Once complete I would probably go back to my Avid Mediafiles Folder and move the "2" folder up a level, so that Avid doesn't recognize it. I would also rename the brand new "1" folder to something like "project name media"
Go back to avid, load your consolidated sequence, and make sure everything is online. If anything is offline, then your source material may have to be dealt with differently. (you may require a transcode pass as well)
If everything is there, and all looks good, you can now copy your "project name media" folder to your archive location.
Find your project folder from your Finder or Windows explorer, and copy it to your archive location. (I typically zip it up first then copy it across)
Copy any elements you need to your archive location.
Its not a bad idea if the project had any "gotchas" to write up a quick text document explaining, or reminding the future you what you did and why.
You can now delete your "project name media" folder from your system, and move your "2" folder back into its original location.
When you launch avid, you should now see your consolidated timeline offline, but your original timeline is back online...this media is coming from the "2" folder. If that is the only thing in the "2" folder, you can delete it. If there are other projects in that folder, you may want to go through media tool to delete the media related to the project you just archived.
As you can see, this is where it is nice to control your media folders, from the start, on a project by project basis. There are plenty of posts here with peoples workflows for naming and managing folders. Takes a bit to get used to, but in the end, makes media management quite easy.
When its time to restore the project, put your "project name media"
folder back into the Avid MediaFiles folder, put your project folder back into your projects location, and launch the program. Your consolidated sequence bin should be back online. You will also see everything else in your project offline.
This tactic does not backup camera originals. That would be a process outside of avid. It also does not backup media not used in the final edit. If you want to do that as well, then your probably looking at simply copying all your original "1" folder...before you renamed it to "2" across...again, depending on how you manage your folders and media from the get go.
There are lots of variables that will come into play when doing your archives. How much original avid media you are dealing with being one of the main ones. If the consolidated version of a project is not much bigger than the original version, I might just archive everything and skip the whole consolidate process. Also, if this is a project or client that is always coming back for changes and versions, or reusing old material for new jobs, I would again probably just archive everything. If it is a one off, cutting 6 hours down to a 30 second spot, I would probably consolidate, and recommend the client has their originals backed up.
Every situation is different.
Do some reading on the managing of folders, naming them for individual projects etc, and then this all becomes quite simple.